Hands-on introduction to NGS variant analysis-2018

From BITS wiki
Jump to: navigation, search

[ Main_Page | NGS_data_analysis ]
# two-days training 2018/11/09-16 session


Technical.png This session is a GenePattern rewritten version of the simplified 2017 version (Hands-on_introduction_to_NGS_variant_analysis-2017) of a more complete and exploratory training given in 2013, 2014 and 2016 (Hands-on introduction to NGS variant analysis)

Technical.png Credits: All GenePattern modules used today have been created by Guy Bottu (VIB Bioinformatics and Nucleomics Core)

Aims of the NGS DNA variant analysis 2-days session

Using a full publicly available chromosome read-set from one of the 1000 genomes[1] samples:

  • Use the graphical environment provided by the BITS GenePattern server to perform all steps of a classical NGS variant workflow and feel the complexity of the task.
  • Perform a simplified analysis workflow including read mapping, variant calling against the current human reference genome
  • Annotate the obtained variant calls and compare them to the public variant file for that genome.
  • implicitly: get the motivation to go to the next level and learn command line and R

Scheme of today's workflow


More BITS Training Info


This training gives an introduction to the use of popular NGS analysis software packages through the GenePattern graphical interface. It reviews several exchangeable tools and provides hints to evaluate quality and content of Genome-Seq data. Much more can be (and should be) done when working at command-line and GenePattern will not replace advanced use of the terminal. However, this simplified workflow will allow unexperienced scientists to discover the practices involved with variant analysis. A recent review by Geraldine Van der Auwera et al develops on many aspects of this [2].

The sequencing data used in this session was obtained from high coverage depth Illumina HiSeq sequencing of gDNA extracted from EBV-transformed B-lymphocytes of a healthy CEPH/UTAH Mormon mother (NA12878). More information about that sample is available from the Coriell repository from which all 1000g gDNA can be obtained [3].

The paired end reads used in this training were extracted from the Genome In A Bottle (GIAB) novoalign BAM mappings of Illumina HiSeq 300x reads for NA12878 available on the ftp://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/ server [4]. This pre-processing step is not covered in the training. A lot of analysis results are available on the dedicated GIAB page [5]

We use this year the state of the art Genome Analysis Toolkit ('GATK') for variant analysis and we strongly advise you to consider it in your work. GATK was compared to varscan used in former training sessions and is clearly superior in sensitivity and specificity [6]. GATK is today the standard workflow for human genome analyses and is referred in most publications. We do not apply here all the available tricks GATK allows but rather present you a streamline pipeline with a single genome that will be a good foundation for most applications. Please read the GATK pages[7] for more information.


The official GATK best practice inspiring this training is as follows:



Translated into separate exercises from today's training (red numbers next to boxes)



Disclaimer: This training does not cover all currently available methods. It does not aim at bringing you to a professional NGS analyst level but provides enough information to allow a motivated biologist understanding what DNA sequencing practically is, and when necessary to communicate knowingly with NGS experts for more in-depth needs.

Handicon.png a nice recent recording of Geraldine's seminar can be found



Skills required to follow this training

  • Linux command line basic skills are required to review some of the long text results under terminal (GenePattern is not handy in reviewing data, it is mainly a job submission and management platform)
  • basic knowledge of human genome structure and nomenclature is necessary to understand the training tasks and the results
  • basic knowledge of Illumina NGS read structure is also required for the same reason

Software used during this training

  • All programs used in this training session were installed on the BITS GenePattern server and specific modules where created for you by Guy Bottu (BITS). These modules are therefore absent from other GenePattern servers like those accessible at the Broad. If you plan to build your own GenePattern Server (GP home page), you may ask Guy copies of these modules contact us.

Hands-On Exercises

REM: At the end of each page, a link to data available on our file server is added and allows downloading some of the data for local use and self-training. Other files, required for the training are present on the server and will be accessed directly from within Genepattern.

Answers to your 2017 requests

Some of you asked about the possibility to call variants from structured experiments like family trios or tumor-normal pairs. We found trio data for one of the public 1000 genome trio (CEPH CEU) as well as a tumor-normal pancreatic cancer dataset (WES) with which we prepared two walk-through tutorial for command-line analysis using Varscan2. Both documents have been linked below.

  • call variants from a family trio (NA12878) and identify inherited and potential non-inherited variants (de-novo) using Varscan 2 (link)
  • call variants from a pair of samples (tumor and normal tissues of the same patient) and identify somatic variants and CNA (copy number abnormalities) in the tumor sample (link)

Handicon.png Please contact us for more info and to report inconsistencies in these documents

PDF version of this training

  • A partial PDF export of these pages can be downloaded here
  • A LatEx tutorial to perform a similar analysis at command-line can be found here

Find more tools and answers

There are many tools out there, finding them is often the easiest part. You are welcome to try as many as you wish and improve results obtained with our selected toolbox. When seeking advice, please consider using:

Technical.png GATK4 should also run on multicore machines using the built-in SPARK system. There will at some point in time be a documentation HERE about it

Extra readings

Other packages have been compared including mappers and callers, here is a starter for more readings, you will find more on Google.

  • A Comparison of Variant Calling Pipelines Using Genome in a Bottle as a Reference [14]
  • Performance Assessment of Variant Calling Pipelines using Human Whole Exome Sequencing and Simulated data [15]
  • VCF-Miner: GUI-based application for mining variants and annotations stored in VCF files [16]
  • Evaluating Variant Calling Tools for Non-Matched Next-Generation Sequencing Data [17]
  • appreci8: a pipeline for precise variant calling integrating 8 tools [18]

What about longer variants (structural!)?

  • Accurate detection of complex structural variations using single-molecule sequencing [19]

What about CNV and RNASeq variants?

GATK performance in numbers

A recent nature biotechnology paper by Poplin et al [20] compared GATK to other callers. Their results concerning GATK (version 3.8) are pictured below from the first two tables.





Conclusion & Contact

Your feedback to this introductory NGS variant analysis using GenePattern session is very important to us and will be used to improve this content for later sessions. If you need more training of this kind, please contact us and we will organise additional hands-on based on your requests. More advance sessions will depend on the availability of expert users within VIB that will accept to prepare specified material.

contact us

  1. http://www.1000genomes.org
  2. Geraldine A Van der Auwera, Mauricio O Carneiro, Chris Hartl, Ryan Poplin, Guillermo Del Angel, Ami Levy-Moonshine, Tadeusz Jordan, Khalid Shakir, David Roazen, Joel Thibault, Eric Banks, Kiran V Garimella, David Altshuler, Stacey Gabriel, Mark A DePristo
    From FastQ data to high confidence variant calls: the Genome Analysis Toolkit best practices pipeline.
    Curr Protoc Bioinformatics: 2013, 43;11.10.1-33
    [PubMed:25431634] ##WORLDCAT## [DOI] (I p)

  3. http://ccr.coriell.org/Sections/Search/Sample_Detail.aspx?Ref=GM12878
  4. ftp://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/NA12878/NIST_NA12878_HG001_HiSeq_300x/NHGRI_Illumina300X_novoalign_bams/
  5. http://jimb.stanford.edu/giab-resources/
  6. Charles D Warden, Aaron W Adamson, Susan L Neuhausen, Xiwei Wu
    Detailed comparison of two popular variant calling packages for exome and targeted exon studies.
    PeerJ: 2014, 2;e600
    [PubMed:25289185] ##WORLDCAT## [DOI] (P e)

  7. https://software.broadinstitute.org/gatk/)
  8. https://software.broadinstitute.org/gatk/best-practices/workflow?id=11145
  9. http://seqanswers.com SeqAnswers
  10. http://www.biostars.org BioStar
  11. http://stackoverflow.com stackoverflow
  12. http://bioinformatics.ca/links_directory
  13. https://broadinstitute.github.io/picard/explain-flags.html
  14. https://www.researchgate.net/publication/283954255_A_Comparison_of_Variant_Calling_Pipelines_Using_Genome_in_a_Bottle_as_a_Reference
  15. https://www.biorxiv.org/content/early/2018/06/29/359109)
  16. Steven N Hart, Patrick Duffy, Daniel J Quest, Asif Hossain, Mike A Meiners, Jean-Pierre Kocher
    VCF-Miner: GUI-based application for mining variants and annotations stored in VCF files.
    Brief. Bioinformatics: 2016, 17(2);346-51
    [PubMed:26210358] ##WORLDCAT## [DOI] (I p)

  17. Sarah Sandmann, Aniek O de Graaf, Mohsen Karimi, Bert A van der Reijden, Eva Hellström-Lindberg, Joop H Jansen, Martin Dugas
    Evaluating Variant Calling Tools for Non-Matched Next-Generation Sequencing Data.
    Sci Rep: 2017, 7;43169
    [PubMed:28233799] ##WORLDCAT## [DOI] (I e)

  18. Sarah Sandmann, Mohsen Karimi, Aniek O de Graaf, Christian Rohde, Stefanie Göllner, Julian Varghese, Jan Ernsting, Gunilla Walldin, Bert A van der Reijden, Carsten Müller-Tidow, Luca Malcovati, Eva Hellström-Lindberg, Joop H Jansen, Martin Dugas
    appreci8: a pipeline for precise variant calling integrating 8 tools.
    Bioinformatics: 2018, 34(24);4205-4212
    [PubMed:29945233] ##WORLDCAT## [DOI] (I p)

  19. Fritz J Sedlazeck, Philipp Rescheneder, Moritz Smolka, Han Fang, Maria Nattestad, Arndt von Haeseler, Michael C Schatz
    Accurate detection of complex structural variations using single-molecule sequencing.
    Nat. Methods: 2018, 15(6);461-468
    [PubMed:29713083] ##WORLDCAT## [DOI] (I p)

  20. Ryan Poplin, Pi-Chuan Chang, David Alexander, Scott Schwartz, Thomas Colthurst, Alexander Ku, Dan Newburger, Jojo Dijamco, Nam Nguyen, Pegah T Afshar, Sam S Gross, Lizzie Dorfman, Cory Y McLean, Mark A DePristo
    A universal SNP and small-indel variant caller using deep neural networks.
    Nat. Biotechnol.: 2018, 36(10);983-987
    [PubMed:30247488] ##WORLDCAT## [DOI] (I p)

[ Main_Page ]