Avoiding Nonsense Resultsin your NGS Variant Studies
James Lyons-Weiler, PhDScientific Director/
Senior Research ScientistBioinformatics Analysis Core
Genomics & Proteomics Core LaboratoriesUniversity of Pittsburgh
Pittsburgh, PAMay 1, 2014
Two Parts
• Identifying sites with low genotypic signal increases concordance among variant callers
• Hazards in finding differentially expressed genes in RNASeq – how to do it more robustly.
23andMe: High risk of RA and psiriosisGTL: Low risk of RA and psiriosis
NYTimes Article, etc.
Data were from Illumina hi-seq 2000
Among method averageConcordance57.5% overall; 32.7% at high coverage
O’Rawe et al.
TRUTH (BIOLOGICAL MOLECULAR SEQUENCE)
SEQUENCER
MAPPER
VARIANT CALLERS
LOW CONCORDANCE (O’Rawe et al., 2013)
Consensus Analysise.g.,2/3, ¾, set analysis
Information Theory(-> modeling)
Improve Callers(fix errors, modeling) Bake Offs
Simulations
Spiked Ins
Entropy of Base Distributions
A T C GA T C G A T C GLow entropyHigh enthalpy
Low entropyHigh enthalpy
High entropyLow enthalpy
Boltzmann Entropy
• s = k ln w (Planck)
• w = antiln(s/k)
http://schneider.ncifcrf.gov/images/boltzmann/boltzmann-tomb-4.html
Rank Sorted Distribution of w(O’Rawe et al. data)
Homozygotes w = 1
Heterozygotes w = 2
Example w Density Distribution
w and FBVCA T C G w pw Zygosity Genotype200 0 0 0 1 0 Homozygote AA
16 158 13 13 2.102558 0 Homozygote TT100 100 0 0 2 0 Heterozygote AT
58 30 1 111 2.768507 0 Heterozygote AG28 80 14 78 3.303636 0 Heterozygote TG76 38 29 57 3.758733 0 Heterozygote AG33 49 60 58 3.895496 0.0126 Heterzygote? CG?50 50 50 50 4 1 noise unknown
Operational*Equiprobable Null Distribution
{f(A) = f(T) = f(G) = f(C)}
Convergence of significance (pw)
What We Expect
TRUTH (BIOLOGICAL MOLECULAR SEQUENCE)
SEQUENCER
MAPPER
VARIANT/BASE CALLERS
Genotypic Signal Filtering
INCREASED CONCORDANCE
Phom Function
gatkConcordance w/ FBVC Hom Het
ALL 0.5762 11868 17670pw<=0.05 0.9976 11282 5676
pw>0.05 0.0074 586 11994samtools
ALL 0.5649 11541 18799pw<=0.05 0.9917 11489 5761
pw>0.05 0.0002 52 13038snver
ALL 0.6006 11904 16729pw<=0.05 0.9934 11812 5470
pw>0.05 0.0007 92 11259
From the O’Rawe et al. generated resultsFBVC = frequency-based variant caller (Lyons-Weiler et al.)
Signal Tx %ConcordanceFBVC_vs_FBVC Marked ALL 85.64
pw<=0.05 91.08pw>0.05 35.66
FBVC_vs_FBVC Realigned ALL 83.82pw<=0.05 91.69
pw>0.05 28.21FBVC_vs_FBVC Recalibrated ALL 93.14
pw<=0.05 ***99.39pw>0.05 48.53
FBVC_vs_FBVC Reduced ALL 21.54pw<=0.05 24.57
pw>0.05 4.25FBVC_vs_FBVC Marked-Realigned ALL 76.91
pw<=0.05 86.11pw>0.05 15.44
FBVC_vs_FBVC Marked-Realigned-Recalibrated ALL 76.73pw<=0.05 85.99
pw>0.05 15.34
FBVC_vs_FBVC Marked-Realigned-Recalibrated-Reduced ALL 19.98pw<=0.05 22.9
pw>0.05 2.66
TRUTH (BIOLOGICAL MOLECULAR SEQUENCE)
SEQUENCER
MAPPER
VARIANT CALLERS
LOW CONCORDANCE (O’Rawe et al., 2013)
Consensus Analysise.g.,2/3, ¾, set analysis
Information Theory(-> modeling)
Improve Callers(fix errors, modeling) Bake Offs
Simulations
Spiked Ins
Lifescope reads (read)
Shrimp2 reads (blue)
Mappers must be systematically evaluated
Part 2: Good and Bad News forRNASeq (and everything else):
The Bad News:
Fold Change is Biased.
The Good News:
We have identified a much less biased method.
T-test is not appropriatefor small N, large P data
(such as RNASeq)
Fold Change > 2.0
Delta > 25
FC(A/B) is Blind to Large Portionsof Your Data
FC(A/B)
Delta(and J5: Patel & Lyons-Weiler, 2004)
Ratio are Hard to Interpret asBiological Differences
Gene A B delta (A-B) FC(A/B)
gene1 5 3 2 1.667
gene2 50 30 20 1.667
gene3 500 300 200 1.667
gene4 5000 3000 2000 1.667
gene5 50000 30000 20000 1.667
A-B is a differenceA/B is a quotient.
Log2 TransformationDoes not Help
Reveals Minor Delta (&J5) Bias
Pink = FC(A/B)Black = Delta
G-Thresholding J5
FC Bias in Amyotrophic Lateral Sclerosis
0
50000
100000
150000
200000
250000
300000
350000
0 50000 100000 150000 200000
Control
ALS DEGy
FCDEGy
Black circles = FC(A/B). Pink = Gthr-J5 genes
Black circles = FC(A/B). Pink = Gthr-J5 genes
FC(A/B) Bias inAlchohol-Induced Hepatitis
Conclusions• Not all NGS/HTS sites have sufficient genotypic signal to warrant
a base call. High coverage alone does not provide a solution.
• By measuring genotypic signal, we can determine which sites we can call with confidence.
• Fold-change(FC(A/B) is blind to highly expressed genes and should be abandoned as a measure of differential expression altogether – even for single gene or single protein studies!
• Published microarray data sets analyzed to date using FC(A/B) only are a gold-mine for re-analysis using less biased methods.
Credits and Contact• pw, pHom, etc: James Lyons-Weiler, Alan Twaddle, Rahil Sethi.
– (MS in preparation)– Our software is called Gconf (not yet available)
• Fold-Change Bias: James Lyons-Weiler, Tamanna Sultana, Rick Jordan, Rahil Sethi– (Paper in review)– For now, read
• Mariani TJ, Budhraja V, Mecham BH, Gu CC, Watson MA, Sadovsky Y. 2003. A variable fold change threshold determines significance for expression microarrays. FASEB J. 17:321-3. doi: 10.1096/fj.02-0351fje
• Pearson, K. 1897. On a form of spurious correlation that may arise when indices are used for the measurement of organs. Proc Roy Soc Lond 60:489-498 doi: 10.1098/rspl.1896.0076
