UsingEnsemblesofHiddenMarkovModelsforGrandChallengesinBioinforma9cs

TandyWarnowFounderProfessorofEngineering

TheUniversityofIllinoisatUrbana-ChampaignhFp://tandy.cs.illinois.edu

From the Tree of the Life Website, University of Arizona

Phylogenomics

1kp:ThousandTranscriptomeProject

l  PlantTreeofLifebasedontranscriptomesof~1200speciesl  Morethan13,000genefamilies(mostnotsinglecopy)GeneTreeIncongruence

G. Ka-Shu Wong U Alberta

N. Wickett Northwestern

J. Leebens-Mack U Georgia

N. Matasci iPlant

T. Warnow, S. Mirarab, N. Nguyen UIUC UCSD UCSD

Challenge: Alignment of datasets with > 100,000 sequences

Plus many many other people…

1000-taxonmodels,orderedbydifficulty(Liuetal.,Science324(5934):1561-1564,2009)

SATéandPASTAAlgorithms

Estimate ML tree on new alignment

Tree

Obtain initial alignment and estimated ML tree

Use tree to compute new alignment

Alignment

Repeatun9ltermina9oncondi9on,and

returnthealignment/treepairwiththebestMLscore

1000taxonmodels,orderedbydifficulty,Liuetal.,Science324(5934):1561-1564,2009

24hourSATé-Ianalysis,ondesktopmachines

(Similarimprovementsforbiologicaldatasets)

1000-taxonmodelsrankedbydifficulty

SATé-2be;erthanSATé-1

SATé-1(Liuetal.,Science2009):cananalyzeupto8KsequencesSATé-2(Liuetal.,SystemaKcBiology2012):cananalyzeupto~50Ksequences

RNASim

0.00

0.05

0.10

0.15

0.20

10000 50000 100000 200000

Tree

Erro

r (FN

Rat

e) Clustal−OmegaMuscleMafftStarting TreeSATe2PASTAReference Alignment

PASTA:Mirarab,Nguyen,andWarnow,JComp.Biol.2015–  SimulatedRNASimdatasetsfrom10Kto200Ktaxa–  Limitedto24hoursusing12CPUs–  Notallmethodscouldrun(missingbarscouldnotfinish)

PASTA:evenbeFerthanSATé-2

1kp:ThousandTranscriptomeProject

l  PlantTreeofLifebasedontranscriptomesof~1200speciesl  Morethan13,000genefamilies(mostnotsinglecopy)GeneTreeIncongruence

G. Ka-Shu Wong U Alberta

N. Wickett Northwestern

J. Leebens-Mack U Georgia

N. Matasci iPlant

T. Warnow, S. Mirarab, N. Nguyen UIUC UCSD UCSD

Challenge: Alignment of datasets with > 100,000 sequences

Plus many many other people…

Length

Counts

0

2000

4000

6000

8000

10000

12000 Mean:317Median:266

0 500 1000 1500 2000

1KPdataset:morethan100,000p450amino-acidsequences,manyfragmentary

Length

Counts

0

2000

4000

6000

8000

10000

12000 Mean:317Median:266

0 500 1000 1500 2000

1KPdataset:morethan100,000p450amino-acidsequences,manyfragmentary

Allstandardmul8plesequencealignmentmethodswetestedperformedpoorlyondatasetswithfragments.

ProfileHiddenMarkovModels•  Probabilis9cmodeltorepresentafamilyofsequences,

representedbyamul9plesequencealignment•  IntroducedforsequenceanalysisinKroghetal.1994.

PopularizedinEddy1996andtextbookDurbinetal.1998•  FundamentalpartofHMMERandotherproteindatabases•  Usedfor:homologydetec9on,proteinfamilyassignment,

mul9plesequencealignment,phylogene9cplacement,proteinstructurepredic9on,alignmentsegmenta9on,etc.

ProfileHMMsl  Genera9vemodelforrepresen9ngaMSA

l  Consistsof:

l  Setofstates(Match,inser9on,anddele9on)

l  Transi9onprobabili9es

l  Emissionprobabili9es

ProfileHiddenMarkovModelforDNAsequencealignment

FromhFp://www.cbs.dtu.dk/~kj/bioinfo_assign2.html

HMMsforMSAl  Givenseedalignment(e.g.,inPFAM)andacollec9onof

sequencesfortheproteinfamily:

l  RepresentseedalignmentusingHMM

l  Aligneachaddi9onalsequencetotheHMM

l  Usetransi9vitytoobtainMSA

l  Canwedosomethinglikethiswithoutaseedalignment?

UPPUPP=“Ultra-largemul9plesequencealignmentusingPhylogeny-awareProfiles”Nguyen,Mirarab,andWarnow.GenomeBiology,2014.Purpose:highlyaccuratelarge-scalemul9plesequencealignments,eveninthepresenceoffragmentarysequences.

UPPUPP=“Ultra-largemul9plesequencealignmentusingPhylogeny-awareProfiles”Nguyen,Mirarab,andWarnow.GenomeBiology,2014.Purpose:highlyaccuratelarge-scalemul9plesequencealignments,eveninthepresenceoffragmentarysequences.

UsesanensembleofHMMs

Simpleidea(notUPP)

•  Selectrandomsubsetofsequences,andbuild“backbonealignment”

•  ConstructaHiddenMarkovModel(HMM)onthebackbonealignment

•  AddallremainingsequencestothebackbonealignmentusingtheHMM

RNASim

0.00

0.05

0.10

0.15

0.20

10000 50000 100000 200000

Tree

Erro

r (FN

Rat

e) Clustal−OmegaMuscleMafftStarting TreeSATe2PASTAReference Alignment

PASTA:Mirarab,Nguyen,andWarnow,JComp.Biol.2015–  SimulatedRNASimdatasetsfrom10Kto200Ktaxa–  Limitedto24hoursusing12CPUs–  Notallmethodscouldrun(missingbarscouldnotfinish)

PASTA:evenbeFerthanSATé-2Star9ngtreeisbasedonUPP(simple):oneprofileHMMforBackbonealignment

•  Selectrandomsubsetofsequences,andbuild“backbonealignment”

•  ConstructaHiddenMarkovModel(HMM)onthebackbonealignment

•  AddallremainingsequencestothebackbonealignmentusingtheHMM

Thisapproachworkswellifthedatasetissmallandhaslowevolu9onaryrates,butisnotveryaccurateotherwise.

One Hidden Markov Model for the entire alignment?

OneHiddenMarkovModelforthebackbonealignment?

HMM1

Or2HMMs?

HMM1

HMM2

HMM1

HMM3 HMM4

HMM2

Or4HMMs?

m

HMM2

HMM3

HMM1

HMM4

HMM5 HMM6

HMM7

Orall7HMMs?

UPPAlgorithmicApproach

1.  Selectrandomsubsetoffull-lengthsequences,andbuild“backbonealignment”

2.  Constructan“EnsembleofHiddenMarkovModels”onthebackbonealignment

3.  AddallremainingsequencestothebackbonealignmentusingtheEnsembleofHMMs

Evalua9on•  Simulateddatasets(somehavefragmentarysequences):–  10Kto1,000,000sequencesinRNASim–complexRNAsequenceevolu9onsimula9on

–  1000-sequencenucleo9dedatasetsfromSATépapers–  5000-sequenceAAdatasets(fromFastTreepaper)–  10,000-sequenceIndeliblenucleo9desimula9on

•  Biologicaldatasets:–  Proteins:largestBaliBASEandHomFam–  RNA:3CRWdatasetsupto28,000sequences

RNASimMillionSequences:treeerror

FromNguyenetal.,GenomeBiology2015

Using 12 processors: •  UPP(Fast,NoDecomp)

took 2.2 days,

•  UPP(Fast) took 11.9 days, and

•  PASTA took 10.3 days

0.0

0.2

0.4

0.6

0 12.5 25 50% Fragmentary

Mea

n al

ignm

ent e

rror

PASTA UPP(Default)

(a) Average alignment error

0.0

0.2

0.4

0 12.5 25 50% Fragmentary

Del

ta F

N tr

ee e

rror

PASTA UPP(Default)

(b) Average tree error

Figure S32: Alignment and tree error of PASTA and UPP on the fragmentary 1000M2datasets.

80

Performanceonfragmentarydatasetsofthe1000M2modelcondi9on

UPPisveryrobusttofragmentarysequences

Underhighratesofevolu9on,PASTAisbadlyimpactedbyfragmentarysequences(thesameistrueforothermethods).UPPcon9nuestohavegoodaccuracyevenondatasetswithmanyfragmentsunderallratesofevolu9on.

FromNguyenetal.,GenomeBiology2015

●

●

●

●

0

5

10

15

50000 100000 150000 200000Number of sequences

Wal

l clo

ck a

lign

time

(hr)

● UPP(Fast)

UPPRunningTime

Wall-clock9meused(inhours)given12processors

FromNguyenetal.,GenomeBiology2015

OtherApplica9onsoftheEnsembleofHMMs

SEPP(phylogene9cplacement,Mirarab,Nguyen,andWarnowPSB2014)

TIPP(metagenomictaxoniden9fica9on,Nguyen,Mirarab,Liu,Pop,andWarnow,Bioinforma9cs2014)

HIPPI(proteinclassifica9onandremotehomologydetec9on),toappearRECOMB-CG2016andBMCGenomics2016(Nguyen,Nute,Mirarab,andWarnow)

Objec9ve:Distribu9onofthespecies(orgenera,orfamilies,etc.)withinthesample.

Forexample:Thedistribu9onofthesampleatthespecies-levelis:

50% speciesA

20% speciesB

15% speciesC

14% speciesD

1% speciesE

AbundanceProfiling

TIPPpipeline

Input:setofreadsfromashotgunsequencingexperimentofametagenomicsample.

1.  AssignreadstomarkergenesusingBLAST2.  Forreadsassignedtomarkergenes,perform

taxonomicanalysis3.  CombineanalysesfromStep2

Highindeldatasetscontainingknowngenomes

Note:NBC,MetaPhlAn,andMetaPhylercannotclassifyanysequencesfromatleastoneofthehighindellongsequencedatasets,andmOTUterminateswithanerrormessageonallthehighindeldatasets.

FromNguyenetal.,Bioinforma9cs2014

“Novel”genomedatasets

Note:mOTUterminateswithanerrormessageonthelongfragmentdatasetsandhighindeldatasets.

FromNguyenetal.,Bioinforma9cs2014

ProteinFamilyAssignment

•  Input:newAAsequence(mightbefragmentary)anddatabaseofproteinfamilies(e.g.,PFAM)

•  Output:assignment(ifjus9fied)ofthesequencetoanexis9ngfamilyinthedatabase

HIPPI

•  HIerarchicalProfileHMMsforProteinfamilyIden9fica9on

•  Nguyen,Nute,Mirarab,andWarnow,toappearRECOMB-CGandBMC-Genomics2016

•  UsesanensembleofHMMstoclassifyproteinsequences

•  TestedonHMMER

Pre

cis

ion

.97

.98

.98

.99

.75 .80 .85 .90 .95 1.00

Seq Length: Full

.96

.97

.98

.99

.50 .60 .70 .80 .90

Recall

Seq Length: 50%

.92

.95

.98

.20 .40 .60 .80

Seq Length: 25%

Method HIPPI HMMER BLAST HHsearch: 1 Iteration HHsearch: 2 Iterations

Toappear,Nguyenetal.,BMCGenomics

FourProblems•  Phylogene9cPlacement(SEPP,PSB2012)

•  Mul9plesequencealignment(UPP,RECOMB2014andGenomeBiology2014)

•  Metagenomictaxoniden9fica9on(TIPP,Bioinforma9cs2014)

•  Genefamilyassignmentandhomologydetec9on(HIPPI,RECOMB-CG2016andtoappearBMCGenomics2016)

Aunifyingtechniqueisthe“EnsembleofHiddenMarkovModels”(introducedbyMirarabetal.,2012)

Summary•  UsinganensembleofHMMstendstoimproveaccuracy,foracostofrunning

9me.Applica9onssofartotaxonomicplacement(SEPP),mul9plesequencealignment(UPP),proteinfamilyclassifica9on(HIPPI).Improvementsaremostlyno9ceableforlargediversedatasets.

•  Phylogene9cally-basedconstruc9onoftheensemblehelpsaccuracy(note:thedecomposi9onsweproducearenotclade-based),butthedesignanduseoftheseensemblesiss9llinitsinfancy.(Manyrela9velysimilarapproacheshavebeenusedbyothers,includingSci-PhyandFlowerPowerbySjolander)

•  Thebasicideacanbeusedwithanykindofprobabilis9cmodel,doesn’thavetoberestrictedtoprofileHMMs.

•  Basicques9on:whydoesithelp?

Acknowledgments

PASTA:SiavashMirarabandNam-phuongNguyenUPP:Nam-phuongNguyen,SiavashMirarab,andKeerthanaKumar(undergrad)TIPP:Nam-phuongNguyen,SiavashMirarab,BoLiu,andMihaiPopHIPPI:Nam-phuongNugyen,MikeNute,andSiavashMirarab.NSFgrants:•  ABI-1458652(mul9plesequencealignment)•  III:AF:1513629(metagenomics–withMaryland)Othersupport:GuggenheimFounda9on,MicrosopResearchNewEngland,NSFDEB:0733029,NSFDBI:1062335,DavidBrutonJr.CentennialProfessorship,theUniversityofAlberta(Canada),GraingerFounda9on(atUIUC),UIUC;HHMI(toSiavashMirarab)ComputaKonalanalysesperformedonTACCandBlueWaters

Nguyen et al. Genome Biology (2015) 16:124 Page 6 of 15

Table 2 Average alignment SP-error, tree error, and TC score across most full-length datasets

Method ROSE RNASim Indelible ROSE CRW 10 AA HomFam HomFam

NT 10K 10K AA (17) (2)

Average alignment SP-error

UPP 7.8 (1) 9.5 (1) 1.7 (2) 2.9 (1) 12.5 (1) 24.2 (1) 23.3 (1) 20.8 (2)

PASTA 7.8 (1) 15.0 (2) 0.4 (1) 3.1 (1) 12.8 (1) 24.0 (1) 22.5 (1) 17.3 (1)

MAFFT 20.6 (2) 25.5 (3) 41.4 (3) 4.9 (2) 28.3 (2) 23.5 (1) 25.3 (2) 20.7 (2)

Muscle 20.6 (2) 64.7 (5) 62.4 (4) 5.5 (3) 30.7 (3) 30.2 (2) 48.1 (4) X

Clustal 49.2 (3) 35.3 (4) X 6.5 (4) 43.3 (4) 24.3 (1) 27.7 (3) 29.4 (3)

Average !FN error

UPP 1.3 (1) 0.8 (1) 0.3 (1) 1.8 (1) 7.8 (2) 3.4 (2) NA NA

PASTA 1.3 (1) 0.4 (1) <0.1 (1) 1.3 (1) 5.1 (1) 3.3 (1) NA NA

MAFFT 5.8 (2) 3.5 (2) 24.8 (3) 4.5 (3) 10.1 (3) 2.3 (1) NA NA

Muscle 8.4 (3) 7.3 (3) 32.5 (4) 3.1 (2) 5.5 (1) 12.6 (3) NA NA

Clustal 24.3 (4) 10.4 (4) X 4.2 (3) 34.1 (4) 3.5 (2) NA NA

Average TC score

UPP 37.8 (1) 0.5 (2) 11.0 (3) 2.6 (2) 1.4 (1) 11.4 (1) 47.3 (1) 40.3 (3)

PASTA 37.8 (1) 2.3 (1) 48.0 (1) 5.4 (1) 2.3 (1) 12.1 (1) 46.1 (2) 50.0 (1)

MAFFT 31.4 (2) 0.4 (2) 7.8 (4) 0.6 (3) 0.7 (2) 12.1 (1) 45.5 (2) 46.9 (2)

Muscle 9.8 (3) <0.0 (2) 18.3 (2) 2.7 (2) 0.7 (2) 10.5 (2) 27.7 (4) X

Clustal 5.7 (4) 0.2 (2) X 3.1 (2) 0.1 (2) 11.8 (1) 38.6 (3) 31.0 (4)

We report the average alignment SP-error (the average of SPFN and SPFP errors) (top), average !FN error (middle), and average TC score (bottom), for the collection offull-length datasets. All scores represent percentages and so are out of 100. Results marked with an X indicate that the method failed to terminate within the time limit(24 hours on a 12-core machine). Muscle failed to align two of the HomFam datasets; we report separate average results on the 17 HomFam datasets for all methods and thetwo HomFam datasets for all but Muscle. We did not test tree error on the HomFam datasets (therefore, the !FN error is indicated by “NA”). The tier ranking for each methodis shown parenthetically

memory error message were marked as failures. Forexperiments on the million-sequence RNASim dataset,we ran the methods on a dedicated machine with 256GBof main memory and 12 cores until an alignment wasgenerated or the method failed. We also performed a lim-ited number of experiments on TACC with UPP’s internal

checkpointing mechanism, to explore performance whentime is not limited. All methods other than Muscle hadparallel implementations and were able to take advantageof the 12 available cores.On full-length datasets (Table 2) where nearly all meth-

ods were able to complete, PASTA was nearly always in

Table 3 Average alignment SP-error and tree error across fragmentary datasets

Method ROSE NT RNASim 10K Indelible 10K CRW

(16S.3 and 16S.T)

Average alignment SP-error

UPP 8.3 (1) 11.8 (1) 2.7 (1) 16.1 (1)

PASTA 25.2 (2) 47.7 (4) 8.8 (2) 23.3 (2)

MAFFT 32.5 (3) 25.5 (2) 51.3 (3) 24.5 (3)

Muscle 35.3 (4) 82.2 (5) 77.6 (4) 70.6 (5)

Clustal 62.0 (5) 35.0 (3) X 46.7 (4)

Average !FN error

UPP 1.9 (1) 3.1 (1) 2.5 (1) 7.4 (2)

PASTA 25.2 (3) 21.9 (3) 9.0 (2) 8.2 (2)

MAFFT 18.0 (2) 6.2 (2) 35.6 (3) 2.5 (1)

Muscle 27.5 (4) 43.6 (5) 45.2 (4) 30.1 (3)

Clustal 47.8 (5) 26.3 (4) X 37.4 (4)

We report the average alignment error (top) and average !FN error (bottom) on the collection of fragmentary datasets. Clustal-Omega failed to align any of the Indelible10000M2 fragmentary datasets and thus we mark the results with an X. The tier ranking for each method is shown in parentheses

.75

.80

.85

.90

.95

1.0

.25 .50 .75 1.0.75

.80

.85

.90

.95

1.0

.25 .50 .75 1.0.75

.80

.85

.90

.95

1.0

.25 .50 .75 1.0

.75

.80

.85

.90

.95

1.0

.25 .50 .75 1.0.75

.80

.85

.90

.95

1.0

.25 .50 .75 1.0.75

.80

.85

.90

.95

1.0

.25 .50 .75 1.0

.80

.90

1.0

.00 .25 .50 .75 1.0

.80

.90

1.0

.00 .25 .50 .75 1.0

.80

.90

1.0

.00 .25 .50 .75 1.0

.80

.90

1.0

.00 .25 .50 .75 1.0

.80

.90

1.0

.00 .25 .50 .75 1.0

.80

.90

1.0

.00 .25 .50 .75 1.0

.70

.80

.90

1.0

.00 .20 .40 .60 .80

.70

.80

.90

1.0

.00 .20 .40 .60 .80

.70

.80

.90

1.0

.00 .20 .40 .60 .80

.70

.80

.90

1.0

.00 .20 .40 .60 .80

.70

.80

.90

1.0

.00 .20 .40 .60 .80

.70

.80

.90

1.0

.00 .20 .40 .60 .80

Pre

cisi

on

Avg. Pairwise Sequence Identiy

> 30% 20−30% < 20%

Size

: 0−

10

0S

ize: >

10

0S

ize: 0

−1

00

Size

: > 1

00

Size

: 0−

10

0S

ize: >

10

0

Sequence

Length

: Full

Sequence

Length

: 50%

Sequence

Length

: 25%

Recall

Method HIPPI HMMER BLAST HHsearch: 1 Iteration HHsearch: 2 Iterations

1. Pre-processing seed alignments

.

.

.

.

Family A

Family B

Family N

.

.

i) Compute an ML tree on each seed alignment

ii) Build ensemble of HMMs for each seed alignment, using its ML tree

HMM A1

HMM B1

HMM N1

HMM A2

HMM A3

HMM N2

HMM N5

HMM N4

HMM N3

iii) Collect HMMs into database

HMM A1 HMM A2HMM A3

HMM B1

......HMM N1HMM N2

HMM N5

HMM N4HMM N3

2. Classification of query sequences

HMM A1 HMM A2HMM A3

HMM B1

......HMM N1HMM N2

HMM N5

HMM N4HMM N3

HMM A1 = 8.9 HMM A2 = 7.3HMM A3 = 9.4HMM B1 = 5.6......HMM N1 = 4.4HMM N2 = 5.6

HMM N5 = 6.6HMM N4 = 4.7HMM N3 = 4.3

1) Family A = 9.4

8) Family B = 5.6 ......

2) Family N = 6.6 ......

Query sequence

HMM database

i) Score query sequence against all HMMs in database, keeping only scores above inclusion threshold

ii) Rank families by best scoring HMM within family

iii) Assign query sequence to top ranking family

Family A

Query

HMM database

Scientific challenges: •  Ultra-large multiple-sequence alignment •  Alignment-free phylogeny estimation •  Supertree estimation •  Estimating species trees from many gene trees •  Genome rearrangement phylogeny •  Reticulate evolution •  Visualization of large trees and alignments •  Data mining techniques to explore multiple optima •  Theoretical guarantees under Markov models of evolution

Techniques: machine learning, applied probability theory, graph theory, combinatorial optimization, supercomputing, and heuristics

The Tree of Life: Multiple Challenges

RelatedResearch

•  Historicallinguis9cs,1994-present•  Absolutefastconvergingmethods1997-2002•  Phylogene9cnetworks,2003-2005•  Genomerearrangements,2000-2006•  Mul9plesequencealignment,2009-present•  Supertreemethods,2009-present•  Metagenomicanalysis,2014-present•  Coalescent-basedspeciestreees9ma9on(2011-present)•  Proteinclassifica9onandremotehomologydetec9on,2015-present

MostMSAmethodsdegradeinaccuracywithevolu9onarydistances,datasetsize,and/orfragmenta9on.

PASTAandSATéusedivide-and-conqueranditera9ontoimproveaccuracyandscalabilityofbaseMSAmethods.Theyhaveexcellentaccuracy,but(liketheirbasemethods)areimpactedbyfragmenta9on.

Bydesign,HMMsarelessimpactedbyfragmenta9on,butsingleHMMsarenotasaccurateasensemblesofHMMs.

UPPusesanEnsembleofHMMstoimproveaccuracycomparedtoasingleHMM,andismuchmoreaccuratethanallothermethodstestedinthepresenceoffragments.

Summarysofar