![Page 1: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/1.jpg)
AssessmentofFMandFM/TBMmodelinginCASP13
LucianoA.AbriataandMatteoDalPeraro
LaboratoryforBiomolecular Modeling– LBMInstituteofBioengineering,SchoolofLifeSciencesEcole Polytechnique Fédérale deLausanne- EPFL
![Page 2: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/2.jpg)
Acknowledgements
• CASP13(andCASP12) organizers– Andriy,John,Torsten,Krzysztof,MayaandAnna!– PredictionCenter– Previousassessorsandallpredictors
§ Greatandconstructiveexperience§ Excitingtowitness2consecutivehugetransformationsinproteinstructureprediction
![Page 3: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/3.jpg)
CASP13tertiarystructuretrack:32FM&13FM/TBM(+4FM-special)
DetailsonclassificationbyLisaKinch &Andriy Kryshtafovych
#groups/servers:107/39#models(1/5):7542/35982
![Page 4: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/4.jpg)
Part1:EU-specificevaluations
![Page 5: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/5.jpg)
StrategyforEU-specificevaluations
TargetEUs,modelsandtablesfromthePredictionCenter
~halfofthemodelsforinitialinspection,representedbytopGDTTSmodels
Clusteringat3Å
WebAppforinteractivenavigationofmodelclusters:6mainscores,othersavailabletoo
JS+HTML
Designationofbestcluster(s) foreachEU
VisualInspection
Model(s)designatedbestforeachEU
Furtherevaluationofmodelsinbestcluster,ifworth
Part1:EU-specificevaluationsChin-HsienTai,Hongjun Bai,ToddJ.Taylor,andByungkook Lee*Proteins 2013
![Page 6: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/6.jpg)
CASP12-likewebapp:facilitatesassessment,andiseasilyopenedtothepublichttp://lucianoabriata.altervista.org/papersdata/casp12fmassessment/casp12-fm-fmtbm-assessment-3Aclusters.html
NEW:morescores,showserversindistinctcolor,andbuiltauxiliarywebappsalsoformodels
clusteredat1Å andforanalysiswithnosplitting
Part1:EU-specificevaluations
![Page 7: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/7.jpg)
Clearbest
Theselookbestbut
theyaren’t
Unclearbest,visualinspectionofmultiplemodels
Verybad
Verygood
Excellent
Part1:EU-specificevaluations
Examplesofcorrelationplots
à GDTTS&QCSturnouttobethetwomostinformativescores,inourexperience*ForQCSseeCongetalBioinformatics2011
![Page 8: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/8.jpg)
Manygoodmodels
Alwayshighr
Differentscoresproposedifferentbestmodels
GDTTS&QCSindeedgroupedseparatelyinanalysisbyOlechnovic etal.Bioinformatics2018
Examplesofcorrelationplots
![Page 9: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/9.jpg)
TopGDTTS TargetTopQCS,secondGDTTS
HHscore 13.98LGA 73.5
Neff/LHHblits 0.01
T0991-D1(FM)
TS366_3(topbyDFM,
designatedbest)
Part1:EU-specificevaluations
Importanceofguidingvisualassessmentbymultiplescores
GDTTS 37.4QCS 68.8DFM 0.82
![Page 10: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/10.jpg)
Target
T1010-D1(FM)
TS117_1
T0990-D3(FM)
Target TS043_1(designatedbest)
HHscore 2.76LGA 23.8Neff/LHHblits 0.2
HHscore 0.69
LGA 39.5
Neff/LHHblits 0.07
Part1:EU-specificevaluations
Severalveryhardtargetswithfoldscaptured
GDTTS 50QCS 80
GDTTS 50QCS 80
![Page 11: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/11.jpg)
OnlytwoverydifficultEUswithnobestmodel
T0981-D2(FM)
Allscoreslow;heremodelofhighestGDTTSlooksreasonablebutismissingthelast
strandwhichisseparatedinsequence.Andmodelsthatarecompletearetoobad…
T0989-D2(FM)
LongextendedN-terminusandC-terminalbetahairpin,noneiswellpositioned;butthecentral
betasheetisquitegoodinsomemodels.
HHscore 14LGA 63.8Neff/LHHblits 0.07 HHscore 4.9LGA 55.1Neff/LHHblits 0.01
Part1:EU-specificevaluations
![Page 12: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/12.jpg)
ImpactofprogressinCASP13:
Examplesof“FM-special”targetsforwhichfullmodelswereverygood
Part1:EU-specificevaluations
![Page 13: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/13.jpg)
Example:T0953s2(D1:FM/TBM,D2&D3:FM)
TS117_4(TopbyTM,2.53ÅRMSDover61%ofsequence)
TS224_3(TopbyGDTTS)TargetbyEU(D1,D2,D3)
![Page 14: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/14.jpg)
Example:T1000(D1:TBMnoteval.,D2:FM)TS043_1(TopbyGDTTS,scoresquitegoodbyallmetrics)
GDTTS 69.5QCS 90.7
Partsmissinginexp targetstructure
There’sanx-raystructureofD1
89%res<2Å
![Page 15: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/15.jpg)
NotableprogressinCASP13:
12hardEUsthatreachednearatomisticresolutionbymanygroups
Part1:EU-specificevaluations
![Page 16: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/16.jpg)
![Page 17: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/17.jpg)
T0968s2-D1(FM)
TS043_1-D1(12models)
2.33Åoverfullsequence(115residues)
HHscore 19LGA50Neff/LHHblits 1.23
GDTTS80&QCS90
T0970-D1(FM/TBM)
TS043_2-D1(5modelsplus4fromTS347)
2.78Åover89%ofsequence(total96residues)
HHscore 17LGA67 GDTTS80&QCS90Neff/LHHblits 1.61
T1001-D1(FM)
TS222_4-D1(106models)
2.32Åoverfullsequence(139residues)
HHscore 11LGA55Neff/LHHblits 0.04GDTTS74&QCS93
T1008-D1(FM/TBM)
TS281_1-D1(126models)
1.14Åoverfullsequence(77residues)
HHscore 61LGA74Neff/LHHblits 0.01GDTTS91QCS95
NMR/MD
Part1:EU-specificevaluations
![Page 18: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/18.jpg)
Part2:Rankings
![Page 19: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/19.jpg)
RankingbasedonZ-scoresofGDTTS&QCS
Ranking=sumZ-scorescombinedfromGDTTS&QCS(asthesearebyfarthetwomostinformativescorestoguidevisualassessment)onallmodelssubmittedas#1,forTBM/FM,FMandFM_sptargetEUs,andconsideringsumofZ-score>-2.
Rankingisveryrobust:scoreswithGDTTSonlyorQCSonlyreturnthesametopgroups.
Allgroups
Servers
Part2:Ranking
![Page 20: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/20.jpg)
Notablehighlights:groupsnotintop5whoprovidedtheonlybestmodelsforsometargets(uponvisualevaluation)
• ZHOU-SPOTforT0998-D1:alone&quitebetterthanrunners-up
• Jones-UCLforT1010-D1:alone&quitebetterthanrunners-up
• RaptorX-DeepModeller forT0949
• KIAS-GdanskforT0957s1-D1
• BAKERforT0975-D1
• Venclovas forT0991-D1Part2:Ranking
QCS78
QCS81
![Page 21: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/21.jpg)
Part3:Progress
![Page 22: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/22.jpg)
ProgressinFreeModeling(FM/TBMnotconsidered)
Notes:- ExactdefinitionofFMEUsmightvaryfromyeartoyear- CASP12andCASP13EUsofroughlyofsimilardifficulty
Median+/- MedianDeviationforGDTTSofbestmodels
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12
GDT
_TSofallbe
stm
odels
19941996 1998200020022004200620082010201220142016
BoxplotsofGDTTSdistributionsforbestmodels
10
15
20
25
30
35
40
45
50
55
60
1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
Med
ianGDT
_TS
Year
Coevolution-basedcontactprediction
methodsinliterature
Med
ianGDT
TSofb
estm
odels
CASP13
Globalanalyses2- Progress
MachineLearningformolecularmodeling
![Page 23: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/23.jpg)
Possiblesourcesofimprovement:alignmentdepth,existingtemplates,domainsize?
![Page 24: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/24.jpg)
Possiblesourcesofimprovement:alignmentdepth,existingtemplates,domainsize?
CASP12
• FromCASP12toCASP13significantimprovementinperformance
• Dosomepredictorshaveaccesstospecial,closemetagenomicsdatabases?
CASP13
![Page 25: Assessment of FM and FM/TBM modeling in CASP13 · 2018-12-12 · GDTTS69.5 QCS90.7 Parts missing in exptarget ... Part 2: Ranking QCS 78 QCS 81. Part 3: Progress. Progress in Free](https://reader033.vdocuments.site/reader033/viewer/2022042003/5e6eab96b4750c6903001fdf/html5/thumbnails/25.jpg)
KeyconclusionsfromCASP13onthetertiarystructurepredictiontrack
• Yetanothersignificantimprovementinpredictionquality,mainlyduetotheriseofmachinelearningmethodscombinedwithcoevolution-basedcontactprediction
• ReachingnearlyatomisticresolutionofthebackboneforsomeverydifficultEUs(<150residues)bymanygroups!
• PredictionsaresogoodthatsplittingEUsisinsomecasesnotnecessary
• AlignmentdepthallowsforbettertopmodelsthaninCASP12,butnowseemtoneedlowernumbersofsequences
• TemplatesofpoorsequencesimilaritymightbebetteridentifiedthaninCASP12
• Remaininglimitations:domainsizeandalignmentdepth