![Page 1: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/1.jpg)
DICEHorizon2020Research&InnovationActionGrantAgreementno.644869http://www.dice-h2020.eu
FundedbytheHorizon2020FrameworkProgrammeoftheEuropeanUnion
TowardstheFormalVerificationofData-IntensiveApplicationsThrough
MetricTemporalLogicICFEM2016,TokyoNov17th,2016
FrancescoMarconi1,MarcelloM.Bersani1,Madalina Erascu2 andMatteoRossi11DEIB, Politecnico di Milano, Italy
2Institute e-Austria Timisoara and West University of Timisoara, Timisoara, Romania
![Page 2: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/2.jpg)
Roadmap
§ ContextandMotivation• Data-IntensiveApplications• StreamingDIAs• Qualityissues
§ OurApproach• FormalModel• DecisionProcedure• Implementedtool:D-VerT
§ Conclusions• ExperimentalAnalysis• Futureworks
2
![Page 3: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/3.jpg)
CONTEXTANDMOTIVATION
TowardstheFormalVerificationofData-IntensiveApplicationsThroughMetricTemporalLogic
3
![Page 4: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/4.jpg)
DICEProject
o Horizon2020Research&InnovationAction(RIA)§ Quality-AwareDevelopmentforData-Intensiveapplications§ Feb2015- Jan2018,4MEurosbudget§ 9partners(Academia&SMEs),7EUcountries
4
![Page 5: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/5.jpg)
Data-IntensiveApplications(DIAs)
o Needtoprocessdatabeing§ Massivelylargeinsize§ Complex§ Rapidlychanging
o DevotemostoftheirprocessingtimetoI/O,movementandmanipulationofdata.
o Relyonso-called"Bigdatatechnologies”
5
![Page 6: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/6.jpg)
TheBigDataLandscape
o HeterogeneousTechnologies§ NoSQL,Spark,Hadoop/MapReduce,Storm,CEP,...
o Lackofstandardmethodologiesfordevelopmentandqualityanalysiso Differentproblemsfordifferent“kinds”ofDIA
§ Batchprocessing,streamprocessing,…o WedecidedtofocusonasubsetofDIA
§ streamingapplications
6
![Page 7: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/7.jpg)
StreamingApplications
o SpecialcaseofDIAso Needtoprocessan(almost)continuousflowofinformation
§ Streamà unboundedsequenceoftuples(messages)
o Usuallydescribedbymeansofatopology§ Graphofcomputationscomposedof
• input nodes(sourceofdatastreams)• computational nodesàmanipulatedatastreams
o Calculate,Filter,Aggregate,Join,Talktodatabases,etc
7
![Page 8: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/8.jpg)
QualityIssuesinStreamingDIAso Importantrequirementsforstreamingapplications
§ Latency§ Throughput
o Criticalpoints§ incorrectdesignoftimingconstraints§ nodefailures
o mightcause§ Highlatencyinprocessingtuples§ Memorysaturation
8
latency
throughput
![Page 9: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/9.jpg)
Questions
o Howcanweanalyzeandverifythepresenceofthesekindsofquality(safety?)issues?§ Which(applicationdependent)propertiescouldweverify?
§ Associatedtowhichtechnology?§ Howcanwemodelthesystemandtheproperties?§ Howcanweautomatetheverification,providinga“userfriendly”supporttoDIAdesigners?
9
![Page 10: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/10.jpg)
Stateoftheart
o Formalverificationofdistributedsystemsisamajorresearchareainsoftwareengineering
o FewworkstryingtoaddressformalverificationinthecontextofDIA§ Mainfocusonverifyingapplication-independentpropertiesrelatedtospecificframeworks
• ReliabilityandloadbalancingofMapReduce• ValidityofmessagingflowinMapReduce
§ nomodelingandverificationofapplication-dependent properties
10
![Page 11: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/11.jpg)
PROPOSEDSOLUTION
TowardstheFormalVerificationofData-IntensiveApplicationsThroughMetricTemporalLogic
11
![Page 12: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/12.jpg)
OurApproacho Focusonaspecificsetoftechnologies
§ Topology-basedstreamingapplicationso Identifyqualityissueso Selectareferencetechnologyà ApacheStormo Deviseaformalmodel
§ Allowingtocapturemeaningfulsystembehaviorandproperties
§ Havinganappropriatelevelofabstraction§ Usingaformalismthatenablesautomaticverification
o Defineatool-supportedmechanismforformalverification§ Startingfromhighlevelapplicationdescription
• Initialversion:JSONformat• Currentversion:annotatedUMLClassdiagram
12
✅
✅
![Page 13: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/13.jpg)
ApacheStorm
o OpenSourceDistributedStreamProcessingSystemo Analytics,LogEventprocessing,etc..o Reliability,at-least-onesemanticso Wideadoptioninproductiono InStormtopologies
§ Sourcenodescalledspouts§ Computationalnodescalledbolts
13
![Page 14: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/14.jpg)
Modelingchoices– 1/2o Allowingforthedefinitionoftopologiesinacompositionalway
§ Formalizebehaviorofspoutsandbolts§ Usethemasbuildingblocksfortopologies
o Abstractingaway§ Deploymentdetails§ Messagecontents§ Multi-layeredmessagebuffers
14
boltspoutSpout Bolt
Bolt
Spout Bolt
![Page 15: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/15.jpg)
Modelingchoices– 2/2
o Relevantfeaturesmodeledforeachcomponent§ evolutionofthestates§ timingconstraints§ evolutionofitsmessagebuffer(inputqueue)
o Propertiestoverify§ “allboltqueueshaveaboundedoccupationlevel”
15
ParallelismFunctionality Proc_timeQueue_threshold:
avg_emit_rate:emit_amount:
![Page 16: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/16.jpg)
Timedcounternetworksmodelo FormalmodelbasedonCLTLoc enrichedwithcountersdescribing:
§ stateevolutionofcomponents
§ timingconstraints
§ quantitiesoftuplesmovingthroughoutthetopology
16
⊆ CLTLoc✅
à ⊆CLTLoc✅
⊈CLTLoc🚫
àLTL
àcounters!
![Page 17: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/17.jpg)
TimedcounternetworksmodelVerifyingtheproperty
o Weformulatedthepropertycheckasasatisfiability problem§ BoundedSatisfiabilityChecking(BSC)
o Goal§ Findanultimatelyperiodic traceviolatingboundednessproperty
• Havingtheform𝛼(𝑠𝛽))
• 𝛼à prefix• 𝑠𝛽à suffixrepeatableinfinitelymanytimes(loop)
o Rationale§ Ifthereisagrowingtrendintheloopà unboundedincreaseadinfinitum
17
![Page 18: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/18.jpg)
Decidabilityissues
o CLTLoc1,2§ SATis decidable anddefined overtimed words§ Computed through Bounded Satisfiability Checking (BSC)§ Implemented procedurebased onSMT3
• UsingZot formal verification tool
o Decidabilityresultscannot beextendedtoCLTLoc +counters• ContainsCLTLoverquantifier-freePresburger formulae4
o Wedefinedapartialassessmentmethodtoguaranteethesoundnessofthesatisfiabilityoutcome.
18
1. AToolforDecidingContinuosTimeMetricTemporalLogic,Bersani,Rossi,SanPietro,20132. AnSMT-basedapproachtosatisfiabilitycheckingofMITL,Bersani,Rossi,SanPietro,20133. ConstraintLTLSatisfiabilityCheckingwithoutAutomata,Bersanietal.,20124. TheeffectsofboundingsyntacticresourcesonPresburger LTL.Demri,Gascon,2006
![Page 19: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/19.jpg)
DecisionProcedure
o Given§ CLTLoc +countersformula§ aboundk
o Trytobuildastructure𝛼𝑠𝛽𝑠 with 𝛼𝑠𝛽𝑠 = 𝑘§ Ifstructureisnotfound(UNSAT)
• Noultimatelyperiodicmodelsoflength<=kexist§ Ifstructureisfound(SAT)
• Performtheassessmenttodetermineitsextensibilitytoinfinitemodel𝜶 𝒔𝜷 𝝎
o Ifchecksucceedsà outcomeisSAT(𝛼𝑠𝛽 iscounterexample)o Ifcheckfailsà spuriousresult,mustlookforanotherstructure
19
![Page 20: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/20.jpg)
Assessmentmethod
o Providessufficientconditionforextendingadinfinitumboundedassignmentofvaluestocounters
o Intuitively,itchecksifintheloopthevalueofeachvariable𝑦 hasthesameshape§ Itmightdifferbyanon-negativeoffsetΔ2
20
s𝛽(loop1)s𝛽(loop2)
α
time
![Page 21: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/21.jpg)
D-VerT – DICEVerificationToolInitialversion(April2016)
21
![Page 22: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/22.jpg)
D-VerT - DICEVerificationToolCurrentVersion
22
https://github.com/dice-project/DICE-Verification/wiki
![Page 23: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/23.jpg)
Experimentalresults
o Validationthroughopenandclosedsourceusecases§ Meaningfulqualitativeresultsinidentifyingcriticalpointsintopologydesign
§ Executiontimestronglydependsonthesizeofthetopologyandontheconfigurationsofsinglecomponents
23http://dice-project.github.io/DICE-Verification/
![Page 24: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/24.jpg)
Usecase:FocusedCrawlerTopologyUMLDesign
24
o TypicalusageexampleofStormo Fetchingandindexingofmediaitemso Fromwebsources
![Page 25: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/25.jpg)
Usecase:FocusedCrawlerTopologyOutputtrace
25
![Page 26: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/26.jpg)
CONCLUSIONS 26
![Page 27: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/27.jpg)
Wrapup
o Approachfortheautomatedverificationoftopology-baseddata-intensiveapplications.§ Definitionofaformalmodel(TCN)
• ExtendingCLTLoc metrictemporallogicwithdiscretecounters
• Enablingautomaticverificationofsafetyproperties
§ Definitionofatool-supportedmechanism• Toautomaticallygenerateformalmodelsfromhighlevelapplicationdescriptionandrunverification
§ Definitionofsufficientconditionsforguaranteeingthesoundnessoftheverificationresults
27
![Page 28: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/28.jpg)
Futureworks
o Identificationandverificationoffurtherproperties
o Modelingdifferenttechnologies§ Spark,CEP,…
o Newresultsonthecorrectnessandcompletenessoftheanalysisofcounternetworks
o Toolandmodelimprovements
28
![Page 29: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/29.jpg)
Thankyou
29
![Page 30: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/30.jpg)
Starting formalism:Constraint LTLoverclocks- CLTLoco Extension of LTLwith TAclocks,where formulae are
§ Propositions (lightOn,lightOff,buttonOn,buttonOff)§ Constraints over clocks (c=0,c<1,…)§ LTLformulae
• X(φ)• φ U ψ
o CLTLoc1,2§ SATis decidable anddefined overtimed words§ Computed through Bounded Satisfiability Checking (BSC)§ Implemented procedurebased onSMT3
• UsingZot formal verification tool
1. AToolforDecidingContinuos Time Metric Temporal Logic,Bersani,Rossi,SanPietro,20132. AnSMT-basedapproachtosatisfiabilitycheckingofMITL,Bersani,Rossi,SanPietro,20133. ConstraintLTLSatisfiabilityChecking without Automata,Bersaniet al.,2012
buttonOn ->X(lightOn U buttonOff)
![Page 31: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/31.jpg)
CLTLoc +counters
o V isafinitesetofvariables overℕo Cisafinitesetofclockvariables overℝo AP isafinitesetofatomicpropositionso 𝜃 areQFPformulaeoverterms𝛼 ≔ 𝑦|𝑋𝑦 where𝑦 ∈ 𝑉
o CLTLoc withcountersformulaearedefinedasfollows:
𝜙 ≔ 𝑝 𝑥~𝑐 𝜃 𝜙 ∧ 𝜙 ¬𝜙 𝑋𝜙 𝑌𝜙 𝜙𝑈𝜙 𝜙𝑆𝜙o where:
§ 𝑝 ∈ 𝐴𝑃, 𝑥 ∈ 𝐶, 𝑐 ∈ ℕ, ~ ∈ <,=§ X,Y,U,SaretheusualLTLoperators.
31
![Page 32: Towards the Formal Verification of Data-Intensive Applications …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/2018/02/ICF… · Towards the Formal Verification of Data-Intensive](https://reader034.vdocuments.site/reader034/viewer/2022042803/5f4b1755591247709a62eae7/html5/thumbnails/32.jpg)
Relatedformalisms
o Timedcounternetworksaremainlyinspiredfrom:§ VectorAdditionSystemswithStates(VASS)
• Subclassofcountersystems• Lossy VASSà takeintoaccountnumberofmessages,nottheirorder
• Onlytheoreticalanalysis,donotenableautomaticverification• Timedcounternetworksallowtospecifytimingconstraintsviaclocks
§ TimedPetriNets• Transitionsfiringwithurgentsemantics• Firingconditionsandnumberoftokenconsumedexpressibleinaquiterigidway
• Forourmodelweneededmoreflexibilityo Possible occurrenceofeventso Expressslightlymoreelaboratefiringconditions
32