test automation with vectorcast · emenda • helping embedded software development teams since...

TestAutomationwithVectorCAST

Emenda

• Helpingembeddedsoftwaredevelopmentteamssince2006

• Maintechnologies:staticanddynamicanalysistools• KlocworkandVectorCAST

• Servicesaroundtoolstoestablishprocessesandhelpteamsmeetdemandingcertificationstandards(ISO,IEC,EN,etc)

• OfficesinEurope,US,China

TestAutomation

• AutomatedRepeatableTests

• Manybenefits,suchas:• Fasterfeedback• Reducedmanualwork• Increasedsystemshelflife

• Unit&Integrationisoneapproach,especiallyusefulforlow-levelsystems

Aquickpoll

• Whocurrentlyperformstestautomation?

• Whoisconsideringtestautomation?

• Whowouldwant(orlike)theirautomatedteststohavecontext(e.g.requirements-based)

Unit&IntegrationTesting

• Sourcefilesarereferredtoas“units”andcanbeintegratedtoperformcomponentormoduletests

• Unittestsverifysingleunits.Integrationtestsverifyunitstogether

BenefitsofUnit/IntegrationTesting

• Automatedregressiontests• Doesthesystemstillworkaftersoftwarechanges?

• Identifiestargets• Whendoyoustopcoding?

• CodeConfidence(Correctness,Reliability,Robustness)

• VeryhighlyrecommendedforanyStandards(ISO,IEC,EN,etc)

• Manymore…

Whydon’tmorepeopleunit/integrationtest?

• Lackofknowledgeofhowandwhy

• Time-to-Marketorotherpressures

• Seenasaverymanualandtimeconsumingprocess• TrueforCorC++• VectorCAST solvesthis…

IllustrationofaTestHarness

Nextsteps

• Soyoucreatedatestharnesstotestsomesourcefiles…

• Whowillmaintainthesetestcases?Requirementsandcodechange

• Howdoweperformregressiontestingagainstchanges?

• Canwerunthesetestcasesontarget(HardwareinLoop)?

• Howmuchcodecoveragedoweget?

• Etc.Etc.Etc.

EnterVectorCAST

• ATestingFrameworkthatautomates:• CreationofTestHarnesses• ManagementofTestCases• RegressionTesting(e.g.withContinuousIntegration)• ReportingofPassed/FailedTestCases• CodeCoverage• TestCaseGeneration(boundaryvalueanalysis,faultinjection,etc)

• Enables:• Requirements-BasedTesting• TargetTesting(Hardware-In-Loop)• TestDrivenDevelopment

• AndMore…

VectorCAST:CertificationKits

• Overview• RegulatedIndustriesRequireProofofSuitabilityforDevelopmentTools

• Industries• AerospaceDO-178B and DO-178C• Railway EN50128• IndustrialControlsIEC61508• AutomotiveISO26262• Medical/FDAIEC62304

• KitContents• Documentsareabout800totalpages• CompleteToolRequirements• TestCasestoProveRequirementsaremet

TÜVSÜDCertifiedSoftwareToolforSafetyRelatedDevelopment

VectorCASTUsersAutolivAutoeversystemsAIVBekoBoschChang'an AutomotiveDelphiDongfeng AutomotiveDENSOFujitsuHyundaiMagnaPowertrainMagneti MarelliMOTONICSiemensTRWValeoYura

AbbottLaboratoriesAcist MedicalBaxterHealthcareBeckmanCoulterBostonScientificBiosafeCardinalHealthCovidienEnteromedicsGEHealthcareHoanaMedicalHospiraMedicalImpactInstrumentationMedtronicPhilipsMedicalSironaDentalSmithsMedicalTerumoHeartVarian

ABBAlliedTelesisAlcatelAnsaldo SignalAlstomBrunelBombardierCAPGeminiElin EBGElectro-MotiveDieselsGETransportationInvensysRailŠKODAELECTRICSafetran SystemsSiemensTriconexThalesUnionSwitch&SignalVerocel

AmazoneAnalogDevicesArçelikBechtelBettisBertholdTechnologiesBritishEnergyBrunelCaterpillarChangzhouScaleLtd.ClaasEndress+HauserFosterMillerGeneralElectricHAWEHydraulikICSTripexInvensysIngersollRandItronKrauss-MaffeiMettler ToledoAGNetAppOakSystemsSanminaSunwaysViaSatWiloZIEHL

AELSistemasAstriumBoeingBellHelicopterBelcanBAESystemsCessnaAircraftEADSElectricBoatGoodrichAerospaceGeneralDynamicsHamiltonSundstrandHoneywellITTCommunicationsIndiaSpaceResearchKoreanAerospaceLockheedMartinL-3CommunicationsNorthropGrummanPratt&WhitneyRaytheonRockwellCollinsRollsRoyceSaabGroupThalesTurkishAerospaceU.S.AirForceXMobots Robotic

DO-178B|ED-12B

Aerospace

FDA|IEC62304

Medical

ISO26262

Automotive

CENELEC|EN50128

Railway

IEC61508

Industrial

Companieswho…• havesafetyorbusinesscriticalapplications• haveprocesseswhichspecifyrigoroustesting• haveunitandintegrationtestingrequirements• needtheabilitytoautomate regressiontesting• haverequirementstoprovecodecoverage• realizethatNOTtesting…isNOTanoption

TimeforaDemo

Polarion andVectorCASTClosingtheLoop

ClosetheLoopwithPolarion andVectorCAST

Requirements TestSpecification

ChangeManagement Report

RequirementsTraceabilitywithPolarion

CodeTraceabilitywithVectorCAST

ManagedinPolarion

ManagedinPolarion orVectorCAST

Requirements:Polarion

Code:VectorCAST

ManualTestingProcess

• FeedbackLoopTime?• Days,weeks,months?

AutomatedTestingProcess

• FeedbackLoopTime?• Seconds,Minutes

AutomatedTestingwithTestManagement

Kiitos /Tack/Thanks!Questions?“MeettheSpeakers”at15:15