pre-con ed: test data management and compliance: is your test data ready for another regulation
TRANSCRIPT
World®’16
TestDataManagementandCompliance:- IsyourTestDataReadyforAnotherRegulation?BenRiley - PrincipalConsultant- CATechnologies
DO5X05E
DEVOPS
2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.
Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.
ForInformationalPurposesOnlyTermsofthisPresentation
3 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Abstract
This90-minutepresentationwilldiscussthenewGDPRregulationandhowitimpactsyouruseofproductiondataintest.ThissessionwilldemonstratetheuseofCATestDataManagertocreatesyntheticdatafromscratchandhowyoucanavoidtheuseofproductiondataaltogether.
BenRiley
CAPrincipalConsultantPresales
4 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Agenda
GDPREXPLAINED
WHAT'SSOIMPORTANTABOUTTHEGDPR
HAVEIDONEENOUGH
WHO'SAFFECTED
DATABREACHES
STRATEGY&APPROACH
1
2
3
4
5
6
5 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GDPR– ExplainedGeneralDataProtectionRegulation
§ NewEUdataregulationsintendstounifylegislation,sothatasinglesetofrulesappliesacrosstheEU.
§ WhentheGDPR comesintoforceon25May2018,allorganisationsthatprocessthepersonallyidentifiableinformationofEUresidentswillberequiredtoabide byanumberofprovisionsorfacesignificantpenalties.
6 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
What’ssoimportantabouttheGDPR?
§ Whoisitgoingtoaffect?
§ WhatarethePenalties&Sanctions
§ DataBreachRegulation&Process
§ RighttoErasure
§ PrivacybyDesign– SDLCImplications.
GDPR– GeneralDataProtectionRegulation
7 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Whoitappliesto?
§ Non-EUorganisationsthatdobusinessintheEUwithEUdatasubjects'personaldatashouldpreparetocomplywiththeRegulation.– DoyouworkwithE.Ubasedcompany’s?– DoyouhaveacustomerbasefromwithintheE.U?– Doyouhaveastrategytocomplyandisitstrongenough?
GeneralDataProtectionRegulation
8 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
WhatarethePenalties&Sanctions?
§ ToughPenalties– FirstBreach2%GlobalTurnoveror$10,000,000 (Whicheverisfirst)
§ Ifacompanystillfailstocomplytoasecondaudit…– 4% Globalturnoveror$20,000,000
§ Company’swillbegiventimeto‘correct’issueswhichcanbecorrected.RegularAudits willalsobeputinplaceforoffendingorhighriskcompanys.
GeneralDataProtectionRegulation
9 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DataBreachRegulation&Process
§ Theconsentdocumentshouldbelaidoutinsimpleterms.– Silenceorinactivitydoesnotconstituteconsent;clearandaffirmative
consenttotheprocessingofprivatedatamustbeprovided.
§ Datacontrollerswillberequiredtoreportdatabreachestotheirdataprotectionauthority.– Thenoticemustbemadewithin72hoursofdatacontrollersbecoming
awareofit.– Regularsupplychainreviewsandauditswillberequiredtoensurethey
arefitforpurposeunderthenewsecurityregime.
GeneralDataProtectionRegulation
10 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
RighttoErasure
§ Therighttoerasureisalsoknownas‘therighttobeforgotten’.– Thebroadprincipleunderpinningthisrightistoenableanindividualto
requestthedeletionorremovalofpersonaldatawhetherthereisnocompellingreasonforitscontinuedprocessing.§ Wherethepersonaldataisnolongernecessaryinrelationtothepurposeforwhichitwasoriginallycollected/processed.
§ LegalObligations§ Thepersonaldatawasunlawfullyprocessed§ Whentheindividualwithdrawsconsent.
GeneralDataProtectionRegulation
11 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
PrivacybyDesign
§ TheGDPRrequiresthatprivacyisincludedinsystemsandprocessbydesign.– Thismeansthatsoftware,systemsandprocessesmustconsider
compliancewiththeprinciplesofdataprotection.– Theessenceofprivacybydesignisthatprivacyinaserviceorproduct
istakenintoaccountnotonlyatthepointofdelivery,butfromtheinceptionoftheproductconcept.
GeneralDataProtectionRegulation
12 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GDPR– KeyTermsGeneralDataProtectionRegulation
§ PersonalData:– Meansanyinformationrelatingtoanidentifiablenaturalperson– Anidentifiablenaturalpersonisonewhocanbeidentified,directlyor
indirectly
§ CoreIdentifiers– Name,anidentificationnumber,locationdata,anonlineidentifieror
tooneormorefactorsspecifictothephysical,physiological,genetic,mental,economic,culturalorsocialidentityofthatnaturalperson.
13 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GDPR– KeyTermsGeneralDataProtectionRegulation
§ DataBreach– Abreachofsecurityleadingtotheaccidentalorunlawfuldestruction,loss,
alteration,unauthorised disclosureof,oraccessto,personaldatatransmitted,storedorotherwiseprocessed;culturalorsocialidentityofthatnaturalperson.
§ RichardHammond,TopGearPresentercrash2006– Aninvestigationhasbegunafterhospitalstaffwereaccusedofspyingon
RichardHammond'smedicalrecords.– LastJuly,healthbossesvowedtocrackdownoncomputersecurityafteritwas
reportedthatpatients'recordsattheTrust'shospitalwereused"inappropriately"morethan70,000 timesinonemonth.
14 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DataBreaches
§ BrightonandSussexUniversityHospitalsNHSTrust(2010)– TheInformationCommissioner(ICO)endedupimposingafineof
£325,000aftersensitivepatient dataofthousandsofpeoplewasdiscoveredonharddrivessoldoneBay.
§ SonyPlayStationNetwork(2011)– Thelargestdatabreachinhistoryatthetime,Sony’sdisastrous2011
breachsawhackersmakeoffwiththecustomerrecordsof77millionpeoplerelatingtoitsPlayStationNetwork,includingasmallnumberrevealingcreditcardnumber
GDPR
15 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DataBreaches
§ Kiddicare (2016)– ParentingretailerKiddicare hassufferedadatabreachthatexposed
thenames,addressesandtelephonenumbersofsomeofitscustomers.§ WhencustomersstartedreceivingsuspiciousSMStextmessagesaskingthemtotakeanonlinesurveyandaninvestigationeventuallyuncoveredtoerror.
§ Thecompanysaidithademailed794,000peoplewhomayhavebeenaffectedbytheincident.
§ Itsaidthedatahadbeentakenfromaversionofitswebsitesetupfortestingpurposes
GDPR
16 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GDPRimpactonTestDataGDPR
§ SinceusershavetherighttowithdrawconsenttousetheirdataforTestingpurposerightfiltersandprecautionsneedtobeplacedwhilecopyingdatafromproduction– Discoverandidentifyallpersonaldataacrossdatabasesandfileformats– Theuseofpersonaldataforapplicationtestingmustbedisclosedtousersasa
“legitimateinterest,”consentobtainedandthedatadeletedwhentestingisfinished.
– Toreduceriskassociatedwithbreachesmask,personaldatawhenitsbeingusedforlowerenvironments
17 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GDPRStrategy+TestDataManagementWhattodo..
§ ThisRegulationdoesnotapplytothepersonaldataofdeceasedpersons.
§ Imposeaproperaudit,process&strategicinitiativeacrosstheorganizationtoimbedkeyprincipals.
18 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Atypicalenvironmentspace+issuesProduction
MainframeCoreapplicationHub
DistributedWorld
Oracle
SQLServer
RelationallyComplex
HighlyIntegrated
Legacy
BigData
Sensitive
Unknownstructures
Hard.
TestingEnvironments
MainframeCoreapplicationHub
DistributedWorld
Oracle
SQLServer
FullDBCopies
ProductPIIData
Lesssecurethan
production
19 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataManagement
§ WhereismyPIIhiding?– Isitsecure?
§ DoIhavetheapplicationknowledgetofindit?
§ AmIconfident,thatifauditedIwouldpass?
DataProfiling
21 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataManagement
CATDMcontainsdatasamplingandadiscoveryfunctionalityforprofilingdata.§ Stringdataanalysis:Numericattributes,
suchasdateformatstimeformatsetc.§ Datavalueanalysis:Comparisons
withexistingreferencedata§ Lexicalanalysis:Analyzingpatterns,
characters&columns
HowdoesCATDMapproachdatadiscovery
22 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Whatnext…?DoImaskdataorgeneratenewdata?
ShouldIfixwhatIhave?
ShouldIgeneratenew?
23 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Fixwhatyou’vegot
§ CanwesecureexistingproductiondataforDev/Test?– Yes.Butit’softendifficult.– MaskingIshighlycomplex– oftensomeinformationisleftinasaformofcompromise.– ThedefinitionofPersonalInformationisgrowingincludinganythingrelatedtogenetic,
mental,economic,culturalorsocialidentity
§ Howhardisit?– Howeasyisittomaskallofthiscontent,whileretainingthereferentialintegrityneeded
fortesting?– Canyoureverseengineerdatafromcomplexrelationshipsusingapieceofexternal
information?
DataMasking
24 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Fixwhatyou’vegot
§ CATDMhasahugenumberofmaskingfunctionstosecuretestdata&environments.
§ Werequireapplicationknowledgeandprofilingiskey.– Maskingisgreatwhendonewell.Ifyoumisssomethingyouareintrouble!!
§ CATDMhandlesreferentialintegrity&databasechanges.
DataMasking
25 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
OrGenerateNewData…?
§ Canwegeneratesecuredataintooursystems?– SyntheticTestDataisgeneratedfromscratch.Itcandrawnfrom‘real’
databutdoesn'tneedto.Itisthereforeentirelysecurefortesting.
§ E-Commerce– Lookuparandomproductandpriceinmytargetsystem,andthen
applyittomygenerateditemsandtheordertotal.
§ BankingandFinancialInstitutions– GeneratePaymentsagainstaninvoice.– Extractasubsetofclaimsandtheirhistory,andmergethemintomy
developmentsystem.
SyntheticTestData
26 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GenerateNewData
§ CustomerCare– Createasetofsyntheticcustomerswiththesamecorecharacteristics
asproduction.– Then,create dataforaperson,addressandneworder.
§ CreditCardManagement– Create asetofcreditcardeventstotestmyfrauddetectionengine.
IncludetheexpectedresultFraud/NotFraudinthecommentssectionofthexmlmessagesoIcantestwhetherthecodehasdetectedtheeventscorrectly.
SyntheticTestData
27 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TheHybridSolution
§ AutomatePIIdiscovery.Don’trelyfullyonSME validation.
§ Makesurealldatais“pseudo-anonymized”.– Theonlycaveatforthisisifyoucanprove consentforuse.– ThismeanscorePIIvaluesaremasked,preventingreverseengineering.
§ CreateTemplatedDataSynthesisroutines– Allowyourtesterstochoosewhatdatatheyneed,earlierinthetesting
phase.– Embracethechangeofapproach
Masking+SyntheticTestData
28 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataManagementPainPoints– EnvironmentFocus
§ RefreshofTestDatausingProductionDataisinefficient,intermsoftimeandcost– FullCopiesofproductionareoftenused– AvailabilityofProductionSystemsislimited– IssuesareamplifiedinMainframeenvironments
§ Anyuseofproductiondatarisksnon-compliance– Dataprivacylawsbeingstrengthened(e.g EuropeanGDPR)– Branddamageandpotentialfinesfordatabreachrepresenta
significantbusinessrisk– Testenvironmentsareinherentlylesssecurethanproduction
29 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataManagementPainPoints– Testing/QualityFocus
§ Thedataisnotofsufficientqualityfortesting– Productiondataisdrawnfrombusinessasusualscenarioswhich
typicallyhaslimitedcoverageoftheteststhatneedtoberun– Itisnotfitbyitsverynatureasitdoesnotcontain“baddata”,future
scenarios,unexpectedresultsandoutliers
§ Toomuchmanualeffort– Manualdatacreationisprohibitivelyslow,andscriptsorworkbooks
havetobeupdatedeachtimeachangeismade
§ Testersspendupto50%oftheirtimemanuallyfinding/creatingdatatomatchtestrequirements/criteria
30 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDatashould…
§ Provideastandardsetofdatatotestwith§ Be“production-like”andyetcoverallpossibleteststhatneed
toberun,includingfutureandnegativescenarios§ Containenoughdatatotestwithrepeatedly§ Beup-to-date,whilealsocontainingallpreviousdataasrequired§ Containnosensitivedata§ Beprovisioned‘On-Demand’
– AspartofEnvironmentBuild– Allocated/reservedfortests&testcases
31 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CATDMCapabilities
3 4 521
Createsmall subsets;Identifysensitivedata&replacewithmeaningful
equivalents
Capture theDataModelAnalyse databasedontest
requirements
Generate SyntheticDatatoimprovedataquality
Find/Allocate/Reservedatafortests
RapidlyprovisionFitforPurposeData
On-Demand
33 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
RecommendedSessions
SESSION# TITLE DATE/TIME
DO5X22E Pre-ConEducation:ManagingTestDataAcrossDistributedandMainframeSystems 11/14/2016at10:30am
DO5X21EPre-ConEducation:AnOverviewofHowCATestDataManagerHelpsDeliverRigorouslyTestedSoftwareEarlierandatLowerCost
11/14/2016at1:00pm
DO5X06L TestDataManager- Masking,subsetting andgeneratingsyntheticdata 11/15/2016at09:00am
34 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
MustSeeDemos
ModernizeAppDeliveryIntegratedCDTheater5- DOV501P
DeliverTestDataFasterTestDataManagerTheater5- DOV511P
DeliverBetterAppsServiceVirtualizationTheater5- DOV507P
OrchestrateYourReleaseReleaseAutomationTheater5- DOV513P