the what, why and how of web analytics testing

The  What,  Why  &  How  of    

Web  Analy3cs  Tes3ng  

Anand  Bagmar  So#ware  Quality  Evangelist  

Principal  Consultant,  ThoughtWorks  

Why  do  we  do  Tes3ng?  

What  is  Web  Analy3cs?  

Why  do  we  need  it?  

Web  Analy*cs  Solu*on  Providers  

Adobe  Marke*ng  Cloud  /  Omniture  

SiteCatalyst  

WebTrends  

AWStats  

Google  Analy*cs  

StatCounter  

…  

How  to  test  Web  Analy3cs?  

The  end  report  

Pros  

•  Ensure  report  is  setup  correctly    

Cons  •  Licensing  •  Reports  not  yet  setup  •  Validate  all  requests  are  sent  /  captured  

Tes3ng  at  the  report  level  

At  the  origin!  

The  problem  

Manual  

The  challenge  

Repea3ng  over-­‐and-­‐over  again  

The  solu3on  

What  does  WAAT  do  for  me?  

• Minimal  changes  to  exisFng  test  framework    •  Does  not  depend  on  junit  /  testng  /  …  • Web  AnalyFc  tool  independent  •  UI  Driver  framework  independent  •  Browser  independent  

Flavors  

Architecture  

•  Original  flavor  •  Source  code  +  jar    •  Supports  3  plugins    •  Omniture  Debugger  

•  HUpSniffer  •  JsSniffer  

•  Available  on  github  

WAAT  -­‐  Java  

•  Ruby  gem  implemented  over  WAAT-­‐Java  

•  Uses  RJB  –  Ruby-­‐Java-­‐Bridge  •  Supports  2  plugins  •  HUpSniffer  •  JsSniffer  

•  Available  on  github  and  rubygems.org  

WAAT  -­‐  Ruby  

Omniture  Debugger  

Omniture  Debugger  How  does  it  work?    Pros  •  OS  independent  •  Run  using  the  regular  test-­‐user    Cons  •  Browser  dependent  –  need  to  implement  

ScriptRunner  for  the  UI-­‐driver  in  use  •  Web-­‐AnalyFc  soluFon  dependent  –  Adobe  MarkeFng  

Cloud  /  Omniture  SiteCatalyst    

HHpSniffer  How  does  it  work?    Pros  •  Web-­‐analyFc  soluFon  independent  •  Browser  independent  •  UI-­‐driver  independent    Cons  •  3rd  party  libraries  are  OS  Dependent  •  HTTPs  not  supported  out-­‐of-­‐the-­‐box  •  Run  tests  as  ‘root’    

JsSniffer  How  does  it  work?    Pros  •  Web-­‐analyFc  soluFon  independent  •  Browser  independent  •  HTTPs  supported  out-­‐of-­‐the-­‐box  •  No  dependency  on  any  3rd  party  library      Cons  •  Need  to  write  javascript  to  get  the  URL  from  the  

browser  •  UI-­‐driver  dependent  

•  Download  zip  from  github  /  rubygems.org  

•  Setup  

•  jpcap  /  ScriptRunner  setup    

•  Update  exis*ng  tests  

How  to  use  WAAT?  

Update  exis3ng  Tests  •  Import  relevant  packages  /  gems  

•  Define  &  provide  parameter  values  

•  Ini*alize  Engine  

•  Enable  Web  Analy*cs  Tes*ng  

•  Define  Test  Data  

•  Verify  Web  Analy*cs  repor*ng  using  WAAT  

•  Valida*ons  

•  Disable  Web  Analy*cs  Tes*ng  

<Section !!!actionName="OpenUpcomingPage_HttpSniffer"!!numberOfEventsTriggered="1"!

!tagList="pageName=upcoming:index||!! ! !cc=USD||!! ! !ch=upcoming||!! ! !events=event2||"/>!

Define  Test  Data  

Ini3alize  Engine  

!engine = getInstance(!! ! !webAnalyticTool, !! ! !inputFileType, !! ! !keepLoadedFileInMemory, !!! ! !log4jPropertiesAbsoluteFilePath!! ! !);!

Enable  Web  Analy3cs  Tes3ng  

engine.enableWebAnalyticsTesting();!

Verify  Web  Analy3cs  Repor3ng  

!Result verificationResult = !!! !engine.verifyWebAnalyticsData (!! ! !inputDataFileName, !! ! !actionName, !! ! !urlPatterns, !! ! !minimumNumberOfPackets);!

Cart.checkout();!

Sample  Valida3ons  assertNotNull(verificationResult.getVerificationStatus(), "Verification status should NOT be NULL"); !!assertNotNull(verificationResult.getListOfErrors(), "Failure details should NOT be NULL"); logVerificationErrors(verificationResult); !!assertEquals(verificationResult.getVerificationStatus(), Status.PASS, "Verification status should be PASS"); !!assertEquals(verificationResult.getListOfErrors().size(), 0, "Failure details should be empty");!

Disable  Web  Analy3cs  Tes3ng  

engine.disableWebAnalyticsTesting();!

hUp://essenceo#esFng.blogspot.com/search/label/waat  

hUps://github.com/anandbagmar/  

hUp://rubygems.org/gems/WAAT  

Anand.Bagmar@thoughtworks.com                        Abagmar@gmail.com  

   @BagmarAnand