Click  to  edit  Master  /tle  style  

Pi2alls  in  Performance  Tes/ng  June  2015  

Wouter  Soubry  

1  #AxxesCC  

Why  are  we  here?  

2  #AxxesCC  

3  

Load  Tes/ng  Failures  –  Taxonweb.be  

#AxxesCC  

4  

Load  Tes/ng  Failures  –  healthcare.gov  

#AxxesCC  

 Technical  point  of  view:   Single  user  performance  Acceptable  slowdown  under  load/stress  

Not  only  websites!  Every  project  should  consider  performance  explicitly  

5  

What  is  (applica/on)  performance?  

#AxxesCC  

hWp://survey.neuronsparks.be  

 hWp://splunk.neuronsparks.be/en-­‐US/app/search/speed_survey    

6  

Experiment  

#AxxesCC  

 “The  site  needs  to  be  sufficiently  fast”   “Page  must  load  in  4s”   PDF  genera/on  should  not  take  >  30s   The  site  must  be  able  to  handle  1000  users  Trading  system  must  be  able  to  do  1000  TPS  

7  

Performance  Requirements  

#AxxesCC  

Pi2all  1  Wrong,  ambiguous  or  non-­‐existent  

performance  requirements  

8  #AxxesCC  

Specific:  200  concurrent  users,  scenario  walkthrough,  4s  @  95  percen/le  response  all  pages  upon  first  /me  visit,  pc  on  dsl  Measurable:  at  the  client  side,  with  preload  Achievable:  ‘heavy’  page  in  200ms?   Relevant  

Above  the  fold  load  /me   “Onload”  event  versus  speed  index  

 Time-­‐bound:  golive,  gradual  adop/on,  campaign  start  

9  

“SMART”  Performance  Requirements  

#AxxesCC  

 Throughput,  load  envelope   Response  /me   User  behaviour  (think  /me,  flow)   Device,  connec/on   Context:  /cket  sale,  tax  deadline  

10  

Aspects  of  Performance  Reqs  

#AxxesCC  

Staging   Produc/on  

11  #AxxesCC  

Pi2all  2  Linear  extrapola/on  

12  #AxxesCC  

Scalability  needs  to  be  built-­‐in  (and  tested)  What  is  the  uncertainty  of  your  expected  load?   Is  scalability  important?  How  fast  can  you  provision?   Business  impact  of  out  of  spec  performance?  

 

13  

Avoiding  Surprises  

#AxxesCC  

14  

Invite  your  Friends?  

#AxxesCC  

15  

Tooling  

#AxxesCC  

16  

L’Embarras  du  Choix  

ApacheBench  ?#AxxesCC  

17  

Tooling:  Cost  /  ROI  

#AxxesCC  

/  

Pi2all  3  A  fool  with  a  tool  is  s/ll  a  fool  

(Grady  Booch)  

18  #AxxesCC  

 Simple  >  complex   Understand  what  the  tool  is  doing  

What  do  the  metrics  mean  What  is  measured  exactly  

 Tools  are  no  subs/tute  for  thinking,  but…  Good  repor/ng  helps  Intui/ve  interface++  

Find  similar  use  cases  –  how  did  they  do  it?  

19  

Choose  Your  Weapon  Wisely  

#AxxesCC  

 Simple  >  complex   Demo  with  ApacheBench  +  nmon  

20  

Choose  Your  Weapon  Wisely  

#AxxesCC  

21  

Load  Test  Recording  

#AxxesCC  

Proxy  Interceptor  

JMeter  recording  demo  

Generate  load  Measure  end-­‐user  perf   “RUM”/synthe/c  

 Transac/on  breakdown   Code-­‐level  profiling  Infrastructure  monitoring   Business  analy/cs  

22  

Tool  Categories  

}  APM  

#AxxesCC  

SaaS  ‘cloud’  load  genera/on  Blazemeter,  loadstorm,…  

 APM  suites     New  Relic  AppDynamics  Dynatrace   …  

 Big  (machine)  data  Splunk,  Loggly,  logentries,  Sumologic,  Logstash/elk  

 Monitoring  infrastructure  23  

Tool  Convergence  

#AxxesCC  

 Domain  knowledge   System  administra/on   Networking  exper/se   Database  adminstra/on   Applica/on  stack  knowledge:  frameworks   Tools  knowledge   Performance  tes/ng  knowhow  (+  some  sta/s/cs)  

24  

Needed  Skills  

#AxxesCC  

25  #AxxesCC  

Pi2all  4  Not  leveraging  in-­‐house  exper/se  

26  #AxxesCC  

 Large  bank  online  security  overhaul   Compliance  log  subsystem  

27  

A  Tale  from  the  Trenches  

#AxxesCC  

Pi2all  5  Premature  or  late  op/miza/on  

28  #AxxesCC  

End-­‐user  performance  sufficient?  

Transac/on  drilldown  

Iden/fy  &  eliminate  boWleneck  

29  

Chasing  the  BoWleneck  

#AxxesCC  

 Week  before  golive   Performance  tes/ng  completed,  green  light  Func/onal  freeze   Cri/cal  bugfixes  only  Launch  goes  

30  

Recipe  for  Disaster  

#AxxesCC  

Pi2all  6  No  con/nuous  profiling  

31  #AxxesCC  

Regression  tes/ng  performance   CI  tools:  Jenkins,  Atlassian  Bamboo,  TFS,  …   Webpagetest/sitespeed.io   Single-­‐user  performance   Performance  budget   Speed  index  

 Demo:  Jmeter/Jenkins  

32  #AxxesCC  

 Integra/on  environment  Not  produc/on  representa/ve  Regression  detec/on  Rule  out  startup  effects  

Iden/fy  cri/cal  measurements   HTTP  APIs   DB  transac/ons   XML  processing  

33  

Jenkins  and  JMeter  

#AxxesCC  

Pi2all  7  No  proper  configura/on  

management  

34  #AxxesCC  

Annota/on  overlays  on  metrics  hWp://play.grafana.org/dashboard/db/annota/ons    What  to  annotate:  

Deploys  Config  changes  Anything  with  poten/al  performance  impact  

Newrelic  deployments  demo  

35  

What  introduced  the  problem?  

#AxxesCC  

 It’s  about  the  user.  The  User.   Tools:  use  what  you  know  best,  simpler  =  beWer   Monitoring:  develop  a  (healthy)  data  addic/on  Config  management  hygiene   Knowledge:  stand  on  the  shoulders  of  giants  

36  

Key  Takeaways  

#AxxesCC  

Thank  You