NGTC  Conference  (c)  

Service  Virtualiza.on:  Reality  is  Overrated.  

a  CA  LISA  Service  Virtualiza.on  Best  Prac.ce:  Transform  Performance  and  Scale  

Min  Fang  min.fang@ca.com+65  9636  8356  

Agenda  

 •  Constraints:  The  Enemy    of  Agility  •  What  is  Service  Virtualiza.on  •  Capabili.es  of  Service  Virtualiza.on  •  Best  Prac.ce:  Transform  Performance  &  Scale  

Next  Genera.on  Tes.ng  Conference  (c)  

Accelera.ng  Applica.on  Time-­‐to-­‐Market  HOW  DO  WE  INCREASE  “TOUCH  TIME”  AND  REDUCE  “IDLE  TIME”?  

DEVELOPMENT   REGRESSION   QA   INTEGRATION  

PERFORMANCE  UAT  PRODUCTION  

§  Constraints  of  dependent  systems  needed  to  move  the  SDLC  forward  

§  Increasing  Complexity  of  today’s  heterogeneous  enterprise  IT  architectures    

§  Lack  of  CollaboraFon  creates  scheduling  conflicts  and  buffers  from  development    through  opera.ons  

§  Need  Complete  visibility  into  what  happens  in  produc.on  

CHALLENGES  

Bridging  the  Gaps  in  DevOps  

RUN  BUILD  PLAN    

DevOps  brings  the  applica.on  porbolio  forward  to  meet  market  challenges    with  collabora.on  and  efficiency  from  design  to  deployment  

ENVIRONMENT    MANAGEMENT  

CONTINUOUS  FEEDBACK  

AGILE  DEVELOP    &  TEST  

SERVICE      VIRTUALIZATION  

RELEASE  AUTOMATION  COLLABORATIVE    

PORTFOLIO  PLANNING  OPERATIONS  MONITORING  

DATA  MINING  

NGTC  Conference  (c)  

Constraints:  The  Enemy  of  Agility    

1.  Insufficient  environment/capacity  for  tes.ng  and  development  

Copyright  ©  2010  CA.  All  rights  reserved.  

SOAP  

AppServer  

UAT  Tester  

100%  u.liza.on  

MainFrame  Performance  Tester  

Developer  

2.  Uncontrollable  environment  growth  

Copyright  ©  2010  CA.  All  rights  reserved.  

SOAP  

UAT  Tester   AppServer1  

MainFrame  AppServer2  

AppServer3  

Dev  Team1  

SOAP  

SOAP  

Training  

Keep  adding  MIPS  to  keep  pace  with  new  requirements  

SYSTEM  UNDER  TEST  

           

3.  System  Dependencies  =  Development  &  tes.ng  constraint  

MainFrame  Frontend  Tester/  Developer  

1.  Upgrading  MQ  version   2.  Timesharing  

Mainframe  

ESB   3.  ESB  Team  will  only  complete  development  in  2  

weeks  .me  

NGTC  Conference  (c)  

What  is  Service  Virtualiza.on  

Service  Virtualiza.on:  How  does  it  work?  

R  Structured Conversations R  Observe R  Understand R  Protocol-Level

§  Recorded traffic

§  Design specs

§  Sample RR pairs

§  Transaction logs

§  Manual Creation

§  Byte-Code

R  Heuristics R  Analytics R  Algorithms R  State

R  Sophisticated Behavior R  Dynamic Properties (Dates,

values, etc.) R  Scenario Support R  Test Data Sanitation R  Compiled Model vs. Stubs R  Automatic Healing

CAPTURE   PROCESS   MODEL  

JDBC  

MQ  

       Source              ApplicaFon  

ERP  

Database  

Mainframe  

Legacy  

SOAP  

§ Automa.cally  capture  realis.c  behavior  § Op.mize  models  as  desired  

Capture    

Create  Virtual  Services  

Captures  and  Models  Realis.c  Virtual  Services  

LISA  Virtualizes  en.re  test  environments  to  eliminate  costly  constraints  

W  W  W  .  I  T  K  O  .  C  O  M  

Virtual  Test  Environments  

JDBC  MQ  

VS1  DB1  

VS2’  

IT  resources  now  available  on  your  terms,  

accelera.ng  .me-­‐to-­‐market  

and  team  produc.vity,  allowing  24/7  

tes.ng    

SOAP/HTTP          Source              ApplicaFon  

ERP  

Database  

Mainframe  

Legacy  

§  Mainframes  &  Mainframe  Components    

§  Risk  Solu.on  Providers  

§  Credit  Bureau      

§  SMS  Gateway  Providers  

§  Teleconference  Service  Provider  

§  Card  Processing  Services  

§  Bill  Payment  Services  

§  Travel  System  (GDS)  

CICS-­‐CICS  

§  Plaborms  

§  SOA  Protocols  

MQ  DB2  

SOAP   JMS   JDBC  HTTP   JAVA  MQ  

UDDI  XML  

CORBA  REST  .NET  

CA  LISA  Virtual  Service  Environments    Customers  can  virtualize  almost  any  IT  asset    

NGTC  Conference  (c)  

How  Can  Service  Virtualiza.on  Help?  

Virtual  Service  

Model  

Design  arFfacts,  transacFon  logs,  

WSDL  files,  flat  files,  and  others  

Source  App  

1.  Model  Applica.on  Under  Development    

Virtual  service  can  be  modeled  from  design  ar.facts.  Allowing  development  and  integra.on  test  to  start  earlier    

2.  Using  SV  for  training  environment  (Use  Client  without  Backend)  

AppServer   MainFrame  

AppServer  

Virtualize  

Virtual  Service  Environment                        

3.  Virtual  Data  Management  -­‐  For  Development  &  Tes.ng  

Copyright  ©  2010  CA.  All  rights  reserved.  

Applica.on  Server  

Abu  Bakar  IC:  1234  Acc:  M123  

Alvin  Koh  IC:  4567  Acc:  456  

Arun  Kailasam  IC:  54354  Acc:D4345  

You  have  full  control  over  data  in  virtual  service,  you  can  add  your  data  directly  into  the  virtual  service  for  the  purpose  of:  a)  Scenario-­‐based  tes.ng  b)  Nega.ve  tes.ng  

3.  Virtual  Data  Management  -­‐  Create  or  Modify  virtual  data  directly  

1.  If  AccountID=  =1313094687566198   2.  Return  user  

details  of  JACKY  LEE  (exis.ng  customer)  

3.  Nega.ve  Tes.ng:  How  do  my  applica.on  handle  -­‐10k  as  account  

balance  

100  tps  (Virtual  

Mainframe)  

4.  Using  SV  for  Performance  Engineering  

SOAP  

AppServer  

1.  Interface  load  tes.ng   2.  Load  tes.ng  

without  u.lizing  mainframe  MIPS  

Test  

Test  

Test  

Virtualize  

3.  At  the  same  .me,  understand  the  txn/sec  requirement  and  to  

size  your  mainframe  more  accurately    

1000  VU  (Interface)  

Aligning  to  CIO  Goals  Do  More  with  Less    

0  

0.5  

1  

1.5  

2  

2.5  

3  

3.5  

4  

Year  0   Year  1   Year  2   Year  3   Year  4   Year  5   Year  6   Year  7  

Normal  

Number  of  Test  Environments  Number  of  Training  Environments    Complexity  in  the  environments  

Halt  growth  by  using  LISA  

3Y  TCO:  Reduce  overall  future  footprint  by  50%  or  more  

5Y  TCO:  Reduce  overall  future  footprint  by  70%  or  more  

CIO  Goal  accomplished!    CONTROL  costs  

#1  consumer  retail  bank  of  America  

eliminated  $100M  in  hardware  

requirements  when  using  LISA  over  5  

years.    Over  10x  ROI,  6x  in  year  1  

NGTC  Conference  (c)  

Best  Prac.ce:  Transform  Performance  &  Scale  

In  the  .me  it  takes  to  blink  an  eye—400  milliseconds  to  be  exact,  online  AND  mobile  users  will  have  begun  to  abandon  your  applica.on  

When  customers  “struggle”  with  an  app…  

You   Your  compeFtor  

Abandon  the  transac.on,  try  later  on  computer  

43%  

Become  more    likely  to  prefer    a  compe.tor  

Log  a  complaint  with  customer  service  

Go  directly  to  a  compe.tor  

16%   14%   12%  

(Harris  Interac.ve  Mobile  Transac.ons  Survey  for  TeaLeaf,    March  2011)  

Performance  will  make  or  break  an  applica.on!.  

When  problems  occur:  

•  Revenue  is  lost.  

•  Produc.vity  decreases.  

•  Customers  leave.  

•  The  longer  it  takes  to  find  the  defect,  the  more  it  Costs  to  fix.  

 

Development  Phase   Cost  of  Fixing  RaFo  

Requirements   1  

Design   3  -­‐  6x  

Coding   10x  

Development  Tes.ng   15  -­‐  40x  

Acceptance  Tes.ng   30  -­‐  70x  

Opera.on/Produc.on   40  -­‐  1000  

80%  of  the  TCO  of  an  applica.on  is  directly  related  to  finding  and  fixing  problems.  (Source:  “The  Economic  Impacts  of  Inadequate  Infrastructure  for  So<ware  Tes>ng,  NIST)  

The  result:  on  average,  a  performance  issue  costs  an  organiza.on  at  least  $45,000/hour.    (Source:  Mul>ple  industry  analysts  have  published  es>mates  ranging  from  EMA’s  $45,000/hr  cost  to  Yankee  Group’s  and  Meta’s  es>mate  of  over  $90,000/hr  or  more  depending  on  the  industry  and  nature  of  the  failing  applica>on.)  

Service  Virtualiza.on:  Where  would  I  use  this?  

Service  Virtualiza.on  

Eliminate  Capacity  Constraints  Major  North  American  Bank  Performance  &  Test  CoE  

Business  Problem  

•  13  performance,  dev,  test  teams  forced  to  .meshare  on  mainframe-­‐based  services  

•  Cost  of  addi.onal  environments  for  L&P  lab  and  the  movement  to  Agile  development/tes.ng  

•  Difficult  to  perform  load  tes.ng  due  to  system  dependencies  

•  Only  2  of  5  major  releases  were    performance  tested    •  Offshore  teams  unable  to  complete  off  hours  

performance  tes.ng  due  to  availability  issues  

Business  Impact  

•  Virtualize  in  8  days  what  took  2  years  of  custom-­‐coded  stub  development  to  create.    

•  Virtualize  new  systems  in  1  hour  with  business  analysts  versus  weeks  of  custom  coding  

•  Virtualized  environments  enabled  all  5  major  releases  plus  5  minor  releases  to  include    L&P  tes.ng  

•  Over  $30  million  in  hardware  cost  avoidance  •  Offshore  teams  performing  300%  more  L&P  tes.ng  due  to  

system  availability  

Or  similar  heavy-­‐weight  implementa.on  environment  

Shared  Mainframe  Service    

   

VSE  

     

VSE  

     

VSE  

Before   A{er  

Avoid  costly  transacFon  fees  when  tesFng  third-­‐party  services    Leading  Global  Airline    

Challenge  §  Test  and  valida.on  of  reserva.on  booking  system  was  incurring  $1  to  2  

million  in  monthly  fees  to  access  live  third-­‐party  services  

SoluFon  using  LISA  Virtualize  §  LISA  simulated  the  behavior  of  third-­‐party  reserva.on  systems  as  virtual  

services  that  were  available  24/7  without  usage  fees  §  No  longer  needed  to  coordinate  test  schedules  or  mock  transac.on  data  

sets  with  third-­‐party  service  providers  §  Provided  freedom  for  development  and  QA  teams  to  test  earlier  in  the  

so{ware  lifecycle  

Client  Results  §  Reduced  service  usage  fees  to  save  over  $12M  annually  §  Lower  tes?ng  costs  by  using  stable,  virtualized  data  sets  §  Higher  quality  by  encouraging  earlier  and  more  frequent  tes.ng  

“We  rely  heavily  on  LISA  to  automate  the  tes?ng,  configura?on  and  virtualiza?on  aspects  of  our  test  and  development  environment.”  -­‐  Lead  Architect  

Customer  Success  Story  

Example:  Performance  Tes.ng  Middle-­‐Tiers  Using  LISA’s  Virtual  Service  Environments  (VSEs)  

   

Frequent  Flyer  System  

Booking  business  service  

PNR    Service  

Ticke.ng  Applica.on  

Integra.on  Bus  (ESB)    

SOAP/HTTP(S     SOAP/JMS  

XML/MQ  

XML/MQ  

LISA  Test    

LISA  Test    

UI  Load  Test  Tools  

USERS:    Agents  /  Adjusters  

Reserva.on  Service    

 GDS  ROI  LISA  

Test    

Isolate,  test  and  tune  components  by  virtualizing  dependent  services    

19.0s  Conven.onal  load  tes.ng  can  

only  expose  overall  performance  .me  

Slow  responding  transac.on  or  component  

pinpointed  with  LISA  

LISA    VSE    

LISA    VSE    

LISA    VSE    

3.3s  

2.5s  

2.5s  

10.7s  

LISA  manages  Test  data  

Enabling  High  Performance  with  Constrained  Resources  &  Time    Leading  P&C  Insurance  Fund    

Challenge  §  Enterprise  wide  “go-­‐live”  of  cri.cal  quo.ng  system  across  50  states  §  Performance  Tes.ng  lags  could  not  be  pinpointed  with  UI  tes.ng  §  Middle-­‐.er  and  3rd  party  credit  systems  too  cri.cal/costly  to  test.  

SoluFon  using  LISA  Virtualize  &  Test  §  Simulated  Web,  IBM  MQ/ESB,  Cloud/3rd  party  Credit  service,  and  Oracle  

apps  with  Virtual  Services  for  24/7  inexpensive  tes.ng  §  Drive  50K  user+  load  tests  directly  against  isolated  components    §  “Prove  It”  quick  ROI  project  with  iTKO  engineers  and  SI  partner    

Client  Results  §  Went  from  test  overruns  to  unlimited  Virtualized  performance  test  access,  

and  star.ng  next  phase  4  months  early  §  Delivered  ROI  within  one  month  with  skilled  resources  §  Improved  test  bed  availability  by  64%  and    increased  number  of  tests  run  by  10x  

“When  you  can  do  isola?on  at  a  component  level  against  the  performance  budget,  you  can  iden?fy  the  weak  link  in  the  chain  and  address  it..”  -­‐  Sr.  Director,  IT  Architecture  &  Performance  

Customer  Success  Story  

Best  Prac.ce:  Decompose  Service  Levels  Increasing  efficiency  by  building  up  to  meet  a  “Performance  Budget”  

Order"

Lookup"

Price"Solution"

Test: "4.0"

Seconds"

Without  component-­‐level  performance  tesFng:  

Isolated  component-­‐level  performance  tesFng  miFgates  risk:  

Timeline 8 mo. 9 mo. PROJECT DUE!

Try adding more test Hardware!!"

Test"3.2

Seconds"

"SLA

2.1 seconds"

Order"

Lookup"

Price"

Solution"

0.5s" 0.4s"

0.5s"

1.2s"

"Test:"

2.1 Seconds"

0.5s"

1.2s"1.9s"

0.4s"

Use VS to isolate & tune components separately from dependencies!

Still not within SLA!!$!$!

Aggregate solution tuned for performance budgets!

Produc.on  vs.  Test:  in  produc.on  

online  banking  shared  infrastructure  

datacenter  –  Kuala  Lumpur    

online  banking  shared  infrastructure  datacenter  –  Singapore  

mobile  banking  shared  infrastructure  datacenter  –  Singapore  

 

4G  SingTel  Singapore  

3G  Celcom  Kaula  Lumpur  

2.5G  Telkomsel  Jakarta  

Mobile  TransacFons   Singapore  

 Kaula  Lumpur  

Jakarta  

Internet  TransacFons  

web  service    

web  service    transac.ons  

online  banking  shared  infrastructure  datacenter  –  Singapore  Simulated  TransacFons  

Simulated  web  services  

online  banking  shared  infrastructure  datacenter  –  Singapore  

mobile  banking  shared  infrastructure  datacenter  –  Singapore  

 

4G  SingTel  Singapore  

3G  Celcom  Kaula  Lumpur  

2.5G  Telkomsel  Jakarta  

TransacFons  with  real    world  network  condiFons  

Singapore  

 Kaula  Lumpur  

Jakarta  

virtual  locaFons  

Load    Generator  

Load    Generator  

Load    Generator  

without  network  virtualizaFon  

with  network  virtualizaFon  

ProducFon  vs.  Test:  in  test  

RealisFc  Mobile/Web/Internet  Network  

CondiFons  

Xyz  Bank  iPad  Edi.on  Mobile  App  Summary  Analysis    

Time  KBs  Data  

0  

10  

20  

30  

40  

50  

60  

70  

80  

7.1  

20.7   17.2  12.3   11.7   8.2  

14.6  

46  

32.8  

20.5   22.4   19.1  21.3  

72.2  

44.6  

27  33.2   31.9  

WiFi  

3G  

3G  Busy  

Performance  Op.miza.on  -­‐  once  bo}lenecks  iden.fied  than  solu.on  is  to  op.mize  the  applica.on.  

References  

Source  :  CA  Press  Release  |  ServiceVirtualiza.on.com  

§  Service  Virtualiza.on  Site  Home  Page  :    

h}p://servicevirtualiza.on.com/profiles/blog/list?tag=best+prac.ces    §  The  Book  -­‐  Service  Virtualiza.on:  Reality  is  Overrated.  

h}p://www.amazon.com/Service-­‐Virtualiza.on-­‐Reality-­‐Is-­‐Overrated-­‐ebook/dp/B009HAKER8/ref=dp_kinw_strp_1    

§  Ge�ng  Started  with  Service  Virtualiza.on  h}p://servicevirtualiza.on.com/video/ge�ng-­‐started-­‐with-­‐service-­‐virtualiza.on-­‐sv-­‐minute-­‐2012    

§  LISA  Introduc.on  –  What  Does  LISA  Do  ?  h}p://www.youtube.com/watch?v=2Nl69c3MLkQ  

   

MIN  FANG  MIN.FANG@CA.COM  

+65  9636  8356  

NGTC  Conference  (c)  

Thank You!