Download - Sibos 2017: Disruptive functional testing - the next frontier in post-trade systems verification
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com1
Disruptive Functional TestingThe next frontier in Post-Trade systems verification
Open Access Quality Assurance & Related Software Development for Financial Markets
Tel: +7 495 640 2460, +1 415 830 38 49
www.exactpro.com
16 October 2017, SIBOS ConferenceIosif Itkin, Exactpro CEO and co-founder, London Stock Exchange Group
• A specialist firm focused on functional and non-functional testing of systems that process
wholesale financial products, particularly market infrastructures
• A UK company with operations in the US and four QA & software development centres in
Russia
• Part of London Stock Exchange Group as of May 29, 2015
• Incorporated in 2009 with 10 people, our company has experienced significant growth as
satisfied clients require more services; now employing 536 specialists
Front office
Listed Derivatives CRM
Dark PoolT25 Clearing
FXFixed income
SEFReconciliation
Onboarding
Ticker PlantCommondities
Smart Order Router
Trading DesktopsIndex Dissemination
FPGA
EquitiesConnectivity
ExchangeMTF
Swaptions
Exactpro Timeline at a Glance
Delivering Quality to London Stock Exchange Group
Some of our projects (start date):
•Turquoise (2010)
•Borsa Italiana (2011)
•CCP.A (2011)
•FPGA-base Ticker Plant (2012)
•LSE (2012)
•Oslo Bors (2012)
•UnaVista (2013)
•FTSE (2013)
•EuroTLX (2014)
•Russell (2015)
•IDEM (2015)
•LSEDM (2015)
•ELITE (since 2015)
•CurveGlobal (2016)
•GATElab (2016)
•LCH (2016)
•TRADEcho (2016)
•MTS (2017)
•RNS (2017)
Open Access Quality Assurance – a Global Client Network
Faster, not Bigger
What is the main difference between incumbent and disruptive?
Incumbent means something you already know and expect.
Disruptive is when we encounter new things that will re-define what we already know.
Post Trade Systems
Software Testing Challenges
With incumbent testing, you are on the hook of pre-conceptions of how your system under test is supposed to work.
It's only when you do disruptive testing that you really gain new knowledge about the system and learn from it.
Get Off the Hook
Software Testing is Relentless Learning
Defect Mining
The goal of disruptive testing is to find defects that are unlikely to be found if testing is done with other methods.
We see ourselves as a mining operation extracting defects hidden in complex scalable technology platforms.
Test Harness
Agile Transformation
As in any other large organization in the financial services industry, technology used at the London Stock
Exchange Group is going through agile transformation.
Developing a Test Harness for Agile Process
What Agile Development Should Be… and Not
NOT LIKE THIS
LIKE THIS
Testing Challenges in Agile
Managing Complex Systems
The Law of Requisite Variety:
a control system must have at least
as many possible states as the
system it wants to control.
It means that Test Harness
development is a truly challenging
and continuous software
development process.
Test Harness Team
Testing Critical Infrastructures
Testing Critical Infrastructures
Safety Rule #1 with Submarines: don't open portholes when underwater!
Testing Critical Infrastructures
Safety Rule #1 with Submarines: don't open portholes when underwater!
Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open
Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force
Testing Critical Infrastructures
Safety Rule #1 with Submarines: don't open portholes when underwater!
Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open
Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force
Disruptive testing:
1) iterate through a huge number of random diverse scenarios under load to prove that it won't open
Testing Critical Infrastructures
Safety Rule #1 with Submarines: don't open portholes when underwater!
Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open
Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force
Disruptive testing:
1) iterate through a huge number of random diverse scenarios under load to prove that it won't open
2) open the porthole
Thank you