embedded systems in algorithmic trading - peoplealanmi/presentations/... · 2015-03-29 ·...
TRANSCRIPT
![Page 1: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/1.jpg)
Algorithmic Trading and Embedded Systems
Sat Chatterjee ([email protected])
Embedded Systems Week / Oct 15th 2014
![Page 2: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/2.jpg)
IMPORTANT DISCLAIMER AND DISCLOSURE INFORMATION
2
This document has been prepared by the author listed on the first page and represents only the opinions of the author. This document is exclusively for informational and educational purposes only and may not be copied, disseminated or distributed to any other person in any fashion without the express prior written consent of the author and Two Sigma Securities, LLC (“TSS”). Under no circumstances should this document or any information herein be construed as investment research, or as an offer to sell or the solicitation of an offer to buy any securities or other financial instruments, including an interest in any investment fund sponsored or managed by TSS, Two Sigma Investments, LLC, Two Sigma Advisers, LLC, or any other affiliate of TSS (each, a “TSS Affiliate”). Further, this document does not constitute and shall not be construed as an advertisement, or an offer or solicitation for any brokerage or investment advisory services, by TSS or any TSS Affiliate. While the information herein was obtained from or based upon sources believed by the author to be reliable, neither the author nor TSS and its affiliates have independently verified the information and provide no assurance as to its accuracy, reliability, suitability or completeness. All information is provided as of the date referenced on the first page, and neither the author nor TSS and its affiliates have an obligation to update the information herein. In no event shall the author, TSS, any TSS Affiliate, or their respective officers, employees or representatives, be liable for any claims, losses, costs or damages of any kind, including direct, indirect, punitive, exemplary, incidental, special or, consequential damages, arising out of or in any way connected with any information contained herein. This limitation of liability applies regardless of any negligence or gross negligence of the author, TSS, its affiliates or any of their respective officers, employees or representatives.
Sat Chatterjee / Algorithmic Trading
![Page 3: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/3.jpg)
A Little History
1792 1797
1987 Today
3 Sat Chatterjee / Algorithmic Trading What is different about the last picture?
![Page 4: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/4.jpg)
The US Equity Market Cloud
4
This picture is not entirely accurate
![Page 5: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/5.jpg)
What is driving this?
Lower Trading Costs
5 Sat Chatterjee / Algorithmic Trading
![Page 6: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/6.jpg)
Market Participants and Interaction
• Buy-side – retail
– mutual funds
– pensions
– insurance
– non-financial
• Sell-Side – execution brokers
– market makers
– exchanges
– dark pools
Different utility curves, horizons
6
Interact through the order book
Sat Chatterjee / Algorithmic Trading
![Page 7: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/7.jpg)
Lower Market Impact
7 Sat Chatterjee / Algorithmic Trading
![Page 8: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/8.jpg)
Lower Spreads
8 Sat Chatterjee / Algorithmic Trading
(spread data: C. Jones, What do we know about High Frequency Trading?, SSRN 2236201)
![Page 9: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/9.jpg)
9
• More efficient price discovery
• Lower operational expenses
Sat Chatterjee / Algorithmic Trading
![Page 10: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/10.jpg)
What are the technical challenges?
Performance
Correctness
10 Sat Chatterjee / Algorithmic Trading
![Page 11: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/11.jpg)
Very High Message Rates
(source: marketdatapeaks.com)
11
Highest 1-second message rate by minute across a set of data
feeds for US Equities, Futures, and Options (30 Sep 2014)
Sat Chatterjee / Algorithmic Trading
![Page 12: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/12.jpg)
Low Latency: Soft Real-time Systems
12
Order acknowledgement latency at BATS (a US Equity exchange)
Sat Chatterjee / Algorithmic Trading
(source: batstrading.com)
![Page 13: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/13.jpg)
13
Bugs can be very costly in short order
(source: nytimes.com)
Sat Chatterjee / Algorithmic Trading
![Page 14: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/14.jpg)
A Simplified
Algorithmic Trading System
14 Sat Chatterjee / Algorithmic Trading
![Page 15: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/15.jpg)
A Simplified Algorithmic Trading System
Market Data Processor
(MD)
Order Management System (OMS)
Trading Logic
Market Cloud
15 Sat Chatterjee / Algorithmic Trading
![Page 16: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/16.jpg)
Market Data Processing
MD OMS Trading
Logic
Market Cloud
16
Responsibilities: filtering, normalization, summarization, logging
• Performance (volume and spikes, O(1) WCET, partitioning, HW/SW co-design)
• Self-healing (connection issues, fail-over, outlier detection)
• Reliable logging (accurate timestamps, compression, integration with storage)
Sat Chatterjee / Algorithmic Trading
![Page 17: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/17.jpg)
Order Management System
MD OMS Trading
Logic
Market Cloud
17
Responsibilities: risk management, regulatory, (de-)normalization
• Distributed algorithms (borrow allocation, fill allocation, risk checks)
• Correctness (trusted component, formal verification)
• Self-monitoring (resilient to internal and external bugs)
• Performance (order bursts, maximal inline checks, HW/SW co-design)
Sat Chatterjee / Algorithmic Trading
![Page 18: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/18.jpg)
Trading Logic
18
MD OMS Trading
Logic
Market Cloud
Responsibilities: buy low, sell high
• Algorithms (machine learning, numerical optimization, state management)
• Time to market (modifiability, exploration, custom DSLs)
• Performance (need to react quickly, for back testing, machine learning)
(Cartoon: JP Rini, New Yorker)
Sat Chatterjee / Algorithmic Trading
![Page 19: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/19.jpg)
The Reality:
Systems of Systems
19 Sat Chatterjee / Algorithmic Trading
![Page 20: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/20.jpg)
Architectural Challenges
20
Market Cloud
MD OMS Trading
Logic
MD OMS Trading
Logic
MD OMS Trading
Logic
MD OMS Trading
Logic
• Buffer bloat, deadlock, livelock (Facebook IPO)
• Quality of service, fairness
• Load balancing, HW/SW partitioning
• Availability
Sat Chatterjee / Algorithmic Trading
![Page 21: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/21.jpg)
Engineering Challenges
21
Market Cloud
MD OMS Trading
Logic
MD OMS Trading
Logic
MD OMS Trading
Logic
MD OMS Trading
Logic
• Early Architectural Exploration
• Back-Testing (fast, accurate simulation)
• QA (continuous integration)
• Deployment and Security
• Measurement and Benchmarking
Sat Chatterjee / Algorithmic Trading
![Page 22: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/22.jpg)
Takeaway
Embedded Systems Research should inform the Engineering of Algorithmic Trading Systems
Hard problems in Algorithmic Trading could be good benchmarks for Embedded Systems Research
in HW/SW Co-Design,
High-level HW Synthesis,
Soft Real-Time SW Synthesis,
and Formal Verification
22 Sat Chatterjee / Algorithmic Trading
![Page 23: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/23.jpg)
Backup
Sat Chatterjee / Algorithmic Trading 23
![Page 24: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/24.jpg)
Markets are linked
24
Contagion spread from the Futures Market to Equities
Flash Crash of 2010
Sat Chatterjee / Algorithmic Trading
![Page 25: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/25.jpg)
25
(source: nytimes.com)
An ongoing process
Sat Chatterjee / Algorithmic Trading
![Page 26: Embedded Systems in Algorithmic Trading - Peoplealanmi/presentations/... · 2015-03-29 · Engineering of Algorithmic Trading Systems Hard problems in Algorithmic Trading could be](https://reader030.vdocuments.site/reader030/viewer/2022040800/5e35e62d2add6b1a917f47fb/html5/thumbnails/26.jpg)
Trading Costs as a Percentage of Transaction Volume
Sat Chatterjee / Algorithmic Trading 26
(spread data: C. Jones, What do we know about High Frequency Trading?, SSRN 2236201)