Download - Reinforcement Learning for Trading
![Page 1: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/1.jpg)
Reinforcement Learningfor Trading
Team:Mariem Ayadi, Shreyas S. Jadhav, Benjamin W. Livingston,Amogh Mishra & Kevin Womack
Industry Mentors:Naftali Cohen, Srijan Sood & Zhen Zeng
DSI Supervisor:Adam Kelleher
Fall 2020
![Page 2: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/2.jpg)
Message from our Industry Mentors
![Page 3: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/3.jpg)
Value Industry
Algorithmic Trading
OpaqueUncontrollable
![Page 4: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/4.jpg)
RL for TradingProblem, Background and Impact- How would you teach a 10
year old to trade stocks?
- How can we transfer this logic to artificial intelligence?
→ Smart trading is nuanced but situational
![Page 5: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/5.jpg)
Technical DetailsRL Algorithm: Q-Table
![Page 6: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/6.jpg)
Technical DetailsQ-Table: Actions
Hold Buy Sell
![Page 7: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/7.jpg)
Technical DetailsQ-Table: States
![Page 8: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/8.jpg)
Technical DetailsQ-Table: q-values
At Initialization After Training
![Page 9: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/9.jpg)
What is RL in the first place?Comparison with Machine Learning
Source: Alexey Poddiachyi - towardsdatascience
Machine Learning: Classification Reinforcement Learning (RL)
Source: Javatpoint.com
![Page 10: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/10.jpg)
What is RL in the first place?Comparison with Machine Learning
Source: Alexey Poddiachyi - towardsdatascience
Reinforcement Learning (RL)
- What is unique about RL?
- How does it related to Q-tables presented?
![Page 11: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/11.jpg)
Research IntegrationDevelopment Visualization
● What technical indicators should be used for our state representation?
● What signals would we expect certain metrics to give an agent?
● What baselines should we construct to compare performance?
● How should the notion of reward be defined?
● What information is needed from the Q-learner examine it’s policy?
● What structure is best for code review and further development?
● Is the Q-Learning agent doing something meaningful/logical?
● What actions were taken by the agent?
![Page 12: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/12.jpg)
Demo
![Page 13: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/13.jpg)
Statistical ResultsQ-Learner Vs Benchmarks
![Page 14: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/14.jpg)
Challenges & Future WorkRL Algorithm: DQN
- Q-learning + Deep Learning = DQN
- But why do we need DQN?
![Page 15: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/15.jpg)
Challenges worth tacklingIf we had more time….
- This is just one stock - just the beginning!- Managing a portfolio
- Further tuning of hyperparameters- Extending state & action space
- Sentiment analysis- New indicators- Buy/Sell in different amounts- Etc...
![Page 16: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/16.jpg)
Appendix
![Page 17: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/17.jpg)
Goal
● Build an RL agent that maximizes cumulative return on a portfolio with a single stock/asset investment (ex: JPMC, AAPL, GOOG, FB)
● Visualize the policy learned by the agent
Environment:
● Initial cash input $100,000● In-sample period: 01/01/2007 – 12/31/2016● Out-of-sample (testing) period: 01/01/2017 – 12/31/2019● Allowable actions: Buy, Sell, Hold
![Page 18: Reinforcement Learning for Trading](https://reader030.vdocuments.site/reader030/viewer/2022012502/617c881cbf3fa51e3731649d/html5/thumbnails/18.jpg)
Source: https://medium.com/@nyxqianl/stock-trader-with-q-learning-91e70161762b