Upload File

reinforcement learning for trading

18
Reinforcement Learning for Trading Team: Mariem Ayadi, Shreyas S. Jadhav, Benjamin W. Livingston, Amogh Mishra & Kevin Womack Industry Mentors: Naftali Cohen, Srijan Sood & Zhen Zeng DSI Supervisor: Adam Kelleher Fall 2020

Upload: others

Post on 30-Oct-2021

4 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Reinforcement Learning for Trading

Reinforcement Learningfor Trading

Team:Mariem Ayadi, Shreyas S. Jadhav, Benjamin W. Livingston,Amogh Mishra & Kevin Womack

Industry Mentors:Naftali Cohen, Srijan Sood & Zhen Zeng

DSI Supervisor:Adam Kelleher

Fall 2020

Page 2: Reinforcement Learning for Trading

Message from our Industry Mentors

Page 3: Reinforcement Learning for Trading

Value Industry

Algorithmic Trading

OpaqueUncontrollable

Page 4: Reinforcement Learning for Trading

RL for TradingProblem, Background and Impact- How would you teach a 10

year old to trade stocks?

- How can we transfer this logic to artificial intelligence?

→ Smart trading is nuanced but situational

Page 5: Reinforcement Learning for Trading

Technical DetailsRL Algorithm: Q-Table

Page 6: Reinforcement Learning for Trading

Technical DetailsQ-Table: Actions

Hold Buy Sell

Page 7: Reinforcement Learning for Trading

Technical DetailsQ-Table: States

Page 8: Reinforcement Learning for Trading

Technical DetailsQ-Table: q-values

At Initialization After Training

Page 9: Reinforcement Learning for Trading

What is RL in the first place?Comparison with Machine Learning

Source: Alexey Poddiachyi - towardsdatascience

Machine Learning: Classification Reinforcement Learning (RL)

Source: Javatpoint.com

Page 10: Reinforcement Learning for Trading

What is RL in the first place?Comparison with Machine Learning

Source: Alexey Poddiachyi - towardsdatascience

Reinforcement Learning (RL)

- What is unique about RL?

- How does it related to Q-tables presented?

Page 11: Reinforcement Learning for Trading

Research IntegrationDevelopment Visualization

● What technical indicators should be used for our state representation?

● What signals would we expect certain metrics to give an agent?

● What baselines should we construct to compare performance?

● How should the notion of reward be defined?

● What information is needed from the Q-learner examine it’s policy?

● What structure is best for code review and further development?

● Is the Q-Learning agent doing something meaningful/logical?

● What actions were taken by the agent?

Page 12: Reinforcement Learning for Trading

Demo

Page 13: Reinforcement Learning for Trading

Statistical ResultsQ-Learner Vs Benchmarks

Page 14: Reinforcement Learning for Trading

Challenges & Future WorkRL Algorithm: DQN

- Q-learning + Deep Learning = DQN

- But why do we need DQN?

Page 15: Reinforcement Learning for Trading

Challenges worth tacklingIf we had more time….

- This is just one stock - just the beginning!- Managing a portfolio

- Further tuning of hyperparameters- Extending state & action space

- Sentiment analysis- New indicators- Buy/Sell in different amounts- Etc...

Page 16: Reinforcement Learning for Trading

Appendix

Page 17: Reinforcement Learning for Trading

Goal

● Build an RL agent that maximizes cumulative return on a portfolio with a single stock/asset investment (ex: JPMC, AAPL, GOOG, FB)

● Visualize the policy learned by the agent

Environment:

● Initial cash input $100,000● In-sample period: 01/01/2007 – 12/31/2016● Out-of-sample (testing) period: 01/01/2017 – 12/31/2019● Allowable actions: Buy, Sell, Hold

Page 18: Reinforcement Learning for Trading

Source: https://medium.com/@nyxqianl/stock-trader-with-q-learning-91e70161762b


Reinforcement Learning for FX trading - Stanford Universitystanford.edu/class/msande448/2019/Final_reports/gr2.pdf · 2019-06-11 · Reinforcement Learning for FX trading Yuqin Dai,

Deep Reinforcement Learning for Active High Frequency Trading

Reinforcement Learning Das Reinforcement Learning-Problem Alexander Schmid

Reinforcement Learning Lecture Inverse Reinforcement Learningipvs.informatik.uni-stuttgart.de/mlr/wp-content/uploads/2017/07/09... · Reinforcement Learning Inverse Reinforcement

Reinforcement Learning in Multi-Party Trading Dialog...Prague, Czech Republic, 2-4 September 2015. c 2015 Association for Computational Linguistics Reinforcement Learning in Multi-Party

Reinforcement Learning - uni-freiburg.degki.informatik.uni-freiburg.de/.../recordings/reinforcement.pdf · Reinforcement Learning 3 What is Reinforcement Learning? Learning from interaction

Reinforcement Learning: Learning algorithms

Optimizing the Pairs-Trading Strategy Using Deep ...downloads.hindawi.com/journals/complexity/2019/3582516.pdf · pairs trading strategy using deep reinforcement learning, especiallydeepQ-networks,sincepairstradingstrategycan

Algorithmic Trading: Reinforcement Learningsebastian.utstat.utoronto.ca/wp-content/uploads/2017/11/RiO-RL.pdf · Algorithmic Trading: Reinforcement Learning Sebastian Jaimungal University

Reinforcement Learning or Active Inference?karl/Reinforcement Learning or Active... · Reinforcement Learning or Active Inference? ... From the point of view of reinforcement learning

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

Reinforcement Learning for Quantitative Trading

Inverse Reinforcement Learning CS885 Reinforcement

Using Reinforcement Learning in the Algorithmic Trading

Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning Applications Summary

Robust Log-Optimal Strategy with Reinforcement Learning · 2 Robust Log-Optimal Strategy with Reinforcement Learning analyzed results from RLOS and observes the trading environment

Generalization in Reinforcement Learning: Successful ...papers.nips.cc/paper/1109-generalization-in-reinforcement-learning... · Generalization in Reinforcement Learning: Successful

Reinforcement Learning Chapter 13 What is Reinforcement Learning? Q-Learning Examples 1

Reinforcement Learning and Deep Reinforcement Learningcse.ucdenver.edu/.../Class-22-Reinforcement-learning-DL.pdf · 2018. 11. 28. · Outlines 1 Principles of Reinforcement Learning

Deep Learning for Reinforcement Learning in · PDF fileDeep Learning for Reinforcement Learning in ... Deep Learning for Reinforcement Learning in Pacman Deep Learning für ... Während

Reinforcement Learning - 4. Model-free reinforcement Learning

Reinforcement Learning for FX trading Font: Roboto 14€¦ · Roboto 14 Reinforcement Learning for FX trading Yuqin Dai, Chris Wang, Iris Wang, Yilun Xu . Font ... The agent may learn

Reinforcement Learning For Automated Trading

From Reinforcement Learning to Deep Reinforcement …fagostin/assets/files/...Keywords: Machine learning · Reinforcement learning Deep learning · Deep reinforcement learning 1 Introduction

Cooperative Inverse Reinforcement Learning...Cooperative Inverse Reinforcement Learning Dylan Hadfield-Menell CS237: Reinforcement Learning May 31, 2017

Eick: Reinforcement Learning. Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning

Deep Reinforcement Learning for Trading · 2020. 6. 16. · Deep Reinforcement Learning for Trading Spring 2020. component of such trading systems is a predictive signal that can

Reinforcement Learning & Apprenticeship Learning

Reinforcement Learning

Reinforcement Learning in Stock Trading

An automated trading system based on Recurrent Reinforcement Learning

Multi-Objective Reinforcement Learning using Sets of Pareto … · 2020. 10. 19. · learning and multi-objective reinforcement learning. 2.1 Reinforcement Learning A reinforcement

Algorithmic Trading and Reinforcement Learning

[DL輪読会]Deep Direct Reinforcement Learning for Financial Signal Representation and Trading

Bayesian Reinforcement Learning - mlg.eng.cam.ac.ukmlg.eng.cam.ac.uk/rowan/files/BayesianReinforcementLearning.pdf · Introduction Bayesian Reinforcement Learning Bayesian Reinforcement