Foundations of Reinforcement Learning with Applications in Finance

Foundations of Reinforcement Learning with Applications in Finance aims to demystify Reinforcement Learning, and to make it a practically useful tool for those studying and working in applied areas ― especially finance.

Reinforcement Learning is emerging as a powerful technique for solving a variety of complex problems across industries that involve Sequential Optimal Decisioning under Uncertainty. Its penetration in high-profile problems like self-driving cars, robotics, and strategy games points to a future where Reinforcement Learning algorithms will have decisioning abilities far superior to humans. But when it comes getting educated in this area, there seems to be a reluctance to jump right in, because Reinforcement Learning appears to have acquired a reputation for being mysterious and technically challenging.

This book strives to impart a lucid and insightful understanding of the topic by emphasizing the foundational mathematics and implementing models and algorithms in well-designed Python code, along with robust coverage of several financial trading problems that can be solved with Reinforcement Learning. This book has been created after years of iterative experimentation on the pedagogy of these topics while being taught to university students as well as industry practitioners.

Features

• Focus on the foundational theory underpinning Reinforcement Learning and software design of the corresponding models and algorithms
• Suitable as a primary text for courses in Reinforcement Learning, but also as supplementary reading for applied/financial mathematics, programming, and other related courses
• Suitable for a professional audience of quantitative analysts or data scientists
• Blends theory/mathematics, programming/algorithms and real-world financial nuances while always striving to maintain simplicity and to build intuitive understanding
• To access the code base for this book, please go to: https://github.com/TikhonJelvis/RL-book

Table of Contents

CHAPTER 1: Overview

CHAPTER 2: Programming and Design

MODULE I: Processes and Planning Algorithms

MODULE II: Modeling Financial Applications

MODULE Ill: Reinforcement Learning Algorithms

MODULE IV: Finishing Touches

APPENDIX A: Moment Generating Function and Its Applications

APPENDIX B: Portfolio Theory

APPENDIX C: Introduction to and Overview of Stochastic Calculus Basics

APPENDIX D: The Hamilton-Jacobi-Bellman (HJB) Equation

APPENDIX E: Black-Scholes Equation and Its Solution for Call/Put Options

APPENDIX F: Function Approximations as Affine Spaces

APPENDIX G: Conjugate Priors for Gaussian and Bernoulli Distributions

About the Authors

Ashwin Rao is the Chief Science Officer of Wayfair, an e-commerce company where he and his team develop mathematical models and algorithms for supply-chain and logistics, merchandising, marketing, search, personalization, pricing and customer service. Ashwin is also an Adjunct Professor at Stanford University, focusing his research and teaching in the area of Stochastic Control, particularly Reinforcement Learning algorithms with applications in Finance and Retail. Previously, Ashwin was a Managing Director at Morgan Stanley and a Trading Strategist at Goldman Sachs. Ashwin holds a Bachelor’s degree in Computer Science and Engineering from IIT-Bombay and a Ph.D in Computer Science from University of Southern California, where he specialized in Algorithms Theory and Abstract Algebra.

Tikhon Jelvis is a programmer who specializes in bringing ideas from programming languages and functional programming to machine learning and data science. He has developed inventory optimization, simulation and demand forecasting systems as a Principal Scientist at Target and is a speaker and open-source contributor in the Haskell community where he serves on the board of directors for Haskell.org.