Deep Reinforcement Learning Hands-On

Maxim Lapan delivers intuitive explanations and insights into complex reinforcement learning (RL) concepts, starting from the basics of RL on simple environments and tasks to modern, state-of-the-art methods

Key Features

• Learn with concise explanations, modern libraries, and diverse applications from games to stock trading and web navigation
• Develop deep RL models, improve their stability, and efficiently solve complex environments
• New content on RL from human feedback (RLHF), MuZero, and transformers

Book Description

Start your journey into reinforcement learning (RL) and reward yourself with the third edition of Deep Reinforcement Learning Hands-On. This book takes you through the basics of RL to more advanced concepts with the help of various applications, including game playing, discrete optimization, stock trading, and web browser navigation. By walking you through landmark research papers in the fi eld, this deep RL book will equip you with practical knowledge of RL and the theoretical foundation to understand and implement most modern RL papers.

The book retains its approach of providing concise and easy-to-follow explanations from the previous editions. You’ll work through practical and diverse examples, from grid environments and games to stock trading and RL agents in web environments, to give you a well-rounded understanding of RL, its capabilities, and its use cases. You’ll learn about key topics, such as deep Q-networks (DQNs), policy gradient methods, continuous control problems, and highly scalable, non-gradient methods.

If you want to learn about RL through a practical approach using OpenAI Gym and PyTorch, concise explanations, and the incremental development of topics, then Deep Reinforcement Learning Hands-On, Third Edition, is your ideal companion

What you will learn

• Stay on the cutting edge with new content on MuZero, RL with human feedback, and LLMs
• Evaluate RL methods, including cross-entropy, DQN, actor-critic, TRPO, PPO, DDPG, and D4PG
• Implement RL algorithms using PyTorch and modern RL libraries
• Build and train deep Q-networks to solve complex tasks in Atari environments
• Speed up RL models using algorithmic and engineering approaches
• Leverage advanced techniques like proximal policy optimization (PPO) for more stable training

Who this book is for

This book is ideal for machine learning engineers, software engineers, and data scientists looking to learn and apply deep reinforcement learning in practice. It assumes familiarity with Python, calculus, and machine learning concepts. With practical examples and high-level overviews, it’s also suitable for experienced professionals looking to deepen their understanding of advanced deep RL methods and apply them across industries, such as gaming and finance

Review

“I have been a devoted fan of Max’s work from the start. I discovered Deep Reinforcement Learning Hands-On while enhancing my understanding of applied reinforcement learning powered by deep neural networks. The book played a significant role in strengthening my skills during that time. I am most happy to see that there is a new edition!”

Dr. Tristan Behrens, AI Hands-On Advisor

“…An excellent piece of work and I really enjoyed reading it. This book offers a great mix of theory (with enough math to understand but not overwhelm) and practical coding exercises that are easy to follow. It comes with tons of visuals to explain complex concepts — super helpful.”

Andreas Horn, Head of AIOps at IBM

Table of Contents

PART I : INTRODUCTIONTORL

Chapter 1: What Is Reinforcement Learning?

Chapter 2: OpenAI Gym API and Gymnasium

Chapter 3: Deep Learning with Py Torch

Chapter 4: The Cross-Entropy Method

PART II: VALUE-BASED METHODS

Chapter 5: Tabular Learning and the Bellman Equation

Chapter 6: Deep Q-Networks

Chapter 7: Higher-Level RL Libraries

Chapter 8: DQN Extensions

Chapter 9: Ways to Speed Up RL

Chapter 10: Stocks Trading Using RL

PART III: POLICY-BASED METHODS

Chapter 11: Policy Gradients

Chapter 12: Actor -Critic Method: A2C and A3C

Chapter 13: The TextWorld Environment

Chapter 14: Web Navigation

PART IV: ADVANCED RL

Chapter 15: Continous Action Space

Chapter 16: Trust Region Methods

Chapter 17: Black-Box Optimizations in RL

Chapter 18: Advanced Exploration

Chapter 19: Reinforcement Learning with Human Feedback

Chapter 20: AlphaGo Zero and Mulero

Chapter 21: RL in Discrete Optimization

Chapter 22: Multi-Agent RL

About the Author

Maxim has been working as a software developer for more than 20 years and was involved in various areas: distributed scientific computing, distributed systems and big data processing. Since 2014 he is actively using machine and deep learning to solve practical industrial tasks, such as NLP problems, RL for web crawling and web pages analysis. He has been living in Germany with his family.