The Science of Deep Learning

The Science of Deep Learning emerged from courses taught by the author that have provided thousands of students with training and experience for their academic studies. prepared them for careers in deep learning, machine learning. artificial intelligence in top companies in industry and academia. The book begins by covering the foundations of deep learning, followed by key deep learning architectures. Subsequent parts on generative models and reinforcement learning may be used as part of a deep learning course or as part of a course on each topic. The book includes state of the art topics such as Transformers, graph neural networks, variational autoencoders. deep reinforcement learning, with a broad range of applications. The appendices provide equations for computing gradients in backpropagation and optimization. best practices in scientific writing and reviewing. The text presents an up to date guide to the field built upon clear visualizations using a unified notation and equations, lowering the barrier to entry for the reader. The accompanying website provides complementary code and hundreds of exercises with solutions.

Table of Contents

Part I Foundations

1 Introduction

2 Forward and Backpropagation

3 Optimization

4 Regularization

Part II Architectures

5 Convolutional Neural Networks

6 Sequence Models

7 Graph Neural Networks

8 Transformers

Part III Generative Models

9 Generative Adversarial Networks

10 Variational Autoencoders

Part IV Reinforcement Learning

11 Reinforcement Learning

12 Deep Reinforcement Learning

Part V Applications

13 Applications

Appendix A: Matrix Calculus

Appendix B: Scientific Writing and Reviewing Best Practices

Review

“This textbook provides an excellent introduction to contemporary methods and models in deep learning. I expect this book to become a key resource in data science education for students and researchers.” Nakul Verma, Lecturer of Computer Science, Columbia University

“Drori’s textbook makes the learning curve for deep learning a whole lot easier to climb. It follows a rigid scientific narrative, accompanied by a trove of code examples and visualizations. These enable a truly multi-modal approach to learning that will allow many students to understand the material better and sets them on a path of exploration.” Joaquin Vanschoren, Assistant Professor of Machine Learning, Eindhoven University of Technology

“This book offers a fascinating tour of the field of deep learning, which in only ten years has come to revolutionize almost every area of computing. Drori provides concise descriptions of many of the most important developments, combining unified mathematical notation and ample figures to form an essential resource for students and practitioners alike.” Jonathan Ventura, Assistant Professor of Computer Science, Cal Poly

‘From the available books on deep learning, this textbook is outstanding. Drori has provided an extensive overview of the field including reinforcement learning – in its technical meaning and in his successful, common-sense approach to teaching and understanding.’ Gilbert Strang, Professor of Mathematics, Massachusetts Institute of Technology

“Drori’s book covers deep learning, from fundamentals to applications. The fundamentals are covered with clear figures and examples, making the underlying algorithms easy to understand for non-specialists. The multidisciplinary applications are thoughtfully selected to illustrate the broad applications of deep neural networks to specialized domains while highlighting the common themes and architectures between them.” Tonio Buonassisi, Professor of Mechanical Engineering, Massachusetts Institute of Technology

“This book covers an impressive breadth of foundational concepts and algorithms behind modern deep learning. By reading this book, readers will quickly but thoroughly learn and appreciate the foundations and advances of modern deep learning.” Kyunghyun Cho, Associate Professor of Computer Science and Data Science, New York University

“This new book by Prof. Drori brings fresh insights from his experience teaching thousands of students at Columbia, MIT, and NYU during the past several years. The book is a unique resource and opportunity for educators and researchers worldwide to build on his highly successful deep learning course.”, Claudio Silva, Professor of Computer Science and Engineering, New York University

“Drori’s textbook goes under the hood of deep learning, covering a broad swath of modern techniques in optimization that are useful for efficiently training neural networks. The book also covers regularization methods to avoid overfitting, a common issue when working with deep learning models. Overall, this is an excellent textbook for students and practitioners who want to gain a deeper understanding of deep learning.” Madeleine Udell, Assistant Professor of Management Science and Engineering, Stanford University

Book Description

Up-to-date guide to deep learning with unique content, rigorous math, unified notation, comprehensive algorithms, and high-quality figures.

About the Author

Iddo Drori is an Associate Professor of Computer Science, faculty of practice at Boston University, visiting at MIT, and adjunct at Columbia University. He was a visiting associate professor at Cornell University in operations research and information engineering, and a research scientist and adjunct professor at NYU Center for Data Science, Courant Institute, and NYU Tandon. He holds a Ph.D. in computer science and was a postdoctoral research fellow at Stanford University in Statistics. He also holds an MBA in organizational behavior and entrepreneurship and has a decade of industry research and leadership experience. His main research is in machine learning, AI, and computer vision, with 70 publications and over 5,200 citations, and has taught over 35 courses in computer science. He has won multiple competitions in computer vision conferences and received multiple best paper awards in machine learning conferences.