Think Bayes

If you know how to program, you’re ready to tackle Bayesian statistics. With this book, you’ll learn how to solve statistical problems with Python code instead of mathematical formulas, using discrete probability distributions rather than continuous mathematics. Once you get the math out of the way, the Bayesian fundamentals will become clearer and you’ll begin to apply these techniques to real-world problems.

Bayesian statistical methods are becoming more common and more important, but there aren’t many resources available to help beginners. Based on undergraduate classes taught by author Allen B. Downey, this book’s computational approach helps you get a solid start.

• Use your programming skills to learn and understand Bayesian statistics
• Work with problems involving estimation, prediction, decision analysis, evidence, and Bayesian hypothesis testing
• Get started with simple examples, using coins, dice, and a bowl of cookies
• Learn computational methods for solving real-world problems

The premise of this book, and the other books in the Think X series, is that if you know how to program, you can use that skill to learn other topics.

Most books on Bayesian statistics use math notation and present ideas using mathematical concepts like calculus. This book uses Python code and discrete approximations instead of continuous mathematics. As a result, what would be an integral in a math book becomes a summation, and most operations on probability distributions are loops or array operations.

I think this presentation is easier to understand, at least for people with programming skills. It is also more general, because when we make modeling decisions, we can choose the most appropriate model without worrying too much about whether the model lends itself to mathematical analysis. Also, it provides a smooth path from simple examples to real-world problems.

Table of Contents

Chapter 1. Probability

Chapter 2. Bayes’s Theorem

Chapter 3. Distributions

Chapter 4. Estimating Proportions

Chapter 5. Estimating Counts

Chapter 6. Odds and Addends

Chapter 7. Minimum, Maximum, and Mixture

Chapter 8. Poisson Processes

Chapter 9. Decision Analysis

Chapter 10. Testing

Chapter 11. Comparison

Chapter 12. Classification

Chapter 13. Inference

Chapter 14. Survival Analysis

Chapter 15. Mark and Recapture

Chapter 16. Logistic Regression

Chapter 17. Regression

Chapter 18. Conjugate Priors

Chapter 19. MCMC

Chapter 20. Approximate Bayesian Computation

Who Is This Book For?

To start this book, you should be comfortable with Python. If you are familiar with NumPy and pandas, that will help, but I’ll explain what you need as we go. You don’t need to know calculus or linear algebra. You don’t need any prior knowledge of statistics. In Chapter 1, I define probability and introduce conditional probability, which is the foundation of Bayes’s theorem. Chapter 3 introduces the probability distribution, which is the foundation of Bayesian statistics.

In later chapters, we use a variety of discrete and continuous distributions, including the binomial, exponential, Poisson, beta, gamma, and normal distributions. I will explain each distribution when it is introduced, and we will use SciPy to compute them, so you don’t need to know about their mathematical properties.

Working with the Code

Reading this book will only get you so far; to really understand it, you have to work with the code. The original form of this book is a series of Jupyter notebooks. After you read each chapter, I encourage you to run the notebook and work on the exercises. If you need help, my solutions are available.

About the Author

Allen Downey is a Professor of Computer Science at Olin College of Engineering. He has taught computer science at Wellesley College, Colby College and U.C. Berkeley. He has a Ph.D. in Computer Science from U.C. Berkeley and Master’s and Bachelor’s degrees from MIT. He is the author of Think Python, Think Bayes, Think DSP, and a blog, Probably Overthinking It.