Bayesian Reasoning in Data Science
Bayesian Reasoning in Data Science#
Prof. C. Fanelli - DATA 340 05
(last update: Dec 9/2022)
Synopsis: No data scientist can work without a solid grasp of conditional probability and Bayesian reasoning. Bayes’ theorem allows to update our beliefs based on the occurrence of new events, steering the inference towards the truth and assessing uncertainty in predictions. This course provides an introduction to Bayesian Reasoning in Data Science (BRDS) and will let you appreciate the basic building blocks of this approach through real-world examples across different areas. During the course you will learn concrete computational implementations, that will help students connect what they have read and heard with what they can program, reinforcing the material.
Mini-projects: Five bonus points on the homework can be obtained with a mini-project done before Thanksgiving week. See mini-project template and check the source code of the html page. We discussed about using PyScript for mini-projects. PyScript is a framework that allows users to create rich Python applications in the browser using HTML’s interface and the power of Pyodide, WASM, and modern web technologies. Examples of PyScript can be found at https://pyscript.net/. Examples of mini-project during the BRDS course can be found here:
Bayesian A/B Testing Webapp with PyScript — by Rini Gupta, BRDS class 2022
This course is based on the following references [D'agostini, 2003, Davidson-Pilon, 2015, Downey, 2021, MacKay et al., 2003, Martin, 2018]
Lectures
- mod1 - Course Structure; Real-world Bayesian applications; Causes and effects; True and Measured - Lecture 1 (9/1/2022)
- mod1 - Bayes Theorem; Exercises; Thinking Probabilistically - Lecture 2 (9/6/2022)
- mod1 - Thinking Probabilistically; Notebooks ; Lecture 3 (9/8/2022)
- mod1 - Distributions; Likelihoods; Priors; Intro to Probabilistic Programming; PyMC - Lecture 4 and 5 (9/13 and 15/2022)
- mod1 - Credible Intervals; ROPE; HDI - Lecture 6 and 7 (9/20 and 22/2022)
- mod1 - Bayesian Inference; Central Limit Theorem; Law of Large Numbers - Lecture 8 and 9 (9/27 and 29/2022, in progress)
- mod2 - Bayesian Linear Regression - Lecture 10 and 11 (10/04 and 10/05)
- mod2 - Bayesian Polynomial Regression - Lecture 12 (10/11)
- mod2 - Bayesian Logistic Regression - Lecture 13 (10/18)
- mod2 - Bayesian Softmax Regression; Intro to Information Criteria - Lecture 14-15 (10/25 and 10/27)
- mod2 - Information Criteria; Bayesian Factors; Models Comparison - Lecture 16-17 (11/1 and 11/3)
- mod3 - Introduction to Gaussian Processes - Lecture 18 and 19 (11/10 - 11/15)
- mod3 - Bayesian Optimization - Lecture 20 (11/17)
- mod3 - Bayesian A/B Testing - Lecture 21 (11/22)
- mod3 - Markov Chain Monte Carlo - Lecture 22, 23 (11/29, 12/1)
- mod3 - Summary - Lecture 24, 25 (12/7, 12/9)
Notebooks
- (mod1 - part1) Thinking Probabilistically
- (mod1 - part2) Intro to Probabilistic Programming
- (mod1 - part3) Gaussian Inferences - t-Student
- (mod2 - part1) Bayesian Linear Regression
- (mod2 - part2) Bayesian Polynomial Regression
- (mod2 - part2b) Bayesian Logistic Regression
- (mod2 - part3) Bayesian Softmax Regression
- (mod3 - part1) Intro to Gaussian Processes
- (mod3 - part1b) Bayesian Optimization
- (mod3 - part2) Bayesian A/B Testing
- (mod3 - part3) Markov Chain Monte Carlo (MCMC)
- (mod3 - part3 extra) MCMC Applications, De-noising Image
- (mod3 - part3b) MCMC Diagnostics
Assignments
Mini-Projects
Additional resources