Bayesian Reasoning in Data Science (PhD Course)
Bayesian Reasoning in Data Science (PhD Course)#
Prof. C. Fanelli - DATA 644 8 (last update: May 8/2024)
Synopsis: No data scientist can work without a solid grasp of conditional probability and Bayesian reasoning. Bayes’ theorem allows to update our beliefs based on the occurrence of new events, steering the inference towards the truth and assessing uncertainty in predictions. This course provides an introduction to Bayesian Reasoning in Data Science (BRDS) and will let you appreciate the basic building blocks of this approach through real-world examples across different areas. During the course you will learn concrete computational implementations, that will help students connect what they have read and heard with what they can program, reinforcing the material.
This course is based on the following references [D'Agostini, 2003, Davidson-Pilon, 2015, Downey, 2021, MacKay et al., 2003, Martin, 2018]
Lectures (integrate with notebooks below)
- Bayes Theorem, priors, likelihood, posterior, distributions - Lec 1 (1/25/2024)
- Bayes Theorem, subjective and objective interpretations of probability, Exercises - Lec 2 (1/30/2024)
- Thinking Probabilistically, Causes and Effects , True and Measured; Credible Intervals, ROPE, HDI - Lec 3 (2/1/2024)
- Inference and Intro to Probabilistic Programming, PyMC; Credible Intervals, ROPE, HDI - Lec 4 (2/6/2024)
- Inference and Intro to Probabilistic Programming, PyMC; Credible Intervals, ROPE, HDI - Lec 5 (2/8/2024)
- Gaussian Inference; t-Student; Update of Beliefs - Lec 6 (2/13/2024)
- Bayesian Linear Regression; Robust Regression - Lec 7 (2/15/2024)
- Bayesian Polynomial Regression, Posterior Predictive Checks - Lec 8 (2/20/2024)
- Bayesian Logistic Regression and Softmax - Lec 9 (2/22/2024), Lec 10 (2/27/2024), Lec 11 (2/29/2024)
- Bayesian Model Comparison and Information Criteria - Lec 12 (3/5/2024)
- Bayes Factors, sequential MC - Lec 13 (3/7/2024)
- MCMC from scratch - Lec 14 (3/19/2024)
- Intro to Gaussian Processes - Lec 15 (3/21/2024), Lec 16 (3/26/2024), Lec 17 (3/28/2024)
- Intro to Bayesian Optimization - Lec 18 (4/2/2024)
- Multi-Objective Bayesian Optimization - Lec 19 (4/4/2024)
- Introduction to Bayes Networks - Lec 20 (4/9/2024)
- Bayes Networks (continue) - Lec 21 (4/11/2024)
- Introduction to Bayesian Neural Networks - Lec 22 (4/16/2024)
- BNN (continue) - Lec 24 (4/25/2024)
- Finale - Lec 25 (5/2/2024)
Notebooks
- Bayes Theorem (2/1/2024)
- Thinking Probabilistically (2/1/2024)
- Intro to Probabilistic Programming (2/6/2024)
- An (Over-)Simplified Intro to MCMC (2/8/2024)
- Bayesian Inference with Gaussian and t-Student models, update of beliefs (2/13/2024)
- Bayesian Linear Regression; Robust Inference (2/15/2024)
- Bayesian Polynomial Regression, Posterior Predictive Checks and Comparison with Observations (2/20/2024)
- Bayesian Logistic Regression (2/22/2024)
- Bayesian Softmax (2/27/2024)
- Model Comparison, Information Criteria (3/5/2024)
- Model Comparison, Bayes Factors (3/7/2024)
- Building MCMC from scratch; a closer look at samplers (3/19/2024)
- Gaussian Processes (3/26/2024)
- Multi-Objective Optimization (4/4/2024)
- Bayesian Networks (4/9/2024) and (4/11/2024)
- Bayesian Neural Networks (4/16/2024)
- Bayesian Convolutional Neural Network (4/25/2024)
Assignments
- Assignment 1 - PDF (Questions 1 and 3)
- Assignment 1 - notebook (Question 2)
- Assignment 2 - PDF
- Assignment 2 - notebook (All Questions)
- Assignment 3 - PDF
- Assignment 3 - notebook (All Questions)
- Assignment 4 - PDF
- Assignment 4 - notebook (Approach 1)
- Assignment 4 - notebook (Approach 2)
- Assignment 5 - PDF
- Assignment 5 - notebook (All Questions)
Question and Answers
Projects
Additional resources