Note: If you are enrolled in this course, please see the detailed syllabus on Canvas.

ECE 6501/4502, CS 6501: Probabilistic Machine Learning (Fall 2026)

Welcome! In this course, we’ll study estimation and machine learning from a probabilistic point of view.

Why a probabilistic view?

Information and uncertainty, which underlie both statistical estimation and machine learning, can be represented via probability in a robust and versatile way. Unknown quantities can be cast as random variables and their relationships to each other and to known information as joint distributions. This provides a unifying framework for setting up estimation and machine learning problems, where we can state our assumptions clearly, design methods, and evaluate performance.

What topics will we study?

We will start with estimation, which can be defined as the problem of learning about the world from data (e.g., finding the chance of getting a disease given one’s genetic make-up) or drawing conclusions about relationships (e.g., what are the best predictors of academic success?). We will then learn about machine learning problems such as regression and classification, where the goal is to predict an unknown quantity, e.g., the price of a house, based on some relevant information. We will also learn how to deal with situations when part of the data is missing. Finally, we will discuss computational methods, which help tackle difficult problems via approximation.

Pre-requisites:

• Fluency in basic probability (e.g., APMA 3100) is needed for the course. You should be comfortable with 70-80% of Chapter 0. In the second week of class, you will be asked to pass a test of basic probability with at least 70%. You will have the chance to retake the test if you score below 70%.

  • Example Probability Review Test

  You can also refer to these pages I developed for a different course to review probability, although these don’t cover everything we’ll need:

  • Basics
  • Random variables, distributions, and independence
  • Expectation and variance
  • Conditional probability
  • And here are some other resources:
    • An online textbook for probability theory: https://www.probabilitycourse.com
    • Random Processes for Engineers by Bruce Hajek
• Familiarity with linear algebra.
• The programming exercises are based on Python.

Course Notes:

Links to the complete notes and individual chapters are listed below and are updated as needed, with the chapter notes being possibly more up-to-date. To ensure you always access the latest versions, please view them via the links below rather than downloading local copies.

Estimation and Probabilistic Learning

0. Review of Probability (pdf)
1. Probability, Inference, and Learning (pdf)
2. Frequentist Parameter Estimation
3. Bayesian Parameter Estimation
4. Multivariate random variables
5. Linear Regression
6. Linear Classification
7. Expectation-Maximization
8. Basics of Graphical Models
9. Independence in Graphical Models
10. Parameter Estimation in Graphical Models
11. Inference in Graphical Models
12. Inference in Hidden Markov Models
13. Factor Graphs and Sum/Max-product Algorithms
14. Markov Chains
15. Sampling Methods
16. Variational Inference
17. Appendix

Textbooks and other resources:

The main resources are lectures and the pdf notes posted on this page. But you may find the following useful:

1. Probabilistic Machine Learning: An introduction by Kevin P. Murphy, 2022.
2. Deep Learning by Ian Goodfellow et al, 2015.
3. Information Theory, Inference, and Learning Algorithms by David MacKay, 2003.
4. Pattern Recognition and Machine Learning by Christopher M. Bishop, 2006.
5. Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman, 2009.

---

Previous offerings:

• Fall 2025