Planning and summaries

Please note that class planning is subject to adjustments.


Class notes are avaliable as a single file here, in pdf format. These notes do not substitute reading the recommended literature.

LectureNotes.pdf Last updated on 2020-11-15.

Note: these notes will be updated during the semester. It is best not to print them and to check this page regularly for updates.

Slides in html format and can be viewed directly from this page. Use the ← and → keys to change slides or Esc to select specific slides.

Lecture videos will be available after each lecture.Each lecture video is also linked from the corresponding summary.

Lecture videos for this year are availale in this YouTube playlist: Aprendizagem Automatica 2020/21

Lecture videos from 2019/20 are availale in this YouTube playlist:Aprendizagem Automática, 2019/20.

Lecture videos from 2018/19 are availale in this YouTube playlist:Aprendizagem Automática, 2018/19.

Lecture videos from 2017/18 are availale in this YouTube playlist: Aprendizagem Automática, 2017/18.


Lectures 1 and 2, 2020-09-22.

Introduction.Supervised learning, minimization (least squares), polynomial regression
Lecture 1 (HTML)Lecture 1 (PDF), Lecture 1 notes (PDF), Video
Lecture 2 (HTML), Lecture 2 (PDF), Lecture 2 notes (PDF), Video
Code for the polynomial regression example:

Lectures 3 and 4, 2020-09-29.

Overfitting and regularization with polynomial regression. Select models: Train, validate, test. Classification. Linear separability and discriminants. Logistic Regression. Using linear classifiers in higher dimensions.
Lecture 3 (HTML),Lecture 3 (PDF), Lecture 3 notes (PDF), Video
Lecture 4 (HTML), Lecture 4 (PDF), Lecture 4 notes (PDF), Video
Data files for the examples:,

Lectures 5 and 6, 2020-10-06.

Scoring classifiers. Cross-validation. Overfitting, model selection and regularization with logistic regression. Lazy learning. K-NN. Kernel regression and kernel density estimation.
Lecture 5 (HTML), Lecture 5 (PDF), Lecture 5 notes (PDF), Video
Lecture 6 (HTML), Lecture 6 (PDF), Lecture 6 notes (PDF), Video
Data for the examples in Lecture 5 and 6:

Lectures 7 and 8, 2020-10-13.

Generative models: naive bayes, bayes. Comparing classifiers. Perceptron and Multilayer Perceptron.
Lecture 7 (HTML), Lecture 7 (PDF), Lecture 7 notes (PDF), Video
Data for the examples in Lecture
Lecture 8 (HTML), Lecture 8 (PDF), Lecture 8 notes (PDF), Video

Lectures 9 and 10, 2020-10-20.

Maximum Margin Classifiers. Support vector machines for linear classification. SVM: soft margins, kernel trick, overfitting and regularization. Assignment 1.
Lecture 9 (HTML), Lecture 9 (PDF), Lecture 9 notes (PDF), Video
Lecture 10 (HTML), Lecture 10 (PDF), Lecture 10 notes (PDF), Video

Lectures 11 and 12, 2020-10-27.

Multiclass classification. Bootstrapping. Bias-variance decomposition and tradeoff. Ensemble methods: bagging and boosting.
Lecture 11 (HTML), Lecture 11 (PDF), Lecture 11 notes (PDF), Video
Lecture 12 (HTML), Lecture 12 (PDF), Lecture 12 notes (PDF), Video
Important note:
I had an outdated slide in lecture 12, about the first test. The test will be online this year, and will not include any question about the assignment.

Lectures 13 and 14, 2020-11-03.

Empirical Risk Minimization. Decision theory. Probably Approximately Correct Learning. VC dimension and shattering. Bayesian Decision theory. Maximum a posteriori estimation. Decisions and costs.
Lecture 13 (HTML), Lecture 13 (PDF), Lecture 13 notes (PDF), Video
Lecture 14 (HTML), Lecture 14 (PDF), Lecture 14 notes (PDF), Video

Lectures 15 and 16, 2020-11-10.

Introduction to unsupervised learning. Data visualization and feature selection. Dimensionality reduction: feature extraction with PCA; self-organzing maps.
Lecture 15 (HTML), Lecture 15 (PDF), Lecture 15 notes (PDF), Video
Lecture 16 (HTML), Lecture 16 (PDF), Lecture 16 notes (PDF), Video

Revisions: 2020-11-17.

This session is reserved for student's questions and revisions.

Lectures 17 and 18, 2020-11-24.

Introduction to clustering. K-means and k-medoids. Expectation maximization. Affinity Propagation clustering and problems with prototype-based clustering. Density Clustering. Clustering validation.
Lecture 17 (HTML), Lecture 17 (PDF), Lecture 17 notes (PDF), Video
Lecture 18 (HTML), Lecture 18 (PDF), Lecture 18 notes (PDF), Video

Lectures 19 and 20, 2020-12-07.

Note: this session will be for questions only, using the Zoom meeting. The lectures will be published earlier

Hierarchical Clustering. Agglomerative and Divisive Clustering. Clustering Features. Fuzzy sets and clustering. Fuzzy c-means. Manifold learning. Second assignment.
Lecture 19 (HTML), Lecture 19 (PDF), Lecture 19 notes (PDF), Video
Lecture 20 (HTML), Lecture 20 (PDF), Lecture 20 notes (PDF), Video

Lectures 21 and 22, 2020-12-15.

Probabilistic Clustering: mixture models. Expectation-Maximization revisited. Graphical methods, Hidden markov models. The Baum-Welch and Vitterbi algorithms. Introduction to deep learning.
Lecture 21 (HTML), Lecture 21 (PDF), Lecture 19 notes (PDF), Video
Lecture 22 (HTML), Lecture 22 (PDF), Lecture 20 notes (PDF), Video

Questions and Revisions (Zoom only): 2020-12-22.