Planning and summaries

Please note that class planning is subject to adjustments.

Files

Class notes are avaliable as a single file here, in pdf format. These notes do not substitute reading the recommended literature.

LectureNotes.pdf Last updated on 2019-10-14.

Note: these notes will be updated during the semester. It is best not to print them and to check this page regularly for updates.

Slides in html format and can be viewed directly from this page. Use the ← and → keys to change slides or Esc to select specific slides.

Lecture videos will be available after each lecture.Each lecture video is also linked from the corresponding summary.

Lecture videos for this year are availale in this YouTube playlist: Aprendizagem Automatica 2019/20

Lecture videos from 2018/19 are availale in this YouTube playlist:Aprendizagem Automática, 2018/19.

Lecture videos from 2017/18 are availale in this YouTube playlist: Aprendizagem Automática, 2017/18.

Summaries

Lectures 1 and 2, 2019-09-10.

Summary
Introduction.Supervised learning, minimization (least squares), polynomial regression
Lecture 1 (HTML)Lecture 1 (PDF), Lecture 1 notes (PDF), Video
Lecture 2 (HTML), Lecture 2 (PDF), Lecture 2 notes (PDF), Video
Code for the polynomial regression example: L2.zip

Lectures 3 and 4, 2019-09-17.

Summary
Overfitting and regularization with polynomial regression. Select models: Train, validate, test. Classification. Linear separability and discriminants. Logistic Regression. Using linear classifiers in higher dimensions.
Lecture 3 (HTML),Lecture 3 (PDF), Lecture 3 notes (PDF), Video
Lecture 4 (HTML), Lecture 4 (PDF), Lecture 4 notes (PDF), Video

Lectures 5 and 6, 2019-09-24.

Summary
Scoring classifiers. Cross-validation. Overfitting, model selection and regularization with logistic regression. Lazy learning. K-NN. Kernel regression and kernel density estimation.
Lecture 5 (HTML), Lecture 5 (PDF), Lecture 5 notes (PDF), Video
Lecture 6 (HTML), Lecture 6 (PDF), Lecture 6 notes (PDF), Video

Lectures 7 and 8, 2019-10-01.

Summary
Generative models: naive bayes, bayes. Comparing classifiers. Perceptron and Multilayer Perceptron.
Lecture 7 (HTML), Lecture 7 (PDF), Lecture 7 notes (PDF), Video
Lecture 8 (HTML), Lecture 8 (PDF), Lecture 8 notes (PDF), Video

Lectures 9 and 10, 2019-10-08.

Summary
Maximum Margin Classifiers. Support vector machines for linear classification. SVM: soft margins, kernel trick, overfitting and regularization. Assignment 1.
Lecture 9 (HTML), Lecture 9 (PDF), Lecture 9 notes (PDF), Video
Lecture 10 (HTML), Lecture 10 (PDF), Lecture 10 notes (PDF), Video

Lectures 11 and 12, 2019-10-15.

Summary
Multiclass classification. Bootstrapping. Bias-variance decomposition and tradeoff. Ensemble methods: bagging and boosting.
Lecture 11 (HTML), Lecture 11 (PDF), Lecture 11 notes (PDF), Video
Lecture 12 (HTML), Lecture 12 (PDF), Lecture 12 notes (PDF), Video

Planned Classes. Please note that planned lectures are subject to changes; download notes and slides after the lecture.

Lectures 13 and 14, 2019-10-22.

Summary
Empirical Risk Minimization. Decision theory. Probably Approximately Correct Learning. VC dimension and shattering. Bayesian Decision theory. Maximum a posteriori estimation. Decisions and costs.
Lecture 13 (HTML)
Lecture 14 (HTML)

Lectures 15 and 16, 2019-10-29.

Summary
Introduction to unsupervised learning. Data visualization and feature selection. Dimensionality reduction: feature extraction with PCA; self-organzing maps.
Lecture 15 (HTML)
Lecture 16 (HTML)

Revisions: 2019-11-05.

Summary
This session is reserved for student's questions and revisions.

Lectures 17 and 18, 2019-11-12.

Summary
Introduction to clustering. K-means and k-medoids. Expectation maximization. Affinity Propagation clustering and problems with prototype-based clustering. Density Clustering. Clustering validation.
Lecture 17 (HTML)
Lecture 18 (HTML)

Lectures 19 and 20, 2019-11-19.

Summary
Hierarchical Clustering. Agglomerative and Divisive Clustering. Clustering Features. Fuzzy sets and clustering. Fuzzy c-means. Probabilistic Clustering: mixture models. Expectation-Maximization revisited. Second assignment.
Lecture 19 (HTML)
Lecture 20 (HTML)

Lectures 21 and 22, 2019-11-26.

Summary
Graphical methods, Hidden markov models. The Baum-Welch and Vitterbi algorithms. Deep learning. The problem of backpropagation. Autoencoders and Stacked Denoising Autoencoders.
Lecture 21 (HTML)
Lecture 22 (HTML)

Assignment 2, questions: 2019-12-03.

Revisions: 2019-12-10.