Probabilistic Machine Learning Contents

  • Introduction to machine learning
    • What do we mean by learning
      • Supervised learning
      • Unsupervised learning
      • Reinforcement learning
      • Active learning
    • Examples
      • Multimedia databases
      • Robotics
      • Statistical machine translation
      • Bio-informatics
      • Probabilistic expert systems
      • Computer graphics
      • Computer games

  • Introduction to probabilistic modelling

  • Probabilistic graphical models

  • Learning
    • Learning discrete and Gaussian models
    • Frequentist approach
      • Maximum likelihood
      • Minimax risk
    • Bayesian approach
      • Conjugate analysis
      • Objective/subjective priors
      • Bayes risk
    • Exponential families and sufficient statistics

  • Linear regression
    • Least-mean-squares algorithm
    • Bias/variance trade-off
    • Least squares
    • Ridge regression
    • Bayesian regression
    • Shrinkage and subset selection
    • Examples

  • Linear classification
    • Discriminative models
    • Generative models
      • Logit
      • Probit
      • Softmax
    • Generalised linear models
    • Examples

  • Basis expansions
    • Radial basis networks
    • Logistic "neural" networks
    • Kernel machines
    • Regularisation
    • Examples: graphics and robotics

  • Constrained optimisation

  • Support vector machines
    • Large margin classifiers
    • Mercer's theorem
    • Slack variables
    • Examples: hand-written digit recognition.

  • Unsupervised learning I
    • K-means
    • Nearest neighbour
    • Vector quantisation
    • Self-organising maps
    • Hierarchical clustering
    • Examples: dimensionality reduction and image compression.

  • Unsupervised learning II
    • Principal component analysis (PCA)
    • Multi-dimensional Scaling (MDS)
    • Independent component analysis (ICA)
    • Latent semantic indexing (LSI)
    • Normalised cuts
    • Examples: information retrieval and image segmentation.

  • Mixture models and the EM algorithm
    • Theory and algorithms
    • Examples: multimedia databases, machine translation and data association

  • Factor analysis

  • Hidden Markov models (HMMs)
    • Theory and EM algorithms
    • Viterbi
    • Examples: speech and bio-informatics

  • Kalman filtering and smoothing
    • Tracking
    • Parameter estimation

  • Particle filtering

  • Monte Carlo methods

  • Markov chains

  • Metropolis and Gibbs algorithms

  • Monte Carlo optimisation

  • Variational methods

  • Time permitting
    • Bagging and boosting
    • Hypothesis testing
    • Model selection
    • Information theory and learning
    • Belief propagation
    • Computational learning theory