Data Estimation and Inference

Course dates – MT – week beginning Monday 08th October 2018 – for Year 1 students
Mike Osborne


We are implicit in an increasingly dense web of data: the rise of big data presents unprecedented research opportunities across science and engineering. Modelling such data presents acute challenges; its complexity demands principled management of uncertainty. As such, this course will provide an introduction to probabilistic inference and modern computational statistics.

  • Understanding how to manage and transform probability distributions as a tool for characterising knowledge and ignorance in light of data.
  • The practical use of decision theory to describe how to take action in an uncertain world.
  • The selection of appropriate approximations to permit the implementation of probabilistic systems within computational constraints.


  • Basic properties of distributions: independence, joint and conditional distributions, marginals. Bayes' theorem. The extension from univariate to multivariate distributions.
  • The representation of a distribution as a belief network.
  • Gaussian processes as a means of inferring functions.
  • Bayesian Decision Theory: the specification of appropriate loss functions, and the minimisation of expected loss to select actions. Classifiers and Decision Surfaces, particularly the discriminant function derived from Normal distributions.
  • Techniques of approximation: Maximum Likelihood; Maximum A-Posteriori; Laplace Approximations
  • K. P. Murphy, 'Machine Learning: A Probabilistic Perspective', 2012, MIT Press
  • David Barber, `Bayesian Reasoning and Machine Learning<>', (pdf available for free), 2012, Cambridge University Press. Chapters: 1; 3.1, 3.3; 7.1-7.2; 8; 9.1-9.2; 10; 17.1-17.4; 24.1-24.4.
  • Phil Gregory, `Bayesian Logical Data Analysis for the Physical Sciences', 2010, Cambridge University Press. Chapters: 1; 3.1-3.4; 5.1-5.12; 9.
  • David MacKay, ' Information Theory, Inference, and Learning Algorithms<>' (pdf available for free), 2003, Cambridge University Press. Chapters: 2.1-2.4; 3; 36.