IE 674 Hot Topics in Machine Learning (HWS 2015)

Organization

  • Lecturer: Prof. Dr. Rainer Gemulla, Dr. Laura Dietz
  • Type of course: Lecture and practical exercises (6 ECTS points)
  • Lecture: Wednesday, 08:30-10:00, A5 C015
  • Tutorium (irregularly): Wednesday, 10:15-11:45, A5 C015
  • Evaluation: Final exam or oral examination, regular exercises
  • Prerequisites: IE 500 Data Mining I (recommended), IE 560 Decision support (recommended), basic knowledge of probability and statistics

News

Content

Machine Learning is about designing algorithms that are able to make predictions about data or extract knowledge from data. The aim of this course is to study algorithms, underlying concepts, and theoretic principles that allow for algorithms to automatically learn how to make predictions. The course focuses on selected "hot topics" and their applications, which include:

  • Basics of machine learning and probability theory
  • Probabilistic graphical models
  • Inference and parameter estimation
  • Neural networks

Lecture Notes

  • 00 Organization (pdf)
  • 01 Introduction (pdf, updated Sep 15)
  • 02 Probability & Bayesian inference (pdf, updated Oct 5)
  • 03 Logistic Regression and GLMs (pdf, model example in Jupyter notebook or python or html)
  • 04 Naive Bayes (pdf, updated Oct 14)
  • 05 Undirected Graphics Models (pdf, updated Oct 20)
  • 06 Conditional Random Fields (pdf, updated Dec 22)
  • 07 Neural Networks (pdf, updated Nov 17)
  • 08 Online Learning under Full and Bandit Information (pdf, Guest Lecture)
  • 09 Topic Models (Part A pdf, part B pdf, further readings: Dietz Extended Chapter 2 of Ph.D. thesis) 

Exercises

  • Week 1: Sheet (pdf), differentiation rules (pdf)
  • Week 2: Introduction to R (R), introduction to Python (Jupyter notebook, python, html, UNdata.csv)
  • Week 3: Sheet (pdf), MNIST data set, solution (zip)
  • Week 4: Sheet (pdf), solution (pdf, R)
  • Week 5: Sheet (pdf), solution (pdf)
  • Week 6: Guest lecture,  Mitul K. Jain (Accenture) on "Predictive Analytics - Introduction and Use Cases"
  • Week 7+8: Sheet (pdf, R, updated 14.10.), solution (pdf, R)
  • Week 9: Sheet (pdf, R), solution (pdf, R)

Assignments

  • 01: Logistic Regression (zip), due Oct 15, solution hints
  • 02: Naive Bayes (zip), due Oct 27
  • 03: linear-chain CRF (zip), due November 22
  • 04: Neural Networks (zip), due Dec 6

Literature

  • K.P. Murphy. Machine Learning: A Probabilistic Perspective, The MIT Press, 2012 (4th printing)
  • D. Koller, N. Friedman. Probabilistic graphical models. The MIT Press, 2009
  • Additional material and articles provided in lecture notes