Machine Learning

COURSE OVERVIEW

This course will provide an introduction to the theory of statistical learning and practical machine learning algorithms with applications in signal processing and data analysis. In contrast?to most traditional approaches to statistical inference and signal processing, in this course we?will focus on how to learn effective models from data and how to apply these models to practical?signal processing problems. We will approach these problems from the perspective of statistical inference. We will study both practical algorithms for statistical inference and theoretical?aspects of how to reason about and work with probabilistic models. We will consider a variety?of applications, including classification, prediction, regression, clustering, modeling, and data?exploration/visualization.

COURSE OBJECTIVES

In the last few years, machine learning has matured from science fiction to reality. We are living in a world where we have already seen industry bringing to reality self-driving cars, face-recognizers that work on a massive scale (Facebook), speech translation systems that can translate from one language to many other simultaneously and in real-time, and more interestingly we have machines that can learn to play atari games in a similar fashion as we do.
A lot of these victories have come from the exciting field of Deep Learning; a learning methodology based on the concept that the human mind captures details at multiple levels or at multiple abstract levels. One property of deep learning is removing the responsibility of humans to design features, instead, Deep Learning is given a task to find the appropriate representation.

GRADING POLICY

• 45% Assignments
• 5% Class participation and Creating Notes
• 20% Final Project
• 10% Quizzes
• 10% Midterm Exam
• 10% Final Exam

COURSE OUTLINE

Topics

Introduction to Machine Learning Supervised learning, Unsupervised Learning Definition of learning problem, Empirical Risk Minimization

Bayes Classifier and Likelihood ratio test Nearest neighbor classifier, Risk of the Nearest neighbor classifier

Naive? Bayes Classifier, LDA Risk of Naive Bayes, Document classification, Laplace smoothing, Parameter estimation

Logistic Regression, Gradient Decent Logistic function, Maximum Likelihood estimation, Newton’s method

Non parametric linear classifiers, Perceptron Learning Linearly separable sets, Geometry of linearly separating hyperplanes

Support Vectors Machines Kernel methods, SVM dual problem

Classification using Neural Networks Computation Graph, Backpropagation

Convolutional Neural Networks CNNs, image classification

Linear Regression Regularized Regression,Overfitting, Generalized Least Squares, Lasso and Ridge Regression.

Dimensionality Reduction PCA, Embedded Methods , Feature Extraction

Multidimensional Scaling Singular Value, Decomposition, Manifold Embedding

Density Estimation K-Means Clustering

Gaussian Mixture Model EM Algorithm

Kernel Ridge Regression

Variational Inference

Graph Neural Networks

Lecture 4: Backpropagation; Dhruv Batra?

• Text Book
  • The elements of statistical learning by Trevor Hastie, Robert Tibshirani, Jerome Friedman
  • Machine learning a probabilistic perspective
  • Learning from Data (A short course) by Yaser S. Abu-Mustafa, Malik Magdon-Ismail, Hsuan-Tien Linhang and co.?Link
• Recommended Online Books

• • http://deeplearning.net/
  • http://neuralnetworksanddeeplearning.com/
• Courses

• • Machine Learning, Oxford  Nando de Freitas?Link
  • Convolutional Neural Networks for Visual Recognition, Stanford (cs231n)?Link
  • A curated list of courses (Recommended)?Link
  • Deep Learning for Natural Language Processing, Stanford?Link
• Video Lectures
  • Essence of Neural Networks  3Blue1Brown?Link
  • Convolutional Neural Networks for Visual Recognition, Stanford (cs231n)  Video Lectures?Link
  • Neural Networks and Deep Learning  deeplearning.ai ?Link

Top Conferences to Follow

• International Conference on Machine Learning (ICML)
• Conference on Neural Information Processing Systems (NIPS)
• International Joint Conference on Artificial Intelligence (ICAI)
• Conference on Computer Vision and Pattern Recognition (CVPR)
• International Conference on Computer Vision (ICCV)
• British Machine Vision Conference (BMVC)

ASSIGNMENTS

• ASSIGNMENT 1:

  • Empirical risk minimization, K-nearest neighbour classifier, LDA