ORIE 4741: Lectures

Slides, notes, demos, and videos for lectures will be posted here. Topics may change based on student interest.

Lectures by date: topics and videos

• 9-3-2020 Introduction video

• 9-8-2020 Exploratory data analysis + finish introduction video

• 9-10-2020 Perceptron VOD (available later) Zoom recording (available earlier, but not in China)

• 9-15-2020 Finish EDA + start linear models VOD Zoom recording

• 9-17-2020 Linear models (gradient descent) VOD Zoom recording

• 9-22-2020 Linear models (parallel and stochastic) VOD Zoom recording

• 9-24-2020 Linear models (convexity, proofs) VOD Zoom recording

• 9-29-2020 Feature engineering (polynomials) VOD Zoom recording

• 10-1-2020 Feature engineering (boolean, ordinal, nominal, text) VOD Zoom recording

• 10-6-2020 Train test validate VOD Zoom recording

• 10-8-2020 Generalization VOD Zoom recording

• 10-13-2020 Underdetermined least squares VOD Zoom recording

• 10-15-2020 Quadratic regularization + bootstrap VOD Zoom recording

• 10-20-2020 Bias variance tradeoff VOD Zoom recording

• 10-22-2020 Real loss functions VOD Zoom recording

• 10-27-2020 Boolean loss functions VOD Zoom recording

• 10-29-2020 Multiclass and ordinal loss functions VOD Zoom recording

• 11-3-2020 Regularization VOD Zoom recording

• 11-5-2020 Limits VOD Zoom recording

• 11-10-2020 Unsupervised learning (PCA) VOD Zoom recording

• 11-12-2020 Generalized low rank models VOD Zoom recording

• 12-1-2020 Missing Data Imputation using Gaussian copula VOD

• 12-3-2020 AutoML VOD

• 12-8-2020 Rich Caruana: InterpretML: Explainable Boosting Machines (EBMs) VOD

• 12-10-2020 Alejandro Schuler: Clinical Trials and Digital Twins VOD

• 12-15-2020 Hamdan Azhar: Telling stories with data VOD

Course topics: slides, demos, and other resources

• Introduction

  • Introduction

  • Demo: SIR model

  • Exploratory data analysis

  • Demo: exploratory data analysis

• Linear models

  • The perceptron algorithm

  • Linear regression

  • Demo: Linear models

  • Demo: Gradient descent

  • Demo: QR decomposition

• Generalization and overfitting

  • Feature engineering and overfitting

  • Demo: Preprocessing crime data set

  • Demo: Predicting crime

  • Train, test, validate

  • Generalization

  • Underdetermined least squares and quadratic regularization

  • Demo: Singular value decomposition

  • The bootstrap and the bias variance tradeoff

• Midterm exam

• Loss functions for messy labels

  • Logistic regression and Support Vector Machines (SVMs)

  • Multiclass and ordinal regression

  • Optimization: subgradient method

• Regularization for messy features

  • Regularization and interpretations

  • Demo: Proximal gradient starter code

  • Demo: Regularized Regression

  • Optimization: proximal gradient method

  • Demo: Proximal gradient method

• Unsupervised learning

  • Missing data and PCA

  • Sparse PCA, NNMF, k-means

  • Optimization: AM and PALM

• Missing value imputation using Gaussian copula

  • Demo: Impute GSS data using Gaussian copula

• Introduction to AutoML

  • Demo: APIs of some AutoML systems

• InterpretML: Explainable Boosting Machines (EBMs)

• Telling stories with data

• Nonlinear models

  • Decision trees

• Learning from prototypes

  • Nearest neighbors

  • Smoothing

  • Graphs and networks

• Trustworthy machine learning

  • Limitations of predictive modeling

  • Fairness

  • Interpretability

  • Causality

Other resources

• Review/overview

  • Midterm review

  • Final review

• Julia

  • Demo tutorial: Julia syntax

• GitHub

  • Demo tutorial: Github basics