ORIE 4741: Lectures

Slides, notes, demos, and videos for lectures will be posted here. Topics may change based on student interest.

Lectures by date: topics and videos

• 9-3-2020 Introduction video

• 9-8-2020 Exploratory data analysis + finish introduction video

• 9-10-2020 Perceptron VOD (available later) Zoom recording (available earlier, but not in China)

• 9-15-2020 Finish EDA + start linear models VOD Zoom recording

• 9-17-2020 Linear models (gradient descent) VOD Zoom recording

• 9-22-2020 Linear models (parallel and stochastic) VOD Zoom recording

• 9-24-2020 Linear models (convexity, proofs) VOD Zoom recording

• 9-29-2020 Feature engineering (polynomials) VOD Zoom recording

• 10-1-2020 Feature engineering (boolean, ordinal, nominal, text) VOD Zoom recording

• 10-6-2020 Train test validate VOD Zoom recording

• 10-8-2020 Generalization VOD Zoom recording

• 10-13-2020 Underdetermined least squares VOD Zoom recording

• 10-15-2020 Quadratic regularization + bootstrap VOD Zoom recording

• 10-20-2020 Bias variance tradeoff VOD Zoom recording

• 10-22-2020 Real loss functions VOD Zoom recording

• 10-27-2020 Boolean loss functions Zoom recording

Course topics: slides, demos, and other resources

• Introduction

  • Introduction

  • Demo: SIR model

  • Exploratory data analysis

  • Demo: exploratory data analysis

• Linear models

  • The perceptron algorithm

  • Linear regression

  • Demo: Linear models

  • Demo: Gradient descent

  • Demo: QR decomposition

• Generalization and overfitting

  • Feature engineering and overfitting

  • Demo: Preprocessing crime data set

  • Demo: Predicting crime

  • Train, test, validate

  • Generalization

  • Underdetermined least squares and quadratic regularization

  • Demo: Singular value decomposition

  • The bootstrap and the bias variance tradeoff

• Midterm exam

• Loss functions for messy labels

  • Logistic regression and Support Vector Machines (SVMs)

  • Multiclass and ordinal regression

  • Optimization: subgradient method

• Regularization for messy features

  • Regularization and interpretations

  • Demo: Proximal gradient starter code

  • Demo: Regularized Regression

  • Optimization: proximal gradient method

  • Demo: Proximal gradient method

• Unsupervised learning

  • Missing data and PCA

  • Sparse PCA, NNMF, k-means

  • Optimization: AM and PALM

• Nonlinear models

  • Decision trees

• Learning from prototypes

  • Nearest neighbors

  • Smoothing

  • Graphs and networks

• Trustworthy machine learning

  • Limitations of predictive modeling

  • Fairness

  • Interpretability

  • Causality

Other resources

• Review/overview

  • Midterm review

  • Final review

• Julia

  • Demo tutorial: Julia syntax

• GitHub

  • Demo tutorial: Github basics