ORIE 4741: Lectures

Slides and notes for lectures will be posted here. Topics may change based on student interest. Lecture videos (Cornell only) available on Videonote one week after lecture.

Course topics

• Introduction

  • Introduction

  • Exploratory data analysis

  • Demo: exploratory data analysis

• Linear models

  • The perceptron algorithm

  • Linear regression

  • Optimization: gradient method and QR

  • Underdetermined least squares and quadratic regularization

  • Optimization: SVD

  • Demo: Linear models

  • Demo: Gradient descent

  • Demo: QR decomposition

  • Demo: Singular value decomposition

  • Demo: Preprocessing crime data set

  • Demo: Predicting crime

• Generalization and overfitting

  • Feature engineering and overfitting

  • Train, test, validate

  • Generalization

  • Bootstrap and maximum likelihood estimation

  • Bias variance tradeoff

• Nonlinear models

  • Decision trees

• Regularization for messy features

  • Regularization and interpretations

  • Optimization: proximal gradient method

  • Demo: Regularized Regression

• Loss functions for messy labels

  • Logistic regression and Support Vector Machines (SVMs)

  • Multiclass and ordinal regression

  • Optimization: proximal subgradient

• Unsupervised learning

  • Missing data and PCA

  • Sparse PCA, NNMF, k-means

  • Optimization: AM and PALM

• Learning from prototypes

  • Nearest neighbors

  • Smoothing

  • Graphs and networks

• Advanced topics

  • Limitations of predictive modeling

  • Parallelism paradigms

  • Data streams

Other resources

• Review/overview

  • Midterm review

  • Final review

• Julia

  • Demo tutorial: Julia syntax

• GitHub

  • Demo tutorial: Github basics