Oregon State University

Corvallis, OR 97331

- Recommended Textbook: Deep Learning. (in preparation) Ian Goodfellow. Yoshua Bengio, Aaron Courville (referred to as Deep Learning Book).
- Week 1 (Intro + ML Refresher)
- Week 2 (Basic Neural Networks)
- Week 3 (Neural Networks Optimization 1 and Convolutional Neural Networks)
- Week 4 (Deep learning toolboxes and Convolutional Neural Networks (cont.))
- Week 5 (Other Deep Architectures and Recurrent Neural Models)
- Week 6 (Recurrent Neural Models)
- Week 7 (Neural Network Optimization)
- Week 8 (Neural Network Optimization + NLP Applications)
- Week 9 (Neural Network Regularization + Unsupervised Learning Methods)
- Week 10 (Deep Reinforcement Learning)
- Week 11 (Final Project Presentations)

Tuesday: Introduction to Deep Learning

About the Class

Thursday: Machine Learning Refresher

Reading: Chapter 2, The Elements of Statistical Learning. Hastie, Friedman, Tibshirani.

Chapter 5, Deep Learning Book.

Participate in the general quiz please! (5 points for participation).

Tuesday: Machine Learning Refresher (cont.) + Optimization Primer #1 (nonconvex optimization, stationary points and saddle points, optima, gradients)

Thursday: Basic Feedforward Neural Networks (backpropagation)

Reading: Chapter 4 and Chapter 6, Deep Learning Book

Tuesday: Neural Networks Optimization 1 (Stochastic mini-batch gradient descent, momentum, early stopping and weight decay)

Thursday: Convolutional Neural Networks

Suggested readings: Sections 8.1 - 8.3 and Chapter 9, Deep Learning Book

VGG Convolutional Networks Practical, great hands-on tutorial of CNN.

Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis P. Simard, D. Steinkraus and J.C. Platt. ICDAR 2003. About elastic deformation.

Chapter 5, Pattern Recognition and Machine Learning. Christopher Bishop. Springer, 2006.

January 25th - 29th

Tuesday: Deep Learning toolboxes (Keras, Theano, Caffe)

Thursday: Convolutional Neural Networks (cont.) (incl. Visualization of Convolutional Networks)

Suggested readings: Zeiler and Fergus. Visualizing and Understanding Convolutional Networks. ECCV 2014

K. Simonyan, A. Vedaldi and A. Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, ICLR 2014.

February 1st - 5th

Tuesday: Deconvolution and Other Vision Problems A brief Introduction to Other Neural Architectures

Thursday: Recurrent Neural Models (RNN, LSTM)

February 8th - 12th

Tuesday: Recurrent Neural Models (cont.) (Slides updated!)

Thursday: Project Proposal Presentations

February 15th - 19th

Tuesday: Project Proposal Presentations

Thursday: Recurrent Neural Models (cont.) + Neural Network Optimization #2

Assignment #3 available on Canvas

February 22th - 26th

Tuesday: Neural Network Optimization #2 + Neural Network Regularization

Thursday: Deep Models in NLP (Guest lecture from the NLP group)

February 29th - Mar 4th

Tuesday: Neural Network Regularization

Thursday: Unsupervised Deep Learning Approaches

Suggested readings: Sections 8.4 - 8.7, Chapter 14, Chapter 20, Deep Learning Book

Slides for ResNet

ResNet paper

Mar 7th - Mar 11th

Tuesday: Deep Reinforcement Learning (Guest lecture from Alan Fern)

Thursday: Deep Reinforcement Learning (Guest lecture from Alan Fern)

Mar 14th - Mar 18th

Tuesday: Final presentations #1 (Tuesday Mar. 15th 4PM - 5:30PM at KEC 1003 (note room change!))

Thursday: Final presentations #2 (Thursday Mar 17th 4PM - 5:30PM at KEC 1003 (note room change!))

Note: Your group will only need to attend 1 session during the final presentations although you are welcome to attend both. Let me and Mingbo know about your time constraints before the end the dead week please.