Oregon State University

Corvallis, OR 97331

- Recommended Textbook: Deep Learning. Ian Goodfellow. Yoshua Bengio, Aaron Courville (referred to as Deep Learning Book). MIT Press. 2016
- Week 1 (Intro + ML Refresher)
- Week 2 (Basic Neural Networks)
- Week 3 (Neural Networks Optimization 1 and Convolutional Neural Networks)
- Week 4 (Convolutional Neural Networks (cont.))
- Week 5 (Other Deep Architectures and Recurrent Neural Models)
- Week 6 (Recurrent Neural Models)
- Week 7 (Neural Network Optimization)
- Week 8 (Neural Network Optimization + NLP Applications)
- Week 9 (Neural Network Regularization + Unsupervised Learning Methods)
- Week 10 (Deep Reinforcement Learning)

Tuesday: Introduction to Deep Learning

About the Class

Thursday: Machine Learning Refresher

Reading: Chapter 2, The Elements of Statistical Learning. Hastie, Friedman, Tibshirani.

Chapter 5, Deep Learning Book.

Participate in the general quiz please! (5 points for participation).

Assignment #1 available (due ~~Jan. 26~~ Jan. 31 4:00PM, submit in paper to TA by the time of the start of the class on ~~Jan. 26~~ Jan. 31)

Tuesday: Machine Learning Refresher (cont.) + Optimization Primer #1

Thursday: Optimization Primer #1 + Basic Feedforward Neural Networks (backpropagation)

Reading: Chapter 4 and Chapter 6, Deep Learning Book

Tuesday: Basic Feedforward Neural Networks + Neural Networks Optimization 1 (Stochastic mini-batch gradient descent, momentum, early stopping and weight decay)

Thursday: Convolutional Neural Networks

Suggested readings: Sections 8.1 - 8.3 and Chapter 9, Deep Learning Book

VGG Convolutional Networks Practical, great hands-on tutorial of CNN.

Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis P. Simard, D. Steinkraus and J.C. Platt. ICDAR 2003. About elastic deformation.

Chapter 5, Pattern Recognition and Machine Learning. Christopher Bishop. Springer, 2006.

Assignment #2 available (due Feb 13th 11:59PM, submit zip file on Canvas)

Tuesday: Convolutional Neural Networks

Thursday: Convolutional Neural Networks (cont.) (Visualization of Convolutional Networks)

Suggested readings: Zeiler and Fergus. Visualizing and Understanding Convolutional Networks. ECCV 2014

K. Simonyan, A. Vedaldi and A. Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, ICLR 2014.

Tuesday: Convolutional Neural Networks (cont.) + Deconvolution and Other Vision Problems

Thursday: A brief Introduction to Other Neural Architectures + Recurrent Neural Models (RNN, LSTM)

Participate in the poll for extra session time please!Suggested readings: Goodfellow. NIPS 2016 Tutorial: Generative Adversarial Networks. arXiv:1701:00160

Tuesday: Recurrent Neural Models (Slides updated!)

Thursday: Project Proposal Presentations

Tuesday: Project Proposal Presentations + Tutorial on Theano

Thursday: Recurrent Neural Models (cont.) + Neural Network Optimization #2

Assignment #3 available on Canvas

February 22th - 26th

Tuesday: Recurrent Neural Models (cont.) + Neural Network Optimization #2

Thursday: Neural Network Optimization #2

Mar 6th - Mar 10th

Tuesday: Deep Learning in NLP (Guest lecture from Dr. Liang Huang)

Thursday: Neural Network Regularization

Suggested readings: Sections 8.4 - 8.7, Chapter 14, Chapter 20, Deep Learning Book

Slides for ResNet

ResNet paper

Mar 13th - Mar 17th

Tuesday: Deep Reinforcement Learning (Guest lecture from Alan Fern)

Thursday: Deep Reinforcement Learning (Guest lecture from Alan Fern)