Monday: Introduction to Deep Learning
About the Class
Wednesday: Introduction to Deep Learning + Optimization Primer 1

Reading: Chapter 2, The Elements of Statistical Learning. Hastie, Friedman, Tibshirani.
Chapter 5, Deep Learning Book.

Wednesday: Basic Feedforward Network

Reading: Chapter 4 and Chapter 6, Deep Learning Book

Monday: Neural Networks Optimization 1 (Stochastic mini-batch gradient descent, momentum, early stopping and weight decay)

Wednesday: Theoretical Implications + Convolutional Neural Networks

Suggested readings: Sections 8.1 - 8.3 and Chapter 9, Deep Learning Book
VGG Convolutional Networks Practical, great hands-on tutorial of CNN.
Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis P. Simard, D. Steinkraus and J.C. Platt. ICDAR 2003. About elastic deformation.
Chapter 5, Pattern Recognition and Machine Learning. Christopher Bishop. Springer, 2006.

Monday: Convolutional Neural Networks

Wednesday: Convolutional Neural Networks (cont.) (Visualization of Convolutional Networks)

Suggested readings: Chapter 9, Deep Learning Book
K Chatfield, K Simonyan, A Vedaldi, A Zisserman. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531
K. Simonyan, A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. ICLR 2015
Zeiler and Fergus. Visualizing and Understanding Convolutional Networks. ECCV 2014
K. Simonyan, A. Vedaldi and A. Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, ICLR 2014.
J. Zhang, J. Lin, J. Brandt, X. Shen, S. Sclaroff. Top-Down Neural Attention by Excitation Backprop. ECCV 2016.

Monday: Quiz + Deep Learning Toolboxes

Wednesday: Deconvolution and Other Vision Problems + A brief Introduction to Other Neural Architectures

Suggested readings: I. Goodfellow. NIPS 2016 Tutorial: Generative Adversarial Networks. arXiv:1701:00160
I. Gulrajani, et al. Improved Training of Wasserstein GANs. NIPS 2017

Monday: Recurrent Neural Models

Wednesday: Project Proposal Presentations

Suggested readings: K. Greff et al. LSTM: A Search Space Odyssey . IEEE Transactions on Neural Networks and Learning Systems. 2017, 28(10):2222-2232.

Monday: Project Proposal Presentations + Recurrent Neural Networks (cont.)

Wednesday: Recurrent Neural Networks (cont.) + Neural Network Regularization

Suggested readings: A. Graves. Generating Sequences with Recurrent Neural Networks . arXiv:1308.0850.
Vinyals, Oriol, et al. Show and tell: A neural image caption generator. CVPR 2015.

Monday: Neural Network Regularization

Wednesday: Quiz + LSTM Training + Neural Network Optimization #2

Suggested readings: Sergey Ioffe, Christian Szegedy. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift ICML 2015.
Diederik Kingma, Jimmy Ba. Adam: A Method for Stochastic Optimization ICLR 2015.

Monday: ResNet and New Architectures

Wednesday: Deep Learning in NLP (Guest lecture from Dr. Liang Huang)

Suggested readings: Slides for ResNet
ResNet paper
Slides for DenseNet
DenseNet Paper

Monday: Unsupervised Deep Learning

Wednesday: Deep Reinforcement Learning (Guest lecture from Dr. Alan Fern)