COMPSCI 697L Deep Learning

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Slides will be posted after each lecture.
Event TypeDateDescriptionCourse Materials
Lecture Sep 7 Intro to Deep Learning, historical context. [slides]
[python/numpy tutorial]
[jupyter tutorial]
Lecture Sep 12 Image classification and the data-driven approach
k-nearest neighbor
Linear classification I		 [slides]
[image classification notes]
[linear classification notes]
Lecture Sep 14 Linear classification II
 [slides]
[linear classification notes]
Lecture Sep 19 Loss functions
Optimization I: Stochastic gradient descent 		[slides]
[optimization notes]
Lecture Sep 21 Backpropagation & Neural Networks I [slides]
[backprop notes]
[Efficient BackProp] (optional)
related: [1], [2], [3] (optional)
Lecture Sep 26 Higher-level representations, image features
Vector, Matrix, and Tensor Derivatives 		[handout]
[slides]
Deep Learning [Nature] (optional)
Lecture Sep 28 Backpropagation & Neural Networks II [slides] (cont.)
tips/tricks: [1], [2] (optional)
Lecture Oct 3 Training Neural Networks I [slides]
[Neural Nets notes 1]
Lecture Oct 5 Training Neural Networks II [slides]
LeNet (optional)
No class Oct 10 Columbus Day; Class will be on Tuesday (Oct 11) instead.
Lecture Oct 11 Training Neural Networks III:
weight initialization, batch normalization 		[slides]
[Neural Nets notes 2]
[Batch Norm]
Lecture Oct 12 Training Neural Network IV:
babysitting the learning process, hyperparameter optimization 		[slides]
[Neural Nets notes 3]
Lecture Oct 17 Training Neural Network IV (cont.):
babysitting the learning process, hyperparameter optimization 		[slides] (cont.)
[Bengio 2012] (optional)
Lecture Oct 19 Project announcement
Training Neural Network V:
parameter updates, model ensembles, dropout 		[slides]
[Stanford cs231n project reports]
Lecture Oct 24 Convolutional Neural Networks: introduction, history, architectures [slides]
[ConvNet notes]
AlexNet (optional)
Guest Lecture Oct 26 Tsung-Yu Lin: Bilinear CNN [slides]
BCNN (optional)
Lecture Oct 31 Convolutional Neural Networks: convolution layer, pooling layer, fully connected layer
Case study of ImageNet challenge winning ConvNets 		[slides] (cont.)
[midterm review sheet]
Midterm Nov 2 In-class midterm
Project proposals due! 		Proposed project topics
Lecture Nov 7 Case study of ImageNet challenge winning ConvNets (cont.)
ConvNets for spatial localization, Object detection 		[slides]
ResNet (optional)
Lecture Nov 9 ConvNets for spatial localization, Object detection (cont.) [slides] (cont.)
FCN (optional)
Lecture Nov 14 Understanding and visualizing Convolutional Neural Networks
Backprop into image: Visualizations, deep dream 		[slides]
[visualization notes]
No class Nov 16 Friday schedule; No class.
No class Nov 21 Thanksgiving; No class.
No class Nov 23 Thanksgiving; No class.
Lecture Nov 28 Artistic style transfer
Adversarial fooling examples
Recurrent Neural Networks (RNN) 		[slides]
DL book RNN chapter (optional)
min-char-rnn, char-rnn, neuraltalk2
Lecture Nov 30 Recurrent Neural Networks (RNN) (cont.)
Long Short Term Memory (LSTM) 		[slides] (cont.)
The Unreasonable Effectiveness of RNN (optional)
Lecture Dec 5 Long Short Term Memory (LSTM) (cont.) [slides] (cont.)
Understanding LSTM Networks (optional)
Lecture Dec 7 Training ConvNets in practice [slides]
Lecture Dec 12 Additional topics in Stanford cs231n
Societal implications of AI 		[slides]
Stanford cs231n slides not covered in our course: Software Packages, Segmentation & Attention, Videos & Unsupervised Learning
Presentation Dec 14 Poster presentations at room 150/151 schedule