CS762 Advanced Deep Learning

CS762, Fall 2022
Department of Computer Sciences
University of Wisconsin–Madison


Tentative Schedule (Subject to Change)

Week Date Topic Reading materials Assignments
1 Thursday, September 8 Course overview and introduction
Sign up for paper presentations and scribes (link to Google sheet)
2 Tuesday, September 13 Evolution of Neural Architecture (lecture) D2L Book Chapter 7 & 10
2 Thursday, September 15 Evolution of Neural Architecture II CNNs
ImageNet Classification with Deep Convolutional Neural Networks
Deep Residual Learning for Image Recognition
Densely Connected Convolutional Networks
A ConvNet for the 2020s (deep dive)
3 Tuesday, September 20 Evolution of Neural Architecture III Transformers
Attention Is All You Need
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (deep dive)
Hierarchical Vision Transformer using Shifted Windows
HOW DO VISION TRANSFORMERS WORK?
3 Thursday, September 22 Trustworthy Deep Learning I (lecture) Goodfellow-Bengio-Courville Chapter 7.5, 7.13
Team registration (link to Google sheet)
4 Tuesday, September 27 Trustworthy Deep Learning II Out-of-distribution Reliability
Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Energy-based Out-of-distribution Detection
VOS: Learning What You Don’t Know by Virtual Outlier Synthesis (deep dive)
4 Thursday, September 29 Trustworthy Deep Learning III Adversarial Robustness
Intriguing properties of neural networks
Towards Deep Learning Models Resistant to Adversarial Attacks
Robust Physical-World Attacks on Deep Learning Models
Unlabeled Data Improves Adversarial Robustness
Defense Against Adversarial Images using Web-Scale Nearest-Neighbor Search
Understanding and Mitigating the Tradeoff Between Robustness and Accuracy (deep dive)
5 Tuesday, October 4 No class (use the time to discuss the project proposal with team)
Project proposal submission deadline on October 7 Midnight (download proposal latex template here)
5 Thursday, October 6 Trustworthy Deep Learning IV Fairness and Group Robustness
Equality of Opportunity in Supervised Learning (deep dive)
An investigation of why overparameterization exacerbates spurious correlations
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
6 Tuesday, October 11 Meetings with Instructor to Review Project Proposals (optional)
Signup sheet for 15-min slot (link to Google sheet)
8 Thursday, October 13 Interpretable Deep Learning I (Lecture)
9 Tuesday, October 18 Interpretable Deep Learning II Learning Deep Features for Discriminative Localization (CAM)
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
Towards A Rigorous Science of Interpretable Machine Learning
"Why Should I Trust You?": Explaining the Predictions of Any Classifier (deep dive)
10 Thursday, October 20 Deep Learning Generalization and Theory I
10 Tuesday, October 25 Deep Learning Generalization and Theory II Exploring Generalization in Deep Learning
Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks
Visualizing the Loss Landscape of Neural Nets
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Deep learning: a statistical viewpoint
Deep Double Descent: Where Bigger Models and More Data Hurt (deep dive)
11 Thursday, October27 Deep Learning Generalization and Theory III Generalization under distributional shift
Invariant Risk Minimization
Self-training Avoids Using Spurious Features Under Domain Shift
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution(deep dive)
6 Tuesday, November 1 Learning with Less Supervision I (Lecture)
6 Thursday, November 3 Learning with Less Supervision II Large-scale pre-training
Exploring the Limits of Weakly Supervised Pretraining
Big Transfer (BiT): General Visual Representation Learning
Rethinking Pre-training and Self-training
Learning Transferable Visual Models From Natural Language Supervision (deep dive)
7 Tuesday, Novermber 8 Learning with Less Supervision III Self-supervised learning
A Simple Framework for Contrastive Learning of Visual Representations
Supervised Contrastive Learning
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere (deep dive)
Understanding self-supervised Learning Dynamics without Contrastive Pairs
7 Thursday, November 10 Learning with Less Supervision IV Data Augmentation
Automating the Art of Data Augmentation: Overview
Automating the Art of Data Augmentation: Practice
Automating the Art of Data Augmentation: Theory
Automating the Art of Data Augmentation: New Direction
Unsupervised Data Augmentation for Consistency Training
On the Generalization Effects of Linear Transformations in Data Augmentation (deep dive)
8 Tuesday, November 15 Continual / Lifelong Learning I (Lecture)
8 Thursday, November 17 Continual / Lifelong Learning II iCaRL - Incremental Classifier and Representation Learning
LwF-Learning without Forgetting
Overcoming catastrophic forgetting in neural networks
Dark Experience for General Continual Learning: a Strong, Simple Baseline (deep dive)
9 Tuesday, November 22 Continual / Lifelong Learning III Representational Continuity for Unsupervised Continual Learning (deep dive)
Self-Supervised Models are Continual Learners
9 Thursday, November 24 No class (Thanksgiving)
10 Tuesday, November 29 Deep Generative Model I (Lecture) Goodfellow-Bengio-Courville Chapter 20
10 Thursday, December 1 Deep Generative Model II Generative Adversarial Networks
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
Stacked Generative Adversarial Networks
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Wasserstein GAN (deep dive)
Self-Attention Generative Adversarial Networks
11 Tuesday, December 6 Deep Generative Model III Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
StarGAN v2: Diverse Image Synthesis for Multiple Domains
A Style-Based Generator Architecture for Generative Adversarial Networks
Hierarchical Text-Conditional Image Generation with CLIP Latents (deep dive)
11 Thursday, December 8 Final project presentation (Part I)
12 Tuesday, December 13 Final project presentation (Part II)
Wednesday, December 21 Final project written report due (by end of the day)