Schedule

Week	Date	Content
1	Mon Jan 20	(Holiday) Martin Luther King, Jr. Day
	Tue Jan 21	During class Introduction Introduction (annotated)
	Thu Jan 23	Before class Read D2L 1. Introduction Complete questionnaire 1 During class Assignment 1: Neural Network Demos
2	Tue Jan 28	Before class Read Project: Individual Proposals During class Machine learning ethics Machine learning ethics (annotated)
	Thu Jan 30	Before class Read The fastai book chapter 3 Complete questionnaire 2 During class Assignment 2: Ethics Discussions Assignment 1 due
3	Tue Feb 4	Before class Read Project Milestone: Introduction Outline During class A single neuron A single neuron (annotated) Individual proposals due
	Thu Feb 6	Before class Read Project Proposals Complete questionnaire 3 During class Project proposals and team creation No assignment (form project teams) Assignment 02 due
4	Tue Feb 11	Before class Read Project Milestone: Related Works Search During class Neural networks Neural networks (annotated) Project Milestone: Introduction outline due
	Thu Feb 13	Before class Read D2L 4.7. Forward Propagation, Backward Propagation, and Computational Graphs Complete questionnaire 4 During class Assignment 3: Backpropagation
5	Tue Feb 18	Before class Read Project Milestone: First Project Check-In During class PyTorch and automatic differentiation PyTorch and automatic differentiation (annotated Project Milestone: Related Works Search due
	Thu Feb 20	Before class Read D2L 2.5. Automatic Differentiation Complete questionnaire 5 During class Assignment 04: Automatic Differentiation with Match Assignment 03 due
6	Tue Feb 25	Before class Read Project Milestone: Outline and Early Draft During class Mini-Batch stochastic gradient descent (SGD) Mini-Batch stochastic gradient descent (SGD) (annotated) Project Milestone: First Project Check-In due
	Thu Feb 27	Before class Read D2L 12.5. Minibatch Stochastic Gradient Descent Complete questionnaire 6 During class Assignment 05: Mini-Batch Stochastic Gradient Descent Assignment 04 due
7	Tue Mar 4	Before class Read Project Milestone: Methods Outline During class Optimization techniques (momentum and adam) Optimization techniques (momentum and adam) (annotated) Project Milestone: Outline and Early Draft due
	Thu Mar 6	Before class Read D2L 12.6. Momentum Complete questionnaire 7 During class Assignment 06: Optimization Assignment 05 due
8	Tue Mar 11	Before class Read An overview of gradient descent optimization algorithms (Blog) During class Activations, initialization, and input normalization Activations, initialization, and input normalization (annotated) Assignment 6 due
	Thu Mar 13	Before class Read How to initialize deep neural networks? Xavier and Kaiming initialization Complete questionnaire 8 During class Assignment 07: Activations, Initialization, and Normalization
9	Tue Mar 18	Spring break
	Thu Mar 20	Spring break
10	Tue Mar 25	Before class Read Project Milestone: Second Project Check-In During class Overfitting and remedies Overfitting and remedies (annotated) Project Milestone: Methods Outline due
	Thu Mar 27	Before class Read D2L 3.7. Weight Decay Complete questionnaire 9 During class Assignment 08: Overfitting Assignment 07 due
	Fri Mar 28	(Holiday) César Chávez Day
11	Tue Apr 1	Before class Read Project Milestone: Discussion Outline During class Convolutional neural networks (CNNs) Convolutional neural networks (CNNs) (annotated) Project Milestone: Second Project Check-In due
	Thu Apr 3	Before class Read Intuitively Understanding Convolutions for Deep Learning (Blog Post) Complete questionnaire 10 During class Assignment 09: Convolutional Neural Networks Assignment 08 due
12	Tue Apr 8	Before class Read Project Milestone: Complete First Draft During class Recurrent neural networks (RNNs) Recurrent neural networks (RNNs) (annotated) Project Milestone: Discussions Outline due
	Thu Apr 10	Before class Read The Unreasonable Effectiveness of Recurrent Neural Networks (Blog Post) Complete questionnaire 11 During class Assignment 10: Recurrent Neural Networks Assignment 09 due
13	Tue Apr 15	Before class Getting Started with gradio No questionnaire During class Inference and Deploying Applications No annotated slides (see the in class coding examples in the course repository)
	Thu Apr 17	Before class What is MLOps? No questionnaire During class Assignment 11: Inference and Applications Assignment 10 due
14	Tue Apr 22	Before class Read Project Milestone: Completed Project During class Attention and Transformers (annotated) Project Milestone: Complete First Draft due
	Thu Apr 24	Before class Read The Illustrated Transformer Complete questionnaire 12 During class Assignment 12: Project Peer Reviews Assignment 11 due
15	Tue Apr 29	Before class Read How GPT3 Works - Visualizations and Animations No questionnaire During class Large language models (LLMs)
	Thu May 1	Before class Read Practical Generative Adversarial Networks for Beginners No questionnaire During class Continue discussion on LLMs Assignment 12 due
16	Tue May 6	Before class Read The Illustrated Stable Diffusion No questionnaire During class Stable Diffusion Project Milestone: Completed Project due
	Thu May 8	Reading day
	Fri May 9	Reading day
17	Mon May 12	Revised projects due (only applicable to groups without a senior)

Modules

Module 1: Introduction, Demos, and Ethics

We will start by digging into demos created using PyTorch, Hugging Face, and Gradio. I want to quickly show you some application areas and give you an idea of the different possibilities for projects.

We will discuss applications of deep learning and ethical implications.

Module 2: Neural Networks from First Principles

Next we will learn how to build neural networks from scratch by deriving the backpropagation algorithm by hand and using Python for implementations.

Module 3: Advanced Topics

The next major chunk of the class will be devoted to higher-level concepts and state-of-the-art techniques.

Module 4: Project Demonstrations

We will end the semester with project presentations/demonstrations.