Assignment 06: Neural Network Optimization

Learning Goals

  • Learn how to implement and train a neural network in PyTorch
  • Learn how to read the PyTorch documentation and make changes to an existing notebook
  • Learn how to implement gradient descent optimization methods
  • Practice converting equations into code
  • Realize that each parameter in a network must have its own values for momentum, squared gradients, etc.

Grading Walk-Throughs

This assignment will be graded as “Nailed It” / “Not Yet” by a TA. To complete (“Nailed It”) the assignment, you must

  1. Complete the assignment and submit your work to gradescope.
  2. Meet with a TA during their mentor session hours.
  3. Complete the walk-through with all group members. I prefer all partners to be present during the walk-through, but you can each meet with the TA separately if needed.
  4. Walk the TA through your answers. Do not expect to make corrections during the walk-through.
  5. The TA will then either
    • mark your assignment as 100% on gradescope, or
    • inform you that you have some corrections to make.
  6. If corrections are needed, then you will need to complete them and conduct a new walk-through with a TA.

If you have concerns about the grading walk-through, you can meet with me after you have first met with a TA.

Overview

You must work on this assignment with a partner; please let me know if you want me to help you find a partner.

For this assignment you will play around with the Optimization notebook. You should clone (or pull) the repository onto the server. You will answer the questions on gradescope (the same questions appear in the notebook).

For anyone curious about exponential moving averages I’ve provided a demo in the form of this notebook.

Submitting Your Assignment

You will submit your responses on gradescope. Only one partner should submit. The submitter will add the other partner through the gradescope interface.

Additional details for using gradescope can be found here: