Assignment 04: Automatic Differentiation with PyMatch

Learning Goals

  • Learn how to read Python code
  • Learn about automatic differentiation
  • Implement automatic differentiation functionality
  • Appreciate that automatic differentiation drives much of the speed behind machine learning research

Grading Walk-Throughs

This assignment will be graded as “Nailed It” / “Not Yet” by a TA. To complete (“Nailed It”) the assignment, you must

  1. Complete the assignment and submit your work to gradescope.
  2. Meet with a TA during their mentor session hours.
  3. Complete the walk-through with all group members. I prefer all partners to be present during the walk-through, but you can each meet with the TA separately if needed.
  4. Walk the TA through your answers. Do not expect to make corrections during the walk-through.
  5. The TA will then either
    • mark your assignment as 100% on gradescope, or
    • inform you that you have some corrections to make.
  6. If corrections are needed, then you will need to complete them and conduct a new walk-through with a TA.

If you have concerns about the grading walk-through, you can meet with me after you have first met with a TA.

Overview

You must work on this assignment with a partner; please let me know if you want me to help you find a partner.

For this assignment you will extend functionality of the PyMatch library. This is a minimal automatic differentiation library I created for this course.

A few notes to get you started:

  1. Login to Jupyter Hub
  2. Open a terminal
  3. Activate the course environment: conda activate cs152
  4. Create a new directory for the assignment (anywhere you’d like, but not inside the current cs152 directory)
  5. Clone the match repository: gh repo clone pymatch/pymatch
  6. Read the library’s readme in its entirety
  7. Read through the library code, test file, and demo

Tasks

For the assignment, you must:

  1. Answer the pen-and-paper questions on gradescope (you should open it now and read through all questions).

  2. Add functionality for a leaky rectified linear unit.

    • add a “LeakyReLU” class to nn.py
    • add a “leakyrelu” method to Matrix
    • add a “leakyrelu” method to List2D
    • add a unit test to test.py

  3. Add functionality for mean-absolute-error (MAE) loss (aka L1Loss).

    • add an “MAELoss” class to nn.py
    • add an “abs” (or __abs__) method to Matrix
    • add an “abs” (or __abs__) method to List2D
    • add a unit test to test.py

  4. Create a new demo (based on linear_demo.ipynb).

    • name the file demo_quadratic.ipynb
    • the data must be quadratic instead of linear
    • use your new leaky ReLU and mean-absolute-error functionality
    • hint: you’ll need to change the data, the model, and the loss function

  5. (Optional.) Add any other functionality that you find interesting and submit a pull-request!

Examples

Here are a few examples that you might find useful as you’re working on your assignment.

Submitting Your Assignment

You will submit your responses on gradescope. Only one partner should submit. The submitter will add the other partner through the gradescope interface.

Additional details for using gradescope can be found here: