**Assignment 04: Automatic Differentiation with Match** *Due date found on gradescope* [Back to Neural Networks](http://cs.pomona.edu/classes/cs152/) # Learning Goals - Learn how to read Python code - Learn about automatic differentiation - Implement automatic differentiation functionality - Appreciate that automatic differentiation drives much of the speed behind machine learning research # Grading Walk-Throughs This assignment will be graded as pass/needs-revisions by a TA. To pass the assignment, you must 1. Complete the assignment and submit your work to gradescope. - You should start this assignment in class on the day shown on the calendar. - **Complete the assignment as early as possible**. 2. Schedule a time to meet with a TA prior to the deadline. - You must book a time to meet with a TA - Sign-up on the Google Sheet **with at least 36 hours of notice**. - Contact your TA on Slack after signing-up. - All partners must meet with the TA. If you can't all make it at the same time, then each of you needs to schedule a time to meet with the TAs. 3. Walk the TA through your solutions prior to the deadline. - Walk-throughs should take no more than 20 minutes. - You should be well prepared to walk a TA through your answers. - You may not make any significant corrections during the walk-through. You should plan on making corrections afterward and scheduling a new walk-through time. Mistakes are expected--nobody is perfect. - You must be prepared to explain your answers and justify your assumptions. TAs do not need to lead you to the correct answer during a walk-through--this is best left to a mentor session. 4. The TA will then either - mark your assignment as "pass" on gradescope, or - mark your assignment as "needs-revisions" and inform you that you have some corrections to make. 5. If corrections are needed, then you will need to complete them and then schedule a new time to meet with the TA. - You will ideally complete any needed revisions by the end of the day the following Monday If you have concerns about the grading walk-through, you can meet with me after you've first met with a TA. # Overview You must work on this assignment with a partner; please let me know if you want me to help you find a partner. For this assignment you will extend functionality of the [Match](https://github.com/anthonyjclark/match) library. This is a minimal automatic differentiation library created for this course. A few notes to get you started: - Login to the course server ([instructions](https://cs.pomona.edu/classes/cs152/pages/pomona-hpc.mdeep.html)). You won't need tmux, but you should use port forwarding and the course mamba environment. - Clone the [match repository](https://github.com/anthonyjclark/match). - Read the library's [readme](https://github.com/anthonyjclark/match/blob/main/README.md). - Read through the library code, test file, and demo. # Tasks The the assignment, you must: 1. Add functionality for a [leaky rectified linear unit]("https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Leaky_ReLU") activation function. + add a "`leakyrelu`" method to [`List2D`](https://github.com/anthonyjclark/match/blob/main/match/list2d.py) + add a "`leakyrelu`" method to [`Matrix`](https://github.com/anthonyjclark/match/blob/main/match/matrix.py) + add a "`LeakyReLU`" class to [`nn.py`](https://github.com/anthonyjclark/match/blob/main/match/nn.py) * Here is the [PyTorch documentation](https://pytorch.org/docs/stable/generated/torch.nn.LeakyReLU.html#torch.nn.LeakyReLU) on their `LeakyReLu` class. + add a unit test to [`test.py`](https://github.com/anthonyjclark/match/blob/main/test.py) 2. Add functionality for mean-absolute-error (MAE) loss class. + add an "`abs`" method to [`List2D`](https://github.com/anthonyjclark/match/blob/main/match/list2d.py) + add an "`abs`" method to [`Matrix`](https://github.com/anthonyjclark/match/blob/main/match/matrix.py) + add an "`MAELoss`" class to [`nn.py`](https://github.com/anthonyjclark/match/blob/main/match/nn.py) * Here is the [PyTorch documentation](https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html#torch.nn.L1Loss) on their `MAELoss` class (called `L1Loss`). + add a unit test to [`test.py`](https://github.com/anthonyjclark/match/blob/main/test.py) 3. Create a new demo (based on [`linear_demo.ipynb`](https://github.com/anthonyjclark/match/blob/main/demo_linear.ipynb)) in which the data is quadratic instead of linear. Call the file `quadratic_demo.ipnb` (or just write it as a Python script). Your demo must use your new functionality. 4. (Optional.) Add any other functionality that you find interesting and submit a pull-request! ## Examples Here are a few examples that you might find useful as you're working on your assignment. - [horch/horch/_operators at master · sbl1996/horch](https://github.com/sbl1996/horch/tree/master/horch/_operators) - [autodiff/ops.py at main · ryanirl/autodiff](https://github.com/ryanirl/autodiff/blob/main/autodiff/ops.py) - [tangent/grads.py at master · google/tangent](https://github.com/google/tangent/blob/master/tangent/grads.py) # Submitting Your Assignment You will submit your code and/or responses on gradescope. **Only one partner should submit.** The submitter will add the other partner through the gradescope interface. To pass the autograder (if one exists for this assignment), your output must exactly match the expected output. Your program output should be similar to the example execution above, and the autograder on gradescope will show you the correct output if yours is not formatted properly. You can use [text-compare](https://text-compare.com/) to compare your output to the expected output and that should give you an idea if you have a misspelled word or extra space (or if I do). Additional details for using gradescope can be found here: - [Submitting an Assignment](https://help.gradescope.com/article/ccbpppziu9-student-submit-work) - [Adding Group Members](https://help.gradescope.com/article/m5qz2xsnjy-student-add-group-members) - [gradescope Student Help Center](https://help.gradescope.com/category/cyk4ij2dwi-student-workflow)