**Assignment 08: Command Line Shell**
# Learning Goals
- Learn how to manage processes with fork and exec.
- Understand child process management.
# Grading Walk-Throughs
This assignment will be graded as "Nailed It" / "Not Yet" by a TA. To pass ("Nailed It") the assignment, you must
1. Complete the assignment and submit your work to gradescope.
+ You should start this assignment in class on the day shown on the calendar.
+ *Complete the assignment as early as possible*.
1. Schedule a time to meet with a TA. You can meet with them after the submission deadline.
+ You must book a time to meet with a TA
+ Sign-up on the Google Sheet *with at least 36 hours of notice*.
+ Contact your TA on Slack after signing up.
+ All partners must meet with the TA. If you cannot all make it at the same time, then each of you needs to schedule a time to meet with the TAs.
1. Walk the TA through your solutions prior to the deadline.
+ Walk-throughs should take no more than 20 minutes.
+ You should be well prepared to walk a TA through your answers.
+ You may not make any significant corrections during the walk-through. You should plan on making corrections afterward and scheduling a new walk-through time. Mistakes are expected---nobody is perfect.
+ You must be prepared to explain your answers and justify your assumptions. TAs do not need to lead you to the correct answer during a walk-through---this is best left to a mentor session.
1. The TA will then either
+ mark your assignment as "Nailed It" on gradescope, or
+ mark your assignment as "Not Yet" and inform you that you have some corrections to make.
1. If corrections are needed, then you will need to complete them and then schedule a new time to meet with the TA.
+ You will ideally complete any needed revisions by the end of the day the following Monday.
If you have concerns about the grading walk-through, you can meet with me after you have first met with a TA.
# Overview
In this assignment, you will be building out the core system-call logic of an interactive shell. You may choose your partner for this assignment; please let me know if you want some help tracking down a partner.
The assignment includes an autograder on gradescope called "Ish Autograder", that you can optionally use it to test your program's functionality ([here are the autograder test files](ishTests.zip)). You **must provide a submission for the "Assignment 08: Command Line Shell" assignment.**
Material for this assignment is available on the course VM in the `/data` directory. You can unpack it directly with:
~~~bash
cd ~/cs105/assignments/
tar xvf /data/A08-Shells.tar
~~~
This command will create the `A08-Shells` directory, which includes three files:
- `Makefile`,
- `ish.c` (the shell you'll code), and
- `snooze.c` (a helper for testing).
# It's shell...-ish
A *shell* is an expert's computer control panel. Here, you'll write the core system-call logic of a very tiny shell (with very few features) we're calling **`ish`**. Common shells include [sh](https://en.wikipedia.org/wiki/Bourne_shell), [bash]("https://en.wikipedia.org/wiki/Bash_%28Unix_shell%29"), [zsh]("https://en.wikipedia.org/wiki/Z_shell"), and [fish]("https://en.wikipedia.org/wiki/Fish_(Unix_shell)").
The starter code includes several parts:
- The `main` function, which consists of a *read-then-evaluate* loop. It uses `getline` to read a line from the user, and then parses the line into a command and its arguments.
- The `setup_signal_handlers` function for setting up interrupt handlers. These handlers can intercept signals from the operating system.
- The definition of `job_t` and its associated functions, `add_job`, `free_job`, and `check_jobs`. You'll use these functions to keep track of background jobs; you'll need to write `check_jobs` yourself.
- The `parse_line` function, which breaks a line up into an array of 'words', the first of which will be interpreted as a command. You should not have to make any changes to this function.
You have six tasks, which will touch three functions in total (plus one you'll write yourself):
1. Get `ish` to run a user-provided command. (`main`)
2. Have `ish` print an exit status if it was non-zero. (`main`)
3. Enable `ish` to run jobs in the background. (`main`)
4. Make `ish` wait for background jobs to finish before exiting the shell. (`main`, `check_jobs`)
5. Make `ish` check on jobs before each prompt. (`main`, `check_jobs`)
6. Make `ish` only check on jobs when something has changed. (`main`, `setup_signal_handlers`)
Start by reading this entire handout. Do not move on from one task until you're confident you have the right behavior.
Note that, throughout, the shell itself only prints to *standard error* (**stderr**, [Unix file descriptor](https://en.wikipedia.org/wiki/File_descriptor) `2`), a special output stream that's different from *standard output* (**stdout**, Unix file descriptor `1`). You will find examples in the provided starter code showing how to use the `fprintf` function with the `FILE` stream `stderr` as its first argument.
## Running the Command
Towards the end of `main`, you'll find the code snippet:
~~~c
// Parse user_input command
int num_words;
char **args = parse_line(user_input, len, &num_words);
assert(args);
assert(args[num_words] == NULL);
// TODO #1: run the user_command
~~~
At this point in the program, `args` is an array of strings (i.e., an array of `char *`).
Your first task is to get `ish` to actually run the user-provided command stored in `args`. There are three steps:
1. Use [`fork`](https://man7.org/linux/man-pages/man2/fork.2.html) to create a new process. This function calls the underlying Linux kernel [system call](https://filippo.io/linux-syscall-table/).
~~~c
pid_t fork(void);
~~~
2. In the child process, use [`execve`](https://man7.org/linux/man-pages/man2/execve.2.html) to execute the program provided as input by the user.
~~~c
int execve(const char *pathname, char *const argv[], char *const envp[]);
~~~
3. In the parent process, use [`waitpid`](https://man7.org/linux/man-pages/man3/wait.3p.html) to wait for the child process to complete.
~~~c
pid_t waitpid(pid_t pid, int *stat_loc, int options);
~~~
You'll want to look at the manpages for each of these commands: run "`man fork`", etc or click the links above. You need to read these pages carefully, especially the `RETURN VALUES` section.
The `execve` function can feel a bit odd until you've seen it a few times. The first item of `args` is what you will pass in as the `pathname`; you will use all of `args` as the second argument to `execve`, `argv`. You should use an empty environment, i.e., an array of `char *` which just has one `NULL` entry (not a null pointer, but a valid `char**` whose first element is `NULL`.
If `execve` fails for some reason, you should indicate failure on standard error; use the [`perror` function](https://man7.org/linux/man-pages/man3/perror.3p.html), following the example below. (Note that `perror` is only for reporting errors. The man pages for `perror` and `errno` should help, if you're confused.)
Note that `execve` doesn't search `PATH` for the `pathname` like a real shell, so we'll use absolute paths to name programs in `ish`.
Here is an implementation in pseudocode:
~~~text
fork
if CHILD
execve
perror
exit
waitpid
~~~
The if-block will only run in the child process. Specifically, code in the block will replace the current running program with whatever is passed in by the user, and we should expect to never hit the `perror` function---though you need to report an error and exit the child process if `execve` fails. You should pass `0` for the last argument to `waitpid`; this means that `waitpid` will "block" and wait for the child process to exit. Later on, you will pass an option to `waitpid` that tells the function to not "block" (don't wait for the child to exit).
### Examples
Your implementation should support the following interaction, where "`^d`" represents pressing control-d in your terminal (which sends an EOF, end-of-file, indicating to `ish` that it should exit). In the examples below, `¢` is the `ish` prompt (where you start typing input).
~~~text
¢ /bin/echo hello
hello
¢ /bin/nonesuch
ish: command error: No such file or directory
¢ ^d
Goodbye!
~~~
And here is a recording of a similar interaction.
If you do not see an animation above this line (or if you see the animation but you don't see the progress bar), you will need to refresh the page (sometimes more than once). Or you can go directly to the player: https://asciinema.org/a/476013
## Printing the Exit Status
Every command has an *exit status*, a number between 0 and 255. Check out [`man 3 exit`](https://man7.org/linux/man-pages/man3/exit.3.html) (where the `3` specifies a section of the manpages---this ensures that you see documentation for the C function and not the shell command).
A process exits with `EXIT_SUCCESS` (defined as `0` in `/usr/include/stdlib.h`) to indicate success; any other value indicates failure. While `main` returns an `int` (a four-byte datatype for our server), it's convention to return values between 0 and 255.
Your next task is to give an informative message when the command exits with failure. Remember to use `fprintf` with `stderr` as the output stream.
The `DESCRIPTION` section of [`waitpid`'s manpage](https://man7.org/linux/man-pages/man3/wait.3p.html) contains important information about how to extract the exit status from the result of `waitpid`. (Specifically, pay attention to `WIFEXITED` and `WEXITSTATUS`.)
Here is an updated implementation in pseudocode:
~~~text
# Task 1
fork
if CHILD
execve
perror
exit
waitpid
# Task 2
ReportExitStatus
~~~
Where `ReportExitStatus` is my psuedocode for what you will implement.
### Examples
Here's an interaction on the VM. On your machine, `tar` might give a different error message.
~~~text
¢ /usr/bin/tar
/usr/bin/tar: You must specify one of the '-Acdtrux' or '–test-label' options
Try '/usr/bin/tar –help' or '/usr/bin/tar –usage' for more information.
ish: status 2
¢ /usr/bin/true
¢ /usr/bin/false
ish: status 1
¢ ^d
Goodbye!`
~~~
And here is a recording of a similar interaction.
If you do not see an animation above this line (or if you see the animation but you don't see the progress bar), you will need to refresh the page (sometimes more than once). Or you can go directly to the player: https://asciinema.org/a/476041
## Running Background Jobs
Now that your shell can run and report on jobs in the foreground, it's time to support running background tasks. Like a real shell, we'll type "`&`" at the end of a command to mark it as "asynchronous", i.e., to run in the background while the shell continues and waits for the user's next input.
The provided code sets the variable `run_in_background` to `1` (true) when `ish` should run the command in the background, and it will otherwise be `0` (false). This logic is right above where you solved tasks 1 and 2.
You *don't* want to wait for background jobs. Instead, you'll add them to a list so that you can keep track of them.
Here is an implementation in pseudocode (including your code from before):
~~~text
# Part of task 1
fork
if CHILD
execve
perror
exit
# Added for task 3
if run_in_background
add_job
else
waitpid # Part of task 1
ReportExitStatus # Task 2
~~~
Notice how your code for task 3 needs to wedge itself within what you completed for tasks 1 and 2.
### Examples
To test background jobs, we need a program that takes some time. A nice way to do this is to write a custom test program---we've provided `snooze.c`, which you should make sure is compiled. In the following example, we first run `snooze` in the foreground, then in the background---while typing `/bin/echo hi` as `snooze` is running. Notice how `snooze`'s output is interleaved with our input!
~~~text
¢ ./snooze
Taking a nap...zzzz...zzzz......yawn! What nice nap.
¢ ./snooze &
Taking a nap.../bin/eczzzz...ho hi
hi
zzzz......yawn! What a nice nap.
¢ ^d
Goodbye!`
~~~
And here is a recording of a similar interaction.
If you do not see an animation above this line (or if you see the animation but you don't see the progress bar), you will need to refresh the page (sometimes more than once). Or you can go directly to the player: https://asciinema.org/a/476998
## Waiting for Background Jobs
Next, we should make our shell wait for background jobs to complete before it fully exits. To see why, look at the following interaction:
~~~text
$ ./ish
¢ ./snooze &
Taking a nap...^d
Goodbye!
$ zzzz...zzzz......yawn! What a nice nap.
~~~
Here "`¢`" is the `ish` prompt and "`$`" is our *actual* shell prompt. And look: somebody is snoring in **our** terminal!
There are two `TODO` marks for task 4: one in `main` and one `check_jobs`.
Let's first address the one in `main`. If there are any background jobs (indicated when `jobs != NULL`) then you should output to **stderr**: "`Jobs are still running...\n`" and call `check_jobs`. For now, we want to use `waitpid` without any options (i.e., `options = 0`).
The `check_jobs` function should wait for every job in the list to complete using the `waitpid`. If a job terminated
- successfully, you should print "`job 'COMMAND' complete`" on its own line,
- otherwise you should print "`job 'COMMAND' status STATUS`".
Here `COMMAND` is the command name (i.e., `job->command`) and `STATUS` is the exit status (i.e., `WEXITSTATUS(j->status)`.
Here is pseudocode for the `check_jobs` function.
~~~text
j = jobs
while j != NULL
waitpid
ReportExitStatus
jobs = jobs->next
free_job
j = jobs
~~~
### Examples
Here, we run `snooze` in the background and immediately exit `ish`. You can see `snooze`'s snoring and wake-up, followed by its wakeup.
~~~text
¢ ./snooze &
¢ Taking a nap...^d
Jobs are still running...
zzzz...zzzz......yawn! What a nice nap.
job ’./snooze’ complete`
Goodbye!`
~~~
Here's another, running `snooze 4` followed by `snooze`, both in the background.
~~~text
¢ ./snooze 4 &
¢ Taking a nap..../snoozzzzz...e &
¢ Taking a nap...zzzz...zzzz...^d
Jobs are still running...
...yawn! What a nice nap.
zzzz......yawn! What a nice nap.
job ’./snooze’ complete
job ’./snooze 4’ status 4`
Goodbye!`
~~~
And here is a recording of a similar interaction.
If you do not see an animation above this line (or if you see the animation but you don't see the progress bar), you will need to refresh the page (sometimes more than once). Or you can go directly to the player: https://asciinema.org/a/477014
## Reporting on Background Job Statuses
We'd like to update the user about background jobs as they complete. To start with, have `main` call `check_jobs` before prompting the user and reading the line (`check_jobs(WNOHANG)`).
Now, alter `check_jobs` to support the new option! You'll need to make a few changes:
1. pass the `options` argument to `waitpid`,
2. save the `pid_t` returned from `waitpid` (`waitpid` does not always return a process ID), and
3. handle the case when the process is still running: print "`job 'COMMAND' still running`".
And now some pseudocode.
~~~text
p = NULL
j = jobs
while j != NULL
pid = waitpid
if pid > 0
ReportExitStatus
# Update linked list (two cases)
if prev == NULL
jobs = jobs->next
free_job
j = jobs
else
prev->next = j->next
free_job
j = prev->next
else
ReportRunning
~~~
### Examples
Here we run a command in the background and hit "return" occasionally over the course of five seconds. Note that `/bin/sleep` is a real, pre-existing utility that's different from the `./snooze` helper we provided; check `man sleep` for more information:
~~~text
¢ /bin/sleep 5 &
¢
job ’/bin/sleep 5’ still running
¢
job ’/bin/sleep 5’ still running
¢
job ’/bin/sleep 5’ still running
¢
job ’/bin/sleep 5’ complete
^d
Goodbye!`
~~~
Here we run two commands in the background, hitting return occasionally.
~~~text
/bin/sleep 5 &
job ’/bin/sleep 5’ still running
/bin/sleep 3 &
job ’/bin/sleep 3’ still running
job ’/bin/sleep 5’ still running
job ’/bin/sleep 3’ still running
job ’/bin/sleep 5’ complete
job ’/bin/sleep 3’ complete
^d
Goodbye!`
~~~
And here is a recording of a similar interaction.
If you do not see an animation above this line (or if you see the animation but you don't see the progress bar), you will need to refresh the page (sometimes more than once). Or you can go directly to the player: https://asciinema.org/a/477023
## Checking Only When Something Changed
It's annoying to update the user unnecessarily---we should only report on background jobs when something has changed. To do so, we'll set up a *signal handler* for the `SIGCHLD` signal. Whenever a child process terminates, the parent process receives a `SIGCHLD` signal. By default, processes ignore these signals... but we'll use them to more cleverly notify the user of changes.
To start, you need to update `setup_signal_handlers` with a signal handler. Signal handlers are very restricted---you shouldn't run a lot of code in them! The standard thing to do is to set a global variable that says, "Hey, something happened!" that your program checks at appropriate points. Just above the `setup_signal_handlers` function, Define a handler function that takes an `int` and returns `void`, and a global variable defaulted to `0` (false). Your handler should set it to `1` to indicate that `SIGCHLD` arrived.
Currently, `setup_signal_handlers` installs a signal handler to ignore `SIGINT` (i.e., control-c). You'll want to set the `action`'s `sa_handler` field to your function, and you'll want to set its `sa_flags` to `SA_RESTART`. (If you *don't* set this flag, your shell might behave strangely when `SIGCHLD` comes in the middle of another system call.) You'll want to:
1. Zero out the `action` struct with `sigemptyset` ([man page](https://man7.org/linux/man-pages/man3/sigemptyset.3p.html)).
2. Set `action.sa_flags` to `SA_RESTART`.
3. Set `action.sa_handler` to your new function.
4. Change the signal action with `sigaction` ([man page](https://www.man7.org/linux/man-pages/man2/sigaction.2.html)).
Finally, use your global variable to condition whether or not you check for jobs before prompting. Specifically, add an if-statement at the top of the while loop that first checks `got_sigchld` before calling `check_jobs`.
### Examples
Here we run a background command that will sleep for five seconds. We hit return a few times and get *no* updates. After waiting four or five seconds, we hit return again and *do* get an update.
~~~text
/bin/sleep 5 &
job ’/bin/sleep 5’ complete
^d
Goodbye!`
~~~
Here's another example, where we run a command in the interim, and then wait several seconds before hitting return gain.
~~~text
/bin/sleep 5 &
/bin/echo hi
hi
job ’/bin/sleep 5’ still running
job ’/bin/sleep 5’ complete
^d
Goodbye!`
~~~
Notice that we get the job update after the call to `echo`. Why? When `echo` terminates, it will send its own `SIGCHLD`, which will get caught by our handler and cause us to update the user about which tasks are running.
And here is a recording of a similar interaction.
If you do not see an animation above this line (or if you see the animation but you don't see the progress bar), you will need to refresh the page (sometimes more than once). Or you can go directly to the player: https://asciinema.org/a/477026
# Submitting Your Assignment
You will submit your code and/or responses on gradescope. **Only one partner should submit.** The submitter will add the other partner through the gradescope interface.
This assignment has two submission parts:
1. An autograder to check your code against.
2. An online submission that you'll use during your walk-through.
To pass the autograder (if one exists for this assignment), your output must exactly match the expected output. Your program output should be similar to the example execution above, and the autograder on gradescope will show you the correct output if yours is not formatted properly. You can use [text-compare](https://text-compare.com/) to compare your output to the expected output and that should give you an idea if you have a misspelled word or extra space (or if I do).
Additional details for using gradescope can be found here:
- [Submitting an Assignment](https://help.gradescope.com/article/ccbpppziu9-student-submit-work)
- [Adding Group Members](https://help.gradescope.com/article/m5qz2xsnjy-student-add-group-members)
- [gradescope Student Help Center](https://help.gradescope.com/category/cyk4ij2dwi-student-workflow)