Project 2: The Unix Shell
Questions about the project? Send them to
Final Deadline: Friday, 2/11, by 9pm.
This project must be done alone. You can talk to your colleagues about it, but every line of code must be written and understood by you. Of course, you can always ask the TAs and professors for help.
IT IS A LOT MORE WORK THAN P1 - YOU HAVE BEEN WARNED.
There are three objectives to this assignment:
In this assignment, you will implement a command line interpreter or, as it is more commonly known, a shell. The shell should operate in this basic way: when you type in a command (in response to its prompt), the shell creates a child process that executes the command you entered and then prompts for more user input when it has finished.
The shells you implement will be similar to, but simpler than,
the one you run every day in Unix. You can find out which shell you
are running by typing
Your basic shell is basically an interactive loop: it repeatedly
prints a prompt
prompt> ./mysh mysh>
You should structure your shell such that it creates a new process for each new command (there are a few exceptions to this, which we will discuss below). There are two advantages of creating a new process. First, it protects the main shell process from any errors that occur in the new command. Second, it allows for concurrency; that is, multiple commands can be started and allowed to execute simultaneously.
Your basic shell should be able to parse a command, and run the
program corresponding to the command. For example, if the user types
Note that the shell itself does not "implement"
The maximum length of a line of input to the shell is
After you get your basic shell running, your shell is not too fun
if you cannot run multiple jobs on a single command line. To do that,
we use the
For example, if the user types
Whenever your shell accepts a command, it should check whether the command
is a built-in command or not. If it is, it should not be executed like
other programs. Instead, your shell will invoke your implementation of the
built-in command. For example, to implement the
So far, you have added your own
exit, cd, and pwd formats. The formats for exit , cd and pwd are:
[optionalSpace]exit[optionalSpace] [optionalSpace]pwd[optionalSpace] [optionalSpace]cd[optionalSpace] [optionalSpace]cd[oneOrMoreSpace]dir[optionalSpace]
When you run "cd" (without arguments), your shell should
change the working directory to the path stored in the $HOME environment
You do not have to support tilde (~). Although in a typical Unix shell you could go to a user's directory by typing "cd ~username", in this project you do not have to deal with tilde. You should treat it like a common character, i.e. you should just pass the whole word (e.g. "~username") to chdir(), and chdir will return error.
Basically, when a user types pwd, you simply call getcwd(). When a user changes the current working directory (e.g. "cd somepath"), you simply call chdir(). Hence, if you run your shell, and then run pwd, it should look like this:
% cd % pwd /afs/cs.wisc.edu/u/m/j/username % echo $PWD /u/m/j/username % ./mysh mysh> pwd /afs/cs.wisc.edu/u/m/j/username
Many times, a shell user prefers to send the output of his/her program
to a file rather than to the screen. The UNIX shell provides this nice
feature with the
For example, if a user types
Here are some redirections that should not work:
ls > out1 out2 ls > out1 out2 out3 ls > out1 > out2
As one final addendum, your shell should able to background processes. To put a process in the background, the user types & after the command. For example:
mysh> ls & mysh>
When a user backgrounds a process, control returns immediately to the
shell, allowing the user to type more. However, sometimes a user wishes to
wait for a backgrounded process to complete. They do this by typing a built-in
shell command we will call
The one and only error message. A section about Error Message has been added. In summary, you should print this one and only error message whenever you encounter an error of any type:
char error_message = "An error has occurred\n"; write(STDERR_FILENO, error_message, strlen(error_message));
The error message should be printed to stderr (standard error). Also, do not add whitespaces or tabs or extra error messages.
There is a difference between errors that your shell catches and those that
the program catches. Your shell should catch all the syntax errors specified
in this project page. If the syntax of the command looks perfect, you simply
run the specified program. If there is any program-related errors
(e.g. invalid arguments to
Zero or more spaces can exist between a command and the shell
special characters (i.e.
mysh> ls;ls;ls mysh> ls ; ls ; ls mysh> ls>a; ls > b; ls> c; ls >d
So far, you have run the shell in interactive mode. Most of the time, testing your shell in interactive mode is time-consuming. To make testing much faster, your shell should support batch mode .
In interactive mode, you display a prompt and the user of the shell will type in one or more commands at the prompt. In batch mode, your shell is started by specifying a batch file on its command line; the batch file contains the same list of commands as you would have typed in the interactive mode.
In batch mode, you should not display a prompt. You should print each line you read from the batch file back to the user before executing it; this will help you when you debug your shells (and us when we test your programs). To print the command line, do not use printf because printf will buffer the string in the C library and will not work as expected when you perform automated testing. To print the command line, use write(STDOUT_FILENO, ...) this way:
write(STDOUT_FILENO, cmdline, strlen(cmdline));
In both interactive and batch mode, your shell should terminates when it
To run the batch mode, your C program must be invoked exactly as follows:
The command line arguments to your shell are to be interpreted as follows.
batchFile: an optional argument (often indicated by square brackets as above). If present, your shell will read each line of the batchFile for commands to be executed. If not present or readable, you should print the one and only error message (see Error Message section below).
Implementing the batch mode should be very straightforward if your shell code is nicely structured. The batch file basically contains the same exact lines that you would have typed interactively in your shell. For example, if in the interactive mode, you test your program with these inputs:
emperor1% ./mysh mysh> ls ; who ; ps some output printed here mysh> ls > /tmp/ls-out;;;; ps > /non-existing-dir/file; some output and error printed here mysh> ls-who-ps some error printed here
then you could cut your testing time by putting the same input lines to a batch file (for example myBatchFile):
ls ; who ; ps ls > /tmp/ls-out;;;; ps > /non-existing-dir/file; ls-who-ps
and run your shell in batch mode:
emperor1% ./mysh myBatchFile
In this example, the output of the batch mode should look like this:
ls ; who ; ps some output printed here ls > /tmp/ls-out;;;; ps > /non-existing-dir/file; some output and error printed here ls-who-ps some error printed here
Important Note: To automate grading, we will heavily use the batch mode . If you do everything correctly except the batch mode, you could be in trouble. Hence, make sure you can read and run the commands in the batch file. Soon, we will provide some batch files for you to test your program.
Defensive Programming and Error Messages
As in the first project, in this project defensive programming is also required. Your program should check all parameters, error-codes, etc. before it trusts them. In general, there should be no circumstances in which your C program will core dump, hang indefinitely, or prematurely terminate. Therefore, your program must respond to all input in a reasonable manner; by "reasonable", we mean print the error message (as specified in the next paragraph) and either continue processing or exit, depending upon the situation.
Since your code will be graded with automated testing, you should print this one and only error message whenever you encounter an error of any type:
char error_message = "An error has occurred\n"; write(STDERR_FILENO, error_message, strlen(error_message));
For this project, the error message should be printed to stderr Also, do not attempt to add whitespaces or tabs or extra error messages.
You should consider the following situations as errors; in each
case, your shell should print the error message to
For the following situation, you should print the error message to
Your shell should also be able to handle the following scenarios below, which are not errors . A reasonable way to check if something is not an error is to run the command line in the real Unix shell.
All of these requirements will be tested extensively.
Writing your shell in a simple manner is a matter of finding the relevant library routines and calling them properly. To simplify things for you in this assignment, we will suggest a few library routines you may want to use to make your coding easier. (Do not expect this detailed of advice for future assignments!) You are free to use these routines if you want or to disregard our suggestions. To find information on these library routines, look at the manual pages (using the Unix command man ).
Parsing: For reading lines of input, you may want to look at fgets(). To open a file and get a handle with type FILE * , look into fopen(). Be sure to check the return code of these routines for errors! (If you see an error, the routine perror() is useful for displaying the problem. But do not print the error message from perror() to the screen. You should only print the one and only error message that we have specified above ). You may find the strtok() routine useful for parsing the command line (i.e., for extracting the arguments within a command separated by whitespace or a tab or ...).
Executing Commands: Look into fork , execvp , and wait/waitpid . See the UNIX man pages for these functions, and also read the Advance Programming in the UNIX Environment, Chapter 8 (specifically, 8.1, 8.2, 8.3, 8.6, 8.10). Before starting this project, you should definitely play around with these functions.
You will note that there are a variety of commands in the
int main(int argc, char *argv);
Note that this argument is an array of strings, or an array of pointers to characters. For example, if you invoke a program with:
foo 205 535
then argv = "foo", argv = "205" and argv = "535".
Important: the list of arguments must be terminated with a NULL pointer; that is, argv = NULL. We strongly recommend that you carefully check that you are constructing this array correctly!
If you get your basic shell running, supporting multiple commands should be straightforward. The only difference here is that you need to wait for the previous process to finish before creating a new one. To do that, you simply use waitpid() again.
For managing the current working directory, you should use getenv ,
chdir , and getcwd . The
Redirection is relatively easy to implement: just use close() on stdout and then open() on a file. More on this during discussion.
With file descriptor, you can perform read and write to a file. Maybe in your life so far, you have only used fopen() , fread() , and fwrite() for reading and writing to a file. Unfortunately, these functions work on FILE* , which is more of a C library support; the file descriptors are hidden.
To work on a file descriptor, you should use open() , read() , and write() system calls. These functions perform their work by using file descriptors. To understand more about file I/O and file descriptors you should read the Advanced UNIX Programming book Section 3 (specifically, 3.2 to 3.5, 3.7, 3.8, and 3.12). Before reading forward, at this point, you should get yourself familiar with file descriptor.
The idea of redirection is to make the stdout descriptor point to
your output file descriptor. First of all, let's understand the
STDOUT_FILENO file descriptor. When a command
Remember to get the basic functionality of your shell working before worrying about all of the error conditions and end cases. For example, first get a single command running (probably first a command with no arguments, such as "ls"). Then try adding more arguments.
Next, try working on multiple commands. Make sure that you are correctly handling all of the cases where there is miscellaneous white space around commands or missing commands. Finally, you add built-in commands and redirection suppors.
We strongly recommend that you check the return codes of all system calls from the very beginning of your work. This will often catch errors in how you are invoking these new system calls. And, it's just good programming sense.
Beat up your own code! You are the best (and in this case, the only) tester of this code. Throw lots of junk at it and make sure the shell behaves well. Good code comes through testing -- you must run all sorts of different tests to make sure things work as desired. Don't be gentle -- other users certainly won't be. Break it now so we don't have to break it later.
Keep versions of your code. More advanced programmers will use a source control system such as CVS . Minimally, when you get a piece of functionality working, make a copy of your .c file (perhaps a subdirectory with a version number, such as v1, v2, etc.). By keeping older, working versions around, you can comfortably work on adding new functionality, safe in the knowledge you can always go back to an older, working version if need be.
To ensure that we compile your C correctly for the demo, you will
need to create a simple makefile; this way our scripts can just
The name of your final executable should be
emperor1% ./mysh emperor1% ./mysh inputTestFile
Please also submit a README file (i.e. filename: "README"). Your README file should describe what functionalities are not working. If you think your code is perfect, simply write "All good".
Hand in your source code (including the README and Makefile). We have created a directory ~cs537-1/handin/NAME/p2, where NAME is your login name. For example, if you are in section 2 and your login is jimmyv, your handin directory is ~cs537-1/handin/jimmyv/p2.
Copy all of your .c source files into the appropriate subdirectory. Do not submit any .o files. Make sure that your code runs correctly on the linux machines in the 13XX labs.
We will run your program on a suite of test cases, some of which will exercise your programs ability to correctly execute commands and some of which will test your programs ability to catch error conditions. Be sure that you thoroughly exercise your program's capabilities on a wide range of test suites, so that you will not be unpleasantly surprised when we run our tests.
The Shell Challenge
With sorting, the fastest code was considered best. With the shell, it's the smallest code. Hence the smallest READABLE code that passes all tests will win the T-shirt prize and fame and glory. The metric will roughly be non-whitespace lines of text in your code (after stripping out comments). However, if we deem the code to be anything but CLEAR, your code will not count. Thus, just write clean properly-abstracted code, not some obfuscated mush, and you can win the coveted prize.