Project 1a: Sorting with Variable-Sized Records
You will write a simple sorting program. This program should be invoked as follows:
shell% ./varsort -i inputfile -o outputfile
The above line means the users typed in the name of the sorting program
Input files are generated by a program we give you called genvar.c .
The file begins with a header. This header is a single integer, R containing the number of records in this file.
After the header, the file contains a
kkkk<8>DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDAgain, the key is four bytes and this data is 8 integers, or 8 * sizeof(integer) = 8 * 4 = 32 bytes.
Note that different records can have keys with different amounts of associated data.
Your goal: to build a sorting program called
shell% gcc -o genvar genvar.c -Wall -Werror
Note: you will also need the header file sort.h to compile this program.
Then you run it:
shell% ./genvar -s 0 -n 100 -m 32 -o /tmp/outfile
There are four flags to
The format of the file generated by the
Another useful tool is dumpvar.c . This program can be used to
dump the contents of a file generated by
For example, if run as follows:
thenshell% ./dumpvar -i /tmp/file
When you create files to sort, we strongly recommend that you place
those files in
You will probably want to the look at the source code for both of these utilities to see how to read and write to files; in particular, they could be useful for seeing how to understand the variable-sized record format.
A common header file sort.h has the detailed description. There are three different versions of the record that you might find useful in different circumstances:
Note that you may NOT simply allocate an array of
In your sorting program, you should just use
To sort the data, use any sort that you'd like to use. An easy way
to go is to use the library routine
Remember to write out the header (i.e., the number of records) for
your sorted output.
For efficiency, you might not want to call
To exit, call
If you don't know how to use these functions, use the man pages!
In addition to testing for correct program behavior, we will also be giving points for good programming style and for careful memory management.
As programmers, we often won't be writing all our own code from scratch, and instead will be making contributions inside of an already existing project which other people are also contributing to. In these situations, other programmers will often need to be able to read and understand the code you've written. Because of this, many companies will require its programmers to adhere to a style guideline, so that the task of reading and understanding another person's code is made easier.
For grading on style, we will mostly stick to the style guidelines
which Google uses for C++ (more info can be found
with a few differences specific to C. These differences are in the
config file in
We will be running it with the following options:
cpplint.py --extensions=c,h varsort.c
Another important skill to develop in C programming is good memory management. This means freeing any heap space you've allocated when you're done using it! Since memory is a limited resource, we will want to free memory when we're done with it so that it can be reused. To check that your code doesn't contain memory leaks, we will be using a tool known as valgrind. It's a simple tool which moniters every call to malloc (and other memory allocation functions) and free to make sure that all memory allocated to our program is subsequently released when we are done with it. Our tests will be running valgrind on your code in the following way:
valgrind --show-reachable=yes ./varsort -i infile -o outfile
Assumptions and Errors
32-bit integer range. You may assume that the keys are unsigned 32-bit integers.
Record size. You may assume that there are no more than USHRT_MAX data items in each record. However, most records may have many fewer data items than the max, so don't allocate this much memory for every record!
File length: May be pretty long! However, there is no need to implement a fancy two-pass sort or anything like that.
Invalid files: If the user specifies an input or output file that you
cannot open (for whatever reason), the sort should EXACTLY print:
Too few or many arguments passed to program: If the user runs varsort
without enough arguments, or in some other way passes incorrect flags and such to
Important: On any error code, you should print the error to the screen
Your grade will primarily depend only on the correctness of your program. However, programs that run signficantly slower than others (i.e., an order of magnitude slower) will be penalized.
History and an Optional Performance Contest
This sorting assignment derives from a yearly competition to make the fastest disk-to-disk sort in the world. See the sort home page for details. If you look closely, you will see that your professor was once -- yes, wait for it -- the fastest sorter in the world.
To continue in this tradition, we will also be holding a sorting competition. The variable-sized record sort that you need to implement for this assignment is a little trickier than the straight-forward sort, since you need to handle more than just 100-byte records. For the contest, we will average together your program's runtime on three input sets:
Whoever wins the performance contest will win a soft-cover copy of the OSTEP text book.
Read more about sorting, including perhaps the NOW-Sort paper , for some hints on how to make a sort run really fast. Or just use your common sense! Hint: you'll have to think a bit about hardware caches.
We will only measure the performance of assignments that sort correctly for all cases!
Start small, and get things working incrementally. For example, first get a program that simply reads in the input file, one line at a time, and prints out what it reads in. Then, slowly add features and test them as you go. Don't worry about performance until you have all of the functionality working correctly.
Testing is critical. Testing your code to make sure it works is crucial. Write tests to see if your code handles all the cases you think it should. Be as comprehensive as you can be. Of course, when grading your projects, we will be. Thus, it is better if you find your bugs first, before we do.
Keep old versions around. Keep copies of older versions of your program
around, as you may introduce bugs and not be able to easily undo them. A
simple way to do this is to keep copies around, by explicitly making copies of
the file at various points during development. For example, let's say you get
a simple version of