CS 537: Fall 2007
Programming Assignment 3: A "Better" Malloc

Test Script Results

Have fun!

• S0 O0 P0
• S1 O0 P0
• S2 O0 P0

• S0 O1 P0
• S1 O1 P0
• S2 O1 P0

• S0 O0 P1
• S1 O0 P1
• S2 O0 P1

• S0 O1 P1
• S1 O1 P1
• S2 O1 P1

Contents

• Objectives
• Overview
• Program Specification
• Unix Hints
• Grading

There are three objectives to this assignment:

1. To understand the nuances of building a memory allocator.
2. To understand the art of performance tuning for different workloads.
3. To create a shared library.

In this project, you will be implementing a memory allocator for the heap of a user-level process. Your functions will be similar to those provided by malloc() and free(), but a little more interesting.

Memory allocators have two distinct tasks. First, the memory allocator asks the operating system to expand the heap portion of the process's address space by calling either sbrk or mmap. Second, the memory allocator doles out this memory to the calling process. This involves managing a free list of memory and finding a contiguous chunk of memory that is large enough for the user's request; when the user later frees memory, it is added back to this list.

This memory allocator is usually provided as part of a standard library and is not part of the OS. To be clear, the memory allocator operates entirely within the address space of a single process and knows nothing about which physical pages have been allocated to this process or the mapping from logical addresses to physical addresses.

When implementing this basic functionality in your project, we have a few guidelines. First, when requesting memory from the OS, you must use mmap (which we think is easier to use than sbrk). Second, although a real memory allocator requests more memory from the OS whenever it can't satisfy a request from the user, your memory allocator must call mmap only one time (when it is first initialized). Third, you are free to use any data structures you want to manage the free list as well as any policy for choosing a chunk of memory. Note that you will be graded partially on performance in this project, so think about these aspects very carefully.

Your memory allocator will be more interesting than the traditional malloc and free int that your allocator will be flexible in how the user can specify what memory should be freed. For comparison, the traditional malloc() and free() are defined as follows.

• void *malloc(size_t size): malloc() allocates size bytes and returns a pointer to the allocated memory. The memory is not cleared.
• void free(void *ptr): free() frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc(), calloc() or realloc(). Otherwise, or if free(ptr) has already been called before, undefined behaviour occurs. If ptr is NULL, no operation is performed.

int *ptr;

// The returned memory object is between ptr and ptr+49
if ((ptr = (int *)Mem_Alloc(50 * sizeof(int))) == NULL) exit(1);

// Could replace 30 with any value from 0 to 49..
Mem_Free(ptr+30);  

We now define each of these routines more precisely.

• int Mem_Init(int sizeOfRegion): Mem_Init is called one time by a process using your routines. sizeOfRegion is the number of bytes that you should request from the OS using mmap. Note that you may need to round up this amount so that you request memory in units of the page size (see the man pages for getpagesize()). Note that you need to use this allocated memory for your own data structures as well; that is, your infrastructure for tracking the mapping from addresses to memory objects has to be placed in this region as well. If you call malloc(), or any other related function, in any of your routines, we will deduct a significant number of points (e.g., 15 points). Similarly, you should not allocate global arrays! However, you may allocate a few global variables (e.g., a pointer to the head of your free list.)
• void *Mem_Alloc(int size): Mem_Alloc() is similar to the library function malloc(). Mem_Alloc takes as input the size in bytes of the object to be allocated and returns a pointer to the start of that object. The function returns NULL if there is not enough free space within sizeOfRegion allocated by Mem_Init to satisfy this request.
• int Mem_Free(void *ptr): Mem_Free frees the memory object that ptr falls within, according to the rules described above. Just like with the standard free(), if ptr is NULL, then no operation is performed. The function returns 0 on success and -1 if ptr to does not fall within a currently allocated object (note that this includes the case where the object was already freed with Mem_Free).
• int Mem_IsValid(void *ptr): This function returns 1 if ptr falls within a currently allocated object and 0 if it does not. You may find this function useful when debugging your memory allocator.
• int Mem_GetSize(void *ptr): If ptr falls within the range of a currently allocated object, then this function returns the size in bytes of that object; otherwise, the function returns -1. You may find this function useful when debugging your memory allocator.

To create a shared library named libmem.so, use the following commands (assuming your library code is in a single file "mem.c"):

gcc -c -fpic mem.c
gcc -shared -o libmem.so mem.o

gcc mymain.c -lmem -L. -o myprogram

Before you run "myprogram", you will need to set the environment variable, LD_LIBRARY_PATH, so that the system can find your library at run-time. Assuming you always run myprogram from this same directory, you can use the command:

setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:.

In project 2, you saw how a web server can use mmap to map a file into its address space; then, the server read the file by simply accessing the memory where that file had been mapped. In this project, you will use mmap to map zero'd pages (i.e., allocate new pages) into the address space of the calling process. Note there are a number of different ways that you can call mmap to achieve this same goal; we give one working example here:

    // open the /dev/zero device 
    int fd = open("/dev/zero", O_RDWR); 

    // size (in bytes) needs to be evenly divisible by the page size
    void *ptr = mmap(NULL, sizeOfRegion, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0); 
    if (ptr == MAP_FAILED) { 
        perror("mmap"); 
        return -1;
    } 

    // close the device (don't worry, mapping should be unaffected) 
    close(fd); 
    return 0; 

• Functionality: Approximately 60% of your project grade will be devoted to how well your implementation matches the specification above; that is, this part of your grade depends upon correctly implementing the specified functions.
• Quality: Approximately 30% of your project grade will be devoted to the quality of your implementation on a range of workloads. By quality, we mean both how quickly your routines execute and how well you minimize fragmentation and space overhead (i.e., are able to satisfy requests from the user). Therefore, you will want to think about how to optimize for various workloads from the very beginning of your design process. When grading quality, your implementation will be compared directly against others in the class.
• Documentation and Style: Approximately 10% of your project grade will be for the documentation in your README file and the style and comments of your code.

The following is a short description of the workloads. These descriptions should be sufficient for you to optimize your design. Each workload can be characterized by three properties: the size of the requested objects, the order of allocates and frees, and whether pointers are always to the beginning of the object or not.

The workloads vary the size of their requests in three different ways:

• Size 1: The user requests all small objects (the sizes are random, uniformly distributed between 8 bytes and 256 bytes).
• Size 2: The user (roughly) alternates between requesting a small object (approximately 64 bytes) and a large object (approximately 64 KB).
• Size 3: The user requests (mostly) objects whose size is a power of two.

The workloads vary the order of allocates and frees in two different ways:

• Order A: The user allocates N objects and then frees all N of them; the user then allocates N more objects and frees all N of them.
• Order B: The user repeatedly allocates N objects and then frees N/2 of the objects until the end at which point it frees all remaining objects.

The workloads vary whether or not they use only the pointer returned by Mem_Alloc:

• Pointer X: The user only calls Mem_Free() with the exact pointer returned by Mem_Alloc, matching the usage of the traditional malloc/free interface. The user does not call Mem_IsValid() or Mem_GetSize().
• Pointer Y: The user calls Mem_Free() with any pointer returned by Mem_Alloc. The user calls Mem_IsValid() or Mem_GetSize() an average of one time per valid object; there are a roughly half as many calls of Mem_IsValid or Mem_GetSize for invalid objects.

Handing in your Code

You should copy all of your server source files (*.c and *.h) and a Makefile to your p3 handin directory. Do not submit any .o files.

In your README file you should have the following four sections:

• The name and login information for both project partners. A brief description of how you divided the work between you.
• Design overview: A few simple paragraphs describing the overall structure of your code and any important structures. For example, include a brief description of the data structures you use to map between addresses and memory objects and a brief description of the policy that use to perform allocations (e.g., first-fit, best-fit, rotating first-fit, buddy, etc),
• Complete specification: Describe how you handled any ambiguities in the specification.
• Known bugs or problems: A list of any features that you did not implement or that you know are not working correctly

After the deadline for this project, you will be prevented from making any changes in these directory. Remember: No late projects will be accepted!