Homework 4: Malloc and Free

Important Dates

Questions about the project? Send them to 354-help@cs.wisc.edu .

Deadline: Wednesday 3/22 Sunday 3/26

Objectives

There are three objectives to this assignment:

  • To understand the nuances of building a memory allocator.
  • To do so in a performance-efficient manner.
  • To create a shared library.

Notes

You are allowed to have a partner.

Added 3/13: You should write your own data structures to manage free space, and you should never call malloc().

Overview

In this project, you will be implementing a memory allocator for the heap of a user-level process. Your functions will be to build your own malloc() and free() functions.

Memory allocators have two distinct tasks. First, the memory allocator asks the operating system to expand the heap portion of the process's address space by calling either sbrk or mmap. Second, the memory allocator doles out this memory to the calling process. This involves managing a free list of memory and finding a contiguous chunk of memory that is large enough for the user's request; when the user later frees memory, it is added back to this list.

This memory allocator is usually provided as part of a standard library and is not part of the OS. To be clear, the memory allocator operates entirely within the address space of a single process.

When implementing this basic functionality in your project, we have a few guidelines. First, when requesting memory from the OS, you must use the mmap() system call; more about this below. Second, although a real memory allocator requests more memory from the OS whenever it can't satisfy a request from the user, your memory allocator must call mmap only one time (when it is first initialized). Third, you are free to use any data structures you want to manage the free list as well as any policy for choosing a chunk of memory. Note that you will be graded partially on performance in this project, so think about these aspects very carefully.

Classic malloc() and free() are defined as follows:

  • void *malloc(size_t size): malloc() allocates size bytes and returns a pointer to the allocated memory. The memory is not cleared.
  • void free(void *ptr): free() frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc() (or calloc() or realloc()). Otherwise, or if free(ptr) has already been called before, undefined behaviour occurs. If ptr is NULL, no operation is performed.

For simplicity, your implementations of Mem_Alloc(int size) and Mem_Free(void *ptr) should basically follow what malloc() and free() do; see below for details.

You will also provide a supporting function, Mem_Dump(), described below; this routine simply prints which regions are currently free and should be used by you for debugging purposes.

Required Reading

This project needs you to understand a few fundamentals before proceeding. Thus, we suggest the following from the free online operating systems book, Operating Systems: Three Easy Pieces:

It would be good It is essential to carefully read these two chapters before beginning on this project.

Program Specifications

For this project, you will be implementing several different routines as part of a shared library. Note that you will not be writing a main() routine for the code that you handin (but you should implement one for your own testing). We have provided the prototypes for these functions in the file mem.h (also available at ~cs354-3/public/hw4/mem.h once you log into a Linux machine); you should include this header file in your code to ensure that you are adhering to the specification exactly. You should not change mem.h in any way! We now define each of these routines more precisely.

  • int Mem_Init(int region_size, int policy): Mem_Init is called one time by a process using your routines. region_size is the number of bytes that you should request from the OS using mmap, and policy is the policy that should be used to manage the region. The options are P_BESTFIT, P_WORSTFIT, and P_FIRSTFIT, which should use a best fit, worst fit, and first fit policy, respectively, as discussed in the book chapters. You should round up this amount so that you request memory in units of the page size (see the man pages for getpagesize()). Note that you need to use this allocated memory for your own data structures as well; that is, your infrastructure for tracking the mapping from addresses to memory objects has to be placed in this region as well. If you call malloc(), or any other related function, in any of your routines, we will deduct a significant number of points. Similarly, you should not allocate global arrays! However, you may allocate a few global variables (e.g., a pointer to the head of your free list.) Return 0 on a success (when call to mmap is successful). Otherwise, return -1. Some cases where Mem_Init should return a failure: Mem_Init is called again after a successful call, region_size is 0, region_size is negative, the policy is not one of the defined three policies, etc.
  • void *Mem_Alloc(int size): Mem_Alloc() is similar to the library function malloc(). Mem_Alloc takes as input the size in bytes of the object to be allocated and returns a pointer to the start of that object. The function returns NULL if there is not enough contiguous free space within region_size allocated by Mem_Init to satisfy this request. For better performance, Mem_Alloc() should return 4-byte aligned chunks of memory. For example if a user allocates 1 byte of memory, your Mem_Alloc() implementation should return 4 bytes of memory so that the next free block will be 4-byte alligned too. To debug whether you return 4-byte aligned pointers, you could print the pointer this way printf("%08x", ptr) . The last digit should be a multiple of 4 (i.e. 0, 4, 8, or C). For example, this is okay: b7b2c04c, and this is not okay: b7b2c043.
  • int Mem_Free(void *ptr): Mem_Free() frees the memory object that ptr points to. Just like with the standard free(), if ptr is NULL, then no operation is performed. The function returns 0 on success and -1 if the ptr was not allocated by Mem_Alloc(). If ptr is NULL, also return -1. Critical: Mem_Free() should always try to coalesce the free space, to make chunks that are as large as possible free. Not doing so will lead to a number of tests failing and hence a bad grade.
  • void Mem_Dump(): This is just a debugging routine for your own use. Have it print the regions of free memory to the screen.

You must provide these routines in a shared library named "libmem.so". Placing the routines in a shared library instead of a simple object file makes it easier for other programmers to link with your code. There are further advantages to shared (dynamic) libraries over static libraries. When you link with a static library, the code for the entire library is merged with your object code to create your executable; if you link to many static libraries, your executable will be enormous. However, when you link to a shared library, the library's code is not merged with your program's object code; instead, a small amount of stub code is inserted into your object code and the stub code finds and invokes the library code when you execute the program. Therefore, shared libraries have two advantages: they lead to smaller executables and they enable users to use the most recent version of the library at run-time. To create a shared library named libmem.so, use the following commands (assuming your library code is in a single file "mem.c"):

gcc -c -fpic mem.c -m32
gcc -shared -o libmem.so mem.o -m32

To link with this library, you simply specify the base name of the library with "-lmem" and the path so that the linker can find the library "-L.".

gcc -m32 -o myprogram mymain.c -L. -lmem

Of course, these commands should be placed in a Makefile. Before you run "myprogram", you will need to set the environment variable, LD_LIBRARY_PATH, so that the system can find your library at run-time. Assuming you always run myprogram from this same directory, you can type:

setenv LD_LIBRARY_PATH .

Of course, you may be using a shell that does not support the above syntax (which is csh, or the C shell), such as bash. In bash, you would have to type:

export LD_LIBRARY_PATH=.

Operating System Hints

In this project, you will use mmap to map zero'd pages (i.e., allocate new pages) into the address space of the calling process. Note there are a number of different ways that you can call mmap to achieve this same goal; we give one working example here:

// open the /dev/zero device
int fd = open("/dev/zero", O_RDWR);

// region_size (in bytes) needs to be evenly divisible by the page size
void *ptr = mmap(NULL, region_size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
if (ptr == MAP_FAILED) {
perror("mmap");
return -1;
}

// close the device (don't worry, mapping is unaffected)
close(fd);
return 0;

Grading

Your implementation will be graded along two main axes:

  • Functionality: Approximately 75% of your project grade will be devoted to how well your implementation matches the specification above; that is, this part of your grade depends upon correctly implementing the specified functions and having policies that behave as expected.
  • Performance: Approximately 25% of your project grade will be devoted to performance of your implementation on a range of workloads.

Handing in your Code

Hand in your source code and a README file. The usual place: ~cs354-3/handin/NAME/hw4/, where NAME is your login name.

You should copy all of your source files (*.c and *.h) and a Makefile to your hw4/ handin directory. Do not submit any .o files.

If you have a partner, put the code in ONE of your directories, and a file called PARTNER.txt in BOTH handin directories (one per partner). In that file should be the login of both partners.

.

.