Hints for Project 3

Readings

Please read the C Programming book, Chapter 8.7, A Storage Allocator. In this chapter, you can see the traditional C implementation of malloc() and free().

If you never use pointers in C, please read the C programming book. Specifically, go to the index page and find "pointer", "struct", and "struct, pointer". You will manipulate struct-pointers a lot! Hence, you need to be an expert in manipulating pointers. In addition, you might want to read the C notes again, part 2 in particular.

The idea of this project is you are given a chunk of memory to manage (this chunk of memory is returned by mmap()). To manage this chunk of memory, you must have internal data structures (e.g. list of free blocks). The rule is to store your internal structures within the given memory. In other words, in your mem.c file, you should only have one global pointer which points to the base address of the given memory. If your design requires you to create global arrays, or you need to call malloc(), it implies your design is wrong!

Memory Management and Pointer Manipulation

Please download this ppt slides [or in pdf] for some illustrations of free space management.

Discussion Recap

In this week's discussion we have covered mainly:

Handling double free

The second Mem_Free() call in this sequence should return an error:
    void *ptr = Mem_Alloc(100);
    ret = Mem_Free(ptr); // ok
    ret = Mem_Free(ptr); // should fail
In the discussion last time, we have talked about how you should add a constant "magic number" in each header of allocated region (let's name this the object header). The magic number is used in this way: In the code sequence above, the first Mem_Free() gets a valid ptr. Hence it will pass the magic number test. The second Mem_Free() unfortunately will pass the magic number test (but, we need to return an error). So, how should you deal with double free? Clear the magic number in the object header when you free the object area so that the second free will not pass the magic number test.

Writing a meta-compiler

Managing memory is not a simple task, mainly because you need to keep track lots of pointers, break a free block into a smaller free block and an object block, merge adjacent free blocks, etc.

If not tested thoroughly, it is highly possible that your code is buggy. For example: there is a hole (lost space) between a free block and an object block, object block and free block are overlapping each other, a pointer points to an address outside the valid memory region, etc.

To test your code, you should create your own tests (e.g. alloc and free hundred and thousand times). We will also provide your some test files. However, if your program fails, these test files are not sufficient to pinpoint where the problem is. Hence, we suggest you to create a meta-compiler in your library to help you debug complex program such as this project.

A meta-compiler is a specific compiler that verifies your program behavior; gcc is a compiler, but it does not know the high-level intent of your program. On the hand, a meta-compiler "knows" the high-level intent of your program.

In this project, your program is supposedly to manage the memory correctly. What does "correct" mean? How do you check correctness? The answer is you need to rewrite the specification from English to C code. Below is a list of some specifications that your program should adhere to. You can add more if you think of any.

  1. Size of memory region = total size of free blocks + total size of object blocks

  2. An object block should not overlap with a free block, e.g.:
      if (free_header < object_header)
         if (free_header + free_header->size > object_header)
             ERROR! return -1, and print a descriptive error message here
      if (...)
        // more possibilities here (add them yourself)
    
  3. (For a better space utilization) there should be no two adjacent free blocks, e.g.:
      if (free_header + free_header->size == free_header->next)
          ERROR!
    
  4. There should be no gaps (no lost memory space), e.g.:
      // Assume this free_header is not the last block in the
      // memory region, then I expect the neighboring block to be
      // an object block. If not, there is a gap.
      object_header = free_header + free_header->size;
      if (object_header->magic_number != YOUR_MAGIC_NUMBER)
          // ERROR!
    
  5. No out-of-bound pointers and out-of-bound blocks, e.g.:
      Iterate all free headers
          if (free_header->next >= base + sizeOfRegion)
              ERROR!
          if (free_header + free_header->size >= base + sizeOfRegion)
              ERROR!
      Iterate all object headers
          if (object_header + sizeof(object_header) + object_header->size >= 
              base + sizeOfRegion)
              ERROR!
    
  6. More, more, and more ...
Remember again that a meta-compiler is often specific to your implementation; you might not want to copy-paste the pseudo-code above if your implementation looks different.

The basic idea here, if your code is complex and buggy, there could be hundreds possibilities of error. By writing your own meta-compiler, you could pinpoint which specific error you should be dealing with. Without a meta-compiler, you will be guessing what the problem is. The more checks you put in your meta-compiler, the more error information you can get. Think more of what checks you should add. These checks often depend on your specific implementation.

How should you implement the meta-compiler? Simply create a Mem_Check() function in your library code. Remember that in your library, you should be able to access all information that you need. Mem_Check() simply peruses that information to find where the problem is. We will not grade your Mem_Check() implementation; it is simply for your own benefit. You can call Mem_Check() anywhere in your test file. Here is an example:

  int main() {

      Mem_Init(someSize);
      
      // some tests here (say testA)

      Mem_Check();  // if no error is printed,
                    // then you pass testA

      // some tests here (say testB)

      Mem_Check();  // if no error is printed,
                    // then you pass testB

      // some tests here (say testC)
  
      Mem_Check();  // if some errors are printed here,
                    // then you know you fail testC,
                    // plus you will know what the problem is
      
  }

Coalescing adjacent free blocks

This is actually just a reminder rather than a hint. When you free an object area, you need to find out if the adjacent blocks are free blocks. If so, you need to merge this object area with the neighboring free block (or blocks) to make a larger free block.