Today : Mon, 06 Jul 20 .


CS537 Student Wiki


Writing

Homeworks

Sections

Projects

Project Discussion

edit SideBar

Project4

Page: PmWiki.Project4 - Last Modified : Fri, 11 Dec 09

The Art of Computer Science Measurement

Due Date: Tuesday, December 15th, at 2 pm This project should be done in groups of three.

Notes

  • Friday, December 11th: Some people have reported problems with the sample rdtsc code below. Here is another version that casts values differently:
#define rdtscll(val) do { \
     unsigned int __a,__d; \
     __asm__ __volatile__("rdtsc" : "=a" (__a), "=d" (__d)); \
     (val) = ((unsigned long long)__a) | (((unsigned long long)__d)<<32); \
} while(0)
  • Friday, December 11th: The TA has requested a particular handin format to speed grading. As always, copy the files to ~cs537-2/handing/<one-of-your-logins>/p4. You should create code that produces a single executable named fs-test that runs a set of experiments based on one integer argument:
    1. = block size
    2. = prefetch
    3. = number of direct pointers in an inode.
    The executable should write to stdout a summary of the results of the experiments at the end. We will provide a file named "/tmp/cs537-test" that is 100 MB in size. If you need other files, you should create them in your program. There should be a makefile to compile the submitted file(s).
  • Wednesday, December 9th: I changed the code samples to also include <sys/types.h> to help make it compile.
  • Tuesday, December 8th: some people have had problems with the rdtsc function. here is an alternate implementation:
#define rdtscll(val) do { \
     unsigned int __a,__d; \
     __asm__ __volatile__("rdtsc" : "=a" (__a), "=d" (__d)); \
     (val) = ((unsigned long)__a) | (((unsigned long)__d)<<32); \
} while(0)


unsigned long long mytime;
rdtscll(mytime);
  • Monday, December 7th: to use the clock_gettime function, pass it CLOCK_REALTIME as the clock identifier. You will need to specify -lrt as an argument to gcc to link against the librt realtime library.
  • Monday, December 7t: Below is some code that can be used to use posix_fadvise to remove pages from cache:
  #include <sys/types.h>
  #include <fcntl.h>
  int evictfile(char * filename) 
  {
     int fd;
     int advice;
     int retval;

     fd = open(filename, 0);
     if (0 > fd) {
        perror("open");
        return(errno);;
     }

     advice = POSIX_FADV_DONTNEED;
     fsync(fd); 

     retval = posix_fadvise(fd, 0, 0, advice);
     if (retval < 0) {
       perror("Posiz_fadvise");
     }
     close(fd);
     sleep(1);
     return(retval);
   } 
  • Monday, December 7th: below is some code to use O_DIRECT to directly access disk. Note that the includes are incompatible with the code above. This will only for for local files (e.g., files on AFS cannot be opened with O_DIRECT). Also, the buffers used for reading and writing files must be aligned to 512 byte boundaries. You have to read in multiples of 512 bytes.
  #define ROUND_UP_POINTER(Ptr,Pow2)					\
        ( (void *) ( (((unsigned long long)(Ptr))+(Pow2)-1) & (~(((long long)(Pow2))-1)) ) )

  #include <sys/types.h>
  #include <linux/fcntl.h>
  int main(int argc, char * argv[])  
  {
    int fd;
    int size = 512; // what size do you want
    char * filename;
    char * buffer, *org_buffer;

    filename = argv[1];

    fd = open(filename, O_RDONLY | O_DIRECT);
    if (fd < 0) {
      perror("open");
      return(-1);
    }

    org_buffer = malloc(size + 511);
    buffer = ROUND_UP_POINTER(org_buffer, 512);

    // do something
    read(fd, buffer, size);
    // do something
    free(org_buffer);

Overview

In this assignment, you will get your feet just a little bit wet with a computer system. One of the most important parts of computer systems is evaluation. Hence, understanding what it takes to perform such an evaluation is a skill we wish to develop.

You're going to run some simple experiments, which you design to bring out various properties of the file system under test. Third, you are going to create some graphs to demonstrate those properties -- call them "empirical proofs". Finally, you will write up what you did. While you can compile and test you code anywhere, the results for this project must be generated from department instructional machines. If you are running the ext3 file system on another machine, you can at least test the code on that machine.

More Detail

In this assignment, we're going to explore the inner-workings of the file system. In a Unix-based file system, assume we have the following system calls to work with: open(), close(), read(), write(), lseek(), and fsync(). If you don't know what those system calls do, then use the man pages to find out.

Our main approach is going to be to write little code snippets that exercise the file system in different ways; then, by measuring how long various operations take, we are going to try to make some deductions about what the file system is doing.

There are two critical pieces of measurement infrastructure you will need to develop: timers and the ability to control the file system cache.

Step 1: Timers

The accuracy and granularity of the timer you use will often have a large affect on your measurements. Therefore, you should use the best timer available. Fortunately, on x86 platforms, a highly accurate cycle counter is available. The instruction to use it is known as rdtsc, and it returns a 64-bit cycle count. The cycle time is the number of processor cycles (similar to instructions) since the machine was booted. By knowing the cycle time, one can easily convert the result of rdtsc into a useful time.

You can convert from cycles to second using the processor frequency. This is available in /proc/cpu_info in the line labeled cpu MHz: divide cycles by (MHz x 1000) to get seconds.

Here is the code you will need to use to access the timestamp counter:

   extern __inline__ unsigned long long int rdtsc()
   {
     unsigned long long int x;
     __asm__ volatile (".byte 0x0f, 0x31" : "=A" (x));
     return x;
   }

Hence, the first thing you should do is: figure out how to use rdtsc Once you know how to call it and get a cycle count, convert the result to seconds and measure how long something takes (e.g., a program that calls sleep(10) and exits should run for about 10 seconds). Note that confirmation of timer accuracy is hugely important! If you don't trust your timer, how can you trust the results of your measurements?

Step 2: Clearing the cache

The file system will cache blocks in memory to avoid expensive disk accesses. To get accurate measurements of performance, though, you will need to control what is in the cache.

One convenient way to do this is direct I/O. You can access this by passing the O_DIRECT flag to the open() system call when opening a file. Here is a snippet from the man page for open().

 O_DIRECT
        Try to minimize cache effects of the I/O to and from this  file.
        In  general  this  will degrade performance, but it is useful in
        special situations, such  as  when  applications  do  their  own
        caching.   File I/O is done directly to/from user space buffers.
        The I/O is synchronous, i.e., at the completion of a read(2)  or
        write(2),  data  is  guaranteed to have been transferred.  Under
        Linux 2.4 transfer sizes, and the alignment of user  buffer  and
        file  offset  must all be multiples of the logical block size of
        the file system. Under Linux 2.6 alignment must  fit  the  block
        size of the device.

Another option is to use a tool to tell the file system that you do not need the pages for a file any more and it can evict them from the cache. The fadvise tool, passed a file name and the dontneed argument will do this.

Step 3: Measuring the File System

After getting our timer in order, we will move on and measure some aspects of the file system proper. All measurements should be done on a file in the tmp directory of an instructional Linux machine. You can learn about the size of various file system data structures by measuring how long certain operations take; if data is in memory, the operations will be quick. If the data is on disk, the operations will be much slower. Remember that disk operations typically take about 1 millisecond while memory operations take about 1 microsecond.

Through experiments that you design, implement, run, and measure, you are to answer the following questions:

  1. How big is the block size used by the file system to read data? A block is the smallest amount of data the file system will read from the disk. Hint: use reads of varying sizes and plot the time it takes to do such reads. Also, be wary of prefetching effects that often kick in during sequential reads.
  2. During a sequential read of a large file, how much data is prefetched by the file system? Prefetching occurs when the file system fetches data off disk before you ask for it. Hint: time each read and plot the time per read. You may want to sleep for a small period between reads to allow the prefetching to complete.
  3. How many direct pointers are in the inode? Hint: think about using write() and fsync(), which forces data to disk, to answer this question. Also think about what happens when you extend a file and suddenly an indirect pointer must be allocated -- how many more writes occur at that point?

Hence, in your write-up, you should have one or more graphs which you use to directly answer the questions above.

A major issue with any data collection is: how convincing are your numbers? How do you make them more convincing? How do you deal with experimental noise? etc. Use your common sense and be critical of your numbers -- do they really convince you that you know the answer?

One thing you will undoubtedly do is to use repetition to increase your confidence, i.e., you will take multiple measurements of an event, and compute (for example) an average over many runs instead of the result from just a single experiment. Be careful when computing averages over numbers -- make sure to always first look at all the data. If you don't, you might use an average where an average doesn't make sense.

Step 4: Writing It Up

After you're done with experiments, you'll need to write up what you've done. For this assignment, the writeup will basically consist (1) a description of your experiments, and (2) graphs showing your results and a paragraph for each of the questions above describing what you found. Make sure that graphs have axes labeled (including units). Also, make sure to draw appropriate conclusions about each graph.

Here are hints on the writeup:

  • When proposing experiments, describe your hypothesis: how does the system work, how do you expose that behavior. Make a prediction on what should happen if your hypothesis is correct.
  • When graphing, change the scale to highlight the useful stuff. If nothing happens at the top, chop it off. Log scale may be useful, but it tends to minimize large percentage differences.
  • Perform multiple runs to make sure the data is good. Depending on the test, you might want best case, average case, median case, or worst case. Understand why. When looking for phenomena, best case may work well. If using randomness, you may want the median or average case.

Turnin

Please turn in your code, a readme file saying how to run your code, and your report in PDF or Microsoft Word format to @@~cs537-2/handin/<your-login>/p4.

Enjoy Yourself

Computer systems are complicated, and careful and accurate measurement is a tricky business. Make sure to have fun! How should you do that? Probably by starting early. As always, feel free to ask me questions if you are having trouble. Good luck!

This page may have a more recent version on pmwiki.org: PmWiki:Project4, and a talk page: PmWiki:Project4-Talk.


Powered by PmWiki
Skin by CarlosAB

looks borrowed from http://haran.freeshell.org/oswd/sinorca
More skins here


PmWiki can't process your request

Cannot acquire lockfile

We are sorry for any inconvenience.

More information

Return to http://pages.cs.wisc.edu/~swift/classes/cs537-fa09/wiki/pmwiki.php