Project 3: Scalable Web Server

CS 537 Project 3: Scalable Web Server

CS 537 Page

IMPORTANT DATES

Due: 11/23.
See Section 2 Specifics if you are in Professor Swift's class, and Section 1 Specifics if you are in Professor Lu's class.

CLARIFICATIONS

If your setenv command does not work, it is because your machine is using bash-shell. In that case, you can try
export LD_LIBRARY_PATH=.

or

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:.
The statistics related to `time' should use milli-second unit. To print out the `arrival time', please convert the second, milli-second, micro-second you got from gettimeofday all to milli-second and print it out.

NOTES

This is a group project. See section-specific pages for information on group sizes and how grades will be assigned to group members.
To start this project, please copy all the code in ~cs537-1/projects/p3 into your working directory, including the 3 sub-directories there.
```
    cd your-working-directory
    cp -r ~cs537-1/projects/p3/* .
    
```
This is a tutorial to pthread libraries.

BACKGROUND

In this assignment, you will be developing a real, working web server. To simplify this project, we are providing you with the code for a very basic web server. This basic web server operates with only a single thread; it will be your job to make the web server multi-threaded so that it is more efficient.

HTTP Background

Before describing what you will be implementing in this project, we will provide a very brief overview of how a web server works and the HTTP protocol. Our goal in providing you with a basic web server is that you should be shielded from all of the details of network connections and the HTTP protocol. The code that we give you already handles everything that we describe in this section. If you are really interested in the full details of the HTTP protocol, you can read the specification, but we do not recommend this for this initial project!

Web browsers and web servers interact using a text-based protocol called HTTP (Hypertext Transfer Protocol). A web browser opens an Internet connection to a web server and requests some content with HTTP. The web server responds with the requested content and closes the connection. The browser reads the content and displays it on the screen.

Each piece of content on the server is associated with a file. If a client requests a specific disk file, then this is referred to as static content. If a client requests that a executable file be run and its output returned, then this is dynamic content. Each file has a unique name known as a URL (Universal Resource Locator). For example, the URL www.cs.wisc.edu:80/index.html identifies an HTML file called “index.html” on Internet host “www.cs.wisc.edu” that is managed by a web server listening on port 80. The port number is optional and defaults to the well-known HTTP port of 80. URLs for executable files can include program arguments after the file name. A “?” character separates the file name from the arguments and each argument is separated by a “&” character. This string of arguments will be passed to a CGI program as part of its “QUERY_STRING” environment variable.

An HTTP request (from the web browser to the server) consists of a request line, followed by zero or more request headers, and finally an empty text line. A request line has the form: method uri version . The method is usually GET (but may be other things, such as POST, OPTIONS, or PUT). The URI is the file name and any optional arguments (for dynamic content). Finally, the version indicates the version of the HTTP protocol that the web client is using (e.g., HTTP/1.0 or HTTP/1.1).

An HTTP response (from the server to the browser) is similar; it consists of a response line, zero or more response headers, an empty text line, and finally the interesting part, the response body. A response line has the form version status message . The status is a three-digit positive integer that indicates the state of the request; some common states are 200 for OK , 403 for Forbidden , and 404 for Not found . Two important lines in the header are Content-Type , which tells the client the MIME type of the content in the response body (e.g., html or gif) and Content-Length , which indicates its size in bytes.

If you would like to see the HTTP protocol in action, you can connect to any web server using telnet. For example, run telnet www.cs.wisc.edu 80 and then type:

    GET / HTTP/1.1
    host: www.cs.wisc.edu

You will then see the HTML text for that web page!

Again, you don't need to know this information about HTTP unless you want to understand the details of the code we have given you. You will not need to modify any of the procedures in the web server that deal with the HTTP protocol or network connections.

Basic Web Server

The code for the web server is available from ~cs537-1/projects/p3. You should copy over all of the files there into your own working directory. You should compile the files by simply typing make . Compile and run this basic web server before making any changes to it! make clean removes .o files and lets you do a clean build.

When you run this basic web server, you need to specify the port number that it will listen on; you should specify port numbers that are greater than about 2000 to avoid active ports. For example, assume that you are running on mumble-21.cs and use port number 2003; copy your favorite html file (e.g. favorite.html) to the directory that you start the web server from. Now you can start the server by:

mumble-21% ./server 2003

You can then view this file from a web browser (either on the same machine or on a different machine). Just make sure that you specify the correct port number. For example, type this url in IE or Firefox or any browser: mumble-21.cs.wisc.edu:2003/favorite.html

We also provide you a command-line client, client, in the same directory as server. You can use it to retrieve files from the server too. You just need to specify the server name, the port number, and the file name in command line. If you want to request for dynamic content (cgi files), you also need to specify the argument, as follows:

king01% ./client mumble-21.cs.wisc.edu 2003 /favorite.html
...
king01% ./client mumble-21.cs.wisc.edu 2003 "/output.cgi?10"
...
king01% ./client mumble-21.cs.wisc.edu 2003 /output.cgi\?10
...

The web server that we are providing you is only about 200 lines of C code, plus some helper functions. To keep the code short and very understandable, we are providing you with the absolute minimum for a web server. For example, the web server does not handle any HTTP requests other than GET, understands only a few content types, and supports only the QUERY_STRING environment variable for CGI programs. This web server is also not very robust; for example, if a web client closes its connection to the server, it may crash. We do not expect you to fix these problems!

The helper functions are simply wrappers for system calls that check the error codes of those system codes and immediately terminate if an error occurs. One should always check error codes! However, many programmer don't like to do it because they believe that it makes their code less readable; the solution, as you know, is to use these wrapper functions. We expect that you will write wrapper functions for the new system routines that you call.

OVERVIEW: NEW FUNCTIONALITY

In this project, you will be adding three key pieces of functionality to the basic web server. First, you make the web server multi-threaded. Second, you will implement different scheduling policies so that requests are serviced in different orders. Third, you will add statistics to measure how the web server is performing. You will also be modifying how the web server is invoked so that it can handle new input parameters (e.g., the number of threads to create).

Part 0: Producer-Consumer Queue Library

Producer-consumer queue is a useful data structure for inter-thread communication, where producer threads keep inserting items into a queue and consumer threads keep removing items from the queue. In this part of the project, you will get a feel for threads, and implement a producer-consumer queue library that can be used later in the multi-threaded web server.

Your library will support one (only one!) producer-consumer queue through three functions. We have provided the prototypes for these functions in myqueue.h in the queue_sem and queue_cv sub-directories. You should include this header file in your code to ensure that you are adhering to the specification exactly. You are not encouraged to change this header file in Part 0. We now explain each of these routines that you need to implement:

	    typedef int buffer_item;
	    buffer_item* myQueue;

	    int PCQueue_Init(int size)
	    {
	      /*Allocate a buffer large enough to hold size-number of buffer items;
	      assign the buffer pointer to myQueue;
	      conduct other necessary initialization here.*/
	      return 0 if successful;
	      return -1 if unsuccessful, such as memory allocation fails.
	    }
	    int PCQueue_Insert(buffer_item item)
	    {
	      /* Insert an item into myQueue, block until space available*/
	      return 0 if successful;
	      return -1 otherwise.
	    }
	    int PCQueue_Remove(buffer_item *item)
	    {
	      /* Remove an item from myQueue following FIFO;
		 block if myQueue is empty.
		The removed item can be retrieved from *item. */
	      return 0 if successful;
	      return -1 otherwise.
	    }

shared library

this simple multi-threaded program

makefile

Note:

What synchronization premitives shall we use?
If you are in Prof. Lu's session, you are required to use condition variables to implement this library; if you are in Prof. Swift's session, you are required to impelment two versions of this library, one using condition variables and one using semaphore. If your implementation performs any busy-waiting (or spin-waiting) instead, you will be heavily penalized.

Please finish your condition-variable implementation in directory queue_cv, and your semaphore implementation in directory queue_sem.
What is `shared library'?
Placing the routines in a shared library instead of a simple object file makes it easier for other programmers to link with your code. There are further advantages to shared (dynamic) libraries over static libraries. When you link with a static library, the code for the entire library is merged with your object code to create your executable; if you link to many static libraries, your executable will be enormous. However, when you link to a shared library, the library's code is not merged with your program's object code; instead, a small amount of stub code is inserted into your object code and the stub code finds and invokes the library code when you execute the program. Therefore, shared libraries have two advantages: they lead to smaller executables and they enable users to use the most recent version of the library at run-time.

To create a shared library named libqueue.so, use the following commands (assuming your library code is in a single file "myqueue.c"):

gcc -c -fpic myqueue.c

gcc -shared -o libqueue.so myqueue.o

To link with this library, you simply specify the base name of the library with "-lqueue" and the path so that the linker can find the library "-L.".

gcc testqueue.c -lqueue -L. -o testqueue

Of course, these commands should be placed in a Makefile. Before you run "testqueue", you will need to set the environment variable, LD_LIBRARY_PATH, so that the system can find your library at run-time. Assuming you always run myprogram from this same directory, you can use the command:

setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:.

If the setenv command returns an error "LD_LIBRARY_PATH: Undefined variable", do not panic. The error implies that your shell has not defined the environment variable. In this case, you simply need to run:

setenv LD_LIBRARY_PATH .

Part 1: Multi-threaded Server

The basic web server that we provided has a single thread of control. Single-threaded web servers suffer from a fundamental performance problem in that only a single HTTP request can be serviced at a time. Thus, every other client that is accessing this web server must wait until the current http request has finished; this is especially a problem if the current http request is a long-running CGI program or is resident only on disk (i.e., is not in memory). Thus, the most important extension that you will be adding is to make the basic web server multi-threaded.

The simplest approach to building a multi-threaded server is to spawn a new thread for every new http request. The OS will then schedule these threads according to its own policy. The advantage of creating these threads is that now short requests will not need to wait for a long request to complete; further, when one thread is blocked (i.e., waiting for disk I/O to finish) the other threads can continue to handle other requests. However, the drawback of the one-thread-per-request approach is that the web server pays the overhead of creating a new thread on every request.

Therefore, the generally preferred approach for a multi-threaded server is to create a fixed-size pool of worker threads when the web server is first started. With the pool-of-threads approach, each thread is blocked until there is an http request for it to handle. Therefore, if there are more worker threads than active requests, then some of the threads will be blocked, waiting for new http requests to arrive; if there are more requests than worker threads, then those requests will need to be buffered until there is a ready thread.

In your implementation, you must have a master thread that begins by creating a pool of worker threads, the number of which is specified on the command line. Your master thread is then responsible for accepting new http connections over the network and placing the descriptor for this connection into a fixed-size buffer (i.e., a producer-consumer queue); in your basic implementation, the master thread should not read from this connection. The number of elements in the buffer will be specified on the command line. Note that the existing web server has a single thread that accepts a connection and then immediately handles the connection; in your web server, this thread should place the connection descriptor into a fixed-size buffer and return to accepting more connections.

Each worker thread is able to handle both static and dynamic requests. A worker thread wakes when there is an http request in the queue; when there are multiple http requests available, which request is handled depends upon the scheduling policy, described below. Once the worker thread wakes, it performs the read on the network descriptor, obtains the specified content (by either reading the static file or executing the CGI process), and then returns the content to the client by writing to the descriptor. The worker thread then waits for another http request.

Note that the master thread and the worker threads are in a producer-consumer relationship. Please synchronize them using the libqueue.so library implemented in Part 0. Of course, modifications to the libqueue.so is needed to support the scheduling policies and the network-descriptor data-type.

Side note: Do not be confused by the fact that the basic web server we provide forks a new process for each CGI process that it runs. Although, in a very limited sense, the web server does use multiple processes, it never handles more than a single request at a time; the parent process in the web server explicitly waits for the child CGI process to complete before continuing and accepting more http requests. When making your server multi-threaded, you should not modify this section of the code.

Part 2: Scheduling Policies

In this project, you will implement two different scheduling policies. Note that when your web server has multiple worker threads running (the number of which is specified on the command line), you will not have any control over which thread is actually scheduled at any given time by the OS. Your role in scheduling is to determine which http request should be handled by each of the waiting worker threads in your web server.

The scheduling policy is determined by a command line argument when the web server is started and are as follows:

First-in-First-out (FIFO): When a worker thread wakes, it handles the first request (i.e., the oldest request) in the buffer. Note that the http requests will not necessarily finish in FIFO order; the order in which the requests complete will depend upon how the OS schedules the active threads.

Shortest-Job-First (SJF): When a worker thread wakes, it handles the request for the smallest file or for the shortest-running CGI script. Specifically, it follows the following policies:

Requests for static content is handled earlier than requests for dynamic content.
A request for a small static file is handled earlier than a request for a larger static file.
We use the argument to a dynamic content to approximate the execution length of a dynamic request. For example, request to ``myprogram?10'' will be handled earlier than ``yourprogram?100'', because 10 is less than 100. You can assume there is only one argument in each request for dynamic content. This algorithm can lead to the starvation of requests for large files.

You will note that the SJF policy requires that something be known about each request (e.g., the size of the file) before the requests can be scheduled. Thus, to support this scheduling policy, you will need to do some initial processing of the request outside of the worker threads; you will probably want the master thread to perform this work, which requires that it read from the network descriptor.

You will also note that you need to modify your producer-consumer queue library a little bit. Please do that inside directory queue_fifo_sjf. Do not modify the code inside part 0 directories.

You might want to modify the type declaration of buffer_item. You can define it with whatever structure type you like.
You should also add a new queue-insertion function, e.g. PCQueue_Insert_SJF, into the queue library. This function will insert an item into the queue and re-arrange the elements in the queue to support the shortest-job-first scheduling.

Part 3: Usage Statistics

You will need to modify your web server to collect a variety of statistics. Some of the statistics will be gathered on a per-request basis and some on a per-thread basis. All statistics will be returned to the web client as part of each http response. Specifically, you will be embedding these statistics in the entity headers; we have already made place-holders in the basic web server code for these headers.

Note that most web browsers will ignore these headers that it doesn't know about; to access these statistics, you will want to run our modified client (You will see a client.c in ~cs537-1/projects/p3. Type make and you will see an executable called client. Execute it by ./client host port filename).

For each request, you will record the following times or durations at the granularity of milliseconds. You may find gettimeofday() useful for gathering these statistics.

Stat-req-arrival: The arrival time, as first seen by the master thread

Stat-req-dispatch: The dispatch interval (the duration between the arrival time and when the request was picked up by worker thread)

Stat-req-read: The read interval (the duration spent reading static content)

Stat-req-complete: The completion interval (the duration between the arrival time and when the worker thread begins writing the response on the socket)

Stat-req-age: The number of requests that were given priority over this request (that is, the number of requests that were dispatched after this request arrived, but before this request was dispatched).

For dynamic requests (that is, CGI scripts), you do not need to return the read or completion interval.

You should also keep the following statistics for each thread:

Stat-thread-id: The id of the responding thread (numbered 0 to number of threads-1)

Stat-thread-count: The total number of http requests this thread has handled

Stat-thread-static: The total number of static requests this thread has handled

Stat-thread-dynamic: The total number of dynamic requests this thread has handled

Thus, for a request handled by thread number i, your web server will return the statistics for that request and the statistics for thread number i.

PROGRAM SPECIFICATIONS

Your C program must be invoked exactly as follows:

prompt> server [portnum] [threads] [buffers] [schedalg]

The command line arguments to your web server are to be interpreted as follows.

portnum: the port number that the web server should listen on; the basic web server already handles this argument. Please specify a portnum that is larger than 2000 to avoid conflicts with other services.

threads: the number of worker threads that should be created within the web server. Must be a positive integer.

buffers: the number of request connections that can be accepted at one time. Must be a positive integer. Note that it is not an error for more or less threads to be created than buffers.

schedalg: the scheduling algorithm to be performed. Must be one of FIFO or SJF.

For example, if you run your program as:

server 5003 8 16 SJF

then your web server will listen to port 5003, create 8 worker threads for handling http requests, allocate 16 buffers for connections that are currently in progress (or waiting), and use SJF scheduling for arriving requests.

HINTS

We recommend understanding how the code that we gave you works. All of the code is available from ~cs537-1/projects/p3 . We provide the following files and directories:

server.c: Contains main() for the basic web server.

request.c: Performs most of the work for handling requests in the basic web server. All procedures in this file begin with the string “request”.

cs537.c: Contains wrapper functions for the system calls invoked by the basic web server and client. The convention is to capitalize the first letter of each routine. Feel free to add to this file as you use new libraries or system calls. You will also find a corresponding header (.h) file that should be included by all of your C files that use the routines defined here.

client.c: Contains main() and the support routines for the very simple web client. To test your server, you may want to change this code so that it can send simultaneous requests to your server. At a minimum, you will want to run multiple copies of this client.

output.c: Code for a CGI program that is almost identical to the output program you used for testing your shell (basically, it repeatedly sleeps for a random amount of time). You may find that having a CGI program that takes a while to complete is useful for testing your server.

queue_cv: This is where you use condition-variables to implement a producer-consumer queue library.

queue_sem: If you are in Prof. Swift's class, this is where you use semaphores to implement a producer-consumer queue library.

queue_fifo_sjf: This is where you implement the producer-consumer queue library that can support FIFO and SJF scheduling.

We also provide you with a sample Makefile that creates server, client, and output.cgi. You can type make to create all of these programs. You can type make clean to remove the object files and the executables. You can type make server to create just the server program, etc. As you create new files, you will need to add them to the Makefile.

The best way to learn about the code is to compile it and run it. Run the server we gave you with your preferred web browser. Run this server with the client code we gave you. You can even have the client code we gave you contact any other server (e.g., www.cs.wisc.edu). Make small changes to the server code (e.g., have it print out more debugging information) to see if you understand how it works.

We have provided a few comments, marked with CS537 , to point you to where we expect you will make changes for this project.

As a third step, we recommend making the server multi-threaded. Finally, add in the different scheduling algorithms, beginning with the easiest (FIFO).

We anticipate that you will find the following routines useful for creating and synchronizing threads: pthread_create, pthread_mutex_init, pthread_mutex_lock, pthread_mutex_unlock, pthread_cond_init, pthread_cond_wait, pthread_cond_signal. You probably will also need fstat to find out the size of a file in your Shortest-Job-First scheduling implementation. To find information on these library routines, please read the manuals.

HANDIN

For professor Lu's class, please put a copy of everything in ALL partners' directories at ~cs537-1/handin/loginID/p3.
For professor Swift's class, please turn your code in only once for each group (it can be any group member) at ~cs537-2/handin/loginID/p3.

You should turn in:

Source files and makefiles for different versions of producer-consumer queue libraries in different sub-directories.
Server source files (*.c and *.h).
A Makefile that can compile all your code.
A textfile named README beginning with a list of the login IDs of your group members. Following that, include the full names of your group members, and then anything you want us to know about your project (such as things you know that do not work).
Do not submit any .o files. You do not need to copy any files for creating clients or CGI programs.

Thank Professor Remzi A.-D. for providing all the project materials.