Benchmark

We run the Wisconsin Proxy Benchmark (WPB) to test the performance of proxies.

Machines

The clients and servers are deployed on the COW (cluster of workstations) machines. For this benchmark, we use four machines to host client processes and two machines to host server processes. The proxy server is deployed on a different machine

Client Number

We run the benchmark with the number of client processes ranges from 8 to 96.  (Some proxies can only support a small number of client processes, so we can not get result after a number of clients) The number of server process is half of the client process number. For each client number, we run the benchmark twice. On the first run, the proxy is not configured with caching, so it will just redirect the client request. On the second run, the proxy is configured with caching.

Configuration parameters

For each run, each client sends 200 requests to the proxy server. The server's latency is set to be 3 seconds.

Running the benchmark

We use a script to start the clients and servers. At the same time, we also start two process on each server machine to monitor the virtual memory and I/O state of the server machine (using iostat and vmstat)

Between two runs, there is a five minutes interval so that all the TCP connections in the client, server and proxy machines will be completely closed.

When the proxy is configured with caching, the proxy cache is cleared before running the benchmark.

Result

As described in Wisconsin Proxy Benchmark, each run has two phases. During the first phase, clients only send new requests to proxy so that there is inherently no cache hit in the requests. During the second phase, clients send some requests that have been sent during the first phase, thus may hit in the proxy cache.

We report for each number of client process both the cache and no-cache version:

Cache Size

For CERN, Squid and Apache, when running the benchmark with caching, we set the cache size of the proxy to be 75M. For each number of client, the benchmark is run 3 times and we results the average of them.

For Mows, JigSaw, Roxen, Wcol and Delegated, when running the benchmark with caching, we set the cache size of proxy to be 20M. For each number of client, we just run benchmark once.