Benchmark
We run the Wisconsin Proxy Benchmark (WPB) to test the performance of proxies.
Machines
The clients and servers are deployed on the COW (cluster of workstations)
machines. For this benchmark, we use four machines to host client processes
and two machines to host server processes. The proxy server is deployed
on a different machine
Client Number
We run the benchmark with the number of client processes ranges from 8
to 96. (Some proxies can only support a small number of client processes,
so we can not get result after a number of clients) The number of server
process is half of the client process number. For each client number, we
run the benchmark twice. On the first run, the proxy is not configured
with caching, so it will just redirect the client request. On the second
run, the proxy is configured with caching.
Configuration parameters
For each run, each client sends 200 requests to the proxy server. The server's
latency is set to be 3 seconds.
Running the benchmark
We use a script to start the clients and servers. At the same time, we
also start two process on each server machine to monitor the virtual memory
and I/O state of the server machine (using iostat and vmstat)
Between two runs, there is a five minutes interval so that all the TCP
connections in the client, server and proxy machines will be completely
closed.
When the proxy is configured with caching, the proxy cache is cleared
before running the benchmark.
Result
As described in Wisconsin Proxy Benchmark, each run has two phases. During
the first phase, clients only send new requests to proxy so that there
is inherently no cache hit in the requests. During the second phase, clients
send some requests that have been sent during the first phase, thus may
hit in the proxy cache.
We report for each number of client process both the cache and no-cache
version:
-
the total requests sent by all clients,
-
the total bytes received by all the clients,
-
the average latency the clients observed during the first phase,
-
the average latency the clients observed during the second phase,
-
the request hit rate if the proxy cache is enabled,
-
the byte hit rate if the proxy cache is enabled.
Cache Size
For CERN, Squid and Apache, when running the benchmark with caching, we
set the cache size of the proxy to be 75M. For each number of client, the
benchmark is run 3 times and we results the average of them.
For Mows, JigSaw, Roxen, Wcol and Delegated, when running the benchmark
with caching, we set the cache size of proxy to be 20M. For each number
of client, we just run benchmark once.