Recommended Benchmarking Steps and Rules

The setup of the benchmarking experiments should follow the rule that the ratio between the number of client machines and the number of server machines is always 2 to 1, and the ratio between the total number of client processes to the total number of server processes is also kept at 4 to 1. The server latency should be at least 3 seconds. Until we have a better understanding of the spatial locality in user access streams, these ratios seem to be a reasonable choice.

Each run should last at least 10 minutes. Thus, the number of operations should be chosen so that this minimum duration is achieved. The total size of HTTP documents fetched by the clients should be at least four times the cache size of the proxy.

The current version of the benchmark does not model DNS lookups, HTTP 1.1 persistent connections, conditional Get and other forms of HTTP requests, realistic path name for URL's and spatial locality in Web accesses (i.e., once a user accesses a document from one Web server, it tends to access other documents at the same Web server). We are continueing the development of the benchmark and hope to eliminate these limitations in future versions of the benchmark.