To measure the overhead incurred by the cache applets, we use the WebStone 2.0 standard Web server benchmark (9, 9) and compare the response times of the original CERN httpd proxy and the Active Cache proxy with various cache applets. In each of our Active Cache tests, we assume that all the documents are associated with the same applet. We test the ``null'' applet, the user access logging applet ``log,'' the advertising banner rotation applet ``ads,'' the access permission checking applet ``apc,'' the server-side include expansion applet ``ssi,'' and the client-specific information distribution applet ``csid.''
Though WebStone is a Web server benchmark, it can be used to test the performance of the proxy upon cache hits of different sizes. We use the filelist.standard in the WebStone 2.0 benchmark and create five files of size 500B, 5KB, 50KB, 500KB and 5MB. The clients request the files with the specified frequency in filelist.standard. The proxy machine is installed between the clients and the server. Since there are only five files involved, once the proxy cache is warmed up, all requests are cache hits and the test stresses the proxy system, instead of the server system. We run experiments with 1, 10 and 20 WebStone clients. The experiments all last over 10 minutes to obtain stable results. Our test platform includes SPARC 20 workstations running Solaris 2.4 as the client and the server machines. Our proxy machine for the ``null,'' ``apc,'' ``log,'' and ``ssi'' applets is a 99MHz Intel x86 workstation running Solaris 2.5. Our proxy machine for the ``ads'' and ``csid'' applets is a SPARC20 running Solaris 2.6; due to a bug in the Java thread library on the Intel platform, these two applets hang on the Intel platform. We are working on resolving the problem.
| Proxy | 1 client | 10 clients | 20 clients |
| null | 1.47 | 1.75 | 1.73 |
| log | 2.16 | 2.31 | 2.24 |
| ads | 1.40 | 2.44 | 3.23 |
| apc | 2.50 | 2.73 | 2.72 |
| ssi | 3.81 | 4.00 | 3.80 |
| csid | 1.06 | 2.04 | 2.23 |
Table 1 lists the response time degradation of each applet, that is, the ratio between the client response time when invoking the applet and the client response time under the original CERN httpd. The ``null'' applet result shows that the mechanics of establishing a Java virtual machine and invoking the applet costs 47% to 75% degradation in response time. There are a number of reasons for it, including the increase in the process image and the corresponding increase in the forking cost, and the CPU overhead for finding the class, finding the method, and invoking the method. The other applets increase the client latency by a factor of 1.5 to 4.
Monitoring the proxy system using ``vmstat'' shows that the performance degradation is mostly caused by CPU overheads. In particular, the ``ads'' applet and the ``ssi'' applet incur CPU overhead that is proportional to the document size because they scan the cached document. The CPU overhead appears to heavily depend on the coding of the applet and the efficiency of Java implementations, particularly string operations. As the speed of Java improves and as we fine-tune our applet implementation, the CPU overhead will be reduced.
The measurements show that for a proxy system to support the Active Cache protocol and yet maintain the same throughput, its CPU resource needs to be increased. Fortunately, the workload is easily parallelizable and multi-processor systems or clusters of workstations can improve the throughput significantly. In most proxy systems today, disk arms and network connections are typically the bottlenecks, not the CPU resources. Whether the Active Cache paradigm will change the situation remains to be seen.
To summarize, our prototype implementation shows that Active Cache is a practical and feasible scheme to implement in proxies. It increases the CPU demands, and in a sense, trades local CPU resources for network bandwidth savings. Given that in today's technology, microprocessors are typically the cheap resources, the tradeoff is well worthwhile. We believe that the benefits of Active Cache greatly outweigh its implementation cost, and every proxy should support the Active Cache protocol.