Research Use
The run rules make provisions for research
use where the rules for reporting SPECjvm98 metrics may not all be followed.
The reporting of any such results is strictly regulated, and in no case
may any derivative metrics be defined based on the SPEC benchmarks or tool
harness.
Execution Modes
The benchmark tool harness provides for several different execution modes.
Some of the non run rule compliant modes may be of interest to those studying
particular aspects of Java performance.
- As an applet loaded from a web server (http://). This is the execuction
mode used for producing reportable metrics.
- As an applet loaded from a local file system (file://)
- Typically you would use a web browser's button or command to open
file and select SpecApplet.html from a locally installed copy of the
benchmark suite.
- A common problem associated with this mode is that the web browser's
security manager may not allow the applet access to the file system.
- When evaluating results from your tests, keep in mind the difference
in speed between local disk and that of your network, and how much work
the security manager is doing verifying class files. Class files loaded
from local disk may be considered more trustworthy, and requiring fewer
security restrictions, than those loaded over the net.
- As an applet loaded from a remotely mounted file system (file:///net/)
- Functionally, this is just like loading from the local file system
above.
- As an application
- From the command line, change into the directory where you installed
the benchmark suite and execute SpecApplication. Be sure that you have
the top level benchmark directory included in your CLASSPATH. If you are
using UNIX and the Bourne (sh), Korn (ksh), or similar shell, you can do
this with the shrc file. E.g.,
cd top_level_directory
. ./shrc
java -mx32m SpecApplication
- You can also use the scripts run.sh (UNIX) or
RUN.BAT (Windows) which attempt to find your Java command
and to set your CLASSPATH appropriately.
You can modify these scripts for your own needs.
- Be sure you don't set your CLASSPATH to load classes locally and then
attempt later to run as an applet from a web server. The benchmarks will run,
but will not comply with the run rules and so the metrics will not be reportable.
See Common Problems.
- On many systems you may instead open the installation directory in
a graphical file browser and double click the icon labeled SpecApplication.class
- When evaluating results from your tests, keep in mind the caveats regarding
running as an applet from a local disk. In addition, consider any differences
in security policy or runtime environment which your system may make between
applets and applications.
- From the command line without GUI controls
- Detailed instructions
- When evaluating results from your tests, keep in mind the caveats regarding
running as an application above. In addition consider any additional overhead
that may be required to maintain (even an inactive) graphical user interface.
If you run benchmarks individually, each in its own Java Virtual Machine,
consider the difference between the presence or absense of interaction
between consecutively executed programs.
Applet Parameters
A single parameter may be specified to the benchmark suite applet, which
specifies an alternate location for the user properties
file, instead of the default "props/user" relative to the
codebase. Then in that alternate user properties file you can specify additional
parameters to control benchmark operation. The property file is specified
for example as:
<applet code="SpecApplet.class" width=530 height=375>
<param name=properties
value=http://localhost:8080/jvm98/props/user.walter>
</applet>
Interactive versus Batch
Normally the benchmarks are run in interactive mode so that you can control
their execution throught the graphic user interface. However, for use in
automated test environments it may be convenient to run them in batch mode
where there is no user interaction. To do this, set spec.initial.interactive
to false, either by editing the user properties file or by specifying an
alternate user properties file as described above. A compliant run will
be performed and the results will be automatically emailed to the address
you specify in spec.testx.emailTo.
You may also run batch mode from the command
line by using the -b flag.
Benchmarks
Programmers' Reference
For research use, you may wish to run your own programs in the SPEC tool
harness. The documentation below as well as the provided source code of
these benchmarks should help you fit programs into the harness. SPEC cannot
provide assistance for this activity.
Next topic: Common Problems
JVM
Client98 Help