here . for other uses see spec disambiguation . standard performance evaluation corporation formation 1988 type not for profit headquarters gainesville virginia membership hardware & software vendors universities research centers staff 4 website http www.spec.org this article needs additional citations for verification . please help improve this article by adding citations to reliable sources . unsourced material may be challenged and removed . april 2011 learn how and when to remove this template message the standard performance evaluation corporation spec is an american non profit organization that aims to produce establish maintain and endorse a standardized set of performance benchmarks for computers . 1 spec was founded in 1988 . 2 3 spec benchmarks are widely used to evaluate the performance of computer systems ; the test results are published on the spec website . results are sometimes informally referred to as specmarks or just spec . spec evolved into an umbrella organization encompassing four diverse groups ; graphics and workstation performance group gwpg the high performance group hpg the open systems group osg and the newest the research group rg . more details are on their website ; 1 . contents 1 membership 2 membership levels 3 spec benchmark suites 4 portability 5 licensing 6 benchmarks 6.1 current benchmarks 6.2 committees for future benchmarks 6.3 retired benchmarks 7 culture 8 references 9 external links membership edit membership in spec is open to any interested company or entity that is willing to commit to spec s standards . it allows participation in benchmark development participation in review of results complimentary software based on group participation the list of members is available on spec s membership page ; 2 . membership levels edit sustaining membership requires dues payment and typically includes hardware or software companies . spec associates pay a reduced fee and typically include universities . spec supporting contributors are invited to participate in development of a single benchmark and do not pay fees . spec benchmark suites edit the benchmarks aim to test real life situations . there are several benchmarks testing java scenarios from simple computation specjbb to a full system with java ee database disk and network specjenterprise . the spec cpu suites test cpu performance by measuring the run time of several programs such as the compiler gcc the chemistry program gamess and the weather program wrf . the various tasks are equally weighted ; no attempt is made to weight them based on their perceived importance . an overall score is based on a geometric mean . portability edit spec benchmarks are written in a portable programming language usually c c java or fortran and the interested parties may compile the code using whatever compiler they prefer for their platform but may not change the code . manufacturers have been known to optimize their compilers to improve performance of the various spec benchmarks . spec has rules that attempt to limit such optimizations . licensing edit in order to use a benchmark a license has to be purchased from spec ; the costs vary from test to test with a typical range from several hundred to several thousand dollars . this pay for license model might seem to be in violation of the gpl as the benchmarks include software such as gcc that is licensed by the gpl . however the gpl does not require software to be distributed for free only that recipients be allowed to redistribute any gpled software that they receive ; the license agreement for spec specifically exempts items that are under licenses that require free distribution and the files themselves are placed in a separate part of the overall software package . benchmarks edit current benchmarks edit specapc for 3ds max 2011 performance evaluation software for systems running autodesk 3ds max 2011 . specapcsm for lightwave 3d 9.6 performance evaluation software for systems running newtek lightwave 3d v9 .6 software . spec cpu2006 combined performance of cpu memory and compiler . designed to provide performance measurements that can be used to compare compute intensive workloads on different computer systems spec cpu2006 contains two benchmark suites cint2006 for measuring and comparing compute intensive integer performance and cfp2006 for measuring and comparing compute intensive floating point performance . cint2006 specint testing integer arithmetic with programs such as compilers interpreters word processors chess programs etc. cfp2006 specfp testing floating point performance with physical simulations 3d graphics image processing computational chemistry etc. spec cpuv6 the cpu search program seeks to encourage those outside of spec to assist them in locating applications that could be used in the next cpu intensive benchmark suite currently designated as spec cpuv6 . specjbb2013 evaluates the performance of server side java by emulating a three tier client server system with emphasis on the middle tier . the specjbb2013 benchmark has been developed from the ground up to measure performance based on the latest java application features . it is relevant to all audiences who are interested in java server performance including jvm vendors hardware developers java application developers researchers and members of the academic community . specjenterprise2010 a multi tier benchmark for measuring the performance of java 2 enterprise edition j2ee technology based application servers . specjenterprise2010 measures full system performance for java enterprise edition java ee 5 or later application servers databases and supporting infrastructure and expands the scope of the specjappserver2004 benchmark . specjms2007 java message service performance . specjms2007 is the first industry standard benchmark for evaluating the performance of enterprise message oriented middleware servers based on jms java message service . it provides a standard workload and performance metrics for competitive product comparisons as well as a framework for indepth performance analysis of enterprise messaging platforms . specjvm2008 measuring basic java performance of a java runtime environment on a wide variety of both client and server systems . specjvm2008 is a benchmark suite for measuring the performance of a java runtime environment jre containing several real life applications and benchmarks focusing on core java functionality . the specjvm2008 workload mimics a variety of common general purpose application computations . specapc performance of several 3d intensive popular applications on a given system spec mpi2007 for evaluating performance of parallel systems using mpi message passing interface applications . spec omp2001 v3 .2 for evaluating performance of parallel systems using openmp http www.openmp.org applications . power specpower ssj2008 evaluates the energy efficiency of server systems . specpower ssj2008 is the first industry standard spec benchmark that evaluates the power and performance characteristics of volume server class computers . the initial benchmark addresses the performance of server side java and additional workloads are planned . server efficiency rating tool sert . the server efficiency rating tool sert was created by standard performance evaluation corporation spec at the request of the us environmental protection agency . it is intended to measure server energy efficiency initially as part of the second generation of the us environmental protection agency epa energy star for computer servers program . designed to be simple to configure and use via a comprehensive graphical user interface the sert uses a set of synthetic worklets to test discrete system components such as memory and storage providing detailed power consumption data at different load levels . other spec benchmarks incorporating power measurement spec accel spec omp2012 specvirt sc2013 network file system specsfs2008 file server throughput and response time supporting both nfs and cifs protocol access . spec s benchmark designed to evaluate the speed and request handling capabilities of file servers utilizing the nfsv3 and cifs protocols . specsip infrastructure2011 sip server performance graphics and workstation performance specviewperf 12 specviewperf 11 performance of an opengl 3d graphics system tested with various rendering tasks from real applications specwpc specapcsm for 3ds max 2015 specapcsm for maya 2012 specapcsm for ptc creo 2.0 specapcsm for siemens nx 8.5 specapcsm for solidworks 2013 previous versions of specapc and specviewperf benchmarks specvirt sc2013 specvirt evaluates the performance of datacenter servers used in virtualized server consolidation environments . virtualization . spec s updated benchmark addressing performance evaluation of datacenter servers used in virtualized server consolidation . specvirt sc2013 measures the end to end performance of all system components including the hardware virtualization platform and the virtualized guest operating system and application software . the benchmark supports hardware virtualization operating system virtualization and hardware partitioning schemes . high performance computing openmp mpi openacc opencl spec accel spec accel tests performance with a suite of computationally intensive parallel applications running under the opencl and openacc apis . the suite exercises the performance of the accelerator host cpu memory transfer between host and accelerator support libraries and drivers and compilers . spec mpi2007 mpi2007 is spec s benchmark suite for evaluating mpi parallel floating point compute intensive performance across a wide range of cluster and smp hardware . the suite consists of the initial mpim2007 suite and mpil2007 which contains larger working sets and longer run times than mpim2007 . spec omp2012 the successor to the omp2001 designed for measuring performance using applications based on the openmp 3.1 standard for shared memory parallel processing . omp2012 also includes an optional metric for measuring energy consumption . sip specsip infrastructure2011 . spec s benchmark designed to evaluate a system s ability to act as a sip server supporting a particular sip application . the application modeled is a voip deployment for an enterprise telco or service provider where the sip server performs proxying and registration . spec tools server efficiency rating tool sert . intended to measure server energy efficiency initially as part of the second generation of the us environmental protection agency epa energy star for computer servers program . designed to be simple to configure and use via a comprehensive graphical user interface the sert uses a set of synthetic worklets to test discrete system components such as memory and storage providing detailed power consumption data at different load levels . chauffeur worklet development kit wdk . chauffeur was designed to simplify the development of workloads for measuring both performance and energy efficiency . because chauffeur contains functions that are common to most workloads developers of new workloads can focus on the actual business logic of the application and take advantage of chauffeur s capabilities for configuration run time data collection validation and reporting . chauffeur was initially designed to meet the requirements of the sert . however spec recognized that the framework would also be useful for research and development purposes . the chauffeur framework is now being made available as the chauffeur worklet development kit wdk . this kit can be used to develop new workloads or worklets in chauffeur terminology . researchers can also use the wdk to configure worklets to run in different ways in order to mimic the behavior of different types of applications . these features can be used in the development and assessment of new technologies such as power management capabilities . ptdaemon . the spec ptdaemon software is used to control power analyzers in benchmarks which contain a power measurement component . committees for future benchmarks edit handheld handheld spec has formed a committee chartered for the development of and support for a compute intensive benchmark suite for handheld devices . soa soa . spec has formed a new subcommittee to develop standard methods of measuring performance for typical middleware database and hardware deployments of applications based on the service oriented architecture soa . retired benchmarks edit spec cpu2000 spec cpu95 spec cpu92 spec 2001 spec hpc2002 spec hpc96 specjappserver2004 specjappserver2002 specjappserver2001 specjbb2005 specjbb2000 specjvm98 specmail2009 specmail2008 specmail2001 spec sdm91 spec sfs97 r1 3.0 spec sfs97 2.0 spec sfs93 laddis spec virt sc2010 specweb2009 specweb2005 specweb96 specweb99 specweb99 ssl culture edit spec attempts to create an environment where arguments are settled by appeal to notions of technical credibility representativeness or the level playing field . spec representatives are typically engineers with expertise in the areas being benchmarked . benchmarks include run rules which describe the conditions of measurement