VMmark: VMmark is the industry’s first virtualization benchmark for x86-based computers. It features a novel tile-based scheme for measuring the scalability of consolidated workloads and provides a consistent methodology that captures both the overall scalability and individual application performance.
Oracle Applications Standard Benchmark: The Oracle Applications Standard Benchmark is focused on ERP applications and represents a mixed workload intended to model the most common transactions operating on the most widely used enterprise application modules. Definitions of transactions that comprise the benchmark load were obtained through collaboration with functional consultants and are representative of typical customer workloads, with batch transactions representing 25% of the total workload.
SAP-SD two-tier: The Sales and Distribution (SD) Benchmark covers a sell-from-stock scenario, which includes the creation of a customer order with five line items and the corresponding delivery with subsequent goods movement and invoicing. It consists of the following transactions:
- Create an order with five line items. (VA01)
- Create a delivery for this order. (VL01N)
- Display the customer order. (VA03)
- Change the delivery (VL02N) and post goods issue.
- List 40 orders for one sold-to party. (VA05)
- Create an invoice. (VF01)
Each benchmark user has his or her own master data, such as material, vendor, or customer master data to avoid data-locking situations.
Important Note: On January 1, 2009, the SAP SD Benchmark was updated. Alongside the upgrade to SAP Business Suite 7 software and the SAP enhancement package 4 for SAP ERP, a number of additional, necessary updates were implemented. These are aimed at bringing the SD benchmark in line with the "sell from stock" scenario as it is productively used by SAP customers. The updates are transparent; that is, the steps of the benchmark scenario remain unchanged. Please be aware that these changes make the SD benchmark more resource-intensive, which has a direct impact on the benchmark results.
TPC-C: The TPC-C benchmark is designed to measure a server’s ability to function as an on-line transaction processing (OLTP) database server. This benchmark simulates a complete computing environment where a population of users executes order-entry transactions against a database. These transactions include placing a new order, making or receiving a payment, checking the order status, monitoring delivery, and tracking stock levels. The TPC-C result is a measurement of the number of new order transactions generated per minute while the system is also executing the other four transaction types.
TPC-E: TPC-E is a new On-Line Transaction Processing (OLTP) workload developed by the TPC. The TPC-E benchmark simulates the OLTP workload of a brokerage firm. The focus of the benchmark is the central database that executes transactions related to the firm’s customer accounts. Although the underlying business model of TPC-E is a brokerage firm, the database schema, data population, transactions, and implementation rules have been designed to be broadly representative of modern OLTP systems.
TPC-H: TPC-H is a decision support benchmark. It consists of a suite of business oriented ad-hoc queries and concurrent data modifications. The queries and the data populating the database have been chosen to have broad industry-wide relevance. This benchmark illustrates decision support systems that examine large volumes of data, execute queries with a high degree of complexity, and give answers to critical business questions.
SPECweb®2005: SPECweb®2005 is the next-generation SPEC benchmark for evaluating the performance of World Wide Web Servers. In response to rapidly advancing Web technology, the SPECweb2005 benchmark includes many sophisticated and state-of-the-art enhancements to meet the modern demands of Web users of today and tomorrow.
SPECpower_ssj™2008: SPECpower_ssj™2008 is the first industry-standard SPEC benchmark that evaluates the power and performance characteristics of volume server class computers. With SPECpower_ssj2008, SPEC is defining server power measurement standards in the same way we have done for performance.
The initial benchmark addresses only one subset of server workloads: the performance of server side Java. It exercises the CPUs, caches, memory hierarchy and the scalability of shared memory processors (SMPs) as well as the implementations of the JVM (Java Virtual Machine), JIT (Just-In-Time) compiler, garbage collection, threads and some aspects of the operating system. The benchmark runs on a wide variety of operating systems and hardware architectures and should not require extensive client or storage infrastructure.
TOP500® Supercomputer Sites: The TOP500 project was started in 1993 to provide a reliable basis for tracking and detecting trends in high-performance computing. Twice a year, a list of the sites operating the 500 most powerful computer systems is assembled and released. The best performance on the Linpack benchmark is used as performance measure for ranking the computer systems. The list contains a variety of information including the system specifications and its major application areas.
FLUENT: Fluent 12 is commercial computational fluid dynamics (CFD) package and is a product of Ansys Inc. Fluent 12 has broad physical modeling capabilities well suited to a variety of industrial applications. Fluent provides a set of standardized benchmarks covering a range of fluid flow simulations and software features.
LS-DYNA: LS-DYNA is a general purpose transient dynamic finite element program capable of simulating complex real world problems, for use in various industries, including Automobile Design, Aerospace, Manufacturing, and Bioengineering. Benchmark data sets and cluster performance quotations are available from the independent web site www.topcrunch.org. The www.topcrunch.org benchmarks associated with structural dynamics (LS-DYNA) address domain decomposition, message passing, load balancing, and dynamic memory allocation in automotive crash safety analysis.
STAR-CD: STAR-CD provides a platform for industrial CFD simulation. Going beyond just a CFD code, the latest release, STAR-CD V4, introduces the capability to perform structural analysis calculations using a methodology based upon its CFD solver technology, a comprehensive solution for flow, thermal and stress simulation that has been available in a single general-purpose commercial finite-volume code. STAR-CD V4 is fully polyhedra enabled and by using the latest polyhedra-solver technology, STAR-CD V4 delivers significant benefits in speed, robustness and usability.
ECLIPSE: Eclipse has been a leading commercial reservoir simulation software for over 25 years because of its breadth of capabilities, parallel scalability, utility computing, and superior platform coverage. Eclipse Version 2008.1 continues to deliver accurate, high performance solutions for all of today's complex reservoir systems. Eclipse 100 is a one million cell black-oil model, and Eclipse 300 is a two million cell compositional model.
SPEComp® 2001: SPEComp® 2001 (OpenMP Benchmark Suite) is SPEC's first benchmark suite for evaluating performance based on OpenMP (http://openmp.org/wp/) applications. SPEComp® 2001 continues the SPEC tradition of giving HPC users the most objective and representative benchmark suite for measuring the performance of SMP (shared memory multi-processor) systems.
STREAM: The Stream benchmark measures sustainable memory bandwidth in megabytes per second by utilizing four computational loops on a user-definable input array size. These loops are Copy, Scale, Add, and Triad. The selected array size must be at least 4x the sum of all the last level processor caches. This forces the CPU to bypass the caches and go out to the memory subsystem for each transaction allowing for a more representative measure of memory bandwidth.
SPECcpu®2006: SPECcpu®2006 is SPEC's next-generation, industry-standardized, CPU-intensive benchmark suite, stressing a system's processor, memory subsystem and compiler. SPEC designed SPECcpu2006 to provide a comparative measure of compute-intensive performance across the widest practical range of hardware using workloads developed from real user applications. These benchmarks are provided as source code and require the user to be comfortable using compiler commands as well as other commands via a command interpreter using a console or command prompt window in order to generate executable binaries. The current version of the benchmark suite is V1.1, released in June 2008. Note that SPEC is currently accepting results submissions based on both SPECcpu®2006 V1.0.1 and V1.1.
SPECjbb®2005: SPECjbb®2005 (Java Server Benchmark) is SPEC's benchmark for evaluating the performance of server side Java. Like its predecessor, SPECjbb®2000, SPECjbb®2005 evaluates the performance of server side Java by emulating a three-tier client/server system (with emphasis on the middle tier). The benchmark exercises the implementations of the JVM (Java Virtual Machine), JIT (Just-In-Time) compiler, garbage collection, threads and some aspects of the operating system. It also measures the performance of CPUs, caches, memory hierarchy and the scalability of shared memory processors (SMPs). SPECjbb®2005 provides a new enhanced workload, implemented in a more object-oriented manner to reflect how real-world applications are designed and introduces new features such as XML processing and BigDecimal computations to make the benchmark a more realistic reflection of today's applications.
SPEC benchmark descriptions are copyright Standard Performance Evaluation Corporation. Used with permission.