Cramer Versus Cramer

Cramer vs Cramer, Java Virtual Machine vs Java Virtual Machine. This article try to give information of what is the best java virtual machine for a given problem. To do this we have choosen two benchmarks so we can give an objective opinion of which are the best virtual machine. There is a lot of benchmarks and a lot of java virtual machines, but we have focused into two virtual machines, Java HotSpot(TM) Client VM (build 1.5.0_05-b05) and IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20051103 (JIT enabled), and two benchmarks, JGF Benchmark http://www.epcc.ed.ac.uk/javagrande/javag.html and Richards and DeltaBlue Benchmark http://research.sun.com/people/mario/java_benchmarking/download2.html.

I would try to explain both benchmarks, but for more information please visit its site.

JGF is composed by three big groups of benchmarks, Sequential Benchmark, Multi-Threaded Benchmark and MPJ Benchmark.

• The sequential benchmarks, suitable for single processor execution.
• The multi-threaded benchmarks, suitable for parallel execution on shared memory multiprocessors.
• The MPJ benchmarks, suitable for parallel execution on distributed memory multiprocessors.

For our porpouse only the first and second benchmarks would be executed.

Sequential Benchmark also have the next subsections:

• Arith: execution of arithmetic operations
• Assign: variable assignment
• Cast: casting
• Create: creating objects and arrays
• Loop: Loop overheads
• Math: execution of maths library operations
• Method: method invocation
• Serial: Serialisation
• Exception Exception handling
• Serial: Read/Write Lists
• Method: executing methods in same object or foreign object

• Series: Fourier coefficient analysis
• LUFact: LU Factorisation
• SOR: Successive over-relaxation
• HeapSort: Integer sorting
• Crypt: IDEA encryption
• FFT: FFT
• Sparse: Sparse Matrix multiplication

• Search: Alpha-beta pruned search
• Euler: Computational Fluid Dynamics
• MD: Molecular Dynamics simulation
• MC: Monte Carlo simulation
• Ray Tracer: 3D Ray Tracer

Multi-Thread Benchmark also have the next subsection:

• ForkJoin: forking and joining threads
• Barrier: barrier synchronisation
• Sync: synchronized blocks and methods

• Series: Fourier coefficient analysis
• LUFact: LU Factorisation
• SOR: Successive over-relaxation
• Crypt: IDEA encryption
• Sparse: Sparse Matrix multiplication

• MolDyn: Molecular Dynamics simulation
• MonteCarlo: Monte Carlo simulation
• RayTracer: 3D Ray Tracer

At the heart of Richards is a task dispatcher (Richards.schedule). Tasks come in 4 different flavors, each represented by a class (DeviceTask, HandlerTask, IdleTask, WorkTask, all subclasses of Task). Each kind of task has an associated work function (fn). At startup (Richards.run), a particular task mix is created, and then the tasks are scheduled, each having its work function invoked. The work functions manipulate work packets and packet queues. At the end of the benchmark (in Richards.run) the number of queued and held packets is checked against the correct value to assist in verifying that the benchmark ran correctly.

For our pourpouse we have executed all sequential benchmark. And only the first group of multi-thread becnhmark. Also, we have runned Richards Benchmark.

All this benchmarks has been runned on a Pentium IV 3 GHz and 512 MB RAM, on Ubuntu Linux 5.10. The optional arguments for java command has been -Xms256m -Xmx512m because there is some tests that needs as memory as possible or a RuntimeException would be thrown.

Lets see the results of the first section of sequencial benchmark.

In this graph we can see that for mathematical operations is faster Java HotSpot than IBM. IBM Java Virtual Machine only wins in creations of objects and rounding numbers, in all other operations, like managing lists, arrays or casting is better Java HotSpot.

About section 2 in sequencial benchmark, we can see that we have the same tests but with three different results, that is because the size of data increases in every test. See JGF homepage to see how the values increases because there is no standard growth between the tests.

We can say that in all tests Java HotSpot is much faster that IBM JVM, this is because HotSpot is better in numerical problems than IBM Java Virtual Machine.

Although the differences has been minimized, Java HotSpot is faster than IBM JVM.

When the load is heavy, IBM JVM in some tests are faster than Java HotSpot, surely we can say that IBM JVM manages memory data better than Java HotSpot.

Now, let's do the same, but with the second group of benchmarks, remember that only the first group, because we are not interested in knowing how fast is a JVM executing mathematical operations in parallel, because we have already executed when I was executing sequential, what we really wants to know is which Java Virtual Machine manages the threads faster.

As we can see, with only one thread, in this case (and all future cases), IBM Virtual Machine is faster than Sun Java HotSpot. See that synchronizers in IBM is spectacular in comparision to Java HotSpot.

Note that in forkjoin a negative value is calculated. I have tried three times the same test and with one thread, always the result has been negative. I think that an overfload has been produced. Of course it is impossible a negative count of forkjoins.

With two threads, the difference between both JVM has been reduced in three first tests, but in synchronizations, IBM still be the best ione in difference.


With 5 , 10 and 20 threads happens the same as 2 threads. IBM still being faster than Java HotSpot.















































Finally, although first tests sun VM is faster than IBM, we can say but that with a lot of threads synchronizating, IBM VM is faster than sun VM.

In summary, in thread environment like web servers, IBM JVM seems to be better than SUN VM. On the other hand, SUN VM in standard applications offer a better throughput.