CoralReactor Performance Numbers

In this article we present the CoralReactor UDP and TCP latency numbers when two separate JVMs exchange messages over loopback.

The machine used for the benchmarks below was an Intel i7 quad-core (4 x 3.50GHz) Ubuntu box overclocked to 4.50Ghz.

The test consists of the following steps:

  • a client running in a JVM sends a 256-byte message to an echo server running in another JVM
  • the first 8 bytes of the message is a long containing the timestamp in nanos when the message was sent by the client
  • the echo server receives the message, reads the the timestamp, reads the remaining 248 bytes and then calculates the elapsed time (now – timestamp)
  • the echo server stores the elapsed time and sends the message back to the client (i.e. the echo)
  • the client gets the echo and sends the next message so the echo server can make another latency measurement
  • the cycle repeats one million times, in other words, one million messages are sent
  • two passes: one to warmup with one million messages and another one to measure with one million messages
  • then the average and the percentiles are calculated and presented

NOTE: Everyone’s network environment is different, and we usually have a hard time comparing over-the-wire benchmark numbers. To make this simple we present loopback numbers (i.e. client and server running on the same machine but different JVMs) which are easy to compare and weed out external factors, isolating the performance of the network code. To calculate total numbers you should add your typical over-the-wire network latency. A 256-byte packet traveling through a 10 Gigabits ethernet will take at least 382 nanoseconds to go from NIC to NIC (ignoring the switch hop). If your ethernet is 1 Gigabits then the latency is at least 3.82 micros on top of CoralReactor numbers. Another factor is the network card latency. Going from JVM to kernel to NIC can be costly and some good network cards optimize that by offering kernel bypass (i.e. Open OnLoad from SolarFlare).

UDP Latencies

Messages: 1,000,000 (size 256 bytes)
Avg Time: 1.747 micros (one-way)
Min Time: 1.486 micros
Max Time: 11.117 micros
Garbage created: ZERO
75% = [avg: 1.696 micros, max: 1.832 micros]
90% = [avg: 1.724 micros, max: 1.899 micros]
99% = [avg: 1.742 micros, max: 1.979 micros]
99.9% = [avg: 1.745 micros, max: 2.784 micros]
99.99% = [avg: 1.746 micros, max: 5.232 micros]
99.999% = [avg: 1.747 micros, max: 6.531 micros]

TCP Latencies

Messages: 1,000,000 (size 256 bytes)
Avg Time: 2.15 micros (one-way)
Min Time: 1.976 micros
Max Time: 64.432 micros
Garbage created: ZERO
75% = [avg: 2.12 micros, max: 2.17 micros]
90% = [avg: 2.131 micros, max: 2.204 micros]
99% = [avg: 2.142 micros, max: 2.679 micros]
99.9% = [avg: 2.147 micros, max: 3.022 micros]
99.99% = [avg: 2.149 micros, max: 5.604 micros]
99.999% = [avg: 2.149 micros, max: 7.072 micros]

Garbage Production and Garbage Collector Overhead

No matter if you send one, one million or one billion messages, no garbage is produced by CoralReactor and the Garbage Collector never kicks in.