Java Development without GC

All products developed by Coral Blocks have the very important feature of leaving ZERO garbage behind. Because the latency imposed by the Java Garbage Collector (i.e. GC) is unacceptable for high-performance systems, the best option for real-time systems in Java is to not produce any garbage at all so that the GC never kicks in. Imagine a high-performance matching engine operating in the microsecond level, sending and receiving several thousand messages per second. If at any given time the GC decides to kick in with its 1+ millisecond latencies, the disruption in the system will be unacceptable. Therefore, if you want to develop real-time systems in Java with minimal variance and latency, the best option is to do it right without creating any garbage for the GC. In this article we will discuss best practices and how you can use Coral Blocks’ MemorySampler utility class to help you accomplish this critical goal.

The MemorySampler utility class

Garbage or dereferenced Java instances can show up in three different places: third-party libraries, JDK classes and your own code. Before discussing each one of them, let’s take a break to introduce Coral Blocks’ MemorySampler utility class which will allow you not just to tell your boss you are producing no trash but to prove it. (Note: MemorySampler is part of the CoralBits component)

		MemorySampler.start();
		// do nothing!
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 0
		 Memory allocated total: 0
		 */
		
		MemorySampler.start();
		// allocate nothing...
		int x = 10;
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 0
		 Memory allocated total: 0
		 */
		
		MemorySampler.start();
		// allocate an object...
		String s = new String("trash");
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 24
		 Memory allocated total: 24

		 Stack Trace for java.lang.String
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:20)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:33)
		 */
		
		MemorySampler.start();
		// allocate 10 objects...
		for(int i = 0; i < 10; i++) new String("trash!");
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 240
	 	 Memory allocated total: 264 // <===== 24 came from the previous pass, not from this pass

		 Stack Trace for java.lang.String
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:26)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:33)
		 */

		Map<String, String> map = new HashMap<String, String>();
		String key = "key";
		String value = "value";
		
		MemorySampler.start();
		map.put(key, value);
		MemorySampler.end();
		MemorySampler.printSituation();
		
		/* == OUTPUT:
		 Memory allocated on last pass: 112
		 Memory allocated total: 376

		 Stack Trace for [Ljava.util.HashMap$Entry;
    		java.util.HashMap.inflateTable(HashMap.java:320)
    		java.util.HashMap.put(HashMap.java:492)
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:69)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:76)

		 Stack Trace for java.util.HashMap$Entry
    		java.util.HashMap.createEntry(HashMap.java:901)
    		java.util.HashMap.addEntry(HashMap.java:888)
    		java.util.HashMap.put(HashMap.java:509)
    		com.coralblocks.coralutils.gcutils.Basics.test1(Basics.java:69)
    		com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:76)
		 */

As you can see by the output above, MemorySampler can tell you some important things:

  • The amount of memory allocated on the last pass
  • The total memory allocated so far
  • Who allocated the memory in the last pass with the source code line number
  • The stack trace leading to the allocation call

Of course the fact that the code is allocating memory does not necessarily mean it is creating garbage as references can be pooled for re-use. For example the code below:

		Map<String, String> map = new HashMap<String, String>();
		
		for(int i = 0; i < 100; i++) {

			MemorySampler.start();
			
			map.put("key", "value");
			map.remove("key");
			
			MemorySampler.end();
			if (MemorySampler.wasMemoryAllocated(true)) { // true => ignore the first pass (init)
				MemorySampler.printSituation();
			}
		}

Prints the output below 99 times:

         Stack Trace for java.util.HashMap$Entry
             java.util.HashMap.createEntry(HashMap.java:901)
             java.util.HashMap.addEntry(HashMap.java:888)
             java.util.HashMap.put(HashMap.java:509)
             com.coralblocks.coralutils.gcutils.Basics.test2(Basics.java:104)
             com.coralblocks.coralutils.gcutils.Basics.main(Basics.java:116)

The total memory allocated is incrementing linearly with iterations, in other words, the memory allocated per pass is always 32 and the total memory allocated increases from 144 to 3280:

Memory allocated on last pass: 32
Memory allocated total: 144

(...)

Memory allocated on last pass: 32
Memory allocated total: 3184

At this point it is clear that java.util.HashMap.createEntry is not pooling its objects, it is creating garbage and it is just a matter of enough iterations before it triggers the GC. We will soon see how to fix that.

Warming up, Checking the GC and Sampling

The key to make sure your system is not creating any garbage is to warm up your critical path from start to finish a couple of million times and then check for memory allocation another couple of million times. If it is allocating memory linearly as the number of iterations increases, it is most likely creating garbage and you should use the stack trace provided by MemorySampler to investigate it further. It might sound more complicated than it really is as it will often be straightforward to verify that as you iterate doing the same thing, the same object is allocated over and over again indicating a garbage leak, as it was the case with the java.util.HashMap.

Real-time applications usually have a critical loop that is executed non-stop by a high-priority thread. You can plug the MemorySampler in that loop to embody your whole critical path. For example, CoralReactor, which is our high-performance asynchronous and non-blocking network I/O library, has the following code at the top of its critical selector (i.e. reactor) thread:

			while (isRunning) {
				
				if (traceAllocation) {
					
    				MemorySampler.end();
    				
    				if (MemorySampler.wasMemoryAllocated()) {

    					MemorySampler.printSituation();
    				}
    				
    				MemorySampler.start();
				}

				// here your critical path starts...

				// (...)
			}

When using CoralReactor or any other application with a MemorySampler configured in the critical path as above, the standard procedure to detect garbage creation or Java garbage leaks is:

  • Turn off the MemorySampler and run your application with -verbose:gc.
  • Send a couple of million messages or exercise your code the same way a couple of million times and check if the GC kicks in.
  • If it doesn’t you most likely do not have a garbage leak.
  • If it does then you should turn on the MemorySampler and investigate to see who is the culprit.

A good system will have the MemorySampler on and will not allocate any more objects after warming up. That’s because after warming up, all instances will start to be served from object pools. It won’t matter if you execute the critical path 1 million, 1 billion or 1 trillion times. No garbage will ever be released to the GC to cause GC overhead.

Getting rid of the trash

Once you have determined that your system has a garbage leak, you have to fix it. As we said in the beginning, there are three scenarios where you can find a garbage leak:

  • In a third-party library

If you are using a third-party library that is producing a lot of garbage you should consider writing your own, contacting the author or the company or fixing the code yourself if it is an open-source project. If that cannot be done, then you should start looking for a better alternative or another way to perform the same task in a more efficient way. Some libraries claim they are real-time libraries that produce no garbage. You should favor these ones and of course test them to see if they are really leaving no trash behind.

  • In the JDK classes

You can bootstrap and patch important JDK classes at runtime. At CoralBlocks we do that for some critical JDK network classes. Contact us for more information on how to do that.

  • In your own code

When coding your own lines of code you must have the discipline together with the right libraries and tools in order to not to leave trash for the GC to clean. For example, you should know that autoboxing produces garbage as well as varargs. Some JDK data structures like java.util.HashMap produce garbage as we saw in an earlier example above and are not efficient to store primitive keys. Therefore you must have your own set of high-performance, clean and fast data-structures if you want to do real-time Java development efficiently. You can use CoralBlocks components (blocks), which are all garbage-free and optimized for speed. CoralBits provides most of the data-structures and utility classes that you need as a foundation to build your own garbage-free code. One of the most important tools to have in hand is a good, clean and fast object pool. By using mutable objects and pooling them for re-use you can eliminate most of the garbage leaks in a Java application. CoralBits provides various object pool implementations that you can use out of the box.

Conclusion

Java Development without GC overhead is very possible and you can use a memory sampler to make sure your application is not leaving any garbage behind. By using real-time libraries you can build a high-performance system from the ground up with minimal GC variance and latency. Coral Blocks provides you with all the libraries and tools you need to build garbage-free systems, to a point that you can even turn off the GC, by using the Epsilon No-Op GC (Java11+).