Getting Started with CoralLog for Event Sourcing

In a previous article we talked about using CoralLog for level logging. In this article we will show you how to use CoralLog for event sourcing, in other words, how to open your own files and log any message or data you want with minimum latency, maximum throughput and zero garbage. Keep in mind that once a Logger object is created, everything we discussed for level logging can be applied to event sourcing and vice-versa because the interface is exactly the same. So make sure you check our article on level logging to get the full picture of the CoralLog features and the many ways to log.

Opening a file through a logger

The basic method to create a Logger and start persisting your events and/or messages without complications is:
NOTE: The methods below create an asynchronous and non-synchronized logger by default. This is most probably what you want. The logger has a shutdown hook to drain and close the file so you don’t lose any information on exit. You don’t have to worry or do anything about that.

// pass a filename to be created in the current directory
Logger myLogger = Log.createLogger("myLogFile.log");

// or you can pass the directory
Logger myLogger = Log.createLogger("/var/mydir/", "myLogFile.log");

Using the LogConfig to create a logger

You can also pass a LogConfig to the createLogger method to be able to create a logger using its various configuration options. For example, if you want to create a logger that does not print a carriage return on the end of each line you can do:

LogConfig logConfig = new LogConfig(filename); // current directory
// or LogConfig logConfig = new LogConfig(dir, filename);

logConfig.includeLogEntrySeparator = false; // default is yes

Logger myLogger = Log.createLogger(logConfig);

Below all the LogConfig options with their defaults:

	public final String filename;
	public String dir = Log.getDir();
	public boolean isSynchronized = false;
	public boolean isMemoryMapped = false;
	public String memoryMappedBufferSize = "64m"; // 64m
	public List<Encoder> encoders = Log.getEncoders();
	public boolean isAsynchronous = true;
	public boolean includeTopHeader = false;
	public boolean includeLogEntrySeparator = true;
	public String outputBufferSize = "64k"; // 64k (optimum)
	public boolean flushImmediately = false;
	public boolean includeTimestamp = true;
	public int secondsToFlush = -1;
	public String sizeToRoll = "-1";
	public boolean isNoSpaceBetweenObjects = false;

Logging Events

Below some examples on how you can log events with CoralLog:

// log the contents of a byte[]
myLogger.log(byteArray);

// log the contents of a ByteBuffer
myLogger.log(byteBuffer);

// log the contents of a StringBuilder (or any CharSequence)
myLogger.log(stringBuilder);

NOTE: None of the methods above generate any garbage.

You can also log more than one object using varargs, which CoralLog implements also without creating any garbage. Please refer to our article on level logging for more details on the different approaches you can choose for logging. As we said in the beginning, the interface is the same so the same strategy applies for both event sourcing and level logging.

myLogger.log("This is a log message!", "user=", "foo", "age=", to_sb(21));

// or

myLogger.log("This is a log message! user={} age={}", "foo", to_sb(21));

// or

StringBuilder sb = new StringBuilder(1024); // cache and re-use this instance
 
sb.setLength(0);
sb.append("This is a log message!");
sb.append(" user=").append("foo");
sb.append(" age=").append(21);
 
myLogger.log(sb);

NOTE: The methods below are optional and are intended for the advanced user. You can safely ignore them because all our asynchronous loggers have a shutdown hook to drain the queue on exit. Also, if you can, you should pin the Async disk I/O thread to an isolated CPU core with the config:

-DlogProcToBindAsyncThread=CORE_ID

Draining and Closing an Asynchronous Logger

When dealing with asynchronous logging, some care must be taken when closing a logger. CoralLog makes it straightforward to drain and close a logger so no data is ever lost due to the system exiting before the writer thread had a chance to drain the queue and log everything to disk. Below the details:

// put a logger close event in the async queue
// this call returns immediately
myLogger.close();

// this method will block, wait for the queue to be drained,
// wait for the logger to be closed and then return
myLogger.drainCloseAndWait();

// if you don't want to close, but just want to drain, you can do:
myLogger.drainAndWait();
// this method will block, wait for the queue to be drained,
// and return

You can use the methods above when you need to coordinate producer (the critical thread doing the logging) and consumer (the writer thread doing file i/o). Such coordination is usually needed when the system is terminating and you want to make sure there are no messages left in the ring-buffer queue.

Closing the Asynchronous Thread

The asynchronous thread (the writer thread doing the actual file i/o), is not a Java daemon thread, in other words, it will prevent the JVM from exiting unless this thread is properly terminated. That’s on purpose because we do not want this thread to be killed by the JVM on exit while it is still draining the queue. Therefore, if you want to graciously exit the JVM, you can use the method below:

// block, drain the queue (all loggers), die and return...
AsyncThread.drainDieAndWait();

You can also drain all loggers and return like below:

// block, drain the queue (all loggers) and return...
AsyncThread.drainAndWait();

There is also a non-blocking version that returns immediately:

// returns immediately and signal the async thread to
// drain and die as soon as it can
AsyncThread.drainAndDie();

The cleanest way to exit the JVM is to close all your loggers and issue a AsyncThread.drainDieAndWait().

Coral Blocks

Opening a file through a logger

Using the LogConfig to create a logger

Logging Events

Draining and Closing an Asynchronous Logger

Closing the Asynchronous Thread