JVM Garbage Collectors: Types, Advantages, Implementation Details, and How to Tune JVM Memory for Large-scale Services

JVM Garbage Collectors

Garbage collection is the process of automatically freeing up memory that is no longer in use by an application. In Java, the garbage collector runs in the background and periodically checks for objects that are no longer in use. Once an object is identified as garbage, the garbage collector reclaims the memory used by that object.

In other words, garbage collection frees the programmer from manually allocating and deallocating memory for objects created by the application. This has several benefits, such as:

• No manual memory allocation/deallocation handling, which reduces the risk of memory leaks, dangling pointers, double frees, and other memory-related errors
• Automatic memory leak management, which prevents the application from running out of memory or crashing due to excessive memory consumption. JVM Garbage Collectors won’t cover all memory leak issues therefore developers still need to watch out for such cases
• Improved code readability and maintainability, as the programmer does not have to worry about low-level memory details

However, Garbage Collection also has some drawbacks, such as:

• Performance overhead, as the JVM has to keep track of object creation and deletion, and periodically run garbage collection cycles to reclaim unused memory
• Unpredictable pauses, as some garbage collectors may stop the application threads while performing garbage collection, which can affect the responsiveness and latency of the application
• Memory fragmentation, as some garbage collectors may leave gaps between live objects after reclaiming unused memory, which can reduce the effective heap space available for allocation
• Tuning complexity, as different garbage collectors have different parameters and options that affect their behavior and performance

Basic Steps of Garbage Collection

There are three basic steps in garbage collection:

• Mark: The garbage collector scans the heap memory segment and marks all the live objects — that is, objects to which the application holds references. All the objects that have no references to them are eligible for removal
• Sweep: The garbage collector recycles all the unreferenced objects from the heap
• Compact: The sweep step tends to leave many empty regions in heap memory, causing memory fragmentation. The compact step moves the live objects closer together to eliminate these gaps and improve heap utilization

These steps may vary depending on the type of garbage collector used by the JVM.

GC Responsibilities

GC manages memory pools, primarily the Heap. While sources vary on the location of PermGen/Metaspace (classloaders), it is partially managed by GC.

• Memory allocation: It’s in the Heap and is managed by the GC. When your code requests memory for an object, the GC selects the best location, allocates it, and returns a reference. The GC is familiar with the generational Heap layout (YoungGen / OldGen / Eden / Survivors) and has full control over memory management
• Memory Cleanup: The JVM runs efficiently with limited resources. Every member must perform exceptionally to deliver the cargo undamaged. The Garbage Collector manages memory and ensures its availability while minimizing resource consumption and impact on performance. The goal is to maintain available memory, protect used memory, and avoid slowing down the system

You can instruct the Garbage Collector on how to manage memory in your application using the following flags:

• -XX:MinHeapFreeRatio=10 allows the GC to commit more memory if only 10% is left unused
• -XX:MaxHeapFreeRatio=60 allows the GC to release memory if 60% is unused

Note that setting these flags doesn’t guarantee that the GC will follow them.

Some GCs ignore flags to shrink the Heap because of how applications allocate memory in RAM. RAM is managed by the OS and applications must issue a syscall to request memory. Syscalls are time-consuming and synchronous, meaning the application must wait for a response. Each memory allocation and release requires at least one syscall. To avoid this overhead, GCs allocate memory once and rarely release it back to the OS. Instead, they flag collected objects as free for reuse. This ensures no unnecessary overhead and available memory for the JVM. This behavior can cause confusion as it may appear that the JVM has a memory leak or is inefficient. However, it is an optimization technique.

GC Tuning

There are 7 GC implementations: SerialGC, ParallelGC, CMS, G1, EpsilonGC, ShenandoahGC, and ZGC. Each uses different algorithms to maintain JVM’s memory. Tuning GCs is complex as each has its own settings. Generational GCs have MinorGC and MajorGC collections. MajorGCs can slow down or stop the JVM.

There are two modes: Throughput mode for better responsiveness and Short-pause times mode for shorter GC pauses. Different GCs can be chosen for YoungGen and OldGen. Larger YoungGen causes fewer MinorGCs but may result in more MajorGCs. Too small survivors may cause large objects to be promoted directly to OldGen.

Challanges

Finding unused objects

An unused object is one that is no longer referred to by another object. The starting node of the graph is called a GC Root. There are several GC Roots, including classes loaded by the system class loader, live threads, local variables and parameters of currently executing methods, and objects used for synchronization. During the marking phase of collection, GC traverses all references starting with each GC root to find and mark all objects still in use. In the sweeping phase, GC removes unmarked objects. Since the graph is constantly changing, marking is often a Stop-The-World phase where only GC threads run.

Fragmentation

Fragmentation occurs when the memory becomes sparse and the JVM must track free regions and decide where to allocate new objects. This can result in a premature OutOfMemoryError when there is no space for large contiguous blocks despite having enough free space. To prevent this, GCs can compact memory blocks into a single contiguous block using different algorithms. Fragmentation can also occur due to Local Allocation Buffers (PLABs and TLABs) in YoungGen (Eden) and OldGen (and YoungGen Survivors). TLABs divide Eden into chunks of various sizes for each thread to reserve, while PLABs avoid locking in OldGen and Survivors by preallocating memory regions for each thread. However, this can result in unused regions and fragmentation.

Generational Collection

Generational garbage collectors prioritize cleaning up memory regions with short-lived objects. By identifying and removing these objects quickly, they can free up over 90% of unneeded memory blocks.

Minor collection

New objects are allocated in the Eden, part of the YoungGen. When Eden fills up, a minor GC is invoked. During this process, the GC collects the Survivor region and cleans it from dead objects. Objects that have survived N minor collections are moved to the OldGen. Remaining objects are moved to the second Survivor region and the regions are swapped. The Eden region is then collected and remaining objects are moved to the active Survivor region or overflow to the OldGen. MinorGC is effective as most garbage is collected. Objects that survive more collections than set with MaxTenuringThreshold are promoted to OldGen during MinorGC.

Typically, MinorGC is mostly StopTheWorld collection but pauses are short and mostly irrelevant.

Major collection

OldGen is a large part of the Heap collected by GC through Major collections. It takes time to fill up as most garbage is collected with MinorGC. However, factors such as Minor Collection moving objects to OldGen, large objects being allocated directly in OldGen, live objects not collected during Eden collection overflowing to OldGen, and garbage objects in Eden with overridden finalize() methods not being released during MinorGC can fill it up.

OldGen collections are typically StopTheWorld collections and can have lengthy pauses due to the amount of data.

Full collection

Although not an official term, “Full Collection” is widely used and feared in the industry. It combines Minor and Major Collections, either concurrently or sequentially. As it collects both YoungGen and OldGen, it’s the longest collection. It’s usually triggered when Minor Collection can’t promote survivors to OldGen due to lack of space, so Major Collection is triggered to free up space.

GC Types

The JVM provides several types of garbage collectors that implement different algorithms and strategies for managing heap memory:

• Serial Garbage Collector: This is the simplest GC implementation that uses a single thread for both young and old generations. It stops all application threads while performing GC (stop-the-world), which can cause long pauses. It is suitable for single-threaded applications or applications with small heaps that do not have strict latency requirements
• Parallel Garbage Collector: This is also known as Throughput Collector or Scavenge Collector. It uses multiple threads for both young and old generations. It also stops all application threads while performing GC (stop-the-world), but it can achieve higher throughput than Serial GC by utilizing multiple CPU cores. It is suitable for multi-threaded applications or applications with large heaps that prioritize throughput over latency
• Concurrent Mark Sweep (CMS) Garbage Collector: This is also known as Low Latency Collector or Mostly Concurrent Collector. It uses multiple threads for both young and old generations. It performs most of its work concurrently with application threads (concurrent), but it still stops all application threads briefly at the beginning and end of each GC cycle (stop-the-world). It can achieve lower pause times than Parallel GC, but it may cause higher CPU overhead and memory fragmentation. It is suitable for applications that require low latency and can tolerate some performance degradation
• Garbage First Garbage Collector (G1 GC): This is a newer GC implementation that aims to provide high throughput and low latency. It divides the heap into multiple regions of equal size and collects the regions with the most garbage first (garbage first). It uses multiple threads for both young and old generations. It performs most of its work concurrently with application threads (concurrent), but it still stops all application threads briefly at the beginning and end of each GC cycle (stop-the-world). It can achieve shorter and more predictable pause times than CMS GC, but it may require more memory and tuning. It is suitable for applications that require both high throughput and low latency and have large heaps
• Z Garbage Collector (ZGC): This is an experimental GC implementation that aims to provide ultra-low latency. It uses a technique called load-barrier to track object references and perform concurrent relocation of objects without stopping application threads (concurrent). It can achieve pause times of less than 10 milliseconds, even for heaps of hundreds of gigabytes. However, it is still in development and may not be compatible with all JVM features and platforms. It is suitable for applications that require extremely low latency and have very large heaps

GC Types Comparison

Comparison table for Garbage Collector Types

Scale up with GC

There are several key takeaways when working with such systems at scale:

• Tune HDFS NameNode memory and garbage collection
• Use a verbose GC log to find out the maximum memory footprint
• Use a profiler to identify memory leaks
• Use the right garbage collector for your application
• Adjust the heap size based on your memory needs
• Monitor the JVM performance regularly
• G1 GC is a low-pause, server-style collector that is available in Java 7 and later. It is designed to provide predictable garbage collection times while avoiding long pause times

Possible Problems and Solutions

Tuning JVM memory can be challenging, and there are several potential problems that can arise:

• Memory leaks: Use a profiler to identify memory leaks and fix them
• Out of memory errors: Adjust the heap size based on your memory needs
• Long garbage collection pauses: Use the right garbage collector for your application
• High CPU usage: Monitor the JVM performance regularly and adjust the GC parameters as needed

Conclusion

JVM garbage collectors play a critical role in managing memory allocation and deallocation in Java applications. By choosing the right garbage collector and tuning the JVM memory settings, developers can optimize the performance of their applications and ensure reliable and efficient operation.