Java REST API Logging: Best Practices and Guidelines

REST API Logging

Logging is essential for understanding how an application works, both in the present and the past. Analyzing logs is the fastest way to detect what went wrong, making logging is critical to ensuring the performance and health of your app, as well as minimizing and reducing any downtime. By following good practices, you will get more value out of your logs and make it easier to use them. You will be able to more easily pinpoint the root cause of errors and poor performance and solve problems before they impact end-users.

Best Practices

Use a Standard Logging Library

Logging in Java can be done in a few different ways. You can use a dedicated logging library, a common API, or even just write logs to file or directly to a dedicated logging service. However, it is recommended to use a standard logging library, such as Log4j or Logback, as it provides a consistent and well-documented API for logging. This makes it easier to maintain and troubleshoot your logging code.

Select Your Appenders Wisely

Appenders are responsible for writing log messages to various destinations, such as files, databases, or remote logging services. It is important to select your appenders wisely, based on your specific use case. For example, if you need to store logs for a long time, you might want to use a database appender. If you need to analyze logs in real-time, you might want to use a remote logging service.

Use Meaningful Messages

When logging messages, it is important to use meaningful messages that provide context and information about what is happening in the application. This makes it easier to understand what is happening when analyzing logs. For example, instead of logging “Error occurred,” log “Failed to connect to database.”

Formatting and structuring log messages

The format and structure of log messages can have a significant impact on the readability and usability of log data. You should follow some best practices and guidelines when formatting and structuring log messages, such as:

• Use a consistent and clear pattern for each log message that includes relevant information (such as timestamp, level, logger name, thread name, message text, exception stack trace, etc
• Use a standard date and time format (ISO 8601) that is easy to parse and compare
• Use a common delimiter (such as space or semicolon) to separate different fields in a log message
• Use a structured format (JSON) to represent complex or nested data in a log message
• Use parameterized messages (such as “{}”) instead of string concatenation (such as “+”) to improve performance and readability
• Use markers (such as Audit or Performance) to tag log messages with specific attributes or categories
• Use MDC (Mapped Diagnostic Context) to add contextual information (such as request ID, user ID, session ID, etc.) to log messages

Implementing logging filters and interceptors

Logging filters and interceptors are components that can intercept and modify the requests and responses of a REST API. You can use logging filters and interceptors to implement various logging functionalities, such as:

• Logging the request and response details (such as method, URL, headers, body, status, etc.)
• Logging the execution time and performance metrics of a REST API
• Logging the exceptions and errors that occur during the processing of a REST API
• Logging the authentication and authorization information of a REST API
• Logging the business logic and validation rules of a REST API

There are different ways to implement logging filters and interceptors in Java, depending on the framework or library you are using. For example, if you are using Spring Boot, you can use the following options:

• HandlerInterceptor: An interface that allows you to intercept the incoming and outgoing HTTP messages at the level of the DispatcherServlet

@Component
public class ExecutionTimeInterceptor implements HandlerInterceptor {
    private long startTime;

    // This method is called before the handler method is executed. It returns a boolean value indicating whether the execution chain should continue or not.
    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
        startTime = System.currentTimeMillis();
        return true;
    }

    // This method is called after the handler method is executed, but before the view is rendered. It allows us to modify the ModelAndView object before it is rendered.
    @Override
    public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
        long endTime = System.currentTimeMillis();
        long executionTime = endTime - startTime;
    }

    
    @Override This method is called after the view is rendered. It allows us to perform any cleanup tasks.
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
        // Cleanup code goes here
    }
}

• Filter: A component that allows you to intercept the incoming and outgoing HTTP messages at the level of the servlet container

@Component
public class LoggingFilter implements Filter {
    @Override
    public void init(FilterConfig filterConfig) throws ServletException {
        // Initialization code goes here
    }

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        var req = (HttpServletRequest) request;
        log.info("Logging request: " + req.getMethod() + " " + req.getRequestURI());
        chain.doFilter(request, response);
        log.info("Logging response: " + response.getContentType());
    }

    @Override
    public void destroy() {
        // Cleanup code goes here
    }
}

• AOP (Aspect-Oriented Programming): A technique that allows you to inject cross-cutting concerns (such as logging) into your code without modifying it

@Aspect
@Component
public class LoggingAspect {

    private final Logger log = LoggerFactory.getLogger(this.getClass());

    @Pointcut("within(@org.springframework.web.bind.annotation.RestController *)")
    public void restController() {}

    @Around("restController()")
    public Object logRequest(ProceedingJoinPoint joinPoint) throws Throwable {
        log.info("Request: {}", joinPoint.getSignature().getName());
        Object result = joinPoint.proceed();
        log.info("Response: {}", result.toString());
        return result;
    }
}

@SpringBootApplication
@EnableAspectJAutoProxy
public class MyApp {
    public static void main(String[] args) {
        SpringApplication.run(MyApp.class, args);
    }
}

• ControllerAdvice attribute: An annotation that allows you to handle global exceptions and errors in your REST API

@ControllerAdvice
public class RequestLoggingInterceptor {

    private static final Logger log = LoggerFactory.getLogger(this::getClass());

    @Autowired
    private HttpServletRequest request;

    @Autowired
    private HttpServletResponse response;

    @Around("@annotation(org.springframework.web.bind.annotation.RequestMapping)")
    public Object logRequest(ProceedingJoinPoint joinPoint) throws Throwable {
        log.info("Request: {} {}", request.getMethod(), request.getRequestURI());
        Object result = joinPoint.proceed();
        log.info("Response: {}", response.getStatus());
        return result;
    }
}

Masking and encrypting sensitive data

Logging sensitive data (such as passwords, credit card numbers, personal information, etc.) can pose a serious security risk for your REST API. You should avoid logging sensitive data or mask or encrypt it before logging it. You can use various techniques to mask or encrypt sensitive data, such as:

• Using regular expressions or patterns to replace sensitive data with asterisks (*) or other symbols
• Using hashing or encryption algorithms (such as MD5 or AES) to transform sensitive data into unreadable strings
• Using annotations or attributes (such as Sensitive or Loggable attributes) to mark sensitive data in your code or configuration
• Using custom converters or serializers (such as MaskingConverter or EncryptingSerializer) to mask or encrypt sensitive data before writing it to appenders

Use the Proper Log Levels

Log levels are labels you can attach to each log entry to indicate their severity. Adding adequate log levels to messages is essential if you want to be able to make sense of them. Log4j 2, besides supporting custom log levels, offers the following standard ones: TRACE, DEBUG, INFO, WARN, ERROR, and FATAL. Use them appropriately to categorize your log messages.

Log All Relevant Things

Log to always have relevant, contextual logs that don’t add overhead. Work smarter, not harder. Logging is often one of the few lifelines you have in production environments where you can’t physically attach and debug. You want to log as much relevant, contextual data as you can.

Use JSON Format

A best practice here would be to ensure all of our logged outputs are in JSON form. This makes searching through log files, categorizing data, and analyzing logs much easier.

Rotating and archiving log files

Log files can grow very large over time and consume a lot of disk space. This can affect the performance and availability of your REST API. You should rotate and archive log files periodically to manage their size and number. You can use various strategies to rotate and archive log files, such as:

• Time-based rotation: Rotating log files based on a fixed time interval (such as daily, weekly, monthly, etc.)
• Size-based rotation: Rotating log files based on a fixed size limit (such as 10 MB, 100 MB, 1 GB, etc.)
• Hybrid rotation: Rotating log files based on a combination of time and size criteria (such as daily or 10 MB, whichever comes first)
• Compression: Compressing log files after rotation to reduce their size
• Deletion: Deleting old log files after a certain period of time or when a certain number of files is reached

Create your own generic wrapper

// example only shows the logic. Adding any Logging Libraries and Tools is straight forward by replacing console output lines with Logging frameworks
interface Log {
    private <R> R execute(String message, Callable<R> fn) throws Exception {
        var stack = StackWalker.getInstance().walk(frames -> frames.skip(2).findFirst().orElse(null));
        var executedBy = stack != null
                         ? format("%s.%s", stack.getClassName(), stack.getMethodName())
                         : "NO_METHOD";

        try {
            out.printf("[%s] start %s%n", executedBy, message);
            var output = fn.call();
            out.printf("[%s] finished %s%n", executedBy, message);

            return output;
        } catch (Exception e) {
            out.printf("[%s] failed %s%n", executedBy, message);
            throw e;
        }
    }

    default void log(@NotBlank String message, @NotNull Runnable fn) throws Exception {
        execute(message, () -> {
            fn.run();
            return null;
        });
    }

    default <R> @NotNull R log(@NotBlank String message, @NotNull Supplier<R> fn) throws Exception {
        return execute(message, fn::get);
    }
}

// usage
log("parsing message", () -> System.out.println("processing"));
var output = log("parsing message", () -> "Hello World".toUpperCase());

Conclusion

Logging is a critical aspect of application development, providing visibility into how the application is working and helping detect issues before they impact end-users.