lixen/log

SHA256

Files

LixenWraith bfed212f40 e1.2.0 Heartbeat feature added.

2025-04-23 11:52:57 -04:00

14 KiB

Raw Blame History

Log

A high-performance, buffered, rotating file logger for Go applications, configured via the LixenWraith/config package or simple overrides. Designed for production-grade reliability with features like disk management, log retention, and lock-free asynchronous processing using atomic operations and channels.

Note: This logger requires creating an instance using NewLogger() and calling methods on that instance (e.g., l.Info(...)). It does not use package-level logging functions.

Features

Instance-Based API: Create logger instances via NewLogger() and use methods like l.Info(), l.Warn(), etc.
Lock-free Asynchronous Logging: Non-blocking log operations with minimal application impact. Logs are sent via a buffered channel, processed by a dedicated background goroutine. Uses atomic operations for state management, avoiding mutexes in the hot path.
External Configuration: Fully configured using github.com/LixenWraith/config, supporting both TOML files and CLI overrides with centralized management. Also supports simple initialization with defaults and string overrides via InitWithDefaults.
Automatic File Rotation: Seamlessly rotates log files when they reach configurable size limits (max_size_mb), generating timestamped filenames.
Comprehensive Disk Management:
- Monitors total log directory size against configured limits (max_total_size_mb)
- Enforces minimum free disk space requirements (min_disk_free_mb)
- Automatically prunes oldest log files to maintain space constraints
- Implements recovery behavior when disk space is exhausted
Adaptive Resource Monitoring: Dynamically adjusts disk check frequency based on logging volume ( enable_adaptive_interval, min_check_interval_ms, max_check_interval_ms), optimizing performance under varying loads.
Operational Heartbeats: Multi-level periodic statistics messages (process, disk, system) that bypass level filtering to ensure operational monitoring even with higher log levels.
Reliable Buffer Management: Periodic buffer flushing with configurable intervals (flush_interval_ms). Detects and reports dropped logs during high-volume scenarios.
Automated Log Retention: Time-based log file cleanup with configurable retention periods (retention_period_hrs, retention_check_mins).
Structured Logging: Support for both human-readable text (txt) and machine-parseable (json) output formats with consistent field handling.
Comprehensive Log Levels: Standard severity levels (Debug, Info, Warn, Error) with numeric values compatible with other logging systems.
Function Call Tracing: Optional function call stack traces with configurable depth (trace_depth) for debugging complex execution flows.
Clean API Design: Straightforward logging methods on the logger instance that don't require context.Context parameters.
Graceful Shutdown: Managed termination with best-effort flushing to minimize log data loss during application shutdown.

Installation

go get github.com/LixenWraith/log
# If using full TOML/CLI config:
# go get github.com/LixenWraith/config

Basic Usage

This example shows minimal initialization using defaults with a single override, logging one message, and shutting down.

package main

import (
    "github.com/LixenWraith/log"
)

func main() {
    logger := log.NewLogger()
    _ = logger.InitWithDefaults("directory=/var/log/myapp")
    logger.Info("Application starting", "pid", 12345)
    _ = logger.Shutdown()
}

Configuration

The log package is configured via keys registered with the config.Config instance passed to (l *Logger) Init, or via string overrides passed to (l *Logger) InitWithDefaults. Init expects these keys relative to the basePath argument.

Key (`basePath` + Key)	Type	Description	Default Value
`level`	`int64`	Minimum log level (-4=Debug, 0=Info, 4=Warn, 8=Error)	`0` (Info)
`name`	`string`	Base name for log files	`"log"`
`directory`	`string`	Directory to store log files	`"./logs"`
`format`	`string`	Log file format (`"txt"`, `"json"`)	`"txt"`
`extension`	`string`	Log file extension (without dot)	`"log"`
`show_timestamp`	`bool`	Show timestamp in log entries	`true`
`show_level`	`bool`	Show log level in entries	`true`
`buffer_size`	`int64`	Channel buffer capacity for log records	`1024`
`max_size_mb`	`int64`	Max size (MB) per log file before rotation	`10`
`max_total_size_mb`	`int64`	Max total size (MB) of log directory (0=unlimited)	`50`
`min_disk_free_mb`	`int64`	Min required free disk space (MB) (0=unlimited)	`100`
`flush_interval_ms`	`int64`	Interval (ms) to force flush buffer to disk via timer	`100`
`trace_depth`	`int64`	Function call trace depth (0=disabled, 1-10)	`0`
`retention_period_hrs`	`float64`	Hours to keep log files (0=disabled)	`0.0`
`retention_check_mins`	`float64`	Minutes between retention checks via timer (if enabled)	`60.0`
`disk_check_interval_ms`	`int64`	Base interval (ms) for periodic disk space checks via timer	`5000`
`enable_adaptive_interval`	`bool`	Adjust disk check interval based on load (within min/max bounds)	`true`
`enable_periodic_sync`	`bool`	Periodic sync with disk based on flush interval	`true`
`min_check_interval_ms`	`int64`	Minimum interval (ms) for adaptive disk checks	`100`
`max_check_interval_ms`	`int64`	Maximum interval (ms) for adaptive disk checks	`60000`
`heartbeat_level`	`int64`	Heartbeat detail level (0=disabled, 1=proc, 2=proc+disk, 3=proc+disk+sys)	`0`
`heartbeat_interval_s`	`int64`	Interval (s) between heartbeat messages	`60`

Example TOML Configuration (Used with (l *Logger) Init and the config package)

Assuming basePath = "logging" passed to Init

[logging]
level = -4 # Debug
directory = "/var/log/my_service"
format = "json"
max_size_mb = 100
retention_period_hrs = 168.0 # 7 days
heartbeat_level = 2 # Process + disk statistics

API Reference

Note: All logging and control functions are methods on a *Logger instance obtained via NewLogger().

Creation

NewLogger() *Logger Creates a new, uninitialized logger instance with default configuration parameters registered internally.

Initialization

(l *Logger) Init(cfg *config.Config, basePath string) error Initializes the logger instance l using settings from the provided config.Config instance under basePath. Starts the background processing goroutine.
(l *Logger) InitWithDefaults(overrides ...string) error Initializes the logger instance l using built-in defaults, applying optional overrides provided as "key=value" strings (e.g., "directory=/tmp/logs"). Starts the background processing goroutine.

Logging Functions

These methods accept ...any arguments, typically used as key-value pairs for structured logging. They are called on an initialized *Logger instance (e.g., l.Info(...)).

(l *Logger) Debug(args ...any): Logs at Debug level (-4).
(l *Logger) Info(args ...any): Logs at Info level (0).
(l *Logger) Warn(args ...any): Logs at Warn level (4).
(l *Logger) Error(args ...any): Logs at Error level (8).

Trace Logging Functions

Temporarily enable function call tracing for a single log entry on an initialized *Logger instance.

(l *Logger) DebugTrace(depth int, args ...any): Logs Debug with trace.
(l *Logger) InfoTrace(depth int, args ...any): Logs Info with trace.
(l *Logger) WarnTrace(depth int, args ...any): Logs Warn with trace.
(l *Logger) ErrorTrace(depth int, args ...any): Logs Error with trace. (depth specifies the number of stack frames, 0-10).

Other Logging Variants

Called on an initialized *Logger instance.

(l *Logger) Log(args ...any): Logs with timestamp only, no level (uses Info internally).
(l *Logger) Message(args ...any): Logs raw message without timestamp or level.
(l *Logger) LogTrace(depth int, args ...any): Logs with timestamp and trace, no level.

Shutdown and Control