lixen/log
SHA256
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

e1.1.0 Refactoring default config and unused functions, global auto-initiated logger instance removed

Browse Source
This commit is contained in:
Lixen Wraith
2025-04-23 01:08:14 -04:00
parent c809396455
commit 0ddfa2c533
10 changed files with 260 additions and 428 deletions
Download Patch File Download Diff File Expand all files Collapse all files

196
README.md
View File

@ -1,26 +1,25 @@
# Log # Log
A robust, buffered, rotating file logger for Go applications, configured via the [LixenWraith/config](https://github.com/LixenWraith/config) package. Designed for performance and reliability with features like disk management, log retention, and asynchronous processing using atomic operations and channels. A high-performance, buffered, rotating file logger for Go applications, configured via the [LixenWraith/config](https://github.com/LixenWraith/config) package. Designed for production-grade reliability with features like disk management, log retention, and lock-free asynchronous processing using atomic operations and channels.
## Features ## Features
- **Buffered Asynchronous Logging:** Logs are sent non-blockingly to a buffered channel, processed by a dedicated background goroutine for minimal application impact. Uses atomic operations for state management, avoiding mutexes in the logging hot path. - **Lock-free Asynchronous Logging:** Non-blocking log operations with minimal application impact. Logs are sent via a buffered channel, processed by a dedicated background goroutine. Uses atomic operations for state management, avoiding mutexes in the hot path.
- **External Configuration:** Fully configured using `github.com/LixenWraith/config`, allowing settings via TOML files and CLI overrides managed centrally. - **External Configuration:** Fully configured using `github.com/LixenWraith/config`, supporting both TOML files and CLI overrides with centralized management.
- **Automatic File Rotation:** Rotates log files when they reach a configurable size (`max_size_mb`). - **Automatic File Rotation:** Seamlessly rotates log files when they reach configurable size limits (`max_size_mb`), generating timestamped filenames.
- **Disk Space Management:** - **Comprehensive Disk Management:**
- Monitors total log directory size against a limit (`max_total_size_mb`). - Monitors total log directory size against configured limits (`max_total_size_mb`)
- Monitors available disk space against a minimum requirement (`min_disk_free_mb`). - Enforces minimum free disk space requirements (`min_disk_free_mb`)
- Automatically attempts to delete the oldest log files (by modification time) to stay within limits during periodic checks or when writes fail. - Automatically prunes oldest log files to maintain space constraints
- Temporarily pauses logging if space cannot be freed, logging an error message. - Implements recovery behavior when disk space is exhausted
- **Adaptive Disk Check Interval:** Optionally adjusts the frequency of disk space checks based on logging load (`enable_adaptive_interval`, `disk_check_interval_ms`, `min_check_interval_ms`, `max_check_interval_ms`) to balance performance and responsiveness. - **Adaptive Resource Monitoring:** Dynamically adjusts disk check frequency based on logging volume (`enable_adaptive_interval`, `min_check_interval_ms`, `max_check_interval_ms`), optimizing performance under varying loads.
- **Periodic Flushing:** Automatically flushes the log buffer to disk at a configured interval (`flush_interval_ms`) using a timer. - **Reliable Buffer Management:** Periodic buffer flushing with configurable intervals (`flush_interval_ms`). Detects and reports dropped logs during high-volume scenarios.
- **Log Retention:** Automatically deletes log files older than a configured duration (`retention_period_hrs`), checked periodically via a timer (`retention_check_mins`). Relies on file modification time. - **Automated Log Retention:** Time-based log file cleanup with configurable retention periods (`retention_period_hrs`, `retention_check_mins`).
- **Dropped Log Detection:** If the internal buffer fills under high load, logs are dropped, and a summary message indicating the number of drops is logged later. - **Structured Logging:** Support for both human-readable text (`txt`) and machine-parseable (`json`) output formats with consistent field handling.
- **Structured Logging:** Supports both plain text (`txt`) and `json` output formats. - **Comprehensive Log Levels:** Standard severity levels (Debug, Info, Warn, Error) with numeric values compatible with other logging systems.
- **Standard Log Levels:** Provides `Debug`, `Info`, `Warn`, `Error` levels (values match `slog`). - **Function Call Tracing:** Optional function call stack traces with configurable depth (`trace_depth`) for debugging complex execution flows.
- **Function Call Tracing:** Optionally include function call traces in logs with configurable depth (`trace_depth`) or enable temporarily via `*Trace` functions. - **Clean API Design:** Straightforward logging methods that don't require `context.Context` parameters.
- **Simplified API:** Public logging functions (`log.Info`, `log.Debug`, etc.) do not require `context.Context`. - **Graceful Shutdown:** Managed termination with best-effort flushing to minimize log data loss during application shutdown.
- **Graceful Shutdown:** `log.Shutdown` signals the background processor to stop by closing the log channel. It then waits for a *brief, fixed duration* (best-effort) before closing the file handle. Note: This is a best-effort flush; logs might be lost if flushing takes longer than the internal wait or if the application exits abruptly.
## Installation ## Installation
@ -29,8 +28,6 @@ go get github.com/LixenWraith/log
go get github.com/LixenWraith/config go get github.com/LixenWraith/config
``` ```
The `config` package has its own dependencies which will be fetched automatically.
## Basic Usage ## Basic Usage
```go ```go
@ -58,7 +55,7 @@ const logConfigPath = "logging" // Base path for logger settings in TOML/config
extension = "log" extension = "log"
max_size_mb = 50 max_size_mb = 50
flush_interval_ms = 100 flush_interval_ms = 100
disk_check_interval_ms = 5000 # Example: Check disk every 5s disk_check_interval_ms = 5000 # Check disk every 5s
enable_adaptive_interval = true enable_adaptive_interval = true
# Other settings will use defaults registered by log.Init # Other settings will use defaults registered by log.Init
*/ */
@ -123,37 +120,36 @@ func main() {
} }
fmt.Println("Shutdown complete.") fmt.Println("Shutdown complete.")
} }
``` ```
## Configuration ## Configuration
The `log` package is configured via keys registered with the `config.Config` instance passed to `log.Init`. `log.Init` expects these keys relative to the `basePath` argument. The `log` package is configured via keys registered with the `config.Config` instance passed to `log.Init`. `log.Init` expects these keys relative to the `basePath` argument.
| Key (`basePath` + Key) | Type | Description | Default Value (Registered by `log.Init`) | | Key (`basePath` + Key) | Type | Description | Default Value |
|:---------------------------| :-------- |:-----------------------------------------------------------------|:-----------------------------------------| |:---------------------------| :-------- |:-----------------------------------------------------------------|:--------------|
| `level` | `int64` | Minimum log level (-4=Debug, 0=Info, 4=Warn, 8=Error) | `0` (LevelInfo) | | `level` | `int64` | Minimum log level (-4=Debug, 0=Info, 4=Warn, 8=Error) | `0` (Info) |
| `name` | `string` | Base name for log files | `"log"` | | `name` | `string` | Base name for log files | `"log"` |
| `directory` | `string` | Directory to store log files | `"./logs"` | | `directory` | `string` | Directory to store log files | `"./logs"` |
| `format` | `string` | Log file format (`"txt"`, `"json"`) | `"txt"` | | `format` | `string` | Log file format (`"txt"`, `"json"`) | `"txt"` |
| `extension` | `string` | Log file extension (e.g., `"log"`, `"app"`) | `"log"` | | `extension` | `string` | Log file extension (without dot) | `"log"` |
| `show_timestamp` | `bool` | Show timestamp in log entries | `true` | | `show_timestamp` | `bool` | Show timestamp in log entries | `true` |
| `show_level` | `bool` | Show log level in entries | `true` | | `show_level` | `bool` | Show log level in entries | `true` |
| `buffer_size` | `int64` | Channel buffer capacity for log records | `1024` | | `buffer_size` | `int64` | Channel buffer capacity for log records | `1024` |
| `max_size_mb` | `int64` | Max size (MB) per log file before rotation | `10` | | `max_size_mb` | `int64` | Max size (MB) per log file before rotation | `10` |
| `max_total_size_mb` | `int64` | Max total size (MB) of log directory (0=unlimited) | `50` | | `max_total_size_mb` | `int64` | Max total size (MB) of log directory (0=unlimited) | `50` |
| `min_disk_free_mb` | `int64` | Min required free disk space (MB) (0=unlimited) | `100` | | `min_disk_free_mb` | `int64` | Min required free disk space (MB) (0=unlimited) | `100` |
| `flush_interval_ms` | `int64` | Interval (ms) to force flush buffer to disk via timer | `100` | | `flush_interval_ms` | `int64` | Interval (ms) to force flush buffer to disk via timer | `100` |
| `trace_depth` | `int64` | Function call trace depth (0=disabled, 1-10) | `0` | | `trace_depth` | `int64` | Function call trace depth (0=disabled, 1-10) | `0` |
| `retention_period_hrs` | `float64` | Hours to keep log files (0=disabled) | `0.0` | | `retention_period_hrs` | `float64` | Hours to keep log files (0=disabled) | `0.0` |
| `retention_check_mins` | `float64` | Minutes between retention checks via timer (if enabled) | `60.0` | | `retention_check_mins` | `float64` | Minutes between retention checks via timer (if enabled) | `60.0` |
| `disk_check_interval_ms` | `int64` | Base interval (ms) for periodic disk space checks via timer | `5000` | | `disk_check_interval_ms` | `int64` | Base interval (ms) for periodic disk space checks via timer | `5000` |
| `enable_adaptive_interval` | `bool` | Adjust disk check interval based on load (within min/max bounds) | `true` | | `enable_adaptive_interval` | `bool` | Adjust disk check interval based on load (within min/max bounds) | `true` |
| `enable_periodic_sync` | `bool` | Periodic sync with disk based on flush interval | `false` | | `enable_periodic_sync` | `bool` | Periodic sync with disk based on flush interval | `false` |
| `min_check_interval_ms` | `int64` | Minimum interval (ms) for adaptive disk checks | `100` | | `min_check_interval_ms` | `int64` | Minimum interval (ms) for adaptive disk checks | `100` |
| `max_check_interval_ms` | `int64` | Maximum interval (ms) for adaptive disk checks | `60000` | | `max_check_interval_ms` | `int64` | Maximum interval (ms) for adaptive disk checks | `60000` |
**Example TOML (`config.toml`)** **Example TOML Configuration (`app_config.toml`)**
```toml ```toml
# Main application settings # Main application settings
@ -180,36 +176,27 @@ app_name = "My Service"
host = "db.example.com" host = "db.example.com"
``` ```
Your application would then initialize the logger like this:
```go
cfg := config.New()
cfg.Load("config.toml", os.Args[1:]) // Load from file & CLI
log.Init(cfg, "logging") // Use "logging" as base path
cfg.Save("config.toml") // Save merged config
```
## API Reference ## API Reference
### Initialization ### Initialization
- **`Init(cfg *config.Config, basePath string) error`** - **`Init(cfg *config.Config, basePath string) error`**
Initializes or reconfigures the logger using settings from the provided `config.Config` instance under `basePath`. Registers required keys with defaults if not present. Handles reconfiguration safely, potentially restarting the background processor goroutine (e.g., if `buffer_size` changes). Must be called before logging. Thread-safe. Initializes or reconfigures the logger using settings from the provided `config.Config` instance under `basePath`. Registers required keys with defaults if not present. Thread-safe.
- **`InitWithDefaults(overrides ...string) error`** - **`InitWithDefaults(overrides ...string) error`**
Initializes or reconfigures the logger using built-in defaults, applying optional overrides provided as "key=value" strings. Useful for simple setups without a config file. Thread-safe. Initializes the logger using built-in defaults, applying optional overrides provided as "key=value" strings. Thread-safe.
### Logging Functions ### Logging Functions
These functions accept `...any` arguments, typically used as key-value pairs for structured logging (e.g., `"user_id", 123, "status", "active"`). They are non-blocking and read configuration/state using atomic operations. These methods accept `...any` arguments, typically used as key-value pairs for structured logging (e.g., `"user_id", 123, "status", "active"`). All logging functions are non-blocking and use atomic operations for state checks.
- **`Debug(args ...any)`**: Logs at Debug level. - **`Debug(args ...any)`**: Logs at Debug level (-4).
- **`Info(args ...any)`**: Logs at Info level. - **`Info(args ...any)`**: Logs at Info level (0).
- **`Warn(args ...any)`**: Logs at Warn level. - **`Warn(args ...any)`**: Logs at Warn level (4).
- **`Error(args ...any)`**: Logs at Error level. - **`Error(args ...any)`**: Logs at Error level (8).
### Trace Logging Functions ### Trace Logging Functions
Temporarily enable function call tracing for a single log entry. Temporarily enable function call tracing for a single log entry, regardless of the configured `trace_depth`.
- **`DebugTrace(depth int, args ...any)`**: Logs Debug with trace. - **`DebugTrace(depth int, args ...any)`**: Logs Debug with trace.
- **`InfoTrace(depth int, args ...any)`**: Logs Info with trace. - **`InfoTrace(depth int, args ...any)`**: Logs Info with trace.
@ -219,52 +206,69 @@ Temporarily enable function call tracing for a single log entry.
### Other Logging Variants ### Other Logging Variants
- **`Log(args ...any)`**: Logs with timestamp, no level (uses Info internally), no trace. - **`Log(args ...any)`**: Logs with timestamp only, no level (uses Info internally).
- **`Message(args ...any)`**: Logs raw message, no timestamp, no level, no trace. - **`Message(args ...any)`**: Logs raw message without timestamp or level.
- **`LogTrace(depth int, args ...any)`**: Logs with timestamp and trace, no level. - **`LogTrace(depth int, args ...any)`**: Logs with timestamp and trace, no level.
### Shutdown ### Shutdown and Control
- **`Shutdown(timeout time.Duration) error`** - **`Shutdown(timeout time.Duration) error`**
Attempts to gracefully shut down the logger. Sets atomic flags to prevent new logs, closes the internal log channel to signal the background processor, waits for a *brief fixed duration* (currently using the `flush_interval_ms` configuration value, `timeout` argument is used as a default if the interval is <= 0), and then closes the current log file. Returns `nil` on success or an error if file operations fail. Note: This provides a *best-effort* flush; logs might be lost if disk I/O is slow or the application exits too quickly after calling Shutdown. Gracefully shuts down the logger. Signals the processor to stop, waits briefly for pending logs to flush, then closes file handles. Returns error details if closing operations fail.
- **`Flush(timeout time.Duration) error`**
Explicitly triggers a sync of the current log file buffer to disk and waits for completion or timeout.
### Constants ### Constants
- **`LevelDebug`, `LevelInfo`, `LevelWarn`, `LevelError` (`int64`)**: Log level constants. - **`LevelDebug (-4)`, `LevelInfo (0)`, `LevelWarn (4)`, `LevelError (8)` (`int64`)**: Log level constants.
- **`FlagShowTimestamp`, `FlagShowLevel`, `FlagDefault`**: Record flag constants controlling output format.
## Implementation Details & Behavior ## Implementation Details
- **Asynchronous Processing:** Log calls (`log.Info`, etc.) are non-blocking. They format a `logRecord` and attempt a non-blocking send to an internal buffered channel (`ActiveLogChannel`). A single background goroutine (`processLogs`) reads from this channel, serializes the record (to TXT or JSON using a reusable buffer), and writes it to the current log file. - **Lock-Free Hot Path:** Log methods (`Info`, `Debug`, etc.) operate without locks, using atomic operations to check logger state and non-blocking channel sends. Only initialization, reconfiguration, and shutdown use a mutex.
- **Configuration Source:** Relies on an initialized `github.com/LixenWraith/config.Config` instance passed to `log.Init` or uses internal defaults with `InitWithDefaults`. It registers expected keys with "log." prefix and retrieves values using the config package's type-specific accessors (Int64, String, Bool, Float64).
- **State Management:** Uses `sync.Mutex` (`initMu`) *only* to protect initialization and reconfiguration logic. Uses `sync/atomic` variables extensively for runtime state (`IsInitialized`, `CurrentFile`, `CurrentSize`, `DroppedLogs`), allowing lock-free reads in logging functions and the processor loop.
- **Timers:** Uses `time.Ticker` internally for:
* Periodic buffer flushing (`flush_interval_ms`).
* Periodic log retention checks (`retention_check_mins`).
* Periodic and potentially adaptive disk space checks (`disk_check_interval_ms`, etc.).
- **File Rotation:** Triggered synchronously within `processLogs` when writing a record would exceed `max_size_mb`. The old file is closed, a new one is created with a timestamped name, and the atomic `CurrentFile` pointer and `CurrentSize` are updated.
- **Disk/Retention Checks:**
* `performDiskCheck` is called periodically by a timer and reactively if writes fail or a byte threshold is crossed. It checks total size and free space limits. If limits are exceeded *and* `forceCleanup` is true (for periodic checks), it calls `cleanOldLogs`. If checks fail, `DiskStatusOK` is set to false, causing subsequent logs to be dropped until the condition resolves.
* `cleanOldLogs` deletes the oldest files (by modification time, skipping the current file) until enough space is freed or no more files can be deleted.
* `cleanExpiredLogs` is called periodically by a timer based on `retention_check_mins`. It deletes files whose modification time is older than `retention_period_hrs`.
- **Shutdown Process:**
1. `Shutdown` sets atomic flags (`ShutdownCalled`, `LoggerDisabled`) to prevent new logs.
2. It closes the current `ActiveLogChannel` (obtained via atomic load).
3. It performs a *fixed short sleep* based on the configured `flush_interval_ms` as a best-effort attempt to allow the processor goroutine time to process remaining items in the channel buffer before the file is closed.
4. The `processLogs` goroutine detects the closed channel, performs a final file sync, and exits.
5. `Shutdown` performs final `Sync` and `Close` on the log file handle after the sleep.
## Limitations, Caveats & Failure Modes - **Channel-Based Architecture:** Log records flow through a buffered channel from producer methods to a single consumer goroutine, preventing contention and serializing file I/O operations.
- **Adaptive Resource Management:**
- Disk checks run periodically via timer and reactively when write volume thresholds are crossed
- Check frequency automatically adjusts based on logging rate when `enable_adaptive_interval` is enabled
- Intelligently backs off during low activity and increases responsiveness during high volume
- **File Management:**
- Log files are rotated when `max_size_mb` is exceeded, with new files named using timestamps
- Oldest files (by modification time) are automatically pruned when space limits are approached
- Files older than `retention_period_hrs` are periodically removed
- **Recovery Behavior:** When disk issues occur, the logger temporarily pauses new logs and attempts recovery on subsequent operations, logging one disk warning message to prevent error spam.
- **Graceful Shutdown Flow:**
1. Sets atomic flags to prevent new logs
2. Closes the active log channel to signal processor shutdown
3. Waits briefly for processor to finish pending records
4. Performs final sync and closes the file handle
## Performance Considerations
- **Non-blocking Design:** The logger is designed to have minimal impact on application performance, with non-blocking log operations and buffered processing.
- **Memory Efficiency:** Uses a reusable buffer for serialization, avoiding unnecessary allocations when formatting log entries.
- **Disk I/O Management:** Batches writes and intelligently schedules disk operations to minimize I/O overhead while maintaining data safety.
- **Concurrent Safety:** Thread-safe through careful use of atomic operations, minimizing mutex usage to initialization and shutdown paths only.
## Caveats & Limitations
- **Dependency:** Requires `github.com/LixenWraith/config` for configuration via `log.Init`.
- **Log Loss Scenarios:** - **Log Loss Scenarios:**
- **Buffer Full:** If the application generates logs faster than they can be written to disk, `ActiveLogChannel` fills up. Subsequent log calls will drop messages until space becomes available. A `"Logs were dropped"` message will be logged later. Increase `buffer_size` or reduce logging volume. - **Buffer Saturation:** Under extreme load, logs may be dropped if the internal buffer fills faster than records can be processed. A summary message will be logged once capacity is available again.
- **Shutdown:** The `Shutdown` function uses a brief, fixed wait, not a guarantee that all logs are flushed. Logs remaining in the buffer or OS buffers after `Shutdown` returns might be lost, especially under heavy load or slow disk I/O. Ensure critical logs are flushed before shutdown if necessary (though this logger doesn't provide an explicit flush mechanism). - **Shutdown Race:** The `Shutdown` function provides a best-effort attempt to process remaining logs, but cannot guarantee all buffered logs will be written if the application terminates quickly.
- **Application Exit:** If the application exits abruptly *before* or *during* `log.Shutdown`, buffered logs will likely be lost. - **Persistent Disk Issues:** If disk space cannot be reclaimed through cleanup, logs will be dropped until the condition is resolved.
- **Disk Full (Unrecoverable):** If `performDiskCheck` detects low space and `cleanOldLogs` *cannot* free enough space (e.g., no old files to delete, permissions issues), `DiskStatusOK` is set to false. Subsequent logs are dropped until the condition resolves. An error message is logged to stderr *once* when this state is entered.
- **Configuration Errors:** `log.Init` or `InitWithDefaults` will return an error and fail if configuration values are invalid (e.g., negative `max_size_mb`, invalid `format`, bad override string) or if the `config.Config` instance is `nil` (for `Init`). The application must handle these errors. - **Configuration Dependencies:** Requires the `github.com/LixenWraith/config` package for advanced configuration management.
- **Cleanup Race Conditions:** Under high load with frequent rotation/cleanup, benign `"failed to remove old log file ... no such file or directory"` errors might appear in stderr if multiple cleanup attempts target the same file.
- **Retention Accuracy:** Log retention is based on file **modification time**. External actions modifying old log files could interfere with accurate retention. - **Retention Accuracy:** Log retention relies on file modification times, which could be affected by external file system operations.
- **Reconfiguration:** Changing `buffer_size` restarts the background processor, involving closing the old channel and creating a new one. Logs sent during this brief transition might be dropped. Other configuration changes are applied live where possible via atomic updates.
- **Reconfiguration Impact:** Changing buffer size during runtime requires restarting the background processor, which may cause a brief period where logs could be dropped.
## License ## License

18
cmd/reconfig/main.go
View File

@ -12,8 +12,10 @@ import (
func main() { func main() {
var count atomic.Int64 var count atomic.Int64
logger := log.NewLogger()
// Initialize the logger with defaults first // Initialize the logger with defaults first
err := log.InitWithDefaults() err := logger.InitWithDefaults()
if err != nil { if err != nil {
fmt.Printf("Initial Init error: %v\n", err) fmt.Printf("Initial Init error: %v\n", err)
return return
@ -22,7 +24,7 @@ func main() {
// Log something constantly // Log something constantly
go func() { go func() {
for i := 0; ; i++ { for i := 0; ; i++ {
log.Info("Test log", i) logger.Info("Test log", i)
count.Add(1) count.Add(1)
time.Sleep(time.Millisecond) time.Sleep(time.Millisecond)
} }
@ -32,7 +34,7 @@ func main() {
for i := 0; i < 10; i++ { for i := 0; i < 10; i++ {
// Use different buffer sizes to trigger channel recreation // Use different buffer sizes to trigger channel recreation
bufSize := fmt.Sprintf("buffer_size=%d", 100*(i+1)) bufSize := fmt.Sprintf("buffer_size=%d", 100*(i+1))
err := log.InitWithDefaults(bufSize) err := logger.InitWithDefaults(bufSize)
if err != nil { if err != nil {
fmt.Printf("Init error: %v\n", err) fmt.Printf("Init error: %v\n", err)
} }
@ -42,14 +44,14 @@ func main() {
// Check if we see any inconsistency // Check if we see any inconsistency
time.Sleep(500 * time.Millisecond) time.Sleep(500 * time.Millisecond)
fmt.Printf("Total logs attempted: %d\n", count.Load()) fmt.Printf("Total logger. attempted: %d\n", count.Load())
// Gracefully shut down the logger // Gracefully shut down the logger.er
err = log.Shutdown(time.Second) err = logger.Shutdown(time.Second)
if err != nil { if err != nil {
fmt.Printf("Shutdown error: %v\n", err) fmt.Printf("Shutdown error: %v\n", err)
} }
// Check for any error messages in the log files // Check for any error messages in the logger.files
// or dropped log count // or dropped logger.count
} }

27
cmd/simple/main.go
View File

@ -58,16 +58,17 @@ func main() {
} }
// --- Initialize Logger --- // --- Initialize Logger ---
logger := log.NewLogger()
// Pass the config instance and the base path for logger settings // Pass the config instance and the base path for logger settings
err = log.Init(cfg, configBasePath) err = logger.Init(cfg, configBasePath)
if err != nil { if err != nil {
fmt.Fprintf(os.Stderr, "Failed to initialize logger: %v\n", err) fmt.Fprintf(os.Stderr, "Failed to initialize logger.er: %v\n", err)
os.Exit(1) os.Exit(1)
} }
fmt.Println("Logger initialized.") fmt.Println("Logger initialized.")
// --- SAVE CONFIGURATION --- // --- SAVE CONFIGURATION ---
// Save the config state *after* log.Init has registered its keys/defaults // Save the config state *after* logger.Init has registered its keys/defaults
// This will write the merged configuration (defaults + file overrides) back. // This will write the merged configuration (defaults + file overrides) back.
err = cfg.Save(configFile) err = cfg.Save(configFile)
if err != nil { if err != nil {
@ -78,10 +79,10 @@ func main() {
// --- End Save Configuration --- // --- End Save Configuration ---
// --- Logging --- // --- Logging ---
log.Debug("This is a debug message.", "user_id", 123) logger.Debug("This is a debug message.", "user_id", 123)
log.Info("Application starting...") logger.Info("Application starting...")
log.Warn("Potential issue detected.", "threshold", 0.95) logger.Warn("Potential issue detected.", "threshold", 0.95)
log.Error("An error occurred!", "code", 500) logger.Error("An error occurred!", "code", 500)
// Logging from goroutines // Logging from goroutines
var wg sync.WaitGroup var wg sync.WaitGroup
@ -89,21 +90,21 @@ func main() {
wg.Add(1) wg.Add(1)
go func(id int) { go func(id int) {
defer wg.Done() defer wg.Done()
log.Info("Goroutine started", "id", id) logger.Info("Goroutine started", "id", id)
time.Sleep(time.Duration(50+id*50) * time.Millisecond) time.Sleep(time.Duration(50+id*50) * time.Millisecond)
log.InfoTrace(1, "Goroutine finished", "id", id) // Log with trace logger.InfoTrace(1, "Goroutine finished", "id", id) // Log with trace
}(i) }(i)
} }
// Wait for goroutines to finish before shutting down logger // Wait for goroutines to finish before shutting down logger.er
wg.Wait() wg.Wait()
fmt.Println("Goroutines finished.") fmt.Println("Goroutines finished.")
// --- Shutdown Logger --- // --- Shutdown Logger ---
fmt.Println("Shutting down logger...") fmt.Println("Shutting down logger.er...")
// Provide a reasonable timeout for logs to flush // Provide a reasonable timeout for logger. to flush
shutdownTimeout := 2 * time.Second shutdownTimeout := 2 * time.Second
err = log.Shutdown(shutdownTimeout) err = logger.Shutdown(shutdownTimeout)
if err != nil { if err != nil {
fmt.Fprintf(os.Stderr, "Logger shutdown error: %v\n", err) fmt.Fprintf(os.Stderr, "Logger shutdown error: %v\n", err)
} else { } else {

15
cmd/stress/main.go
View File

@ -53,6 +53,8 @@ var levels = []int64{
log.LevelError, log.LevelError,
} }
var logger *log.Logger
func generateRandomMessage(size int) string { func generateRandomMessage(size int) string {
const chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 " const chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 "
var sb strings.Builder var sb strings.Builder
@ -78,13 +80,13 @@ func logBurst(burstID int) {
} }
switch level { switch level {
case log.LevelDebug: case log.LevelDebug:
log.Debug(args...) logger.Debug(args...)
case log.LevelInfo: case log.LevelInfo:
log.Info(args...) logger.Info(args...)
case log.LevelWarn: case log.LevelWarn:
log.Warn(args...) logger.Warn(args...)
case log.LevelError: case log.LevelError:
log.Error(args...) logger.Error(args...)
} }
} }
} }
@ -126,7 +128,8 @@ func main() {
} }
// --- Initialize Logger --- // --- Initialize Logger ---
err = log.Init(cfg, configBasePath) logger = log.NewLogger()
err = logger.Init(cfg, configBasePath)
if err != nil { if err != nil {
fmt.Fprintf(os.Stderr, "Failed to initialize logger: %v\n", err) fmt.Fprintf(os.Stderr, "Failed to initialize logger: %v\n", err)
os.Exit(1) os.Exit(1)
@ -195,7 +198,7 @@ endLoop:
// --- Shutdown Logger --- // --- Shutdown Logger ---
fmt.Println("Shutting down logger (allowing up to 10s)...") fmt.Println("Shutting down logger (allowing up to 10s)...")
shutdownTimeout := 10 * time.Second shutdownTimeout := 10 * time.Second
err = log.Shutdown(shutdownTimeout) err = logger.Shutdown(shutdownTimeout)
if err != nil { if err != nil {
fmt.Fprintf(os.Stderr, "Logger shutdown error: %v\n", err) fmt.Fprintf(os.Stderr, "Logger shutdown error: %v\n", err)
} else { } else {

73
config.go
View File

@ -38,45 +38,44 @@ type Config struct {
MaxCheckIntervalMs int64 `toml:"max_check_interval_ms"` // Maximum adaptive interval MaxCheckIntervalMs int64 `toml:"max_check_interval_ms"` // Maximum adaptive interval
} }
// DefaultConfig returns a LogConfig with sensible defaults. // defaultConfig is the single source of truth for all default values
// These defaults are primarily used if config registration or loading fails, var defaultConfig = Config{
// or before the first configuration is applied. The primary default mechanism // Basic settings
// is config.Register. Level: LevelInfo,
func DefaultConfig() *Config { Name: "log",
return &Config{ Directory: "./logs",
Level: LevelInfo, Format: "txt",
Name: "log", Extension: "log",
Directory: "./logs",
Format: "txt", // Formatting
Extension: "log", ShowTimestamp: true,
ShowTimestamp: true, ShowLevel: true,
ShowLevel: true,
BufferSize: 1024, // Buffer and size limits
MaxSizeMB: 10, BufferSize: 1024,
MaxTotalSizeMB: 50, MaxSizeMB: 10,
MinDiskFreeMB: 100, MaxTotalSizeMB: 50,
FlushIntervalMs: 100, MinDiskFreeMB: 100,
TraceDepth: 0,
RetentionPeriodHrs: 0.0, // Timers
RetentionCheckMins: 60.0, FlushIntervalMs: 100,
DiskCheckIntervalMs: 5000, TraceDepth: 0,
EnableAdaptiveInterval: true, RetentionPeriodHrs: 0.0,
EnablePeriodicSync: false, RetentionCheckMins: 60.0,
MinCheckIntervalMs: 100,
MaxCheckIntervalMs: 60000, // Disk check settings
} DiskCheckIntervalMs: 5000,
EnableAdaptiveInterval: true,
EnablePeriodicSync: false,
MinCheckIntervalMs: 100,
MaxCheckIntervalMs: 60000,
} }
// Clone creates a deep copy of the Config. // DefaultConfig returns a copy of the default configuration
// Used internally to avoid modifying the shared config object directly. func DefaultConfig() *Config {
func (c *Config) Clone() *Config { // Create a copy to prevent modifications to the original
if c == nil { config := defaultConfig
// Should ideally not happen if Load() returns default, but defensive copy return &config
return DefaultConfig()
}
// Create a shallow copy, which is sufficient as all fields are basic types
clone := *c
return &clone
} }
// validate performs basic sanity checks on the configuration values. // validate performs basic sanity checks on the configuration values.

98
default.go
View File

@ -1,98 +0,0 @@
// --- File: default.go ---
package log
import (
"time"
"github.com/LixenWraith/config"
)
// Global instance for package-level functions
var defaultLogger = NewLogger()
// Default package-level functions that delegate to the default logger
// Init initializes or reconfigures the logger using the provided config.Config instance
func Init(cfg *config.Config, basePath string) error {
return defaultLogger.Init(cfg, basePath)
}
// InitWithDefaults initializes the logger with built-in defaults and optional overrides
func InitWithDefaults(overrides ...string) error {
return defaultLogger.InitWithDefaults(overrides...)
}
// Shutdown gracefully closes the logger, attempting to flush pending records
func Shutdown(timeout time.Duration) error {
return defaultLogger.Shutdown(timeout)
}
// Debug logs a message at debug level
func Debug(args ...any) {
defaultLogger.Debug(args...)
}
// Info logs a message at info level
func Info(args ...any) {
defaultLogger.Info(args...)
}
// Warn logs a message at warning level
func Warn(args ...any) {
defaultLogger.Warn(args...)
}
// Error logs a message at error level
func Error(args ...any) {
defaultLogger.Error(args...)
}
// DebugTrace logs a debug message with function call trace
func DebugTrace(depth int, args ...any) {
defaultLogger.DebugTrace(depth, args...)
}
// InfoTrace logs an info message with function call trace
func InfoTrace(depth int, args ...any) {
defaultLogger.InfoTrace(depth, args...)
}
// WarnTrace logs a warning message with function call trace
func WarnTrace(depth int, args ...any) {
defaultLogger.WarnTrace(depth, args...)
}
// ErrorTrace logs an error message with function call trace
func ErrorTrace(depth int, args ...any) {
defaultLogger.ErrorTrace(depth, args...)
}
// Log writes a timestamp-only record without level information
func Log(args ...any) {
defaultLogger.Log(args...)
}
// Message writes a plain record without timestamp or level info
func Message(args ...any) {
defaultLogger.Message(args...)
}
// LogTrace writes a timestamp record with call trace but no level info
func LogTrace(depth int, args ...any) {
defaultLogger.LogTrace(depth, args...)
}
// SaveConfig saves the current logger configuration to a file
func SaveConfig(path string) error {
return defaultLogger.SaveConfig(path)
}
// LoadConfig loads logger configuration from a file with optional CLI overrides
func LoadConfig(path string, args []string) error {
return defaultLogger.LoadConfig(path, args)
}
// Flush triggers a sync of the current log file buffer to disk and waits for completion or timeout
func Flush(timeout time.Duration) error {
return defaultLogger.Flush(timeout)
}

2
go.mod
View File

@ -2,7 +2,7 @@ module github.com/LixenWraith/log
go 1.24.2 go 1.24.2
require github.com/LixenWraith/config v0.0.0-20250422065842-0c5b33a935d3 require github.com/LixenWraith/config v0.0.0-20250423043415-925ccb5f1748
require ( require (
github.com/LixenWraith/tinytoml v0.0.0-20250422065624-8aa28720f04a // indirect github.com/LixenWraith/tinytoml v0.0.0-20250422065624-8aa28720f04a // indirect

4
go.sum
View File

@ -1,5 +1,5 @@
github.com/LixenWraith/config v0.0.0-20250422065842-0c5b33a935d3 h1:FosLYzJhQRB5skEvG50gZb5gALUS1zn7jzA6bWLxjB4= github.com/LixenWraith/config v0.0.0-20250423043415-925ccb5f1748 h1:d5Kq0OSqsJM8eSwA4xvoAOAWwniKBgZOy3h4e4fjiPo=
github.com/LixenWraith/config v0.0.0-20250422065842-0c5b33a935d3/go.mod h1:LWz2FXeYAN1IxmPFAmbMZLhL/5LbHzJgnj4m7l5jGvc= github.com/LixenWraith/config v0.0.0-20250423043415-925ccb5f1748/go.mod h1:LWz2FXeYAN1IxmPFAmbMZLhL/5LbHzJgnj4m7l5jGvc=
github.com/LixenWraith/tinytoml v0.0.0-20250422065624-8aa28720f04a h1:m+lhpIexwlJa5m1QuEveRmaGIE+wp87T97PyX1IWbMw= github.com/LixenWraith/tinytoml v0.0.0-20250422065624-8aa28720f04a h1:m+lhpIexwlJa5m1QuEveRmaGIE+wp87T97PyX1IWbMw=
github.com/LixenWraith/tinytoml v0.0.0-20250422065624-8aa28720f04a/go.mod h1:Vax79K0I//Klsa8POjua/XHbsMUiIdjJHr59VFbc0/8= github.com/LixenWraith/tinytoml v0.0.0-20250422065624-8aa28720f04a/go.mod h1:Vax79K0I//Klsa8POjua/XHbsMUiIdjJHr59VFbc0/8=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY= github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=

117
interface.go
View File

@ -3,8 +3,6 @@ package log
import ( import (
"time" "time"
"github.com/LixenWraith/config"
) )
// Log level constants // Log level constants
@ -31,56 +29,71 @@ type logRecord struct {
Args []any Args []any
} }
// LoggerInterface defines the public methods for a logger implementation. // Logger instance methods for logging at different levels
type LoggerInterface interface {
// Init initializes or reconfigures the logger using the provided config.Config instance
Init(cfg *config.Config, basePath string) error
// InitWithDefaults initializes the logger with built-in defaults and optional overrides // Debug logs a message at debug level.
InitWithDefaults(overrides ...string) error func (l *Logger) Debug(args ...any) {
flags := l.getFlags()
// Shutdown gracefully closes the logger, attempting to flush pending records traceDepth, _ := l.config.Int64("log.trace_depth")
Shutdown(timeout time.Duration) error l.log(flags, LevelDebug, traceDepth, args...)
// Debug logs a message at debug level
Debug(args ...any)
// Info logs a message at info level
Info(args ...any)
// Warn logs a message at warning level
Warn(args ...any)
// Error logs a message at error level
Error(args ...any)
// DebugTrace logs a debug message with function call trace
DebugTrace(depth int, args ...any)
// InfoTrace logs an info message with function call trace
InfoTrace(depth int, args ...any)
// WarnTrace logs a warning message with function call trace
WarnTrace(depth int, args ...any)
// ErrorTrace logs an error message with function call trace
ErrorTrace(depth int, args ...any)
// Log writes a timestamp-only record without level information
Log(args ...any)
// Message writes a plain record without timestamp or level info
Message(args ...any)
// LogTrace writes a timestamp record with call trace but no level info
LogTrace(depth int, args ...any)
// SaveConfig saves the current logger configuration to a file
SaveConfig(path string) error
// LoadConfig loads logger configuration from a file with optional CLI overrides
LoadConfig(path string, args []string) error
} }
// Compile-time check to ensure Logger implements LoggerInterface // Info logs a message at info level.
var _ LoggerInterface = (*Logger)(nil) func (l *Logger) Info(args ...any) {
flags := l.getFlags()
traceDepth, _ := l.config.Int64("log.trace_depth")
l.log(flags, LevelInfo, traceDepth, args...)
}
// Warn logs a message at warning level.
func (l *Logger) Warn(args ...any) {
flags := l.getFlags()
traceDepth, _ := l.config.Int64("log.trace_depth")
l.log(flags, LevelWarn, traceDepth, args...)
}
// Error logs a message at error level.
func (l *Logger) Error(args ...any) {
flags := l.getFlags()
traceDepth, _ := l.config.Int64("log.trace_depth")
l.log(flags, LevelError, traceDepth, args...)
}
// DebugTrace logs a debug message with function call trace.
func (l *Logger) DebugTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelDebug, int64(depth), args...)
}
// InfoTrace logs an info message with function call trace.
func (l *Logger) InfoTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelInfo, int64(depth), args...)
}
// WarnTrace logs a warning message with function call trace.
func (l *Logger) WarnTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelWarn, int64(depth), args...)
}
// ErrorTrace logs an error message with function call trace.
func (l *Logger) ErrorTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelError, int64(depth), args...)
}
// Log writes a timestamp-only record without level information.
func (l *Logger) Log(args ...any) {
l.log(FlagShowTimestamp, LevelInfo, 0, args...)
}
// Message writes a plain record without timestamp or level info.
func (l *Logger) Message(args ...any) {
l.log(0, LevelInfo, 0, args...)
}
// LogTrace writes a timestamp record with call trace but no level info.
func (l *Logger) LogTrace(depth int, args ...any) {
l.log(FlagShowTimestamp, LevelInfo, int64(depth), args...)
}

138
logger.go
View File

@ -19,29 +19,6 @@ type Logger struct {
serializer *serializer // Encapsulated serializer instance serializer *serializer // Encapsulated serializer instance
} }
// configDefaults holds the default values for logger configuration
var configDefaults = map[string]interface{}{
"log.level": LevelInfo,
"log.name": "log",
"log.directory": "./logs",
"log.format": "txt",
"log.extension": "log",
"log.show_timestamp": true,
"log.show_level": true,
"log.buffer_size": int64(1024),
"log.max_size_mb": int64(10),
"log.max_total_size_mb": int64(50),
"log.min_disk_free_mb": int64(100),
"log.flush_interval_ms": int64(100),
"log.trace_depth": int64(0),
"log.retention_period_hrs": float64(0.0),
"log.retention_check_mins": float64(60.0),
"log.disk_check_interval_ms": int64(5000),
"log.enable_adaptive_interval": true,
"log.min_check_interval_ms": int64(100),
"log.max_check_interval_ms": int64(60000),
}
// NewLogger creates a new Logger instance with default settings // NewLogger creates a new Logger instance with default settings
func NewLogger() *Logger { func NewLogger() *Logger {
l := &Logger{ l := &Logger{
@ -96,37 +73,39 @@ func (l *Logger) SaveConfig(path string) error {
// registerConfigValues registers all configuration parameters with the config instance // registerConfigValues registers all configuration parameters with the config instance
func (l *Logger) registerConfigValues() { func (l *Logger) registerConfigValues() {
// Register each configuration value with its default // Register the entire config struct at once
for path, defaultValue := range configDefaults { err := l.config.RegisterStruct("log.", defaultConfig)
err := l.config.Register(path, defaultValue) if err != nil {
if err != nil { fmt.Fprintf(os.Stderr, "log: warning - failed to register config values: %v\n", err)
// If registration fails, we'll handle it gracefully
fmt.Fprintf(os.Stderr, "log: warning - failed to register config key '%s': %v\n", path, err)
}
} }
} }
// updateConfigFromExternal updates the logger config from an external config.Config instance // updateConfigFromExternal updates the logger config from an external config.Config instance
func (l *Logger) updateConfigFromExternal(extCfg *config.Config, basePath string) error { func (l *Logger) updateConfigFromExternal(extCfg *config.Config, basePath string) error {
// For each config key, get value from external config and update local config // Get our registered config paths (already registered during initialization)
for path := range configDefaults { registeredPaths := l.config.GetRegisteredPaths("log.")
// Extract the local name without the "log." prefix if len(registeredPaths) == 0 {
localName := strings.TrimPrefix(path, "log.") // Register defaults first if not already done
l.registerConfigValues()
registeredPaths = l.config.GetRegisteredPaths("log.")
}
// Create the full path for the external config // For each registered path
fullPath := localName for path := range registeredPaths {
if basePath != "" { // Extract local name and build external path
fullPath = basePath + "." + localName localName := strings.TrimPrefix(path, "log.")
fullPath := basePath + "." + localName
if basePath == "" {
fullPath = localName
} }
// Get current value from our config to use as default in external config // Get current value to use as default in external config
currentVal, found := l.config.Get(path) currentVal, found := l.config.Get(path)
if !found { if !found {
// Use the original default if not found in current config continue // Skip if not found (shouldn't happen)
currentVal = configDefaults[path]
} }
// Register in external config with our current value as the default // Register in external config with current value as default
err := extCfg.Register(fullPath, currentVal) err := extCfg.Register(fullPath, currentVal)
if err != nil { if err != nil {
return fmtErrorf("failed to register config key '%s': %w", fullPath, err) return fmtErrorf("failed to register config key '%s': %w", fullPath, err)
@ -138,14 +117,12 @@ func (l *Logger) updateConfigFromExternal(extCfg *config.Config, basePath string
continue // Use existing value if not found in external config continue // Use existing value if not found in external config
} }
// Validate the value before updating // Validate and update
if err := validateConfigValue(localName, val); err != nil { if err := validateConfigValue(localName, val); err != nil {
return fmtErrorf("invalid value for '%s': %w", localName, err) return fmtErrorf("invalid value for '%s': %w", localName, err)
} }
// Update our config with the new value if err := l.config.Set(path, val); err != nil {
err = l.config.Set(path, val)
if err != nil {
return fmtErrorf("failed to update config value for '%s': %w", path, err) return fmtErrorf("failed to update config value for '%s': %w", path, err)
} }
} }
@ -277,75 +254,6 @@ func (l *Logger) getCurrentLogChannel() chan logRecord {
return chVal.(chan logRecord) return chVal.(chan logRecord)
} }
// Logger instance methods for logging at different levels
// Debug logs a message at debug level.
func (l *Logger) Debug(args ...any) {
flags := l.getFlags()
traceDepth, _ := l.config.Int64("log.trace_depth")
l.log(flags, LevelDebug, traceDepth, args...)
}
// Info logs a message at info level.
func (l *Logger) Info(args ...any) {
flags := l.getFlags()
traceDepth, _ := l.config.Int64("log.trace_depth")
l.log(flags, LevelInfo, traceDepth, args...)
}
// Warn logs a message at warning level.
func (l *Logger) Warn(args ...any) {
flags := l.getFlags()
traceDepth, _ := l.config.Int64("log.trace_depth")
l.log(flags, LevelWarn, traceDepth, args...)
}
// Error logs a message at error level.
func (l *Logger) Error(args ...any) {
flags := l.getFlags()
traceDepth, _ := l.config.Int64("log.trace_depth")
l.log(flags, LevelError, traceDepth, args...)
}
// DebugTrace logs a debug message with function call trace.
func (l *Logger) DebugTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelDebug, int64(depth), args...)
}
// InfoTrace logs an info message with function call trace.
func (l *Logger) InfoTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelInfo, int64(depth), args...)
}
// WarnTrace logs a warning message with function call trace.
func (l *Logger) WarnTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelWarn, int64(depth), args...)
}
// ErrorTrace logs an error message with function call trace.
func (l *Logger) ErrorTrace(depth int, args ...any) {
flags := l.getFlags()
l.log(flags, LevelError, int64(depth), args...)
}
// Log writes a timestamp-only record without level information.
func (l *Logger) Log(args ...any) {
l.log(FlagShowTimestamp, LevelInfo, 0, args...)
}
// Message writes a plain record without timestamp or level info.
func (l *Logger) Message(args ...any) {
l.log(0, LevelInfo, 0, args...)
}
// LogTrace writes a timestamp record with call trace but no level info.
func (l *Logger) LogTrace(depth int, args ...any) {
l.log(FlagShowTimestamp, LevelInfo, int64(depth), args...)
}
// Helper method to get flags from config // Helper method to get flags from config
func (l *Logger) getFlags() int64 { func (l *Logger) getFlags() int64 {
var flags int64 = 0 var flags int64 = 0