lixen/log
SHA256
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

e3.0.0 Tests added, optimization, bug fixes, builder changed.

Browse Source
This commit is contained in:
Lixen Wraith
2025-07-20 18:11:03 -04:00
parent 97b85995e9
commit 98402cce37
43 changed files with 2469 additions and 1373 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
doc/api-reference.md
View File

@ -43,10 +43,10 @@ cfg.Directory = "/var/log/app"
err := logger.ApplyConfig(cfg)
```
### ApplyOverride
### ApplyConfigString
```go
func (l *Logger) ApplyOverride(overrides ...string) error
func (l *Logger) ApplyConfigString(overrides ...string) error
```
Applies key-value overrides to the logger. Convenient interface for minor changes.
@ -61,7 +61,7 @@ Applies key-value overrides to the logger. Convenient interface for minor change
```go
logger := log.NewLogger()
err := logger.ApplyOverride("directory=/var/log/app", "name=app")
err := logger.ApplyConfigString("directory=/var/log/app", "name=app")
```
## Logging Methods
@ -358,7 +358,7 @@ type Service struct {
func NewService() (*Service, error) {
logger := log.NewLogger()
err := logger.ApplyOverride(
err := logger.ApplyConfigString(
"directory=/var/log/service",
"format=json",
"buffer_size=2048",

4
doc/compatibility-adapters.md
View File

@ -261,7 +261,7 @@ adapter.Infof("Connected to server")
// → {"msg": "Connected to server"}
```
## Advanced Configuration
## Example Configuration
### High-Performance Setup
@ -282,7 +282,7 @@ builder := compat.NewBuilder().
```go
builder := compat.NewBuilder().
WithOptions(
"directory=./logs",
"directory=./log",
"format=txt", // Human-readable
"level=-4", // Debug level
"trace_depth=3", // Include traces

2
doc/config-builder.md
View File

@ -27,7 +27,7 @@ All builder methods return `*ConfigBuilder` for chaining. Errors are accumulated
| `Directory(dir string)` | `dir`: Path | Sets log directory |
| `Format(format string)` | `format`: Output format | Sets format ("txt", "json", "raw") |
| `BufferSize(size int64)` | `size`: Buffer size | Sets channel buffer size |
| `MaxSizeMB(size int64)` | `size`: Size in MB | Sets max file size |
| `MaxSizeKB(size int64)` | `size`: Size in MB | Sets max file size |
| `EnableStdout(enable bool)` | `enable`: Boolean | Enables console output |
| `DisableFile(disable bool)` | `disable`: Boolean | Disables file output |
| `HeartbeatLevel(level int64)` | `level`: 0-3 | Sets monitoring level |

20
doc/configuration.md
View File

@ -12,14 +12,14 @@ logger := log.NewLogger()
## Configuration Methods
### ApplyConfig & ApplyOverride
### ApplyConfig & ApplyConfigString
Direct struct configuration using the Config struct, or key-value overrides:
```go
logger := log.NewLogger() // logger instance created with DefaultConfig (using default values)
logger.Info("info txt log record written to ./logs/log.log")
logger.Info("info txt log record written to ./log/log.log")
// Directly change config struct
cfg := log.GetConfig()
@ -27,13 +27,13 @@ cfg.Level = log.LevelDebug
cfg.Name = "myapp"
cfg.Directory = "/var/log/myapp"
cfg.Format = "json"
cfg.MaxSizeMB = 100
cfg.MaxSizeKB = 100
err := logger.ApplyConfig(cfg)
logger.Info("info json log record written to /var/log/myapp/myapp.log")
// Override values with key-value string
err = logger.ApplyOverride(
err = logger.ApplyConfigString(
"directory=/var/log/",
"extension=txt"
"format=txt")
@ -49,7 +49,7 @@ logger.Info("info txt log record written to /var/log/myapp.txt")
|-----------|------|-------------|------------|
| `level` | `int64` | Minimum log level (-4=Debug, 0=Info, 4=Warn, 8=Error) | `0` |
| `name` | `string` | Base name for log files | `"log"` |
| `directory` | `string` | Directory to store log files | `"./logs"` |
| `directory` | `string` | Directory to store log files | `"./log"` |
| `format` | `string` | Output format: `"txt"` or `"json"` | `"txt"` |
| `extension` | `string` | Log file extension (without dot) | `"log"` |
| `internal_errors_to_stderr` | `bool` | Write logger's internal errors to stderr | `false` |
@ -78,11 +78,11 @@ logger.Info("info txt log record written to /var/log/myapp.txt")
### File Management
| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| `max_size_mb` | `int64` | Maximum size per log file (MB) | `10` |
| `max_total_size_mb` | `int64` | Maximum total log directory size (MB) | `50` |
| `min_disk_free_mb` | `int64` | Minimum required free disk space (MB) | `100` |
| `retention_period_hrs` | `float64` | Hours to keep log files (0=disabled) | `0.0` |
|-----------|------|-------------|--------|
| `max_size_kb` | `int64` | Maximum size per log file (KB) | `1000` |
| `max_total_size_kb` | `int64` | Maximum total log directory size (KB) | `5000` |
| `min_disk_free_kb` | `int64` | Minimum required free disk space (KB) | `10000` |
| `retention_period_hrs` | `float64` | Hours to keep log files (0=disabled) | `0.0` |
| `retention_check_mins` | `float64` | Retention check interval (minutes) | `60.0` |
### Disk Monitoring

153
doc/disk-management.md
View File

@ -9,7 +9,7 @@ Comprehensive guide to log file rotation, retention policies, and disk space man
Log files are automatically rotated when they reach the configured size limit:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"max_size_mb=100", // Rotate at 100MB
)
```
@ -43,7 +43,7 @@ Components:
The logger enforces two types of space limits:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"max_total_size_mb=1000", // Total log directory size
"min_disk_free_mb=5000", // Minimum free disk space
)
@ -61,21 +61,21 @@ When limits are exceeded, the logger:
```go
// Conservative: Strict limits
logger.ApplyOverride(
logger.ApplyConfigString(
"max_size_mb=50", // 50MB files
"max_total_size_mb=500", // 500MB total
"min_disk_free_mb=1000", // 1GB free required
)
// Generous: Large files, external archival
logger.ApplyOverride(
logger.ApplyConfigString(
"max_size_mb=1000", // 1GB files
"max_total_size_mb=0", // No total limit
"min_disk_free_mb=100", // 100MB free required
)
// Balanced: Production defaults
logger.ApplyOverride(
logger.ApplyConfigString(
"max_size_mb=100", // 100MB files
"max_total_size_mb=5000", // 5GB total
"min_disk_free_mb=500", // 500MB free required
@ -89,7 +89,7 @@ logger.ApplyOverride(
Automatically delete logs older than a specified duration:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"retention_period_hrs=168", // Keep 7 days
"retention_check_mins=60", // Check hourly
)
@ -99,21 +99,21 @@ logger.ApplyOverride(
```go
// Daily logs, keep 30 days
logger.ApplyOverride(
logger.ApplyConfigString(
"retention_period_hrs=720", // 30 days
"retention_check_mins=60", // Check hourly
"max_size_mb=1000", // 1GB daily files
)
// High-frequency logs, keep 24 hours
logger.ApplyOverride(
logger.ApplyConfigString(
"retention_period_hrs=24", // 1 day
"retention_check_mins=15", // Check every 15 min
"max_size_mb=100", // 100MB files
)
// Compliance: Keep 90 days
logger.ApplyOverride(
logger.ApplyConfigString(
"retention_period_hrs=2160", // 90 days
"retention_check_mins=360", // Check every 6 hours
"max_total_size_mb=100000", // 100GB total
@ -134,7 +134,7 @@ When multiple policies conflict, cleanup priority is:
The logger adjusts disk check frequency based on logging volume:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"enable_adaptive_interval=true",
"disk_check_interval_ms=5000", // Base: 5 seconds
"min_check_interval_ms=100", // Minimum: 100ms
@ -153,7 +153,7 @@ logger.ApplyOverride(
Check disk-related heartbeat messages:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=2", // Enable disk stats
"heartbeat_interval_s=300", // Every 5 minutes
)
@ -164,9 +164,7 @@ Output:
2024-01-15T10:30:00Z DISK type="disk" sequence=1 rotated_files=5 deleted_files=2 total_log_size_mb="487.32" log_file_count=8 current_file_size_mb="23.45" disk_status_ok=true disk_free_mb="5234.67"
```
## Recovery Behavior
### Manual Intervention
## Manual Recovery
If automatic cleanup fails:
@ -184,133 +182,6 @@ ls -t /var/log/myapp/*.log | tail -n 20 | xargs rm
df -h /var/log
```
## Best Practices
### 1. Plan for Growth
Estimate log volume and set appropriate limits:
```go
// Calculate required space:
// - Average log entry: 200 bytes
// - Entries per second: 100
// - Daily volume: 200 * 100 * 86400 = 1.7GB
logger.ApplyOverride(
"max_size_mb=2000", // 2GB files (~ 1 day)
"max_total_size_mb=15000", // 15GB (~ 1 week)
"retention_period_hrs=168", // 7 days
)
```
### 2. External Archival
For long-term storage, implement external archival:
```go
// Configure for archival
logger.ApplyOverride(
"max_size_mb=1000", // 1GB files for easy transfer
"max_total_size_mb=10000", // 10GB local buffer
"retention_period_hrs=48", // 2 days local
)
// Archive completed files
func archiveCompletedLogs(archivePath string) error {
files, _ := filepath.Glob("/var/log/myapp/*.log")
for _, file := range files {
if !isCurrentLogFile(file) {
// Move to archive storage (S3, NFS, etc.)
if err := archiveFile(file, archivePath); err != nil {
return err
}
os.Remove(file)
}
}
return nil
}
```
### 3. Monitor Disk Health
Set up alerts for disk issues:
```go
// Parse heartbeat logs for monitoring
type DiskStats struct {
TotalSizeMB float64
FileCount int
DiskFreeMB float64
DiskStatusOK bool
}
func monitorDiskHealth(logLine string) {
if strings.Contains(logLine, "type=\"disk\"") {
stats := parseDiskHeartbeat(logLine)
if !stats.DiskStatusOK {
alert("Log disk unhealthy")
}
if stats.DiskFreeMB < 1000 {
alert("Low disk space: %.0fMB free", stats.DiskFreeMB)
}
if stats.FileCount > 100 {
alert("Too many log files: %d", stats.FileCount)
}
}
}
```
### 4. Separate Log Volumes
Use dedicated volumes for logs:
```bash
# Create dedicated log volume
mkdir -p /mnt/logs
mount /dev/sdb1 /mnt/logs
# Configure logger
logger.ApplyOverride(
"directory=/mnt/logs/myapp",
"max_total_size_mb=50000", # Use most of volume
"min_disk_free_mb=1000", # Leave 1GB free
)
```
### 5. Test Cleanup Behavior
Verify cleanup works before production:
```go
// Test configuration
func TestDiskCleanup(t *testing.T) {
logger := log.NewLogger()
logger.ApplyOverride(
"directory=./test_logs",
"max_size_mb=1", // Small files
"max_total_size_mb=5", // Low limit
"retention_period_hrs=0.01", // 36 seconds
"retention_check_mins=0.5", // 30 seconds
)
// Generate logs to trigger cleanup
for i := 0; i < 1000; i++ {
logger.Info(strings.Repeat("x", 1000))
}
time.Sleep(45 * time.Second)
// Verify cleanup occurred
files, _ := filepath.Glob("./test_logs/*.log")
if len(files) > 5 {
t.Errorf("Cleanup failed: %d files remain", len(files))
}
}
```
---
[← Logging Guide](logging-guide.md) | [← Back to README](../README.md) | [Heartbeat Monitoring →](heartbeat-monitoring.md)

4
doc/getting-started.md
View File

@ -29,7 +29,7 @@ import (
func main() {
// Create a new logger instance with default configuration
// Writes to file ./logs/log.log
// Writes to file ./log/log.log
logger := log.NewLogger()
defer logger.Shutdown()
@ -57,7 +57,7 @@ type Service struct {
func NewService() (*Service, error) {
logger := log.NewLogger()
if err := logger.ApplyOverride(
if err := logger.ApplyConfigString(
"directory=/var/log/service",
"name=service",
"format=json",

16
doc/heartbeat-monitoring.md
View File

@ -20,7 +20,7 @@ Heartbeats are periodic log messages that provide operational statistics about t
No heartbeat messages are generated.
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=0", // No heartbeats
)
```
@ -30,7 +30,7 @@ logger.ApplyOverride(
Basic logger operation metrics:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=1",
"heartbeat_interval_s=300", // Every 5 minutes
)
@ -52,7 +52,7 @@ logger.ApplyOverride(
Includes file and disk usage information:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=2",
"heartbeat_interval_s=300",
)
@ -77,7 +77,7 @@ logger.ApplyOverride(
Includes runtime and memory metrics:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=3",
"heartbeat_interval_s=60", // Every minute for detailed monitoring
)
@ -99,7 +99,7 @@ logger.ApplyOverride(
### Basic Configuration
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=2", // Process + Disk stats
"heartbeat_interval_s=300", // Every 5 minutes
)
@ -118,19 +118,19 @@ logger.ApplyOverride(
```go
// Start with basic monitoring
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=1",
"heartbeat_interval_s=600",
)
// During incident, increase detail
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=3",
"heartbeat_interval_s=60",
)
// After resolution, reduce back
logger.ApplyOverride(
logger.ApplyConfigString(
"heartbeat_level=1",
"heartbeat_interval_s=600",
)

284
doc/llm-guide.md
View File

@ -1,284 +0,0 @@
# lixenwraith/log LLM Usage Guide
High-performance, thread-safe logging library for Go with file rotation, disk management, and compatibility adapters for popular frameworks.
## Core Types
### Logger
```go
// Primary logger instance. All operations are thread-safe.
type Logger struct {
// Internal fields - thread-safe logging implementation
}
```
### Config
```go
// Logger configuration with validation support.
type Config struct {
// Basic settings
Level int64 `toml:"level"`
Name string `toml:"name"`
Directory string `toml:"directory"`
Format string `toml:"format"` // "txt", "json", or "raw"
Extension string `toml:"extension"`
// Formatting
ShowTimestamp bool `toml:"show_timestamp"`
ShowLevel bool `toml:"show_level"`
TimestampFormat string `toml:"timestamp_format"`
// Buffer and size limits
BufferSize int64 `toml:"buffer_size"`
MaxSizeMB int64 `toml:"max_size_mb"`
MaxTotalSizeMB int64 `toml:"max_total_size_mb"`
MinDiskFreeMB int64 `toml:"min_disk_free_mb"`
// Timers
FlushIntervalMs int64 `toml:"flush_interval_ms"`
TraceDepth int64 `toml:"trace_depth"`
RetentionPeriodHrs float64 `toml:"retention_period_hrs"`
RetentionCheckMins float64 `toml:"retention_check_mins"`
// Disk check settings
DiskCheckIntervalMs int64 `toml:"disk_check_interval_ms"`
EnableAdaptiveInterval bool `toml:"enable_adaptive_interval"`
EnablePeriodicSync bool `toml:"enable_periodic_sync"`
MinCheckIntervalMs int64 `toml:"min_check_interval_ms"`
MaxCheckIntervalMs int64 `toml:"max_check_interval_ms"`
// Heartbeat configuration
HeartbeatLevel int64 `toml:"heartbeat_level"`
HeartbeatIntervalS int64 `toml:"heartbeat_interval_s"`
// Stdout/console output settings
EnableStdout bool `toml:"enable_stdout"`
StdoutTarget string `toml:"stdout_target"` // "stdout", "stderr", or "split"
DisableFile bool `toml:"disable_file"`
// Internal error handling
InternalErrorsToStderr bool `toml:"internal_errors_to_stderr"`
}
```
## Constants
### Log Levels
```go
const (
LevelDebug int64 = -4
LevelInfo int64 = 0
LevelWarn int64 = 4
LevelError int64 = 8
)
```
### Heartbeat Levels
```go
const (
LevelProc int64 = 12 // Process statistics
LevelDisk int64 = 16 // Disk usage statistics
LevelSys int64 = 20 // System statistics
)
```
## Core Methods
### Creation
```go
func NewLogger() *Logger
func DefaultConfig() *Config
```
### Configuration
```go
func (l *Logger) ApplyConfig(cfg *Config) error
func (l *Logger) ApplyOverride(overrides ...string) error
func (l *Logger) GetConfig() *Config
```
### Logging Methods
```go
func (l *Logger) Debug(args ...any)
func (l *Logger) Info(args ...any)
func (l *Logger) Warn(args ...any)
func (l *Logger) Error(args ...any)
func (l *Logger) LogStructured(level int64, message string, fields map[string]any)
func (l *Logger) Write(args ...any) // Raw output, no formatting
func (l *Logger) Log(args ...any) // Timestamp only, no level
func (l *Logger) Message(args ...any) // No timestamp or level
```
### Trace Logging
```go
func (l *Logger) DebugTrace(depth int, args ...any)
func (l *Logger) InfoTrace(depth int, args ...any)
func (l *Logger) WarnTrace(depth int, args ...any)
func (l *Logger) ErrorTrace(depth int, args ...any)
func (l *Logger) LogTrace(depth int, args ...any)
```
### Control Methods
```go
func (l *Logger) Shutdown(timeout ...time.Duration) error
func (l *Logger) Flush(timeout time.Duration) error
```
### Utilities
```go
func Level(levelStr string) (int64, error)
```
## Configuration Builder
### ConfigBuilder
```go
type ConfigBuilder struct {
// Internal builder state
}
```
### Builder Methods
```go
func NewConfigBuilder() *ConfigBuilder
func (b *ConfigBuilder) Build() (*Config, error)
func (b *ConfigBuilder) Level(level int64) *ConfigBuilder
func (b *ConfigBuilder) LevelString(level string) *ConfigBuilder
func (b *ConfigBuilder) Directory(dir string) *ConfigBuilder
func (b *ConfigBuilder) Format(format string) *ConfigBuilder
func (b *ConfigBuilder) BufferSize(size int64) *ConfigBuilder
func (b *ConfigBuilder) MaxSizeMB(size int64) *ConfigBuilder
func (b *ConfigBuilder) EnableStdout(enable bool) *ConfigBuilder
func (b *ConfigBuilder) DisableFile(disable bool) *ConfigBuilder
func (b *ConfigBuilder) HeartbeatLevel(level int64) *ConfigBuilder
func (b *ConfigBuilder) HeartbeatIntervalS(seconds int64) *ConfigBuilder
```
## Compatibility Adapters (log/compat)
### Builder
```go
type Builder struct {
// Internal adapter builder state
}
```
### Builder Methods
```go
func NewBuilder() *Builder
func (b *Builder) WithLogger(l *log.Logger) *Builder
func (b *Builder) WithConfig(cfg *log.Config) *Builder
func (b *Builder) BuildGnet(opts ...GnetOption) (*GnetAdapter, error)
func (b *Builder) BuildStructuredGnet(opts ...GnetOption) (*StructuredGnetAdapter, error)
func (b *Builder) BuildFastHTTP(opts ...FastHTTPOption) (*FastHTTPAdapter, error)
func (b *Builder) GetLogger() (*log.Logger, error)
```
### gnet Adapters
```go
type GnetAdapter struct {
// Implements gnet.Logger interface
}
type StructuredGnetAdapter struct {
*GnetAdapter
// Enhanced with field extraction
}
type GnetOption func(*GnetAdapter)
func WithFatalHandler(handler func(string)) GnetOption
```
### gnet Interface Implementation
```go
func (a *GnetAdapter) Debugf(format string, args ...any)
func (a *GnetAdapter) Infof(format string, args ...any)
func (a *GnetAdapter) Warnf(format string, args ...any)
func (a *GnetAdapter) Errorf(format string, args ...any)
func (a *GnetAdapter) Fatalf(format string, args ...any)
```
### fasthttp Adapter
```go
type FastHTTPAdapter struct {
// Implements fasthttp.Logger interface
}
type FastHTTPOption func(*FastHTTPAdapter)
func WithDefaultLevel(level int64) FastHTTPOption
func WithLevelDetector(detector func(string) int64) FastHTTPOption
```
### fasthttp Interface Implementation
```go
func (a *FastHTTPAdapter) Printf(format string, args ...any)
```
### Helper Functions
```go
func NewGnetAdapter(logger *log.Logger, opts ...GnetOption) *GnetAdapter
func NewStructuredGnetAdapter(logger *log.Logger, opts ...GnetOption) *StructuredGnetAdapter
func NewFastHTTPAdapter(logger *log.Logger, opts ...FastHTTPOption) *FastHTTPAdapter
func DetectLogLevel(msg string) int64
```
## File Management
### Rotation
Files rotate automatically when `MaxSizeMB` is reached. Rotated files use naming pattern: `{name}_{YYMMDD}_{HHMMSS}_{nanoseconds}.{extension}`
### Disk Management
- Enforces `MaxTotalSizeMB` for total log directory size
- Maintains `MinDiskFreeMB` free disk space
- Deletes oldest logs when limits exceeded
### Retention
- Time-based cleanup with `RetentionPeriodHrs`
- Periodic checks via `RetentionCheckMins`
## Heartbeat Monitoring
### Levels
- **0**: Disabled (default)
- **1**: Process stats (logs processed, dropped, uptime)
- **2**: + Disk stats (rotations, deletions, sizes, free space)
- **3**: + System stats (memory, GC, goroutines)
### Output
Heartbeats bypass log level filtering and use special levels (PROC, DISK, SYS).
## Output Formats
### Text Format
Human-readable with configurable timestamp and level display.
### JSON Format
Machine-parseable with structured fields array.
### Raw Format
Space-separated values without metadata, triggered by `Write()` method or `format=raw`.
## Thread Safety
All public methods are thread-safe. Concurrent logging from multiple goroutines is supported without external synchronization.
## Configuration Overrides
String key-value pairs for runtime configuration changes:
```
"level=-4" // Numeric level
"level=debug" // Named level
"directory=/var/log" // String value
"buffer_size=2048" // Integer value
"enable_stdout=true" // Boolean value
```
## Error Handling
- Configuration errors prefixed with "log: "
- Failed initialization disables logger
- Dropped logs tracked and reported periodically
- Internal errors optionally written to stderr
## Performance Characteristics
- Non-blocking log submission (buffered channel)
- Adaptive disk checking based on load
- Batch file writes with configurable flush interval
- Automatic log dropping under extreme load with tracking

10
doc/logging-guide.md
View File

@ -31,13 +31,13 @@ logger.Error("Database query failed", "query", query, "error", err)
```go
// Development: See everything
logger.ApplyOverride("level=-4") // Debug and above
logger.ApplyConfigString("level=-4") // Debug and above
// Production: Reduce noise
logger.ApplyOverride("level=0") // Info and above
logger.ApplyConfigString("level=0") // Info and above
// Critical systems: Errors only
logger.ApplyOverride("level=8") // Error only
logger.ApplyConfigString("level=8") // Error only
```
## Structured Logging
@ -137,7 +137,7 @@ Default format for development and debugging:
Configuration:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"format=txt",
"show_timestamp=true",
"show_level=true",
@ -155,7 +155,7 @@ Ideal for log aggregation and analysis:
Configuration:
```go
logger.ApplyOverride(
logger.ApplyConfigString(
"format=json",
"show_timestamp=true",
"show_level=true",