v0.1.6 fix to formatter and sanitizer for async use, doc update
This commit is contained in:
+9
-7
@@ -267,13 +267,13 @@ if err != nil {
|
||||
func (l *Logger) Flush(timeout time.Duration) error
|
||||
```
|
||||
|
||||
Explicitly triggers a sync of the current log file buffer to disk.
|
||||
Explicitly triggers a sync of the current log file buffer to disk. Uses **barrier semantics**: it guarantees that all log records enqueued *before* the `Flush` call are fully processed and formatted before the disk sync occurs and confirmation is returned.
|
||||
|
||||
**Parameters:**
|
||||
- `timeout`: Maximum time to wait for flush completion
|
||||
|
||||
**Returns:**
|
||||
- `error`: Flush error if timeout exceeded
|
||||
- `error`: Flush error if timeout exceeded or logger is uninitialized.
|
||||
|
||||
**Example:**
|
||||
```go
|
||||
@@ -331,10 +331,12 @@ level, err := log.Level("debug") // Returns -4
|
||||
|
||||
```go
|
||||
const (
|
||||
FlagRaw = formatter.FlagRaw // Bypass formatting
|
||||
FlagShowTimestamp = formatter.FlagShowTimestamp // Include timestamp
|
||||
FlagShowLevel = formatter.FlagShowLevel // Include level
|
||||
FlagStructuredJSON = formatter.FlagStructuredJSON // Structured JSON
|
||||
FlagRaw = formatter.FlagRaw // Bypass formatting and sanitization
|
||||
FlagShowTimestamp = formatter.FlagShowTimestamp // Force include timestamp
|
||||
FlagShowLevel = formatter.FlagShowLevel // Force include level
|
||||
FlagStructuredJSON = formatter.FlagStructuredJSON // Structured JSON output
|
||||
FlagNoTimestamp = formatter.FlagNoTimestamp // Suppress timestamp
|
||||
FlagNoLevel = formatter.FlagNoLevel // Suppress level
|
||||
FlagDefault = formatter.FlagDefault // Default flags
|
||||
)
|
||||
```
|
||||
@@ -412,4 +414,4 @@ func (s *Service) ProcessRequest(id string) error {
|
||||
func (s *Service) Shutdown() error {
|
||||
return s.logger.Shutdown(5 * time.Second)
|
||||
}
|
||||
```
|
||||
```
|
||||
|
||||
+76
-20
@@ -35,31 +35,40 @@ data := f.Format(
|
||||
)
|
||||
```
|
||||
|
||||
### Formatter Methods
|
||||
### Formatter Methods and Concurrency
|
||||
|
||||
#### Format Configuration
|
||||
- `Type(format string)` - Set output format: "txt", "json", or "raw"
|
||||
- `TimestampFormat(format string)` - Set timestamp format (Go time format)
|
||||
- `ShowLevel(show bool)` - Include level in output
|
||||
- `ShowTimestamp(show bool)` - Include timestamp in output
|
||||
The formatter provides two classes of methods. **You must understand the concurrency contract** when using these standalone:
|
||||
|
||||
#### Formatting Methods
|
||||
- `Format(flags int64, timestamp time.Time, level int64, trace string, args []any) []byte`
|
||||
- `FormatWithOptions(format string, flags int64, timestamp time.Time, level int64, trace string, args []any) []byte`
|
||||
- `FormatValue(v any) []byte` - Format a single value
|
||||
- `FormatArgs(args ...any) []byte` - Format multiple arguments
|
||||
**1. Buffered Methods (Single Goroutine Only)**
|
||||
These methods reuse an internal buffer to prevent allocations. **The returned byte slice is valid ONLY until the next buffered call.** You must copy the result (`bytes.Clone()`) before retention or async hand-off.
|
||||
* `Format(flags int64, timestamp time.Time, level int64, trace string, args []any) []byte`
|
||||
* `FormatWithOptions(format string, flags int64, timestamp time.Time, level int64, trace string, args []any) []byte`
|
||||
* `FormatValue(v any) []byte`
|
||||
* `FormatArgs(args ...any) []byte`
|
||||
|
||||
**2. Append Methods (Thread-Safe)**
|
||||
These methods write to a caller-provided destination buffer. Once the formatter is configured, these are **safe for concurrent use** and are preferred for async sinks.
|
||||
* `AppendFormat(dst []byte, flags int64, timestamp time.Time, level int64, trace string, args []any) []byte`
|
||||
* `AppendFormatWithOptions(dst []byte, format string, flags int64, timestamp time.Time, level int64, trace string, args []any) []byte`
|
||||
* `AppendValue(dst []byte, v any) []byte`
|
||||
* `AppendArgs(dst []byte, args ...any) []byte`
|
||||
|
||||
### Format Flags
|
||||
|
||||
Flags use an additive resolution system. `No*` flags suppress output, `Show*` flags force output, and if neither is specified, the configured default applies. (`No*` flags win on conflicts).
|
||||
|
||||
```go
|
||||
const (
|
||||
FlagRaw int64 = 0b0001 // Bypass formatter and sanitizer
|
||||
FlagShowTimestamp int64 = 0b0010 // Include timestamp
|
||||
FlagShowLevel int64 = 0b0100 // Include level
|
||||
FlagStructuredJSON int64 = 0b1000 // Use structured JSON with message/fields
|
||||
FlagRaw int64 = 0b0001 // Bypass formatter and sanitizer completely
|
||||
FlagShowTimestamp int64 = 0b0010 // Force include timestamp
|
||||
FlagShowLevel int64 = 0b0100 // Force include level
|
||||
FlagStructuredJSON int64 = 0b1000 // Use structured JSON with message/fields
|
||||
FlagNoTimestamp int64 = 0b010000 // Suppress timestamp
|
||||
FlagNoLevel int64 = 0b100000 // Suppress level
|
||||
FlagDefault = FlagShowTimestamp | FlagShowLevel
|
||||
)
|
||||
```
|
||||
*Note: `FormatWithOptions` and `AppendFormatWithOptions` bypass configured defaults entirely. Unset `Show*` bits in these methods mean the feature is off.*
|
||||
|
||||
### Level Constants
|
||||
|
||||
@@ -105,7 +114,7 @@ const (
|
||||
- **PolicyRaw**: Pass through all characters unchanged
|
||||
- **PolicyTxt**: Hex-encode non-printable characters as `<XX>`
|
||||
- **PolicyJSON**: Escape control characters with JSON-style backslashes
|
||||
- **PolicyShell**: Strip shell metacharacters and whitespace
|
||||
- **PolicyShell**: Strips shell metacharacters (``` ` $ ; | & > < ( ) # ' " \ * ? [ ] { } ~ ! ```), whitespace, and control characters. *Note: Used for defense-in-depth logging, NOT for safely constructing executable shell commands.*
|
||||
|
||||
### Filter Flags
|
||||
|
||||
@@ -158,6 +167,44 @@ serializer.WriteBool(&buf, true) // "true"
|
||||
serializer.WriteNil(&buf) // "null"
|
||||
```
|
||||
|
||||
## JSON Escaping Layers
|
||||
|
||||
The sanitizer is a content transform; JSON string escaping is transport
|
||||
encoding applied afterward, unconditionally. Output is valid JSON for any
|
||||
sanitization policy. Multi-byte UTF-8 passes through unescaped.
|
||||
|
||||
- `format=json` + `sanitization=raw`: recommended; transport escaping only.
|
||||
- `format=json` + `sanitization=txt`: non-printables appear as `<XX>` inside
|
||||
JSON strings.
|
||||
- `format=json` + `sanitization=json`: redundant; produces visible `\\n`
|
||||
double escapes. Use `raw` instead.
|
||||
- Structured JSON (`FlagStructuredJSON`) marshals the fields map via
|
||||
`encoding/json` and bypasses the sanitizer; validity is guaranteed,
|
||||
content-level sanitization is not applied to field values.
|
||||
|
||||
## PolicyShell Scope
|
||||
|
||||
`PolicyShell` strips metacharacters, whitespace, and control characters as
|
||||
defense-in-depth for logged values. It is not sufficient for constructing
|
||||
shell commands from untrusted input; pass arguments via exec argv.
|
||||
|
||||
## Hex Marker Integrity
|
||||
|
||||
`PolicyTxt` hex-encodes literal `<` as `<3c>`. Every `<` in sanitized output
|
||||
therefore starts a genuine marker; encoded sequences cannot be spoofed by
|
||||
input containing literal `<XX>` text.
|
||||
|
||||
## Format Flags
|
||||
|
||||
| Flag | Effect |
|
||||
|---|---|
|
||||
| `FlagShowTimestamp` / `FlagShowLevel` | Force display on |
|
||||
| `FlagNoTimestamp` / `FlagNoLevel` | Force display off (wins over Show) |
|
||||
| neither | Configured default applies (`Format`/`AppendFormat` only) |
|
||||
|
||||
`FormatWithOptions`/`AppendFormatWithOptions` ignore configured defaults:
|
||||
unset Show bits mean off. Unknown format strings fall back to `"txt"`.
|
||||
|
||||
## Integration with Logger
|
||||
|
||||
The logger uses these packages internally but configuration remains simple:
|
||||
@@ -227,8 +274,17 @@ scriptLog := txtFormatter.Format(...)
|
||||
- Formatter reuses internal buffers via `Reset()`
|
||||
- No regex or reflection in hot paths
|
||||
|
||||
## Thread Safety
|
||||
## Ownership and Thread Safety
|
||||
|
||||
- `Formatter` instances are **NOT** thread-safe (use separate instances per goroutine)
|
||||
- `Sanitizer` instances **ARE** thread-safe (immutable after creation)
|
||||
- For concurrent formatting, create a formatter per goroutine or use sync.Pool
|
||||
- Configuration (`Type`, `ShowLevel`, `Rule`, `RuleFunc`, `Policy`, ...) must
|
||||
complete before an instance is shared between goroutines.
|
||||
- `Formatter` buffered methods (`Format`, `FormatWithOptions`, `FormatValue`,
|
||||
`FormatArgs`) reuse an internal buffer. The returned slice is valid only
|
||||
until the next buffered call. Copy (`bytes.Clone`) before retaining or
|
||||
handing off to async queues. Single goroutine only.
|
||||
- `Formatter` append methods (`AppendFormat`, `AppendFormatWithOptions`,
|
||||
`AppendValue`, `AppendArgs`) write to a caller-provided buffer and are safe
|
||||
for concurrent use. Preferred for async sinks and multi-goroutine callers.
|
||||
- `Sanitizer` is immutable after configuration; `Sanitize`/`AppendSanitize`
|
||||
are safe for concurrent use. `Sanitize` returns the input unchanged
|
||||
(allocation-free) when no rule matches.
|
||||
|
||||
+47
-1
@@ -53,6 +53,52 @@ func main() {
|
||||
}
|
||||
```
|
||||
|
||||
## Recommended Usage (Builder Pattern)
|
||||
|
||||
The Builder pattern provides a fluent API with compile-time safety and deferred validation, making it the most robust way to initialize your logger:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/lixenwraith/log"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// Build logger with fluent configuration
|
||||
logger, err := log.NewBuilder().
|
||||
Directory("/var/log/myapp"). // Log directory path
|
||||
LevelString("info"). // Minimum log level
|
||||
Format("json"). // Output format
|
||||
Sanitization("json"). // Sanitization policy
|
||||
EnableFile(true). // Enable file output (disabled by default)
|
||||
BufferSize(2048). // Channel buffer size
|
||||
MaxSizeMB(10). // Max file size before rotation
|
||||
HeartbeatLevel(1). // Enable operational monitoring
|
||||
HeartbeatIntervalS(300). // Every 5 minutes
|
||||
Build() // Build the logger instance
|
||||
|
||||
if err != nil {
|
||||
panic(fmt.Errorf("logger build failed: %w", err))
|
||||
}
|
||||
|
||||
// Ensure logs are flushed on shutdown
|
||||
defer logger.Shutdown(5 * time.Second)
|
||||
|
||||
// Start the background async processor (required before logging)
|
||||
if err := logger.Start(); err != nil {
|
||||
panic(fmt.Errorf("logger start failed: %w", err))
|
||||
}
|
||||
|
||||
// Begin logging with structured key-value pairs
|
||||
logger.Info("Application started", "version", "1.0.0", "pid", os.Getpid())
|
||||
}
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **[Learn about configuration options](configuration.md)** - Customize behavior for your needs
|
||||
@@ -114,4 +160,4 @@ func loggingMiddleware(logger *log.Logger) func(http.Handler) http.Handler {
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
```
|
||||
|
||||
@@ -1,518 +0,0 @@
|
||||
I'll search the project knowledge to understand the current state of the log package and update the quick-guide documentation accordingly.# FILE: doc/quick-guide_lixenwraith_log.md
|
||||
|
||||
# lixenwraith/log Quick Reference Guide
|
||||
|
||||
High-performance buffered rotating file logger with disk management, operational monitoring, and exported formatter/sanitizer packages.
|
||||
|
||||
## Quick Start: Recommended Usage
|
||||
|
||||
Builder pattern with type-safe configuration (compile-time safety, no runtime errors):
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/lixenwraith/log"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// Build logger with configuration
|
||||
logger, err := log.NewBuilder().
|
||||
Directory("/var/log/myapp"). // Log directory path
|
||||
LevelString("info"). // Minimum log level
|
||||
Format("json"). // Output format
|
||||
Sanitization("json"). // Sanitization policy
|
||||
EnableFile(true). // Enable file output (disabled by default)
|
||||
BufferSize(2048). // Channel buffer size
|
||||
MaxSizeMB(10). // Max file size before rotation
|
||||
HeartbeatLevel(1). // Enable operational monitoring
|
||||
HeartbeatIntervalS(300). // Every 5 minutes
|
||||
Build() // Build the logger instance
|
||||
if err != nil {
|
||||
panic(fmt.Errorf("logger build failed: %w", err))
|
||||
}
|
||||
defer logger.Shutdown(5 * time.Second)
|
||||
|
||||
// Start the logger (required before logging)
|
||||
if err := logger.Start(); err != nil {
|
||||
panic(fmt.Errorf("logger start failed: %w", err))
|
||||
}
|
||||
|
||||
// Begin logging with structured key-value pairs
|
||||
logger.Info("Application started", "version", "1.0.0", "pid", os.Getpid())
|
||||
logger.Debug("Debug information", "user_id", 12345)
|
||||
logger.Warn("High memory usage", "used_mb", 1800, "limit_mb", 2048)
|
||||
logger.Error("Connection failed", "host", "db.example.com", "error", err)
|
||||
}
|
||||
```
|
||||
|
||||
## Alternative Initialization Methods
|
||||
|
||||
### Using ApplyConfigString (Quick Configuration)
|
||||
|
||||
```go
|
||||
logger := log.NewLogger()
|
||||
err := logger.ApplyConfigString(
|
||||
"directory=/var/log/app",
|
||||
"format=json",
|
||||
"sanitization=json",
|
||||
"level=debug",
|
||||
"max_size_kb=5000",
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("config failed: %w", err)
|
||||
}
|
||||
defer logger.Shutdown()
|
||||
logger.Start()
|
||||
```
|
||||
|
||||
### Using ApplyConfig (Full Control)
|
||||
|
||||
```go
|
||||
logger := log.NewLogger()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Directory = "/var/log/app"
|
||||
cfg.Format = "json"
|
||||
cfg.Sanitization = log.PolicyJSON
|
||||
cfg.Level = log.LevelDebug
|
||||
cfg.MaxSizeKB = 5000
|
||||
cfg.HeartbeatLevel = 2 // Process + disk stats
|
||||
err := logger.ApplyConfig(cfg)
|
||||
if err != nil {
|
||||
return fmt.Errorf("config failed: %w", err)
|
||||
}
|
||||
defer logger.Shutdown()
|
||||
logger.Start()
|
||||
```
|
||||
|
||||
## Builder Pattern
|
||||
|
||||
```go
|
||||
func NewBuilder() *Builder
|
||||
func (b *Builder) Build() (*Logger, error)
|
||||
```
|
||||
|
||||
### Builder Methods
|
||||
|
||||
All builder methods return `*Builder` for chaining.
|
||||
|
||||
**Basic Configuration:**
|
||||
- `Level(level int64)`: Set numeric log level (-4 to 8)
|
||||
- `LevelString(level string)`: Set level by name ("debug", "info", "warn", "error")
|
||||
- `Directory(dir string)`: Set log directory path
|
||||
- `Name(name string)`: Set base filename (default: "log")
|
||||
- `Format(format string)`: Set format ("txt", "json", "raw")
|
||||
- `Sanitization(policy string)`: Set sanitization policy ("txt", "json", "raw", "shell")
|
||||
- `Extension(ext string)`: Set file extension (default: ".log")
|
||||
|
||||
**Buffer and Performance:**
|
||||
- `BufferSize(size int64)`: Channel buffer size (default: 1024)
|
||||
- `FlushIntervalMs(ms int64)`: Buffer flush interval (default: 100ms)
|
||||
- `TraceDepth(depth int64)`: Default function trace depth 0-10 (default: 0)
|
||||
|
||||
**File Management:**
|
||||
- `MaxSizeKB(size int64)` / `MaxSizeMB(size int64)`: Max file size before rotation
|
||||
- `MaxTotalSizeKB(size int64)` / `MaxTotalSizeMB(size int64)`: Max total directory size
|
||||
- `MinDiskFreeKB(size int64)` / `MinDiskFreeMB(size int64)`: Required free disk space
|
||||
- `RetentionPeriodHrs(hours float64)`: Hours to keep logs (0=disabled)
|
||||
- `RetentionCheckMins(mins float64)`: Retention check interval
|
||||
|
||||
**Output Control:**
|
||||
- `EnableConsole(enable bool)`: Enable stdout/stderr output
|
||||
- `EnableFile(enable bool)`: Enable file output
|
||||
- `ConsoleTarget(target string)`: "stdout", "stderr", or "split"
|
||||
|
||||
**Formatting:**
|
||||
- `ShowTimestamp(show bool)`: Add timestamps
|
||||
- `ShowLevel(show bool)`: Add level labels
|
||||
- `TimestampFormat(format string)`: Go time format string
|
||||
|
||||
**Monitoring:**
|
||||
- `HeartbeatLevel(level int64)`: 0=off, 1=proc, 2=+disk, 3=+sys
|
||||
- `HeartbeatIntervalS(seconds int64)`: Heartbeat interval
|
||||
|
||||
**Disk Monitoring:**
|
||||
- `DiskCheckIntervalMs(ms int64)`: Base disk check interval
|
||||
- `EnableAdaptiveInterval(enable bool)`: Adjust interval based on load
|
||||
- `MinCheckIntervalMs(ms int64)`: Minimum adaptive interval
|
||||
- `MaxCheckIntervalMs(ms int64)`: Maximum adaptive interval
|
||||
- `EnablePeriodicSync(enable bool)`: Periodic disk sync
|
||||
|
||||
**Error Handling:**
|
||||
- `InternalErrorsToStderr(enable bool)`: Send internal errors to stderr
|
||||
|
||||
## API Reference
|
||||
|
||||
### Logger Creation
|
||||
|
||||
```go
|
||||
func NewLogger() *Logger
|
||||
```
|
||||
Creates a new uninitialized logger with default configuration.
|
||||
|
||||
### Configuration Methods
|
||||
|
||||
```go
|
||||
func (l *Logger) ApplyConfig(cfg *Config) error
|
||||
func (l *Logger) ApplyConfigString(overrides ...string) error
|
||||
func (l *Logger) GetConfig() *Config
|
||||
```
|
||||
|
||||
### Lifecycle Methods
|
||||
|
||||
```go
|
||||
func (l *Logger) Start() error // Start log processing
|
||||
func (l *Logger) Stop(timeout ...time.Duration) error // Stop (can restart)
|
||||
func (l *Logger) Shutdown(timeout ...time.Duration) error // Terminal shutdown
|
||||
func (l *Logger) Flush(timeout time.Duration) error // Force buffer flush
|
||||
```
|
||||
|
||||
### Standard Logging Methods
|
||||
|
||||
```go
|
||||
func (l *Logger) Debug(args ...any) // Level -4
|
||||
func (l *Logger) Info(args ...any) // Level 0
|
||||
func (l *Logger) Warn(args ...any) // Level 4
|
||||
func (l *Logger) Error(args ...any) // Level 8
|
||||
```
|
||||
|
||||
### Trace Logging Methods
|
||||
|
||||
Include function call traces (depth 0-10):
|
||||
|
||||
```go
|
||||
func (l *Logger) DebugTrace(depth int, args ...any)
|
||||
func (l *Logger) InfoTrace(depth int, args ...any)
|
||||
func (l *Logger) WarnTrace(depth int, args ...any)
|
||||
func (l *Logger) ErrorTrace(depth int, args ...any)
|
||||
```
|
||||
|
||||
### Special Logging Methods
|
||||
|
||||
```go
|
||||
func (l *Logger) LogStructured(level int64, message string, fields map[string]any)
|
||||
func (l *Logger) Write(args ...any) // Raw output, no formatting
|
||||
func (l *Logger) Log(args ...any) // Timestamp only, no level
|
||||
func (l *Logger) Message(args ...any) // No timestamp or level
|
||||
func (l *Logger) LogTrace(depth int, args ...any) // Timestamp + trace, no level
|
||||
```
|
||||
|
||||
## Constants and Levels
|
||||
|
||||
### Standard Log Levels
|
||||
|
||||
```go
|
||||
const (
|
||||
LevelDebug int64 = -4 // Verbose debugging
|
||||
LevelInfo int64 = 0 // Informational messages
|
||||
LevelWarn int64 = 4 // Warning conditions
|
||||
LevelError int64 = 8 // Error conditions
|
||||
)
|
||||
```
|
||||
|
||||
### Heartbeat Monitoring Levels
|
||||
|
||||
Special levels that bypass filtering:
|
||||
|
||||
```go
|
||||
const (
|
||||
LevelProc int64 = 12 // Process statistics
|
||||
LevelDisk int64 = 16 // Disk usage statistics
|
||||
LevelSys int64 = 20 // System statistics
|
||||
)
|
||||
```
|
||||
|
||||
### Sanitization Policies
|
||||
|
||||
```go
|
||||
const (
|
||||
PolicyRaw = "raw" // No-op passthrough
|
||||
PolicyJSON = "json" // JSON-safe output
|
||||
PolicyTxt = "txt" // Text file safe
|
||||
PolicyShell = "shell" // Shell-safe output
|
||||
)
|
||||
```
|
||||
|
||||
### Level Helper
|
||||
|
||||
```go
|
||||
func Level(levelStr string) (int64, error)
|
||||
```
|
||||
Converts level string to numeric constant: "debug", "info", "warn", "error", "proc", "disk", "sys".
|
||||
|
||||
## Output Formats
|
||||
|
||||
### JSON Format
|
||||
|
||||
```json
|
||||
{"timestamp":"2024-01-01T12:00:00Z","level":"INFO","fields":["Application started","version","1.0.0"]}
|
||||
```
|
||||
|
||||
### TXT Format
|
||||
|
||||
```
|
||||
2024-01-01T12:00:00Z INFO Application started version="1.0.0" pid=1234
|
||||
```
|
||||
|
||||
### RAW Format
|
||||
|
||||
Minimal format without timestamps or levels:
|
||||
```
|
||||
Application started version="1.0.0" pid=1234
|
||||
Connection failed host="db.example.com" error="timeout"
|
||||
```
|
||||
|
||||
## Standalone Formatter/Sanitizer Packages
|
||||
|
||||
### Formatter Package
|
||||
|
||||
```go
|
||||
import (
|
||||
"time"
|
||||
"github.com/lixenwraith/log/formatter"
|
||||
"github.com/lixenwraith/log/sanitizer"
|
||||
)
|
||||
|
||||
// Create formatter with sanitizer
|
||||
s := sanitizer.New().Policy(sanitizer.PolicyJSON)
|
||||
f := formatter.New(s)
|
||||
|
||||
// Configure and format
|
||||
f.Type("json").ShowTimestamp(true)
|
||||
data := f.Format(
|
||||
formatter.FlagDefault,
|
||||
time.Now(),
|
||||
0, // Info level
|
||||
"", // No trace
|
||||
[]any{"User action", "user_id", 42},
|
||||
)
|
||||
```
|
||||
|
||||
### Sanitizer Package
|
||||
|
||||
```go
|
||||
import "github.com/lixenwraith/log/sanitizer"
|
||||
|
||||
// Predefined policy
|
||||
s := sanitizer.New().Policy(sanitizer.PolicyJSON)
|
||||
clean := s.Sanitize("hello\nworld") // "hello\\nworld"
|
||||
|
||||
// Custom rules
|
||||
s = sanitizer.New().
|
||||
Rule(sanitizer.FilterControl, sanitizer.TransformStrip).
|
||||
Rule(sanitizer.FilterNonPrintable, sanitizer.TransformHexEncode)
|
||||
```
|
||||
|
||||
## Framework Adapters (compat package)
|
||||
|
||||
### gnet v2 Adapter
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/lixenwraith/log"
|
||||
"github.com/lixenwraith/log/compat"
|
||||
"github.com/panjf2000/gnet/v2"
|
||||
)
|
||||
|
||||
// Create adapter
|
||||
adapter := compat.NewGnetAdapter(logger)
|
||||
|
||||
// Use with gnet
|
||||
gnet.Run(handler, "tcp://127.0.0.1:9000", gnet.WithLogger(adapter))
|
||||
```
|
||||
|
||||
### fasthttp Adapter
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/lixenwraith/log"
|
||||
"github.com/lixenwraith/log/compat"
|
||||
"github.com/valyala/fasthttp"
|
||||
)
|
||||
|
||||
// Create adapter
|
||||
adapter := compat.NewFastHTTPAdapter(logger)
|
||||
|
||||
// Use with fasthttp
|
||||
server := &fasthttp.Server{
|
||||
Handler: requestHandler,
|
||||
Logger: adapter,
|
||||
}
|
||||
```
|
||||
|
||||
### Adapter Builder Pattern
|
||||
|
||||
```go
|
||||
// Share logger across adapters
|
||||
builder := compat.NewBuilder().WithLogger(logger)
|
||||
|
||||
gnetAdapter, err := builder.BuildGnet()
|
||||
fasthttpAdapter, err := builder.BuildFastHTTP()
|
||||
|
||||
// Or create structured adapters
|
||||
structuredGnet, err := builder.BuildStructuredGnet()
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Service with Shared Logger
|
||||
|
||||
```go
|
||||
type Service struct {
|
||||
logger *log.Logger
|
||||
}
|
||||
|
||||
func NewService() (*Service, error) {
|
||||
logger, err := log.NewBuilder().
|
||||
Directory("/var/log/service").
|
||||
Format("json").
|
||||
BufferSize(2048).
|
||||
HeartbeatLevel(2).
|
||||
Build()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := logger.Start(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &Service{logger: logger}, nil
|
||||
}
|
||||
|
||||
func (s *Service) Close() error {
|
||||
return s.logger.Shutdown(5 * time.Second)
|
||||
}
|
||||
|
||||
func (s *Service) ProcessRequest(id string) {
|
||||
s.logger.Info("Processing", "request_id", id)
|
||||
// ... process ...
|
||||
s.logger.Info("Completed", "request_id", id)
|
||||
}
|
||||
```
|
||||
|
||||
### HTTP Middleware
|
||||
|
||||
```go
|
||||
func loggingMiddleware(logger *log.Logger) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
wrapped := &responseWriter{ResponseWriter: w, status: 200}
|
||||
|
||||
next.ServeHTTP(wrapped, r)
|
||||
|
||||
logger.Info("HTTP request",
|
||||
"method", r.Method,
|
||||
"path", r.URL.Path,
|
||||
"status", wrapped.status,
|
||||
"duration_ms", time.Since(start).Milliseconds(),
|
||||
"remote_addr", r.RemoteAddr,
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Hot Reconfiguration
|
||||
|
||||
```go
|
||||
// Initial configuration
|
||||
logger.ApplyConfigString("level=info")
|
||||
|
||||
// Debugging reconfiguration
|
||||
logger.ApplyConfigString(
|
||||
"level=debug",
|
||||
"heartbeat_level=3",
|
||||
"heartbeat_interval_s=60",
|
||||
)
|
||||
|
||||
// Revert to normal
|
||||
logger.ApplyConfigString(
|
||||
"level=info",
|
||||
"heartbeat_level=1",
|
||||
"heartbeat_interval_s=300",
|
||||
)
|
||||
```
|
||||
|
||||
### Security-Focused Sanitization
|
||||
|
||||
```go
|
||||
// User input logging with shell-safe sanitization
|
||||
userInput := getUserInput()
|
||||
s := sanitizer.New().Policy(sanitizer.PolicyShell)
|
||||
logger.Info("User command", "input", s.Sanitize(userInput))
|
||||
|
||||
// Or configure logger-wide
|
||||
logger.ApplyConfigString("sanitization=shell")
|
||||
```
|
||||
|
||||
### Graceful Shutdown
|
||||
|
||||
```go
|
||||
// Setup signal handling
|
||||
sigChan := make(chan os.Signal, 1)
|
||||
signal.Notify(sigChan, syscall.SIGTERM, syscall.SIGINT)
|
||||
|
||||
// Shutdown sequence
|
||||
<-sigChan
|
||||
logger.Info("Shutdown initiated")
|
||||
|
||||
// Flush pending logs with timeout
|
||||
if err := logger.Shutdown(5 * time.Second); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Logger shutdown error: %v\n", err)
|
||||
}
|
||||
```
|
||||
|
||||
## Thread Safety
|
||||
|
||||
All public methods are thread-safe. The logger uses:
|
||||
- Atomic operations for state management
|
||||
- Channels for log record passing
|
||||
- No locks in the critical logging path
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
- **Zero-allocation logging path**: Pre-allocated buffers
|
||||
- **Lock-free async design**: Non-blocking sends to buffered channel
|
||||
- **Adaptive disk checks**: Adjusts I/O based on load
|
||||
- **Batch writes**: Flushes buffer periodically, not per-record
|
||||
- **Drop tracking**: Counts dropped logs when buffer full
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### From standard log package
|
||||
|
||||
```go
|
||||
// Before: standard log
|
||||
log.Printf("User login: id=%d name=%s", id, name)
|
||||
|
||||
// After: lixenwraith/log
|
||||
logger.Info("User login", "id", id, "name", name)
|
||||
```
|
||||
|
||||
### From other structured loggers
|
||||
|
||||
```go
|
||||
// Before: zap
|
||||
zap.Info("User login",
|
||||
zap.Int("id", id),
|
||||
zap.String("name", name))
|
||||
|
||||
// After: lixenwraith/log
|
||||
logger.Info("User login", "id", id, "name", name)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use Builder pattern** for configuration - compile-time safety
|
||||
2. **Use structured logging** - consistent key-value pairs
|
||||
3. **Use appropriate levels** - filter noise in logs
|
||||
4. **Configure sanitization** - prevent log injection attacks
|
||||
5. **Monitor heartbeats** - track logger health in production
|
||||
6. **Handle shutdown** - always call Shutdown() to flush logs
|
||||
7. **Use standalone packages** - reuse formatter/sanitizer for other needs
|
||||
+12
-12
@@ -6,11 +6,11 @@ Comprehensive guide to log file rotation, retention policies, and disk space man
|
||||
|
||||
### Automatic Rotation
|
||||
|
||||
Log files are automatically rotated when they reach the configured size limit:
|
||||
Log files are automatically rotated when they reach the configured size limit in Kilobytes:
|
||||
|
||||
```go
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=100", // Rotate at 100MB
|
||||
"max_size_kb=102400", // Rotate at 100MB (102400 KB)
|
||||
)
|
||||
```
|
||||
|
||||
@@ -62,23 +62,23 @@ When limits are exceeded, the logger:
|
||||
```go
|
||||
// Conservative: Strict limits
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=500", // 500KB files
|
||||
"max_total_size_kb=5000", // 5MB total
|
||||
"min_disk_free_kb=1000000", // 1GB free required
|
||||
"max_size_kb=500", // 500 KB files
|
||||
"max_total_size_kb=5000", // 5 MB total log directory limit
|
||||
"min_disk_free_kb=1048576", // 1 GB free space required on disk
|
||||
)
|
||||
|
||||
// Generous: Large files, external archival
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=100000", // 100MB files
|
||||
"max_total_size_kb=0", // No total limit
|
||||
"min_disk_free_kb=10000", // 10MB free required
|
||||
"max_size_kb=102400", // 100 MB files
|
||||
"max_total_size_kb=0", // No total limit
|
||||
"min_disk_free_kb=10240", // 10 MB free required
|
||||
)
|
||||
|
||||
// Balanced: Production defaults
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=100000", // 100MB files
|
||||
"max_total_size_kb=5000000", // 5GB total
|
||||
"min_disk_free_kb=500000", // 500MB free required
|
||||
"max_size_kb=102400", // 100 MB files
|
||||
"max_total_size_kb=5242880", // 5 GB total limit
|
||||
"min_disk_free_kb=512000", // 500 MB free required
|
||||
)
|
||||
```
|
||||
|
||||
@@ -180,4 +180,4 @@ ls -t /var/log/myapp/*.log | tail -n 20 | xargs rm
|
||||
|
||||
# Verify space
|
||||
df -h /var/log
|
||||
```
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user