v0.1.11 harden logging lifecycle, sanitization, and formatter edge cases
Test / test (push) Canceled after 0s
Test / fuzz (./compat, FuzzParseFormat) (push) Canceled after 0s
Test / fuzz (./formatter, FuzzFormatJSON) (push) Canceled after 0s
Test / fuzz (./formatter, FuzzFormatTextPolicy) (push) Canceled after 0s
Test / fuzz (./sanitizer, FuzzSanitize) (push) Canceled after 0s
Test / fuzz (./sanitizer, FuzzSerializerJSON) (push) Canceled after 0s
Test / test (push) Canceled after 0s
Test / fuzz (./compat, FuzzParseFormat) (push) Canceled after 0s
Test / fuzz (./formatter, FuzzFormatJSON) (push) Canceled after 0s
Test / fuzz (./formatter, FuzzFormatTextPolicy) (push) Canceled after 0s
Test / fuzz (./sanitizer, FuzzSanitize) (push) Canceled after 0s
Test / fuzz (./sanitizer, FuzzSerializerJSON) (push) Canceled after 0s
This commit is contained in:
lixen committed
1 parent
79a81db3ca
commit
2417e51da1
46 files changed
+1846
-1289
No files matched your search
+75
-319
@@ -1,359 +1,112 @@
|
||||
# Compatibility Adapters
|
||||
# Compatibility adapters
|
||||
|
||||
Guide to using lixenwraith/log with popular Go networking frameworks through compatibility adapters.
|
||||
The `compat` package provides logging adapters for gnet v2, fasthttp, and Fiber v2. Framework dependencies are not imported by this module; applications supply them.
|
||||
|
||||
## Overview
|
||||
## Shared logger setup
|
||||
|
||||
The `compat` package provides adapters that allow the lixenwraith/log logger to work seamlessly with:
|
||||
|
||||
- **gnet v2**: High-performance event-driven networking framework
|
||||
- **fasthttp**: Fast HTTP implementation
|
||||
|
||||
### Features
|
||||
|
||||
- Full interface compatibility
|
||||
- Preserves structured logging
|
||||
- Configurable behavior
|
||||
- Shared logger instances
|
||||
- Optional field extraction
|
||||
|
||||
## gnet Adapter
|
||||
|
||||
### Basic Usage
|
||||
Configure and start the logger before passing an adapter to a framework:
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/lixenwraith/log"
|
||||
"github.com/lixenwraith/log/compat"
|
||||
"github.com/panjf2000/gnet/v2"
|
||||
)
|
||||
|
||||
// Create logger
|
||||
logger := log.NewLogger()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Directory = "/var/log/gnet"
|
||||
logger.ApplyConfig(cfg)
|
||||
defer logger.Shutdown()
|
||||
|
||||
// Create adapter
|
||||
adapter := compat.NewGnetAdapter(logger)
|
||||
|
||||
// Use with gnet
|
||||
gnet.Run(eventHandler, "tcp://127.0.0.1:9000",
|
||||
gnet.WithLogger(adapter),
|
||||
)
|
||||
```
|
||||
|
||||
### gnet Interface Implementation
|
||||
|
||||
The adapter implements all gnet logger methods:
|
||||
|
||||
```go
|
||||
type GnetAdapter struct {
|
||||
logger *log.Logger
|
||||
logger, err := log.NewBuilder().
|
||||
Directory("/var/log/service").
|
||||
EnableFile(true).
|
||||
Format("json").
|
||||
LevelString("info").
|
||||
Build()
|
||||
if err != nil { return err }
|
||||
if err := logger.Start(); err != nil {
|
||||
_ = logger.Shutdown()
|
||||
return err
|
||||
}
|
||||
defer logger.Shutdown(5 * time.Second)
|
||||
|
||||
// Methods implemented:
|
||||
// - Debugf(format string, args ...any)
|
||||
// - Infof(format string, args ...any)
|
||||
// - Warnf(format string, args ...any)
|
||||
// - Errorf(format string, args ...any)
|
||||
// - Fatalf(format string, args ...any)
|
||||
```
|
||||
|
||||
### Custom Fatal Behavior
|
||||
|
||||
Override default fatal handling:
|
||||
|
||||
```go
|
||||
adapter := compat.NewGnetAdapter(logger,
|
||||
compat.WithFatalHandler(func(msg string) {
|
||||
// Custom cleanup
|
||||
saveApplicationState()
|
||||
notifyOperations(msg)
|
||||
gracefulShutdown()
|
||||
os.Exit(1)
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
### Complete gnet Example
|
||||
|
||||
```go
|
||||
type echoServer struct {
|
||||
gnet.BuiltinEventEngine
|
||||
logger gnet.Logger
|
||||
}
|
||||
|
||||
func (es *echoServer) OnBoot(eng gnet.Engine) gnet.Action {
|
||||
es.logger.Infof("Server started on %s", eng.Addrs)
|
||||
return gnet.None
|
||||
}
|
||||
|
||||
func (es *echoServer) OnTraffic(c gnet.Conn) gnet.Action {
|
||||
buf, _ := c.Next(-1)
|
||||
es.logger.Debugf("Received %d bytes from %s", len(buf), c.RemoteAddr())
|
||||
c.Write(buf)
|
||||
return gnet.None
|
||||
}
|
||||
|
||||
func main() {
|
||||
logger := log.NewLogger()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Directory = "/var/log/gnet"
|
||||
cfg.Format = "json"
|
||||
cfg.BufferSize = 2048
|
||||
logger.ApplyConfig(cfg)
|
||||
defer logger.Shutdown()
|
||||
|
||||
adapter := compat.NewGnetAdapter(logger)
|
||||
|
||||
gnet.Run(
|
||||
&echoServer{logger: adapter},
|
||||
"tcp://127.0.0.1:9000",
|
||||
gnet.WithMulticore(true),
|
||||
gnet.WithLogger(adapter),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
## fasthttp Adapter
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/lixenwraith/log"
|
||||
"github.com/lixenwraith/log/compat"
|
||||
"github.com/valyala/fasthttp"
|
||||
)
|
||||
|
||||
// Create logger
|
||||
logger := log.NewLogger()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Directory = "/var/log/fasthttp"
|
||||
logger.ApplyConfig(cfg)
|
||||
defer logger.Shutdown()
|
||||
|
||||
// Create adapter
|
||||
adapter := compat.NewFastHTTPAdapter(logger)
|
||||
|
||||
// Configure server
|
||||
server := &fasthttp.Server{
|
||||
Handler: requestHandler,
|
||||
Logger: adapter,
|
||||
}
|
||||
```
|
||||
|
||||
### Level Detection
|
||||
|
||||
The adapter automatically detects log levels from message content:
|
||||
|
||||
```go
|
||||
// Default detection rules:
|
||||
// - Contains "error", "failed", "fatal", "panic" → ERROR
|
||||
// - Contains "warn", "warning", "deprecated" → WARN
|
||||
// - Contains "debug", "trace" → DEBUG
|
||||
// - Otherwise → INFO
|
||||
```
|
||||
|
||||
### Custom Level Detection
|
||||
|
||||
```go
|
||||
adapter := compat.NewFastHTTPAdapter(logger,
|
||||
compat.WithDefaultLevel(log.LevelInfo),
|
||||
compat.WithLevelDetector(func(msg string) int64 {
|
||||
// Custom detection logic
|
||||
if strings.Contains(msg, "CRITICAL") {
|
||||
return log.LevelError
|
||||
}
|
||||
if strings.Contains(msg, "performance") {
|
||||
return log.LevelWarn
|
||||
}
|
||||
// Return 0 to use the adapter's default log level (log.LevelInfo by default)
|
||||
return 0
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
## Builder Pattern
|
||||
|
||||
### Using Existing Logger (Recommended)
|
||||
|
||||
Share a configured logger across adapters:
|
||||
|
||||
```go
|
||||
// Create and configure your main logger
|
||||
logger := log.NewLogger()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Level = log.LevelDebug
|
||||
logger.ApplyConfig(cfg)
|
||||
logger.Start()
|
||||
defer logger.Shutdown()
|
||||
|
||||
// Create builder with existing logger
|
||||
builder := compat.NewBuilder().WithLogger(logger)
|
||||
|
||||
// Build adapters
|
||||
gnetAdapter, _ := builder.BuildGnet()
|
||||
gnetAdapter, err := builder.BuildGnet()
|
||||
if err != nil { return err }
|
||||
|
||||
fasthttpAdapter, _ := builder.BuildFastHTTP()
|
||||
fastHTTPAdapter, err := builder.BuildFastHTTP()
|
||||
if err != nil { return err }
|
||||
fiberAdapter, err := builder.BuildFiber()
|
||||
if err != nil { return err }
|
||||
```
|
||||
|
||||
### Creating New Logger
|
||||
Alternatively, `compat.NewBuilder().WithConfig(cfg)` creates an initialized logger on the first build. Retrieve it with `GetLogger()` and call `Start()` explicitly. `WithLogger` takes precedence over `WithConfig`. Builders and option application are single-goroutine operations; completed adapters may be shared with a shared Logger.
|
||||
|
||||
Let the builder create a logger with config:
|
||||
## gnet
|
||||
|
||||
`GnetAdapter` implements `Debugf`, `Infof`, `Warnf`, `Errorf`, and `Fatalf`. Pass it through `gnet.WithLogger(adapter)`:
|
||||
|
||||
```go
|
||||
// Option 1: With custom config
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Directory = "/var/log/app"
|
||||
builder := compat.NewBuilder().WithConfig(cfg)
|
||||
|
||||
// Option 2: Default config (created on first build)
|
||||
builder := compat.NewBuilder()
|
||||
if err != nil { return err }
|
||||
|
||||
// Build adapters
|
||||
gnetAdapter, _ := builder.BuildGnet()
|
||||
logger, _ := builder.GetLogger() // Retrieve for direct use
|
||||
adapter := compat.NewGnetAdapter(logger)
|
||||
err := gnet.Run(eventHandler, "tcp://127.0.0.1:9000", gnet.WithLogger(adapter))
|
||||
```
|
||||
|
||||
### Structured gnet Adapter
|
||||
Ordinary messages include `msg` and `source` (`gnet`) as positional arguments. With JSON output, these appear in the `fields` array. Use the logger's `LogStructured` or `LogContext` with `FlagKV` directly if a keyed JSON object is required.
|
||||
|
||||
Extract fields from printf-style formats:
|
||||
### Structured field extraction
|
||||
|
||||
```go
|
||||
structuredAdapter, _ := builder.BuildStructuredGnet()
|
||||
// "client=%s port=%d" → {"client": "...", "port": ...}
|
||||
```
|
||||
|
||||
## Structured Logging
|
||||
|
||||
### Field Extraction
|
||||
|
||||
Structured adapters can extract fields from printf-style formats:
|
||||
|
||||
```go
|
||||
// Regular adapter output:
|
||||
// "client=192.168.1.1 port=8080"
|
||||
|
||||
// Structured adapter output:
|
||||
// {"client": "192.168.1.1", "port": 8080, "source": "gnet"}
|
||||
```
|
||||
|
||||
### Pattern Detection
|
||||
|
||||
The structured adapter recognizes patterns like:
|
||||
- `key=%v`
|
||||
- `key: %v`
|
||||
- `key = %v`
|
||||
`NewStructuredGnetAdapter` extracts simple `key=%v` or `key: %v` expressions with unindexed, single-argument verbs:
|
||||
|
||||
```go
|
||||
adapter := compat.NewStructuredGnetAdapter(logger)
|
||||
|
||||
// These will extract structured fields:
|
||||
adapter.Infof("client=%s port=%d", "192.168.1.1", 8080)
|
||||
// → {"client": "192.168.1.1", "port": 8080}
|
||||
|
||||
adapter.Errorf("user: %s, error: %s", "john", "auth failed")
|
||||
// → {"user": "john", "error": "auth failed"}
|
||||
|
||||
// These remain as messages:
|
||||
adapter.Infof("Connected to server")
|
||||
// → {"msg": "Connected to server"}
|
||||
// JSON fields: ["client", "192.168.1.1", "port", 8080, "source", "gnet"]
|
||||
```
|
||||
|
||||
### Integration Examples
|
||||
Text outside the expressions is retained in `msg`. Formats mixing unrelated conversions, argument indexes, width/precision, escaped percent signs, or a different argument count fall back to a complete `fmt.Sprintf` message. Extraction never guesses which argument belongs to a key.
|
||||
|
||||
#### Microservice with Both Frameworks
|
||||
## fasthttp
|
||||
|
||||
`FastHTTPAdapter` implements the `Printf` method accepted by `fasthttp.Server.Logger`:
|
||||
|
||||
```go
|
||||
type Service struct {
|
||||
gnetAdapter *compat.GnetAdapter
|
||||
fasthttpAdapter *compat.FastHTTPAdapter
|
||||
logger *log.Logger
|
||||
}
|
||||
|
||||
func NewService() (*Service, error) {
|
||||
// Create and configure logger
|
||||
logger := log.NewLogger()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Directory = "/var/log/service"
|
||||
cfg.Format = "json"
|
||||
cfg.HeartbeatLevel = 2
|
||||
if err := logger.ApplyConfig(cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := logger.Start(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Create builder with the logger
|
||||
builder := compat.NewBuilder().WithLogger(logger)
|
||||
|
||||
// Build adapters
|
||||
gnetAdapter, err := builder.BuildGnet()
|
||||
if err != nil {
|
||||
logger.Shutdown()
|
||||
return nil, err
|
||||
}
|
||||
|
||||
fasthttpAdapter, err := builder.BuildFastHTTP()
|
||||
if err != nil {
|
||||
logger.Shutdown()
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &Service{
|
||||
gnetAdapter: gnetAdapter,
|
||||
fasthttpAdapter: fasthttpAdapter,
|
||||
logger: logger,
|
||||
}, nil
|
||||
adapter := compat.NewFastHTTPAdapter(logger)
|
||||
server := &fasthttp.Server{
|
||||
Handler: requestHandler,
|
||||
Logger: adapter,
|
||||
}
|
||||
```
|
||||
|
||||
#### Middleware Integration
|
||||
The default detector performs a case-insensitive substring check:
|
||||
|
||||
| Message contains | Level |
|
||||
|---|---|
|
||||
| `error`, `failed`, `fatal`, or `panic` | Error |
|
||||
| `warn` or `deprecated` | Warn |
|
||||
| `debug` or `trace` | Debug |
|
||||
| Otherwise | Configured default (Info initially) |
|
||||
|
||||
This heuristic does not interpret HTTP status numbers. Disable it with `WithLevelDetector(nil)`, or provide a custom function. A detector result of zero selects the configured default; nonzero custom levels are preserved.
|
||||
|
||||
```go
|
||||
// gnet middleware
|
||||
func loggingMiddleware(adapter *compat.GnetAdapter) gnet.EventHandler {
|
||||
return func(c gnet.Conn) gnet.Action {
|
||||
start := time.Now()
|
||||
addr := c.RemoteAddr()
|
||||
|
||||
// Process connection
|
||||
action := next(c)
|
||||
|
||||
adapter.Infof("conn_duration=%v remote=%s action=%v",
|
||||
time.Since(start), addr, action)
|
||||
|
||||
return action
|
||||
}
|
||||
}
|
||||
|
||||
// fasthttp middleware
|
||||
func requestLogger(adapter *compat.FastHTTPAdapter) fasthttp.RequestHandler {
|
||||
return func(ctx *fasthttp.RequestCtx) {
|
||||
start := time.Now()
|
||||
|
||||
// Process request
|
||||
next(ctx)
|
||||
|
||||
// Adapter will detect level from status
|
||||
adapter.Printf("method=%s path=%s status=%d duration=%v",
|
||||
ctx.Method(), ctx.Path(),
|
||||
ctx.Response.StatusCode(),
|
||||
time.Since(start))
|
||||
}
|
||||
}
|
||||
adapter := compat.NewFastHTTPAdapter(logger,
|
||||
compat.WithDefaultLevel(log.LevelWarn),
|
||||
compat.WithLevelDetector(nil),
|
||||
)
|
||||
```
|
||||
|
||||
## Fiber
|
||||
|
||||
`FiberAdapter` provides plain, printf-style (`Infof`), and key/value (`Infow`) logging methods, and implements `io.Writer` through `Write([]byte)`. Trace methods currently map to Debug with a `level`, `trace` marker. Use it directly in Fiber middleware:
|
||||
|
||||
```go
|
||||
adapter := compat.NewFiberAdapter(logger)
|
||||
app.Use(func(c *fiber.Ctx) error {
|
||||
err := c.Next()
|
||||
adapter.Infow("request", "method", c.Method(), "path", c.Path(),
|
||||
"status", c.Response().StatusCode())
|
||||
return err
|
||||
})
|
||||
```
|
||||
|
||||
## Fatal and panic behavior
|
||||
|
||||
Fatal methods enqueue an Error record, attempt a flush for 100 ms, and invoke the configured handler. Defaults call `os.Exit(1)` for fatal and `panic(msg)` for Fiber panic methods. These effects still occur when the log level suppresses the record. A timed-out or failed flush cannot guarantee that the final message was written.
|
||||
|
||||
Override behavior with `WithFatalHandler`, `WithFiberFatalHandler`, or `WithFiberPanicHandler`. A nil handler disables that action. Shut down the application/framework before the shared logger so shutdown messages can still be recorded.
|
||||
|
||||
### Simple integration example suite
|
||||
|
||||
Below simple client and server examples can be used to test the basic functionality of the adapters. They are not included in the package to avoid dependency creep.
|
||||
These client and server examples can be used to test the basic functionality of the adapters. They are not included in the package to avoid dependency creep.
|
||||
|
||||
|
||||
#### gnet server
|
||||
@@ -391,6 +144,7 @@ func main() {
|
||||
// Minimal logger config
|
||||
logger, err := log.NewBuilder().
|
||||
Directory("./logs_gnet").
|
||||
EnableFile(true).
|
||||
Format("json").
|
||||
LevelString("info").
|
||||
HeartbeatLevel(0).
|
||||
@@ -453,6 +207,7 @@ func main() {
|
||||
// Minimal logger config
|
||||
logger, err := log.NewBuilder().
|
||||
Directory("./logs_fasthttp").
|
||||
EnableFile(true).
|
||||
Format("json").
|
||||
LevelString("info").
|
||||
HeartbeatLevel(0).
|
||||
@@ -521,6 +276,7 @@ func main() {
|
||||
// Minimal logger config
|
||||
logger, err := log.NewBuilder().
|
||||
Directory("./logs_fiber").
|
||||
EnableFile(true).
|
||||
Format("json").
|
||||
LevelString("info").
|
||||
HeartbeatLevel(0).
|
||||
|
||||
+27
-9
@@ -37,7 +37,7 @@ Applies a validated configuration to the logger. This is the recommended method
|
||||
```go
|
||||
logger := log.NewLogger()
|
||||
|
||||
cfg := log.GetConfig()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Level = log.LevelDebug
|
||||
cfg.Directory = "/var/log/app"
|
||||
err := logger.ApplyConfig(cfg)
|
||||
@@ -66,7 +66,7 @@ err := logger.ApplyConfigString("directory=/var/log/app", "name=app")
|
||||
|
||||
## Logging Methods
|
||||
|
||||
All logging methods accept variadic arguments, typically used as key-value pairs for structured logging.
|
||||
Logging methods accept variadic arguments. Ordinary methods preserve them as positional values (a JSON array in JSON mode). Use `LogContext` with `FlagKV` for a keyed object, or `LogStructured` for a message and nested fields.
|
||||
|
||||
### Debug
|
||||
|
||||
@@ -217,7 +217,7 @@ logger.Log("Checkpoint reached", "step", 5)
|
||||
func (l *Logger) Message(args ...any)
|
||||
```
|
||||
|
||||
Logs raw message without timestamp or level.
|
||||
Logs using the configured format and sanitization, without timestamp or level.
|
||||
|
||||
**Example:**
|
||||
```go
|
||||
@@ -239,13 +239,21 @@ logger.LogTrace(2, "Function boundary", "entering", true)
|
||||
|
||||
## Control Methods
|
||||
|
||||
### Start and Stop
|
||||
|
||||
`Start() error` begins processing after configuration; repeated starts are no-ops. `Stop(timeout ...time.Duration) error` closes emission, drains accepted records, and joins the processor. A timeout does not permit a second processor to start while the first still owns resources. Retry `Stop` or `Shutdown` to wait again. Lifecycle timeouts bound the processor join after acquiring the lifecycle lock.
|
||||
|
||||
### Context and fast filtering
|
||||
|
||||
`Enabled(level int64) bool` checks the current emit gate before constructing expensive arguments. `SetLevel(level int64)` updates the threshold without restarting. `SetContextKeys(tag string, vals ...string)` names a tag and up to three counters; `LogContext(ctx Context, flags, level, depth int64, args ...any)` stamps them onto a record. `Flags()` returns display defaults suitable for combining with `FlagKV`.
|
||||
|
||||
### Shutdown
|
||||
|
||||
```go
|
||||
func (l *Logger) Shutdown(timeout ...time.Duration) error
|
||||
```
|
||||
|
||||
Gracefully shuts down the logger, attempting to flush pending logs.
|
||||
Gracefully drains accepted records and closes the log file. If the processor times out, its resources remain open; retry `Shutdown` to finish cleanup. A fresh `ApplyConfig` permits reuse after shutdown.
|
||||
|
||||
**Parameters:**
|
||||
- `timeout`: Optional timeout duration (defaults to 2x flush interval)
|
||||
@@ -267,7 +275,7 @@ if err != nil {
|
||||
func (l *Logger) Flush(timeout time.Duration) error
|
||||
```
|
||||
|
||||
Explicitly triggers a sync of the current log file buffer to disk. Uses **barrier semantics**: it guarantees that all log records enqueued *before* the `Flush` call are fully processed and formatted before the disk sync occurs and confirmation is returned.
|
||||
Drains records enqueued before the request and syncs the file. Later arrivals do not extend the drain indefinitely. The timeout covers both queueing and confirmation. A successful return reports successful sync; previously dropped records cannot be recovered. A concurrent stop may return a stopped-processor error.
|
||||
|
||||
**Parameters:**
|
||||
- `timeout`: Maximum time to wait for flush completion
|
||||
@@ -286,6 +294,7 @@ err := logger.Flush(1 * time.Second)
|
||||
|
||||
```go
|
||||
const (
|
||||
LevelTrace int64 = -8
|
||||
LevelDebug int64 = -4
|
||||
LevelInfo int64 = 0
|
||||
LevelWarn int64 = 4
|
||||
@@ -316,7 +325,7 @@ func Level(levelStr string) (int64, error)
|
||||
Converts level string to numeric constant.
|
||||
|
||||
**Parameters:**
|
||||
- `levelStr`: Level name ("debug", "info", "warn", "error", "proc", "disk", "sys")
|
||||
- `levelStr`: Level name ("trace", "debug", "info", "warn", "error", "proc", "disk", "sys")
|
||||
|
||||
**Returns:**
|
||||
- `int64`: Numeric level value
|
||||
@@ -337,6 +346,7 @@ const (
|
||||
FlagStructuredJSON = formatter.FlagStructuredJSON // Structured JSON output
|
||||
FlagNoTimestamp = formatter.FlagNoTimestamp // Suppress timestamp
|
||||
FlagNoLevel = formatter.FlagNoLevel // Suppress level
|
||||
FlagKV = formatter.FlagKV // Alternating string keys and values
|
||||
FlagDefault = formatter.FlagDefault // Default flags
|
||||
)
|
||||
```
|
||||
@@ -348,9 +358,9 @@ Control output formatting behavior. These flags are re-exported from the formatt
|
||||
```go
|
||||
const (
|
||||
PolicyRaw = sanitizer.PolicyRaw // No sanitization
|
||||
PolicyJSON = sanitizer.PolicyJSON // JSON-safe output
|
||||
PolicyJSON = sanitizer.PolicyJSON // Content control-character escapes
|
||||
PolicyTxt = sanitizer.PolicyTxt // Text file safe
|
||||
PolicyShell = sanitizer.PolicyShell // Shell-safe output
|
||||
PolicyShell = sanitizer.PolicyShell // Lossy filter, not shell quoting
|
||||
)
|
||||
```
|
||||
|
||||
@@ -376,7 +386,11 @@ The logger returns errors prefixed with "log: " for easy identification:
|
||||
|
||||
## Thread Safety
|
||||
|
||||
All public methods are thread-safe and can be called concurrently from multiple goroutines. The logger uses atomic operations and channels to ensure safe concurrent access without locks in the critical path.
|
||||
Logger methods support concurrent calls. Lifecycle and configuration changes are serialized. Enqueues hold a short read lock to prevent records being stranded during a stop. Logger, Formatter, Sanitizer, and Builder instances must not be copied after use.
|
||||
|
||||
`ApplyConfig` takes a snapshot. Emission copies the argument slice, and `LogStructured` also copies the top-level fields map. Referenced values (byte slices, nested maps/slices, pointers, Stringers, and custom marshalers) remain caller-owned: keep them immutable until a successful `Flush` or `Stop`, or pass an application-created snapshot. Do not mutate configuration while an `ApplyConfig` call is reading it.
|
||||
|
||||
Builders and standalone formatter/sanitizer configuration are not synchronized; finish configuration before sharing. `SetSpawn` callbacks must launch the function asynchronously and return promptly. Error handlers must return promptly and must not invoke lifecycle/configuration methods or `Flush`.
|
||||
|
||||
### Usage Pattern Example
|
||||
|
||||
@@ -396,6 +410,10 @@ func NewService() (*Service, error) {
|
||||
return nil, fmt.Errorf("logger init: %w", err)
|
||||
}
|
||||
|
||||
if err := logger.Start(); err != nil {
|
||||
_ = logger.Shutdown()
|
||||
return nil, err
|
||||
}
|
||||
return &Service{logger: logger}, nil
|
||||
}
|
||||
|
||||
|
||||
+3
-3
@@ -14,7 +14,7 @@ builder := log.NewBuilder()
|
||||
|
||||
## Builder Methods
|
||||
|
||||
All builder methods return `*Builder` for chaining. Errors are accumulated and returned by `Build()`.
|
||||
All builder methods return `*Builder` for chaining. Validation errors are returned by `Build()`. Builders are not safe for concurrent mutation; each built logger owns a configuration snapshot.
|
||||
|
||||
### Common Methods
|
||||
|
||||
@@ -25,7 +25,7 @@ All builder methods return `*Builder` for chaining. Errors are accumulated and r
|
||||
| `Name(name string)` | `name`: Base filename | Sets log file base name |
|
||||
| `Directory(dir string)` | `dir`: Path | Sets log directory |
|
||||
| `Format(format string)` | `format`: Output format | Sets format ("txt", "json", "raw") |
|
||||
| `Sanitization(policy string)` | `policy`: Sanitization policy | Sets policy ("txt", "json", "raw", "shell") |
|
||||
| `Sanitization(policy sanitizer.PolicyPreset)` | `policy`: Sanitization policy | Sets policy ("txt", "json", "raw", "shell") |
|
||||
| `Extension(ext string)` | `ext`: File extension | Sets log file extension |
|
||||
| `BufferSize(size int64)` | `size`: Buffer size | Sets channel buffer size |
|
||||
| `MaxSizeKB(size int64)` | `size`: Size in KB | Sets max file size in KB |
|
||||
@@ -36,7 +36,7 @@ All builder methods return `*Builder` for chaining. Errors are accumulated and r
|
||||
| `MinDiskFreeMB(size int64)` | `size`: Size in MB | Sets minimum required free disk space in MB |
|
||||
| `EnableConsole(enable bool)` | `enable`: Boolean | Enables console output |
|
||||
| `EnableFile(enable bool)` | `enable`: Boolean | Enables file output |
|
||||
| `ConsoleTarget(target string)` | `target`: "stdout"/"stderr" | Sets console output target |
|
||||
| `ConsoleTarget(target string)` | `target`: "stdout"/"stderr"/"split" | Sets console output target |
|
||||
| `ShowTimestamp(show bool)` | `show`: Boolean | Controls timestamp display |
|
||||
| `ShowLevel(show bool)` | `show`: Boolean | Controls log level display |
|
||||
| `TimestampFormat(format string)` | `format`: Time format | Sets timestamp format (Go time format) |
|
||||
|
||||
+17
-20
@@ -17,39 +17,36 @@ logger := log.NewLogger()
|
||||
Direct struct configuration using the Config struct, or key-value overrides:
|
||||
|
||||
```go
|
||||
logger := log.NewLogger() // logger instance created with DefaultConfig (using default values)
|
||||
|
||||
// Note: with default config, logs only go to stderr (file output disabled by default)
|
||||
logger.Start() // Required before logging
|
||||
logger.Info("info raw log record written to stderr")
|
||||
|
||||
// Directly change config struct
|
||||
cfg := log.GetConfig()
|
||||
logger := log.NewLogger()
|
||||
cfg := log.DefaultConfig()
|
||||
cfg.Level = log.LevelDebug
|
||||
cfg.Name = "myapp"
|
||||
cfg.Directory = "/var/log/myapp"
|
||||
cfg.EnableFile = true
|
||||
cfg.Format = "json"
|
||||
cfg.MaxSizeKB = 100
|
||||
err := logger.ApplyConfig(cfg)
|
||||
if err := logger.ApplyConfig(cfg); err != nil { return err }
|
||||
if err := logger.Start(); err != nil { return err }
|
||||
defer logger.Shutdown()
|
||||
logger.Info("application started")
|
||||
|
||||
logger.Info("info json log record written to /var/log/myapp/myapp.log")
|
||||
|
||||
// Override values with key-value string
|
||||
err = logger.ApplyConfigString(
|
||||
"directory=/var/log/",
|
||||
"extension=txt"
|
||||
"format=txt")
|
||||
|
||||
logger.Info("info txt log record written to /var/log/myapp.txt")
|
||||
// Apply overrides to the current configuration atomically.
|
||||
if err := logger.ApplyConfigString("extension=txt", "format=txt"); err != nil {
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
`ApplyConfig` copies the supplied configuration. The caller may reuse or modify it after the call returns. Failed validation, directory creation, or file opening leaves the active configuration unchanged. Changes requiring a restart drain accepted records using the old file and configuration before publishing the replacement. Calls made while emission is stopped are ignored; a full queue drops records.
|
||||
|
||||
Names and extensions must be filename components, without path separators or NUL. The queue size must be between 1 and 1,048,576 records. Numeric settings are checked for byte-count and duration overflow; positive fractional retention settings must resolve to at least one nanosecond. Size settings use decimal units: 1 KB = 1,000 bytes and 1 MB = 1,000 KB.
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
### Basic Settings
|
||||
|
||||
| Parameter | Type | Description | Default |
|
||||
|-----------|------|-------------|------------|
|
||||
| `level` | `int64` | Minimum log level (-4=Debug, 0=Info, 4=Warn, 8=Error) | `0` |
|
||||
| `level` | `int64` | Minimum log level (-8=Trace, -4=Debug, 0=Info, 4=Warn, 8=Error) | `0` |
|
||||
| `name` | `string` | Base name for log files | `"log"` |
|
||||
| `extension` | `string` | Log file extension (without dot) | `"log"` |
|
||||
| `directory` | `string` | Directory to store log files | `"./log"` |
|
||||
@@ -66,7 +63,7 @@ logger.Info("info txt log record written to /var/log/myapp.txt")
|
||||
| `show_level` | `bool` | Include log level in entries | `true` |
|
||||
| `enable_console` | `bool` | Enable console output (stdout/stderr) | `true` |
|
||||
| `console_target` | `string` | Console target: `"stdout"`, `"stderr"`, or `"split"` | `"stderr"` |
|
||||
| `enable_file` | `bool` | Enable file output (console-only) | `false` |
|
||||
| `enable_file` | `bool` | Enable file output | `false` |
|
||||
|
||||
**Note:** When `console_target="split"`, INFO/DEBUG logs go to stdout while WARN/ERROR logs go to stderr.
|
||||
|
||||
|
||||
+34
-13
@@ -65,6 +65,7 @@ const (
|
||||
FlagStructuredJSON int64 = 0b1000 // Use structured JSON with message/fields
|
||||
FlagNoTimestamp int64 = 0b010000 // Suppress timestamp
|
||||
FlagNoLevel int64 = 0b100000 // Suppress level
|
||||
FlagKV int64 = 0b1000000 // Alternating string keys and values
|
||||
FlagDefault = FlagShowTimestamp | FlagShowLevel
|
||||
)
|
||||
```
|
||||
@@ -105,9 +106,9 @@ clean = s.Sanitize("cmd; echo test") // "cmd echo test"
|
||||
```go
|
||||
const (
|
||||
PolicyRaw PolicyPreset = "raw" // No-op passthrough
|
||||
PolicyJSON PolicyPreset = "json" // JSON-safe strings
|
||||
PolicyJSON PolicyPreset = "json" // Visible control-character escapes
|
||||
PolicyTxt PolicyPreset = "txt" // Text file safe
|
||||
PolicyShell PolicyPreset = "shell" // Shell command safe
|
||||
PolicyShell PolicyPreset = "shell" // Lossy filter, not shell quoting
|
||||
)
|
||||
```
|
||||
|
||||
@@ -171,7 +172,26 @@ serializer.WriteNil(&buf) // "null"
|
||||
|
||||
The sanitizer is a content transform; JSON string escaping is transport
|
||||
encoding applied afterward, unconditionally. Output is valid JSON for any
|
||||
sanitization policy. Multi-byte UTF-8 passes through unescaped.
|
||||
sanitization policy. Multi-byte UTF-8 passes through unescaped. Malformed UTF-8
|
||||
is replaced before rule-based sanitization, preventing stripped text from joining
|
||||
invalid byte fragments into control characters. A sanitizer with no rules remains
|
||||
byte-preserving; the JSON transport layer always replaces malformed UTF-8.
|
||||
|
||||
JSON keys and custom timestamp layouts are escaped. Non-finite scalar floats
|
||||
are emitted as strings (`"NaN"`, `"+Inf"`, `"-Inf"`); non-finite values in structured
|
||||
maps follow `encoding/json` and produce a marshal error. As in the existing API,
|
||||
`int32` values are treated as runes; cast to `int64` when numeric output is needed.
|
||||
Choose unique context and KV keys; avoid context keys named `time`, `level`,
|
||||
`trace`, `message`, or `fields`. Content policies can make distinct keys identical.
|
||||
|
||||
`PolicyTxt` covers text timestamps, trace, context, KV keys, and values. `PolicyRaw`
|
||||
is intentionally unfiltered, and `FlagRaw` bypasses formatting and sanitization.
|
||||
Generic complex-value text formatting is bounded to 64 levels and 10,000 visited
|
||||
values; cycles and larger graphs produce `<cyclic or oversized value>`. This also
|
||||
applies to complex values under `FlagRaw`, which use `%+v` text formatting.
|
||||
Custom formatting and marshal callbacks must terminate and be safe for shared use.
|
||||
A custom rule with no recognized transform preserves its matched rune. Predicate
|
||||
callbacks must be safe for concurrent use when the sanitizer is shared.
|
||||
|
||||
- `format=json` + `sanitization=raw`: recommended; transport escaping only.
|
||||
- `format=json` + `sanitization=txt`: non-printables appear as `<XX>` inside
|
||||
@@ -179,8 +199,9 @@ sanitization policy. Multi-byte UTF-8 passes through unescaped.
|
||||
- `format=json` + `sanitization=json`: redundant; produces visible `\\n`
|
||||
double escapes. Use `raw` instead.
|
||||
- Structured JSON (`FlagStructuredJSON`) marshals the fields map via
|
||||
`encoding/json` and bypasses the sanitizer; validity is guaranteed,
|
||||
content-level sanitization is not applied to field values.
|
||||
`encoding/json`, then applies the content policy to every string, including
|
||||
nested values and object keys. Value types and deterministic map order are preserved.
|
||||
Marshal failures produce a valid `fields._marshal_error` string.
|
||||
|
||||
## PolicyShell Scope
|
||||
|
||||
@@ -215,13 +236,13 @@ logger := log.NewLogger()
|
||||
// Configure sanitization policy
|
||||
logger.ApplyConfigString(
|
||||
"format=json",
|
||||
"sanitization=json", // Uses PolicyJSON
|
||||
"sanitization=raw", // JSON transport escaping remains enabled
|
||||
)
|
||||
|
||||
// Or with custom formatter (advanced)
|
||||
s := sanitizer.New().Policy(sanitizer.PolicyShell)
|
||||
customFormatter := formatter.New(s).Type("txt")
|
||||
// Note: Direct formatter injection requires using lower-level APIs
|
||||
// Use customFormatter directly; Logger has no formatter injection API.
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
@@ -247,7 +268,7 @@ f := formatter.New()
|
||||
f.Type("json").ShowTimestamp(false).ShowLevel(false)
|
||||
|
||||
// Create custom log entry
|
||||
entry := f.FormatArgs("action", "purchase", "amount", 99.99)
|
||||
entry := f.Format(formatter.FlagKV, time.Now(), 0, "", []any{"action", "purchase", "amount", 99.99})
|
||||
sendToExternalSystem(entry)
|
||||
```
|
||||
|
||||
@@ -262,17 +283,17 @@ shellSanitizer := sanitizer.New().Policy(sanitizer.PolicyShell)
|
||||
jsonFormatter := formatter.New(jsonSanitizer).Type("json")
|
||||
apiLog := jsonFormatter.Format(...)
|
||||
|
||||
// For shell script generation
|
||||
// For a filtered text log
|
||||
txtFormatter := formatter.New(shellSanitizer).Type("txt")
|
||||
scriptLog := txtFormatter.Format(...)
|
||||
textLog := txtFormatter.Format(...)
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- Both packages use pre-allocated buffers for efficiency
|
||||
- Sanitizer rules are applied in a single pass
|
||||
- Formatter append methods reuse caller-provided buffers
|
||||
- Sanitizer avoids allocations when no rule matches valid UTF-8
|
||||
- Formatter reuses internal buffers via `Reset()`
|
||||
- No regex or reflection in hot paths
|
||||
- Scalar formatting avoids reflection; complex values and structured marshaling may use it
|
||||
|
||||
## Ownership and Thread Safety
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ This guide will help you get started with the lixenwraith/log package, from inst
|
||||
|
||||
## Installation
|
||||
|
||||
Install the logger package:
|
||||
Requires Go 1.27.1 or later. Install the logger package:
|
||||
|
||||
```bash
|
||||
go get github.com/lixenwraith/log
|
||||
@@ -74,7 +74,7 @@ func main() {
|
||||
Directory("/var/log/myapp"). // Log directory path
|
||||
LevelString("info"). // Minimum log level
|
||||
Format("json"). // Output format
|
||||
Sanitization("json"). // Sanitization policy
|
||||
Sanitization(log.PolicyRaw). // Sanitization policy
|
||||
EnableFile(true). // Enable file output (disabled by default)
|
||||
BufferSize(2048). // Channel buffer size
|
||||
MaxSizeMB(10). // Max file size before rotation
|
||||
@@ -102,8 +102,8 @@ func main() {
|
||||
## Next Steps
|
||||
|
||||
1. **[Learn about configuration options](configuration.md)** - Customize behavior for your needs
|
||||
2. **[Explore the API](api-reference.md)** - See all available methods
|
||||
3. **[Logging patterns and examples](logging-guide.md)** - Write better logs
|
||||
2. **[Explore the API](api.md)** - See all available methods
|
||||
3. **[Logging patterns and examples](logging.md)** - Write better logs
|
||||
|
||||
## Common Patterns
|
||||
|
||||
@@ -125,7 +125,10 @@ func NewService() (*Service, error) {
|
||||
return nil, fmt.Errorf("logger init failed: %w", err)
|
||||
}
|
||||
|
||||
logger.Start()
|
||||
if err := logger.Start(); err != nil {
|
||||
_ = logger.Shutdown()
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &Service{
|
||||
logger: logger,
|
||||
|
||||
+18
-15
@@ -8,7 +8,7 @@ Heartbeats are periodic log messages that provide operational statistics about t
|
||||
|
||||
### Key Features
|
||||
|
||||
- **Always Visible**: Heartbeats use special log levels that bypass filtering
|
||||
- **Independent of level filtering**: Heartbeats can still be dropped on queue overflow or output failure
|
||||
- **Multi-Level Detail**: Choose from process, disk, or system statistics
|
||||
- **Production Monitoring**: Track logger health without debug logs
|
||||
- **Metrics Source**: Parse heartbeats for monitoring dashboards
|
||||
@@ -38,14 +38,15 @@ logger.ApplyConfigString(
|
||||
|
||||
**Output:**
|
||||
```
|
||||
2024-01-15T10:30:00Z PROC type="proc" sequence=1 uptime_hours="24.50" processed_logs=1847293 dropped_logs=0
|
||||
2024-01-15T10:30:00Z PROC type=proc sequence=1 uptime_hours="24.50" processed_logs=1847293 total_dropped_logs=0
|
||||
```
|
||||
|
||||
**Fields:**
|
||||
- `sequence`: Incrementing counter
|
||||
- `uptime_hours`: Logger uptime
|
||||
- `processed_logs`: Successfully written logs
|
||||
- `dropped_logs`: Logs lost due to buffer overflow
|
||||
- `processed_logs`: Application records processed successfully, excluding heartbeats
|
||||
- `total_dropped_logs`: Cumulative queue and output-processing drops, including heartbeats
|
||||
- `dropped_since_last`: Interval drops, emitted only when non-zero; reset at each PROC heartbeat attempt
|
||||
|
||||
### Level 2: Process + Disk Statistics (DISK)
|
||||
|
||||
@@ -60,7 +61,7 @@ logger.ApplyConfigString(
|
||||
|
||||
**Additional Output:**
|
||||
```
|
||||
2024-01-15T10:30:00Z DISK type="disk" sequence=1 rotated_files=12 deleted_files=5 total_log_size_mb="487.32" log_file_count=8 current_file_size_mb="23.45" disk_status_ok=true disk_free_mb="5234.67"
|
||||
2024-01-15T10:30:00Z DISK type=disk sequence=1 rotated_files=12 deleted_files=5 total_log_size_mb="487.32" log_file_count=8 current_file_size_mb="23.45" disk_status_ok=true disk_free_mb="5234.67"
|
||||
```
|
||||
|
||||
**Additional Fields:**
|
||||
@@ -85,7 +86,7 @@ logger.ApplyConfigString(
|
||||
|
||||
**Additional Output:**
|
||||
```
|
||||
2024-01-15T10:30:00Z SYS type="sys" sequence=1 alloc_mb="45.23" sys_mb="128.45" num_gc=1523 num_goroutine=42
|
||||
2024-01-15T10:30:00Z SYS type=sys sequence=1 alloc_mb="45.23" sys_mb="128.45" num_gc=1523 num_goroutine=42
|
||||
```
|
||||
|
||||
**Additional Fields:**
|
||||
@@ -146,13 +147,13 @@ With `format=json`, heartbeats are structured for easy parsing:
|
||||
{
|
||||
"time": "2024-01-15T10:30:00.123456789Z",
|
||||
"level": "PROC",
|
||||
"fields": [
|
||||
"type", "proc",
|
||||
"sequence", 42,
|
||||
"uptime_hours", "24.50",
|
||||
"processed_logs", 1847293,
|
||||
"dropped_logs", 0
|
||||
]
|
||||
"fields": {
|
||||
"type": "proc",
|
||||
"sequence": 42,
|
||||
"uptime_hours": "24.50",
|
||||
"processed_logs": 1847293,
|
||||
"total_dropped_logs": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -161,5 +162,7 @@ With `format=json`, heartbeats are structured for easy parsing:
|
||||
With `format=txt`, heartbeats are human-readable:
|
||||
|
||||
```
|
||||
2024-01-15T10:30:00.123456789Z PROC type="proc" sequence=42 uptime_hours="24.50" processed_logs=1847293 dropped_logs=0
|
||||
```
|
||||
2024-01-15T10:30:00.123456789Z PROC type=proc sequence=42 uptime_hours="24.50" processed_logs=1847293 total_dropped_logs=0
|
||||
```
|
||||
|
||||
PROC uptime and totals span the lifetime of the Logger instance, including stop/start cycles. DISK fields ending in `_mb` retain historical binary MiB units (1,048,576 bytes); SYS memory fields use decimal MB. Configuration size limits use decimal KB.
|
||||
+8
-8
@@ -44,7 +44,7 @@ logger.ApplyConfigString("level=8") // Error only
|
||||
|
||||
### Key-Value Pairs
|
||||
|
||||
Use structured key-value pairs for machine-parseable logs:
|
||||
Ordinary methods emit positional arguments (JSON arrays). Use `LogStructured`, or `LogContext` with `FlagKV`, when named JSON fields are required:
|
||||
|
||||
```go
|
||||
logger.Info("User login",
|
||||
@@ -117,9 +117,9 @@ func logWithContext(ctx context.Context, logger *log.Logger, level string, msg s
|
||||
|
||||
switch level {
|
||||
case "info":
|
||||
logger.Info(msg, fields...)
|
||||
logger.Info(append([]any{msg}, fields...)...)
|
||||
case "error":
|
||||
logger.Error(msg, fields...)
|
||||
logger.Error(append([]any{msg}, fields...)...)
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -130,11 +130,11 @@ The logger supports three output formats, each with configurable sanitization. T
|
||||
|
||||
### Txt Format (Human-Readable)
|
||||
|
||||
Default format for development and debugging:
|
||||
Text format for development and debugging (select `format=txt`):
|
||||
|
||||
```
|
||||
2024-01-15T10:30:45.123456789Z INFO User login user_id=42 email=user@example.com ip=192.168.1.100
|
||||
2024-01-15T10:30:45.234567890Z WARN Rate limit approaching user_id=42 requests=95 limit=100
|
||||
2024-01-15T10:30:45.123456789Z INFO "User login" user_id 42 email user@example.com ip 192.168.1.100
|
||||
2024-01-15T10:30:45.234567890Z WARN "Rate limit approaching" user_id 42 requests 95 limit 100
|
||||
```
|
||||
|
||||
Note: The txt format applies the configured sanitization policy (default: raw). Non-printable characters can be hex-encoded using `sanitization=txt` configuration.
|
||||
@@ -204,8 +204,8 @@ func processPayment(amount float64) error {
|
||||
|
||||
Output includes function names:
|
||||
```
|
||||
2024-01-15T10:30:45.123456789Z INFO processPayment Processing payment amount=99.99
|
||||
2024-01-15T10:30:45.234567890Z ERROR validateAmount -> processPayment -> main Payment validation failed amount=-10 error="negative amount"
|
||||
2024-01-15T10:30:45.123456789Z INFO processPayment "Processing payment" amount 99.99
|
||||
2024-01-15T10:30:45.234567890Z ERROR main -> processPayment "Payment validation failed" amount -10 error "negative amount"
|
||||
```
|
||||
|
||||
### Trace Depth Guidelines
|
||||
|
||||
+17
-28
@@ -10,31 +10,19 @@ Log files are automatically rotated when they reach the configured size limit in
|
||||
|
||||
```go
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=102400", // Rotate at 100MB (102400 KB)
|
||||
"max_size_kb=100000", // Rotate at 100 MB (100,000 KB)
|
||||
)
|
||||
```
|
||||
|
||||
### Rotation Behavior
|
||||
|
||||
1. **Size Check**: Before each write, the logger checks if the file would exceed `max_size_kb`
|
||||
2. **New File Creation**: Creates a new file with timestamp: `appname_240115_103045_123456789.log`
|
||||
3. **Seamless Transition**: No logs are lost during rotation
|
||||
4. **Old File Closure**: Previous file is properly closed and synced
|
||||
1. Before each write, the logger checks whether the active file would exceed `max_size_kb`.
|
||||
2. It closes the active `{name}.{extension}` file and renames it to `{name}_{YYMMDD}_{HHMMSS}[_N].{extension}`.
|
||||
3. It opens a new active file. A counter suffix avoids existing names when rotations share a second.
|
||||
|
||||
### File Naming Convention
|
||||
For example: `myapp.log` becomes `myapp_240115_143022.log`, then `myapp_240115_143022_1.log` on the next collision. Rotation does not split a record, so a single oversized record can exceed the size limit. File errors can drop records and are counted. Periodic sync, explicit `Flush`, and shutdown provide sync points; closing a file alone does not guarantee crash durability.
|
||||
|
||||
```
|
||||
{name}_{YYMMDD}_{HHMMSS}_{nanoseconds}.{extension}
|
||||
|
||||
Example: myapp_240115_143022_987654321.log
|
||||
```
|
||||
|
||||
Components:
|
||||
- `name`: Configured log name
|
||||
- `YYMMDD`: Date (year, month, day)
|
||||
- `HHMMSS`: Time (hour, minute, second)
|
||||
- `nanoseconds`: For uniqueness
|
||||
- `extension`: Configured extension
|
||||
Use a directory owned exclusively by this logger. Cleanup and retention select regular files by extension, including files from earlier runs with different base names. They exclude the active filename, directories, and symlinks. With an empty extension, only extensionless files match. Console-only configurations never perform retention deletion. Total-size and free-space limits are checked periodically and can be exceeded between checks.
|
||||
|
||||
## Disk Space Management
|
||||
|
||||
@@ -45,7 +33,7 @@ The logger enforces two types of space limits:
|
||||
```go
|
||||
logger.ApplyConfigString(
|
||||
"max_total_size_kb=1000", // Total log directory size
|
||||
"min_disk_free_kb=5000", // Minimum free disk space
|
||||
"min_disk_free_mb=5000", // Minimum free disk space
|
||||
)
|
||||
```
|
||||
|
||||
@@ -55,7 +43,7 @@ When limits are exceeded, the logger:
|
||||
1. Identifies oldest log files
|
||||
2. Deletes them until space requirements are met
|
||||
3. Preserves the current active log file
|
||||
4. Logs cleanup actions for audit
|
||||
4. Increments deletion counters reported in disk heartbeats
|
||||
|
||||
### Example Configuration
|
||||
|
||||
@@ -64,21 +52,21 @@ When limits are exceeded, the logger:
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=500", // 500 KB files
|
||||
"max_total_size_kb=5000", // 5 MB total log directory limit
|
||||
"min_disk_free_kb=1048576", // 1 GB free space required on disk
|
||||
"min_disk_free_mb=1000000", // 1 GB free space required on disk
|
||||
)
|
||||
|
||||
// Generous: Large files, external archival
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=102400", // 100 MB files
|
||||
"max_size_kb=100000", // 100 MB files
|
||||
"max_total_size_kb=0", // No total limit
|
||||
"min_disk_free_kb=10240", // 10 MB free required
|
||||
"min_disk_free_mb=10000", // 10 MB free required
|
||||
)
|
||||
|
||||
// Balanced: Production defaults
|
||||
logger.ApplyConfigString(
|
||||
"max_size_kb=102400", // 100 MB files
|
||||
"max_total_size_kb=5242880", // 5 GB total limit
|
||||
"min_disk_free_kb=512000", // 500 MB free required
|
||||
"max_size_kb=100000", // 100 MB files
|
||||
"max_total_size_kb=5000000", // 5 GB total limit
|
||||
"min_disk_free_mb=500000", // 500 MB free required
|
||||
)
|
||||
```
|
||||
|
||||
@@ -161,7 +149,7 @@ logger.ApplyConfigString(
|
||||
|
||||
Output:
|
||||
```
|
||||
2024-01-15T10:30:00Z DISK type="disk" sequence=1 rotated_files=5 deleted_files=2 total_log_size_kb="487.32" log_file_count=8 current_file_size_kb="23.45" disk_status_ok=true disk_free_kb="5234.67"
|
||||
2024-01-15T10:30:00Z DISK type="disk" sequence=1 rotated_files=5 deleted_files=2 total_log_size_mb="487.32" log_file_count=8 current_file_size_mb="23.45" disk_status_ok=true disk_free_mb="5234.67"
|
||||
```
|
||||
|
||||
## Manual Recovery
|
||||
@@ -176,7 +164,8 @@ df -h /var/log
|
||||
find /var/log/myapp -name "*.log" -size +100M
|
||||
|
||||
# Manual cleanup (oldest first)
|
||||
ls -t /var/log/myapp/*.log | tail -n 20 | xargs rm
|
||||
# Review archive paths before deleting; preserve the active log file.
|
||||
find /var/log/myapp -maxdepth 1 -type f -name 'myapp_*.log' -print
|
||||
|
||||
# Verify space
|
||||
df -h /var/log
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Testing and performance
|
||||
|
||||
Use Go 1.27.1 or later. The module has no external dependencies.
|
||||
|
||||
```sh
|
||||
go vet ./...
|
||||
go test -race -count=1 -timeout=3m ./...
|
||||
go test -race -run 'TestStress' -count=10 -timeout=3m .
|
||||
```
|
||||
|
||||
`stress_test.go` uses fixed record counts and synchronized starts. It verifies exact queue overflow, unique IDs, processed-plus-dropped accounting across lifecycle changes, and byte preservation across rotations. It has no throughput pass threshold and no timed soak loop. Heartbeat and adaptive-timer tests use `testing/synctest` or explicit state transitions instead of wall-clock sleeps. Run benchmarks separately when comparing performance:
|
||||
|
||||
```sh
|
||||
go test ./... -run '^$' -bench . -benchmem -count=5
|
||||
```
|
||||
|
||||
Fuzz targets test malformed UTF-8, control characters, keys, custom timestamp layouts, numeric edge cases, structured marshal failures, and printf argument association. Each target runs independently:
|
||||
|
||||
```sh
|
||||
go test ./sanitizer -run '^$' -fuzz '^FuzzSanitize$' -fuzztime=30s -parallel=2
|
||||
go test ./sanitizer -run '^$' -fuzz '^FuzzSerializerJSON$' -fuzztime=30s -parallel=2
|
||||
go test ./formatter -run '^$' -fuzz '^FuzzFormatJSON$' -fuzztime=30s -parallel=2
|
||||
go test ./formatter -run '^$' -fuzz '^FuzzFormatTextPolicy$' -fuzztime=30s -parallel=2
|
||||
go test ./compat -run '^$' -fuzz '^FuzzParseFormat$' -fuzztime=30s -parallel=2
|
||||
```
|
||||
|
||||
Fuzzing is bounded by time in CI; execution counts depend on the runner. All committed seeds run during ordinary `go test`, including minimized failures retained under `testdata/fuzz`.
|
||||
|
||||
The GitHub workflow runs on pushes and pull requests. It checks formatting, vet, race detection, repeated lifecycle/stress workloads, bounded fuzzing, and compilation for FreeBSD, Windows, and browser WASM. Runtime tests execute on Linux; cross-compilation does not validate other operating systems' runtime behavior.
|
||||
Reference in new issue
Block a user