diff --git a/.gitignore b/.gitignore index 602ecb9..ca0f6df 100644 --- a/.gitignore +++ b/.gitignore @@ -9,6 +9,7 @@ script/ build/ *.log *.toml +!config/*.toml build.sh catalog.txt combined.txt diff --git a/README.md b/README.md index ac73015..48771da 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@

LogWisp

- Go + Go License Documentation

@@ -16,79 +16,128 @@ # LogWisp -A high-performance, pipeline-based log transport and processing system built in Go. LogWisp provides flexible log collection, filtering, formatting, and distribution with enterprise-grade security and reliability features. +A pipeline-based log transport and processing system written in Go. LogWisp +collects log entries from files, stdin, or other LogWisp nodes; rate-limits, +filters, and formats them; and distributes them to files, consoles, live network +streams, or downstream LogWisp nodes. ## Features -### Core Capabilities -- **Pipeline Architecture**: Independent processing pipelines with source(s) → filter → format → sink(s) flow -- **Multiple Input Sources**: Directory monitoring, stdin, HTTP, TCP -- **Flexible Output Sinks**: Console, file, HTTP SSE, TCP streaming, HTTP/TCP forwarding -- **Real-time Processing**: Sub-millisecond latency with configurable buffering -- **Hot Configuration Reload**: Update pipelines without service restart +### Pipeline -### Data Processing -- **Pattern-based Filtering**: Chainable include/exclude filters with regex support -- **Multiple Formatters**: Raw, JSON, and template-based text formatting -- **Rate Limiting**: Pipeline rate control +- **Independent pipelines**, each `sources → flow → sinks`, running concurrently + in one process +- **Fan-in and fan-out**: many sources and many sinks per pipeline +- **Never blocks**: a stalled sink drops its own events and is counted, rather + than stalling the pipeline or its sibling sinks +- **Hot reload** via `SIGHUP`/`SIGUSR1` or a config file watch, with the new + configuration validated before the old service is torn down -### Security & Reliability -- **Authentication**: mTLS support for HTTPS -- **TLS Encryption**: TLS 1.2/1.3 support for HTTP connections -- **Access Control**: IP whitelisting/blacklisting, connection limits -- **Automatic Reconnection**: Resilient client connections with exponential backoff -- **File Rotation**: Size-based rotation with retention policies +### Inputs -### Operational Features -- **Status Monitoring**: Real-time statistics and health endpoints -- **Signal Handling**: Graceful shutdown and configuration reload via signals -- **Background Mode**: Daemon operation with proper signal handling -- **Quiet Mode**: Silent operation for automated deployments +`file` (directory tail with rotation detection and JSON line parsing), +`console` (stdin), `random` (synthetic generator), `null`, and the chain ingest +listeners `tcp_chain` and `http_chain`. + +### Outputs + +`console`, `file` (rotating with retention), `http` (Server-Sent Events plus a +JSON status endpoint), `tcp` (broadcast server), `null`, and the chain +forwarders `tcp_chain` and `http_chain`. + +### Processing + +- **Filters**: chainable include/exclude RE2 patterns with `or`/`and` logic +- **Formatters**: `raw`, `txt`, and `json` with selectable sanitizer policies +- **Rate limiting**: token bucket with an optional per-entry size cap +- **Heartbeats**: flow-level keep-alive entries that reach every sink + +### Chaining + +Multi-node topologies over a versioned protocol. Chain links carry the +**structured entry**, not the formatted text, so a relay can filter and reformat +as if the entries were local. Entries keep a `node` label identifying their +origin across any number of hops. Chain sinks reconnect automatically with +exponential backoff and jitter. + +### Transport security + +- TLS 1.2/1.3 on every network source and sink, listener and dialer alike +- Mutual TLS: listeners can require and verify client certificates; dialers can + present a client identity + +mTLS is currently a CA-wide membership check — any certificate the configured CA +issued is accepted, and the peer's Common Name is recorded but not used for +authorization. See [Security](doc/security.md) for the exact boundary and +[the mTLS authentication plan](doc/mtls-auth-plan.md) for the proposed work. +Password, token, and SCRAM authentication were removed during the restructure +and are not currently available. ## Documentation -Available in `doc/` directory. +| Document | Contents | +|----------|----------| +| [Installation](doc/installation.md) | Building, installing, running as a service | +| [Architecture](doc/architecture.md) | Component model, data flow, concurrency, back-pressure | +| [Configuration](doc/configuration.md) | TOML structure, precedence, environment and CLI overrides | +| [Sources](doc/sources.md) | Every input plugin and its options | +| [Sinks](doc/sinks.md) | Every output plugin and its options | +| [Filters](doc/filters.md) | Pattern-based inclusion and exclusion | +| [Formatters](doc/formatters.md) | Output shaping and sanitization | +| [Chaining](doc/chaining.md) | Multi-node topologies and the chain wire protocol | +| [Networking](doc/networking.md) | Listeners, dialers, timeouts, connection limits | +| [Security](doc/security.md) | TLS and mTLS configuration, threat model, current limits | +| [mTLS Authentication Plan](doc/mtls-auth-plan.md) | Design for certificate-based authorization | +| [CLI](doc/cli.md) | Flags, signals, exit codes | +| [Operations](doc/operations.md) | Running, monitoring, tuning, troubleshooting | -- [Installation Guide](doc/installation.md) - Platform setup and service configuration -- [Architecture Overview](doc/architecture.md) - System design and component interaction -- [Configuration Reference](doc/configuration.md) - TOML structure and configuration methods -- [Input Sources](doc/sources.md) - Available source types and configurations -- [Output Sinks](doc/sinks.md) - Sink types and output options -- [Filters](doc/filters.md) - Pattern-based log filtering -- [Formatters](doc/formatters.md) - Log formatting and transformation -- [Security](doc/security.md) - mTLS configurations and access control -- [Networking](doc/networking.md) - TLS, rate limiting, and network features -- [Command Line Interface](doc/cli.md) - CLI flags and subcommands -- [Operations Guide](doc/operations.md) - Running and maintaining LogWisp +A fully annotated configuration covering every option ships as +[`config/logwisp.toml`](config/logwisp.toml). ## Quick Start -Install LogWisp and create a basic configuration: +```bash +make +``` ```toml +# logwisp.toml [[pipelines]] name = "default" -[[pipelines.sources]] -type = "directory" -[pipelines.sources.directory] -path = "./" +[pipelines.flow.format] +type = "json" +sanitizer_policy = "json" + +[[pipelines.plugin_sources]] +id = "app_logs" +type = "file" +[pipelines.plugin_sources.config] +directory = "/var/log/myapp" pattern = "*.log" -[[pipelines.sinks]] +[[pipelines.plugin_sinks]] +id = "stdout" type = "console" -[pipelines.sinks.console] +[pipelines.plugin_sinks.config] target = "stdout" ``` -Run with: `logwisp -c config.toml` +```bash +logwisp -c logwisp.toml +``` + +Running with no configuration file starts a self-demonstrating pipeline: a +synthetic generator writing JSON to stdout. ## System Requirements -- **Operating Systems**: Linux (kernel 6.10+), FreeBSD (14.0+) +- **Operating systems**: Linux (kernel 6.10+), FreeBSD (14.0+) - **Architecture**: amd64 -- **Go Version**: 1.25+ (for building from source) +- **Go**: 1.26+ to build from source + +Network sources and sinks bind and dial over IPv4 only. ## License -BSD 3-Clause License \ No newline at end of file +BSD 3-Clause License diff --git a/config/logwisp.toml b/config/logwisp.toml new file mode 100644 index 0000000..37af9db --- /dev/null +++ b/config/logwisp.toml @@ -0,0 +1,305 @@ +############################################################################### +### LogWisp Configuration +### Default location: ~/.config/logwisp/logwisp.toml +### Precedence: CLI flags > Environment > File > Defaults +### +### Commented values are the built-in defaults unless marked "example". +### Uncommenting a default is a no-op. +### +### NOTE: authentication (password/token/SCRAM), network access control (ACL), +### http/tcp ingest sources, and http_client/tcp_client sinks were removed in +### the restructure and are not available in this version. TLS and mTLS ARE +### available on every network source and sink; see the [...tls] blocks below. +### +### Environment overrides are currently read WITHOUT the LOGWISP_ prefix +### (QUIET, LOGGING_LEVEL, ...). LOGWISP_CONFIG_FILE and LOGWISP_CONFIG_DIR +### are the exceptions and do carry it. Array-indexed paths such as +### pipelines.0.name cannot be set from the CLI or environment at all. +############################################################################### + +############################################################################### +### Global Settings +############################################################################### + +quiet = false # Suppress console output +status_reporter = true # Periodic status logging (30s, DEBUG level) +auto_reload = false # Config auto-reload on file change + +############################################################################### +### Logging (LogWisp's internal operational logging) +############################################################################### + +[logging] +output = "stdout" # file|stdout|stderr|split|all|none +level = "info" # debug|info|warn|error +# format = "txt" # raw|txt|json +# sanitization = "" # raw|json|txt|shell (empty = logger default) + +# [logging.file] # Used when output is "file" or "all" +# directory = "./log" +# name = "logwisp" +# max_size_mb = 100 +# max_total_size_mb = 1000 +# retention_hours = 168.0 # 7 days + +## Validated but NOT applied: the console destination comes from `output` above. +# [logging.console] +# target = "stdout" # stdout|stderr|split + +############################################################################### +### Pipelines +### Each pipeline: plugin_sources -> flow (rate_limit|filters|format) -> plugin_sinks +### Names must be unique. 1+ source and 1+ sink required. +############################################################################### + +[[pipelines]] +name = "default" + +###============================================================================ +### Flow (processing between sources and sinks) +###============================================================================ + +## policy="pass" short-circuits the size check too: enforcing a size cap needs +## rate > 0 AND policy = "drop". +# [pipelines.flow.rate_limit] +# rate = 0.0 # Entries/second (0 = limiter disabled) +# burst = 0.0 # Burst capacity (defaults to rate) +# policy = "pass" # pass|drop +# max_entry_size_bytes = 0 # 0 = unlimited + +## Filters: sequential include/exclude chain, matched against " " +# [[pipelines.flow.filters]] +# type = "include" # include|exclude +# logic = "or" # or|and +# patterns = [".*ERROR.*", ".*WARN.*"] # example; RE2 syntax + +# [pipelines.flow.format] +# type = "raw" # raw|json|txt +# flags = 0 # Formatter flags (0 = defaults per type) +# timestamp_format = "" # Go time layout (formatter default if empty) +# sanitizer_policy = "" # raw|json|txt|shell (defaults per type) + +## Flow-level heartbeat (fan-out to all sinks, traverses chain links) +# [pipelines.flow.heartbeat] +# enabled = false +# interval_ms = 1000 # Minimum 100 +# include_timestamp = false +# include_stats = false +# format = "txt" # txt|json|raw ("comment" is rejected) + +###============================================================================ +### TLS (shared shape; applies to every network source and sink) +### +### Listeners (tcp/http sinks, tcp_chain/http_chain sources): +### cert_file + key_file are REQUIRED; client_auth + client_ca_file enable mTLS. +### Dialers (tcp_chain/http_chain sinks): +### ca_file + server_name verify the server; cert_file + key_file present a +### client identity for mTLS. +### +### enabled = false Master switch +### cert_file = "" Local certificate +### key_file = "" Private key for cert_file (set together) +### client_auth = false Listener: require and verify a client certificate +### client_ca_file = "" Listener: CA bundle verifying client certs +### ca_file = "" Dialer: CA bundle verifying the server (empty = system) +### server_name = "" Dialer: SNI / name to verify (empty = configured host) +### insecure_skip_verify = false Dialer: disable verification (never in prod) +### min_version = "1.3" "1.2" or "1.3". No max_version, no cipher_suites. +### +### Any certificate signed by client_ca_file is accepted; the peer CN is +### recorded but not used for authorization. See doc/security.md. +###============================================================================ + +###============================================================================ +### Sources (1+ required) +###============================================================================ + +## Null source (testing) +# [[pipelines.plugin_sources]] +# id = "null_in" +# type = "null" + +[[pipelines.plugin_sources]] +id = "default_source" +type = "file" +[pipelines.plugin_sources.config] +directory = "./" # Directory to monitor (required, not recursive) +pattern = "*.log" # Glob pattern (* and ? only) +## Tailing an already-open file polls at a fixed 100ms, regardless of this value. +check_interval_ms = 100 # Directory rescan interval (min 10) + +## Console source (stdin, single instance per pipeline) +# [[pipelines.plugin_sources]] +# id = "console_in" +# type = "console" +# [pipelines.plugin_sources.config] +# buffer_size = 1000 + +## Random source (testing; special=true exercises sanitizer policies) +# [[pipelines.plugin_sources]] +# id = "random_in" +# type = "random" +# [pipelines.plugin_sources.config] +# interval_ms = 500 +# jitter_ms = 0 # Clamped to interval_ms +# format = "txt" # raw|txt|json +# length = 20 +# special = false + +## TCP chain source (stdlib listener; receives NDJSON from upstream tcp_chain sinks) +## Topology: jail [file source -> tcp_chain sink] -> host [tcp_chain source -> aggregate sink] +# [[pipelines.plugin_sources]] +# id = "chain_in" +# type = "tcp_chain" +# [pipelines.plugin_sources.config] +# host = "0.0.0.0" # IPv4 only +# port = 9440 # Required +# buffer_size = 1000 +# max_connections = 0 # 0 = unlimited +# read_timeout_ms = 0 # idle deadline, 0 = none +# hello_timeout_ms = 10000 # Protocol preamble deadline +# trust_node = true # false: label entries by remote address +# [pipelines.plugin_sources.config.tls] +# enabled = true # example: mTLS listener +# cert_file = "/etc/logwisp/tls/server.crt" +# key_file = "/etc/logwisp/tls/server.key" +# client_auth = true +# client_ca_file = "/etc/logwisp/tls/client-ca.crt" +# min_version = "1.3" + +## HTTP chain source (stdlib listener; receives NDJSON batches from upstream http_chain sinks) +# [[pipelines.plugin_sources]] +# id = "hchain_in" +# type = "http_chain" +# [pipelines.plugin_sources.config] +# host = "0.0.0.0" +# port = 9441 # Required +# ingest_path = "/ingest" # Must start with "/" +# buffer_size = 1000 +# max_body_bytes = 8388608 # per-request cap (8 MiB) +# read_timeout_ms = 30000 +# trust_node = true # false: label entries by remote address +# [pipelines.plugin_sources.config.tls] +# enabled = true # example: mTLS listener +# cert_file = "/etc/logwisp/tls/server.crt" +# key_file = "/etc/logwisp/tls/server.key" +# client_auth = true +# client_ca_file = "/etc/logwisp/tls/client-ca.crt" + +###============================================================================ +### Sinks (1+ required, fan-out) +###============================================================================ + +## Null sink (testing) +# [[pipelines.plugin_sinks]] +# id = "null_out" +# type = "null" + +[[pipelines.plugin_sinks]] +id = "default_sink" +type = "console" +[pipelines.plugin_sinks.config] +target = "stdout" # stdout|stderr ("split" NOT supported) +# buffer_size = 1000 + +## File sink (rotating) +# [[pipelines.plugin_sinks]] +# id = "file_out" +# type = "file" +# [pipelines.plugin_sinks.config] +# directory = "./logs" # Required +# name = "output" # Required +# max_size_mb = 100 +# max_total_size_mb = 1000 +# min_disk_free_mb = 0 # 0 = no floor (only negatives become 100) +# retention_hours = 168.0 +# buffer_size = 1000 +# flush_interval_ms = 100 + +## HTTP sink (SSE streaming server + JSON status endpoint; IPv4 clients only) +## Both endpoints are UNAUTHENTICATED and the stream sends +## Access-Control-Allow-Origin: *. Bind to a trusted interface. +# [[pipelines.plugin_sinks]] +# id = "http_out" +# type = "http" +# [pipelines.plugin_sinks.config] +# host = "0.0.0.0" +# port = 8081 # Required +# stream_path = "/stream" # Must start with "/" +# status_path = "/status" # Must differ from stream_path +# buffer_size = 1000 # Sink input queue +# client_buffer_size = 256 # Per-client send queue +# write_timeout_ms = 0 # Per-event deadline, 0 = none +# max_connections = 0 # 0 = unlimited +# [pipelines.plugin_sinks.config.tls] +# enabled = true # example +# cert_file = "/etc/logwisp/tls/server.crt" +# key_file = "/etc/logwisp/tls/server.key" +# client_auth = false +# client_ca_file = "" + +## TCP sink (streaming server, IPv4 clients only) +# [[pipelines.plugin_sinks]] +# id = "tcp_out" +# type = "tcp" +# [pipelines.plugin_sinks.config] +# host = "0.0.0.0" +# port = 9090 # Required +# buffer_size = 1000 +# client_buffer_size = 256 +# write_timeout_ms = 5000 # Missed deadline disconnects the client +# keep_alive = true +# keep_alive_period_ms = 30000 +# max_connections = 0 +# [pipelines.plugin_sinks.config.tls] +# enabled = true # example +# cert_file = "/etc/logwisp/tls/server.crt" +# key_file = "/etc/logwisp/tls/server.key" + +## TCP chain sink (stdlib client; forwards to downstream tcp_chain source) +## Do NOT point a chain sink at a chain source in the SAME pipeline: entries +## loop back in and amplify without bound. +# [[pipelines.plugin_sinks]] +# id = "chain_out" +# type = "tcp_chain" +# [pipelines.plugin_sinks.config] +# host = "10.0.0.1" # Required +# port = 9440 # Required +# node = "" # origin label, default hostname; preserved across hops +# buffer_size = 1000 +# dial_timeout_ms = 5000 +# write_timeout_ms = 5000 +# backoff_min_ms = 500 +# backoff_max_ms = 30000 +# keep_alive = true +# keep_alive_period_ms = 30000 +# [pipelines.plugin_sinks.config.tls] +# enabled = true # example: mTLS dialer +# ca_file = "/etc/logwisp/tls/ca.crt" +# server_name = "" +# insecure_skip_verify = false +# cert_file = "/etc/logwisp/tls/client.crt" +# key_file = "/etc/logwisp/tls/client.key" +# min_version = "1.3" + +## HTTP chain sink (stdlib client; batched NDJSON POST to downstream http_chain source) +# [[pipelines.plugin_sinks]] +# id = "hchain_out" +# type = "http_chain" +# [pipelines.plugin_sinks.config] +# host = "10.0.0.1" # Required +# port = 9441 # Required +# ingest_path = "/ingest" +# node = "" # origin label, default hostname; preserved across hops +# buffer_size = 1000 +# max_batch_count = 100 +# max_batch_bytes = 1048576 +# flush_interval_ms = 1000 +# request_timeout_ms = 10000 # Covers dial + write + response +# backoff_min_ms = 500 +# backoff_max_ms = 30000 +# [pipelines.plugin_sinks.config.tls] +# enabled = true # example: mTLS dialer +# ca_file = "/etc/logwisp/tls/ca.crt" +# cert_file = "/etc/logwisp/tls/client.crt" +# key_file = "/etc/logwisp/tls/client.key" diff --git a/doc/README.md b/doc/README.md index 294c569..1fc90eb 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,73 +1,103 @@ -# LogWisp +# LogWisp Documentation -A pipeline-based log transport and processing system built in Go. LogWisp provides flexible log collection, filtering, formatting, and distribution with security and reliability features. +LogWisp is a pipeline-based log transport and processing system written in Go. +It collects log entries from files, stdin, or other LogWisp nodes; rate-limits, +filters, and formats them; and distributes them to files, consoles, live network +streams, or downstream LogWisp nodes. -## Features +## Documentation Map -### Core Capabilities -- **Pipeline Architecture**: Independent processing pipelines with source(s) → filter → format → sink(s) flow -- **Multiple Input Sources**: File monitoring, console (stdin), random log generation, null -- **Flexible Output Sinks**: Console, file, HTTP SSE, TCP streaming, null -- **Real-time Processing**: Sub-millisecond latency with configurable buffering -- **Hot Configuration Reload**: Update pipelines without service restart -- **Session Management**: Built-in session tracking for multiple client connections +| Document | Contents | +|----------|----------| +| [Installation](installation.md) | Building, installing, and running as a service | +| [Architecture](architecture.md) | Component model, data flow, concurrency, back-pressure | +| [Configuration](configuration.md) | TOML structure, precedence, environment and CLI overrides | +| [Sources](sources.md) | Every input plugin and its options | +| [Sinks](sinks.md) | Every output plugin and its options | +| [Filters](filters.md) | Pattern-based inclusion and exclusion | +| [Formatters](formatters.md) | Output shaping and sanitization | +| [Chaining](chaining.md) | Multi-node topologies and the chain wire protocol | +| [Networking](networking.md) | Listeners, dialers, timeouts, connection limits | +| [Security](security.md) | TLS and mTLS configuration, threat model, current limits | +| [mTLS Authentication Plan](mtls-auth-plan.md) | Design for certificate-based authorization | +| [CLI](cli.md) | Flags, signals, exit codes | +| [Operations](operations.md) | Running, monitoring, tuning, troubleshooting | -### Data Processing -- **Pattern-based Filtering**: Chainable include/exclude filters with regex support -- **Multiple Formatters**: Raw, JSON, and text formatting with integrated sanitizer policies -- **Rate Limiting**: Pipeline rate controls -- **Heartbeat Generation**: Flow-level heartbeat events for keep-alives +A fully annotated configuration covering every option lives at +[`config/logwisp.toml`](../config/logwisp.toml). -### Security & Reliability -- **File Rotation**: Size-based rotation with retention policies -- **Status Monitoring**: Real-time statistics and health endpoints -- **Signal Handling**: Graceful shutdown and configuration reload via signals -- **Background Mode**: Daemon operation with proper signal handling -- **Quiet Mode**: Silent operation for automated deployments +## Capabilities -## Documentation +### Pipeline -- [Installation Guide](installation.md) - Platform setup and service configuration -- [Architecture Overview](architecture.md) - System design and component interaction -- [Configuration Reference](configuration.md) - TOML structure and configuration methods -- [Input Sources](sources.md) - Available source types and configurations -- [Output Sinks](sinks.md) - Sink types and output options -- [Filters](filters.md) - Pattern-based log filtering -- [Formatters](formatters.md) - Log formatting and transformation -- [Networking & Security](networking.md) - Network features (Note: TLS and Auth are currently placeholders in the new architecture) -- [Command Line Interface](cli.md) - CLI flags and subcommands -- [Operations Guide](operations.md) - Running and maintaining LogWisp +- Independent named pipelines, each `sources → flow → sinks` +- Fan-in (many sources per pipeline) and fan-out (many sinks per pipeline) +- Non-blocking sink dispatch: a stalled sink drops its own events and never + stalls the pipeline or its sibling sinks +- Hot reload of pipeline configuration via `SIGHUP`/`SIGUSR1` or a file watch + +### Inputs + +`file` (directory tail with rotation detection), `console` (stdin), +`random` (synthetic generator), `null`, and the chain ingest listeners +`tcp_chain` and `http_chain`. + +### Outputs + +`console`, `file` (rotating), `http` (Server-Sent Events plus a JSON status +endpoint), `tcp` (broadcast server), `null`, and the chain forwarders +`tcp_chain` and `http_chain`. + +### Processing + +- Token-bucket rate limiting with an optional per-entry size cap +- Chainable include/exclude regex filters with `or`/`and` logic +- `raw`, `txt`, and `json` formatting with selectable sanitizer policies +- Optional flow-level heartbeat entries + +### Transport security + +- TLS 1.2/1.3 on every network source and sink, listener and dialer alike +- Mutual TLS: listeners can require and verify client certificates; dialers can + present a client identity. See [Security](security.md) for what this does and + does not currently give you. ## Quick Start -Install LogWisp and create a basic configuration: - ```toml [[pipelines]] name = "default" +[pipelines.flow.format] +type = "json" +sanitizer_policy = "json" + [[pipelines.plugin_sources]] -id = "default_source" +id = "app_logs" type = "file" [pipelines.plugin_sources.config] -directory = "./" +directory = "/var/log/myapp" pattern = "*.log" [[pipelines.plugin_sinks]] -id = "default_sink" +id = "stdout" type = "console" [pipelines.plugin_sinks.config] target = "stdout" ``` -Run with: `logwisp -c config.toml` +```bash +logwisp -c config.toml +``` ## System Requirements -- **Operating Systems**: Linux (kernel 6.10+), FreeBSD (14.0+) +- **Operating systems**: Linux (kernel 6.10+), FreeBSD (14.0+) - **Architecture**: amd64 -- **Go Version**: 1.25+ (for building from source) +- **Go**: 1.26+ to build from source + +Network sources and sinks bind and dial over IPv4 only. ## License -BSD 3-Clause License +BSD 3-Clause. diff --git a/doc/architecture.md b/doc/architecture.md index 9b26436..7353171 100644 --- a/doc/architecture.md +++ b/doc/architecture.md @@ -1,171 +1,195 @@ # Architecture Overview -LogWisp implements a pipeline-based architecture for flexible log processing and distribution. +LogWisp moves log entries through independent pipelines. Everything else — +plugins, sessions, TLS, statistics — hangs off that spine. -## Core Concepts - -### Pipeline Model - -Each pipeline operates independently with a source → filter → format → sink flow. Multiple pipelines can run concurrently within a single LogWisp instance, each processing different log streams with unique configurations. - -### Component Hierarchy +## Component Hierarchy ``` -Service (Main Process) -├── Pipeline 1 -│ ├── Plugin Sources (1 or more) -│ ├── Flow -│ │ ├── Heartbeat Generator (optional) -│ │ ├── Rate Limiter (optional) -│ │ ├── Filter Chain (optional) -│ │ └── Formatter (optional) -│ └── Plugin Sinks (1 or more) -├── Pipeline 2 -│ └── [Similar structure] -└── Status Reporter (optional) +main +└── Service + ├── Pipeline "app" + │ ├── Registry instance tracking, single-instance enforcement + │ ├── Session Manager per-pipeline connection/session bookkeeping + │ ├── Sources[] plugin instances, keyed by id + │ ├── Flow + │ │ ├── Rate Limiter optional, token bucket + │ │ ├── Filter Chain optional, ordered + │ │ ├── Formatter raw | txt | json, with sanitizer + │ │ └── Heartbeat optional generator + │ └── Sinks[] plugin instances, keyed by id + ├── Pipeline "audit" + │ └── ... + └── Status Reporter optional, 30s interval ``` +Package map: + +| Package | Responsibility | +|---------|----------------| +| `cmd/logwisp` | Entry point, help, logger bootstrap, signal loop, status reporter | +| `internal/config` | Typed config schema, loading, top-level validation | +| `internal/service` | Owns the pipeline set; start, stop, shutdown, global stats | +| `internal/pipeline` | Pipeline runtime and per-pipeline plugin registry | +| `internal/flow` | Rate limiter, filter chain invocation, formatting, heartbeat | +| `internal/filter` | Regex filter and filter chain | +| `internal/format` | Adapter over `lixenwraith/log` formatter + sanitizer | +| `internal/source/*` | Source plugins | +| `internal/sink/*` | Sink plugins | +| `internal/plugin` | Global factory registry populated by plugin `init()` | +| `internal/chain` | Chain wire protocol: hello preamble, entry codec, backoff | +| `internal/tlsx` | The single seam between `TLSOptions` and `crypto/tls` | +| `internal/session` | Session manager and per-instance proxy | +| `internal/core` | Shared types (`LogEntry`, `TransportEvent`), capabilities, constants | +| `internal/tokenbucket` | Rate limiter primitive | +| `internal/sanitize` | Standalone hex-escaping helpers | + +## Plugin Registration + +Every plugin registers itself in an `init()` function, and +`cmd/logwisp/bootstrap.go` blank-imports each package to trigger those +`init()`s. Adding a plugin therefore means writing the package, calling +`plugin.RegisterSource` / `plugin.RegisterSink`, and adding one blank import. + +Registration may attach metadata. The `console` source declares +`MaxInstances: 1`, because a process has only one stdin; the per-pipeline +registry rejects a second instance of any such type. + ## Data Flow -### Processing Stages +### Entry lifecycle -1. **Source Stage**: Plugin sources monitor inputs and generate log entries -2. **Flow - Rate Limiting**: Optional pipeline-level rate control -3. **Flow - Filtering**: Pattern-based inclusion/exclusion -4. **Flow - Formatting**: Transform entries to desired output format with sanitization -5. **Distribution**: Fan-out to multiple plugin sinks +1. **Source** produces a `core.LogEntry` and publishes it to every subscriber + channel it has handed out. Publication is non-blocking: a full subscriber + channel increments the source's `dropped_entries` counter. +2. **Flow** applies, in order: rate limit → filter chain → formatter. A drop at + any stage ends the entry's life and increments `flow.total_dropped`. +3. The formatter output becomes a `core.TransportEvent`, which carries both the + formatted `Payload` and the original structured `Entry`. +4. **Dispatch** sends the event to every sink's input channel with a + non-blocking send. -### Entry Lifecycle +`LogEntry` fields: -Log entries flow through the pipeline as `core.LogEntry` structures containing: -- **Time**: Entry timestamp -- **Level**: Log level (DEBUG, INFO, WARN, ERROR) -- **Source**: Origin identifier -- **Message**: Log content -- **Fields**: Additional metadata (JSON) -- **RawSize**: Original entry size +| Field | Purpose | +|-------|---------| +| `Time` | Entry timestamp | +| `Node` | Origin node label for chained topologies; stamped at the first hop, preserved by relays | +| `Source` | Origin identifier within the node (filename, plugin id, …) | +| `Level` | `DEBUG`/`INFO`/`WARN`/`ERROR`/`TRACE`, when detected | +| `Message` | Log content | +| `Fields` | Optional structured metadata as raw JSON | +| `RawSize` | Original byte size, used by the entry-size cap | -### Buffering Strategy +Carrying `Entry` alongside `Payload` is what makes chain sinks +format-independent: a `tcp_chain` or `http_chain` sink re-serializes the +structured entry rather than shipping whatever text the local formatter chose. -Each component maintains internal buffers to handle burst traffic: -- Sources: Configurable buffer size (default 1000 entries) -- Sinks: Independent buffers per sink -- Network components: Additional TCP/HTTP buffers +### Back-pressure and drops -*"Sink dispatch uses non-blocking sends. When a sink's input buffer is full, the event is dropped for that sink only and counted in pipeline statistics (`total_dropped_by_sink`). The policy is uniform and not configurable; a slow sink does not stall the pipeline or other sinks."* +There is exactly one drop policy and it is not configurable: **never block**. -## Component Types +| Stage | Full-buffer behaviour | Counter | +|-------|----------------------|---------| +| Source → subscriber | Drop the entry | source `dropped_entries` | +| Flow | Drop on rate limit, filter, or format error | `flow.total_dropped` | +| Pipeline → sink | Drop for that sink only | pipeline `total_dropped_by_sink` | +| TCP/HTTP sink → client queue | Drop for that client only | sink `dropped_writes` | -### Sources (Input) - -- **File Source**: File system directory monitoring with rotation detection -- **Console Source**: Standard input processing (stdin) -- **Random Source**: Generates random log entries for testing -- **Null Source**: Discards logs, used for testing - -### Sinks (Output) - -- **Console Sink**: stdout/stderr output -- **File Sink**: Rotating file writer -- **HTTP Sink**: Server-Sent Events (SSE) streaming -- **TCP Sink**: TCP server for client connections -- **Null Sink**: Discards all received events - -### Processing Components - -- **Rate Limiter**: Token bucket algorithm for flow control -- **Filter Chain**: Sequential pattern matching -- **Formatters**: Raw, JSON, or text transformation with sanitizer policies +The `tcp_chain` sink is the one deliberate exception. It holds a line across +reconnects until it is written or the process shuts down, so a downstream +outage propagates backwards as a full input buffer and surfaces as +`total_dropped_by_sink` on the pipeline rather than as silent data loss inside +the sink. The `http_chain` sink retries a batch with backoff, and drops it only +on a non-retryable response or on shutdown (`dropped_batches`). ## Concurrency Model -### Goroutine Architecture +- One goroutine per source drains that source's subscription and feeds the flow. +- The flow's formatter holds a mutex; the underlying formatter reuses an + internal buffer and is not goroutine-safe. +- Each network sink runs one broadcast/broker goroutine plus, per connection, a + writer goroutine (and for TCP, a reader goroutine that exists only to detect + disconnects and refresh session activity). +- Chain sinks run a single run-loop goroutine that exclusively owns the + connection or the pending batch, so no locking is needed around either. +- Statistics are atomics; configuration and registries use RW mutexes; + shutdown is context cancellation plus wait groups. -- Each source runs in dedicated goroutines for monitoring -- Sinks operate independently with their own processing loops -- Network listeners use optimized event loops (gnet for TCP) -- Pipeline processing uses channel-based communication +### Shutdown ordering -### Synchronization +`Pipeline.Stop` is deliberately ordered so in-flight data drains: -- Atomic counters for statistics -- Read-write mutexes for configuration access -- Context-based cancellation for graceful shutdown -- Wait groups for coordinated startup/shutdown +1. Stop all sources concurrently; each closes its subscriber channels. +2. Wait for the run loop, which ends when every subscription channel closes. +3. Stop all sinks concurrently. ## Network Architecture -### Connection Patterns +All listeners bind `tcp4` and all dialers dial `tcp4`. IPv6 clients cannot +connect; this is deliberate, not an oversight. -**Chaining Design**: -- Future plan +| Plugin | Role | Protocol | +|--------|------|----------| +| `tcp` sink | Listener | Raw broadcast of formatted payloads | +| `http` sink | Listener | HTTP/1.1 SSE; HTTP/2 negotiated via ALPN when TLS is on | +| `tcp_chain` source | Listener | Chain protocol, persistent NDJSON stream | +| `http_chain` source | Listener | Chain protocol, NDJSON batches over POST | +| `tcp_chain` sink | Dialer | Chain protocol, persistent stream, auto-reconnect | +| `http_chain` sink | Dialer | Chain protocol, batched POST with retry | -**Monitoring Design**: -- TCP Sink: Debugging interface -- HTTP Sink: Browser-based live monitoring +TLS is built in exactly one place, `internal/tlsx`, which exposes +`Server(opts)` for listeners and `Client(opts, host)` for dialers. See +[Security](security.md). -### Protocol Support +## Sessions -- HTTP/1.1 and HTTP/2 for HTTP connections -- Raw TCP connections +Each pipeline owns a `session.Manager`. Plugins receive a `session.Proxy` +scoped to their instance id, so one plugin cannot see or remove another's +sessions. A session records the remote address, creation and last-activity +timestamps, and metadata — including `tls` and `tls_peer_cn` for TLS peers. + +Idle sessions are reaped every 5 minutes against a 30-minute idle limit. The +HTTP sink's broker treats a vanished session as an eviction signal and closes +the corresponding SSE client. + +Session metadata is currently bookkeeping only: nothing in the pipeline makes +an authorization decision from it. Closing that gap is the subject of the +[mTLS authentication plan](mtls-auth-plan.md). + +## Configuration Reload + +Reload (signal or file watch) rebuilds the entire service: + +1. Re-read the config through the config manager. +2. Build a **new** service from it. If construction fails, the old service keeps + running untouched. +3. Shut the old service down, start the new one, and restart the status + reporter if it is enabled. + +Because this is a full rebuild, listening sockets close and reopen and all +clients are disconnected. Application logging is configured once at startup and +is **not** re-applied on reload. ## Resource Management -### Memory Management +- Every buffer is bounded; the drop-not-block policy keeps memory flat under + load. +- Network sinks and chain sources accept a `max_connections` cap. Admission is + a load-then-check, so a burst can over-admit by roughly one connection. +- Chain listeners bound a single line at `core.MaxLogEntryBytes` (1 MiB); an + oversized line is a protocol violation and terminates the connection. +- The `http_chain` source caps each request body at `max_body_bytes`. +- File sinks rotate on size, cap total rotated size, and honour a retention + window. -- Bounded buffers prevent unbounded growth -- Automatic garbage collection via Go runtime -- Connection limits prevent resource exhaustion +## Performance Notes -### File Management - -- Automatic rotation based on size thresholds -- Retention policies for old log files -- Minimum disk space checks before writing - -### Connection Management - -- HTTP sink silenty drops IPv6 connections (deliberate IPv4-only enforcement) - -*Note:* Placeholder; below features are removed in restructuring and will be added in the future release. - -- Per-IP connection limits -- Global connection caps -- Automatic reconnection with exponential backoff -- Keep-alive for persistent connections - -## Reliability Features - -### Fault Tolerance - -- Panic recovery in pipeline processing -- Independent pipeline operation -- Sink failure isolation - -### Data Integrity - -- Entry validation at ingestion -- Size limits for entries and batches -- Duplicate detection in file monitoring -- Position tracking for file reads - -## Performance Characteristics - -### Throughput - -- Pipeline rate limiting: Configurable (default 1000 entries/second) -- Network throughput: Limited by network and sink capacity -- File monitoring: Sub-second detection (default 100ms interval) - -### Latency - -- Entry processing: Sub-millisecond in-memory -- Network forwarding: Depends on batch configuration -- File detection: Configurable check interval - -### Scalability - -- Horizontal: Multiple LogWisp instances with different configurations -- Vertical: Multiple pipelines per instance -- Fan-out: Multiple sinks per pipeline -- Fan-in: Multiple sources per pipeline +- In-memory entry processing is sub-millisecond; the formatter mutex is the + only shared serialization point in the hot path. +- File tailing detects new content within roughly 100 ms (fixed poll), while + `check_interval_ms` governs how quickly a *newly created* file is noticed. +- `http_chain` trades latency for efficiency: entries wait up to + `flush_interval_ms` (default 1 s) before a batch is sent. +- Scale out with more pipelines per process, more sinks per pipeline, or more + nodes chained together. diff --git a/doc/chaining.md b/doc/chaining.md new file mode 100644 index 0000000..49285d7 --- /dev/null +++ b/doc/chaining.md @@ -0,0 +1,212 @@ +# Chaining + +Chaining links LogWisp nodes together. An edge node forwards its entries to a +relay or collector, which can filter, reformat, fan out, or forward them again. +Unlike the `tcp` and `http` sinks — which emit *formatted text* for humans and +generic clients — chain links carry the **structured entry**, so downstream +nodes can filter and reformat as if the entries were local. + +## Topology + +``` + edge-01 relay consumers + ┌───────────────┐ ┌────────────────────┐ ┌──────────────┐ + │ file source │ │ tcp_chain source │ │ browser (SSE)│ + │ ↓ │ TCP/TLS │ ↓ │ ───► │ nc / telnet │ + │ tcp_chain sink├────────────►│ flow │ │ archive file │ + └───────────────┘ :15801 │ ↓ │ └──────────────┘ + │ http sink, tcp sink│ + edge-02 │ file sink │ + ┌───────────────┐ │ http_chain sink ───┼──► upstream collector + │ file source │ HTTP/TLS │ │ + │ ↓ ├────────────►│ http_chain source │ + │ http_chain sink│ :15802 └────────────────────┘ + └───────────────┘ +``` + +Both chain sources can feed a single pipeline (fan-in) whose sinks then fan the +merged stream out. `test/chain-test.sh` builds the two-independent-pipelines +variant; `test/chain-aggregate-test.sh` builds the fan-in variant. + +## Node Identity + +Chained entries carry a `node` label identifying where they originated. + +- A chain **sink** stamps `node` on any entry that does not already have one. + The label comes from the `node` option, defaulting to `os.Hostname()`. +- A chain **source** either honours the sender's label or overrides it, + according to `trust_node`: + +| `trust_node` | Behaviour | +|--------------|-----------| +| `true` (default) | Keep the label the sender declared; fall back to the remote address when absent | +| `false` | Always overwrite with the sender's remote address | + +Relays preserve `node`, so a label survives any number of hops and identifies +the original producer rather than the last relay. + +Formatters render node identity as a syslog-style prefix on the source field: +`edge-01/app.log`. In JSON output the node therefore appears inside the source +field, not as a separate top-level key. + +> `trust_node = true` means an authenticated peer can claim **any** node label, +> including one belonging to another host. On an untrusted network use +> `trust_node = false`, or read the +> [mTLS authentication plan](mtls-auth-plan.md), which proposes binding the +> label to the peer's certificate identity. + +## Wire Protocol + +Protocol version: **1**. Both transports carry the same canonical entry +encoding, and differ only in how the preamble and framing are expressed. + +### TCP transport + +A persistent connection carrying newline-delimited JSON. + +1. The dialer connects and, under TLS, completes the handshake. +2. The dialer immediately writes the hello preamble as one JSON line: + + ```json + {"logwisp":1,"node":"edge-01"} + ``` + +3. The listener reads that line within `hello_timeout_ms` and rejects the + connection if it is malformed or declares a different protocol version. +4. Every subsequent line is one JSON-encoded `LogEntry`. + +Line size is bounded at 1 MiB. An oversized line is a protocol violation and +terminates the connection, because the scanner cannot resynchronize afterwards. + +### HTTP transport + +Batches of NDJSON delivered by `POST`, with the preamble expressed as headers. + +| Header | Direction | Meaning | +|--------|-----------|---------| +| `X-Logwisp-Protocol` | request | Protocol version; must be `1` | +| `X-Logwisp-Node` | request | Origin node label | +| `Content-Type` | request | `application/x-ndjson` | +| `X-Logwisp-Accepted` | response | Number of entries ingested | + +Responses: `204` on success, `400` for a bad protocol version or a malformed +body, `413` when the body cap is exceeded, `405` for a non-`POST` method. + +### Entry encoding + +```json +{ + "time": "2026-01-02T15:04:05.123456789Z", + "node": "edge-01", + "source": "app.log", + "level": "ERROR", + "message": "connection refused", + "fields": {"attempt": 3} +} +``` + +`node`, `level`, and `fields` are omitted when empty. A missing `time` is filled +in at ingest. + +## Delivery Semantics + +| Transport | Guarantee | Failure behaviour | +|-----------|-----------|-------------------| +| `tcp_chain` | Per-line, held across reconnects | Retries with exponential backoff plus ±20 % jitter until written or shutdown; back-pressure appears upstream as `total_dropped_by_sink` | +| `http_chain` | At-least-once per batch | Retries transport errors, `408`, `429`, `5xx`; drops on any other non-2xx (`dropped_batches`) | + +`http_chain` batches can be delivered twice when a successful request's response +is lost. There is no de-duplication downstream; design your consumers to +tolerate it, or use `tcp_chain` where each line is written once per successful +write. + +Neither transport persists anything to disk. Entries buffered in memory during +an outage are lost if the process exits. + +## Worked Example + +**Edge node** — tail files, forward over mTLS: + +```toml +[[pipelines]] +name = "edge" + +[[pipelines.plugin_sources]] +id = "app" +type = "file" +[pipelines.plugin_sources.config] +directory = "/var/log/myapp" +pattern = "*.log" + +[[pipelines.plugin_sinks]] +id = "forward" +type = "tcp_chain" +[pipelines.plugin_sinks.config] +host = "relay.internal" +port = 15801 +node = "edge-01" +[pipelines.plugin_sinks.config.tls] +enabled = true +ca_file = "/etc/logwisp/tls/ca.crt" +cert_file = "/etc/logwisp/tls/edge-01.crt" +key_file = "/etc/logwisp/tls/edge-01.key" +``` + +**Relay** — ingest, keep errors only, archive and stream: + +```toml +[[pipelines]] +name = "relay" + +[[pipelines.flow.filters]] +type = "include" +patterns = ["ERROR", "FATAL"] + +[pipelines.flow.format] +type = "json" +sanitizer_policy = "json" + +[[pipelines.plugin_sources]] +id = "ingest" +type = "tcp_chain" +[pipelines.plugin_sources.config] +host = "0.0.0.0" +port = 15801 +trust_node = true +[pipelines.plugin_sources.config.tls] +enabled = true +cert_file = "/etc/logwisp/tls/relay.crt" +key_file = "/etc/logwisp/tls/relay.key" +client_auth = true +client_ca_file = "/etc/logwisp/tls/ca.crt" + +[[pipelines.plugin_sinks]] +id = "archive" +type = "file" +[pipelines.plugin_sinks.config] +directory = "/var/log/logwisp" +name = "errors" + +[[pipelines.plugin_sinks]] +id = "live" +type = "http" +[pipelines.plugin_sinks.config] +host = "127.0.0.1" +port = 8080 +``` + +## Operational Notes + +- **Formatting is a relay decision.** Because chain links carry structured + entries, the edge node's `flow.format` affects only its own local sinks. Set + the output shape on the node that owns the human-facing sink. +- **Filtering early saves bandwidth.** A filter on the edge drops entries before + they cross the network; a filter on the relay is easier to change centrally. +- **Rate limits are per pipeline.** An edge limit protects the link; a relay + limit protects the relay from a noisy edge. +- **Heartbeats traverse chain links** as ordinary structured entries and keep + otherwise-idle links and their sessions warm. +- **Ports** used by the bundled test scripts: `15801` tcp_chain ingest, `15802` + http_chain ingest, `15803` tcp sink, `15804` http sink. +- **Use `127.0.0.1`, not `localhost`**, when testing locally: all listeners and + dialers are IPv4-only, and `localhost` may resolve to `::1`. diff --git a/doc/cli.md b/doc/cli.md index f557372..68dbb91 100644 --- a/doc/cli.md +++ b/doc/cli.md @@ -1,196 +1,184 @@ # Command Line Interface -LogWisp CLI reference for commands and options. - -## Synopsis - -```bash -logwisp [command] [options] -logwisp [options] ``` - -## Commands - -### Main Commands - -| Command | Description | -|---------|-------------| -| `--version` | Display version information | -| `--help` | Show help information | - -### version Command - -Display version information. - -```bash -logwisp version -logwisp -v +logwisp [options] +logwisp help | -h | --help logwisp --version ``` -Output includes: -- Version number -- Build date -- Git commit hash -- Go version +LogWisp has no subcommands. Earlier releases shipped `logwisp auth` and +`logwisp tls` for credential and certificate generation; both were removed +during the restructure. Use `openssl` or your PKI tooling instead — see +[Security](security.md). -## Global Options +## Options -### Configuration Options +Any scalar configuration key is settable as a flag using its TOML path: + +``` +--= e.g. --logging.level=debug +-- e.g. --logging.level debug +-- bare flag, means true +``` + +### Common | Flag | Description | Default | |------|-------------|---------| -| `-c, --config` | Configuration file path | `./logwisp.toml` | -| `-q, --quiet` | Suppress console output | false | -| `--status-reporter` | Status logging | true | -| `--auto-reload` | Enable config hot reload | false | +| `-c ` | Configuration file | `./logwisp.toml` | +| `--config=` | Configuration file (equals form only) | `./logwisp.toml` | +| `--quiet` | Suppress all application output | `false` | +| `--status_reporter=` | Periodic status logging | `true` | +| `--auto_reload=` | Reload config when the file changes | `false` | +| `--version` | Print version and exit | — | +| `-h`, `--help`, `help` | Print usage and exit | — | -### Logging Options +> `--config ` with a space is not recognized. The path resolver +> understands `-c ` and `--config=` only; the space form is treated +> as an unknown flag, warned about, and ignored, after which LogWisp silently +> falls back to `./logwisp.toml`. +> +> `-c` as the final argument, with no path after it, crashes with an index +> panic rather than reporting a usage error. -| Flag | Description | Values | -|------|-------------|--------| -| `--logging.output` | Log output mode | file, stdout, stderr, split, all, none | -| `--logging.level` | Log level | debug, info, warn, error | -| `--logging.file.directory` | Log directory | Path | -| `--logging.file.name` | Log filename | String | -| `--logging.file.max_size_mb` | Max file size | Integer | -| `--logging.file.max_total_size_mb` | Total size limit | Integer | -| `--logging.file.retention_hours` | Retention period | Float | -| `--logging.console.target` | Console target | stdout, stderr, split | -| `--logging.console.format` | Output format | txt, json | +### Logging -### Pipeline Options +| Flag | Values | +|------|--------| +| `--logging.output` | `file`, `stdout`, `stderr`, `split`, `all`, `none` | +| `--logging.level` | `debug`, `info`, `warn`, `error` | +| `--logging.format` | `raw`, `txt`, `json` | +| `--logging.sanitization` | `raw`, `json`, `txt`, `shell` | +| `--logging.file.directory` | path | +| `--logging.file.name` | string | +| `--logging.file.max_size_mb` | integer | +| `--logging.file.max_total_size_mb` | integer | +| `--logging.file.retention_hours` | float | -Configure pipelines via CLI (N = array index, 0-based). +`--logging.console.target` is accepted but has no effect; the console +destination is derived from `--logging.output`. -**Pipeline Configuration:** +### Pipelines -| Flag | Description | -|------|-------------| -| `--pipelines.N.name` | Pipeline name | -| `--pipelines.N.plugin_sources.N.type` | Source type | -| `--pipelines.N.flow.filters.N.type` | Filter type | -| `--pipelines.N.plugin_sinks.N.type` | Sink type | +Pipelines, sources, sinks, and filters **cannot** be configured from the command +line. Array-indexed paths such as `--pipelines.0.name=app` or +`--pipelines.0.plugin_sinks.0.type=null` are reported as unrecognized and +ignored: -## Flag Formats - -### Boolean Flags - -```bash -logwisp --quiet -logwisp --quiet=true -logwisp --pipelines.0.plugin_sources.0.type=console +``` +Warning: unrecognized flags ignored: [pipelines.0.name] ``` -### String Flags - -```bash -logwisp --config /etc/logwisp/config.toml -logwisp -c config.toml -``` - -### Nested Configuration - -```bash -logwisp --logging.level=debug -logwisp --pipelines.0.name=myapp -logwisp --pipelines.0.sources.0.type=console -``` - -### Array Values (JSON) - -```bash -logwisp --pipelines.0.flow.filters.0.patterns='["ERROR","WARN"]' -``` +Use a configuration file. Older documentation described CLI pipeline overrides +that the current loader does not implement. ## Environment Variables -All flags can be set via environment: +Configuration paths map to environment variables by replacing `.` with `_` and +uppercasing: ```bash -export LOGWISP_QUIET=true -export LOGWISP_LOGGING_LEVEL=debug -export LOGWISP_PIPELINES_0_NAME=myapp +export QUIET=true +export LOGGING_LEVEL=debug +export LOGGING_FILE_DIRECTORY=/var/log/logwisp ``` -## Configuration Precedence +> The `LOGWISP_` prefix is **not** currently applied to these — see +> [Configuration](configuration.md#environment-variables). Bare names like +> `QUIET` are what LogWisp actually reads, which is worth knowing both to make +> overrides work and to avoid accidental collisions. -1. Command-line flags (highest) +The two variables that do carry the prefix are read directly by the path +resolver: + +| Variable | Effect | +|----------|--------| +| `LOGWISP_CONFIG_FILE` | Configuration file path; joined onto `LOGWISP_CONFIG_DIR` when both are set | +| `LOGWISP_CONFIG_DIR` | Configuration directory; alone, implies `/logwisp.toml` | + +As with flags, array elements cannot be set this way. + +## Precedence + +1. Command-line flags 2. Environment variables 3. Configuration file -4. Built-in defaults (lowest) +4. Built-in defaults -## Exit Codes - -| Code | Description | -|------|-------------| -| 0 | Success | -| 1 | General error | -| 2 | Configuration file not found | -| 137 | SIGKILL received | - -## Signal Handling +## Signals | Signal | Action | |--------|--------| -| SIGINT (Ctrl+C) | Graceful shutdown | -| SIGTERM | Graceful shutdown | -| SIGHUP | Reload configuration | -| SIGUSR1 | Reload configuration | -| SIGKILL | Immediate termination | +| `SIGINT` | Graceful shutdown | +| `SIGTERM` | Graceful shutdown | +| `SIGHUP` | Reload configuration | +| `SIGUSR1` | Reload configuration | + +`SIGHUP` is ignored during startup, before the signal handler is installed, so +LogWisp survives a terminal hang-up like `nohup`. Once running, it triggers a +reload rather than terminating. + +Reload rebuilds the whole service. A configuration error leaves the running +service untouched; see [Configuration](configuration.md#hot-reload). + +## Exit Codes + +| Code | Meaning | +|------|---------| +| `0` | Clean shutdown, or `--version` / `--help` | +| `1` | General error: config load or validation failure, logger init failure, service bootstrap failure | +| `2` | Explicitly requested configuration file not found | + +Exit code 2 applies only when the file was named explicitly (`-c`, +`--config=`, or the `LOGWISP_CONFIG_*` variables). A missing discovered default +is not an error, and LogWisp starts on built-in defaults. + +## Built-in Defaults + +With no configuration file present, LogWisp runs one pipeline named +`default_pipeline`: a `random` source with `special = true`, JSON formatting, +a rate limit of 5 entries/second with a burst of 10 and `policy = "drop"`, and a +`console` sink on stdout. It is a self-demonstrating idle mode, not a useful +production configuration. + +Note that as soon as your file defines `[[pipelines]]`, that entire default +pipeline — rate limit included — is replaced rather than merged. ## Usage Patterns -### Development Mode +**Development** ```bash -# Verbose logging to console -logwisp --logging.output=stderr --logging.level=debug +# verbose, everything to stderr +logwisp -c dev.toml --logging.output=stderr --logging.level=debug -# Quick test with stdin -logwisp --pipelines.0.plugin_sources.0.type=console --pipelines.0.plugin_sinks.0.type=console +# no config at all: synthetic generator to stdout +logwisp ``` -### Production Deployment +**Configuration check** ```bash -# Background with file logging -logwisp --background --config /etc/logwisp/prod.toml --logging.output=file - -# Systemd service -ExecStart=/usr/local/bin/logwisp --config /etc/logwisp/config.toml +# starts the service; a config error exits non-zero before any pipeline runs +logwisp -c /etc/logwisp/logwisp.toml --logging.level=debug ``` -### Debugging +There is no dry-run or validate-only mode. The closest approximation is starting +with debug logging and stopping once the pipelines report as started. + +**Production** ```bash -# Check configuration -logwisp --config test.toml --logging.level=debug --disable-status-reporter - -# Dry run (verify config only) -logwisp --config test.toml --quiet +logwisp -c /etc/logwisp/logwisp.toml --logging.output=file ``` -## Help System +Run under a supervisor (systemd, rc.d) rather than backgrounding it — there is +no `--background` flag; earlier releases had one and it was removed. See +[Installation](installation.md). -### General Help +**Reload** ```bash -logwisp --help -logwisp -h -logwisp help +kill -HUP $(pidof logwisp) +kill -USR1 $(pidof logwisp) ``` - -## Special Flags - -### Internal Flags - -These flags are for internal use: -- `--background-daemon`: Child process indicator -- `--config-save-on-exit`: Save config on shutdown - -### Hidden Behaviors - -- SIGHUP ignored ignored during startup (after startup triggers config reload) -- Automatic panic recovery in pipelines -- Resource cleanup on shutdown diff --git a/doc/configuration.md b/doc/configuration.md index 219a254..f31e275 100644 --- a/doc/configuration.md +++ b/doc/configuration.md @@ -1,203 +1,285 @@ # Configuration Reference -LogWisp configuration uses TOML format with flexible override mechanisms. +LogWisp is configured with TOML. A complete annotated file listing every option +and its default ships as [`config/logwisp.toml`](../config/logwisp.toml). ## Configuration Precedence -Configuration sources are evaluated in order: -1. **Command-line flags** (highest priority) -2. **Environment variables** -3. **Configuration file** -4. **Built-in defaults** (lowest priority) +Sources are merged in this order, highest priority first: + +1. Command-line flags +2. Environment variables +3. Configuration file +4. Built-in defaults + +The `pipelines` array is replaced wholesale, not merged: as soon as your file +defines `[[pipelines]]`, the built-in default pipeline (and its default rate +limit and formatter) disappears entirely. ## File Location -LogWisp searches for configuration in order: -1. Path specified via `--config` flag -2. Path from `LOGWISP_CONFIG_FILE` environment variable -3. `~/.config/logwisp/logwisp.toml` -4. `./logwisp.toml` in current directory +The path is resolved before any other configuration is read: + +1. `-c ` on the command line +2. `--config=` on the command line +3. `$LOGWISP_CONFIG_FILE`, joined onto `$LOGWISP_CONFIG_DIR` when both are set +4. `$LOGWISP_CONFIG_DIR/logwisp.toml` +5. `~/.config/logwisp/logwisp.toml`, if it exists +6. `./logwisp.toml` + +Missing file behaviour differs by how it was chosen. An explicitly requested +file that does not exist is a fatal error (exit code 2); a missing discovered +default is not an error, and LogWisp starts on built-in defaults. + +> `--config ` with a space is **not** recognized as a config path. It is +> parsed as an unknown flag, warned about, and ignored — LogWisp then silently +> falls back to `./logwisp.toml`. Use `-c ` or `--config=`. ## Global Settings -Top-level configuration options: - | Setting | Type | Default | Description | |---------|------|---------|-------------| -| `quiet` | bool | false | Suppress console output | -| `status_reporter` | bool | true | Periodic status logging | -| `auto_reload` | bool | false | Enable file watch for auto-reload | +| `quiet` | bool | `false` | Disable all application logging and console diagnostics | +| `status_reporter` | bool | `true` | Emit a periodic status report every 30 s at DEBUG level | +| `auto_reload` | bool | `false` | Watch the config file and reload pipelines on change | -## Logging Configuration +`--version` prints version information and exits; it is not a persistent +setting. -LogWisp's internal operational logging: +Note that `status_reporter` writes at DEBUG level, so it produces nothing unless +`logging.level = "debug"`. + +## Application Logging + +This configures LogWisp's own operational log, not the log data it transports. ```toml [logging] -output = "stdout" # file|stdout|stderr|split|all|none -level = "info" # debug|info|warn|error +output = "stdout" # file | stdout | stderr | split | all | none +level = "info" # debug | info | warn | error +format = "txt" # raw | txt | json +# sanitization = "" # raw | json | txt | shell [logging.file] -directory = "./log" -name = "logwisp" -max_size_mb = 100 +directory = "./log" +name = "logwisp" +max_size_mb = 100 max_total_size_mb = 1000 -retention_hours = 168.0 - -[logging.console] -target = "stdout" # stdout|stderr|split +retention_hours = 168.0 ``` -### Output Modes +### Output modes -- **file**: Write to log files only -- **stdout**: Write to standard output -- **stderr**: Write to standard error -- **split**: INFO/DEBUG to stdout, WARN/ERROR to stderr -- **all**: Write to both file and console -- **none**: Disable all logging +| Mode | Behaviour | +|------|-----------| +| `file` | Files only | +| `stdout` | Standard output only | +| `stderr` | Standard error only | +| `split` | DEBUG/INFO to stdout, WARN/ERROR to stderr | +| `all` | Files plus split console | +| `none` | No application logging | + +`[logging.file]` applies only to the `file` and `all` modes. + +> `[logging.console].target` is accepted and validated (`stdout`, `stderr`, +> `split`) but **not applied**. The console destination is derived from +> `logging.output`. The key is retained for compatibility; setting it has no +> effect. + +`quiet = true` overrides every logging setting and disables both file and +console output. ## Pipeline Configuration -Each `[[pipelines]]` section defines an independent processing pipeline: - ```toml [[pipelines]] -name = "pipeline-name" +name = "app" # required, unique across pipelines -# Rate limiting (optional) +# --- flow: everything between sources and sinks --- [pipelines.flow.rate_limit] -rate = 1000.0 -burst = 2000.0 -policy = "drop" # pass|drop -max_entry_size_bytes = 0 # 0=unlimited +rate = 1000.0 +burst = 2000.0 +policy = "drop" +max_entry_size_bytes = 65536 -# Format configuration (optional) -[pipelines.flow.format] -type = "json" # raw|json|txt -sanitizer_policy = "json" - -[[pipelines.plugin_sources]] -id = "my_source" -type = "file" -[pipelines.plugin_sources.config] -# ... source-specific config - -# Filters (optional) [[pipelines.flow.filters]] -type = "include" -logic = "or" +type = "include" +logic = "or" patterns = ["ERROR", "WARN"] -# Sinks (required, 1+) +[pipelines.flow.format] +type = "json" +sanitizer_policy = "json" + +[pipelines.flow.heartbeat] +enabled = true +interval_ms = 30000 + +# --- sources: one or more --- +[[pipelines.plugin_sources]] +id = "app_logs" # unique within the pipeline +type = "file" +[pipelines.plugin_sources.config] +directory = "/var/log/myapp" + +# --- sinks: one or more --- [[pipelines.plugin_sinks]] -id = "my_sink" +id = "sse" type = "http" [pipelines.plugin_sinks.config] -# ... sink-specific config +port = 8080 ``` +Every source and sink is a plugin instance with three keys: + +| Key | Meaning | +|-----|---------| +| `id` | Instance identifier, unique within the pipeline; appears in logs and stats | +| `type` | Registered plugin type | +| `config` | Plugin-specific table; see [Sources](sources.md) and [Sinks](sinks.md) | + +`config_file` is reserved on both structures for a future include mechanism and +is not implemented. + +### Flow stages + +| Block | Optional | Reference | +|-------|----------|-----------| +| `flow.rate_limit` | yes | below | +| `flow.filters` | yes | [Filters](filters.md) | +| `flow.format` | yes (defaults to `raw`) | [Formatters](formatters.md) | +| `flow.heartbeat` | yes | below | + +#### Rate limiting + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `rate` | float | `0` | Entries per second; `<= 0` disables the limiter entirely | +| `burst` | float | `rate` | Token bucket capacity | +| `policy` | string | `pass` | `pass` allows everything through, `drop` discards over-limit entries | +| `max_entry_size_bytes` | int | `0` | Per-entry byte cap; `0` = unlimited | + +Two behaviours are easy to trip over: + +- The limiter is constructed only when `rate > 0`. With `rate = 0`, + `max_entry_size_bytes` is never enforced. +- `policy = "pass"` short-circuits the whole check, including the size cap. + To enforce a size cap you need `rate > 0` **and** `policy = "drop"`. + +#### Heartbeat + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `enabled` | bool | `false` | Enable heartbeat generation | +| `interval_ms` | int | `1000` | Interval; minimum `100` | +| `include_timestamp` | bool | `false` | `false` formats with level only, no timestamp | +| `include_stats` | bool | `false` | Attach `beat_count` and measured `interval_ms` as fields | +| `format` | string | `txt` | `txt`, `json`, or `raw` | + +Heartbeats are ordinary entries with source `heartbeat` and level `INFO`. They +are generated after the flow's filter and rate-limit stages, so filters do not +suppress them, and they reach every sink in the pipeline. + +> `format = "comment"` (SSE comment framing) appears in older documentation and +> in a code path in the generator, but the validator rejects it and the pipeline +> fails to start. Use `txt`, `json`, or `raw`. + ## Environment Variables -All configuration options support environment variable overrides: +Environment overrides are derived from the TOML path: `.` becomes `_` and the +result is uppercased. -### Naming Convention - -- Prefix: `LOGWISP_` -- Path separator: `_` (underscore) -- Array indices: Numeric suffix (0-based) -- Case: UPPERCASE - -### Mapping Examples - -| TOML Path | Environment Variable | +| TOML path | Environment variable | |-----------|---------------------| -| `quiet` | `LOGWISP_QUIET` | -| `logging.level` | `LOGWISP_LOGGING_LEVEL` | -| `pipelines[0].name` | `LOGWISP_PIPELINES_0_NAME` | -| `pipelines[0].plugin_sources[0].type` | `LOGWISP_PIPELINES_0_PLUGIN_SOURCES_0_TYPE` | +| `quiet` | `QUIET` | +| `status_reporter` | `STATUS_REPORTER` | +| `logging.level` | `LOGGING_LEVEL` | +| `logging.file.directory` | `LOGGING_FILE_DIRECTORY` | + +> **The `LOGWISP_` prefix is not currently applied.** The configuration loader +> requests it, but supplying a custom path-to-variable transform replaces the +> prefixing step rather than composing with it, so LogWisp reads bare +> `QUIET`, `LOGGING_LEVEL`, and so on from the environment. Treat this as +> current behaviour to be aware of — bare names like `QUIET` can collide with +> unrelated variables — rather than as intended design. +> +> The two exceptions are `LOGWISP_CONFIG_FILE` and `LOGWISP_CONFIG_DIR`, which +> are read directly by the path resolver and **do** carry the prefix. + +Only scalar paths that exist in the configuration schema can be set this way. +Array elements cannot: `PIPELINES_0_NAME` has no effect. ## Command-Line Overrides -All configuration options can be overridden via CLI flags: +Any scalar configuration path is settable as a flag using its TOML path: ```bash -logwisp --quiet \ - --logging.level=debug \ - --pipelines.0.name=myapp \ - --pipelines.0.plugin_sources.0.type=console +logwisp --logging.level=debug --status_reporter=false +logwisp --logging.level debug # space form also works +logwisp --quiet # bare flag means true ``` -## Configuration Validation +Unrecognized flags are reported on stderr before the logger exists and are then +ignored: -LogWisp validates configuration at startup: -- Rpipelines non-empty, name non-empty, ≥1 source, ≥1 sink, logging enum values.equired fields presence +``` +Warning: unrecognized flags ignored: [pipelines.0.name] +``` -Partial check in plugin constructor: -- Type correctness -- Port conflicts -- Path accessibility -- Pattern compilation -- Network address formats +> Array-indexed paths are **not** settable from the command line. +> `--pipelines.0.name=x`, `--pipelines.0.plugin_sinks.0.type=null`, and similar +> flags are reported as unrecognized and ignored. Pipelines, sources, sinks, and +> filters can only be defined in the configuration file. Older documentation +> claimed otherwise. + +## Validation + +Startup validation is intentionally split. + +`internal/config` validates only global structure: + +- at least one pipeline +- unique, non-empty pipeline names +- at least one source and one sink per pipeline +- `logging.output`, `logging.level`, `logging.format`, `logging.sanitization`, + and `logging.console.target` enum membership + +Everything else is validated by the plugin constructor that owns it — port +range, required paths, path prefixes, enum values, regex compilation, TLS file +loading. A failure there aborts pipeline construction with a message naming the +pipeline, plugin id, and offending key. + +There is **no** cross-pipeline port-conflict detection. Two sinks bound to the +same port fail at listener bind time, when the pipeline starts. ## Hot Reload -Enable configuration hot reload: - ```toml auto_reload = true ``` -Or via command line: -```bash -logwisp --auto-reload -``` +or send `SIGHUP` / `SIGUSR1`. -Reload triggers: -- File modification detection -- SIGHUP or SIGUSR1 signals +Reload rebuilds the whole service: a new service is constructed from the new +configuration first, and only if that succeeds is the old one shut down. A +configuration error therefore leaves the running service untouched. -Reloadable items: -- Pipeline configurations -- Sources and sinks -- Filters and formatters -- Rate limits +| Reloaded | Not reloaded | +|----------|--------------| +| Pipelines, sources, sinks | `logging.*` (applied once at startup) | +| Filters, formatters, rate limits, heartbeats | `quiet` | +| `status_reporter` | `auto_reload` (the watcher is not restarted) | -Non-reloadable (requires restart): -- Logging configuration -- Global settings +Because the rebuild is total, listeners close and reopen and every connected +client is disconnected. Chain sinks reconnect on their own backoff schedule. -## Default Configuration +## Type Reference -Minimal working configuration: - -```toml -[[pipelines]] -name = "default" - -[[pipelines.plugin_sources]] -id = "default_source" -type = "file" -[pipelines.plugin_sources.config] -directory = "./" -pattern = "*.log" - -[[pipelines.plugin_sinks]] -id = "default_sink" -type = "console" -[pipelines.plugin_sinks.config] -target = "stdout" -``` - -## Configuration Schema - -### Type Reference - -| TOML Type | Go Type | Environment Format | -|-----------|---------|-------------------| -| String | string | Plain text | -| Integer | int64 | Numeric string | -| Float | float64 | Decimal string | -| Boolean | bool | true/false | -| Array | []T | JSON array string | -| Table | struct | Nested with `_` | +| TOML type | Go type | Command-line / environment form | +|-----------|---------|-------------------------------| +| String | `string` | Plain text | +| Integer | `int64` | Decimal string | +| Float | `float64` | Decimal string | +| Boolean | `bool` | `true` / `false`, or a bare flag for `true` | +| Array | `[]T` | Not settable outside the file | +| Table | struct | Nested path with `.` (flags) or `_` (environment) | diff --git a/doc/filters.md b/doc/filters.md index 961700b..1cf37bf 100644 --- a/doc/filters.md +++ b/doc/filters.md @@ -1,185 +1,192 @@ # Filters -LogWisp filters control which log entries pass through the pipeline using pattern matching. - -## Filter Types - -### Include Filter - -Only entries matching patterns pass through. +Filters decide which entries continue through a pipeline. They run in the flow, +after rate limiting and before formatting. ```toml [[pipelines.flow.filters]] -type = "include" -logic = "or" # or|and -patterns = [ - "ERROR", - "WARN", - "CRITICAL" -] +type = "include" +logic = "or" +patterns = ["ERROR", "WARN"] ``` -### Exclude Filter - -Entries matching patterns are dropped. - -```toml -[[pipelines.flow.filters]] -type = "exclude" -patterns = [ - "DEBUG", - "TRACE", - "health-check" -] -``` - -## Configuration Options +## Options | Option | Type | Default | Description | |--------|------|---------|-------------| -| `type` | string | Required | Filter type (include/exclude) | -| `logic` | string | "or" | Pattern matching logic (or/and) | -| `patterns` | []string | Required | Pattern list | +| `type` | string | `include` | `include` (only matches pass) or `exclude` (matches are dropped) | +| `logic` | string | `or` | `or` (any pattern matches) or `and` (every pattern matches) | +| `patterns` | []string | `[]` | Go RE2 regular expressions | + +A filter with no patterns passes everything. Invalid patterns fail at startup +with the filter index and the offending pattern in the message. + +## What Gets Matched + +Patterns are matched against a single string assembled from the entry: + +``` +" " +``` + +Empty parts are omitted, so an entry with no detected level matches +`" "`. This means a pattern can target the source name or the +level as easily as the message body: + +| Pattern | Matches | +|---------|---------| +| `"^app\\.log "` | Entries whose source is `app.log` | +| `"ERROR"` | Level `ERROR`, or the word `ERROR` anywhere in the message | + +The structured `fields` payload is **not** part of the match text. + +For entries that arrived over a chain link, the `source` used here is the bare +source — the `node/source` prefix is applied later, by the formatter — so +filtering by originating node requires matching on the message, or filtering on +the node that produces the entries. + +## Filter Types + +### include + +Only matching entries pass. Everything else is dropped. + +```toml +[[pipelines.flow.filters]] +type = "include" +patterns = ["ERROR", "WARN", "FATAL"] +``` + +### exclude + +Matching entries are dropped. Everything else passes. + +```toml +[[pipelines.flow.filters]] +type = "exclude" +patterns = ["/healthz", "TRACE"] +``` + +## Logic + +### or (default) + +```toml +logic = "or" +patterns = ["ERROR", "WARN"] +# passes: "ERROR in module" "WARN: low memory" +# blocks: "INFO: started" +``` + +### and + +```toml +logic = "and" +patterns = ["database", "ERROR"] +# passes: "ERROR: database connection failed" +# blocks: "ERROR: file not found" +``` + +With `logic = "and"` on an `exclude` filter, an entry is dropped only when it +matches *every* pattern. + +## Filter Chains + +Filters are evaluated in declaration order and an entry must survive all of +them. The first filter to reject an entry ends its life; later filters never see +it. + +```toml +# 1. keep only production traffic +[[pipelines.flow.filters]] +type = "include" +patterns = ["prod-", "production"] + +# 2. of that, keep only failures +[[pipelines.flow.filters]] +type = "include" +patterns = ["ERROR", "EXCEPTION", "FATAL"] + +# 3. minus known noise +[[pipelines.flow.filters]] +type = "exclude" +patterns = ["ECONNRESET", "broken pipe"] +``` + +Order matters for cost, not for correctness: put the most selective filter first +so later ones evaluate fewer entries. ## Pattern Syntax -Patterns support regular expression syntax: +Go's RE2 syntax. No backreferences and no lookaround — RE2 guarantees linear +time, which is exactly what you want in a log hot path. -### Basic Patterns -- **Literal match**: `"ERROR"` - matches "ERROR" anywhere -- **Case-insensitive**: `"(?i)error"` - matches "error", "ERROR", "Error" -- **Word boundary**: `"\\berror\\b"` - matches whole word only +| Need | Pattern | +|------|---------| +| Literal substring | `ERROR` | +| Case-insensitive | `(?i)error` | +| Whole word | `\\berror\\b` | +| Alternation | `ERROR\|WARN\|FATAL` | +| Character class | `[0-9]{3}` | +| Anchors | `^ERROR`, `ERROR$` | +| Any characters | `.*exception.*` | -### Advanced Patterns -- **Alternation**: `"ERROR|WARN|FATAL"` -- **Character classes**: `"[0-9]{3}"` -- **Wildcards**: `".*exception.*"` -- **Line anchors**: `"^ERROR"` (start), `"ERROR$"` (end) +Remember that TOML basic strings process escapes, so a regex backslash needs +doubling: `"\\berror\\b"`. TOML literal strings avoid the issue: +`'\berror\b'`. -### Special Characters -Escape special regex characters with backslash: -- `.` → `\\.` -- `*` → `\\*` -- `[` → `\\[` -- `(` → `\\(` +Anchors apply to the assembled match text, which begins with the source name — +so `^ERROR` will not match an entry whose source is non-empty. Use +`\\bERROR\\b` instead unless you mean to anchor on the source. -## Filter Logic +## Common Recipes -### OR Logic (default) -Entry passes if ANY pattern matches: -```toml -logic = "or" -patterns = ["ERROR", "WARN"] -# Passes: "ERROR in module", "WARN: low memory" -# Blocks: "INFO: started" -``` - -### AND Logic -Entry passes only if ALL patterns match: -```toml -logic = "and" -patterns = ["database", "ERROR"] -# Passes: "ERROR: database connection failed" -# Blocks: "ERROR: file not found" -``` - -## Filter Chain - -Multiple filters execute sequentially: +**Severity floor** ```toml -# First filter: Include errors and warnings [[pipelines.flow.filters]] -type = "include" -patterns = ["ERROR", "WARN"] - -# Second filter: Exclude test environments -[[pipelines.flow.filters]] -type = "exclude" -patterns = ["test-env", "staging"] +type = "include" +patterns = ["ERROR", "FATAL", "CRITICAL"] ``` -Processing order: -1. Entry arrives from source -2. Include filter evaluates -3. If passed, exclude filter evaluates -4. If passed all filters, entry continues to sink +**Noise reduction** -## Performance Considerations - -### Pattern Compilation -- Patterns compile once at startup -- Invalid patterns cause startup failure -- Complex patterns may impact performance - -### Optimization Tips -- Place most selective filters first -- Use simple patterns when possible -- Combine related patterns with alternation -- Avoid excessive wildcards (`.*`) - -## Filter Statistics - -Filters track: -- Total entries evaluated -- Entries passed -- Entries blocked -- Processing time per pattern - -## Common Use Cases - -### Log Level Filtering -```toml -[[pipelines.filters]] -type = "include" -patterns = ["ERROR", "WARN", "FATAL", "CRITICAL"] -``` - -### Application Filtering ```toml [[pipelines.flow.filters]] -type = "include" -patterns = ["app1", "app2", "app3"] +type = "exclude" +patterns = ["/healthz", "/metrics", "\\bping\\b"] ``` -### Noise Reduction +**Secret suppression** — see [Security](security.md); filters are the only +redaction mechanism LogWisp currently offers. + ```toml [[pipelines.flow.filters]] -type = "exclude" -patterns = [ - "health-check", - "ping", - "/metrics", - "heartbeat" -] +type = "exclude" +patterns = ["password", "api[_-]?key", "authorization", "bearer ", "secret", "token"] ``` -### Security Filtering -```toml -[[pipelines.flow.filters]] -type = "exclude" -patterns = [ - "password", - "token", - "api[_-]key", - "secret" -] -``` +Note this drops the whole entry, it does not redact part of it. -### Multi-stage Filtering -```toml -# Include production logs -[[pipelines.flow.filters]] -type = "include" -patterns = ["prod-", "production"] +**Per-application routing** — run one pipeline per application, each with its +own include filter, rather than trying to route inside one pipeline. Sinks fan +out to *all* sinks in a pipeline; there is no conditional routing. -# Include only errors -[[pipelines.flow.filters]] -type = "include" -patterns = ["ERROR", "EXCEPTION", "FATAL"] +## Statistics -# Exclude known issues -[[pipelines.flow.filters]] -type = "exclude" -patterns = ["ECONNRESET", "broken pipe"] -``` +Each filter reports `type`, `logic`, `pattern_count`, `total_processed`, +`total_matched`, and `total_dropped`. The chain reports `filter_count`, +`total_processed`, and `total_passed`; the pipeline derives +`total_filtered` as the difference. + +## Performance + +Patterns compile once at startup. Every entry that reaches the filter stage is +evaluated against every filter until one rejects it, so cost scales with the +number of patterns and their complexity. Prefer literal substrings and simple +alternations over broad `.*` wildcards. + +Filters log at DEBUG on every entry — pattern text, match results, and the +final decision. That is invaluable when a filter is not behaving as expected and +very expensive in production; keep `logging.level` at `info` or higher on a busy +pipeline. diff --git a/doc/formatters.md b/doc/formatters.md index 0918559..a68967e 100644 --- a/doc/formatters.md +++ b/doc/formatters.md @@ -1,180 +1,175 @@ # Formatters -LogWisp formatters transform log entries before output to sinks. +The formatter is the last flow stage. It turns a `core.LogEntry` into the byte +payload that sinks write, applying a sanitizer policy on the way. -## Formatter Types +```toml +[pipelines.flow.format] +type = "json" +sanitizer_policy = "json" +flags = 0 +timestamp_format = "" +``` -### Raw Formatter +One formatter serves the whole pipeline. Sinks receive an identical payload; +there is no per-sink formatting. When you need two shapes of the same data, run +two pipelines, or chain to a node that formats differently. -Outputs the log message as-is with optional newline. +Omitting `[pipelines.flow.format]` entirely selects `raw`. + +## Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `type` | string | `raw` | `raw`, `txt` (alias `text`), or `json` | +| `sanitizer_policy` | string | derived from `type` | `raw`, `txt`, `json`, or `shell` | +| `flags` | int64 | `0` | Bitmask override; `0` selects a per-type default | +| `timestamp_format` | string | formatter default | Go reference layout, e.g. `"2006-01-02T15:04:05Z07:00"` | + +## Types + +### raw + +Passthrough. `FlagRaw` bypasses both formatting and sanitization, so the +message reaches the sink exactly as the source produced it. ```toml [pipelines.flow.format] type = "raw" -sanitizer_policy = "raw" -flags = 1 ``` -**Configuration Options:** +Fastest option, and the right one when you are relaying text that is already in +its final form. Note that it also bypasses sanitization, so control characters +in the source data reach your sinks intact. -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `add_new_line` | bool | true | Append newline to messages | -| `type` | string | "raw" | raw, json, or txt | -| `flags` | int64 | 0 | log/formatter flags override | -| `sanitizer_policy` | string | | Sanitizer policy (e.g. "json", "raw", "txt", "shell") | +### txt -### JSON Formatter - -Produces structured JSON output. +Human-readable line output with a timestamp and level. ```toml -[pipelines.format] -type = "json" +[pipelines.flow.format] +type = "txt" +sanitizer_policy = "txt" +timestamp_format = "2006-01-02 15:04:05" +``` -+[pipelines.flow.format] - type = "json" +### json + +Structured output, the natural choice for downstream ingestion. + +```toml +[pipelines.flow.format] +type = "json" sanitizer_policy = "json" ``` -**Output Structure:** +Output has the shape: + ```json -{ - "timestamp": "2024-01-01T12:00:00Z", - "level": "ERROR", - "source": "app", - "message": "Connection failed" -} +{"time":"2026-01-02T15:04:05.123Z","level":"ERROR","trace":"edge-01/app.log","fields":["connection refused"]} ``` -### Text Formatter +The exact key names and structure come from the `lixenwraith/log` formatter, not +from LogWisp; they are stable for a given dependency version but are not part of +LogWisp's own configuration surface. -Template-based text formatting. +## Flags -```toml -[pipelines.flow.format] -type = "txt" -sanitizer_policy = "txt" -timestamp_format = "2006-01-02T15:04:05.000Z07:00" +`flags` is a bitmask passed to the underlying formatter. Leave it at `0` unless +you need to override the defaults. + +| Value | Name | Effect | +|-------|------|--------| +| `1` | Raw | Bypass formatting and sanitization entirely | +| `2` | ShowTimestamp | Emit the timestamp | +| `4` | ShowLevel | Emit the level | +| `8` | StructuredJSON | Render attached fields as a JSON object | +| `16` | NoTimestamp | Suppress the timestamp | +| `32` | NoLevel | Suppress the level | + +With `flags = 0` the formatter selects `1` for `type = "raw"` and `6` +(timestamp + level) for every other type. `8` is added automatically whenever an +entry carries parseable `fields`. + +Examples: `flags = 4` for level only, no timestamp; `flags = 2` for timestamp +only, no level. + +## Sanitizer Policies + +The sanitizer runs before serialization and neutralizes control characters that +would otherwise break framing or reach a terminal. + +| Policy | Behaviour | Use with | +|--------|-----------|----------| +| `raw` | No-op passthrough | `type = "raw"` where you control the data | +| `txt` | Escapes non-printable characters | File and console sinks | +| `json` | Escapes control characters for safe JSON embedding | `type = "json"`, chain links | +| `shell` | Strips shell metacharacters, whitespace, and control characters | Data that will be passed to a command | + +When `sanitizer_policy` is omitted, the policy is derived from `type`: `json` +for `json`, `txt` for `txt`/`text`, and `raw` for anything else — so the safe +pairing is the default. + +> `shell` strips dangerous characters but is **not** sufficient to make a string +> safe for shell construction. Pass arguments through `exec` argv instead of +> building command lines. + +To see a policy working, point a pipeline at the `random` source with +`special = true`, which injects control bytes and multi-byte Unicode into every +message. + +## Node Identity in Output + +Entries that arrived over a chain link carry a `Node` label. The formatter +renders it as a syslog-style prefix on the source field: + +``` +edge-01/app.log ``` -**Configuration Options:** +Entries with no node label show the bare source. Node identity therefore appears +*inside* the source field rather than as a separate output key — worth knowing +when writing downstream parsers or grep patterns. -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `timestamp_format` | string | "" | Time format override | +## Structured Fields -**Default Template:** -``` -[{{.Timestamp | FmtTime}}] [{{.Level | ToUpper}}] {{.Source}} - {{.Message}}{{ if .Fields }} {{.Fields}}{{ end }} -``` +When an entry carries `Fields` (raw JSON), the formatter parses it and switches +to structured rendering by adding the `StructuredJSON` flag automatically. +Fields reach a pipeline in two ways: from the `file` source when a tailed line +parses as JSON with a `fields` key, and from the heartbeat generator when +`include_stats = true`. -## Template Functions +## Choosing a Configuration -Available functions in text templates: +| Goal | Configuration | +|------|---------------| +| Maximum throughput, data already formatted | `type = "raw"` | +| Human reading in a terminal or file | `type = "txt"`, `sanitizer_policy = "txt"` | +| Downstream ingestion (Loki, Elasticsearch, jq) | `type = "json"`, `sanitizer_policy = "json"` | +| Compact console output | `type = "txt"`, `flags = 4` | +| Untrusted log content | never `raw`; pick `txt` or `json` and set the matching policy | -| Function | Description | Example | -|----------|-------------|---------| -| `FmtTime` | Format timestamp | `{{.Timestamp \| FmtTime}}` | -| `ToUpper` | Convert to uppercase | `{{.Level \| ToUpper}}` | -| `ToLower` | Convert to lowercase | `{{.Source \| ToLower}}` | -| `TrimSpace` | Remove whitespace | `{{.Message \| TrimSpace}}` | +## Formatting and Chain Links -## Template Variables +Chain sinks (`tcp_chain`, `http_chain`) do **not** ship the formatted payload. +They re-serialize the structured entry into the canonical chain encoding, which +makes them independent of the local formatter. -Available variables in templates: +The practical consequence: setting `flow.format` on an edge node changes only +that node's own local sinks. The output shape seen by a human or a downstream +system is decided on the node that owns the sink they read. -| Variable | Type | Description | -|----------|------|-------------| -| `.Timestamp` | time.Time | Entry timestamp | -| `.Level` | string | Log level | -| `.Source` | string | Source identifier | -| `.Message` | string | Log message | -| `.Fields` | string | Additional fields (JSON) | +If an event ever reaches a chain sink without a structured entry, the sink wraps +the formatted payload into a synthetic entry and counts it in `synthesized`. +A non-zero `synthesized` count means something upstream lost structure. -## Time Format Strings +## Performance -Common Go time format patterns: +Relative cost, cheapest first: `raw` (passthrough) → `txt` (line assembly) → +`json` (serialization). Sanitization adds a scan of the message; the `raw` +policy skips it. -| Pattern | Example Output | -|---------|---------------| -| `2006-01-02T15:04:05Z07:00` | 2024-01-02T15:04:05Z | -| `2006-01-02 15:04:05` | 2024-01-02 15:04:05 | -| `Jan 2 15:04:05` | Jan 2 15:04:05 | -| `15:04:05.000` | 15:04:05.123 | -| `2006/01/02` | 2024/01/02 | - -## Format Selection - -### Default Behavior - -If no formatter specified: -- **HTTP/TCP sinks**: JSON format -- **Console/File sinks**: Raw format -- **Client sinks**: JSON format - -### Per-Pipeline Configuration - -Each pipeline can have its own formatter: - -```toml -[[pipelines]] -name = "json-pipeline" -[pipelines.flow.format] -type = "json" - -[[pipelines]] -name = "text-pipeline" -[pipelines.flow.format] -type = "txt" -``` - -## Message Processing - -### JSON Message Handling - -When using JSON formatter with JSON log messages: -1. Attempts to parse message as JSON -2. Merges fields with LogWisp metadata -3. LogWisp fields take precedence -4. Falls back to string if parsing fails - -### Field Preservation - -LogWisp metadata always includes: -- Timestamp (from source or current time) -- Level (detected or default) -- Source (origin identifier) -- Message (original content) - -## Performance Characteristics - -### Formatter Performance - -Relative performance (fastest to slowest): -1. **Raw**: Direct passthrough -2. **Text**: Template execution -3. **JSON**: Serialization -4. **JSON (pretty)**: Formatted serialization - -### Optimization Tips - -- Use raw format for high throughput -- Cache template compilation (automatic) -- Minimize template complexity -- Avoid pretty JSON in production - -## Common Configurations - -### Structured Logging -```toml -[pipelines.flow.format] -type = "json" -``` - -### Human-Readable Logs -```toml -[pipelines.flow.format] -type = "txt" -timestamp_format = "15:04:05" -``` +The formatter holds a mutex because the underlying implementation reuses an +internal buffer and is not goroutine-safe. It is the only shared serialization +point in the hot path, and the reason a single pipeline formats entries one at +a time. diff --git a/doc/installation.md b/doc/installation.md index 393bea2..f370604 100644 --- a/doc/installation.md +++ b/doc/installation.md @@ -1,49 +1,62 @@ # Installation Guide -LogWisp installation and service configuration for Linux and FreeBSD systems. +## Requirements -## Installation Methods +- **Operating systems**: Linux (kernel 6.10+), FreeBSD (14.0+) +- **Architecture**: amd64 +- **Go**: 1.26 or newer, to build from source -### Pre-built Binaries - -Download the latest release binary for your platform and install to `/usr/local/bin`: +## Building from Source ```bash -# Linux amd64 -wget https://github.com/yourusername/logwisp/releases/latest/download/logwisp-linux-amd64 -chmod +x logwisp-linux-amd64 -sudo mv logwisp-linux-amd64 /usr/local/bin/logwisp - -# FreeBSD amd64 -fetch https://github.com/yourusername/logwisp/releases/latest/download/logwisp-freebsd-amd64 -chmod +x logwisp-freebsd-amd64 -sudo mv logwisp-freebsd-amd64 /usr/local/bin/logwisp -``` - -### Building from Source - -Requires Go 1.24 or newer: - -```bash -git clone https://github.com/yourusername/logwisp.git +git clone https://github.com/lixenwraith/logwisp.git cd logwisp -go build -o logwisp ./src/cmd/logwisp -sudo install -m 755 logwisp /usr/local/bin/ +make +sudo make install # installs to $PREFIX/bin, default /usr/local/bin ``` -### Go Install Method +The Makefile works with both GNU make and BSD make. Targets: -Install directly using Go (version information will not be embedded): +| Target | Effect | +|--------|--------| +| `make` / `make build` | Build `bin/logwisp` with version metadata | +| `make dev` | Build with the race detector enabled | +| `make install` | Install the binary to `$(PREFIX)/bin` (default `/usr/local`) | +| `make uninstall` | Intended to remove the installed binary — currently broken: it expands to `$(BINDIR)/bin/logwisp` instead of `$(BINDIR)/logwisp`, so it removes nothing. Delete the binary by hand | +| `make clean` | Remove the built binary | +| `make version` | Print the version, commit, and build time that would be embedded | + +Version, commit hash, and build time are injected via `-ldflags` from `git +describe` and `git rev-parse`. A plain `go build` produces a working binary that +reports `dev` for all three: ```bash -go install github.com/yourusername/logwisp/cmd/logwisp@latest +go build -o bin/logwisp ./cmd/logwisp ``` -## Service Configuration +`go install github.com/lixenwraith/logwisp/cmd/logwisp@latest` also works, with +the same loss of version metadata. + +## Configuration + +Copy the annotated reference configuration and edit it: + +```bash +sudo mkdir -p /etc/logwisp +sudo cp config/logwisp.toml /etc/logwisp/logwisp.toml +``` + +LogWisp searches, in order: `-c `, `--config=`, +`$LOGWISP_CONFIG_DIR`/`$LOGWISP_CONFIG_FILE`, `~/.config/logwisp/logwisp.toml`, +`./logwisp.toml`. See [Configuration](configuration.md). + +## Running as a Service + +LogWisp has no daemon mode; run it in the foreground under a supervisor. ### Linux (systemd) -Create systemd service file `/etc/systemd/system/logwisp.service`: +`/etc/systemd/system/logwisp.service`: ```ini [Unit] @@ -55,30 +68,47 @@ Type=simple User=logwisp Group=logwisp ExecStart=/usr/local/bin/logwisp -c /etc/logwisp/logwisp.toml +ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure RestartSec=10 +WorkingDirectory=/var/lib/logwisp StandardOutput=journal StandardError=journal -WorkingDirectory=/var/lib/logwisp + +# Hardening +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/var/log/logwisp /var/lib/logwisp [Install] WantedBy=multi-user.target ``` -Setup service user and directories: +`ExecReload` gives you `systemctl reload logwisp` for configuration and +certificate rotation without dropping the process. + +If a pipeline binds a port below 1024, add +`AmbientCapabilities=CAP_NET_BIND_SERVICE` rather than running as root. + +Setup: ```bash -sudo useradd -r -s /bin/false logwisp +sudo useradd -r -s /usr/sbin/nologin logwisp sudo mkdir -p /etc/logwisp /var/lib/logwisp /var/log/logwisp sudo chown logwisp:logwisp /var/lib/logwisp /var/log/logwisp sudo systemctl daemon-reload -sudo systemctl enable logwisp -sudo systemctl start logwisp +sudo systemctl enable --now logwisp ``` +The service account needs **read** access to every directory a `file` source +watches and **write** access to every directory a `file` sink or +`logging.file` writes to. + ### FreeBSD (rc.d) -Create rc script `/usr/local/etc/rc.d/logwisp`: +`/usr/local/etc/rc.d/logwisp`: ```sh #!/bin/sh @@ -92,8 +122,9 @@ Create rc script `/usr/local/etc/rc.d/logwisp`: name="logwisp" rcvar="${name}_enable" pidfile="/var/run/${name}.pid" -command="/usr/local/bin/logwisp" -command_args="-c /usr/local/etc/logwisp/logwisp.toml" +procname="/usr/local/bin/logwisp" +command="/usr/sbin/daemon" +command_args="-p ${pidfile} -f ${procname} -c /usr/local/etc/logwisp/logwisp.toml" load_rc_config $name : ${logwisp_enable:="NO"} @@ -101,7 +132,7 @@ load_rc_config $name run_rc_command "$1" ``` -Setup service: +Setup: ```bash sudo chmod +x /usr/local/etc/rc.d/logwisp @@ -112,45 +143,66 @@ sudo sysrc logwisp_enable="YES" sudo service logwisp start ``` -## Directory Structure - -Standard installation directories: +## Directory Layout | Purpose | Linux | FreeBSD | |---------|-------|---------| | Binary | `/usr/local/bin/logwisp` | `/usr/local/bin/logwisp` | | Configuration | `/etc/logwisp/` | `/usr/local/etc/logwisp/` | -| Working Directory | `/var/lib/logwisp/` | `/var/db/logwisp/` | -| Log Files | `/var/log/logwisp/` | `/var/log/logwisp/` | -| PID File | `/var/run/logwisp.pid` | `/var/run/logwisp.pid` | +| TLS material | `/etc/logwisp/tls/` | `/usr/local/etc/logwisp/tls/` | +| Working directory | `/var/lib/logwisp/` | `/var/db/logwisp/` | +| Application logs | `/var/log/logwisp/` | `/var/log/logwisp/` | -## Post-Installation Verification +Key files should be mode `0600` and owned by the service account. -Verify the installation: +## Verification ```bash -# Check version -logwisp version +logwisp --version -# Test configuration -logwisp -c /etc/logwisp/logwisp.toml --disable-status-reporter +# start in the foreground with debug logging and watch pipelines come up +logwisp -c /etc/logwisp/logwisp.toml --logging.level=debug --logging.output=stderr -# Check service status (Linux) -sudo systemctl status logwisp - -# Check service status (FreeBSD) -sudo service logwisp status +sudo systemctl status logwisp # Linux +sudo service logwisp status # FreeBSD ``` -## Uninstallation +Expect `Created source instance`, `Created sink instance`, and +`Starting pipeline` for each configured pipeline. There is no validate-only +mode; see [Operations](operations.md#checking-a-configuration). + +## Test Scripts + +Two end-to-end scripts under `test/` build multi-node chain topologies against a +local build: + +```bash +make +./test/chain-test.sh --auto # two independent relay pipelines +./test/chain-aggregate-test.sh --auto # fan-in: both edges into one pipeline +``` + +Without `--auto` they run the relay in the foreground for interactive +inspection. They need bash 5+, coreutils, and curl, and they bind ports +15801–15804. Generated configuration and logs land in `test/run/`. + +> Two of the three `--auto` assertions currently report `FAIL` against a +> working build. They grep the sink output for `"source":"edge-tcp/` and +> `"node":"edge-http"`, but the JSON formatter emits the `node/source` label +> under the key `trace`. The transport itself is healthy — the +> `total_processed` assertion passes and the streamed entries carry +> `"trace":"edge-tcp/random_rand"` as expected. Until the assertions are +> updated, verify the streams by eye with `nc 127.0.0.1 15803` and +> `curl -sN http://127.0.0.1:15804/stream`. + +## Uninstall ### Linux ```bash -sudo systemctl stop logwisp -sudo systemctl disable logwisp -sudo rm /usr/local/bin/logwisp -sudo rm /etc/systemd/system/logwisp.service +sudo systemctl disable --now logwisp +sudo rm /usr/local/bin/logwisp /etc/systemd/system/logwisp.service +sudo systemctl daemon-reload sudo rm -rf /etc/logwisp /var/lib/logwisp /var/log/logwisp sudo userdel logwisp ``` @@ -160,8 +212,7 @@ sudo userdel logwisp ```bash sudo service logwisp stop sudo sysrc -x logwisp_enable -sudo rm /usr/local/bin/logwisp -sudo rm /usr/local/etc/rc.d/logwisp +sudo rm /usr/local/bin/logwisp /usr/local/etc/rc.d/logwisp sudo rm -rf /usr/local/etc/logwisp /var/db/logwisp /var/log/logwisp sudo pw userdel logwisp -``` \ No newline at end of file +``` diff --git a/doc/mtls-auth-plan.md b/doc/mtls-auth-plan.md new file mode 100644 index 0000000..0940334 --- /dev/null +++ b/doc/mtls-auth-plan.md @@ -0,0 +1,368 @@ +# Implementation Plan: mTLS as Authentication + +**Status:** proposal, not implemented. +**Scope:** turn the existing transport-level mutual TLS into a real +authentication and authorization mechanism. + +## Problem + +LogWisp already does mutual TLS at the transport layer. A listener with +`client_auth = true` refuses any peer that cannot present a certificate +chaining to `client_ca_file`, and `internal/tlsx` already extracts the peer's +Common Name and stashes it in session metadata as `tls_peer_cn`. + +Nothing reads it back. The result is a CA-wide membership check with no notion +of *which* peer connected: + +1. **No per-identity authorization.** Every certificate the CA issues is + equivalent. There is no way to say "only `edge-01` and `edge-02` may write to + this ingest port", so one CA cannot serve several trust domains, and + withdrawing one peer means rotating the CA bundle for all of them. +2. **Node labels are unauthenticated.** With `trust_node = true` (the default) a + peer declares its own `node` label in the chain hello or the + `X-Logwisp-Node` header. Any certificate holder can claim any label, + including another host's, and every downstream consumer will attribute those + entries accordingly. The only current defence, `trust_node = false`, replaces + the label with a remote address — unforgeable, but useless for identifying a + host behind NAT or a load balancer. +3. **The `http` sink has no authentication at all**, even with TLS on. Its + stream and status endpoints are readable by anyone who can reach the port. + +Password, token, and SCRAM authentication were removed during the plugin/flow +restructure and the move to standard-library networking. Certificates are the +one credential the current transport already carries, which makes mTLS the +cheapest path back to authenticated peers. + +## Goals + +- Authorize peers by certificate identity, per listener. +- Bind the chain `node` label to the authenticated identity, so origin + attribution is trustworthy. +- Gate the `http` sink's endpoints on client certificates. +- Make identity visible in sessions, statistics, and logs. +- Change nothing for existing configurations that omit the new block. + +## Non-Goals + +- Reviving password, token, or SCRAM authentication. The reserved hooks + (`chain.Hello.Auth`, the `Authorization` header comment in the `http_chain` + sink, the "Future: password auth block" comments in the network options + structs) stay reserved. +- IP allow/deny lists and per-peer rate limits. Related, but a separate feature + with its own config surface. +- OCSP. See [Revocation](#revocation) for what is proposed instead. +- Authorization *within* a stream — the unit of decision is a connection (TCP) + or a request (HTTP), never an individual entry. + +## Design + +### Identity + +The authenticated identity is a single string derived from the peer's verified +leaf certificate. Because `tls.RequireAndVerifyClientCert` has already validated +the chain, signature, and validity window by the time we look, extraction is +pure field selection. + +| `identity` mode | Source | Notes | +|-----------------|--------|-------| +| `cn` (default) | `Subject.CommonName` | Matches the existing `tls_peer_cn` metadata | +| `san_dns` | first `DNSNames` entry | Preferred for host identities | +| `san_uri` | first `URIs` entry | SPIFFE-style IDs | +| `san_email` | first `EmailAddresses` entry | Operator identities | + +An empty identity is a rejection, not an empty match: a certificate with no +usable identity field cannot satisfy any policy. + +Identities are not secrets, so ordinary string comparison is fine; there is no +timing side channel worth defending here. + +### Configuration + +A new `auth` table sits beside `tls` in every network plugin's `config`. Keeping +it separate from `tls` matters: TLS answers "is this channel private and is the +peer chained to a CA", auth answers "may *this* peer do *this*", and a later +non-certificate method should be able to reuse the block. + +```toml +[pipelines.plugin_sources.config.auth] +type = "mtls" # none (default) | mtls +identity = "cn" # cn | san_dns | san_uri | san_email +allow = ["edge-01", "edge-02"] # exact identities +allow_patterns = ["^edge-\\d{2}$"] # RE2, anchored by the author +node_binding = "force" # none | assert | force +``` + +| Option | Type | Default | Meaning | +|--------|------|---------|---------| +| `type` | string | `none` | `none` preserves today's behaviour exactly; `mtls` enables the policy | +| `identity` | string | `cn` | Which certificate field is the identity | +| `allow` | []string | `[]` | Exact identity matches | +| `allow_patterns` | []string | `[]` | RE2 patterns matched against the identity | +| `node_binding` | string | `force` when `type = "mtls"` | See below | + +Empty `allow` **and** empty `allow_patterns` under `type = "mtls"` means "any +identity the CA vouches for" — that is, today's behaviour, but with the identity +now recorded and node binding available. It is a deliberate, documented default +rather than a silent deny-all, and startup logs say so plainly. + +`node_binding` applies only to the chain sources, where a `node` label is +declared: + +| Value | Behaviour | +|-------|-----------| +| `none` | `trust_node` governs, as today | +| `assert` | The declared label must equal the identity; a mismatch is rejected | +| `force` | The declared label is ignored and the identity is used | + +`force` is the default under `type = "mtls"` because it is the only setting +where a misconfigured or hostile edge cannot mislabel its entries. `assert` +exists for operators who want the mismatch to be loud rather than silently +corrected. `node_binding` overrides `trust_node`; when both are set, the +constructor logs that `trust_node` is being ignored. + +For the `tcp` and `http` sinks, which have no node concept, `node_binding` is +ignored. + +Dialer-side plugins (`tcp_chain` and `http_chain` sinks) accept the same block +to pin the *server's* identity beyond hostname verification. This is deferred to +phase 4 and does nothing before then. + +### Validation + +At plugin construction, before anything binds: + +- `type = "mtls"` on a listener requires `tls.enabled = true` and + `tls.client_auth = true`. Silently accepting an auth policy the transport + cannot enforce is the failure mode worth designing out. +- `identity` must be one of the four modes. +- Every entry in `allow_patterns` must compile. +- `node_binding` must be one of the three values. + +Errors follow existing style: `auth: type "mtls" requires tls.client_auth`. + +### New package: `internal/authz` + +```go +package authz + +// Policy is the compiled form of config.AuthOptions. +type Policy struct { /* mode, identity selector, exact set, patterns, binding */ } + +// New compiles a policy. Returns (nil, nil) when auth is disabled, matching +// the tlsx.Server / tlsx.Client convention so callers can nil-check. +func New(o *config.AuthOptions) (*Policy, error) + +// Identity is the outcome of a successful authorization. +type Identity struct { + Name string // the selected certificate field + Method string // "mtls" +} + +// Authorize extracts and checks the peer identity from a completed handshake. +func (p *Policy) Authorize(cs *tls.ConnectionState) (Identity, error) + +// ResolveNode applies node_binding to a declared label. +func (p *Policy) ResolveNode(declared string, id Identity) (string, error) + +// Stats reports counters for the sink/source stats map. +func (p *Policy) Stats() map[string]any +``` + +This mirrors `internal/tlsx`: one small package that is the single seam between +declarative config and a cross-cutting concern, with `(nil, nil)` for the +disabled case so every call site is a nil check rather than a branch on config. + +### Enforcement Points + +Each plugin already has the right spot, and in two cases the code says so. + +**`tcp_chain` source** (`internal/source/tcpchain/tcpchain.go`, `handleConn`) + +Today: handshake → read hello → decode → resolve node from `trust_node` → +create session. Insert authorization between the handshake and the hello read, +so an unauthorized peer never gets a preamble parsed on its behalf, and replace +the node resolution with `Policy.ResolveNode`. + +```go +if tlsState != nil && s.authPolicy != nil { + id, err := s.authPolicy.Authorize(*tlsState) + if err != nil { + s.authRejected.Add(1) + s.logger.Warn("msg", "Connection rejected by auth policy", + "component", "tcp_chain_source", "remote_addr", remote, "error", err) + return // deferred cleanup closes conn + } + ident = id +} +// ... read and decode hello ... +connNode, err = s.authPolicy.ResolveNode(hello.Node, ident) +``` + +**`http_chain` source** (`internal/source/httpchain/httpchain.go`, `handleIngest`) + +Per request, from `r.TLS`, before the body is read — an unauthorized sender +should not get to stream 8 MiB into the process. Rejection is `403`, distinct +from the `400` used for protocol errors, so a sender can tell "you are not +allowed" from "your batch was malformed". `ResolveNode` then governs the +`X-Logwisp-Node` header exactly as it governs the TCP hello. + +**`tcp` sink** (`internal/sink/tcp/tcp.go`, `handleConn`) + +There is already a comment marking the place: *"Password-auth extension point: +preamble verification runs in handleConn post-handshake, pre-registration."* +Authorization goes precisely there — after the explicit handshake, before the +session is created and the client is registered — so an unauthorized peer never +appears in the client map and never receives a broadcast. + +**`http` sink** (`internal/sink/http/http.go`, `Start`) + +Also already marked: *"Auth extension point: wrap mux with auth middleware once +credentials land, e.g. handler = authMiddleware(cfg)(handler)."* A middleware +around the mux covers both the stream and the status endpoint with one wrapper, +and keeps the handlers themselves unaware of authorization. + +```go +var handler http.Handler = mux +if h.authPolicy != nil { + handler = authMiddleware(h.authPolicy, h.logger)(handler) +} +``` + +The middleware rejects with `403` and no body detail — the status endpoint +leaks host, port, and throughput counters, so a rejection should not leak policy +shape on top of it. + +### Capabilities + +`core.CapAuth` is currently derived from `tlsConfig.ClientAuth`. It should +reflect the auth policy instead: + +```go +if s.authPolicy != nil { + caps = append(caps, core.CapAuth) +} +``` + +`Pipeline.initSourceCapabilities` and `initSinkCapabilities` treat `CapAuth` as +a no-op placeholder today. They become the natural place for a cross-cutting +check: a plugin advertising `CapAuth` without `CapTLS` is a contradiction and +should fail pipeline construction rather than start. + +### Observability + +Every authorization decision must be visible, because a silent deny is +indistinguishable from a network fault at 3am. + +- **Session metadata** gains `auth_method` and `auth_identity` alongside the + existing `tls` and `tls_peer_cn`. +- **Statistics** gain `auth_enabled`, `auth_rejected`, and `node_binding` in the + `details` map of every affected source and sink, so rejections show up in the + status reporter and the `http` sink's status endpoint. +- **Logs** record a WARN per rejection with the remote address, the extracted + identity (or the reason extraction failed), and the policy that rejected it. + Accepted connections log the identity at INFO on the chain sources and at + DEBUG on the sinks, matching each plugin's existing verbosity. + +### Revocation + +Certificate revocation is deliberately handled by the allow-list rather than by +CRL or OCSP: + +1. Remove the identity from `allow` / `allow_patterns`. +2. `kill -HUP`. + +The reload path already rebuilds every pipeline, so the policy takes effect on +the next connection and existing connections are dropped by the rebuild itself. +This is one moving part instead of three, it needs no network calls on the +handshake path, and it is exact — no window between revocation and the next CRL +publication. + +A CRL file loaded next to `client_ca_file` and re-read on reload is a reasonable +phase-4 addition for operators with existing CRL infrastructure. OCSP stapling +is out of scope: it adds a network dependency to the handshake path for a +system whose entire design is "never block". + +## Phases + +### Phase 1 — Identity and policy (no enforcement) + +- `internal/config`: add `AuthOptions` plus an `Auth *AuthOptions` field to the + four listener option structs. +- `internal/tlsx`: add `PeerIdentity(cs tls.ConnectionState, mode string) string` + beside the existing `PeerCN`. +- `internal/authz`: new package — `Policy`, `New`, `Authorize`, `ResolveNode`, + `Stats`. +- Unit tests: identity extraction per mode, exact and pattern matching, empty + policy, malformed patterns, node binding in all three modes. + +Nothing behaves differently yet, which makes this phase safe to merge alone. + +### Phase 2 — Chain sources + +- Wire `authz` into `tcp_chain` and `http_chain` sources at the points above. +- Replace the `trust_node` node resolution with `ResolveNode`. +- Add validation, capabilities, statistics, and session metadata. +- Add `test/mtls-chain-test.sh`, modelled on `test/chain-test.sh`: generate a + CA, a server certificate, and two client certificates with `openssl`; assert + that an allowed identity delivers entries, a non-allowed identity is rejected, + a peer with no certificate fails the handshake, and a peer declaring another + node's label is rejected under `assert` and corrected under `force`. + +This phase alone closes the node-spoofing hole, which is the sharpest of the +three problems. + +### Phase 3 — Sinks + +- `tcp` sink: authorize in `handleConn` before registration. +- `http` sink: `authMiddleware` around the mux, covering stream and status. +- Extend the test script to cover both. + +### Phase 4 — Optional hardening + +- Dialer-side server identity pinning for the chain sinks. +- CRL file support alongside `client_ca_file`, re-read on reload. +- Certificate expiry warnings at startup and on reload — a leaf expiring inside + 30 days logged at WARN, since nothing warns today. +- Surface `tls_peer_cn` / `auth_identity` in the `http` sink's status output for + connected clients. + +## Compatibility + +No configuration breaks. Omitting the `auth` block, or setting +`type = "none"`, reproduces current behaviour byte for byte: `Policy` is nil, +every call site short-circuits, and `trust_node` continues to govern node +labels. + +The one behavioural note for adopters: turning on `type = "mtls"` defaults +`node_binding` to `force`, so entries from a peer whose certificate identity +differs from its configured `node` label will be relabelled. That is the point +of the feature, but it will move data between labels in a dashboard, so the +release note should say it in those terms. + +## Estimated Cost + +| Phase | Files touched | Rough size | +|-------|--------------|-----------| +| 1 | `config/config.go`, `config/validate.go`, `tlsx/tlsx.go`, new `authz/` + tests | ~400 lines | +| 2 | Two chain sources, new test script | ~200 lines | +| 3 | `sink/tcp`, `sink/http`, test script extension | ~150 lines | +| 4 | `tlsx`, both chain sinks, `sink/http` status | ~250 lines | + +Phases 1–2 are the security-relevant core; phase 3 closes the read-side +exposure; phase 4 is discretionary. + +## Open Questions + +1. **Should an empty allow-list deny instead of allow?** Deny-by-default is the + safer instinct, but it makes `type = "mtls"` with no list a footgun that + silently drops all traffic. The proposal is allow-with-a-loud-startup-log; + the alternative is requiring a non-empty list and erroring at construction, + which is arguably better and costs one line. +2. **Should `identity` accept a list of modes** (try `san_uri`, fall back to + `cn`)? Simpler as a single mode; heterogeneous PKI is the argument against. +3. **Per-identity rate limits.** The natural follow-on once identity exists, and + the natural home for the per-IP limiting that was also removed. Deliberately + out of scope here so this feature stays reviewable. +4. **Whether `assert` should reject or warn-and-correct.** As proposed it + rejects, which is unambiguous but turns a certificate/config mismatch into an + outage. `force` is the forgiving option, and it is the default. diff --git a/doc/networking.md b/doc/networking.md index 272a892..060bbec 100644 --- a/doc/networking.md +++ b/doc/networking.md @@ -1,95 +1,185 @@ # Networking -*Note: Under redesign* +Everything LogWisp does over a socket, and the knobs that shape it. For +certificates and trust see [Security](security.md); for multi-node topologies +see [Chaining](chaining.md). -## TLS Configuration +## Address Family -*Note: As of the latest architecture updates, network TLS, mTLS, and Rate Limiting features are undergoing a redesign and are currently acting as placeholders. The documentation below details the structure for future updates.* - -## Connection Management +**All listeners bind `tcp4` and all dialers dial `tcp4`.** IPv6 is not +supported, deliberately. An IPv6 client cannot connect and will simply see a +connection failure. -### TCP Keep-Alive +When testing locally use `127.0.0.1`, not `localhost` — the latter may resolve +to `::1` and appear as an unexplained connection refusal. -```toml -[[pipelines.plugin_sinks]] -id = "tcp_out" -type = "tcp" -[pipelines.plugin_sinks.config] -keep_alive = true -keep_alive_period_ms = 30000 # 30 seconds +## Network Plugins + +| Plugin | Role | Protocol | Purpose | +|--------|------|----------|---------| +| `tcp` sink | Listener | Raw stream | Broadcast formatted payloads to clients | +| `http` sink | Listener | HTTP SSE | Browser-friendly live stream plus status JSON | +| `tcp_chain` source | Listener | Chain v1 | Ingest a persistent NDJSON stream | +| `http_chain` source | Listener | Chain v1 | Ingest NDJSON batches over POST | +| `tcp_chain` sink | Dialer | Chain v1 | Forward entries over a persistent connection | +| `http_chain` sink | Dialer | Chain v1 | Forward entries as batched POSTs | + +There is no port registry and no default port: `port` is required on every +network plugin. There is also no cross-pipeline conflict detection — two sinks +on the same port fail at bind time when the pipeline starts: + +``` +ERROR msg="Failed to start sink" error="tcp sink bind 0.0.0.0:9090: listen tcp4 0.0.0.0:9090: bind: address already in use" ``` -### Connection Timeouts +## Timeouts + +Every network plugin exposes the deadlines relevant to its role. Zero means "no +deadline" wherever the table says so. + +| Plugin | Option | Default | Bounds | +|--------|--------|---------|--------| +| `tcp` sink | `write_timeout_ms` | `5000` | One write to one client; a miss disconnects that client | +| `http` sink | `write_timeout_ms` | `0` (none) | One SSE event write | +| `tcp_chain` source | `hello_timeout_ms` | `10000` | Reading the protocol preamble | +| `tcp_chain` source | `read_timeout_ms` | `0` (none) | Idle time between entries | +| `http_chain` source | `read_timeout_ms` | `30000` | Reading a whole request body | +| `tcp_chain` sink | `dial_timeout_ms` | `5000` | TCP connect | +| `tcp_chain` sink | `write_timeout_ms` | `5000` | One line write | +| `http_chain` sink | `request_timeout_ms` | `10000` | Dial plus write plus response | + +Fixed, non-configurable bounds: + +| Bound | Value | Applies to | +|-------|-------|------------| +| TLS handshake | 10 s | All TLS listeners and dialers | +| HTTP read-header timeout | 10 s | `http` sink, `http_chain` source | +| HTTP server shutdown grace | 2 s | `http` sink, `http_chain` source | +| Max single entry line | 1 MiB | Chain listeners | + +The `http` sink deliberately leaves the server's `WriteTimeout` unset, since it +would terminate long-lived SSE streams; per-event deadlines come from +`write_timeout_ms` instead. + +## Connection Limits + +`max_connections` caps concurrent connections on the `tcp` sink, the `http` +sink, and the `tcp_chain` source. `0` means unlimited. + +- Admission is a load-then-check, so a burst can over-admit by roughly one + connection. This is accepted, not a bug to work around. +- On the `tcp` sink and `tcp_chain` source the count is taken at accept, so it + bounds concurrent TLS handshakes as well as established sessions. +- Over-limit connections are closed immediately and counted in `rejected_conns` + (TCP) or `rejected_clients` (HTTP, which first answers `503`). + +The `http_chain` source has no connection cap; it bounds work with +`max_body_bytes` and `read_timeout_ms` instead. + +There is **no** per-IP limiting and no IP allow/deny list. `flow.rate_limit` is +a pipeline-wide entry rate limit, not a network-level one — it cannot +distinguish or throttle an individual peer. + +## Keep-Alive + +TCP keep-alive is available on the `tcp` sink (for accepted connections) and the +`tcp_chain` sink (for its outbound connection): ```toml -[[pipelines.plugin_sinks]] -id = "http_out" -type = "http" -[pipelines.plugin_sinks.config] -write_timeout_ms = 10000 # 10 seconds +keep_alive = true +keep_alive_period_ms = 30000 ``` -## Heartbeat Configuration +This is kernel-level keep-alive; it detects a dead peer but does not keep an +application-level stream flowing. For that, use a heartbeat. -Keep connections alive with periodic heartbeats. Note that Heartbeat is a flow-level feature in the new architecture. +## Heartbeats + +Heartbeats are a **flow-level** feature, not a per-sink one. Enabling one +injects a synthetic entry into the pipeline at a fixed interval; it reaches +every sink and traverses chain links as an ordinary structured entry. ```toml [pipelines.flow.heartbeat] -enabled = true -interval_ms = 30000 +enabled = true +interval_ms = 30000 include_timestamp = true -include_stats = false -format = "comment" # comment|event|json +include_stats = false +format = "txt" # txt | json | raw ``` -## Network Protocols +Use it to keep idle SSE clients, TCP clients, and chain links from being +reaped by intermediate NAT or proxy timeouts, and to make an idle pipeline +visibly alive. -### HTTP/HTTPS +> `format = "comment"` (SSE `:` comment framing) is rejected by validation +> despite appearing in older documentation and in a still-present code branch. +> A pipeline configured with it fails to start. -- HTTP/1.1 support -- Persistent connections -- Server-Sent Events (SSE) +## Reconnection -### TCP +Chain sinks reconnect on their own. Both use exponential backoff between +`backoff_min_ms` and `backoff_max_ms` with ±20 % jitter, and both are +interruptible by shutdown. -- Raw TCP sockets -- Newline-delimited protocol +```toml +backoff_min_ms = 500 +backoff_max_ms = 30000 +``` -## Port Configuration +The connection is established lazily, so an edge node starts cleanly even when +its relay is down and connects as soon as the relay appears. Reconnect counts +are reported in the sink's `reconnects` statistic. -### Default Ports +Server-side sinks (`tcp`, `http`) do not reconnect; clients are expected to +retry. Browsers reconnect SSE streams automatically. -| Service | Default Port | Protocol | -|---------|--------------|----------| -| HTTP Sink | 8080 | HTTP | -| TCP Sink | 9090 | TCP | +## Protocol Details -### Port Conflict Prevention +**HTTP sink (SSE)** — HTTP/1.1 in plaintext; HTTP/2 is negotiated via ALPN when +TLS is enabled. Only `GET` is routed to the stream and status paths. Each event +is framed as one `data:` line per newline in the payload, so multi-line entries +stream intact. Response headers set `Cache-Control: no-cache`, +`X-Accel-Buffering: no`, and `Access-Control-Allow-Origin: *`. -LogWisp validates port usage at startup: -- Detects port conflicts across pipelines -- Prevents duplicate bindings +**TCP sink** — raw payload bytes, no framing added by the sink. Whether entries +are newline-delimited depends on the formatter. + +**Chain transports** — see [Chaining](chaining.md) for the hello preamble, +headers, and entry encoding. ## Troubleshooting -### Common Issues +**Connection refused** +- Confirm the pipeline started; a bind failure is logged at ERROR. +- Confirm you are dialing IPv4. `localhost` may resolve to `::1`. +- Check the port is not already bound by another pipeline in the same process. -**Connection Refused** -- Check firewall rules -- Verify service is running -- Confirm correct port/host +**TLS handshake failure** +- `client didn't provide a certificate` — the listener has `client_auth = true` + and the dialer has no `cert_file`/`key_file`. +- `certificate signed by unknown authority` — the dialer's `ca_file` does not + contain the issuer of the server certificate, or the listener's + `client_ca_file` does not contain the issuer of the client certificate. +- `certificate is not valid for any names` / SAN mismatch — the dialed `host` + is not covered by the server certificate's SANs; set `server_name`. +- `protocol version not supported` — one side is pinned to `min_version = "1.3"` + and the other cannot negotiate it. +- Handshake failures appear as WARN with the remote address, and increment + `tls_handshake_errors`. -**TLS Handshake Failure** -- Verify certificate validity -- Check certificate chain -- Confirm TLS versions match +**Entries not arriving over a chain link** +- Check the sink's `connected` statistic and its `reconnects` count. +- Check the source's `parse_errors` — a version skew shows up here. +- On `http_chain`, remember entries wait up to `flush_interval_ms` before a + batch is sent. -**Rate Limit Exceeded** -- Adjust rate limit parameters -- Add IP to whitelist -- Implement client-side throttling +**Clients connect but see nothing** +- The pipeline may be filtering everything out; check `flow.filters` stats. +- The rate limiter may be dropping everything; check `rate_limiter` stats. +- Nothing may be arriving from the sources; check source `total_entries`. -**Connection Timeout** -- Increase timeout values -- Check network latency -- Verify keep-alive settings +**Entries missing under load** +- Compare `dropped_writes` (per-client queue full — raise + `client_buffer_size`) against `total_dropped_by_sink` (sink input queue full + — raise `buffer_size` or reduce sink latency). diff --git a/doc/operations.md b/doc/operations.md index a08841d..0ec8d79 100644 --- a/doc/operations.md +++ b/doc/operations.md @@ -1,327 +1,305 @@ # Operations Guide -Running, monitoring, and maintaining LogWisp in production. +Running, monitoring, and maintaining LogWisp. -*Note: TLS, acccess control under redesign* - -## Starting LogWisp - -### Manual Start +## Starting ```bash -# Foreground with default config +# foreground, explicit config +logwisp -c /etc/logwisp/logwisp.toml + +# no config: built-in demo pipeline (random source -> stdout) logwisp - -# Background mode -logwisp --background - -# With specific configuration -logwisp --config /etc/logwisp/production.toml ``` -### Service Management +There is no built-in daemon mode. Run LogWisp in the foreground under a +supervisor — systemd, rc.d, or a container runtime — which is where restart, +log capture, and resource limits belong. See [Installation](installation.md). + +**systemd** -**Linux (systemd):** ```bash sudo systemctl start logwisp -sudo systemctl stop logwisp -sudo systemctl restart logwisp sudo systemctl status logwisp +sudo journalctl -u logwisp -f ``` -**FreeBSD (rc.d):** +**FreeBSD rc.d** + ```bash sudo service logwisp start -sudo service logwisp stop -sudo service logwisp restart sudo service logwisp status ``` -## Configuration Management +## Configuration Changes -### Hot Reload +### Hot reload -Enable automatic configuration reload: ```toml -config_auto_reload = true +auto_reload = true ``` -Or via command line: -```bash -logwisp --config-auto-reload -``` +or send a signal: -Trigger manual reload: ```bash kill -HUP $(pidof logwisp) -# or -kill -USR1 $(pidof logwisp) ``` -### Configuration Validation +Reload constructs a new service from the new configuration **before** tearing +the old one down, so a broken configuration leaves the running service intact +and logs the failure: + +``` +ERROR msg="Failed to bootstrap new service, keeping old service running" error=... +``` + +What reload does *not* do: + +- Re-apply `logging.*`; application logging is configured once at startup. +- Preserve connections. Listeners close and reopen, and every SSE, TCP, and + chain client is disconnected. Chain sinks reconnect on their own backoff; + browsers reconnect SSE automatically; raw TCP consumers must retry themselves. +- Reload certificates without a reload — certificate files are read at plugin + construction, so rotation requires `SIGHUP`. + +Plan reloads on a busy relay the way you would plan a restart. + +### Checking a configuration + +There is no validate-only mode. To check a file, start it with debug logging and +watch for pipeline startup: -Test configuration without starting: ```bash -logwisp --config test.toml --quiet --status-reporter=false +logwisp -c candidate.toml --logging.level=debug --logging.output=stderr ``` -Check for errors: -- Port conflicts -- Invalid patterns -- Missing required fields -- File permissions +Success looks like `Created source instance`, `Created sink instance`, and +`Starting pipeline` for each pipeline. Failures name the pipeline and the +offending key: + +``` +ERROR msg="Failed to create pipeline" pipeline=app error="failed to create sink out: port: must be 1-65535, got 0" +``` + +Remember that most validation lives in plugin constructors, so a config only +proves itself when the pipeline is actually built. ## Monitoring -### Status Reporter +### Status reporter -Built-in periodic status logging (30-second intervals): +Enabled by default, every 30 seconds. It logs at **DEBUG**, so it produces +nothing unless `logging.level = "debug"` — a common surprise. -``` -[INFO] Status report active_pipelines=2 time=15:04:05 -[INFO] Pipeline status pipeline=app entries_processed=10523 -[INFO] Pipeline status pipeline=system entries_processed=5231 -``` - -Disable if not needed: ```toml -disable_status_reporter = true +status_reporter = true + +[logging] +level = "debug" ``` -### HTTP Status Endpoint +It emits a service summary and then walks each pipeline, flattening scalar +statistics into log fields and recursing into flow, rate limiter, filter, +source, and sink stats. + +Disable with `status_reporter = false`. + +### HTTP status endpoint + +When a pipeline has an `http` sink: -When using HTTP sink: ```bash -curl http://localhost:8080/status | jq . +curl -s http://127.0.0.1:8080/status | jq . ``` -Response structure: ```json { - "uptime": "2h15m30s", - "pipelines": { - "default": { - "sources": 1, - "sinks": 2, - "processed": 15234, - "filtered": 523, - "dropped": 12 - } + "service": "LogWisp", + "version": "v0.16.0", + "instance_id": "sse", + "server": { + "type": "http", + "host": "0.0.0.0", + "port": 8080, + "tls": false, + "active_clients": 3, + "buffer_size": 1000, + "uptime_seconds": 8130 + }, + "endpoints": { "stream": "/stream", "status": "/status" }, + "statistics": { + "total_processed": 15234, + "dropped_writes": 12, + "rejected_clients": 0 } } ``` -### Metrics Collection +This endpoint is scoped to one sink, not to the whole process, and it is +**unauthenticated**. Bind it to a trusted interface. -Track via logs: -- Total entries processed -- Entries filtered -- Entries dropped -- Active connections -- Buffer utilization +### Metrics worth watching + +| Metric | Where | Meaning if rising | +|--------|-------|-------------------| +| `dropped_entries` | source | Downstream cannot keep up with the source | +| `total_dropped` | flow | Rate limit or filters are discarding entries (often intended) | +| `total_dropped_by_sink` | pipeline | A sink's input queue is full | +| `dropped_writes` | tcp/http sink | A specific client is too slow | +| `rejected_conns` / `rejected_clients` | tcp/http sink, tcp_chain source | `max_connections` is being hit | +| `tls_handshake_errors` | tcp sink, tcp_chain source | Certificate or version mismatch, or scanning | +| `parse_errors` | chain source | Protocol or version skew upstream | +| `reconnects` | chain sink | Unstable link or a flapping downstream | +| `dropped_batches` | http_chain sink | Downstream rejecting batches permanently | +| `synthesized` | chain sink | Events reaching the sink without structure | ## Log Management -### LogWisp's Operational Logs - -Configuration for LogWisp's own logs: +LogWisp's own operational log: ```toml [logging] output = "file" -level = "info" +level = "info" [logging.file] -directory = "/var/log/logwisp" -name = "logwisp" -max_size_mb = 100 -retention_hours = 168 +directory = "/var/log/logwisp" +name = "logwisp" +max_size_mb = 100 +max_total_size_mb = 1000 +retention_hours = 168.0 ``` -### Log Rotation +Rotation is automatic on size, with a total-size cap and a retention window. +There is no signal to reopen log files, so do not move files out from under +LogWisp and expect it to reattach — let it rotate, or restart it. -Automatic rotation based on: -- File size threshold -- Total size limit -- Retention period - -Manual rotation: -```bash -# Move current log -mv /var/log/logwisp/logwisp.log /var/log/logwisp/logwisp.log.1 -# Send signal to reopen -kill -USR1 $(pidof logwisp) -``` - -### Log Levels - -Operational log levels: -- **debug**: Detailed debugging information -- **info**: General operational messages -- **warn**: Warning conditions -- **error**: Error conditions - -Production recommendation: `info` or `warn` +Production level: `info`, or `warn` on a busy relay. Avoid `debug` under load: +the filter stage logs several lines per entry evaluated. ## Performance Tuning -### Buffer Sizing +### Buffers -Adjust buffers based on load: +Raise `buffer_size` when `total_dropped_by_sink` is climbing but the sink itself +is healthy — that is a burst-absorption problem. Raise `client_buffer_size` when +`dropped_writes` is climbing for network sinks; that is a slow-consumer problem, +and a bigger buffer only buys time. ```toml -[[pipelines.plugin_sources]] -id = "file_in" -type = "file" -[pipelines.plugin_sources.config] +[pipelines.plugin_sinks.config] +buffer_size = 5000 +client_buffer_size = 1024 ``` -### Rate Limiting - -Protect against overload: +### Rate limiting ```toml [pipelines.flow.rate_limit] -rate = 1000.0 # Entries per second -burst = 2000.0 # Burst capacity -policy = "drop" # Drop excess entries +rate = 1000.0 +burst = 2000.0 +policy = "drop" +max_entry_size_bytes = 65536 ``` +Two behaviours to keep in mind: the limiter does not exist at all when +`rate <= 0`, and `policy = "pass"` short-circuits the size cap as well as the +rate check. Enforcing `max_entry_size_bytes` therefore requires `rate > 0` and +`policy = "drop"`. + +### Formatting + +`raw` is the cheapest and skips sanitization; `json` costs the most. The +formatter serializes on a mutex, so it is the one shared bottleneck in a +pipeline — splitting work across pipelines parallelizes it. + +### Chain batching + +`http_chain` trades latency for efficiency. Lower `flush_interval_ms` for +freshness, raise `max_batch_count` and `max_batch_bytes` for throughput. Use +`tcp_chain` when per-entry latency matters. + ## Troubleshooting -### Common Issues +**Nothing appears at the sink** -**High Memory Usage** -- Check buffer sizes -- Monitor goroutine count -- Review retention settings +Walk the pipeline in order and read the counters: source `total_entries` (is +anything being produced?), flow `total_dropped` (filters or rate limit?), +pipeline `total_dropped_by_sink` (sink backed up?), sink `total_processed`. -**Dropped Entries** -- Increase buffer sizes -- Add rate limiting -- Check sink performance +**File source reads nothing** -**Connection Errors** -- Verify network connectivity -- Check firewall rules -- Review TLS certificates +- The watcher seeks to end-of-file on start; only content appended afterwards is + read. Positions are in memory, so a restart re-seeks to end and anything + written during the downtime is lost. +- `pattern` is a filename glob with `*` and `?` only, and matching is not + recursive. +- `check_interval_ms` governs how quickly a *new file* is noticed; tailing an + open file polls at a fixed 100 ms. -### Debug Mode +**High memory use** -Enable detailed logging: -```bash -logwisp --logging.level=debug --logging.output=stderr -``` +Buffers are bounded, so unbounded growth almost always means many buffers: +count sinks × `buffer_size`, plus clients × `client_buffer_size`. A `tcp_chain` +sink blocked on an unreachable downstream also holds its full input queue. -### Health Checks +**Chain link not delivering** -Implement external monitoring: -```bash -#!/bin/bash -# Health check script -if ! curl -sf http://localhost:8080/status > /dev/null; then - echo "LogWisp health check failed" - exit 1 -fi -``` +Check `connected` and `reconnects` on the sink, `parse_errors` on the source, +and remember `http_chain` waits up to `flush_interval_ms`. For TLS problems see +[Networking](networking.md#troubleshooting). -## Backup and Recovery +**Environment variable override has no effect** -### Configuration Backup - -```bash -# Backup configuration -cp /etc/logwisp/logwisp.toml /backup/logwisp-$(date +%Y%m%d).toml - -# Version control -git add /etc/logwisp/ -git commit -m "LogWisp config update" -``` - -### State Recovery - -LogWisp maintains minimal state: -- File read positions (automatic) -- Connection state (automatic) - -Recovery after crash: -1. Service automatically restarts (systemd/rc.d) -2. File sources resume from last position -3. Network sources accept new connections -4. Clients reconnect automatically +LogWisp currently reads these **without** the `LOGWISP_` prefix — `QUIET`, +`LOGGING_LEVEL`, and so on. Array-indexed paths cannot be set from the +environment or the command line at all. ## Security Operations -### Certificate Management +**Certificate rotation** -Monitor certificate expiration: ```bash -openssl x509 -in /path/to/cert.pem -noout -enddate +openssl x509 -in /etc/logwisp/tls/relay.crt -noout -enddate ``` -Rotate certificates: -1. Generate new certificates -2. Update configuration -3. Reload service (SIGHUP) +Certificates load at plugin construction, so rotation is: write the new files, +then `kill -HUP`. Automate the expiry check; nothing in LogWisp warns you. -### Access Auditing +**Access review** -Monitor access patterns: -- Review connection logs -- Monitor rate limit hits +With mTLS, any certificate signed by the configured `client_ca_file` is +accepted — there is no per-identity allow-list, so "access review" means +reviewing what your CA has issued. Peer Common Names are recorded in session +metadata but are not surfaced in statistics and are not used for authorization. +See [Security](security.md) and the +[mTLS authentication plan](mtls-auth-plan.md). + +**Secret leakage** + +Filters are the only redaction mechanism, and they drop whole entries rather +than masking parts of them. See [Filters](filters.md#common-recipes). ## Maintenance -### Planned Maintenance +**Upgrades** -1. Notify users of maintenance window -2. Stop accepting new connections -3. Drain existing connections -4. Perform maintenance -5. Restart service +1. Read the changelog for configuration-schema changes. +2. Start the new binary against the current configuration in a scratch + environment. +3. Stop the old process, install the new binary, start it. +4. Confirm each pipeline started and that counters are advancing. -### Upgrade Process +**Backup** -1. Download new version -2. Test with current configuration -3. Stop old version -4. Install new version -5. Start service -6. Verify operation +Configuration files and TLS material are the only durable state worth backing +up. LogWisp keeps no persistent runtime state: file read positions live in +memory, connections are re-established on restart, and in-flight entries are +lost. -### Cleanup Tasks +**Redundancy** -Regular maintenance: -- Remove old log files -- Clean temporary files -- Verify disk space -- Update documentation - -## Disaster Recovery - -### Backup Strategy - -- Configuration files: Daily -- TLS certificates: After generation -- Authentication credentials: Secure storage - -### Recovery Procedures - -Service failure: -1. Check service status -2. Review error logs -3. Verify configuration -4. Restart service - -Data loss: -1. Restore configuration from backup -2. Regenerate certificates if needed -3. Recreate authentication credentials -4. Restart service - -### Business Continuity - -- Run multiple instances for redundancy -- Use load balancer for distribution -- Implement monitoring alerts -- Document recovery procedures +Because there is no persistence, availability comes from topology, not from +LogWisp itself. Give each edge two chain sinks pointing at two relays if you +need to survive a relay outage, and accept that this duplicates entries +downstream. diff --git a/doc/security.md b/doc/security.md index 1a5097b..f1e9b1d 100644 --- a/doc/security.md +++ b/doc/security.md @@ -1,4 +1,242 @@ # Security -*Note: Security features like mTLS and IP-Based Access Control are currently under redesign and act as placeholders in the new architecture. Future versions will reintroduce full security capabilities.* +This page covers LogWisp's transport security: what it protects, how to +configure it, and — equally important — what it does not yet do. +## Current State + +| Capability | Status | +|------------|--------| +| TLS 1.2 / 1.3 on all network sources and sinks | Implemented | +| Server certificate verification by dialers | Implemented | +| Mutual TLS (client certificate required and verified) | Implemented at the transport layer | +| Peer identity (certificate CN) recorded per session | Implemented | +| Authorization from peer identity (CN allow-lists, node binding) | **Not implemented** — see [mtls-auth-plan.md](mtls-auth-plan.md) | +| Password, token, or SCRAM authentication | **Removed**; not currently available | +| IP allow/deny lists, per-IP connection or request limits | **Not implemented** | +| Authentication on the `http` sink's stream and status endpoints | **Not implemented** | + +Earlier releases carried basic-auth, bearer-token, and SCRAM authentication. +Those were removed during the move to the plugin/flow architecture and the +switch to standard-library networking. Only certificate-based transport +security survived that transition. + +## The TLS Block + +One option shape serves both roles, so the configuration reads the same +wherever it appears. Which keys matter depends on whether the plugin listens or +dials. + +```toml +[pipelines.plugin_sources.config.tls] # or plugin_sinks.config.tls +enabled = false +cert_file = "" +key_file = "" +client_auth = false +client_ca_file = "" +ca_file = "" +server_name = "" +insecure_skip_verify = false +min_version = "1.3" +``` + +| Option | Role | Default | Description | +|--------|------|---------|-------------| +| `enabled` | both | `false` | Master switch; when false the whole block is ignored | +| `cert_file` | both | — | Local certificate. **Required** for listeners; optional client identity for dialers | +| `key_file` | both | — | Private key for `cert_file`. Must be set together with it | +| `client_auth` | listener | `false` | Require and verify a client certificate (mTLS) | +| `client_ca_file` | listener | — | CA bundle used to verify client certificates. **Required** when `client_auth` is true | +| `ca_file` | dialer | system store | CA bundle used to verify the server certificate | +| `server_name` | dialer | the configured `host` | SNI and certificate name to verify against | +| `insecure_skip_verify` | dialer | `false` | Disable server verification | +| `min_version` | both | `"1.3"` | `"1.2"` or `"1.3"` | + +**Roles by plugin:** + +| Plugin | Role | Keys that apply | +|--------|------|-----------------| +| `tcp` sink, `http` sink | Listener | `cert_file`, `key_file`, `client_auth`, `client_ca_file`, `min_version` | +| `tcp_chain` source, `http_chain` source | Listener | same as above | +| `tcp_chain` sink, `http_chain` sink | Dialer | `ca_file`, `server_name`, `insecure_skip_verify`, `cert_file`, `key_file`, `min_version` | + +> `min_version` takes `"1.2"` or `"1.3"`. The older `"TLS1.2"` spelling from +> pre-restructure releases is rejected. There is no `max_version` and no +> `cipher_suites` option; TLS 1.3 suites are not configurable in Go, and the +> 1.2 defaults are the standard library's. + +### Validation + +Misconfiguration fails at plugin construction, before the pipeline starts: + +- a listener with `enabled = true` and no `cert_file`/`key_file` +- `client_auth = true` with no `client_ca_file` +- a dialer with only one of `cert_file` / `key_file` +- a certificate or key that will not load, or a CA file containing no + certificates +- a `min_version` that is neither `"1.2"` nor `"1.3"` + +## Enabling mTLS + +### 1. Generate a CA and certificates + +LogWisp no longer ships a certificate-generation subcommand; the `logwisp tls` +command was removed with the rest of the CLI restructure. Use `openssl`, +`cfssl`, `step-cli`, or your existing PKI. + +```bash +# CA +openssl req -x509 -newkey rsa:4096 -nodes -days 3650 \ + -keyout ca.key -out ca.crt -subj "/CN=LogWisp CA" + +# Relay (server) certificate — SAN must match how clients address it +openssl req -newkey rsa:2048 -nodes -keyout relay.key -out relay.csr \ + -subj "/CN=relay.internal" +openssl x509 -req -in relay.csr -CA ca.crt -CAkey ca.key -CAcreateserial \ + -out relay.crt -days 825 \ + -extfile <(printf "subjectAltName=DNS:relay.internal\nextendedKeyUsage=serverAuth") + +# Edge (client) certificate — CN identifies the node +openssl req -newkey rsa:2048 -nodes -keyout edge-01.key -out edge-01.csr \ + -subj "/CN=edge-01" +openssl x509 -req -in edge-01.csr -CA ca.crt -CAkey ca.key -CAcreateserial \ + -out edge-01.crt -days 825 \ + -extfile <(printf "extendedKeyUsage=clientAuth") +``` + +The server certificate's SAN must cover the address clients dial. Dialers seed +`ServerName` from the configured `host`, so an IP literal in `host` requires an +IP SAN, and a DNS name requires a DNS SAN. Override with `server_name` when the +dialed address and the certificate name legitimately differ. + +### 2. Configure the listener + +```toml +[pipelines.plugin_sources.config.tls] +enabled = true +cert_file = "/etc/logwisp/tls/relay.crt" +key_file = "/etc/logwisp/tls/relay.key" +client_auth = true +client_ca_file = "/etc/logwisp/tls/ca.crt" +min_version = "1.3" +``` + +### 3. Configure the dialer + +```toml +[pipelines.plugin_sinks.config.tls] +enabled = true +ca_file = "/etc/logwisp/tls/ca.crt" +cert_file = "/etc/logwisp/tls/edge-01.crt" +key_file = "/etc/logwisp/tls/edge-01.key" +min_version = "1.3" +``` + +### 4. Verify + +Startup logs report both flags: + +``` +INFO msg="TCP chain source initialized" ... tls=true mtls=true +INFO msg="TCP chain sink initialized" ... tls=true mtls=true +``` + +A client that presents no certificate is refused during the handshake: + +``` +WARN msg="TLS handshake failed" component=tcp_chain_source + remote_addr=127.0.0.1:53840 error="tls: client didn't provide a certificate" +``` + +Handshake failures are counted in the `tls_handshake_errors` statistic on the +`tcp` sink and the `tcp_chain` source. + +## What mTLS Currently Buys You + +With `client_auth = true`, the transport enforces: + +- the peer holds a certificate chaining to `client_ca_file` +- the certificate is within its validity window and not structurally broken +- the peer holds the matching private key + +That is a real membership check: an attacker without a CA-issued certificate +cannot connect at all. + +## What It Does Not Buy You + +**Any** valid certificate from the configured CA is accepted. LogWisp extracts +the peer's Common Name into session metadata (`tls_peer_cn`) but never consults +it, so within one CA there is no way to express: + +- "only `edge-01` and `edge-02` may connect to this ingest port" +- "the node label `edge-01` may only be claimed by the holder of the `edge-01` + certificate" +- "this certificate may connect but only at this rate" + +Two consequences follow. + +1. **A compromised edge can impersonate any other edge.** With + `trust_node = true` (the default) a peer declares its own node label. Any + certificate holder can claim `edge-99`, or `relay`, and downstream consumers + will attribute its entries accordingly. Setting `trust_node = false` replaces + the label with the remote address, which is coarse but not forgeable at the + application layer. +2. **Revocation is CA-wide.** With no CRL or OCSP checking and no per-identity + allow-list, withdrawing one node's access means re-issuing the CA or rotating + the CA bundle for every peer. + +Closing both gaps is the subject of the +[mTLS authentication plan](mtls-auth-plan.md). + +## Unauthenticated Surfaces + +These endpoints have no access control at all. Bind them to a trusted interface +or front them with an authenticating proxy. + +| Surface | Exposure | +|---------|----------| +| `http` sink `stream_path` | Full log stream, with `Access-Control-Allow-Origin: *`, so any browser origin can read it | +| `http` sink `status_path` | Host, port, TLS flag, uptime, client counts, throughput counters | +| `tcp` sink | Full log stream to any client that connects | + +`max_connections` bounds concurrency on all three but does not distinguish +callers. + +## Operational Guidance + +**Certificates** + +- Use a dedicated CA for LogWisp so its trust decisions stay independent. +- Keep leaf lifetimes short (90–825 days) and automate renewal. +- Key files should be `0600` and owned by the service account. +- Rotation requires a reload (`SIGHUP`), because certificates are loaded once at + plugin construction; there is no on-disk watch for certificate files. +- Check expiry: `openssl x509 -in relay.crt -noout -enddate`. + +**Deployment** + +- Prefer `min_version = "1.3"`. Drop to `"1.2"` only for a peer that genuinely + cannot do 1.3. +- Never enable `insecure_skip_verify` outside a lab; it disables server + verification entirely and makes the connection trivially interceptable. +- Bind listeners to specific interfaces rather than `0.0.0.0` where you can. +- Use `trust_node = false` on any ingest port reachable from a network you do + not fully control. +- Run LogWisp as an unprivileged user with write access only to its own log and + configuration directories. + +**Log content** + +Logs routinely contain secrets that were never meant to leave the host. Filters +are the available tool: + +```toml +[[pipelines.flow.filters]] +type = "exclude" +patterns = ["password", "api[_-]?key", "authorization", "bearer ", "secret"] +``` + +Choose a sanitizer policy that matches the sink — `json` for JSON output, +`txt` for files and consoles — so control characters in log data cannot break +framing or inject terminal escapes downstream. See +[Formatters](formatters.md). diff --git a/doc/sinks.md b/doc/sinks.md index df7e3b8..b0fcab6 100644 --- a/doc/sinks.md +++ b/doc/sinks.md @@ -1,154 +1,335 @@ # Output Sinks -LogWisp sinks deliver processed log entries to various destinations. +Sinks consume `core.TransportEvent` values — a formatted `Payload` plus the +original structured `Entry` — and deliver them somewhere. Each sink is declared +as a `[[pipelines.plugin_sinks]]` entry with an `id`, a `type`, and a +type-specific `config` table. -## Sink Types +Registered types: `console`, `file`, `http`, `tcp`, `null`, `tcp_chain`, +`http_chain`. -### Console Sink +Dispatch into a sink is non-blocking. A sink whose input queue is full drops the +event *for itself only* and the pipeline counts it in `total_dropped_by_sink`; +sibling sinks are unaffected. -Output to stdout/stderr. +--- + +## console + +Writes formatted payloads to stdout or stderr. ```toml [[pipelines.plugin_sinks]] -id = "console_out" +id = "stdout" type = "console" [pipelines.plugin_sinks.config] -target = "stdout" # stdout|stderr|split +target = "stdout" buffer_size = 1000 ``` -**Configuration Options:** - | Option | Type | Default | Description | |--------|------|---------|-------------| -| `target` | string | "stdout" | Output target (stdout/stderr/split) | -| `buffer_size` | int | 1000 | Internal buffer size | +| `target` | string | `stdout` | `stdout` or `stderr` | +| `buffer_size` | int | `1000` | Sink input queue depth | -**Target Modes:** -- **stdout**: All output to standard output -- **stderr**: All output to standard error -- **split**: INFO/DEBUG to stdout, WARN/ERROR to stderr +> `split` is **not** a valid target for this sink and is rejected at startup. +> Level-based splitting exists only for LogWisp's own application log +> (`logging.output = "split"`). -### File Sink +Payloads are written verbatim; the sink adds no framing. Whether entries are +newline-terminated is decided by the formatter. -Write logs to rotating files. +--- + +## file + +Rotating file writer. ```toml [[pipelines.plugin_sinks]] -id = "file_out" +id = "archive" type = "file" [pipelines.plugin_sinks.config] -directory = "./logs" -name = "output" -max_size_mb = 100 +directory = "/var/log/logwisp" +name = "output" +max_size_mb = 100 max_total_size_mb = 1000 -min_disk_free_mb = 500 -retention_hours = 168.0 -buffer_size = 1000 -flush_interval_ms = 1000 +min_disk_free_mb = 0 +retention_hours = 168.0 +buffer_size = 1000 +flush_interval_ms = 100 ``` -**Configuration Options:** - | Option | Type | Default | Description | |--------|------|---------|-------------| -| `directory` | string | Required | Output directory | -| `name` | string | Required | Base filename | -| `max_size_mb` | int | 100 | Rotation threshold | -| `max_total_size_mb` | int | 1000 | Total size limit | -| `min_disk_free_mb` | int | 500 | Minimum free disk space | -| `retention_hours` | float | 168 | Delete files older than | -| `buffer_size` | int | 1000 | Internal buffer size | -| `flush_interval_ms` | int | 1000 | Force flush interval | +| `directory` | string | **required** | Output directory | +| `name` | string | **required** | Base filename | +| `max_size_mb` | int | `100` | Rotate when the active file reaches this size | +| `max_total_size_mb` | int | `1000` | Cap across all rotated files | +| `min_disk_free_mb` | int | `0` | Free-space floor before writing; `0` = no floor | +| `retention_hours` | float | `168.0` | Delete rotated files older than this | +| `buffer_size` | int | `1000` | Sink input queue depth | +| `flush_interval_ms` | int | `100` | Forced flush interval | -**Features:** -- Automatic rotation on size -- Retention management -- Disk space monitoring -- Periodic flushing +> `min_disk_free_mb` has an unusual default. The constructor replaces only +> *negative* values with `100`; leaving the key unset yields `0`, which means no +> free-space floor. Set it explicitly if you want one. -### HTTP Sink +The sink drives an internal writer configured for raw output with timestamps and +levels disabled, so what lands on disk is exactly the formatted payload. -SSE (Server-Sent Events) streaming server. +--- + +## null + +Discards everything, counting entries and bytes. Useful for benchmarking a +source or flow in isolation. ```toml [[pipelines.plugin_sinks]] -id = "http_out" -type = "http" -[pipelines.plugin_sinks.config] -host = "0.0.0.0" -port = 8080 -stream_path = "/stream" -status_path = "/status" -buffer_size = 1000 -client_buffer_size = 256 -write_timeout_ms = 0 -max_connections = 0 -``` - -**Configuration Options:** - -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `host` | string | "0.0.0.0" | Bind address | -| `port` | int | Required | Listen port | -| `stream_path` | string | "/stream" | SSE stream endpoint | -| `status_path` | string | "/status" | Status endpoint | -| `buffer_size` | int | 1000 | Sink input queue size | -| `client_buffer_size` | int | 256 | Per-client send queue size | -| `write_timeout_ms` | int | 0 | Write deadline per event (0 = none) | -| `max_connections` | int | 0 | Concurrent connection cap (0 = unlimited) | - -### TCP Sink - -TCP streaming server for debugging and raw client forwarding. - -```toml -[[pipelines.plugin_sinks]] -id = "tcp_out" -type = "tcp" -[pipelines.plugin_sinks.config] -host = "0.0.0.0" -port = 9090 -buffer_size = 1000 -client_buffer_size = 256 -write_timeout_ms = 5000 -keep_alive = true -keep_alive_period_ms = 30000 -max_connections = 0 -``` - -**Configuration Options:** - -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `host` | string | "0.0.0.0" | Bind address | -| `port` | int | Required | Listen port | -| `buffer_size` | int | 1000 | Sink input queue size | -| `client_buffer_size` | int | 256 | Per-client send queue size | -| `write_timeout_ms` | int | 5000 | Write timeout | -| `keep_alive` | bool | true | Enable TCP keep-alive | -| `keep_alive_period_ms` | int | 30000 | Keep-alive interval | -| `max_connections` | int | 0 | Concurrent connection cap (0 = unlimited) | -``` - -### Null Sink - -```toml -[[pipelines.plugin_sinks]] -id = "null_out" +id = "discard" type = "null" ``` -## Buffer Management +No options. The input queue is fixed at 1000. -- Full input buffer: entry dropped for that sink only (counted per pipeline as `total_dropped_by_sink`)" +--- + +## http + +Server-Sent Events stream plus a JSON status endpoint. + +```toml +[[pipelines.plugin_sinks]] +id = "sse" +type = "http" +[pipelines.plugin_sinks.config] +host = "0.0.0.0" +port = 8080 +stream_path = "/stream" +status_path = "/status" +buffer_size = 1000 +client_buffer_size = 256 +write_timeout_ms = 0 +max_connections = 0 + +[pipelines.plugin_sinks.config.tls] +enabled = true +cert_file = "/etc/logwisp/tls/server.crt" +key_file = "/etc/logwisp/tls/server.key" +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `host` | string | `0.0.0.0` | Bind address; IPv4 only | +| `port` | int | **required** | Listen port | +| `stream_path` | string | `/stream` | SSE endpoint; must start with `/` | +| `status_path` | string | `/status` | Status endpoint; must start with `/` and differ from `stream_path` | +| `buffer_size` | int | `1000` | Sink input queue depth | +| `client_buffer_size` | int | `256` | Per-client send queue depth | +| `write_timeout_ms` | int | `0` | Per-event write deadline; `0` = none | +| `max_connections` | int | `0` | Concurrent stream cap; `0` = unlimited | +| `tls` | table | — | Listener TLS; see [Security](security.md) | + +**Behaviour** + +- Only `GET` is routed to either path; anything else gets `405`. +- On connect the client receives an `event: connected` frame carrying its + client id, session id, sink instance id, endpoint paths, and buffer size. +- Payloads are framed per the SSE spec, one `data:` line per newline in the + payload, so multi-line entries stream correctly. +- The server sets no `WriteTimeout` (that would kill long-lived streams); + per-write deadlines come from `write_timeout_ms` via `http.ResponseController`. +- A client whose send queue is full has that event dropped + (`dropped_writes`); it is not disconnected. +- Clients whose session has been idle-expired by the session manager are + evicted by the broker. +- On shutdown, connected clients receive + `event: disconnect / data: {"reason":"server_shutdown"}`. +- HTTP/2 is negotiated via ALPN when TLS is enabled; plaintext is HTTP/1.1. + +**Status endpoint** returns service and version identity, host, port, TLS flag, +active client count, buffer size, uptime, endpoint paths, and the +`total_processed` / `dropped_writes` / `rejected_clients` counters. + +> Both endpoints are unauthenticated, and the stream response carries +> `Access-Control-Allow-Origin: *`, so any web origin can read it. Bind to a +> trusted interface, or put an authenticating reverse proxy in front. + +--- + +## tcp + +Broadcasts formatted payloads to every connected TCP client. + +```toml +[[pipelines.plugin_sinks]] +id = "tap" +type = "tcp" +[pipelines.plugin_sinks.config] +host = "0.0.0.0" +port = 9090 +buffer_size = 1000 +client_buffer_size = 256 +write_timeout_ms = 5000 +keep_alive = true +keep_alive_period_ms = 30000 +max_connections = 0 + +[pipelines.plugin_sinks.config.tls] +enabled = true +cert_file = "/etc/logwisp/tls/server.crt" +key_file = "/etc/logwisp/tls/server.key" +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `host` | string | `0.0.0.0` | Bind address; IPv4 only | +| `port` | int | **required** | Listen port | +| `buffer_size` | int | `1000` | Sink input queue depth | +| `client_buffer_size` | int | `256` | Per-client send queue depth | +| `write_timeout_ms` | int | `5000` | Per-write deadline | +| `keep_alive` | bool | `true` | Enable TCP keep-alive on accepted connections | +| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period | +| `max_connections` | int | `0` | Concurrent connection cap; `0` = unlimited | +| `tls` | table | — | Listener TLS | + +**Behaviour** + +- The sink is write-only. Each connection also runs a reader that discards + inbound bytes; it exists to detect disconnects and to refresh session + activity when a client sends anything. +- A write that misses its deadline means the kernel buffer stayed full for the + whole timeout, so the client is disconnected immediately rather than retried. +- A client whose send queue is full has that event dropped (`dropped_writes`) + and stays connected. +- With TLS enabled the handshake runs under a 10 s bound *after* the + `max_connections` check, so concurrent handshakes are bounded too. + +--- + +## tcp_chain + +Forwards structured entries to a downstream LogWisp `tcp_chain` source over one +persistent connection. See [Chaining](chaining.md). + +```toml +[[pipelines.plugin_sinks]] +id = "to_relay" +type = "tcp_chain" +[pipelines.plugin_sinks.config] +host = "relay.internal" +port = 15801 +node = "edge-01" +buffer_size = 1000 +dial_timeout_ms = 5000 +write_timeout_ms = 5000 +backoff_min_ms = 500 +backoff_max_ms = 30000 +keep_alive = true +keep_alive_period_ms = 30000 + +[pipelines.plugin_sinks.config.tls] +enabled = true +ca_file = "/etc/logwisp/tls/ca.crt" +cert_file = "/etc/logwisp/tls/client.crt" +key_file = "/etc/logwisp/tls/client.key" +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `host` | string | **required** | Downstream host | +| `port` | int | **required** | Downstream port | +| `node` | string | `os.Hostname()` | Origin label stamped on first-hop entries | +| `buffer_size` | int | `1000` | Sink input queue depth | +| `dial_timeout_ms` | int | `5000` | TCP connect timeout | +| `write_timeout_ms` | int | `5000` | Per-write deadline | +| `backoff_min_ms` | int | `500` | Reconnect backoff floor | +| `backoff_max_ms` | int | `30000` | Reconnect backoff ceiling | +| `keep_alive` | bool | `true` | Enable TCP keep-alive | +| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period | +| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity | + +**Behaviour** + +- The connection is established lazily, so pipeline start does not depend on the + downstream being up. +- Each entry is serialized as one canonical JSON line. Delivery holds the line + across reconnects until it is written or the process shuts down, with + exponential backoff plus ±20 % jitter between attempts. +- Because delivery blocks the sink's run loop during an outage, back-pressure + surfaces as a full input queue and is counted by the pipeline as + `total_dropped_by_sink`. +- With TLS, dial and handshake are bounded together by + `dial_timeout_ms` + 10 s. +- Events arriving without a structured entry are wrapped from the formatted + payload and counted in `synthesized`. + +**Statistics**: `target`, `node`, `tls`, `connected`, `reconnects`, +`write_errors`, `synthesized`. + +--- + +## http_chain + +Batches structured entries as NDJSON and POSTs them to a downstream LogWisp +`http_chain` source. + +```toml +[[pipelines.plugin_sinks]] +id = "to_collector" +type = "http_chain" +[pipelines.plugin_sinks.config] +host = "collector.internal" +port = 15802 +ingest_path = "/ingest" +node = "edge-01" +buffer_size = 1000 +max_batch_count = 100 +max_batch_bytes = 1048576 +flush_interval_ms = 1000 +request_timeout_ms = 10000 +backoff_min_ms = 500 +backoff_max_ms = 30000 + +[pipelines.plugin_sinks.config.tls] +enabled = true +ca_file = "/etc/logwisp/tls/ca.crt" +cert_file = "/etc/logwisp/tls/client.crt" +key_file = "/etc/logwisp/tls/client.key" +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `host` | string | **required** | Downstream host | +| `port` | int | **required** | Downstream port | +| `ingest_path` | string | `/ingest` | Endpoint path; must start with `/` | +| `node` | string | `os.Hostname()` | Origin label stamped on first-hop entries | +| `buffer_size` | int | `1000` | Sink input queue depth | +| `max_batch_count` | int | `100` | Flush after this many entries | +| `max_batch_bytes` | int | `1048576` | Flush after this many bytes (1 MiB) | +| `flush_interval_ms` | int | `1000` | Flush after this long | +| `request_timeout_ms` | int | `10000` | Covers dial, write, and response | +| `backoff_min_ms` | int | `500` | Retry backoff floor | +| `backoff_max_ms` | int | `30000` | Retry backoff ceiling | +| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity | + +**Behaviour** + +- Delivery is at-least-once per batch: a retried batch can be delivered twice if + the first attempt succeeded but the response was lost. +- Retries apply to transport errors, `408`, `429`, and `5xx`. Any other + non-2xx response is treated as permanent, and the batch is dropped and counted + in `dropped_batches`. +- HTTP/2 is off by design; batched NDJSON POSTs gain nothing from it. +- On shutdown a single best-effort flush of the pending batch is attempted. + +**Statistics**: `target`, `node`, `tls`, `batches_sent`, `request_errors`, +`dropped_batches`, `synthesized`. + +--- ## Sink Statistics -All sinks track: -- Total entries processed -- Active connections -- Failed sends -- Retry attempts -- Last processed timestamp +Every sink reports: `id`, `type`, `total_processed`, `active_connections`, +`start_time`, `last_processed`, and a type-specific `details` map. diff --git a/doc/sources.md b/doc/sources.md index 422e177..4a4256d 100644 --- a/doc/sources.md +++ b/doc/sources.md @@ -1,107 +1,249 @@ # Input Sources -LogWisp sources monitor various inputs and generate log entries for pipeline processing. - -## Source Types - -### Directory Source - -Monitors a directory for log files matching a pattern. (type: `file`) +Sources produce `core.LogEntry` values for a pipeline. Every source is declared +as a `[[pipelines.plugin_sources]]` entry with an `id`, a `type`, and a +type-specific `config` table. ```toml [[pipelines.plugin_sources]] -id = "file_in" +id = "app_logs" type = "file" [pipelines.plugin_sources.config] directory = "/var/log/myapp" -pattern = "*.log" # Glob pattern -check_interval_ms = 100 # Poll interval ``` -**Configuration Options:** +Registered types: `file`, `console`, `random`, `null`, `tcp_chain`, +`http_chain`. -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `directory` | string | Required | Directory to monitor | -| `pattern` | string | "*" | File pattern (glob) | -| `check_interval_ms` | int | 100 | File check interval in milliseconds | +Publication from any source is non-blocking. When a subscriber channel is full +the entry is dropped and counted in `dropped_entries`. -**Features:** -- Automatic rotation detection (inode + size tracking) -- In-memory position tracking; on restart, monitoring resumes from the current end of each file (offsets are not persisted) -- Concurrent file monitoring -- Pattern-based file selection +--- -### Stdin Source +## file -Reads log entries from standard input. +Tails every file in a directory whose name matches a glob. ```toml [[pipelines.plugin_sources]] -id = "console_in" +id = "app_logs" +type = "file" +[pipelines.plugin_sources.config] +directory = "/var/log/myapp" +pattern = "*.log" +check_interval_ms = 100 +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `directory` | string | **required** | Directory to scan; not recursive | +| `pattern` | string | `*` | Glob over filenames; `*` and `?` only | +| `check_interval_ms` | int | `100` | Directory rescan interval; minimum `10` | + +**Behaviour** + +- `check_interval_ms` governs how often the directory is rescanned for new or + removed files. Tailing an already-open file polls on a **fixed 100 ms** + interval that this option does not change. +- Each matched file gets its own watcher. Watchers for files that disappear are + stopped and removed on the next scan. +- A new watcher seeks to end-of-file. Positions live in memory only, so a + restart resumes from the current end of each file and content written while + LogWisp was down is not read. +- Rotation is detected from size decrease, modification-time reset, a position + beyond end-of-file, or an inode change. An inode change where the new file is + already larger than the recorded position is treated as an atomic save, not a + rotation, and the position is preserved. +- Lines are parsed as JSON when they contain `time`, `level`, `msg`, and + `fields` keys; `time` is read as RFC3339Nano. Anything else is kept as plain + text with the level inferred from common markers (`[ERROR]`, `WARN:`, and so + on). +- `Source` is set to the file's base name. + +**Statistics**: per-watcher size, position, entries read, rotation count, and +last read time, plus `active_watchers`. + +--- + +## console + +Reads newline-delimited entries from standard input. + +```toml +[[pipelines.plugin_sources]] +id = "stdin" type = "console" [pipelines.plugin_sources.config] buffer_size = 1000 ``` -**Configuration Options:** - | Option | Type | Default | Description | |--------|------|---------|-------------| -| `buffer_size` | int | 1000 | Internal buffer size | +| `buffer_size` | int | `1000` | Subscriber channel depth | -**Features:** -- Line-based processing -- Automatic level detection -- Non-blocking reads +At most **one** instance per pipeline: the type is registered with +`MaxInstances: 1`, and a second instance is rejected at pipeline construction. +The level is inferred from the line text, and `Source` is set to `console`. -### Random Source +--- + +## random + +Synthetic entry generator for development, smoke tests, and sanitizer testing. ```toml [[pipelines.plugin_sources]] -id = "random_in" +id = "generator" type = "random" [pipelines.plugin_sources.config] interval_ms = 500 -jitter_ms = 0 -format = "txt" -length = 20 -special = false +jitter_ms = 0 +format = "txt" +length = 20 +special = false ``` - **Configuration Options:** - | Option | Type | Default | Description | |--------|------|---------|-------------| -| `interval_ms` | int | 500 | Generation interval | -| `jitter_ms` | int | 0 | Random jitter interval | -| `format` | string | "txt" | "txt", "json", "raw" | -| `length` | int | 20 | Log length | -| `special` | bool | false | Include special characters | - -## Source Statistics +| `interval_ms` | int | `500` | Emission period | +| `jitter_ms` | int | `0` | Symmetric jitter; clamped to `interval_ms`, must be non-negative | +| `format` | string | `txt` | `raw` (message only), `txt` (bracketed line), `json` (JSON object as the message) | +| `length` | int | `20` | Message length in characters | +| `special` | bool | `false` | Inject control and non-ASCII characters | -All sources track: -- Total entries received -- Dropped entries (buffer full) -- Invalid entries -- Last entry timestamp -- Active connections (network sources) -- Source-specific metrics +`special = true` is the intended way to exercise sanitizer policies: it inserts +control bytes and multi-byte Unicode into otherwise ordinary messages. Levels +are chosen at random from DEBUG, INFO, WARN, ERROR. -### Null Source +--- + +## null + +Produces nothing. Useful as a placeholder so a sink-only pipeline satisfies the +"at least one source" requirement. ```toml [[pipelines.plugin_sources]] -id = "null_in" +id = "void" type = "null" -[pipelines.plugin_sources.config] ``` -## Buffer Management +No options. -Each source maintains internal buffers: -- Default size: 1000 entries -- Drop policy when full -- Configurable per source -- Non-blocking writes +--- + +## tcp_chain + +Listens for persistent NDJSON streams from upstream LogWisp `tcp_chain` sinks. +See [Chaining](chaining.md) for the protocol. + +```toml +[[pipelines.plugin_sources]] +id = "ingest_tcp" +type = "tcp_chain" +[pipelines.plugin_sources.config] +host = "0.0.0.0" +port = 15801 +buffer_size = 1000 +max_connections = 0 +read_timeout_ms = 0 +hello_timeout_ms = 10000 +trust_node = true + +[pipelines.plugin_sources.config.tls] +enabled = true +cert_file = "/etc/logwisp/tls/server.crt" +key_file = "/etc/logwisp/tls/server.key" +client_auth = true +client_ca_file = "/etc/logwisp/tls/client-ca.crt" +min_version = "1.3" +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `host` | string | `0.0.0.0` | Bind address; IPv4 only | +| `port` | int | **required** | Listen port, 1–65535 | +| `buffer_size` | int | `1000` | Subscriber channel depth | +| `max_connections` | int | `0` | Concurrent connection cap; `0` = unlimited | +| `read_timeout_ms` | int | `0` | Per-connection idle read deadline; `0` = none | +| `hello_timeout_ms` | int | `10000` | Deadline for the hello preamble | +| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address | +| `tls` | table | — | Listener TLS; see [Security](security.md) | + +**Behaviour** + +- TLS handshakes run explicitly with a 10 s bound before the preamble is read, + after the `max_connections` admission check. +- A connection is rejected if the first line is not a valid hello with a + matching protocol version. +- Each accepted connection gets a session recording the remote address, node + label, and — under TLS — `tls` and `tls_peer_cn`. +- A malformed entry line increments `parse_errors` and is skipped; the + connection survives. A line over 1 MiB is a protocol violation and terminates + the connection. + +**Statistics**: `active_connections`, `rejected_conns`, `parse_errors`, +`tls_handshake_errors`, `trust_node`. + +--- + +## http_chain + +Accepts NDJSON batches POSTed by upstream LogWisp `http_chain` sinks. + +```toml +[[pipelines.plugin_sources]] +id = "ingest_http" +type = "http_chain" +[pipelines.plugin_sources.config] +host = "0.0.0.0" +port = 15802 +ingest_path = "/ingest" +buffer_size = 1000 +max_body_bytes = 8388608 +read_timeout_ms = 30000 +trust_node = true + +[pipelines.plugin_sources.config.tls] +enabled = true +cert_file = "/etc/logwisp/tls/server.crt" +key_file = "/etc/logwisp/tls/server.key" +client_auth = true +client_ca_file = "/etc/logwisp/tls/client-ca.crt" +``` + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `host` | string | `0.0.0.0` | Bind address; IPv4 only | +| `port` | int | **required** | Listen port | +| `ingest_path` | string | `/ingest` | Endpoint path; must start with `/` | +| `buffer_size` | int | `1000` | Subscriber channel depth | +| `max_body_bytes` | int | `8388608` | Per-request body cap (8 MiB) | +| `read_timeout_ms` | int | `30000` | Full request read deadline | +| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address | +| `tls` | table | — | Listener TLS | + +**Behaviour** + +- Only `POST` to `ingest_path` is routed; other methods get `405` with an + `Allow` header, and other paths get `404`. +- A missing or mismatched `X-Logwisp-Protocol` header is rejected with `400`. +- Batch acceptance is atomic: entries are published only after the body reads + cleanly end to end. A transfer error rejects the whole batch (`400`, or `413` + when the body cap is hit) so the sender retries it. A malformed *line* inside + an otherwise clean transfer is skipped and counted in `parse_errors`. +- Success is `204 No Content` with `X-Logwisp-Accepted` set to the number of + entries ingested. +- Sessions are cached per remote host + declared node and recreated after idle + expiry. + +**Statistics**: `total_requests`, `rejected_requests`, `parse_errors`, +`cached_sessions`, `trust_node`. + +--- + +## Source Statistics + +Every source reports: `id`, `type`, `total_entries`, `dropped_entries`, +`start_time`, `last_entry_time`, and a type-specific `details` map. These appear +in the status reporter output and in the `http` sink's status endpoint. diff --git a/go.mod b/go.mod index 456e0c4..22d66d1 100644 --- a/go.mod +++ b/go.mod @@ -1,10 +1,10 @@ module logwisp -go 1.26.0 +go 1.26.5 require ( github.com/lixenwraith/config v0.1.1-0.20260712172228-ccd280ba6a98 - github.com/lixenwraith/log v0.1.1-0.20260717175128-82eea9846ccd + github.com/lixenwraith/log v0.1.1-0.20260801090951-2c40643523b3 ) require ( diff --git a/go.sum b/go.sum index 203dd83..d286597 100644 --- a/go.sum +++ b/go.sum @@ -6,12 +6,14 @@ github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPE github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= github.com/lixenwraith/config v0.1.1-0.20260712172228-ccd280ba6a98 h1:VEFo2WsYgM5YYfzAfvXRKigCWr3zEz6h4M59N5TXMpk= github.com/lixenwraith/config v0.1.1-0.20260712172228-ccd280ba6a98/go.mod h1:J9ydxY7he4Dz+S59xKs/kDQ2YAv+zOLF05b3Rr2/ogE= -github.com/lixenwraith/log v0.1.1-0.20260717175128-82eea9846ccd h1:06Rk4DLvJW1kdeclH5HwywizgtMI64PHNE/2sBeuiwA= -github.com/lixenwraith/log v0.1.1-0.20260717175128-82eea9846ccd/go.mod h1:2+qURSVdWcX7REFH+3jUtIbtc+NtsAPecSvXLeGyK6U= +github.com/lixenwraith/log v0.1.1-0.20260724174821-e688c5a07207 h1:EnDMcnFkgTwTGo1ojoJ85/b6aH3yxlBUAg6AefaBcrI= +github.com/lixenwraith/log v0.1.1-0.20260724174821-e688c5a07207/go.mod h1:fvGT3IxlJQLJ1is4OP8Iet8gVzTJTGcuT90FFB2WBtc= +github.com/lixenwraith/log v0.1.1-0.20260801090951-2c40643523b3 h1:5ojkxyuOKiUBRYhqPqKJS1lrqkjckYqYMNGBB49kdY0= +github.com/lixenwraith/log v0.1.1-0.20260801090951-2c40643523b3/go.mod h1:fvGT3IxlJQLJ1is4OP8Iet8gVzTJTGcuT90FFB2WBtc= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= -github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=