v0.16.1 doc update
This commit is contained in:
@@ -9,6 +9,7 @@ script/
|
||||
build/
|
||||
*.log
|
||||
*.toml
|
||||
!config/*.toml
|
||||
build.sh
|
||||
catalog.txt
|
||||
combined.txt
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<td>
|
||||
<h1>LogWisp</h1>
|
||||
<p>
|
||||
<a href="https://golang.org"><img src="https://img.shields.io/badge/Go-1.25-00ADD8?style=flat&logo=go" alt="Go"></a>
|
||||
<a href="https://golang.org"><img src="https://img.shields.io/badge/Go-1.26-00ADD8?style=flat&logo=go" alt="Go"></a>
|
||||
<a href="https://opensource.org/licenses/BSD-3-Clause"><img src="https://img.shields.io/badge/License-BSD_3--Clause-blue.svg" alt="License"></a>
|
||||
<a href="doc/"><img src="https://img.shields.io/badge/Docs-Available-green.svg" alt="Documentation"></a>
|
||||
</p>
|
||||
@@ -16,78 +16,127 @@
|
||||
|
||||
# LogWisp
|
||||
|
||||
A high-performance, pipeline-based log transport and processing system built in Go. LogWisp provides flexible log collection, filtering, formatting, and distribution with enterprise-grade security and reliability features.
|
||||
A pipeline-based log transport and processing system written in Go. LogWisp
|
||||
collects log entries from files, stdin, or other LogWisp nodes; rate-limits,
|
||||
filters, and formats them; and distributes them to files, consoles, live network
|
||||
streams, or downstream LogWisp nodes.
|
||||
|
||||
## Features
|
||||
|
||||
### Core Capabilities
|
||||
- **Pipeline Architecture**: Independent processing pipelines with source(s) → filter → format → sink(s) flow
|
||||
- **Multiple Input Sources**: Directory monitoring, stdin, HTTP, TCP
|
||||
- **Flexible Output Sinks**: Console, file, HTTP SSE, TCP streaming, HTTP/TCP forwarding
|
||||
- **Real-time Processing**: Sub-millisecond latency with configurable buffering
|
||||
- **Hot Configuration Reload**: Update pipelines without service restart
|
||||
### Pipeline
|
||||
|
||||
### Data Processing
|
||||
- **Pattern-based Filtering**: Chainable include/exclude filters with regex support
|
||||
- **Multiple Formatters**: Raw, JSON, and template-based text formatting
|
||||
- **Rate Limiting**: Pipeline rate control
|
||||
- **Independent pipelines**, each `sources → flow → sinks`, running concurrently
|
||||
in one process
|
||||
- **Fan-in and fan-out**: many sources and many sinks per pipeline
|
||||
- **Never blocks**: a stalled sink drops its own events and is counted, rather
|
||||
than stalling the pipeline or its sibling sinks
|
||||
- **Hot reload** via `SIGHUP`/`SIGUSR1` or a config file watch, with the new
|
||||
configuration validated before the old service is torn down
|
||||
|
||||
### Security & Reliability
|
||||
- **Authentication**: mTLS support for HTTPS
|
||||
- **TLS Encryption**: TLS 1.2/1.3 support for HTTP connections
|
||||
- **Access Control**: IP whitelisting/blacklisting, connection limits
|
||||
- **Automatic Reconnection**: Resilient client connections with exponential backoff
|
||||
- **File Rotation**: Size-based rotation with retention policies
|
||||
### Inputs
|
||||
|
||||
### Operational Features
|
||||
- **Status Monitoring**: Real-time statistics and health endpoints
|
||||
- **Signal Handling**: Graceful shutdown and configuration reload via signals
|
||||
- **Background Mode**: Daemon operation with proper signal handling
|
||||
- **Quiet Mode**: Silent operation for automated deployments
|
||||
`file` (directory tail with rotation detection and JSON line parsing),
|
||||
`console` (stdin), `random` (synthetic generator), `null`, and the chain ingest
|
||||
listeners `tcp_chain` and `http_chain`.
|
||||
|
||||
### Outputs
|
||||
|
||||
`console`, `file` (rotating with retention), `http` (Server-Sent Events plus a
|
||||
JSON status endpoint), `tcp` (broadcast server), `null`, and the chain
|
||||
forwarders `tcp_chain` and `http_chain`.
|
||||
|
||||
### Processing
|
||||
|
||||
- **Filters**: chainable include/exclude RE2 patterns with `or`/`and` logic
|
||||
- **Formatters**: `raw`, `txt`, and `json` with selectable sanitizer policies
|
||||
- **Rate limiting**: token bucket with an optional per-entry size cap
|
||||
- **Heartbeats**: flow-level keep-alive entries that reach every sink
|
||||
|
||||
### Chaining
|
||||
|
||||
Multi-node topologies over a versioned protocol. Chain links carry the
|
||||
**structured entry**, not the formatted text, so a relay can filter and reformat
|
||||
as if the entries were local. Entries keep a `node` label identifying their
|
||||
origin across any number of hops. Chain sinks reconnect automatically with
|
||||
exponential backoff and jitter.
|
||||
|
||||
### Transport security
|
||||
|
||||
- TLS 1.2/1.3 on every network source and sink, listener and dialer alike
|
||||
- Mutual TLS: listeners can require and verify client certificates; dialers can
|
||||
present a client identity
|
||||
|
||||
mTLS is currently a CA-wide membership check — any certificate the configured CA
|
||||
issued is accepted, and the peer's Common Name is recorded but not used for
|
||||
authorization. See [Security](doc/security.md) for the exact boundary and
|
||||
[the mTLS authentication plan](doc/mtls-auth-plan.md) for the proposed work.
|
||||
Password, token, and SCRAM authentication were removed during the restructure
|
||||
and are not currently available.
|
||||
|
||||
## Documentation
|
||||
|
||||
Available in `doc/` directory.
|
||||
| Document | Contents |
|
||||
|----------|----------|
|
||||
| [Installation](doc/installation.md) | Building, installing, running as a service |
|
||||
| [Architecture](doc/architecture.md) | Component model, data flow, concurrency, back-pressure |
|
||||
| [Configuration](doc/configuration.md) | TOML structure, precedence, environment and CLI overrides |
|
||||
| [Sources](doc/sources.md) | Every input plugin and its options |
|
||||
| [Sinks](doc/sinks.md) | Every output plugin and its options |
|
||||
| [Filters](doc/filters.md) | Pattern-based inclusion and exclusion |
|
||||
| [Formatters](doc/formatters.md) | Output shaping and sanitization |
|
||||
| [Chaining](doc/chaining.md) | Multi-node topologies and the chain wire protocol |
|
||||
| [Networking](doc/networking.md) | Listeners, dialers, timeouts, connection limits |
|
||||
| [Security](doc/security.md) | TLS and mTLS configuration, threat model, current limits |
|
||||
| [mTLS Authentication Plan](doc/mtls-auth-plan.md) | Design for certificate-based authorization |
|
||||
| [CLI](doc/cli.md) | Flags, signals, exit codes |
|
||||
| [Operations](doc/operations.md) | Running, monitoring, tuning, troubleshooting |
|
||||
|
||||
- [Installation Guide](doc/installation.md) - Platform setup and service configuration
|
||||
- [Architecture Overview](doc/architecture.md) - System design and component interaction
|
||||
- [Configuration Reference](doc/configuration.md) - TOML structure and configuration methods
|
||||
- [Input Sources](doc/sources.md) - Available source types and configurations
|
||||
- [Output Sinks](doc/sinks.md) - Sink types and output options
|
||||
- [Filters](doc/filters.md) - Pattern-based log filtering
|
||||
- [Formatters](doc/formatters.md) - Log formatting and transformation
|
||||
- [Security](doc/security.md) - mTLS configurations and access control
|
||||
- [Networking](doc/networking.md) - TLS, rate limiting, and network features
|
||||
- [Command Line Interface](doc/cli.md) - CLI flags and subcommands
|
||||
- [Operations Guide](doc/operations.md) - Running and maintaining LogWisp
|
||||
A fully annotated configuration covering every option ships as
|
||||
[`config/logwisp.toml`](config/logwisp.toml).
|
||||
|
||||
## Quick Start
|
||||
|
||||
Install LogWisp and create a basic configuration:
|
||||
```bash
|
||||
make
|
||||
```
|
||||
|
||||
```toml
|
||||
# logwisp.toml
|
||||
[[pipelines]]
|
||||
name = "default"
|
||||
|
||||
[[pipelines.sources]]
|
||||
type = "directory"
|
||||
[pipelines.sources.directory]
|
||||
path = "./"
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
sanitizer_policy = "json"
|
||||
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "app_logs"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "/var/log/myapp"
|
||||
pattern = "*.log"
|
||||
|
||||
[[pipelines.sinks]]
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "stdout"
|
||||
type = "console"
|
||||
[pipelines.sinks.console]
|
||||
[pipelines.plugin_sinks.config]
|
||||
target = "stdout"
|
||||
```
|
||||
|
||||
Run with: `logwisp -c config.toml`
|
||||
```bash
|
||||
logwisp -c logwisp.toml
|
||||
```
|
||||
|
||||
Running with no configuration file starts a self-demonstrating pipeline: a
|
||||
synthetic generator writing JSON to stdout.
|
||||
|
||||
## System Requirements
|
||||
|
||||
- **Operating Systems**: Linux (kernel 6.10+), FreeBSD (14.0+)
|
||||
- **Operating systems**: Linux (kernel 6.10+), FreeBSD (14.0+)
|
||||
- **Architecture**: amd64
|
||||
- **Go Version**: 1.25+ (for building from source)
|
||||
- **Go**: 1.26+ to build from source
|
||||
|
||||
Network sources and sinks bind and dial over IPv4 only.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
###############################################################################
|
||||
### LogWisp Configuration
|
||||
### Default location: ~/.config/logwisp/logwisp.toml
|
||||
### Precedence: CLI flags > Environment > File > Defaults
|
||||
###
|
||||
### Commented values are the built-in defaults unless marked "example".
|
||||
### Uncommenting a default is a no-op.
|
||||
###
|
||||
### NOTE: authentication (password/token/SCRAM), network access control (ACL),
|
||||
### http/tcp ingest sources, and http_client/tcp_client sinks were removed in
|
||||
### the restructure and are not available in this version. TLS and mTLS ARE
|
||||
### available on every network source and sink; see the [...tls] blocks below.
|
||||
###
|
||||
### Environment overrides are currently read WITHOUT the LOGWISP_ prefix
|
||||
### (QUIET, LOGGING_LEVEL, ...). LOGWISP_CONFIG_FILE and LOGWISP_CONFIG_DIR
|
||||
### are the exceptions and do carry it. Array-indexed paths such as
|
||||
### pipelines.0.name cannot be set from the CLI or environment at all.
|
||||
###############################################################################
|
||||
|
||||
###############################################################################
|
||||
### Global Settings
|
||||
###############################################################################
|
||||
|
||||
quiet = false # Suppress console output
|
||||
status_reporter = true # Periodic status logging (30s, DEBUG level)
|
||||
auto_reload = false # Config auto-reload on file change
|
||||
|
||||
###############################################################################
|
||||
### Logging (LogWisp's internal operational logging)
|
||||
###############################################################################
|
||||
|
||||
[logging]
|
||||
output = "stdout" # file|stdout|stderr|split|all|none
|
||||
level = "info" # debug|info|warn|error
|
||||
# format = "txt" # raw|txt|json
|
||||
# sanitization = "" # raw|json|txt|shell (empty = logger default)
|
||||
|
||||
# [logging.file] # Used when output is "file" or "all"
|
||||
# directory = "./log"
|
||||
# name = "logwisp"
|
||||
# max_size_mb = 100
|
||||
# max_total_size_mb = 1000
|
||||
# retention_hours = 168.0 # 7 days
|
||||
|
||||
## Validated but NOT applied: the console destination comes from `output` above.
|
||||
# [logging.console]
|
||||
# target = "stdout" # stdout|stderr|split
|
||||
|
||||
###############################################################################
|
||||
### Pipelines
|
||||
### Each pipeline: plugin_sources -> flow (rate_limit|filters|format) -> plugin_sinks
|
||||
### Names must be unique. 1+ source and 1+ sink required.
|
||||
###############################################################################
|
||||
|
||||
[[pipelines]]
|
||||
name = "default"
|
||||
|
||||
###============================================================================
|
||||
### Flow (processing between sources and sinks)
|
||||
###============================================================================
|
||||
|
||||
## policy="pass" short-circuits the size check too: enforcing a size cap needs
|
||||
## rate > 0 AND policy = "drop".
|
||||
# [pipelines.flow.rate_limit]
|
||||
# rate = 0.0 # Entries/second (0 = limiter disabled)
|
||||
# burst = 0.0 # Burst capacity (defaults to rate)
|
||||
# policy = "pass" # pass|drop
|
||||
# max_entry_size_bytes = 0 # 0 = unlimited
|
||||
|
||||
## Filters: sequential include/exclude chain, matched against "<source> <level> <message>"
|
||||
# [[pipelines.flow.filters]]
|
||||
# type = "include" # include|exclude
|
||||
# logic = "or" # or|and
|
||||
# patterns = [".*ERROR.*", ".*WARN.*"] # example; RE2 syntax
|
||||
|
||||
# [pipelines.flow.format]
|
||||
# type = "raw" # raw|json|txt
|
||||
# flags = 0 # Formatter flags (0 = defaults per type)
|
||||
# timestamp_format = "" # Go time layout (formatter default if empty)
|
||||
# sanitizer_policy = "" # raw|json|txt|shell (defaults per type)
|
||||
|
||||
## Flow-level heartbeat (fan-out to all sinks, traverses chain links)
|
||||
# [pipelines.flow.heartbeat]
|
||||
# enabled = false
|
||||
# interval_ms = 1000 # Minimum 100
|
||||
# include_timestamp = false
|
||||
# include_stats = false
|
||||
# format = "txt" # txt|json|raw ("comment" is rejected)
|
||||
|
||||
###============================================================================
|
||||
### TLS (shared shape; applies to every network source and sink)
|
||||
###
|
||||
### Listeners (tcp/http sinks, tcp_chain/http_chain sources):
|
||||
### cert_file + key_file are REQUIRED; client_auth + client_ca_file enable mTLS.
|
||||
### Dialers (tcp_chain/http_chain sinks):
|
||||
### ca_file + server_name verify the server; cert_file + key_file present a
|
||||
### client identity for mTLS.
|
||||
###
|
||||
### enabled = false Master switch
|
||||
### cert_file = "" Local certificate
|
||||
### key_file = "" Private key for cert_file (set together)
|
||||
### client_auth = false Listener: require and verify a client certificate
|
||||
### client_ca_file = "" Listener: CA bundle verifying client certs
|
||||
### ca_file = "" Dialer: CA bundle verifying the server (empty = system)
|
||||
### server_name = "" Dialer: SNI / name to verify (empty = configured host)
|
||||
### insecure_skip_verify = false Dialer: disable verification (never in prod)
|
||||
### min_version = "1.3" "1.2" or "1.3". No max_version, no cipher_suites.
|
||||
###
|
||||
### Any certificate signed by client_ca_file is accepted; the peer CN is
|
||||
### recorded but not used for authorization. See doc/security.md.
|
||||
###============================================================================
|
||||
|
||||
###============================================================================
|
||||
### Sources (1+ required)
|
||||
###============================================================================
|
||||
|
||||
## Null source (testing)
|
||||
# [[pipelines.plugin_sources]]
|
||||
# id = "null_in"
|
||||
# type = "null"
|
||||
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "default_source"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "./" # Directory to monitor (required, not recursive)
|
||||
pattern = "*.log" # Glob pattern (* and ? only)
|
||||
## Tailing an already-open file polls at a fixed 100ms, regardless of this value.
|
||||
check_interval_ms = 100 # Directory rescan interval (min 10)
|
||||
|
||||
## Console source (stdin, single instance per pipeline)
|
||||
# [[pipelines.plugin_sources]]
|
||||
# id = "console_in"
|
||||
# type = "console"
|
||||
# [pipelines.plugin_sources.config]
|
||||
# buffer_size = 1000
|
||||
|
||||
## Random source (testing; special=true exercises sanitizer policies)
|
||||
# [[pipelines.plugin_sources]]
|
||||
# id = "random_in"
|
||||
# type = "random"
|
||||
# [pipelines.plugin_sources.config]
|
||||
# interval_ms = 500
|
||||
# jitter_ms = 0 # Clamped to interval_ms
|
||||
# format = "txt" # raw|txt|json
|
||||
# length = 20
|
||||
# special = false
|
||||
|
||||
## TCP chain source (stdlib listener; receives NDJSON from upstream tcp_chain sinks)
|
||||
## Topology: jail [file source -> tcp_chain sink] -> host [tcp_chain source -> aggregate sink]
|
||||
# [[pipelines.plugin_sources]]
|
||||
# id = "chain_in"
|
||||
# type = "tcp_chain"
|
||||
# [pipelines.plugin_sources.config]
|
||||
# host = "0.0.0.0" # IPv4 only
|
||||
# port = 9440 # Required
|
||||
# buffer_size = 1000
|
||||
# max_connections = 0 # 0 = unlimited
|
||||
# read_timeout_ms = 0 # idle deadline, 0 = none
|
||||
# hello_timeout_ms = 10000 # Protocol preamble deadline
|
||||
# trust_node = true # false: label entries by remote address
|
||||
# [pipelines.plugin_sources.config.tls]
|
||||
# enabled = true # example: mTLS listener
|
||||
# cert_file = "/etc/logwisp/tls/server.crt"
|
||||
# key_file = "/etc/logwisp/tls/server.key"
|
||||
# client_auth = true
|
||||
# client_ca_file = "/etc/logwisp/tls/client-ca.crt"
|
||||
# min_version = "1.3"
|
||||
|
||||
## HTTP chain source (stdlib listener; receives NDJSON batches from upstream http_chain sinks)
|
||||
# [[pipelines.plugin_sources]]
|
||||
# id = "hchain_in"
|
||||
# type = "http_chain"
|
||||
# [pipelines.plugin_sources.config]
|
||||
# host = "0.0.0.0"
|
||||
# port = 9441 # Required
|
||||
# ingest_path = "/ingest" # Must start with "/"
|
||||
# buffer_size = 1000
|
||||
# max_body_bytes = 8388608 # per-request cap (8 MiB)
|
||||
# read_timeout_ms = 30000
|
||||
# trust_node = true # false: label entries by remote address
|
||||
# [pipelines.plugin_sources.config.tls]
|
||||
# enabled = true # example: mTLS listener
|
||||
# cert_file = "/etc/logwisp/tls/server.crt"
|
||||
# key_file = "/etc/logwisp/tls/server.key"
|
||||
# client_auth = true
|
||||
# client_ca_file = "/etc/logwisp/tls/client-ca.crt"
|
||||
|
||||
###============================================================================
|
||||
### Sinks (1+ required, fan-out)
|
||||
###============================================================================
|
||||
|
||||
## Null sink (testing)
|
||||
# [[pipelines.plugin_sinks]]
|
||||
# id = "null_out"
|
||||
# type = "null"
|
||||
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "default_sink"
|
||||
type = "console"
|
||||
[pipelines.plugin_sinks.config]
|
||||
target = "stdout" # stdout|stderr ("split" NOT supported)
|
||||
# buffer_size = 1000
|
||||
|
||||
## File sink (rotating)
|
||||
# [[pipelines.plugin_sinks]]
|
||||
# id = "file_out"
|
||||
# type = "file"
|
||||
# [pipelines.plugin_sinks.config]
|
||||
# directory = "./logs" # Required
|
||||
# name = "output" # Required
|
||||
# max_size_mb = 100
|
||||
# max_total_size_mb = 1000
|
||||
# min_disk_free_mb = 0 # 0 = no floor (only negatives become 100)
|
||||
# retention_hours = 168.0
|
||||
# buffer_size = 1000
|
||||
# flush_interval_ms = 100
|
||||
|
||||
## HTTP sink (SSE streaming server + JSON status endpoint; IPv4 clients only)
|
||||
## Both endpoints are UNAUTHENTICATED and the stream sends
|
||||
## Access-Control-Allow-Origin: *. Bind to a trusted interface.
|
||||
# [[pipelines.plugin_sinks]]
|
||||
# id = "http_out"
|
||||
# type = "http"
|
||||
# [pipelines.plugin_sinks.config]
|
||||
# host = "0.0.0.0"
|
||||
# port = 8081 # Required
|
||||
# stream_path = "/stream" # Must start with "/"
|
||||
# status_path = "/status" # Must differ from stream_path
|
||||
# buffer_size = 1000 # Sink input queue
|
||||
# client_buffer_size = 256 # Per-client send queue
|
||||
# write_timeout_ms = 0 # Per-event deadline, 0 = none
|
||||
# max_connections = 0 # 0 = unlimited
|
||||
# [pipelines.plugin_sinks.config.tls]
|
||||
# enabled = true # example
|
||||
# cert_file = "/etc/logwisp/tls/server.crt"
|
||||
# key_file = "/etc/logwisp/tls/server.key"
|
||||
# client_auth = false
|
||||
# client_ca_file = ""
|
||||
|
||||
## TCP sink (streaming server, IPv4 clients only)
|
||||
# [[pipelines.plugin_sinks]]
|
||||
# id = "tcp_out"
|
||||
# type = "tcp"
|
||||
# [pipelines.plugin_sinks.config]
|
||||
# host = "0.0.0.0"
|
||||
# port = 9090 # Required
|
||||
# buffer_size = 1000
|
||||
# client_buffer_size = 256
|
||||
# write_timeout_ms = 5000 # Missed deadline disconnects the client
|
||||
# keep_alive = true
|
||||
# keep_alive_period_ms = 30000
|
||||
# max_connections = 0
|
||||
# [pipelines.plugin_sinks.config.tls]
|
||||
# enabled = true # example
|
||||
# cert_file = "/etc/logwisp/tls/server.crt"
|
||||
# key_file = "/etc/logwisp/tls/server.key"
|
||||
|
||||
## TCP chain sink (stdlib client; forwards to downstream tcp_chain source)
|
||||
## Do NOT point a chain sink at a chain source in the SAME pipeline: entries
|
||||
## loop back in and amplify without bound.
|
||||
# [[pipelines.plugin_sinks]]
|
||||
# id = "chain_out"
|
||||
# type = "tcp_chain"
|
||||
# [pipelines.plugin_sinks.config]
|
||||
# host = "10.0.0.1" # Required
|
||||
# port = 9440 # Required
|
||||
# node = "" # origin label, default hostname; preserved across hops
|
||||
# buffer_size = 1000
|
||||
# dial_timeout_ms = 5000
|
||||
# write_timeout_ms = 5000
|
||||
# backoff_min_ms = 500
|
||||
# backoff_max_ms = 30000
|
||||
# keep_alive = true
|
||||
# keep_alive_period_ms = 30000
|
||||
# [pipelines.plugin_sinks.config.tls]
|
||||
# enabled = true # example: mTLS dialer
|
||||
# ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
# server_name = ""
|
||||
# insecure_skip_verify = false
|
||||
# cert_file = "/etc/logwisp/tls/client.crt"
|
||||
# key_file = "/etc/logwisp/tls/client.key"
|
||||
# min_version = "1.3"
|
||||
|
||||
## HTTP chain sink (stdlib client; batched NDJSON POST to downstream http_chain source)
|
||||
# [[pipelines.plugin_sinks]]
|
||||
# id = "hchain_out"
|
||||
# type = "http_chain"
|
||||
# [pipelines.plugin_sinks.config]
|
||||
# host = "10.0.0.1" # Required
|
||||
# port = 9441 # Required
|
||||
# ingest_path = "/ingest"
|
||||
# node = "" # origin label, default hostname; preserved across hops
|
||||
# buffer_size = 1000
|
||||
# max_batch_count = 100
|
||||
# max_batch_bytes = 1048576
|
||||
# flush_interval_ms = 1000
|
||||
# request_timeout_ms = 10000 # Covers dial + write + response
|
||||
# backoff_min_ms = 500
|
||||
# backoff_max_ms = 30000
|
||||
# [pipelines.plugin_sinks.config.tls]
|
||||
# enabled = true # example: mTLS dialer
|
||||
# ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
# cert_file = "/etc/logwisp/tls/client.crt"
|
||||
# key_file = "/etc/logwisp/tls/client.key"
|
||||
+71
-41
@@ -1,73 +1,103 @@
|
||||
# LogWisp
|
||||
# LogWisp Documentation
|
||||
|
||||
A pipeline-based log transport and processing system built in Go. LogWisp provides flexible log collection, filtering, formatting, and distribution with security and reliability features.
|
||||
LogWisp is a pipeline-based log transport and processing system written in Go.
|
||||
It collects log entries from files, stdin, or other LogWisp nodes; rate-limits,
|
||||
filters, and formats them; and distributes them to files, consoles, live network
|
||||
streams, or downstream LogWisp nodes.
|
||||
|
||||
## Features
|
||||
## Documentation Map
|
||||
|
||||
### Core Capabilities
|
||||
- **Pipeline Architecture**: Independent processing pipelines with source(s) → filter → format → sink(s) flow
|
||||
- **Multiple Input Sources**: File monitoring, console (stdin), random log generation, null
|
||||
- **Flexible Output Sinks**: Console, file, HTTP SSE, TCP streaming, null
|
||||
- **Real-time Processing**: Sub-millisecond latency with configurable buffering
|
||||
- **Hot Configuration Reload**: Update pipelines without service restart
|
||||
- **Session Management**: Built-in session tracking for multiple client connections
|
||||
| Document | Contents |
|
||||
|----------|----------|
|
||||
| [Installation](installation.md) | Building, installing, and running as a service |
|
||||
| [Architecture](architecture.md) | Component model, data flow, concurrency, back-pressure |
|
||||
| [Configuration](configuration.md) | TOML structure, precedence, environment and CLI overrides |
|
||||
| [Sources](sources.md) | Every input plugin and its options |
|
||||
| [Sinks](sinks.md) | Every output plugin and its options |
|
||||
| [Filters](filters.md) | Pattern-based inclusion and exclusion |
|
||||
| [Formatters](formatters.md) | Output shaping and sanitization |
|
||||
| [Chaining](chaining.md) | Multi-node topologies and the chain wire protocol |
|
||||
| [Networking](networking.md) | Listeners, dialers, timeouts, connection limits |
|
||||
| [Security](security.md) | TLS and mTLS configuration, threat model, current limits |
|
||||
| [mTLS Authentication Plan](mtls-auth-plan.md) | Design for certificate-based authorization |
|
||||
| [CLI](cli.md) | Flags, signals, exit codes |
|
||||
| [Operations](operations.md) | Running, monitoring, tuning, troubleshooting |
|
||||
|
||||
### Data Processing
|
||||
- **Pattern-based Filtering**: Chainable include/exclude filters with regex support
|
||||
- **Multiple Formatters**: Raw, JSON, and text formatting with integrated sanitizer policies
|
||||
- **Rate Limiting**: Pipeline rate controls
|
||||
- **Heartbeat Generation**: Flow-level heartbeat events for keep-alives
|
||||
A fully annotated configuration covering every option lives at
|
||||
[`config/logwisp.toml`](../config/logwisp.toml).
|
||||
|
||||
### Security & Reliability
|
||||
- **File Rotation**: Size-based rotation with retention policies
|
||||
- **Status Monitoring**: Real-time statistics and health endpoints
|
||||
- **Signal Handling**: Graceful shutdown and configuration reload via signals
|
||||
- **Background Mode**: Daemon operation with proper signal handling
|
||||
- **Quiet Mode**: Silent operation for automated deployments
|
||||
## Capabilities
|
||||
|
||||
## Documentation
|
||||
### Pipeline
|
||||
|
||||
- [Installation Guide](installation.md) - Platform setup and service configuration
|
||||
- [Architecture Overview](architecture.md) - System design and component interaction
|
||||
- [Configuration Reference](configuration.md) - TOML structure and configuration methods
|
||||
- [Input Sources](sources.md) - Available source types and configurations
|
||||
- [Output Sinks](sinks.md) - Sink types and output options
|
||||
- [Filters](filters.md) - Pattern-based log filtering
|
||||
- [Formatters](formatters.md) - Log formatting and transformation
|
||||
- [Networking & Security](networking.md) - Network features (Note: TLS and Auth are currently placeholders in the new architecture)
|
||||
- [Command Line Interface](cli.md) - CLI flags and subcommands
|
||||
- [Operations Guide](operations.md) - Running and maintaining LogWisp
|
||||
- Independent named pipelines, each `sources → flow → sinks`
|
||||
- Fan-in (many sources per pipeline) and fan-out (many sinks per pipeline)
|
||||
- Non-blocking sink dispatch: a stalled sink drops its own events and never
|
||||
stalls the pipeline or its sibling sinks
|
||||
- Hot reload of pipeline configuration via `SIGHUP`/`SIGUSR1` or a file watch
|
||||
|
||||
### Inputs
|
||||
|
||||
`file` (directory tail with rotation detection), `console` (stdin),
|
||||
`random` (synthetic generator), `null`, and the chain ingest listeners
|
||||
`tcp_chain` and `http_chain`.
|
||||
|
||||
### Outputs
|
||||
|
||||
`console`, `file` (rotating), `http` (Server-Sent Events plus a JSON status
|
||||
endpoint), `tcp` (broadcast server), `null`, and the chain forwarders
|
||||
`tcp_chain` and `http_chain`.
|
||||
|
||||
### Processing
|
||||
|
||||
- Token-bucket rate limiting with an optional per-entry size cap
|
||||
- Chainable include/exclude regex filters with `or`/`and` logic
|
||||
- `raw`, `txt`, and `json` formatting with selectable sanitizer policies
|
||||
- Optional flow-level heartbeat entries
|
||||
|
||||
### Transport security
|
||||
|
||||
- TLS 1.2/1.3 on every network source and sink, listener and dialer alike
|
||||
- Mutual TLS: listeners can require and verify client certificates; dialers can
|
||||
present a client identity. See [Security](security.md) for what this does and
|
||||
does not currently give you.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Install LogWisp and create a basic configuration:
|
||||
|
||||
```toml
|
||||
[[pipelines]]
|
||||
name = "default"
|
||||
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
sanitizer_policy = "json"
|
||||
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "default_source"
|
||||
id = "app_logs"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "./"
|
||||
directory = "/var/log/myapp"
|
||||
pattern = "*.log"
|
||||
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "default_sink"
|
||||
id = "stdout"
|
||||
type = "console"
|
||||
[pipelines.plugin_sinks.config]
|
||||
target = "stdout"
|
||||
```
|
||||
|
||||
Run with: `logwisp -c config.toml`
|
||||
```bash
|
||||
logwisp -c config.toml
|
||||
```
|
||||
|
||||
## System Requirements
|
||||
|
||||
- **Operating Systems**: Linux (kernel 6.10+), FreeBSD (14.0+)
|
||||
- **Operating systems**: Linux (kernel 6.10+), FreeBSD (14.0+)
|
||||
- **Architecture**: amd64
|
||||
- **Go Version**: 1.25+ (for building from source)
|
||||
- **Go**: 1.26+ to build from source
|
||||
|
||||
Network sources and sinks bind and dial over IPv4 only.
|
||||
|
||||
## License
|
||||
|
||||
BSD 3-Clause License
|
||||
BSD 3-Clause.
|
||||
|
||||
+160
-136
@@ -1,171 +1,195 @@
|
||||
# Architecture Overview
|
||||
|
||||
LogWisp implements a pipeline-based architecture for flexible log processing and distribution.
|
||||
LogWisp moves log entries through independent pipelines. Everything else —
|
||||
plugins, sessions, TLS, statistics — hangs off that spine.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Pipeline Model
|
||||
|
||||
Each pipeline operates independently with a source → filter → format → sink flow. Multiple pipelines can run concurrently within a single LogWisp instance, each processing different log streams with unique configurations.
|
||||
|
||||
### Component Hierarchy
|
||||
## Component Hierarchy
|
||||
|
||||
```
|
||||
Service (Main Process)
|
||||
├── Pipeline 1
|
||||
│ ├── Plugin Sources (1 or more)
|
||||
│ ├── Flow
|
||||
│ │ ├── Heartbeat Generator (optional)
|
||||
│ │ ├── Rate Limiter (optional)
|
||||
│ │ ├── Filter Chain (optional)
|
||||
│ │ └── Formatter (optional)
|
||||
│ └── Plugin Sinks (1 or more)
|
||||
├── Pipeline 2
|
||||
│ └── [Similar structure]
|
||||
└── Status Reporter (optional)
|
||||
main
|
||||
└── Service
|
||||
├── Pipeline "app"
|
||||
│ ├── Registry instance tracking, single-instance enforcement
|
||||
│ ├── Session Manager per-pipeline connection/session bookkeeping
|
||||
│ ├── Sources[] plugin instances, keyed by id
|
||||
│ ├── Flow
|
||||
│ │ ├── Rate Limiter optional, token bucket
|
||||
│ │ ├── Filter Chain optional, ordered
|
||||
│ │ ├── Formatter raw | txt | json, with sanitizer
|
||||
│ │ └── Heartbeat optional generator
|
||||
│ └── Sinks[] plugin instances, keyed by id
|
||||
├── Pipeline "audit"
|
||||
│ └── ...
|
||||
└── Status Reporter optional, 30s interval
|
||||
```
|
||||
|
||||
Package map:
|
||||
|
||||
| Package | Responsibility |
|
||||
|---------|----------------|
|
||||
| `cmd/logwisp` | Entry point, help, logger bootstrap, signal loop, status reporter |
|
||||
| `internal/config` | Typed config schema, loading, top-level validation |
|
||||
| `internal/service` | Owns the pipeline set; start, stop, shutdown, global stats |
|
||||
| `internal/pipeline` | Pipeline runtime and per-pipeline plugin registry |
|
||||
| `internal/flow` | Rate limiter, filter chain invocation, formatting, heartbeat |
|
||||
| `internal/filter` | Regex filter and filter chain |
|
||||
| `internal/format` | Adapter over `lixenwraith/log` formatter + sanitizer |
|
||||
| `internal/source/*` | Source plugins |
|
||||
| `internal/sink/*` | Sink plugins |
|
||||
| `internal/plugin` | Global factory registry populated by plugin `init()` |
|
||||
| `internal/chain` | Chain wire protocol: hello preamble, entry codec, backoff |
|
||||
| `internal/tlsx` | The single seam between `TLSOptions` and `crypto/tls` |
|
||||
| `internal/session` | Session manager and per-instance proxy |
|
||||
| `internal/core` | Shared types (`LogEntry`, `TransportEvent`), capabilities, constants |
|
||||
| `internal/tokenbucket` | Rate limiter primitive |
|
||||
| `internal/sanitize` | Standalone hex-escaping helpers |
|
||||
|
||||
## Plugin Registration
|
||||
|
||||
Every plugin registers itself in an `init()` function, and
|
||||
`cmd/logwisp/bootstrap.go` blank-imports each package to trigger those
|
||||
`init()`s. Adding a plugin therefore means writing the package, calling
|
||||
`plugin.RegisterSource` / `plugin.RegisterSink`, and adding one blank import.
|
||||
|
||||
Registration may attach metadata. The `console` source declares
|
||||
`MaxInstances: 1`, because a process has only one stdin; the per-pipeline
|
||||
registry rejects a second instance of any such type.
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Processing Stages
|
||||
### Entry lifecycle
|
||||
|
||||
1. **Source Stage**: Plugin sources monitor inputs and generate log entries
|
||||
2. **Flow - Rate Limiting**: Optional pipeline-level rate control
|
||||
3. **Flow - Filtering**: Pattern-based inclusion/exclusion
|
||||
4. **Flow - Formatting**: Transform entries to desired output format with sanitization
|
||||
5. **Distribution**: Fan-out to multiple plugin sinks
|
||||
1. **Source** produces a `core.LogEntry` and publishes it to every subscriber
|
||||
channel it has handed out. Publication is non-blocking: a full subscriber
|
||||
channel increments the source's `dropped_entries` counter.
|
||||
2. **Flow** applies, in order: rate limit → filter chain → formatter. A drop at
|
||||
any stage ends the entry's life and increments `flow.total_dropped`.
|
||||
3. The formatter output becomes a `core.TransportEvent`, which carries both the
|
||||
formatted `Payload` and the original structured `Entry`.
|
||||
4. **Dispatch** sends the event to every sink's input channel with a
|
||||
non-blocking send.
|
||||
|
||||
### Entry Lifecycle
|
||||
`LogEntry` fields:
|
||||
|
||||
Log entries flow through the pipeline as `core.LogEntry` structures containing:
|
||||
- **Time**: Entry timestamp
|
||||
- **Level**: Log level (DEBUG, INFO, WARN, ERROR)
|
||||
- **Source**: Origin identifier
|
||||
- **Message**: Log content
|
||||
- **Fields**: Additional metadata (JSON)
|
||||
- **RawSize**: Original entry size
|
||||
| Field | Purpose |
|
||||
|-------|---------|
|
||||
| `Time` | Entry timestamp |
|
||||
| `Node` | Origin node label for chained topologies; stamped at the first hop, preserved by relays |
|
||||
| `Source` | Origin identifier within the node (filename, plugin id, …) |
|
||||
| `Level` | `DEBUG`/`INFO`/`WARN`/`ERROR`/`TRACE`, when detected |
|
||||
| `Message` | Log content |
|
||||
| `Fields` | Optional structured metadata as raw JSON |
|
||||
| `RawSize` | Original byte size, used by the entry-size cap |
|
||||
|
||||
### Buffering Strategy
|
||||
Carrying `Entry` alongside `Payload` is what makes chain sinks
|
||||
format-independent: a `tcp_chain` or `http_chain` sink re-serializes the
|
||||
structured entry rather than shipping whatever text the local formatter chose.
|
||||
|
||||
Each component maintains internal buffers to handle burst traffic:
|
||||
- Sources: Configurable buffer size (default 1000 entries)
|
||||
- Sinks: Independent buffers per sink
|
||||
- Network components: Additional TCP/HTTP buffers
|
||||
### Back-pressure and drops
|
||||
|
||||
*"Sink dispatch uses non-blocking sends. When a sink's input buffer is full, the event is dropped for that sink only and counted in pipeline statistics (`total_dropped_by_sink`). The policy is uniform and not configurable; a slow sink does not stall the pipeline or other sinks."*
|
||||
There is exactly one drop policy and it is not configurable: **never block**.
|
||||
|
||||
## Component Types
|
||||
| Stage | Full-buffer behaviour | Counter |
|
||||
|-------|----------------------|---------|
|
||||
| Source → subscriber | Drop the entry | source `dropped_entries` |
|
||||
| Flow | Drop on rate limit, filter, or format error | `flow.total_dropped` |
|
||||
| Pipeline → sink | Drop for that sink only | pipeline `total_dropped_by_sink` |
|
||||
| TCP/HTTP sink → client queue | Drop for that client only | sink `dropped_writes` |
|
||||
|
||||
### Sources (Input)
|
||||
|
||||
- **File Source**: File system directory monitoring with rotation detection
|
||||
- **Console Source**: Standard input processing (stdin)
|
||||
- **Random Source**: Generates random log entries for testing
|
||||
- **Null Source**: Discards logs, used for testing
|
||||
|
||||
### Sinks (Output)
|
||||
|
||||
- **Console Sink**: stdout/stderr output
|
||||
- **File Sink**: Rotating file writer
|
||||
- **HTTP Sink**: Server-Sent Events (SSE) streaming
|
||||
- **TCP Sink**: TCP server for client connections
|
||||
- **Null Sink**: Discards all received events
|
||||
|
||||
### Processing Components
|
||||
|
||||
- **Rate Limiter**: Token bucket algorithm for flow control
|
||||
- **Filter Chain**: Sequential pattern matching
|
||||
- **Formatters**: Raw, JSON, or text transformation with sanitizer policies
|
||||
The `tcp_chain` sink is the one deliberate exception. It holds a line across
|
||||
reconnects until it is written or the process shuts down, so a downstream
|
||||
outage propagates backwards as a full input buffer and surfaces as
|
||||
`total_dropped_by_sink` on the pipeline rather than as silent data loss inside
|
||||
the sink. The `http_chain` sink retries a batch with backoff, and drops it only
|
||||
on a non-retryable response or on shutdown (`dropped_batches`).
|
||||
|
||||
## Concurrency Model
|
||||
|
||||
### Goroutine Architecture
|
||||
- One goroutine per source drains that source's subscription and feeds the flow.
|
||||
- The flow's formatter holds a mutex; the underlying formatter reuses an
|
||||
internal buffer and is not goroutine-safe.
|
||||
- Each network sink runs one broadcast/broker goroutine plus, per connection, a
|
||||
writer goroutine (and for TCP, a reader goroutine that exists only to detect
|
||||
disconnects and refresh session activity).
|
||||
- Chain sinks run a single run-loop goroutine that exclusively owns the
|
||||
connection or the pending batch, so no locking is needed around either.
|
||||
- Statistics are atomics; configuration and registries use RW mutexes;
|
||||
shutdown is context cancellation plus wait groups.
|
||||
|
||||
- Each source runs in dedicated goroutines for monitoring
|
||||
- Sinks operate independently with their own processing loops
|
||||
- Network listeners use optimized event loops (gnet for TCP)
|
||||
- Pipeline processing uses channel-based communication
|
||||
### Shutdown ordering
|
||||
|
||||
### Synchronization
|
||||
`Pipeline.Stop` is deliberately ordered so in-flight data drains:
|
||||
|
||||
- Atomic counters for statistics
|
||||
- Read-write mutexes for configuration access
|
||||
- Context-based cancellation for graceful shutdown
|
||||
- Wait groups for coordinated startup/shutdown
|
||||
1. Stop all sources concurrently; each closes its subscriber channels.
|
||||
2. Wait for the run loop, which ends when every subscription channel closes.
|
||||
3. Stop all sinks concurrently.
|
||||
|
||||
## Network Architecture
|
||||
|
||||
### Connection Patterns
|
||||
All listeners bind `tcp4` and all dialers dial `tcp4`. IPv6 clients cannot
|
||||
connect; this is deliberate, not an oversight.
|
||||
|
||||
**Chaining Design**:
|
||||
- Future plan
|
||||
| Plugin | Role | Protocol |
|
||||
|--------|------|----------|
|
||||
| `tcp` sink | Listener | Raw broadcast of formatted payloads |
|
||||
| `http` sink | Listener | HTTP/1.1 SSE; HTTP/2 negotiated via ALPN when TLS is on |
|
||||
| `tcp_chain` source | Listener | Chain protocol, persistent NDJSON stream |
|
||||
| `http_chain` source | Listener | Chain protocol, NDJSON batches over POST |
|
||||
| `tcp_chain` sink | Dialer | Chain protocol, persistent stream, auto-reconnect |
|
||||
| `http_chain` sink | Dialer | Chain protocol, batched POST with retry |
|
||||
|
||||
**Monitoring Design**:
|
||||
- TCP Sink: Debugging interface
|
||||
- HTTP Sink: Browser-based live monitoring
|
||||
TLS is built in exactly one place, `internal/tlsx`, which exposes
|
||||
`Server(opts)` for listeners and `Client(opts, host)` for dialers. See
|
||||
[Security](security.md).
|
||||
|
||||
### Protocol Support
|
||||
## Sessions
|
||||
|
||||
- HTTP/1.1 and HTTP/2 for HTTP connections
|
||||
- Raw TCP connections
|
||||
Each pipeline owns a `session.Manager`. Plugins receive a `session.Proxy`
|
||||
scoped to their instance id, so one plugin cannot see or remove another's
|
||||
sessions. A session records the remote address, creation and last-activity
|
||||
timestamps, and metadata — including `tls` and `tls_peer_cn` for TLS peers.
|
||||
|
||||
Idle sessions are reaped every 5 minutes against a 30-minute idle limit. The
|
||||
HTTP sink's broker treats a vanished session as an eviction signal and closes
|
||||
the corresponding SSE client.
|
||||
|
||||
Session metadata is currently bookkeeping only: nothing in the pipeline makes
|
||||
an authorization decision from it. Closing that gap is the subject of the
|
||||
[mTLS authentication plan](mtls-auth-plan.md).
|
||||
|
||||
## Configuration Reload
|
||||
|
||||
Reload (signal or file watch) rebuilds the entire service:
|
||||
|
||||
1. Re-read the config through the config manager.
|
||||
2. Build a **new** service from it. If construction fails, the old service keeps
|
||||
running untouched.
|
||||
3. Shut the old service down, start the new one, and restart the status
|
||||
reporter if it is enabled.
|
||||
|
||||
Because this is a full rebuild, listening sockets close and reopen and all
|
||||
clients are disconnected. Application logging is configured once at startup and
|
||||
is **not** re-applied on reload.
|
||||
|
||||
## Resource Management
|
||||
|
||||
### Memory Management
|
||||
- Every buffer is bounded; the drop-not-block policy keeps memory flat under
|
||||
load.
|
||||
- Network sinks and chain sources accept a `max_connections` cap. Admission is
|
||||
a load-then-check, so a burst can over-admit by roughly one connection.
|
||||
- Chain listeners bound a single line at `core.MaxLogEntryBytes` (1 MiB); an
|
||||
oversized line is a protocol violation and terminates the connection.
|
||||
- The `http_chain` source caps each request body at `max_body_bytes`.
|
||||
- File sinks rotate on size, cap total rotated size, and honour a retention
|
||||
window.
|
||||
|
||||
- Bounded buffers prevent unbounded growth
|
||||
- Automatic garbage collection via Go runtime
|
||||
- Connection limits prevent resource exhaustion
|
||||
## Performance Notes
|
||||
|
||||
### File Management
|
||||
|
||||
- Automatic rotation based on size thresholds
|
||||
- Retention policies for old log files
|
||||
- Minimum disk space checks before writing
|
||||
|
||||
### Connection Management
|
||||
|
||||
- HTTP sink silenty drops IPv6 connections (deliberate IPv4-only enforcement)
|
||||
|
||||
*Note:* Placeholder; below features are removed in restructuring and will be added in the future release.
|
||||
|
||||
- Per-IP connection limits
|
||||
- Global connection caps
|
||||
- Automatic reconnection with exponential backoff
|
||||
- Keep-alive for persistent connections
|
||||
|
||||
## Reliability Features
|
||||
|
||||
### Fault Tolerance
|
||||
|
||||
- Panic recovery in pipeline processing
|
||||
- Independent pipeline operation
|
||||
- Sink failure isolation
|
||||
|
||||
### Data Integrity
|
||||
|
||||
- Entry validation at ingestion
|
||||
- Size limits for entries and batches
|
||||
- Duplicate detection in file monitoring
|
||||
- Position tracking for file reads
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Throughput
|
||||
|
||||
- Pipeline rate limiting: Configurable (default 1000 entries/second)
|
||||
- Network throughput: Limited by network and sink capacity
|
||||
- File monitoring: Sub-second detection (default 100ms interval)
|
||||
|
||||
### Latency
|
||||
|
||||
- Entry processing: Sub-millisecond in-memory
|
||||
- Network forwarding: Depends on batch configuration
|
||||
- File detection: Configurable check interval
|
||||
|
||||
### Scalability
|
||||
|
||||
- Horizontal: Multiple LogWisp instances with different configurations
|
||||
- Vertical: Multiple pipelines per instance
|
||||
- Fan-out: Multiple sinks per pipeline
|
||||
- Fan-in: Multiple sources per pipeline
|
||||
- In-memory entry processing is sub-millisecond; the formatter mutex is the
|
||||
only shared serialization point in the hot path.
|
||||
- File tailing detects new content within roughly 100 ms (fixed poll), while
|
||||
`check_interval_ms` governs how quickly a *newly created* file is noticed.
|
||||
- `http_chain` trades latency for efficiency: entries wait up to
|
||||
`flush_interval_ms` (default 1 s) before a batch is sent.
|
||||
- Scale out with more pipelines per process, more sinks per pipeline, or more
|
||||
nodes chained together.
|
||||
|
||||
+212
@@ -0,0 +1,212 @@
|
||||
# Chaining
|
||||
|
||||
Chaining links LogWisp nodes together. An edge node forwards its entries to a
|
||||
relay or collector, which can filter, reformat, fan out, or forward them again.
|
||||
Unlike the `tcp` and `http` sinks — which emit *formatted text* for humans and
|
||||
generic clients — chain links carry the **structured entry**, so downstream
|
||||
nodes can filter and reformat as if the entries were local.
|
||||
|
||||
## Topology
|
||||
|
||||
```
|
||||
edge-01 relay consumers
|
||||
┌───────────────┐ ┌────────────────────┐ ┌──────────────┐
|
||||
│ file source │ │ tcp_chain source │ │ browser (SSE)│
|
||||
│ ↓ │ TCP/TLS │ ↓ │ ───► │ nc / telnet │
|
||||
│ tcp_chain sink├────────────►│ flow │ │ archive file │
|
||||
└───────────────┘ :15801 │ ↓ │ └──────────────┘
|
||||
│ http sink, tcp sink│
|
||||
edge-02 │ file sink │
|
||||
┌───────────────┐ │ http_chain sink ───┼──► upstream collector
|
||||
│ file source │ HTTP/TLS │ │
|
||||
│ ↓ ├────────────►│ http_chain source │
|
||||
│ http_chain sink│ :15802 └────────────────────┘
|
||||
└───────────────┘
|
||||
```
|
||||
|
||||
Both chain sources can feed a single pipeline (fan-in) whose sinks then fan the
|
||||
merged stream out. `test/chain-test.sh` builds the two-independent-pipelines
|
||||
variant; `test/chain-aggregate-test.sh` builds the fan-in variant.
|
||||
|
||||
## Node Identity
|
||||
|
||||
Chained entries carry a `node` label identifying where they originated.
|
||||
|
||||
- A chain **sink** stamps `node` on any entry that does not already have one.
|
||||
The label comes from the `node` option, defaulting to `os.Hostname()`.
|
||||
- A chain **source** either honours the sender's label or overrides it,
|
||||
according to `trust_node`:
|
||||
|
||||
| `trust_node` | Behaviour |
|
||||
|--------------|-----------|
|
||||
| `true` (default) | Keep the label the sender declared; fall back to the remote address when absent |
|
||||
| `false` | Always overwrite with the sender's remote address |
|
||||
|
||||
Relays preserve `node`, so a label survives any number of hops and identifies
|
||||
the original producer rather than the last relay.
|
||||
|
||||
Formatters render node identity as a syslog-style prefix on the source field:
|
||||
`edge-01/app.log`. In JSON output the node therefore appears inside the source
|
||||
field, not as a separate top-level key.
|
||||
|
||||
> `trust_node = true` means an authenticated peer can claim **any** node label,
|
||||
> including one belonging to another host. On an untrusted network use
|
||||
> `trust_node = false`, or read the
|
||||
> [mTLS authentication plan](mtls-auth-plan.md), which proposes binding the
|
||||
> label to the peer's certificate identity.
|
||||
|
||||
## Wire Protocol
|
||||
|
||||
Protocol version: **1**. Both transports carry the same canonical entry
|
||||
encoding, and differ only in how the preamble and framing are expressed.
|
||||
|
||||
### TCP transport
|
||||
|
||||
A persistent connection carrying newline-delimited JSON.
|
||||
|
||||
1. The dialer connects and, under TLS, completes the handshake.
|
||||
2. The dialer immediately writes the hello preamble as one JSON line:
|
||||
|
||||
```json
|
||||
{"logwisp":1,"node":"edge-01"}
|
||||
```
|
||||
|
||||
3. The listener reads that line within `hello_timeout_ms` and rejects the
|
||||
connection if it is malformed or declares a different protocol version.
|
||||
4. Every subsequent line is one JSON-encoded `LogEntry`.
|
||||
|
||||
Line size is bounded at 1 MiB. An oversized line is a protocol violation and
|
||||
terminates the connection, because the scanner cannot resynchronize afterwards.
|
||||
|
||||
### HTTP transport
|
||||
|
||||
Batches of NDJSON delivered by `POST`, with the preamble expressed as headers.
|
||||
|
||||
| Header | Direction | Meaning |
|
||||
|--------|-----------|---------|
|
||||
| `X-Logwisp-Protocol` | request | Protocol version; must be `1` |
|
||||
| `X-Logwisp-Node` | request | Origin node label |
|
||||
| `Content-Type` | request | `application/x-ndjson` |
|
||||
| `X-Logwisp-Accepted` | response | Number of entries ingested |
|
||||
|
||||
Responses: `204` on success, `400` for a bad protocol version or a malformed
|
||||
body, `413` when the body cap is exceeded, `405` for a non-`POST` method.
|
||||
|
||||
### Entry encoding
|
||||
|
||||
```json
|
||||
{
|
||||
"time": "2026-01-02T15:04:05.123456789Z",
|
||||
"node": "edge-01",
|
||||
"source": "app.log",
|
||||
"level": "ERROR",
|
||||
"message": "connection refused",
|
||||
"fields": {"attempt": 3}
|
||||
}
|
||||
```
|
||||
|
||||
`node`, `level`, and `fields` are omitted when empty. A missing `time` is filled
|
||||
in at ingest.
|
||||
|
||||
## Delivery Semantics
|
||||
|
||||
| Transport | Guarantee | Failure behaviour |
|
||||
|-----------|-----------|-------------------|
|
||||
| `tcp_chain` | Per-line, held across reconnects | Retries with exponential backoff plus ±20 % jitter until written or shutdown; back-pressure appears upstream as `total_dropped_by_sink` |
|
||||
| `http_chain` | At-least-once per batch | Retries transport errors, `408`, `429`, `5xx`; drops on any other non-2xx (`dropped_batches`) |
|
||||
|
||||
`http_chain` batches can be delivered twice when a successful request's response
|
||||
is lost. There is no de-duplication downstream; design your consumers to
|
||||
tolerate it, or use `tcp_chain` where each line is written once per successful
|
||||
write.
|
||||
|
||||
Neither transport persists anything to disk. Entries buffered in memory during
|
||||
an outage are lost if the process exits.
|
||||
|
||||
## Worked Example
|
||||
|
||||
**Edge node** — tail files, forward over mTLS:
|
||||
|
||||
```toml
|
||||
[[pipelines]]
|
||||
name = "edge"
|
||||
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "app"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "/var/log/myapp"
|
||||
pattern = "*.log"
|
||||
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "forward"
|
||||
type = "tcp_chain"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "relay.internal"
|
||||
port = 15801
|
||||
node = "edge-01"
|
||||
[pipelines.plugin_sinks.config.tls]
|
||||
enabled = true
|
||||
ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
cert_file = "/etc/logwisp/tls/edge-01.crt"
|
||||
key_file = "/etc/logwisp/tls/edge-01.key"
|
||||
```
|
||||
|
||||
**Relay** — ingest, keep errors only, archive and stream:
|
||||
|
||||
```toml
|
||||
[[pipelines]]
|
||||
name = "relay"
|
||||
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["ERROR", "FATAL"]
|
||||
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
sanitizer_policy = "json"
|
||||
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "ingest"
|
||||
type = "tcp_chain"
|
||||
[pipelines.plugin_sources.config]
|
||||
host = "0.0.0.0"
|
||||
port = 15801
|
||||
trust_node = true
|
||||
[pipelines.plugin_sources.config.tls]
|
||||
enabled = true
|
||||
cert_file = "/etc/logwisp/tls/relay.crt"
|
||||
key_file = "/etc/logwisp/tls/relay.key"
|
||||
client_auth = true
|
||||
client_ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "archive"
|
||||
type = "file"
|
||||
[pipelines.plugin_sinks.config]
|
||||
directory = "/var/log/logwisp"
|
||||
name = "errors"
|
||||
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "live"
|
||||
type = "http"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "127.0.0.1"
|
||||
port = 8080
|
||||
```
|
||||
|
||||
## Operational Notes
|
||||
|
||||
- **Formatting is a relay decision.** Because chain links carry structured
|
||||
entries, the edge node's `flow.format` affects only its own local sinks. Set
|
||||
the output shape on the node that owns the human-facing sink.
|
||||
- **Filtering early saves bandwidth.** A filter on the edge drops entries before
|
||||
they cross the network; a filter on the relay is easier to change centrally.
|
||||
- **Rate limits are per pipeline.** An edge limit protects the link; a relay
|
||||
limit protects the relay from a noisy edge.
|
||||
- **Heartbeats traverse chain links** as ordinary structured entries and keep
|
||||
otherwise-idle links and their sessions warm.
|
||||
- **Ports** used by the bundled test scripts: `15801` tcp_chain ingest, `15802`
|
||||
http_chain ingest, `15803` tcp sink, `15804` http sink.
|
||||
- **Use `127.0.0.1`, not `localhost`**, when testing locally: all listeners and
|
||||
dialers are IPv4-only, and `localhost` may resolve to `::1`.
|
||||
+130
-142
@@ -1,196 +1,184 @@
|
||||
# Command Line Interface
|
||||
|
||||
LogWisp CLI reference for commands and options.
|
||||
|
||||
## Synopsis
|
||||
|
||||
```bash
|
||||
logwisp [command] [options]
|
||||
logwisp [options]
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
### Main Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `--version` | Display version information |
|
||||
| `--help` | Show help information |
|
||||
|
||||
### version Command
|
||||
|
||||
Display version information.
|
||||
|
||||
```bash
|
||||
logwisp version
|
||||
logwisp -v
|
||||
logwisp [options]
|
||||
logwisp help | -h | --help
|
||||
logwisp --version
|
||||
```
|
||||
|
||||
Output includes:
|
||||
- Version number
|
||||
- Build date
|
||||
- Git commit hash
|
||||
- Go version
|
||||
LogWisp has no subcommands. Earlier releases shipped `logwisp auth` and
|
||||
`logwisp tls` for credential and certificate generation; both were removed
|
||||
during the restructure. Use `openssl` or your PKI tooling instead — see
|
||||
[Security](security.md).
|
||||
|
||||
## Global Options
|
||||
## Options
|
||||
|
||||
### Configuration Options
|
||||
Any scalar configuration key is settable as a flag using its TOML path:
|
||||
|
||||
```
|
||||
--<path>=<value> e.g. --logging.level=debug
|
||||
--<path> <value> e.g. --logging.level debug
|
||||
--<path> bare flag, means true
|
||||
```
|
||||
|
||||
### Common
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `-c, --config` | Configuration file path | `./logwisp.toml` |
|
||||
| `-q, --quiet` | Suppress console output | false |
|
||||
| `--status-reporter` | Status logging | true |
|
||||
| `--auto-reload` | Enable config hot reload | false |
|
||||
| `-c <path>` | Configuration file | `./logwisp.toml` |
|
||||
| `--config=<path>` | Configuration file (equals form only) | `./logwisp.toml` |
|
||||
| `--quiet` | Suppress all application output | `false` |
|
||||
| `--status_reporter=<bool>` | Periodic status logging | `true` |
|
||||
| `--auto_reload=<bool>` | Reload config when the file changes | `false` |
|
||||
| `--version` | Print version and exit | — |
|
||||
| `-h`, `--help`, `help` | Print usage and exit | — |
|
||||
|
||||
### Logging Options
|
||||
> `--config <path>` with a space is not recognized. The path resolver
|
||||
> understands `-c <path>` and `--config=<path>` only; the space form is treated
|
||||
> as an unknown flag, warned about, and ignored, after which LogWisp silently
|
||||
> falls back to `./logwisp.toml`.
|
||||
>
|
||||
> `-c` as the final argument, with no path after it, crashes with an index
|
||||
> panic rather than reporting a usage error.
|
||||
|
||||
| Flag | Description | Values |
|
||||
|------|-------------|--------|
|
||||
| `--logging.output` | Log output mode | file, stdout, stderr, split, all, none |
|
||||
| `--logging.level` | Log level | debug, info, warn, error |
|
||||
| `--logging.file.directory` | Log directory | Path |
|
||||
| `--logging.file.name` | Log filename | String |
|
||||
| `--logging.file.max_size_mb` | Max file size | Integer |
|
||||
| `--logging.file.max_total_size_mb` | Total size limit | Integer |
|
||||
| `--logging.file.retention_hours` | Retention period | Float |
|
||||
| `--logging.console.target` | Console target | stdout, stderr, split |
|
||||
| `--logging.console.format` | Output format | txt, json |
|
||||
### Logging
|
||||
|
||||
### Pipeline Options
|
||||
| Flag | Values |
|
||||
|------|--------|
|
||||
| `--logging.output` | `file`, `stdout`, `stderr`, `split`, `all`, `none` |
|
||||
| `--logging.level` | `debug`, `info`, `warn`, `error` |
|
||||
| `--logging.format` | `raw`, `txt`, `json` |
|
||||
| `--logging.sanitization` | `raw`, `json`, `txt`, `shell` |
|
||||
| `--logging.file.directory` | path |
|
||||
| `--logging.file.name` | string |
|
||||
| `--logging.file.max_size_mb` | integer |
|
||||
| `--logging.file.max_total_size_mb` | integer |
|
||||
| `--logging.file.retention_hours` | float |
|
||||
|
||||
Configure pipelines via CLI (N = array index, 0-based).
|
||||
`--logging.console.target` is accepted but has no effect; the console
|
||||
destination is derived from `--logging.output`.
|
||||
|
||||
**Pipeline Configuration:**
|
||||
### Pipelines
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--pipelines.N.name` | Pipeline name |
|
||||
| `--pipelines.N.plugin_sources.N.type` | Source type |
|
||||
| `--pipelines.N.flow.filters.N.type` | Filter type |
|
||||
| `--pipelines.N.plugin_sinks.N.type` | Sink type |
|
||||
Pipelines, sources, sinks, and filters **cannot** be configured from the command
|
||||
line. Array-indexed paths such as `--pipelines.0.name=app` or
|
||||
`--pipelines.0.plugin_sinks.0.type=null` are reported as unrecognized and
|
||||
ignored:
|
||||
|
||||
## Flag Formats
|
||||
|
||||
### Boolean Flags
|
||||
|
||||
```bash
|
||||
logwisp --quiet
|
||||
logwisp --quiet=true
|
||||
logwisp --pipelines.0.plugin_sources.0.type=console
|
||||
```
|
||||
Warning: unrecognized flags ignored: [pipelines.0.name]
|
||||
```
|
||||
|
||||
### String Flags
|
||||
|
||||
```bash
|
||||
logwisp --config /etc/logwisp/config.toml
|
||||
logwisp -c config.toml
|
||||
```
|
||||
|
||||
### Nested Configuration
|
||||
|
||||
```bash
|
||||
logwisp --logging.level=debug
|
||||
logwisp --pipelines.0.name=myapp
|
||||
logwisp --pipelines.0.sources.0.type=console
|
||||
```
|
||||
|
||||
### Array Values (JSON)
|
||||
|
||||
```bash
|
||||
logwisp --pipelines.0.flow.filters.0.patterns='["ERROR","WARN"]'
|
||||
```
|
||||
Use a configuration file. Older documentation described CLI pipeline overrides
|
||||
that the current loader does not implement.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All flags can be set via environment:
|
||||
Configuration paths map to environment variables by replacing `.` with `_` and
|
||||
uppercasing:
|
||||
|
||||
```bash
|
||||
export LOGWISP_QUIET=true
|
||||
export LOGWISP_LOGGING_LEVEL=debug
|
||||
export LOGWISP_PIPELINES_0_NAME=myapp
|
||||
export QUIET=true
|
||||
export LOGGING_LEVEL=debug
|
||||
export LOGGING_FILE_DIRECTORY=/var/log/logwisp
|
||||
```
|
||||
|
||||
## Configuration Precedence
|
||||
> The `LOGWISP_` prefix is **not** currently applied to these — see
|
||||
> [Configuration](configuration.md#environment-variables). Bare names like
|
||||
> `QUIET` are what LogWisp actually reads, which is worth knowing both to make
|
||||
> overrides work and to avoid accidental collisions.
|
||||
|
||||
1. Command-line flags (highest)
|
||||
The two variables that do carry the prefix are read directly by the path
|
||||
resolver:
|
||||
|
||||
| Variable | Effect |
|
||||
|----------|--------|
|
||||
| `LOGWISP_CONFIG_FILE` | Configuration file path; joined onto `LOGWISP_CONFIG_DIR` when both are set |
|
||||
| `LOGWISP_CONFIG_DIR` | Configuration directory; alone, implies `<dir>/logwisp.toml` |
|
||||
|
||||
As with flags, array elements cannot be set this way.
|
||||
|
||||
## Precedence
|
||||
|
||||
1. Command-line flags
|
||||
2. Environment variables
|
||||
3. Configuration file
|
||||
4. Built-in defaults (lowest)
|
||||
4. Built-in defaults
|
||||
|
||||
## Exit Codes
|
||||
|
||||
| Code | Description |
|
||||
|------|-------------|
|
||||
| 0 | Success |
|
||||
| 1 | General error |
|
||||
| 2 | Configuration file not found |
|
||||
| 137 | SIGKILL received |
|
||||
|
||||
## Signal Handling
|
||||
## Signals
|
||||
|
||||
| Signal | Action |
|
||||
|--------|--------|
|
||||
| SIGINT (Ctrl+C) | Graceful shutdown |
|
||||
| SIGTERM | Graceful shutdown |
|
||||
| SIGHUP | Reload configuration |
|
||||
| SIGUSR1 | Reload configuration |
|
||||
| SIGKILL | Immediate termination |
|
||||
| `SIGINT` | Graceful shutdown |
|
||||
| `SIGTERM` | Graceful shutdown |
|
||||
| `SIGHUP` | Reload configuration |
|
||||
| `SIGUSR1` | Reload configuration |
|
||||
|
||||
`SIGHUP` is ignored during startup, before the signal handler is installed, so
|
||||
LogWisp survives a terminal hang-up like `nohup`. Once running, it triggers a
|
||||
reload rather than terminating.
|
||||
|
||||
Reload rebuilds the whole service. A configuration error leaves the running
|
||||
service untouched; see [Configuration](configuration.md#hot-reload).
|
||||
|
||||
## Exit Codes
|
||||
|
||||
| Code | Meaning |
|
||||
|------|---------|
|
||||
| `0` | Clean shutdown, or `--version` / `--help` |
|
||||
| `1` | General error: config load or validation failure, logger init failure, service bootstrap failure |
|
||||
| `2` | Explicitly requested configuration file not found |
|
||||
|
||||
Exit code 2 applies only when the file was named explicitly (`-c`,
|
||||
`--config=`, or the `LOGWISP_CONFIG_*` variables). A missing discovered default
|
||||
is not an error, and LogWisp starts on built-in defaults.
|
||||
|
||||
## Built-in Defaults
|
||||
|
||||
With no configuration file present, LogWisp runs one pipeline named
|
||||
`default_pipeline`: a `random` source with `special = true`, JSON formatting,
|
||||
a rate limit of 5 entries/second with a burst of 10 and `policy = "drop"`, and a
|
||||
`console` sink on stdout. It is a self-demonstrating idle mode, not a useful
|
||||
production configuration.
|
||||
|
||||
Note that as soon as your file defines `[[pipelines]]`, that entire default
|
||||
pipeline — rate limit included — is replaced rather than merged.
|
||||
|
||||
## Usage Patterns
|
||||
|
||||
### Development Mode
|
||||
**Development**
|
||||
|
||||
```bash
|
||||
# Verbose logging to console
|
||||
logwisp --logging.output=stderr --logging.level=debug
|
||||
# verbose, everything to stderr
|
||||
logwisp -c dev.toml --logging.output=stderr --logging.level=debug
|
||||
|
||||
# Quick test with stdin
|
||||
logwisp --pipelines.0.plugin_sources.0.type=console --pipelines.0.plugin_sinks.0.type=console
|
||||
# no config at all: synthetic generator to stdout
|
||||
logwisp
|
||||
```
|
||||
|
||||
### Production Deployment
|
||||
**Configuration check**
|
||||
|
||||
```bash
|
||||
# Background with file logging
|
||||
logwisp --background --config /etc/logwisp/prod.toml --logging.output=file
|
||||
|
||||
# Systemd service
|
||||
ExecStart=/usr/local/bin/logwisp --config /etc/logwisp/config.toml
|
||||
# starts the service; a config error exits non-zero before any pipeline runs
|
||||
logwisp -c /etc/logwisp/logwisp.toml --logging.level=debug
|
||||
```
|
||||
|
||||
### Debugging
|
||||
There is no dry-run or validate-only mode. The closest approximation is starting
|
||||
with debug logging and stopping once the pipelines report as started.
|
||||
|
||||
**Production**
|
||||
|
||||
```bash
|
||||
# Check configuration
|
||||
logwisp --config test.toml --logging.level=debug --disable-status-reporter
|
||||
|
||||
# Dry run (verify config only)
|
||||
logwisp --config test.toml --quiet
|
||||
logwisp -c /etc/logwisp/logwisp.toml --logging.output=file
|
||||
```
|
||||
|
||||
## Help System
|
||||
Run under a supervisor (systemd, rc.d) rather than backgrounding it — there is
|
||||
no `--background` flag; earlier releases had one and it was removed. See
|
||||
[Installation](installation.md).
|
||||
|
||||
### General Help
|
||||
**Reload**
|
||||
|
||||
```bash
|
||||
logwisp --help
|
||||
logwisp -h
|
||||
logwisp help
|
||||
kill -HUP $(pidof logwisp)
|
||||
kill -USR1 $(pidof logwisp)
|
||||
```
|
||||
|
||||
## Special Flags
|
||||
|
||||
### Internal Flags
|
||||
|
||||
These flags are for internal use:
|
||||
- `--background-daemon`: Child process indicator
|
||||
- `--config-save-on-exit`: Save config on shutdown
|
||||
|
||||
### Hidden Behaviors
|
||||
|
||||
- SIGHUP ignored ignored during startup (after startup triggers config reload)
|
||||
- Automatic panic recovery in pipelines
|
||||
- Resource cleanup on shutdown
|
||||
|
||||
+220
-138
@@ -1,203 +1,285 @@
|
||||
# Configuration Reference
|
||||
|
||||
LogWisp configuration uses TOML format with flexible override mechanisms.
|
||||
LogWisp is configured with TOML. A complete annotated file listing every option
|
||||
and its default ships as [`config/logwisp.toml`](../config/logwisp.toml).
|
||||
|
||||
## Configuration Precedence
|
||||
|
||||
Configuration sources are evaluated in order:
|
||||
1. **Command-line flags** (highest priority)
|
||||
2. **Environment variables**
|
||||
3. **Configuration file**
|
||||
4. **Built-in defaults** (lowest priority)
|
||||
Sources are merged in this order, highest priority first:
|
||||
|
||||
1. Command-line flags
|
||||
2. Environment variables
|
||||
3. Configuration file
|
||||
4. Built-in defaults
|
||||
|
||||
The `pipelines` array is replaced wholesale, not merged: as soon as your file
|
||||
defines `[[pipelines]]`, the built-in default pipeline (and its default rate
|
||||
limit and formatter) disappears entirely.
|
||||
|
||||
## File Location
|
||||
|
||||
LogWisp searches for configuration in order:
|
||||
1. Path specified via `--config` flag
|
||||
2. Path from `LOGWISP_CONFIG_FILE` environment variable
|
||||
3. `~/.config/logwisp/logwisp.toml`
|
||||
4. `./logwisp.toml` in current directory
|
||||
The path is resolved before any other configuration is read:
|
||||
|
||||
1. `-c <path>` on the command line
|
||||
2. `--config=<path>` on the command line
|
||||
3. `$LOGWISP_CONFIG_FILE`, joined onto `$LOGWISP_CONFIG_DIR` when both are set
|
||||
4. `$LOGWISP_CONFIG_DIR/logwisp.toml`
|
||||
5. `~/.config/logwisp/logwisp.toml`, if it exists
|
||||
6. `./logwisp.toml`
|
||||
|
||||
Missing file behaviour differs by how it was chosen. An explicitly requested
|
||||
file that does not exist is a fatal error (exit code 2); a missing discovered
|
||||
default is not an error, and LogWisp starts on built-in defaults.
|
||||
|
||||
> `--config <path>` with a space is **not** recognized as a config path. It is
|
||||
> parsed as an unknown flag, warned about, and ignored — LogWisp then silently
|
||||
> falls back to `./logwisp.toml`. Use `-c <path>` or `--config=<path>`.
|
||||
|
||||
## Global Settings
|
||||
|
||||
Top-level configuration options:
|
||||
|
||||
| Setting | Type | Default | Description |
|
||||
|---------|------|---------|-------------|
|
||||
| `quiet` | bool | false | Suppress console output |
|
||||
| `status_reporter` | bool | true | Periodic status logging |
|
||||
| `auto_reload` | bool | false | Enable file watch for auto-reload |
|
||||
| `quiet` | bool | `false` | Disable all application logging and console diagnostics |
|
||||
| `status_reporter` | bool | `true` | Emit a periodic status report every 30 s at DEBUG level |
|
||||
| `auto_reload` | bool | `false` | Watch the config file and reload pipelines on change |
|
||||
|
||||
## Logging Configuration
|
||||
`--version` prints version information and exits; it is not a persistent
|
||||
setting.
|
||||
|
||||
LogWisp's internal operational logging:
|
||||
Note that `status_reporter` writes at DEBUG level, so it produces nothing unless
|
||||
`logging.level = "debug"`.
|
||||
|
||||
## Application Logging
|
||||
|
||||
This configures LogWisp's own operational log, not the log data it transports.
|
||||
|
||||
```toml
|
||||
[logging]
|
||||
output = "stdout" # file|stdout|stderr|split|all|none
|
||||
level = "info" # debug|info|warn|error
|
||||
output = "stdout" # file | stdout | stderr | split | all | none
|
||||
level = "info" # debug | info | warn | error
|
||||
format = "txt" # raw | txt | json
|
||||
# sanitization = "" # raw | json | txt | shell
|
||||
|
||||
[logging.file]
|
||||
directory = "./log"
|
||||
name = "logwisp"
|
||||
max_size_mb = 100
|
||||
directory = "./log"
|
||||
name = "logwisp"
|
||||
max_size_mb = 100
|
||||
max_total_size_mb = 1000
|
||||
retention_hours = 168.0
|
||||
|
||||
[logging.console]
|
||||
target = "stdout" # stdout|stderr|split
|
||||
retention_hours = 168.0
|
||||
```
|
||||
|
||||
### Output Modes
|
||||
### Output modes
|
||||
|
||||
- **file**: Write to log files only
|
||||
- **stdout**: Write to standard output
|
||||
- **stderr**: Write to standard error
|
||||
- **split**: INFO/DEBUG to stdout, WARN/ERROR to stderr
|
||||
- **all**: Write to both file and console
|
||||
- **none**: Disable all logging
|
||||
| Mode | Behaviour |
|
||||
|------|-----------|
|
||||
| `file` | Files only |
|
||||
| `stdout` | Standard output only |
|
||||
| `stderr` | Standard error only |
|
||||
| `split` | DEBUG/INFO to stdout, WARN/ERROR to stderr |
|
||||
| `all` | Files plus split console |
|
||||
| `none` | No application logging |
|
||||
|
||||
`[logging.file]` applies only to the `file` and `all` modes.
|
||||
|
||||
> `[logging.console].target` is accepted and validated (`stdout`, `stderr`,
|
||||
> `split`) but **not applied**. The console destination is derived from
|
||||
> `logging.output`. The key is retained for compatibility; setting it has no
|
||||
> effect.
|
||||
|
||||
`quiet = true` overrides every logging setting and disables both file and
|
||||
console output.
|
||||
|
||||
## Pipeline Configuration
|
||||
|
||||
Each `[[pipelines]]` section defines an independent processing pipeline:
|
||||
|
||||
```toml
|
||||
[[pipelines]]
|
||||
name = "pipeline-name"
|
||||
name = "app" # required, unique across pipelines
|
||||
|
||||
# Rate limiting (optional)
|
||||
# --- flow: everything between sources and sinks ---
|
||||
[pipelines.flow.rate_limit]
|
||||
rate = 1000.0
|
||||
burst = 2000.0
|
||||
policy = "drop" # pass|drop
|
||||
max_entry_size_bytes = 0 # 0=unlimited
|
||||
rate = 1000.0
|
||||
burst = 2000.0
|
||||
policy = "drop"
|
||||
max_entry_size_bytes = 65536
|
||||
|
||||
# Format configuration (optional)
|
||||
[pipelines.flow.format]
|
||||
type = "json" # raw|json|txt
|
||||
sanitizer_policy = "json"
|
||||
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "my_source"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
# ... source-specific config
|
||||
|
||||
# Filters (optional)
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
logic = "or"
|
||||
type = "include"
|
||||
logic = "or"
|
||||
patterns = ["ERROR", "WARN"]
|
||||
|
||||
# Sinks (required, 1+)
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
sanitizer_policy = "json"
|
||||
|
||||
[pipelines.flow.heartbeat]
|
||||
enabled = true
|
||||
interval_ms = 30000
|
||||
|
||||
# --- sources: one or more ---
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "app_logs" # unique within the pipeline
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "/var/log/myapp"
|
||||
|
||||
# --- sinks: one or more ---
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "my_sink"
|
||||
id = "sse"
|
||||
type = "http"
|
||||
[pipelines.plugin_sinks.config]
|
||||
# ... sink-specific config
|
||||
port = 8080
|
||||
```
|
||||
|
||||
Every source and sink is a plugin instance with three keys:
|
||||
|
||||
| Key | Meaning |
|
||||
|-----|---------|
|
||||
| `id` | Instance identifier, unique within the pipeline; appears in logs and stats |
|
||||
| `type` | Registered plugin type |
|
||||
| `config` | Plugin-specific table; see [Sources](sources.md) and [Sinks](sinks.md) |
|
||||
|
||||
`config_file` is reserved on both structures for a future include mechanism and
|
||||
is not implemented.
|
||||
|
||||
### Flow stages
|
||||
|
||||
| Block | Optional | Reference |
|
||||
|-------|----------|-----------|
|
||||
| `flow.rate_limit` | yes | below |
|
||||
| `flow.filters` | yes | [Filters](filters.md) |
|
||||
| `flow.format` | yes (defaults to `raw`) | [Formatters](formatters.md) |
|
||||
| `flow.heartbeat` | yes | below |
|
||||
|
||||
#### Rate limiting
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `rate` | float | `0` | Entries per second; `<= 0` disables the limiter entirely |
|
||||
| `burst` | float | `rate` | Token bucket capacity |
|
||||
| `policy` | string | `pass` | `pass` allows everything through, `drop` discards over-limit entries |
|
||||
| `max_entry_size_bytes` | int | `0` | Per-entry byte cap; `0` = unlimited |
|
||||
|
||||
Two behaviours are easy to trip over:
|
||||
|
||||
- The limiter is constructed only when `rate > 0`. With `rate = 0`,
|
||||
`max_entry_size_bytes` is never enforced.
|
||||
- `policy = "pass"` short-circuits the whole check, including the size cap.
|
||||
To enforce a size cap you need `rate > 0` **and** `policy = "drop"`.
|
||||
|
||||
#### Heartbeat
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `enabled` | bool | `false` | Enable heartbeat generation |
|
||||
| `interval_ms` | int | `1000` | Interval; minimum `100` |
|
||||
| `include_timestamp` | bool | `false` | `false` formats with level only, no timestamp |
|
||||
| `include_stats` | bool | `false` | Attach `beat_count` and measured `interval_ms` as fields |
|
||||
| `format` | string | `txt` | `txt`, `json`, or `raw` |
|
||||
|
||||
Heartbeats are ordinary entries with source `heartbeat` and level `INFO`. They
|
||||
are generated after the flow's filter and rate-limit stages, so filters do not
|
||||
suppress them, and they reach every sink in the pipeline.
|
||||
|
||||
> `format = "comment"` (SSE comment framing) appears in older documentation and
|
||||
> in a code path in the generator, but the validator rejects it and the pipeline
|
||||
> fails to start. Use `txt`, `json`, or `raw`.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All configuration options support environment variable overrides:
|
||||
Environment overrides are derived from the TOML path: `.` becomes `_` and the
|
||||
result is uppercased.
|
||||
|
||||
### Naming Convention
|
||||
|
||||
- Prefix: `LOGWISP_`
|
||||
- Path separator: `_` (underscore)
|
||||
- Array indices: Numeric suffix (0-based)
|
||||
- Case: UPPERCASE
|
||||
|
||||
### Mapping Examples
|
||||
|
||||
| TOML Path | Environment Variable |
|
||||
| TOML path | Environment variable |
|
||||
|-----------|---------------------|
|
||||
| `quiet` | `LOGWISP_QUIET` |
|
||||
| `logging.level` | `LOGWISP_LOGGING_LEVEL` |
|
||||
| `pipelines[0].name` | `LOGWISP_PIPELINES_0_NAME` |
|
||||
| `pipelines[0].plugin_sources[0].type` | `LOGWISP_PIPELINES_0_PLUGIN_SOURCES_0_TYPE` |
|
||||
| `quiet` | `QUIET` |
|
||||
| `status_reporter` | `STATUS_REPORTER` |
|
||||
| `logging.level` | `LOGGING_LEVEL` |
|
||||
| `logging.file.directory` | `LOGGING_FILE_DIRECTORY` |
|
||||
|
||||
> **The `LOGWISP_` prefix is not currently applied.** The configuration loader
|
||||
> requests it, but supplying a custom path-to-variable transform replaces the
|
||||
> prefixing step rather than composing with it, so LogWisp reads bare
|
||||
> `QUIET`, `LOGGING_LEVEL`, and so on from the environment. Treat this as
|
||||
> current behaviour to be aware of — bare names like `QUIET` can collide with
|
||||
> unrelated variables — rather than as intended design.
|
||||
>
|
||||
> The two exceptions are `LOGWISP_CONFIG_FILE` and `LOGWISP_CONFIG_DIR`, which
|
||||
> are read directly by the path resolver and **do** carry the prefix.
|
||||
|
||||
Only scalar paths that exist in the configuration schema can be set this way.
|
||||
Array elements cannot: `PIPELINES_0_NAME` has no effect.
|
||||
|
||||
## Command-Line Overrides
|
||||
|
||||
All configuration options can be overridden via CLI flags:
|
||||
Any scalar configuration path is settable as a flag using its TOML path:
|
||||
|
||||
```bash
|
||||
logwisp --quiet \
|
||||
--logging.level=debug \
|
||||
--pipelines.0.name=myapp \
|
||||
--pipelines.0.plugin_sources.0.type=console
|
||||
logwisp --logging.level=debug --status_reporter=false
|
||||
logwisp --logging.level debug # space form also works
|
||||
logwisp --quiet # bare flag means true
|
||||
```
|
||||
|
||||
## Configuration Validation
|
||||
Unrecognized flags are reported on stderr before the logger exists and are then
|
||||
ignored:
|
||||
|
||||
LogWisp validates configuration at startup:
|
||||
- Rpipelines non-empty, name non-empty, ≥1 source, ≥1 sink, logging enum values.equired fields presence
|
||||
```
|
||||
Warning: unrecognized flags ignored: [pipelines.0.name]
|
||||
```
|
||||
|
||||
Partial check in plugin constructor:
|
||||
- Type correctness
|
||||
- Port conflicts
|
||||
- Path accessibility
|
||||
- Pattern compilation
|
||||
- Network address formats
|
||||
> Array-indexed paths are **not** settable from the command line.
|
||||
> `--pipelines.0.name=x`, `--pipelines.0.plugin_sinks.0.type=null`, and similar
|
||||
> flags are reported as unrecognized and ignored. Pipelines, sources, sinks, and
|
||||
> filters can only be defined in the configuration file. Older documentation
|
||||
> claimed otherwise.
|
||||
|
||||
## Validation
|
||||
|
||||
Startup validation is intentionally split.
|
||||
|
||||
`internal/config` validates only global structure:
|
||||
|
||||
- at least one pipeline
|
||||
- unique, non-empty pipeline names
|
||||
- at least one source and one sink per pipeline
|
||||
- `logging.output`, `logging.level`, `logging.format`, `logging.sanitization`,
|
||||
and `logging.console.target` enum membership
|
||||
|
||||
Everything else is validated by the plugin constructor that owns it — port
|
||||
range, required paths, path prefixes, enum values, regex compilation, TLS file
|
||||
loading. A failure there aborts pipeline construction with a message naming the
|
||||
pipeline, plugin id, and offending key.
|
||||
|
||||
There is **no** cross-pipeline port-conflict detection. Two sinks bound to the
|
||||
same port fail at listener bind time, when the pipeline starts.
|
||||
|
||||
## Hot Reload
|
||||
|
||||
Enable configuration hot reload:
|
||||
|
||||
```toml
|
||||
auto_reload = true
|
||||
```
|
||||
|
||||
Or via command line:
|
||||
```bash
|
||||
logwisp --auto-reload
|
||||
```
|
||||
or send `SIGHUP` / `SIGUSR1`.
|
||||
|
||||
Reload triggers:
|
||||
- File modification detection
|
||||
- SIGHUP or SIGUSR1 signals
|
||||
Reload rebuilds the whole service: a new service is constructed from the new
|
||||
configuration first, and only if that succeeds is the old one shut down. A
|
||||
configuration error therefore leaves the running service untouched.
|
||||
|
||||
Reloadable items:
|
||||
- Pipeline configurations
|
||||
- Sources and sinks
|
||||
- Filters and formatters
|
||||
- Rate limits
|
||||
| Reloaded | Not reloaded |
|
||||
|----------|--------------|
|
||||
| Pipelines, sources, sinks | `logging.*` (applied once at startup) |
|
||||
| Filters, formatters, rate limits, heartbeats | `quiet` |
|
||||
| `status_reporter` | `auto_reload` (the watcher is not restarted) |
|
||||
|
||||
Non-reloadable (requires restart):
|
||||
- Logging configuration
|
||||
- Global settings
|
||||
Because the rebuild is total, listeners close and reopen and every connected
|
||||
client is disconnected. Chain sinks reconnect on their own backoff schedule.
|
||||
|
||||
## Default Configuration
|
||||
## Type Reference
|
||||
|
||||
Minimal working configuration:
|
||||
|
||||
```toml
|
||||
[[pipelines]]
|
||||
name = "default"
|
||||
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "default_source"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "./"
|
||||
pattern = "*.log"
|
||||
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "default_sink"
|
||||
type = "console"
|
||||
[pipelines.plugin_sinks.config]
|
||||
target = "stdout"
|
||||
```
|
||||
|
||||
## Configuration Schema
|
||||
|
||||
### Type Reference
|
||||
|
||||
| TOML Type | Go Type | Environment Format |
|
||||
|-----------|---------|-------------------|
|
||||
| String | string | Plain text |
|
||||
| Integer | int64 | Numeric string |
|
||||
| Float | float64 | Decimal string |
|
||||
| Boolean | bool | true/false |
|
||||
| Array | []T | JSON array string |
|
||||
| Table | struct | Nested with `_` |
|
||||
| TOML type | Go type | Command-line / environment form |
|
||||
|-----------|---------|-------------------------------|
|
||||
| String | `string` | Plain text |
|
||||
| Integer | `int64` | Decimal string |
|
||||
| Float | `float64` | Decimal string |
|
||||
| Boolean | `bool` | `true` / `false`, or a bare flag for `true` |
|
||||
| Array | `[]T` | Not settable outside the file |
|
||||
| Table | struct | Nested path with `.` (flags) or `_` (environment) |
|
||||
|
||||
+157
-150
@@ -1,185 +1,192 @@
|
||||
# Filters
|
||||
|
||||
LogWisp filters control which log entries pass through the pipeline using pattern matching.
|
||||
|
||||
## Filter Types
|
||||
|
||||
### Include Filter
|
||||
|
||||
Only entries matching patterns pass through.
|
||||
Filters decide which entries continue through a pipeline. They run in the flow,
|
||||
after rate limiting and before formatting.
|
||||
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
logic = "or" # or|and
|
||||
patterns = [
|
||||
"ERROR",
|
||||
"WARN",
|
||||
"CRITICAL"
|
||||
]
|
||||
type = "include"
|
||||
logic = "or"
|
||||
patterns = ["ERROR", "WARN"]
|
||||
```
|
||||
|
||||
### Exclude Filter
|
||||
|
||||
Entries matching patterns are dropped.
|
||||
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = [
|
||||
"DEBUG",
|
||||
"TRACE",
|
||||
"health-check"
|
||||
]
|
||||
```
|
||||
|
||||
## Configuration Options
|
||||
## Options
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `type` | string | Required | Filter type (include/exclude) |
|
||||
| `logic` | string | "or" | Pattern matching logic (or/and) |
|
||||
| `patterns` | []string | Required | Pattern list |
|
||||
| `type` | string | `include` | `include` (only matches pass) or `exclude` (matches are dropped) |
|
||||
| `logic` | string | `or` | `or` (any pattern matches) or `and` (every pattern matches) |
|
||||
| `patterns` | []string | `[]` | Go RE2 regular expressions |
|
||||
|
||||
A filter with no patterns passes everything. Invalid patterns fail at startup
|
||||
with the filter index and the offending pattern in the message.
|
||||
|
||||
## What Gets Matched
|
||||
|
||||
Patterns are matched against a single string assembled from the entry:
|
||||
|
||||
```
|
||||
"<source> <level> <message>"
|
||||
```
|
||||
|
||||
Empty parts are omitted, so an entry with no detected level matches
|
||||
`"<source> <message>"`. This means a pattern can target the source name or the
|
||||
level as easily as the message body:
|
||||
|
||||
| Pattern | Matches |
|
||||
|---------|---------|
|
||||
| `"^app\\.log "` | Entries whose source is `app.log` |
|
||||
| `"ERROR"` | Level `ERROR`, or the word `ERROR` anywhere in the message |
|
||||
|
||||
The structured `fields` payload is **not** part of the match text.
|
||||
|
||||
For entries that arrived over a chain link, the `source` used here is the bare
|
||||
source — the `node/source` prefix is applied later, by the formatter — so
|
||||
filtering by originating node requires matching on the message, or filtering on
|
||||
the node that produces the entries.
|
||||
|
||||
## Filter Types
|
||||
|
||||
### include
|
||||
|
||||
Only matching entries pass. Everything else is dropped.
|
||||
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["ERROR", "WARN", "FATAL"]
|
||||
```
|
||||
|
||||
### exclude
|
||||
|
||||
Matching entries are dropped. Everything else passes.
|
||||
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = ["/healthz", "TRACE"]
|
||||
```
|
||||
|
||||
## Logic
|
||||
|
||||
### or (default)
|
||||
|
||||
```toml
|
||||
logic = "or"
|
||||
patterns = ["ERROR", "WARN"]
|
||||
# passes: "ERROR in module" "WARN: low memory"
|
||||
# blocks: "INFO: started"
|
||||
```
|
||||
|
||||
### and
|
||||
|
||||
```toml
|
||||
logic = "and"
|
||||
patterns = ["database", "ERROR"]
|
||||
# passes: "ERROR: database connection failed"
|
||||
# blocks: "ERROR: file not found"
|
||||
```
|
||||
|
||||
With `logic = "and"` on an `exclude` filter, an entry is dropped only when it
|
||||
matches *every* pattern.
|
||||
|
||||
## Filter Chains
|
||||
|
||||
Filters are evaluated in declaration order and an entry must survive all of
|
||||
them. The first filter to reject an entry ends its life; later filters never see
|
||||
it.
|
||||
|
||||
```toml
|
||||
# 1. keep only production traffic
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["prod-", "production"]
|
||||
|
||||
# 2. of that, keep only failures
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["ERROR", "EXCEPTION", "FATAL"]
|
||||
|
||||
# 3. minus known noise
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = ["ECONNRESET", "broken pipe"]
|
||||
```
|
||||
|
||||
Order matters for cost, not for correctness: put the most selective filter first
|
||||
so later ones evaluate fewer entries.
|
||||
|
||||
## Pattern Syntax
|
||||
|
||||
Patterns support regular expression syntax:
|
||||
Go's RE2 syntax. No backreferences and no lookaround — RE2 guarantees linear
|
||||
time, which is exactly what you want in a log hot path.
|
||||
|
||||
### Basic Patterns
|
||||
- **Literal match**: `"ERROR"` - matches "ERROR" anywhere
|
||||
- **Case-insensitive**: `"(?i)error"` - matches "error", "ERROR", "Error"
|
||||
- **Word boundary**: `"\\berror\\b"` - matches whole word only
|
||||
| Need | Pattern |
|
||||
|------|---------|
|
||||
| Literal substring | `ERROR` |
|
||||
| Case-insensitive | `(?i)error` |
|
||||
| Whole word | `\\berror\\b` |
|
||||
| Alternation | `ERROR\|WARN\|FATAL` |
|
||||
| Character class | `[0-9]{3}` |
|
||||
| Anchors | `^ERROR`, `ERROR$` |
|
||||
| Any characters | `.*exception.*` |
|
||||
|
||||
### Advanced Patterns
|
||||
- **Alternation**: `"ERROR|WARN|FATAL"`
|
||||
- **Character classes**: `"[0-9]{3}"`
|
||||
- **Wildcards**: `".*exception.*"`
|
||||
- **Line anchors**: `"^ERROR"` (start), `"ERROR$"` (end)
|
||||
Remember that TOML basic strings process escapes, so a regex backslash needs
|
||||
doubling: `"\\berror\\b"`. TOML literal strings avoid the issue:
|
||||
`'\berror\b'`.
|
||||
|
||||
### Special Characters
|
||||
Escape special regex characters with backslash:
|
||||
- `.` → `\\.`
|
||||
- `*` → `\\*`
|
||||
- `[` → `\\[`
|
||||
- `(` → `\\(`
|
||||
Anchors apply to the assembled match text, which begins with the source name —
|
||||
so `^ERROR` will not match an entry whose source is non-empty. Use
|
||||
`\\bERROR\\b` instead unless you mean to anchor on the source.
|
||||
|
||||
## Filter Logic
|
||||
## Common Recipes
|
||||
|
||||
### OR Logic (default)
|
||||
Entry passes if ANY pattern matches:
|
||||
```toml
|
||||
logic = "or"
|
||||
patterns = ["ERROR", "WARN"]
|
||||
# Passes: "ERROR in module", "WARN: low memory"
|
||||
# Blocks: "INFO: started"
|
||||
```
|
||||
|
||||
### AND Logic
|
||||
Entry passes only if ALL patterns match:
|
||||
```toml
|
||||
logic = "and"
|
||||
patterns = ["database", "ERROR"]
|
||||
# Passes: "ERROR: database connection failed"
|
||||
# Blocks: "ERROR: file not found"
|
||||
```
|
||||
|
||||
## Filter Chain
|
||||
|
||||
Multiple filters execute sequentially:
|
||||
**Severity floor**
|
||||
|
||||
```toml
|
||||
# First filter: Include errors and warnings
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["ERROR", "WARN"]
|
||||
|
||||
# Second filter: Exclude test environments
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = ["test-env", "staging"]
|
||||
type = "include"
|
||||
patterns = ["ERROR", "FATAL", "CRITICAL"]
|
||||
```
|
||||
|
||||
Processing order:
|
||||
1. Entry arrives from source
|
||||
2. Include filter evaluates
|
||||
3. If passed, exclude filter evaluates
|
||||
4. If passed all filters, entry continues to sink
|
||||
**Noise reduction**
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Pattern Compilation
|
||||
- Patterns compile once at startup
|
||||
- Invalid patterns cause startup failure
|
||||
- Complex patterns may impact performance
|
||||
|
||||
### Optimization Tips
|
||||
- Place most selective filters first
|
||||
- Use simple patterns when possible
|
||||
- Combine related patterns with alternation
|
||||
- Avoid excessive wildcards (`.*`)
|
||||
|
||||
## Filter Statistics
|
||||
|
||||
Filters track:
|
||||
- Total entries evaluated
|
||||
- Entries passed
|
||||
- Entries blocked
|
||||
- Processing time per pattern
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### Log Level Filtering
|
||||
```toml
|
||||
[[pipelines.filters]]
|
||||
type = "include"
|
||||
patterns = ["ERROR", "WARN", "FATAL", "CRITICAL"]
|
||||
```
|
||||
|
||||
### Application Filtering
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["app1", "app2", "app3"]
|
||||
type = "exclude"
|
||||
patterns = ["/healthz", "/metrics", "\\bping\\b"]
|
||||
```
|
||||
|
||||
### Noise Reduction
|
||||
**Secret suppression** — see [Security](security.md); filters are the only
|
||||
redaction mechanism LogWisp currently offers.
|
||||
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = [
|
||||
"health-check",
|
||||
"ping",
|
||||
"/metrics",
|
||||
"heartbeat"
|
||||
]
|
||||
type = "exclude"
|
||||
patterns = ["password", "api[_-]?key", "authorization", "bearer ", "secret", "token"]
|
||||
```
|
||||
|
||||
### Security Filtering
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = [
|
||||
"password",
|
||||
"token",
|
||||
"api[_-]key",
|
||||
"secret"
|
||||
]
|
||||
```
|
||||
Note this drops the whole entry, it does not redact part of it.
|
||||
|
||||
### Multi-stage Filtering
|
||||
```toml
|
||||
# Include production logs
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["prod-", "production"]
|
||||
**Per-application routing** — run one pipeline per application, each with its
|
||||
own include filter, rather than trying to route inside one pipeline. Sinks fan
|
||||
out to *all* sinks in a pipeline; there is no conditional routing.
|
||||
|
||||
# Include only errors
|
||||
[[pipelines.flow.filters]]
|
||||
type = "include"
|
||||
patterns = ["ERROR", "EXCEPTION", "FATAL"]
|
||||
## Statistics
|
||||
|
||||
# Exclude known issues
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = ["ECONNRESET", "broken pipe"]
|
||||
```
|
||||
Each filter reports `type`, `logic`, `pattern_count`, `total_processed`,
|
||||
`total_matched`, and `total_dropped`. The chain reports `filter_count`,
|
||||
`total_processed`, and `total_passed`; the pipeline derives
|
||||
`total_filtered` as the difference.
|
||||
|
||||
## Performance
|
||||
|
||||
Patterns compile once at startup. Every entry that reaches the filter stage is
|
||||
evaluated against every filter until one rejects it, so cost scales with the
|
||||
number of patterns and their complexity. Prefer literal substrings and simple
|
||||
alternations over broad `.*` wildcards.
|
||||
|
||||
Filters log at DEBUG on every entry — pattern text, match results, and the
|
||||
final decision. That is invaluable when a filter is not behaving as expected and
|
||||
very expensive in production; keep `logging.level` at `info` or higher on a busy
|
||||
pipeline.
|
||||
|
||||
+138
-143
@@ -1,180 +1,175 @@
|
||||
# Formatters
|
||||
|
||||
LogWisp formatters transform log entries before output to sinks.
|
||||
The formatter is the last flow stage. It turns a `core.LogEntry` into the byte
|
||||
payload that sinks write, applying a sanitizer policy on the way.
|
||||
|
||||
## Formatter Types
|
||||
```toml
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
sanitizer_policy = "json"
|
||||
flags = 0
|
||||
timestamp_format = ""
|
||||
```
|
||||
|
||||
### Raw Formatter
|
||||
One formatter serves the whole pipeline. Sinks receive an identical payload;
|
||||
there is no per-sink formatting. When you need two shapes of the same data, run
|
||||
two pipelines, or chain to a node that formats differently.
|
||||
|
||||
Outputs the log message as-is with optional newline.
|
||||
Omitting `[pipelines.flow.format]` entirely selects `raw`.
|
||||
|
||||
## Options
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `type` | string | `raw` | `raw`, `txt` (alias `text`), or `json` |
|
||||
| `sanitizer_policy` | string | derived from `type` | `raw`, `txt`, `json`, or `shell` |
|
||||
| `flags` | int64 | `0` | Bitmask override; `0` selects a per-type default |
|
||||
| `timestamp_format` | string | formatter default | Go reference layout, e.g. `"2006-01-02T15:04:05Z07:00"` |
|
||||
|
||||
## Types
|
||||
|
||||
### raw
|
||||
|
||||
Passthrough. `FlagRaw` bypasses both formatting and sanitization, so the
|
||||
message reaches the sink exactly as the source produced it.
|
||||
|
||||
```toml
|
||||
[pipelines.flow.format]
|
||||
type = "raw"
|
||||
sanitizer_policy = "raw"
|
||||
flags = 1
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
Fastest option, and the right one when you are relaying text that is already in
|
||||
its final form. Note that it also bypasses sanitization, so control characters
|
||||
in the source data reach your sinks intact.
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `add_new_line` | bool | true | Append newline to messages |
|
||||
| `type` | string | "raw" | raw, json, or txt |
|
||||
| `flags` | int64 | 0 | log/formatter flags override |
|
||||
| `sanitizer_policy` | string | | Sanitizer policy (e.g. "json", "raw", "txt", "shell") |
|
||||
### txt
|
||||
|
||||
### JSON Formatter
|
||||
|
||||
Produces structured JSON output.
|
||||
Human-readable line output with a timestamp and level.
|
||||
|
||||
```toml
|
||||
[pipelines.format]
|
||||
type = "json"
|
||||
[pipelines.flow.format]
|
||||
type = "txt"
|
||||
sanitizer_policy = "txt"
|
||||
timestamp_format = "2006-01-02 15:04:05"
|
||||
```
|
||||
|
||||
+[pipelines.flow.format]
|
||||
type = "json"
|
||||
### json
|
||||
|
||||
Structured output, the natural choice for downstream ingestion.
|
||||
|
||||
```toml
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
sanitizer_policy = "json"
|
||||
```
|
||||
|
||||
**Output Structure:**
|
||||
Output has the shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-01-01T12:00:00Z",
|
||||
"level": "ERROR",
|
||||
"source": "app",
|
||||
"message": "Connection failed"
|
||||
}
|
||||
{"time":"2026-01-02T15:04:05.123Z","level":"ERROR","trace":"edge-01/app.log","fields":["connection refused"]}
|
||||
```
|
||||
|
||||
### Text Formatter
|
||||
The exact key names and structure come from the `lixenwraith/log` formatter, not
|
||||
from LogWisp; they are stable for a given dependency version but are not part of
|
||||
LogWisp's own configuration surface.
|
||||
|
||||
Template-based text formatting.
|
||||
## Flags
|
||||
|
||||
```toml
|
||||
[pipelines.flow.format]
|
||||
type = "txt"
|
||||
sanitizer_policy = "txt"
|
||||
timestamp_format = "2006-01-02T15:04:05.000Z07:00"
|
||||
`flags` is a bitmask passed to the underlying formatter. Leave it at `0` unless
|
||||
you need to override the defaults.
|
||||
|
||||
| Value | Name | Effect |
|
||||
|-------|------|--------|
|
||||
| `1` | Raw | Bypass formatting and sanitization entirely |
|
||||
| `2` | ShowTimestamp | Emit the timestamp |
|
||||
| `4` | ShowLevel | Emit the level |
|
||||
| `8` | StructuredJSON | Render attached fields as a JSON object |
|
||||
| `16` | NoTimestamp | Suppress the timestamp |
|
||||
| `32` | NoLevel | Suppress the level |
|
||||
|
||||
With `flags = 0` the formatter selects `1` for `type = "raw"` and `6`
|
||||
(timestamp + level) for every other type. `8` is added automatically whenever an
|
||||
entry carries parseable `fields`.
|
||||
|
||||
Examples: `flags = 4` for level only, no timestamp; `flags = 2` for timestamp
|
||||
only, no level.
|
||||
|
||||
## Sanitizer Policies
|
||||
|
||||
The sanitizer runs before serialization and neutralizes control characters that
|
||||
would otherwise break framing or reach a terminal.
|
||||
|
||||
| Policy | Behaviour | Use with |
|
||||
|--------|-----------|----------|
|
||||
| `raw` | No-op passthrough | `type = "raw"` where you control the data |
|
||||
| `txt` | Escapes non-printable characters | File and console sinks |
|
||||
| `json` | Escapes control characters for safe JSON embedding | `type = "json"`, chain links |
|
||||
| `shell` | Strips shell metacharacters, whitespace, and control characters | Data that will be passed to a command |
|
||||
|
||||
When `sanitizer_policy` is omitted, the policy is derived from `type`: `json`
|
||||
for `json`, `txt` for `txt`/`text`, and `raw` for anything else — so the safe
|
||||
pairing is the default.
|
||||
|
||||
> `shell` strips dangerous characters but is **not** sufficient to make a string
|
||||
> safe for shell construction. Pass arguments through `exec` argv instead of
|
||||
> building command lines.
|
||||
|
||||
To see a policy working, point a pipeline at the `random` source with
|
||||
`special = true`, which injects control bytes and multi-byte Unicode into every
|
||||
message.
|
||||
|
||||
## Node Identity in Output
|
||||
|
||||
Entries that arrived over a chain link carry a `Node` label. The formatter
|
||||
renders it as a syslog-style prefix on the source field:
|
||||
|
||||
```
|
||||
edge-01/app.log
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
Entries with no node label show the bare source. Node identity therefore appears
|
||||
*inside* the source field rather than as a separate output key — worth knowing
|
||||
when writing downstream parsers or grep patterns.
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `timestamp_format` | string | "" | Time format override |
|
||||
## Structured Fields
|
||||
|
||||
**Default Template:**
|
||||
```
|
||||
[{{.Timestamp | FmtTime}}] [{{.Level | ToUpper}}] {{.Source}} - {{.Message}}{{ if .Fields }} {{.Fields}}{{ end }}
|
||||
```
|
||||
When an entry carries `Fields` (raw JSON), the formatter parses it and switches
|
||||
to structured rendering by adding the `StructuredJSON` flag automatically.
|
||||
Fields reach a pipeline in two ways: from the `file` source when a tailed line
|
||||
parses as JSON with a `fields` key, and from the heartbeat generator when
|
||||
`include_stats = true`.
|
||||
|
||||
## Template Functions
|
||||
## Choosing a Configuration
|
||||
|
||||
Available functions in text templates:
|
||||
| Goal | Configuration |
|
||||
|------|---------------|
|
||||
| Maximum throughput, data already formatted | `type = "raw"` |
|
||||
| Human reading in a terminal or file | `type = "txt"`, `sanitizer_policy = "txt"` |
|
||||
| Downstream ingestion (Loki, Elasticsearch, jq) | `type = "json"`, `sanitizer_policy = "json"` |
|
||||
| Compact console output | `type = "txt"`, `flags = 4` |
|
||||
| Untrusted log content | never `raw`; pick `txt` or `json` and set the matching policy |
|
||||
|
||||
| Function | Description | Example |
|
||||
|----------|-------------|---------|
|
||||
| `FmtTime` | Format timestamp | `{{.Timestamp \| FmtTime}}` |
|
||||
| `ToUpper` | Convert to uppercase | `{{.Level \| ToUpper}}` |
|
||||
| `ToLower` | Convert to lowercase | `{{.Source \| ToLower}}` |
|
||||
| `TrimSpace` | Remove whitespace | `{{.Message \| TrimSpace}}` |
|
||||
## Formatting and Chain Links
|
||||
|
||||
## Template Variables
|
||||
Chain sinks (`tcp_chain`, `http_chain`) do **not** ship the formatted payload.
|
||||
They re-serialize the structured entry into the canonical chain encoding, which
|
||||
makes them independent of the local formatter.
|
||||
|
||||
Available variables in templates:
|
||||
The practical consequence: setting `flow.format` on an edge node changes only
|
||||
that node's own local sinks. The output shape seen by a human or a downstream
|
||||
system is decided on the node that owns the sink they read.
|
||||
|
||||
| Variable | Type | Description |
|
||||
|----------|------|-------------|
|
||||
| `.Timestamp` | time.Time | Entry timestamp |
|
||||
| `.Level` | string | Log level |
|
||||
| `.Source` | string | Source identifier |
|
||||
| `.Message` | string | Log message |
|
||||
| `.Fields` | string | Additional fields (JSON) |
|
||||
If an event ever reaches a chain sink without a structured entry, the sink wraps
|
||||
the formatted payload into a synthetic entry and counts it in `synthesized`.
|
||||
A non-zero `synthesized` count means something upstream lost structure.
|
||||
|
||||
## Time Format Strings
|
||||
## Performance
|
||||
|
||||
Common Go time format patterns:
|
||||
Relative cost, cheapest first: `raw` (passthrough) → `txt` (line assembly) →
|
||||
`json` (serialization). Sanitization adds a scan of the message; the `raw`
|
||||
policy skips it.
|
||||
|
||||
| Pattern | Example Output |
|
||||
|---------|---------------|
|
||||
| `2006-01-02T15:04:05Z07:00` | 2024-01-02T15:04:05Z |
|
||||
| `2006-01-02 15:04:05` | 2024-01-02 15:04:05 |
|
||||
| `Jan 2 15:04:05` | Jan 2 15:04:05 |
|
||||
| `15:04:05.000` | 15:04:05.123 |
|
||||
| `2006/01/02` | 2024/01/02 |
|
||||
|
||||
## Format Selection
|
||||
|
||||
### Default Behavior
|
||||
|
||||
If no formatter specified:
|
||||
- **HTTP/TCP sinks**: JSON format
|
||||
- **Console/File sinks**: Raw format
|
||||
- **Client sinks**: JSON format
|
||||
|
||||
### Per-Pipeline Configuration
|
||||
|
||||
Each pipeline can have its own formatter:
|
||||
|
||||
```toml
|
||||
[[pipelines]]
|
||||
name = "json-pipeline"
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
|
||||
[[pipelines]]
|
||||
name = "text-pipeline"
|
||||
[pipelines.flow.format]
|
||||
type = "txt"
|
||||
```
|
||||
|
||||
## Message Processing
|
||||
|
||||
### JSON Message Handling
|
||||
|
||||
When using JSON formatter with JSON log messages:
|
||||
1. Attempts to parse message as JSON
|
||||
2. Merges fields with LogWisp metadata
|
||||
3. LogWisp fields take precedence
|
||||
4. Falls back to string if parsing fails
|
||||
|
||||
### Field Preservation
|
||||
|
||||
LogWisp metadata always includes:
|
||||
- Timestamp (from source or current time)
|
||||
- Level (detected or default)
|
||||
- Source (origin identifier)
|
||||
- Message (original content)
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Formatter Performance
|
||||
|
||||
Relative performance (fastest to slowest):
|
||||
1. **Raw**: Direct passthrough
|
||||
2. **Text**: Template execution
|
||||
3. **JSON**: Serialization
|
||||
4. **JSON (pretty)**: Formatted serialization
|
||||
|
||||
### Optimization Tips
|
||||
|
||||
- Use raw format for high throughput
|
||||
- Cache template compilation (automatic)
|
||||
- Minimize template complexity
|
||||
- Avoid pretty JSON in production
|
||||
|
||||
## Common Configurations
|
||||
|
||||
### Structured Logging
|
||||
```toml
|
||||
[pipelines.flow.format]
|
||||
type = "json"
|
||||
```
|
||||
|
||||
### Human-Readable Logs
|
||||
```toml
|
||||
[pipelines.flow.format]
|
||||
type = "txt"
|
||||
timestamp_format = "15:04:05"
|
||||
```
|
||||
The formatter holds a mutex because the underlying implementation reuses an
|
||||
internal buffer and is not goroutine-safe. It is the only shared serialization
|
||||
point in the hot path, and the reason a single pipeline formats entries one at
|
||||
a time.
|
||||
|
||||
+113
-62
@@ -1,49 +1,62 @@
|
||||
# Installation Guide
|
||||
|
||||
LogWisp installation and service configuration for Linux and FreeBSD systems.
|
||||
## Requirements
|
||||
|
||||
## Installation Methods
|
||||
- **Operating systems**: Linux (kernel 6.10+), FreeBSD (14.0+)
|
||||
- **Architecture**: amd64
|
||||
- **Go**: 1.26 or newer, to build from source
|
||||
|
||||
### Pre-built Binaries
|
||||
|
||||
Download the latest release binary for your platform and install to `/usr/local/bin`:
|
||||
## Building from Source
|
||||
|
||||
```bash
|
||||
# Linux amd64
|
||||
wget https://github.com/yourusername/logwisp/releases/latest/download/logwisp-linux-amd64
|
||||
chmod +x logwisp-linux-amd64
|
||||
sudo mv logwisp-linux-amd64 /usr/local/bin/logwisp
|
||||
|
||||
# FreeBSD amd64
|
||||
fetch https://github.com/yourusername/logwisp/releases/latest/download/logwisp-freebsd-amd64
|
||||
chmod +x logwisp-freebsd-amd64
|
||||
sudo mv logwisp-freebsd-amd64 /usr/local/bin/logwisp
|
||||
```
|
||||
|
||||
### Building from Source
|
||||
|
||||
Requires Go 1.24 or newer:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/yourusername/logwisp.git
|
||||
git clone https://github.com/lixenwraith/logwisp.git
|
||||
cd logwisp
|
||||
go build -o logwisp ./src/cmd/logwisp
|
||||
sudo install -m 755 logwisp /usr/local/bin/
|
||||
make
|
||||
sudo make install # installs to $PREFIX/bin, default /usr/local/bin
|
||||
```
|
||||
|
||||
### Go Install Method
|
||||
The Makefile works with both GNU make and BSD make. Targets:
|
||||
|
||||
Install directly using Go (version information will not be embedded):
|
||||
| Target | Effect |
|
||||
|--------|--------|
|
||||
| `make` / `make build` | Build `bin/logwisp` with version metadata |
|
||||
| `make dev` | Build with the race detector enabled |
|
||||
| `make install` | Install the binary to `$(PREFIX)/bin` (default `/usr/local`) |
|
||||
| `make uninstall` | Intended to remove the installed binary — currently broken: it expands to `$(BINDIR)/bin/logwisp` instead of `$(BINDIR)/logwisp`, so it removes nothing. Delete the binary by hand |
|
||||
| `make clean` | Remove the built binary |
|
||||
| `make version` | Print the version, commit, and build time that would be embedded |
|
||||
|
||||
Version, commit hash, and build time are injected via `-ldflags` from `git
|
||||
describe` and `git rev-parse`. A plain `go build` produces a working binary that
|
||||
reports `dev` for all three:
|
||||
|
||||
```bash
|
||||
go install github.com/yourusername/logwisp/cmd/logwisp@latest
|
||||
go build -o bin/logwisp ./cmd/logwisp
|
||||
```
|
||||
|
||||
## Service Configuration
|
||||
`go install github.com/lixenwraith/logwisp/cmd/logwisp@latest` also works, with
|
||||
the same loss of version metadata.
|
||||
|
||||
## Configuration
|
||||
|
||||
Copy the annotated reference configuration and edit it:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /etc/logwisp
|
||||
sudo cp config/logwisp.toml /etc/logwisp/logwisp.toml
|
||||
```
|
||||
|
||||
LogWisp searches, in order: `-c <path>`, `--config=<path>`,
|
||||
`$LOGWISP_CONFIG_DIR`/`$LOGWISP_CONFIG_FILE`, `~/.config/logwisp/logwisp.toml`,
|
||||
`./logwisp.toml`. See [Configuration](configuration.md).
|
||||
|
||||
## Running as a Service
|
||||
|
||||
LogWisp has no daemon mode; run it in the foreground under a supervisor.
|
||||
|
||||
### Linux (systemd)
|
||||
|
||||
Create systemd service file `/etc/systemd/system/logwisp.service`:
|
||||
`/etc/systemd/system/logwisp.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
@@ -55,30 +68,47 @@ Type=simple
|
||||
User=logwisp
|
||||
Group=logwisp
|
||||
ExecStart=/usr/local/bin/logwisp -c /etc/logwisp/logwisp.toml
|
||||
ExecReload=/bin/kill -HUP $MAINPID
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
WorkingDirectory=/var/lib/logwisp
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
WorkingDirectory=/var/lib/logwisp
|
||||
|
||||
# Hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/log/logwisp /var/lib/logwisp
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Setup service user and directories:
|
||||
`ExecReload` gives you `systemctl reload logwisp` for configuration and
|
||||
certificate rotation without dropping the process.
|
||||
|
||||
If a pipeline binds a port below 1024, add
|
||||
`AmbientCapabilities=CAP_NET_BIND_SERVICE` rather than running as root.
|
||||
|
||||
Setup:
|
||||
|
||||
```bash
|
||||
sudo useradd -r -s /bin/false logwisp
|
||||
sudo useradd -r -s /usr/sbin/nologin logwisp
|
||||
sudo mkdir -p /etc/logwisp /var/lib/logwisp /var/log/logwisp
|
||||
sudo chown logwisp:logwisp /var/lib/logwisp /var/log/logwisp
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable logwisp
|
||||
sudo systemctl start logwisp
|
||||
sudo systemctl enable --now logwisp
|
||||
```
|
||||
|
||||
The service account needs **read** access to every directory a `file` source
|
||||
watches and **write** access to every directory a `file` sink or
|
||||
`logging.file` writes to.
|
||||
|
||||
### FreeBSD (rc.d)
|
||||
|
||||
Create rc script `/usr/local/etc/rc.d/logwisp`:
|
||||
`/usr/local/etc/rc.d/logwisp`:
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
@@ -92,8 +122,9 @@ Create rc script `/usr/local/etc/rc.d/logwisp`:
|
||||
name="logwisp"
|
||||
rcvar="${name}_enable"
|
||||
pidfile="/var/run/${name}.pid"
|
||||
command="/usr/local/bin/logwisp"
|
||||
command_args="-c /usr/local/etc/logwisp/logwisp.toml"
|
||||
procname="/usr/local/bin/logwisp"
|
||||
command="/usr/sbin/daemon"
|
||||
command_args="-p ${pidfile} -f ${procname} -c /usr/local/etc/logwisp/logwisp.toml"
|
||||
|
||||
load_rc_config $name
|
||||
: ${logwisp_enable:="NO"}
|
||||
@@ -101,7 +132,7 @@ load_rc_config $name
|
||||
run_rc_command "$1"
|
||||
```
|
||||
|
||||
Setup service:
|
||||
Setup:
|
||||
|
||||
```bash
|
||||
sudo chmod +x /usr/local/etc/rc.d/logwisp
|
||||
@@ -112,45 +143,66 @@ sudo sysrc logwisp_enable="YES"
|
||||
sudo service logwisp start
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
Standard installation directories:
|
||||
## Directory Layout
|
||||
|
||||
| Purpose | Linux | FreeBSD |
|
||||
|---------|-------|---------|
|
||||
| Binary | `/usr/local/bin/logwisp` | `/usr/local/bin/logwisp` |
|
||||
| Configuration | `/etc/logwisp/` | `/usr/local/etc/logwisp/` |
|
||||
| Working Directory | `/var/lib/logwisp/` | `/var/db/logwisp/` |
|
||||
| Log Files | `/var/log/logwisp/` | `/var/log/logwisp/` |
|
||||
| PID File | `/var/run/logwisp.pid` | `/var/run/logwisp.pid` |
|
||||
| TLS material | `/etc/logwisp/tls/` | `/usr/local/etc/logwisp/tls/` |
|
||||
| Working directory | `/var/lib/logwisp/` | `/var/db/logwisp/` |
|
||||
| Application logs | `/var/log/logwisp/` | `/var/log/logwisp/` |
|
||||
|
||||
## Post-Installation Verification
|
||||
Key files should be mode `0600` and owned by the service account.
|
||||
|
||||
Verify the installation:
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Check version
|
||||
logwisp version
|
||||
logwisp --version
|
||||
|
||||
# Test configuration
|
||||
logwisp -c /etc/logwisp/logwisp.toml --disable-status-reporter
|
||||
# start in the foreground with debug logging and watch pipelines come up
|
||||
logwisp -c /etc/logwisp/logwisp.toml --logging.level=debug --logging.output=stderr
|
||||
|
||||
# Check service status (Linux)
|
||||
sudo systemctl status logwisp
|
||||
|
||||
# Check service status (FreeBSD)
|
||||
sudo service logwisp status
|
||||
sudo systemctl status logwisp # Linux
|
||||
sudo service logwisp status # FreeBSD
|
||||
```
|
||||
|
||||
## Uninstallation
|
||||
Expect `Created source instance`, `Created sink instance`, and
|
||||
`Starting pipeline` for each configured pipeline. There is no validate-only
|
||||
mode; see [Operations](operations.md#checking-a-configuration).
|
||||
|
||||
## Test Scripts
|
||||
|
||||
Two end-to-end scripts under `test/` build multi-node chain topologies against a
|
||||
local build:
|
||||
|
||||
```bash
|
||||
make
|
||||
./test/chain-test.sh --auto # two independent relay pipelines
|
||||
./test/chain-aggregate-test.sh --auto # fan-in: both edges into one pipeline
|
||||
```
|
||||
|
||||
Without `--auto` they run the relay in the foreground for interactive
|
||||
inspection. They need bash 5+, coreutils, and curl, and they bind ports
|
||||
15801–15804. Generated configuration and logs land in `test/run/`.
|
||||
|
||||
> Two of the three `--auto` assertions currently report `FAIL` against a
|
||||
> working build. They grep the sink output for `"source":"edge-tcp/` and
|
||||
> `"node":"edge-http"`, but the JSON formatter emits the `node/source` label
|
||||
> under the key `trace`. The transport itself is healthy — the
|
||||
> `total_processed` assertion passes and the streamed entries carry
|
||||
> `"trace":"edge-tcp/random_rand"` as expected. Until the assertions are
|
||||
> updated, verify the streams by eye with `nc 127.0.0.1 15803` and
|
||||
> `curl -sN http://127.0.0.1:15804/stream`.
|
||||
|
||||
## Uninstall
|
||||
|
||||
### Linux
|
||||
|
||||
```bash
|
||||
sudo systemctl stop logwisp
|
||||
sudo systemctl disable logwisp
|
||||
sudo rm /usr/local/bin/logwisp
|
||||
sudo rm /etc/systemd/system/logwisp.service
|
||||
sudo systemctl disable --now logwisp
|
||||
sudo rm /usr/local/bin/logwisp /etc/systemd/system/logwisp.service
|
||||
sudo systemctl daemon-reload
|
||||
sudo rm -rf /etc/logwisp /var/lib/logwisp /var/log/logwisp
|
||||
sudo userdel logwisp
|
||||
```
|
||||
@@ -160,8 +212,7 @@ sudo userdel logwisp
|
||||
```bash
|
||||
sudo service logwisp stop
|
||||
sudo sysrc -x logwisp_enable
|
||||
sudo rm /usr/local/bin/logwisp
|
||||
sudo rm /usr/local/etc/rc.d/logwisp
|
||||
sudo rm /usr/local/bin/logwisp /usr/local/etc/rc.d/logwisp
|
||||
sudo rm -rf /usr/local/etc/logwisp /var/db/logwisp /var/log/logwisp
|
||||
sudo pw userdel logwisp
|
||||
```
|
||||
@@ -0,0 +1,368 @@
|
||||
# Implementation Plan: mTLS as Authentication
|
||||
|
||||
**Status:** proposal, not implemented.
|
||||
**Scope:** turn the existing transport-level mutual TLS into a real
|
||||
authentication and authorization mechanism.
|
||||
|
||||
## Problem
|
||||
|
||||
LogWisp already does mutual TLS at the transport layer. A listener with
|
||||
`client_auth = true` refuses any peer that cannot present a certificate
|
||||
chaining to `client_ca_file`, and `internal/tlsx` already extracts the peer's
|
||||
Common Name and stashes it in session metadata as `tls_peer_cn`.
|
||||
|
||||
Nothing reads it back. The result is a CA-wide membership check with no notion
|
||||
of *which* peer connected:
|
||||
|
||||
1. **No per-identity authorization.** Every certificate the CA issues is
|
||||
equivalent. There is no way to say "only `edge-01` and `edge-02` may write to
|
||||
this ingest port", so one CA cannot serve several trust domains, and
|
||||
withdrawing one peer means rotating the CA bundle for all of them.
|
||||
2. **Node labels are unauthenticated.** With `trust_node = true` (the default) a
|
||||
peer declares its own `node` label in the chain hello or the
|
||||
`X-Logwisp-Node` header. Any certificate holder can claim any label,
|
||||
including another host's, and every downstream consumer will attribute those
|
||||
entries accordingly. The only current defence, `trust_node = false`, replaces
|
||||
the label with a remote address — unforgeable, but useless for identifying a
|
||||
host behind NAT or a load balancer.
|
||||
3. **The `http` sink has no authentication at all**, even with TLS on. Its
|
||||
stream and status endpoints are readable by anyone who can reach the port.
|
||||
|
||||
Password, token, and SCRAM authentication were removed during the plugin/flow
|
||||
restructure and the move to standard-library networking. Certificates are the
|
||||
one credential the current transport already carries, which makes mTLS the
|
||||
cheapest path back to authenticated peers.
|
||||
|
||||
## Goals
|
||||
|
||||
- Authorize peers by certificate identity, per listener.
|
||||
- Bind the chain `node` label to the authenticated identity, so origin
|
||||
attribution is trustworthy.
|
||||
- Gate the `http` sink's endpoints on client certificates.
|
||||
- Make identity visible in sessions, statistics, and logs.
|
||||
- Change nothing for existing configurations that omit the new block.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Reviving password, token, or SCRAM authentication. The reserved hooks
|
||||
(`chain.Hello.Auth`, the `Authorization` header comment in the `http_chain`
|
||||
sink, the "Future: password auth block" comments in the network options
|
||||
structs) stay reserved.
|
||||
- IP allow/deny lists and per-peer rate limits. Related, but a separate feature
|
||||
with its own config surface.
|
||||
- OCSP. See [Revocation](#revocation) for what is proposed instead.
|
||||
- Authorization *within* a stream — the unit of decision is a connection (TCP)
|
||||
or a request (HTTP), never an individual entry.
|
||||
|
||||
## Design
|
||||
|
||||
### Identity
|
||||
|
||||
The authenticated identity is a single string derived from the peer's verified
|
||||
leaf certificate. Because `tls.RequireAndVerifyClientCert` has already validated
|
||||
the chain, signature, and validity window by the time we look, extraction is
|
||||
pure field selection.
|
||||
|
||||
| `identity` mode | Source | Notes |
|
||||
|-----------------|--------|-------|
|
||||
| `cn` (default) | `Subject.CommonName` | Matches the existing `tls_peer_cn` metadata |
|
||||
| `san_dns` | first `DNSNames` entry | Preferred for host identities |
|
||||
| `san_uri` | first `URIs` entry | SPIFFE-style IDs |
|
||||
| `san_email` | first `EmailAddresses` entry | Operator identities |
|
||||
|
||||
An empty identity is a rejection, not an empty match: a certificate with no
|
||||
usable identity field cannot satisfy any policy.
|
||||
|
||||
Identities are not secrets, so ordinary string comparison is fine; there is no
|
||||
timing side channel worth defending here.
|
||||
|
||||
### Configuration
|
||||
|
||||
A new `auth` table sits beside `tls` in every network plugin's `config`. Keeping
|
||||
it separate from `tls` matters: TLS answers "is this channel private and is the
|
||||
peer chained to a CA", auth answers "may *this* peer do *this*", and a later
|
||||
non-certificate method should be able to reuse the block.
|
||||
|
||||
```toml
|
||||
[pipelines.plugin_sources.config.auth]
|
||||
type = "mtls" # none (default) | mtls
|
||||
identity = "cn" # cn | san_dns | san_uri | san_email
|
||||
allow = ["edge-01", "edge-02"] # exact identities
|
||||
allow_patterns = ["^edge-\\d{2}$"] # RE2, anchored by the author
|
||||
node_binding = "force" # none | assert | force
|
||||
```
|
||||
|
||||
| Option | Type | Default | Meaning |
|
||||
|--------|------|---------|---------|
|
||||
| `type` | string | `none` | `none` preserves today's behaviour exactly; `mtls` enables the policy |
|
||||
| `identity` | string | `cn` | Which certificate field is the identity |
|
||||
| `allow` | []string | `[]` | Exact identity matches |
|
||||
| `allow_patterns` | []string | `[]` | RE2 patterns matched against the identity |
|
||||
| `node_binding` | string | `force` when `type = "mtls"` | See below |
|
||||
|
||||
Empty `allow` **and** empty `allow_patterns` under `type = "mtls"` means "any
|
||||
identity the CA vouches for" — that is, today's behaviour, but with the identity
|
||||
now recorded and node binding available. It is a deliberate, documented default
|
||||
rather than a silent deny-all, and startup logs say so plainly.
|
||||
|
||||
`node_binding` applies only to the chain sources, where a `node` label is
|
||||
declared:
|
||||
|
||||
| Value | Behaviour |
|
||||
|-------|-----------|
|
||||
| `none` | `trust_node` governs, as today |
|
||||
| `assert` | The declared label must equal the identity; a mismatch is rejected |
|
||||
| `force` | The declared label is ignored and the identity is used |
|
||||
|
||||
`force` is the default under `type = "mtls"` because it is the only setting
|
||||
where a misconfigured or hostile edge cannot mislabel its entries. `assert`
|
||||
exists for operators who want the mismatch to be loud rather than silently
|
||||
corrected. `node_binding` overrides `trust_node`; when both are set, the
|
||||
constructor logs that `trust_node` is being ignored.
|
||||
|
||||
For the `tcp` and `http` sinks, which have no node concept, `node_binding` is
|
||||
ignored.
|
||||
|
||||
Dialer-side plugins (`tcp_chain` and `http_chain` sinks) accept the same block
|
||||
to pin the *server's* identity beyond hostname verification. This is deferred to
|
||||
phase 4 and does nothing before then.
|
||||
|
||||
### Validation
|
||||
|
||||
At plugin construction, before anything binds:
|
||||
|
||||
- `type = "mtls"` on a listener requires `tls.enabled = true` and
|
||||
`tls.client_auth = true`. Silently accepting an auth policy the transport
|
||||
cannot enforce is the failure mode worth designing out.
|
||||
- `identity` must be one of the four modes.
|
||||
- Every entry in `allow_patterns` must compile.
|
||||
- `node_binding` must be one of the three values.
|
||||
|
||||
Errors follow existing style: `auth: type "mtls" requires tls.client_auth`.
|
||||
|
||||
### New package: `internal/authz`
|
||||
|
||||
```go
|
||||
package authz
|
||||
|
||||
// Policy is the compiled form of config.AuthOptions.
|
||||
type Policy struct { /* mode, identity selector, exact set, patterns, binding */ }
|
||||
|
||||
// New compiles a policy. Returns (nil, nil) when auth is disabled, matching
|
||||
// the tlsx.Server / tlsx.Client convention so callers can nil-check.
|
||||
func New(o *config.AuthOptions) (*Policy, error)
|
||||
|
||||
// Identity is the outcome of a successful authorization.
|
||||
type Identity struct {
|
||||
Name string // the selected certificate field
|
||||
Method string // "mtls"
|
||||
}
|
||||
|
||||
// Authorize extracts and checks the peer identity from a completed handshake.
|
||||
func (p *Policy) Authorize(cs *tls.ConnectionState) (Identity, error)
|
||||
|
||||
// ResolveNode applies node_binding to a declared label.
|
||||
func (p *Policy) ResolveNode(declared string, id Identity) (string, error)
|
||||
|
||||
// Stats reports counters for the sink/source stats map.
|
||||
func (p *Policy) Stats() map[string]any
|
||||
```
|
||||
|
||||
This mirrors `internal/tlsx`: one small package that is the single seam between
|
||||
declarative config and a cross-cutting concern, with `(nil, nil)` for the
|
||||
disabled case so every call site is a nil check rather than a branch on config.
|
||||
|
||||
### Enforcement Points
|
||||
|
||||
Each plugin already has the right spot, and in two cases the code says so.
|
||||
|
||||
**`tcp_chain` source** (`internal/source/tcpchain/tcpchain.go`, `handleConn`)
|
||||
|
||||
Today: handshake → read hello → decode → resolve node from `trust_node` →
|
||||
create session. Insert authorization between the handshake and the hello read,
|
||||
so an unauthorized peer never gets a preamble parsed on its behalf, and replace
|
||||
the node resolution with `Policy.ResolveNode`.
|
||||
|
||||
```go
|
||||
if tlsState != nil && s.authPolicy != nil {
|
||||
id, err := s.authPolicy.Authorize(*tlsState)
|
||||
if err != nil {
|
||||
s.authRejected.Add(1)
|
||||
s.logger.Warn("msg", "Connection rejected by auth policy",
|
||||
"component", "tcp_chain_source", "remote_addr", remote, "error", err)
|
||||
return // deferred cleanup closes conn
|
||||
}
|
||||
ident = id
|
||||
}
|
||||
// ... read and decode hello ...
|
||||
connNode, err = s.authPolicy.ResolveNode(hello.Node, ident)
|
||||
```
|
||||
|
||||
**`http_chain` source** (`internal/source/httpchain/httpchain.go`, `handleIngest`)
|
||||
|
||||
Per request, from `r.TLS`, before the body is read — an unauthorized sender
|
||||
should not get to stream 8 MiB into the process. Rejection is `403`, distinct
|
||||
from the `400` used for protocol errors, so a sender can tell "you are not
|
||||
allowed" from "your batch was malformed". `ResolveNode` then governs the
|
||||
`X-Logwisp-Node` header exactly as it governs the TCP hello.
|
||||
|
||||
**`tcp` sink** (`internal/sink/tcp/tcp.go`, `handleConn`)
|
||||
|
||||
There is already a comment marking the place: *"Password-auth extension point:
|
||||
preamble verification runs in handleConn post-handshake, pre-registration."*
|
||||
Authorization goes precisely there — after the explicit handshake, before the
|
||||
session is created and the client is registered — so an unauthorized peer never
|
||||
appears in the client map and never receives a broadcast.
|
||||
|
||||
**`http` sink** (`internal/sink/http/http.go`, `Start`)
|
||||
|
||||
Also already marked: *"Auth extension point: wrap mux with auth middleware once
|
||||
credentials land, e.g. handler = authMiddleware(cfg)(handler)."* A middleware
|
||||
around the mux covers both the stream and the status endpoint with one wrapper,
|
||||
and keeps the handlers themselves unaware of authorization.
|
||||
|
||||
```go
|
||||
var handler http.Handler = mux
|
||||
if h.authPolicy != nil {
|
||||
handler = authMiddleware(h.authPolicy, h.logger)(handler)
|
||||
}
|
||||
```
|
||||
|
||||
The middleware rejects with `403` and no body detail — the status endpoint
|
||||
leaks host, port, and throughput counters, so a rejection should not leak policy
|
||||
shape on top of it.
|
||||
|
||||
### Capabilities
|
||||
|
||||
`core.CapAuth` is currently derived from `tlsConfig.ClientAuth`. It should
|
||||
reflect the auth policy instead:
|
||||
|
||||
```go
|
||||
if s.authPolicy != nil {
|
||||
caps = append(caps, core.CapAuth)
|
||||
}
|
||||
```
|
||||
|
||||
`Pipeline.initSourceCapabilities` and `initSinkCapabilities` treat `CapAuth` as
|
||||
a no-op placeholder today. They become the natural place for a cross-cutting
|
||||
check: a plugin advertising `CapAuth` without `CapTLS` is a contradiction and
|
||||
should fail pipeline construction rather than start.
|
||||
|
||||
### Observability
|
||||
|
||||
Every authorization decision must be visible, because a silent deny is
|
||||
indistinguishable from a network fault at 3am.
|
||||
|
||||
- **Session metadata** gains `auth_method` and `auth_identity` alongside the
|
||||
existing `tls` and `tls_peer_cn`.
|
||||
- **Statistics** gain `auth_enabled`, `auth_rejected`, and `node_binding` in the
|
||||
`details` map of every affected source and sink, so rejections show up in the
|
||||
status reporter and the `http` sink's status endpoint.
|
||||
- **Logs** record a WARN per rejection with the remote address, the extracted
|
||||
identity (or the reason extraction failed), and the policy that rejected it.
|
||||
Accepted connections log the identity at INFO on the chain sources and at
|
||||
DEBUG on the sinks, matching each plugin's existing verbosity.
|
||||
|
||||
### Revocation
|
||||
|
||||
Certificate revocation is deliberately handled by the allow-list rather than by
|
||||
CRL or OCSP:
|
||||
|
||||
1. Remove the identity from `allow` / `allow_patterns`.
|
||||
2. `kill -HUP`.
|
||||
|
||||
The reload path already rebuilds every pipeline, so the policy takes effect on
|
||||
the next connection and existing connections are dropped by the rebuild itself.
|
||||
This is one moving part instead of three, it needs no network calls on the
|
||||
handshake path, and it is exact — no window between revocation and the next CRL
|
||||
publication.
|
||||
|
||||
A CRL file loaded next to `client_ca_file` and re-read on reload is a reasonable
|
||||
phase-4 addition for operators with existing CRL infrastructure. OCSP stapling
|
||||
is out of scope: it adds a network dependency to the handshake path for a
|
||||
system whose entire design is "never block".
|
||||
|
||||
## Phases
|
||||
|
||||
### Phase 1 — Identity and policy (no enforcement)
|
||||
|
||||
- `internal/config`: add `AuthOptions` plus an `Auth *AuthOptions` field to the
|
||||
four listener option structs.
|
||||
- `internal/tlsx`: add `PeerIdentity(cs tls.ConnectionState, mode string) string`
|
||||
beside the existing `PeerCN`.
|
||||
- `internal/authz`: new package — `Policy`, `New`, `Authorize`, `ResolveNode`,
|
||||
`Stats`.
|
||||
- Unit tests: identity extraction per mode, exact and pattern matching, empty
|
||||
policy, malformed patterns, node binding in all three modes.
|
||||
|
||||
Nothing behaves differently yet, which makes this phase safe to merge alone.
|
||||
|
||||
### Phase 2 — Chain sources
|
||||
|
||||
- Wire `authz` into `tcp_chain` and `http_chain` sources at the points above.
|
||||
- Replace the `trust_node` node resolution with `ResolveNode`.
|
||||
- Add validation, capabilities, statistics, and session metadata.
|
||||
- Add `test/mtls-chain-test.sh`, modelled on `test/chain-test.sh`: generate a
|
||||
CA, a server certificate, and two client certificates with `openssl`; assert
|
||||
that an allowed identity delivers entries, a non-allowed identity is rejected,
|
||||
a peer with no certificate fails the handshake, and a peer declaring another
|
||||
node's label is rejected under `assert` and corrected under `force`.
|
||||
|
||||
This phase alone closes the node-spoofing hole, which is the sharpest of the
|
||||
three problems.
|
||||
|
||||
### Phase 3 — Sinks
|
||||
|
||||
- `tcp` sink: authorize in `handleConn` before registration.
|
||||
- `http` sink: `authMiddleware` around the mux, covering stream and status.
|
||||
- Extend the test script to cover both.
|
||||
|
||||
### Phase 4 — Optional hardening
|
||||
|
||||
- Dialer-side server identity pinning for the chain sinks.
|
||||
- CRL file support alongside `client_ca_file`, re-read on reload.
|
||||
- Certificate expiry warnings at startup and on reload — a leaf expiring inside
|
||||
30 days logged at WARN, since nothing warns today.
|
||||
- Surface `tls_peer_cn` / `auth_identity` in the `http` sink's status output for
|
||||
connected clients.
|
||||
|
||||
## Compatibility
|
||||
|
||||
No configuration breaks. Omitting the `auth` block, or setting
|
||||
`type = "none"`, reproduces current behaviour byte for byte: `Policy` is nil,
|
||||
every call site short-circuits, and `trust_node` continues to govern node
|
||||
labels.
|
||||
|
||||
The one behavioural note for adopters: turning on `type = "mtls"` defaults
|
||||
`node_binding` to `force`, so entries from a peer whose certificate identity
|
||||
differs from its configured `node` label will be relabelled. That is the point
|
||||
of the feature, but it will move data between labels in a dashboard, so the
|
||||
release note should say it in those terms.
|
||||
|
||||
## Estimated Cost
|
||||
|
||||
| Phase | Files touched | Rough size |
|
||||
|-------|--------------|-----------|
|
||||
| 1 | `config/config.go`, `config/validate.go`, `tlsx/tlsx.go`, new `authz/` + tests | ~400 lines |
|
||||
| 2 | Two chain sources, new test script | ~200 lines |
|
||||
| 3 | `sink/tcp`, `sink/http`, test script extension | ~150 lines |
|
||||
| 4 | `tlsx`, both chain sinks, `sink/http` status | ~250 lines |
|
||||
|
||||
Phases 1–2 are the security-relevant core; phase 3 closes the read-side
|
||||
exposure; phase 4 is discretionary.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Should an empty allow-list deny instead of allow?** Deny-by-default is the
|
||||
safer instinct, but it makes `type = "mtls"` with no list a footgun that
|
||||
silently drops all traffic. The proposal is allow-with-a-loud-startup-log;
|
||||
the alternative is requiring a non-empty list and erroring at construction,
|
||||
which is arguably better and costs one line.
|
||||
2. **Should `identity` accept a list of modes** (try `san_uri`, fall back to
|
||||
`cn`)? Simpler as a single mode; heterogeneous PKI is the argument against.
|
||||
3. **Per-identity rate limits.** The natural follow-on once identity exists, and
|
||||
the natural home for the per-IP limiting that was also removed. Deliberately
|
||||
out of scope here so this feature stays reviewable.
|
||||
4. **Whether `assert` should reject or warn-and-correct.** As proposed it
|
||||
rejects, which is unambiguous but turns a certificate/config mismatch into an
|
||||
outage. `force` is the forgiving option, and it is the default.
|
||||
+149
-59
@@ -1,95 +1,185 @@
|
||||
# Networking
|
||||
|
||||
*Note: Under redesign*
|
||||
Everything LogWisp does over a socket, and the knobs that shape it. For
|
||||
certificates and trust see [Security](security.md); for multi-node topologies
|
||||
see [Chaining](chaining.md).
|
||||
|
||||
## TLS Configuration
|
||||
## Address Family
|
||||
|
||||
*Note: As of the latest architecture updates, network TLS, mTLS, and Rate Limiting features are undergoing a redesign and are currently acting as placeholders. The documentation below details the structure for future updates.*
|
||||
**All listeners bind `tcp4` and all dialers dial `tcp4`.** IPv6 is not
|
||||
supported, deliberately. An IPv6 client cannot connect and will simply see a
|
||||
connection failure.
|
||||
|
||||
## Connection Management
|
||||
When testing locally use `127.0.0.1`, not `localhost` — the latter may resolve
|
||||
to `::1` and appear as an unexplained connection refusal.
|
||||
|
||||
### TCP Keep-Alive
|
||||
## Network Plugins
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "tcp_out"
|
||||
type = "tcp"
|
||||
[pipelines.plugin_sinks.config]
|
||||
keep_alive = true
|
||||
keep_alive_period_ms = 30000 # 30 seconds
|
||||
| Plugin | Role | Protocol | Purpose |
|
||||
|--------|------|----------|---------|
|
||||
| `tcp` sink | Listener | Raw stream | Broadcast formatted payloads to clients |
|
||||
| `http` sink | Listener | HTTP SSE | Browser-friendly live stream plus status JSON |
|
||||
| `tcp_chain` source | Listener | Chain v1 | Ingest a persistent NDJSON stream |
|
||||
| `http_chain` source | Listener | Chain v1 | Ingest NDJSON batches over POST |
|
||||
| `tcp_chain` sink | Dialer | Chain v1 | Forward entries over a persistent connection |
|
||||
| `http_chain` sink | Dialer | Chain v1 | Forward entries as batched POSTs |
|
||||
|
||||
There is no port registry and no default port: `port` is required on every
|
||||
network plugin. There is also no cross-pipeline conflict detection — two sinks
|
||||
on the same port fail at bind time when the pipeline starts:
|
||||
|
||||
```
|
||||
ERROR msg="Failed to start sink" error="tcp sink bind 0.0.0.0:9090: listen tcp4 0.0.0.0:9090: bind: address already in use"
|
||||
```
|
||||
|
||||
### Connection Timeouts
|
||||
## Timeouts
|
||||
|
||||
Every network plugin exposes the deadlines relevant to its role. Zero means "no
|
||||
deadline" wherever the table says so.
|
||||
|
||||
| Plugin | Option | Default | Bounds |
|
||||
|--------|--------|---------|--------|
|
||||
| `tcp` sink | `write_timeout_ms` | `5000` | One write to one client; a miss disconnects that client |
|
||||
| `http` sink | `write_timeout_ms` | `0` (none) | One SSE event write |
|
||||
| `tcp_chain` source | `hello_timeout_ms` | `10000` | Reading the protocol preamble |
|
||||
| `tcp_chain` source | `read_timeout_ms` | `0` (none) | Idle time between entries |
|
||||
| `http_chain` source | `read_timeout_ms` | `30000` | Reading a whole request body |
|
||||
| `tcp_chain` sink | `dial_timeout_ms` | `5000` | TCP connect |
|
||||
| `tcp_chain` sink | `write_timeout_ms` | `5000` | One line write |
|
||||
| `http_chain` sink | `request_timeout_ms` | `10000` | Dial plus write plus response |
|
||||
|
||||
Fixed, non-configurable bounds:
|
||||
|
||||
| Bound | Value | Applies to |
|
||||
|-------|-------|------------|
|
||||
| TLS handshake | 10 s | All TLS listeners and dialers |
|
||||
| HTTP read-header timeout | 10 s | `http` sink, `http_chain` source |
|
||||
| HTTP server shutdown grace | 2 s | `http` sink, `http_chain` source |
|
||||
| Max single entry line | 1 MiB | Chain listeners |
|
||||
|
||||
The `http` sink deliberately leaves the server's `WriteTimeout` unset, since it
|
||||
would terminate long-lived SSE streams; per-event deadlines come from
|
||||
`write_timeout_ms` instead.
|
||||
|
||||
## Connection Limits
|
||||
|
||||
`max_connections` caps concurrent connections on the `tcp` sink, the `http`
|
||||
sink, and the `tcp_chain` source. `0` means unlimited.
|
||||
|
||||
- Admission is a load-then-check, so a burst can over-admit by roughly one
|
||||
connection. This is accepted, not a bug to work around.
|
||||
- On the `tcp` sink and `tcp_chain` source the count is taken at accept, so it
|
||||
bounds concurrent TLS handshakes as well as established sessions.
|
||||
- Over-limit connections are closed immediately and counted in `rejected_conns`
|
||||
(TCP) or `rejected_clients` (HTTP, which first answers `503`).
|
||||
|
||||
The `http_chain` source has no connection cap; it bounds work with
|
||||
`max_body_bytes` and `read_timeout_ms` instead.
|
||||
|
||||
There is **no** per-IP limiting and no IP allow/deny list. `flow.rate_limit` is
|
||||
a pipeline-wide entry rate limit, not a network-level one — it cannot
|
||||
distinguish or throttle an individual peer.
|
||||
|
||||
## Keep-Alive
|
||||
|
||||
TCP keep-alive is available on the `tcp` sink (for accepted connections) and the
|
||||
`tcp_chain` sink (for its outbound connection):
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "http_out"
|
||||
type = "http"
|
||||
[pipelines.plugin_sinks.config]
|
||||
write_timeout_ms = 10000 # 10 seconds
|
||||
keep_alive = true
|
||||
keep_alive_period_ms = 30000
|
||||
```
|
||||
|
||||
## Heartbeat Configuration
|
||||
This is kernel-level keep-alive; it detects a dead peer but does not keep an
|
||||
application-level stream flowing. For that, use a heartbeat.
|
||||
|
||||
Keep connections alive with periodic heartbeats. Note that Heartbeat is a flow-level feature in the new architecture.
|
||||
## Heartbeats
|
||||
|
||||
Heartbeats are a **flow-level** feature, not a per-sink one. Enabling one
|
||||
injects a synthetic entry into the pipeline at a fixed interval; it reaches
|
||||
every sink and traverses chain links as an ordinary structured entry.
|
||||
|
||||
```toml
|
||||
[pipelines.flow.heartbeat]
|
||||
enabled = true
|
||||
interval_ms = 30000
|
||||
enabled = true
|
||||
interval_ms = 30000
|
||||
include_timestamp = true
|
||||
include_stats = false
|
||||
format = "comment" # comment|event|json
|
||||
include_stats = false
|
||||
format = "txt" # txt | json | raw
|
||||
```
|
||||
|
||||
## Network Protocols
|
||||
Use it to keep idle SSE clients, TCP clients, and chain links from being
|
||||
reaped by intermediate NAT or proxy timeouts, and to make an idle pipeline
|
||||
visibly alive.
|
||||
|
||||
### HTTP/HTTPS
|
||||
> `format = "comment"` (SSE `:` comment framing) is rejected by validation
|
||||
> despite appearing in older documentation and in a still-present code branch.
|
||||
> A pipeline configured with it fails to start.
|
||||
|
||||
- HTTP/1.1 support
|
||||
- Persistent connections
|
||||
- Server-Sent Events (SSE)
|
||||
## Reconnection
|
||||
|
||||
### TCP
|
||||
Chain sinks reconnect on their own. Both use exponential backoff between
|
||||
`backoff_min_ms` and `backoff_max_ms` with ±20 % jitter, and both are
|
||||
interruptible by shutdown.
|
||||
|
||||
- Raw TCP sockets
|
||||
- Newline-delimited protocol
|
||||
```toml
|
||||
backoff_min_ms = 500
|
||||
backoff_max_ms = 30000
|
||||
```
|
||||
|
||||
## Port Configuration
|
||||
The connection is established lazily, so an edge node starts cleanly even when
|
||||
its relay is down and connects as soon as the relay appears. Reconnect counts
|
||||
are reported in the sink's `reconnects` statistic.
|
||||
|
||||
### Default Ports
|
||||
Server-side sinks (`tcp`, `http`) do not reconnect; clients are expected to
|
||||
retry. Browsers reconnect SSE streams automatically.
|
||||
|
||||
| Service | Default Port | Protocol |
|
||||
|---------|--------------|----------|
|
||||
| HTTP Sink | 8080 | HTTP |
|
||||
| TCP Sink | 9090 | TCP |
|
||||
## Protocol Details
|
||||
|
||||
### Port Conflict Prevention
|
||||
**HTTP sink (SSE)** — HTTP/1.1 in plaintext; HTTP/2 is negotiated via ALPN when
|
||||
TLS is enabled. Only `GET` is routed to the stream and status paths. Each event
|
||||
is framed as one `data:` line per newline in the payload, so multi-line entries
|
||||
stream intact. Response headers set `Cache-Control: no-cache`,
|
||||
`X-Accel-Buffering: no`, and `Access-Control-Allow-Origin: *`.
|
||||
|
||||
LogWisp validates port usage at startup:
|
||||
- Detects port conflicts across pipelines
|
||||
- Prevents duplicate bindings
|
||||
**TCP sink** — raw payload bytes, no framing added by the sink. Whether entries
|
||||
are newline-delimited depends on the formatter.
|
||||
|
||||
**Chain transports** — see [Chaining](chaining.md) for the hello preamble,
|
||||
headers, and entry encoding.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
**Connection refused**
|
||||
- Confirm the pipeline started; a bind failure is logged at ERROR.
|
||||
- Confirm you are dialing IPv4. `localhost` may resolve to `::1`.
|
||||
- Check the port is not already bound by another pipeline in the same process.
|
||||
|
||||
**Connection Refused**
|
||||
- Check firewall rules
|
||||
- Verify service is running
|
||||
- Confirm correct port/host
|
||||
**TLS handshake failure**
|
||||
- `client didn't provide a certificate` — the listener has `client_auth = true`
|
||||
and the dialer has no `cert_file`/`key_file`.
|
||||
- `certificate signed by unknown authority` — the dialer's `ca_file` does not
|
||||
contain the issuer of the server certificate, or the listener's
|
||||
`client_ca_file` does not contain the issuer of the client certificate.
|
||||
- `certificate is not valid for any names` / SAN mismatch — the dialed `host`
|
||||
is not covered by the server certificate's SANs; set `server_name`.
|
||||
- `protocol version not supported` — one side is pinned to `min_version = "1.3"`
|
||||
and the other cannot negotiate it.
|
||||
- Handshake failures appear as WARN with the remote address, and increment
|
||||
`tls_handshake_errors`.
|
||||
|
||||
**TLS Handshake Failure**
|
||||
- Verify certificate validity
|
||||
- Check certificate chain
|
||||
- Confirm TLS versions match
|
||||
**Entries not arriving over a chain link**
|
||||
- Check the sink's `connected` statistic and its `reconnects` count.
|
||||
- Check the source's `parse_errors` — a version skew shows up here.
|
||||
- On `http_chain`, remember entries wait up to `flush_interval_ms` before a
|
||||
batch is sent.
|
||||
|
||||
**Rate Limit Exceeded**
|
||||
- Adjust rate limit parameters
|
||||
- Add IP to whitelist
|
||||
- Implement client-side throttling
|
||||
**Clients connect but see nothing**
|
||||
- The pipeline may be filtering everything out; check `flow.filters` stats.
|
||||
- The rate limiter may be dropping everything; check `rate_limiter` stats.
|
||||
- Nothing may be arriving from the sources; check source `total_entries`.
|
||||
|
||||
**Connection Timeout**
|
||||
- Increase timeout values
|
||||
- Check network latency
|
||||
- Verify keep-alive settings
|
||||
**Entries missing under load**
|
||||
- Compare `dropped_writes` (per-client queue full — raise
|
||||
`client_buffer_size`) against `total_dropped_by_sink` (sink input queue full
|
||||
— raise `buffer_size` or reduce sink latency).
|
||||
|
||||
+200
-222
@@ -1,327 +1,305 @@
|
||||
# Operations Guide
|
||||
|
||||
Running, monitoring, and maintaining LogWisp in production.
|
||||
Running, monitoring, and maintaining LogWisp.
|
||||
|
||||
*Note: TLS, acccess control under redesign*
|
||||
|
||||
## Starting LogWisp
|
||||
|
||||
### Manual Start
|
||||
## Starting
|
||||
|
||||
```bash
|
||||
# Foreground with default config
|
||||
# foreground, explicit config
|
||||
logwisp -c /etc/logwisp/logwisp.toml
|
||||
|
||||
# no config: built-in demo pipeline (random source -> stdout)
|
||||
logwisp
|
||||
|
||||
# Background mode
|
||||
logwisp --background
|
||||
|
||||
# With specific configuration
|
||||
logwisp --config /etc/logwisp/production.toml
|
||||
```
|
||||
|
||||
### Service Management
|
||||
There is no built-in daemon mode. Run LogWisp in the foreground under a
|
||||
supervisor — systemd, rc.d, or a container runtime — which is where restart,
|
||||
log capture, and resource limits belong. See [Installation](installation.md).
|
||||
|
||||
**systemd**
|
||||
|
||||
**Linux (systemd):**
|
||||
```bash
|
||||
sudo systemctl start logwisp
|
||||
sudo systemctl stop logwisp
|
||||
sudo systemctl restart logwisp
|
||||
sudo systemctl status logwisp
|
||||
sudo journalctl -u logwisp -f
|
||||
```
|
||||
|
||||
**FreeBSD (rc.d):**
|
||||
**FreeBSD rc.d**
|
||||
|
||||
```bash
|
||||
sudo service logwisp start
|
||||
sudo service logwisp stop
|
||||
sudo service logwisp restart
|
||||
sudo service logwisp status
|
||||
```
|
||||
|
||||
## Configuration Management
|
||||
## Configuration Changes
|
||||
|
||||
### Hot Reload
|
||||
### Hot reload
|
||||
|
||||
Enable automatic configuration reload:
|
||||
```toml
|
||||
config_auto_reload = true
|
||||
auto_reload = true
|
||||
```
|
||||
|
||||
Or via command line:
|
||||
```bash
|
||||
logwisp --config-auto-reload
|
||||
```
|
||||
or send a signal:
|
||||
|
||||
Trigger manual reload:
|
||||
```bash
|
||||
kill -HUP $(pidof logwisp)
|
||||
# or
|
||||
kill -USR1 $(pidof logwisp)
|
||||
```
|
||||
|
||||
### Configuration Validation
|
||||
Reload constructs a new service from the new configuration **before** tearing
|
||||
the old one down, so a broken configuration leaves the running service intact
|
||||
and logs the failure:
|
||||
|
||||
```
|
||||
ERROR msg="Failed to bootstrap new service, keeping old service running" error=...
|
||||
```
|
||||
|
||||
What reload does *not* do:
|
||||
|
||||
- Re-apply `logging.*`; application logging is configured once at startup.
|
||||
- Preserve connections. Listeners close and reopen, and every SSE, TCP, and
|
||||
chain client is disconnected. Chain sinks reconnect on their own backoff;
|
||||
browsers reconnect SSE automatically; raw TCP consumers must retry themselves.
|
||||
- Reload certificates without a reload — certificate files are read at plugin
|
||||
construction, so rotation requires `SIGHUP`.
|
||||
|
||||
Plan reloads on a busy relay the way you would plan a restart.
|
||||
|
||||
### Checking a configuration
|
||||
|
||||
There is no validate-only mode. To check a file, start it with debug logging and
|
||||
watch for pipeline startup:
|
||||
|
||||
Test configuration without starting:
|
||||
```bash
|
||||
logwisp --config test.toml --quiet --status-reporter=false
|
||||
logwisp -c candidate.toml --logging.level=debug --logging.output=stderr
|
||||
```
|
||||
|
||||
Check for errors:
|
||||
- Port conflicts
|
||||
- Invalid patterns
|
||||
- Missing required fields
|
||||
- File permissions
|
||||
Success looks like `Created source instance`, `Created sink instance`, and
|
||||
`Starting pipeline` for each pipeline. Failures name the pipeline and the
|
||||
offending key:
|
||||
|
||||
```
|
||||
ERROR msg="Failed to create pipeline" pipeline=app error="failed to create sink out: port: must be 1-65535, got 0"
|
||||
```
|
||||
|
||||
Remember that most validation lives in plugin constructors, so a config only
|
||||
proves itself when the pipeline is actually built.
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Status Reporter
|
||||
### Status reporter
|
||||
|
||||
Built-in periodic status logging (30-second intervals):
|
||||
Enabled by default, every 30 seconds. It logs at **DEBUG**, so it produces
|
||||
nothing unless `logging.level = "debug"` — a common surprise.
|
||||
|
||||
```
|
||||
[INFO] Status report active_pipelines=2 time=15:04:05
|
||||
[INFO] Pipeline status pipeline=app entries_processed=10523
|
||||
[INFO] Pipeline status pipeline=system entries_processed=5231
|
||||
```
|
||||
|
||||
Disable if not needed:
|
||||
```toml
|
||||
disable_status_reporter = true
|
||||
status_reporter = true
|
||||
|
||||
[logging]
|
||||
level = "debug"
|
||||
```
|
||||
|
||||
### HTTP Status Endpoint
|
||||
It emits a service summary and then walks each pipeline, flattening scalar
|
||||
statistics into log fields and recursing into flow, rate limiter, filter,
|
||||
source, and sink stats.
|
||||
|
||||
Disable with `status_reporter = false`.
|
||||
|
||||
### HTTP status endpoint
|
||||
|
||||
When a pipeline has an `http` sink:
|
||||
|
||||
When using HTTP sink:
|
||||
```bash
|
||||
curl http://localhost:8080/status | jq .
|
||||
curl -s http://127.0.0.1:8080/status | jq .
|
||||
```
|
||||
|
||||
Response structure:
|
||||
```json
|
||||
{
|
||||
"uptime": "2h15m30s",
|
||||
"pipelines": {
|
||||
"default": {
|
||||
"sources": 1,
|
||||
"sinks": 2,
|
||||
"processed": 15234,
|
||||
"filtered": 523,
|
||||
"dropped": 12
|
||||
}
|
||||
"service": "LogWisp",
|
||||
"version": "v0.16.0",
|
||||
"instance_id": "sse",
|
||||
"server": {
|
||||
"type": "http",
|
||||
"host": "0.0.0.0",
|
||||
"port": 8080,
|
||||
"tls": false,
|
||||
"active_clients": 3,
|
||||
"buffer_size": 1000,
|
||||
"uptime_seconds": 8130
|
||||
},
|
||||
"endpoints": { "stream": "/stream", "status": "/status" },
|
||||
"statistics": {
|
||||
"total_processed": 15234,
|
||||
"dropped_writes": 12,
|
||||
"rejected_clients": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Metrics Collection
|
||||
This endpoint is scoped to one sink, not to the whole process, and it is
|
||||
**unauthenticated**. Bind it to a trusted interface.
|
||||
|
||||
Track via logs:
|
||||
- Total entries processed
|
||||
- Entries filtered
|
||||
- Entries dropped
|
||||
- Active connections
|
||||
- Buffer utilization
|
||||
### Metrics worth watching
|
||||
|
||||
| Metric | Where | Meaning if rising |
|
||||
|--------|-------|-------------------|
|
||||
| `dropped_entries` | source | Downstream cannot keep up with the source |
|
||||
| `total_dropped` | flow | Rate limit or filters are discarding entries (often intended) |
|
||||
| `total_dropped_by_sink` | pipeline | A sink's input queue is full |
|
||||
| `dropped_writes` | tcp/http sink | A specific client is too slow |
|
||||
| `rejected_conns` / `rejected_clients` | tcp/http sink, tcp_chain source | `max_connections` is being hit |
|
||||
| `tls_handshake_errors` | tcp sink, tcp_chain source | Certificate or version mismatch, or scanning |
|
||||
| `parse_errors` | chain source | Protocol or version skew upstream |
|
||||
| `reconnects` | chain sink | Unstable link or a flapping downstream |
|
||||
| `dropped_batches` | http_chain sink | Downstream rejecting batches permanently |
|
||||
| `synthesized` | chain sink | Events reaching the sink without structure |
|
||||
|
||||
## Log Management
|
||||
|
||||
### LogWisp's Operational Logs
|
||||
|
||||
Configuration for LogWisp's own logs:
|
||||
LogWisp's own operational log:
|
||||
|
||||
```toml
|
||||
[logging]
|
||||
output = "file"
|
||||
level = "info"
|
||||
level = "info"
|
||||
|
||||
[logging.file]
|
||||
directory = "/var/log/logwisp"
|
||||
name = "logwisp"
|
||||
max_size_mb = 100
|
||||
retention_hours = 168
|
||||
directory = "/var/log/logwisp"
|
||||
name = "logwisp"
|
||||
max_size_mb = 100
|
||||
max_total_size_mb = 1000
|
||||
retention_hours = 168.0
|
||||
```
|
||||
|
||||
### Log Rotation
|
||||
Rotation is automatic on size, with a total-size cap and a retention window.
|
||||
There is no signal to reopen log files, so do not move files out from under
|
||||
LogWisp and expect it to reattach — let it rotate, or restart it.
|
||||
|
||||
Automatic rotation based on:
|
||||
- File size threshold
|
||||
- Total size limit
|
||||
- Retention period
|
||||
|
||||
Manual rotation:
|
||||
```bash
|
||||
# Move current log
|
||||
mv /var/log/logwisp/logwisp.log /var/log/logwisp/logwisp.log.1
|
||||
# Send signal to reopen
|
||||
kill -USR1 $(pidof logwisp)
|
||||
```
|
||||
|
||||
### Log Levels
|
||||
|
||||
Operational log levels:
|
||||
- **debug**: Detailed debugging information
|
||||
- **info**: General operational messages
|
||||
- **warn**: Warning conditions
|
||||
- **error**: Error conditions
|
||||
|
||||
Production recommendation: `info` or `warn`
|
||||
Production level: `info`, or `warn` on a busy relay. Avoid `debug` under load:
|
||||
the filter stage logs several lines per entry evaluated.
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### Buffer Sizing
|
||||
### Buffers
|
||||
|
||||
Adjust buffers based on load:
|
||||
Raise `buffer_size` when `total_dropped_by_sink` is climbing but the sink itself
|
||||
is healthy — that is a burst-absorption problem. Raise `client_buffer_size` when
|
||||
`dropped_writes` is climbing for network sinks; that is a slow-consumer problem,
|
||||
and a bigger buffer only buys time.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "file_in"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
[pipelines.plugin_sinks.config]
|
||||
buffer_size = 5000
|
||||
client_buffer_size = 1024
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
Protect against overload:
|
||||
### Rate limiting
|
||||
|
||||
```toml
|
||||
[pipelines.flow.rate_limit]
|
||||
rate = 1000.0 # Entries per second
|
||||
burst = 2000.0 # Burst capacity
|
||||
policy = "drop" # Drop excess entries
|
||||
rate = 1000.0
|
||||
burst = 2000.0
|
||||
policy = "drop"
|
||||
max_entry_size_bytes = 65536
|
||||
```
|
||||
|
||||
Two behaviours to keep in mind: the limiter does not exist at all when
|
||||
`rate <= 0`, and `policy = "pass"` short-circuits the size cap as well as the
|
||||
rate check. Enforcing `max_entry_size_bytes` therefore requires `rate > 0` and
|
||||
`policy = "drop"`.
|
||||
|
||||
### Formatting
|
||||
|
||||
`raw` is the cheapest and skips sanitization; `json` costs the most. The
|
||||
formatter serializes on a mutex, so it is the one shared bottleneck in a
|
||||
pipeline — splitting work across pipelines parallelizes it.
|
||||
|
||||
### Chain batching
|
||||
|
||||
`http_chain` trades latency for efficiency. Lower `flush_interval_ms` for
|
||||
freshness, raise `max_batch_count` and `max_batch_bytes` for throughput. Use
|
||||
`tcp_chain` when per-entry latency matters.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
**Nothing appears at the sink**
|
||||
|
||||
**High Memory Usage**
|
||||
- Check buffer sizes
|
||||
- Monitor goroutine count
|
||||
- Review retention settings
|
||||
Walk the pipeline in order and read the counters: source `total_entries` (is
|
||||
anything being produced?), flow `total_dropped` (filters or rate limit?),
|
||||
pipeline `total_dropped_by_sink` (sink backed up?), sink `total_processed`.
|
||||
|
||||
**Dropped Entries**
|
||||
- Increase buffer sizes
|
||||
- Add rate limiting
|
||||
- Check sink performance
|
||||
**File source reads nothing**
|
||||
|
||||
**Connection Errors**
|
||||
- Verify network connectivity
|
||||
- Check firewall rules
|
||||
- Review TLS certificates
|
||||
- The watcher seeks to end-of-file on start; only content appended afterwards is
|
||||
read. Positions are in memory, so a restart re-seeks to end and anything
|
||||
written during the downtime is lost.
|
||||
- `pattern` is a filename glob with `*` and `?` only, and matching is not
|
||||
recursive.
|
||||
- `check_interval_ms` governs how quickly a *new file* is noticed; tailing an
|
||||
open file polls at a fixed 100 ms.
|
||||
|
||||
### Debug Mode
|
||||
**High memory use**
|
||||
|
||||
Enable detailed logging:
|
||||
```bash
|
||||
logwisp --logging.level=debug --logging.output=stderr
|
||||
```
|
||||
Buffers are bounded, so unbounded growth almost always means many buffers:
|
||||
count sinks × `buffer_size`, plus clients × `client_buffer_size`. A `tcp_chain`
|
||||
sink blocked on an unreachable downstream also holds its full input queue.
|
||||
|
||||
### Health Checks
|
||||
**Chain link not delivering**
|
||||
|
||||
Implement external monitoring:
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Health check script
|
||||
if ! curl -sf http://localhost:8080/status > /dev/null; then
|
||||
echo "LogWisp health check failed"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
Check `connected` and `reconnects` on the sink, `parse_errors` on the source,
|
||||
and remember `http_chain` waits up to `flush_interval_ms`. For TLS problems see
|
||||
[Networking](networking.md#troubleshooting).
|
||||
|
||||
## Backup and Recovery
|
||||
**Environment variable override has no effect**
|
||||
|
||||
### Configuration Backup
|
||||
|
||||
```bash
|
||||
# Backup configuration
|
||||
cp /etc/logwisp/logwisp.toml /backup/logwisp-$(date +%Y%m%d).toml
|
||||
|
||||
# Version control
|
||||
git add /etc/logwisp/
|
||||
git commit -m "LogWisp config update"
|
||||
```
|
||||
|
||||
### State Recovery
|
||||
|
||||
LogWisp maintains minimal state:
|
||||
- File read positions (automatic)
|
||||
- Connection state (automatic)
|
||||
|
||||
Recovery after crash:
|
||||
1. Service automatically restarts (systemd/rc.d)
|
||||
2. File sources resume from last position
|
||||
3. Network sources accept new connections
|
||||
4. Clients reconnect automatically
|
||||
LogWisp currently reads these **without** the `LOGWISP_` prefix — `QUIET`,
|
||||
`LOGGING_LEVEL`, and so on. Array-indexed paths cannot be set from the
|
||||
environment or the command line at all.
|
||||
|
||||
## Security Operations
|
||||
|
||||
### Certificate Management
|
||||
**Certificate rotation**
|
||||
|
||||
Monitor certificate expiration:
|
||||
```bash
|
||||
openssl x509 -in /path/to/cert.pem -noout -enddate
|
||||
openssl x509 -in /etc/logwisp/tls/relay.crt -noout -enddate
|
||||
```
|
||||
|
||||
Rotate certificates:
|
||||
1. Generate new certificates
|
||||
2. Update configuration
|
||||
3. Reload service (SIGHUP)
|
||||
Certificates load at plugin construction, so rotation is: write the new files,
|
||||
then `kill -HUP`. Automate the expiry check; nothing in LogWisp warns you.
|
||||
|
||||
### Access Auditing
|
||||
**Access review**
|
||||
|
||||
Monitor access patterns:
|
||||
- Review connection logs
|
||||
- Monitor rate limit hits
|
||||
With mTLS, any certificate signed by the configured `client_ca_file` is
|
||||
accepted — there is no per-identity allow-list, so "access review" means
|
||||
reviewing what your CA has issued. Peer Common Names are recorded in session
|
||||
metadata but are not surfaced in statistics and are not used for authorization.
|
||||
See [Security](security.md) and the
|
||||
[mTLS authentication plan](mtls-auth-plan.md).
|
||||
|
||||
**Secret leakage**
|
||||
|
||||
Filters are the only redaction mechanism, and they drop whole entries rather
|
||||
than masking parts of them. See [Filters](filters.md#common-recipes).
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Planned Maintenance
|
||||
**Upgrades**
|
||||
|
||||
1. Notify users of maintenance window
|
||||
2. Stop accepting new connections
|
||||
3. Drain existing connections
|
||||
4. Perform maintenance
|
||||
5. Restart service
|
||||
1. Read the changelog for configuration-schema changes.
|
||||
2. Start the new binary against the current configuration in a scratch
|
||||
environment.
|
||||
3. Stop the old process, install the new binary, start it.
|
||||
4. Confirm each pipeline started and that counters are advancing.
|
||||
|
||||
### Upgrade Process
|
||||
**Backup**
|
||||
|
||||
1. Download new version
|
||||
2. Test with current configuration
|
||||
3. Stop old version
|
||||
4. Install new version
|
||||
5. Start service
|
||||
6. Verify operation
|
||||
Configuration files and TLS material are the only durable state worth backing
|
||||
up. LogWisp keeps no persistent runtime state: file read positions live in
|
||||
memory, connections are re-established on restart, and in-flight entries are
|
||||
lost.
|
||||
|
||||
### Cleanup Tasks
|
||||
**Redundancy**
|
||||
|
||||
Regular maintenance:
|
||||
- Remove old log files
|
||||
- Clean temporary files
|
||||
- Verify disk space
|
||||
- Update documentation
|
||||
|
||||
## Disaster Recovery
|
||||
|
||||
### Backup Strategy
|
||||
|
||||
- Configuration files: Daily
|
||||
- TLS certificates: After generation
|
||||
- Authentication credentials: Secure storage
|
||||
|
||||
### Recovery Procedures
|
||||
|
||||
Service failure:
|
||||
1. Check service status
|
||||
2. Review error logs
|
||||
3. Verify configuration
|
||||
4. Restart service
|
||||
|
||||
Data loss:
|
||||
1. Restore configuration from backup
|
||||
2. Regenerate certificates if needed
|
||||
3. Recreate authentication credentials
|
||||
4. Restart service
|
||||
|
||||
### Business Continuity
|
||||
|
||||
- Run multiple instances for redundancy
|
||||
- Use load balancer for distribution
|
||||
- Implement monitoring alerts
|
||||
- Document recovery procedures
|
||||
Because there is no persistence, availability comes from topology, not from
|
||||
LogWisp itself. Give each edge two chain sinks pointing at two relays if you
|
||||
need to survive a relay outage, and accept that this duplicates entries
|
||||
downstream.
|
||||
|
||||
+239
-1
@@ -1,4 +1,242 @@
|
||||
# Security
|
||||
|
||||
*Note: Security features like mTLS and IP-Based Access Control are currently under redesign and act as placeholders in the new architecture. Future versions will reintroduce full security capabilities.*
|
||||
This page covers LogWisp's transport security: what it protects, how to
|
||||
configure it, and — equally important — what it does not yet do.
|
||||
|
||||
## Current State
|
||||
|
||||
| Capability | Status |
|
||||
|------------|--------|
|
||||
| TLS 1.2 / 1.3 on all network sources and sinks | Implemented |
|
||||
| Server certificate verification by dialers | Implemented |
|
||||
| Mutual TLS (client certificate required and verified) | Implemented at the transport layer |
|
||||
| Peer identity (certificate CN) recorded per session | Implemented |
|
||||
| Authorization from peer identity (CN allow-lists, node binding) | **Not implemented** — see [mtls-auth-plan.md](mtls-auth-plan.md) |
|
||||
| Password, token, or SCRAM authentication | **Removed**; not currently available |
|
||||
| IP allow/deny lists, per-IP connection or request limits | **Not implemented** |
|
||||
| Authentication on the `http` sink's stream and status endpoints | **Not implemented** |
|
||||
|
||||
Earlier releases carried basic-auth, bearer-token, and SCRAM authentication.
|
||||
Those were removed during the move to the plugin/flow architecture and the
|
||||
switch to standard-library networking. Only certificate-based transport
|
||||
security survived that transition.
|
||||
|
||||
## The TLS Block
|
||||
|
||||
One option shape serves both roles, so the configuration reads the same
|
||||
wherever it appears. Which keys matter depends on whether the plugin listens or
|
||||
dials.
|
||||
|
||||
```toml
|
||||
[pipelines.plugin_sources.config.tls] # or plugin_sinks.config.tls
|
||||
enabled = false
|
||||
cert_file = ""
|
||||
key_file = ""
|
||||
client_auth = false
|
||||
client_ca_file = ""
|
||||
ca_file = ""
|
||||
server_name = ""
|
||||
insecure_skip_verify = false
|
||||
min_version = "1.3"
|
||||
```
|
||||
|
||||
| Option | Role | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `enabled` | both | `false` | Master switch; when false the whole block is ignored |
|
||||
| `cert_file` | both | — | Local certificate. **Required** for listeners; optional client identity for dialers |
|
||||
| `key_file` | both | — | Private key for `cert_file`. Must be set together with it |
|
||||
| `client_auth` | listener | `false` | Require and verify a client certificate (mTLS) |
|
||||
| `client_ca_file` | listener | — | CA bundle used to verify client certificates. **Required** when `client_auth` is true |
|
||||
| `ca_file` | dialer | system store | CA bundle used to verify the server certificate |
|
||||
| `server_name` | dialer | the configured `host` | SNI and certificate name to verify against |
|
||||
| `insecure_skip_verify` | dialer | `false` | Disable server verification |
|
||||
| `min_version` | both | `"1.3"` | `"1.2"` or `"1.3"` |
|
||||
|
||||
**Roles by plugin:**
|
||||
|
||||
| Plugin | Role | Keys that apply |
|
||||
|--------|------|-----------------|
|
||||
| `tcp` sink, `http` sink | Listener | `cert_file`, `key_file`, `client_auth`, `client_ca_file`, `min_version` |
|
||||
| `tcp_chain` source, `http_chain` source | Listener | same as above |
|
||||
| `tcp_chain` sink, `http_chain` sink | Dialer | `ca_file`, `server_name`, `insecure_skip_verify`, `cert_file`, `key_file`, `min_version` |
|
||||
|
||||
> `min_version` takes `"1.2"` or `"1.3"`. The older `"TLS1.2"` spelling from
|
||||
> pre-restructure releases is rejected. There is no `max_version` and no
|
||||
> `cipher_suites` option; TLS 1.3 suites are not configurable in Go, and the
|
||||
> 1.2 defaults are the standard library's.
|
||||
|
||||
### Validation
|
||||
|
||||
Misconfiguration fails at plugin construction, before the pipeline starts:
|
||||
|
||||
- a listener with `enabled = true` and no `cert_file`/`key_file`
|
||||
- `client_auth = true` with no `client_ca_file`
|
||||
- a dialer with only one of `cert_file` / `key_file`
|
||||
- a certificate or key that will not load, or a CA file containing no
|
||||
certificates
|
||||
- a `min_version` that is neither `"1.2"` nor `"1.3"`
|
||||
|
||||
## Enabling mTLS
|
||||
|
||||
### 1. Generate a CA and certificates
|
||||
|
||||
LogWisp no longer ships a certificate-generation subcommand; the `logwisp tls`
|
||||
command was removed with the rest of the CLI restructure. Use `openssl`,
|
||||
`cfssl`, `step-cli`, or your existing PKI.
|
||||
|
||||
```bash
|
||||
# CA
|
||||
openssl req -x509 -newkey rsa:4096 -nodes -days 3650 \
|
||||
-keyout ca.key -out ca.crt -subj "/CN=LogWisp CA"
|
||||
|
||||
# Relay (server) certificate — SAN must match how clients address it
|
||||
openssl req -newkey rsa:2048 -nodes -keyout relay.key -out relay.csr \
|
||||
-subj "/CN=relay.internal"
|
||||
openssl x509 -req -in relay.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
|
||||
-out relay.crt -days 825 \
|
||||
-extfile <(printf "subjectAltName=DNS:relay.internal\nextendedKeyUsage=serverAuth")
|
||||
|
||||
# Edge (client) certificate — CN identifies the node
|
||||
openssl req -newkey rsa:2048 -nodes -keyout edge-01.key -out edge-01.csr \
|
||||
-subj "/CN=edge-01"
|
||||
openssl x509 -req -in edge-01.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
|
||||
-out edge-01.crt -days 825 \
|
||||
-extfile <(printf "extendedKeyUsage=clientAuth")
|
||||
```
|
||||
|
||||
The server certificate's SAN must cover the address clients dial. Dialers seed
|
||||
`ServerName` from the configured `host`, so an IP literal in `host` requires an
|
||||
IP SAN, and a DNS name requires a DNS SAN. Override with `server_name` when the
|
||||
dialed address and the certificate name legitimately differ.
|
||||
|
||||
### 2. Configure the listener
|
||||
|
||||
```toml
|
||||
[pipelines.plugin_sources.config.tls]
|
||||
enabled = true
|
||||
cert_file = "/etc/logwisp/tls/relay.crt"
|
||||
key_file = "/etc/logwisp/tls/relay.key"
|
||||
client_auth = true
|
||||
client_ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
min_version = "1.3"
|
||||
```
|
||||
|
||||
### 3. Configure the dialer
|
||||
|
||||
```toml
|
||||
[pipelines.plugin_sinks.config.tls]
|
||||
enabled = true
|
||||
ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
cert_file = "/etc/logwisp/tls/edge-01.crt"
|
||||
key_file = "/etc/logwisp/tls/edge-01.key"
|
||||
min_version = "1.3"
|
||||
```
|
||||
|
||||
### 4. Verify
|
||||
|
||||
Startup logs report both flags:
|
||||
|
||||
```
|
||||
INFO msg="TCP chain source initialized" ... tls=true mtls=true
|
||||
INFO msg="TCP chain sink initialized" ... tls=true mtls=true
|
||||
```
|
||||
|
||||
A client that presents no certificate is refused during the handshake:
|
||||
|
||||
```
|
||||
WARN msg="TLS handshake failed" component=tcp_chain_source
|
||||
remote_addr=127.0.0.1:53840 error="tls: client didn't provide a certificate"
|
||||
```
|
||||
|
||||
Handshake failures are counted in the `tls_handshake_errors` statistic on the
|
||||
`tcp` sink and the `tcp_chain` source.
|
||||
|
||||
## What mTLS Currently Buys You
|
||||
|
||||
With `client_auth = true`, the transport enforces:
|
||||
|
||||
- the peer holds a certificate chaining to `client_ca_file`
|
||||
- the certificate is within its validity window and not structurally broken
|
||||
- the peer holds the matching private key
|
||||
|
||||
That is a real membership check: an attacker without a CA-issued certificate
|
||||
cannot connect at all.
|
||||
|
||||
## What It Does Not Buy You
|
||||
|
||||
**Any** valid certificate from the configured CA is accepted. LogWisp extracts
|
||||
the peer's Common Name into session metadata (`tls_peer_cn`) but never consults
|
||||
it, so within one CA there is no way to express:
|
||||
|
||||
- "only `edge-01` and `edge-02` may connect to this ingest port"
|
||||
- "the node label `edge-01` may only be claimed by the holder of the `edge-01`
|
||||
certificate"
|
||||
- "this certificate may connect but only at this rate"
|
||||
|
||||
Two consequences follow.
|
||||
|
||||
1. **A compromised edge can impersonate any other edge.** With
|
||||
`trust_node = true` (the default) a peer declares its own node label. Any
|
||||
certificate holder can claim `edge-99`, or `relay`, and downstream consumers
|
||||
will attribute its entries accordingly. Setting `trust_node = false` replaces
|
||||
the label with the remote address, which is coarse but not forgeable at the
|
||||
application layer.
|
||||
2. **Revocation is CA-wide.** With no CRL or OCSP checking and no per-identity
|
||||
allow-list, withdrawing one node's access means re-issuing the CA or rotating
|
||||
the CA bundle for every peer.
|
||||
|
||||
Closing both gaps is the subject of the
|
||||
[mTLS authentication plan](mtls-auth-plan.md).
|
||||
|
||||
## Unauthenticated Surfaces
|
||||
|
||||
These endpoints have no access control at all. Bind them to a trusted interface
|
||||
or front them with an authenticating proxy.
|
||||
|
||||
| Surface | Exposure |
|
||||
|---------|----------|
|
||||
| `http` sink `stream_path` | Full log stream, with `Access-Control-Allow-Origin: *`, so any browser origin can read it |
|
||||
| `http` sink `status_path` | Host, port, TLS flag, uptime, client counts, throughput counters |
|
||||
| `tcp` sink | Full log stream to any client that connects |
|
||||
|
||||
`max_connections` bounds concurrency on all three but does not distinguish
|
||||
callers.
|
||||
|
||||
## Operational Guidance
|
||||
|
||||
**Certificates**
|
||||
|
||||
- Use a dedicated CA for LogWisp so its trust decisions stay independent.
|
||||
- Keep leaf lifetimes short (90–825 days) and automate renewal.
|
||||
- Key files should be `0600` and owned by the service account.
|
||||
- Rotation requires a reload (`SIGHUP`), because certificates are loaded once at
|
||||
plugin construction; there is no on-disk watch for certificate files.
|
||||
- Check expiry: `openssl x509 -in relay.crt -noout -enddate`.
|
||||
|
||||
**Deployment**
|
||||
|
||||
- Prefer `min_version = "1.3"`. Drop to `"1.2"` only for a peer that genuinely
|
||||
cannot do 1.3.
|
||||
- Never enable `insecure_skip_verify` outside a lab; it disables server
|
||||
verification entirely and makes the connection trivially interceptable.
|
||||
- Bind listeners to specific interfaces rather than `0.0.0.0` where you can.
|
||||
- Use `trust_node = false` on any ingest port reachable from a network you do
|
||||
not fully control.
|
||||
- Run LogWisp as an unprivileged user with write access only to its own log and
|
||||
configuration directories.
|
||||
|
||||
**Log content**
|
||||
|
||||
Logs routinely contain secrets that were never meant to leave the host. Filters
|
||||
are the available tool:
|
||||
|
||||
```toml
|
||||
[[pipelines.flow.filters]]
|
||||
type = "exclude"
|
||||
patterns = ["password", "api[_-]?key", "authorization", "bearer ", "secret"]
|
||||
```
|
||||
|
||||
Choose a sanitizer policy that matches the sink — `json` for JSON output,
|
||||
`txt` for files and consoles — so control characters in log data cannot break
|
||||
framing or inject terminal escapes downstream. See
|
||||
[Formatters](formatters.md).
|
||||
|
||||
+294
-113
@@ -1,154 +1,335 @@
|
||||
# Output Sinks
|
||||
|
||||
LogWisp sinks deliver processed log entries to various destinations.
|
||||
Sinks consume `core.TransportEvent` values — a formatted `Payload` plus the
|
||||
original structured `Entry` — and deliver them somewhere. Each sink is declared
|
||||
as a `[[pipelines.plugin_sinks]]` entry with an `id`, a `type`, and a
|
||||
type-specific `config` table.
|
||||
|
||||
## Sink Types
|
||||
Registered types: `console`, `file`, `http`, `tcp`, `null`, `tcp_chain`,
|
||||
`http_chain`.
|
||||
|
||||
### Console Sink
|
||||
Dispatch into a sink is non-blocking. A sink whose input queue is full drops the
|
||||
event *for itself only* and the pipeline counts it in `total_dropped_by_sink`;
|
||||
sibling sinks are unaffected.
|
||||
|
||||
Output to stdout/stderr.
|
||||
---
|
||||
|
||||
## console
|
||||
|
||||
Writes formatted payloads to stdout or stderr.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "console_out"
|
||||
id = "stdout"
|
||||
type = "console"
|
||||
[pipelines.plugin_sinks.config]
|
||||
target = "stdout" # stdout|stderr|split
|
||||
target = "stdout"
|
||||
buffer_size = 1000
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `target` | string | "stdout" | Output target (stdout/stderr/split) |
|
||||
| `buffer_size` | int | 1000 | Internal buffer size |
|
||||
| `target` | string | `stdout` | `stdout` or `stderr` |
|
||||
| `buffer_size` | int | `1000` | Sink input queue depth |
|
||||
|
||||
**Target Modes:**
|
||||
- **stdout**: All output to standard output
|
||||
- **stderr**: All output to standard error
|
||||
- **split**: INFO/DEBUG to stdout, WARN/ERROR to stderr
|
||||
> `split` is **not** a valid target for this sink and is rejected at startup.
|
||||
> Level-based splitting exists only for LogWisp's own application log
|
||||
> (`logging.output = "split"`).
|
||||
|
||||
### File Sink
|
||||
Payloads are written verbatim; the sink adds no framing. Whether entries are
|
||||
newline-terminated is decided by the formatter.
|
||||
|
||||
Write logs to rotating files.
|
||||
---
|
||||
|
||||
## file
|
||||
|
||||
Rotating file writer.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "file_out"
|
||||
id = "archive"
|
||||
type = "file"
|
||||
[pipelines.plugin_sinks.config]
|
||||
directory = "./logs"
|
||||
name = "output"
|
||||
max_size_mb = 100
|
||||
directory = "/var/log/logwisp"
|
||||
name = "output"
|
||||
max_size_mb = 100
|
||||
max_total_size_mb = 1000
|
||||
min_disk_free_mb = 500
|
||||
retention_hours = 168.0
|
||||
buffer_size = 1000
|
||||
flush_interval_ms = 1000
|
||||
min_disk_free_mb = 0
|
||||
retention_hours = 168.0
|
||||
buffer_size = 1000
|
||||
flush_interval_ms = 100
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `directory` | string | Required | Output directory |
|
||||
| `name` | string | Required | Base filename |
|
||||
| `max_size_mb` | int | 100 | Rotation threshold |
|
||||
| `max_total_size_mb` | int | 1000 | Total size limit |
|
||||
| `min_disk_free_mb` | int | 500 | Minimum free disk space |
|
||||
| `retention_hours` | float | 168 | Delete files older than |
|
||||
| `buffer_size` | int | 1000 | Internal buffer size |
|
||||
| `flush_interval_ms` | int | 1000 | Force flush interval |
|
||||
| `directory` | string | **required** | Output directory |
|
||||
| `name` | string | **required** | Base filename |
|
||||
| `max_size_mb` | int | `100` | Rotate when the active file reaches this size |
|
||||
| `max_total_size_mb` | int | `1000` | Cap across all rotated files |
|
||||
| `min_disk_free_mb` | int | `0` | Free-space floor before writing; `0` = no floor |
|
||||
| `retention_hours` | float | `168.0` | Delete rotated files older than this |
|
||||
| `buffer_size` | int | `1000` | Sink input queue depth |
|
||||
| `flush_interval_ms` | int | `100` | Forced flush interval |
|
||||
|
||||
**Features:**
|
||||
- Automatic rotation on size
|
||||
- Retention management
|
||||
- Disk space monitoring
|
||||
- Periodic flushing
|
||||
> `min_disk_free_mb` has an unusual default. The constructor replaces only
|
||||
> *negative* values with `100`; leaving the key unset yields `0`, which means no
|
||||
> free-space floor. Set it explicitly if you want one.
|
||||
|
||||
### HTTP Sink
|
||||
The sink drives an internal writer configured for raw output with timestamps and
|
||||
levels disabled, so what lands on disk is exactly the formatted payload.
|
||||
|
||||
SSE (Server-Sent Events) streaming server.
|
||||
---
|
||||
|
||||
## null
|
||||
|
||||
Discards everything, counting entries and bytes. Useful for benchmarking a
|
||||
source or flow in isolation.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "http_out"
|
||||
type = "http"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "0.0.0.0"
|
||||
port = 8080
|
||||
stream_path = "/stream"
|
||||
status_path = "/status"
|
||||
buffer_size = 1000
|
||||
client_buffer_size = 256
|
||||
write_timeout_ms = 0
|
||||
max_connections = 0
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | "0.0.0.0" | Bind address |
|
||||
| `port` | int | Required | Listen port |
|
||||
| `stream_path` | string | "/stream" | SSE stream endpoint |
|
||||
| `status_path` | string | "/status" | Status endpoint |
|
||||
| `buffer_size` | int | 1000 | Sink input queue size |
|
||||
| `client_buffer_size` | int | 256 | Per-client send queue size |
|
||||
| `write_timeout_ms` | int | 0 | Write deadline per event (0 = none) |
|
||||
| `max_connections` | int | 0 | Concurrent connection cap (0 = unlimited) |
|
||||
|
||||
### TCP Sink
|
||||
|
||||
TCP streaming server for debugging and raw client forwarding.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "tcp_out"
|
||||
type = "tcp"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "0.0.0.0"
|
||||
port = 9090
|
||||
buffer_size = 1000
|
||||
client_buffer_size = 256
|
||||
write_timeout_ms = 5000
|
||||
keep_alive = true
|
||||
keep_alive_period_ms = 30000
|
||||
max_connections = 0
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | "0.0.0.0" | Bind address |
|
||||
| `port` | int | Required | Listen port |
|
||||
| `buffer_size` | int | 1000 | Sink input queue size |
|
||||
| `client_buffer_size` | int | 256 | Per-client send queue size |
|
||||
| `write_timeout_ms` | int | 5000 | Write timeout |
|
||||
| `keep_alive` | bool | true | Enable TCP keep-alive |
|
||||
| `keep_alive_period_ms` | int | 30000 | Keep-alive interval |
|
||||
| `max_connections` | int | 0 | Concurrent connection cap (0 = unlimited) |
|
||||
```
|
||||
|
||||
### Null Sink
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "null_out"
|
||||
id = "discard"
|
||||
type = "null"
|
||||
```
|
||||
|
||||
## Buffer Management
|
||||
No options. The input queue is fixed at 1000.
|
||||
|
||||
- Full input buffer: entry dropped for that sink only (counted per pipeline as `total_dropped_by_sink`)"
|
||||
---
|
||||
|
||||
## http
|
||||
|
||||
Server-Sent Events stream plus a JSON status endpoint.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "sse"
|
||||
type = "http"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "0.0.0.0"
|
||||
port = 8080
|
||||
stream_path = "/stream"
|
||||
status_path = "/status"
|
||||
buffer_size = 1000
|
||||
client_buffer_size = 256
|
||||
write_timeout_ms = 0
|
||||
max_connections = 0
|
||||
|
||||
[pipelines.plugin_sinks.config.tls]
|
||||
enabled = true
|
||||
cert_file = "/etc/logwisp/tls/server.crt"
|
||||
key_file = "/etc/logwisp/tls/server.key"
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | `0.0.0.0` | Bind address; IPv4 only |
|
||||
| `port` | int | **required** | Listen port |
|
||||
| `stream_path` | string | `/stream` | SSE endpoint; must start with `/` |
|
||||
| `status_path` | string | `/status` | Status endpoint; must start with `/` and differ from `stream_path` |
|
||||
| `buffer_size` | int | `1000` | Sink input queue depth |
|
||||
| `client_buffer_size` | int | `256` | Per-client send queue depth |
|
||||
| `write_timeout_ms` | int | `0` | Per-event write deadline; `0` = none |
|
||||
| `max_connections` | int | `0` | Concurrent stream cap; `0` = unlimited |
|
||||
| `tls` | table | — | Listener TLS; see [Security](security.md) |
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Only `GET` is routed to either path; anything else gets `405`.
|
||||
- On connect the client receives an `event: connected` frame carrying its
|
||||
client id, session id, sink instance id, endpoint paths, and buffer size.
|
||||
- Payloads are framed per the SSE spec, one `data:` line per newline in the
|
||||
payload, so multi-line entries stream correctly.
|
||||
- The server sets no `WriteTimeout` (that would kill long-lived streams);
|
||||
per-write deadlines come from `write_timeout_ms` via `http.ResponseController`.
|
||||
- A client whose send queue is full has that event dropped
|
||||
(`dropped_writes`); it is not disconnected.
|
||||
- Clients whose session has been idle-expired by the session manager are
|
||||
evicted by the broker.
|
||||
- On shutdown, connected clients receive
|
||||
`event: disconnect / data: {"reason":"server_shutdown"}`.
|
||||
- HTTP/2 is negotiated via ALPN when TLS is enabled; plaintext is HTTP/1.1.
|
||||
|
||||
**Status endpoint** returns service and version identity, host, port, TLS flag,
|
||||
active client count, buffer size, uptime, endpoint paths, and the
|
||||
`total_processed` / `dropped_writes` / `rejected_clients` counters.
|
||||
|
||||
> Both endpoints are unauthenticated, and the stream response carries
|
||||
> `Access-Control-Allow-Origin: *`, so any web origin can read it. Bind to a
|
||||
> trusted interface, or put an authenticating reverse proxy in front.
|
||||
|
||||
---
|
||||
|
||||
## tcp
|
||||
|
||||
Broadcasts formatted payloads to every connected TCP client.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "tap"
|
||||
type = "tcp"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "0.0.0.0"
|
||||
port = 9090
|
||||
buffer_size = 1000
|
||||
client_buffer_size = 256
|
||||
write_timeout_ms = 5000
|
||||
keep_alive = true
|
||||
keep_alive_period_ms = 30000
|
||||
max_connections = 0
|
||||
|
||||
[pipelines.plugin_sinks.config.tls]
|
||||
enabled = true
|
||||
cert_file = "/etc/logwisp/tls/server.crt"
|
||||
key_file = "/etc/logwisp/tls/server.key"
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | `0.0.0.0` | Bind address; IPv4 only |
|
||||
| `port` | int | **required** | Listen port |
|
||||
| `buffer_size` | int | `1000` | Sink input queue depth |
|
||||
| `client_buffer_size` | int | `256` | Per-client send queue depth |
|
||||
| `write_timeout_ms` | int | `5000` | Per-write deadline |
|
||||
| `keep_alive` | bool | `true` | Enable TCP keep-alive on accepted connections |
|
||||
| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period |
|
||||
| `max_connections` | int | `0` | Concurrent connection cap; `0` = unlimited |
|
||||
| `tls` | table | — | Listener TLS |
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- The sink is write-only. Each connection also runs a reader that discards
|
||||
inbound bytes; it exists to detect disconnects and to refresh session
|
||||
activity when a client sends anything.
|
||||
- A write that misses its deadline means the kernel buffer stayed full for the
|
||||
whole timeout, so the client is disconnected immediately rather than retried.
|
||||
- A client whose send queue is full has that event dropped (`dropped_writes`)
|
||||
and stays connected.
|
||||
- With TLS enabled the handshake runs under a 10 s bound *after* the
|
||||
`max_connections` check, so concurrent handshakes are bounded too.
|
||||
|
||||
---
|
||||
|
||||
## tcp_chain
|
||||
|
||||
Forwards structured entries to a downstream LogWisp `tcp_chain` source over one
|
||||
persistent connection. See [Chaining](chaining.md).
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "to_relay"
|
||||
type = "tcp_chain"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "relay.internal"
|
||||
port = 15801
|
||||
node = "edge-01"
|
||||
buffer_size = 1000
|
||||
dial_timeout_ms = 5000
|
||||
write_timeout_ms = 5000
|
||||
backoff_min_ms = 500
|
||||
backoff_max_ms = 30000
|
||||
keep_alive = true
|
||||
keep_alive_period_ms = 30000
|
||||
|
||||
[pipelines.plugin_sinks.config.tls]
|
||||
enabled = true
|
||||
ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
cert_file = "/etc/logwisp/tls/client.crt"
|
||||
key_file = "/etc/logwisp/tls/client.key"
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | **required** | Downstream host |
|
||||
| `port` | int | **required** | Downstream port |
|
||||
| `node` | string | `os.Hostname()` | Origin label stamped on first-hop entries |
|
||||
| `buffer_size` | int | `1000` | Sink input queue depth |
|
||||
| `dial_timeout_ms` | int | `5000` | TCP connect timeout |
|
||||
| `write_timeout_ms` | int | `5000` | Per-write deadline |
|
||||
| `backoff_min_ms` | int | `500` | Reconnect backoff floor |
|
||||
| `backoff_max_ms` | int | `30000` | Reconnect backoff ceiling |
|
||||
| `keep_alive` | bool | `true` | Enable TCP keep-alive |
|
||||
| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period |
|
||||
| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity |
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- The connection is established lazily, so pipeline start does not depend on the
|
||||
downstream being up.
|
||||
- Each entry is serialized as one canonical JSON line. Delivery holds the line
|
||||
across reconnects until it is written or the process shuts down, with
|
||||
exponential backoff plus ±20 % jitter between attempts.
|
||||
- Because delivery blocks the sink's run loop during an outage, back-pressure
|
||||
surfaces as a full input queue and is counted by the pipeline as
|
||||
`total_dropped_by_sink`.
|
||||
- With TLS, dial and handshake are bounded together by
|
||||
`dial_timeout_ms` + 10 s.
|
||||
- Events arriving without a structured entry are wrapped from the formatted
|
||||
payload and counted in `synthesized`.
|
||||
|
||||
**Statistics**: `target`, `node`, `tls`, `connected`, `reconnects`,
|
||||
`write_errors`, `synthesized`.
|
||||
|
||||
---
|
||||
|
||||
## http_chain
|
||||
|
||||
Batches structured entries as NDJSON and POSTs them to a downstream LogWisp
|
||||
`http_chain` source.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sinks]]
|
||||
id = "to_collector"
|
||||
type = "http_chain"
|
||||
[pipelines.plugin_sinks.config]
|
||||
host = "collector.internal"
|
||||
port = 15802
|
||||
ingest_path = "/ingest"
|
||||
node = "edge-01"
|
||||
buffer_size = 1000
|
||||
max_batch_count = 100
|
||||
max_batch_bytes = 1048576
|
||||
flush_interval_ms = 1000
|
||||
request_timeout_ms = 10000
|
||||
backoff_min_ms = 500
|
||||
backoff_max_ms = 30000
|
||||
|
||||
[pipelines.plugin_sinks.config.tls]
|
||||
enabled = true
|
||||
ca_file = "/etc/logwisp/tls/ca.crt"
|
||||
cert_file = "/etc/logwisp/tls/client.crt"
|
||||
key_file = "/etc/logwisp/tls/client.key"
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | **required** | Downstream host |
|
||||
| `port` | int | **required** | Downstream port |
|
||||
| `ingest_path` | string | `/ingest` | Endpoint path; must start with `/` |
|
||||
| `node` | string | `os.Hostname()` | Origin label stamped on first-hop entries |
|
||||
| `buffer_size` | int | `1000` | Sink input queue depth |
|
||||
| `max_batch_count` | int | `100` | Flush after this many entries |
|
||||
| `max_batch_bytes` | int | `1048576` | Flush after this many bytes (1 MiB) |
|
||||
| `flush_interval_ms` | int | `1000` | Flush after this long |
|
||||
| `request_timeout_ms` | int | `10000` | Covers dial, write, and response |
|
||||
| `backoff_min_ms` | int | `500` | Retry backoff floor |
|
||||
| `backoff_max_ms` | int | `30000` | Retry backoff ceiling |
|
||||
| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity |
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Delivery is at-least-once per batch: a retried batch can be delivered twice if
|
||||
the first attempt succeeded but the response was lost.
|
||||
- Retries apply to transport errors, `408`, `429`, and `5xx`. Any other
|
||||
non-2xx response is treated as permanent, and the batch is dropped and counted
|
||||
in `dropped_batches`.
|
||||
- HTTP/2 is off by design; batched NDJSON POSTs gain nothing from it.
|
||||
- On shutdown a single best-effort flush of the pending batch is attempted.
|
||||
|
||||
**Statistics**: `target`, `node`, `tls`, `batches_sent`, `request_errors`,
|
||||
`dropped_batches`, `synthesized`.
|
||||
|
||||
---
|
||||
|
||||
## Sink Statistics
|
||||
|
||||
All sinks track:
|
||||
- Total entries processed
|
||||
- Active connections
|
||||
- Failed sends
|
||||
- Retry attempts
|
||||
- Last processed timestamp
|
||||
Every sink reports: `id`, `type`, `total_processed`, `active_connections`,
|
||||
`start_time`, `last_processed`, and a type-specific `details` map.
|
||||
|
||||
+203
-61
@@ -1,107 +1,249 @@
|
||||
# Input Sources
|
||||
|
||||
LogWisp sources monitor various inputs and generate log entries for pipeline processing.
|
||||
|
||||
## Source Types
|
||||
|
||||
### Directory Source
|
||||
|
||||
Monitors a directory for log files matching a pattern. (type: `file`)
|
||||
Sources produce `core.LogEntry` values for a pipeline. Every source is declared
|
||||
as a `[[pipelines.plugin_sources]]` entry with an `id`, a `type`, and a
|
||||
type-specific `config` table.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "file_in"
|
||||
id = "app_logs"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "/var/log/myapp"
|
||||
pattern = "*.log" # Glob pattern
|
||||
check_interval_ms = 100 # Poll interval
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
Registered types: `file`, `console`, `random`, `null`, `tcp_chain`,
|
||||
`http_chain`.
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `directory` | string | Required | Directory to monitor |
|
||||
| `pattern` | string | "*" | File pattern (glob) |
|
||||
| `check_interval_ms` | int | 100 | File check interval in milliseconds |
|
||||
Publication from any source is non-blocking. When a subscriber channel is full
|
||||
the entry is dropped and counted in `dropped_entries`.
|
||||
|
||||
**Features:**
|
||||
- Automatic rotation detection (inode + size tracking)
|
||||
- In-memory position tracking; on restart, monitoring resumes from the current end of each file (offsets are not persisted)
|
||||
- Concurrent file monitoring
|
||||
- Pattern-based file selection
|
||||
---
|
||||
|
||||
### Stdin Source
|
||||
## file
|
||||
|
||||
Reads log entries from standard input.
|
||||
Tails every file in a directory whose name matches a glob.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "console_in"
|
||||
id = "app_logs"
|
||||
type = "file"
|
||||
[pipelines.plugin_sources.config]
|
||||
directory = "/var/log/myapp"
|
||||
pattern = "*.log"
|
||||
check_interval_ms = 100
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `directory` | string | **required** | Directory to scan; not recursive |
|
||||
| `pattern` | string | `*` | Glob over filenames; `*` and `?` only |
|
||||
| `check_interval_ms` | int | `100` | Directory rescan interval; minimum `10` |
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- `check_interval_ms` governs how often the directory is rescanned for new or
|
||||
removed files. Tailing an already-open file polls on a **fixed 100 ms**
|
||||
interval that this option does not change.
|
||||
- Each matched file gets its own watcher. Watchers for files that disappear are
|
||||
stopped and removed on the next scan.
|
||||
- A new watcher seeks to end-of-file. Positions live in memory only, so a
|
||||
restart resumes from the current end of each file and content written while
|
||||
LogWisp was down is not read.
|
||||
- Rotation is detected from size decrease, modification-time reset, a position
|
||||
beyond end-of-file, or an inode change. An inode change where the new file is
|
||||
already larger than the recorded position is treated as an atomic save, not a
|
||||
rotation, and the position is preserved.
|
||||
- Lines are parsed as JSON when they contain `time`, `level`, `msg`, and
|
||||
`fields` keys; `time` is read as RFC3339Nano. Anything else is kept as plain
|
||||
text with the level inferred from common markers (`[ERROR]`, `WARN:`, and so
|
||||
on).
|
||||
- `Source` is set to the file's base name.
|
||||
|
||||
**Statistics**: per-watcher size, position, entries read, rotation count, and
|
||||
last read time, plus `active_watchers`.
|
||||
|
||||
---
|
||||
|
||||
## console
|
||||
|
||||
Reads newline-delimited entries from standard input.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "stdin"
|
||||
type = "console"
|
||||
[pipelines.plugin_sources.config]
|
||||
buffer_size = 1000
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `buffer_size` | int | 1000 | Internal buffer size |
|
||||
| `buffer_size` | int | `1000` | Subscriber channel depth |
|
||||
|
||||
**Features:**
|
||||
- Line-based processing
|
||||
- Automatic level detection
|
||||
- Non-blocking reads
|
||||
At most **one** instance per pipeline: the type is registered with
|
||||
`MaxInstances: 1`, and a second instance is rejected at pipeline construction.
|
||||
The level is inferred from the line text, and `Source` is set to `console`.
|
||||
|
||||
### Random Source
|
||||
---
|
||||
|
||||
## random
|
||||
|
||||
Synthetic entry generator for development, smoke tests, and sanitizer testing.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "random_in"
|
||||
id = "generator"
|
||||
type = "random"
|
||||
[pipelines.plugin_sources.config]
|
||||
interval_ms = 500
|
||||
jitter_ms = 0
|
||||
format = "txt"
|
||||
length = 20
|
||||
special = false
|
||||
jitter_ms = 0
|
||||
format = "txt"
|
||||
length = 20
|
||||
special = false
|
||||
```
|
||||
|
||||
**Configuration Options:**
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `interval_ms` | int | 500 | Generation interval |
|
||||
| `jitter_ms` | int | 0 | Random jitter interval |
|
||||
| `format` | string | "txt" | "txt", "json", "raw" |
|
||||
| `length` | int | 20 | Log length |
|
||||
| `special` | bool | false | Include special characters |
|
||||
| `interval_ms` | int | `500` | Emission period |
|
||||
| `jitter_ms` | int | `0` | Symmetric jitter; clamped to `interval_ms`, must be non-negative |
|
||||
| `format` | string | `txt` | `raw` (message only), `txt` (bracketed line), `json` (JSON object as the message) |
|
||||
| `length` | int | `20` | Message length in characters |
|
||||
| `special` | bool | `false` | Inject control and non-ASCII characters |
|
||||
|
||||
## Source Statistics
|
||||
`special = true` is the intended way to exercise sanitizer policies: it inserts
|
||||
control bytes and multi-byte Unicode into otherwise ordinary messages. Levels
|
||||
are chosen at random from DEBUG, INFO, WARN, ERROR.
|
||||
|
||||
All sources track:
|
||||
- Total entries received
|
||||
- Dropped entries (buffer full)
|
||||
- Invalid entries
|
||||
- Last entry timestamp
|
||||
- Active connections (network sources)
|
||||
- Source-specific metrics
|
||||
---
|
||||
|
||||
### Null Source
|
||||
## null
|
||||
|
||||
Produces nothing. Useful as a placeholder so a sink-only pipeline satisfies the
|
||||
"at least one source" requirement.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "null_in"
|
||||
id = "void"
|
||||
type = "null"
|
||||
[pipelines.plugin_sources.config]
|
||||
```
|
||||
|
||||
## Buffer Management
|
||||
No options.
|
||||
|
||||
Each source maintains internal buffers:
|
||||
- Default size: 1000 entries
|
||||
- Drop policy when full
|
||||
- Configurable per source
|
||||
- Non-blocking writes
|
||||
---
|
||||
|
||||
## tcp_chain
|
||||
|
||||
Listens for persistent NDJSON streams from upstream LogWisp `tcp_chain` sinks.
|
||||
See [Chaining](chaining.md) for the protocol.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "ingest_tcp"
|
||||
type = "tcp_chain"
|
||||
[pipelines.plugin_sources.config]
|
||||
host = "0.0.0.0"
|
||||
port = 15801
|
||||
buffer_size = 1000
|
||||
max_connections = 0
|
||||
read_timeout_ms = 0
|
||||
hello_timeout_ms = 10000
|
||||
trust_node = true
|
||||
|
||||
[pipelines.plugin_sources.config.tls]
|
||||
enabled = true
|
||||
cert_file = "/etc/logwisp/tls/server.crt"
|
||||
key_file = "/etc/logwisp/tls/server.key"
|
||||
client_auth = true
|
||||
client_ca_file = "/etc/logwisp/tls/client-ca.crt"
|
||||
min_version = "1.3"
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | `0.0.0.0` | Bind address; IPv4 only |
|
||||
| `port` | int | **required** | Listen port, 1–65535 |
|
||||
| `buffer_size` | int | `1000` | Subscriber channel depth |
|
||||
| `max_connections` | int | `0` | Concurrent connection cap; `0` = unlimited |
|
||||
| `read_timeout_ms` | int | `0` | Per-connection idle read deadline; `0` = none |
|
||||
| `hello_timeout_ms` | int | `10000` | Deadline for the hello preamble |
|
||||
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address |
|
||||
| `tls` | table | — | Listener TLS; see [Security](security.md) |
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- TLS handshakes run explicitly with a 10 s bound before the preamble is read,
|
||||
after the `max_connections` admission check.
|
||||
- A connection is rejected if the first line is not a valid hello with a
|
||||
matching protocol version.
|
||||
- Each accepted connection gets a session recording the remote address, node
|
||||
label, and — under TLS — `tls` and `tls_peer_cn`.
|
||||
- A malformed entry line increments `parse_errors` and is skipped; the
|
||||
connection survives. A line over 1 MiB is a protocol violation and terminates
|
||||
the connection.
|
||||
|
||||
**Statistics**: `active_connections`, `rejected_conns`, `parse_errors`,
|
||||
`tls_handshake_errors`, `trust_node`.
|
||||
|
||||
---
|
||||
|
||||
## http_chain
|
||||
|
||||
Accepts NDJSON batches POSTed by upstream LogWisp `http_chain` sinks.
|
||||
|
||||
```toml
|
||||
[[pipelines.plugin_sources]]
|
||||
id = "ingest_http"
|
||||
type = "http_chain"
|
||||
[pipelines.plugin_sources.config]
|
||||
host = "0.0.0.0"
|
||||
port = 15802
|
||||
ingest_path = "/ingest"
|
||||
buffer_size = 1000
|
||||
max_body_bytes = 8388608
|
||||
read_timeout_ms = 30000
|
||||
trust_node = true
|
||||
|
||||
[pipelines.plugin_sources.config.tls]
|
||||
enabled = true
|
||||
cert_file = "/etc/logwisp/tls/server.crt"
|
||||
key_file = "/etc/logwisp/tls/server.key"
|
||||
client_auth = true
|
||||
client_ca_file = "/etc/logwisp/tls/client-ca.crt"
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `host` | string | `0.0.0.0` | Bind address; IPv4 only |
|
||||
| `port` | int | **required** | Listen port |
|
||||
| `ingest_path` | string | `/ingest` | Endpoint path; must start with `/` |
|
||||
| `buffer_size` | int | `1000` | Subscriber channel depth |
|
||||
| `max_body_bytes` | int | `8388608` | Per-request body cap (8 MiB) |
|
||||
| `read_timeout_ms` | int | `30000` | Full request read deadline |
|
||||
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address |
|
||||
| `tls` | table | — | Listener TLS |
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Only `POST` to `ingest_path` is routed; other methods get `405` with an
|
||||
`Allow` header, and other paths get `404`.
|
||||
- A missing or mismatched `X-Logwisp-Protocol` header is rejected with `400`.
|
||||
- Batch acceptance is atomic: entries are published only after the body reads
|
||||
cleanly end to end. A transfer error rejects the whole batch (`400`, or `413`
|
||||
when the body cap is hit) so the sender retries it. A malformed *line* inside
|
||||
an otherwise clean transfer is skipped and counted in `parse_errors`.
|
||||
- Success is `204 No Content` with `X-Logwisp-Accepted` set to the number of
|
||||
entries ingested.
|
||||
- Sessions are cached per remote host + declared node and recreated after idle
|
||||
expiry.
|
||||
|
||||
**Statistics**: `total_requests`, `rejected_requests`, `parse_errors`,
|
||||
`cached_sessions`, `trust_node`.
|
||||
|
||||
---
|
||||
|
||||
## Source Statistics
|
||||
|
||||
Every source reports: `id`, `type`, `total_entries`, `dropped_entries`,
|
||||
`start_time`, `last_entry_time`, and a type-specific `details` map. These appear
|
||||
in the status reporter output and in the `http` sink's status endpoint.
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
module logwisp
|
||||
|
||||
go 1.26.0
|
||||
go 1.26.5
|
||||
|
||||
require (
|
||||
github.com/lixenwraith/config v0.1.1-0.20260712172228-ccd280ba6a98
|
||||
github.com/lixenwraith/log v0.1.1-0.20260717175128-82eea9846ccd
|
||||
github.com/lixenwraith/log v0.1.1-0.20260801090951-2c40643523b3
|
||||
)
|
||||
|
||||
require (
|
||||
|
||||
@@ -6,12 +6,14 @@ github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPE
|
||||
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
||||
github.com/lixenwraith/config v0.1.1-0.20260712172228-ccd280ba6a98 h1:VEFo2WsYgM5YYfzAfvXRKigCWr3zEz6h4M59N5TXMpk=
|
||||
github.com/lixenwraith/config v0.1.1-0.20260712172228-ccd280ba6a98/go.mod h1:J9ydxY7he4Dz+S59xKs/kDQ2YAv+zOLF05b3Rr2/ogE=
|
||||
github.com/lixenwraith/log v0.1.1-0.20260717175128-82eea9846ccd h1:06Rk4DLvJW1kdeclH5HwywizgtMI64PHNE/2sBeuiwA=
|
||||
github.com/lixenwraith/log v0.1.1-0.20260717175128-82eea9846ccd/go.mod h1:2+qURSVdWcX7REFH+3jUtIbtc+NtsAPecSvXLeGyK6U=
|
||||
github.com/lixenwraith/log v0.1.1-0.20260724174821-e688c5a07207 h1:EnDMcnFkgTwTGo1ojoJ85/b6aH3yxlBUAg6AefaBcrI=
|
||||
github.com/lixenwraith/log v0.1.1-0.20260724174821-e688c5a07207/go.mod h1:fvGT3IxlJQLJ1is4OP8Iet8gVzTJTGcuT90FFB2WBtc=
|
||||
github.com/lixenwraith/log v0.1.1-0.20260801090951-2c40643523b3 h1:5ojkxyuOKiUBRYhqPqKJS1lrqkjckYqYMNGBB49kdY0=
|
||||
github.com/lixenwraith/log v0.1.1-0.20260801090951-2c40643523b3/go.mod h1:fvGT3IxlJQLJ1is4OP8Iet8gVzTJTGcuT90FFB2WBtc=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
|
||||
Reference in New Issue
Block a user