LogWisp Documentation

LogWisp is a pipeline-based log transport and processing system written in Go. It collects log entries from files, stdin, or other LogWisp nodes; rate-limits, filters, and formats them; and distributes them to files, consoles, live network streams, or downstream LogWisp nodes.

Documentation Map

Document	Contents
Installation	Building, installing, and running as a service
Architecture	Component model, data flow, concurrency, back-pressure
Configuration	TOML structure, precedence, environment and CLI overrides
Sources	Every input plugin and its options
Sinks	Every output plugin and its options
Filters	Pattern-based inclusion and exclusion
Formatters	Output shaping and sanitization
Chaining	Multi-node topologies and the chain wire protocol
Networking	Listeners, dialers, timeouts, connection limits
Security	TLS, mTLS, and peer authorization; threat model and current limits
mTLS Authentication	Design and rationale for certificate-based authorization
Password Authentication	Design and rationale for Argon2id-SCRAM authentication; mTLS hardening
CLI	Flags, signals, exit codes, `logwisp auth`
Operations	Running, monitoring, tuning, troubleshooting

A fully annotated configuration covering every option lives at config/logwisp.toml.

Capabilities

Pipeline

Independent named pipelines, each sources → flow → sinks
Fan-in (many sources per pipeline) and fan-out (many sinks per pipeline)
Non-blocking sink dispatch: a stalled sink drops its own events and never stalls the pipeline or its sibling sinks
Hot reload of pipeline configuration via SIGHUP/SIGUSR1 or a file watch

Inputs

file (directory tail with rotation detection), console (stdin), random (synthetic generator), null, and the chain ingest listeners tcp_chain and http_chain.

Outputs

console, file (rotating), http (Server-Sent Events plus a JSON status endpoint), tcp (broadcast server), null, and the chain forwarders tcp_chain and http_chain.

Processing

Token-bucket rate limiting with an optional per-entry size cap
Chainable include/exclude regex filters with or/and logic
raw, txt, and json formatting with selectable sanitizer policies
Optional flow-level heartbeat entries

Transport security and authentication

TLS 1.2/1.3 on every network source and sink, listener and dialer alike
Mutual TLS: listeners can require and verify client certificates; dialers can present a client identity
Authorization by certificate identity, per listener: named peers rather than everything the CA issued, with the http sink's endpoints gated too
Password authentication (Argon2id-SCRAM) bound to the listener's certificate, with bearer tokens on HTTP listeners and a logwisp auth CLI, and browser logins behind a TLS-terminating proxy
Node binding, so a chain source labels entries from the sender's certificate identity or username rather than from what the sender claims

See Security for what each layer does and does not give you.

Quick Start

[[pipelines]]
name = "default"

[pipelines.flow.format]
type = "json"
sanitizer_policy = "json"

[[pipelines.plugin_sources]]
id = "app_logs"
type = "file"
[pipelines.plugin_sources.config]
directory = "/var/log/myapp"
pattern = "*.log"

[[pipelines.plugin_sinks]]
id = "stdout"
type = "console"
[pipelines.plugin_sinks.config]
target = "stdout"

logwisp -c config.toml

System Requirements

Operating systems: Linux (kernel 6.10+), FreeBSD (14.0+)
Architecture: amd64
Go: 1.26+ to build from source

Network sources and sinks bind and dial over IPv4 only.

License

BSD 3-Clause.

4.0 KiB Raw Blame History