A pipeline-based log transport and processing system written in Go. LogWisp collects log entries from files, stdin, or other LogWisp nodes; rate-limits, filters, and formats them; and distributes them to files, consoles, live network streams, or downstream LogWisp nodes.

Features

Pipeline

Independent pipelines, each sources → flow → sinks, running concurrently in one process
Fan-in and fan-out: many sources and many sinks per pipeline
Never blocks: a stalled sink drops its own events and is counted, rather than stalling the pipeline or its sibling sinks
Hot reload via SIGHUP/SIGUSR1 or a config file watch, with the new configuration validated before the old service is torn down

Inputs

file (directory tail with rotation detection and JSON line parsing), console (stdin), random (synthetic generator), null, and the chain ingest listeners tcp_chain and http_chain.

Outputs

console, file (rotating with retention), http (Server-Sent Events plus a JSON status endpoint), tcp (broadcast server), null, and the chain forwarders tcp_chain and http_chain.

Processing

Filters: chainable include/exclude RE2 patterns with or/and logic
Formatters: raw, txt, and json with selectable sanitizer policies
Rate limiting: token bucket with an optional per-entry size cap
Heartbeats: flow-level keep-alive entries that reach every sink

Chaining

Multi-node topologies over a versioned protocol. Chain links carry the structured entry, not the formatted text, so a relay can filter and reformat as if the entries were local. Entries keep a node label identifying their origin across any number of hops. Chain sinks reconnect automatically with exponential backoff and jitter.

Transport security and authentication

TLS 1.2/1.3 on every network source and sink, listener and dialer alike
Mutual TLS: listeners can require and verify client certificates; dialers can present a client identity
Authorization by certificate identity: an auth block admits named peers (exact or RE2) rather than everything the CA issued, gates the http sink's stream and status endpoints, and lets a dialer pin the server it talks to
Node binding: a chain source can label entries from the sender's certificate instead of from what the sender claims, so origin attribution is not forgeable

See Security for configuration and the exact boundary, and the mTLS authentication design for the rationale and what is deliberately left out. Password, token, and SCRAM authentication were removed during the restructure and are not currently available.

Documentation

Document	Contents
Installation	Building, installing, running as a service
Architecture	Component model, data flow, concurrency, back-pressure
Configuration	TOML structure, precedence, environment and CLI overrides
Sources	Every input plugin and its options
Sinks	Every output plugin and its options
Filters	Pattern-based inclusion and exclusion
Formatters	Output shaping and sanitization
Chaining	Multi-node topologies and the chain wire protocol
Networking	Listeners, dialers, timeouts, connection limits
Security	TLS, mTLS, and peer authorization; threat model and current limits
mTLS Authentication	Design and rationale for certificate-based authorization
CLI	Flags, signals, exit codes
Operations	Running, monitoring, tuning, troubleshooting

A fully annotated configuration covering every option ships as config/logwisp.toml.

Quick Start

make

# logwisp.toml
[[pipelines]]
name = "default"

[pipelines.flow.format]
type = "json"
sanitizer_policy = "json"

[[pipelines.plugin_sources]]
id = "app_logs"
type = "file"
[pipelines.plugin_sources.config]
directory = "/var/log/myapp"
pattern = "*.log"

[[pipelines.plugin_sinks]]
id = "stdout"
type = "console"
[pipelines.plugin_sinks.config]
target = "stdout"

logwisp -c logwisp.toml

Running with no configuration file starts a self-demonstrating pipeline: a synthetic generator writing JSON to stdout.

System Requirements

Operating systems: Linux (kernel 6.10+), FreeBSD (14.0+)
Architecture: amd64
Go: 1.26+ to build from source

Network sources and sinks bind and dial over IPv4 only.

License

BSD 3-Clause License