logwisp/doc/configuration.md

Configuration Reference

LogWisp is configured with TOML. A complete annotated file listing every option and its default ships as config/logwisp.toml.

Configuration Precedence

Sources are merged in this order, highest priority first:

1. Command-line flags
2. Environment variables
3. Configuration file
4. Built-in defaults

The pipelines array is replaced wholesale, not merged: as soon as your file defines [[pipelines]], the built-in default pipeline (and its default rate limit and formatter) disappears entirely.

File Location

The path is resolved before any other configuration is read:

1. -c <path> on the command line
2. --config=<path> on the command line
3. $LOGWISP_CONFIG_FILE, joined onto $LOGWISP_CONFIG_DIR when both are set
4. $LOGWISP_CONFIG_DIR/logwisp.toml
5. ~/.config/logwisp/logwisp.toml, if it exists
6. ./logwisp.toml

Missing file behaviour differs by how it was chosen. An explicitly requested file that does not exist is a fatal error (exit code 2); a missing discovered default is not an error, and LogWisp starts on built-in defaults.

--config <path> with a space is not recognized as a config path. It is parsed as an unknown flag, warned about, and ignored — LogWisp then silently falls back to ./logwisp.toml. Use -c <path> or --config=<path>.

Global Settings

Setting	Type	Default	Description
quiet	bool	false	Disable all application logging and console diagnostics
status_reporter	bool	true	Emit a periodic status report every 30 s at DEBUG level
auto_reload	bool	false	Watch the config file and reload pipelines on change

--version prints version information and exits; it is not a persistent setting.

Note that status_reporter writes at DEBUG level, so it produces nothing unless logging.level = "debug".

Application Logging

This configures LogWisp's own operational log, not the log data it transports.

[logging]
output = "stdout"      # file | stdout | stderr | split | all | none
level  = "info"        # debug | info | warn | error
format = "txt"         # raw | txt | json
# sanitization = ""    # raw | json | txt | shell

[logging.file]
directory         = "./log"
name              = "logwisp"
max_size_mb       = 100
max_total_size_mb = 1000
retention_hours   = 168.0

Output modes

Mode	Behaviour
file	Files only
stdout	Standard output only
stderr	Standard error only
split	DEBUG/INFO to stdout, WARN/ERROR to stderr
all	Files plus split console
none	No application logging

[logging.file] applies only to the file and all modes.

[logging.console].target is accepted and validated (stdout, stderr, split) but not applied. The console destination is derived from logging.output. The key is retained for compatibility; setting it has no effect.

quiet = true overrides every logging setting and disables both file and console output.

Pipeline Configuration

[[pipelines]]
name = "app"                       # required, unique across pipelines

# --- flow: everything between sources and sinks ---
[pipelines.flow.rate_limit]
rate                 = 1000.0
burst                = 2000.0
policy               = "drop"
max_entry_size_bytes = 65536

[[pipelines.flow.filters]]
type     = "include"
logic    = "or"
patterns = ["ERROR", "WARN"]

[pipelines.flow.format]
type             = "json"
sanitizer_policy = "json"

[pipelines.flow.heartbeat]
enabled     = true
interval_ms = 30000

# --- sources: one or more ---
[[pipelines.plugin_sources]]
id   = "app_logs"                  # unique within the pipeline
type = "file"
[pipelines.plugin_sources.config]
directory = "/var/log/myapp"

# --- sinks: one or more ---
[[pipelines.plugin_sinks]]
id   = "sse"
type = "http"
[pipelines.plugin_sinks.config]
port = 8080

Every source and sink is a plugin instance with three keys:

Key	Meaning
id	Instance identifier, unique within the pipeline; appears in logs and stats
type	Registered plugin type
config	Plugin-specific table; see Sources and Sinks

config_file is reserved on both structures for a future include mechanism and is not implemented.

Flow stages

Block	Optional	Reference
flow.rate_limit	yes	below
flow.filters	yes	Filters
flow.format	yes (defaults to raw)	Formatters
flow.heartbeat	yes	below

Rate limiting

Option	Type	Default	Description
rate	float	0	Entries per second; <= 0 disables the limiter entirely
burst	float	rate	Token bucket capacity
policy	string	pass	pass allows everything through, drop discards over-limit entries
max_entry_size_bytes	int	0	Per-entry byte cap; 0 = unlimited

Two behaviours are easy to trip over:

• The limiter is constructed only when rate > 0. With rate = 0, max_entry_size_bytes is never enforced.
• policy = "pass" short-circuits the whole check, including the size cap. To enforce a size cap you need rate > 0 and policy = "drop".

Heartbeat

Option	Type	Default	Description
enabled	bool	false	Enable heartbeat generation
interval_ms	int	1000	Interval; minimum 100
include_timestamp	bool	false	false formats with level only, no timestamp
include_stats	bool	false	Attach beat_count and measured interval_ms as fields
format	string	txt	txt, json, or raw

Heartbeats are ordinary entries with source heartbeat and level INFO. They are generated after the flow's filter and rate-limit stages, so filters do not suppress them, and they reach every sink in the pipeline.

format = "comment" (SSE comment framing) appears in older documentation and in a code path in the generator, but the validator rejects it and the pipeline fails to start. Use txt, json, or raw.

Environment Variables

Environment overrides are derived from the TOML path: . becomes _ and the result is uppercased.

TOML path	Environment variable
quiet	QUIET
status_reporter	STATUS_REPORTER
logging.level	LOGGING_LEVEL
logging.file.directory	LOGGING_FILE_DIRECTORY

The LOGWISP_ prefix is not currently applied. The configuration loader requests it, but supplying a custom path-to-variable transform replaces the prefixing step rather than composing with it, so LogWisp reads bare QUIET, LOGGING_LEVEL, and so on from the environment. Treat this as current behaviour to be aware of — bare names like QUIET can collide with unrelated variables — rather than as intended design.

The two exceptions are LOGWISP_CONFIG_FILE and LOGWISP_CONFIG_DIR, which are read directly by the path resolver and do carry the prefix.

Only scalar paths that exist in the configuration schema can be set this way. Array elements cannot: PIPELINES_0_NAME has no effect.

Command-Line Overrides

Any scalar configuration path is settable as a flag using its TOML path:

logwisp --logging.level=debug --status_reporter=false
logwisp --logging.level debug          # space form also works
logwisp --quiet                        # bare flag means true

Unrecognized flags are reported on stderr before the logger exists and are then ignored:

Warning: unrecognized flags ignored: [pipelines.0.name]

Array-indexed paths are not settable from the command line. --pipelines.0.name=x, --pipelines.0.plugin_sinks.0.type=null, and similar flags are reported as unrecognized and ignored. Pipelines, sources, sinks, and filters can only be defined in the configuration file. Older documentation claimed otherwise.

Validation

Startup validation is intentionally split.

internal/config validates only global structure:

• at least one pipeline
• unique, non-empty pipeline names
• at least one source and one sink per pipeline
• logging.output, logging.level, logging.format, logging.sanitization, and logging.console.target enum membership

Everything else is validated by the plugin constructor that owns it — port range, required paths, path prefixes, enum values, regex compilation, TLS file loading. A failure there aborts pipeline construction with a message naming the pipeline, plugin id, and offending key.

There is no cross-pipeline port-conflict detection. Two sinks bound to the same port fail at listener bind time, when the pipeline starts.

Hot Reload

auto_reload = true

or send SIGHUP / SIGUSR1.

Reload rebuilds the whole service: a new service is constructed from the new configuration first, and only if that succeeds is the old one shut down. A configuration error therefore leaves the running service untouched.

Reloaded	Not reloaded
Pipelines, sources, sinks	logging.* (applied once at startup)
Filters, formatters, rate limits, heartbeats	quiet
status_reporter	auto_reload (the watcher is not restarted)

Because the rebuild is total, listeners close and reopen and every connected client is disconnected. Chain sinks reconnect on their own backoff schedule.

Type Reference

TOML type	Go type	Command-line / environment form
String	string	Plain text
Integer	int64	Decimal string
Float	float64	Decimal string
Boolean	bool	true / false, or a bare flag for true
Array	[]T	Not settable outside the file
Table	struct	Nested path with . (flags) or _ (environment)