logwisp/doc/cli.md

Command Line Interface

lw [options]
lw --check [options]
lw auth <command> [flags]
lw help | -h | --help
lw --version

lw auth manages SCRAM credentials and logs in to scram listeners; see below. There is no certificate-generation command: use openssl or your PKI tooling — see Security.

Options

Any scalar configuration key is settable as a flag using its TOML path; pipelines have their own flags:

--<path>=<value>        e.g. --logging.level=debug
--<path> <value>        e.g. --logging.level debug
--<path>                bare flag, means true

Common

• -c <path>, --config <path>, --config=<path>: the configuration file, default ./logwisp.toml
• --check: build every pipeline and plugin, bind nothing, report and exit; see Usage Patterns
• --quiet: suppress all application output, default false
• --status_reporter=<bool>: periodic status logging, default true
• --auto_reload=<bool>: reload when the configuration file changes, default false
• --version: print the version and exit
• -h, --help, help: print usage and exit

The last file-selection flag wins. -c=<path> also works. A missing or empty path returns an error, and -- ends option parsing.

Logging

• --logging.output: file, stdout, stderr, split, all or none
• --logging.level: debug, info, warn or error
• --logging.format: raw, txt or json
• --logging.sanitization: raw, json, txt or shell
• --logging.file.directory: a path
• --logging.file.name: a string
• --logging.file.max_size_mb, --logging.file.max_total_size_mb: integers
• --logging.file.retention_hours: a float

--logging.console.target is accepted but has no effect; the console destination is derived from --logging.output.

Pipelines

Pipeline flags define whole pipelines. They replace the configuration file's pipelines and the built-in default; every other key keeps its precedence.

• --pipeline NAME starts a pipeline; specs before the first one go to a pipeline named cli
• --source SPEC, --sink SPEC and --filter SPEC add a stage, repeatable
• --format SPEC, --rate-limit SPEC and --heartbeat SPEC, once per pipeline
• both --flag SPEC and --flag=SPEC work; -- ends options

A SPEC is a comma-separated list:

• source, sink, filter and format specs start with a TYPE
  • the plugin type: Sources, Sinks
  • include or exclude: Filters
  • json, txt or raw: Formatters
• rate-limit and heartbeat specs have no TYPE
• the rest are key=value pairs with the TOML keys of the plugin's config table or of the flow stage
  • a dotted key reaches a nested table: tls.cert_file=..., auth.type=scram
  • a repeated key makes a list: patterns=ERROR,patterns=WARN
  • each value is one list entry, commas included: auth.allow_patterns=^a{1\,3}$ is one regex, auth.allow=a,auth.allow=b two entries
  • values convert to the option's type: port=8080, raw=true
• \ escapes ,, = and \ in a value; any other backslash stays, so regex escapes such as \d pass unchanged
• id=NAME names a source or sink; the default is its TYPE, then TYPE_2, TYPE_3, ...
• naming a stage turns it on: --rate-limit defaults policy to drop, --heartbeat defaults enabled to true

Errors name the flag and the offending part, and nothing starts:

--sink http,port: missing "="
--rate-limit rate=100,polcy=drop: unknown key "polcy"

A misspelled plugin key fails when the plugin is built, before any listener opens: failed to create sink http: ... unknown key "tls.enabeld".

# tail a directory, serve it over SSE
lw --source 'file,directory=/var/log/app,pattern=*.log' \
   --sink http,host=0.0.0.0,port=8080

# errors and warnings as JSON, over TLS with SCRAM logins
lw --source file,directory=/var/log/app \
   --filter include,patterns=ERROR,patterns=WARN --format json \
   --sink "http,port=8443,auth.type=scram,auth.credentials_file=/etc/logwisp/users.toml,\
tls.enabled=true,tls.cert_file=/etc/logwisp/server.crt,tls.key_file=/etc/logwisp/server.key"

# two pipelines
lw --pipeline app --source file,directory=/var/log/app --sink console \
   --pipeline relay --source tcp_chain,port=9000 \
   --sink file,directory=/var/log/relay,name=relay

Environment Variables

Configuration paths map to environment variables by replacing . with _, uppercasing and adding LOGWISP_:

export LOGWISP_QUIET=true
export LOGWISP_LOGGING_LEVEL=debug
export LOGWISP_LOGGING_FILE_DIRECTORY=/var/log/logwisp

Bare names used by older versions are now ignored. Rename them to the prefixed forms when upgrading; see Configuration.

The path resolver reads these variables directly:

• LOGWISP_CONFIG_FILE: the configuration file path, joined onto LOGWISP_CONFIG_DIR when both are set
• LOGWISP_CONFIG_DIR: the configuration directory; alone, it implies <dir>/logwisp.toml

Pipeline Variables

One pipeline can come from the environment, with the SPEC syntax. Any pipeline flag on the command line makes LogWisp ignore all of them.

• LOGWISP_PIPELINE: the pipeline's name, default cli
• LOGWISP_SOURCE, LOGWISP_SINK, LOGWISP_FILTER: one stage each
  • LOGWISP_SOURCE_1 .. LOGWISP_SOURCE_N add more, after the unnumbered one in numeric order; likewise LOGWISP_SINK_N and LOGWISP_FILTER_N
• LOGWISP_FORMAT, LOGWISP_RATE_LIMIT, LOGWISP_HEARTBEAT
• an empty variable counts as unset

A container needs no configuration file:

docker run --rm -p 8080:8080 -v /var/log/app:/logs:ro \
  -e LOGWISP_PIPELINE=app \
  -e LOGWISP_SOURCE='file,directory=/logs,pattern=*.log' \
  -e LOGWISP_FILTER='exclude,patterns=DEBUG' \
  -e LOGWISP_SINK='http,host=0.0.0.0,port=8080' \
  -e LOGWISP_SINK_1=console \
  logwisp

Precedence

• Pipelines
  • pipeline flags, if any are given
  • else pipeline variables, if any are set
  • else the configuration file's [[pipelines]]
  • else the built-in default
• Every other key
  1. command-line flags
  2. environment variables
  3. configuration file
  4. built-in defaults

A reload rereads the file and keeps the command-line or environment pipelines.

Signals

• SIGINT, SIGTERM: graceful shutdown
• SIGHUP, SIGUSR1: reload the configuration

SIGHUP is ignored during startup, before the signal handler is installed, so LogWisp survives a terminal hang-up like nohup. Once running, it triggers a reload rather than terminating.

Reload rebuilds the whole service. A configuration error leaves the running service untouched; see Configuration.

Exit Codes

• 0: clean shutdown, --version / --help, or a valid --check
• 1: general error: a configuration load or validation failure (--check included), a logger init failure, a service bootstrap failure
• 2: an explicitly requested configuration file is not found

Exit code 2 applies only when the file was named explicitly (-c, --config=, or the LOGWISP_CONFIG_* variables). A missing discovered default is not an error, and LogWisp starts on built-in defaults.

Built-in Defaults

With no configuration file present, LogWisp runs one pipeline named default_pipeline: a random source with special = true, JSON formatting, a rate limit of 5 entries/second with a burst of 10 and policy = "drop", and a console sink on stdout. It is a self-demonstrating idle mode, not a useful production configuration.

As soon as the file defines [[pipelines]], or pipeline flags or variables are given, that entire default pipeline — rate limit included — is replaced rather than merged.

Usage Patterns

Development

# verbose, everything to stderr
lw -c dev.toml --logging.output=stderr --logging.level=debug

# no config at all: synthetic generator to stdout
lw

Configuration check

lw --check validates without running: it builds every pipeline and plugin as a start would (options, TLS files, credentials files, startup warnings), but binds no port and opens, reads or creates no log file; it prints configuration ok and exits 0, or the error and exits 1 (2 when a named file is missing, see Exit Codes):

lw --check -c /etc/logwisp/logwisp.toml
lw --check --source file,directory=/var/log/app --sink http,port=8080

Production

lw -c /etc/logwisp/logwisp.toml --logging.output=file

Run under a supervisor (systemd, rc.d) rather than backgrounding it — there is no --background flag; earlier releases had one and it was removed. See Installation.

Reload

kill -HUP  $(pidof lw)
kill -USR1 $(pidof lw)

lw auth

Manages the credentials files of auth.type = "scram" listeners and logs in to them as a viewer. See Password Authentication.

lw auth add-user    -credentials FILE -user NAME [-password-file FILE] [-generate]
lw auth remove-user -credentials FILE -user NAME
lw auth token       -url https://HOST:PORT[/PATH] -user NAME -password-file FILE
                    [-unbound] [TLS flags]
lw auth stream      -addr HOST:PORT -user NAME -password-file FILE [TLS flags]

• add-user: adds a user, or replaces its password
• remove-user: removes a user; refuses the last one
• token: logs in to an http sink or http_chain source and prints a bearer token
• stream: logs in to a tcp sink and copies its stream to stdout until interrupted

lw auth <command> -h lists a command's flags. The exit status is 0 on success, 1 on failure and 2 on a usage error.

add-user creates the credentials file, with a fresh decoy_key, when it does not exist or is empty (create it empty first to choose its owner). The password comes from -password-file when that file exists (at least 8 bytes; one trailing line break is trimmed). Otherwise a random 26-character password (130 bits) is generated and written to -password-file, or printed once to stdout when there is none. Replacing an existing user's password needs an existing -password-file or -generate, so a mistyped path cannot replace a working password; -generate always generates, overwriting -password-file. New users take the file's existing Argon2 profile.

Both file commands rewrite atomically (a temporary file in the same directory as the file, or a symlink's target, then a rename), create files 0600, keep an existing file's mode and owner, and refuse to write anything the daemon would not load. A change that cannot keep the owner fails: run it as root or as the owner. A symlink to a missing file is refused: create the target first. Neither touches a running LogWisp: send SIGHUP, since auto_reload does not watch the credentials file.

token and stream build the same TLS and SCRAM client as a chain sink. TLS flags: -ca-file (default: system roots), -server-name (default: the host), and -cert-file / -key-file for a listener with tls.client_auth. There is no flag to skip verification: an unverified server could relay the login. Redirects are not followed. -unbound logs in to an http sink behind a TLS-terminating proxy (auth.trusted_proxies), still pinning the proxy's certificate across the two requests; only then may -url carry the path the proxy mounts LogWisp at. stream exits 0 on SIGINT or SIGTERM, and 1 when the server ends the stream (a reload or shutdown).

Addresses, as for the plugins (Networking):

• An IPv6 address goes in brackets: -addr [::1]:PORT, -url https://[::1]:PORT; in a URL a link-local zone is escaped, https://[fe80::1%25eth0]:PORT.
• The dial keeps to the address's family; a hostname resolves.
• -server-name defaults to the host; an address, without brackets or zone, must then be among the certificate's IP SANs.

# listener host: a file the service user can read, users, then apply
install -m 0640 -o root -g logwisp /dev/null /etc/logwisp/users.toml
lw auth add-user -credentials /etc/logwisp/users.toml -user edge-01 \
  -password-file /etc/logwisp/edge-01.pass
lw auth add-user -credentials /etc/logwisp/users.toml -user viewer \
  -password-file viewer.pass
kill -HUP $(pidof lw)

# rotate: new password into the file; SIGHUP, deploy the file, SIGHUP the edge
lw auth add-user -credentials /etc/logwisp/users.toml -user edge-01 \
  -password-file /etc/logwisp/edge-01.pass -generate

# http sink status, with the token kept out of curl's argv
curl --cacert ca.crt -H @<(printf 'Authorization: Bearer %s\n' \
  "$(lw auth token -url https://HOST:PORT -user viewer \
     -password-file viewer.pass -ca-file ca.crt)") https://HOST:PORT/status

# follow a tcp sink
lw auth stream -addr HOST:PORT -user viewer -password-file viewer.pass \
  -ca-file ca.crt