logwisp/doc/lw.1

.Dd October 3, 2026
.Dt LW 1
.Os
.Sh NAME
.Nm lw
.Nd LogWisp log transport
.Sh SYNOPSIS
.Nm
.Op Fl \-check
.Op Fl c Ar file
.Op Fl \-quiet
.Op Fl \- Ns Ar key Ns = Ns Ar value ...
.Op Fl \-pipeline Ar name
.Op Fl \-source Ar spec ...
.Op Fl \-sink Ar spec ...
.Op Fl \-filter Ar spec ...
.Op Fl \-format Ar spec
.Op Fl \-rate\-limit Ar spec
.Op Fl \-heartbeat Ar spec
.Nm
.Cm auth
.Ar command
.Op Ar flags
.Nm
.Fl \-version
.Nm
.Cm help | Fl h | Fl \-help
.Sh DESCRIPTION
.Nm
runs LogWisp: independent pipelines, each reading entries from sources,
passing them through a flow of rate limiting, filters and formatting, and
writing them to sinks.
Sources tail files, read standard input or accept other LogWisp nodes;
sinks write to the console or files, serve live streams over HTTP or TCP, or
forward to other nodes.
Every network source and sink can use TLS, mutual TLS and
Argon2id-SCRAM password authentication.
.Pp
With no configuration file and no pipeline options,
.Nm
is a filter like
.Xr cat 1 :
it copies standard input to standard output line by line and exits 0 at the
end of input.
This built-in pipeline,
.Dq pipe ,
joins the console source
.Dq stdin
to the console sink
.Dq stdout
in the raw format, without a filter or rate limit; a configuration file that
defines no pipelines runs it too.
Each line is written with one newline, whether it ended in a newline, a
carriage return and newline, or nothing; blank lines are skipped, and a line
over 1 MiB continues in the next one.
When standard input is a terminal,
.Nm
prints a reminder on standard error that Control-D ends the input.
.Pp
A pipeline finishes when all its sources have ended.
Only a console source ends, at the end of standard input; every other
source runs until shutdown.
When every pipeline has finished,
.Nm
shuts down gracefully and exits 0.
Console and file sinks first write what they hold; the
.Cm http
and
.Cm tcp
sinks do not yet, so a client can miss the last lines of a finite input.
Under a supervisor, standard input is usually
.Pa /dev/null :
a console source there ends at once.
.Pp
The console source and sink never drop a line.
A busy pipeline slows the reading of standard input; a console sink whose
output is slow makes its pipeline wait, other sinks included.
On a terminal the console sink writes control characters, escape sequences
included, as the hex of their bytes in angle brackets, such as
.Ql <1b> ,
so that a log line cannot drive the terminal; pipes and files get the bytes
unchanged.
Its
.Cm escape
option sets this:
.Cm auto
.Pq the default ,
.Cm always
or
.Cm never .
.Pp
.Nm
stays in the foreground and logs to standard error; run a configuration as a
service under a supervisor such as
.Xr systemd 1
or
.Xr daemon 8 .
.Pp
The options are as follows:
.Bl -tag -width Ds
.It Fl c Ar file , Fl \-config Ar file , Fl \-config= Ns Ar file
Read the configuration from
.Ar file ,
which must exist.
The last one given wins.
.It Fl \- Ns Ar key Ns = Ns Ar value , Fl \- Ns Ar key Ar value
Set any scalar configuration key by its TOML path, such as
.Fl \-logging.level=debug .
A bare
.Fl \- Ns Ar key
sets a boolean to true.
.It Fl \-quiet
Turn off the log of
.Nm
and the terminal reminder; console sinks still write.
.It Fl \-status_reporter= Ns Ar bool
Log pipeline statistics every 30 seconds at the debug level
.Pq default true with a configuration file, false without .
.It Fl \-auto_reload= Ns Ar bool
Reload when the configuration file changes
.Pq default false .
.It Fl \-logging.output= Ns Ar mode
One of
.Cm stderr
.Pq the default ,
.Cm stdout ,
.Cm split
.Pq warnings and errors to standard error, the rest to standard output ,
.Cm file ,
.Cm all
.Pq split and file
or
.Cm none .
The modes that use standard output mix the log into what a console sink
writes there.
.It Fl \-logging.level= Ns Ar level
One of
.Cm debug , info , warn
or
.Cm error ;
the default is
.Cm info
with a configuration file and
.Cm warn
without.
.It Fl \-check
Build every pipeline and plugin as a start would, loading options, TLS files
and credentials files and printing startup warnings, without binding a port
or opening, reading or creating a log file; print
.Dq configuration ok
and exit 0, or the error and exit 1, or 2 when a named configuration file is
missing
.Pq see Sx EXIT STATUS .
.It Fl \-version
Print the version, commit and build time, and exit.
.It Fl h , Fl \-help , Cm help
Print usage and exit.
.It Fl \-
End option processing.
.El
.Ss Pipeline specifications
Pipelines can be given on the command line or in the environment instead of
the configuration file.
They then replace the file's pipelines and the built-in default; every other
key keeps its usual precedence.
A reload rereads the file but keeps these pipelines.
A pipeline without a source reads standard input with the console source
.Dq stdin ;
one without a sink writes standard output with the console sink
.Dq stdout .
Standard input has one reader: a second console source anywhere in the
configuration fails validation.
.Bl -tag -width Ds
.It Fl \-pipeline Ar name
Start a new pipeline.
Specifications before the first
.Fl \-pipeline
belong to a pipeline named
.Dq cli .
.It Fl \-source Ar spec , Fl \-sink Ar spec , Fl \-filter Ar spec
Add a source, sink or filter; each is repeatable.
.It Fl \-format Ar spec , Fl \-rate\-limit Ar spec , Fl \-heartbeat Ar spec
Set the pipeline's formatter, rate limit and heartbeat.
.El
.Pp
A source, sink, filter or format
.Ar spec
is a type followed by comma-separated
.Ar key Ns = Ns Ar value
pairs; a rate limit or heartbeat
.Ar spec
has the pairs only:
.Bd -literal -offset indent
\-\-source file,directory=/var/log/app,pattern=*.log
\-\-sink http,host=127.0.0.1,port=8080,id=web
\-\-filter include,patterns=ERROR,patterns=WARN
\-\-format json
\-\-rate\-limit rate=100,burst=200,policy=drop
.Ed
.Pp
Keys are dotted paths into the plugin's options, such as
.Cm tls.cert_file .
A repeated key makes a list.
A backslash escapes
.Ql \&, ,
.Ql =
and
.Ql \e
inside a value; quote such a
.Ar spec
so the shell keeps the backslash.
.Cm id= Ns Ar name
names a source or sink; the default is its type, then
.Ar type Ns _2 ,
.Ar type Ns _3
and so on.
Any pipeline flag on the command line makes
.Nm
ignore the pipeline variables in
.Sx ENVIRONMENT .
.Ss Authentication commands
.Nm
.Cm auth
manages SCRAM credentials files and logs in to SCRAM listeners.
Changes to a credentials file apply when the server receives
.Dv SIGHUP .
.Bl -tag -width Ds
.It Cm add-user Fl credentials Ar file Fl user Ar name Oo Fl password-file Ar file Oc Op Fl generate
Add a user, or replace its password.
The password is read from the
.Fl password-file
when it exists; otherwise one is generated and written there, or printed
once to standard output.
.Fl generate
replaces an existing password without reading one.
A missing or empty credentials file is created with mode 0600; an existing
one keeps its owner and mode.
.It Cm remove-user Fl credentials Ar file Fl user Ar name
Remove a user.
The last user cannot be removed.
.It Cm token Fl url Ar https://host:port Fl user Ar name Fl password-file Ar file Oo Ar tls-flags Oc Op Fl unbound
Log in to an
.Cm http
sink or
.Cm http_chain
source and print a bearer token.
.Fl unbound
logs in without channel binding, to an
.Cm http
sink behind a TLS-terminating proxy; the URL may then carry the proxy's path.
.It Cm stream Fl addr Ar host:port Fl user Ar name Fl password-file Ar file Op Ar tls-flags
Log in to a
.Cm tcp
sink and copy its stream to standard output until interrupted.
.El
.Pp
The
.Ar tls-flags
are
.Fl ca-file Ar file
.Pq default: the system roots ,
.Fl server-name Ar name ,
and
.Fl cert-file Ar file
with
.Fl key-file Ar file
for a listener that requires client certificates.
There is no flag to skip server verification.
An IPv6
.Ar host
is written in brackets, as in
.Li [2001:db8::1]:8443 .
.Sh ENVIRONMENT
.Bl -tag -width Ds
.It Ev LOGWISP_ Ns Ar KEY
Any scalar configuration key: its TOML path in upper case with
.Ql \&.
replaced by
.Ql _ ,
as in
.Ev LOGWISP_LOGGING_LEVEL=debug .
Command-line options take precedence, and both take precedence over the
configuration file.
.It Ev LOGWISP_CONFIG_FILE
The configuration file, relative to
.Ev LOGWISP_CONFIG_DIR
when both are set.
.It Ev LOGWISP_CONFIG_DIR
The directory holding
.Pa logwisp.toml .
.It Ev LOGWISP_PIPELINE
The name of the one pipeline the variables below define.
All of these are ignored when a pipeline option is on the command line.
.It Ev LOGWISP_SOURCE , LOGWISP_SINK , LOGWISP_FILTER
Source, sink and filter specifications, in the syntax of the matching
options; a suffix
.Ar _N ,
counting from 1, adds more, as in
.Ev LOGWISP_SINK_1 .
.It Ev LOGWISP_FORMAT , LOGWISP_RATE_LIMIT , LOGWISP_HEARTBEAT
The pipeline's format, rate limit and heartbeat specifications.
.El
.Pp
Secrets never go in the environment: TLS keys, credentials files and
password files are named by path
.Pq Cm tls.key_file , auth.credentials_file , auth.password_file .
.Sh FILES
Without
.Fl c
or the
.Ev LOGWISP_CONFIG_*
variables,
.Nm
reads the first of these that exists:
.Bl -enum -compact
.It
.Pa ~/.config/logwisp/logwisp.toml
.It
.Pa ./logwisp.toml
.El
.Pp
Packages install a sample configuration and service files that pass it with
.Fl c :
.Bl -tag -width Ds
.It Pa /etc/logwisp/logwisp.toml
The configuration on Linux, used by
.Pa logwisp.service .
.It Pa /usr/local/etc/logwisp/logwisp.toml
The configuration on
.Fx ,
installed as
.Pa logwisp.toml.sample
and used by
.Pa /usr/local/etc/rc.d/logwisp .
.It Pa /var/lib/logwisp , /var/db/logwisp
The service's working directory on Linux and on
.Fx .
.El
.Sh EXIT STATUS
.Nm
exits 0 at the end of input, after a clean shutdown,
.Fl \-version
or help; 1 on a configuration, logger or startup error; and 2 when a
configuration file named by
.Fl c
or
.Ev LOGWISP_CONFIG_*
does not exist.
A reader of standard output that goes away, as in
.Ql lw | head ,
ends
.Nm
with
.Dv SIGPIPE ,
as it ends
.Xr cat 1 ;
the shell reports status 141.
.Pp
.Nm
.Cm auth
exits 0 on success or help, 1 on failure, and 2 on a usage error.
.Sh SIGNALS
.Bl -tag -width "SIGINT, SIGTERM"
.It Dv SIGINT , SIGTERM
Shut down gracefully.
.It Dv SIGHUP , SIGUSR1
Reread the configuration file, certificates and credentials, and rebuild
every pipeline.
An invalid configuration leaves the running service untouched.
.El
.Sh EXAMPLES
Copy a file line by line:
.Bd -literal -offset indent
$ lw < app.log > copy.log
.Ed
.Pp
Follow a log and keep only errors and warnings:
.Bd -literal -offset indent
$ tail \-F app.log | lw \-\-filter include,patterns=ERROR,patterns=WARN
.Ed
.Pp
Prefix each line with the time, the level found in it and the source:
.Bd -literal -offset indent
$ tail \-F app.log | lw \-\-format txt
.Ed
.Pp
Serve standard input as a live stream on the loopback address only, since
the
.Cm http
sink listens on every address by default, and read it from another terminal
as server-sent events; a client gets the lines that arrive while it is
connected:
.Bd -literal -offset indent
$ journalctl \-f | lw \-\-sink http,host=127.0.0.1,port=8080
$ curl \-N http://127.0.0.1:8080/stream
.Ed
.Pp
Follow every
.Pa .log
file in a directory, as
.Ql tail \-F
does, and write the new lines unparsed to one file:
.Bd -literal -offset indent
$ lw \-\-source 'file,directory=/var/log/app,pattern=*.log,raw=true' \e
    > all.log
.Ed
.Pp
Check a configuration without binding a port or touching a log file:
.Bd -literal -offset indent
$ lw \-\-check \-c /etc/logwisp/logwisp.toml
.Ed
.Pp
Run a configuration in the foreground with debug logging:
.Bd -literal -offset indent
$ lw \-c /etc/logwisp/logwisp.toml \-\-logging.level=debug
.Ed
.Pp
Create a credentials file the service account can read, add a user,
letting
.Nm
generate the password into a file, then reload the server:
.Bd -literal -offset indent
# install \-m 0600 \-o logwisp \-g logwisp \e
    /dev/null /etc/logwisp/users.toml
# lw auth add-user \-credentials /etc/logwisp/users.toml \e
    \-user alice \-password-file /root/alice.pass
# systemctl reload logwisp
.Ed
.Sh SEE ALSO
.Xr rc.conf 5 ,
.Xr systemd.exec 5 ,
.Xr sysusers.d 5 ,
.Xr tmpfiles.d 5 ,
.Xr daemon 8
.Pp
The documentation installed in
.Pa share/doc/logwisp
under the installation prefix, and
.Lk https://github.com/lixenwraith/logwisp
.Sh SECURITY CONSIDERATIONS
A listener without an
.Cm auth
block serves or accepts logs from anyone who can reach it.
Keep key, credentials and password files at mode 0600, owned by the account
.Nm
runs as; startup warns about world-readable ones.