v0.22.0 several enhancements and fixes for security, network, doc, test, packaging
This commit is contained in:
lixen committed
1 parent
9ab8c356d2
commit
71d1b82f83
102 files changed
+7102
-2178
No files matched your search
@@ -0,0 +1,328 @@
|
||||
.Dd October 2, 2026
|
||||
.Dt LW 1
|
||||
.Os
|
||||
.Sh NAME
|
||||
.Nm lw
|
||||
.Nd LogWisp log transport
|
||||
.Sh SYNOPSIS
|
||||
.Nm
|
||||
.Op Fl \-check
|
||||
.Op Fl c Ar file
|
||||
.Op Fl \-quiet
|
||||
.Op Fl \- Ns Ar key Ns = Ns Ar value ...
|
||||
.Op Fl \-pipeline Ar name
|
||||
.Op Fl \-source Ar spec ...
|
||||
.Op Fl \-sink Ar spec ...
|
||||
.Op Fl \-filter Ar spec ...
|
||||
.Op Fl \-format Ar spec
|
||||
.Op Fl \-rate\-limit Ar spec
|
||||
.Op Fl \-heartbeat Ar spec
|
||||
.Nm
|
||||
.Cm auth
|
||||
.Ar command
|
||||
.Op Ar flags
|
||||
.Nm
|
||||
.Fl \-version
|
||||
.Nm
|
||||
.Cm help | Fl h | Fl \-help
|
||||
.Sh DESCRIPTION
|
||||
.Nm
|
||||
runs LogWisp: independent pipelines, each reading entries from sources,
|
||||
passing them through a flow of rate limiting, filters and formatting, and
|
||||
writing them to sinks.
|
||||
Sources tail files, read standard input or accept other LogWisp nodes;
|
||||
sinks write to the console or files, serve live streams over HTTP or TCP, or
|
||||
forward to other nodes.
|
||||
Every network source and sink can use TLS, mutual TLS and
|
||||
Argon2id-SCRAM password authentication.
|
||||
.Pp
|
||||
.Nm
|
||||
stays in the foreground and logs to standard output; run it under a
|
||||
supervisor such as
|
||||
.Xr systemd 1
|
||||
or
|
||||
.Xr daemon 8 .
|
||||
With no configuration file it runs a demonstration pipeline: a synthetic
|
||||
generator writing JSON to standard output.
|
||||
.Pp
|
||||
The options are as follows:
|
||||
.Bl -tag -width Ds
|
||||
.It Fl c Ar file , Fl \-config Ar file , Fl \-config= Ns Ar file
|
||||
Read the configuration from
|
||||
.Ar file ,
|
||||
which must exist.
|
||||
The last one given wins.
|
||||
.It Fl \- Ns Ar key Ns = Ns Ar value , Fl \- Ns Ar key Ar value
|
||||
Set any scalar configuration key by its TOML path, such as
|
||||
.Fl \-logging.level=debug .
|
||||
A bare
|
||||
.Fl \- Ns Ar key
|
||||
sets a boolean to true.
|
||||
.It Fl \-quiet
|
||||
Suppress console output.
|
||||
.It Fl \-status_reporter= Ns Ar bool
|
||||
Log pipeline statistics every 30 seconds at the debug level
|
||||
.Pq default true .
|
||||
.It Fl \-auto_reload= Ns Ar bool
|
||||
Reload when the configuration file changes
|
||||
.Pq default false .
|
||||
.It Fl \-logging.output= Ns Ar mode
|
||||
One of
|
||||
.Cm file , stdout , stderr , split , all
|
||||
or
|
||||
.Cm none .
|
||||
.It Fl \-logging.level= Ns Ar level
|
||||
One of
|
||||
.Cm debug , info , warn
|
||||
or
|
||||
.Cm error .
|
||||
.It Fl \-check
|
||||
Build every pipeline and plugin as a start would, loading options, TLS files
|
||||
and credentials files and printing startup warnings, without binding a port
|
||||
or opening, reading or creating a log file; print
|
||||
.Dq configuration ok
|
||||
and exit 0, or the error and exit 1, or 2 when a named configuration file is
|
||||
missing
|
||||
.Pq see Sx EXIT STATUS .
|
||||
.It Fl \-version
|
||||
Print the version, commit and build time, and exit.
|
||||
.It Fl h , Fl \-help , Cm help
|
||||
Print usage and exit.
|
||||
.It Fl \-
|
||||
End option processing.
|
||||
.El
|
||||
.Ss Pipeline specifications
|
||||
Pipelines can be given on the command line or in the environment instead of
|
||||
the configuration file.
|
||||
They then replace the file's pipelines and the built-in default; every other
|
||||
key keeps its usual precedence.
|
||||
A reload rereads the file but keeps these pipelines.
|
||||
.Bl -tag -width Ds
|
||||
.It Fl \-pipeline Ar name
|
||||
Start a new pipeline.
|
||||
Specifications before the first
|
||||
.Fl \-pipeline
|
||||
belong to a pipeline named
|
||||
.Dq cli .
|
||||
.It Fl \-source Ar spec , Fl \-sink Ar spec , Fl \-filter Ar spec
|
||||
Add a source, sink or filter; each is repeatable.
|
||||
.It Fl \-format Ar spec , Fl \-rate\-limit Ar spec , Fl \-heartbeat Ar spec
|
||||
Set the pipeline's formatter, rate limit and heartbeat.
|
||||
.El
|
||||
.Pp
|
||||
A source, sink, filter or format
|
||||
.Ar spec
|
||||
is a type followed by comma-separated
|
||||
.Ar key Ns = Ns Ar value
|
||||
pairs; a rate limit or heartbeat
|
||||
.Ar spec
|
||||
has the pairs only:
|
||||
.Bd -literal -offset indent
|
||||
\-\-source file,directory=/var/log/app,pattern=*.log
|
||||
\-\-sink http,host=127.0.0.1,port=8080,id=web
|
||||
\-\-filter include,patterns=ERROR,patterns=WARN
|
||||
\-\-format json
|
||||
\-\-rate\-limit rate=100,burst=200,policy=drop
|
||||
.Ed
|
||||
.Pp
|
||||
Keys are dotted paths into the plugin's options, such as
|
||||
.Cm tls.cert_file .
|
||||
A repeated key makes a list.
|
||||
A backslash escapes
|
||||
.Ql \&, ,
|
||||
.Ql =
|
||||
and
|
||||
.Ql \e
|
||||
inside a value; quote such a
|
||||
.Ar spec
|
||||
so the shell keeps the backslash.
|
||||
.Cm id= Ns Ar name
|
||||
names a source or sink; the default is its type, then
|
||||
.Ar type Ns _2 ,
|
||||
.Ar type Ns _3
|
||||
and so on.
|
||||
Any pipeline flag on the command line makes
|
||||
.Nm
|
||||
ignore the pipeline variables in
|
||||
.Sx ENVIRONMENT .
|
||||
.Ss Authentication commands
|
||||
.Nm
|
||||
.Cm auth
|
||||
manages SCRAM credentials files and logs in to SCRAM listeners.
|
||||
Changes to a credentials file apply when the server receives
|
||||
.Dv SIGHUP .
|
||||
.Bl -tag -width Ds
|
||||
.It Cm add-user Fl credentials Ar file Fl user Ar name Oo Fl password-file Ar file Oc Op Fl generate
|
||||
Add a user, or replace its password.
|
||||
The password is read from the
|
||||
.Fl password-file
|
||||
when it exists; otherwise one is generated and written there, or printed
|
||||
once to standard output.
|
||||
.Fl generate
|
||||
replaces an existing password without reading one.
|
||||
A missing or empty credentials file is created with mode 0600; an existing
|
||||
one keeps its owner and mode.
|
||||
.It Cm remove-user Fl credentials Ar file Fl user Ar name
|
||||
Remove a user.
|
||||
The last user cannot be removed.
|
||||
.It Cm token Fl url Ar https://host:port Fl user Ar name Fl password-file Ar file Oo Ar tls-flags Oc Op Fl unbound
|
||||
Log in to an
|
||||
.Cm http
|
||||
sink or
|
||||
.Cm http_chain
|
||||
source and print a bearer token.
|
||||
.Fl unbound
|
||||
logs in without channel binding, to an
|
||||
.Cm http
|
||||
sink behind a TLS-terminating proxy; the URL may then carry the proxy's path.
|
||||
.It Cm stream Fl addr Ar host:port Fl user Ar name Fl password-file Ar file Op Ar tls-flags
|
||||
Log in to a
|
||||
.Cm tcp
|
||||
sink and copy its stream to standard output until interrupted.
|
||||
.El
|
||||
.Pp
|
||||
The
|
||||
.Ar tls-flags
|
||||
are
|
||||
.Fl ca-file Ar file
|
||||
.Pq default: the system roots ,
|
||||
.Fl server-name Ar name ,
|
||||
and
|
||||
.Fl cert-file Ar file
|
||||
with
|
||||
.Fl key-file Ar file
|
||||
for a listener that requires client certificates.
|
||||
There is no flag to skip server verification.
|
||||
An IPv6
|
||||
.Ar host
|
||||
is written in brackets, as in
|
||||
.Li [2001:db8::1]:8443 .
|
||||
.Sh ENVIRONMENT
|
||||
.Bl -tag -width Ds
|
||||
.It Ev LOGWISP_ Ns Ar KEY
|
||||
Any scalar configuration key: its TOML path in upper case with
|
||||
.Ql \&.
|
||||
replaced by
|
||||
.Ql _ ,
|
||||
as in
|
||||
.Ev LOGWISP_LOGGING_LEVEL=debug .
|
||||
Command-line options take precedence, and both take precedence over the
|
||||
configuration file.
|
||||
.It Ev LOGWISP_CONFIG_FILE
|
||||
The configuration file, relative to
|
||||
.Ev LOGWISP_CONFIG_DIR
|
||||
when both are set.
|
||||
.It Ev LOGWISP_CONFIG_DIR
|
||||
The directory holding
|
||||
.Pa logwisp.toml .
|
||||
.It Ev LOGWISP_PIPELINE
|
||||
The name of the one pipeline the variables below define.
|
||||
All of these are ignored when a pipeline option is on the command line.
|
||||
.It Ev LOGWISP_SOURCE , LOGWISP_SINK , LOGWISP_FILTER
|
||||
Source, sink and filter specifications, in the syntax of the matching
|
||||
options; a suffix
|
||||
.Ar _N ,
|
||||
counting from 1, adds more, as in
|
||||
.Ev LOGWISP_SINK_1 .
|
||||
.It Ev LOGWISP_FORMAT , LOGWISP_RATE_LIMIT , LOGWISP_HEARTBEAT
|
||||
The pipeline's format, rate limit and heartbeat specifications.
|
||||
.El
|
||||
.Pp
|
||||
Secrets never go in the environment: TLS keys, credentials files and
|
||||
password files are named by path
|
||||
.Pq Cm tls.key_file , auth.credentials_file , auth.password_file .
|
||||
.Sh FILES
|
||||
Without
|
||||
.Fl c
|
||||
or the
|
||||
.Ev LOGWISP_CONFIG_*
|
||||
variables,
|
||||
.Nm
|
||||
reads the first of these that exists:
|
||||
.Bl -enum -compact
|
||||
.It
|
||||
.Pa ~/.config/logwisp/logwisp.toml
|
||||
.It
|
||||
.Pa ./logwisp.toml
|
||||
.El
|
||||
.Pp
|
||||
Packages install a sample configuration and service files that pass it with
|
||||
.Fl c :
|
||||
.Bl -tag -width Ds
|
||||
.It Pa /etc/logwisp/logwisp.toml
|
||||
The configuration on Linux, used by
|
||||
.Pa logwisp.service .
|
||||
.It Pa /usr/local/etc/logwisp/logwisp.toml
|
||||
The configuration on
|
||||
.Fx ,
|
||||
installed as
|
||||
.Pa logwisp.toml.sample
|
||||
and used by
|
||||
.Pa /usr/local/etc/rc.d/logwisp .
|
||||
.It Pa /var/lib/logwisp , /var/db/logwisp
|
||||
The service's working directory on Linux and on
|
||||
.Fx .
|
||||
.El
|
||||
.Sh EXIT STATUS
|
||||
.Nm
|
||||
exits 0 after a clean shutdown,
|
||||
.Fl \-version
|
||||
or help; 1 on a configuration, logger or startup error; and 2 when a
|
||||
configuration file named by
|
||||
.Fl c
|
||||
or
|
||||
.Ev LOGWISP_CONFIG_*
|
||||
does not exist.
|
||||
.Pp
|
||||
.Nm
|
||||
.Cm auth
|
||||
exits 0 on success or help, 1 on failure, and 2 on a usage error.
|
||||
.Sh SIGNALS
|
||||
.Bl -tag -width "SIGINT, SIGTERM"
|
||||
.It Dv SIGINT , SIGTERM
|
||||
Shut down gracefully.
|
||||
.It Dv SIGHUP , SIGUSR1
|
||||
Reread the configuration file, certificates and credentials, and rebuild
|
||||
every pipeline.
|
||||
An invalid configuration leaves the running service untouched.
|
||||
.El
|
||||
.Sh EXAMPLES
|
||||
Run a configuration in the foreground with debug logging:
|
||||
.Bd -literal -offset indent
|
||||
$ lw \-c /etc/logwisp/logwisp.toml \-\-logging.level=debug
|
||||
.Ed
|
||||
.Pp
|
||||
Tail a directory and serve it as a live stream, without a file:
|
||||
.Bd -literal -offset indent
|
||||
$ lw \-\-source 'file,directory=/var/log/app,pattern=*.log' \e
|
||||
\-\-sink http,host=127.0.0.1,port=8080
|
||||
.Ed
|
||||
.Pp
|
||||
Create a credentials file the service account can read, add a user,
|
||||
letting
|
||||
.Nm
|
||||
generate the password into a file, then reload the server:
|
||||
.Bd -literal -offset indent
|
||||
# install \-m 0600 \-o logwisp \-g logwisp /dev/null /etc/logwisp/users.toml
|
||||
# lw auth add-user \-credentials /etc/logwisp/users.toml \e
|
||||
\-user alice \-password-file /root/alice.pass
|
||||
# systemctl reload logwisp
|
||||
.Ed
|
||||
.Sh SEE ALSO
|
||||
.Xr rc.conf 5 ,
|
||||
.Xr systemd.exec 5 ,
|
||||
.Xr sysusers.d 5 ,
|
||||
.Xr tmpfiles.d 5 ,
|
||||
.Xr daemon 8
|
||||
.Pp
|
||||
The documentation installed in
|
||||
.Pa share/doc/logwisp
|
||||
under the installation prefix, and
|
||||
.Lk https://github.com/lixenwraith/logwisp
|
||||
.Sh SECURITY CONSIDERATIONS
|
||||
A listener without an
|
||||
.Cm auth
|
||||
block serves or accepts logs from anyone who can reach it.
|
||||
Keep key, credentials and password files at mode 0600, owned by the account
|
||||
.Nm
|
||||
runs as; startup warns about world-readable ones.
|
||||
Reference in new issue
Block a user