Package / arch (push) Canceled after 0s
Package / user (push) Canceled after 0s
Package / deb (push) Canceled after 0s
Package / freebsd (push) Canceled after 0s
Package / image (push) Canceled after 0s
Test / test (push) Canceled after 0s
Test / e2e (push) Canceled after 0s
lwconf: sparse stages, every problem of a part, pastes read by form
963 lines
26 KiB
Groff
963 lines
26 KiB
Groff
.Dd October 14, 2026
|
|
.Dt LW 1
|
|
.Os
|
|
.Sh NAME
|
|
.Nm lw
|
|
.Nd LogWisp log transport
|
|
.Sh SYNOPSIS
|
|
.Nm
|
|
.Op Fl t | Fl \-dump
|
|
.Op Fl q
|
|
.Op Fl c Ar file
|
|
.Op Fl \-color Op Ar when
|
|
.Op Fl \- Ns Ar key Ns = Ns Ar value ...
|
|
.Op Fl p Ar spec
|
|
.Op Fl \-pipeline Ar name
|
|
.Op Fl \-source Ar spec ...
|
|
.Op Fl \-sink Ar spec ...
|
|
.Op Fl \-filter Ar spec ...
|
|
.Op Fl \-format Ar spec
|
|
.Op Fl \-rate\-limit Ar spec
|
|
.Op Fl \-heartbeat Ar spec
|
|
.Nm
|
|
.Fl T | Fl \-tui
|
|
.Op Ar options
|
|
.Nm
|
|
.Cm auth | tls | preset | config
|
|
.Ar command
|
|
.Op Ar flags
|
|
.Nm
|
|
.Fl V | Fl \-version | Fl \-schema
|
|
.Nm
|
|
.Cm help | Fl h | Fl \-help
|
|
.Sh DESCRIPTION
|
|
.Nm
|
|
runs LogWisp: independent pipelines, each reading entries from sources,
|
|
passing them through a flow of rate limiting, filters and formatting, and
|
|
writing them to sinks.
|
|
Sources tail files, read standard input or accept other LogWisp nodes;
|
|
sinks write to the console or files, serve live streams over HTTP or TCP, or
|
|
forward to other nodes.
|
|
Every network source and sink can use TLS, mutual TLS and
|
|
Argon2id-SCRAM password authentication.
|
|
.Pp
|
|
With no configuration file and no pipeline options,
|
|
.Nm
|
|
is a filter like
|
|
.Xr cat 1 :
|
|
it copies standard input to standard output line by line and exits 0 at the
|
|
end of input.
|
|
This built-in pipeline,
|
|
.Dq pipe ,
|
|
joins a console source to a console sink in the raw format, without a filter
|
|
or rate limit; a configuration file that
|
|
defines no pipelines runs it too.
|
|
Each line is written with one newline, whether it ended in a newline, a
|
|
carriage return and newline, or nothing; empty lines are skipped, and a line
|
|
over 1 MiB continues in the next one.
|
|
When standard input is a terminal,
|
|
.Nm
|
|
prints a reminder on standard error that Control-D ends the input.
|
|
.Pp
|
|
A pipeline finishes when all its sources have ended.
|
|
Only a console source ends, at the end of standard input; every other
|
|
source runs until shutdown.
|
|
When every pipeline has finished,
|
|
.Nm
|
|
shuts down gracefully and exits 0.
|
|
Every sink first writes what it holds; a network sink gives its clients or
|
|
its downstream
|
|
.Cm write_timeout_ms
|
|
.Po Cm request_timeout_ms No for Cm http_chain Pc ,
|
|
at most 2 seconds, to take it.
|
|
Under a supervisor, standard input is usually
|
|
.Pa /dev/null :
|
|
a console source there ends at once.
|
|
.Pp
|
|
While their pipeline runs, the console source and sink never drop a line;
|
|
a stop or reload can drop lines still in flight.
|
|
A busy pipeline slows the reading of standard input; a console sink whose
|
|
output is slow makes its pipeline wait, other sinks included.
|
|
On a terminal the console sink writes control characters, escape sequences
|
|
included, as the hex of their bytes in angle brackets, such as
|
|
.Ql <1b> ,
|
|
so that a log line cannot drive the terminal; pipes and files get the bytes
|
|
unchanged.
|
|
Its
|
|
.Cm escape
|
|
option sets this:
|
|
.Cm auto
|
|
.Pq the default ,
|
|
.Cm always
|
|
or
|
|
.Cm never .
|
|
Its
|
|
.Cm color
|
|
option, by default the top-level
|
|
.Cm color ,
|
|
paints the level name in each line: debug cyan, info green, warn yellow,
|
|
error bold red, trace grey, in the 16 colors every terminal and text console
|
|
renders, with the terminal theme choosing the shades.
|
|
.Pp
|
|
.Nm
|
|
stays in the foreground and logs to standard error; run a configuration as a
|
|
service under a supervisor such as
|
|
.Xr systemd 1
|
|
or
|
|
.Xr daemon 8 .
|
|
.Pp
|
|
Every option has a long form, and the frequent ones a one-letter short form
|
|
that stands for it in every command that has it.
|
|
A value follows as the next argument or after
|
|
.Ql = ;
|
|
a value that starts with
|
|
.Ql - ,
|
|
and a boolean key's, follows
|
|
.Ql = ;
|
|
.Fl t , Fl \-dump , Fl T , Fl \-schema
|
|
and
|
|
.Fl V
|
|
take none.
|
|
Short options do not combine.
|
|
.Nm
|
|
takes options only: an unknown option, or a word no option takes, is a usage
|
|
error.
|
|
The options of
|
|
.Nm
|
|
join words with
|
|
.Ql - ;
|
|
a configuration or preset key keeps its TOML spelling.
|
|
The commands also take a long option after a single dash.
|
|
.Pp
|
|
The options are as follows:
|
|
.Bl -tag -width Ds
|
|
.It Fl c Ar file , Fl \-config Ar file , Fl \-config= Ns Ar file
|
|
Read the configuration from
|
|
.Ar file ,
|
|
which must exist; a relative path that climbs above the current directory
|
|
with
|
|
.Ql ..
|
|
is refused, so give such a path absolute.
|
|
The last one given wins.
|
|
.It Fl t , Fl \-check
|
|
Build every pipeline and plugin as a start would, loading options, TLS files
|
|
and credentials files and printing startup warnings, without binding a port
|
|
or opening, reading or creating a log file; print
|
|
.Dq configuration ok
|
|
and exit 0.
|
|
Otherwise print every key's problem of each setting, flow stage, source,
|
|
sink and pipeline, and the first broken rule between a table's keys, a line
|
|
each named by its key's path, or else the first file that fails to open, and
|
|
exit 1, or 2 when a named configuration file is missing
|
|
.Pq see Sx EXIT STATUS .
|
|
.It Fl \-dump
|
|
Print the effective configuration, with the options, environment and
|
|
presets resolved into it, as a file
|
|
.Fl c
|
|
reads back, and exit.
|
|
.It Fl T , Fl \-tui
|
|
Compose pipelines on a full-screen terminal, starting from the ones
|
|
.Nm
|
|
would run with the other options given, or from a preset when that is only
|
|
the built-in standard input to standard output pipe.
|
|
On leaving, run them with the other options, as
|
|
.Fl \-check
|
|
and
|
|
.Fl \-dump
|
|
also do, or print them as a command line, environment variables or a
|
|
configuration file's pipelines.
|
|
On the screen,
|
|
.Ic t
|
|
runs them in place, for real: listeners bind, dialers connect, file sinks
|
|
write.
|
|
A live view then shows each entry with its fate, passed or the flow stage
|
|
that dropped it (dropped ones dimmed when all show), and the log of
|
|
.Nm
|
|
from warnings up;
|
|
.Ic i
|
|
shows an entry full screen and
|
|
.Ic e
|
|
edits the stage that dropped it.
|
|
A console sink writing to the terminal runs as a null sink there, and a
|
|
console source needs piped input.
|
|
The screen draws on the controlling terminal, so standard input and output
|
|
stay data:
|
|
.Ql lw \-T > pipelines.toml .
|
|
A file is read, never written; the shell empties a redirect's target before
|
|
.Nm
|
|
reads it, so the redirect names a file it does not load.
|
|
.It Fl \-schema
|
|
Print every setting, pipeline flag, flow stage, plugin option, preset and
|
|
output form as one JSON document, with kinds, defaults, rules and help, and
|
|
exit; tools build their option lists from it.
|
|
A plugin's table holds the keys of its side; a flow stage carries
|
|
.Ql on ,
|
|
the values that turn it on; a preset key carries the choices and hint of the
|
|
option it sets.
|
|
.It Fl q , Fl \-quiet
|
|
Turn off the log of
|
|
.Nm
|
|
and the terminal reminder; console sinks still write.
|
|
.It Fl \-color Op Ar when
|
|
Color the level names console sinks write:
|
|
.Cm auto
|
|
.Pq the default: on a terminal, with Ev NO_COLOR empty or unset and Ev TERM not Cm dumb ,
|
|
.Cm always
|
|
.Pq what a bare Fl \-color No means
|
|
or
|
|
.Cm never .
|
|
A console sink's own
|
|
.Cm color
|
|
option takes precedence.
|
|
The codes go into whatever the format makes, JSON included, so output another
|
|
program parses keeps
|
|
.Cm auto
|
|
or
|
|
.Cm never .
|
|
.It Fl V , Fl \-version
|
|
Print the version, commit and build time, and exit.
|
|
.It Fl h , Fl \-help , Cm help
|
|
Print a summary of every option and exit.
|
|
.It Fl \-
|
|
End option processing.
|
|
.El
|
|
.Ss Configuration keys
|
|
Every scalar configuration key is an option by its TOML path,
|
|
.Fl \- Ns Ar key Ns = Ns Ar value
|
|
or
|
|
.Fl \- Ns Ar key Ar value ,
|
|
such as
|
|
.Fl \-logging.level=debug ;
|
|
a bare
|
|
.Fl \- Ns Ar key
|
|
sets a boolean to true.
|
|
Besides the options above, the keys are:
|
|
.Bl -tag -width Ds
|
|
.It Fl \-status_reporter= Ns Ar bool
|
|
Log pipeline statistics every 30 seconds at the debug level
|
|
.Pq default true with a configuration file, false without .
|
|
.It Fl \-auto_reload= Ns Ar bool
|
|
Reload when the configuration file changes
|
|
.Pq default false .
|
|
.It Fl \-logging.output= Ns Ar mode
|
|
Where the log of
|
|
.Nm
|
|
itself goes:
|
|
.Cm stderr
|
|
.Pq the default ,
|
|
.Cm stdout ,
|
|
.Cm split
|
|
.Pq warnings and errors to standard error, the rest to standard output ,
|
|
.Cm file ,
|
|
.Cm all
|
|
.Pq split and file
|
|
or
|
|
.Cm none .
|
|
The modes that use standard output mix the log into what a console sink
|
|
writes there.
|
|
.It Fl \-logging.level= Ns Ar level
|
|
One of
|
|
.Cm debug , info , warn
|
|
or
|
|
.Cm error ;
|
|
the default is
|
|
.Cm info
|
|
with a configuration file and
|
|
.Cm warn
|
|
without.
|
|
.It Fl \-logging.format= Ns Ar format
|
|
One of
|
|
.Cm txt
|
|
.Pq the default ,
|
|
.Cm json
|
|
or
|
|
.Cm raw .
|
|
.It Fl \-logging.sanitization= Ns Ar policy
|
|
How the log's text is cleaned:
|
|
.Cm raw
|
|
.Pq the default: unchanged ,
|
|
.Cm txt
|
|
.Pq Ql < No and non-printable characters as hex ,
|
|
.Cm json
|
|
.Pq control characters escaped
|
|
or
|
|
.Cm shell
|
|
.Pq shell metacharacters, whitespace and control characters removed; lossy .
|
|
.It Fl \-logging.file.directory= Ns Ar dir
|
|
The directory the
|
|
.Cm file
|
|
and
|
|
.Cm all
|
|
modes write in
|
|
.Pq default Pa ./log .
|
|
It must be
|
|
.Nm Ns 's
|
|
alone: the limits below count and delete every
|
|
.Pa .log
|
|
file in it but the one being written, and old ones are also deleted when the
|
|
disk has less than 10 MB free.
|
|
.It Fl \-logging.file.name= Ns Ar name
|
|
The log file's name before
|
|
.Pa .log
|
|
.Pq default Cm logwisp ;
|
|
see
|
|
.Sx FILES .
|
|
.It Fl \-logging.file.max_size_mb= Ns Ar n
|
|
Start a new file when the current one would pass
|
|
.Ar n
|
|
megabytes
|
|
.Pq default 100, 0 for no limit .
|
|
.It Fl \-logging.file.max_total_size_mb= Ns Ar n
|
|
Delete the oldest
|
|
.Pa .log
|
|
files in the directory when all of them pass
|
|
.Ar n
|
|
megabytes
|
|
.Pq default 1000, 0 for no limit .
|
|
.It Fl \-logging.file.retention_hours= Ns Ar hours
|
|
Delete the
|
|
.Pa .log
|
|
files in the directory older than
|
|
.Ar hours
|
|
.Pq default 168, 0 to keep them .
|
|
.It Fl \-logging.console.target= Ns Ar target
|
|
Accepted, as
|
|
.Cm stdout ,
|
|
.Cm stderr
|
|
or
|
|
.Cm split ,
|
|
and ignored:
|
|
.Fl \-logging.output
|
|
chooses the stream.
|
|
.El
|
|
.Ss Pipeline specifications
|
|
Pipelines can be given on the command line or in the environment instead of
|
|
the configuration file.
|
|
They then replace the file's pipelines and the built-in default; every other
|
|
key keeps its usual precedence.
|
|
A reload rereads the file but keeps these pipelines.
|
|
A pipeline without a source reads standard input with a console source;
|
|
one without a sink writes standard output with a console sink.
|
|
Standard input has one reader: a second console source anywhere in the
|
|
configuration fails validation.
|
|
.Bl -tag -width Ds
|
|
.It Fl \-pipeline Ar name
|
|
Start a pipeline named
|
|
.Ar name ,
|
|
which the pipeline options after it configure.
|
|
Those before any
|
|
.Fl \-pipeline ,
|
|
and the variables without
|
|
.Ev LOGWISP_PIPELINE ,
|
|
make a pipeline named after its preset, or
|
|
.Dq cli
|
|
without one.
|
|
.It Fl p Ar spec , Fl \-preset Ar spec
|
|
Begin the pipeline with a preset, as its first option, described in
|
|
.Sx Presets ;
|
|
later options add sources, sinks and filters to it, set its rate limit and
|
|
heartbeat, or replace its format.
|
|
A second preset needs its own
|
|
.Fl \-pipeline .
|
|
.It Fl \-source Ar spec , Fl \-sink Ar spec , Fl \-filter Ar spec
|
|
Add a source, sink or filter; each is repeatable.
|
|
.It Fl \-format Ar spec , Fl \-rate\-limit Ar spec , Fl \-heartbeat Ar spec
|
|
Set the pipeline's formatter, rate limit and heartbeat.
|
|
.El
|
|
.Pp
|
|
A source, sink, filter or format
|
|
.Ar spec
|
|
is a type, then a colon and comma-separated
|
|
.Ar key Ns = Ns Ar value
|
|
pairs; the first
|
|
.Ql \&:
|
|
ends the type, and a type alone takes the defaults.
|
|
A rate limit or heartbeat
|
|
.Ar spec
|
|
has the pairs only:
|
|
.Bd -literal -offset indent
|
|
\-\-source file:directory=/var/log/app,pattern=*.log
|
|
\-\-sink http:host=127.0.0.1,port=8080,id=web
|
|
\-\-filter include:patterns=ERROR,patterns=WARN
|
|
\-\-format json
|
|
\-\-rate\-limit entries_per_second=100,burst_entries=200,policy=drop
|
|
.Ed
|
|
.Pp
|
|
Keys are dotted paths into the plugin's options, such as
|
|
.Cm tls.cert_file .
|
|
A repeated key makes a list.
|
|
A backslash escapes
|
|
.Ql \&, ,
|
|
.Ql =
|
|
and
|
|
.Ql \e
|
|
inside a value; quote such a
|
|
.Ar spec
|
|
so the shell keeps the backslash.
|
|
A colon in a value needs no escape, as in
|
|
.Cm host=::1 .
|
|
.Cm id= Ns Ar name
|
|
names a source or sink; the default is its type, then
|
|
.Ar type Ns _2 ,
|
|
.Ar type Ns _3
|
|
and so on.
|
|
Any pipeline flag on the command line makes
|
|
.Nm
|
|
ignore the pipeline variables in
|
|
.Sx ENVIRONMENT .
|
|
.Ss Presets
|
|
A preset builds a common pipeline from a few keys:
|
|
.Fl \-preset Ar name Ns Op : Ns Ar key Ns = Ns Ar value ... ,
|
|
in the syntax of the other specifications;
|
|
.Nm
|
|
.Cm preset Ar name Op Fl \- Ns Ar key Ar value ...
|
|
prints the pipeline it expands to, and
|
|
.Fl h
|
|
there lists its keys and defaults; its flags are the keys, spelled as in the
|
|
specification, and
|
|
.Fl u
|
|
is
|
|
.Fl \-user .
|
|
A
|
|
.Cm path
|
|
is a file, a directory, whose files are followed, or a glob, and this host
|
|
decides which: an existing directory, or a path ending in a slash, is
|
|
followed whole, any other path is a file name or glob in its directory; for a
|
|
directory not here, as in a
|
|
.Fl \-dump
|
|
for another host, write
|
|
.Ql dir/ .
|
|
Without one,
|
|
.Cm serve
|
|
and
|
|
.Cm edge
|
|
read standard input.
|
|
.Cm from
|
|
.Pq Cm end No or Cm start
|
|
is where reading a file starts.
|
|
.Cm user
|
|
names one SCRAM user, an edge's login or a listener's only user in place of
|
|
.Cm users ;
|
|
.Cm password_file
|
|
holds its password, read once at startup when it is
|
|
.Pa /dev/fd/ Ns Ar N
|
|
or a pipe, and defaults to
|
|
.Pa /dev/tty ,
|
|
which asks at startup; on a listener, Enter alone generates a password and
|
|
shows it there.
|
|
.Bl -tag -width Ds
|
|
.It Cm pipe Op Cm format
|
|
Standard input to standard output, the pipeline
|
|
.Nm
|
|
runs without a configuration.
|
|
.It Cm tail path Op Cm from , format
|
|
Follow files to standard output, like
|
|
.Ql tail \-F .
|
|
.It Cm serve Oo Cm path , from , listen , format , tls , cert , key , issuer_cert , issuer_key , hosts , users , user , password_file , proxy , viewer , allow , deny Oc
|
|
Serve a live HTTP stream on
|
|
.Cm listen
|
|
.Pq default Li 127.0.0.1:8080 ,
|
|
each stream starting with the last 1000 entries.
|
|
.Cm tls
|
|
is
|
|
.Cm off ,
|
|
.Cm self
|
|
.Pq a self-signed certificate made at startup ,
|
|
.Cm issuer
|
|
.Pq one signed by Cm issuer_cert No and Cm issuer_key
|
|
or
|
|
.Cm files
|
|
.Pq Cm cert No and Cm key ;
|
|
.Cm hosts
|
|
adds names and addresses to a certificate made at startup;
|
|
.Cm users
|
|
names a credentials file, or
|
|
.Cm user
|
|
one user, so readers log in with SCRAM, which needs
|
|
.Cm tls
|
|
or
|
|
.Cm proxy .
|
|
A browser at the root gets a live viewer; with a user,
|
|
only behind a TLS-terminating proxy named by
|
|
.Cm proxy ,
|
|
with
|
|
.Cm viewer=true ,
|
|
since a browser cannot bind its login to the TLS channel.
|
|
.Cm allow
|
|
and
|
|
.Cm deny
|
|
list the addresses or CIDRs the listener admits and refuses.
|
|
.It Cm edge to Oo Cm path , from , transport , ca | pin , server_name , user , password_file , cert , key , node Oc
|
|
Forward to an aggregator at
|
|
.Cm to
|
|
over TLS, by
|
|
.Cm tcp_chain
|
|
or, with
|
|
.Cm transport=http ,
|
|
.Cm http_chain ,
|
|
verifying it by
|
|
.Cm ca ,
|
|
by default the system roots, or
|
|
.Cm pin ;
|
|
.Cm server_name
|
|
is the name its certificate carries, by default the host of
|
|
.Cm to .
|
|
It refuses to run without credentials:
|
|
.Cm user ,
|
|
or a client
|
|
.Cm cert
|
|
and
|
|
.Cm key .
|
|
.It Cm aggregator Oo Cm listen , transport , tls , cert , key , issuer_cert , issuer_key , hosts , users , user , password_file , client_ca , out , format , allow , deny Oc
|
|
Receive from edges on
|
|
.Cm listen
|
|
.Pq default Li 0.0.0.0:9000
|
|
over TLS, never off
|
|
.Po default Cm tls=self ;
|
|
.Cm cert , key , issuer_cert , issuer_key
|
|
and
|
|
.Cm hosts
|
|
as for
|
|
.Cm serve Pc ,
|
|
authenticated by the
|
|
.Cm users
|
|
credentials file, the one
|
|
.Cm user ,
|
|
or the client certificates
|
|
.Cm client_ca
|
|
vouches for, and write to standard output or to files in
|
|
.Cm out ;
|
|
.Cm allow
|
|
and
|
|
.Cm deny
|
|
as for
|
|
.Cm serve .
|
|
.El
|
|
.Ss Certificate commands
|
|
.Nm
|
|
.Cm tls
|
|
makes ECDSA P-256 certificates; it never replaces a file, and writes keys
|
|
with mode 0600.
|
|
.Bl -tag -width Ds
|
|
.It Cm ca Fl \-dir Ar dir Oo Fl \-name Ar name Oc Op Fl \-days Ar n
|
|
Create a certificate authority,
|
|
.Pa dir/ca.crt
|
|
and
|
|
.Pa dir/ca.key ,
|
|
valid 3650 days.
|
|
.It Cm cert Fl \-ca\-dir Ar dir Fl \-name Ar name Oo Fl \-hosts Ar names Oc Oo Fl \-server Oc Oo Fl \-client Oc Oo Fl \-days Ar n Oc Op Fl \-out Ar dir
|
|
Issue
|
|
.Pa name.crt
|
|
and
|
|
.Pa name.key
|
|
from that authority, valid 397 days and never past it.
|
|
.Fl \-hosts
|
|
.Pq or Fl \-host
|
|
lists the DNS names and addresses a server certificate carries
|
|
.Pq default: Ar name ;
|
|
.Ar name
|
|
is also the subject, the identity
|
|
.Cm auth.type=mtls
|
|
checks.
|
|
One of
|
|
.Fl \-server
|
|
and
|
|
.Fl \-client
|
|
is required; both may be given.
|
|
.El
|
|
.Pp
|
|
A listener with
|
|
.Cm tls.enabled=true
|
|
needs no files:
|
|
.Cm tls.self_signed=true ,
|
|
or
|
|
.Cm tls.issuer_cert_file
|
|
with
|
|
.Cm tls.issuer_key_file ,
|
|
make its certificate at startup, for
|
|
.Cm tls.hosts ,
|
|
its
|
|
.Cm host ,
|
|
this machine's name, localhost and the loopback addresses.
|
|
Its key lasts until
|
|
.Nm
|
|
exits, across reloads.
|
|
A self-signed listener logs the
|
|
.Cm tls.pin_sha256
|
|
that dialers set in place of
|
|
.Cm tls.ca_file ;
|
|
it changes when
|
|
.Nm
|
|
restarts, so a fleet uses an issuer instead.
|
|
.Ss Authentication commands
|
|
.Nm
|
|
.Cm auth
|
|
manages SCRAM credentials files and logs in to SCRAM listeners.
|
|
.Fl u
|
|
is short for
|
|
.Fl \-user .
|
|
Changes to a credentials file apply when the server receives
|
|
.Dv SIGHUP .
|
|
.Bl -tag -width Ds
|
|
.It Cm add-user Fl \-credentials Ar file Fl u Ar name Oo Fl \-password\-file Ar file Oc Op Fl \-generate
|
|
Add a user, or replace its password.
|
|
The password is read from the
|
|
.Fl \-password\-file
|
|
when it exists; otherwise one is generated and written there, or printed
|
|
once to standard output.
|
|
.Fl \-generate
|
|
replaces an existing password without reading one.
|
|
A missing credentials file is created with mode 0600; an existing one keeps
|
|
its owner and mode, including an empty one made beforehand to choose them.
|
|
.It Cm remove-user Fl \-credentials Ar file Fl u Ar name
|
|
Remove a user.
|
|
The last user cannot be removed.
|
|
.It Cm token Fl \-url Ar https://host:port Fl u Ar name Oo Fl \-password\-file Ar file Oc Oo Ar tls-flags Oc Op Fl \-unbound
|
|
Log in to an
|
|
.Cm http
|
|
sink or
|
|
.Cm http_chain
|
|
source and print a bearer token.
|
|
.Fl \-unbound
|
|
logs in without channel binding, to an
|
|
.Cm http
|
|
sink behind a TLS-terminating proxy; the URL may then carry the proxy's path.
|
|
.It Cm stream Fl \-addr Ar host:port Fl u Ar name Oo Fl \-password\-file Ar file Oc Op Ar tls-flags
|
|
Log in to a
|
|
.Cm tcp
|
|
sink and copy its stream to standard output until interrupted.
|
|
.El
|
|
.Pp
|
|
For
|
|
.Cm token
|
|
and
|
|
.Cm stream ,
|
|
.Fl \-password\-file
|
|
defaults to
|
|
.Pa /dev/tty ,
|
|
where the password is asked without echo.
|
|
.Pp
|
|
The
|
|
.Ar tls-flags
|
|
are
|
|
.Fl \-ca\-file Ar file
|
|
.Pq default: the system roots
|
|
or
|
|
.Fl \-pin\-sha256 Ar pin ,
|
|
.Fl \-server\-name Ar name ,
|
|
and
|
|
.Fl \-cert\-file Ar file
|
|
with
|
|
.Fl \-key\-file Ar file
|
|
for a listener that requires client certificates.
|
|
There is no flag to skip server verification.
|
|
An IPv6
|
|
.Ar host
|
|
is written in brackets, as in
|
|
.Li [2001:db8::1]:8443 .
|
|
.Ss Configuration command
|
|
.Bl -tag -width Ds
|
|
.It Cm config init Op Fl \-out Ar file
|
|
Write the annotated configuration
|
|
.Nm
|
|
ships, every option with its default, to
|
|
.Ar file ,
|
|
by default
|
|
.Pa ~/.config/logwisp/logwisp.toml ,
|
|
the first file in
|
|
.Sx FILES .
|
|
An existing file is never replaced.
|
|
.El
|
|
.Sh ENVIRONMENT
|
|
.Bl -tag -width Ds
|
|
.It Ev LOGWISP_ Ns Ar KEY
|
|
Any scalar configuration key: its TOML path in upper case with
|
|
.Ql \&.
|
|
replaced by
|
|
.Ql _ ,
|
|
as in
|
|
.Ev LOGWISP_LOGGING_LEVEL=debug .
|
|
Command-line options take precedence, and both take precedence over the
|
|
configuration file.
|
|
.It Ev LOGWISP_CONFIG_FILE
|
|
The configuration file, relative to
|
|
.Ev LOGWISP_CONFIG_DIR
|
|
when both are set.
|
|
.It Ev LOGWISP_CONFIG_DIR
|
|
The directory holding
|
|
.Pa logwisp.toml .
|
|
.It Ev LOGWISP_PIPELINE
|
|
The name of the one pipeline the variables below define.
|
|
.It Ev LOGWISP_PRESET
|
|
The preset that begins it.
|
|
All of these are ignored when a pipeline option is on the command line.
|
|
.It Ev LOGWISP_SOURCE , LOGWISP_SINK , LOGWISP_FILTER
|
|
Source, sink and filter specifications, in the syntax of the matching
|
|
options; a suffix
|
|
.Ar _N ,
|
|
counting from 1, adds more, as in
|
|
.Ev LOGWISP_SINK_1 .
|
|
.It Ev LOGWISP_FORMAT , LOGWISP_RATE_LIMIT , LOGWISP_HEARTBEAT
|
|
The pipeline's format, rate limit and heartbeat specifications.
|
|
.It Ev NO_COLOR , TERM
|
|
Set to anything but the empty string,
|
|
.Ev NO_COLOR
|
|
turns
|
|
.Cm auto
|
|
color off, as does
|
|
.Ev TERM=dumb ,
|
|
which also keeps the
|
|
.Fl \-tui
|
|
screen from starting.
|
|
.It Ev LC_ALL , LC_CTYPE , LANG
|
|
The first one set names the locale; one other than UTF-8 makes the
|
|
.Fl \-tui
|
|
screen draw in ASCII.
|
|
.It Ev HOME
|
|
Where
|
|
.Pa ~/.config/logwisp/logwisp.toml
|
|
is looked for, and what a leading
|
|
.Ql ~
|
|
names in a plugin's directory and file options, a preset's
|
|
.Cm path
|
|
and
|
|
.Fl \-logging.file.directory .
|
|
.El
|
|
.Pp
|
|
Secrets never go in the environment: TLS keys, credentials files and
|
|
password files are named by path
|
|
.Pq Cm tls.key_file , auth.credentials_file , auth.password_file ,
|
|
or asked on the terminal.
|
|
.Sh FILES
|
|
Without
|
|
.Fl c
|
|
or the
|
|
.Ev LOGWISP_CONFIG_*
|
|
variables,
|
|
.Nm
|
|
reads the first of these that exists:
|
|
.Bl -enum -compact
|
|
.It
|
|
.Pa ~/.config/logwisp/logwisp.toml
|
|
.It
|
|
.Pa ./logwisp.toml
|
|
.El
|
|
.Pp
|
|
.Nm
|
|
.Cm config init
|
|
writes the first.
|
|
A service installation places the annotated configuration, keeping one
|
|
already there, and service files that pass it with
|
|
.Fl c :
|
|
.Bl -tag -width Ds
|
|
.It Pa /etc/logwisp/logwisp.toml
|
|
The configuration on Linux, used by
|
|
.Pa logwisp.service .
|
|
.It Pa /usr/local/etc/logwisp/logwisp.toml
|
|
The configuration on
|
|
.Fx ,
|
|
used by
|
|
.Pa /usr/local/etc/rc.d/logwisp .
|
|
.It Pa /var/lib/logwisp , /var/db/logwisp
|
|
The service's working directory on Linux and on
|
|
.Fx .
|
|
.It Pa ./log/logwisp.log
|
|
The log of
|
|
.Nm
|
|
itself with
|
|
.Fl \-logging.output=file
|
|
or
|
|
.Cm all ,
|
|
in
|
|
.Fl \-logging.file.directory
|
|
and named by
|
|
.Fl \-logging.file.name ;
|
|
a full one is renamed with its time, as
|
|
.Pa logwisp_YYMMDD_HHMMSS.log .
|
|
.El
|
|
.Sh EXIT STATUS
|
|
.Nm
|
|
exits 0 at the end of input, after a clean shutdown, and for
|
|
.Fl \-version ,
|
|
.Fl \-schema ,
|
|
help,
|
|
.Fl \-dump ,
|
|
a
|
|
.Fl \-tui
|
|
screen left by printing or quitting,
|
|
and a configuration
|
|
.Fl \-check
|
|
accepts; 1 on a failure, such as a configuration that does not load, or a
|
|
logger or startup error; and 2 on a usage error or when a configuration file
|
|
named by
|
|
.Fl c
|
|
or
|
|
.Ev LOGWISP_CONFIG_*
|
|
does not exist.
|
|
A reader of standard output that goes away, as in
|
|
.Ql lw | head ,
|
|
ends
|
|
.Nm
|
|
with
|
|
.Dv SIGPIPE ,
|
|
as it ends
|
|
.Xr cat 1 ;
|
|
the shell reports status 141.
|
|
.Pp
|
|
.Nm
|
|
.Cm auth ,
|
|
.Cm tls ,
|
|
.Cm preset
|
|
and
|
|
.Cm config
|
|
do the same: 0 on success or help, 1 on failure, and 2 on a usage error.
|
|
.Sh SIGNALS
|
|
.Bl -tag -width "SIGINT, SIGTERM"
|
|
.It Dv SIGINT , SIGTERM
|
|
Shut down gracefully.
|
|
.It Dv SIGHUP , SIGUSR1
|
|
Reread the configuration file, certificates and credentials, and rebuild
|
|
every pipeline.
|
|
An invalid configuration leaves the running service untouched; one that builds
|
|
but fails to start, as on a port another process holds, leaves nothing running
|
|
until a later reload succeeds.
|
|
Before the service runs,
|
|
.Dv SIGHUP
|
|
is ignored, as under
|
|
.Xr nohup 1 .
|
|
.El
|
|
.Pp
|
|
On the
|
|
.Fl \-tui
|
|
screen,
|
|
.Dv SIGTERM
|
|
and
|
|
.Dv SIGHUP
|
|
close it, stopping what it runs, and
|
|
.Nm
|
|
exits 0.
|
|
.Sh EXAMPLES
|
|
Copy a file line by line:
|
|
.Bd -literal -offset indent
|
|
$ lw < app.log > copy.log
|
|
.Ed
|
|
.Pp
|
|
Follow a log and keep only errors and warnings:
|
|
.Bd -literal -offset indent
|
|
$ tail \-F app.log | lw \-\-filter include:patterns=ERROR,patterns=WARN
|
|
.Ed
|
|
.Pp
|
|
Prefix each line with the time, the level found in it and the source:
|
|
.Bd -literal -offset indent
|
|
$ tail \-F app.log | lw \-\-format txt
|
|
.Ed
|
|
.Pp
|
|
Serve standard input as a live stream on the loopback address only, since
|
|
the
|
|
.Cm http
|
|
sink listens on every IPv4 address by default, and read it from another
|
|
terminal as server-sent events; a client gets the lines that arrive while it
|
|
is connected:
|
|
.Bd -literal -offset indent
|
|
$ journalctl \-f | lw \-\-sink http:host=127.0.0.1,port=8080
|
|
$ curl \-N http://127.0.0.1:8080/stream
|
|
.Ed
|
|
.Pp
|
|
Follow every
|
|
.Pa .log
|
|
file in a directory, as
|
|
.Ql tail \-F
|
|
does, and write the new lines unparsed to one file:
|
|
.Bd -literal -offset indent
|
|
$ lw \-\-source 'file:directory=/var/log/app,pattern=*.log,raw=true' \e
|
|
> all.log
|
|
.Ed
|
|
.Pp
|
|
Follow every file in a directory, with the level names in color on a
|
|
terminal:
|
|
.Bd -literal -offset indent
|
|
$ lw \-\-preset tail:path=/var/log/app
|
|
.Ed
|
|
.Pp
|
|
Print the configuration a set of options makes, to keep as a file:
|
|
.Bd -literal -offset indent
|
|
$ lw \-\-preset 'serve:path=/var/log/app/*,tls=self' \-\-dump > serve.toml
|
|
.Ed
|
|
.Pp
|
|
Ship a directory from an edge to an aggregator: on the aggregator, create
|
|
the authority, its certificate and a user for the edge; copy
|
|
.Pa ca.crt
|
|
and the password file to the edge; then run both:
|
|
.Bd -literal -offset indent
|
|
# lw tls ca \-\-dir /etc/logwisp/pki
|
|
# lw tls cert \-\-ca-dir /etc/logwisp/pki \-\-name agg.example.org \-\-server
|
|
# lw auth add-user \-\-credentials /etc/logwisp/users.toml \e
|
|
\-\-user edge-01 \-\-password-file /root/edge-01.pass
|
|
agg# lw \-\-preset aggregator:tls=files,users=/etc/logwisp/users.toml,\e
|
|
cert=/etc/logwisp/pki/agg.example.org.crt,\e
|
|
key=/etc/logwisp/pki/agg.example.org.key,out=/var/log/edges
|
|
edge# lw \-\-preset edge:path=/var/log/app,to=agg.example.org:9000,\e
|
|
ca=/etc/logwisp/pki/ca.crt,user=edge-01,password_file=/root/edge-01.pass
|
|
.Ed
|
|
.Pp
|
|
A temporary pipe with nothing on disk: the receiver makes a self-signed
|
|
certificate, logs its pin, and generates the password when Enter alone
|
|
answers its prompt; the sender pins the certificate and is asked for the
|
|
password:
|
|
.Bd -literal -offset indent
|
|
recv$ lw \-p aggregator:listen=0.0.0.0:9000,user=pipe,format=raw
|
|
send$ cmd | lw \-p edge:to=RECEIVER:9000,pin=sha256//BASE64,user=pipe
|
|
.Ed
|
|
.Pp
|
|
Write the annotated configuration to the file
|
|
.Nm
|
|
reads without
|
|
.Fl c ,
|
|
edit it, then check it:
|
|
.Bd -literal -offset indent
|
|
$ lw config init
|
|
$ lw \-\-check
|
|
.Ed
|
|
.Pp
|
|
Check a configuration without binding a port or touching a log file:
|
|
.Bd -literal -offset indent
|
|
$ lw \-\-check \-c /etc/logwisp/logwisp.toml
|
|
.Ed
|
|
.Pp
|
|
Run a configuration in the foreground with debug logging:
|
|
.Bd -literal -offset indent
|
|
$ lw \-c /etc/logwisp/logwisp.toml \-\-logging.level=debug
|
|
.Ed
|
|
.Pp
|
|
Create a credentials file the service account can read, add a user,
|
|
letting
|
|
.Nm
|
|
generate the password into a file, then reload the server:
|
|
.Bd -literal -offset indent
|
|
# install \-m 0640 \-o root \-g logwisp \e
|
|
/dev/null /etc/logwisp/users.toml
|
|
# lw auth add-user \-\-credentials /etc/logwisp/users.toml \e
|
|
\-\-user alice \-\-password-file /root/alice.pass
|
|
# systemctl reload logwisp
|
|
.Ed
|
|
.Sh SEE ALSO
|
|
.Xr rc.conf 5 ,
|
|
.Xr systemd.exec 5 ,
|
|
.Xr sysusers.d 5 ,
|
|
.Xr tmpfiles.d 5 ,
|
|
.Xr daemon 8
|
|
.Pp
|
|
The guides to configuration, plugins, security and deployment, installed in
|
|
.Pa share/doc/logwisp/doc
|
|
under the installation prefix and kept at
|
|
.Lk https://github.com/lixenwraith/logwisp/tree/main/doc
|
|
.Sh SECURITY CONSIDERATIONS
|
|
A listener without an
|
|
.Cm auth
|
|
block serves or accepts logs from anyone who can reach it.
|
|
Keep key, credentials and password files unreadable by others: mode 0600,
|
|
owned by the account
|
|
.Nm
|
|
runs as, or 0640, owned by root and a group that account is in; startup
|
|
warns about world-readable ones.
|