v0.24.0 cli long and short options, http sink viewer and new logo
Test / test (push) Canceled after 0s
Test / e2e (push) Canceled after 0s

This commit is contained in:
lixen committed 2026-10-04 18:28:47 -04:00
1 parent bf410e1225
commit 9e86d7999d
42 files changed
+786 -404

No files matched your search

+1 -1
View File
@@ -31,7 +31,7 @@ streams, or downstream LogWisp nodes.
manual, [`lw.1`](lw.1)
- [Operations](operations.md): running, monitoring, tuning, troubleshooting
- [To Do](todo.md): planned work in priority order: network access control
and the PROXY protocol, packaging
and the PROXY protocol, packaging, smaller follow-ups
A fully annotated configuration covering every option lives at
[`config/logwisp.toml`](../config/logwisp.toml).
+100 -73
View File
@@ -2,15 +2,15 @@
```
lw [options] [< input]
lw -c <file> [options]
lw --check [options]
lw -c|--config FILE [options]
lw -p|--preset NAME[,KEY=VALUE...] [options]
lw -t|--check [options]
lw --dump [options]
lw --preset NAME[,key=value...] [options]
lw auth <command> [flags]
lw tls <command> [flags]
lw preset <name> [flags]
lw auth COMMAND [flags]
lw tls COMMAND [flags]
lw preset NAME [flags]
lw help | -h | --help
lw --version
lw -V | --version
```
Without a configuration file or pipeline options, lw is a filter: stdin to
@@ -25,6 +25,23 @@ manages SCRAM credentials and logs in to `scram` listeners
## Options
- lw's own options have a long form (`--config`), and the frequent ones a
one-letter short form that stands for it in every command that has it:
- `-c` `--config`, `-p` `--preset`, `-q` `--quiet`, `-t` `--check`,
`-V` `--version`, `-h` `--help`
- `-u` `--user` in `lw auth` and `lw preset edge`
- A value follows as the next argument or after `=` (`-c FILE`, `-c=FILE`,
`--config FILE`, `--config=FILE`); a value that starts with `-` follows `=`,
and `--` ends the options.
- Short options do not combine (`-q -t`, not `-qt`) or take an attached value
(`-cFILE`); an unknown single-dash option is an error.
- lw's multiword options join words with `-` (`--rate-limit`,
`--password-file`); configuration keys, spec keys and preset keys keep
TOML's `_` and `.` (`--status_reporter`, `--sink http,tls.cert_file=F`,
`lw preset edge --password_file F`).
- `lw auth`, `lw tls` and `lw preset` also take a long option after one dash
(`-user NAME`), as earlier releases documented.
Any scalar configuration key is settable as a flag using its TOML path;
pipelines have [their own flags](#pipelines):
@@ -36,27 +53,27 @@ pipelines have [their own flags](#pipelines):
### Common
- `-c <path>`, `--config <path>`, `--config=<path>`: the configuration file;
unnamed, `~/.config/logwisp/logwisp.toml` if it exists, else
`./logwisp.toml` ([Configuration](configuration.md#file-location))
- `--check`: build every pipeline and plugin, bind nothing, report and exit;
see [Usage Patterns](#usage-patterns)
- `-c FILE`, `--config FILE`: the configuration file; unnamed,
`~/.config/logwisp/logwisp.toml` if it exists, else `./logwisp.toml`
([Configuration](configuration.md#file-location))
- `-t`, `--check`: build every pipeline and plugin, bind nothing, report and
exit; see [Usage Patterns](#usage-patterns)
- `--dump`: print the effective configuration, flags, environment and presets
resolved into it, as TOML that `-c` reads back, and exit
- `--color [auto|always|never]`: level names in color on console sinks,
default `auto` (a terminal, `NO_COLOR` unset, `TERM` not `dumb`); bare,
`always`. A console sink's own `color` wins ([console sink](sinks.md#console))
- `--quiet`: silence lw's own log and notices, default `false`; pipeline
output still flows
- `--status_reporter=<bool>`: periodic status logging, default `true` with a
- `--color [WHEN]`, WHEN `auto`, `always` or `never`: level names in color on
console sinks, default `auto` (a terminal, `NO_COLOR` unset, `TERM` not
`dumb`); bare, `always`. A console sink's own `color` wins
([console sink](sinks.md#console))
- `-q`, `--quiet`: silence lw's own log and notices, default `false`;
pipeline output still flows
- `--status_reporter=BOOL`: periodic status logging, default `true` with a
configuration file, `false` without
- `--auto_reload=<bool>`: reload when the configuration file changes, default
- `--auto_reload=BOOL`: reload when the configuration file changes, default
`false`
- `--version`: print the version and exit
- `-V`, `--version`: print the version and exit
- `-h`, `--help`, `help`: print usage and exit
The last file-selection flag wins. `-c=<path>` also works. A missing or empty
path returns an error, and `--` ends option parsing.
The last `-c` wins; a missing or empty path is an error.
### Logging
@@ -81,12 +98,11 @@ pipelines and the built-in default; every other key keeps its precedence.
- `--pipeline NAME` starts a pipeline; specs before the first one go to a
pipeline named `cli`, or after the preset that starts them
- `--preset SPEC` starts a pipeline with a [preset](#presets); later flags add
sources, sinks and filters to it, or replace its format, rate limit and
heartbeat
- `-p SPEC`, `--preset SPEC` starts a pipeline with a [preset](#presets);
later flags add sources, sinks and filters to it, or replace its format,
rate limit and heartbeat
- `--source SPEC`, `--sink SPEC` and `--filter SPEC` add a stage, repeatable
- `--format SPEC`, `--rate-limit SPEC` and `--heartbeat SPEC`, once per pipeline
- both `--flag SPEC` and `--flag=SPEC` work; `--` ends options
- a pipeline without `--source` reads stdin (console source `stdin`), one
without `--sink` writes stdout (console sink `stdout`)
- stdin has one reader, so the whole configuration holds at most one console
@@ -124,7 +140,8 @@ A misspelled plugin key fails when the plugin is built, before any listener
opens: `failed to create sink http: ... unknown key "tls.enabeld"`.
```bash
# tail a directory, serve it over SSE on every interface
# tail a directory, serve it over SSE on every interface; a browser at
# http://HOST:8080/ shows it
lw --source 'file,directory=/var/log/app,pattern=*.log' \
--sink http,host=0.0.0.0,port=8080
@@ -146,8 +163,10 @@ Pipelines over stdin and stdout are under [Usage Patterns](#usage-patterns).
`--preset NAME,key=value,...` takes the [SPEC](#pipelines) syntax. A list
key (`hosts`, `proxy`) takes several values, repeated (`hosts=a,hosts=b`) or
`,`-separated; any other key takes one. `lw preset NAME [-key value ...]`
prints the pipeline a preset expands to, ready for a configuration file;
`,`-separated; any other key takes one. `lw preset NAME [--key value ...]`
prints the pipeline a preset expands to, ready for a configuration file; its
flags are the keys, spelled as in the SPEC (`--password_file`), and `-u` is
`--user`;
`lw preset NAME -h` lists its keys and defaults, `lw preset` the presets. An
unknown key fails and lists the valid ones. A `path` is a file, a directory
(its files) or a glob; a preset that takes one reads stdin without it, and
@@ -157,15 +176,17 @@ unknown key fails and lists the valid ones. A `path` is a file, a directory
- `format` (`raw`)
- `tail`: follow files to stdout, like `tail -F`
- `path` (required), `from`, `format` (`raw`)
- `serve`: a live HTTP stream (SSE), an [http sink](sinks.md#http)
- `serve`: a live stream, an [http sink](sinks.md#http): a browser at
`listen` gets the viewer, `/stream` is SSE
- `path`, `from`, `format` (`json`), `listen` (`127.0.0.1:8080`)
- `tls`: `off` (default), `self` (a self-signed certificate made at
startup), `issuer` (one signed by `issuer_cert` and `issuer_key`), or
`files` (`cert` and `key`); `hosts` adds names to a made certificate
- `users`: a credentials file; readers then log in with SCRAM
- `proxy`, `viewer=true`: the built-in login and viewer pages, which work
only behind the TLS-terminating proxies `proxy` names (a browser cannot
bind its login to the TLS channel); there is no standalone viewer
- without `users` the viewer needs no login, over plain http too
- `users`: a credentials file; readers then log in with SCRAM, and browsers
need `proxy` and `viewer=true`: the login page and viewer behind the
TLS-terminating proxies `proxy` names, since a browser cannot bind its
login to the TLS channel
- `edge`: forward to an aggregator over TLS (`tcp_chain`, or `http_chain`
with `transport=http`)
- `to` (required), `path`, `from`, `transport` (`tcp`), `node`
@@ -183,6 +204,9 @@ unknown key fails and lists the valid ones. A `path` is a file, a directory
# follow a directory, level names in color on a terminal
lw --preset tail,path=/var/log/app
# serve it to a browser at http://127.0.0.1:8080/
lw --preset serve,path=/var/log/app
# serve it over HTTPS, self-signed, to SCRAM users; keep the result as a file
lw --preset serve,path=/var/log/app,tls=self,users=/etc/logwisp/users.toml --dump > serve.toml
@@ -281,7 +305,8 @@ service untouched; see [Configuration](configuration.md#hot-reload).
- `0`: the end of input, a clean shutdown (`SIGINT`, `SIGTERM`), `--version` /
`--help`, `--dump`, or a valid `--check`
- `1`: general error: a configuration load or validation failure (`--check`
- `1`: general error: a command-line error (an unknown option, a missing
`-c` path), a configuration load or validation failure (`--check`
included), a logger init failure, a service bootstrap failure
- `2`: an explicitly requested configuration file is not found
- killed by `SIGPIPE` (shell status `141`): the reader of stdout went away, as
@@ -395,13 +420,15 @@ them as a viewer. See
[Password Authentication](security.md#password-authentication-scram).
```
lw auth add-user -credentials FILE -user NAME [-password-file FILE] [-generate]
lw auth remove-user -credentials FILE -user NAME
lw auth token -url https://HOST:PORT[/PATH] -user NAME -password-file FILE
[-unbound] [TLS flags]
lw auth stream -addr HOST:PORT -user NAME -password-file FILE [TLS flags]
lw auth add-user --credentials FILE -u NAME [--password-file FILE] [--generate]
lw auth remove-user --credentials FILE -u NAME
lw auth token --url https://HOST:PORT[/PATH] -u NAME --password-file FILE
[--unbound] [TLS flags]
lw auth stream --addr HOST:PORT -u NAME --password-file FILE [TLS flags]
```
`-u` is short for `--user`.
- `add-user`: adds a user, or replaces its password
- `remove-user`: removes a user; refuses the last one
- `token`: logs in to an `http` sink or `http_chain` source and prints a
@@ -409,18 +436,18 @@ lw auth stream -addr HOST:PORT -user NAME -password-file FILE [TLS flags]
- `stream`: logs in to a `tcp` sink and copies its stream to stdout until
interrupted
`lw auth <command> -h` lists a command's flags. The exit status is `0` on
`lw auth COMMAND -h` lists a command's flags. The exit status is `0` on
success, `1` on failure and `2` on a usage error.
**`add-user`** creates the credentials file, with a fresh `decoy_key`, when it
does not exist or is empty (create it empty first to choose its owner). The
password comes from `-password-file` when that file exists (at least 8 bytes;
password comes from `--password-file` when that file exists (at least 8 bytes;
one trailing line break is trimmed). Otherwise a random
26-character password (130 bits) is generated and written to `-password-file`,
26-character password (130 bits) is generated and written to `--password-file`,
or printed once to stdout when there is none. Replacing an existing user's
password needs an existing `-password-file` or `-generate`, so a mistyped path
cannot replace a working password; `-generate` always generates, overwriting
`-password-file`. New users take the file's existing Argon2 profile.
password needs an existing `--password-file` or `--generate`, so a mistyped path
cannot replace a working password; `--generate` always generates, overwriting
`--password-file`. New users take the file's existing Argon2 profile.
Both file commands rewrite atomically (a temporary file in the same directory
as the file, or a symlink's target, then a rename), create files `0600`, keep an
@@ -431,46 +458,46 @@ Neither touches a running LogWisp: send `SIGHUP`, since `auto_reload` does not
watch the credentials file.
**`token`** and **`stream`** build the same TLS and SCRAM client as a chain
sink. TLS flags: `-ca-file` (default: system roots) or `-pin-sha256` (the
`tls.pin_sha256` a self-signed listener logs), `-server-name` (default: the
host), and `-cert-file` / `-key-file` for a listener with `tls.client_auth`.
sink. TLS flags: `--ca-file` (default: system roots) or `--pin-sha256` (the
`tls.pin_sha256` a self-signed listener logs), `--server-name` (default: the
host), and `--cert-file` / `--key-file` for a listener with `tls.client_auth`.
There is no flag to skip verification: an unverified server could relay the
login. Redirects are not followed. `-unbound` logs in to an `http` sink behind
login. Redirects are not followed. `--unbound` logs in to an `http` sink behind
a TLS-terminating proxy (`auth.trusted_proxies`), still pinning the proxy's
certificate across the two requests; only then may `-url` carry the path the
certificate across the two requests; only then may `--url` carry the path the
proxy mounts LogWisp at. `stream` exits `0` on `SIGINT` or `SIGTERM`, and `1`
when the server ends the stream (a reload or shutdown).
Addresses, as for the plugins ([Networking](networking.md#address-family)):
- An IPv6 address goes in brackets: `-addr [::1]:PORT`,
`-url https://[::1]:PORT`; in a URL a link-local zone is escaped,
- An IPv6 address goes in brackets: `--addr [::1]:PORT`,
`--url https://[::1]:PORT`; in a URL a link-local zone is escaped,
`https://[fe80::1%25eth0]:PORT`.
- The dial keeps to the address's family; a hostname resolves.
- `-server-name` defaults to the host; an address, without brackets or zone,
- `--server-name` defaults to the host; an address, without brackets or zone,
must then be among the certificate's IP SANs.
```bash
# listener host: a file the service user can read, users, then apply
install -m 0640 -o root -g logwisp /dev/null /etc/logwisp/users.toml
lw auth add-user -credentials /etc/logwisp/users.toml -user edge-01 \
-password-file /etc/logwisp/edge-01.pass
lw auth add-user -credentials /etc/logwisp/users.toml -user viewer \
-password-file viewer.pass
lw auth add-user --credentials /etc/logwisp/users.toml --user edge-01 \
--password-file /etc/logwisp/edge-01.pass
lw auth add-user --credentials /etc/logwisp/users.toml --user viewer \
--password-file viewer.pass
kill -HUP $(pidof lw)
# rotate: new password into the file; SIGHUP, deploy the file, SIGHUP the edge
lw auth add-user -credentials /etc/logwisp/users.toml -user edge-01 \
-password-file /etc/logwisp/edge-01.pass -generate
lw auth add-user --credentials /etc/logwisp/users.toml --user edge-01 \
--password-file /etc/logwisp/edge-01.pass --generate
# http sink status, with the token kept out of curl's argv
curl --cacert ca.crt -H @<(printf 'Authorization: Bearer %s\n' \
"$(lw auth token -url https://HOST:PORT -user viewer \
-password-file viewer.pass -ca-file ca.crt)") https://HOST:PORT/status
"$(lw auth token --url https://HOST:PORT --user viewer \
--password-file viewer.pass --ca-file ca.crt)") https://HOST:PORT/status
# follow a tcp sink
lw auth stream -addr HOST:PORT -user viewer -password-file viewer.pass \
-ca-file ca.crt
lw auth stream --addr HOST:PORT --user viewer --password-file viewer.pass \
--ca-file ca.crt
```
## `lw tls`
@@ -479,25 +506,25 @@ Makes ECDSA P-256 certificates for `tls` blocks. It never replaces a file
(remove one to replace it) and writes keys `0600`, certificates `0644`.
```
lw tls ca -dir DIR [-name NAME] [-days 3650]
lw tls cert -ca-dir DIR -name NAME [-host NAME,...] [-server] [-client]
[-days 397] [-out DIR]
lw tls ca --dir DIR [--name NAME] [--days 3650]
lw tls cert --ca-dir DIR --name NAME [--host NAME,...] [--server] [--client]
[--days 397] [--out DIR]
```
- `ca`: `DIR/ca.crt` and `DIR/ca.key`, a CA that signs leaves only (path
length 0). `ca.crt` is what dialers set as `tls.ca_file` and listeners as
`tls.client_ca_file`; with `ca.key` it is a listener's
`tls.issuer_cert_file` and `tls.issuer_key_file`.
- `cert`: `NAME.crt` and `NAME.key`, in `-out` (default: `-ca-dir`).
- `-server` for a listener: `-host` lists the DNS names and IP addresses it
carries, default `NAME`; `-client` for a dialer presenting a certificate;
- `cert`: `NAME.crt` and `NAME.key`, in `--out` (default: `--ca-dir`).
- `--server` for a listener: `--host` lists the DNS names and IP addresses it
carries, default `NAME`; `--client` for a dialer presenting a certificate;
both may be given.
- `NAME` is the subject CN, the identity `auth.type = "mtls"` matches.
- Valid 397 days (the longest browsers accept), never past the CA. It prints
the certificate's `pin_sha256`.
```bash
lw tls ca -dir /etc/logwisp/pki
lw tls cert -ca-dir /etc/logwisp/pki -name agg.example.org -server
lw tls cert -ca-dir /etc/logwisp/pki -name edge-01 -client
lw tls ca --dir /etc/logwisp/pki
lw tls cert --ca-dir /etc/logwisp/pki --name agg.example.org --server
lw tls cert --ca-dir /etc/logwisp/pki --name edge-01 --client
```
+9 -6
View File
@@ -235,19 +235,22 @@ lw --quiet # bare flag means true
```
A bare flag is accepted only for a boolean key: `--logging.file.directory
--quiet` fails with `--logging.file.directory needs a value`. Unrecognized flags,
positional words and single-dash arguments are reported on stderr before the
logger exists and are then ignored:
--quiet` fails with `--logging.file.directory needs a value`. Unrecognized `--`
flags and positional words are reported on stderr before the logger exists and
are then ignored:
```
Warning: unrecognized flags ignored: [pipelines.0.name -q]
Warning: unrecognized flags ignored: [nosuch.key]
```
Single-dash options are lw's own ([CLI](cli.md#options)); an unknown one is
an error.
An empty `LOGWISP_<PATH>` variable counts as unset, as the pipeline variables
do.
Array-indexed paths such as `--pipelines.0.name=x` are unrecognized; whole
pipelines have [their own flags](cli.md#pipelines).
An array-indexed path such as `--pipelines.0.name=x` is an error (exit 1);
whole pipelines have [their own flags](cli.md#pipelines).
## Validation
+2 -2
View File
@@ -147,8 +147,8 @@ already, and the `logs` jail has an address the others reach (`AGG_ADDR`).
4. Check: `jexec logs service logwisp status`, the files in the `logs` jail's
`/var/log/logwisp`, and `https://AGG_ADDR:8080/status` with a token from
`lw auth token -url https://AGG_ADDR:8080 -user ops -password-file
~/logwisp-secrets/ops.pass -ca-file ca.crt` ([CLI](cli.md#lw-auth)).
`lw auth token --url https://AGG_ADDR:8080 --user ops --password-file
~/logwisp-secrets/ops.pass --ca-file ca.crt` ([CLI](cli.md#lw-auth)).
Variants:
+3 -3
View File
@@ -116,8 +116,8 @@ both:
```bash
sudo install -m 0600 -o logwisp -g logwisp /dev/null /etc/logwisp/users.toml
sudo lw auth add-user -credentials /etc/logwisp/users.toml -user <name> \
-password-file <file>
sudo lw auth add-user --credentials /etc/logwisp/users.toml --user <name> \
--password-file <file>
sudo systemctl reload logwisp
```
@@ -230,7 +230,7 @@ docker run -d --name logwisp \
```bash
docker run --rm --read-only --cap-drop ALL --network none \
-v /srv/logwisp/secrets:/work -v /srv/logwisp/passwords:/pw logwisp:<tag> \
auth add-user -credentials /work/users.toml -user <name> -password-file /pw/<name>.pass
auth add-user --credentials /work/users.toml --user <name> --password-file /pw/<name>.pass
```
## Packaging Status
+66 -36
View File
@@ -6,12 +6,12 @@
.Nd LogWisp log transport
.Sh SYNOPSIS
.Nm
.Op Fl \-check | Fl \-dump
.Op Fl t | Fl \-dump
.Op Fl q
.Op Fl c Ar file
.Op Fl \-quiet
.Op Fl \-color Op Ar when
.Op Fl \- Ns Ar key Ns = Ns Ar value ...
.Op Fl \-preset Ar spec
.Op Fl p Ar spec
.Op Fl \-pipeline Ar name
.Op Fl \-source Ar spec ...
.Op Fl \-sink Ar spec ...
@@ -24,7 +24,7 @@
.Ar command
.Op Ar flags
.Nm
.Fl \-version
.Fl V | Fl \-version
.Nm
.Cm help | Fl h | Fl \-help
.Sh DESCRIPTION
@@ -107,6 +107,27 @@ service under a supervisor such as
or
.Xr daemon 8 .
.Pp
Every option has a long form, and the frequent ones a one-letter short form
that stands for it in every command that has it.
A value follows as the next argument or after
.Ql = ;
a value that starts with
.Ql -
follows
.Ql = .
Short options do not combine, and an unknown one is an error.
The options of
.Nm
join words with
.Ql - ;
a configuration or preset key keeps its TOML spelling.
The
.Cm auth ,
.Cm tls
and
.Cm preset
commands also take a long option after a single dash.
.Pp
The options are as follows:
.Bl -tag -width Ds
.It Fl c Ar file , Fl \-config Ar file , Fl \-config= Ns Ar file
@@ -136,7 +157,7 @@ program parses keeps
.Cm auto
or
.Cm never .
.It Fl \-quiet
.It Fl q , Fl \-quiet
Turn off the log of
.Nm
and the terminal reminder; console sinks still write.
@@ -170,7 +191,7 @@ the default is
with a configuration file and
.Cm warn
without.
.It Fl \-check
.It Fl t , Fl \-check
Build every pipeline and plugin as a start would, loading options, TLS files
and credentials files and printing startup warnings, without binding a port
or opening, reading or creating a log file; print
@@ -183,7 +204,7 @@ Print the effective configuration, with the options, environment and
presets resolved into it, as a file
.Fl c
reads back, and exit.
.It Fl \-version
.It Fl V , Fl \-version
Print the version, commit and build time, and exit.
.It Fl h , Fl \-help , Cm help
Print usage and exit.
@@ -210,7 +231,7 @@ Specifications before the first
belong to a pipeline named
.Dq cli ,
or after the preset that starts them.
.It Fl \-preset Ar spec
.It Fl p Ar spec , Fl \-preset Ar spec
Start the pipeline with a preset, described in
.Sx Presets ;
later options add sources, sinks and filters to it, or replace its format.
@@ -260,10 +281,14 @@ A preset builds a common pipeline from a few keys:
.Fl \-preset Ar name Ns Op , Ns Ar key Ns = Ns Ar value ... ,
in the syntax of the other specifications;
.Nm
.Cm preset Ar name Op Fl Ns Ar key Ar value ...
.Cm preset Ar name Op Fl \- Ns Ar key Ar value ...
prints the pipeline it expands to, and
.Fl h
there lists its keys and defaults.
there lists its keys and defaults; its flags are the keys, spelled as in the
specification, and
.Fl u
is
.Fl \-user .
A
.Ar path
is a file, a directory, whose files are followed, or a glob; without one, a
@@ -292,10 +317,12 @@ or
.Pq Cm cert No and Cm key ;
.Cm users
names a credentials file, so readers log in with SCRAM.
The built-in login and viewer pages,
.Cm viewer=true ,
work only behind a TLS-terminating proxy named by
A browser at the root gets a live viewer; with
.Cm users ,
only behind a TLS-terminating proxy named by
.Cm proxy ,
with
.Cm viewer=true ,
since a browser cannot bind its login to the TLS channel.
.It Cm edge to Oo Cm path , transport , ca | pin , user , password_file , cert , key , node Oc
Forward to an aggregator at
@@ -332,19 +359,19 @@ vouches for, and write to standard output or to files in
makes ECDSA P-256 certificates; it never replaces a file, and writes keys
with mode 0600.
.Bl -tag -width Ds
.It Cm ca Fl dir Ar dir Oo Fl name Ar name Oc Op Fl days Ar n
.It Cm ca Fl \-dir Ar dir Oo Fl \-name Ar name Oc Op Fl \-days Ar n
Create a certificate authority,
.Pa dir/ca.crt
and
.Pa dir/ca.key ,
valid 3650 days.
.It Cm cert Fl ca-dir Ar dir Fl name Ar name Oo Fl host Ar names Oc Oo Fl server Oc Oo Fl client Oc Oo Fl days Ar n Oc Op Fl out Ar dir
.It Cm cert Fl \-ca\-dir Ar dir Fl \-name Ar name Oo Fl \-host Ar names Oc Oo Fl \-server Oc Oo Fl \-client Oc Oo Fl \-days Ar n Oc Op Fl \-out Ar dir
Issue
.Pa name.crt
and
.Pa name.key
from that authority, valid 397 days and never past it.
.Fl host
.Fl \-host
lists the DNS names and addresses a server certificate carries
.Pq default: Ar name ;
.Ar name
@@ -378,33 +405,36 @@ restarts, so a fleet uses an issuer instead.
.Nm
.Cm auth
manages SCRAM credentials files and logs in to SCRAM listeners.
.Fl u
is short for
.Fl \-user .
Changes to a credentials file apply when the server receives
.Dv SIGHUP .
.Bl -tag -width Ds
.It Cm add-user Fl credentials Ar file Fl user Ar name Oo Fl password-file Ar file Oc Op Fl generate
.It Cm add-user Fl \-credentials Ar file Fl u Ar name Oo Fl \-password\-file Ar file Oc Op Fl \-generate
Add a user, or replace its password.
The password is read from the
.Fl password-file
.Fl \-password\-file
when it exists; otherwise one is generated and written there, or printed
once to standard output.
.Fl generate
.Fl \-generate
replaces an existing password without reading one.
A missing or empty credentials file is created with mode 0600; an existing
one keeps its owner and mode.
.It Cm remove-user Fl credentials Ar file Fl user Ar name
.It Cm remove-user Fl \-credentials Ar file Fl u Ar name
Remove a user.
The last user cannot be removed.
.It Cm token Fl url Ar https://host:port Fl user Ar name Fl password-file Ar file Oo Ar tls-flags Oc Op Fl unbound
.It Cm token Fl \-url Ar https://host:port Fl u Ar name Fl \-password\-file Ar file Oo Ar tls-flags Oc Op Fl \-unbound
Log in to an
.Cm http
sink or
.Cm http_chain
source and print a bearer token.
.Fl unbound
.Fl \-unbound
logs in without channel binding, to an
.Cm http
sink behind a TLS-terminating proxy; the URL may then carry the proxy's path.
.It Cm stream Fl addr Ar host:port Fl user Ar name Fl password-file Ar file Op Ar tls-flags
.It Cm stream Fl \-addr Ar host:port Fl u Ar name Fl \-password\-file Ar file Op Ar tls-flags
Log in to a
.Cm tcp
sink and copy its stream to standard output until interrupted.
@@ -413,15 +443,15 @@ sink and copy its stream to standard output until interrupted.
The
.Ar tls-flags
are
.Fl ca-file Ar file
.Fl \-ca\-file Ar file
.Pq default: the system roots
or
.Fl pin-sha256 Ar pin ,
.Fl server-name Ar name ,
.Fl \-pin\-sha256 Ar pin ,
.Fl \-server\-name Ar name ,
and
.Fl cert-file Ar file
.Fl \-cert\-file Ar file
with
.Fl key-file Ar file
.Fl \-key\-file Ar file
for a listener that requires client certificates.
There is no flag to skip server verification.
An IPv6
@@ -507,8 +537,8 @@ The service's working directory on Linux and on
.Nm
exits 0 at the end of input, after a clean shutdown,
.Fl \-version
or help; 1 on a configuration, logger or startup error; and 2 when a
configuration file named by
or help; 1 on a command-line, configuration, logger or startup error; and 2
when a configuration file named by
.Fl c
or
.Ev LOGWISP_CONFIG_*
@@ -591,10 +621,10 @@ the authority, its certificate and a user for the edge; copy
.Pa ca.crt
and the password file to the edge; then run both:
.Bd -literal -offset indent
# lw tls ca \-dir /etc/logwisp/pki
# lw tls cert \-ca-dir /etc/logwisp/pki \-name agg.example.org \-server
# lw auth add-user \-credentials /etc/logwisp/users.toml \e
\-user edge-01 \-password-file /root/edge-01.pass
# lw tls ca \-\-dir /etc/logwisp/pki
# lw tls cert \-\-ca-dir /etc/logwisp/pki \-\-name agg.example.org \-\-server
# lw auth add-user \-\-credentials /etc/logwisp/users.toml \e
\-\-user edge-01 \-\-password-file /root/edge-01.pass
agg# lw \-\-preset aggregator,tls=files,users=/etc/logwisp/users.toml,\e
cert=/etc/logwisp/pki/agg.example.org.crt,\e
key=/etc/logwisp/pki/agg.example.org.key,out=/var/log/edges
@@ -619,8 +649,8 @@ generate the password into a file, then reload the server:
.Bd -literal -offset indent
# install \-m 0600 \-o logwisp \-g logwisp \e
/dev/null /etc/logwisp/users.toml
# lw auth add-user \-credentials /etc/logwisp/users.toml \e
\-user alice \-password-file /root/alice.pass
# lw auth add-user \-\-credentials /etc/logwisp/users.toml \e
\-\-user alice \-\-password-file /root/alice.pass
# systemctl reload logwisp
.Ed
.Sh SEE ALSO
+4 -3
View File
@@ -25,7 +25,7 @@ Every listener and dialer keeps strictly to the family of its `host`:
- `host` takes the bare address: `host = "::1"`. A bracketed host, one
with a port, or an IPv4-mapped one (`::ffff:10.0.0.1`) fails at load.
- Addresses and URLs bracket it: logs print `[::1]:8443`, and `lw auth`
takes `-addr [::1]:8443` and `-url https://[::1]:8443`. Older `curl`
takes `--addr [::1]:8443` and `--url https://[::1]:8443`. Older `curl`
releases read the brackets as a glob; `-g` stops that and is harmless.
- A link-local address carries its zone, `fe80::1%eth0`, escaped in URLs as
`https://[fe80::1%25eth0]:8443`.
@@ -46,7 +46,8 @@ Every listener and dialer keeps strictly to the family of its `host`:
Listeners:
- `tcp` sink: raw stream; broadcasts formatted payloads to clients.
- `http` sink: HTTP SSE; a browser-friendly live stream plus status JSON.
- `http` sink: HTTP SSE; a browser viewer at `/`, the live stream and status
JSON.
- `tcp_chain` source: chain v1; ingests a persistent NDJSON stream.
- `http_chain` source: chain v1; ingests NDJSON batches over POST.
@@ -249,7 +250,7 @@ headers, and entry encoding.
between the peers terminates TLS. Pass TLS through to LogWisp; the login
cannot work otherwise, except on an `http` sink in proxy mode
(`auth.trusted_proxies`), where `the client bound its proof to the proxy's
certificate` means a client that needs `lw auth token -unbound`, and
certificate` means a client that needs `lw auth token --unbound`, and
`the client sent an unbound proof` the reverse. On an `http_chain` sink,
`server certificate differs from the one the SCRAM login was bound to` means
the certificate changed after the login: a rotation (the retry binds anew),
+18 -16
View File
@@ -204,16 +204,16 @@ closed.
## CLI
```
lw auth add-user -credentials F -user U [-password-file P] [-generate]
lw auth remove-user -credentials F -user U
lw auth token -url https://host:port[/path] -user U -password-file P [-unbound] [TLS flags]
lw auth stream -addr host:port -user U -password-file P [TLS flags]
lw auth add-user --credentials F --user U [--password-file P] [--generate]
lw auth remove-user --credentials F --user U
lw auth token --url https://host:port[/path] --user U --password-file P [--unbound] [TLS flags]
lw auth stream --addr host:port --user U --password-file P [TLS flags]
```
- `add-user` takes the password from `-password-file` when it exists (at least
- `add-user` takes the password from `--password-file` when it exists (at least
8 bytes); a new user without one gets a generated 130-bit password, written to
that file (0600) or printed once. Rotating an existing user needs the file or
`-generate`, so a mistyped path cannot replace a password. The file's KDF
`--generate`, so a mistyped path cannot replace a password. The file's KDF
profile and decoy key are reused, the result is validated with the daemon's
own parser, and the rewrite (temp file beside the file or a symlink's target,
rename) keeps the existing mode and owner, or fails.
@@ -222,10 +222,10 @@ lw auth stream -addr host:port -user U -password-file P [TLS flags]
credentials file.
- `token` prints a bearer token for curl (`-H @<(...)` keeps it out of argv);
`stream` authenticates to a `tcp` sink and copies the stream to stdout until
interrupted. `-url` takes a path only with `-unbound`, and redirects are not
interrupted. `--url` takes a path only with `--unbound`, and redirects are not
followed.
- TLS flags: `-ca-file`, `-server-name`, `-cert-file`, `-key-file`. There is no
insecure flag.
- TLS flags: `--ca-file` or `--pin-sha256`, `--server-name`, `--cert-file`,
`--key-file`. There is no insecure flag.
- Exit status: 0 success, 1 failure, 2 usage error.
## Rollout and Rotation
@@ -233,7 +233,7 @@ lw auth stream -addr host:port -user U -password-file P [TLS flags]
- Turning SCRAM on is per listener and cuts off unconfigured dialers. To migrate
without a gap, upgrade binaries, add a second SCRAM listener, move dialers,
then remove the old listener.
- Password rotation: `add-user -generate`, `SIGHUP` the listener, deploy the new
- Password rotation: `add-user --generate`, `SIGHUP` the listener, deploy the new
password file, `SIGHUP` the dialer. The dialer retries under backoff in
between.
- Revocation: `remove-user`, `SIGHUP`. All tokens die with the reload.
@@ -261,7 +261,7 @@ sink into proxy mode; it is refused on every other plugin and with `identity`.
proxy-to-LogWisp hop could relay a login, hence the address restriction and a
startup warning for a plaintext hop to a proxy off this host. `tls` may stay
off. Outside proxy mode nothing changes: proofs stay bound, and a bound proof
in proxy mode fails with a hint naming `-unbound`.
in proxy mode fails with a hint naming `--unbound`.
**Session.** The proof step may ask for a cookie (`"session": "cookie"`, proxy
mode only). The answer sets `logwisp_session` (`HttpOnly; Secure;
@@ -289,8 +289,10 @@ token in a page variable only: it lasts no longer than the page or its lifetime.
pages is relative, so a proxy prefix works unchanged.
- `POST /auth`: SCRAM hello, proof or logout; unbound in proxy mode.
- `GET /auth/scram.js`: the client library, always served in proxy mode.
- `GET /auth/login`, `login.js`, `style.css`: the login page (`login_page`).
- `GET /auth/scram.js`, `style.css`: the client library and the pages' style,
always served in proxy mode, and with the viewer.
- `GET /`: `303` to `auth/view`, else `auth/login`, when either is served.
- `GET /auth/login`, `login.js`: the login page (`login_page`).
- `GET /auth/view`, `view.js`: a minimal live viewer (`viewer_page`, needs
`login_page`).
@@ -316,8 +318,8 @@ refuses redirects, and reports success only after verifying the server's
signature. A 64 MiB login takes about 2 s on a desktop. The wire protocol above
is the contract: a site may implement its own client.
**CLI.** `lw auth token -unbound` logs in through the proxy, still pinning
its certificate across the two requests; only then may `-url` carry the mount
**CLI.** `lw auth token --unbound` logs in through the proxy, still pinning
its certificate across the two requests; only then may `--url` carry the mount
path.
## Verification
@@ -367,7 +369,7 @@ by sign-out, and no CSP violations; a cookie the browser drops leading to the
viewer's own form, not a loop, its token kept out of storage and the URL; a
profile blocking every cookie signing in on the viewer, its proofs asking for
no cookie, streaming with a bearer, sign-out revoking the token, and a reload's
`401` asking again; `token -unbound` and curl through the proxy; direct peers
`401` asking again; `token --unbound` and curl through the proxy; direct peers
and plaintext-forwarded requests `403`. The existing scripts keep passing.
## Not Implemented
+15 -10
View File
@@ -360,9 +360,9 @@ process and, at startup and every reload, a certificate for it valid 397 days
to that certificate.
```bash
lw tls ca -dir /etc/logwisp/pki # ca.crt, ca.key (0600)
lw tls cert -ca-dir /etc/logwisp/pki -name agg.example.org -server
lw tls cert -ca-dir /etc/logwisp/pki -name edge-01 -client
lw tls ca --dir /etc/logwisp/pki # ca.crt, ca.key (0600)
lw tls cert --ca-dir /etc/logwisp/pki --name agg.example.org --server
lw tls cert --ca-dir /etc/logwisp/pki --name edge-01 --client
```
## Enabling mTLS
@@ -594,8 +594,8 @@ different users.
- **Rollout.** Turning `scram` on cuts off every dialer of that listener that
has no credentials. Upgrade every binary first, then add a second listener
with `scram` on another port, move the dialers to it, and remove the old one.
- **Password rotation.** `lw auth add-user -generate` with the user's
`-password-file` replaces verifier and password together. `SIGHUP` the
- **Password rotation.** `lw auth add-user --generate` with the user's
`--password-file` replaces verifier and password together. `SIGHUP` the
listener, deploy the password file, `SIGHUP` the dialer. Logins fail in
between and the dialer retries under backoff, holding its current entry or
batch while its input queue fills. To avoid the gap, add a second user, move
@@ -664,7 +664,7 @@ location /logs/ {
`Path`, so it scopes itself to the mount (`/logs` above). Stream and status
accept it or a bearer token, so `new EventSource("/logs/stream")` works on any
page of the site, beside the site's own Basic auth too, and `lw auth token
-unbound` keeps working. `POST /auth` with `{"logout": true}` clears the
--unbound` keeps working. `POST /auth` with `{"logout": true}` clears the
cookie and revokes the token until it expires. `/auth` takes only
`application/json`, which a cross-origin page cannot send without a preflight.
- **Private windows.** Private or incognito windows keep cookies in memory,
@@ -698,9 +698,10 @@ location /logs/ {
- `cookiesUsable()` tells which applies: false when the session cookie would
not stick. `loginUnavailable()`, asked first, names why neither can run
(no secure context), or is empty.
- **Pages.** Under `/auth/` the sink serves `scram.js` always, and with
`login_page` / `viewer_page` the login page and a minimal live viewer with
their script and style, under `default-src 'none'; script-src 'self';
- **Pages.** Under `/auth/` the sink serves `scram.js` and `style.css` always,
and with `login_page` / `viewer_page` the login page and a minimal live
viewer with their scripts; `GET /` leads to the viewer, else the login page.
All are served under `default-src 'none'; script-src 'self';
connect-src 'self'; style-src 'self'; form-action 'self'; frame-ancestors
'none'; base-uri 'none'`.
@@ -771,7 +772,7 @@ sets `proxy_protocol` per stream `server`, so route LogWisp SNIs without it:
`listen`s with `proxy_protocol` and proxy_passes to LogWisp without it; that
hop consumes the header.
- Either way dialers must send the name: a `host` that is one, or
`tls.server_name` (`-server-name` for `lw auth`); an IP literal sends no SNI.
`tls.server_name` (`--server-name` for `lw auth`); an IP literal sends no SNI.
- With HAProxy, leave `send-proxy` off the LogWisp backends.
```nginx
@@ -864,6 +865,10 @@ What each exposes when `auth.type = "none"`:
header is omitted once an auth policy is set).
- `http` sink `status_path`: host, port, TLS flag, uptime, client counts,
throughput counters.
- `http` sink `GET /` (`303` to `/auth/view`) and the viewer's files: static
pages that read the two endpoints above under the CSP and expose nothing
beyond them. Under `mtls` they are served before the allow list, while the
stream and status stay gated.
- `tcp` sink: the full log stream to any client that connects.
- `tcp_chain` and `http_chain` sources: ingest from any peer that can connect
(with `client_auth`, any the CA vouches for), under any node label it
+24 -8
View File
@@ -128,7 +128,7 @@ No options. The input queue is fixed at 1000.
## http
Server-Sent Events stream plus a JSON status endpoint.
Server-Sent Events stream, a JSON status endpoint, and a browser viewer.
```toml
[[pipelines.plugin_sinks]]
@@ -161,17 +161,20 @@ Options, each as type and default:
- `host` (string, `0.0.0.0`): bind address, IPv4 or IPv6 (`::`); the listener
keeps to its family ([Networking](networking.md#address-family)).
- `port` (int, **required**): listen port.
- `stream_path` (string, `/stream`): SSE endpoint; must start with `/`.
- `status_path` (string, `/status`): status endpoint; must start with `/` and
differ from `stream_path`.
- `stream_path` (string, `/stream`): SSE endpoint, matched exactly (`/logs/`
does not answer `/logs/x`); it starts with `/`, is clean (no `//`, `.` or
`..` segment) and holds none of `{`, `}`, `%`, `?`, `#`.
- `status_path` (string, `/status`): status endpoint, under the same rules;
differs from `stream_path`.
- `buffer_size` (int, `1000`): sink input queue depth.
- `client_buffer_size` (int, `256`): per-client send queue depth.
- `write_timeout_ms` (int, `0`): per-event write deadline; `0` = none.
- `max_connections` (int, `0`): concurrent stream cap; `0` = unlimited.
- `login_page` (bool, `false`): serve the browser login page at `/auth/login`;
needs `auth.trusted_proxies`.
- `viewer_page` (bool, `false`): serve a minimal live viewer at `/auth/view`;
needs `login_page`.
- `login_page` (bool, `false`): `scram` in proxy mode only (an error
elsewhere): the browser login page at `/auth/login`.
- `viewer_page` (bool, `false`): `scram` in proxy mode only (an error
elsewhere): the live viewer at `/auth/view`; needs `login_page`. Without
auth, or under `mtls`, the viewer is always served.
- `tls` (table): listener TLS; see [Security](security.md).
- `auth` (table): client authentication (`mtls` or `scram`); see
[Security](security.md#the-auth-block).
@@ -194,6 +197,19 @@ Options, each as type and default:
through `/auth/login` or the site's own copy of `/auth/scram.js`, and stream
and status also accept the `logwisp_session` cookie. See
[Browsers behind a TLS-terminating proxy](security.md#browsers-behind-a-tls-terminating-proxy).
- `GET /` answers `303` to `auth/view`, a relative `Location` that a proxy
prefix keeps, wherever a browser can read the stream:
- without auth, over plain http too: the viewer needs no secure context, so
`http://ADDRESS:PORT/` works;
- under `mtls`, with the browser's client certificate (`openssl pkcs12
-export -in NAME.crt -inkey NAME.key -out NAME.p12` imports it);
- in proxy mode with `viewer_page`, or to `auth/login` with `login_page`
alone;
- never under `scram` on the sink's own TLS, which a browser cannot log in
to: `/` is refused like any other path;
- an endpoint at `/` keeps the root, and the viewer stays at `/auth/view`.
The viewer shows entries from when it connects (the sink keeps no backlog)
and holds one stream against `max_connections`.
- Refusals are logged at WARN and counted in `auth_rejected`. The authorized
identity is recorded in the client's session as `auth_method` /
`auth_identity`.
+37
View File
@@ -178,3 +178,40 @@ identity, and both skeletons download the tagged source.
5. Shell completion for bash, zsh and fish, installed by `make install`.
6. A packaging CI job: build the AUR package in an Arch container and the port
in a FreeBSD VM, run `lw --version` and `make image-check`.
## 3. Follow-ups
Smaller items found along the way; each is independent of the ACL.
1. Chain sinks at the end of a finite input. The `tcp_chain` sink's `Stop`
closes its loop without draining its queue, and the `http_chain` sink makes
one best-effort flush, so `seq 1 N | lw --preset edge,...` may lose the
tail. Verify with a finite stdin into each and an aggregator counting lines;
give both a bounded drain like `sink.FlushBound`.
2. A late browser sees no backlog: the `http` sink keeps none, so the viewer
shows entries from when it connects. A bounded replay (a `replay_lines`
option, sent after `event: connected`) would fill it. The viewer also hides
a `503` from `max_connections` behind "retrying"; the status JSON it already
fetches could name it.
3. Level detection. `source.ExtractLogLevel` needs a delimiter after the name
(`ERROR:`, `[WARN]`, ` INFO `), so `warning disk full` or `DBG x` get no
level, hence no color. Match the names as words, as the console sink's
painter does, with one table for both.
4. Command line:
- a positional word at the top level (`lw tail`, `lw -t FILE`) still reaches
config with an unclear message; name it ("unexpected argument X; the file
is -c FILE");
- Go's flag errors keep single dashes ("flag needs an argument: -u");
- an unknown `--key` only warns and then runs the stdin pipe; consider an
error when stdin is a terminal;
- `lw tls cert --host` could become `--hosts` (keeping `--host`), to match
`tls.hosts` and the preset key;
- top-level usage errors exit 1, subcommand ones 2.
5. `GET /favicon.ico` answers 404; the logo could serve as the icon once the
pages' CSP allows `img-src 'self'`.
6. TLS: the generated-certificate key is one per process, so a pin taken from a
`self_signed` listener also matches an `issuer` listener of the same
process; document it or key per listener. Pins are checked in
`VerifyPeerCertificate`, which Go skips on resumption: guard it if a dialer
ever keeps a `ClientSessionCache`.
7. `core.ShutdownTimeout` is unused.