v0.27.1 packaging update, install and initial config

doc/cli.md

@@ -9,6 +9,7 @@ lw --dump [options]
 lw auth COMMAND [flags]
 lw tls COMMAND [flags]
 lw preset NAME [flags]
+lw config init [--out FILE]
 lw help | -h | --help
 lw -V | --version
 ```
@@ -21,7 +22,9 @@ service.
 A [preset](#presets) builds a common pipeline from a few keys. `lw auth`
 manages SCRAM credentials and logs in to `scram` listeners
 ([below](#lw-auth)); `lw tls` makes a certificate authority and certificates
-([below](#lw-tls)). A command is recognized only as the first argument.
+([below](#lw-tls)); `lw config init` writes the annotated default
+configuration ([below](#lw-config)). A command is recognized only as the
+first argument.
 
 ## Options
 
@@ -43,8 +46,7 @@ manages SCRAM credentials and logs in to `scram` listeners
   `--password-file`); configuration keys, spec keys and preset keys keep
   TOML's `_` and `.` (`--status_reporter`, `--sink http,tls.cert_file=F`,
   `lw preset edge --password_file F`).
-- `lw auth`, `lw tls` and `lw preset` also take a long option after one dash
-  (`-user NAME`), as earlier releases documented.
+- The commands also take a long option after one dash (`-user NAME`).
 
 Any scalar configuration key is settable as a flag using its TOML path;
 pipelines have [their own flags](#pipelines):
@@ -540,3 +542,23 @@ lw tls ca --dir /etc/logwisp/pki
 lw tls cert --ca-dir /etc/logwisp/pki --name agg.example.org --server
 lw tls cert --ca-dir /etc/logwisp/pki --name edge-01 --client
 ```
+
+## `lw config`
+
+```
+lw config init [--out FILE]
+```
+
+`init` writes the annotated configuration that ships as
+`config/logwisp.toml`, every option with its default, to edit and run. It
+never replaces a file (remove one to replace it). Without `--out` it writes
+`~/.config/logwisp/logwisp.toml`, the file lw reads without `-c` unless a
+`LOGWISP_CONFIG_*` variable names another; with that file in place, lw
+without pipeline options runs its pipelines rather than filtering stdin.
+Any other path is run with `-c FILE`; lw prints which applies. `lw --dump` prints the
+effective configuration instead, flags and environment included.
+
+```bash
+lw config init
+sudo lw config init --out /usr/local/etc/logwisp/logwisp.toml
+```

doc/installation.md

@@ -42,7 +42,8 @@ The Makefile works with GNU make and BSD make alike. Targets:
 - Container
   - `make image`, `make image-check`: see [Container Image](#container-image)
 - Install
-  - `make install`, `make uninstall`, `make deb`: see [Installing](#installing)
+  - `make install`, `make uninstall`, `make deb`, `make arch`: see
+    [Installing](#installing)
 
 Variables:
 
@@ -64,8 +65,9 @@ checkout.
 ## Installing
 
 `make install` copies what a package ships and compiles nothing, so build
-first. It honours `DESTDIR`, `PREFIX`, `SYSCONFDIR` and `BINDIR` (default
-`$PREFIX/bin`, the path the systemd unit and the rc.d script run `lw` from):
+first. It honours `DESTDIR`, `PREFIX`, `SYSCONFDIR`, `BINDIR` (default
+`$PREFIX/bin`, the path the systemd unit and the rc.d script run `lw` from)
+and `SERVICE`:
 
 - Linux defaults: `PREFIX=/usr`, `SYSCONFDIR=/etc`. Outside a package
   manager, prefer `PREFIX=/usr/local`; systemd reads units, sysusers and
@@ -73,6 +75,8 @@ first. It honours `DESTDIR`, `PREFIX`, `SYSCONFDIR` and `BINDIR` (default
 - FreeBSD defaults: `PREFIX=/usr/local`, `SYSCONFDIR=/usr/local/etc`.
 - `INSTALL_OS` (default: `uname -s`) picks the layout, so
   `INSTALL_OS=FreeBSD DESTDIR=...` stages the FreeBSD one on another system.
+- `SERVICE`: `yes` (default) adds the service and its configuration; `no`
+  installs `lw` alone ([Without a Service](#without-a-service)).
 - `BASHCOMPDIR`, `ZSHCOMPDIR`, `FISHCOMPDIR`: where the shell completion
   goes, by default `$PREFIX/share/bash-completion/completions`,
   `$PREFIX/share/zsh/site-functions` (`/usr/share/zsh/vendor-completions`
@@ -84,7 +88,7 @@ make release
 sudo make install PREFIX=/usr/local
 ```
 
-Every system gets:
+Every install gets:
 
 - `$BINDIR/lw` and the manual `$PREFIX/share/man/man1/lw.1`
 - completion for `lw`, its commands, flags, files and preset names:
@@ -92,10 +96,13 @@ Every system gets:
 - `$PREFIX/share/doc/logwisp/` (this documentation) and
   `$PREFIX/share/licenses/logwisp/LICENSE`
 
+A service install also gets the service files below and, on Linux and
+FreeBSD alike, `$SYSCONFDIR/logwisp/logwisp.toml`: the annotated sample
+configuration, placed when none is there and otherwise kept. Edit it before
+starting the service.
+
 ### Linux (systemd)
 
-- `$SYSCONFDIR/logwisp/logwisp.toml`: the annotated sample configuration,
-  kept when one is already there. Edit it before starting the service.
 - `$PREFIX/lib/systemd/system/logwisp.service`: runs
   `lw -c $SYSCONFDIR/logwisp/logwisp.toml` as the `logwisp` user, from
   `/var/lib/logwisp`; `systemctl reload` sends `SIGHUP`.
@@ -133,11 +140,11 @@ sudo systemctl reload logwisp
 
 ### FreeBSD (rc.d)
 
-- `$SYSCONFDIR/logwisp/logwisp.toml.sample`, copied to `logwisp.toml` when
-  none exists and `DESTDIR` is empty
-- `$PREFIX/etc/rc.d/logwisp`: runs `lw` through `daemon(8)` as
-  `logwisp_user`, with its output in syslog under the tag `logwisp`;
-  `service logwisp reload` sends `SIGHUP`
+`$PREFIX/etc/rc.d/logwisp` runs `lw` through `daemon(8)` as `logwisp_user`,
+with its output in syslog under the tag `logwisp`; `service logwisp reload`
+sends `SIGHUP`. The port in `deploy/package/freebsd/` places the
+configuration the ports way, to the same effect: `@sample` installs
+`logwisp.toml.sample` and copies it to `logwisp.toml` when none exists.
 
 ```bash
 sudo pw useradd logwisp -d /nonexistent -s /usr/sbin/nologin -c "LogWisp log transport"
@@ -172,18 +179,93 @@ sudo systemctl enable --now logwisp
 
 - Installing creates the `logwisp` account and its directories where
   `systemd-sysusers` and `systemd-tmpfiles` exist; the service waits to be
-  enabled, and an upgrade restarts it if it runs.
-- `/etc/logwisp/logwisp.toml` is a conffile: an upgrade keeps your edits,
-  and only a purge removes it.
+  enabled, and an upgrade or reinstall restarts it if it runs.
+- `/etc/logwisp/logwisp.toml` is a conffile: an upgrade or reinstall keeps
+  your edits, and only a purge removes it.
+- The same version again: `sudo apt install --reinstall ./bin/logwisp_*.deb`.
 - Removing stops and disables the service. The account, its directories
   and any credentials in `/etc/logwisp` stay, as with `make uninstall`.
 
+### Arch Linux (makepkg)
+
+`make arch` builds the PKGBUILD in `deploy/package/arch/` from the working
+tree, committed or not, into `bin/arch/`, then prints the `pacman -U` line
+that installs the package. It needs `base-devel` and `go`, and runs as a
+user: `makepkg` refuses root.
+
+```bash
+make arch
+sudo pacman -U bin/arch/logwisp-<version>-1-x86_64.pkg.tar.zst
+sudo systemctl enable --now logwisp
+```
+
+- pacman's hooks create the `logwisp` account and its directories; the
+  service waits to be enabled.
+- `/etc/logwisp/logwisp.toml` is a `backup` file: an upgrade or reinstall
+  keeps your edits, and a shipped version that changed lands beside it as
+  `logwisp.toml.pacnew`. `pacman -R` keeps an edited one as
+  `logwisp.toml.pacsave`.
+- Reinstall: `make arch` and the same `pacman -U`. pacman does not restart
+  the service: `sudo systemctl restart logwisp`.
+- An earlier `sudo make install` conflicts with the package
+  (`PREFIX=/usr`) or overrides it (`PREFIX=/usr/local`, whose unit and `lw`
+  come first): `sudo make uninstall` with that `PREFIX` first. The
+  configuration it keeps is no conflict: pacman keeps it, and when it was
+  edited, the shipped one lands beside it as `logwisp.toml.pacnew`.
+
+### Reinstalling
+
+Every method runs again over its own installation, for an upgrade or the
+same version, and keeps an edited configuration: `make install` leaves an
+existing `$SYSCONFDIR/logwisp/logwisp.toml` alone, and the packages treat it
+as above. Only the `.deb` restarts a running service; otherwise restart it
+(`sudo systemctl restart logwisp`, `sudo service logwisp restart`). To see
+what a release changed in the shipped configuration:
+
+```bash
+d=$(mktemp -d); lw config init --out "$d/logwisp.toml"; diff "$d/logwisp.toml" /etc/logwisp/logwisp.toml
+```
+
+### Without a Service
+
+`SERVICE=no` installs `lw`, its manual, completion and documentation, and
+nothing else: no service files, account or configuration. For one user,
+without root:
+
+```bash
+make release
+make install SERVICE=no PREFIX="$HOME/.local"
+lw config init
+```
+
+- `~/.local/bin` goes on `PATH`. `man lw` then finds the manual, and bash
+  and fish the completion; zsh needs
+  `fpath=(~/.local/share/zsh/site-functions $fpath)` before `compinit`.
+- `make uninstall PREFIX="$HOME/.local"` removes it.
+
+For every user: `sudo make install SERVICE=no PREFIX=/usr/local`.
+
+`lw config init` writes the shipped, annotated configuration, never over an
+existing file, where you choose:
+
+- `lw config init`: `~/.config/logwisp/logwisp.toml`, which `lw` reads
+  without `-c` unless a `LOGWISP_CONFIG_*` variable names another file. With
+  it in place, `lw` without pipeline options runs its
+  pipelines instead of copying stdin to stdout; pipeline options and presets
+  still replace them.
+- `sudo lw config init --out /usr/local/etc/logwisp/logwisp.toml`: a file
+  for every user, named with `-c`, or without it by
+  `LOGWISP_CONFIG_DIR=/usr/local/etc/logwisp`
+  ([File Location](configuration.md#file-location)).
+
+`lw --check` (with the same `-c`) then validates it.
+
 ### Uninstall
 
 `sudo make uninstall` (with the `PREFIX` used to install) removes everything
 `make install` wrote except `$SYSCONFDIR/logwisp`, which holds configuration
-and credentials. Remove that, the working directories and the account by
-hand.
+and credentials, and any file `lw config init` wrote. Remove those, the
+working directories and the account by hand.
 
 ## Container Image
 
@@ -269,15 +351,21 @@ docker run -d --name logwisp \
 
 In place:
 
-- `make install` with `DESTDIR`, `PREFIX` and `SYSCONFDIR`, the manual,
-  shell completion and the service files in `deploy/package/`
-- `make deb`
+- `make install` with `DESTDIR`, `PREFIX`, `SYSCONFDIR` and `SERVICE`, the
+  manual, shell completion and the service files in `deploy/package/`
+- `make deb`, and `make arch` building the PKGBUILD from the working tree
 - skeletons: `deploy/package/arch/PKGBUILD` and a `sysutils/logwisp` port in
   `deploy/package/freebsd/`
-- `.github/workflows/package.yml`, on changes to them: `makepkg` on the
-  PKGBUILD in an Arch container, the `.deb` installed, started, removed and
-  purged on Ubuntu, BSD make building, installing and running the rc.d
-  service on FreeBSD, and `make image image-check`
+- `.github/workflows/package.yml`, on changes to them:
+  - Arch container: `make arch`, installed over an edited configuration,
+    reinstalled keeping it, removed
+  - Ubuntu: the `.deb` installed, started, reinstalled keeping an edited
+    configuration, removed and purged
+  - Ubuntu, without a service: `make install SERVICE=no` into a home twice,
+    then `lw config init` writing the file `lw` reads, once
+  - FreeBSD: BSD make building, installing twice keeping an edited
+    configuration, and running the rc.d service
+  - `make image image-check`
 
 Still missing: release tags, which both skeletons download; submitting the
 AUR package and the port; a Debian source package for a PPA or the archive.

doc/lw.1

@@ -20,7 +20,7 @@
 .Op Fl \-rate\-limit Ar spec
 .Op Fl \-heartbeat Ar spec
 .Nm
-.Cm auth | tls | preset
+.Cm auth | tls | preset | config
 .Ar command
 .Op Ar flags
 .Nm
@@ -124,12 +124,7 @@ The options of
 join words with
 .Ql - ;
 a configuration or preset key keeps its TOML spelling.
-The
-.Cm auth ,
-.Cm tls
-and
-.Cm preset
-commands also take a long option after a single dash.
+The commands also take a long option after a single dash.
 .Pp
 The options are as follows:
 .Bl -tag -width Ds
@@ -549,6 +544,19 @@ An IPv6
 .Ar host
 is written in brackets, as in
 .Li [2001:db8::1]:8443 .
+.Ss Configuration command
+.Bl -tag -width Ds
+.It Cm config init Op Fl \-out Ar file
+Write the annotated configuration
+.Nm
+ships, every option with its default, to
+.Ar file ,
+by default
+.Pa ~/.config/logwisp/logwisp.toml ,
+the first file in
+.Sx FILES .
+An existing file is never replaced.
+.El
 .Sh ENVIRONMENT
 .Bl -tag -width Ds
 .It Ev LOGWISP_ Ns Ar KEY
@@ -611,7 +619,11 @@ reads the first of these that exists:
 .Pa ./logwisp.toml
 .El
 .Pp
-Packages install a sample configuration and service files that pass it with
+.Nm
+.Cm config init
+writes the first.
+A service installation places the annotated configuration, keeping one
+already there, and service files that pass it with
 .Fl c :
 .Bl -tag -width Ds
 .It Pa /etc/logwisp/logwisp.toml
@@ -620,9 +632,7 @@ The configuration on Linux, used by
 .It Pa /usr/local/etc/logwisp/logwisp.toml
 The configuration on
 .Fx ,
-installed as
-.Pa logwisp.toml.sample
-and used by
+used by
 .Pa /usr/local/etc/rc.d/logwisp .
 .It Pa /var/lib/logwisp , /var/db/logwisp
 The service's working directory on Linux and on
@@ -668,9 +678,10 @@ the shell reports status 141.
 .Pp
 .Nm
 .Cm auth ,
-.Cm tls
-and
+.Cm tls ,
 .Cm preset
+and
+.Cm config
 do the same: 0 on success or help, 1 on failure, and 2 on a usage error.
 .Sh SIGNALS
 .Bl -tag -width "SIGINT, SIGTERM"
@@ -749,6 +760,16 @@ edge# lw \-\-preset edge,path=/var/log/app,to=agg.example.org:9000,\e
 ca=/etc/logwisp/pki/ca.crt,user=edge-01,password_file=/root/edge-01.pass
 .Ed
 .Pp
+Write the annotated configuration to the file
+.Nm
+reads without
+.Fl c ,
+edit it, then check it:
+.Bd -literal -offset indent
+$ lw config init
+$ lw \-\-check
+.Ed
+.Pp
 Check a configuration without binding a port or touching a log file:
 .Bd -literal -offset indent
 $ lw \-\-check \-c /etc/logwisp/logwisp.toml

doc/todo.md

@@ -11,15 +11,18 @@ The foundation exists:
   24.04 and 26.04, and the FreeBSD 15.1 ports tree.
 - The canonical module path `github.com/lixenwraith/logwisp`, which
   `go install`, FreeBSD's `USES=go:modules` and Debian's dh-golang expect.
-- `make install` with `DESTDIR`, `PREFIX` and `SYSCONFDIR`: the binary, the
-  `doc/lw.1` manual, bash, zsh and fish completion generated from `cmd/lw`'s
-  `shorts` and `commands` tables, and the service files in `deploy/package/`
-  (systemd unit, sysusers, tmpfiles, FreeBSD rc.d script).
-- `make deb`, a binary package built with `dpkg-deb`.
+- `make install` with `DESTDIR`, `PREFIX`, `SYSCONFDIR` and `SERVICE`: the
+  binary, the `doc/lw.1` manual, bash, zsh and fish completion generated from
+  `cmd/lw`'s `shorts` and `commands` tables, and the service files in
+  `deploy/package/` (systemd unit, sysusers, tmpfiles, FreeBSD rc.d script)
+  with the configuration, kept on reinstall.
+- `make deb`, a binary package built with `dpkg-deb`, and `make arch`, the
+  PKGBUILD built from the working tree.
 - Skeletons in `deploy/package/arch/` and `deploy/package/freebsd/`.
-- `.github/workflows/package.yml`: `makepkg` on the PKGBUILD, the `.deb`
-  installed, started and purged, BSD make and the rc.d service on FreeBSD
-  15.1, and `make image image-check`.
+- `.github/workflows/package.yml`: `make arch`, the `.deb` and BSD make on
+  FreeBSD 15.1, each reinstalled over an edited configuration; the systemd
+  and rc.d services run; a home install with `SERVICE=no` and
+  `lw config init`; and `make image image-check`.
 
 What remains is the maintainer's: it needs push rights, an identity or a
 release tag, and both skeletons download the tagged source.