v0.32.0 lwconf engine update: config by path, schema from code, wasm

doc/cli.md

@@ -69,9 +69,18 @@ pipelines have [their own flags](#pipelines):
   resolved into it, as TOML that `-c` reads back, and exit
 - `-T`, `--tui`: compose the pipelines on screen, then run or print them
   ([below](#lw---tui))
-- `--schema`: print every setting, pipeline flag, flow stage, plugin option
-  and preset as one JSON document (kinds, defaults, rules, help), and exit;
-  tools build their option lists from it
+- `--schema`: print every setting, pipeline flag, flow stage, plugin option,
+  preset and output form as one JSON document, and exit; tools build their
+  option lists from it:
+  - a key: its kind, default, rules and help; a table (`tls`, `auth`, `acl`)
+    holds its own keys, those of the plugin's side (listener or dialer),
+    HTTP's only on a plugin that speaks HTTP. Every listener lists
+    `auth.node_binding` and `auth.trusted_proxies`, which validation admits
+    only on chain sources and on the http sink
+  - a flow stage: its keys, and `on`, the values that turn it on
+  - a preset key: its default and help, and the choices (`enum`) and `hint`
+    of the option it sets
+  - `forms`: `command`, `environment` and `file`, each with its hint
 - `--color [WHEN]`, WHEN `auto`, `always` or `never`: level names in color on
   console sinks, default `auto` (a terminal, `NO_COLOR` unset, `TERM` not
   `dumb`); bare, `always`. A console sink's own `color` wins
@@ -421,14 +430,22 @@ lw -c dev.toml --logging.level=debug
 `lw --check` validates without running: it builds every pipeline and plugin as
 a start would (options, TLS files, credentials files, startup warnings), but
 binds no port, reads no stdin, and opens, reads or creates no log file; it
-prints `configuration ok: N pipeline(s)` and exits 0, or the error and exits 1
-(2 when a named file is missing, see [Exit Codes](#exit-codes)):
+prints `configuration ok: N pipeline(s)` and exits 0. Otherwise it prints the
+first problem of each setting, flow stage, source, sink and pipeline, a line
+each named by its key's path, or else the first file that fails to open, and
+exits 1 (2 when a named file is missing, see [Exit Codes](#exit-codes)):
 
 ```bash
 lw --check -c /etc/logwisp/logwisp.toml
 lw --check --source file:directory=/var/log/app --sink http:port=8080
 ```
 
+```text
+Error: Failed to load config: invalid configuration:
+pipelines[0].plugin_sources[file].config.directory: must be set
+pipelines[0].plugin_sinks[tcp].config.port: must be from 1 to 65535, got 70000
+```
+
 **Production**
 
 ```bash
@@ -472,8 +489,8 @@ WebAssembly.
   Space flips it; Esc goes back. `a` adds a source, sink or filter; `c`
   changes a source's or sink's type in its place; `d` deletes, or returns a
   value to its default; Space turns a flow stage on or off; `J`/`K` move a
-  filter; `[` `]` switch pipelines; `p` presets; `!` shows the first
-  problem; `r` runs; `o` output; `?` keys; `q` quits.
+  filter; `[` `]` switch pipelines; `p` presets; `!` shows every problem,
+  moving to the first; `r` runs; `o` output; `?` keys; `q` quits.
 - Pipelines: Up from a column's top row reaches the bar, where `[` `]` or
   Left/Right switch, `a` adds one (empty or from a preset), Enter renames and
   `d` deletes, asking first when it has parts.

doc/installation.md

@@ -26,9 +26,12 @@ The Makefile works with GNU make and BSD make alike. Targets:
   - `make build`: `bin/lw`, with version, commit and build time from git
   - `make release`: the same, static (`CGO_ENABLED=0`), `-trimpath`, stripped
   - `make dev`: built with the race detector
-  - `make wasm`: `bin/lwconf.wasm`, the configuration engine for the
-    website, version stamped, beside the toolchain's `wasm_exec.js`; prints
-    both checksums
+  - `make wasm`: `bin/lwconf/`, the configuration engine a website drops in:
+    `lwconf.wasm` (version stamped), the toolchain's `wasm_exec.js`,
+    `lwconf.js` that loads and calls it, `schema.json` (`lw --schema`) and
+    `lwconf.json`, their manifest: version, commit, Go version and each
+    file's sha256. Stamped with the commit's time, a commit's files are the
+    same on every build
   - `make version`: the metadata a build would embed
   - `make completion`: regenerate the bash, zsh and fish completion in
     `deploy/package/completion/` from `cmd/lw`'s `shorts` and `commands`

doc/lw.1

@@ -141,7 +141,10 @@ Build every pipeline and plugin as a start would, loading options, TLS files
 and credentials files and printing startup warnings, without binding a port
 or opening, reading or creating a log file; print
 .Dq configuration ok
-and exit 0, or the error and exit 1, or 2 when a named configuration file is
+and exit 0.
+Otherwise print the first problem of each setting, flow stage, source, sink
+and pipeline, a line each named by its key's path, or else the first file
+that fails to open, and exit 1, or 2 when a named configuration file is
 missing
 .Pq see Sx EXIT STATUS .
 .It Fl \-dump
@@ -167,9 +170,13 @@ A file is read, never written; the shell empties a redirect's target before
 .Nm
 reads it, so the redirect names a file it does not load.
 .It Fl \-schema
-Print every setting, pipeline flag, flow stage, plugin option and preset as
-one JSON document, with kinds, defaults, rules and help, and exit; tools build
-their option lists from it.
+Print every setting, pipeline flag, flow stage, plugin option, preset and
+output form as one JSON document, with kinds, defaults, rules and help, and
+exit; tools build their option lists from it.
+A plugin's table holds the keys of its side; a flow stage carries
+.Ql on ,
+the values that turn it on; a preset key carries the choices and hint of the
+option it sets.
 .It Fl q , Fl \-quiet
 Turn off the log of
 .Nm

doc/operations.md

@@ -100,15 +100,16 @@ lw --check -c candidate.toml
 ```
 
 Before a reload, check the edited file this way: a failed reload keeps the
-running service, but `--check` says why without touching it. Failures name the
-pipeline and the offending key:
+running service, but `--check` says why without touching it. It prints the
+first problem of each setting, flow stage, source, sink and pipeline, a line
+each named by its key's path; once none is left, the files they name are
+opened as a start opens them:
 
+```text
+Error: Failed to load config: invalid configuration:
+pipelines[0].plugin_sources[in].config.directory: must be set
+pipelines[0].plugin_sinks[out].config.port: must be from 1 to 65535, got 70000
 ```
-ERROR msg="Failed to create pipeline" pipeline=app error="failed to create sink out: port: must be 1-65535, got 0"
-```
-
-Remember that most validation lives in plugin constructors, so a config only
-proves itself when the pipeline is actually built.
 
 ## Monitoring
 

doc/todo.md

@@ -89,17 +89,18 @@ What exists:
   on the option structs, one catalogue row per plugin type, `Decode` (every
   constructor decodes through it and keeps only I/O), `Coerce` for typed
   command-line values, and a `ValidateConfig` that decodes every plugin
-  offline and names the refused key's path.
+  offline and names the first problem of each part by its key's path.
 - `internal/config` already compiles for `GOOS=js`; it links no net/http,
   crypto/tls or terminal code.
 - The engine, `internal/compose`: starts empty, from a preset (whose path
-  `config.IsDir` resolves, nil off the host), a loaded configuration or a
-  pasted command line; adds, removes and moves sources, sinks and filters,
-  sets and unsets keys, validates; writes a command line, environment
-  variables or a file, each loading back to the same pipelines. Its test
-  fails when it links TLS, HTTP, process, signal or terminal code.
-- `lw --schema`: settings, pipeline flags, flow stages in order, the tls,
-  auth and acl tables, sources, sinks and presets, as one JSON document.
+  `config.IsDir` resolves, from answers off the host), a loaded
+  configuration or pasted text of any form; adds, removes and moves sources,
+  sinks and filters, sets and unsets keys, validates; writes a command line,
+  environment variables or a file, each reading back to the same pipelines.
+  Its test fails when it links TLS, HTTP, process, signal or terminal code.
+- `lw --schema`: settings, pipeline flags, flow stages in order with what
+  turns each on, sources and sinks with their tables' keys of their side,
+  presets and forms, as one JSON document.
 - `github.com/lixenwraith/terminal` (pinned) and its `tui` package: regions,
   layout, boxes, List, Tree, TabBar, StatusBar, TextField, Form, Modal,
   ConfirmDialog, scrolling, mouse; themed Radio, Field, TextInput,
@@ -118,21 +119,46 @@ Steps:
    radio buttons, narrow form fields and option lists, menus, frames, rules,
    wires and windows, moved from `lw --tui`.
 5. `lw --tui`: done ([doc/cli.md](cli.md#lw---tui)).
-6. WebAssembly: done. `make wasm` builds `bin/lwconf.wasm`, version stamped,
-   beside the toolchain's `wasm_exec.js`, and prints both checksums: about
-   6.9 MB, 1.9 MB gzipped. `wasm_exec.js` uses no eval, so the page needs only
-   `'wasm-unsafe-eval'` in its `script-src`. The page runs the module with
-   `new Go().run(instance)`, then calls `globalThis.lwconf`; each call takes
-   strings and returns a JSON object, `{"value": ...}` or `{"error": "..."}`:
+6. WebAssembly: done. `make wasm` writes `bin/lwconf/`, which a site drops
+   in whole:
+   - `lwconf.wasm`, version stamped: 7.1 MB, 1.9 MB at gzip -9.
+   - `wasm_exec.js`, the toolchain's, which uses no eval: the page needs only
+     `'wasm-unsafe-eval'` in its `script-src`.
+   - `lwconf.js`, one dependency-free ES module that knows the calls and
+     shapes, never a key: `load({execURL, execSRI, wasmURL, wasmSRI})`
+     fetches both by their integrity and resolves to the engine;
+     `fields(schema, role, type)` is a node's form rows, a table's a group;
+     `value`, `at` and `group` read a node's value and sort problems by node.
+   - `schema.json`, equal to `schema()`, and `lwconf.json`, the manifest:
+     version, commit, Go version and each file's sha256.
+
+   The engine's calls take strings (`lwconf.js` writes objects as JSON) and
+   answer `{"value": ...}` or `{"error", "path", "pipeline", "node", "key"}`,
+   as far as the engine places the refusal:
    - `schema()`: what `lw --schema` prints.
-   - `preset(name, values)`: a preset's pipelines; `values` is a JSON object
-     of strings, and a directory path ends in `/`.
-   - `parse(line)`: a pasted command line's pipelines.
-   - `validate(pipelines)`: the pipelines settled (defaults, kinds, ids), or
-     the first problem by its key path.
-   - `emit(pipelines, form)`: `command`, `environment` or `file`.
+   - `preset(name, values, dirs)`: a preset's pipelines; `values` maps a key
+     to a string, a list key to strings.
+   - `parse(text, form, dirs)`: the pipelines of a form's pasted text.
+   - `validate(pipelines)`: `{pipelines, problems}`, the pipelines settled
+     (defaults, kinds, ids) and the first problem of each part (a value of
+     the wrong kind too) as `{path, pipeline, node, key, text}`.
+   - `emit(pipelines, form)`: the text of `command`, `environment` or `file`.
+   - `edit(pipelines, op)`: `{pipelines, node}`, the node left chosen. `op` is
+     `{op, pipeline, ...}`: `add_pipeline` and `rename_pipeline` (`name`),
+     `remove_pipeline`, `add` (`role`, `type`), `retype` (`node`, `type`),
+     `remove` (`node`), `move_filter` (`from`, `to`), `set` (`node`, `key`,
+     `values`), `unset` (`node`, `key`).
+   - `dirs` answers whether a path a preset names is a directory,
+     `{"/var/log/app.log": false}`; unanswered, a path ending in `/` is one,
+     one with a glob a pattern, and any other is refused at `path`.
    - Pipelines are a JSON list in the file's shape (`name`, `flow`,
-     `plugin_sources`, `plugin_sinks`); a misspelled key is refused.
+     `plugin_sources`, `plugin_sinks`); a misspelled key is refused. A node
+     is `{role, id, index}`: a source's or sink's id, a filter's index.
 
 Still to verify: a scripted session on a pseudo-terminal for Run and each
 output.
+
+Still to derive: `AuthOptions.Check` alone limits `auth.node_binding` to chain
+sources and `auth.trusted_proxies` to the http sink, so the schema lists both
+on every listener; a side tag naming those sides would filter them as `http`
+filters HTTP's keys.