v0.33.0 tui tap for live view and console enhancements

doc/README.md

@@ -27,8 +27,8 @@ streams, or downstream LogWisp nodes.
   certificate-based authorization
 - [Password Authentication](scram-auth-plan.md): design and rationale for
   Argon2id-SCRAM authentication; mTLS hardening
-- [CLI](cli.md): flags, signals, exit codes, `lw auth`; also the `lw(1)`
-  manual, [`lw.1`](lw.1)
+- [CLI](cli.md): flags, signals, exit codes, `lw --tui`, `lw auth`; also
+  the `lw(1)` manual, [`lw.1`](lw.1)
 - [Operations](operations.md): running, monitoring, tuning, troubleshooting
 - [To Do](todo.md): planned work: packaging, `lw --tui` and its shared configuration engine
 

doc/architecture.md

@@ -48,7 +48,11 @@ Package map:
 - `internal/core`: shared types (`LogEntry`, `TransportEvent`), capabilities,
   constants.
 - `internal/tokenbucket`: rate limiter primitive.
-- `internal/sanitize`: standalone hex-escaping helpers.
+- `internal/tui`: `lw --tui`, the diagram and the live view of what `t`
+  runs; `cmd/lw/screen.go` runs the pipelines for it.
+- `internal/logline`: reads a line of any common shape into time, level,
+  tag, message and fields for the live view; holds the one rule for runes a
+  terminal acts on, which the console sink shares.
 
 ## Plugin Registration
 
@@ -74,7 +78,9 @@ instance of a single type fails, naming both pipelines.
    the source's `dropped_entries` counter. The console source alone waits,
    since stdin can simply be read later.
 2. **Flow** applies, in order: rate limit → filter chain → formatter. A drop at
-   any stage ends the entry's life and increments `flow.total_dropped`.
+   any stage ends the entry's life and increments `flow.total_dropped`. The
+   verdict, passed or the stage that dropped it (a filter by its index), goes
+   to the pipeline's tap when a viewer set one, as `lw --tui`'s live view does.
 3. The formatter output becomes a `core.TransportEvent`, which carries both the
    formatted `Payload` and the original structured `Entry`.
 4. **Dispatch** sends the event to every sink's input channel with a
@@ -107,10 +113,15 @@ loses nothing. Each stage drops and counts instead of waiting:
   `flow.total_dropped`.
 - Pipeline → sink: a full sink input drops it for that sink only; pipeline
   `total_dropped_by_sink`. A sink declaring `core.CapBackpressure`, the console
-  sink, makes dispatch wait instead, so a filter loses no line to a slow
-  reader and its pipeline runs at the reader's pace; `Stop` ends the wait.
+  and null sinks, makes dispatch wait instead, so a filter loses no line to a
+  slow reader and its pipeline runs at the reader's pace; `Stop` ends the wait.
 - TCP/HTTP sink → client queue: a full queue drops it for that client only;
   sink `dropped_writes`.
+- Flow → tap: a full tap drops the record, copying nothing, which the live
+  view counts as "screen missed". It copies the first 16 KiB of a longer
+  message, fields or payload and 256 bytes of a level, source or node, so a
+  viewer's rows pin no source's line, and takes dropped entries only while
+  the view shows them.
 
 The `tcp_chain` sink is the one deliberate exception. It holds a line across
 reconnects until it is written or the process shuts down, so a downstream
@@ -202,7 +213,8 @@ Reload (signal or file watch) rebuilds the entire service:
 
 Because this is a full rebuild, listening sockets close and reopen and all
 clients are disconnected. Application logging is configured once at startup and
-is **not** re-applied on reload.
+is **not** re-applied on reload. `t` on the `lw --tui` screen reruns through
+the same path, giving each new service the screen's tap before it starts.
 
 Watch errors do not rebuild services. Queued changes are combined, and an
 unchanged pipeline/status snapshot is skipped. Signals always rebuild for

doc/cli.md

@@ -335,6 +335,8 @@ A reload rereads the file and keeps the command-line or environment pipelines.
 
 - `SIGINT`, `SIGTERM`: graceful shutdown
 - `SIGHUP`, `SIGUSR1`: reload the configuration
+- On the `lw --tui` screen, `SIGTERM` and `SIGHUP` close it, stopping what
+  `t` runs, and lw exits 0
 
 `SIGHUP` is ignored during startup, before the signal handler is installed, so
 LogWisp survives a terminal hang-up like `nohup`. Once running, it triggers a
@@ -469,7 +471,7 @@ kill -USR1 $(pidof lw)
 
 `lw --tui` (`-T`) composes pipelines on a full-screen terminal, drawn as the
 website's configurator draws them, with the engine the website runs as
-WebAssembly.
+WebAssembly; `t` runs them under the screen and shows what flows.
 
 - Start: the pipelines `lw` would run with the same options (`-c FILE`,
   pipeline flags, `LOGWISP_SOURCE`, a discovered file); with none but the
@@ -480,8 +482,10 @@ WebAssembly.
   type, whether it listens or dials, and a summary; the four flow stages in
   their order in a dashed box, the ones off dimmed. The inspector, the
   chosen part's options with radio buttons for few choices, sits below the
-  canvas, and beside it from 110 columns. Narrow panes stack labels, wrap
-  text and scroll dialogs; the status line wraps to three rows.
+  canvas, and beside it from 110 columns, where each part also lists up to
+  four options it sets; wide screens size the columns to their content and
+  centre them. Narrow panes stack labels, wrap text and scroll dialogs; the
+  status line wraps to three rows.
 - Keys: arrows or `hjkl` move; Enter inspects a part, opening on its first
   required key, then edits a value (empty returns it to its default), opens
   a list's choices or adds on an add row; in a value, Enter sets it, and Tab
@@ -490,23 +494,71 @@ WebAssembly.
   changes a source's or sink's type in its place; `d` deletes, or returns a
   value to its default; Space turns a flow stage on or off; `J`/`K` move a
   filter; `[` `]` switch pipelines; `p` presets; `!` shows every problem,
-  moving to the first; `r` runs; `o` output; `?` keys; `q` quits.
+  moving to the first; `r` runs; `t` taps; `o` output; `?` keys; `q` quits.
 - Pipelines: Up from a column's top row reaches the bar, where `[` `]` or
   Left/Right switch, `a` adds one (empty or from a preset), Enter renames and
   `d` deletes, asking first when it has parts.
 - Output, `o`, each picked by its letter: `r` runs the pipelines with the
   other options given (so `--check` and `--dump` apply to them, and reloads
-  keep them), or `c`, `e` and `f` print the command line, the environment
-  variables (one pipeline) or the file's pipelines on standard output once
-  the screen closes.
+  keep them), `t` taps them, or `c`, `e` and `f` print the command line, the
+  environment variables (one pipeline) or the file's pipelines on standard
+  output once the screen closes.
+- Tap, `t`: runs the pipelines in this process as `r` would, without the
+  status reporter, and opens the live view. It runs for real: listeners
+  bind, dialers connect and log in, file sinks write.
+  - The terminal stays the screen's: a console sink writing to it runs as a
+    null sink with its id, and the view shows what it would print; a console
+    source needs piped input (`cmd | lw -T`); lw's own log, when it writes to
+    the terminal, shows in the view as source `lw`; a password the terminal
+    would be asked for is asked in a masked dialog.
+  - Leaving the view keeps the pipelines running. `t` after an edit reruns
+    them as a reload does: when the new ones do not build, the earlier ones
+    keep running and the error chooses the part it names. `q` asks before it
+    stops them, `r` and `o`'s choices stop them first, and `SIGTERM` or
+    `SIGHUP` (an ssh session's end) stops them and closes the screen.
+  - The bar counts the shown pipeline's entries in, filtered by the flow,
+    dropped by sinks, and missed by the screen: the view never slows a
+    pipeline.
+- Live view: from 40 rows it sits under the diagram; shorter, it takes the
+  screen while it has the keys. `t` or Esc gives them back to the diagram.
+  - The canvas cursor picks its point, so the view opens on the first
+    source: a source shows its entries, a flow stage (or the filter the
+    inspector is on) what reaches it, the heartbeat its own, the sinks (or
+    no part) what they got. `v` switches between all, what passed and what
+    was dropped there, with no restart. lw's own log shows from info up at
+    every point; `s` also offers it alone, debug too.
+  - A row is the entry's time, level, tag, message and fields, read from any
+    line shape: LogWisp's json, lw's own log, vif records, pino, zap, slog,
+    docker, journald, ECS, GCP, or text after an RFC3339 stamp. A narrow
+    screen drops columns; a dropped entry is marked; a rune a terminal would
+    act on or draw wider than a cell shows as a placeholder.
+  - Keys: arrows or `jk` move (up pauses, down onto the newest follows),
+    PgUp/PgDn; `g` the oldest, `G` or `f` the newest; Enter the row's
+    detail, `J`/`K` scroll it; `/` only rows matching a pattern, ignoring
+    case when it is all lower case (clearing it, or a level, keeps the
+    focused row in place, paused there unless it is the newest); `1`-`5`
+    trace to error and up, `0` all; `s` chooses the point;
+    `i` inspects the stage that dropped the row; `c` clears; `q`, `?`, `r`,
+    `o`, `[` and `]` act as on the diagram.
+  - The detail, beside the rows from 160 columns and under them otherwise:
+    time and the stamp as written, level, origin, the verdict with a
+    filter's patterns, the layout read, message, fields, and the payload as
+    the sinks receive it, with escapes for what a cell cannot show.
+  - It keeps 32 MiB of rows, each entry's message, fields and payload cut to
+    16 KiB, and redraws at most every 100 ms while entries flow, at once on
+    a key, never when idle. A closed standard output or error fails the
+    write (logged) rather than end lw with the terminal raw.
 - The screen draws on the controlling terminal: `lw -T > pipelines.toml` and
   `cmd | lw -T` keep standard input and output as data. lw never writes the
   file it starts from, but the shell empties a redirect's target before lw
   reads it: redirect to a file lw does not load.
 - Color follows the `color` setting: `auto` the terminal and `NO_COLOR`,
   `always` the terminal, `never` none. True color and 256 colors paint the
-  website's dark theme; 16 colors and none keep the terminal's own; a text
-  console draws with CP437 glyphs, and a terminal outside UTF-8 with ASCII.
+  website's dark theme; 16 colors a console look on the terminal's own
+  background (bright white text, its grey for hints, a blue cursor bar that
+  keeps each tint); none the terminal's own colors, the cursor row one
+  reverse bar. A text console draws with CP437 glyphs, and a terminal outside
+  UTF-8 with ASCII.
 
 ## `lw auth`
 

doc/lw.1

@@ -1,4 +1,4 @@
-.Dd October 5, 2026
+.Dd October 10, 2026
 .Dt LW 1
 .Os
 .Sh NAME
@@ -163,6 +163,15 @@ and
 .Fl \-dump
 also do, or print them as a command line, environment variables or a
 configuration file's pipelines.
+On the screen,
+.Ic t
+runs them in place, for real: listeners bind, dialers connect, file sinks
+write.
+A live view then shows each entry with its fate, passed or the flow stage
+that dropped it, and the log of
+.Nm ;
+a console sink writing to the terminal runs as a null sink there, and a
+console source needs piped input.
 The screen draws on the controlling terminal, so standard input and output
 stay data:
 .Ql lw \-T > pipelines.toml .
@@ -756,6 +765,16 @@ Before the service runs,
 is ignored, as under
 .Xr nohup 1 .
 .El
+.Pp
+On the
+.Fl \-tui
+screen,
+.Dv SIGTERM
+and
+.Dv SIGHUP
+close it, stopping what it runs, and
+.Nm
+exits 0.
 .Sh EXAMPLES
 Copy a file line by line:
 .Bd -literal -offset indent

doc/security.md

@@ -745,6 +745,8 @@ edge. `password_file` is read by what it names, on either side:
     forwarded, and stderr is often a file.
   - With no terminal (systemd, cron) the plugin fails. `lw ... &` stops on
     `SIGTTIN` at the prompt.
+  - Under `lw --tui`'s `t`, the screen asks in a masked dialog instead; a
+    generated password stays shown until a key closes it.
 
 Trade-offs:
 
@@ -1094,4 +1096,5 @@ The `raw` format passes control characters through. A console sink writing to
 a terminal escapes them itself (`escape = "auto"`), so a log line cannot move
 the cursor, retitle the window, write the clipboard (OSC 52) or reorder text
 with bidi controls; pipes and files get the bytes unchanged. See
-[Sinks](sinks.md#console).
+[Sinks](sinks.md#console). `lw --tui`'s live view applies the same rule to
+everything it draws.

doc/sinks.md

@@ -122,7 +122,8 @@ id   = "discard"
 type = "null"
 ```
 
-No options. The input queue is fixed at 1000.
+No options. The input queue is fixed at 1000; a full one makes the pipeline
+wait, which costs nothing as it empties at once, rather than count a drop.
 
 ---
 

doc/todo.md

@@ -74,11 +74,12 @@ release tag, and both skeletons download the tagged source.
 
 ## 2. `lw --tui` and the configuration engine it shares with lixen.com
 
-`lw --tui` composes pipelines in a full-screen terminal UI, then either runs
-them at once or prints one form of them, chosen on exit: a command line,
-environment variables, or a configuration file. Its logic is a pure engine that
-lixen.com's planned Configure tab (lixencom `doc/todo.md`) runs as WebAssembly,
-so no part of LogWisp is rewritten in JavaScript.
+`lw --tui` composes pipelines in a full-screen terminal UI, runs them under
+the screen with a live view (`t`), and on exit runs them at once or prints one
+form of them: a command line, environment variables, or a configuration file.
+Its logic is a pure engine that lixen.com's planned Configure tab (lixencom
+`doc/todo.md`) runs as WebAssembly, so no part of LogWisp is rewritten in
+JavaScript.
 
 What exists:
 - Option structs with toml tags for every plugin and flow stage
@@ -117,7 +118,8 @@ Steps:
    optional, in `color`: export the 16 xterm reference shades (RGBTo16 maps
    the site's hues badly, so the 16-color tier is chosen by hand). v0.2.1:
    radio buttons, narrow form fields and option lists, menus, frames, rules,
-   wires and windows, moved from `lw --tui`.
+   wires and windows, moved from `lw --tui`. Pinned at the untagged v0.2.2
+   commit: one glyph per cell at output, a reverse cursor bar in one style.
 5. `lw --tui`: done ([doc/cli.md](cli.md#lw---tui)).
 6. WebAssembly: done. `make wasm` writes `bin/lwconf/`, which a site drops
    in whole:
@@ -154,11 +156,23 @@ Steps:
    - Pipelines are a JSON list in the file's shape (`name`, `flow`,
      `plugin_sources`, `plugin_sinks`); a misspelled key is refused. A node
      is `{role, id, index}`: a source's or sink's id, a filter's index.
+7. `t`, the tap: done ([doc/cli.md](cli.md#lw---tui)): `core.Tap` takes each
+   flow verdict, `internal/logline` reads any line shape, and the live view
+   shows them under the diagram.
 
-Still to verify: a scripted session on a pseudo-terminal for Run and each
-output.
-
-Still to derive: `AuthOptions.Check` alone limits `auth.node_binding` to chain
-sources and `auth.trusted_proxies` to the http sink, so the schema lists both
-on every listener; a side tag naming those sides would filter them as `http`
-filters HTTP's keys.
+Not yet:
+- Pins and export in the live view, as vif-log has them: pinned rows survive
+  filter changes and show alone on a key; the pins or the shown rows go to a
+  new file.
+- Per-sink taps: what each sink took, where the tap at the flow's exit shows
+  what dispatch offered (a full sink input drops for that sink only).
+- Live filter patching: a filter edited while `t` runs takes effect through
+  `Filter.UpdatePatterns` (unused), not a rerun that rebinds every listener.
+- The wide-rune table `internal/tui/view.go` copies from
+  `lixenwraith/terminal`'s one-cell rule goes once terminal exports that rule.
+- A scripted session on a pseudo-terminal for Run, each output and `t` (keys,
+  the password dialog, `SIGHUP`), on a text console and over ssh.
+- `AuthOptions.Check` alone limits `auth.node_binding` to chain sources and
+  `auth.trusted_proxies` to the http sink, so the schema lists both on every
+  listener; a side tag naming those sides would filter them as `http`
+  filters HTTP's keys.