v0.18.0 raw and json formatter improvements, dockerfile, go bump to 1.27.1

doc/formatters.md

@@ -30,8 +30,10 @@ Omitting `[pipelines.flow.format]` entirely selects `raw`.
 
 ### raw
 
-Passthrough. `FlagRaw` bypasses both formatting and sanitization, so the
-message reaches the sink exactly as the source produced it.
+Passthrough. `FlagRaw` bypasses formatting and sanitization: the message reaches
+the sink exactly as the source produced it, with no timestamp, level or source
+prefix added. An entry that also carries `fields` gets the fields JSON appended
+verbatim after a single space — `raw` never drops data and never re-encodes it.
 
 ```toml
 [pipelines.flow.format]
@@ -42,6 +44,12 @@ Fastest option, and the right one when you are relaying text that is already in
 its final form. Note that it also bypasses sanitization, so control characters
 in the source data reach your sinks intact.
 
+Byte-exact transport needs a source that does not split the line: the `console`
+source, or the `file` source with `raw = true`. Both put the whole line —
+newline included — in the message and leave `fields` empty. The `file` source's
+JSON branch splits a line into message and fields, so `raw` reassembles it as
+`<msg> <fields>` rather than reproducing the original object.
+
 ### txt
 
 Human-readable line output with a timestamp and level.
@@ -89,7 +97,7 @@ you need to override the defaults.
 
 With `flags = 0` the formatter selects `1` for `type = "raw"` and `6`
 (timestamp + level) for every other type. `8` is added automatically whenever an
-entry carries parseable `fields`.
+entry carries parseable `fields` and `1` is not set; `1` always wins.
 
 Examples: `flags = 4` for level only, no timestamp; `flags = 2` for timestamp
 only, no level.
@@ -133,11 +141,11 @@ when writing downstream parsers or grep patterns.
 
 ## Structured Fields
 
-When an entry carries `Fields` (raw JSON), the formatter parses it and switches
-to structured rendering by adding the `StructuredJSON` flag automatically.
-Fields reach a pipeline in two ways: from the `file` source when a tailed line
-parses as JSON with a `fields` key, and from the heartbeat generator when
-`include_stats = true`.
+When an entry carries `Fields` (raw JSON) and `FlagRaw` is not set, the
+formatter parses it and switches to structured rendering by adding the
+`StructuredJSON` flag automatically. Fields reach a pipeline in two ways: from
+the `file` source when a tailed line parses as JSON with a `fields` key, and
+from the heartbeat generator when `include_stats = true`.
 
 ## Choosing a Configuration
 

doc/installation.md

@@ -4,7 +4,7 @@
 
 - **Operating systems**: Linux (kernel 6.10+), FreeBSD (14.0+)
 - **Architecture**: amd64
-- **Go**: 1.26 or newer, to build from source
+- **Go**: 1.27.1 or newer, to build from source
 
 ## Building from Source
 
@@ -37,6 +37,24 @@ go build -o bin/logwisp ./cmd/logwisp
 `go install github.com/lixenwraith/logwisp/cmd/logwisp@latest` also works, with
 the same loss of version metadata.
 
+## Container Image
+
+The root `Dockerfile` builds the same package into `scratch` under UID 65532,
+static and stripped. There is no shell and no config in the image: mount one and
+name it, as the binary has no daemon mode and no built-in defaults worth running.
+
+```bash
+REV=$(git rev-parse HEAD)
+docker build -t "logwisp:$(git rev-parse --short HEAD)" \
+  --build-arg VERSION="$(git describe --tags --always)" \
+  --build-arg REVISION="$REV" .
+docker run --rm -v /etc/logwisp:/etc/logwisp:ro logwisp:... -c /etc/logwisp/logwisp.toml
+```
+
+Sinks that listen (`http`, `tcp`) need their ports published; the read-only
+root filesystem and dropped capabilities a restricted runtime imposes are all
+compatible with it, provided a `file` sink's directory is writable by 65532.
+
 ## Configuration
 
 Copy the annotated reference configuration and edit it:
@@ -173,27 +191,20 @@ mode; see [Operations](operations.md#checking-a-configuration).
 
 ## Test Scripts
 
-Two end-to-end scripts under `test/` build multi-node chain topologies against a
-local build:
+End-to-end scripts under `test/` run against a local build:
 
 ```bash
 make
 ./test/chain-test.sh --auto             # two independent relay pipelines
 ./test/chain-aggregate-test.sh --auto   # fan-in: both edges into one pipeline
+./test/mtls-chain-test.sh --auto        # the same fan-in under mTLS
+./test/passthrough-test.sh              # file source relays a wide envelope intact
 ```
 
-Without `--auto` they run the relay in the foreground for interactive
-inspection. They need bash 5+, coreutils, and curl, and they bind ports
-15801–15804. Generated configuration and logs land in `test/run/`.
-
-> Two of the three `--auto` assertions currently report `FAIL` against a
-> working build. They grep the sink output for `"source":"edge-tcp/` and
-> `"node":"edge-http"`, but the JSON formatter emits the `node/source` label
-> under the key `trace`. The transport itself is healthy — the
-> `total_processed` assertion passes and the streamed entries carry
-> `"trace":"edge-tcp/random_rand"` as expected. Until the assertions are
-> updated, verify the streams by eye with `nc 127.0.0.1 15803` and
-> `curl -sN http://127.0.0.1:15804/stream`.
+Without `--auto` the chain scripts run the relay in the foreground for
+interactive inspection. They need bash 5+, coreutils, and curl, and they bind
+ports 15801–15804. The pass-through test binds nothing. Generated configuration
+and logs land in `test/run/`.
 
 ## Uninstall
 

doc/operations.md

@@ -231,7 +231,8 @@ pipeline `total_dropped_by_sink` (sink backed up?), sink `total_processed`.
 
 - The watcher seeks to end-of-file on start; only content appended afterwards is
   read. Positions are in memory, so a restart re-seeks to end and anything
-  written during the downtime is lost.
+  written during the downtime is lost. `from = "start"` reads each file whole
+  instead, and replays it on every restart.
 - `pattern` is a filename glob with `*` and `?` only, and matching is not
   recursive.
 - `check_interval_ms` governs how quickly a *new file* is noticed; tailing an

doc/sources.md

@@ -32,6 +32,8 @@ type = "file"
 directory         = "/var/log/myapp"
 pattern           = "*.log"
 check_interval_ms = 100
+raw               = false
+from              = "end"
 ```
 
 | Option | Type | Default | Description |
@@ -39,6 +41,8 @@ check_interval_ms = 100
 | `directory` | string | **required** | Directory to scan; not recursive |
 | `pattern` | string | `*` | Glob over filenames; `*` and `?` only |
 | `check_interval_ms` | int | `100` | Directory rescan interval; minimum `10` |
+| `raw` | bool | `false` | Never parse a line: the whole line is the message |
+| `from` | string | `end` | Where a new watcher starts: `end` or `start` of the file |
 
 **Behaviour**
 
@@ -49,15 +53,23 @@ check_interval_ms = 100
   stopped and removed on the next scan.
 - A new watcher seeks to end-of-file. Positions live in memory only, so a
   restart resumes from the current end of each file and content written while
-  LogWisp was down is not read.
+  LogWisp was down is not read. `from = "start"` reads each file whole when its
+  watcher is created instead — what a process writing beside LogWisp needs, at
+  the cost of replaying a file already on disk at every restart.
 - Rotation is detected from size decrease, modification-time reset, a position
   beyond end-of-file, or an inode change. An inode change where the new file is
   already larger than the recorded position is treated as an atomic save, not a
   rotation, and the position is preserved.
-- Lines are parsed as JSON when they contain `time`, `level`, `msg`, and
-  `fields` keys; `time` is read as RFC3339Nano. Anything else is kept as plain
-  text with the level inferred from common markers (`[ERROR]`, `WARN:`, and so
-  on).
+- A line is parsed as JSON only when it is an object whose top-level keys are
+  all drawn from `time`, `level`, `msg` and `fields` — the four an entry can
+  carry. `time` is read as RFC3339Nano. Any other key, and any non-object line,
+  is kept whole as text with the level inferred from common markers
+  (`[ERROR]`, `WARN:`, and so on), because parsing it would drop the rest.
+- `raw = true` skips the JSON branch entirely. The line, plus its newline,
+  becomes the message; `fields` stays empty, the time is the read time, and the
+  level is inferred from the text as for any unparsed line. Paired with
+  `format.type = "raw"` this is byte-exact transport for records LogWisp's
+  envelope cannot hold — see [Formatters](formatters.md#raw).
 - `Source` is set to the file's base name.
 
 **Statistics**: per-watcher size, position, entries read, rotation count, and