v0.21.0 scram password auth, cred and token management, browser tls-terminating proxy, viewer client
Test / test (push) Canceled after 0s

This commit is contained in:
lixen committed 2026-10-02 15:25:51 -04:00
1 parent f192c26fa2
commit 9ab8c356d2
60 files changed
+6791 -747

No files matched your search

+6 -3
View File
@@ -20,8 +20,8 @@ streams, or downstream LogWisp nodes.
| [Networking](networking.md) | Listeners, dialers, timeouts, connection limits |
| [Security](security.md) | TLS, mTLS, and peer authorization; threat model and current limits |
| [mTLS Authentication](mtls-auth-plan.md) | Design and rationale for certificate-based authorization |
| [Password Authentication](scram-auth-plan.md) | Planned Argon2id-SCRAM design; mTLS hardening |
| [CLI](cli.md) | Flags, signals, exit codes |
| [Password Authentication](scram-auth-plan.md) | Design and rationale for Argon2id-SCRAM authentication; mTLS hardening |
| [CLI](cli.md) | Flags, signals, exit codes, `logwisp auth` |
| [Operations](operations.md) | Running, monitoring, tuning, troubleshooting |
A fully annotated configuration covering every option lives at
@@ -63,8 +63,11 @@ endpoint), `tcp` (broadcast server), `null`, and the chain forwarders
present a client identity
- Authorization by certificate identity, per listener: named peers rather than
everything the CA issued, with the `http` sink's endpoints gated too
- Password authentication (Argon2id-SCRAM) bound to the listener's certificate,
with bearer tokens on HTTP listeners and a `logwisp auth` CLI, and browser
logins behind a TLS-terminating proxy
- Node binding, so a chain source labels entries from the sender's certificate
rather than from what the sender claims
identity or username rather than from what the sender claims
See [Security](security.md) for what each layer does and does not give you.
+6 -5
View File
@@ -154,11 +154,12 @@ Idle sessions are reaped every 5 minutes against a 30-minute idle limit. The
HTTP sink's broker treats a vanished session as an eviction signal and closes
the corresponding SSE client.
Authorization decisions do not read session metadata — they are made from the
handshake by `internal/authz`, at the point of connection or request, and their
outcome is *recorded* in the session. That ordering matters: a session exists
only for a peer that was already admitted. See the
[mTLS authentication design](mtls-auth-plan.md).
Authorization decisions do not read session metadata — `internal/authz` makes
them from the handshake, the SCRAM exchange or a bearer token, at the point of
connection or request, and their outcome is *recorded* in the session. That
ordering matters: a session exists only for a peer that was already admitted.
See the [mTLS](mtls-auth-plan.md) and [SCRAM](scram-auth-plan.md)
authentication designs.
## Configuration Reload
+43 -9
View File
@@ -45,14 +45,15 @@ Chained entries carry a `node` label identifying where they originated.
Relays preserve `node`, so a label survives any number of hops and identifies
the original producer rather than the last relay.
Under mTLS the source can instead bind the label to the sender's certificate,
which overrides `trust_node` entirely:
With an `auth` block the source can instead bind the label to the sender's
authenticated identity — its certificate identity under `mtls`, its username
under `scram` — which overrides `trust_node` entirely:
| `auth.node_binding` | Connection label | Per-entry `node` field |
|---------------------|------------------|------------------------|
| `none` | `trust_node` governs | `trust_node` governs |
| `assert` | Must equal the certificate identity, or the peer is rejected | `trust_node` governs |
| `force` (default under `mtls`) | The certificate identity | Overwritten with the identity |
| `assert` | Must equal the identity, or the peer is rejected | `trust_node` governs |
| `force` (default with an `auth` block) | The identity | Overwritten with the identity |
Pick `force` at an ingest boundary you do not trust — it is the only setting
where a compromised edge cannot mislabel its entries, including through the
@@ -64,10 +65,11 @@ Formatters render node identity as a syslog-style prefix on the source field:
`edge-01/app.log`. In JSON output the node therefore appears inside the source
field, not as a separate top-level key.
> `trust_node = true` with no `auth` block means any peer the CA vouches for can
> `trust_node = true` with no `auth` block means any peer that can connect can
> claim **any** node label, including one belonging to another host. On an
> untrusted network set `auth.type = "mtls"` with `node_binding = "force"`;
> `trust_node = false` is the fallback when certificates are not an option.
> untrusted network set an `auth` block (`mtls` or `scram`) and keep
> `node_binding = "force"`; `trust_node = false` is the fallback when neither is
> an option.
## Wire Protocol
@@ -92,6 +94,24 @@ A persistent connection carrying newline-delimited JSON.
Line size is bounded at 1 MiB. An oversized line is a protocol violation and
terminates the connection, because the scanner cannot resynchronize afterwards.
Under `auth.type = "scram"` the hello also carries the SCRAM client-first
message, and three more lines precede the entries, each one JSON object of at
most 4 KiB:
```
dialer {"logwisp":1,"node":"edge-01","scram":{"username":"edge-01","client_nonce":"…"}}
listener {"challenge":{"full_nonce":"…","salt":"…","argon_time":3,"argon_memory":65536,"argon_threads":4}}
dialer {"proof":{"full_nonce":"…","client_proof":"…"},"binding":"<base64 SHA-256 of the server certificate>"}
listener {"final":{"server_signature":"…","username":"edge-01"}}
```
The whole exchange runs within `hello_timeout_ms`; the listener sends the final
line only after node binding agrees, and a refusal at any step is one
`{"error":"…"}` line before the connection closes. The field is additive, so
the protocol version stays `1`: a listener without `scram` answers a hello
carrying it with `{"error":"authentication not enabled"}`, while an older
binary ignores it and the dialer gives up after 10 s.
### HTTP transport
Batches of NDJSON delivered by `POST`, with the preamble expressed as headers.
@@ -101,10 +121,18 @@ Batches of NDJSON delivered by `POST`, with the preamble expressed as headers.
| `X-Logwisp-Protocol` | request | Protocol version; must be `1` |
| `X-Logwisp-Node` | request | Origin node label |
| `Content-Type` | request | `application/x-ndjson` |
| `Authorization` | request | `Bearer <token>`, under `scram` |
| `X-Logwisp-Accepted` | response | Number of entries ingested |
Responses: `204` on success, `400` for a bad protocol version or a malformed
body, `413` when the body cap is exceeded, `405` for a non-`POST` method.
body, `413` when the body cap is exceeded, `405` for a non-`POST` method, `401`
for a missing or expired token, `403` for a refused certificate or node label.
Under `scram` the sink first logs in with two `POST /auth` requests carrying the
same JSON as the TCP lines: the hello (`200` and the challenge), then the proof
(`200` with the final message, `token` and `expires_in` in seconds). A refusal is
`{"error":"…"}`: `401` for a failed proof, `429` when throttled, `503` when the
listener is busy, `400` or `413` for a malformed or oversized body.
### Entry encoding
@@ -127,7 +155,7 @@ in at ingest.
| Transport | Guarantee | Failure behaviour |
|-----------|-----------|-------------------|
| `tcp_chain` | Per-line, held across reconnects | Retries with exponential backoff plus ±20 % jitter until written or shutdown; back-pressure appears upstream as `total_dropped_by_sink` |
| `http_chain` | At-least-once per batch | Retries transport errors, `408`, `429`, `5xx`; drops on any other non-2xx (`dropped_batches`) |
| `http_chain` | At-least-once per batch | Retries transport errors, `408`, `429`, `5xx` and, under `scram`, failed logins and `401`; drops on any other non-2xx (`dropped_batches`) |
`http_chain` batches can be delivered twice when a successful request's response
is lost. There is no de-duplication downstream; design your consumers to
@@ -219,6 +247,12 @@ Entries arriving on this relay are labelled `edge-01` or `edge-02` because that
is what their certificates say, regardless of the `node` each edge configured.
`test/mtls-chain-test.sh` builds exactly this shape against a throwaway PKI.
To use passwords instead of client certificates, replace both `auth` blocks:
`type = "scram"` with `username` and `password_file` on the edge, and with a
`credentials_file` on the relay. `client_auth` and the edge's certificate become
optional, and entries are labelled with the username. See
[Password Authentication](security.md#password-authentication-scram).
## Operational Notes
- **Formatting is a relay decision.** Because chain links carry structured
+71 -4
View File
@@ -2,14 +2,14 @@
```
logwisp [options]
logwisp auth <command> [flags]
logwisp help | -h | --help
logwisp --version
```
LogWisp has no subcommands. Earlier releases shipped `logwisp auth` and
`logwisp tls` for credential and certificate generation; both were removed
during the restructure. Use `openssl` or your PKI tooling instead — see
[Security](security.md).
`logwisp auth` manages SCRAM credentials and logs in to `scram` listeners; see
[below](#logwisp-auth). There is no certificate-generation command: use
`openssl` or your PKI tooling — see [Security](security.md#enabling-mtls).
## Options
@@ -174,3 +174,70 @@ no `--background` flag; earlier releases had one and it was removed. See
kill -HUP $(pidof logwisp)
kill -USR1 $(pidof logwisp)
```
## `logwisp auth`
Manages the credentials files of `auth.type = "scram"` listeners and logs in to
them as a viewer. See
[Password Authentication](security.md#password-authentication-scram).
| Command | Does |
|---------|------|
| `add-user -credentials FILE -user NAME [-password-file FILE] [-generate]` | Adds a user, or replaces its password |
| `remove-user -credentials FILE -user NAME` | Removes a user; refuses the last one |
| `token -url https://HOST:PORT[/PATH] -user NAME -password-file FILE [-unbound] [TLS flags]` | Logs in to an `http` sink or `http_chain` source and prints a bearer token |
| `stream -addr HOST:PORT -user NAME -password-file FILE [TLS flags]` | Logs in to a `tcp` sink and copies its stream to stdout until interrupted |
`logwisp auth <command> -h` lists a command's flags. The exit status is `0` on
success, `1` on failure and `2` on a usage error.
**`add-user`** creates the credentials file, with a fresh `decoy_key`, when it
does not exist or is empty (create it empty first to choose its owner). The password comes from `-password-file` when that file exists
(at least 8 bytes; one trailing line break is trimmed). Otherwise a random
26-character password (130 bits) is generated and written to `-password-file`,
or printed once to stdout when there is none. Replacing an existing user's
password needs an existing `-password-file` or `-generate`, so a mistyped path
cannot replace a working password; `-generate` always generates, overwriting
`-password-file`. New users take the file's existing Argon2 profile.
Both file commands rewrite atomically (a temporary file in the same directory
as the file, or a symlink's target, then a rename), create files `0600`, keep an
existing file's mode and owner, and refuse to write anything the daemon would
not load. A change that cannot keep the owner fails: run it as root or as the
owner. A symlink to a missing file is refused: create the target first.
Neither touches a running LogWisp: send `SIGHUP`, since `auto_reload` does not
watch the credentials file.
**`token`** and **`stream`** build the same TLS and SCRAM client as a chain
sink. TLS flags: `-ca-file` (default: system roots), `-server-name` (default:
the host), and `-cert-file` / `-key-file` for a listener with `tls.client_auth`.
There is no flag to skip verification: an unverified server could relay the
login. Redirects are not followed. `-unbound` logs in to an `http` sink behind
a TLS-terminating proxy (`auth.trusted_proxies`), still pinning the proxy's
certificate across the two requests; only then may `-url` carry the path the
proxy mounts LogWisp at. `stream` exits `0`
on `SIGINT` or `SIGTERM`, and `1` when the server ends the stream (a reload or
shutdown).
```bash
# listener host: a file the service user can read, users, then apply
install -m 0640 -o root -g logwisp /dev/null /etc/logwisp/users.toml
logwisp auth add-user -credentials /etc/logwisp/users.toml -user edge-01 \
-password-file /etc/logwisp/edge-01.pass
logwisp auth add-user -credentials /etc/logwisp/users.toml -user viewer \
-password-file viewer.pass
kill -HUP $(pidof logwisp)
# rotate: new password into the file; SIGHUP, deploy the file, SIGHUP the edge
logwisp auth add-user -credentials /etc/logwisp/users.toml -user edge-01 \
-password-file /etc/logwisp/edge-01.pass -generate
# http sink status, with the token kept out of curl's argv
curl --cacert ca.crt -H @<(printf 'Authorization: Bearer %s\n' \
"$(logwisp auth token -url https://HOST:PORT -user viewer \
-password-file viewer.pass -ca-file ca.crt)") https://HOST:PORT/status
# follow a tcp sink
logwisp auth stream -addr HOST:PORT -user viewer -password-file viewer.pass \
-ca-file ca.crt
```
+15 -6
View File
@@ -232,6 +232,10 @@ Startup validation is intentionally split.
`internal/config` validates only global structure:
- no key the file schema does not declare, at any depth outside a plugin's
`config` table: `config file "…": unknown key
"pipelines[0].plugin_sinks[0].confg"`. A misspelled table path would otherwise
drop the whole table it heads. A top-level `config_file` is ignored
- at least one pipeline
- unique, non-empty pipeline names
- at least one source and one sink per pipeline
@@ -239,11 +243,15 @@ Startup validation is intentionally split.
and `logging.console.target` enum membership
Everything else is validated by the plugin constructor that owns it — port
range, required paths, path prefixes, enum values, regex compilation, TLS file
loading. A failure there aborts pipeline construction with a message naming the
pipeline, plugin id, and offending key. A key the plugin does not declare, at
any depth, is such a failure (`unknown key "tls.enabeld"`): a misspelled option
must not silently fall back to its default.
range, required paths, path prefixes, enum values, regex compilation, TLS and
credentials file loading. A failure there aborts pipeline construction with a
message naming the pipeline, plugin id, and offending key. A key the plugin does
not declare, at any depth, is such a failure (`unknown key "tls.enabeld"`): a
misspelled option must not silently fall back to its default. For the same
reason an `auth` block that names peers or credentials (`allow`,
`allow_patterns`, `credentials_file`, `token_lifetime_ms`, `username`,
`password_file`, `trusted_proxies`) while its `type` is `none` or unset is refused: auth was
intended and the type forgotten. See [Security](security.md#the-auth-block).
There is **no** cross-pipeline port-conflict detection. Two sinks bound to the
same port fail at listener bind time, when the pipeline starts.
@@ -255,7 +263,8 @@ auto_reload = true
```
or send `SIGHUP` / `SIGUSR1`. Signals reread the selected file even when watching
is disabled and always rebuild, allowing certificate rotation without TOML edits.
is disabled and always rebuild, allowing certificate and credentials rotation
without TOML edits; `auto_reload` watches only the configuration file.
CLI and environment overrides are captured at startup and keep their precedence.
Reload rebuilds the whole service: a new service is constructed from the new
+6 -3
View File
@@ -198,13 +198,16 @@ make
./test/chain-test.sh --auto # two independent relay pipelines
./test/chain-aggregate-test.sh --auto # fan-in: both edges into one pipeline
./test/mtls-chain-test.sh --auto # the same fan-in under mTLS
./test/scram-chain-test.sh --auto # SCRAM logins on every network plugin
./test/passthrough-test.sh # file source relays a wide envelope intact
```
Without `--auto` the chain scripts run the relay in the foreground for
interactive inspection. They need bash 5+, coreutils, and curl, and they bind
ports 15801–15804. The pass-through test binds nothing. Generated configuration
and logs land in `test/run/`.
interactive inspection. They need bash 5+, coreutils, and curl (the mTLS and
SCRAM scripts also openssl). The plain chain scripts bind ports 15801–15804 and
write to `test/run/`; the mTLS script uses 15811–15814 and `test/run-mtls/`, the
SCRAM script 15821–15825 and `test/run-scram/`. The pass-through test binds
nothing.
## Uninstall
+17 -58
View File
@@ -48,10 +48,10 @@ back to authenticated peers.
## Non-Goals
- Password authentication. It is designed separately in
[scram-auth-plan.md](scram-auth-plan.md), reusing the `auth` block and the
reserved hooks (`chain.Hello`, the `Authorization` header in the `http_chain`
sink, the "Future: password auth block" comments in the options structs).
- Password authentication. It is implemented separately as `type = "scram"`
([scram-auth-plan.md](scram-auth-plan.md)) on the same `auth` block and
`internal/authz` seam, carried by `chain.Hello` on TCP and a bearer token on
HTTP.
- IP allow/deny lists and per-peer rate limits. Related, but a separate feature
with its own config surface.
- OCSP. See [Revocation](#revocation) for what is done instead.
@@ -156,68 +156,25 @@ Errors follow existing style: `auth: type "mtls" requires tls.client_auth`.
### The `internal/authz` package
```go
package authz
// Policy is the compiled form of config.AuthOptions.
type Policy struct { /* role, identity mode, exact set, patterns, binding, counters */ }
// Role selects the validation and behavior appropriate to the call site.
const ( RoleListener Role = iota; RoleChainListener; RoleDialer )
// New compiles a policy. Returns (nil, nil) when auth is disabled, matching
// the tlsx.Server / tlsx.Client convention. tlsOpts is the sibling `tls`
// block, so an unenforceable policy fails here rather than at run time.
func New(o *config.AuthOptions, tlsOpts *config.TLSOptions, role Role) (*Policy, error)
// Identity is the outcome of a successful authorization.
type Identity struct {
Name string // the selected certificate field
Method string // "mtls"
}
// Apply stamps an identity onto session metadata.
func (id Identity) Apply(meta map[string]any)
// Authorize extracts and checks the peer identity from a completed handshake.
func (p *Policy) Authorize(cs *tls.ConnectionState) (Identity, error)
// VerifyConnection is assignable to tls.Config.VerifyConnection on a dialer.
func (p *Policy) VerifyConnection(cs tls.ConnectionState) error
// ResolveNode applies node_binding to the label a peer declared.
func (p *Policy) ResolveNode(declared, fallback string, trustNode bool, id Identity) (string, error)
// TrustsEntryNode reports whether per-entry node labels survive the policy.
func (p *Policy) TrustsEntryNode(trustNode bool) bool
// Stats reports counters for the sink/source stats map.
func (p *Policy) Stats() map[string]any
```
This mirrors `internal/tlsx`: one small package that is the single seam between
declarative config and a cross-cutting concern. `New` returns `(nil, nil)` for
the disabled case and **every method tolerates a nil receiver**, so a call site
reads identically whether or not auth is configured — no nil checks, no branch
on config:
```go
id, err := s.auth.Authorize(tlsState) // nil policy: (zero Identity, nil)
if err != nil { /* reject */ }
```
One small package, mirroring `internal/tlsx`, is the single seam between the
`auth` block and the plugins. `New(o, tlsCfg, role, transport)` compiles a policy
against the sibling TLS config and returns `(nil, nil)` when auth is disabled;
every method tolerates a nil receiver, so call sites never branch on config.
Listeners admit only through `Admit` (TCP) and `AuthorizeRequest` (HTTP); the
current API, including SCRAM, is in [scram-auth-plan.md](scram-auth-plan.md#the-authz-seam).
### Enforcement Points
**`tcp_chain` source** (`internal/source/tcpchain/tcpchain.go`, `handleConn`)
Handshake → **authorize** → read hello → `ResolveNode` → create session. The
authorization sits between the handshake and the hello read, so an unauthorized
peer never gets a preamble parsed on its behalf. `chain.DecodeEntry` is then
Handshake → `Admit` (authorize, then read hello) → `ResolveNode` → create
session. Under `mtls` the certificate is checked before the hello is read, so
an unauthorized peer never gets a preamble parsed on its behalf. `chain.DecodeEntry` is then
called with `auth.TrustsEntryNode(trust_node)` rather than `trust_node` itself.
**`http_chain` source** (`internal/source/httpchain/httpchain.go`, `handleIngest`)
Per request, from `r.TLS`, before the body is read — an unauthorized sender does
Per request, through `AuthorizeRequest`, before the body is read — an unauthorized sender does
not get to stream `max_body_bytes` into the process. Rejection is `403`,
distinct from the `400` used for protocol errors, so a sender can tell "you are
not allowed" from "your batch was malformed". `ResolveNode` then governs the
@@ -256,7 +213,9 @@ still reports `CapTLS`, the `mtls=true` field on the startup log line, and the
`Pipeline.initSourceCapabilities` and `initSinkCapabilities` treat this as a
cross-cutting check: a plugin advertising `CapAuth` without `CapTLS` is a
contradiction and fails pipeline construction rather than starting.
contradiction and fails pipeline construction rather than starting. An `http`
sink in SCRAM proxy mode also reports `CapProxyTLS`, which satisfies the check
without `tls`: TLS ends at a trusted proxy.
### Observability
+74 -16
View File
@@ -41,12 +41,12 @@ deadline" wherever the table says so.
|--------|--------|---------|--------|
| `tcp` sink | `write_timeout_ms` | `5000` | One write to one client; a miss disconnects that client |
| `http` sink | `write_timeout_ms` | `0` (none) | One SSE event write |
| `tcp_chain` source | `hello_timeout_ms` | `10000` | Reading the protocol preamble |
| `tcp_chain` source | `hello_timeout_ms` | `10000` | Reading the protocol preamble, and under `scram` the whole login |
| `tcp_chain` source | `read_timeout_ms` | `0` (none) | Idle time between entries |
| `http_chain` source | `read_timeout_ms` | `30000` | Reading a whole request body |
| `tcp_chain` sink | `dial_timeout_ms` | `5000` | TCP connect |
| `tcp_chain` sink | `write_timeout_ms` | `5000` | One line write |
| `http_chain` sink | `request_timeout_ms` | `10000` | Dial plus write plus response |
| `http_chain` sink | `request_timeout_ms` | `10000` | Dial plus write plus response, and a SCRAM login when one is due |
Fixed, non-configurable bounds:
@@ -56,6 +56,8 @@ Fixed, non-configurable bounds:
| HTTP read-header timeout | 10 s | `http` sink, `http_chain` source |
| HTTP server shutdown grace | 2 s | `http` sink, `http_chain` source |
| Max single entry line | 1 MiB | Chain listeners |
| SCRAM login | 10 s | `tcp` sink, chain sinks, `logwisp auth`, each `/auth` request |
| SCRAM line or `/auth` body | 4 KiB | `scram` listeners and dialers |
The `http` sink deliberately leaves the server's `WriteTimeout` unset, since it
would terminate long-lived SSE streams; per-event deadlines come from
@@ -76,9 +78,10 @@ sink, and the `tcp_chain` source. `0` means unlimited.
The `http_chain` source has no connection cap; it bounds work with
`max_body_bytes` and `read_timeout_ms` instead.
There is **no** per-IP limiting and no IP allow/deny list. `flow.rate_limit` is
a pipeline-wide entry rate limit, not a network-level one — it cannot
distinguish or throttle an individual peer.
Apart from SCRAM logins, which are throttled per address (see
[Security](security.md#throttling)), there is **no** per-IP limiting and no IP
allow/deny list. `flow.rate_limit` is a pipeline-wide entry rate limit, not a
network-level one — it cannot distinguish or throttle an individual peer.
## Keep-Alive
@@ -141,10 +144,12 @@ TLS is enabled. Only `GET` is routed to the stream and status paths. Each event
is framed as one `data:` line per newline in the payload, so multi-line entries
stream intact. Response headers set `Cache-Control: no-cache`,
`X-Accel-Buffering: no`, and, without an auth policy,
`Access-Control-Allow-Origin: *`.
`Access-Control-Allow-Origin: *`. Under `scram`, `POST /auth` serves the login
and both paths take `Authorization: Bearer <token>`.
**TCP sink** — raw payload bytes, no framing added by the sink. Whether entries
are newline-delimited depends on the formatter.
are newline-delimited depends on the formatter. Under `scram` the client's hello
and login lines come first, as on a [chain link](chaining.md#tcp-transport).
**Chain transports** — see [Chaining](chaining.md) for the hello preamble,
headers, and entry encoding.
@@ -183,25 +188,78 @@ headers, and entry encoding.
- `auth: node_binding "assert": declared node "..." does not match identity` —
the sender's `node` option and its certificate disagree. Fix one, or use
`node_binding = "force"` to let the certificate win silently.
- On a dialer, the same message inside `Chain connect failed` means the
*server* was refused: its certificate identity is not in the sink's
- On a `tcp_chain` sink the same message inside `Chain connect refused` means
the *server* was refused: its certificate identity is not in the sink's
`auth.allow`.
- Rejections appear as WARN and increment `auth_rejected`.
**SCRAM login fails**
- `authentication failed` — the dialer is not told why, by design. The
listener's WARN line (`Connection rejected by auth policy` on TCP,
`Login rejected` on HTTP) is: `invalid credentials` for a wrong password, an
unknown user, or one removed or rotated on the listener (compare the
`password_file` with the last `add-user`, and check the listener got
`SIGHUP`); `certificate cn "…" does not match user "…"` for the `identity`
binding.
- `the client saw another server certificate: TLS interception or a terminating
proxy` in the listener's log, with `auth_binding_mismatch` rising — something
between the peers terminates TLS. Pass TLS through to LogWisp; the login
cannot work otherwise, except on an `http` sink in proxy mode
(`auth.trusted_proxies`), where `the client bound its proof to the proxy's
certificate` means a client that needs `logwisp auth token -unbound`, and
`the client sent an unbound proof` the reverse. On an `http_chain` sink,
`server certificate differs from the one the SCRAM login was bound to` means
the certificate changed after the login: a rotation (the retry binds anew),
several backends, or interception.
- `403` from an `http` sink in proxy mode, with `not a trusted proxy` or
`X-Forwarded-Proto` in its WARN line — the request did not come from a listed
proxy, or the proxy did not forward `X-Forwarded-Proto: https` and
`X-Forwarded-For`.
- `too many attempts` — the address failed or abandoned logins faster than one
per second beyond a burst of 10, or has 4 unfinished; it clears within seconds
once the failing peer stops. Peers behind one NAT share the budget. `busy` —
4,096 logins in flight, or the listener is stopping.
- `authentication not enabled` — the dialer has `scram`; the listener's
`auth.type` is `none` or `mtls`.
- `no challenge within 10s (older logwisp, or not a scram listener)` — the
server read the hello and said nothing. Over HTTP the same case reads
`no auth endpoint at …/auth`.
- `Argon2 parameters below client minimum` — the credentials file holds a
cheaper Argon2 profile than the defaults; recreate it with `logwisp auth`.
- `peer offered no credentials` (`authentication required` on the wire) in the
listener's log — a peer without `scram` reached a `scram` listener; a client
that sends nothing logs `read hello: … i/o timeout` after 10 s instead. A
`tcp_chain` sink without `scram` logs WARN `Chain link refused` with that
reason and retries under growing backoff; what it wrote before reading the
refusal is lost. An `http_chain` sink without `scram` gets `401` and drops
every batch.
**`401` on HTTP under `scram`**
- An `http_chain` sink renews its token before it expires, so a `401` means
the listener reloaded (every reload revokes all tokens); the retry logs in
again.
- Continuous `401` means the token never sticks: several LogWisp processes
behind one address, a proxy that strips `Authorization`, or a
`token_lifetime_ms` shorter than a request takes. A token used with `curl`
dies with every listener reload.
- `403` is not the token: it is the `identity` binding, node binding or, in
proxy mode, the proxy gate.
**Entries not arriving over a chain link**
- Check the sink's `connected` statistic and its `reconnects` count.
- Check the source's `auth_rejected` — an allow-list miss looks exactly like a
network fault from the sender's side.
- Check the source's `auth_rejected` — an allow-list miss or a failed login
looks exactly like a network fault from the sender's side — and the sink's
`last_auth_error`.
- Check the source's `parse_errors` — a version skew shows up here.
- On `http_chain`, remember entries wait up to `flush_interval_ms` before a
batch is sent.
**Entries arriving under an unexpected node label**
- `auth.node_binding` defaults to `force` when `auth.type = "mtls"`, which
relabels every entry with the sender's certificate identity. If a dashboard
suddenly shows a different label, that is why. Use `node_binding = "assert"`
to keep upstream origin labels on relay-to-relay hops, or `"none"` to leave
`trust_node` in charge.
- `auth.node_binding` defaults to `force` with any `auth` block, which
relabels every entry with the sender's certificate identity or username. If
a dashboard suddenly shows a different label, that is why. Use
`node_binding = "assert"` to keep upstream origin labels on relay-to-relay
hops, or `"none"` to leave `trust_node` in charge.
**Clients connect but see nothing**
- The pipeline may be filtering everything out; check `flow.filters` stats.
+16 -12
View File
@@ -104,7 +104,8 @@ level = "debug"
It emits a service summary and then walks each pipeline, flattening scalar
statistics into log fields and recursing into flow, rate limiter, filter,
source, and sink stats.
source, and sink stats. Each plugin's `details` are spread into its line, so
the auth and TLS rejection counters show up there.
Disable with `status_reporter = false`.
@@ -142,8 +143,9 @@ curl -s http://127.0.0.1:8080/status | jq .
}
```
This endpoint is scoped to one sink, not to the whole process, and it is
**unauthenticated**. Bind it to a trusted interface.
This endpoint is scoped to one sink, not to the whole process, and without an
`auth` block it is **unauthenticated**. Bind it to a trusted interface, or
query it with a client certificate or a token from `logwisp auth token`.
### Metrics worth watching
@@ -280,20 +282,22 @@ environment or the command line at all.
openssl x509 -in /etc/logwisp/tls/relay.crt -noout -enddate
```
Certificates load at plugin construction, so rotation is: write the new files,
then `kill -HUP`. Startup and every reload warn at WARN once a certificate is
within 30 days of expiry; automate the check anyway.
Certificates and SCRAM credentials files load at plugin construction, so
rotation is: write the new files, then `kill -HUP`. Startup and every reload
warn at WARN once a certificate is within 30 days of expiry; automate the check
anyway. For passwords see
[rotation and revocation](security.md#rollout-rotation-and-revocation).
**Access review**
With `tls` alone, any certificate signed by the configured `client_ca_file` is
accepted, so "access review" means reviewing what your CA has issued. Add an
`auth` block with an explicit `allow` list and the review becomes the config
file itself: the identities listed there are the ones that can connect, and
removing one plus a `SIGHUP` is the revocation path. Authorized identities are
recorded in session metadata as `auth_identity`; rejections are counted in
`auth_rejected` and logged at WARN. See
[Security](security.md#the-auth-block).
`auth` block with an explicit `allow` list, or `scram` with its credentials
file, and the review becomes those files: the identities or users listed there
are the ones that can connect, and removing one plus a `SIGHUP` is the
revocation path. Authorized identities are recorded in session metadata as
`auth_identity`; rejections are counted in `auth_rejected` and logged at WARN.
See [Security](security.md#the-auth-block).
**Secret leakage**
+220 -81
View File
@@ -1,11 +1,12 @@
# Password Authentication (Argon2id-SCRAM)
**Status:** planned. Ships in two batches:
1. **Done:** `lixenwraith/auth` gained SCRAM channel binding, and the mTLS layer
was hardened (see [mTLS Hardening](#mtls-hardening)).
2. **Next:** the SCRAM integration described here, pinned to the tagged `auth`
release that carries channel binding.
**Status:** implemented, in three batches: `lixenwraith/auth` gained SCRAM
channel binding and the mTLS layer was hardened (see
[mTLS Hardening](#mtls-hardening)); then the SCRAM integration described here,
on the `auth` release that carries channel binding; then browser logins behind
a TLS-terminating proxy. Operator documentation lives in
[Security](security.md#password-authentication-scram) and
[`logwisp auth`](cli.md#logwisp-auth).
**Scope:** optional username/password authentication on every network plugin,
next to the existing certificate method, without weakening what mTLS gives.
@@ -24,8 +25,10 @@ next to the existing certificate method, without weakening what mTLS gives.
- Standard SASL SCRAM-SHA-256 interoperability. The `auth` exchange is
logwisp-to-logwisp (or logwisp CLI) only.
- Browser `EventSource` viewers under SCRAM: they cannot set `Authorization` or
run Argon2. mTLS remains the browser path.
- Browser viewers of a logwisp that terminates TLS itself: a browser cannot see
the server certificate, so its proof cannot be channel-bound. Browsers log in
through [proxy mode](#browser-viewers-behind-a-tls-terminating-proxy), or use
mTLS.
- Multiple backends behind a non-sticky L7 balancer: handshake state and the
token key live in one process.
@@ -35,23 +38,27 @@ next to the existing certificate method, without weakening what mTLS gives.
`StoredKey = SHA-256(HMAC(K, "Client Key"))`, `ServerKey = HMAC(K, "Server Key")`,
`K = Argon2id(password, salt)`. The Argon2id digest itself is never stored: it is
the salted password, from which both keys derive. Verifying a proof is HMAC only;
Argon2 runs on the dialer or CLI (64 MiB, t=3, the `auth` defaults).
Argon2 runs on the dialer or CLI (t=3, 64 MiB, 4 threads, the `auth` defaults).
The dialer refuses a challenge below that cost, so a rogue server cannot ask for
a cheaply guessable proof, and trusts the link only after the server's final
signature proves it holds the user's `ServerKey`.
**Channel binding.** Both peers pass `auth.WithChannelBinding(SHA-256(server leaf
DER))` at the proof step. The server hashes its configured certificate; the
client hashes the certificate it saw. A relay presenting any other certificate,
even one the CA trusts, makes the server reject the proof, so it never obtains a
link or a token. A mismatch is indistinguishable from a wrong password by design;
the client also sends its hash as a diagnostic field so the listener can log
"TLS interception or terminating proxy" and count it. TLS must terminate at
logwisp.
link or a token. A mismatch is indistinguishable from a wrong password on the
wire by design; the client also sends its hash as a diagnostic field, so the
listener logs "TLS interception or a terminating proxy" and counts it. TLS must
terminate at logwisp, except in [proxy mode](#browser-viewers-behind-a-tls-terminating-proxy).
**HTTP pinning.** HTTP auth takes two requests and every later request may use a
new pooled connection, so binding the exchange is not enough. After step 1 the
dialer pins the bound hash on its `VerifyConnection`: step 2 and every ingest
request must present the same leaf. A mismatch clears token and pin and
re-authenticates, which also covers certificate rotation. A token handed to curl
by the CLI is unpinned.
new pooled connection, so binding the exchange is not enough. Once the challenge
arrives the dialer pins the certificate that answered on its `VerifyConnection`:
the proof and every ingest request must present the same leaf, or the handshake
fails before anything is sent. The `http_chain` sink then clears token and pin
and logs in again, which also covers certificate rotation. A token handed to
curl by the CLI is unpinned.
## Configuration
@@ -72,86 +79,107 @@ password_file = "/etc/logwisp/edge-01.pass"
Validation at construction:
- `scram` requires TLS; a dialer also forbids `insecure_skip_verify`.
- Listeners require `credentials_file` and reject `allow`/`allow_patterns`: the
credentials file is the allow list. Use one file per listener when listeners
admit different users.
- Dialers require `username` + `password_file`, and reject `credentials_file`,
`allow*` and `identity` (server pinning stays an mTLS feature).
- `scram` requires TLS, except on an `http` sink in proxy mode; a dialer also
forbids `insecure_skip_verify`.
- Listeners require `credentials_file` and reject `allow`/`allow_patterns` (the
credentials file is the allow list), `username` and `password_file`. Use one
file per listener when listeners admit different users.
- Dialers require `username` + `password_file` (1–1024 bytes after trimming one
line break), and reject `credentials_file`, `token_lifetime_ms`, `allow*` and
`identity` (server pinning stays an mTLS feature).
- `identity` on a listener requires `tls.client_auth` and **binds the certificate
to the user**: the certificate's identity field must equal the SCRAM username,
so a peer needs its own certificate and its own password (PostgreSQL's
`clientcert=verify-full`). Without it the factors are independent: any
CA-issued certificate plus any valid password (`clientcert=verify-ca`).
- `token_lifetime_ms` only on HTTP listeners; default 15 minutes.
- `/auth` is reserved; no configured path may equal it.
- `/auth` and every path under it are reserved.
- `mtls` rejects the four SCRAM keys. A block that names peers or credentials
with `type = "none"` is refused: the type was forgotten, not the block.
**Credentials file** (TOML, written by the CLI, mode 0600):
```toml
decoy_key = "<base64, 32 random bytes>"
[[users]]
username = "edge-01"
salt = "<base64>"
argon_time = 3
argon_memory = 65536
argon_threads = 4
stored_key = "<base64>"
argon_time = 3
salt = "<base64>"
server_key = "<base64>"
stored_key = "<base64>"
username = "edge-01"
```
`decoy_key` keeps unknown-user challenges stable across restarts and edits, so
probing usernames reveals nothing. It is created by the first `add-user` and
preserved by every rewrite. An empty user set, a duplicate user, a missing
`decoy_key` or mixed KDF profiles are configuration errors. The file is read at
construction; the SCRAM server starts in the plugin's `Start` and stops in `Stop`,
so a rejected reload leaks nothing.
a single probe cannot tell a real user from an unknown one (a real user's salt
changes when it is added, rotated or removed). It is created by the first `add-user` and
preserved by every rewrite. An empty user set, a duplicate user, a missing or
short `decoy_key`, an unknown key, or users differing in KDF profile or salt
length are configuration errors. The file is read at construction; the SCRAM
server starts in the plugin's `Start` and stops in `Stop`, so a rejected reload
leaks nothing.
## Wire Protocol
`chain.Hello` gains `scram` (raw JSON, decoded by `authz`); the protocol version
stays 1 because the field is additive. Every message after the hello is one
`authStep`: `error`, `challenge`, `proof` (+ `binding`), `final` (+ `token`,
`expires_in` on HTTP). Pre-auth lines are capped at 4 KiB.
`expires_in` on HTTP). Pre-auth lines and `/auth` bodies are capped at 4 KiB.
**TCP** (`tcp_chain`, `tcp` sink): hello carrying the client-first message,
challenge, proof, final — all under one deadline (`hello_timeout_ms` on the
`tcp_chain` source, 10 s on the `tcp` sink and every dialer, with the dialer's
context able to cut it short). The final is written only after every admission
check (certificate binding, node binding); the stream then continues on the same
buffered reader. A source without SCRAM answers a hello carrying credentials with
`error: authentication not enabled`; only an old binary stays silent, and the
dialer's deadline covers that.
check (certificate binding, node binding); a later refusal is an `error` line
instead. The stream then continues on the same buffered reader. A listener
without SCRAM that reads a hello (the `tcp_chain` source) answers one carrying
credentials with `error: authentication not enabled`; an old binary stays
silent, and the dialer gives up after its deadline with "no challenge within
10s".
**HTTP** (`http_chain` source, `http` sink): `POST /auth`, outside the auth
middleware, with its own 10 s read/write deadline and a 4 KiB body limit.
middleware, with its own 10 s read/write deadline. It answers `404` unless the
policy is `scram`.
| Step | Request body | Success | Failure |
|------|--------------|---------|---------|
| 1 | hello | `200` challenge | `429` throttled, `503` handshake table full, `400` malformed |
| 2 | proof | `200` final, token, expires_in | `401 {"error":"authentication failed"}` |
| 1 | hello | `200` challenge | `429` throttled, `503` busy, `400` malformed, `413` oversized |
| 2 | proof | `200` final, token, expires_in | `401 {"error":"authentication failed"}`, `503` if the server stopped |
Protected endpoints take `Authorization: Bearer <token>`: missing or invalid is
`401` with `WWW-Authenticate: Bearer realm="logwisp"`, a certificate-binding miss
is `403`. Tokens are HS256 JWTs from `auth.NewJWT` with a random per-instance key,
so every reload revokes them. An SSE stream is checked at connect and outlives its
token; a reload ends it.
is `403`. Tokens are HS256 JWTs from `auth.NewJWT` (issuer `logwisp`, no leeway)
with a random per-instance key, so every reload revokes them. An SSE stream is
checked at connect and outlives its token; a reload ends it.
**Dialer outcomes** (`http_chain` sink): every `/auth` failure is transient (the
batch is held under backoff); `404`/`405` from `/auth` logs "no auth endpoint:
older logwisp or auth.type is not scram"; an ingest `401` clears the token and is
retried; `403` drops the batch; a pin mismatch clears token and pin. Rejections
are logged at WARN once per change of state and counted in `auth_failures` /
`last_auth_error`.
batch is held under backoff); `404`/`405` from `/auth` reads "no auth endpoint
at <url> (older logwisp, or not a scram listener)"; an ingest `401` clears the
token and is retried (tokens are renewed ahead of expiry, so this follows a
listener reload); `403` drops
the batch; a pin mismatch clears token and pin. Refusals are logged at WARN on
every attempt, paced by the backoff (`Chain connect refused` on `tcp_chain`,
`Chain batch delivery failed` on `http_chain`), and failed logins are counted
in `auth_failures` / `last_auth_error`.
## Throttling
Per remote IP (the socket address, never a forwarded header), on handshake
Per remote IP (the socket address; in proxy mode the forwarded client, an IPv6
one per /64), on handshake
starts: a token bucket (burst 10, 1/s) refunded on success, and at most 4
unfinished exchanges. The table is bounded and fails closed when full. Abandoned
TCP exchanges release their slot in the `auth` handshake table immediately.
Counters: `auth_allowed` and `auth_rejected` once per exchange outcome;
`auth_throttled`, `auth_busy` and `auth_binding_mismatch` separately.
unfinished exchanges. An unanswered HTTP challenge holds its slot for the
`auth` handshake timeout (30 s); abandoned TCP exchanges release their slot,
and their entry in the `auth` handshake table, immediately. The address table
holds 65,536 entries, drops one idle for a minute once its challenges have
expired, and fails closed when full; the SCRAM server itself caps in-flight
handshakes at 4,096 (`busy`).
Counters: `auth_allowed` counts logins; `auth_rejected` every refusal — failed
proofs, malformed requests, missing credentials and, on HTTP, refused tokens;
`auth_binding_mismatch` is the subset of failed proofs whose client saw another
certificate. `auth_throttled` and `auth_busy` count starts refused before a
challenge and are not in `auth_rejected`.
## The `authz` Seam
@@ -161,11 +189,13 @@ closed.
- `Admit(conn, cs, wantHello, timeout)` returns an admission (identity, hello,
reader) whose `Accept` or `Reject` writes the final message.
- `AuthorizeRequest(r)` covers mTLS certificates and bearer tokens;
`ServeAuth(w, r)` serves `/auth`.
- Dialers: `Greet(ctx, conn, node)` writes the hello (plain or SCRAM);
`Authenticate`, `SetAuthorization`, `ClearToken` and `VerifyConnection` serve
HTTP.
- `AuthorizeRequest(r)` covers mTLS certificates and bearer tokens, returning
the refusal status for `Refuse`; `ServeAuth(w, r)` serves `/auth`.
- Dialers: `Greet(ctx, conn, node)` writes the hello and, under SCRAM, runs the
TCP exchange. Over HTTP `Prepare` logs in when no token is held and sets the
bearer header, `Invalidate` drops the token after a `401` or a pin mismatch,
`Token` is the login itself (behind `Prepare`, and for the CLI), and
`VerifyConnection` pins.
- `Start`/`Close` bracket the SCRAM server and are nil-safe like everything else.
## CLI
@@ -173,24 +203,27 @@ closed.
```
logwisp auth add-user -credentials F -user U [-password-file P] [-generate]
logwisp auth remove-user -credentials F -user U
logwisp auth token -url https://host:port -user U -password-file P [TLS flags]
logwisp auth token -url https://host:port[/path] -user U -password-file P [-unbound] [TLS flags]
logwisp auth stream -addr host:port -user U -password-file P [TLS flags]
```
- `add-user` takes the password from `-password-file` when it exists; a new user
without one gets a generated 130-bit password, written to that file (0600) or
printed once. Rotating an existing user needs the file or `-generate`, so a
mistyped path cannot replace a password. The file's KDF profile and decoy key
are reused, the result is validated with the daemon's own loader, and the
rewrite (temp file in the same directory, rename) keeps the existing mode and
owner.
- `add-user` takes the password from `-password-file` when it exists (at least
8 bytes); a new user without one gets a generated 130-bit password, written to
that file (0600) or printed once. Rotating an existing user needs the file or
`-generate`, so a mistyped path cannot replace a password. The file's KDF
profile and decoy key are reused, the result is validated with the daemon's
own parser, and the rewrite (temp file beside the file or a symlink's target,
rename) keeps the existing mode and owner, or fails.
- `remove-user` refuses to remove the last user.
- Both print that changes apply on `SIGHUP`: `auto_reload` does not watch the
credentials file.
- `token` prints a bearer token for curl (`-H @<(...)` keeps it out of argv);
`stream` authenticates to a `tcp` sink and copies the stream to stdout.
`stream` authenticates to a `tcp` sink and copies the stream to stdout until
interrupted. `-url` takes a path only with `-unbound`, and redirects are not
followed.
- TLS flags: `-ca-file`, `-server-name`, `-cert-file`, `-key-file`. There is no
insecure flag.
- Exit status: 0 success, 1 failure, 2 usage error.
## Rollout and Rotation
@@ -202,18 +235,108 @@ logwisp auth stream -addr host:port -user U -password-file P [TLS flags]
between.
- Revocation: `remove-user`, `SIGHUP`. All tokens die with the reload.
## Browser Viewers Behind a TLS-Terminating Proxy
The deployer serves the log stream inside a website whose TLS ends at its
reverse proxy; LogWisp holds no private key and listens on a trusted hop.
Viewers log in from a browser with the client LogWisp ships, and the password
reaches neither the proxy nor LogWisp. Operator view and example:
[Security](security.md#browsers-behind-a-tls-terminating-proxy).
**Trust model.** `auth.trusted_proxies` (addresses or CIDRs) switches the `http`
sink into proxy mode; it is refused on every other plugin and with `identity`.
- Only listed peers may connect: `403` on every path, the pages included.
- `X-Forwarded-Proto` must be `https` on every hop, so a site served in
plaintext by mistake fails closed instead of exposing sessions.
- The client is the rightmost `X-Forwarded-For` hop that is not a proxy (hops to
its left are the client's own claims), or the leftmost when every hop is
inside a proxy range; throttling (an IPv6 client per /64, as one host holds
it whole), sessions and logs use it. Outside proxy mode forwarded headers
stay ignored.
- SCRAM runs unbound: a browser cannot read the site's certificate, and LogWisp
sees none. The proof and the session ride on the site's TLS; a party on the
proxy-to-LogWisp hop could relay a login, hence the address restriction and a
startup warning for a plaintext hop to a proxy off this host. `tls` may stay
off. Outside proxy mode nothing changes: proofs stay bound, and a bound proof
in proxy mode fails with a hint naming `-unbound`.
**Session.** The proof step may ask for a cookie (`"session": "cookie"`, proxy
mode only). The answer sets `logwisp_session` (`HttpOnly; Secure;
SameSite=Strict; Max-Age=<lifetime>`) and carries no token in the body. It has
no `Path`: RFC 6265 defaults it to the directory of `POST <mount>/auth`, which
is the mount behind any proxy prefix. Stream and status accept the cookie or
`Authorization: Bearer`; another `Authorization` scheme, such as the site's own
Basic auth, leaves the cookie in charge. Logout is `{"logout": true}` on `POST /auth` itself,
because a clearing cookie set from `/auth/logout` would default to another path
and miss the login cookie. It clears the cookie and revokes a valid token until
it expires (a set of at most 65,536 hashes, gone on reload like every token). An
open stream outlives its token; a reconnect after expiry gets `401`. `/auth`
takes only `application/json`, which a cross-origin page cannot send without a
preflight nobody answers.
**Endpoints.** `/auth` and every path under it are reserved; every URL in the
pages is relative, so a proxy prefix works unchanged.
| Path | Purpose |
|------|---------|
| `POST /auth` | SCRAM hello, proof, or logout; unbound in proxy mode |
| `GET /auth/scram.js` | the client library, always in proxy mode |
| `GET /auth/login`, `login.js`, `style.css` | login page (`login_page`) |
| `GET /auth/view`, `view.js` | minimal live viewer (`viewer_page`, needs `login_page`) |
Files carry `default-src 'none'; script-src 'self'; connect-src 'self';
style-src 'self'; form-action 'self'; frame-ancestors 'none'; base-uri 'none'`,
`nosniff` and `no-referrer`; nothing is inline. The viewer learns a custom
`status_path` from a meta tag the sink fills in, renders entries as text, and
sends a signed-out visitor to `login?next=view`; the login page follows `next`
only within its origin. The `proxy_tls` capability satisfies the pipeline's
"auth needs TLS" check.
**Client library.** `internal/sink/http/web/scram.js`, embedded with `go:embed`,
is one dependency-free ES module, so a site with its own CSP can copy it into
its bundle. It exports `login(base, username, password, {onProgress})` and
`logout(base)`. BLAKE2b and Argon2id are plain JS (no WebAssembly, which would
need `'wasm-unsafe-eval'`), yielding to the page every 50 ms; SHA-256 and HMAC
come from WebCrypto, so it needs a secure context. It checks the challenge as
the Go client does (nonce, salt, cost limits and the dialer's Argon2 floor),
refuses redirects, and reports success only after verifying the server's
signature. A 64 MiB login takes about 2 s on a desktop. The wire protocol above
is the contract: a site may implement its own client.
**CLI.** `logwisp auth token -unbound` logs in through the proxy, still pinning
its certificate across the two requests; only then may `-url` carry the mount
path.
## Verification
Go tests, one per rule: binding (match, mismatch, one-sided, length) in `auth`;
in `authz` the validation table, credential loading, decoy stability across
edits, TCP exchange over `net.Pipe` (success, wrong password equals unknown
user, binding mismatch, missing hello, silent peer within the deadline,
abandoned exchange frees its slot, final and first entry in one write, assert
mismatch gets an error rather than a final), HTTP exchange on a TLS test server
Go tests, one per rule: binding (match, mismatch, one-sided, length) in `auth`.
In `authz`: the validation table (each row valid but for its rule, the error
naming it) and credentials file validation; the TCP exchange over `net.Pipe`
(admission with the first entry in the same read, wrong password
indistinguishable from an unknown user, another certificate refused and counted,
hello and policy disagreeing either way, a dialer cut short by its context, an
abandoned exchange freeing its handshake and limiter slots, a refusal after the
exchange sent as a reason rather than a final, certificate-to-user binding for
logins and tokens); only failed attempts throttled, the limiter bounds
(reservation under concurrency, the sweep of abandoned challenges); pre-auth
input capped at 4 KiB; the decoy salt stable across restarts; a dialer refusing
a challenge below its Argon2 floor; the HTTP exchange on a TLS test server
(token accepted, garbage and foreign tokens `401`, forged final yields no token,
a different certificate after step 1 refused before the body is sent), the
limiter bound; `http_chain` sink re-authentication and outcome table; CLI
add/remove/rotate, mode preservation, last-user refusal, password trimming.
a different certificate after the challenge refused before the proof is sent,
renewal ahead of expiry); `Authorize` failing closed under SCRAM; password
trimming; refused hellos freeing their slot; unbound logins pinned too. Proxy
mode: trusted peers, https and the client hop; throttling on the forwarded
client and IPv6 /64; an unbound browser login with the cookie's attributes, the
cookie opening endpoints (beside Basic auth) until logout revokes it, cookie
sessions refused elsewhere; only unbound logins behind a proxy; `/auth` taking
only JSON. In the `http` sink, every SSE line break framed as data.
In the plugins: `/auth` outside the `http` sink's gate, the pages served with
their types and CSP, direct peers refused, pages only in proxy mode, and the
`http_chain` sink logging in again after a `401` and delivering once. In the
CLI: private files with a matching verifier, no password replaced without a
source, mode, decoy key and symlink kept across rewrites, an empty file filled,
last-user refusal. Under node, `scram.js` against the RFC 9106 vector, Go's
`argon2.IDKey`, and `auth`'s known-answer proof.
`test/scram-chain-test.sh --auto` (ports 15821-15825): a relay with SCRAM on all
four listener types plus one `client_auth` + SCRAM port; authorized edges over
@@ -222,7 +345,20 @@ user and no-auth edges deliver nothing; a `tcp` sink viewer through `logwisp aut
stream` held past the exchange deadline; a raw TLS client without a hello gets
nothing; the `http` sink through `logwisp auth token` and curl; missing and bad
tokens get `401`; `remove-user` + `SIGHUP` revokes token and login; throttling
last. The existing scripts keep passing.
last. `test/scram-proxy-test.sh --auto` (ports 15831-15832): a Go reverse proxy
ending TLS in front of a plaintext `http` sink mounted at `/logs/`; headless
Chromium sent to the login page, refused a wrong password, streaming events
after login, with the cookie scoped to `/logs` and hidden from scripts, cleared
by sign-out, and no CSP violations; `token -unbound` and curl through the
proxy; direct peers and plaintext-forwarded requests `403`. The existing
scripts keep passing.
## Not Implemented
1. **PROXY protocol behind TLS passthrough.** Every client then shares the
proxy's address, and so one throttling budget that a single client can
exhaust. Accepting PROXY v2 from listed proxies would restore per-client
throttling on the TCP listeners and on HTTP listeners outside proxy mode.
## mTLS Hardening
@@ -234,10 +370,13 @@ Shipped with this plan, independent of SCRAM:
- The `tcp` sink tracks connections from accept, so `Stop` and reloads no longer
wait out the 10 s handshake timeout of a silent peer.
- Unknown keys in any plugin's config, at any depth, fail construction: a typo
such as `[...tls] enabeld = true` used to switch protection off silently.
such as `[...tls] enabeld = true` used to switch protection off silently. The
configuration file itself is checked the same way, so a misspelled table path
above a plugin's config fails too.
- Startup and every reload warn about certificates that expired, are not yet
valid, or expire within 30 days; about `insecure_skip_verify`; about
`allow_patterns` that are not anchored at both ends of every alternative; and,
once per path, about key files every local user can read.
once per path, about key, credentials and password files every local user can
read.
- An `http` sink with an auth policy no longer sends
`Access-Control-Allow-Origin: *`.
+334 -86
View File
@@ -11,19 +11,21 @@ configure it, and — equally important — what it does not yet do.
| Server certificate verification by dialers | Implemented |
| Mutual TLS (client certificate required and verified) | Implemented at the transport layer |
| Peer identity recorded per session | Implemented |
| Authorization from peer identity (allow-lists, node binding) | Implemented — see [The Auth Block](#the-auth-block) |
| Authentication on the `http` sink's stream and status endpoints | Implemented, via the auth block |
| Server identity pinning by dialers | Implemented, via the auth block |
| Authorization from certificate identity (allow-lists, node binding) | Implemented — see [The Auth Block](#the-auth-block) |
| Password (Argon2id-SCRAM) authentication, bound to the TLS channel | Implemented — see [Password Authentication](#password-authentication-scram) |
| Authentication on the `http` sink's stream and status endpoints | Implemented: client certificate or bearer token |
| Browser logins behind a site's TLS-terminating proxy | Implemented — see [Browsers behind a TLS-terminating proxy](#browsers-behind-a-tls-terminating-proxy) |
| Server pinning by dialers | Implemented: certificate identity (`mtls`), bound certificate (`scram`) |
| Startup warnings for expiring certificates and risky settings | Implemented — see [Startup Warnings](#startup-warnings) |
| Unknown configuration keys rejected | Implemented — a typo in `tls` or `auth` fails startup |
| Certificate revocation lists (CRL) or OCSP | **Not implemented** — revoke by editing the allow-list |
| Password (Argon2id-SCRAM) authentication | **Planned** — see [scram-auth-plan.md](scram-auth-plan.md) |
| IP allow/deny lists, per-IP connection or request limits | **Not implemented** |
| Unknown configuration keys rejected | Implemented — a typo in `tls`, `auth` or a table path fails startup |
| Certificate revocation lists (CRL) or OCSP | **Not implemented** — revoke by editing the allow-list or credentials file |
| IP allow/deny lists, per-IP connection or request limits | **Not implemented** — only SCRAM logins are throttled per address |
Certificates are the credential the transport already carries, so they are what
authentication is built on today: the `tls` block establishes that a peer chains
to your CA, and the `auth` block decides which peers that CA vouches for may
actually do what.
Two credentials are supported. Certificates (`mtls`) are the one the transport
already carries: the `tls` block establishes that a peer chains to your CA, and
the `auth` block decides which of those peers may do what. Passwords (`scram`)
need no client PKI: the listener holds verifiers, never passwords, and every
login is bound to the listener's certificate.
## The TLS Block
@@ -56,13 +58,8 @@ min_version = "1.3"
| `insecure_skip_verify` | dialer | `false` | Disable server verification |
| `min_version` | both | `"1.3"` | `"1.2"` or `"1.3"` |
**Roles by plugin:**
| Plugin | Role | Keys that apply |
|--------|------|-----------------|
| `tcp` sink, `http` sink | Listener | `cert_file`, `key_file`, `client_auth`, `client_ca_file`, `min_version` |
| `tcp_chain` source, `http_chain` source | Listener | same as above |
| `tcp_chain` sink, `http_chain` sink | Dialer | `ca_file`, `server_name`, `insecure_skip_verify`, `cert_file`, `key_file`, `min_version` |
Listeners are the `tcp` and `http` sinks and the `tcp_chain` and `http_chain`
sources; dialers are the `tcp_chain` and `http_chain` sinks.
> `min_version` takes `"1.2"` or `"1.3"`. The older `"TLS1.2"` spelling from
> pre-restructure releases is rejected. There is no `max_version` and no
@@ -86,25 +83,34 @@ Misconfiguration fails at plugin construction, before the pipeline starts:
TLS answers "is this channel private, and does the peer chain to a CA". Auth
answers "may *this* peer do *this*". They are separate blocks because they are
separate questions, and because a later non-certificate method should be able to
reuse the second one.
separate questions.
```toml
[pipelines.plugin_sources.config.auth] # or plugin_sinks.config.auth
type = "none" # none | mtls
identity = "cn" # cn | san_dns | san_uri | san_email
allow = []
allow_patterns = []
node_binding = "force" # chain sources only
type = "none" # none | mtls | scram
identity = "cn" # cn | san_dns | san_uri | san_email
allow = [] # mtls
allow_patterns = [] # mtls
node_binding = "force" # chain sources only
credentials_file = "" # scram listeners
token_lifetime_ms = 0 # scram, HTTP listeners; 10 s to 24 h, 0 = 15 minutes
username = "" # scram dialers
password_file = "" # scram dialers
trusted_proxies = [] # scram http sink behind a TLS-terminating proxy
```
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `type` | string | `none` | `none` ignores the whole block; `mtls` authorizes by certificate identity |
| `identity` | string | `cn` | Which certificate field carries the identity |
| `allow` | []string | `[]` | Exact identities to admit |
| `allow_patterns` | []string | `[]` | RE2 patterns matched against the identity; anchor them yourself |
| `node_binding` | string | `force` under `mtls` | Chain sources only: `none`, `assert`, or `force` |
| Option | Applies to | Default | Description |
|--------|------------|---------|-------------|
| `type` | all | `none` | `none` ignores the block; `mtls` authorizes by certificate identity; `scram` by password |
| `identity` | `mtls`; `scram` listeners | `cn` (`mtls`), unset (`scram`) | Certificate field carrying the identity. Under `scram` it binds the client certificate to the user |
| `allow` | `mtls` | `[]` | Exact identities to admit |
| `allow_patterns` | `mtls` | `[]` | RE2 patterns matched against the identity; anchor them yourself |
| `node_binding` | chain sources | `force` | `none`, `assert`, or `force` |
| `credentials_file` | `scram` listeners | — | Verifiers written by [`logwisp auth add-user`](cli.md#logwisp-auth) |
| `token_lifetime_ms` | `scram` on the `http` sink and `http_chain` source | 15 minutes | Bearer token lifetime, 10 s to 24 h |
| `username` | `scram` dialers | — | User to log in as |
| `password_file` | `scram` dialers | — | File holding the password; one trailing line break is trimmed |
| `trusted_proxies` | `scram` on the `http` sink | `[]` | Addresses or CIDRs of the reverse proxies that end the browsers' TLS; see [Browsers behind a TLS-terminating proxy](#browsers-behind-a-tls-terminating-proxy) |
**Roles by plugin:**
@@ -112,13 +118,14 @@ node_binding = "force" # chain sources only
|--------|------|---------|
| `tcp_chain` source, `http_chain` source | Listener | Which senders may ingest, and what node label their entries carry |
| `tcp` sink, `http` sink | Listener | Which clients may read the stream (and, on `http`, the status endpoint) |
| `tcp_chain` sink, `http_chain` sink | Dialer | Which server identity to accept, beyond hostname verification |
| `tcp_chain` sink, `http_chain` sink | Dialer | `mtls`: which server identity to accept, beyond hostname verification. `scram`: which user to log in as |
### Identity
The identity is one string pulled from the peer's verified leaf certificate.
The handshake has already checked the chain, signature, and validity window, so
this is pure field selection.
Under `mtls` the identity is one string pulled from the peer's verified leaf
certificate; under `scram` it is the username. The handshake has already
checked the chain, signature, and validity window, so this is pure field
selection.
| Mode | Source | Typical use |
|------|--------|-------------|
@@ -133,7 +140,8 @@ identity is a refusal, not an empty match.
### The allow list
`allow` is an exact-match set; `allow_patterns` holds RE2 patterns. An identity
passes if it appears in either.
passes if it appears in either. Under `scram` the credentials file is the allow
list, and both keys are refused.
Leaving **both** empty under `type = "mtls"` admits any identity the CA vouches
for. That is deliberate — it is how you enable node binding without enumerating
@@ -161,7 +169,8 @@ reported at startup.
Use **`force`** on an ingest boundary you do not trust. Every entry is
relabelled, so a compromised edge cannot smuggle a foreign origin through the
per-entry `node` field either. It is the default under `type = "mtls"`.
per-entry `node` field either. It is the default whenever `type` is `mtls` or
`scram`.
Use **`assert`** on a relay-to-relay hop. The relay must prove its own identity —
a mismatch fails loudly instead of being silently corrected — but the entries it
@@ -177,7 +186,7 @@ INFO msg="Node labels bound to peer identity; trust_node is ignored"
### Dialer-side pinning
On a chain sink, the same block pins the *server's* identity. Hostname
Under `mtls`, the block on a chain sink pins the *server's* identity. Hostname
verification already proves the server holds a certificate valid for the address
you dialed; pinning additionally requires that certificate to name an identity
you listed.
@@ -190,20 +199,40 @@ allow = ["relay.internal"]
The check runs as part of the handshake, so a server the policy rejects never
receives an entry — the sink's normal backoff loop handles it like any other
connect failure. `insecure_skip_verify` is refused alongside `type = "mtls"`:
an identity read from an unverified chain is a claim, not a fact.
connect failure. Under `scram` the dialer binds its login to the certificate it
was shown instead; see [Channel binding](#channel-binding).
### Validation
Misconfiguration fails at plugin construction, before the pipeline starts:
- `type = "mtls"` on a listener without `tls.enabled` **and** `tls.client_auth`
- `type = "mtls"` on a dialer without `tls.enabled`, or with
`tls.insecure_skip_verify`
- `type` `mtls` or `scram` without `tls.enabled` (proxy mode aside), or on a dialer with
`tls.insecure_skip_verify`: an identity read from an unverified chain is a
claim, and an unverified server could relay a login
- `type = "mtls"` on a listener without `tls.client_auth`
- a block naming peers or credentials (`allow`, `allow_patterns`,
`credentials_file`, `token_lifetime_ms`, `username`, `password_file`,
`trusted_proxies`) whose
`type` is `none` or unset: auth was intended and the type forgotten
- a key of the other method: `allow` or `allow_patterns` under `scram`, a
`scram` key under `mtls`
- a `scram` listener without a loadable `credentials_file` (see
[Credentials file](#credentials-file)), with `username` or `password_file`,
with a `token_lifetime_ms` outside 10 s to 24 h or on a TCP listener, or with
`identity` but no `tls.client_auth`
- a `scram` dialer without `username` and `password_file`, with
`credentials_file`, `token_lifetime_ms`, `trusted_proxies` or `identity`
(server pinning is `mtls` only), or with a password file that is empty or
over 1024 bytes
- `trusted_proxies` on anything but the `http` sink, with `identity`, or with
an entry that is neither an address nor a CIDR; `login_page` or
`viewer_page` without `trusted_proxies`, `viewer_page` without `login_page`
- an `identity` that is not one of the four modes
- an `allow_patterns` entry that does not compile
- a `node_binding` that is not one of the three values, or one set on a plugin
that has no node concept
- an HTTP path (`stream_path`, `status_path`, `ingest_path`) equal to `/auth`,
which is reserved for logins
Errors read like `auth: type "mtls" requires tls.client_auth`.
@@ -217,15 +246,15 @@ when it is constructed, so startup and every reload repeat them:
- `insecure_skip_verify` on a dialer
- an `allow_patterns` entry not anchored at both ends of every alternative
(`^a|b$` admits `a…` and `…b`)
- a `key_file` every local user can read (once per path per process)
- a `key_file`, `credentials_file` or `password_file` every local user can read
(once per path per process)
## Enabling mTLS
### 1. Generate a CA and certificates
LogWisp no longer ships a certificate-generation subcommand; the `logwisp tls`
command was removed with the rest of the CLI restructure. Use `openssl`,
`cfssl`, `step-cli`, or your existing PKI.
LogWisp does not generate certificates; use `openssl`, `cfssl`, `step-cli`, or
your existing PKI.
```bash
# CA
@@ -299,48 +328,265 @@ INFO msg="TCP chain sink initialized" ... tls=true mtls=true
auth="mtls identity=cn allow=[1 exact, 0 pattern(s)] node_binding=none"
```
A client that presents no certificate is refused during the handshake:
```
WARN msg="TLS handshake failed" component=tcp_chain_source
remote_addr=127.0.0.1:53840 error="tls: client didn't provide a certificate"
```
A client that presents no certificate is refused during the handshake
(`tls: client didn't provide a certificate`, counted in `tls_handshake_errors`).
A client whose certificate is valid but whose identity is not authorized gets
past the handshake and is refused by the policy:
```
WARN msg="Connection rejected by auth policy" component=tcp_chain_source
remote_addr=127.0.0.1:33946 error="auth: identity \"edge-99\" is not allowed"
remote_addr=<addr> error="auth: refused: identity \"edge-99\" is not allowed"
```
Handshake failures are counted in `tls_handshake_errors`; policy rejections in
`auth_rejected`. Both appear in the status reporter and in the `http` sink's
status endpoint. Accepted peers are recorded in session metadata as
`auth_method` and `auth_identity`.
Policy rejections are counted in the plugin's `auth_rejected` statistic; the
`http` sink's status endpoint reports its own. Accepted peers are recorded in
session metadata as `auth_method` and `auth_identity`.
`test/mtls-chain-test.sh` builds a throwaway PKI and exercises the whole surface
end to end — run it with `--auto` to see each guarantee asserted.
## Password Authentication (SCRAM)
`type = "scram"` authenticates peers by username and password with
Argon2id-SCRAM from `lixenwraith/auth`. It needs TLS but no client
certificates, works on all six network plugins, and is LogWisp's own protocol,
not standard SASL: viewers use the [`logwisp auth`](cli.md#logwisp-auth) CLI or,
behind a TLS-terminating proxy, the shipped browser client.
```toml
# Listener: tcp_chain/http_chain source, tcp/http sink
[pipelines.plugin_sources.config.tls]
enabled = true
cert_file = "/etc/logwisp/tls/relay.crt"
key_file = "/etc/logwisp/tls/relay.key"
[pipelines.plugin_sources.config.auth]
type = "scram"
credentials_file = "/etc/logwisp/users.toml"
# Dialer: tcp_chain/http_chain sink
[pipelines.plugin_sinks.config.tls]
enabled = true
ca_file = "/etc/logwisp/tls/ca.crt"
[pipelines.plugin_sinks.config.auth]
type = "scram"
username = "edge-01"
password_file = "/etc/logwisp/edge-01.pass"
```
Startup reports `auth="scram users=2 certificate=independent node_binding=force"`
on such a chain source, and `auth="scram user=edge-01"` on the dialer.
### Mechanism
For each user the listener keeps a salt, the Argon2id cost, and two keys derived
from `K = Argon2id(password, salt)`: `StoredKey = SHA-256(HMAC(K, "Client
Key"))` and `ServerKey = HMAC(K, "Server Key")`. Neither logs anyone in, but
each password guess against them costs one Argon2id, so keep the file private.
Checking a login is HMAC work only; Argon2id (3 passes, 64 MiB, 4 threads) runs
on the dialer or the CLI. The proof is mutual: the dialer accepts the server's
final message only when it is signed with this user's `ServerKey`, and it
refuses a challenge asking for less Argon2 work than those defaults, so a rogue
server cannot request a cheaply crackable proof. An unknown user gets a stable
decoy challenge and fails exactly like a wrong password: `authentication
failed`.
### Channel binding
Both sides bind the proof to the SHA-256 of the listener's leaf certificate:
the listener hashes the certificate it serves, the dialer the one it was shown.
A TLS-terminating proxy or interceptor shows another certificate — CA-valid or
not — so the proof fails and the relay never obtains a link or a token. The
dialer is told `authentication failed` either way; the listener compares the
hash the dialer reports, logs `the client saw another server certificate: TLS
interception or a terminating proxy`, and counts `auth_binding_mismatch`.
An HTTP login takes two requests, and every later request may open a new pooled
connection. After the challenge the dialer pins the certificate it was bound to:
the proof and every ingest request must meet the same certificate, or the TLS
handshake fails before anything is sent. The `http_chain` sink then drops token
and pin and logs in again, which also covers a rotated server certificate. A
token printed by `logwisp auth token` is not pinned.
### Certificates and users
`tls.client_auth` is optional under `scram`, and on its own independent of the
password: a peer needs *some* certificate the CA issued and *some* valid
password, like PostgreSQL's `clientcert=verify-ca`. Setting `identity` (which
requires `tls.client_auth`) binds the two: the certificate's identity field must
equal the username, so every peer needs its own certificate *and* its own
password, like `clientcert=verify-full`. A mismatch fails the login; an HTTP
request whose certificate does not match its token's user gets `403`. Startup
shows the mode as `certificate=independent` or `certificate=cn=username`.
### Tokens
The `http` sink and `http_chain` source answer a login at `POST /auth` with a
bearer token: an HS256 JWT valid for `token_lifetime_ms` (default 15 minutes)
and signed with a key drawn when the plugin is built, so **every reload revokes
every token** (a rejected reload keeps the running service and its tokens). A
missing, invalid or expired token gets `401` with `WWW-Authenticate: Bearer
realm="logwisp"`. An SSE stream is checked when it connects and then outlives
its token; a reload ends it. TCP has no tokens: each connection runs its own
exchange.
### Throttling
Logins are throttled per remote socket address or, on an `http` sink in proxy
mode, per forwarded client (an IPv6 client per /64); forwarded headers are read
only from `trusted_proxies`. Each exchange takes a token from a bucket of 10 that refills at one per
second, and a successful login gives it back, so only failed or abandoned
attempts drain it. At most 4 exchanges per address may be unfinished: an HTTP
challenge never answered holds its slot for up to 30 s, a TCP connection that
ends mid-exchange frees it at once. A refused start answers `too many attempts`
(HTTP `429`) and counts `auth_throttled`; the table holds 65,536 addresses and
refuses new ones when full. Separately, at most 4,096 exchanges may be in flight
per listener; beyond that, or while the plugin stops, a login gets `busy` (HTTP
`503`) and counts `auth_busy`. Peers behind one NAT or one passthrough proxy
share a bucket: there, one client can exhaust every other client's logins.
### Credentials file
```toml
# logwisp SCRAM verifiers, written by `logwisp auth add-user`. Keep it private.
decoy_key = "<base64, 32 random bytes>"
[[users]]
argon_memory = 65536
argon_threads = 4
argon_time = 3
salt = "<base64>"
server_key = "<base64>"
stored_key = "<base64>"
username = "edge-01"
```
Write it with [`logwisp auth add-user`](cli.md#logwisp-auth) rather than by
hand. The whole file is validated when the plugin is built: a `decoy_key` of at
least 32 bytes, at least one user, unique names, no unknown keys, and one Argon2
profile and salt length for every user, since mixed profiles would tell a prober
which users exist. `decoy_key` keeps unknown-user challenges stable across
restarts and edits; the CLI creates it once and preserves it. A single probe
cannot tell a real user from an unknown one, but a prober polling a name sees
its salt change when the user is added, rotated or removed. The file is read
only when the plugin is built and `auto_reload` does not watch it: send
`SIGHUP` after every change. Use one file per listener when listeners admit
different users.
### Rollout, rotation and revocation
- **Rollout.** Turning `scram` on cuts off every dialer of that listener that
has no credentials. Upgrade every binary first, then add a second listener
with `scram` on another port, move the dialers to it, and remove the old one.
- **Password rotation.** `logwisp auth add-user -generate` with the user's
`-password-file` replaces verifier and password together. `SIGHUP` the
listener, deploy the password file, `SIGHUP` the dialer. Logins fail in
between and the dialer retries under backoff, holding its current entry or
batch while its input queue fills. To avoid the gap, add a second user, move
the dialer to it, then remove the first; under `node_binding = "force"` the
node label follows the username.
- **Revocation.** `logwisp auth remove-user`, then `SIGHUP`. The reload drops
every connection and revokes every token; other dialers log in again on
their own.
### Limits
- TLS must terminate at LogWisp, except on an `http` sink in proxy mode. Behind
any other terminating proxy or load balancer every login fails by design;
pass TLS through instead (TCP or SNI routing). All clients then share the
proxy's address, and so one throttling budget.
- One process per HTTP address. Handshake state and the token key live in one
instance, so behind a balancer the proof or the token can reach an instance
that never saw the login.
- Browsers log in only to an `http` sink in proxy mode, below; elsewhere use
`mtls` for browsers.
- `nc` and `openssl s_client` cannot read a `scram` `tcp` sink; use
`logwisp auth stream`.
`test/scram-chain-test.sh --auto` exercises these guarantees end to end.
### Browsers behind a TLS-terminating proxy
A site that serves the log stream inside its own pages ends TLS at its reverse
proxy and need not give LogWisp a private key. `trusted_proxies` puts the `http`
sink in proxy mode, where browsers log in with the client LogWisp ships; the
password reaches neither the proxy nor LogWisp.
```toml
[pipelines.plugin_sinks.config]
host = "127.0.0.1"
port = 8081
login_page = true # GET /auth/login
viewer_page = true # GET /auth/view, needs login_page
[pipelines.plugin_sinks.config.auth]
type = "scram"
credentials_file = "/etc/logwisp/users.toml"
trusted_proxies = ["127.0.0.1"] # addresses or CIDRs
```
```nginx
location /logs/ {
proxy_pass http://127.0.0.1:8081/;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
```
- **Trust.** Every other peer gets `403` on every path. `X-Forwarded-Proto`
must be `https` on every hop, so a site accidentally served in plaintext fails
closed. The client is the rightmost `X-Forwarded-For` hop that is not a
proxy; throttling, sessions and logs use it. List only the proxies: a client
inside a listed range is trusted to name its own hops.
- **Unbound logins.** The browser cannot see a certificate LogWisp could bind
to, so proofs are unbound and a party on the hop between proxy and LogWisp
could relay a login. Keep that hop on loopback or a trusted network, or
enable `tls` on the sink; a plaintext hop to a proxy off this host is
warned about at startup. `identity` cannot be combined with proxy mode.
- **Sessions.** The page asks for a cookie: `logwisp_session`, `HttpOnly`,
`Secure`, `SameSite=Strict`, `Max-Age` the token lifetime, and no `Path`, so
it scopes itself to the mount (`/logs` above). Stream and status accept it or
a bearer token, so `new EventSource("/logs/stream")` works on any page of the
site, beside the site's own Basic auth too, and `logwisp auth token -unbound`
keeps working. `POST /auth` with
`{"logout": true}` clears the cookie and revokes the token until it expires.
`/auth` takes only `application/json`, which a cross-origin page cannot send
without a preflight.
- **Pages.** Under `/auth/` the sink serves `scram.js` always, and with
`login_page` / `viewer_page` the login page and a minimal live viewer with
their script and style, under `default-src 'none'; script-src 'self';
connect-src 'self'; style-src 'self'; form-action 'self'; frame-ancestors
'none'; base-uri 'none'`. A site with its own CSP can copy `scram.js` from
`internal/sink/http/web/` into its bundle: one dependency-free ES module
exporting `login(base, username, password, {onProgress})` and
`logout(base)`, where `base` is the mount URL ending in `/`. It needs a
secure context for WebCrypto and takes about 2 s of Argon2 per login on a
desktop.
`test/scram-proxy-test.sh --auto` logs in from headless Chromium through such a
proxy.
## What Each Layer Enforces
**`tls` with `client_auth = true`** — a membership check. The peer holds a
certificate chaining to `client_ca_file`, within its validity window, and holds
the matching private key. An attacker without a CA-issued certificate cannot
connect at all. What it does *not* decide is which CA-issued certificate: every
one is equivalent at this layer.
the matching private key. Every CA-issued certificate is equivalent at this
layer.
**`auth` with `type = "mtls"`** — an identity check, per listener:
**`auth` with `type = "mtls"`** — an identity check, per listener: only the
identities you list may connect, so one CA can serve several trust domains and
a single peer can be withdrawn without touching the others.
- only the identities you list may connect, so one CA can serve several trust
domains and a single peer can be withdrawn without touching the others
- the chain `node` label is bound to the certificate, so a compromised edge
cannot attribute its entries to another host
- the `http` sink's stream and status endpoints stop being open to anyone who
can reach the port
**`auth` with `type = "scram"`** — a password check, per listener: only the
users in its credentials file may connect, the login cannot be relayed through
another certificate (proxy mode, where TLS ends at the site's proxy, aside),
and with `identity` each user is tied to its own certificate.
**Revocation** is the allow-list, not a CRL. Remove the identity from `allow` /
`allow_patterns` and send `SIGHUP`: the reload rebuilds every pipeline, so the
Under either method the chain `node` label can be bound to the identity, so a
compromised edge cannot attribute its entries to another host, and the `http`
sink's stream and status endpoints stop being open to anyone who can reach the
port.
**Revocation** is the allow-list or the credentials file, not a CRL. Remove the
identity or user and send `SIGHUP`: the reload rebuilds every pipeline, so the
change takes effect on the next connection and existing ones are dropped by the
rebuild. No network call on the handshake path, and no window between revocation
and the next CRL publication. See
@@ -349,32 +595,33 @@ would add.
## Surfaces Without Access Control
An `auth` block closes each of these. Without one, bind them to a trusted
interface or front them with an authenticating proxy.
An `auth` block (`mtls` or `scram`) closes each of these. Without one, bind them
to a trusted interface or front them with an authenticating proxy.
| Surface | Exposure when `auth.type = "none"` |
|---------|-----------------------------------|
| `http` sink `stream_path` | Full log stream, with `Access-Control-Allow-Origin: *`, so any browser origin can read it (the header is omitted once an auth policy is set) |
| `http` sink `status_path` | Host, port, TLS flag, uptime, client counts, throughput counters |
| `tcp` sink | Full log stream to any client that connects |
| `tcp_chain` / `http_chain` source | Ingest from any peer the CA vouches for, under any node label it claims |
| `tcp_chain` / `http_chain` source | Ingest from any peer that can connect (with `client_auth`, any the CA vouches for), under any node label it claims |
`max_connections` bounds concurrency on all of them but does not distinguish
callers.
Note that `auth` requires `client_auth = true`, which requires TLS. There is no
way to authenticate a plaintext listener.
Both methods require TLS on the listener, except an `http` sink in proxy mode,
where TLS ends at the trusted proxy; `mtls` also requires `client_auth`.
## Operational Guidance
**Certificates**
**Certificates and secrets**
- Use a dedicated CA for LogWisp so its trust decisions stay independent.
- Keep leaf lifetimes short (90–825 days) and automate renewal.
- Key files should be `0600` and owned by the service account; a world-readable
one is reported at startup.
- Rotation requires a reload (`SIGHUP`), because certificates are loaded once at
plugin construction; there is no on-disk watch for certificate files.
- Key, credentials and password files should be `0600` and owned by the service
account; a world-readable one is reported at startup. `logwisp auth` creates
them `0600` and keeps the mode of an existing file.
- Rotation requires a reload (`SIGHUP`), because certificates and credentials
are loaded once at plugin construction; there is no on-disk watch for them.
- Startup and every reload warn 30 days before a certificate lapses. Still
automate renewal; the warning only reaches someone reading the log.
- Keep the identity field you authorize on stable across rotations. Reissuing a
@@ -388,10 +635,11 @@ way to authenticate a plaintext listener.
verification entirely and makes the connection trivially interceptable. It is
reported at startup.
- Bind listeners to specific interfaces rather than `0.0.0.0` where you can.
- On any ingest port reachable from a network you do not fully control, set
`auth.type = "mtls"` with an explicit `allow` list. `trust_node = false` is the
fallback when certificates are not an option; it is unforgeable but labels
entries by remote address, which is useless behind NAT or a load balancer.
- On any ingest port reachable from a network you do not fully control, set an
`auth` block: `mtls` with an explicit `allow` list, or `scram`.
`trust_node = false` is the fallback when neither is an option; it is
unforgeable but labels entries by remote address, which is useless behind NAT
or a load balancer.
- Run LogWisp as an unprivileged user with write access only to its own log and
configuration directories.
+71 -21
View File
@@ -135,22 +135,37 @@ allow = ["viewer-01"]
| `client_buffer_size` | int | `256` | Per-client send queue depth |
| `write_timeout_ms` | int | `0` | Per-event write deadline; `0` = none |
| `max_connections` | int | `0` | Concurrent stream cap; `0` = unlimited |
| `login_page` | bool | `false` | Serve the browser login page at `/auth/login`; needs `auth.trusted_proxies` |
| `viewer_page` | bool | `false` | Serve a minimal live viewer at `/auth/view`; needs `login_page` |
| `tls` | table | — | Listener TLS; see [Security](security.md) |
| `auth` | table | — | Client authorization; see [Security](security.md#the-auth-block) |
| `auth` | table | — | Client authentication (`mtls` or `scram`); see [Security](security.md#the-auth-block) |
**Behaviour**
- Only `GET` is routed to either path; anything else gets `405`, `HEAD` on
`stream_path` included — a stream is a body, and a client registered to have
its body discarded never reads and never leaves.
- With an `auth` block, one middleware gates **both** endpoints: an
unauthorized client gets `403` with no body detail, and the rejection is
logged at WARN and counted in `auth_rejected`. The authorized identity is
recorded in the client's session as `auth_method` / `auth_identity`.
- With an `auth` block, one middleware gates **both** endpoints, with no body
detail in a refusal. Under `mtls` a refused certificate gets `403`. Under
`scram` a client logs in at `POST /auth`, which sits outside the gate (see
[`logwisp auth token`](cli.md#logwisp-auth)), and sends
`Authorization: Bearer <token>`; a missing, invalid or expired token gets
`401` with `WWW-Authenticate`, a certificate that does not match the token's
user `403`. A stream is checked when it connects and outlives its token; a
reload ends it. `/auth` and every path under it are reserved.
- With `auth.trusted_proxies` (proxy mode) the sink sits behind a site's
TLS-terminating reverse proxy: only the proxies may connect, browsers log in
through `/auth/login` or the site's own copy of `/auth/scram.js`, and stream
and status also accept the `logwisp_session` cookie. See
[Browsers behind a TLS-terminating proxy](security.md#browsers-behind-a-tls-terminating-proxy).
- Refusals are logged at WARN and counted in `auth_rejected`. The authorized
identity is recorded in the client's session as `auth_method` /
`auth_identity`.
- On connect the client receives an `event: connected` frame carrying its
client id, session id, sink instance id, endpoint paths, and buffer size.
- Payloads are framed per the SSE spec, one `data:` line per newline in the
payload, so multi-line entries stream correctly.
- Payloads are framed per the SSE spec, one `data:` line per line break in the
payload (CRLF, LF or a lone CR), so multi-line entries stream correctly and
no entry can inject an `event:`, `id:` or `retry:` field.
- The server sets no `WriteTimeout` (that would kill long-lived streams);
per-write deadlines come from `write_timeout_ms` via `http.ResponseController`
and cover the connected frame, every payload, and the idle comment.
@@ -176,10 +191,15 @@ connection limit, write timeout, uptime, endpoint paths, and the
`total_processed` / `dropped_writes` / `rejected_clients` / `auth_rejected`
counters.
**Statistics**: `dropped_writes`, `rejected_clients`, `auth`, `auth_allowed`,
`auth_rejected`; under `scram` also `auth_users`, `auth_throttled`,
`auth_busy`, `auth_binding_mismatch` and `auth_token_lifetime_ms`, and in
proxy mode `auth_trusted_proxies`.
> Without an `auth` block both endpoints are unauthenticated, and the stream
> response carries `Access-Control-Allow-Origin: *`, so any web origin can read
> it. With one, the header is omitted. Set `auth.type = "mtls"` (which requires `tls.client_auth`), bind to a
> trusted interface, or put an authenticating reverse proxy in front.
> it. With one, the header is omitted. Set an `auth` block, bind to a trusted
> interface, or put an authenticating reverse proxy in front.
---
@@ -224,13 +244,17 @@ allow = ["viewer-01"]
| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period |
| `max_connections` | int | `0` | Concurrent connection cap; `0` = unlimited |
| `tls` | table | — | Listener TLS |
| `auth` | table | — | Client authorization; see [Security](security.md#the-auth-block) |
| `auth` | table | — | Client authentication (`mtls` or `scram`); see [Security](security.md#the-auth-block) |
**Behaviour**
- The sink is write-only. Each connection also runs a reader that discards
inbound bytes; it exists to detect disconnects and to refresh session
activity when a client sends anything.
activity when a client sends anything. Under `scram` the client must log in
first: a hello carrying the login, then challenge, proof and final, all
within 10 s, after which the stream follows on the same connection. `nc` and
`openssl s_client` cannot do this; use
[`logwisp auth stream`](cli.md#logwisp-auth).
- A write that misses its deadline means the kernel buffer stayed full for the
whole timeout, so the client is disconnected immediately rather than retried.
- A client whose send queue is full has that event dropped (`dropped_writes`)
@@ -242,6 +266,10 @@ allow = ["viewer-01"]
receives a broadcast. Its connection is closed, the rejection logged at WARN,
and `rejected_conns` incremented.
**Statistics**: `write_errors`, `dropped_writes`, `rejected_conns`,
`tls_handshake_errors`, `auth`, `auth_allowed`, `auth_rejected`; under `scram`
also `auth_users`, `auth_throttled`, `auth_busy` and `auth_binding_mismatch`.
---
## tcp_chain
@@ -285,15 +313,27 @@ key_file = "/etc/logwisp/tls/client.key"
| `keep_alive` | bool | `true` | Enable TCP keep-alive |
| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period |
| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity |
| `auth` | table | — | Server identity pinning; see [Security](security.md#dialer-side-pinning) |
| `auth` | table | — | Server identity pinning (`mtls`) or a login (`scram`); see [Security](security.md#the-auth-block) |
**Behaviour**
- The connection is established lazily, so pipeline start does not depend on the
downstream being up.
- Under `scram` the hello carries the login, and the link counts as established
only once the server's final message proves it holds the user's verifier. The
exchange is bounded at 10 s and cut short by shutdown. A refusal —
`authentication failed`, `authentication not enabled`, `no challenge within
10s` — is logged at WARN as `Chain connect refused` on every attempt; other
connect failures log at DEBUG. Both retry under the normal backoff.
- Each entry is serialized as one canonical JSON line. Delivery holds the line
across reconnects until it is written or the process shuts down, with
exponential backoff plus ±20 % jitter between attempts.
exponential backoff plus ±20 % jitter between attempts. The delay keeps
growing until a link outlives `backoff_min_ms`, so a source that drops every
link right after accepting it is not retried in a tight loop.
- A source never writes once a link is up, so the sink watches each link: a
line from the source is a refusal (a sink without `scram` facing a `scram`
source), logged at WARN as `Chain link refused`; EOF ends the link at once
instead of at the next failed write.
- Because delivery blocks the sink's run loop during an outage, back-pressure
surfaces as a full input queue and is counted by the pipeline as
`total_dropped_by_sink`.
@@ -302,12 +342,12 @@ key_file = "/etc/logwisp/tls/client.key"
- Events arriving without a structured entry are wrapped from the formatted
payload and counted in `synthesized`.
An `auth` block on a dialer pins the server's identity: the policy runs as part
of the handshake, so a server it rejects is treated like any other connect
failure and retried under the normal backoff.
Under `mtls` the policy pins the server's identity as part of the handshake, so
a server it rejects is refused like a failed login above.
**Statistics**: `target`, `node`, `tls`, `auth`, `connected`, `reconnects`,
`write_errors`, `synthesized`.
`write_errors`, `synthesized`; under `scram` also `auth_username`,
`auth_failures` and `last_auth_error` (empty after a successful login).
---
@@ -354,7 +394,7 @@ key_file = "/etc/logwisp/tls/client.key"
| `backoff_min_ms` | int | `500` | Retry backoff floor |
| `backoff_max_ms` | int | `30000` | Retry backoff ceiling |
| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity |
| `auth` | table | — | Server identity pinning; see [Security](security.md#dialer-side-pinning) |
| `auth` | table | — | Server identity pinning (`mtls`) or a login (`scram`); see [Security](security.md#the-auth-block) |
**Behaviour**
@@ -362,18 +402,28 @@ key_file = "/etc/logwisp/tls/client.key"
the first attempt succeeded but the response was lost.
- Retries apply to transport errors, `408`, `429`, and `5xx`. Any other
non-2xx response is treated as permanent, and the batch is dropped and counted
in `dropped_batches`.
in `dropped_batches`. A retry is logged at WARN, a dropped batch at ERROR.
- Under `scram` the sink logs in at `POST /auth` whenever it holds no token and
sends `Authorization: Bearer <token>`. A failed login is retried like a
transport error, holding the batch. The token is renewed ahead of its expiry;
a `401` on ingest (the source reloaded) drops it and the retry logs in again.
A `403` is permanent. A
connection presenting a certificate other than the one the login was bound
to drops token and pin, and the retry logs in anew. A sink without `scram`
facing a `scram` source gets `401` and drops every batch.
- Redirects are never followed; a `3xx` is permanent too. Following one would
resend the batch wherever the response points, plaintext `http` included.
- HTTP/2 is off by design; batched NDJSON POSTs gain nothing from it.
- On shutdown a single best-effort flush of the pending batch is attempted.
**Statistics**: `target`, `node`, `tls`, `auth`, `batches_sent`,
`request_errors`, `dropped_batches`, `synthesized`.
`request_errors`, `dropped_batches`, `synthesized`; under `scram` also
`auth_username`, `auth_failures` and `last_auth_error`.
---
## Sink Statistics
Every sink reports: `id`, `type`, `total_processed`, `active_connections`,
`start_time`, `last_processed`, and a type-specific `details` map.
`start_time`, `last_processed`, and a type-specific `details` map. The DEBUG
status report spreads `details` into each plugin's line.
+31 -18
View File
@@ -190,22 +190,30 @@ node_binding = "force"
| `hello_timeout_ms` | int | `10000` | Deadline for the hello preamble |
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address. Ignored when `auth.node_binding` is active |
| `tls` | table | — | Listener TLS; see [Security](security.md) |
| `auth` | table | — | Peer authorization and node binding; see [Security](security.md#the-auth-block) |
| `auth` | table | — | Peer authentication (`mtls` or `scram`) and node binding; see [Security](security.md#the-auth-block) |
For passwords instead of certificates, the `auth` block takes `type = "scram"`
and a `credentials_file`, and `client_auth` becomes optional; see
[Password Authentication](security.md#password-authentication-scram).
**Behaviour**
- TLS handshakes run explicitly with a 10 s bound before the preamble is read,
after the `max_connections` admission check.
- Authorization runs between the handshake and the hello read, so an
unauthorized peer never gets a preamble parsed on its behalf. A rejection is
logged at WARN and counted in `rejected_conns`.
- A connection is rejected if the first line is not a valid hello with a
matching protocol version.
- Under `mtls` the certificate is checked before the hello is read, so an
unauthorized peer never gets a preamble parsed on its behalf. Under `scram`
the hello opens the login: challenge, proof and final follow on the same
connection, the whole exchange within `hello_timeout_ms`, and the final is
sent only once node binding agrees. A source without `scram` answers a hello
offering credentials with `authentication not enabled`. A rejection is logged
at WARN and counted in `rejected_conns`.
- The node label is then resolved: under `auth.node_binding` it comes from the
peer's certificate, otherwise `trust_node` governs. `force` also overrides the
`node` field on every individual entry; `assert` leaves per-entry labels to
`trust_node`, so a relay can forward other nodes' entries while proving its
own identity.
peer's certificate identity or username, otherwise `trust_node` governs.
`force` also overrides the `node` field on every individual entry; `assert`
leaves per-entry labels to `trust_node`, so a relay can forward other nodes'
entries while proving its own identity.
- Each accepted connection gets a session recording the remote address, node
label, — under TLS — `tls` and `tls_peer_cn`, and — under auth —
`auth_method` and `auth_identity`.
@@ -215,7 +223,8 @@ node_binding = "force"
**Statistics**: `active_connections`, `rejected_conns`, `parse_errors`,
`tls_handshake_errors`, `trust_node`, `auth`, `auth_allowed`, `auth_rejected`,
`node_binding`.
`node_binding`; under `scram` also `auth_users`, `auth_throttled`, `auth_busy`
and `auth_binding_mismatch`.
---
@@ -259,16 +268,19 @@ node_binding = "force"
| `read_timeout_ms` | int | `30000` | Full request read deadline |
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address. Ignored when `auth.node_binding` is active |
| `tls` | table | — | Listener TLS |
| `auth` | table | — | Peer authorization and node binding; see [Security](security.md#the-auth-block) |
| `auth` | table | — | Peer authentication (`mtls` or `scram`) and node binding; see [Security](security.md#the-auth-block) |
**Behaviour**
- Only `POST` to `ingest_path` is routed; other methods get `405` with an
`Allow` header, and other paths get `404`.
- Only `POST` to `ingest_path` and to `/auth` is routed; other methods get `405`
with an `Allow` header, and other paths get `404`. `/auth` is the SCRAM login
and answers `404` unless `auth.type = "scram"`.
- Authorization runs before the body is read, so an unauthorized sender does not
get to stream `max_body_bytes` into the process. Both a policy rejection and a
node-binding failure answer `403`, distinct from the `400` used for protocol
errors, so a sender can tell "not allowed" from "malformed batch".
get to stream `max_body_bytes` into the process. A missing, invalid or expired
bearer token (`scram`) answers `401`, so the sender logs in again; a refused
certificate, a certificate that does not match the token's user, and a
node-binding failure answer `403`. Both are distinct from the `400` used for
protocol errors, so a sender can tell "not allowed" from "malformed batch".
- A missing or mismatched `X-Logwisp-Protocol` header is rejected with `400`.
- Batch acceptance is atomic: entries are published only after the body reads
cleanly end to end. A transfer error rejects the whole batch (`400`, or `413`
@@ -282,12 +294,13 @@ node_binding = "force"
**Statistics**: `total_requests`, `rejected_requests`, `parse_errors`,
`cached_sessions`, `trust_node`, `auth`, `auth_allowed`, `auth_rejected`,
`node_binding`.
`node_binding`; under `scram` also `auth_users`, `auth_throttled`, `auth_busy`,
`auth_binding_mismatch` and `auth_token_lifetime_ms`.
---
## Source Statistics
Every source reports: `id`, `type`, `total_entries`, `dropped_entries`,
`start_time`, `last_entry_time`, and a type-specific `details` map. These appear
in the status reporter output and in the `http` sink's status endpoint.
`start_time`, `last_entry_time`, and a type-specific `details` map. The DEBUG
status report spreads `details` into each plugin's line.