lixen/logwisp
SHA256
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.17.0 mtls added to network chain, sinks, and sources

Browse Source
This commit is contained in:
lixen
2026-08-29 18:44:44 -04:00
parent 5fbd5c71cf
commit dd665bb339
26 changed files with 2293 additions and 488 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/README.md
+10 -5
View File
@@ -18,8 +18,8 @@ streams, or downstream LogWisp nodes.
| [Formatters](formatters.md) | Output shaping and sanitization |
| [Chaining](chaining.md) | Multi-node topologies and the chain wire protocol |
| [Networking](networking.md) | Listeners, dialers, timeouts, connection limits |
| [Security](security.md) | TLS and mTLS configuration, threat model, current limits |
| [mTLS Authentication Plan](mtls-auth-plan.md) | Design for certificate-based authorization |
| [Security](security.md) | TLS, mTLS, and peer authorization; threat model and current limits |
| [mTLS Authentication](mtls-auth-plan.md) | Design and rationale for certificate-based authorization |
| [CLI](cli.md) | Flags, signals, exit codes |
| [Operations](operations.md) | Running, monitoring, tuning, troubleshooting |
@@ -55,12 +55,17 @@ endpoint), `tcp` (broadcast server), `null`, and the chain forwarders
- `raw`, `txt`, and `json` formatting with selectable sanitizer policies
- Optional flow-level heartbeat entries
### Transport security
### Transport security and authentication
- TLS 1.2/1.3 on every network source and sink, listener and dialer alike
- Mutual TLS: listeners can require and verify client certificates; dialers can
present a client identity. See [Security](security.md) for what this does and
does not currently give you.
present a client identity
- Authorization by certificate identity, per listener: named peers rather than
everything the CA issued, with the `http` sink's endpoints gated too
- Node binding, so a chain source labels entries from the sender's certificate
rather than from what the sender claims
See [Security](security.md) for what each layer does and does not give you.
## Quick Start
doc/architecture.md
+7 -4
View File
@@ -147,15 +147,18 @@ TLS is built in exactly one place, `internal/tlsx`, which exposes
Each pipeline owns a `session.Manager`. Plugins receive a `session.Proxy`
scoped to their instance id, so one plugin cannot see or remove another's
sessions. A session records the remote address, creation and last-activity
timestamps, and metadata — including `tls` and `tls_peer_cn` for TLS peers.
timestamps, and metadata — including `tls` and `tls_peer_cn` for TLS peers, and
`auth_method` / `auth_identity` for authorized ones.
Idle sessions are reaped every 5 minutes against a 30-minute idle limit. The
HTTP sink's broker treats a vanished session as an eviction signal and closes
the corresponding SSE client.
Session metadata is currently bookkeeping only: nothing in the pipeline makes
an authorization decision from it. Closing that gap is the subject of the
[mTLS authentication plan](mtls-auth-plan.md).
Authorization decisions do not read session metadata — they are made from the
handshake by `internal/authz`, at the point of connection or request, and their
outcome is *recorded* in the session. That ordering matters: a session exists
only for a peer that was already admitted. See the
[mTLS authentication design](mtls-auth-plan.md).
## Configuration Reload
doc/chaining.md
+30 -6
View File
@@ -45,15 +45,29 @@ Chained entries carry a `node` label identifying where they originated.
Relays preserve `node`, so a label survives any number of hops and identifies
the original producer rather than the last relay.
Under mTLS the source can instead bind the label to the sender's certificate,
which overrides `trust_node` entirely:
| `auth.node_binding` | Connection label | Per-entry `node` field |
|---------------------|------------------|------------------------|
| `none` | `trust_node` governs | `trust_node` governs |
| `assert` | Must equal the certificate identity, or the peer is rejected | `trust_node` governs |
| `force` (default under `mtls`) | The certificate identity | Overwritten with the identity |
Pick `force` at an ingest boundary you do not trust — it is the only setting
where a compromised edge cannot mislabel its entries, including through the
per-entry `node` field. Pick `assert` on a relay-to-relay hop, where the relay
should prove its own identity but the origin labels it forwards must survive.
See [Security](security.md#node-binding).
Formatters render node identity as a syslog-style prefix on the source field:
`edge-01/app.log`. In JSON output the node therefore appears inside the source
field, not as a separate top-level key.
> `trust_node = true` means an authenticated peer can claim **any** node label,
> including one belonging to another host. On an untrusted network use
> `trust_node = false`, or read the
> [mTLS authentication plan](mtls-auth-plan.md), which proposes binding the
> label to the peer's certificate identity.
> `trust_node = true` with no `auth` block means any peer the CA vouches for can
> claim **any** node label, including one belonging to another host. On an
> untrusted network set `auth.type = "mtls"` with `node_binding = "force"`;
> `trust_node = false` is the fallback when certificates are not an option.
## Wire Protocol
@@ -150,6 +164,9 @@ enabled = true
ca_file = "/etc/logwisp/tls/ca.crt"
cert_file = "/etc/logwisp/tls/edge-01.crt"
key_file = "/etc/logwisp/tls/edge-01.key"
[pipelines.plugin_sinks.config.auth]
type = "mtls"
allow = ["relay.internal"] # pin the relay, not just its hostname
```
**Relay** — ingest, keep errors only, archive and stream:
@@ -172,13 +189,16 @@ type = "tcp_chain"
[pipelines.plugin_sources.config]
host = "0.0.0.0"
port = 15801
trust_node = true
[pipelines.plugin_sources.config.tls]
enabled = true
cert_file = "/etc/logwisp/tls/relay.crt"
key_file = "/etc/logwisp/tls/relay.key"
client_auth = true
client_ca_file = "/etc/logwisp/tls/ca.crt"
[pipelines.plugin_sources.config.auth]
type = "mtls"
allow = ["edge-01", "edge-02"]
node_binding = "force" # entries are labelled from the certificate
[[pipelines.plugin_sinks]]
id = "archive"
@@ -195,6 +215,10 @@ host = "127.0.0.1"
port = 8080
```
Entries arriving on this relay are labelled `edge-01` or `edge-02` because that
is what their certificates say, regardless of the `node` each edge configured.
`test/mtls-chain-test.sh` builds exactly this shape against a throwaway PKI.
## Operational Notes
- **Formatting is a relay decision.** Because chain links carry structured
doc/mtls-auth-plan.md
+219 -214
View File
@@ -1,46 +1,50 @@
# Implementation Plan: mTLS as Authentication
# mTLS as Authentication
**Status:** proposal, not implemented.
**Scope:** turn the existing transport-level mutual TLS into a real
authentication and authorization mechanism.
**Status:** implemented. Phases 1–3 of the original proposal, plus dialer-side
server identity pinning from phase 4, are in the tree and covered by
`test/mtls-chain-test.sh`. The remaining phase-4 items are listed under
[Not Implemented](#not-implemented).
**Scope:** turn transport-level mutual TLS into an authentication and
authorization mechanism.
## Problem
LogWisp already does mutual TLS at the transport layer. A listener with
`client_auth = true` refuses any peer that cannot present a certificate
chaining to `client_ca_file`, and `internal/tlsx` already extracts the peer's
Common Name and stashes it in session metadata as `tls_peer_cn`.
LogWisp already did mutual TLS at the transport layer. A listener with
`client_auth = true` refused any peer that could not present a certificate
chaining to `client_ca_file`, and `internal/tlsx` extracted the peer's Common
Name into session metadata as `tls_peer_cn`.
Nothing reads it back. The result is a CA-wide membership check with no notion
Nothing read it back. The result was a CA-wide membership check with no notion
of *which* peer connected:
1. **No per-identity authorization.** Every certificate the CA issues is
equivalent. There is no way to say "only `edge-01` and `edge-02` may write to
this ingest port", so one CA cannot serve several trust domains, and
withdrawing one peer means rotating the CA bundle for all of them.
2. **Node labels are unauthenticated.** With `trust_node = true` (the default) a
peer declares its own `node` label in the chain hello or the
`X-Logwisp-Node` header. Any certificate holder can claim any label,
including another host's, and every downstream consumer will attribute those
entries accordingly. The only current defence, `trust_node = false`, replaces
1. **No per-identity authorization.** Every certificate the CA issued was
equivalent. There was no way to say "only `edge-01` and `edge-02` may write
to this ingest port", so one CA could not serve several trust domains, and
withdrawing one peer meant rotating the CA bundle for all of them.
2. **Node labels were unauthenticated.** With `trust_node = true` (the default)
a peer declares its own `node` label in the chain hello or the
`X-Logwisp-Node` header. Any certificate holder could claim any label,
including another host's, and every downstream consumer would attribute
those entries accordingly. The only defence, `trust_node = false`, replaced
the label with a remote address — unforgeable, but useless for identifying a
host behind NAT or a load balancer.
3. **The `http` sink has no authentication at all**, even with TLS on. Its
stream and status endpoints are readable by anyone who can reach the port.
3. **The `http` sink had no authentication at all**, even with TLS on. Its
stream and status endpoints were readable by anyone who could reach the port.
Password, token, and SCRAM authentication were removed during the plugin/flow
restructure and the move to standard-library networking. Certificates are the
one credential the current transport already carries, which makes mTLS the
cheapest path back to authenticated peers.
one credential the transport already carries, which made mTLS the cheapest path
back to authenticated peers.
## Goals
- Authorize peers by certificate identity, per listener.
- Authorize peers by certificate identity, per listener. ✅
- Bind the chain `node` label to the authenticated identity, so origin
attribution is trustworthy.
- Gate the `http` sink's endpoints on client certificates.
- Make identity visible in sessions, statistics, and logs.
- Change nothing for existing configurations that omit the new block.
attribution is trustworthy. ✅
- Gate the `http` sink's endpoints on client certificates. ✅
- Make identity visible in sessions, statistics, and logs. ✅
- Change nothing for existing configurations that omit the new block. ✅
## Non-Goals
@@ -50,7 +54,7 @@ cheapest path back to authenticated peers.
structs) stay reserved.
- IP allow/deny lists and per-peer rate limits. Related, but a separate feature
with its own config surface.
- OCSP. See [Revocation](#revocation) for what is proposed instead.
- OCSP. See [Revocation](#revocation) for what is done instead.
- Authorization *within* a stream — the unit of decision is a connection (TCP)
or a request (HTTP), never an individual entry.
@@ -61,7 +65,7 @@ cheapest path back to authenticated peers.
The authenticated identity is a single string derived from the peer's verified
leaf certificate. Because `tls.RequireAndVerifyClientCert` has already validated
the chain, signature, and validity window by the time we look, extraction is
pure field selection.
pure field selection — `tlsx.PeerIdentity`.
| `identity` mode | Source | Notes |
|-----------------|--------|-------|
@@ -73,15 +77,15 @@ pure field selection.
An empty identity is a rejection, not an empty match: a certificate with no
usable identity field cannot satisfy any policy.
Identities are not secrets, so ordinary string comparison is fine; there is no
Identities are not secrets, so ordinary string comparison is used; there is no
timing side channel worth defending here.
### Configuration
A new `auth` table sits beside `tls` in every network plugin's `config`. Keeping
it separate from `tls` matters: TLS answers "is this channel private and is the
peer chained to a CA", auth answers "may *this* peer do *this*", and a later
non-certificate method should be able to reuse the block.
An `auth` table sits beside `tls` in every network plugin's `config`. Keeping it
separate from `tls` matters: TLS answers "is this channel private and does the
peer chain to a CA", auth answers "may *this* peer do *this*", and a later
non-certificate method can reuse the block.
```toml
[pipelines.plugin_sources.config.auth]
@@ -94,63 +98,77 @@ node_binding = "force" # none | assert | force
| Option | Type | Default | Meaning |
|--------|------|---------|---------|
| `type` | string | `none` | `none` preserves today's behaviour exactly; `mtls` enables the policy |
| `type` | string | `none` | `none` preserves pre-auth behaviour exactly; `mtls` enables the policy |
| `identity` | string | `cn` | Which certificate field is the identity |
| `allow` | []string | `[]` | Exact identity matches |
| `allow_patterns` | []string | `[]` | RE2 patterns matched against the identity |
| `node_binding` | string | `force` when `type = "mtls"` | See below |
| `node_binding` | string | `force` when `type = "mtls"` | Chain sources only; see below |
Empty `allow` **and** empty `allow_patterns` under `type = "mtls"` means "any
identity the CA vouches for" — that is, today's behaviour, but with the identity
now recorded and node binding available. It is a deliberate, documented default
rather than a silent deny-all, and startup logs say so plainly.
identity the CA vouches for" — that is, the pre-auth behaviour, but with the
identity now recorded and node binding available. It is a deliberate, documented
default rather than a silent deny-all, and the plugin logs a WARN at startup
saying so.
`node_binding` applies only to the chain sources, where a `node` label is
declared:
declared. Setting it on any other plugin is a configuration error.
| Value | Behaviour |
|-------|-----------|
| `none` | `trust_node` governs, as today |
| `assert` | The declared label must equal the identity; a mismatch is rejected |
| `force` | The declared label is ignored and the identity is used |
| Value | Connection label | Per-entry `node` field |
|-------|------------------|------------------------|
| `none` | `trust_node` governs, as before | `trust_node` governs |
| `assert` | Must equal the identity; a mismatch or an omission is rejected | `trust_node` governs |
| `force` | The declared label is ignored and the identity is used | Overwritten with the identity |
`force` is the default under `type = "mtls"` because it is the only setting
where a misconfigured or hostile edge cannot mislabel its entries. `assert`
exists for operators who want the mismatch to be loud rather than silently
corrected. `node_binding` overrides `trust_node`; when both are set, the
constructor logs that `trust_node` is being ignored.
The split between `assert` and `force` is what makes both worth having:
For the `tcp` and `http` sinks, which have no node concept, `node_binding` is
ignored.
- **`force`** is for an ingest boundary that does not trust its peer. Every
entry is relabeled, so a compromised edge cannot smuggle a foreign origin
through the per-entry `node` field either. It is the default under
`type = "mtls"` because it is the only setting where a misconfigured or
hostile edge cannot mislabel its entries.
- **`assert`** is for a relay-to-relay hop. The relay must prove *its own*
identity — a mismatch is loud rather than silently corrected — but the entries
it forwards keep the origin labels stamped at the first hop, so multi-hop
attribution survives.
`node_binding` overrides `trust_node`; when binding is active the constructor
logs that `trust_node` is being ignored.
Dialer-side plugins (`tcp_chain` and `http_chain` sinks) accept the same block
to pin the *server's* identity beyond hostname verification. This is deferred to
phase 4 and does nothing before then.
to pin the *server's* identity beyond hostname verification. There
`node_binding` does not apply, and `tls.insecure_skip_verify` is rejected:
identity read from an unverified chain is a claim, not a fact.
### Validation
At plugin construction, before anything binds:
- `type = "mtls"` on a listener requires `tls.enabled = true` and
`tls.client_auth = true`. Silently accepting an auth policy the transport
cannot enforce is the failure mode worth designing out.
`tls.client_auth = true`; on a dialer it requires `tls.enabled = true` and
forbids `tls.insecure_skip_verify`. Silently accepting an auth policy the
transport cannot enforce is the failure mode worth designing out.
- `identity` must be one of the four modes.
- Every entry in `allow_patterns` must compile.
- `node_binding` must be one of the three values.
- `node_binding` must be one of the three values, and must be absent or `none`
outside the chain sources.
Errors follow existing style: `auth: type "mtls" requires tls.client_auth`.
### New package: `internal/authz`
### The `internal/authz` package
```go
package authz
// Policy is the compiled form of config.AuthOptions.
type Policy struct { /* mode, identity selector, exact set, patterns, binding */ }
type Policy struct { /* role, identity mode, exact set, patterns, binding, counters */ }
// Role selects the validation and behavior appropriate to the call site.
const ( RoleListener Role = iota; RoleChainListener; RoleDialer )
// New compiles a policy. Returns (nil, nil) when auth is disabled, matching
// the tlsx.Server / tlsx.Client convention so callers can nil-check.
func New(o *config.AuthOptions) (*Policy, error)
// the tlsx.Server / tlsx.Client convention. tlsOpts is the sibling `tls`
// block, so an unenforceable policy fails here rather than at run time.
func New(o *config.AuthOptions, tlsOpts *config.TLSOptions, role Role) (*Policy, error)
// Identity is the outcome of a successful authorization.
type Identity struct {
@@ -158,211 +176,198 @@ type Identity struct {
Method string // "mtls"
}
// Apply stamps an identity onto session metadata.
func (id Identity) Apply(meta map[string]any)
// Authorize extracts and checks the peer identity from a completed handshake.
func (p *Policy) Authorize(cs *tls.ConnectionState) (Identity, error)
// ResolveNode applies node_binding to a declared label.
func (p *Policy) ResolveNode(declared string, id Identity) (string, error)
// VerifyConnection is assignable to tls.Config.VerifyConnection on a dialer.
func (p *Policy) VerifyConnection(cs tls.ConnectionState) error
// ResolveNode applies node_binding to the label a peer declared.
func (p *Policy) ResolveNode(declared, fallback string, trustNode bool, id Identity) (string, error)
// TrustsEntryNode reports whether per-entry node labels survive the policy.
func (p *Policy) TrustsEntryNode(trustNode bool) bool
// Stats reports counters for the sink/source stats map.
func (p *Policy) Stats() map[string]any
```
This mirrors `internal/tlsx`: one small package that is the single seam between
declarative config and a cross-cutting concern, with `(nil, nil)` for the
disabled case so every call site is a nil check rather than a branch on config.
declarative config and a cross-cutting concern. `New` returns `(nil, nil)` for
the disabled case and **every method tolerates a nil receiver**, so a call site
reads identically whether or not auth is configured — no nil checks, no branch
on config:
```go
id, err := s.auth.Authorize(tlsState) // nil policy: (zero Identity, nil)
if err != nil { /* reject */ }
```
### Enforcement Points
Each plugin already has the right spot, and in two cases the code says so.
**`tcp_chain` source** (`internal/source/tcpchain/tcpchain.go`, `handleConn`)
Today: handshake → read hello → decode → resolve node from `trust_node` →
create session. Insert authorization between the handshake and the hello read,
so an unauthorized peer never gets a preamble parsed on its behalf, and replace
the node resolution with `Policy.ResolveNode`.
```go
if tlsState != nil && s.authPolicy != nil {
id, err := s.authPolicy.Authorize(*tlsState)
if err != nil {
s.authRejected.Add(1)
s.logger.Warn("msg", "Connection rejected by auth policy",
"component", "tcp_chain_source", "remote_addr", remote, "error", err)
return // deferred cleanup closes conn
}
ident = id
}
// ... read and decode hello ...
connNode, err = s.authPolicy.ResolveNode(hello.Node, ident)
```
Handshake → **authorize** → read hello → `ResolveNode` → create session. The
authorization sits between the handshake and the hello read, so an unauthorized
peer never gets a preamble parsed on its behalf. `chain.DecodeEntry` is then
called with `auth.TrustsEntryNode(trust_node)` rather than `trust_node` itself.
**`http_chain` source** (`internal/source/httpchain/httpchain.go`, `handleIngest`)
Per request, from `r.TLS`, before the body is read — an unauthorized sender
should not get to stream 8 MiB into the process. Rejection is `403`, distinct
from the `400` used for protocol errors, so a sender can tell "you are not
allowed" from "your batch was malformed". `ResolveNode` then governs the
`X-Logwisp-Node` header exactly as it governs the TCP hello.
Per request, from `r.TLS`, before the body is read — an unauthorized sender does
not get to stream `max_body_bytes` into the process. Rejection is `403`,
distinct from the `400` used for protocol errors, so a sender can tell "you are
not allowed" from "your batch was malformed". `ResolveNode` then governs the
`X-Logwisp-Node` header exactly as it governs the TCP hello. The session cache
key includes the identity, so two peers sharing a remote address never share a
session.
**`tcp` sink** (`internal/sink/tcp/tcp.go`, `handleConn`)
There is already a comment marking the place: *"Password-auth extension point:
preamble verification runs in handleConn post-handshake, pre-registration."*
Authorization goes precisely there — after the explicit handshake, before the
session is created and the client is registered — so an unauthorized peer never
appears in the client map and never receives a broadcast.
After the explicit handshake, before the session is created and the client is
registered — so an unauthorized peer never appears in the client map and never
receives a broadcast.
**`http` sink** (`internal/sink/http/http.go`, `Start`)
**`http` sink** (`internal/sink/http/http.go`, `authMiddleware`)
Also already marked: *"Auth extension point: wrap mux with auth middleware once
credentials land, e.g. handler = authMiddleware(cfg)(handler)."* A middleware
around the mux covers both the stream and the status endpoint with one wrapper,
and keeps the handlers themselves unaware of authorization.
A middleware around the mux covers both the stream and the status endpoint with
one wrapper and keeps the handlers themselves unaware of authorization. The
authorized identity is passed down through the request context for session
metadata. Rejections are `403` with no body detail — the status endpoint leaks
host, port, and throughput counters, so a rejection should not leak policy shape
on top of that.
```go
var handler http.Handler = mux
if h.authPolicy != nil {
handler = authMiddleware(h.authPolicy, h.logger)(handler)
}
```
**`tcp_chain` / `http_chain` sinks** (dialers)
The middleware rejects with `403` and no body detail — the status endpoint
leaks host, port, and throughput counters, so a rejection should not leak policy
shape on top of it.
The policy is installed as `tls.Config.VerifyConnection`, which runs after the
standard chain and hostname checks. A server whose identity the policy rejects
fails the handshake itself rather than the first write, and the chain sink's
existing backoff loop handles it as any other connect failure.
### Capabilities
`core.CapAuth` is currently derived from `tlsConfig.ClientAuth`. It should
reflect the auth policy instead:
`core.CapAuth` now means "this plugin authorizes peers" — it is derived from the
policy, not from `tlsConfig.ClientAuth`. Transport-level mTLS without a policy
still reports `CapTLS`, the `mtls=true` field on the startup log line, and the
`tls` statistic.
```go
if s.authPolicy != nil {
caps = append(caps, core.CapAuth)
}
```
`Pipeline.initSourceCapabilities` and `initSinkCapabilities` treat `CapAuth` as
a no-op placeholder today. They become the natural place for a cross-cutting
check: a plugin advertising `CapAuth` without `CapTLS` is a contradiction and
should fail pipeline construction rather than start.
`Pipeline.initSourceCapabilities` and `initSinkCapabilities` treat this as a
cross-cutting check: a plugin advertising `CapAuth` without `CapTLS` is a
contradiction and fails pipeline construction rather than starting.
### Observability
Every authorization decision must be visible, because a silent deny is
Every authorization decision is visible, because a silent deny is
indistinguishable from a network fault at 3am.
- **Session metadata** gains `auth_method` and `auth_identity` alongside the
existing `tls` and `tls_peer_cn`.
- **Statistics** gain `auth_enabled`, `auth_rejected`, and `node_binding` in the
`details` map of every affected source and sink, so rejections show up in the
status reporter and the `http` sink's status endpoint.
- **Logs** record a WARN per rejection with the remote address, the extracted
identity (or the reason extraction failed), and the policy that rejected it.
Accepted connections log the identity at INFO on the chain sources and at
DEBUG on the sinks, matching each plugin's existing verbosity.
- **Statistics** gain `auth`, `auth_identity` (the mode), `auth_unrestricted`,
`auth_allowed`, `auth_rejected`, and — on chain sources — `node_binding`, in
the `details` map of every affected source and sink. They surface in the
status reporter and in the `http` sink's status endpoint.
- **Logs** record a WARN per rejection with the remote address and the reason.
The startup line carries a rendered policy summary
(`auth="mtls identity=cn allow=[1 exact, 0 pattern(s)] node_binding=force"`),
a WARN when the allow list is empty, and an INFO when node binding overrides
`trust_node`.
### Revocation
Certificate revocation is deliberately handled by the allow-list rather than by
CRL or OCSP:
Certificate revocation is handled by the allow-list rather than by CRL or OCSP:
1. Remove the identity from `allow` / `allow_patterns`.
2. `kill -HUP`.
The reload path already rebuilds every pipeline, so the policy takes effect on
the next connection and existing connections are dropped by the rebuild itself.
This is one moving part instead of three, it needs no network calls on the
handshake path, and it is exact — no window between revocation and the next CRL
The reload path rebuilds every pipeline, so the policy takes effect on the next
connection and existing connections are dropped by the rebuild itself. This is
one moving part instead of three, it needs no network calls on the handshake
path, and it is exact — no window between revocation and the next CRL
publication.
A CRL file loaded next to `client_ca_file` and re-read on reload is a reasonable
phase-4 addition for operators with existing CRL infrastructure. OCSP stapling
is out of scope: it adds a network dependency to the handshake path for a
system whose entire design is "never block".
## Phases
### Phase 1 — Identity and policy (no enforcement)
- `internal/config`: add `AuthOptions` plus an `Auth *AuthOptions` field to the
four listener option structs.
- `internal/tlsx`: add `PeerIdentity(cs tls.ConnectionState, mode string) string`
beside the existing `PeerCN`.
- `internal/authz`: new package — `Policy`, `New`, `Authorize`, `ResolveNode`,
`Stats`.
- Unit tests: identity extraction per mode, exact and pattern matching, empty
policy, malformed patterns, node binding in all three modes.
Nothing behaves differently yet, which makes this phase safe to merge alone.
### Phase 2 — Chain sources
- Wire `authz` into `tcp_chain` and `http_chain` sources at the points above.
- Replace the `trust_node` node resolution with `ResolveNode`.
- Add validation, capabilities, statistics, and session metadata.
- Add `test/mtls-chain-test.sh`, modelled on `test/chain-test.sh`: generate a
CA, a server certificate, and two client certificates with `openssl`; assert
that an allowed identity delivers entries, a non-allowed identity is rejected,
a peer with no certificate fails the handshake, and a peer declaring another
node's label is rejected under `assert` and corrected under `force`.
This phase alone closes the node-spoofing hole, which is the sharpest of the
three problems.
### Phase 3 — Sinks
- `tcp` sink: authorize in `handleConn` before registration.
- `http` sink: `authMiddleware` around the mux, covering stream and status.
- Extend the test script to cover both.
### Phase 4 — Optional hardening
- Dialer-side server identity pinning for the chain sinks.
- CRL file support alongside `client_ca_file`, re-read on reload.
- Certificate expiry warnings at startup and on reload — a leaf expiring inside
30 days logged at WARN, since nothing warns today.
- Surface `tls_peer_cn` / `auth_identity` in the `http` sink's status output for
connected clients.
## Compatibility
No configuration breaks. Omitting the `auth` block, or setting
`type = "none"`, reproduces current behaviour byte for byte: `Policy` is nil,
every call site short-circuits, and `trust_node` continues to govern node
labels.
No configuration breaks. Omitting the `auth` block, or setting `type = "none"`,
reproduces the previous behaviour exactly: `Policy` is nil, every call site
short-circuits, and `trust_node` continues to govern node labels.
The one behavioural note for adopters: turning on `type = "mtls"` defaults
`node_binding` to `force`, so entries from a peer whose certificate identity
differs from its configured `node` label will be relabelled. That is the point
of the feature, but it will move data between labels in a dashboard, so the
release note should say it in those terms.
of the feature, but it moves data between labels in a dashboard, so plan for it.
Use `node_binding = "assert"` on relay-to-relay hops where upstream origin
labels must survive.
## Estimated Cost
## Verification
| Phase | Files touched | Rough size |
|-------|--------------|-----------|
| 1 | `config/config.go`, `config/validate.go`, `tlsx/tlsx.go`, new `authz/` + tests | ~400 lines |
| 2 | Two chain sources, new test script | ~200 lines |
| 3 | `sink/tcp`, `sink/http`, test script extension | ~150 lines |
| 4 | `tlsx`, both chain sinks, `sink/http` status | ~250 lines |
`test/mtls-chain-test.sh` builds a full PKI with `openssl` and exercises both
target topologies end to end:
Phases 1–2 are the security-relevant core; phase 3 closes the read-side
exposure; phase 4 is discretionary.
```
./test/mtls-chain-test.sh --auto
```
## Open Questions
Scenario 1 — chained instances, client authenticating with mTLS:
1. **Should an empty allow-list deny instead of allow?** Deny-by-default is the
safer instinct, but it makes `type = "mtls"` with no list a footgun that
silently drops all traffic. The proposal is allow-with-a-loud-startup-log;
the alternative is requiring a non-empty list and erroring at construction,
which is arguably better and costs one line.
2. **Should `identity` accept a list of modes** (try `san_uri`, fall back to
`cn`)? Simpler as a single mode; heterogeneous PKI is the argument against.
3. **Per-identity rate limits.** The natural follow-on once identity exists, and
the natural home for the per-IP limiting that was also removed. Deliberately
out of scope here so this feature stays reviewable.
4. **Whether `assert` should reject or warn-and-correct.** As proposed it
rejects, which is unambiguous but turns a certificate/config mismatch into an
outage. `force` is the forgiving option, and it is the default.
- an authorized edge (`edge-01`) delivers entries through both the `tcp_chain`
and `http_chain` ingest ports into a file sink
- `node_binding = "force"` overrides the label the sender configured
- an identity outside the allow list (`edge-99`) is refused, even while claiming
to be `edge-01`
- a peer presenting no certificate fails the handshake
- a dialer that pins a server identity the relay does not hold refuses to
connect, even though the server certificate chains to the trusted CA
Scenario 2 — a viewer client reading a streaming sink over mTLS:
- an authorized viewer streams from the `tcp` sink and from the `http` sink's
SSE endpoint, and reads `/status`
- a CA-valid but unauthorized viewer gets nothing from the `tcp` sink and `403`
from both `http` sink endpoints
- a client with no certificate fails the handshake
- the status endpoint reports the policy and its rejection count
`test/chain-test.sh` and `test/chain-aggregate-test.sh` continue to pass
unchanged, which is the regression check for the auth-disabled path.
## Not Implemented
The remaining phase-4 items, in rough order of value:
1. **CRL file support** alongside `client_ca_file`, re-read on reload, for
operators with existing CRL infrastructure. The allow-list covers the same
ground with fewer moving parts, so this is only worth doing for a fleet whose
revocation already flows through a CRL.
2. **Certificate expiry warnings** at startup and on reload — a leaf expiring
inside 30 days logged at WARN. Nothing warns today; expiry shows up as a
handshake failure.
3. **Per-identity rate limits.** The natural follow-on now that identity exists,
and the natural home for the per-IP limiting that was also removed. Kept out
of scope here so this feature stayed reviewable.
4. **Per-client identity in the `http` sink's status output.** The endpoint
reports the policy and counters, but not which identities are currently
connected; session metadata has the data.
5. **A list of `identity` modes** (try `san_uri`, fall back to `cn`) for
heterogeneous PKI. A single mode is simpler and covers a uniform CA.
## Decisions Taken
Four questions were left open by the proposal. What was chosen, and why:
1. **An empty allow-list allows rather than denies.** Deny-by-default is the
safer instinct, but `type = "mtls"` with no list is a legitimate
configuration — "any peer this CA issued, but bind the node labels" — and
node binding alone is worth enabling without enumerating every node.
Erroring on it would force operators to list their whole fleet to get
trustworthy attribution. The compromise is a WARN at startup naming the
condition and the fix.
2. **`identity` takes a single mode.** Simpler, and a uniform CA is the common
case. Listed above as a possible extension.
3. **Per-identity rate limits are out of scope**, as proposed.
4. **`assert` rejects rather than warns and corrects.** A certificate/config
mismatch under `assert` is an outage, which is the point: `force` is the
forgiving option and it is the default, so an operator reaches for `assert`
precisely when they want the mismatch to be loud.
doc/networking.md
+23
View File
@@ -168,12 +168,35 @@ headers, and entry encoding.
- Handshake failures appear as WARN with the remote address, and increment
`tls_handshake_errors`.
**Rejected after a successful handshake**
- `auth: identity "..." is not allowed` — the certificate is valid and chains to
the CA, but the identity is not in `auth.allow` / `auth.allow_patterns`. On
TCP the connection is closed; on HTTP the answer is `403`.
- `auth: peer certificate carries no <mode> identity` — `auth.identity` names a
field the certificate does not populate, e.g. `san_dns` on a CN-only leaf.
- `auth: node_binding "assert": declared node "..." does not match identity` —
the sender's `node` option and its certificate disagree. Fix one, or use
`node_binding = "force"` to let the certificate win silently.
- On a dialer, the same message inside `Chain connect failed` means the
*server* was refused: its certificate identity is not in the sink's
`auth.allow`.
- Rejections appear as WARN and increment `auth_rejected`.
**Entries not arriving over a chain link**
- Check the sink's `connected` statistic and its `reconnects` count.
- Check the source's `auth_rejected` — an allow-list miss looks exactly like a
network fault from the sender's side.
- Check the source's `parse_errors` — a version skew shows up here.
- On `http_chain`, remember entries wait up to `flush_interval_ms` before a
batch is sent.
**Entries arriving under an unexpected node label**
- `auth.node_binding` defaults to `force` when `auth.type = "mtls"`, which
relabels every entry with the sender's certificate identity. If a dashboard
suddenly shows a different label, that is why. Use `node_binding = "assert"`
to keep upstream origin labels on relay-to-relay hops, or `"none"` to leave
`trust_node` in charge.
**Clients connect but see nothing**
- The pipeline may be filtering everything out; check `flow.filters` stats.
- The rate limiter may be dropping everything; check `rate_limiter` stats.
doc/operations.md
+8 -6
View File
@@ -268,12 +268,14 @@ then `kill -HUP`. Automate the expiry check; nothing in LogWisp warns you.
**Access review**
With mTLS, any certificate signed by the configured `client_ca_file` is
accepted — there is no per-identity allow-list, so "access review" means
reviewing what your CA has issued. Peer Common Names are recorded in session
metadata but are not surfaced in statistics and are not used for authorization.
See [Security](security.md) and the
[mTLS authentication plan](mtls-auth-plan.md).
With `tls` alone, any certificate signed by the configured `client_ca_file` is
accepted, so "access review" means reviewing what your CA has issued. Add an
`auth` block with an explicit `allow` list and the review becomes the config
file itself: the identities listed there are the ones that can connect, and
removing one plus a `SIGHUP` is the revocation path. Authorized identities are
recorded in session metadata as `auth_identity`; rejections are counted in
`auth_rejected` and logged at WARN. See
[Security](security.md#the-auth-block).
**Secret leakage**
doc/security.md
+199 -46
View File
@@ -10,16 +10,20 @@ configure it, and — equally important — what it does not yet do.
| TLS 1.2 / 1.3 on all network sources and sinks | Implemented |
| Server certificate verification by dialers | Implemented |
| Mutual TLS (client certificate required and verified) | Implemented at the transport layer |
| Peer identity (certificate CN) recorded per session | Implemented |
| Authorization from peer identity (CN allow-lists, node binding) | **Not implemented** — see [mtls-auth-plan.md](mtls-auth-plan.md) |
| Peer identity recorded per session | Implemented |
| Authorization from peer identity (allow-lists, node binding) | Implemented — see [The Auth Block](#the-auth-block) |
| Authentication on the `http` sink's stream and status endpoints | Implemented, via the auth block |
| Server identity pinning by dialers | Implemented, via the auth block |
| Certificate revocation lists (CRL) or OCSP | **Not implemented** — revoke by editing the allow-list |
| Password, token, or SCRAM authentication | **Removed**; not currently available |
| IP allow/deny lists, per-IP connection or request limits | **Not implemented** |
| Authentication on the `http` sink's stream and status endpoints | **Not implemented** |
Earlier releases carried basic-auth, bearer-token, and SCRAM authentication.
Those were removed during the move to the plugin/flow architecture and the
switch to standard-library networking. Only certificate-based transport
security survived that transition.
switch to standard-library networking. Certificates are the one credential the
transport still carries, so they are what authentication is built on: the `tls`
block establishes that a peer chains to your CA, and the `auth` block decides
which peers that CA vouches for may actually do what.
## The TLS Block
@@ -76,6 +80,130 @@ Misconfiguration fails at plugin construction, before the pipeline starts:
certificates
- a `min_version` that is neither `"1.2"` nor `"1.3"`
## The Auth Block
TLS answers "is this channel private, and does the peer chain to a CA". Auth
answers "may *this* peer do *this*". They are separate blocks because they are
separate questions, and because a later non-certificate method should be able to
reuse the second one.
```toml
[pipelines.plugin_sources.config.auth] # or plugin_sinks.config.auth
type = "none" # none | mtls
identity = "cn" # cn | san_dns | san_uri | san_email
allow = []
allow_patterns = []
node_binding = "force" # chain sources only
```
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `type` | string | `none` | `none` ignores the whole block; `mtls` authorizes by certificate identity |
| `identity` | string | `cn` | Which certificate field carries the identity |
| `allow` | []string | `[]` | Exact identities to admit |
| `allow_patterns` | []string | `[]` | RE2 patterns matched against the identity; anchor them yourself |
| `node_binding` | string | `force` under `mtls` | Chain sources only: `none`, `assert`, or `force` |
**Roles by plugin:**
| Plugin | Role | Decides |
|--------|------|---------|
| `tcp_chain` source, `http_chain` source | Listener | Which senders may ingest, and what node label their entries carry |
| `tcp` sink, `http` sink | Listener | Which clients may read the stream (and, on `http`, the status endpoint) |
| `tcp_chain` sink, `http_chain` sink | Dialer | Which server identity to accept, beyond hostname verification |
### Identity
The identity is one string pulled from the peer's verified leaf certificate.
The handshake has already checked the chain, signature, and validity window, so
this is pure field selection.
| Mode | Source | Typical use |
|------|--------|-------------|
| `cn` (default) | `Subject.CommonName` | Matches the existing `tls_peer_cn` metadata |
| `san_dns` | first DNS SAN | Host identities |
| `san_uri` | first URI SAN | SPIFFE-style IDs |
| `san_email` | first email SAN | Operator identities |
A certificate with no usable value in the chosen field is rejected. An empty
identity is a refusal, not an empty match.
### The allow list
`allow` is an exact-match set; `allow_patterns` holds RE2 patterns. An identity
passes if it appears in either.
Leaving **both** empty under `type = "mtls"` admits any identity the CA vouches
for. That is deliberate — it is how you enable node binding without enumerating
a whole fleet — but it is announced rather than silent:
```
WARN msg="Auth policy admits any identity the configured CA vouches for"
component=tcp_chain_source instance_id=in_tcp
hint="set auth.allow or auth.allow_patterns to authorize named peers"
```
Anchor your patterns. `allow_patterns = ["edge-\\d{2}"]` matches
`evil-edge-01-impostor`; `["^edge-\\d{2}$"]` does not.
### Node binding
`node_binding` applies only to the chain sources, and it overrides `trust_node`.
| Value | Connection label | Per-entry `node` field |
|-------|------------------|------------------------|
| `none` | `trust_node` governs | `trust_node` governs |
| `assert` | Must equal the identity; a mismatch or an omission is rejected | `trust_node` governs |
| `force` | Ignored; the identity is used | Overwritten with the identity |
Use **`force`** on an ingest boundary you do not trust. Every entry is
relabelled, so a compromised edge cannot smuggle a foreign origin through the
per-entry `node` field either. It is the default under `type = "mtls"`.
Use **`assert`** on a relay-to-relay hop. The relay must prove its own identity —
a mismatch fails loudly instead of being silently corrected — but the entries it
forwards keep the origin labels stamped at the first hop, so multi-hop
attribution survives.
When binding is active the source says so at startup:
```
INFO msg="Node labels bound to peer identity; trust_node is ignored"
component=tcp_chain_source node_binding=force trust_node=true
```
### Dialer-side pinning
On a chain sink, the same block pins the *server's* identity. Hostname
verification already proves the server holds a certificate valid for the address
you dialed; pinning additionally requires that certificate to name an identity
you listed.
```toml
[pipelines.plugin_sinks.config.auth]
type = "mtls"
allow = ["relay.internal"]
```
The check runs as part of the handshake, so a server the policy rejects never
receives an entry — the sink's normal backoff loop handles it like any other
connect failure. `insecure_skip_verify` is refused alongside `type = "mtls"`:
an identity read from an unverified chain is a claim, not a fact.
### Validation
Misconfiguration fails at plugin construction, before the pipeline starts:
- `type = "mtls"` on a listener without `tls.enabled` **and** `tls.client_auth`
- `type = "mtls"` on a dialer without `tls.enabled`, or with
`tls.insecure_skip_verify`
- an `identity` that is not one of the four modes
- an `allow_patterns` entry that does not compile
- a `node_binding` that is not one of the three values, or one set on a plugin
that has no node concept
Errors read like `auth: type "mtls" requires tls.client_auth`.
## Enabling mTLS
### 1. Generate a CA and certificates
@@ -119,8 +247,17 @@ key_file = "/etc/logwisp/tls/relay.key"
client_auth = true
client_ca_file = "/etc/logwisp/tls/ca.crt"
min_version = "1.3"
[pipelines.plugin_sources.config.auth]
type = "mtls"
allow = ["edge-01", "edge-02"]
node_binding = "force"
```
Without the `auth` block the listener accepts every certificate the CA issued.
With it, only `edge-01` and `edge-02` may ingest, and their entries are labelled
from their certificates rather than from whatever they declare.
### 3. Configure the dialer
```toml
@@ -130,15 +267,21 @@ ca_file = "/etc/logwisp/tls/ca.crt"
cert_file = "/etc/logwisp/tls/edge-01.crt"
key_file = "/etc/logwisp/tls/edge-01.key"
min_version = "1.3"
[pipelines.plugin_sinks.config.auth]
type = "mtls"
allow = ["relay.internal"]
```
### 4. Verify
Startup logs report both flags:
Startup logs report the transport flags and the compiled policy:
```
INFO msg="TCP chain source initialized" ... tls=true mtls=true
auth="mtls identity=cn allow=[2 exact, 0 pattern(s)] node_binding=force"
INFO msg="TCP chain sink initialized" ... tls=true mtls=true
auth="mtls identity=cn allow=[1 exact, 0 pattern(s)] node_binding=none"
```
A client that presents no certificate is refused during the handshake:
@@ -148,60 +291,65 @@ WARN msg="TLS handshake failed" component=tcp_chain_source
remote_addr=127.0.0.1:53840 error="tls: client didn't provide a certificate"
```
Handshake failures are counted in the `tls_handshake_errors` statistic on the
`tcp` sink and the `tcp_chain` source.
A client whose certificate is valid but whose identity is not authorized gets
past the handshake and is refused by the policy:
## What mTLS Currently Buys You
```
WARN msg="Connection rejected by auth policy" component=tcp_chain_source
remote_addr=127.0.0.1:33946 error="auth: identity \"edge-99\" is not allowed"
```
With `client_auth = true`, the transport enforces:
Handshake failures are counted in `tls_handshake_errors`; policy rejections in
`auth_rejected`. Both appear in the status reporter and in the `http` sink's
status endpoint. Accepted peers are recorded in session metadata as
`auth_method` and `auth_identity`.
- the peer holds a certificate chaining to `client_ca_file`
- the certificate is within its validity window and not structurally broken
- the peer holds the matching private key
`test/mtls-chain-test.sh` builds a throwaway PKI and exercises the whole surface
end to end — run it with `--auto` to see each guarantee asserted.
That is a real membership check: an attacker without a CA-issued certificate
cannot connect at all.
## What Each Layer Enforces
## What It Does Not Buy You
**`tls` with `client_auth = true`** — a membership check. The peer holds a
certificate chaining to `client_ca_file`, within its validity window, and holds
the matching private key. An attacker without a CA-issued certificate cannot
connect at all. What it does *not* decide is which CA-issued certificate: every
one is equivalent at this layer.
**Any** valid certificate from the configured CA is accepted. LogWisp extracts
the peer's Common Name into session metadata (`tls_peer_cn`) but never consults
it, so within one CA there is no way to express:
**`auth` with `type = "mtls"`** — an identity check, per listener:
- "only `edge-01` and `edge-02` may connect to this ingest port"
- "the node label `edge-01` may only be claimed by the holder of the `edge-01`
certificate"
- "this certificate may connect but only at this rate"
- only the identities you list may connect, so one CA can serve several trust
domains and a single peer can be withdrawn without touching the others
- the chain `node` label is bound to the certificate, so a compromised edge
cannot attribute its entries to another host
- the `http` sink's stream and status endpoints stop being open to anyone who
can reach the port
Two consequences follow.
**Revocation** is the allow-list, not a CRL. Remove the identity from `allow` /
`allow_patterns` and send `SIGHUP`: the reload rebuilds every pipeline, so the
change takes effect on the next connection and existing ones are dropped by the
rebuild. No network call on the handshake path, and no window between revocation
and the next CRL publication. See
[mtls-auth-plan.md](mtls-auth-plan.md#not-implemented) for what CRL support
would add.
1. **A compromised edge can impersonate any other edge.** With
`trust_node = true` (the default) a peer declares its own node label. Any
certificate holder can claim `edge-99`, or `relay`, and downstream consumers
will attribute its entries accordingly. Setting `trust_node = false` replaces
the label with the remote address, which is coarse but not forgeable at the
application layer.
2. **Revocation is CA-wide.** With no CRL or OCSP checking and no per-identity
allow-list, withdrawing one node's access means re-issuing the CA or rotating
the CA bundle for every peer.
## Surfaces Without Access Control
Closing both gaps is the subject of the
[mTLS authentication plan](mtls-auth-plan.md).
An `auth` block closes each of these. Without one, bind them to a trusted
interface or front them with an authenticating proxy.
## Unauthenticated Surfaces
These endpoints have no access control at all. Bind them to a trusted interface
or front them with an authenticating proxy.
| Surface | Exposure |
|---------|----------|
| Surface | Exposure when `auth.type = "none"` |
|---------|-----------------------------------|
| `http` sink `stream_path` | Full log stream, with `Access-Control-Allow-Origin: *`, so any browser origin can read it |
| `http` sink `status_path` | Host, port, TLS flag, uptime, client counts, throughput counters |
| `tcp` sink | Full log stream to any client that connects |
| `tcp_chain` / `http_chain` source | Ingest from any peer the CA vouches for, under any node label it claims |
`max_connections` bounds concurrency on all three but does not distinguish
`max_connections` bounds concurrency on all of them but does not distinguish
callers.
Note that `auth` requires `client_auth = true`, which requires TLS. There is no
way to authenticate a plaintext listener.
## Operational Guidance
**Certificates**
@@ -211,7 +359,10 @@ callers.
- Key files should be `0600` and owned by the service account.
- Rotation requires a reload (`SIGHUP`), because certificates are loaded once at
plugin construction; there is no on-disk watch for certificate files.
- Check expiry: `openssl x509 -in relay.crt -noout -enddate`.
- Check expiry yourself: `openssl x509 -in relay.crt -noout -enddate`. Nothing
warns before a certificate lapses; it surfaces as a handshake failure.
- Keep the identity field you authorize on stable across rotations. Reissuing a
leaf with a different CN silently drops the peer out of the allow list.
**Deployment**
@@ -220,8 +371,10 @@ callers.
- Never enable `insecure_skip_verify` outside a lab; it disables server
verification entirely and makes the connection trivially interceptable.
- Bind listeners to specific interfaces rather than `0.0.0.0` where you can.
- Use `trust_node = false` on any ingest port reachable from a network you do
not fully control.
- On any ingest port reachable from a network you do not fully control, set
`auth.type = "mtls"` with an explicit `allow` list. `trust_node = false` is the
fallback when certificates are not an option; it is unforgeable but labels
entries by remote address, which is useless behind NAT or a load balancer.
- Run LogWisp as an unprivileged user with write access only to its own log and
configuration directories.
doc/sinks.md
+43 -13
View File
@@ -114,9 +114,15 @@ write_timeout_ms = 0
max_connections = 0
[pipelines.plugin_sinks.config.tls]
enabled = true
cert_file = "/etc/logwisp/tls/server.crt"
key_file = "/etc/logwisp/tls/server.key"
enabled = true
cert_file = "/etc/logwisp/tls/server.crt"
key_file = "/etc/logwisp/tls/server.key"
client_auth = true
client_ca_file = "/etc/logwisp/tls/client-ca.crt"
[pipelines.plugin_sinks.config.auth]
type = "mtls"
allow = ["viewer-01"]
```
| Option | Type | Default | Description |
@@ -130,10 +136,15 @@ key_file = "/etc/logwisp/tls/server.key"
| `write_timeout_ms` | int | `0` | Per-event write deadline; `0` = none |
| `max_connections` | int | `0` | Concurrent stream cap; `0` = unlimited |
| `tls` | table | — | Listener TLS; see [Security](security.md) |
| `auth` | table | — | Client authorization; see [Security](security.md#the-auth-block) |
**Behaviour**
- Only `GET` is routed to either path; anything else gets `405`.
- With an `auth` block, one middleware gates **both** endpoints: an
unauthorized client gets `403` with no body detail, and the rejection is
logged at WARN and counted in `auth_rejected`. The authorized identity is
recorded in the client's session as `auth_method` / `auth_identity`.
- On connect the client receives an `event: connected` frame carrying its
client id, session id, sink instance id, endpoint paths, and buffer size.
- Payloads are framed per the SSE spec, one `data:` line per newline in the
@@ -149,11 +160,13 @@ key_file = "/etc/logwisp/tls/server.key"
- HTTP/2 is negotiated via ALPN when TLS is enabled; plaintext is HTTP/1.1.
**Status endpoint** returns service and version identity, host, port, TLS flag,
active client count, buffer size, uptime, endpoint paths, and the
`total_processed` / `dropped_writes` / `rejected_clients` counters.
the compiled auth policy, active client count, buffer size, uptime, endpoint
paths, and the `total_processed` / `dropped_writes` / `rejected_clients` /
`auth_rejected` counters.
> Both endpoints are unauthenticated, and the stream response carries
> `Access-Control-Allow-Origin: *`, so any web origin can read it. Bind to a
> Without an `auth` block both endpoints are unauthenticated, and the stream
> response carries `Access-Control-Allow-Origin: *`, so any web origin can read
> it. Set `auth.type = "mtls"` (which requires `tls.client_auth`), bind to a
> trusted interface, or put an authenticating reverse proxy in front.
---
@@ -177,9 +190,15 @@ keep_alive_period_ms = 30000
max_connections = 0
[pipelines.plugin_sinks.config.tls]
enabled = true
cert_file = "/etc/logwisp/tls/server.crt"
key_file = "/etc/logwisp/tls/server.key"
enabled = true
cert_file = "/etc/logwisp/tls/server.crt"
key_file = "/etc/logwisp/tls/server.key"
client_auth = true
client_ca_file = "/etc/logwisp/tls/client-ca.crt"
[pipelines.plugin_sinks.config.auth]
type = "mtls"
allow = ["viewer-01"]
```
| Option | Type | Default | Description |
@@ -193,6 +212,7 @@ key_file = "/etc/logwisp/tls/server.key"
| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period |
| `max_connections` | int | `0` | Concurrent connection cap; `0` = unlimited |
| `tls` | table | — | Listener TLS |
| `auth` | table | — | Client authorization; see [Security](security.md#the-auth-block) |
**Behaviour**
@@ -205,6 +225,10 @@ key_file = "/etc/logwisp/tls/server.key"
and stays connected.
- With TLS enabled the handshake runs under a 10 s bound *after* the
`max_connections` check, so concurrent handshakes are bounded too.
- With an `auth` block, authorization runs after that handshake and *before*
registration, so an unauthorized client never enters the client map and never
receives a broadcast. Its connection is closed, the rejection logged at WARN,
and `rejected_conns` incremented.
---
@@ -249,6 +273,7 @@ key_file = "/etc/logwisp/tls/client.key"
| `keep_alive` | bool | `true` | Enable TCP keep-alive |
| `keep_alive_period_ms` | int | `30000` | Keep-alive idle period |
| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity |
| `auth` | table | — | Server identity pinning; see [Security](security.md#dialer-side-pinning) |
**Behaviour**
@@ -265,7 +290,11 @@ key_file = "/etc/logwisp/tls/client.key"
- Events arriving without a structured entry are wrapped from the formatted
payload and counted in `synthesized`.
**Statistics**: `target`, `node`, `tls`, `connected`, `reconnects`,
An `auth` block on a dialer pins the server's identity: the policy runs as part
of the handshake, so a server it rejects is treated like any other connect
failure and retried under the normal backoff.
**Statistics**: `target`, `node`, `tls`, `auth`, `connected`, `reconnects`,
`write_errors`, `synthesized`.
---
@@ -313,6 +342,7 @@ key_file = "/etc/logwisp/tls/client.key"
| `backoff_min_ms` | int | `500` | Retry backoff floor |
| `backoff_max_ms` | int | `30000` | Retry backoff ceiling |
| `tls` | table | — | Dialer TLS; `cert_file`/`key_file` present a client identity |
| `auth` | table | — | Server identity pinning; see [Security](security.md#dialer-side-pinning) |
**Behaviour**
@@ -324,8 +354,8 @@ key_file = "/etc/logwisp/tls/client.key"
- HTTP/2 is off by design; batched NDJSON POSTs gain nothing from it.
- On shutdown a single best-effort flush of the pending batch is attempted.
**Statistics**: `target`, `node`, `tls`, `batches_sent`, `request_errors`,
`dropped_batches`, `synthesized`.
**Statistics**: `target`, `node`, `tls`, `auth`, `batches_sent`,
`request_errors`, `dropped_batches`, `synthesized`.
---
doc/sources.md
+35 -7
View File
@@ -157,6 +157,11 @@ key_file = "/etc/logwisp/tls/server.key"
client_auth = true
client_ca_file = "/etc/logwisp/tls/client-ca.crt"
min_version = "1.3"
[pipelines.plugin_sources.config.auth]
type = "mtls"
allow = ["edge-01", "edge-02"]
node_binding = "force"
```
| Option | Type | Default | Description |
@@ -167,23 +172,34 @@ min_version = "1.3"
| `max_connections` | int | `0` | Concurrent connection cap; `0` = unlimited |
| `read_timeout_ms` | int | `0` | Per-connection idle read deadline; `0` = none |
| `hello_timeout_ms` | int | `10000` | Deadline for the hello preamble |
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address |
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address. Ignored when `auth.node_binding` is active |
| `tls` | table | — | Listener TLS; see [Security](security.md) |
| `auth` | table | — | Peer authorization and node binding; see [Security](security.md#the-auth-block) |
**Behaviour**
- TLS handshakes run explicitly with a 10 s bound before the preamble is read,
after the `max_connections` admission check.
- Authorization runs between the handshake and the hello read, so an
unauthorized peer never gets a preamble parsed on its behalf. A rejection is
logged at WARN and counted in `rejected_conns`.
- A connection is rejected if the first line is not a valid hello with a
matching protocol version.
- The node label is then resolved: under `auth.node_binding` it comes from the
peer's certificate, otherwise `trust_node` governs. `force` also overrides the
`node` field on every individual entry; `assert` leaves per-entry labels to
`trust_node`, so a relay can forward other nodes' entries while proving its
own identity.
- Each accepted connection gets a session recording the remote address, node
label, and — under TLS — `tls` and `tls_peer_cn`.
label, — under TLS — `tls` and `tls_peer_cn`, and — under auth —
`auth_method` and `auth_identity`.
- A malformed entry line increments `parse_errors` and is skipped; the
connection survives. A line over 1 MiB is a protocol violation and terminates
the connection.
**Statistics**: `active_connections`, `rejected_conns`, `parse_errors`,
`tls_handshake_errors`, `trust_node`.
`tls_handshake_errors`, `trust_node`, `auth`, `auth_allowed`, `auth_rejected`,
`node_binding`.
---
@@ -210,6 +226,11 @@ cert_file = "/etc/logwisp/tls/server.crt"
key_file = "/etc/logwisp/tls/server.key"
client_auth = true
client_ca_file = "/etc/logwisp/tls/client-ca.crt"
[pipelines.plugin_sources.config.auth]
type = "mtls"
allow = ["edge-01", "edge-02"]
node_binding = "force"
```
| Option | Type | Default | Description |
@@ -220,13 +241,18 @@ client_ca_file = "/etc/logwisp/tls/client-ca.crt"
| `buffer_size` | int | `1000` | Subscriber channel depth |
| `max_body_bytes` | int | `8388608` | Per-request body cap (8 MiB) |
| `read_timeout_ms` | int | `30000` | Full request read deadline |
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address |
| `trust_node` | bool | `true` | `false` overrides the sender's node label with its remote address. Ignored when `auth.node_binding` is active |
| `tls` | table | — | Listener TLS |
| `auth` | table | — | Peer authorization and node binding; see [Security](security.md#the-auth-block) |
**Behaviour**
- Only `POST` to `ingest_path` is routed; other methods get `405` with an
`Allow` header, and other paths get `404`.
- Authorization runs before the body is read, so an unauthorized sender does not
get to stream `max_body_bytes` into the process. Both a policy rejection and a
node-binding failure answer `403`, distinct from the `400` used for protocol
errors, so a sender can tell "not allowed" from "malformed batch".
- A missing or mismatched `X-Logwisp-Protocol` header is rejected with `400`.
- Batch acceptance is atomic: entries are published only after the body reads
cleanly end to end. A transfer error rejects the whole batch (`400`, or `413`
@@ -234,11 +260,13 @@ client_ca_file = "/etc/logwisp/tls/client-ca.crt"
an otherwise clean transfer is skipped and counted in `parse_errors`.
- Success is `204 No Content` with `X-Logwisp-Accepted` set to the number of
entries ingested.
- Sessions are cached per remote host + declared node and recreated after idle
expiry.
- Sessions are cached per remote host + node + authenticated identity, and
recreated after idle expiry. Including the identity in the key means two peers
sharing a remote address never share a session.
**Statistics**: `total_requests`, `rejected_requests`, `parse_errors`,
`cached_sessions`, `trust_node`.
`cached_sessions`, `trust_node`, `auth`, `auth_allowed`, `auth_rejected`,
`node_binding`.
---