v0.20.0 mtls hardening, fail-closed config, startup warnings, sink fixes

This commit is contained in:
lixen committed 2026-10-02 05:00:33 -04:00
1 parent 815ef345ea
commit f192c26fa2
37 files changed
+1035 -145

No files matched your search

+89
View File
@@ -0,0 +1,89 @@
# Working rules
It is the contract, not a suggestion.
## Comments
- A comment block is **at most 5 lines**. If the explanation needs more, the code
is wrong or the explanation belongs in `doc/`.
- One line is the default. Comment *why*, never *what* — the code says what.
- No narrative, no history ("this used to…", "the plan proposed…"), no restating
the diff, no essays on design philosophy. Git log and `doc/` hold those.
- No comment on a self-evident function. `// Lookup returns the address` above
`func Lookup` is noise.
- Package doc blocks: 5 lines. Files do not get their own prologue.
## Files
- Do not create a file for one or a few helpers. Put them next to what uses them,
or, if shared, in the package's main or type file.
- A new file needs a new concept, not a new function.
- Prefer editing an existing file over adding one.
- When you edit a file, bring its comments up to these rules; older files predate
them.
## Scope
- Implement what was asked. Do not add tables, indirection, telemetry, tests or
abstraction that nothing asked for.
Exception: a task that explicitly invites additions gets the obvious ones, each
named in the PR body; ask before any that is not obvious.
- One mechanism per problem. Two mechanisms doing one job is a bug or a refactor
opportunity.
- Delete before adding. If a change is net-positive lines for a fix, justify it.
- Reverting an existing API to "improve" it is not a fix. Leave working code alone.
- Per-kind data lives in one table indexed by its kind; a new kind adds a row,
never a switch or a parallel field.
## Tests
- One test per rule, named for the rule. No test that restates another.
- Test comments follow the 5-line limit.
- Do not pin lists that a human has to hand-maintain unless the pin prevents a
real regression.
- Collapse tests: when a broader test covers a narrower one, delete the narrower
one or do not add it. No test for simple behaviour that is unlikely to fail or
is already exercised elsewhere.
## Docs and commits
- `doc/` is already long. Condense when you touch it; append only for a new concept
or scope.
- A gap you are deferring goes in the "Not Implemented" list of the design doc it
belongs to: extend the item, or add a concise one in the list's pattern.
- The repository describes the architecture, never a machine: docs and commands
use placeholders for addresses, node names and credentials.
- PR bodies: what changed, why, how it was verified.
- One task, one branch, one PR. Divide the work into commits on that branch; do not
split it across stacked PRs or multiple branches.
- Work that continues after its PR merged starts again from the latest main; never
stack new commits on merged history.
## Gates
`go build ./...`, `gofmt -l` on changed files, and `go test` on the packages the
change touches.
- A change to platform-sensitive code also cross-builds the production target:
`GOOS=freebsd GOARCH=amd64 go build ./cmd/logwisp`.
- Do not run `-race` tests unless investigating a known/suspected race issue; CI
runs them.
## LogWisp
- Plugin options decode through `config.Scan`, never `lconfig.ScanMap`: unknown
keys must fail at any depth. Set defaults on the options struct before the
call; `TestEveryPluginRejectsUnknownKeys` covers every registered plugin.
- Network security has two seams: `tlsx` (TLS configs, startup warnings) and
`authz` (every admission decision). Both return nil when disabled and every
method tolerates a nil receiver; keep call sites branch-free on config.
- A plugin starts goroutines in `Start`, never in its constructor: a rejected
reload discards constructed plugins without calling `Stop`. Reload is the only
rotation path for certificates and secrets.
- The chain `Hello` is the protocol's extension point: optional fields only (old
peers ignore unknown JSON); anything a peer must understand bumps
`chain.ProtocolVersion`.
- Listeners and dialers are IPv4-only (`tcp4`). E2E scripts in `test/` need
`bin/logwisp` and `--auto`, and each owns a port range and a gitignored run
directory.
- Many older files lack a trailing newline and fail `gofmt -l`; format the files
you change, not the tree.
+5 -2
View File
@@ -64,11 +64,13 @@ exponential backoff and jitter.
stream and status endpoints, and lets a dialer pin the server it talks to stream and status endpoints, and lets a dialer pin the server it talks to
- Node binding: a chain source can label entries from the sender's certificate - Node binding: a chain source can label entries from the sender's certificate
instead of from what the sender claims, so origin attribution is not forgeable instead of from what the sender claims, so origin attribution is not forgeable
- Fail-closed configuration: unknown keys are rejected, and startup warns about
expiring certificates, disabled verification and unanchored allow patterns
See [Security](doc/security.md) for configuration and the exact boundary, and See [Security](doc/security.md) for configuration and the exact boundary, and
the [mTLS authentication design](doc/mtls-auth-plan.md) for the rationale and the [mTLS authentication design](doc/mtls-auth-plan.md) for the rationale and
what is deliberately left out. Password, token, and SCRAM authentication were what is deliberately left out. Password authentication (Argon2id-SCRAM) is
removed during the restructure and are not currently available. planned; see the [SCRAM design](doc/scram-auth-plan.md).
## Documentation ## Documentation
@@ -85,6 +87,7 @@ removed during the restructure and are not currently available.
| [Networking](doc/networking.md) | Listeners, dialers, timeouts, connection limits | | [Networking](doc/networking.md) | Listeners, dialers, timeouts, connection limits |
| [Security](doc/security.md) | TLS, mTLS, and peer authorization; threat model and current limits | | [Security](doc/security.md) | TLS, mTLS, and peer authorization; threat model and current limits |
| [mTLS Authentication](doc/mtls-auth-plan.md) | Design and rationale for certificate-based authorization | | [mTLS Authentication](doc/mtls-auth-plan.md) | Design and rationale for certificate-based authorization |
| [Password Authentication](doc/scram-auth-plan.md) | Planned Argon2id-SCRAM design; mTLS hardening |
| [CLI](doc/cli.md) | Flags, signals, exit codes | | [CLI](doc/cli.md) | Flags, signals, exit codes |
| [Operations](doc/operations.md) | Running, monitoring, tuning, troubleshooting | | [Operations](doc/operations.md) | Running, monitoring, tuning, troubleshooting |
+29
View File
@@ -3,8 +3,12 @@ package main
import ( import (
"context" "context"
"path/filepath" "path/filepath"
"strings"
"testing" "testing"
"time"
"logwisp/internal/plugin"
"logwisp/internal/session"
"logwisp/internal/testutil" "logwisp/internal/testutil"
lconfig "github.com/lixenwraith/config" lconfig "github.com/lixenwraith/config"
@@ -77,3 +81,28 @@ func TestInvalidReloadLeavesStatusReporterRunning(t *testing.T) {
}) })
} }
} }
// Every registered plugin decodes through the checked decoder, so a typo in
// any plugin's config, security tables included, refuses to start.
func TestEveryPluginRejectsUnknownKeys(t *testing.T) {
testLogger(t)
manager := session.NewManager(time.Hour)
t.Cleanup(manager.Stop)
proxy := session.NewProxy(manager, "typo")
bad := map[string]any{"no_such_key": true}
check := func(kind, name string, err error) {
if err == nil || !strings.Contains(err.Error(), "no_such_key") {
t.Errorf("%s %q accepted an unknown key: %v", kind, name, err)
}
}
for _, name := range plugin.ListSources() {
factory, _ := plugin.GetSource(name)
_, err := factory("typo", bad, logger, proxy)
check("source", name, err)
}
for _, name := range plugin.ListSinks() {
factory, _ := plugin.GetSink(name)
_, err := factory("typo", bad, logger, proxy)
check("sink", name, err)
}
}
+6 -5
View File
@@ -6,11 +6,12 @@
### Commented values are the built-in defaults unless marked "example". ### Commented values are the built-in defaults unless marked "example".
### Uncommenting a default is a no-op. ### Uncommenting a default is a no-op.
### ###
### NOTE: password/token/SCRAM authentication, network access control (ACL), ### NOTE: password authentication (planned: doc/scram-auth-plan.md), network
### http/tcp ingest sources, and http_client/tcp_client sinks were removed in ### access control (ACL), http/tcp ingest sources, and http_client/tcp_client
### the restructure and are not available in this version. TLS and mTLS ARE ### sinks are not available in this version. TLS and mTLS ARE available on
### available on every network source and sink; see the [...tls] blocks below, ### every network source and sink; see the [...tls] blocks below, and the
### and the [...auth] blocks to authorize peers by certificate identity. ### [...auth] blocks to authorize peers by certificate identity.
### Unknown keys in any plugin config are rejected at startup.
### ###
### Environment overrides use the LOGWISP_ prefix ### Environment overrides use the LOGWISP_ prefix
### (LOGWISP_QUIET, LOGWISP_LOGGING_LEVEL, ...). Bare names are ignored. ### (LOGWISP_QUIET, LOGWISP_LOGGING_LEVEL, ...). Bare names are ignored.
+1
View File
@@ -20,6 +20,7 @@ streams, or downstream LogWisp nodes.
| [Networking](networking.md) | Listeners, dialers, timeouts, connection limits | | [Networking](networking.md) | Listeners, dialers, timeouts, connection limits |
| [Security](security.md) | TLS, mTLS, and peer authorization; threat model and current limits | | [Security](security.md) | TLS, mTLS, and peer authorization; threat model and current limits |
| [mTLS Authentication](mtls-auth-plan.md) | Design and rationale for certificate-based authorization | | [mTLS Authentication](mtls-auth-plan.md) | Design and rationale for certificate-based authorization |
| [Password Authentication](scram-auth-plan.md) | Planned Argon2id-SCRAM design; mTLS hardening |
| [CLI](cli.md) | Flags, signals, exit codes | | [CLI](cli.md) | Flags, signals, exit codes |
| [Operations](operations.md) | Running, monitoring, tuning, troubleshooting | | [Operations](operations.md) | Running, monitoring, tuning, troubleshooting |
+3 -1
View File
@@ -241,7 +241,9 @@ Startup validation is intentionally split.
Everything else is validated by the plugin constructor that owns it — port Everything else is validated by the plugin constructor that owns it — port
range, required paths, path prefixes, enum values, regex compilation, TLS file range, required paths, path prefixes, enum values, regex compilation, TLS file
loading. A failure there aborts pipeline construction with a message naming the loading. A failure there aborts pipeline construction with a message naming the
pipeline, plugin id, and offending key. pipeline, plugin id, and offending key. A key the plugin does not declare, at
any depth, is such a failure (`unknown key "tls.enabeld"`): a misspelled option
must not silently fall back to its default.
There is **no** cross-pipeline port-conflict detection. Two sinks bound to the There is **no** cross-pipeline port-conflict detection. Two sinks bound to the
same port fail at listener bind time, when the pipeline starts. same port fail at listener bind time, when the pipeline starts.
+7 -10
View File
@@ -48,10 +48,10 @@ back to authenticated peers.
## Non-Goals ## Non-Goals
- Reviving password, token, or SCRAM authentication. The reserved hooks - Password authentication. It is designed separately in
(`chain.Hello.Auth`, the `Authorization` header comment in the `http_chain` [scram-auth-plan.md](scram-auth-plan.md), reusing the `auth` block and the
sink, the "Future: password auth block" comments in the network options reserved hooks (`chain.Hello`, the `Authorization` header in the `http_chain`
structs) stay reserved. sink, the "Future: password auth block" comments in the options structs).
- IP allow/deny lists and per-peer rate limits. Related, but a separate feature - IP allow/deny lists and per-peer rate limits. Related, but a separate feature
with its own config surface. with its own config surface.
- OCSP. See [Revocation](#revocation) for what is done instead. - OCSP. See [Revocation](#revocation) for what is done instead.
@@ -341,16 +341,13 @@ The remaining phase-4 items, in rough order of value:
operators with existing CRL infrastructure. The allow-list covers the same operators with existing CRL infrastructure. The allow-list covers the same
ground with fewer moving parts, so this is only worth doing for a fleet whose ground with fewer moving parts, so this is only worth doing for a fleet whose
revocation already flows through a CRL. revocation already flows through a CRL.
2. **Certificate expiry warnings** at startup and on reload — a leaf expiring 2. **Per-identity rate limits.** The natural follow-on now that identity exists,
inside 30 days logged at WARN. Nothing warns today; expiry shows up as a
handshake failure.
3. **Per-identity rate limits.** The natural follow-on now that identity exists,
and the natural home for the per-IP limiting that was also removed. Kept out and the natural home for the per-IP limiting that was also removed. Kept out
of scope here so this feature stayed reviewable. of scope here so this feature stayed reviewable.
4. **Per-client identity in the `http` sink's status output.** The endpoint 3. **Per-client identity in the `http` sink's status output.** The endpoint
reports the policy and counters, but not which identities are currently reports the policy and counters, but not which identities are currently
connected; session metadata has the data. connected; session metadata has the data.
5. **A list of `identity` modes** (try `san_uri`, fall back to `cn`) for 4. **A list of `identity` modes** (try `san_uri`, fall back to `cn`) for
heterogeneous PKI. A single mode is simpler and covers a uniform CA. heterogeneous PKI. A single mode is simpler and covers a uniform CA.
## Decisions Taken ## Decisions Taken
+7 -1
View File
@@ -140,7 +140,8 @@ retry. Browsers reconnect SSE streams automatically.
TLS is enabled. Only `GET` is routed to the stream and status paths. Each event TLS is enabled. Only `GET` is routed to the stream and status paths. Each event
is framed as one `data:` line per newline in the payload, so multi-line entries is framed as one `data:` line per newline in the payload, so multi-line entries
stream intact. Response headers set `Cache-Control: no-cache`, stream intact. Response headers set `Cache-Control: no-cache`,
`X-Accel-Buffering: no`, and `Access-Control-Allow-Origin: *`. `X-Accel-Buffering: no`, and, without an auth policy,
`Access-Control-Allow-Origin: *`.
**TCP sink** — raw payload bytes, no framing added by the sink. Whether entries **TCP sink** — raw payload bytes, no framing added by the sink. Whether entries
are newline-delimited depends on the formatter. are newline-delimited depends on the formatter.
@@ -150,6 +151,11 @@ headers, and entry encoding.
## Troubleshooting ## Troubleshooting
**Plugin fails to start with `unknown key "..."`**
- The key is misspelled or belongs to another plugin type. Every plugin rejects
keys it does not declare, nested tables included, so a typo in `tls` or `auth`
cannot leave a protection silently off.
**Connection refused** **Connection refused**
- Confirm the pipeline started; a bind failure is logged at ERROR. - Confirm the pipeline started; a bind failure is logged at ERROR.
- Confirm you are dialing IPv4. `localhost` may resolve to `::1`. - Confirm you are dialing IPv4. `localhost` may resolve to `::1`.
+2 -1
View File
@@ -281,7 +281,8 @@ openssl x509 -in /etc/logwisp/tls/relay.crt -noout -enddate
``` ```
Certificates load at plugin construction, so rotation is: write the new files, Certificates load at plugin construction, so rotation is: write the new files,
then `kill -HUP`. Automate the expiry check; nothing in LogWisp warns you. then `kill -HUP`. Startup and every reload warn at WARN once a certificate is
within 30 days of expiry; automate the check anyway.
**Access review** **Access review**
+243
View File
@@ -0,0 +1,243 @@
# Password Authentication (Argon2id-SCRAM)
**Status:** planned. Ships in two batches:
1. **Done:** `lixenwraith/auth` gained SCRAM channel binding, and the mTLS layer
was hardened (see [mTLS Hardening](#mtls-hardening)).
2. **Next:** the SCRAM integration described here, pinned to the tagged `auth`
release that carries channel binding.
**Scope:** optional username/password authentication on every network plugin,
next to the existing certificate method, without weakening what mTLS gives.
## Goals
- Password authentication for chain links (`tcp_chain`, `http_chain`: sources
verify, sinks present) and for viewers of the `tcp` and `http` sinks.
- No password or password-equivalent on the wire, no KDF work on listeners, and
no credential relay through a TLS-terminating MITM.
- Default stays `auth.type = "none"`; existing configurations are unaffected.
- Config in TOML, credential management and viewer clients in a `logwisp auth`
CLI.
## Non-Goals
- Standard SASL SCRAM-SHA-256 interoperability. The `auth` exchange is
logwisp-to-logwisp (or logwisp CLI) only.
- Browser `EventSource` viewers under SCRAM: they cannot set `Authorization` or
run Argon2. mTLS remains the browser path.
- Multiple backends behind a non-sticky L7 balancer: handshake state and the
token key live in one process.
## Mechanism
`lixenwraith/auth` Argon2id-SCRAM. The listener stores a verifier per user:
`StoredKey = SHA-256(HMAC(K, "Client Key"))`, `ServerKey = HMAC(K, "Server Key")`,
`K = Argon2id(password, salt)`. The Argon2id digest itself is never stored: it is
the salted password, from which both keys derive. Verifying a proof is HMAC only;
Argon2 runs on the dialer or CLI (64 MiB, t=3, the `auth` defaults).
**Channel binding.** Both peers pass `auth.WithChannelBinding(SHA-256(server leaf
DER))` at the proof step. The server hashes its configured certificate; the
client hashes the certificate it saw. A relay presenting any other certificate,
even one the CA trusts, makes the server reject the proof, so it never obtains a
link or a token. A mismatch is indistinguishable from a wrong password by design;
the client also sends its hash as a diagnostic field so the listener can log
"TLS interception or terminating proxy" and count it. TLS must terminate at
logwisp.
**HTTP pinning.** HTTP auth takes two requests and every later request may use a
new pooled connection, so binding the exchange is not enough. After step 1 the
dialer pins the bound hash on its `VerifyConnection`: step 2 and every ingest
request must present the same leaf. A mismatch clears token and pin and
re-authenticates, which also covers certificate rotation. A token handed to curl
by the CLI is unpinned.
## Configuration
```toml
# listener (tcp_chain/http_chain source, tcp/http sink)
[pipelines.plugin_sources.config.auth]
type = "scram"
credentials_file = "/etc/logwisp/users.toml"
token_lifetime_ms = 900000 # http sink and http_chain source only
node_binding = "force" # chain sources; binds to the username
# dialer (tcp_chain/http_chain sink)
[pipelines.plugin_sinks.config.auth]
type = "scram"
username = "edge-01"
password_file = "/etc/logwisp/edge-01.pass"
```
Validation at construction:
- `scram` requires TLS; a dialer also forbids `insecure_skip_verify`.
- Listeners require `credentials_file` and reject `allow`/`allow_patterns`: the
credentials file is the allow list. Use one file per listener when listeners
admit different users.
- Dialers require `username` + `password_file`, and reject `credentials_file`,
`allow*` and `identity` (server pinning stays an mTLS feature).
- `identity` on a listener requires `tls.client_auth` and **binds the certificate
to the user**: the certificate's identity field must equal the SCRAM username,
so a peer needs its own certificate and its own password (PostgreSQL's
`clientcert=verify-full`). Without it the factors are independent: any
CA-issued certificate plus any valid password (`clientcert=verify-ca`).
- `token_lifetime_ms` only on HTTP listeners; default 15 minutes.
- `/auth` is reserved; no configured path may equal it.
**Credentials file** (TOML, written by the CLI, mode 0600):
```toml
decoy_key = "<base64, 32 random bytes>"
[[users]]
username = "edge-01"
salt = "<base64>"
argon_time = 3
argon_memory = 65536
argon_threads = 4
stored_key = "<base64>"
server_key = "<base64>"
```
`decoy_key` keeps unknown-user challenges stable across restarts and edits, so
probing usernames reveals nothing. It is created by the first `add-user` and
preserved by every rewrite. An empty user set, a duplicate user, a missing
`decoy_key` or mixed KDF profiles are configuration errors. The file is read at
construction; the SCRAM server starts in the plugin's `Start` and stops in `Stop`,
so a rejected reload leaks nothing.
## Wire Protocol
`chain.Hello` gains `scram` (raw JSON, decoded by `authz`); the protocol version
stays 1 because the field is additive. Every message after the hello is one
`authStep`: `error`, `challenge`, `proof` (+ `binding`), `final` (+ `token`,
`expires_in` on HTTP). Pre-auth lines are capped at 4 KiB.
**TCP** (`tcp_chain`, `tcp` sink): hello carrying the client-first message,
challenge, proof, final — all under one deadline (`hello_timeout_ms` on the
`tcp_chain` source, 10 s on the `tcp` sink and every dialer, with the dialer's
context able to cut it short). The final is written only after every admission
check (certificate binding, node binding); the stream then continues on the same
buffered reader. A source without SCRAM answers a hello carrying credentials with
`error: authentication not enabled`; only an old binary stays silent, and the
dialer's deadline covers that.
**HTTP** (`http_chain` source, `http` sink): `POST /auth`, outside the auth
middleware, with its own 10 s read/write deadline and a 4 KiB body limit.
| Step | Request body | Success | Failure |
|------|--------------|---------|---------|
| 1 | hello | `200` challenge | `429` throttled, `503` handshake table full, `400` malformed |
| 2 | proof | `200` final, token, expires_in | `401 {"error":"authentication failed"}` |
Protected endpoints take `Authorization: Bearer <token>`: missing or invalid is
`401` with `WWW-Authenticate: Bearer realm="logwisp"`, a certificate-binding miss
is `403`. Tokens are HS256 JWTs from `auth.NewJWT` with a random per-instance key,
so every reload revokes them. An SSE stream is checked at connect and outlives its
token; a reload ends it.
**Dialer outcomes** (`http_chain` sink): every `/auth` failure is transient (the
batch is held under backoff); `404`/`405` from `/auth` logs "no auth endpoint:
older logwisp or auth.type is not scram"; an ingest `401` clears the token and is
retried; `403` drops the batch; a pin mismatch clears token and pin. Rejections
are logged at WARN once per change of state and counted in `auth_failures` /
`last_auth_error`.
## Throttling
Per remote IP (the socket address, never a forwarded header), on handshake
starts: a token bucket (burst 10, 1/s) refunded on success, and at most 4
unfinished exchanges. The table is bounded and fails closed when full. Abandoned
TCP exchanges release their slot in the `auth` handshake table immediately.
Counters: `auth_allowed` and `auth_rejected` once per exchange outcome;
`auth_throttled`, `auth_busy` and `auth_binding_mismatch` separately.
## The `authz` Seam
Listeners never call `Authorize` directly any more; one entry point per transport
decides, and `Authorize` itself refuses under `scram` so a missed call site fails
closed.
- `Admit(conn, cs, wantHello, timeout)` returns an admission (identity, hello,
reader) whose `Accept` or `Reject` writes the final message.
- `AuthorizeRequest(r)` covers mTLS certificates and bearer tokens;
`ServeAuth(w, r)` serves `/auth`.
- Dialers: `Greet(ctx, conn, node)` writes the hello (plain or SCRAM);
`Authenticate`, `SetAuthorization`, `ClearToken` and `VerifyConnection` serve
HTTP.
- `Start`/`Close` bracket the SCRAM server and are nil-safe like everything else.
## CLI
```
logwisp auth add-user -credentials F -user U [-password-file P] [-generate]
logwisp auth remove-user -credentials F -user U
logwisp auth token -url https://host:port -user U -password-file P [TLS flags]
logwisp auth stream -addr host:port -user U -password-file P [TLS flags]
```
- `add-user` takes the password from `-password-file` when it exists; a new user
without one gets a generated 130-bit password, written to that file (0600) or
printed once. Rotating an existing user needs the file or `-generate`, so a
mistyped path cannot replace a password. The file's KDF profile and decoy key
are reused, the result is validated with the daemon's own loader, and the
rewrite (temp file in the same directory, rename) keeps the existing mode and
owner.
- `remove-user` refuses to remove the last user.
- Both print that changes apply on `SIGHUP`: `auto_reload` does not watch the
credentials file.
- `token` prints a bearer token for curl (`-H @<(...)` keeps it out of argv);
`stream` authenticates to a `tcp` sink and copies the stream to stdout.
- TLS flags: `-ca-file`, `-server-name`, `-cert-file`, `-key-file`. There is no
insecure flag.
## Rollout and Rotation
- Turning SCRAM on is per listener and cuts off unconfigured dialers. To migrate
without a gap, upgrade binaries, add a second SCRAM listener, move dialers,
then remove the old listener.
- Password rotation: `add-user -generate`, `SIGHUP` the listener, deploy the new
password file, `SIGHUP` the dialer. The dialer retries under backoff in
between.
- Revocation: `remove-user`, `SIGHUP`. All tokens die with the reload.
## Verification
Go tests, one per rule: binding (match, mismatch, one-sided, length) in `auth`;
in `authz` the validation table, credential loading, decoy stability across
edits, TCP exchange over `net.Pipe` (success, wrong password equals unknown
user, binding mismatch, missing hello, silent peer within the deadline,
abandoned exchange frees its slot, final and first entry in one write, assert
mismatch gets an error rather than a final), HTTP exchange on a TLS test server
(token accepted, garbage and foreign tokens `401`, forged final yields no token,
a different certificate after step 1 refused before the body is sent), the
limiter bound; `http_chain` sink re-authentication and outcome table; CLI
add/remove/rotate, mode preservation, last-user refusal, password trimming.
`test/scram-chain-test.sh --auto` (ports 15821-15825): a relay with SCRAM on all
four listener types plus one `client_auth` + SCRAM port; authorized edges over
both chain transports; node labels forced to usernames; wrong password, unknown
user and no-auth edges deliver nothing; a `tcp` sink viewer through `logwisp auth
stream` held past the exchange deadline; a raw TLS client without a hello gets
nothing; the `http` sink through `logwisp auth token` and curl; missing and bad
tokens get `401`; `remove-user` + `SIGHUP` revokes token and login; throttling
last. The existing scripts keep passing.
## mTLS Hardening
Shipped with this plan, independent of SCRAM:
- The `http_chain` sink no longer follows redirects. Go's client resends the
body on a 307/308, including from `https` to plaintext `http`.
- Its response drain is bounded (64 KiB).
- The `tcp` sink tracks connections from accept, so `Stop` and reloads no longer
wait out the 10 s handshake timeout of a silent peer.
- Unknown keys in any plugin's config, at any depth, fail construction: a typo
such as `[...tls] enabeld = true` used to switch protection off silently.
- Startup and every reload warn about certificates that expired, are not yet
valid, or expire within 30 days; about `insecure_skip_verify`; about
`allow_patterns` that are not anchored at both ends of every alternative; and,
once per path, about key files every local user can read.
- An `http` sink with an auth policy no longer sends
`Access-Control-Allow-Origin: *`.
+30 -13
View File
@@ -14,16 +14,16 @@ configure it, and — equally important — what it does not yet do.
| Authorization from peer identity (allow-lists, node binding) | Implemented — see [The Auth Block](#the-auth-block) | | Authorization from peer identity (allow-lists, node binding) | Implemented — see [The Auth Block](#the-auth-block) |
| Authentication on the `http` sink's stream and status endpoints | Implemented, via the auth block | | Authentication on the `http` sink's stream and status endpoints | Implemented, via the auth block |
| Server identity pinning by dialers | Implemented, via the auth block | | Server identity pinning by dialers | Implemented, via the auth block |
| Startup warnings for expiring certificates and risky settings | Implemented — see [Startup Warnings](#startup-warnings) |
| Unknown configuration keys rejected | Implemented — a typo in `tls` or `auth` fails startup |
| Certificate revocation lists (CRL) or OCSP | **Not implemented** — revoke by editing the allow-list | | Certificate revocation lists (CRL) or OCSP | **Not implemented** — revoke by editing the allow-list |
| Password, token, or SCRAM authentication | **Removed**; not currently available | | Password (Argon2id-SCRAM) authentication | **Planned** — see [scram-auth-plan.md](scram-auth-plan.md) |
| IP allow/deny lists, per-IP connection or request limits | **Not implemented** | | IP allow/deny lists, per-IP connection or request limits | **Not implemented** |
Earlier releases carried basic-auth, bearer-token, and SCRAM authentication. Certificates are the credential the transport already carries, so they are what
Those were removed during the move to the plugin/flow architecture and the authentication is built on today: the `tls` block establishes that a peer chains
switch to standard-library networking. Certificates are the one credential the to your CA, and the `auth` block decides which peers that CA vouches for may
transport still carries, so they are what authentication is built on: the `tls` actually do what.
block establishes that a peer chains to your CA, and the `auth` block decides
which peers that CA vouches for may actually do what.
## The TLS Block ## The TLS Block
@@ -79,6 +79,8 @@ Misconfiguration fails at plugin construction, before the pipeline starts:
- a certificate or key that will not load, or a CA file containing no - a certificate or key that will not load, or a CA file containing no
certificates certificates
- a `min_version` that is neither `"1.2"` nor `"1.3"` - a `min_version` that is neither `"1.2"` nor `"1.3"`
- a key the plugin does not know, at any depth: `unknown key "tls.enabeld"`. A
misspelled option must not silently leave a protection switched off
## The Auth Block ## The Auth Block
@@ -144,7 +146,8 @@ WARN msg="Auth policy admits any identity the configured CA vouches for"
``` ```
Anchor your patterns. `allow_patterns = ["edge-\\d{2}"]` matches Anchor your patterns. `allow_patterns = ["edge-\\d{2}"]` matches
`evil-edge-01-impostor`; `["^edge-\\d{2}$"]` does not. `evil-edge-01-impostor`; `["^edge-\\d{2}$"]` does not. An unanchored pattern is
reported at startup.
### Node binding ### Node binding
@@ -204,6 +207,18 @@ Misconfiguration fails at plugin construction, before the pipeline starts:
Errors read like `auth: type "mtls" requires tls.client_auth`. Errors read like `auth: type "mtls" requires tls.client_auth`.
## Startup Warnings
Some settings work but are usually mistakes. Each plugin reports them at WARN
when it is constructed, so startup and every reload repeat them:
- a certificate in `cert_file`, `ca_file` or `client_ca_file` that has expired,
is not yet valid, or expires within 30 days
- `insecure_skip_verify` on a dialer
- an `allow_patterns` entry not anchored at both ends of every alternative
(`^a|b$` admits `a…` and `…b`)
- a `key_file` every local user can read (once per path per process)
## Enabling mTLS ## Enabling mTLS
### 1. Generate a CA and certificates ### 1. Generate a CA and certificates
@@ -339,7 +354,7 @@ interface or front them with an authenticating proxy.
| Surface | Exposure when `auth.type = "none"` | | Surface | Exposure when `auth.type = "none"` |
|---------|-----------------------------------| |---------|-----------------------------------|
| `http` sink `stream_path` | Full log stream, with `Access-Control-Allow-Origin: *`, so any browser origin can read it | | `http` sink `stream_path` | Full log stream, with `Access-Control-Allow-Origin: *`, so any browser origin can read it (the header is omitted once an auth policy is set) |
| `http` sink `status_path` | Host, port, TLS flag, uptime, client counts, throughput counters | | `http` sink `status_path` | Host, port, TLS flag, uptime, client counts, throughput counters |
| `tcp` sink | Full log stream to any client that connects | | `tcp` sink | Full log stream to any client that connects |
| `tcp_chain` / `http_chain` source | Ingest from any peer the CA vouches for, under any node label it claims | | `tcp_chain` / `http_chain` source | Ingest from any peer the CA vouches for, under any node label it claims |
@@ -356,11 +371,12 @@ way to authenticate a plaintext listener.
- Use a dedicated CA for LogWisp so its trust decisions stay independent. - Use a dedicated CA for LogWisp so its trust decisions stay independent.
- Keep leaf lifetimes short (90–825 days) and automate renewal. - Keep leaf lifetimes short (90–825 days) and automate renewal.
- Key files should be `0600` and owned by the service account. - Key files should be `0600` and owned by the service account; a world-readable
one is reported at startup.
- Rotation requires a reload (`SIGHUP`), because certificates are loaded once at - Rotation requires a reload (`SIGHUP`), because certificates are loaded once at
plugin construction; there is no on-disk watch for certificate files. plugin construction; there is no on-disk watch for certificate files.
- Check expiry yourself: `openssl x509 -in relay.crt -noout -enddate`. Nothing - Startup and every reload warn 30 days before a certificate lapses. Still
warns before a certificate lapses; it surfaces as a handshake failure. automate renewal; the warning only reaches someone reading the log.
- Keep the identity field you authorize on stable across rotations. Reissuing a - Keep the identity field you authorize on stable across rotations. Reissuing a
leaf with a different CN silently drops the peer out of the allow list. leaf with a different CN silently drops the peer out of the allow list.
@@ -369,7 +385,8 @@ way to authenticate a plaintext listener.
- Prefer `min_version = "1.3"`. Drop to `"1.2"` only for a peer that genuinely - Prefer `min_version = "1.3"`. Drop to `"1.2"` only for a peer that genuinely
cannot do 1.3. cannot do 1.3.
- Never enable `insecure_skip_verify` outside a lab; it disables server - Never enable `insecure_skip_verify` outside a lab; it disables server
verification entirely and makes the connection trivially interceptable. verification entirely and makes the connection trivially interceptable. It is
reported at startup.
- Bind listeners to specific interfaces rather than `0.0.0.0` where you can. - Bind listeners to specific interfaces rather than `0.0.0.0` where you can.
- On any ingest port reachable from a network you do not fully control, set - On any ingest port reachable from a network you do not fully control, set
`auth.type = "mtls"` with an explicit `allow` list. `trust_node = false` is the `auth.type = "mtls"` with an explicit `allow` list. `trust_node = false` is the
+3 -1
View File
@@ -178,7 +178,7 @@ counters.
> Without an `auth` block both endpoints are unauthenticated, and the stream > Without an `auth` block both endpoints are unauthenticated, and the stream
> response carries `Access-Control-Allow-Origin: *`, so any web origin can read > response carries `Access-Control-Allow-Origin: *`, so any web origin can read
> it. Set `auth.type = "mtls"` (which requires `tls.client_auth`), bind to a > it. With one, the header is omitted. Set `auth.type = "mtls"` (which requires `tls.client_auth`), bind to a
> trusted interface, or put an authenticating reverse proxy in front. > trusted interface, or put an authenticating reverse proxy in front.
--- ---
@@ -363,6 +363,8 @@ key_file = "/etc/logwisp/tls/client.key"
- Retries apply to transport errors, `408`, `429`, and `5xx`. Any other - Retries apply to transport errors, `408`, `429`, and `5xx`. Any other
non-2xx response is treated as permanent, and the batch is dropped and counted non-2xx response is treated as permanent, and the batch is dropped and counted
in `dropped_batches`. in `dropped_batches`.
- Redirects are never followed; a `3xx` is permanent too. Following one would
resend the batch wherever the response points, plaintext `http` included.
- HTTP/2 is off by design; batched NDJSON POSTs gain nothing from it. - HTTP/2 is off by design; batched NDJSON POSTs gain nothing from it.
- On shutdown a single best-effort flush of the pending batch is attempted. - On shutdown a single best-effort flush of the pending batch is attempted.
+3 -3
View File
@@ -3,8 +3,8 @@ module logwisp
go 1.27.1 go 1.27.1
require ( require (
github.com/lixenwraith/config v0.1.1-0.20260929051144-5132718bd627 github.com/lixenwraith/config v0.2.1
github.com/lixenwraith/log v0.1.1-0.20260926182514-9ad925351aa1 github.com/lixenwraith/log v0.1.11
) )
require github.com/lixenwraith/toml v0.0.0-20260928180512-b59d068ff6a4 // indirect require github.com/lixenwraith/toml v0.0.0-20261001212158-cede215365c0 // indirect
+6 -6
View File
@@ -1,6 +1,6 @@
github.com/lixenwraith/config v0.1.1-0.20260929051144-5132718bd627 h1:PkSnFbvJWaf7cl4txDwglYFreet0QJypBD2XNOo5cc4= github.com/lixenwraith/config v0.2.1 h1:a1VLWhtAi1F7FWtRtFVpPkvemnBEf+4xrvDupvwHYtg=
github.com/lixenwraith/config v0.1.1-0.20260929051144-5132718bd627/go.mod h1:yfJQwKS+/6Dg70GIoHeYc0eYztOPETpqe6nUIbakcyg= github.com/lixenwraith/config v0.2.1/go.mod h1:yfJQwKS+/6Dg70GIoHeYc0eYztOPETpqe6nUIbakcyg=
github.com/lixenwraith/log v0.1.1-0.20260926182514-9ad925351aa1 h1:fdVY7Jbg+GR61BA6zlvX4N2RFtgzCvSuWeOscicTb8I= github.com/lixenwraith/log v0.1.11 h1:dKWLnaR8dhXxj4l/aU5XZiFIsr8geaJQbOf8m7SiKhU=
github.com/lixenwraith/log v0.1.1-0.20260926182514-9ad925351aa1/go.mod h1:EE8i5nSAW2yZBTytKuhdgehmgzrby6kW1k5DyH1SvBg= github.com/lixenwraith/log v0.1.11/go.mod h1:EE8i5nSAW2yZBTytKuhdgehmgzrby6kW1k5DyH1SvBg=
github.com/lixenwraith/toml v0.0.0-20260928180512-b59d068ff6a4 h1:9qZU0iBIsqintDh39vieio0ygLiRrPmCEo6Ra45EQyY= github.com/lixenwraith/toml v0.0.0-20261001212158-cede215365c0 h1:ue1pILlQAovrxTMSaKII4AbZ7yY4mNs3cyivoSBwMmA=
github.com/lixenwraith/toml v0.0.0-20260928180512-b59d068ff6a4/go.mod h1:itcT/Vt5mwmX9KVGCMsWZr6vbU/JgZ24wV/I0J5cMt0= github.com/lixenwraith/toml v0.0.0-20261001212158-cede215365c0/go.mod h1:itcT/Vt5mwmX9KVGCMsWZr6vbU/JgZ24wV/I0J5cMt0=
+70
View File
@@ -12,11 +12,14 @@ import (
"crypto/tls" "crypto/tls"
"fmt" "fmt"
"regexp" "regexp"
"regexp/syntax"
"strings" "strings"
"sync/atomic" "sync/atomic"
"logwisp/internal/config" "logwisp/internal/config"
"logwisp/internal/tlsx" "logwisp/internal/tlsx"
"github.com/lixenwraith/log"
) )
// Authentication methods // Authentication methods
@@ -276,6 +279,73 @@ func (p *Policy) Unrestricted() bool {
return p != nil && len(p.allow) == 0 && len(p.patterns) == 0 return p != nil && len(p.allow) == 0 && len(p.patterns) == 0
} }
// LogStartup reports, once per plugin construction, what a listener policy
// admits, how it labels nodes, and allow patterns that admit more than they
// appear to. trustNode matters only for chain sources.
func (p *Policy) LogStartup(l *log.Logger, component, id string, trustNode bool) {
if p == nil || p.role == RoleDialer {
return
}
if p.Unrestricted() {
l.Warn("msg", "Auth policy admits any identity the configured CA vouches for",
"component", component,
"instance_id", id,
"hint", "set auth.allow or auth.allow_patterns to authorize named peers")
}
if p.BindsNode() {
l.Info("msg", "Node labels bound to peer identity; trust_node is ignored",
"component", component,
"instance_id", id,
"node_binding", p.binding,
"trust_node", trustNode)
}
for _, re := range p.patterns {
if !anchored(re.String()) {
l.Warn("msg", "auth.allow_patterns entry is not anchored and matches any identity containing it",
"component", component,
"instance_id", id,
"pattern", re.String(),
"hint", "anchor every alternative, e.g. ^(edge-01|edge-02)$")
}
}
}
// anchored reports whether every match of pattern spans the whole identity:
// each alternative must start with ^ and end with $. "^a|b$" is the classic
// miss, admitting "a..." and "...b".
func anchored(pattern string) bool {
re, err := syntax.Parse(pattern, syntax.Perl)
if err != nil {
return false
}
re = re.Simplify()
return anchoredAt(re, syntax.OpBeginText) && anchoredAt(re, syntax.OpEndText)
}
// anchoredAt follows the matching edge of re (first or last element of a
// concatenation, every branch of an alternation) down to the anchor op.
func anchoredAt(re *syntax.Regexp, anchor syntax.Op) bool {
switch re.Op {
case anchor:
return true
case syntax.OpCapture:
return anchoredAt(re.Sub[0], anchor)
case syntax.OpConcat:
if anchor == syntax.OpBeginText {
return anchoredAt(re.Sub[0], anchor)
}
return anchoredAt(re.Sub[len(re.Sub)-1], anchor)
case syntax.OpAlternate:
for _, sub := range re.Sub {
if !anchoredAt(sub, anchor) {
return false
}
}
return true
}
return false
}
// Describe renders the policy for a startup log line // Describe renders the policy for a startup log line
func (p *Policy) Describe() string { func (p *Policy) Describe() string {
if p == nil { if p == nil {
+19
View File
@@ -296,3 +296,22 @@ func TestVerifyConnectionPinsServer(t *testing.T) {
t.Error("unpinned server accepted") t.Error("unpinned server accepted")
} }
} }
// Only patterns whose every alternative is pinned at both ends stay silent at
// startup; "^a|b$" reads as anchored but admits "a..." and "...b".
func TestAllowPatternAnchoring(t *testing.T) {
for pattern, want := range map[string]bool{
`^edge-\d{2}$`: true,
`^(edge-01|edge-02)$`: true,
`^edge-a$|^edge-bc$`: true, // the parser factors this into ^edge-(?:a$|bc$)
`\Aedge\z`: true,
`edge-\d{2}`: false,
`^a|b$`: false,
`^ab$|^ac`: false,
`^a$|`: false,
} {
if got := anchored(pattern); got != want {
t.Errorf("anchored(%q) = %v, want %v", pattern, got, want)
}
}
}
+38 -3
View File
@@ -9,8 +9,6 @@ import (
"time" "time"
"logwisp/internal/testutil" "logwisp/internal/testutil"
lconfig "github.com/lixenwraith/config"
) )
func TestResolveConfigArguments(t *testing.T) { func TestResolveConfigArguments(t *testing.T) {
@@ -185,7 +183,7 @@ allow = ["one", "two"]
t.Fatal(err) t.Fatal(err)
} }
opts := TCPSinkOptions{Host: "127.0.0.1", BufferSize: 1000} opts := TCPSinkOptions{Host: "127.0.0.1", BufferSize: 1000}
if err := lconfig.ScanMap(cfg.Pipelines[0].PluginSinks[0].Config, &opts); err != nil { if err := Scan(cfg.Pipelines[0].PluginSinks[0].Config, &opts); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if opts.Host != "127.0.0.1" || opts.BufferSize != 1000 || opts.Port != 9000 || opts.WriteTimeoutMS != 9007199254740993 || if opts.Host != "127.0.0.1" || opts.BufferSize != 1000 || opts.Port != 9000 || opts.WriteTimeoutMS != 9007199254740993 ||
@@ -193,3 +191,40 @@ allow = ["one", "two"]
t.Fatalf("plugin defaults, nested tables or integer precision lost: %+v", opts) t.Fatalf("plugin defaults, nested tables or integer precision lost: %+v", opts)
} }
} }
// A misspelled key, even inside a nested tls or auth table, fails plugin
// construction instead of leaving the protection it named switched off.
func TestPluginConfigRejectsUnknownKeys(t *testing.T) {
isolateConfig(t)
for key, table := range map[string]string{
"bogus": "bogus = true",
"tls.enabeld": "[pipelines.plugin_sinks.config.tls]\nenabeld = true",
"auth.tpye": "[pipelines.plugin_sinks.config.auth]\ntpye = \"mtls\"",
} {
testutil.WriteFile(t, "plugins.toml", `
[[pipelines]]
name = "typo"
[[pipelines.plugin_sources]]
id = "in"
type = "null"
[[pipelines.plugin_sinks]]
id = "out"
type = "tcp"
[pipelines.plugin_sinks.config]
port = 9000
`+table+"\n")
m, err := Load([]string{"-c", "plugins.toml"})
if err != nil {
t.Fatal(err)
}
cfg, err := m.Snapshot()
m.Close()
if err != nil {
t.Fatal(err)
}
err = Scan(cfg.Pipelines[0].PluginSinks[0].Config, &TCPSinkOptions{})
if err == nil || !strings.Contains(err.Error(), `"`+key+`"`) {
t.Errorf("unknown key %s: err = %v", key, err)
}
}
}
+56
View File
@@ -2,6 +2,10 @@ package config
import ( import (
"fmt" "fmt"
"maps"
"reflect"
"slices"
"strings"
lconfig "github.com/lixenwraith/config" lconfig "github.com/lixenwraith/config"
) )
@@ -81,3 +85,55 @@ func validateLogConfig(cfg *LogConfig) error {
return nil return nil
} }
// Scan decodes a plugin's config map into target, rejecting keys target does
// not declare: a misspelled tls or auth key must fail, not silently disable
// the protection it was meant to configure.
func Scan(configMap map[string]any, target any) error {
if err := unknownKeys(configMap, reflect.TypeOf(target), ""); err != nil {
return err
}
return lconfig.ScanMap(configMap, target)
}
// unknownKeys walks nested tables against the toml tags of t. Map-typed fields
// hold free-form keys and are not descended into.
func unknownKeys(m map[string]any, t reflect.Type, prefix string) error {
for t.Kind() == reflect.Pointer || t.Kind() == reflect.Slice {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return nil
}
fields := make(map[string]reflect.Type, t.NumField())
for f := range t.Fields() {
if name, _, _ := strings.Cut(f.Tag.Get("toml"), ","); name != "" && name != "-" {
fields[name] = f.Type
}
}
for _, key := range slices.Sorted(maps.Keys(m)) {
ft, ok := fields[key]
if !ok {
return fmt.Errorf("unknown key %q", prefix+key)
}
var tables []map[string]any
switch v := m[key].(type) {
case map[string]any:
tables = []map[string]any{v}
case []map[string]any:
tables = v
case []any:
for _, e := range v {
if table, ok := e.(map[string]any); ok {
tables = append(tables, table)
}
}
}
for _, table := range tables {
if err := unknownKeys(table, ft, prefix+key+"."); err != nil {
return err
}
}
}
return nil
}
+1 -1
View File
@@ -65,7 +65,7 @@ func NewConsoleSinkPlugin(
opts := &config.ConsoleSinkOptions{} opts := &config.ConsoleSinkOptions{}
// Scan config map into struct // Scan config map into struct
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
+1 -1
View File
@@ -68,7 +68,7 @@ func NewFileSinkPlugin(
opts := &config.FileSinkOptions{} opts := &config.FileSinkOptions{}
// Scan config map into struct // Scan config map into struct
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
+6 -7
View File
@@ -107,7 +107,7 @@ func NewHTTPSinkPlugin(
Host: DefaultHTTPHost, Host: DefaultHTTPHost,
WriteTimeoutMS: 0, // SSE indefinite streaming WriteTimeoutMS: 0, // SSE indefinite streaming
} }
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
if err := lconfig.Port(opts.Port); err != nil { if err := lconfig.Port(opts.Port); err != nil {
@@ -167,12 +167,8 @@ func NewHTTPSinkPlugin(
"tls", tlsCfg != nil, "tls", tlsCfg != nil,
"mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert, "mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert,
"auth", authPolicy.Describe()) "auth", authPolicy.Describe())
if authPolicy.Unrestricted() { tlsx.LogWarnings(logger, "http_sink", id, opts.TLS, true)
logger.Warn("msg", "Auth policy admits any identity the configured CA vouches for", authPolicy.LogStartup(logger, "http_sink", id, false)
"component", "http_sink",
"instance_id", id,
"hint", "set auth.allow or auth.allow_patterns to authorize named clients")
}
return h, nil return h, nil
} }
@@ -407,7 +403,10 @@ func (h *HTTPSink) handleStream(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream") w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache") w.Header().Set("Cache-Control", "no-cache")
if !h.auth.Enabled() {
// An authenticated stream is not offered to every web origin
w.Header().Set("Access-Control-Allow-Origin", "*") w.Header().Set("Access-Control-Allow-Origin", "*")
}
w.Header().Set("X-Accel-Buffering", "no") w.Header().Set("X-Accel-Buffering", "no")
w.WriteHeader(http.StatusOK) w.WriteHeader(http.StatusOK)
+50
View File
@@ -2,13 +2,18 @@ package http
import ( import (
"bufio" "bufio"
"crypto/tls"
"crypto/x509"
"encoding/json" "encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os"
"strings"
"testing" "testing"
"time" "time"
"logwisp/internal/sink" "logwisp/internal/sink"
"logwisp/internal/testutil"
) )
func TestStatusReportsQueueAndConnectionBounds(t *testing.T) { func TestStatusReportsQueueAndConnectionBounds(t *testing.T) {
@@ -119,3 +124,48 @@ func TestHeadOnStreamPathIsRefused(t *testing.T) {
t.Errorf("sessions after HEAD = %d, want 0", n) t.Errorf("sessions after HEAD = %d, want 0", n)
} }
} }
// A stream gated by an auth policy is not offered to every web origin; an
// open stream keeps the wildcard so browser dashboards still work.
func TestWildcardCORSOnlyWithoutAuth(t *testing.T) {
pki := testutil.NewPKI(t, "viewer-01")
caPEM, err := os.ReadFile(pki.CA)
if err != nil {
t.Fatal(err)
}
roots := x509.NewCertPool()
roots.AppendCertsFromPEM(caPEM)
clientCert, err := tls.LoadX509KeyPair(pki.ClientCert, pki.ClientKey)
if err != nil {
t.Fatal(err)
}
open, _ := newTestHTTPSink(t, nil)
gated, _ := newTestHTTPSink(t, map[string]any{
"tls": map[string]any{"enabled": true, "cert_file": pki.ServerCert, "key_file": pki.ServerKey,
"client_auth": true, "client_ca_file": pki.CA},
"auth": map[string]any{"type": "mtls", "allow": []any{"viewer-01"}},
})
for _, tc := range []struct {
name string
sink *HTTPSink
want string
}{{"open", open, "*"}, {"gated", gated, ""}} {
client, baseURL := serveTestHTTPSink(t, tc.sink)
if tc.sink.tlsConfig != nil {
client.Transport = &http.Transport{TLSClientConfig: &tls.Config{RootCAs: roots, Certificates: []tls.Certificate{clientCert}}}
baseURL = "https" + strings.TrimPrefix(baseURL, "http")
}
resp, err := client.Get(baseURL + "/stream")
if err != nil {
t.Fatalf("%s: %v", tc.name, err)
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("%s: GET /stream = %d", tc.name, resp.StatusCode)
}
if got := resp.Header.Get("Access-Control-Allow-Origin"); got != tc.want {
t.Errorf("%s: Access-Control-Allow-Origin = %q, want %q", tc.name, got, tc.want)
}
}
}
+10 -4
View File
@@ -44,6 +44,7 @@ const (
DefaultHTTPChainSinkRequestTimeoutMS = 10000 DefaultHTTPChainSinkRequestTimeoutMS = 10000
DefaultHTTPChainSinkBackoffMinMS = 500 DefaultHTTPChainSinkBackoffMinMS = 500
DefaultHTTPChainSinkBackoffMaxMS = 30000 DefaultHTTPChainSinkBackoffMaxMS = 30000
maxResponseDrain = 64 * 1024
) )
// HTTPChainSink batches structured entries and posts NDJSON to a downstream // HTTPChainSink batches structured entries and posts NDJSON to a downstream
@@ -92,7 +93,7 @@ func NewHTTPChainSinkPlugin(
proxy *session.Proxy, proxy *session.Proxy,
) (sink.Sink, error) { ) (sink.Sink, error) {
opts := &config.HTTPChainSinkOptions{} opts := &config.HTTPChainSinkOptions{}
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
if err := lconfig.NonEmpty(opts.Host); err != nil { if err := lconfig.NonEmpty(opts.Host); err != nil {
@@ -182,7 +183,11 @@ func NewHTTPChainSinkPlugin(
mtls: tlsCfg != nil && len(tlsCfg.Certificates) > 0, mtls: tlsCfg != nil && len(tlsCfg.Certificates) > 0,
auth: authPolicy, auth: authPolicy,
url: scheme + "://" + addr + opts.IngestPath, url: scheme + "://" + addr + opts.IngestPath,
client: &http.Client{Transport: transport}, // A redirect would resend the batch, even from https to plaintext http
client: &http.Client{
Transport: transport,
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
},
input: make(chan core.TransportEvent, opts.BufferSize), input: make(chan core.TransportEvent, opts.BufferSize),
done: make(chan struct{}), done: make(chan struct{}),
logger: logger, logger: logger,
@@ -208,6 +213,7 @@ func NewHTTPChainSinkPlugin(
"tls", t.tlsEnabled, "tls", t.tlsEnabled,
"mtls", t.mtls, "mtls", t.mtls,
"auth", authPolicy.Describe()) "auth", authPolicy.Describe())
tlsx.LogWarnings(logger, "http_chain_sink", id, opts.TLS, false)
return t, nil return t, nil
} }
@@ -436,8 +442,8 @@ func (t *HTTPChainSink) post(ctx context.Context, body []byte) (transient bool,
return true, err return true, err
} }
defer resp.Body.Close() defer resp.Body.Close()
// Drain for connection reuse // Drain for connection reuse, bounded so a hostile peer cannot stream forever
io.Copy(io.Discard, resp.Body) io.Copy(io.Discard, io.LimitReader(resp.Body, maxResponseDrain))
switch { switch {
case resp.StatusCode >= 200 && resp.StatusCode < 300: case resp.StatusCode >= 200 && resp.StatusCode < 300:
+47
View File
@@ -0,0 +1,47 @@
package httpchain
import (
"net"
"net/http"
"net/http/httptest"
"strconv"
"sync/atomic"
"testing"
"time"
"logwisp/internal/session"
"github.com/lixenwraith/log"
)
// A redirect is never followed: it would resend the batch wherever the
// response points, plaintext http included.
func TestRedirectIsNotFollowed(t *testing.T) {
var leaked atomic.Bool
target := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { leaked.Store(true) }))
t.Cleanup(target.Close)
redirector := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, target.URL+r.URL.Path, http.StatusPermanentRedirect)
}))
t.Cleanup(redirector.Close)
host, port, err := net.SplitHostPort(redirector.Listener.Addr().String())
if err != nil {
t.Fatal(err)
}
p, _ := strconv.ParseInt(port, 10, 64)
manager := session.NewManager(time.Hour)
t.Cleanup(manager.Stop)
created, err := NewHTTPChainSinkPlugin("fwd", map[string]any{"host": host, "port": p},
log.NewLogger(), session.NewProxy(manager, "fwd"))
if err != nil {
t.Fatal(err)
}
transient, err := created.(*HTTPChainSink).post(t.Context(), []byte("{}\n"))
if err == nil || transient {
t.Fatalf("post = (transient %v, %v), want a permanent error", transient, err)
}
if leaked.Load() {
t.Fatal("the batch followed the redirect")
}
}
+4
View File
@@ -6,6 +6,7 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"logwisp/internal/config"
"logwisp/internal/core" "logwisp/internal/core"
"logwisp/internal/plugin" "logwisp/internal/plugin"
"logwisp/internal/session" "logwisp/internal/session"
@@ -49,6 +50,9 @@ func NewNullSinkPlugin(
logger *log.Logger, logger *log.Logger,
proxy *session.Proxy, proxy *session.Proxy,
) (sink.Sink, error) { ) (sink.Sink, error) {
if err := config.Scan(configMap, &config.NullSinkOptions{}); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err)
}
ns := &NullSink{ ns := &NullSink{
id: id, id: id,
proxy: proxy, proxy: proxy,
+25 -18
View File
@@ -58,8 +58,10 @@ type TCPSink struct {
input chan core.TransportEvent input chan core.TransportEvent
logger *log.Logger logger *log.Logger
// Client registry // Client registry. conns holds every accepted connection, so shutdown also
// reaches peers still in the handshake, before they become clients.
clients map[uint64]*tcpClient clients map[uint64]*tcpClient
conns map[net.Conn]struct{}
clientsMu sync.Mutex clientsMu sync.Mutex
nextClientID atomic.Uint64 nextClientID atomic.Uint64
writeTimeout time.Duration writeTimeout time.Duration
@@ -86,11 +88,10 @@ type TCPSink struct {
lastProcessed atomic.Value // time.Time lastProcessed atomic.Value // time.Time
} }
// tcpClient pairs a connection with its bounded send queue. // tcpClient is a registered connection's bounded send queue.
// send is written by the broadcast loop (non-blocking) and drained by the // send is written by the broadcast loop (non-blocking) and drained by the
// writer goroutine; closed signals reader-detected disconnect. // writer goroutine; closed signals reader-detected disconnect.
type tcpClient struct { type tcpClient struct {
conn net.Conn
send chan []byte send chan []byte
sessionID string sessionID string
closed chan struct{} closed chan struct{}
@@ -107,7 +108,7 @@ func NewTCPSinkPlugin(
Host: DefaultTCPHost, Host: DefaultTCPHost,
KeepAlive: true, KeepAlive: true,
} }
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
if err := lconfig.Port(opts.Port); err != nil { if err := lconfig.Port(opts.Port); err != nil {
@@ -143,6 +144,7 @@ func NewTCPSinkPlugin(
done: make(chan struct{}), done: make(chan struct{}),
logger: logger, logger: logger,
clients: make(map[uint64]*tcpClient), clients: make(map[uint64]*tcpClient),
conns: make(map[net.Conn]struct{}),
writeTimeout: time.Duration(opts.WriteTimeoutMS) * time.Millisecond, writeTimeout: time.Duration(opts.WriteTimeoutMS) * time.Millisecond,
tlsConfig: tlsCfg, tlsConfig: tlsCfg,
auth: authPolicy, auth: authPolicy,
@@ -157,12 +159,8 @@ func NewTCPSinkPlugin(
"tls", tlsCfg != nil, "tls", tlsCfg != nil,
"mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert, "mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert,
"auth", authPolicy.Describe()) "auth", authPolicy.Describe())
if authPolicy.Unrestricted() { tlsx.LogWarnings(logger, "tcp_sink", id, opts.TLS, true)
logger.Warn("msg", "Auth policy admits any identity the configured CA vouches for", authPolicy.LogStartup(logger, "tcp_sink", id, false)
"component", "tcp_sink",
"instance_id", id,
"hint", "set auth.allow or auth.allow_patterns to authorize named clients")
}
return t, nil return t, nil
} }
@@ -258,8 +256,8 @@ func (t *TCPSink) shutdown() {
t.listener.Close() // unblocks acceptLoop t.listener.Close() // unblocks acceptLoop
} }
t.clientsMu.Lock() t.clientsMu.Lock()
for _, c := range t.clients { for conn := range t.conns {
c.conn.Close() // unblocks per-connection readers conn.Close() // unblocks handshakes and per-connection readers
} }
t.clientsMu.Unlock() t.clientsMu.Unlock()
}) })
@@ -287,9 +285,17 @@ func (t *TCPSink) acceptLoop() {
continue continue
} }
// Certificate authorization runs in handleConn post-handshake, t.clientsMu.Lock()
// pre-registration. Password-auth extension point: preamble select {
// verification belongs at the same place. case <-t.done:
// shutdown already swept conns; this one would outlive it
t.clientsMu.Unlock()
conn.Close()
return
default:
}
t.conns[conn] = struct{}{}
t.clientsMu.Unlock()
t.wg.Add(1) t.wg.Add(1)
go t.handleConn(conn) go t.handleConn(conn)
@@ -305,6 +311,10 @@ func (t *TCPSink) handleConn(conn net.Conn) {
// Counted from accept: max_connections bounds concurrent handshakes too // Counted from accept: max_connections bounds concurrent handshakes too
count := t.activeConns.Add(1) count := t.activeConns.Add(1)
defer func() { defer func() {
conn.Close()
t.clientsMu.Lock()
delete(t.conns, conn)
t.clientsMu.Unlock()
newCount := t.activeConns.Add(-1) newCount := t.activeConns.Add(-1)
t.logger.Debug("msg", "TCP connection closed", t.logger.Debug("msg", "TCP connection closed",
"component", "tcp_sink", "component", "tcp_sink",
@@ -327,7 +337,6 @@ func (t *TCPSink) handleConn(conn net.Conn) {
"component", "tcp_sink", "component", "tcp_sink",
"remote_addr", remote, "remote_addr", remote,
"error", err) "error", err)
conn.Close()
return return
} }
cs := tc.ConnectionState() cs := tc.ConnectionState()
@@ -348,14 +357,12 @@ func (t *TCPSink) handleConn(conn net.Conn) {
"instance_id", t.id, "instance_id", t.id,
"remote_addr", remote, "remote_addr", remote,
"error", err) "error", err)
conn.Close()
return return
} }
ident.Apply(meta) ident.Apply(meta)
sess := t.proxy.CreateSession(remote, meta) sess := t.proxy.CreateSession(remote, meta)
c := &tcpClient{ c := &tcpClient{
conn: conn,
send: make(chan []byte, t.config.ClientBufferSize), send: make(chan []byte, t.config.ClientBufferSize),
sessionID: sess.ID, sessionID: sess.ID,
closed: make(chan struct{}), closed: make(chan struct{}),
+59
View File
@@ -0,0 +1,59 @@
package tcp
import (
"net"
"strconv"
"testing"
"time"
"logwisp/internal/session"
"logwisp/internal/testutil"
"github.com/lixenwraith/log"
)
// A peer that connects and never completes the TLS handshake does not hold
// Stop for the handshake timeout: shutdown reaches connections before they
// become clients.
func TestStopDoesNotWaitForSilentHandshake(t *testing.T) {
pki := testutil.NewPKI(t, "viewer")
probe, err := net.Listen("tcp4", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
addr := probe.Addr().String()
probe.Close()
_, port, _ := net.SplitHostPort(addr)
p, _ := strconv.ParseInt(port, 10, 64)
manager := session.NewManager(time.Hour)
t.Cleanup(manager.Stop)
created, err := NewTCPSinkPlugin("out", map[string]any{
"host": "127.0.0.1",
"port": p,
"tls": map[string]any{"enabled": true, "cert_file": pki.ServerCert, "key_file": pki.ServerKey},
}, log.NewLogger(), session.NewProxy(manager, "out"))
if err != nil {
t.Fatal(err)
}
sink := created.(*TCPSink)
if err := sink.Start(t.Context()); err != nil {
t.Fatal(err)
}
conn, err := net.Dial("tcp4", addr)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { conn.Close() })
for deadline := time.Now().Add(2 * time.Second); sink.activeConns.Load() == 0; time.Sleep(10 * time.Millisecond) {
if time.Now().After(deadline) {
t.Fatal("connection never reached the handshake")
}
}
start := time.Now()
sink.Stop()
if d := time.Since(start); d > 2*time.Second {
t.Fatalf("Stop took %v with a silent peer in the handshake", d)
}
}
+2 -34
View File
@@ -6,7 +6,6 @@ import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"maps" "maps"
"math/rand/v2"
"net" "net"
"os" "os"
"strconv" "strconv"
@@ -88,7 +87,7 @@ func NewTCPChainSinkPlugin(
opts := &config.TCPChainSinkOptions{ opts := &config.TCPChainSinkOptions{
KeepAlive: true, KeepAlive: true,
} }
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
if err := lconfig.NonEmpty(opts.Host); err != nil { if err := lconfig.NonEmpty(opts.Host); err != nil {
@@ -179,6 +178,7 @@ func NewTCPChainSinkPlugin(
"tls", tlsCfg != nil, "tls", tlsCfg != nil,
"mtls", tlsCfg != nil && len(tlsCfg.Certificates) > 0, "mtls", tlsCfg != nil && len(tlsCfg.Certificates) > 0,
"auth", authPolicy.Describe()) "auth", authPolicy.Describe())
tlsx.LogWarnings(logger, "tcp_chain_sink", id, opts.TLS, false)
return t, nil return t, nil
} }
@@ -299,24 +299,6 @@ func (t *TCPChainSink) runLoop(ctx context.Context) {
} }
} }
// toEntry extracts the structured entry, stamping node identity at first hop
func (t *TCPChainSink) toEntry(event core.TransportEvent) core.LogEntry {
entry := event.Entry
if entry.Time.IsZero() {
// Defensive: event without structured entry, wrap formatted payload
t.synthesized.Add(1)
entry = core.LogEntry{
Time: event.Time,
Source: t.id,
Message: string(event.Payload),
}
}
if entry.Node == "" {
entry.Node = t.node
}
return entry
}
// deliver writes one line, holding it across reconnects until sent or shutdown. // deliver writes one line, holding it across reconnects until sent or shutdown.
// Backpressure during outage propagates to the pipeline dispatch drop counter. // Backpressure during outage propagates to the pipeline dispatch drop counter.
func (t *TCPChainSink) deliver(ctx context.Context, line []byte) bool { func (t *TCPChainSink) deliver(ctx context.Context, line []byte) bool {
@@ -427,17 +409,3 @@ func (t *TCPChainSink) waitBackoff(ctx context.Context, failures int) bool {
return false return false
} }
} }
// backoffDelay computes exponential backoff with ±20% jitter
func (t *TCPChainSink) backoffDelay(failures int) time.Duration {
minD := time.Duration(t.config.BackoffMinMS) * time.Millisecond
maxD := time.Duration(t.config.BackoffMaxMS) * time.Millisecond
d := maxD
if failures < 63 {
if v := minD << uint(failures-1); v > 0 && v < maxD {
d = v
}
}
return d - d/5 + time.Duration(rand.Int64N(int64(2*d/5)+1))
}
+1 -2
View File
@@ -13,7 +13,6 @@ import (
"logwisp/internal/session" "logwisp/internal/session"
"logwisp/internal/source" "logwisp/internal/source"
lconfig "github.com/lixenwraith/config"
"github.com/lixenwraith/log" "github.com/lixenwraith/log"
) )
@@ -70,7 +69,7 @@ func NewConsoleSourcePlugin(
opts := &config.ConsoleSourceOptions{} opts := &config.ConsoleSourceOptions{}
// Scan config map // Scan config map
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
+1 -1
View File
@@ -75,7 +75,7 @@ func NewFileSourcePlugin(
opts := &config.FileSourceOptions{} opts := &config.FileSourceOptions{}
// Use lconfig to scan map into struct (overriding defaults) // Use lconfig to scan map into struct (overriding defaults)
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
+3 -14
View File
@@ -85,7 +85,7 @@ func NewHTTPChainSourcePlugin(
Host: "0.0.0.0", Host: "0.0.0.0",
TrustNode: true, TrustNode: true,
} }
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
if err := lconfig.Port(opts.Port); err != nil { if err := lconfig.Port(opts.Port); err != nil {
@@ -135,19 +135,8 @@ func NewHTTPChainSourcePlugin(
"tls", tlsCfg != nil, "tls", tlsCfg != nil,
"mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert, "mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert,
"auth", authPolicy.Describe()) "auth", authPolicy.Describe())
if authPolicy.Unrestricted() { tlsx.LogWarnings(logger, "http_chain_source", id, opts.TLS, true)
logger.Warn("msg", "Auth policy admits any identity the configured CA vouches for", authPolicy.LogStartup(logger, "http_chain_source", id, opts.TrustNode)
"component", "http_chain_source",
"instance_id", id,
"hint", "set auth.allow or auth.allow_patterns to authorize named peers")
}
if authPolicy.BindsNode() {
logger.Info("msg", "Node labels bound to peer identity; trust_node is ignored",
"component", "http_chain_source",
"instance_id", id,
"node_binding", authPolicy.NodeBinding(),
"trust_node", opts.TrustNode)
}
return s, nil return s, nil
} }
+4
View File
@@ -5,6 +5,7 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"logwisp/internal/config"
"logwisp/internal/core" "logwisp/internal/core"
"logwisp/internal/plugin" "logwisp/internal/plugin"
"logwisp/internal/session" "logwisp/internal/session"
@@ -47,6 +48,9 @@ func NewNullSourcePlugin(
logger *log.Logger, logger *log.Logger,
proxy *session.Proxy, proxy *session.Proxy,
) (source.Source, error) { ) (source.Source, error) {
if err := config.Scan(configMap, &config.NullSourceOptions{}); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err)
}
ns := &NullSource{ ns := &NullSource{
id: id, id: id,
proxy: proxy, proxy: proxy,
+1 -1
View File
@@ -76,7 +76,7 @@ func NewRandomSourcePlugin(
} }
// Scan config map // Scan config map
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
+3 -14
View File
@@ -80,7 +80,7 @@ func NewTCPChainSourcePlugin(
Host: "0.0.0.0", Host: "0.0.0.0",
TrustNode: true, TrustNode: true,
} }
if err := lconfig.ScanMap(configMap, opts); err != nil { if err := config.Scan(configMap, opts); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err) return nil, fmt.Errorf("failed to parse config: %w", err)
} }
if err := lconfig.Port(opts.Port); err != nil { if err := lconfig.Port(opts.Port); err != nil {
@@ -121,19 +121,8 @@ func NewTCPChainSourcePlugin(
"tls", tlsCfg != nil, "tls", tlsCfg != nil,
"mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert, "mtls", tlsCfg != nil && tlsCfg.ClientAuth == tls.RequireAndVerifyClientCert,
"auth", authPolicy.Describe()) "auth", authPolicy.Describe())
if authPolicy.Unrestricted() { tlsx.LogWarnings(logger, "tcp_chain_source", id, opts.TLS, true)
logger.Warn("msg", "Auth policy admits any identity the configured CA vouches for", authPolicy.LogStartup(logger, "tcp_chain_source", id, opts.TrustNode)
"component", "tcp_chain_source",
"instance_id", id,
"hint", "set auth.allow or auth.allow_patterns to authorize named peers")
}
if authPolicy.BindsNode() {
logger.Info("msg", "Node labels bound to peer identity; trust_node is ignored",
"component", "tcp_chain_source",
"instance_id", id,
"node_binding", authPolicy.NodeBinding(),
"trust_node", opts.TrustNode)
}
return s, nil return s, nil
} }
+71
View File
@@ -3,9 +3,19 @@
package testutil package testutil
import ( import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"net"
"os" "os"
"path/filepath"
"strings" "strings"
"testing" "testing"
"time"
) )
// ClearEnvPrefix isolates overrides while preserving the caller's environment. // ClearEnvPrefix isolates overrides while preserving the caller's environment.
@@ -29,3 +39,64 @@ func WriteFile(t testing.TB, path, contents string) {
t.Fatalf("write fixture %q: %v", path, err) t.Fatalf("write fixture %q: %v", path, err)
} }
} }
// PKI is a throwaway CA with a server certificate for 127.0.0.1 and one client
// certificate, written as PEM files for plugins that load TLS from disk.
type PKI struct {
CA, ServerCert, ServerKey, ClientCert, ClientKey string
}
// NewPKI issues the PKI into a test directory; clientCN names the client.
func NewPKI(t testing.TB, clientCN string) PKI {
t.Helper()
dir := t.TempDir()
now := time.Now()
caKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
caTmpl := &x509.Certificate{
SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "LogWisp Test CA"},
NotBefore: now.Add(-time.Hour), NotAfter: now.Add(24 * time.Hour),
IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign,
}
caDER, err := x509.CreateCertificate(rand.Reader, caTmpl, caTmpl, &caKey.PublicKey, caKey)
if err != nil {
t.Fatal(err)
}
ca, err := x509.ParseCertificate(caDER)
if err != nil {
t.Fatal(err)
}
pki := PKI{CA: writePEM(t, dir, "ca.crt", "CERTIFICATE", caDER)}
issue := func(name, cn string, serial int64, usage x509.ExtKeyUsage, ips []net.IP) (string, string) {
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
tmpl := &x509.Certificate{
SerialNumber: big.NewInt(serial), Subject: pkix.Name{CommonName: cn},
NotBefore: now.Add(-time.Hour), NotAfter: now.Add(24 * time.Hour),
KeyUsage: x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{usage}, IPAddresses: ips,
}
der, err := x509.CreateCertificate(rand.Reader, tmpl, ca, &key.PublicKey, caKey)
if err != nil {
t.Fatal(err)
}
keyDER, err := x509.MarshalPKCS8PrivateKey(key)
if err != nil {
t.Fatal(err)
}
return writePEM(t, dir, name+".crt", "CERTIFICATE", der), writePEM(t, dir, name+".key", "PRIVATE KEY", keyDER)
}
pki.ServerCert, pki.ServerKey = issue("server", "relay.internal", 2, x509.ExtKeyUsageServerAuth, []net.IP{net.IPv4(127, 0, 0, 1)})
pki.ClientCert, pki.ClientKey = issue("client", clientCN, 3, x509.ExtKeyUsageClientAuth, nil)
return pki
}
func writePEM(t testing.TB, dir, name, blockType string, der []byte) string {
t.Helper()
path := filepath.Join(dir, name)
WriteFile(t, path, string(pem.EncodeToMemory(&pem.Block{Type: blockType, Bytes: der})))
return path
}
+95 -2
View File
@@ -6,10 +6,12 @@ package tlsx
import ( import (
"crypto/tls" "crypto/tls"
"crypto/x509" "crypto/x509"
"encoding/pem"
"fmt" "fmt"
stdlog "log" stdlog "log"
"os" "os"
"strings" "strings"
"sync"
"time" "time"
"logwisp/internal/config" "logwisp/internal/config"
@@ -17,8 +19,12 @@ import (
"github.com/lixenwraith/log" "github.com/lixenwraith/log"
) )
// HandshakeTimeout bounds TLS handshakes on both accept and dial paths const (
const HandshakeTimeout = 10 * time.Second // HandshakeTimeout bounds TLS handshakes on both accept and dial paths
HandshakeTimeout = 10 * time.Second
// expiryWindow is how far ahead startup announces certificate expiry
expiryWindow = 30 * 24 * time.Hour
)
// Server builds the *tls.Config for listener plugins // Server builds the *tls.Config for listener plugins
// (tcp/http sinks, tcp_chain/http_chain sources). Returns (nil, nil) when disabled. // (tcp/http sinks, tcp_chain/http_chain sources). Returns (nil, nil) when disabled.
@@ -136,6 +142,93 @@ func PeerIdentity(cs tls.ConnectionState, mode string) string {
return "" return ""
} }
// LogWarnings reports TLS settings that work but are likely mistakes, once per
// plugin construction, so each reload re-announces expiring certificates.
func LogWarnings(l *log.Logger, component, id string, o *config.TLSOptions, listener bool) {
if o == nil || !o.Enabled {
return
}
var warnings []string
if !listener && o.InsecureSkipVerify {
warnings = append(warnings, "tls.insecure_skip_verify disables server verification: any peer can impersonate the server")
}
files := []struct{ key, path string }{{"cert_file", o.CertFile}, {"ca_file", o.CAFile}}
if listener {
files[1] = struct{ key, path string }{"client_ca_file", o.ClientCAFile}
}
now := time.Now()
for _, f := range files {
for _, c := range readCerts(f.path) {
if w := expiryWarning(c, now); w != "" {
warnings = append(warnings, fmt.Sprintf("tls.%s %s: %s", f.key, f.path, w))
}
}
}
if w := SecretFileWarning("tls.key_file", o.KeyFile); w != "" {
warnings = append(warnings, w)
}
for _, w := range warnings {
l.Warn("msg", w, "component", component, "instance_id", id)
}
}
// expiryWarning describes a certificate outside, or about to leave, its
// validity window; a handshake failure is otherwise the first sign.
func expiryWarning(c *x509.Certificate, now time.Time) string {
name := c.Subject.CommonName
switch {
case now.After(c.NotAfter):
return fmt.Sprintf("certificate %q expired on %s", name, c.NotAfter.UTC().Format(time.DateOnly))
case now.Before(c.NotBefore):
return fmt.Sprintf("certificate %q is not valid until %s", name, c.NotBefore.UTC().Format(time.DateTime))
case c.NotAfter.Sub(now) < expiryWindow:
return fmt.Sprintf("certificate %q expires on %s", name, c.NotAfter.UTC().Format(time.DateOnly))
}
return ""
}
// readCerts parses every certificate in a PEM file. Constructors already
// loaded it successfully, so an unreadable file yields nothing here.
func readCerts(path string) []*x509.Certificate {
if path == "" {
return nil
}
data, err := os.ReadFile(path)
if err != nil {
return nil
}
var certs []*x509.Certificate
for block, rest := pem.Decode(data); block != nil; block, rest = pem.Decode(rest) {
if block.Type != "CERTIFICATE" {
continue
}
if c, err := x509.ParseCertificate(block.Bytes); err == nil {
certs = append(certs, c)
}
}
return certs
}
// warnedSecrets keeps reloads from repeating a permission warning per path
var warnedSecrets sync.Map
// SecretFileWarning describes a secret file every local user can read, once
// per path per process. Group access is left alone: container secret mounts
// commonly grant it on purpose.
func SecretFileWarning(key, path string) string {
if path == "" {
return ""
}
fi, err := os.Stat(path)
if err != nil || fi.Mode().Perm()&0o004 == 0 {
return ""
}
if _, seen := warnedSecrets.LoadOrStore(path, struct{}{}); seen {
return ""
}
return fmt.Sprintf("%s %s is readable by all users (mode %04o)", key, path, fi.Mode().Perm())
}
// HTTPErrorLog adapts the structured logger for http.Server.ErrorLog so TLS // HTTPErrorLog adapts the structured logger for http.Server.ErrorLog so TLS
// handshake failures don't bypass log routing straight to stderr (which would // handshake failures don't bypass log routing straight to stderr (which would
// violate the console sanitization policy). // violate the console sanitization policy).
+34
View File
@@ -0,0 +1,34 @@
package tlsx
import (
"crypto/x509"
"crypto/x509/pkix"
"strings"
"testing"
"time"
)
// Startup names a certificate outside its validity window or within 30 days
// of leaving it; a healthy one stays silent.
func TestExpiryWarning(t *testing.T) {
now := time.Date(2026, 10, 1, 12, 0, 0, 0, time.UTC)
day := 24 * time.Hour
cert := func(from, until time.Duration) *x509.Certificate {
return &x509.Certificate{Subject: pkix.Name{CommonName: "relay"}, NotBefore: now.Add(from), NotAfter: now.Add(until)}
}
for _, tc := range []struct {
name string
cert *x509.Certificate
want string
}{
{"valid", cert(-day, 90*day), ""},
{"expiring", cert(-day, 10*day), "expires on 2026-10-11"},
{"expired", cert(-90*day, -day), "expired on 2026-09-30"},
{"not yet valid", cert(day, 90*day), "not valid until"},
} {
got := expiryWarning(tc.cert, now)
if tc.want == "" && got != "" || !strings.Contains(got, tc.want) {
t.Errorf("%s: warning = %q, want %q", tc.name, got, tc.want)
}
}
}