v0.26.1 per-client connection caps

doc/README.md

@@ -30,8 +30,8 @@ streams, or downstream LogWisp nodes.
 - [CLI](cli.md): flags, signals, exit codes, `lw auth`; also the `lw(1)`
   manual, [`lw.1`](lw.1)
 - [Operations](operations.md): running, monitoring, tuning, troubleshooting
-- [To Do](todo.md): planned work in priority order: per-client limits,
-  packaging, smaller follow-ups
+- [To Do](todo.md): planned work in priority order: packaging, smaller
+  follow-ups
 
 A fully annotated configuration covering every option lives at
 [`config/logwisp.toml`](../config/logwisp.toml).

doc/architecture.md

@@ -42,6 +42,8 @@ Package map:
 - `internal/tlsx`: the single seam between `TLSOptions` and `crypto/tls`.
 - `internal/authz`: the single seam between `AuthOptions` and the network
   plugins: `Admit`, `AuthorizeRequest`, `Greet`, `Prepare`.
+- `internal/netacl`: a listener's `acl` block: address rules, PROXY headers
+  and per-client limits; its `Table` is SCRAM throttling's limiter too.
 - `internal/session`: session manager and per-instance proxy.
 - `internal/core`: shared types (`LogEntry`, `TransportEvent`), capabilities,
   constants.

doc/deployment.md

@@ -184,7 +184,7 @@ every key.
 - filters, rate limits, heartbeat, formatter flags and timestamp layouts
 - more than one pipeline; both chain transports, or a further hop, on one node
 - proxy mode: browser logins through a TLS-terminating proxy
-- `acl` on the listeners: address rules, PROXY headers
+- `acl` on the listeners: address rules, PROXY headers, per-client limits
 - mtls identities other than the certificate CN; `node_binding` other than
   `force`
 - creating certificates

doc/installation.md

@@ -314,5 +314,6 @@ or SKIP (exit 77, e.g. no IPv6 loopback); only a failure fails the target.
 - `acl-test.sh`: each listener admitting or refusing peers by address before
   TLS, with clients from 127.0.0.1 and 127.0.0.2, then by the client a PROXY
   header names behind a stub proxy on 127.0.0.11-13 (and nginx `stream` on
-  127.0.0.14 when installed), per-client login throttling included; ports
+  127.0.0.14 when installed) or an L7 proxy's `X-Forwarded-For`, with
+  per-client connection caps, request rates and login throttling; ports
   15881-15889; skips (exit 77) where 127.0.0.2 cannot be bound

doc/mtls-auth-plan.md

@@ -305,9 +305,10 @@ The remaining phase-4 items, in rough order of value:
    operators with existing CRL infrastructure. The allow-list covers the same
    ground with fewer moving parts, so this is only worth doing for a fleet whose
    revocation already flows through a CRL.
-2. **Per-identity rate limits.** The natural follow-on now that identity exists,
-   and the natural home for the per-IP limiting that was also removed. Kept out
-   of scope here so this feature stayed reviewable.
+2. **Per-identity rate limits.** The natural follow-on now that identity exists:
+   the `acl` block's per-client limits count addresses
+   ([Security](security.md#per-client-limits)), and would be keyed by identity
+   once a peer is authenticated.
 3. **Per-client identity in the `http` sink's status output.** The endpoint
    reports the policy and counters, but not which identities are currently
    connected; session metadata has the data.

doc/networking.md

@@ -116,11 +116,10 @@ sink, and the `tcp_chain` source. `0` means unlimited.
 The `http_chain` source has no connection cap; it bounds work with
 `max_body_bytes` and `read_timeout_ms` instead.
 
-Apart from SCRAM logins, which are throttled per address (see
-[Security](security.md#throttling)), there is **no** per-IP limiting; `acl`
-admits or refuses peers by address ([Security](security.md#the-acl-block)).
-`flow.rate_limit` is a pipeline-wide entry rate limit, not a network-level
-one — it cannot distinguish or throttle an individual peer.
+Per client, the `acl` block caps connections on every listener and requests
+on the HTTP ones ([Security](security.md#per-client-limits)), and SCRAM logins
+are throttled ([Security](security.md#throttling)). `flow.rate_limit` is a
+pipeline-wide entry rate limit: it cannot distinguish or throttle a peer.
 
 ## Keep-Alive
 
@@ -208,13 +207,16 @@ headers, and entry encoding.
 - Check the port is not already bound by another pipeline in the same process.
 
 **Closed as soon as it connects, before TLS**
-- The listener's `acl` refused it: `acl_denied` rises, and its log has
-  `Connection refused by acl` at WARN with a `reason`, once a minute at most.
-  The dialer sees `EOF` or `connection reset by peer` (curl: exit 52 or 56).
+- The listener's `acl` refused it: `acl_denied` or `acl_limited` rises, and
+  its log has `Connection refused by acl` at WARN with a `reason`, once a
+  minute at most. The dialer sees `EOF` or `connection reset by peer` (curl:
+  exit 52 or 56).
   - `address rules` — `allow` or `deny` excludes the client: the one a PROXY
     header names (`remote_addr`, the proxy as `peer_addr`), else the socket
     peer. Behind the `http` sink's `auth.trusted_proxies` that is the proxy:
     allow it.
+  - `max_connections_per_client` — the client holds that many already, or the
+    listener tracks 65,536 clients; HTTP keep-alive connections count too.
   - `no PROXY header` — a peer in `proxy_from` sent none under
     `proxy_protocol = "required"`: the proxy lacks `proxy_protocol on` (nginx)
     or `send-proxy` (HAProxy), or the proxy's host dialed directly.
@@ -223,7 +225,14 @@ headers, and entry encoding.
     the connection opened: list the proxy in `proxy_from`.
 - `acl: allow entry "...": not of the listener's family` at startup — an IPv4
   entry on an IPv6 `host`, or the reverse: each family's listener takes its own.
-  Behind `proxy_from`, `allow` and `deny` take both; `proxy_from` its own.
+  Behind `proxy_from` or `trusted_proxies`, `allow` and `deny` take both;
+  `proxy_from` its own.
+
+**HTTP `429 Too Many Requests` with `Retry-After`**
+- `requests_per_second_per_client`: `acl_limited` rises and the log has
+  `Request refused by acl`, once a minute at most. A browser's page load
+  takes several requests; raise the rate rather than retry faster. From
+  `/auth` with `too many attempts`, it is SCRAM throttling (below).
 
 **TLS handshake failure**
 - `client didn't provide a certificate` — the listener has `client_auth = true`
@@ -277,7 +286,8 @@ headers, and entry encoding.
 - `403` from an `http` sink in proxy mode, with `not a trusted proxy` or
   `X-Forwarded-Proto` in its WARN line — the request did not come from a listed
   proxy, or the proxy did not forward `X-Forwarded-Proto: https` and
-  `X-Forwarded-For`.
+  `X-Forwarded-For`. With `Request refused by acl` (once a minute at most) or
+  none, the `acl` rules refuse the forwarded client (`acl_denied`).
 - `too many attempts` — the address failed or abandoned logins faster than one
   per second beyond a burst of 10, or has 4 unfinished; it clears within seconds
   once the failing peer stops. Peers behind one NAT, or one passthrough proxy

doc/scram-auth-plan.md

@@ -174,8 +174,9 @@ one per /64), on handshake starts: a token bucket (burst 10, 1/s) refunded on
 success, and at most 4 unfinished exchanges. An unanswered HTTP challenge holds
 its slot for the `auth` handshake timeout (30 s); abandoned TCP exchanges
 release their slot, and their entry in the `auth` handshake table,
-immediately. The address table holds 65,536 entries, drops one idle for a
-minute once its challenges have expired, and fails closed when full; the SCRAM
+immediately. The address table, `netacl.Table` as for the `acl` block's
+per-client limits, holds 65,536 entries, drops one once its bucket has
+refilled and its challenges have expired, and fails closed when full; the SCRAM
 server itself caps in-flight handshakes at 4,096 (`busy`).
 
 Counters: `auth_allowed` counts logins; `auth_rejected` every refusal — failed
@@ -332,8 +333,8 @@ indistinguishable from an unknown user, another certificate refused and counted,
 hello and policy disagreeing either way, a dialer cut short by its context, an
 abandoned exchange freeing its handshake and limiter slots, a refusal after the
 exchange sent as a reason rather than a final, certificate-to-user binding for
-logins and tokens); only failed attempts throttled, the limiter bounds
-(reservation under concurrency, the sweep of abandoned challenges); pre-auth
+logins and tokens); only failed attempts throttled (the limiter's bounds are
+`netacl`'s, under the per-client limits); pre-auth
 input capped at 4 KiB; the decoy salt stable across restarts; a dialer refusing
 a challenge below its Argon2 floor; the HTTP exchange on a TLS test server
 (token accepted, garbage and foreign tokens `401`, forged final yields no token,

doc/security.md

@@ -26,9 +26,10 @@ configure it, and — equally important — what it does not yet do.
   [Browsers behind a TLS-terminating proxy](#browsers-behind-a-tls-terminating-proxy).
 - Server pinning by dialers: certificate identity (`mtls`), bound certificate
   (`scram`).
-- Address rules on every listener, applied before TLS, to the client a PROXY
-  header (v1 or v2) from a listed L4 proxy names: see
-  [The ACL Block](#the-acl-block).
+- Address rules and per-client connection and request limits on every
+  listener, applied before TLS to the client a PROXY header (v1 or v2) from a
+  listed L4 proxy names, and per request to the client an L7 proxy forwards:
+  see [The ACL Block](#the-acl-block).
 - Startup warnings for expiring certificates and risky settings: see
   [Startup Warnings](#startup-warnings).
 - Unknown configuration keys rejected: a typo in `tls`, `auth`, `acl` or a
@@ -52,10 +53,11 @@ configure it, and — equally important — what it does not yet do.
 
 - Certificate revocation lists (CRL) or OCSP: revoke by editing the allow-list
   or credentials file.
-- Per-address connection or request limits: only SCRAM logins are throttled
-  per address.
-- Address rules for `X-Forwarded-For` clients: in the `http` sink's proxy mode
-  `acl` sees the L7 proxy, never the client it forwards.
+- Per-identity limits: the per-client limits count addresses, not the
+  identities `auth` proves.
+- Per-client stream caps: under HTTP/2 `max_connections_per_client` bounds a
+  client's connections, not the streams each carries; the request rate only
+  slows how fast it opens them.
 
 Two credentials are supported. Certificates (`mtls`) are the one the transport
 already carries: the `tls` block establishes that a peer chains to your CA, and
@@ -334,26 +336,54 @@ allow = ["192.0.2.0/24", "198.51.100.7"] # addresses or CIDRs; empty: all
 deny  = ["192.0.2.66"]                   # refused, listed in allow or not
 proxy_protocol = "required"              # off (default), optional, required
 proxy_from = ["10.0.0.5"]                # L4 proxies sending PROXY headers
+max_connections_per_client = 4           # at once; 0 (default): no cap
+requests_per_second_per_client = 10      # HTTP listeners; 0 (default): none
 ```
 
 - `deny` wins; then a non-empty `allow` admits only its entries, an empty one
   every address `deny` does not list.
 - Entries are of the listener's [family](networking.md#address-family): IPv4
   on an IPv4 listener, IPv6 on an IPv6 one, either behind a hostname (only the
-  family it binds ever matches). Behind `proxy_from` they take either, as a
-  header may name a client of either. An entry of the other family, an
+  family it binds ever matches). Behind `proxy_from` or the `http` sink's
+  `trusted_proxies` they take either, as the proxy may name a client of
+  either. An entry of the other family, an
   IPv4-mapped address, one with a zone, or anything but an address or CIDR
   fails construction. A link-local peer matches without its zone.
-- A refused connection is closed at once and counted in `acl_denied` (plugin
-  stats, the `http` sink's status). A WARN names the peer, the reason and the
-  count, at most once a minute per listener; the refusals in between are only
-  counted.
+- A refused connection is closed at once and counted in `acl_denied`, or
+  `acl_limited` past a [limit](#per-client-limits) (plugin stats, the `http`
+  sink's status). A WARN names the client, the reason and the counts, at most
+  once a minute per listener; the refusals in between are only counted.
 - The rules see the client: the socket peer, or the client a PROXY header
-  names. Behind the `http` sink's `trusted_proxies` they see the L7 proxy.
+  names. Behind the `http` sink's `trusted_proxies` they judge the L7 proxy
+  as the peer, then each request's forwarded client (`403` when refused,
+  counted in `acl_denied`): allow both.
 - The `serve` and `aggregator` presets take `allow` and `deny` list keys.
 - Dialers have no `acl`: on a `tcp_chain` or `http_chain` sink it is an
   unknown key.
 
+### Per-client limits
+
+- `max_connections_per_client` caps the connections a client holds at once,
+  on every listener; past it a connection is closed like a refused one.
+  - HTTP keep-alive connections count, idle ones too, and a browser opens up
+    to six; under HTTP/2 one connection carries a client's many requests.
+  - Behind `trusted_proxies` every connection is the proxy's, so the key
+    fails construction: cap clients at the proxy.
+- `requests_per_second_per_client`, on the `http` sink and the `http_chain`
+  source, admits a second's worth of requests at once (at least one),
+  refilled at that rate; past it a request gets `429`, with `Retry-After` the
+  seconds until its next (1 at a rate of 1 or more).
+  - Every request counts but the SCRAM exchange (`POST /auth`), which
+    [throttling](#throttling) limits: stream, status, ingest and the viewer's
+    files, about six for a viewer's first load.
+- A client is the address the rules see, an IPv6 one by its /64 and a
+  link-local one per link, so the hosts of one /64 (a SLAAC LAN) share a
+  budget; behind `trusted_proxies` the request rate counts the forwarded
+  client.
+- Each limit tracks up to 65,536 clients, a client until its budget is whole
+  again, and refuses new ones when full, failing closed. SCRAM throttling
+  keeps its budgets the same way.
+
 ### PROXY protocol
 
 Behind an L4 proxy that passes TLS through (nginx `stream`, HAProxy
@@ -906,7 +936,8 @@ sink's stream and status endpoints stop being open to anyone who can reach the
 port.
 
 **`acl`** — an address check, per listener, before TLS: it narrows who can
-reach the other layers, never proves who a peer is.
+reach the other layers and how much each address may take, never proves who a
+peer is.
 
 **Revocation** is the allow-list or the credentials file, not a CRL. Remove the
 identity or user and send `SIGHUP`: the reload rebuilds every pipeline, so the
@@ -938,8 +969,9 @@ What each exposes when `auth.type = "none"`:
   (with `client_auth`, any the CA vouches for), under any node label it
   claims.
 
-`max_connections` bounds concurrency on all of them but does not distinguish
-callers.
+`max_connections` bounds concurrency on all but the `http_chain` source; the
+`acl` block's [per-client limits](#per-client-limits) bound each caller's
+share.
 
 Both methods require TLS on the listener, except an `http` sink in proxy mode,
 where TLS ends at the trusted proxy; `mtls` also requires `client_auth`.

doc/sinks.md

@@ -179,8 +179,9 @@ Options, each as type and default:
 - `auth` (table): client authentication (`mtls` or `scram`); see
   [Security](security.md#the-auth-block).
 - `acl` (table): `allow` and `deny`, addresses or CIDRs admitted and refused
-  before TLS, and `proxy_protocol` with `proxy_from` for the client a PROXY
-  header names; see [Security](security.md#the-acl-block).
+  before TLS, `proxy_protocol` with `proxy_from` for the client a PROXY header
+  names, and the per-client `max_connections_per_client` and
+  `requests_per_second_per_client`; see [Security](security.md#the-acl-block).
 
 **Behaviour**
 
@@ -247,13 +248,13 @@ Options, each as type and default:
 the compiled auth policy and acl, active client count, sink and per-client
 buffer sizes, connection limit, write timeout, uptime, endpoint paths, and the
 `total_processed` / `dropped_writes` / `rejected_clients` / `auth_rejected` /
-`acl_denied` counters.
+`acl_denied` / `acl_limited` counters.
 
 **Statistics**: `dropped_writes`, `rejected_clients`, `auth`, `auth_allowed`,
 `auth_rejected`; under `scram` also `auth_users`, `auth_throttled`,
 `auth_busy`, `auth_binding_mismatch` and `auth_token_lifetime_ms`, and in
-proxy mode `auth_trusted_proxies`; with `acl` rules or `proxy_from` also
-`acl`, `acl_denied` and `acl_proxy_headers`.
+proxy mode `auth_trusted_proxies`; with a non-empty `acl` block also `acl`,
+`acl_denied`, `acl_limited` and `acl_proxy_headers`.
 
 > Without an `auth` block both endpoints are unauthenticated, and the stream
 > response carries `Access-Control-Allow-Origin: *`, so any web origin can read
@@ -307,8 +308,9 @@ Options, each as type and default:
 - `auth` (table): client authentication (`mtls` or `scram`); see
   [Security](security.md#the-auth-block).
 - `acl` (table): `allow` and `deny`, addresses or CIDRs admitted and refused
-  before TLS, and `proxy_protocol` with `proxy_from` for the client a PROXY
-  header names; see [Security](security.md#the-acl-block).
+  before TLS, `proxy_protocol` with `proxy_from` for the client a PROXY header
+  names, and `max_connections_per_client`; see
+  [Security](security.md#the-acl-block).
 
 **Behaviour**
 
@@ -335,7 +337,7 @@ Options, each as type and default:
 **Statistics**: `write_errors`, `dropped_writes`, `rejected_conns`,
 `tls_handshake_errors`, `auth`, `auth_allowed`, `auth_rejected`; under `scram`
 also `auth_users`, `auth_throttled`, `auth_busy` and `auth_binding_mismatch`;
-with `acl` rules or `proxy_from` also `acl`, `acl_denied` and
+with a non-empty `acl` block also `acl`, `acl_denied`, `acl_limited` and
 `acl_proxy_headers`.
 
 ---

doc/sources.md

@@ -215,8 +215,9 @@ Options, each as type and default:
 - `auth` (table): peer authentication (`mtls` or `scram`) and node binding;
   see [Security](security.md#the-auth-block).
 - `acl` (table): `allow` and `deny`, addresses or CIDRs admitted and refused
-  before TLS, and `proxy_protocol` with `proxy_from` for the client a PROXY
-  header names; see [Security](security.md#the-acl-block).
+  before TLS, `proxy_protocol` with `proxy_from` for the client a PROXY header
+  names, and `max_connections_per_client`; see
+  [Security](security.md#the-acl-block).
 
 For passwords instead of certificates, the `auth` block takes `type = "scram"`
 and a `credentials_file`, and `client_auth` becomes optional; see
@@ -251,8 +252,8 @@ and a `credentials_file`, and `client_auth` becomes optional; see
 **Statistics**: `active_connections`, `rejected_conns`, `parse_errors`,
 `tls_handshake_errors`, `trust_node`, `auth`, `auth_allowed`, `auth_rejected`,
 `node_binding`; under `scram` also `auth_users`, `auth_throttled`, `auth_busy`
-and `auth_binding_mismatch`; with `acl` rules or `proxy_from` also `acl`,
-`acl_denied` and `acl_proxy_headers`.
+and `auth_binding_mismatch`; with a non-empty `acl` block also `acl`,
+`acl_denied`, `acl_limited` and `acl_proxy_headers`.
 
 ---
 
@@ -304,8 +305,9 @@ Options, each as type and default:
 - `auth` (table): peer authentication (`mtls` or `scram`) and node binding;
   see [Security](security.md#the-auth-block).
 - `acl` (table): `allow` and `deny`, addresses or CIDRs admitted and refused
-  before TLS, and `proxy_protocol` with `proxy_from` for the client a PROXY
-  header names; see [Security](security.md#the-acl-block).
+  before TLS, `proxy_protocol` with `proxy_from` for the client a PROXY header
+  names, and the per-client `max_connections_per_client` and
+  `requests_per_second_per_client`; see [Security](security.md#the-acl-block).
 
 **Behaviour**
 
@@ -332,8 +334,8 @@ Options, each as type and default:
 **Statistics**: `total_requests`, `rejected_requests`, `parse_errors`,
 `cached_sessions`, `trust_node`, `auth`, `auth_allowed`, `auth_rejected`,
 `node_binding`; under `scram` also `auth_users`, `auth_throttled`, `auth_busy`,
-`auth_binding_mismatch` and `auth_token_lifetime_ms`; with `acl` rules or
-`proxy_from` also `acl`, `acl_denied` and `acl_proxy_headers`.
+`auth_binding_mismatch` and `auth_token_lifetime_ms`; with a non-empty `acl`
+block also `acl`, `acl_denied`, `acl_limited` and `acl_proxy_headers`.
 
 ---
 

doc/todo.md

@@ -4,45 +4,7 @@ Planned work, in order of priority. Each item names what to build, how it fits
 the existing seams, and how to verify it. Finished items move to the design doc
 they belong to.
 
-## 1. Network access control (ACL)
-
-The seam exists: `internal/netacl` wraps each listener's socket before TLS,
-reads the PROXY header of the L4 proxies `proxy_from` lists, and the `acl`
-block's `allow` and `deny` refuse the client by address
-([Security](security.md#the-acl-block)). Every limit except SCRAM throttling
-is still global.
-
-### 1.1 Per-client limits
-
-- Generalize the SCRAM limiter in `internal/authz/scram.go` (token bucket per
-  key, bounded table that fails closed, idle sweep, IPv6 per /64 via
-  `throttleKey`) into the shared per-key limiter. One mechanism (AGENTS.md):
-  SCRAM throttling then uses it too.
-- Keys in the `acl` block:
-  - `max_connections_per_client` on every listener (concurrent connections per
-    client), applied by the netacl listener after the rules.
-  - `requests_per_second_per_client` on HTTP listeners (ingest, stream and
-    status requests).
-- A full table refuses new clients and counts them in `acl_limited`, beside
-  `acl_denied` and `acl_proxy_headers`.
-- Later: per-identity limits once a peer is authenticated (the mtls plan's
-  deferred item 2), keyed by identity instead of address.
-
-### 1.2 Address rules for forwarded clients
-
-- In HTTP proxy mode, `X-Forwarded-For` is applied by `authz.ClientAddr` after
-  the connection-level decision, so rules for forwarded HTTP clients belong in
-  a second, request-level check there, reusing the compiled `netacl` rules.
-  `authz.parseProxies` then takes netacl's address-or-CIDR parsing too.
-
-### 1.3 Verification
-
-- Go tests, one per rule: the per-client connection cap; the request rate; the
-  limiter table failing closed; a forwarded client refused by the rules.
-- `test/acl-test.sh` adds per-client connection caps, behind its stub proxy
-  too.
-
-## 2. Packaging: AUR, FreeBSD ports, Debian
+## 1. Packaging: AUR, FreeBSD ports, Debian
 
 The foundation exists:
 - The `lw` binary name, free in Arch (official repositories and AUR), Ubuntu
@@ -99,9 +61,9 @@ identity, and both skeletons download the tagged source.
 6. A packaging CI job: build the AUR package in an Arch container and the port
    in a FreeBSD VM, run `lw --version` and `make image-check`.
 
-## 3. Follow-ups
+## 2. Follow-ups
 
-Smaller items found along the way; each is independent of the ACL.
+Smaller items found along the way, each independent of the others.
 
 1. A late browser sees no backlog: the `http` sink keeps none, so the viewer
    shows entries from when it connects. A bounded replay (a `replay_lines`