v0.18.2 resume rotated files instead of replay, keep quiet SSE alive, refuse HEAD on strem path
This commit is contained in:
+12
-5
@@ -148,7 +148,7 @@ This endpoint is scoped to one sink, not to the whole process, and it is
|
||||
| `dropped_entries` | source | Downstream cannot keep up with the source |
|
||||
| `total_dropped` | flow | Rate limit or filters are discarding entries (often intended) |
|
||||
| `total_dropped_by_sink` | pipeline | A sink's input queue is full |
|
||||
| `dropped_writes` | tcp/http sink | A specific client is too slow |
|
||||
| `dropped_writes` | tcp/http sink | A client's queue overflowed: either it is too slow, or one burst exceeded `client_buffer_size` |
|
||||
| `rejected_conns` / `rejected_clients` | tcp/http sink, tcp_chain source | `max_connections` is being hit |
|
||||
| `tls_handshake_errors` | tcp sink, tcp_chain source | Certificate or version mismatch, or scanning |
|
||||
| `parse_errors` | chain source | Protocol or version skew upstream |
|
||||
@@ -185,11 +185,18 @@ the filter stage logs several lines per entry evaluated.
|
||||
### Buffers
|
||||
|
||||
Raise `buffer_size` when `total_dropped_by_sink` is climbing but the sink itself
|
||||
is healthy — that is a burst-absorption problem. Raise `client_buffer_size` when
|
||||
`dropped_writes` is climbing for network sinks; that is a slow-consumer or
|
||||
startup-replay burst problem, and a bigger buffer only buys time. The HTTP
|
||||
is healthy — that is a burst-absorption problem.
|
||||
|
||||
`dropped_writes` on a network sink has two causes that a counter alone does not
|
||||
separate. A consumer slower than the sustained rate cannot be bought off with
|
||||
buffer, and drops are the intended outcome. A burst the consumer would have
|
||||
drained, arriving faster than it reads, is configuration: the sink queues a
|
||||
whole burst while the client writes one frame at a time, so the part of a burst
|
||||
above `client_buffer_size` is lost even to a loopback reader.
|
||||
Where a `rate_limit` bounds the pipeline, its `burst` is that number — keep
|
||||
`client_buffer_size` at or above it and the second cause disappears. The HTTP
|
||||
status endpoint reports both queue bounds alongside the counters so an operator
|
||||
can distinguish configuration from demand.
|
||||
can tell which one is in play.
|
||||
|
||||
```toml
|
||||
[pipelines.plugin_sinks.config]
|
||||
|
||||
+15
-4
@@ -140,7 +140,9 @@ allow = ["viewer-01"]
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- Only `GET` is routed to either path; anything else gets `405`.
|
||||
- Only `GET` is routed to either path; anything else gets `405`, `HEAD` on
|
||||
`stream_path` included — a stream is a body, and a client registered to have
|
||||
its body discarded never reads and never leaves.
|
||||
- With an `auth` block, one middleware gates **both** endpoints: an
|
||||
unauthorized client gets `403` with no body detail, and the rejection is
|
||||
logged at WARN and counted in `auth_rejected`. The authorized identity is
|
||||
@@ -150,11 +152,20 @@ allow = ["viewer-01"]
|
||||
- Payloads are framed per the SSE spec, one `data:` line per newline in the
|
||||
payload, so multi-line entries stream correctly.
|
||||
- The server sets no `WriteTimeout` (that would kill long-lived streams);
|
||||
per-write deadlines come from `write_timeout_ms` via `http.ResponseController`.
|
||||
per-write deadlines come from `write_timeout_ms` via `http.ResponseController`
|
||||
and cover the connected frame, every payload, and the idle comment.
|
||||
- A quiet stream emits an SSE comment every 15 s. It refreshes the client's
|
||||
session and is how a peer that stopped reading is noticed.
|
||||
- A client whose send queue is full has that event dropped
|
||||
(`dropped_writes`); it is not disconnected.
|
||||
(`dropped_writes`); it is not disconnected. A `dropped_writes` that rises while
|
||||
no client is behind is a burst larger than `client_buffer_size`, not
|
||||
backpressure: size the queue at or above whatever burst the pipeline's
|
||||
`rate_limit` releases at once.
|
||||
- A client is registered only once its connected frame has flushed, so the
|
||||
broker never queues into a buffer whose reader has not started.
|
||||
- Clients whose session has been idle-expired by the session manager are
|
||||
evicted by the broker.
|
||||
evicted by the broker. With the idle comment above, that reaches only a peer
|
||||
that has stopped accepting bytes on a sink configured `write_timeout_ms = 0`.
|
||||
- On shutdown, connected clients receive
|
||||
`event: disconnect / data: {"reason":"server_shutdown"}`.
|
||||
- HTTP/2 is negotiated via ALPN when TLS is enabled; plaintext is HTTP/1.1.
|
||||
|
||||
@@ -60,6 +60,10 @@ from = "end"
|
||||
beyond end-of-file, or an inode change. An inode change where the new file is
|
||||
already larger than the recorded position is treated as an atomic save, not a
|
||||
rotation, and the position is preserved.
|
||||
- A rotation that renames in place — what a size-capped writer does — puts the
|
||||
same inode back under a name `pattern` also matches. Its watcher resumes at
|
||||
the position the original reached, so `from = "start"` reads the tail an
|
||||
unfinished read left behind rather than the whole archive a second time.
|
||||
- A line is parsed as JSON only when it is an object whose top-level keys are
|
||||
all drawn from `time`, `level`, `msg` and `fields` — the four an entry can
|
||||
carry. `time` is read as RFC3339Nano. Any other key, and any non-object line,
|
||||
|
||||
Reference in New Issue
Block a user