| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Clean up and speed up pub/sub logic (#990) * some cleanup * nits * nits * wip * updates * updates * more cleaning * clean clean clean * nits * improvements * updates * update * fix * remove bytearraykey * naming * updates to log based publish * updates * use ConcurrentList to make pub sub allocation free * fix format * updates * add logging * fix tsavorite tests * fix test * format * updates * updates * fixes * cleanup | 1 年前 | |
Enhance file logging to include exception details (#1155) Added a conditional check in `FileLoggerProvider.cs` to append the string representation of an `exception` to the log message if it is not null. This improvement provides better context for debugging by ensuring that exceptions encountered during logging are captured in the output. Co-authored-by: Tal Zaccai <talzacc@microsoft.com> | 1 年前 | |
Bound per-connection memory: adaptive network buffer budget, session buffer caps, and connection admission (#2157) * Bound connection-scaled network buffer memory Per-connection network buffers were unbounded in practice: the receive buffer only shrank when it exceeded maxReceiveBufferSize, so a buffer that doubled 128K -> 256K -> 512K never shrank and every connection permanently retained its all-time peak. The TLS transport buffer had no shrink path at all. GarnetServerTcp also discarded its networkBufferSize argument and hardcoded 128 KB, so the per-connection floor was unconfigurable. - Add --network-buffer-size, --network-max-receive-buffer-size and --network-buffer-pool-size, plumbed through GarnetServerOptions into GarnetServerTcp, which now honors them. Defaults match today's behavior. - Shrink a grown receive buffer to the smallest size class that keeps 2x headroom over the buffered bytes, after 16 consecutive receives that fit. Buffers above maxReceiveBufferSize skip the hysteresis since the pool cannot recycle them. - Add the same shrink path for the TLS transport receive buffer, called only from the reader sites that already consider doubling. - Track liveBytes, peakLiveBytes, pooledBytes and a shared byte budget in LimitedFixedBufferPool, exposed through INFO BPSTATS. - Return the outstanding responseObject in DisposeNetworkSender so a session torn down mid-response does not leak its PoolEntry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Document new network buffer options and add config parsing tests Adds the three --network-buffer-* options to the configuration table, warns instead of silently correcting an inverted size pair, and covers defaults, overrides, power-of-2 rounding, the inverted pair and invalid values in GarnetServerConfigTests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bound per-session parse-state and scratch buffers A RESP session sizes SessionParseState.rootBuffer to the largest array arity the client has ever sent and keeps it for the life of the connection. One high-arity command therefore enlarges the session permanently, and the cost scales with connection count. The same ratchet exists in ScratchBufferBuilder, whose Reset() zeroes the offset but never releases the backing array. Add BufferShrinkPolicy, a shared cap-plus-hysteresis helper, and apply it at both sites. A buffer above the cap is released only after the configured number of consecutive batches that did not need the extra capacity, so a session with recurring large requests keeps its buffer. Both caps are configurable and default-on: --session-parse-state-max-retained-args (default 1024) --session-scratch-buffer-max-retained-size (default 64k) Measured end to end with a forced compacting GC, 200 sessions each issuing one 20,000-argument EXISTS followed by sustained single-key work: 327 KB/session retained before, 98 KB/session after. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover buffer shrink against transaction and watch aliasing The batch-boundary shrink releases a pinned array, so anything holding a reference into it across that boundary would dangle rather than merely read stale bytes. Two paths cross the boundary by design: a MULTI..EXEC window, which re-parses queued commands from the network buffer, and WATCH, which copies key bytes into the transaction scratch allocator so they outlive the receive buffer. Add tests that force the shrink to fire inside both windows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized response elements instead of killing the session A response element larger than the send buffer cannot be written atomically. The chunking helper for this already exists and SCAN uses it, but several commands still wrote unbounded user data through the atomic path, where TryWriteBulkString cannot make progress and SendAndReset throws, killing the session mid-response. KEYS fails today with any key larger than the 128 KB default send buffer. Give the shared WriteBulkString helper an atomic-first fallback: write in place, retry once on a flushed buffer, and only chunk an element that cannot fit an empty buffer. Small elements keep exactly today's path, so multi-element replies pay nothing. Route KEYS, COMMAND GETKEYS, COMMAND GETKEYSANDFLAGS and the pub/sub channel and pattern echoes through it. Two related defects found while covering this: ClusterSession.SendAndReset silently did nothing on an empty buffer, so a caller's retry loop spun forever rather than failing. It now throws, as the RESP session already does. MainStoreOps.GETDEL asserted its output was inline immediately after the RMW, before the spill-to-pooled-memory case is handled. Any value large enough to spill tripped it, so GETDEL of a large value failed on every debug build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Enter the chunking fallback when the send buffer is empty The fallback added for over-sized response elements flushed before retrying, but SendAndReset throws when there is nothing to send. That is exactly the over-sized case where the element is the first thing written to the response, so the chunking path was unreachable there and the session still died. Every earlier test emitted an array header first, which left the buffer non-empty and masked it. Flush only when output is actually buffered, otherwise chunk directly. CLIENT GETNAME is the reachable instance: the name is the whole response, so a name larger than the send buffer killed the session. Give the ASCII bulk-string helper the same fallback and route CLIENT GETNAME through it. Move both fallbacks to NoInlining helpers so the 49 call sites of the inlined fast path keep their previous code size. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate latency histograms on first record Every session was given twelve HdrHistograms when its connection was accepted, before it had recorded anything. At two significant digits each rents a 32 KB array from the shared pool, so a connection cost 384 KB of latency monitoring whether or not it ever issued a command. A dump of a server holding several thousand connections showed 887 MB in these arrays. Allocate the histograms for a latency type on the first value recorded into it. A session typically records into three of the six types, and an idle connection records into none, so it now pays nothing. Add --latency-monitor-precision to size the histograms. It defaults to 2, which is what the histograms were built with, so accuracy is unchanged unless it is lowered. Dropping it to 1 shrinks each histogram several-fold in exchange for reporting percentiles to 10% rather than 1% resolution. Fix three defects on the release path that lazy allocation widens the window for. LongHistogram.Return() now drops its reference to the array it hands back, so a later record throws rather than silently writing into an array the pool has re-rented to another session. The session's pending work is cancelled before its histograms are released rather than after. GarnetLatencyMetricsSession publishes its metrics as unavailable before releasing them, and the hot path reads the field once and checks it. Dropping the array reference exposed a use-after-return in GarnetClient, which copied its histogram after Dispose had already handed the array back; it now reports no metrics instead of reading a recycled array. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the latency precision to the histograms it sizes The precision knob was only covered by tests that constructed histograms directly, so a broken binding between the command line and the allocation would have gone unnoticed and silently degraded resolution. Assert the size the server reports through LATENCY HISTOGRAM, the size of the per-session histograms reached through the live sessions, and that the option parses to 2 by default and rejects values outside the 0-5 range HdrHistogram accepts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Wait for the session to record before reading its histograms A session records its latency after the reply has been written, so a test that read the histograms as soon as the reply arrived could find none allocated yet. Poll until one appears. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Publish a process-wide budget for live network buffers The buffer pool's byte ceiling bounds only its idle free list. Buffers checked out by live connections are bounded by nothing, so their footprint is connections times per-connection size, which is what grew unchecked with thousands of clients. Add NetworkBufferBudget, which divides a configured total by the live buffer count and publishes the largest base size that keeps the two consistent. It is a division rather than a control loop: buffer sizes move in factor-of-two steps, so any controller with a narrower deadband cannot converge and limit-cycles between two size classes. Growth therefore requires the quotient to reach twice the published target, matching the step it actuates. One budget is shared by every listener. Each endpoint previously built its own pool and was handed the whole ceiling, so with N endpoints the effective limit was N times the configured value. Only server listener pools participate; replication and migration keep their own unbounded behaviour so that a sync burst cannot drive live client connections down. The target is published but consumed by no allocation site yet. Landing the signal on its own makes a drift in the live buffer count observable in isolation, which matters because the target is budget divided by that count: a count that ratchets up does not degrade gracefully, it collapses the target to the floor permanently. The count rides exactly the two paths that already maintain the pool's live byte accounting and nothing else. Defaults are inert: 1 GB against a 128 KB buffer size means adaptation does not begin until 8,192 live buffers, and the target clamps at the configured size so it can only ever lower it. --network-buffer-memory-budget 0 disables it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new receive buffers from the network buffer budget The budget was published but consumed by nothing. Wire it to the receive side: a new connection's buffer starts at the published target rather than the configured size, and a grown buffer shrinks back toward that target. Only the base size is governed. Demand-driven doubling is untouched, so a connection that needs a large buffer still reaches it; pressure changes what a connection starts and settles at, never what it is allowed to grow to. The allocation size and the shrink floor move together. Clamping only the allocation would leave a connection allocated below the configured size unable to shrink at all, because the shrink path both floors at that size and returns early when the buffer is already at or under it. A pool cannot recycle a buffer outside its size classes, so NetworkBufferSettings gains an explicit minimum. Deriving it from the three configured sizes, as before, gives no way to express "recycle down to 16 KB while new connections still start at 128 KB", and a clamped buffer would have been dropped rather than pooled. The per-level idle entry count derived from a byte budget is capped in the same change, since a smaller minimum size class otherwise raises it proportionally; the cap sits above the value today's defaults produce, so it is inert until the two are combined. Budget accounting is now inert when disabled rather than merely unread. The disabled budget is a shared singleton, so unbudgeted pools were accumulating a count on it that was meaningless but would have become misleading the moment anything read it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new send buffers from the network buffer budget GarnetSaeaBuffer is the single allocation point behind both GarnetTcpNetworkSender call sites, so clamping there covers the send side entirely. Send buffers never grow, so an oversized response is chunked through whatever buffer it was given, and every consumer reads the length off the pool entry rather than off the configured setting -- which is what makes the size safe to adapt. The clamp uses the separate, higher send floor so that a shrunken send buffer does not push custom raw string responses into repeated ArrayPool rentals. Adds an end-to-end test that a 1 MB value and a key larger than the send buffer both survive while the budget is pinned at its floor, and tightens the shrink assertion now that both directions adapt. Neutering the clamp fails that assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized cluster response elements The chunking audit converted the RESP session but left the cluster session on the atomic path, where an element larger than the response buffer cannot be written and SendAndReset has nothing to flush. CLUSTER GETKEYSINSLOT echoes client-supplied keys, so it kills the session for any key larger than the send buffer -- reproducible today at the 128 KB default, independent of the adaptive budget. The gossip reply echoes the serialized cluster config and grows with node count. Give the cluster session the same atomic-first helper the RESP session has: write in place, retry once on a flushed buffer, and chunk only an element that cannot fit an empty buffer, so small elements keep exactly today's path. Route both sites through it. Covered by a new fixture run at both the default send buffer and one floored by the budget; reverting either arm's fix fails it. Also extends the oversized-response fixture to run at a floored send buffer, with a guard asserting each arm actually binds the size it claims to, and corrects a stale comment about the per-level entry cap being inert -- the 16 KB adaptive floor makes it bind at the defaults. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Shrink receive buffers immediately while the budget is binding The idle hysteresis exists so a connection with bursty-but-recurring large payloads keeps its buffer through the burst, and it is raised from 16 to 256 receives to make that hold over a genuinely long quiet stretch. But a countdown that long cannot converge the aggregate when the process is actually short of memory, so shrinking now bypasses it entirely while the budget is binding. The pressure signal is the already-published target having been driven below the configured size, not a live-byte read: it costs one predictable branch on the receive path and is ordered after the read-mostly size comparisons, so an unpressured server evaluates nothing new. Wires the pressure and idle shrink counters that INFO BPSTATS already reports. The new test drives a burst and then ten small rounds -- far short of the 256-receive hysteresis -- and requires the budgeted arm to give the buffers back while the unbudgeted control still holds them. Neutering the pressure signal fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound pressure shrinking so alternating payloads do not churn Shrinking immediately under pressure was wrong. Pressure is sticky -- the published target stays below the ceiling for as long as the connections are live -- so an alternating large/small workload reallocated a pinned buffer on every large request. Measured at 640 shrinks over 640 large/small pairs, i.e. one per request, precisely when the server is most loaded. Apply a short countdown under pressure instead of none. A strictly alternating workload never trips it, because each large receive resets the countdown, while sustained small traffic still converges in eight receives rather than two hundred and fifty-six. The countdown is clamped rather than reloaded so that pressure arriving mid-countdown does not wait out whatever remained of the idle one. Same measurement now reports zero shrinks. Adds a TLS arm, since the incident is thousands of TLS connections and the decrypted transport buffer shrinks through a separate path that the plain-socket test cannot reach. Disabling the transport clamp leaves the grown buffers at their burst size and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Converge live connections onto the adapted buffer size Clamping the allocation size governs new buffers only. A connection established before the budget started binding keeps its full-size buffers: the send side hands them straight back out from its own stack, and the receive side moves them to the pool free list, where they stay pinned. Neither converges, so at incident scale the adaptation applies only to churn. Gate both return paths on the budget actually binding, using the same IsUnderPressure signal the shrink policy shipped with: - LimitedFixedBufferPool.Return drops an over-target entry instead of pooling it. Unpressured it is pooled exactly as before, so short-lived connections handling moderately large payloads keep their reuse. Send and receive adapt to separate floors, so the comparison reads the entry's own buffer type off PoolEntry.source: measuring against the receive target alone would treat a correctly sized send buffer as over-sized and make send buffers un-poolable for as long as the budget binds. Accounting is unaffected -- the drop reuses the existing non-pooled fall-through, below the unconditional live-byte and live-count release. - GarnetTcpNetworkSender.ReturnBuffer disposes an over-target send buffer rather than pushing it back onto the per-connection stack, so the next acquire allocates at the adapted size. Also clear the vacated slot in LightConcurrentStack.TryPop. Without it the array keeps the popped buffer, and through it the pinned pool entry, strongly referenced for the connection's lifetime -- so releasing a buffer freed the accounting but not the memory. Measured over ten connections established while the budget was slack, then driven into pressure by two hundred more: 2.0 MB released with both gates against 1.0 MB with the send gate neutered. Pooled bytes after the same burst: 0 against 10.5 MB with the pool gate neutered, and the free list after connection churn holds 80 KB against 32 KB when the send target is ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address adversarial review findings on the network buffer budget Five findings from two independent reviews, each reproduced before being fixed. Recompute the adapted target inside the live buffer count's own increment and decrement. It previously had a single production caller on the pool's allocate-miss path, which a pooled reuse never reaches, so the target never recovered when connections drained: after a spike, four live buffers still published a 16 KB target and reconnecting did not clear it. Clamp the TLS plaintext send buffer to the send target. It was allocated at the configured size while the pool measured it against the target on return, so under pressure every TLS connection's buffer was dropped and freshly pinned on the next connect. 64 connections now hold 11.5 MB where they held 15.7 MB. Sample the receive shrink signal before the buffer is processed rather than after. Consumption and compaction happen inside Process on the plain path, so a fully consumed large request was read as a zero-byte receive and advanced the shrink countdown: 160 shrinks over 640 alternating large/small pairs, now zero. Route 26 further response sites through the chunking helper. Set operation results and every blocking variant wrote unbounded elements atomically, which kills the session above the send buffer size at the shipped default; BLPOP loses the popped element outright. The new tests fail in both fixture arms, so this is not budget-induced. Drop the aborted-session response release added earlier in this branch. The batch's finally already returns the object on every teardown path, so 200 protocol-aborted sessions conserve the buffer count with and without it, and handing a possibly in-use buffer back to the pool is not worth an unmeasurable fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix stale transport alias, above-max release ordering, and scratch demand tracking Round 2 adversarial review found five defects, two of which were introduced by the round 1 fixes. All five reproduced before being fixed, and every new assertion is mutation-checked. Without TLS the transport receive buffer is an alias of the network buffer, refreshed only at the top of each receive. A resize left the alias pointing at the entry just returned to the pool, so a connection that grew once and then idled kept the old pinned array rooted indefinitely -- the accounting showed the release and the memory never happened. Measured at 64 connections the stale roots exceed 16 MB. Both resize paths now refresh the alias. The above-max immediate release sat behind the demand guard, so a fully consumed oversized request made target equal current and held the out-of-pool buffer until another receive arrived. This regressed C4 via the round 1 demand-capture fix. The branch now runs first and is sized from the residual rather than the pass's demand. CLUSTER SLOTS still wrote the whole serialized topology atomically, so a sufficiently fragmented cluster overran the send buffer with no oversized key involved. The new test fails at today's 128 KB default too, which is what proves it pre-existing. ScratchBufferBuilder only raised its demand high-water when a write did not fit, so a batch that kept filling an already-large buffer reported no usage at all and the shrink policy released it every time. The popped-entry collectability test could pass for the wrong reason under optimization, since the stack itself was collectable once unused. GC.KeepAlive pins it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reset the shrink countdown on the above-max early return The above-max branch returns without shrinking when the buffer is still full of unconsumed data. Leaving the countdown untouched there is the same guard-ordering class as the defect the branch reorder just fixed, so restore the reset the demand guard used to perform. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Redesign session buffer caps as countdown checkpoints The committed high-water design tracked per-command demand on the hot path and cost ~9-11 ns on Network.BasicOperations.InlinePing at batchSize=1. Replace it at both sites with a countdown checkpoint: every batch decrements a counter, and only when it reaches zero does a cold NoInlining method run. Demand is inferred from whether the buffer grew since the last checkpoint rather than tracked per command, so the hot path holds one decrement and one predictable branch and never touches the parse state. This trades bounded churn for zero hot-path tracking: reclaim takes up to two checkpoint intervals, and a session that needs the capacity every batch reallocates at most once per two intervals. Removes BufferShrinkPolicy, SessionParseState.batchHighWater and ScratchBufferBuilder.AdvanceOffset as dead code. Also re-applies the SLOWLOG chunking fix (review round 2 finding 1) for sps.GetString and entry.ClientName, tightens the parse-state retention assertion from a bare Assert.Less that passed with the shrink neutered to a magnitude bound that fails when mutated, and tightens the scratch-buffer equivalence assertion from a 2x slack bound to unbounded + 64 KB. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the unpressured common case against an unbudgeted server The central promise of the budget is that with few connections nothing changes. Two gaps in the coverage left that unpinned. UnpressuredGrownBuffersAreRetainedExactlyAsWithNoBudget runs the same grow- then-small-traffic workload with the default budget and with the budget disabled, and asserts the two hold identical bytes both immediately after growth and after the small traffic that follows. Ungating the pressure clamp in MaybeShrinkNetworkReceiveBuffer fails its retention assertion. UnpressuredConnectionChurnKeepsRecyclingGrownBuffers guards correction C3: dropping an over-target entry on return must be gated on pressure, or connection churn re-allocates pinned arrays that are recyclable today. Ungating the drop in LimitedFixedBufferPool.Return collapses pooledBytes to 128 KB and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the scratch shrink checkpoint from the batch boundary only The countdown interval is counted in batches, but it was decremented from ScratchBufferBuilder.Reset(), which several callers invoke far more often than once per batch. The Lua interpreter resets once per string while decoding a JSON document and once per response through ScratchBufferNetworkSender, so a single cjson.decode of a wide object fired many checkpoints inside one command -- releasing a buffer the very next element re-grows, which is the churn the interval exists to bound. Split the countdown into ResetAtBatchBoundary(), called only from RespServerSession's batch boundary. Plain Reset() is now free again on the per-element paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the Lua script processor's scratch checkpoint from the session The Lua interpreter writes into the scratch buffer of the script cache's inner RespServerSession, not the network session's. That inner session never reads from a socket, so after the batch-boundary change its builder was never checkpointed at all and one cjson.encode of a large value permanently enlarged every session that ran a script. Fold the Lua checkpoint into the existing cold session checkpoint, which already runs once per 64 outer batches, so this costs nothing on the hot path and is definitionally outside any script execution. Rename the countdown trio accordingly, since it is no longer parse-state-only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk the remaining unbounded response sites and cap the Lua reply buffer A full sweep of the 70 atomic write loops in libs/server/Resp and libs/cluster found six writing unbounded data: ACL LIST, ACL USERS and ACL GETUSER echo client-supplied usernames and rule descriptions, and CLUSTER SLAVES, CLUSTER REPLICAS and CLUSTER INFO write topology that grows with the cluster. All six kill the session at today's 128 KB default, which the new tests prove by failing in both fixture arms. The remaining sites write literals, numbers or fixed-width identifiers. The script cache's dummy network sender owns a second scratch buffer that the RESP fallback path for redis.call writes replies into, built uncapped and never checkpointed. It now takes the configured session cap and is checkpointed alongside the request buffer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Clear retained references on pop and on the cluster parse-state copy Three round-3 review findings, all in the same aliasing class the earlier rounds kept surfacing: a reference survives the release that was supposed to free it. SimpleStack.Pop left the vacated slot populated. Its only user stacks ScratchBufferAllocator's outgrown pinned buffers there, so every buffer a session ever outgrew stayed rooted for the life of the connection and a reset freed the accounting but not the memory. The clear is gated on IsReferenceOrContainsReferences so a struct-only instantiation is unaffected. ClusterSession took a copy of the caller's SessionParseState and left it populated after the command finished. The copy holds its own reference to the session's pinned root buffer, so a wide cluster command kept that buffer alive past the owning session's shrink. NetworkBufferBudget.Recompute validated the previous target rather than the count it derived from, so a lost exchange could publish a target derived from a stale count. The retry loop re-derives on a lost exchange and re-reads the count after a won one. This is defensive: the race was not reproduced in 20 runs of a 256-thread oversubscribed churn, and the test that ships pins the invariant rather than discriminating the fix. LatencyHistogramMemoryTests' idle arm was vacuous. GarnetServerTcp needs four bytes before it creates a RespServerSession, so an idle socket never created one and the arm measured a server with no sessions at all -- it would have passed with eager allocation restored. Sessions are now opened with a partial command and the count asserted against INFO CLIENTS, and both assertions are restated as comparisons between the arms rather than against an absolute. Restoring eager allocation now fails three tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make three assertions test what they claim, and correct three doc statements The send-buffer pooling test accepted any 64 KB on the free list, which four pooled 16 KB receive buffers also satisfy -- so it could pass with no send buffer pooled at all, which is the defect it exists to catch. It now reads the per-level entry counts the pool already reports and asserts an entry at the send size class specifically. Ignoring the entry's buffer type in the drop rule leaves it at zero entries and fails. The cluster chunking tests stopped reading at a byte count, which samples only the part of the reply the first chunk boundary has passed and cannot see a lost tail -- the half a chunking fix is most likely to get wrong. Both now pipeline a PING and read to it: RESP replies are ordered, so the sentinel cannot arrive before the whole reply is written. GETKEYSINSLOT additionally asserts the key contents rather than its length header, and CLUSTER SLOTS asserts every one of the 2,000 ranges is present. Three statements were false. GetSessionScratchBufferMaxRetainedSize says unset disables shrinking; unset returns the 64 KB default and only zero disables. MaxEntriesPerLevelCap says it is inert at the shipped defaults; the budget is on by default, so the smallest size class is the 16 KB receive floor and the derived value is 4,096, which the cap binds. defaults.conf still described the high-water design the countdown checkpoint replaced. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the reachable cluster chunking site and count sessions, not sockets Review round 4 found the cluster chunking fixture green with all three of the round-3 cluster conversions reverted, and both latency histogram tests green with the session-opening helper made a no-op. The three round-3 cluster conversions are bounded and unreachable: CLUSTER INFO is a fixed eleven-line string, CLUSTER REPLICAS elements are node lines carrying no slots, and CLUSTER BANLIST entries are a node id plus a number. The reachable caller of the same helper is CLUSTER NODES, whose node line embeds slot ranges, so a fragmented assignment overruns the send buffer at the 128 KB default with no oversized key anywhere. Covered with 4,000 alternating slots, asserting the declared bulk length against the received payload and every range. connected_clients counts accepted network handlers, not sessions, so the latency guard could not distinguish a measured session from a bare socket. CLIENT LIST enumerates the sessions themselves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Stop the Lua reply cap being undone, and re-derive an abandoned budget target Three defects on the same theme: a bound that does not bind. The Lua reply sender re-requests a response window at the full server buffer size on every use, so capping its builder below one window could not save anything -- the checkpoint released the buffer and the next redis.call immediately reallocated it larger than it started, churning a pinned array every checkpoint. The cap is now floored at one window, so it releases only the oversized replies it was added for. The existing test stopped at the shrink and could not see this. ScratchBufferBuilder rounded one byte past the request, so a request for exactly a size class allocated twice it -- and the Lua reply window, at exactly 128 KiB, is one. Every caller that must strictly grow already asks for one byte more than the current length, so exact rounding still grows them. NetworkBufferBudget.Recompute abandoned its target when its attempts ran out, leaving whatever the last contended exchange wrote, derived from a count that had since moved. Since nothing else recomputes, that value stood until the next acquire or release, and at a band boundary it could report no pressure while the budget was binding. It now re-derives from the freshest count. Contention cannot reach that path reliably, so the attempt limit is settable to force it. The ACL chunking tests stopped reading as soon as the oversized username arrived and never checked the tail; they now parse the whole array and require the pipelined PING reply to begin exactly where it ends. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the recompute attempt limit immutable Every connection reads the budget concurrently, so the attempt limit belongs in the constructor rather than in a settable field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Compare the CLUSTER NODES slot set exactly Substring matching is near-vacuous over 4,000 alternating slots: " 2" matches inside " 20" and " 2000", so almost any surviving fragment of the line satisfies it. Dropping every slot below 1,000 from the node line leaves the old assertion green and fails the new one at 3,500 slots against 4,000. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report a malformed reply as an assertion, not a timeout The hand-rolled RESP parsers in the chunking and latency tests assumed well-formed input. A truncated array, an error reply where a bulk string was expected, or a null bulk string surfaced as a fifteen-second receive timeout or an ArgumentOutOfRangeException rather than as a named assertion about the reply. Both parsers now check every bound before slicing, reject a reply whose type byte is not the expected one, and handle a null bulk string by recording it and advancing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Yield to a fresher publisher instead of overwriting it The exhaustion fallback published the target with an unconditional write, so a thread that sampled the live count before another thread published could land afterwards and replace the right answer for a fresher count with its own stale one. The remark argued the race in the wrong direction: it reasoned about being overtaken, not about overtaking. The publication is now a compare-exchange against the target that was observed alongside the count, and it does not retry. The only writer that can beat it also exchanges, so a lost exchange means a value derived from a count no older than this one is already published. The publish step takes the count and the observed target as parameters so the interleaving can be induced directly rather than raced into. Two assertions were weaker than the fixes they named: the size class test exercised the arithmetic helper but never an allocation, and the Lua reply test showed the cap survived another use without pinning it at one response window. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the cluster chunking arm small enough to chunk The CLUSTER NODES body is about 19 KB, which is below both the 128 KB default send buffer and the 64 KB the floored fixture arm actually resolved to for a single connection. Replacing the chunked write with a spinning atomic one left both arms green, so neither covered the path the test is named for. The floored arm's budget drops to 16k so its send target reaches the floor. The mutation now fails there. The default arm cannot be made to cover chunking here at all -- a single node's line is bounded by the 16,384 slots it can hold, which is well under 128 KB -- so the comment no longer claims it does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the target before the count, not after The compare-exchange was doing less than its remark claimed. It proves the target has not moved since this thread read it, which two threads that sampled the same target concurrently both satisfy regardless of whose count is fresher. Reading the target first and the count second makes the stronger statement true: a successful exchange proves no other publisher completed between the target read and the write, and the count was sampled inside that window, so it is no older than the count behind any value published earlier. The ordering is not observable from a single-threaded test, so it ships on the argument rather than on an assertion, as the other reasoning-level fixes in this work do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Abandon a contended recompute instead of publishing from a stale sample The single-attempt fallback had no correct form. An unconditional write let a stale count land after a fresher one; the compare-exchange that replaced it dropped a fresher publication when the staler one won the race; and its hysteresis band tested a target that could already have moved. Each fix traded one wrong interleaving for another. The fallback was also strictly weaker than one more iteration of the loop it backstopped: no post-write count revalidation and no retry. Remove it. Exhausting the attempts now publishes nothing, which is safe rather than resigned. Every write to the target is made by a thread that read both the previous target and the count immediately beforehand, so exhaustion leaves a value some recent count justified. Both count movers move the count before recomputing, so whichever thread invalidates a published target must itself re-derive it. A stale target left too high cannot hide behind the band either: clearing pressure requires the target to equal the ceiling, while the budget binding requires the quotient to be below it, and the band suppresses a write only when the quotient is at least the target. Replace the two tests that pinned the deleted methods with one that pins the ordering the argument rests on, and tighten two reply parsers that accepted malformed framing: any negative bulk length read as null, and a bulk string whose terminator was never checked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Remove the recompute attempt knob and harden two reply parsers Deleting the stale-publish fallback made a zero or negative attempt limit permanently stick the published target, since nothing else republishes on exhaustion. No caller passed the parameter once the two tests pinning the deleted methods were removed, so the constructor parameter is gone and the bound is a constant. A reply carrying its declared payload with no trailing CRLF blocked in Receive until the socket timed out, so the terminator assertion was unreachable. Both parsers now read to a pipelined PING sentinel and parse declared lengths as long, so an out-of-range length reports a named assertion instead of overflowing the bounds arithmetic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count sessions directly instead of parsing CLIENT LIST off the wire The hand-rolled RESP parser behind the session count produced a review finding in four consecutive rounds: an unreachable terminator check, a null-length case accepted too broadly, and twice an overflow in its bounds arithmetic. None were about the property under test. The fixture already enumerates live sessions for another test, so the count now uses that: it opens no probe connection, so it needs no self-discount, and it cannot mis-frame a reply. The remaining array parser compares its declared length by subtraction. Summing the offset, the length and the terminator wraps negative at long.MaxValue, which passes the bounds check and reaches the narrowing cast with an invalid length. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cut the per-receive and per-batch cost the buffer caps added The session buffer caps and the network budget each added work to a path that runs once per receive or once per batch, which at batchSize=1 is once per command. Three changes, none of which alter behaviour: - UpdateNetworkBuffers called MaybeShrinkNetworkReceiveBuffer on every receive, so a connection whose buffer is still at its base size paid a call and a chain of loads to discover it had nothing to do. The entry test is hoisted to the call site and the callee marked NoInlining. - BaseReceiveBufferSize walked networkPool.Budget and two properties on every read. The budget and the configured sizes are cached as readonly fields on the handler, and the clamp is one inlined call on the budget. - The scratch builder and the parse state each counted their own batches. One countdown on the session now drives both, so the batch boundary pays a single decrement regardless of how many buffers are capped, and ScratchBufferBuilder.Reset returns to exactly what it was before. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Release quiesced sessions' latency histograms A session that records latencies and then goes quiet holds its histograms for the connection's lifetime: lazy allocation defers the cost but never returns it, and the monitor's per-window Reset only zeroes the counts while keeping the array. At thousands of connections that is hundreds of MB retained for histograms recording nothing, plus a memset of all of it every sampling interval. The latency monitor already sweeps every session every window and already evaluates the exact predicate needed, so no tracking is added to the record path. After a configured number of consecutive empty windows the monitor drops that type's histograms. The reference is dropped rather than returned to the pool. The record path is deliberately unsynchronised, so a writer racing the monitor can hold the array past the release; dropping leaves that writer with the only remaining reference and its write unobservable, whereas returning would let it write into an array another session has re-rented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the histogram release threshold's hysteresis margin The component test supplies its own threshold, so the configured value was pinned by nothing: setting it to one window left the fixture green. A session issuing requests more slowly than the monitor sweeps must keep its histograms across the isolated empty windows that produces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's timing margin At a two-second request interval against a one-second sweep, an overshoot on a loaded box can manufacture the four consecutive empty windows the test asserts cannot happen. At 1.5 seconds the mutation still fires in three seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's absolute stall tolerance Scaling the sweep and the request interval together keeps the ratio the mutation needs while doubling how long the machine may stall before the test manufactures the consecutive empty windows it asserts cannot occur. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reduce the network buffer config surface from nine flags to two The branch had grown nine new command-line switches. Seven of them were tuning knobs for an adaptive mechanism that sizes itself, exposed because they were easy to expose rather than because an operator needs them. Demoted to properties on GarnetServerOptions, reachable programmatically but no longer settable at startup: --network-buffer-size --network-buffer-min-size --network-max-receive-buffer-size --network-send-buffer-min-size --network-buffer-pool-size --session-parse-state-max-retained-args --session-scratch-buffer-max-retained-size None was CONFIG SET-able, so no live mitigation is lost, and none has shipped, so no operator .conf can reference one. Kept: --network-buffer-memory-budget the process-wide ceiling this work adds, and the only knob whose right value depends on the deployment --latency-monitor-precision a genuine accuracy/memory trade-off with no correct default Renamed for direction-explicit consistency with the pre-existing NetworkSendBufferMinSize: NetworkMaxReceiveBufferSize -> NetworkReceiveBufferMaxSize NetworkBufferMinSize -> NetworkReceiveBufferMinSize NetworkBufferSize keeps its direction-agnostic name: GarnetServerBase already exposes a public NetworkBufferSize for the same concept, so renaming here would diverge from established vocabulary. Removing the defaults.conf key would have changed behaviour rather than preserved it. An unset NetworkBufferPoolSize does not mean "unbounded" -- LimitedFixedBufferPool derives a byte ceiling from its per-level entry bound instead, which at the shipped 16-entry default is ~32.5 MB rather than the 64 MB the standalone server has always run. Options.GetServerOptions() now assigns "64m" directly, so the CLI path resolves exactly as before and embedded hosts keep the derived geometry they already had. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tell a refused connection why, and count it A connection refused by --network-connection-limit was disposed with no error written and counted in no metric, so an operator hitting the limit saw an unexplained reset and had nothing to confirm the cause. Send the Redis-compatible -ERR max number of clients reached before closing, and add rejected_connections to INFO stats. The write is gated to non-TLS. A plaintext error to a peer that has only sent a ClientHello is a protocol violation, so the client reports a handshake failure and the operator is pointed at certificates instead of capacity -- worse than a clean reset. Under TLS the metric is the whole remedy. The counter is incremented only in the limit branch, not in the catch that handles sockets dying during setup, so it stays trustworthy in the incident it exists for. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the connection limit process-wide and settable as maxclients The limit was per-listener, so N endpoints gave N times the configured ceiling -- the same defect this branch fixed for buffers, on the adjacent line of GarnetServer.cs. ConnectionLimit sums the listeners' existing activeHandlerCount values rather than maintaining a second counter, so no new accounting can drift; a connection counter that ratchets fails closed, refusing everything until restart. Default is now 10000, matching Redis, and set on GarnetServerOptions rather than only defaults.conf so embedded and test hosts resolve the same value as CLI hosts. CONFIG SET maxclients pushes the new ceiling through StoreWrapper.Servers into the live accept path. Lowering it disconnects nobody, which is also how Redis treats a lowered maxclients. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the refusal message, the counter, maxclients and the shared ceiling Seven assertions. The fixture holds one persistent probe connection so INFO and CONFIG can still be issued after the server is full, and fills only the remaining slots -- a probe that connected on demand would itself be refused. metricsSamplingFreq is set because rejected_connections is published by the monitor's sampling loop and reads "0" unconditionally without it, which would have made every counter assertion vacuous. Commands go over the wire as RESP arrays: Garnet's inline parsing handles single-token commands but silently produces no reply for a multi-token one, so an inline CONFIG GET returned nothing and the first version of these tests failed for that reason rather than for anything about the limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the shared-ceiling test able to fail The second listener refusing is equally consistent with it having reached a ceiling of its own, so the test now makes it accept a connection first. Also adds the drain-and-recover assertion and asserts several connections past the old ceiling when the limit is lifted, since admitting exactly one more is what raising the limit by one would also do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Build the connection limit on net8.0, and document the maxclients semantics System.Threading.Lock is .NET 9+, so ConnectionLimit did not compile for the net8.0 target the solution also builds. A plain object monitor is what the rest of the server uses and is equivalent here -- the lock is taken only by listener registration at startup, never on the accept path. Also records the three semantics an operator will ask about: replica and cluster peer links count against the limit (as they do in Redis, whose accept path adds getClusterConnectionsCount()); lowering the limit refuses new connections rather than culling established ones; and the value is not clamped to the file descriptor limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix final newlines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover maxclients across listeners and TLS rejection accounting MaxClientsReachesTheSharedLimitFromAMultiListenerHost fills the shared ceiling from two endpoints, raises maxclients at runtime, and asserts both listeners admit again, so CONFIG SET is pinned as reaching every listener rather than the one that happened to be first. TlsRejectionsAreCountedEvenThoughTheClientCannotBeTold pins the other half of the TLS gating: a refused TLS connection is closed without a RESP error, but still increments rejected_connections, which is the only signal an operator has on that transport. The TLS fillers are real TLS clients rather than bare sockets. The handshake runs on the accept path, so a connection that never handshakes stalls the accept loop and is never admitted, which makes bare sockets useless as fillers on a TLS listener. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS accept-path stall with concurrent and partial-handshake cases Upstream now starts the handler without waiting for the handshake, so the fix this branch carried is redundant and is dropped in favour of it. The tests are not: they add a peer that stalls after a partial record header, and sixteen concurrent silent peers, neither of which the upstream regression test reaches. Each asserts on a completed RESP exchange rather than on connect, because a connect against a wedged listener is absorbed by the backlog and succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin that a failed TLS handshake releases its admission slot Starting the handshake off the accept loop moves where a handshake failure is noticed: it no longer propagates out of Start into the accept site's catch, so the slot the accept path already took is released by the handshake's own failure path instead. A leak there would let an unauthenticated peer permanently fill a connection-limited server. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Delete the non-discriminating handshake slot test Three mutations could not break it: removing the observer's Dispose, removing Dispose from the handshake failure path, and holding the peer socket open all left it green. Slot release on a failed handshake is over-determined, so the test observed nothing its subject could change. Replacing the fillers with silent sockets does fail, which is the residual the handshake timeout addresses rather than a property to pin here. Also corrects the ConnectionLimitTlsTests rationale, which cited the accept-loop stall that no longer exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound the per-session Lua script cache with an LRU cap Each cached script holds a whole Lua VM, so an unbounded per-session cache lets a session that runs many distinct scripts accumulate VMs for its lifetime. Cap the cache and evict the least recently used entry. Recency is a monotonic stamp updated in place on each hit, so a cache hit costs one store and eviction scans only at capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the script cache bound, its LRU order, and the evicted-script path Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate transaction key entries unpinned on both paths TxnKeyEntries allocated its initial array with GC.AllocateArray(pinned: true) while the growth path used a plain array, so the pinning was silently dropped at the first resize. Every use of the array is through AsSpan, indexing or Array.Copy, and no caller takes a stable address into it, so the unpinned form the growth path has always used is the correct one. Dropping the pinned initial allocation also keeps these arrays out of the pinned object heap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count and report response buffer overflow rentals ReallocateOutput rents a pooled buffer whenever a response outgrows the buffer it is being written into. That traffic was only inferable, so the amount of overflow a given send buffer size causes could not be observed directly. Adds a process-wide rental count, reported as total_output_buffer_rentals in INFO STATS. It is read straight from the counter rather than from the monitor's sampled global metrics, so unlike the neighbouring rows it stays accurate when no monitor is configured. Measured through the new counter: ordinary responses rent nothing, and an oversized response rents exactly one buffer rather than climbing a doubling ladder, which is the property that makes a smaller send buffer affordable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop trailing newlines to match the repo's final-newline convention Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report the output buffer rental count under BPSTATS rather than STATS The counter is read live from its source rather than sampled by the monitor, so under STATS it was the one row that stayed non-zero when no monitor is configured. MetricsDisabledTest asserts every STATS row reads zero in that state, which is Garnet's default, and it caught this. BPSTATS is where the rest of the buffer instrumentation on this branch already lives and is read live for the same reason, so the counter is consistent with its section there instead of being an exception inside a sampled one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count the refusals a capacity wait legitimately incurs WaitForAcceptance polls until the server has capacity, and capacity returns when the server observes the closes rather than when the client disposes. An attempt made before it does is refused for real and advances rejected_connections, so the two tests asserting an exact count were racing a teardown they do not control. Report the refusals from the wait and add them to the expectation, keeping the assertions exact rather than loosening them to a bound that a build which stopped counting would also satisfy. Only a genuine max-clients reply counts, so an unrelated socket error cannot inflate the allowance and mask a miscount. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Capture the session latency histograms once at every reader The monitor sweep nulls LatencyMetricsEntrySession.latency when a session quiesces, but every reader check-then-dereferenced the field, so nothing guaranteed the two reads saw the same value. Both RecordValue overloads run on the session hot path with no lock by design, and GarnetLatencyMetrics.Merge runs on the disposing session's thread under rwLock while the sweep's reclaim holds only disposeLock, so the two do not exclude each other. Load the field into a local once and work from that. Allocate() now returns the array so the miss path reads the field once too. The type's remark claimed the reclaim was safe because a racing writer holds the only reference to the graph; that is true only if the writer captured it, so the remark now states single-capture as the invariant to preserve. Also gate the shrink counters on the budget being enabled, matching OnBufferAcquired: every pool handed no budget shares the Disabled singleton, so counting there put a contended process-wide interlocked write on the shrink path of connections the budget does not govern. Drop two unintended UTF-8 BOMs, and correct the maxclients documentation. Garnet takes every connection on one listener and cannot tell the kinds apart at accept time, so at the limit it refuses replica and gossip links alike; Redis counts cluster bus links but accepts them on a separate port that applies no limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Name the single-capture invariant at the latency record path The two RecordValue overloads load `latency` into a local exactly once because the monitor thread can null it concurrently. The expression reads as a convenience idiom, so state why it is load-bearing at the point a future edit would touch. The invariant itself is documented on ReclaimIfQuiesced, which is the writer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * State the single-capture requirement on the latency field itself The record-path comments guard the readers that exist. An author adding a new reader passes through the field declaration instead, so the constraint belongs there too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tighten the comments added by this work Cut justifications-against-alternatives and narrative framing from the doc comments and inline comments this change introduces, keeping the constraints and invariants a reader needs in order to change the code correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Restate why Recompute needs no fallback on attempt exhaustion Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Inherit TestBase in the concurrent stack fixture Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the buffer pool's stats directly rather than over the wire Serving an INFO reply takes a send buffer and a receive buffer off the free list the assertion reads, so the probe consumed what it reported and the assertion had no margin against its own instrument. Read the listener's pool directly, and poll the size class the assertion reads rather than the byte total, returning the snapshot every assertion is then taken from. Cover the same rule against the pool directly, where nothing can take an entry off the free list between the return and the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address PR review comments on histogram release, scratch cap, and test sends Release the session latency histograms by dropping the reference rather than returning their counts arrays, so a racing recorder cannot write into an array another session has since rented. `LongHistogram.Release()` does the same for the client, whose `latency` field is readonly. `ScratchBufferAllocator.Reset()` no longer enforces the retained-capacity cap at every batch boundary; a `ShrinkCheckpoint()` on the session's coarse countdown does, mirroring `ScratchBufferBuilder`. The session and transaction allocators are both wired to it, with a guard so a WATCH spanning batches keeps its slices. Test socket sends loop until the whole request is written, and the RESP reply reader reads until the expected length has arrived. `NetworkBufferBudget` documents the budget as an adaptive target on the base buffer size rather than a ceiling on the aggregate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the scratch allocator's checkpoint interval and its outstanding-slice guard A functional WATCH test cannot observe a buffer released underneath a live slice: the pinned array is collectable but its bytes stay readable until a collection reuses them. Both invariants are asserted on the allocator directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Update the scratch allocator tests for the checkpoint release contract The allocator now keeps its current buffer across Reset and releases an over-sized one at a ShrinkCheckpoint, so the fixture asserts the retained capacity at each point and drives two checkpoints to clear the hysteresis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Floor the above-max receive shrink at the largest poolable size The residual left after a fully consumed request is zero, so sizing the replacement from it always resolved to the base. A connection issuing one oversized request per batch then re-grew through every intermediate size class on every request. Floor the replacement at the largest poolable size at both the socket and TLS transport sites; the idle hysteresis path still takes it down to the base once the traffic no longer needs the capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the client latency histogram once at every consumer Release drops the counts array while the reply-processing thread may still be recording, so the record path tolerates a released array and the copy path reads the field once instead of testing IsReturned and then dereferencing. Also make the quiesce-threshold test observe empty windows: it recorded a value and never cleared it, so the windows it asserted over were not empty and a threshold that never restarted its run still passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the released-histogram contract on the record and copy paths Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Derive the copied histogram's tracking values from the counts it carries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Correct the rationale for deriving the copied histogram's total Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS transport above-max receive replacement A TLS connection decrypts into a second receive buffer that grows through its own code path, so the above-max release there was unexercised while its plain-socket twin was pinned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Gate the above-max receive floor on budget pressure The above-max branch releases a buffer the pool cannot recycle. Flooring the replacement at the largest poolable size matches origin/main and saves a connection that needs the capacity on every request from re-growing through every size class, but it must not apply while the budget is binding: the target is derived from a buffer count, so a connection parked at the maximum costs the same single unit as one at the floor and the budget gets no feedback from it. Under pressure the replacement is now the adapted base, which the pool keeps and the budget can see. Also capture the counts array once in ClearCounts and CopyCountsInto, read the client call count off the snapshot the percentiles come from, and record the single-read contract on the field itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS half of the above-max pressure gate The plaintext and TLS receive paths gate the above-max release independently. Only the plaintext site was exercised, so the TLS gate at NetworkHandler.cs:801 could be removed with the whole fixture still green. Parameterize the test over both transports. Guard the counts array in CopyCountsInto, matching the single-read invariant documented on the field and the five sibling capture sites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop the scratch retention assertion that measures test ordering `ScratchBufferCapDoesNotChangeLargeArgumentResults` compared a per-session memory figure across two arms and failed intermittently. Two measurements show it cannot discriminate: swapping the arms moves each figure by about 130 KB per session and flips the sign of the difference, because the arm that runs second also measures the residue of the server instance the first one disposed; and releasing the buffer at every reset rather than at the checkpoint -- the churn such an assertion claims to catch -- moves it by less than that, because churn leaves garbage the forced collection reclaims rather than retention. The test keeps both arms and the reply validation they drive, which is what its name has always claimed, and the remarks state why no magnitude is asserted. `NetworkBufferBudget` documents the per-direction floors as the third reason the budget is a target rather than a ceiling: once the target reaches them, the aggregate grows with the connection count again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Sample the shrink churn either side of the batch boundary Comparing capacity across iterations cannot see the pathology the test exists to catch: a policy that released on every batch regrows before the next sample, so consecutive post-boundary capacities are equal and the churn counts as zero. Sampling before and after each boundary counts every release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Say what the latency precision number actually buys The setting is HdrHistogram significant digits: relative accuracy at every magnitude, not a fixed time unit. Stating it that way, with the memory each step costs, lets an operator pick a value without knowing HdrHistogram. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Pin that an over-sized request is accepted whole The receive buffer grows to fit and maxReceiveBufferSize bounds only what a connection retains, so a chunk far above it must still be accepted. Dribbled in small slices so the request sits partially buffered across many receive passes, where the above-max shrink runs against a live partial request. Also drop a qualifier from the latency precision help text. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Keep the byte-order marks as main has them Earlier edits flipped the BOM on seven files, which showed up as a one-line change to each file header with no content behind it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Wait for the buffer release instead of racing it A receive releases its buffer in UpdateNetworkBuffers, which runs after Process has already written the reply, so a client that has read its response can still be ahead of the release. Sampling BPSTATS once at that point read the pre-release numbers on a loaded runner and failed with the full oversized buffer still live. Poll until the counters settle. The wait uses only the out-of-band INFO connection, so it does not hand the connection under test the extra receive that would let a buffer released one pass too late off the hook: suppressing the immediate release still fails the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Fold the two receive-buffer shrink paths into one policy The network and transport receive buffers ran the same three-regime shrink policy over their own occupancy and countdown, written out twice. Extract it as TryPlanReceiveBufferShrink, which owns the countdown bookkeeping and the budget's shrink accounting, and leave each caller with nothing but the buffer swap. Rename both entry points to TryShrink*, and give the transport buffer a ShrinkTransportReceiveBuffer mirroring ShrinkNetworkReceiveBuffer. Drop the cached configuredReceiveBufferSize/configuredSendBufferSize fields and read them off networkBufferSettings, as maxReceiveBufferSize already is on the same paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Keep the configured base sizes cached on the handler BaseReceiveBufferSize is read on every receive, so reaching the configured size through networkBufferSettings puts a dependent load on that path. BDN Network.BasicOperations.InlinePing measures it at +1.1ns (89.8 -> 90.9) on net10 and +1.9ns (85.7 -> 87.6) on net8, the net8 range clearing baseline entirely. Restoring the fields returns both to baseline (89.8 / 85.8). The shrink consolidation is unaffected and stays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Capture the metrics array in ReclaimQuiescedHistograms Every other method on the session captures the metrics field once before dereferencing it. ReclaimQuiescedHistograms re-read it for the null check and again inside the loop, so it relied on the dispose lock for that safety where its peers do not. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Checkpoint the transaction scratch allocator during AOF replay Replayed transaction procedures run their prepare phase against the watch API, and every watched key is copied into the transaction scratch allocator. The replay session never reaches the network batch boundary that drives the shrink checkpoint, so the buffer one wide procedure grew stayed pinned for the lifetime of a replica. The replay path now reports a boundary per AOF record and the window is counted in TransactionManager, so the caller does not carry the cadence. The allocator shrinks only when nothing is outstanding, so a boundary reported mid-transaction is safe. Also apply two review nits: a u8 span literal for the max-clients error instead of a materialized array, and a corrected comment on the connection-limit registration lock, which was crediting the lock with reader safety that the volatile publication provides. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Release the session scratch allocator at the AOF replay boundary Replayed procedures grow both scratch allocators, not just the transaction one: the prepare phase copies watched keys into the transaction allocator, and the procedure builds its arguments from the session allocator. The session allocator is reset at the start of every procedure rather than when one ends, so a completed procedure leaves it oversized with a non-zero offset, and a shrink checkpoint declines in that state. Reset it at the replay boundary, as the network batch boundary does, and only outside a transaction, whose slices are still live. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Rename the buffer shrink hook off "checkpoint" and count replay records at the call site "Checkpoint" is load-bearing in this codebase: Tsavorite uses it for durable, versioned, recoverable snapshots, and Garnet's AOF replay handles genuine CheckpointStartCommit markers a few lines from where this hook was called. The scratch-buffer hook means something unrelated -- periodically sample a buffer's capacity and release it if it did not grow -- so it is renamed to Trim, matching ArrayPool<T>.Trim, which applies the same hysteresis. The replay hook also counted its own interval while the network one had the caller count, so TrimAfterReplayedRecord ran on every record but trimmed on one in sixty-four. The counter moves to AofReplayContext, alongside the session it governs, so both paths show the cadence at the call site and TrimReplayBuffers means "trim now". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 --------- Co-authored-by: badrishc <badrishc@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 | 1 个月前 | |
Exponential backoff for gossip reconnect, TLS read fix, and pub/sub large-payload forwarding (#2225) * exponential backoff mechanism for Gossip and misc fixes * Fix final newline in ClusterServerNodeOpsTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 146f7b3c-7f4e-41f9-b8c1-836075480226 * ensure linked cts is re-used accordingly on reconnect * update testing and push cts down to reconnect * fix tls partial frame regression with addition of test * add comment for out-of-order delivery of pubsub that does not fit AOF page * fix reset tcsArray and add test * simplified credential rotation in gossip clients * fix tls issue * fix issues with double framing --------- Copilot-Session: 146f7b3c-7f4e-41f9-b8c1-836075480226 | 20 天前 | |
Improve network connection handling (#2268) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> | 11 天前 | |
Support and accelerate bitmap operations on more platforms (#942) * Initial BITOP refactor * Remove TensorPrimitives invoke logic for now * format whitespace & remove incorrect todo comment * Remove unnecessary usings * Add HW acceleration guards and V128 core path for non V256 CPUs * fix * Add S.N.TensorPrimitives and use it for BITOP NOT * Use TensorPrimitives to specialize two key scenario And fast-path memcpy for single key * Make BitOp tests more granular with parameterization * Also increase coverage to show failure in current PR to handle uneven bitmap lengths on the two-key accelerated path * Make uneven 2-input specialization fallback to the multi-key path for tail handling * Simplify bitmap operation result memory logic * Add out-of-proc tests for the bitop HW accelerated paths * Add benchmarks * dotnet format * Add Vector512 path * Handle environment variables containing '=' Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove the two-input special path * The TensorPrimitives are much much more mature implementations, but the control flow here is a hassle with them. If TensorPrimitives expose the InvokeSpan helper methods they are largely based on, this may be worth it to add back. Or we have greater need for the InvokeSpanInto* and we'll copy the implementation to Garnet. * Add BITOP DIFF support * StackExchange.Redis to 2.9.25 for the new bitops in tests * Wire up DIFF with rest of the BITOPs * forgot to set the Lua registry constant --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Vasileios Zois <96085550+vazois@users.noreply.github.com> | 1 年前 | |
Cleanup VectorSet TODOs (#1810) * VectorSet todos Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com> * upd Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com> * address comment Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com> --------- Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com> Co-authored-by: Tiago Napoli <tiagonapoli@microsoft.com> | 5 个月前 | |
Introduce hint to prevent exclusive lockers from being starved in `ReadOptimizedLock` (#2156) * add test for exclusive lock acquisition starvation in ReadOptimizedLock * add an exclusive lock hint slow path to ReadOptimizedLock * extend hinting to AcquireAllExclusiveLock * address feedback * per CoPilot, RI has a single re-entrant case; guard against it * track actual index for re-entrant check, no guarantee same index is what's locked for flushing * move to threadlocal * remove implicit re-entrant test case | 1 个月前 | |
[Cluster] Fix stale offsets and write admission during primary failover (#2253) * fix stale replication offset bug for FAILOVER TO * introduce pause variable * ensure pauseWrites is logical and does not change the node role * fix race when running session misses primary to replica transition at FAILOVER * enforce no checkpoints during failover and ensure that config ref is updated when operations spin on write-pause * refresh snapshot on slot verification retry * ensure that FAILOVER forces writes to spinWait instead of blockin and affecting availability * register background failover task with TaskManager and hook-up timeout to FAILOVER command * mitigate waiter starvation through async await * fix cancellation clearing the pause after demotion * ensure primary tasks are suspended before failover initiates to capture AOF tail * ensure txn safety when flushdb executes while FAILOVER is running * ensure old primary initiates replication stream at FAILOVER success * ensure task cancellation targets only primary tasks for role changing operations * add regression test for REPLICA OF * fix build * ensure commits drain on CommitTask Suspension | 9 天前 | |
Cluster epoch: isolate GarnetEpoch and replace the busy-spin barrier with a jittered self-poll (#2203) * separate epoch to isolated instance * stage 1; implement async wait wake up GarnetEpoch * BDN bench for Async GarnetEpoch * cleanup and renaming * add cpu diagnoser * add proof BDN for CPU contention * Remove uncommitted EpochBarrierLitmus project from solution The litmus harness was a dev scratch artifact never committed; BDN (EpochBumpContention) demonstrates the spin-vs-wake CPU issue cleanly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a071df49-2309-4a9f-a4c4-dc6a77ea05aa * add waker latency BDN * add more tests * simplify waitAsync on epoch bump * use valuetask; harden concurrency guarantees using volatile * introduce declarative spinWait primitive * ensure memory ordering and atomicity for epoch acquisition and release * bump version --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a071df49-2309-4a9f-a4c4-dc6a77ea05aa | 25 天前 | |
Dynamically Set Memory Limits with CONFIG SET (#1279) * wip * wip * wip * wip * wip * wip * wip * Added some comments * Update libs/server/ServerConfig.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * wip * wip * wip * wip * WIP * comment fixes * wip * wip * wip * wip * fixed comment * Test fix * fix * Fixing comments * Fix --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> | 1 年前 | |
Improve network connection handling (#2268) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> | 11 天前 | |
Support for Parallel Replication (#1556) * implement sequence number generator * More work on LogRecord.RecordLength * format code * Finish LogRecord.RecordLength * add checkpoint marker process coordination * add fuzzy region flag to replay context * trigger exception when remove of checkpoint barrier fails * throw exception if SignalAndWait times out * add flag to differentiate between sharded and single log AOF * AOF explorer * wait for aofSync and refreshTail tasks on Dispose * validate base replication tests v1 * Remove ObjectDb* from Sample*Txn * test fixes for disk-based replication v2 * add synchronized replay for remaining operations (i.e. FLUSHDB, STREAMING_CHECKPOINT) * sequential node dispose on teardown * ensure dispose client precedes iterator * MultiLevelPageArray stress fixes; improved LogRecord and RecordDataHeader ToString() * WIP * Fix GetObjectLogSegmentSize() * wait for aof sync driver and connection to drop before recovering * consolidate reset replica replay task group * update docs * Fix a couple comments * address data loss on synchronization * use unified synchronized operation for regulard txn replay * consistent read for scan commands * Trim ObjectIdMap arrays on ClearPage * remove unused variable * fix single log tests * fix sharded log tests * adding comments to AofReplayCoordinator * adjust ROLE command response for sharded log implementation * WIP; multi phase read consistency protocol * added basic consistent read protocol context * Add transactional consistent read context * md readmed for mlog design * add design notes * WIP; change session at state change * allow disklogRecord upserts for streamless sync * implement consistent read protocol for scan commands * add sharded diskless sync tests * eliminated interface declaration for APIs with consistent read * cleanup and refactoring * remove sharded option since we have AofBench * update tail witness task to ping replica for latest replayed sequence number information * remove unused code * wip; logical subtask replay * WIP; re-design AofHeaders * WIP; update sequence replay info * wip; v1 parallel replay within physical sublog * wip; running sharded log tests with single replay task * wip; refactor to use virtual sublog replay state and a virtual sublog idx calculation * wip; update coordinated operations to trigger when multilog is enabled * wip; add multi-replay test for single key operations * wip; cleanup * remove AofExplorer * wip; index coordination barriers with distinct key to mixing separate transactions on same session * add indexing for coordination barriers using sequence number * rename sublog count to physical sublog count * remove cproj ref * separate AofBenchType * wip; mlog description * wip; mlog description * wip; adding comments and writing dev replication website content * wip; updating documentation and description in code * wip; AofSyncTask section * wip; replay details summary and fixes * wip; renaming and updating todo items * wip; skip hashing when configured with one physical sublog and multiple replay tasks * add test for custom txn and multi-exec txn with parallel replay * refactor/rename and cleanup replica replay code * wip; adding bulk consume recovery * renaming and cleanup * simplify intra page replay and synchronization primitive * implement recover intra page replay * assert throw when configuring multi-log without FastCommit * add description of prefix consistent recovery * push log specific operations in GarnetLog instance * wip; * implement prefix consistent recovery * refactor code to use single log when multi-replay is enabled with one physical log * remove cookie generation callbacks * cleanup and documentation write-up * more documentation * refactor advance time background task * TaskCompletionSource based wait for read consistent read * fix read protocol edge cases * code comments and cleanup * add cluster command to retrieve key sequence number for debugging purposes * PR description initial draft * advance time using high precision timestamp and background task * PR details * fix formatting * revert HashUtils location * HashUtils mv * fix formatting * cleanup * single log legacy implementation updates * add docs for mlog key time * legacy checkpoint cookie serialization support * fix test delay * run aofsynctask directly with single log * append replication vector to ROLE command when using multi-log * add support for intra-cluster connection identification and fix failing management tests * set client info when clientname parameter != null * support negative values in AofAddress * add ConsistentReadContext definition into Globals * fix formatting * fix ClusterDisklessSyncResetSyncManagerCts * fix ClusterResetHardDuringDisklessReplicationAttach * fix consistent read context bugs * wip; refactor read-protocol integration * Implement read key batch consistent protocol * improve robustness of background advance time consumer * remove unused playground project * cleanup clientname setup * fix tsavorite ISessionFunctions * remove unused assembly * simplify AofAddress and check for correctness * fix typo * eliminate bytes from informational message * simplify and cleanup leader barrier * cleanup LeaderFollowerBarrier * ensure commit for multilog recovery * replace tcs with semaphore slim * fix ClusterReplicationCheckpointCleanupTest * Fix copy-paste bug sending beginAddress instead of tailAddress in disk-based replica sync (#1633) In TryReplicateDiskbasedSync, ExecuteClusterInitiateReplicaSync was sending beginAddress.Span for both the aofBeginAddress and aofTailAddress parameters. This was introduced in commit 6fb99e58 when converting from ToByteArray() to Span-based calls. The primary uses the replica's tail address to compute the AOF sync replay range. With both parameters being the begin address, the primary couldn't determine where the replica's AOF actually ended, causing the replica to never receive AOF records and remain stuck at offset 64 (kFirstValidAofAddress). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix ClusterResetHardDuring tests by using ConfigureAwait for ExceptionInjection * enforce ConfigureAwait(false) across Garnet * fix AofBench * add website description * add migration GarnetLog upgrade test * fix formatting * fix website links * add object option for replication scheme upgrade test * set default logging to Error * pre-compute enforceConsistentRead * add AOF option for cluster BDN * update global.json * consolidate enqueue calls * fix upgrade test null check * add custom stop bulk consume callback * fixing ClusterReplicationMultiRestartRecover * fixing base replication tests * use throttle to check consumer liveness during bulk consume * refactor advance time background task * ensure WorkQueueLIFO waits for process bg task to complete on dispose * add activeWorkerMonitor * cleanup and refactoring * wip; adding tracking * fix hanging ClusterReplicationCheckpointCleanupTest * fix Tsavorite formatting * fix Garnet formatting * fix breaking build * improve log message * throw the caught AsyncFlushSnapshotPage exception * ensure failover suspends replica only tasks * usingSharded when replaying with multiple tasks * add no-inlining exception throw * fix AofBench to work with AofReplayCount * wip; add variant to parallel replay a physical sublog * wip; fixing ClusterSRNoCheckpointRestartSecondary * fix defaults * classify sharded tests under same namespace * throw at TakeOverAsPrimary to fix test * log startAddress on dispose of AofSyncDriver * add better logging at ClusterAppendLog initialization * reuse parseState at replayOp * avoid cache invalidation when updating replicationOffset * ensure get slots parsing deals with unstable slots * nit; cleanup * fix formatting * better logging * eliminate monotonic updates on the enqueue hot path when tracking sequence numbers for advance time. * make aof-replay-task-count default=1 for AofBench * add ReplayNoResp * add ReplayDirect * fix Enqueue Bench * add better logging for replica receive primary AOF stream * fix InProc Bench wait for server * implemented targetted logging API for testing * enable LogPrimaryStream for ClusterSRNoCheckpointRestartSecondary * refactor cluster cmdstrings * remove unused methods * cleanup * fix logging event flag checking * update aof bench * optimistically return if not ReadWaiters * improving AofBench * Pass around keyHash and re-use it during replay * remove notes * increase thread pool for cluster tls replication * expand logging at AofSyncTask initialization * wip; cleanup replay path * use owned epoch for multilog * use owned epoch for tsavorite log and cleanup aof recover * make vector fixes for replication * fix aof gen * JIT replayOp refactor * fix range index AOF enqueue * use barrier on AofSync connection with multi-log only * adding async for connect and gossip * addressing first set of comments * addressing second set of comments * add throw cancellation * make light epoch in client compatible with tsavorite epoch implementation * simplify object RMW enqueue * separate appendLog initialization --------- Co-authored-by: TedHartMS <15467143+TedHartMS@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> | 6 个月前 | |
Add Unix socket support (#944) * Add Unix socket support * More cluster test adjustments * Remove warnings * dotnet format whitespace * fix merge * fix merge * Add very basic test for the unix socket support * Try fix long-path error & sync GarnetClientSession connection logic with GarnetClient * Dispose the GarnetClient objects in tests * Sync LightClient connection logic with GarnetClient(Session) * fix * Add exception message to StoreWrapper.GetIp * Fix merge * Add TLS test path * Add more Unix socket tests with SE.Redis as client * Add config parsing for --unixsocketperm and tests * attempt fix import ordering * Set unix socket permission after listener is bound * attempt fix import ordering * attempt fix import ordering * attempt fix import ordering * use ILogger in LightClient --------- Co-authored-by: Vasileios Zois <96085550+vazois@users.noreply.github.com> | 1 年前 | |
Cache DateTime.UtcNow in AAD authenticator hot path (#1855) * Cache UTC time in GarnetAadAuthenticator.IsAuthorized to remove syscall from hot path IsAuthorized() is checked twice per command (AdminCommands.cs:32 and :131) via the IsAuthenticated property. Under AAD auth, DateTime.UtcNow showed ~60% exclusive CPU in cluster traces because it issues a clock_gettime syscall via vDSO on every call. Introduce Garnet.common.CachedTime, a process-wide UTC time cache refreshed every 100ms by a background Timer (modeled after Redis's mstime). Reads are a pure memory load via Volatile.Read of a static long. GarnetAadAuthenticator now precomputes the token validity window in ticks once in Authenticate() and compares against CachedTime.UtcNowTicks in IsAuthorized(). 100ms granularity is operationally invisible against AAD token windows measured in hours. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix misleading Redis hz=10 reference in CachedTime doc Redis's server.mstime is refreshed both from serverCron (hz floor, default 100ms) and from beforeSleep (every event loop iteration), so under load it updates far more often than 100ms. Rephrase as an illustrative example rather than a precise cadence match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop underscore prefix on CachedTime private fields libs/common consistently uses bare camelCase for private fields; align. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop unnecessary tick-cached validity fields GarnetAadAuthenticator is per-RespServerSession (single-threaded), so _validFrom/_validateTo are never accessed cross-thread; there's no DateTime atomicity concern and DateTime comparison compiles to the same mask+compare as long comparison. Strip the fields and inline CachedTime.UtcNow into the original DateTime check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Extract Timer callback into named UpdateCachedUtc method; tidy stale docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Collapse Timer ctor to single line; document benign overlap race Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop Redis reference from RefreshPeriodMs doc Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Use block-scoped namespace in CachedTime (matches libs/common convention) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop platform syscall names from CachedTime summary Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Note CachedTime staleness can exceed RefreshPeriodMs under high load Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop Timer comment from CachedTime Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename CachedTime to CoarseUtcNow; switch accessors to .Value/.Ticks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Correct CoarseUtcNow doc: DateTime.UtcNow is vDSO/KUSER_SHARED_DATA, not syscall Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add BDN benchmark for IsAuthorized hot path: DateTime.UtcNow vs CoarseUtcNow Mirrors GarnetAadAuthenticator.IsAuthorized() body verbatim plus the raw clock primitives for context. Results on i7-12700K / .NET 10: IsAuthorized_DateTimeUtcNow : 19.34 ns (baseline) IsAuthorized_CoarseUtcNow : 1.35 ns (0.070x — ~14.4x faster) Raw_DateTimeUtcNow : 18.06 ns Raw_CoarseUtcNowValue : 0.51 ns Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename CoarseUtcNow to CoarseDateTime (UtcNow/UtcNowTicks); use Ticks in AAD hot path CoarseDateTime exposes both UtcNow (DateTime) and UtcNowTicks (long) accessors. GarnetAadAuthenticator now caches token validity as long ticks and compares against CoarseDateTime.UtcNowTicks, eliminating DateTime's per-call Kind-bit masking on the hot path. BDN benchmark shows 1.35 ns -> 0.32 ns vs the DateTime variant (~60x speedup vs DateTime.UtcNow baseline). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop raw clock primitive benchmarks; keep only IsAuthorized variants Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add blank lines around tick-fields block in GarnetAadAuthenticator Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Refactor coarse time to TimeProvider; inject into AAD authenticator; add tests + TickCount64 benchmark Addresses PR review feedback: 1. CoarseDateTime becomes a thin static wrapper around the new CoarseTimeProvider (instance) which wraps an arbitrary TimeProvider and uses TimeProvider.CreateTimer for cache refresh. CoarseTimeProvider.System is the process-wide singleton backed by TimeProvider.System -- production behaviour unchanged. 2. GarnetAadAuthenticator now takes a CoarseTimeProvider (default = .System). Hot path is identical in production and tests; tests inject a FakeTimeProvider-backed CoarseTimeProvider so advancing the fake fires the same Timer-refresh callback the production cache uses. 3. AadAuthenticationSettings exposes the public TimeProvider parameter, internally owning/disposing the CoarseTimeProvider lifecycle (reuses .System singleton when no override). 4. New GarnetAadAuthenticatorTests covers: default fallback, valid window, expiry after Advance(), staying valid across Advances, malformed token, failed reauthentication clears state. 5. IsAuthorizedBenchmark adds the Environment.TickCount64 variant suggested in PR review (Tornhoof) -- monotonic ms-since-boot, validity window projected at Authenticate() time. * CoarseTimeProvider extends TimeProvider; drop CoarseDateTime facade - CoarseTimeProvider now inherits TimeProvider: overrides GetUtcNow to return the cached coarse value; delegates GetTimestamp / TimestampFrequency / LocalTimeZone / CreateTimer to the wrapped provider. Adds an explicit fast UtcNowTicks accessor (single Volatile.Read) that the AAD hot path keeps using. - Dispose() no-ops on the .System singleton so callers can dispose unconditionally; removes the _ownsCoarseTime bool from AadAuthenticationSettings. - Drop the CoarseDateTime static facade -- redundant now that CoarseTimeProvider.System exposes UtcNow/UtcNowTicks directly. Benchmark renamed (CoarseTimeProvider / CoarseTimeProviderTicks). - libs/common naming convention: drop underscore prefix on new private fields in CoarseTimeProvider (matches the rest of libs/common). * Tighten CoarseTimeProvider API; add Stopwatch.GetTimestamp benchmark - Make CoarseTimeProvider ctor private; CoarseTimeProvider.Create is now the only entry point, structurally preventing accidental Timer duplication. - Replace RefreshPeriodMs (int) with RefreshPeriod (static readonly TimeSpan); pass directly to CreateTimer. - Add IsAuthorized_StopwatchTimestamp benchmark for completeness. - Test updated to use CoarseTimeProvider.Create(fakeProvider). * Verify AAD AUTH succeeded in BDN OperationsBase setup; tighten comments/formatting - Add EmbeddedNetworkSender.GetLastResponse() that returns the bytes written by the most recent SendResponse(int, int) call. - EmbeddedRespServer.GetRespSession now accepts an optional sender so callers can hold a reference for response inspection. - OperationsBase.GlobalSetup, on the AAD path, asserts the AUTH response is exactly '+OK\\r\\n' after sending the AUTH command. A failed AUTH would otherwise silently leave the session unauthenticated and the benchmark would measure auth-rejection overhead instead of the authenticated hot path. Verified by deliberately corrupting the JWT: BDN aborts with InvalidOperationException and the AAD result becomes NA — failure is loud, not silent. - Drop the redundant 'matches the production GarnetAadAuthenticator code path exactly' comment and collapse two multi-line constructor invocations to one line per repo formatting preference. * Revert "Verify AAD AUTH succeeded in BDN OperationsBase setup; tighten comments/formatting" This reverts commit fb9464ceff606f5e95985f5ce076fd7a9f6684a7. * Reject ACL+AAD combo in OperationParams ctor; collapse formatting - OperationParams.ctor now throws ArgumentException when both useACLs and useAad are true. Per Copilot review (#1859): useAad was documented as mutually exclusive with useACLs but nothing enforced it, so a caller passing (true, false, true) would silently get the ACL path with a confusing 'ACL+AAD' ToString label. - Drop the redundant comment on BuildAadAuthSettings and collapse two multi-line constructor invocations to one line per repo formatting preference (lost in the revert of fb9464cef). * dotnet format: strip trailing newlines per .editorconfig insert_final_newline=false * Clarify in ClusterAadAuthTests that the tenant ID is a public identifier Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump CoarseTimeProvider.RefreshPeriod from 100ms to 1s Coarseness is already documented as ~RefreshPeriod best-effort; AAD token validity windows are measured in hours, so a 1s lag is operationally invisible and reduces Timer wakeups 10x. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add the same not-a-secret comment to GarnetAadAuthenticatorTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Simplify CoarseTimeProvider; switch authenticator to TimeProvider abstraction CoarseTimeProvider is no longer parameterized on a wrapped TimeProvider: it always caches TimeProvider.System, the refresh Timer is process-wide static, and instances are stateless (Create/Dispose and the wrapping ctor are gone). A new Instance singleton replaces the former System field. GarnetAadAuthenticator and AadAuthenticationSettings now hold an abstract TimeProvider (defaulting to CoarseTimeProvider.Instance) instead of a concrete CoarseTimeProvider. IsAuthorized reads ticks via _timeProvider.GetUtcNow().UtcTicks so tests can inject a FakeTimeProvider directly, and production keeps the cached hot-path read through Instance. Tests updated accordingly, plus a dedicated CoarseTimeProviderTests fixture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Cache DateTimeOffset directly in CoarseTimeProvider via Snapshot wrapper Switch the cache from long utcTicks (reconstructed per read) to a heap-allocated Snapshot holding the immutable DateTimeOffset. Readers become a single Volatile.Read on a reference + field load, avoiding the DateTimeOffset constructor's range-checks (ticks bounds, offset bounds) on the hot path. The Timer allocates one Snapshot per refresh tick (~24 B/s, gen-0). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Simplify CoarseTimeProvider: drop UtcNowTicks cache and redundant overrides - Remove the parallel long-ticks Volatile cache (UtcNowTicks). Benchmarks showed the heap-allocated Snapshot<DateTimeOffset> reaches parity with direct ticks under PGO, so the dual cache is unnecessary complexity. - Drop the four delegating TimeProvider overrides (GetTimestamp, TimestampFrequency, LocalTimeZone, CreateTimer). The base class defaults already match TimeProvider.System, so the overrides are pure noise. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop spurious allocation-rate estimate in Snapshot comment Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix DateTimeOffset size in Snapshot comment (it's 16 bytes, not 12) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename Snapshot to DateTimeOffsetSnapshot for clarity Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Use volatile field for snapshot instead of Volatile.Read/Write The volatile keyword gives reference fields the same acquire/release semantics as Volatile.Read/Write at the call sites, but reads cleaner. The two writes (init + timer tick) and two reads (GetUtcNow + UtcNow) now look like ordinary field access. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Move DateTimeOffsetSnapshot to end and trim comment verbosity Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Trim verbose comments in GarnetAadAuthenticator Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert TimeProvider passthrough in AadAuthenticationSettings GarnetAadAuthenticator's ctor already defaults TimeProvider to CoarseTimeProvider.Instance when null, so the settings-level passthrough is unnecessary in production. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove RefreshPeriod XML doc Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop quantified speedup from tick comparison comment Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove _timeProvider comment Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Make UTC explicit in token validity tick projection DateTime.Ticks ignores Kind, so .Ticks on a non-UTC DateTime would silently miscompare against UtcTicks. Construct DateTimeOffset with TimeSpan.Zero so UTC interpretation is explicit at the assignment, and a Local-kind DateTime would throw instead of silently mismatching. Also trim the verbose XML doc on GarnetAadAuthenticatorTests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Comment UTC contract on token validity tick projection Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Drop redundant CoarseTimeProvider/default-clock tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update IsAuthorized BDN to match CoarseTimeProvider API Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Trim IsAuthorized BDN to DateTime vs CoarseTimeProvider Drop the Stopwatch.GetTimestamp variant and a verbose test comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply dotnet format Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Make CoarseTimeProvider constructor private Singleton via Instance is the only intended use; private ctor prevents accidental allocations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Trim CoarseTimeProvider.Instance xmldoc Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add minimal CoarseTimeProvider tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply dotnet format Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Tiago Napoli <tiagonapoli@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> | 4 个月前 | |
Bump version to 2.2.4; combined RESP fixes: HINCRBY overflow, integer-parse protocol error, expired-key checks, large PEXPIRE, SET KEEPTTL (#2259) | 10 天前 | |
Adding basic version of DUMP and RESTORE commands (#899) * Implement basic version of DUMP and RESTORE redis commands * refactor and add dump, restore to cluster slot verification test * Update comments to use 'RESP' encoding terminology * fix formating * fix tests * add acl and tests and update default config to include the skip checksum validation flag * rm accidentally commited dump.rdb file * Remove trailing whitespace in RespCommandTests.cs * fix comments * run CommandInfoUpdater and replace docs / info files * Remove trailing whitespace in Options.cs * fix RestoreACLsAsync test * fix comments * optimize RespLengthEncodingUtils * implement suggestions * fix comments * use SET_Conditional directly * rename SkipChecksumValidation * fix cluster restore test * directly write to the output buffer for non-large objects * Refactor WriteDirect call in KeyAdminCommands * Mark multiple commands as deprecated in JSON * Mark commands as deprecated in documentation --------- Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com> | 1 年前 | |
Add runtime management of eligible server options via CONFIG GET/SET (#1965) * add support for runtime config management * addressing comments * add frozen dictionary and validate kind * add formal enum validation * canonical format check of enums for runtime config * wip; readonly fall through for CONFIG GET off non-runtime options * de-duplication using hashset * add some AOF read-only parameters to support CONFIG GET * support restart action for read-write configuration affecting backgroudn tasks * install infinite timespan with negative value * harden test | 2 个月前 | |
Add runtime management of eligible server options via CONFIG GET/SET (#1965) * add support for runtime config management * addressing comments * add frozen dictionary and validate kind * add formal enum validation * canonical format check of enums for runtime config * wip; readonly fall through for CONFIG GET off non-runtime options * de-duplication using hashset * add some AOF read-only parameters to support CONFIG GET * support restart action for read-write configuration affecting backgroudn tasks * install infinite timespan with negative value * harden test | 2 个月前 | |
Kill AOF BulkConsume When Connection is Dead (#1656) * add custom stop bulk consume callback * overload throttle to check for killed client * use exception utils | 6 个月前 | |
Exponential backoff for gossip reconnect, TLS read fix, and pub/sub large-payload forwarding (#2225) * exponential backoff mechanism for Gossip and misc fixes * Fix final newline in ClusterServerNodeOpsTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 146f7b3c-7f4e-41f9-b8c1-836075480226 * ensure linked cts is re-used accordingly on reconnect * update testing and push cts down to reconnect * fix tls partial frame regression with addition of test * add comment for out-of-order delivery of pubsub that does not fit AOF page * fix reset tcsArray and add test * simplified credential rotation in gossip clients * fix tls issue * fix issues with double framing --------- Copilot-Session: 146f7b3c-7f4e-41f9-b8c1-836075480226 | 20 天前 | |
Configuration & error-handling related fixes (#1229) * wip * wip * wip * wip * fix * wip * wip * wip * wip * wip * wip * wip * format * fix * Logging command line args * Removing logging of command line args | 1 年前 | |
Remove .NET 8 support (#2230) * Remove .NET 8 support Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve multi-framework release and benchmark infrastructure Keep release packaging and publish paths framework-aware while targeting only .NET 10. Retain benchmark job selection for future runtimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Minimize .NET 8 release-script removal Retain the original publish and archive structure, removing only .NET 8 work while keeping the framework-aware archive loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve existing release-script formatting Restore the original whitespace on unchanged .NET 10 publish commands and archive staging lines so the PR diff only shows the required .NET 8 removal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark framework options while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark README update to .NET 10 commands Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave sorted-set implementation unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave GeoHash unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while dropping .NET 8 entries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Revert publish-profile conditional path changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark config note to .NET 10 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark overview change limited to .NET 10 default Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove redundant benchmark framework exclusion guard Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Load benchmark issuer certificate without warning suppression Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 Tsavorite warning properties Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Fix .NET 10 formatting in Vector Set key accessors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 feature guards Drop conditional branches now that .NET 10 is the sole target framework. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep ref struct anti-constraints on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep alternate comparer interfaces on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep vector read batch ref struct declaration on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e --------- Co-authored-by: Tiago Napoli <tiagonapoli@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e | 17 天前 | |
Remove sync-over-async where possible, consolidate blocking into helpers, add analyzers (#1714) * knock out some of the 'easy' .Result uses * convert some easy GetResult() calls to async * proper conversion of a lot of migration code to async * convert cluster epoch polls to async, and everything that's downstream of that * async some of checkpointing and replication * remove some more .Result, mostly by shifting to await helper methods * all .Results that can be removed (or turned into .GetAwaiter().GetResult()) have been removed * another audit of .GetResult(); converting more to tasks where appropriate * formatting * standardize on GetAwaiter().GetResult(); propogates exceptions correctly, but also more unique for searching * move all .GetResult()'s to a helper for easier auditing; cleanup more 'could be async' code * remove explicit .Wait() calls where possible, switch to helper where not * adopt Microsoft.VisualStudio.Threading analyzers; fix or suppress all findings * address feedback * fix nit * change NetworkHandler.Start() so auth proceeds asynchronously - introduce IsAuthenticated(...) to allow polling for auth completion, which many callers assume | 6 个月前 | |
Bound per-connection memory: adaptive network buffer budget, session buffer caps, and connection admission (#2157) * Bound connection-scaled network buffer memory Per-connection network buffers were unbounded in practice: the receive buffer only shrank when it exceeded maxReceiveBufferSize, so a buffer that doubled 128K -> 256K -> 512K never shrank and every connection permanently retained its all-time peak. The TLS transport buffer had no shrink path at all. GarnetServerTcp also discarded its networkBufferSize argument and hardcoded 128 KB, so the per-connection floor was unconfigurable. - Add --network-buffer-size, --network-max-receive-buffer-size and --network-buffer-pool-size, plumbed through GarnetServerOptions into GarnetServerTcp, which now honors them. Defaults match today's behavior. - Shrink a grown receive buffer to the smallest size class that keeps 2x headroom over the buffered bytes, after 16 consecutive receives that fit. Buffers above maxReceiveBufferSize skip the hysteresis since the pool cannot recycle them. - Add the same shrink path for the TLS transport receive buffer, called only from the reader sites that already consider doubling. - Track liveBytes, peakLiveBytes, pooledBytes and a shared byte budget in LimitedFixedBufferPool, exposed through INFO BPSTATS. - Return the outstanding responseObject in DisposeNetworkSender so a session torn down mid-response does not leak its PoolEntry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Document new network buffer options and add config parsing tests Adds the three --network-buffer-* options to the configuration table, warns instead of silently correcting an inverted size pair, and covers defaults, overrides, power-of-2 rounding, the inverted pair and invalid values in GarnetServerConfigTests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bound per-session parse-state and scratch buffers A RESP session sizes SessionParseState.rootBuffer to the largest array arity the client has ever sent and keeps it for the life of the connection. One high-arity command therefore enlarges the session permanently, and the cost scales with connection count. The same ratchet exists in ScratchBufferBuilder, whose Reset() zeroes the offset but never releases the backing array. Add BufferShrinkPolicy, a shared cap-plus-hysteresis helper, and apply it at both sites. A buffer above the cap is released only after the configured number of consecutive batches that did not need the extra capacity, so a session with recurring large requests keeps its buffer. Both caps are configurable and default-on: --session-parse-state-max-retained-args (default 1024) --session-scratch-buffer-max-retained-size (default 64k) Measured end to end with a forced compacting GC, 200 sessions each issuing one 20,000-argument EXISTS followed by sustained single-key work: 327 KB/session retained before, 98 KB/session after. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover buffer shrink against transaction and watch aliasing The batch-boundary shrink releases a pinned array, so anything holding a reference into it across that boundary would dangle rather than merely read stale bytes. Two paths cross the boundary by design: a MULTI..EXEC window, which re-parses queued commands from the network buffer, and WATCH, which copies key bytes into the transaction scratch allocator so they outlive the receive buffer. Add tests that force the shrink to fire inside both windows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized response elements instead of killing the session A response element larger than the send buffer cannot be written atomically. The chunking helper for this already exists and SCAN uses it, but several commands still wrote unbounded user data through the atomic path, where TryWriteBulkString cannot make progress and SendAndReset throws, killing the session mid-response. KEYS fails today with any key larger than the 128 KB default send buffer. Give the shared WriteBulkString helper an atomic-first fallback: write in place, retry once on a flushed buffer, and only chunk an element that cannot fit an empty buffer. Small elements keep exactly today's path, so multi-element replies pay nothing. Route KEYS, COMMAND GETKEYS, COMMAND GETKEYSANDFLAGS and the pub/sub channel and pattern echoes through it. Two related defects found while covering this: ClusterSession.SendAndReset silently did nothing on an empty buffer, so a caller's retry loop spun forever rather than failing. It now throws, as the RESP session already does. MainStoreOps.GETDEL asserted its output was inline immediately after the RMW, before the spill-to-pooled-memory case is handled. Any value large enough to spill tripped it, so GETDEL of a large value failed on every debug build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Enter the chunking fallback when the send buffer is empty The fallback added for over-sized response elements flushed before retrying, but SendAndReset throws when there is nothing to send. That is exactly the over-sized case where the element is the first thing written to the response, so the chunking path was unreachable there and the session still died. Every earlier test emitted an array header first, which left the buffer non-empty and masked it. Flush only when output is actually buffered, otherwise chunk directly. CLIENT GETNAME is the reachable instance: the name is the whole response, so a name larger than the send buffer killed the session. Give the ASCII bulk-string helper the same fallback and route CLIENT GETNAME through it. Move both fallbacks to NoInlining helpers so the 49 call sites of the inlined fast path keep their previous code size. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate latency histograms on first record Every session was given twelve HdrHistograms when its connection was accepted, before it had recorded anything. At two significant digits each rents a 32 KB array from the shared pool, so a connection cost 384 KB of latency monitoring whether or not it ever issued a command. A dump of a server holding several thousand connections showed 887 MB in these arrays. Allocate the histograms for a latency type on the first value recorded into it. A session typically records into three of the six types, and an idle connection records into none, so it now pays nothing. Add --latency-monitor-precision to size the histograms. It defaults to 2, which is what the histograms were built with, so accuracy is unchanged unless it is lowered. Dropping it to 1 shrinks each histogram several-fold in exchange for reporting percentiles to 10% rather than 1% resolution. Fix three defects on the release path that lazy allocation widens the window for. LongHistogram.Return() now drops its reference to the array it hands back, so a later record throws rather than silently writing into an array the pool has re-rented to another session. The session's pending work is cancelled before its histograms are released rather than after. GarnetLatencyMetricsSession publishes its metrics as unavailable before releasing them, and the hot path reads the field once and checks it. Dropping the array reference exposed a use-after-return in GarnetClient, which copied its histogram after Dispose had already handed the array back; it now reports no metrics instead of reading a recycled array. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the latency precision to the histograms it sizes The precision knob was only covered by tests that constructed histograms directly, so a broken binding between the command line and the allocation would have gone unnoticed and silently degraded resolution. Assert the size the server reports through LATENCY HISTOGRAM, the size of the per-session histograms reached through the live sessions, and that the option parses to 2 by default and rejects values outside the 0-5 range HdrHistogram accepts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Wait for the session to record before reading its histograms A session records its latency after the reply has been written, so a test that read the histograms as soon as the reply arrived could find none allocated yet. Poll until one appears. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Publish a process-wide budget for live network buffers The buffer pool's byte ceiling bounds only its idle free list. Buffers checked out by live connections are bounded by nothing, so their footprint is connections times per-connection size, which is what grew unchecked with thousands of clients. Add NetworkBufferBudget, which divides a configured total by the live buffer count and publishes the largest base size that keeps the two consistent. It is a division rather than a control loop: buffer sizes move in factor-of-two steps, so any controller with a narrower deadband cannot converge and limit-cycles between two size classes. Growth therefore requires the quotient to reach twice the published target, matching the step it actuates. One budget is shared by every listener. Each endpoint previously built its own pool and was handed the whole ceiling, so with N endpoints the effective limit was N times the configured value. Only server listener pools participate; replication and migration keep their own unbounded behaviour so that a sync burst cannot drive live client connections down. The target is published but consumed by no allocation site yet. Landing the signal on its own makes a drift in the live buffer count observable in isolation, which matters because the target is budget divided by that count: a count that ratchets up does not degrade gracefully, it collapses the target to the floor permanently. The count rides exactly the two paths that already maintain the pool's live byte accounting and nothing else. Defaults are inert: 1 GB against a 128 KB buffer size means adaptation does not begin until 8,192 live buffers, and the target clamps at the configured size so it can only ever lower it. --network-buffer-memory-budget 0 disables it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new receive buffers from the network buffer budget The budget was published but consumed by nothing. Wire it to the receive side: a new connection's buffer starts at the published target rather than the configured size, and a grown buffer shrinks back toward that target. Only the base size is governed. Demand-driven doubling is untouched, so a connection that needs a large buffer still reaches it; pressure changes what a connection starts and settles at, never what it is allowed to grow to. The allocation size and the shrink floor move together. Clamping only the allocation would leave a connection allocated below the configured size unable to shrink at all, because the shrink path both floors at that size and returns early when the buffer is already at or under it. A pool cannot recycle a buffer outside its size classes, so NetworkBufferSettings gains an explicit minimum. Deriving it from the three configured sizes, as before, gives no way to express "recycle down to 16 KB while new connections still start at 128 KB", and a clamped buffer would have been dropped rather than pooled. The per-level idle entry count derived from a byte budget is capped in the same change, since a smaller minimum size class otherwise raises it proportionally; the cap sits above the value today's defaults produce, so it is inert until the two are combined. Budget accounting is now inert when disabled rather than merely unread. The disabled budget is a shared singleton, so unbudgeted pools were accumulating a count on it that was meaningless but would have become misleading the moment anything read it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new send buffers from the network buffer budget GarnetSaeaBuffer is the single allocation point behind both GarnetTcpNetworkSender call sites, so clamping there covers the send side entirely. Send buffers never grow, so an oversized response is chunked through whatever buffer it was given, and every consumer reads the length off the pool entry rather than off the configured setting -- which is what makes the size safe to adapt. The clamp uses the separate, higher send floor so that a shrunken send buffer does not push custom raw string responses into repeated ArrayPool rentals. Adds an end-to-end test that a 1 MB value and a key larger than the send buffer both survive while the budget is pinned at its floor, and tightens the shrink assertion now that both directions adapt. Neutering the clamp fails that assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized cluster response elements The chunking audit converted the RESP session but left the cluster session on the atomic path, where an element larger than the response buffer cannot be written and SendAndReset has nothing to flush. CLUSTER GETKEYSINSLOT echoes client-supplied keys, so it kills the session for any key larger than the send buffer -- reproducible today at the 128 KB default, independent of the adaptive budget. The gossip reply echoes the serialized cluster config and grows with node count. Give the cluster session the same atomic-first helper the RESP session has: write in place, retry once on a flushed buffer, and chunk only an element that cannot fit an empty buffer, so small elements keep exactly today's path. Route both sites through it. Covered by a new fixture run at both the default send buffer and one floored by the budget; reverting either arm's fix fails it. Also extends the oversized-response fixture to run at a floored send buffer, with a guard asserting each arm actually binds the size it claims to, and corrects a stale comment about the per-level entry cap being inert -- the 16 KB adaptive floor makes it bind at the defaults. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Shrink receive buffers immediately while the budget is binding The idle hysteresis exists so a connection with bursty-but-recurring large payloads keeps its buffer through the burst, and it is raised from 16 to 256 receives to make that hold over a genuinely long quiet stretch. But a countdown that long cannot converge the aggregate when the process is actually short of memory, so shrinking now bypasses it entirely while the budget is binding. The pressure signal is the already-published target having been driven below the configured size, not a live-byte read: it costs one predictable branch on the receive path and is ordered after the read-mostly size comparisons, so an unpressured server evaluates nothing new. Wires the pressure and idle shrink counters that INFO BPSTATS already reports. The new test drives a burst and then ten small rounds -- far short of the 256-receive hysteresis -- and requires the budgeted arm to give the buffers back while the unbudgeted control still holds them. Neutering the pressure signal fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound pressure shrinking so alternating payloads do not churn Shrinking immediately under pressure was wrong. Pressure is sticky -- the published target stays below the ceiling for as long as the connections are live -- so an alternating large/small workload reallocated a pinned buffer on every large request. Measured at 640 shrinks over 640 large/small pairs, i.e. one per request, precisely when the server is most loaded. Apply a short countdown under pressure instead of none. A strictly alternating workload never trips it, because each large receive resets the countdown, while sustained small traffic still converges in eight receives rather than two hundred and fifty-six. The countdown is clamped rather than reloaded so that pressure arriving mid-countdown does not wait out whatever remained of the idle one. Same measurement now reports zero shrinks. Adds a TLS arm, since the incident is thousands of TLS connections and the decrypted transport buffer shrinks through a separate path that the plain-socket test cannot reach. Disabling the transport clamp leaves the grown buffers at their burst size and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Converge live connections onto the adapted buffer size Clamping the allocation size governs new buffers only. A connection established before the budget started binding keeps its full-size buffers: the send side hands them straight back out from its own stack, and the receive side moves them to the pool free list, where they stay pinned. Neither converges, so at incident scale the adaptation applies only to churn. Gate both return paths on the budget actually binding, using the same IsUnderPressure signal the shrink policy shipped with: - LimitedFixedBufferPool.Return drops an over-target entry instead of pooling it. Unpressured it is pooled exactly as before, so short-lived connections handling moderately large payloads keep their reuse. Send and receive adapt to separate floors, so the comparison reads the entry's own buffer type off PoolEntry.source: measuring against the receive target alone would treat a correctly sized send buffer as over-sized and make send buffers un-poolable for as long as the budget binds. Accounting is unaffected -- the drop reuses the existing non-pooled fall-through, below the unconditional live-byte and live-count release. - GarnetTcpNetworkSender.ReturnBuffer disposes an over-target send buffer rather than pushing it back onto the per-connection stack, so the next acquire allocates at the adapted size. Also clear the vacated slot in LightConcurrentStack.TryPop. Without it the array keeps the popped buffer, and through it the pinned pool entry, strongly referenced for the connection's lifetime -- so releasing a buffer freed the accounting but not the memory. Measured over ten connections established while the budget was slack, then driven into pressure by two hundred more: 2.0 MB released with both gates against 1.0 MB with the send gate neutered. Pooled bytes after the same burst: 0 against 10.5 MB with the pool gate neutered, and the free list after connection churn holds 80 KB against 32 KB when the send target is ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address adversarial review findings on the network buffer budget Five findings from two independent reviews, each reproduced before being fixed. Recompute the adapted target inside the live buffer count's own increment and decrement. It previously had a single production caller on the pool's allocate-miss path, which a pooled reuse never reaches, so the target never recovered when connections drained: after a spike, four live buffers still published a 16 KB target and reconnecting did not clear it. Clamp the TLS plaintext send buffer to the send target. It was allocated at the configured size while the pool measured it against the target on return, so under pressure every TLS connection's buffer was dropped and freshly pinned on the next connect. 64 connections now hold 11.5 MB where they held 15.7 MB. Sample the receive shrink signal before the buffer is processed rather than after. Consumption and compaction happen inside Process on the plain path, so a fully consumed large request was read as a zero-byte receive and advanced the shrink countdown: 160 shrinks over 640 alternating large/small pairs, now zero. Route 26 further response sites through the chunking helper. Set operation results and every blocking variant wrote unbounded elements atomically, which kills the session above the send buffer size at the shipped default; BLPOP loses the popped element outright. The new tests fail in both fixture arms, so this is not budget-induced. Drop the aborted-session response release added earlier in this branch. The batch's finally already returns the object on every teardown path, so 200 protocol-aborted sessions conserve the buffer count with and without it, and handing a possibly in-use buffer back to the pool is not worth an unmeasurable fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix stale transport alias, above-max release ordering, and scratch demand tracking Round 2 adversarial review found five defects, two of which were introduced by the round 1 fixes. All five reproduced before being fixed, and every new assertion is mutation-checked. Without TLS the transport receive buffer is an alias of the network buffer, refreshed only at the top of each receive. A resize left the alias pointing at the entry just returned to the pool, so a connection that grew once and then idled kept the old pinned array rooted indefinitely -- the accounting showed the release and the memory never happened. Measured at 64 connections the stale roots exceed 16 MB. Both resize paths now refresh the alias. The above-max immediate release sat behind the demand guard, so a fully consumed oversized request made target equal current and held the out-of-pool buffer until another receive arrived. This regressed C4 via the round 1 demand-capture fix. The branch now runs first and is sized from the residual rather than the pass's demand. CLUSTER SLOTS still wrote the whole serialized topology atomically, so a sufficiently fragmented cluster overran the send buffer with no oversized key involved. The new test fails at today's 128 KB default too, which is what proves it pre-existing. ScratchBufferBuilder only raised its demand high-water when a write did not fit, so a batch that kept filling an already-large buffer reported no usage at all and the shrink policy released it every time. The popped-entry collectability test could pass for the wrong reason under optimization, since the stack itself was collectable once unused. GC.KeepAlive pins it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reset the shrink countdown on the above-max early return The above-max branch returns without shrinking when the buffer is still full of unconsumed data. Leaving the countdown untouched there is the same guard-ordering class as the defect the branch reorder just fixed, so restore the reset the demand guard used to perform. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Redesign session buffer caps as countdown checkpoints The committed high-water design tracked per-command demand on the hot path and cost ~9-11 ns on Network.BasicOperations.InlinePing at batchSize=1. Replace it at both sites with a countdown checkpoint: every batch decrements a counter, and only when it reaches zero does a cold NoInlining method run. Demand is inferred from whether the buffer grew since the last checkpoint rather than tracked per command, so the hot path holds one decrement and one predictable branch and never touches the parse state. This trades bounded churn for zero hot-path tracking: reclaim takes up to two checkpoint intervals, and a session that needs the capacity every batch reallocates at most once per two intervals. Removes BufferShrinkPolicy, SessionParseState.batchHighWater and ScratchBufferBuilder.AdvanceOffset as dead code. Also re-applies the SLOWLOG chunking fix (review round 2 finding 1) for sps.GetString and entry.ClientName, tightens the parse-state retention assertion from a bare Assert.Less that passed with the shrink neutered to a magnitude bound that fails when mutated, and tightens the scratch-buffer equivalence assertion from a 2x slack bound to unbounded + 64 KB. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the unpressured common case against an unbudgeted server The central promise of the budget is that with few connections nothing changes. Two gaps in the coverage left that unpinned. UnpressuredGrownBuffersAreRetainedExactlyAsWithNoBudget runs the same grow- then-small-traffic workload with the default budget and with the budget disabled, and asserts the two hold identical bytes both immediately after growth and after the small traffic that follows. Ungating the pressure clamp in MaybeShrinkNetworkReceiveBuffer fails its retention assertion. UnpressuredConnectionChurnKeepsRecyclingGrownBuffers guards correction C3: dropping an over-target entry on return must be gated on pressure, or connection churn re-allocates pinned arrays that are recyclable today. Ungating the drop in LimitedFixedBufferPool.Return collapses pooledBytes to 128 KB and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the scratch shrink checkpoint from the batch boundary only The countdown interval is counted in batches, but it was decremented from ScratchBufferBuilder.Reset(), which several callers invoke far more often than once per batch. The Lua interpreter resets once per string while decoding a JSON document and once per response through ScratchBufferNetworkSender, so a single cjson.decode of a wide object fired many checkpoints inside one command -- releasing a buffer the very next element re-grows, which is the churn the interval exists to bound. Split the countdown into ResetAtBatchBoundary(), called only from RespServerSession's batch boundary. Plain Reset() is now free again on the per-element paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the Lua script processor's scratch checkpoint from the session The Lua interpreter writes into the scratch buffer of the script cache's inner RespServerSession, not the network session's. That inner session never reads from a socket, so after the batch-boundary change its builder was never checkpointed at all and one cjson.encode of a large value permanently enlarged every session that ran a script. Fold the Lua checkpoint into the existing cold session checkpoint, which already runs once per 64 outer batches, so this costs nothing on the hot path and is definitionally outside any script execution. Rename the countdown trio accordingly, since it is no longer parse-state-only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk the remaining unbounded response sites and cap the Lua reply buffer A full sweep of the 70 atomic write loops in libs/server/Resp and libs/cluster found six writing unbounded data: ACL LIST, ACL USERS and ACL GETUSER echo client-supplied usernames and rule descriptions, and CLUSTER SLAVES, CLUSTER REPLICAS and CLUSTER INFO write topology that grows with the cluster. All six kill the session at today's 128 KB default, which the new tests prove by failing in both fixture arms. The remaining sites write literals, numbers or fixed-width identifiers. The script cache's dummy network sender owns a second scratch buffer that the RESP fallback path for redis.call writes replies into, built uncapped and never checkpointed. It now takes the configured session cap and is checkpointed alongside the request buffer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Clear retained references on pop and on the cluster parse-state copy Three round-3 review findings, all in the same aliasing class the earlier rounds kept surfacing: a reference survives the release that was supposed to free it. SimpleStack.Pop left the vacated slot populated. Its only user stacks ScratchBufferAllocator's outgrown pinned buffers there, so every buffer a session ever outgrew stayed rooted for the life of the connection and a reset freed the accounting but not the memory. The clear is gated on IsReferenceOrContainsReferences so a struct-only instantiation is unaffected. ClusterSession took a copy of the caller's SessionParseState and left it populated after the command finished. The copy holds its own reference to the session's pinned root buffer, so a wide cluster command kept that buffer alive past the owning session's shrink. NetworkBufferBudget.Recompute validated the previous target rather than the count it derived from, so a lost exchange could publish a target derived from a stale count. The retry loop re-derives on a lost exchange and re-reads the count after a won one. This is defensive: the race was not reproduced in 20 runs of a 256-thread oversubscribed churn, and the test that ships pins the invariant rather than discriminating the fix. LatencyHistogramMemoryTests' idle arm was vacuous. GarnetServerTcp needs four bytes before it creates a RespServerSession, so an idle socket never created one and the arm measured a server with no sessions at all -- it would have passed with eager allocation restored. Sessions are now opened with a partial command and the count asserted against INFO CLIENTS, and both assertions are restated as comparisons between the arms rather than against an absolute. Restoring eager allocation now fails three tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make three assertions test what they claim, and correct three doc statements The send-buffer pooling test accepted any 64 KB on the free list, which four pooled 16 KB receive buffers also satisfy -- so it could pass with no send buffer pooled at all, which is the defect it exists to catch. It now reads the per-level entry counts the pool already reports and asserts an entry at the send size class specifically. Ignoring the entry's buffer type in the drop rule leaves it at zero entries and fails. The cluster chunking tests stopped reading at a byte count, which samples only the part of the reply the first chunk boundary has passed and cannot see a lost tail -- the half a chunking fix is most likely to get wrong. Both now pipeline a PING and read to it: RESP replies are ordered, so the sentinel cannot arrive before the whole reply is written. GETKEYSINSLOT additionally asserts the key contents rather than its length header, and CLUSTER SLOTS asserts every one of the 2,000 ranges is present. Three statements were false. GetSessionScratchBufferMaxRetainedSize says unset disables shrinking; unset returns the 64 KB default and only zero disables. MaxEntriesPerLevelCap says it is inert at the shipped defaults; the budget is on by default, so the smallest size class is the 16 KB receive floor and the derived value is 4,096, which the cap binds. defaults.conf still described the high-water design the countdown checkpoint replaced. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the reachable cluster chunking site and count sessions, not sockets Review round 4 found the cluster chunking fixture green with all three of the round-3 cluster conversions reverted, and both latency histogram tests green with the session-opening helper made a no-op. The three round-3 cluster conversions are bounded and unreachable: CLUSTER INFO is a fixed eleven-line string, CLUSTER REPLICAS elements are node lines carrying no slots, and CLUSTER BANLIST entries are a node id plus a number. The reachable caller of the same helper is CLUSTER NODES, whose node line embeds slot ranges, so a fragmented assignment overruns the send buffer at the 128 KB default with no oversized key anywhere. Covered with 4,000 alternating slots, asserting the declared bulk length against the received payload and every range. connected_clients counts accepted network handlers, not sessions, so the latency guard could not distinguish a measured session from a bare socket. CLIENT LIST enumerates the sessions themselves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Stop the Lua reply cap being undone, and re-derive an abandoned budget target Three defects on the same theme: a bound that does not bind. The Lua reply sender re-requests a response window at the full server buffer size on every use, so capping its builder below one window could not save anything -- the checkpoint released the buffer and the next redis.call immediately reallocated it larger than it started, churning a pinned array every checkpoint. The cap is now floored at one window, so it releases only the oversized replies it was added for. The existing test stopped at the shrink and could not see this. ScratchBufferBuilder rounded one byte past the request, so a request for exactly a size class allocated twice it -- and the Lua reply window, at exactly 128 KiB, is one. Every caller that must strictly grow already asks for one byte more than the current length, so exact rounding still grows them. NetworkBufferBudget.Recompute abandoned its target when its attempts ran out, leaving whatever the last contended exchange wrote, derived from a count that had since moved. Since nothing else recomputes, that value stood until the next acquire or release, and at a band boundary it could report no pressure while the budget was binding. It now re-derives from the freshest count. Contention cannot reach that path reliably, so the attempt limit is settable to force it. The ACL chunking tests stopped reading as soon as the oversized username arrived and never checked the tail; they now parse the whole array and require the pipelined PING reply to begin exactly where it ends. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the recompute attempt limit immutable Every connection reads the budget concurrently, so the attempt limit belongs in the constructor rather than in a settable field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Compare the CLUSTER NODES slot set exactly Substring matching is near-vacuous over 4,000 alternating slots: " 2" matches inside " 20" and " 2000", so almost any surviving fragment of the line satisfies it. Dropping every slot below 1,000 from the node line leaves the old assertion green and fails the new one at 3,500 slots against 4,000. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report a malformed reply as an assertion, not a timeout The hand-rolled RESP parsers in the chunking and latency tests assumed well-formed input. A truncated array, an error reply where a bulk string was expected, or a null bulk string surfaced as a fifteen-second receive timeout or an ArgumentOutOfRangeException rather than as a named assertion about the reply. Both parsers now check every bound before slicing, reject a reply whose type byte is not the expected one, and handle a null bulk string by recording it and advancing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Yield to a fresher publisher instead of overwriting it The exhaustion fallback published the target with an unconditional write, so a thread that sampled the live count before another thread published could land afterwards and replace the right answer for a fresher count with its own stale one. The remark argued the race in the wrong direction: it reasoned about being overtaken, not about overtaking. The publication is now a compare-exchange against the target that was observed alongside the count, and it does not retry. The only writer that can beat it also exchanges, so a lost exchange means a value derived from a count no older than this one is already published. The publish step takes the count and the observed target as parameters so the interleaving can be induced directly rather than raced into. Two assertions were weaker than the fixes they named: the size class test exercised the arithmetic helper but never an allocation, and the Lua reply test showed the cap survived another use without pinning it at one response window. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the cluster chunking arm small enough to chunk The CLUSTER NODES body is about 19 KB, which is below both the 128 KB default send buffer and the 64 KB the floored fixture arm actually resolved to for a single connection. Replacing the chunked write with a spinning atomic one left both arms green, so neither covered the path the test is named for. The floored arm's budget drops to 16k so its send target reaches the floor. The mutation now fails there. The default arm cannot be made to cover chunking here at all -- a single node's line is bounded by the 16,384 slots it can hold, which is well under 128 KB -- so the comment no longer claims it does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the target before the count, not after The compare-exchange was doing less than its remark claimed. It proves the target has not moved since this thread read it, which two threads that sampled the same target concurrently both satisfy regardless of whose count is fresher. Reading the target first and the count second makes the stronger statement true: a successful exchange proves no other publisher completed between the target read and the write, and the count was sampled inside that window, so it is no older than the count behind any value published earlier. The ordering is not observable from a single-threaded test, so it ships on the argument rather than on an assertion, as the other reasoning-level fixes in this work do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Abandon a contended recompute instead of publishing from a stale sample The single-attempt fallback had no correct form. An unconditional write let a stale count land after a fresher one; the compare-exchange that replaced it dropped a fresher publication when the staler one won the race; and its hysteresis band tested a target that could already have moved. Each fix traded one wrong interleaving for another. The fallback was also strictly weaker than one more iteration of the loop it backstopped: no post-write count revalidation and no retry. Remove it. Exhausting the attempts now publishes nothing, which is safe rather than resigned. Every write to the target is made by a thread that read both the previous target and the count immediately beforehand, so exhaustion leaves a value some recent count justified. Both count movers move the count before recomputing, so whichever thread invalidates a published target must itself re-derive it. A stale target left too high cannot hide behind the band either: clearing pressure requires the target to equal the ceiling, while the budget binding requires the quotient to be below it, and the band suppresses a write only when the quotient is at least the target. Replace the two tests that pinned the deleted methods with one that pins the ordering the argument rests on, and tighten two reply parsers that accepted malformed framing: any negative bulk length read as null, and a bulk string whose terminator was never checked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Remove the recompute attempt knob and harden two reply parsers Deleting the stale-publish fallback made a zero or negative attempt limit permanently stick the published target, since nothing else republishes on exhaustion. No caller passed the parameter once the two tests pinning the deleted methods were removed, so the constructor parameter is gone and the bound is a constant. A reply carrying its declared payload with no trailing CRLF blocked in Receive until the socket timed out, so the terminator assertion was unreachable. Both parsers now read to a pipelined PING sentinel and parse declared lengths as long, so an out-of-range length reports a named assertion instead of overflowing the bounds arithmetic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count sessions directly instead of parsing CLIENT LIST off the wire The hand-rolled RESP parser behind the session count produced a review finding in four consecutive rounds: an unreachable terminator check, a null-length case accepted too broadly, and twice an overflow in its bounds arithmetic. None were about the property under test. The fixture already enumerates live sessions for another test, so the count now uses that: it opens no probe connection, so it needs no self-discount, and it cannot mis-frame a reply. The remaining array parser compares its declared length by subtraction. Summing the offset, the length and the terminator wraps negative at long.MaxValue, which passes the bounds check and reaches the narrowing cast with an invalid length. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cut the per-receive and per-batch cost the buffer caps added The session buffer caps and the network budget each added work to a path that runs once per receive or once per batch, which at batchSize=1 is once per command. Three changes, none of which alter behaviour: - UpdateNetworkBuffers called MaybeShrinkNetworkReceiveBuffer on every receive, so a connection whose buffer is still at its base size paid a call and a chain of loads to discover it had nothing to do. The entry test is hoisted to the call site and the callee marked NoInlining. - BaseReceiveBufferSize walked networkPool.Budget and two properties on every read. The budget and the configured sizes are cached as readonly fields on the handler, and the clamp is one inlined call on the budget. - The scratch builder and the parse state each counted their own batches. One countdown on the session now drives both, so the batch boundary pays a single decrement regardless of how many buffers are capped, and ScratchBufferBuilder.Reset returns to exactly what it was before. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Release quiesced sessions' latency histograms A session that records latencies and then goes quiet holds its histograms for the connection's lifetime: lazy allocation defers the cost but never returns it, and the monitor's per-window Reset only zeroes the counts while keeping the array. At thousands of connections that is hundreds of MB retained for histograms recording nothing, plus a memset of all of it every sampling interval. The latency monitor already sweeps every session every window and already evaluates the exact predicate needed, so no tracking is added to the record path. After a configured number of consecutive empty windows the monitor drops that type's histograms. The reference is dropped rather than returned to the pool. The record path is deliberately unsynchronised, so a writer racing the monitor can hold the array past the release; dropping leaves that writer with the only remaining reference and its write unobservable, whereas returning would let it write into an array another session has re-rented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the histogram release threshold's hysteresis margin The component test supplies its own threshold, so the configured value was pinned by nothing: setting it to one window left the fixture green. A session issuing requests more slowly than the monitor sweeps must keep its histograms across the isolated empty windows that produces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's timing margin At a two-second request interval against a one-second sweep, an overshoot on a loaded box can manufacture the four consecutive empty windows the test asserts cannot happen. At 1.5 seconds the mutation still fires in three seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's absolute stall tolerance Scaling the sweep and the request interval together keeps the ratio the mutation needs while doubling how long the machine may stall before the test manufactures the consecutive empty windows it asserts cannot occur. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reduce the network buffer config surface from nine flags to two The branch had grown nine new command-line switches. Seven of them were tuning knobs for an adaptive mechanism that sizes itself, exposed because they were easy to expose rather than because an operator needs them. Demoted to properties on GarnetServerOptions, reachable programmatically but no longer settable at startup: --network-buffer-size --network-buffer-min-size --network-max-receive-buffer-size --network-send-buffer-min-size --network-buffer-pool-size --session-parse-state-max-retained-args --session-scratch-buffer-max-retained-size None was CONFIG SET-able, so no live mitigation is lost, and none has shipped, so no operator .conf can reference one. Kept: --network-buffer-memory-budget the process-wide ceiling this work adds, and the only knob whose right value depends on the deployment --latency-monitor-precision a genuine accuracy/memory trade-off with no correct default Renamed for direction-explicit consistency with the pre-existing NetworkSendBufferMinSize: NetworkMaxReceiveBufferSize -> NetworkReceiveBufferMaxSize NetworkBufferMinSize -> NetworkReceiveBufferMinSize NetworkBufferSize keeps its direction-agnostic name: GarnetServerBase already exposes a public NetworkBufferSize for the same concept, so renaming here would diverge from established vocabulary. Removing the defaults.conf key would have changed behaviour rather than preserved it. An unset NetworkBufferPoolSize does not mean "unbounded" -- LimitedFixedBufferPool derives a byte ceiling from its per-level entry bound instead, which at the shipped 16-entry default is ~32.5 MB rather than the 64 MB the standalone server has always run. Options.GetServerOptions() now assigns "64m" directly, so the CLI path resolves exactly as before and embedded hosts keep the derived geometry they already had. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tell a refused connection why, and count it A connection refused by --network-connection-limit was disposed with no error written and counted in no metric, so an operator hitting the limit saw an unexplained reset and had nothing to confirm the cause. Send the Redis-compatible -ERR max number of clients reached before closing, and add rejected_connections to INFO stats. The write is gated to non-TLS. A plaintext error to a peer that has only sent a ClientHello is a protocol violation, so the client reports a handshake failure and the operator is pointed at certificates instead of capacity -- worse than a clean reset. Under TLS the metric is the whole remedy. The counter is incremented only in the limit branch, not in the catch that handles sockets dying during setup, so it stays trustworthy in the incident it exists for. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the connection limit process-wide and settable as maxclients The limit was per-listener, so N endpoints gave N times the configured ceiling -- the same defect this branch fixed for buffers, on the adjacent line of GarnetServer.cs. ConnectionLimit sums the listeners' existing activeHandlerCount values rather than maintaining a second counter, so no new accounting can drift; a connection counter that ratchets fails closed, refusing everything until restart. Default is now 10000, matching Redis, and set on GarnetServerOptions rather than only defaults.conf so embedded and test hosts resolve the same value as CLI hosts. CONFIG SET maxclients pushes the new ceiling through StoreWrapper.Servers into the live accept path. Lowering it disconnects nobody, which is also how Redis treats a lowered maxclients. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the refusal message, the counter, maxclients and the shared ceiling Seven assertions. The fixture holds one persistent probe connection so INFO and CONFIG can still be issued after the server is full, and fills only the remaining slots -- a probe that connected on demand would itself be refused. metricsSamplingFreq is set because rejected_connections is published by the monitor's sampling loop and reads "0" unconditionally without it, which would have made every counter assertion vacuous. Commands go over the wire as RESP arrays: Garnet's inline parsing handles single-token commands but silently produces no reply for a multi-token one, so an inline CONFIG GET returned nothing and the first version of these tests failed for that reason rather than for anything about the limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the shared-ceiling test able to fail The second listener refusing is equally consistent with it having reached a ceiling of its own, so the test now makes it accept a connection first. Also adds the drain-and-recover assertion and asserts several connections past the old ceiling when the limit is lifted, since admitting exactly one more is what raising the limit by one would also do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Build the connection limit on net8.0, and document the maxclients semantics System.Threading.Lock is .NET 9+, so ConnectionLimit did not compile for the net8.0 target the solution also builds. A plain object monitor is what the rest of the server uses and is equivalent here -- the lock is taken only by listener registration at startup, never on the accept path. Also records the three semantics an operator will ask about: replica and cluster peer links count against the limit (as they do in Redis, whose accept path adds getClusterConnectionsCount()); lowering the limit refuses new connections rather than culling established ones; and the value is not clamped to the file descriptor limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix final newlines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover maxclients across listeners and TLS rejection accounting MaxClientsReachesTheSharedLimitFromAMultiListenerHost fills the shared ceiling from two endpoints, raises maxclients at runtime, and asserts both listeners admit again, so CONFIG SET is pinned as reaching every listener rather than the one that happened to be first. TlsRejectionsAreCountedEvenThoughTheClientCannotBeTold pins the other half of the TLS gating: a refused TLS connection is closed without a RESP error, but still increments rejected_connections, which is the only signal an operator has on that transport. The TLS fillers are real TLS clients rather than bare sockets. The handshake runs on the accept path, so a connection that never handshakes stalls the accept loop and is never admitted, which makes bare sockets useless as fillers on a TLS listener. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS accept-path stall with concurrent and partial-handshake cases Upstream now starts the handler without waiting for the handshake, so the fix this branch carried is redundant and is dropped in favour of it. The tests are not: they add a peer that stalls after a partial record header, and sixteen concurrent silent peers, neither of which the upstream regression test reaches. Each asserts on a completed RESP exchange rather than on connect, because a connect against a wedged listener is absorbed by the backlog and succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin that a failed TLS handshake releases its admission slot Starting the handshake off the accept loop moves where a handshake failure is noticed: it no longer propagates out of Start into the accept site's catch, so the slot the accept path already took is released by the handshake's own failure path instead. A leak there would let an unauthenticated peer permanently fill a connection-limited server. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Delete the non-discriminating handshake slot test Three mutations could not break it: removing the observer's Dispose, removing Dispose from the handshake failure path, and holding the peer socket open all left it green. Slot release on a failed handshake is over-determined, so the test observed nothing its subject could change. Replacing the fillers with silent sockets does fail, which is the residual the handshake timeout addresses rather than a property to pin here. Also corrects the ConnectionLimitTlsTests rationale, which cited the accept-loop stall that no longer exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound the per-session Lua script cache with an LRU cap Each cached script holds a whole Lua VM, so an unbounded per-session cache lets a session that runs many distinct scripts accumulate VMs for its lifetime. Cap the cache and evict the least recently used entry. Recency is a monotonic stamp updated in place on each hit, so a cache hit costs one store and eviction scans only at capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the script cache bound, its LRU order, and the evicted-script path Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate transaction key entries unpinned on both paths TxnKeyEntries allocated its initial array with GC.AllocateArray(pinned: true) while the growth path used a plain array, so the pinning was silently dropped at the first resize. Every use of the array is through AsSpan, indexing or Array.Copy, and no caller takes a stable address into it, so the unpinned form the growth path has always used is the correct one. Dropping the pinned initial allocation also keeps these arrays out of the pinned object heap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count and report response buffer overflow rentals ReallocateOutput rents a pooled buffer whenever a response outgrows the buffer it is being written into. That traffic was only inferable, so the amount of overflow a given send buffer size causes could not be observed directly. Adds a process-wide rental count, reported as total_output_buffer_rentals in INFO STATS. It is read straight from the counter rather than from the monitor's sampled global metrics, so unlike the neighbouring rows it stays accurate when no monitor is configured. Measured through the new counter: ordinary responses rent nothing, and an oversized response rents exactly one buffer rather than climbing a doubling ladder, which is the property that makes a smaller send buffer affordable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop trailing newlines to match the repo's final-newline convention Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report the output buffer rental count under BPSTATS rather than STATS The counter is read live from its source rather than sampled by the monitor, so under STATS it was the one row that stayed non-zero when no monitor is configured. MetricsDisabledTest asserts every STATS row reads zero in that state, which is Garnet's default, and it caught this. BPSTATS is where the rest of the buffer instrumentation on this branch already lives and is read live for the same reason, so the counter is consistent with its section there instead of being an exception inside a sampled one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count the refusals a capacity wait legitimately incurs WaitForAcceptance polls until the server has capacity, and capacity returns when the server observes the closes rather than when the client disposes. An attempt made before it does is refused for real and advances rejected_connections, so the two tests asserting an exact count were racing a teardown they do not control. Report the refusals from the wait and add them to the expectation, keeping the assertions exact rather than loosening them to a bound that a build which stopped counting would also satisfy. Only a genuine max-clients reply counts, so an unrelated socket error cannot inflate the allowance and mask a miscount. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Capture the session latency histograms once at every reader The monitor sweep nulls LatencyMetricsEntrySession.latency when a session quiesces, but every reader check-then-dereferenced the field, so nothing guaranteed the two reads saw the same value. Both RecordValue overloads run on the session hot path with no lock by design, and GarnetLatencyMetrics.Merge runs on the disposing session's thread under rwLock while the sweep's reclaim holds only disposeLock, so the two do not exclude each other. Load the field into a local once and work from that. Allocate() now returns the array so the miss path reads the field once too. The type's remark claimed the reclaim was safe because a racing writer holds the only reference to the graph; that is true only if the writer captured it, so the remark now states single-capture as the invariant to preserve. Also gate the shrink counters on the budget being enabled, matching OnBufferAcquired: every pool handed no budget shares the Disabled singleton, so counting there put a contended process-wide interlocked write on the shrink path of connections the budget does not govern. Drop two unintended UTF-8 BOMs, and correct the maxclients documentation. Garnet takes every connection on one listener and cannot tell the kinds apart at accept time, so at the limit it refuses replica and gossip links alike; Redis counts cluster bus links but accepts them on a separate port that applies no limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Name the single-capture invariant at the latency record path The two RecordValue overloads load `latency` into a local exactly once because the monitor thread can null it concurrently. The expression reads as a convenience idiom, so state why it is load-bearing at the point a future edit would touch. The invariant itself is documented on ReclaimIfQuiesced, which is the writer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * State the single-capture requirement on the latency field itself The record-path comments guard the readers that exist. An author adding a new reader passes through the field declaration instead, so the constraint belongs there too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tighten the comments added by this work Cut justifications-against-alternatives and narrative framing from the doc comments and inline comments this change introduces, keeping the constraints and invariants a reader needs in order to change the code correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Restate why Recompute needs no fallback on attempt exhaustion Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Inherit TestBase in the concurrent stack fixture Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the buffer pool's stats directly rather than over the wire Serving an INFO reply takes a send buffer and a receive buffer off the free list the assertion reads, so the probe consumed what it reported and the assertion had no margin against its own instrument. Read the listener's pool directly, and poll the size class the assertion reads rather than the byte total, returning the snapshot every assertion is then taken from. Cover the same rule against the pool directly, where nothing can take an entry off the free list between the return and the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address PR review comments on histogram release, scratch cap, and test sends Release the session latency histograms by dropping the reference rather than returning their counts arrays, so a racing recorder cannot write into an array another session has since rented. `LongHistogram.Release()` does the same for the client, whose `latency` field is readonly. `ScratchBufferAllocator.Reset()` no longer enforces the retained-capacity cap at every batch boundary; a `ShrinkCheckpoint()` on the session's coarse countdown does, mirroring `ScratchBufferBuilder`. The session and transaction allocators are both wired to it, with a guard so a WATCH spanning batches keeps its slices. Test socket sends loop until the whole request is written, and the RESP reply reader reads until the expected length has arrived. `NetworkBufferBudget` documents the budget as an adaptive target on the base buffer size rather than a ceiling on the aggregate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the scratch allocator's checkpoint interval and its outstanding-slice guard A functional WATCH test cannot observe a buffer released underneath a live slice: the pinned array is collectable but its bytes stay readable until a collection reuses them. Both invariants are asserted on the allocator directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Update the scratch allocator tests for the checkpoint release contract The allocator now keeps its current buffer across Reset and releases an over-sized one at a ShrinkCheckpoint, so the fixture asserts the retained capacity at each point and drives two checkpoints to clear the hysteresis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Floor the above-max receive shrink at the largest poolable size The residual left after a fully consumed request is zero, so sizing the replacement from it always resolved to the base. A connection issuing one oversized request per batch then re-grew through every intermediate size class on every request. Floor the replacement at the largest poolable size at both the socket and TLS transport sites; the idle hysteresis path still takes it down to the base once the traffic no longer needs the capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the client latency histogram once at every consumer Release drops the counts array while the reply-processing thread may still be recording, so the record path tolerates a released array and the copy path reads the field once instead of testing IsReturned and then dereferencing. Also make the quiesce-threshold test observe empty windows: it recorded a value and never cleared it, so the windows it asserted over were not empty and a threshold that never restarted its run still passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the released-histogram contract on the record and copy paths Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Derive the copied histogram's tracking values from the counts it carries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Correct the rationale for deriving the copied histogram's total Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS transport above-max receive replacement A TLS connection decrypts into a second receive buffer that grows through its own code path, so the above-max release there was unexercised while its plain-socket twin was pinned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Gate the above-max receive floor on budget pressure The above-max branch releases a buffer the pool cannot recycle. Flooring the replacement at the largest poolable size matches origin/main and saves a connection that needs the capacity on every request from re-growing through every size class, but it must not apply while the budget is binding: the target is derived from a buffer count, so a connection parked at the maximum costs the same single unit as one at the floor and the budget gets no feedback from it. Under pressure the replacement is now the adapted base, which the pool keeps and the budget can see. Also capture the counts array once in ClearCounts and CopyCountsInto, read the client call count off the snapshot the percentiles come from, and record the single-read contract on the field itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS half of the above-max pressure gate The plaintext and TLS receive paths gate the above-max release independently. Only the plaintext site was exercised, so the TLS gate at NetworkHandler.cs:801 could be removed with the whole fixture still green. Parameterize the test over both transports. Guard the counts array in CopyCountsInto, matching the single-read invariant documented on the field and the five sibling capture sites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop the scratch retention assertion that measures test ordering `ScratchBufferCapDoesNotChangeLargeArgumentResults` compared a per-session memory figure across two arms and failed intermittently. Two measurements show it cannot discriminate: swapping the arms moves each figure by about 130 KB per session and flips the sign of the difference, because the arm that runs second also measures the residue of the server instance the first one disposed; and releasing the buffer at every reset rather than at the checkpoint -- the churn such an assertion claims to catch -- moves it by less than that, because churn leaves garbage the forced collection reclaims rather than retention. The test keeps both arms and the reply validation they drive, which is what its name has always claimed, and the remarks state why no magnitude is asserted. `NetworkBufferBudget` documents the per-direction floors as the third reason the budget is a target rather than a ceiling: once the target reaches them, the aggregate grows with the connection count again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Sample the shrink churn either side of the batch boundary Comparing capacity across iterations cannot see the pathology the test exists to catch: a policy that released on every batch regrows before the next sample, so consecutive post-boundary capacities are equal and the churn counts as zero. Sampling before and after each boundary counts every release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Say what the latency precision number actually buys The setting is HdrHistogram significant digits: relative accuracy at every magnitude, not a fixed time unit. Stating it that way, with the memory each step costs, lets an operator pick a value without knowing HdrHistogram. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Pin that an over-sized request is accepted whole The receive buffer grows to fit and maxReceiveBufferSize bounds only what a connection retains, so a chunk far above it must still be accepted. Dribbled in small slices so the request sits partially buffered across many receive passes, where the above-max shrink runs against a live partial request. Also drop a qualifier from the latency precision help text. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Keep the byte-order marks as main has them Earlier edits flipped the BOM on seven files, which showed up as a one-line change to each file header with no content behind it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Wait for the buffer release instead of racing it A receive releases its buffer in UpdateNetworkBuffers, which runs after Process has already written the reply, so a client that has read its response can still be ahead of the release. Sampling BPSTATS once at that point read the pre-release numbers on a loaded runner and failed with the full oversized buffer still live. Poll until the counters settle. The wait uses only the out-of-band INFO connection, so it does not hand the connection under test the extra receive that would let a buffer released one pass too late off the hook: suppressing the immediate release still fails the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Fold the two receive-buffer shrink paths into one policy The network and transport receive buffers ran the same three-regime shrink policy over their own occupancy and countdown, written out twice. Extract it as TryPlanReceiveBufferShrink, which owns the countdown bookkeeping and the budget's shrink accounting, and leave each caller with nothing but the buffer swap. Rename both entry points to TryShrink*, and give the transport buffer a ShrinkTransportReceiveBuffer mirroring ShrinkNetworkReceiveBuffer. Drop the cached configuredReceiveBufferSize/configuredSendBufferSize fields and read them off networkBufferSettings, as maxReceiveBufferSize already is on the same paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Keep the configured base sizes cached on the handler BaseReceiveBufferSize is read on every receive, so reaching the configured size through networkBufferSettings puts a dependent load on that path. BDN Network.BasicOperations.InlinePing measures it at +1.1ns (89.8 -> 90.9) on net10 and +1.9ns (85.7 -> 87.6) on net8, the net8 range clearing baseline entirely. Restoring the fields returns both to baseline (89.8 / 85.8). The shrink consolidation is unaffected and stays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Capture the metrics array in ReclaimQuiescedHistograms Every other method on the session captures the metrics field once before dereferencing it. ReclaimQuiescedHistograms re-read it for the null check and again inside the loop, so it relied on the dispose lock for that safety where its peers do not. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Checkpoint the transaction scratch allocator during AOF replay Replayed transaction procedures run their prepare phase against the watch API, and every watched key is copied into the transaction scratch allocator. The replay session never reaches the network batch boundary that drives the shrink checkpoint, so the buffer one wide procedure grew stayed pinned for the lifetime of a replica. The replay path now reports a boundary per AOF record and the window is counted in TransactionManager, so the caller does not carry the cadence. The allocator shrinks only when nothing is outstanding, so a boundary reported mid-transaction is safe. Also apply two review nits: a u8 span literal for the max-clients error instead of a materialized array, and a corrected comment on the connection-limit registration lock, which was crediting the lock with reader safety that the volatile publication provides. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Release the session scratch allocator at the AOF replay boundary Replayed procedures grow both scratch allocators, not just the transaction one: the prepare phase copies watched keys into the transaction allocator, and the procedure builds its arguments from the session allocator. The session allocator is reset at the start of every procedure rather than when one ends, so a completed procedure leaves it oversized with a non-zero offset, and a shrink checkpoint declines in that state. Reset it at the replay boundary, as the network batch boundary does, and only outside a transaction, whose slices are still live. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Rename the buffer shrink hook off "checkpoint" and count replay records at the call site "Checkpoint" is load-bearing in this codebase: Tsavorite uses it for durable, versioned, recoverable snapshots, and Garnet's AOF replay handles genuine CheckpointStartCommit markers a few lines from where this hook was called. The scratch-buffer hook means something unrelated -- periodically sample a buffer's capacity and release it if it did not grow -- so it is renamed to Trim, matching ArrayPool<T>.Trim, which applies the same hysteresis. The replay hook also counted its own interval while the network one had the caller count, so TrimAfterReplayedRecord ran on every record but trimmed on one in sixty-four. The counter moves to AofReplayContext, alongside the session it governs, so both paths show the cadence at the call site and TrimReplayBuffers means "trim now". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 --------- Co-authored-by: badrishc <badrishc@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 | 1 个月前 | |
Tedhar/dev fixes (#1662) * Fix issue 1630 Move CurrentRepeatCount output to AllureTestBase Add SetupFixture to list running tests on unhandled exceptions * format * Allow turning off AOF,ACL params to speed up BDN Add logRec.TrySetInlineValueLength; current use improves perf of MainSessionFunctions.InPlaceUpdateNumber when logRec.IsPinnedValue, and other areas will be modified accordingly. Minor improvement to performance related to RecordSizeInfo * Improvements to TrySetPinnedValueLength and InPlaceUpdateNumber * Tweaks to TrySetPinnedValueLength * More WIP on RawStringOps perf * fixes to TrySetPinnedValueSpan calls with ETag/Expiration * Clean up Tsavorite BDN tests for better debugging and Diagnosers RecordDataHeader and RecordSizeInfo perf tweaks * Fix Tsavorite BDN ISF implementation * Perf tweak in RecordSource; more streamlining in Tsavorite BDN * More perf tweaks: Make HandleOperationStatus NoInlining and InternalRead AggressiveInlining * Make YCSB key size consistent with main * Propagate perf changes through all IPU, including on-demand creation of RecordSizeInfo as needed * format --------- Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com> | 6 个月前 | |
Remove .NET 8 support (#2230) * Remove .NET 8 support Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve multi-framework release and benchmark infrastructure Keep release packaging and publish paths framework-aware while targeting only .NET 10. Retain benchmark job selection for future runtimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Minimize .NET 8 release-script removal Retain the original publish and archive structure, removing only .NET 8 work while keeping the framework-aware archive loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve existing release-script formatting Restore the original whitespace on unchanged .NET 10 publish commands and archive staging lines so the PR diff only shows the required .NET 8 removal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark framework options while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark README update to .NET 10 commands Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave sorted-set implementation unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave GeoHash unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while dropping .NET 8 entries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Revert publish-profile conditional path changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark config note to .NET 10 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark overview change limited to .NET 10 default Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove redundant benchmark framework exclusion guard Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Load benchmark issuer certificate without warning suppression Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 Tsavorite warning properties Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Fix .NET 10 formatting in Vector Set key accessors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 feature guards Drop conditional branches now that .NET 10 is the sole target framework. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep ref struct anti-constraints on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep alternate comparer interfaces on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep vector read batch ref struct declaration on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e --------- Co-authored-by: Tiago Napoli <tiagonapoli@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e | 17 天前 | |
Use cryptographically random values where required (#88) * Make cluster connection identifiers cryptographically random * Make Generator.cs generate cryptographically random values * Make SortedSetObject.SortedSetRandomMember cryptographically random * Make HashObject.HashRandomField cryptographically random * Make SetObject.SetPop cryptographically random * Fix Generator.cs regression | 2 年前 | |
LogRecord and SpanByte changes (#1186) * Refactors prior to storage-v2: - Move CountdownWrapper to its own file - Rename Lockable* to Transactional* - Rename Transient* to Ephemeral* * WIP; first cut at LogRecord etc. implementations * WIP: more LogRecord * More *LogRecord, and DiskRecord * WIP: Convert allocators to new LogRecord and DiskRecord * WIP: incorporate LogRecord into ISessionFunctions * More logRecord WIP * More WIP on MainSessionFunctions RMWMethods (and PrivateMethods) with LogRecord * WIP LogRecord (MainStore RMWMethods) * WIP: Consolidate to a single LogRecord; convert InternalRMW * TKey removal * A few more TKey removals * WIP more logRecord * Remove pendingConterxt.recordInfo as it is no longer in RecordMetadata Other work to get CreateNewRecordRMW clean * WIP; Internal(RUMD) converted. * added PendingContext implementation of ISourceLogRecord * Finish ContinuePending * Iterator changes - consolidate to RecordScanIterator * Mostly IO work * WIP: Reintroduce TValue through ObjectAllocator, LogRecord, etc. * WIP Convert Garnet to LogRecord and ObjectAllocator (MainStoreFunctions first pass done) * WIP converting Garnet to non-ref SpanByte-only keys * More WIP on Garnet migration to LogRecord * WIP Tsavorite LogRecord odds and ends * Rename TsavoriteLog to TsavoriteAof * More processing-layer LogRecord etc. fixes * fix iterator as ISourceLogRecord * More Expiration etc. cleanup in processing layer * WIP BlockAllocate work * FInalize removal of SpanByte.Metadata * More param fixes in processing layer * SpanField and OptionalFieldsShift structs * Propagate some SpanField etc. changes * First pass at LogRecord-based Revivification * Finish OverflowAllocator * OK now I finished this iteration of OverflowAllocator * improve callers' use of OverflowAllocator * More LogRecord-related cleanup * // TODOMigrate: temporarily clear errors related to Expiration in Key Migration * Fixes for first test runs * More fixes for CopyRecord and setting of Expiration etc * Cleanup OversizeAllocator.BlockHeader and usage * SpanField and LogRecord documentation and cleanup; BlockHeader GetUserSize * More improvements to overflow allocator and usage * More fixes for tests * Remove unneeded IAllocator(Callbacks) methods (superseded by LogRecord) * more overflow allocator fixes * sizeInfo on ISessionFunctions and TrySetValueSpanLength improvements * Remove SpanByte serialized usage (but leave an assert on .Serialized in case I missed some). Also fix CopyUpdater copies from old record and some more OverflowAllocator tweaks * Improved FreeList handling and ObjectIdMap growth/FreeList via SimpleConcurrentStack and TailAndLatch * Make sure DiskLogRecord Value and Record lengths are long * Remove ISerializer (not used and not planned to be) * TIghten up SpanByte !IsSerialized assert and fix SETEX "expiration set mismatch" * Some TODO cleanup * Begin converting Tsavorite tests and benchmarks to LogRecord * Tsavorite RevivificationTests and ObjectTests * More ObjectIdMap and OverflowAllocator work and tests * More overflowAllocator work: FixedSizePages and other fixes for YCSB, add tests and fixes for OversizePages * Clean up Locking.md (had some referenced to obsolete locking modes); start of LogRecord-related docs * support for "inline struct" as SpanByte for Object value optimization * finish "inline struct" testing; update processing layer to Tsavorite changes; update LogRecord docs * Convert RecordFieldInfo Key/Value TotalSize to DataSize - caller should not need to know about the length prefix (especially bc it changes between Inline, Overflow, ObjectId) * Fixes from merging main * More testing and fixes * More merge fixes * fix more tests * minor comment cleanup * some doc-related updates, mostly comments * Add DisposeReason.Deleted, .Elided Add YCSB --di (Delete and reInsert) option * WIP Span<byte> * WIP Span<byte> * WIP Span<byte> * WIP Span<byte> * WIP Span<byte> * WIP Span<byte> * WIP Span<byte> * WIP Span<byte> (Tsavorite builds, with a couple hacks) * WIP Span<byte>: YCSB fixes * WIP Span<byte>: more fixes found during YCSB * Switching CTT, CTRC, TempKv insertions to TSourceLogRecord direct copies (eliminating need for ISessionFunctions calls for those operations) * WIP Span<byte>: more cleanup in Compaction (no TSessionFunctions needed), other fixes related to prior changes * WIP Span<byte>: LogRecord revisions in functions and prep Migrate/Repl for change to LogRecord based * ISessionFunctions revisions - Rename SingleWriter() => InitialWriter(), ConcurrentWriter() => InPlaceWriter() - Rename SinglecDeleter() => InitialDeleter(), ConcurrentDeleter() => InPlaceDeleter() - Consolidate SingleReader() and ConcurrentReader() to just Reader() * Remove TInput, TOutput, TContext from Compact and PrepareIOForConditionalOperation chain * Remove WriteReason as it is no longer needed with Compaction, CopyToTail, and CopyToReadCache now being fully internal operations. * Missed a few WriteReason * WIP Span<byte>: Cleanup DiskLogRecord deserialization and remove some no-longer-needed allocator functions that are now done by DLR * Varebyte in DiskLogRecord * More LogRecord testing * FInish DiskLogRecord serialization tests * Rename SpanField to LogField * Various cleanup, including PrepareForRevivification * WIP on converting Replication and Migration to LogRecord * WIP on Repl (done-ish) and Migrate (mostly done except ISF) conversion to DiskLogRecord * Split IHO.Size into .MemorySize, DiskSize Almost done with Migrate changeover to LogRecord * Finish LogRecordization of Migrate * First fixes for Migration testing * More Migrate testing; use SBAM.MemorySpan * SImple Migrate tests now run * Simple diskless replica runs * Fixes for in-memory tests * Add UpsertValueSelector * Remove PendingContext.IsAsync (a remnant of earlier, since-removed async APIs) from Tsavorite * Cleanup Read initialization of PendingContext * Fix record alignment in YCSB test * tune HashBytes * Fix initial inserting in YCSB * Update LogRecord.md and some cleanup * Restore TsavoriteAof naming back to TsavoriteLog * Add changes for AddressType in logicalAddresses and enable minimal IO for SpanByteAllocator. The AddressType change is a breaking on-disk format change: it shuffles bits around in RecordInfo to add an additional bit adjacent to the old ReadCache bit to mark an address as: - 00: Reserved - 11: ReadCache - 10: InMemory portion of the main log - 01: On-Disk * Fix a few more cursor tests (int => long) * Fixes for review comments (except for "ref RecordSizeInfo") * TSourceLogRecord and RecordSizeInfo parameters are readonly so make them "in" rather than "ref" * Fix some missing ref => in for TSourceLogRecord * Merge fixes; MigrateKeysWithObjects test still fails * Fix Object store migration typo * format * ObjectAllocator serialization - not yet complete * Garnet Storage v2 (#1384) * wip * wip * wip * Added unified store session * Correcting generic typing * Added MEMORY USAGE + TYPE to unified ops * Added TTL, EXPIRETIME and EXISTS to unified ops * implemented DEL in unified ops * wip - expire & persist (broken) * wip - adding expire to unified ops * wip - expire * add cref to server-side replication inter-node commands * fix server-side BeginRecoverReplica * wip * Fix transaction key locking * format * Some test fixes * Fixing tests * reverting a couple of unnecessary changes * Eliminating more multi-context methods from API * Removed some unnecessary stuff * Some more cleanup to TransactionManager * merge tedhar/storage-v2 (ObjectAllocator serialization updates) * Updating memory usage values * format * Handling wrong type ops * Revert "Updating memory usage values" This reverts commit 88ba30721a18cc89bece54983459f7a31b30621e. * fix no-object-log case * Fixes for Tsavorite UTs * Fixes for Tsavorite UTs (mostly ReadCache, TsavoriteLog, Compaction) * Tsavorite Iterator work and UT re-enabling * Fixes to Object iteration, LogRecord.ToString() * Add RecordMetadata.ETag * Readding --no-obj config * fix * test fix * Prep for Recovery * wip * wip * fix * More fixes for UT (mostly Recovery, Migration) * Moving DELIFEXPIM to unified store * ObjectLogTail in Recovery, and more UT fixes (Migration record serialization, ReadCache size and tailAddress verification calculations, etc.) * More Tsavorite recovery tests * Removing unnecessary isObject flag from record serialization --------- Co-authored-by: Vasileios Zois <vazois@microsoft.com> Co-authored-by: TedHartMS <15467143+TedHartMS@users.noreply.github.com> * Removing unnecessary output flag * Fixes for StateMachineTests (Recovery) * Fixes, mostly for Replication to include object log for UnifiedStore; correctly initialized SegmentSize for ObjectLogDevice; other fixes to UTs to work with UnifiedStore * fixes * Fixes for RESP UTs for Unified; clear logRecord on deserialization error * Remove SerializedSize(IsExact) from IHeapObject; various UT fixes * All Cluster.test now passes * small fixes * Fix TailAddress from Rescovery not being PageHeader-aligned * Move RENAME to Unified Store and modify for LogRecord * Fixing rename with etag + removing remaining unnecessary GarnetObjectTypes * Add ITsavoriteRecordObserver and fix Readcache size tracking * Convert LogRecord to use RecordLength * More work on LogRecord.RecordLength * Finish LogRecord.RecordLength * Remove ObjectDb* from Sample*Txn * MultiLevelPageArray stress fixes; improved LogRecord and RecordDataHeader ToString() * Fix GetObjectLogSegmentSize() * update docs * Fix a couple comments * Trim ObjectIdMap arrays on ClearPage * wip * wip * wip * wip * wip * wip * wip * wip * formatting tsavo * Fixing website build error * some renames * small rename * some small changes * Version update * Address review comments; implement OA Truncate; add --value-overflow-threshold configuration option * ObjectAllocator testing * Add large (multi-GB) object tests (and fix associated issues, mostly object-log segments); PendingContext is no longer itself a TSourceLogRecord; * readcache fixes * Move KVSettings from GarnetServer to the individual Database, so we can dispose the log devices. Remove KVSettings from CachedSizeTracker as it was unused Fix some Tsavorite LogRecord-lifetime issues on scan and conditional insert/push * WIP to fix replica for object log segments * Fixes for ObjectLog recovery * Some additional ObjectRecovery test conversion, not ready yet * Convert and re-enable more Tsavorite ObjectRecovery tests. Fix CacheSerializedData to not overwrite if CopyUpdater already set the object Fix foldover checkpoint storage of object log tail * format * A couple fixes for DiskLogRecord serialization * Fix SimpleIncrementByFloatWithOutOfRangeFloat exception verification to use Throws.TypeOf<>().Or.TypeOf<>()) Make SpanByteRcMultiThreadTest [Explicit] until I figure out the request_key issue * Address review comments Fix merge error that resulted in Cluster timeouts Minor cleanup * format * Comment out [Repeat] --------- Co-authored-by: Tal Zaccai <talzacc@microsoft.com> Co-authored-by: Vasileios Zois <vazois@microsoft.com> | 10 个月前 | |
Bugfix: MurmurHash3: 1) wrong suffixLen; 2) should cast to ulong before shift (#1864) | 4 个月前 | |
Add cluster benchmark support to Resp.benchmark (#1906) * wip; add cluster bench * split ClientRequestProvider * nit; cleanup * generate overlapping workload for same shard clients * fix offline bench rung * add different reporting format for --online * change bench summary * improve reporting * add tpt tracking in GB/s * improve verbosity of loading * fix load to improve hit rate * add dbsize load validation and hit rate reporting * fix encoding for dbsize validation * add first stage of issuing workload to replicas * implement load on cluster with replicas * hookup dual connection workload for offline and online * cleanup and better parameter presentation * implement worker-pool infrastructure for cluster-bench * implement online and offline with workerpool for cluster bench * finalize fixed * ref cleanup fix * pipeline implementation * batch send across connections * improved clarity on benchmark config output * fix pool set issue * fix data tpt accounting * added bench parameters * add support for MSET/MGET * fix pool with MSET/MGET * add support for different client types in offline * fix larger batches bug * change pipeline to broadcast * add batchsize => pipeline alias * refactor and cleanup * cleanup and refactoring to simplify offline * add support for mixed workload for offline * formatting * add mixed workload support for worker pool * broadcast in parallel across primary and replica * fixed mixed workload * fix formatting * use shared workload for offline * set replica-ops-percent=0 * pick randomly from shared request buffer per shard * use random seed per process * fix --online cluster-bench to utilize --op-percent and --op-workload parameters * fix issue with --online where ExecuteBatch was necessary * add benchmark completion marker * add zipf distribution option for cluster-bench | 2 个月前 | |
Add Unix socket support (#944) * Add Unix socket support * More cluster test adjustments * Remove warnings * dotnet format whitespace * fix merge * fix merge * Add very basic test for the unix socket support * Try fix long-path error & sync GarnetClientSession connection logic with GarnetClient * Dispose the GarnetClient objects in tests * Sync LightClient connection logic with GarnetClient(Session) * fix * Add exception message to StoreWrapper.GetIp * Fix merge * Add TLS test path * Add more Unix socket tests with SE.Redis as client * Add config parsing for --unixsocketperm and tests * attempt fix import ordering * Set unix socket permission after listener is bound * attempt fix import ordering * attempt fix import ordering * attempt fix import ordering * use ILogger in LightClient --------- Co-authored-by: Vasileios Zois <96085550+vazois@users.noreply.github.com> | 1 年前 | |
Reuse Large Buffers in MigrateSession (#623) * add buffer pool to migration manager * expose separate receive and send buffers from NetworkHandler * add network buffers wrapper to gcs * push NetworkBuffers to NetworkHandler * add networkBuffers instance in migration manager * add constructor * reuse buffer for replication communication * consolidate network buffer pool * nit * add gc collect for migration buffer pool * add ACL test for MIGRATEGC * make migrategc management commands * add verbose logging for IOCallback * expose maxEntries from NetworkBuffers.Allocate * configure GarnetClient for Failover * cleanup ReplicationManager * Cancel cts before disposing in AofSyncTask * consolidate buffer pool purge under a single command * purge without disposing pool * introduce info bpstats metrics * ensure shared bp uses correct allocation size for send and recv * fix bp stats call * change format of info bpstats * fix out of bounds level request for LFBP * introduce purge for server buffer pool * move PURGEBP in server namespace * fix migrate bench keys option * fix formatting * add outOfBound allocation request metric * augment migrate bench * separate pool definition from buffer spec * rename to NetworkBufferSettings * rename NetworkBuffers to NetworkBufferSettings * revert dispose order * revert replication networkSettings * add timeout to dispose of LFBP * add timeout in cluster test TearDown * addressing review comments * add constants for NetworkBufferSettings in ReplicationManager * rename types for purgebp * rename to createBufferPool * bump Garnet version * add comments to LFBP * remove unused variables | 2 年前 | |
Initial commit | 2 年前 | |
Bound per-connection memory: adaptive network buffer budget, session buffer caps, and connection admission (#2157) * Bound connection-scaled network buffer memory Per-connection network buffers were unbounded in practice: the receive buffer only shrank when it exceeded maxReceiveBufferSize, so a buffer that doubled 128K -> 256K -> 512K never shrank and every connection permanently retained its all-time peak. The TLS transport buffer had no shrink path at all. GarnetServerTcp also discarded its networkBufferSize argument and hardcoded 128 KB, so the per-connection floor was unconfigurable. - Add --network-buffer-size, --network-max-receive-buffer-size and --network-buffer-pool-size, plumbed through GarnetServerOptions into GarnetServerTcp, which now honors them. Defaults match today's behavior. - Shrink a grown receive buffer to the smallest size class that keeps 2x headroom over the buffered bytes, after 16 consecutive receives that fit. Buffers above maxReceiveBufferSize skip the hysteresis since the pool cannot recycle them. - Add the same shrink path for the TLS transport receive buffer, called only from the reader sites that already consider doubling. - Track liveBytes, peakLiveBytes, pooledBytes and a shared byte budget in LimitedFixedBufferPool, exposed through INFO BPSTATS. - Return the outstanding responseObject in DisposeNetworkSender so a session torn down mid-response does not leak its PoolEntry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Document new network buffer options and add config parsing tests Adds the three --network-buffer-* options to the configuration table, warns instead of silently correcting an inverted size pair, and covers defaults, overrides, power-of-2 rounding, the inverted pair and invalid values in GarnetServerConfigTests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bound per-session parse-state and scratch buffers A RESP session sizes SessionParseState.rootBuffer to the largest array arity the client has ever sent and keeps it for the life of the connection. One high-arity command therefore enlarges the session permanently, and the cost scales with connection count. The same ratchet exists in ScratchBufferBuilder, whose Reset() zeroes the offset but never releases the backing array. Add BufferShrinkPolicy, a shared cap-plus-hysteresis helper, and apply it at both sites. A buffer above the cap is released only after the configured number of consecutive batches that did not need the extra capacity, so a session with recurring large requests keeps its buffer. Both caps are configurable and default-on: --session-parse-state-max-retained-args (default 1024) --session-scratch-buffer-max-retained-size (default 64k) Measured end to end with a forced compacting GC, 200 sessions each issuing one 20,000-argument EXISTS followed by sustained single-key work: 327 KB/session retained before, 98 KB/session after. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover buffer shrink against transaction and watch aliasing The batch-boundary shrink releases a pinned array, so anything holding a reference into it across that boundary would dangle rather than merely read stale bytes. Two paths cross the boundary by design: a MULTI..EXEC window, which re-parses queued commands from the network buffer, and WATCH, which copies key bytes into the transaction scratch allocator so they outlive the receive buffer. Add tests that force the shrink to fire inside both windows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized response elements instead of killing the session A response element larger than the send buffer cannot be written atomically. The chunking helper for this already exists and SCAN uses it, but several commands still wrote unbounded user data through the atomic path, where TryWriteBulkString cannot make progress and SendAndReset throws, killing the session mid-response. KEYS fails today with any key larger than the 128 KB default send buffer. Give the shared WriteBulkString helper an atomic-first fallback: write in place, retry once on a flushed buffer, and only chunk an element that cannot fit an empty buffer. Small elements keep exactly today's path, so multi-element replies pay nothing. Route KEYS, COMMAND GETKEYS, COMMAND GETKEYSANDFLAGS and the pub/sub channel and pattern echoes through it. Two related defects found while covering this: ClusterSession.SendAndReset silently did nothing on an empty buffer, so a caller's retry loop spun forever rather than failing. It now throws, as the RESP session already does. MainStoreOps.GETDEL asserted its output was inline immediately after the RMW, before the spill-to-pooled-memory case is handled. Any value large enough to spill tripped it, so GETDEL of a large value failed on every debug build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Enter the chunking fallback when the send buffer is empty The fallback added for over-sized response elements flushed before retrying, but SendAndReset throws when there is nothing to send. That is exactly the over-sized case where the element is the first thing written to the response, so the chunking path was unreachable there and the session still died. Every earlier test emitted an array header first, which left the buffer non-empty and masked it. Flush only when output is actually buffered, otherwise chunk directly. CLIENT GETNAME is the reachable instance: the name is the whole response, so a name larger than the send buffer killed the session. Give the ASCII bulk-string helper the same fallback and route CLIENT GETNAME through it. Move both fallbacks to NoInlining helpers so the 49 call sites of the inlined fast path keep their previous code size. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate latency histograms on first record Every session was given twelve HdrHistograms when its connection was accepted, before it had recorded anything. At two significant digits each rents a 32 KB array from the shared pool, so a connection cost 384 KB of latency monitoring whether or not it ever issued a command. A dump of a server holding several thousand connections showed 887 MB in these arrays. Allocate the histograms for a latency type on the first value recorded into it. A session typically records into three of the six types, and an idle connection records into none, so it now pays nothing. Add --latency-monitor-precision to size the histograms. It defaults to 2, which is what the histograms were built with, so accuracy is unchanged unless it is lowered. Dropping it to 1 shrinks each histogram several-fold in exchange for reporting percentiles to 10% rather than 1% resolution. Fix three defects on the release path that lazy allocation widens the window for. LongHistogram.Return() now drops its reference to the array it hands back, so a later record throws rather than silently writing into an array the pool has re-rented to another session. The session's pending work is cancelled before its histograms are released rather than after. GarnetLatencyMetricsSession publishes its metrics as unavailable before releasing them, and the hot path reads the field once and checks it. Dropping the array reference exposed a use-after-return in GarnetClient, which copied its histogram after Dispose had already handed the array back; it now reports no metrics instead of reading a recycled array. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the latency precision to the histograms it sizes The precision knob was only covered by tests that constructed histograms directly, so a broken binding between the command line and the allocation would have gone unnoticed and silently degraded resolution. Assert the size the server reports through LATENCY HISTOGRAM, the size of the per-session histograms reached through the live sessions, and that the option parses to 2 by default and rejects values outside the 0-5 range HdrHistogram accepts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Wait for the session to record before reading its histograms A session records its latency after the reply has been written, so a test that read the histograms as soon as the reply arrived could find none allocated yet. Poll until one appears. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Publish a process-wide budget for live network buffers The buffer pool's byte ceiling bounds only its idle free list. Buffers checked out by live connections are bounded by nothing, so their footprint is connections times per-connection size, which is what grew unchecked with thousands of clients. Add NetworkBufferBudget, which divides a configured total by the live buffer count and publishes the largest base size that keeps the two consistent. It is a division rather than a control loop: buffer sizes move in factor-of-two steps, so any controller with a narrower deadband cannot converge and limit-cycles between two size classes. Growth therefore requires the quotient to reach twice the published target, matching the step it actuates. One budget is shared by every listener. Each endpoint previously built its own pool and was handed the whole ceiling, so with N endpoints the effective limit was N times the configured value. Only server listener pools participate; replication and migration keep their own unbounded behaviour so that a sync burst cannot drive live client connections down. The target is published but consumed by no allocation site yet. Landing the signal on its own makes a drift in the live buffer count observable in isolation, which matters because the target is budget divided by that count: a count that ratchets up does not degrade gracefully, it collapses the target to the floor permanently. The count rides exactly the two paths that already maintain the pool's live byte accounting and nothing else. Defaults are inert: 1 GB against a 128 KB buffer size means adaptation does not begin until 8,192 live buffers, and the target clamps at the configured size so it can only ever lower it. --network-buffer-memory-budget 0 disables it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new receive buffers from the network buffer budget The budget was published but consumed by nothing. Wire it to the receive side: a new connection's buffer starts at the published target rather than the configured size, and a grown buffer shrinks back toward that target. Only the base size is governed. Demand-driven doubling is untouched, so a connection that needs a large buffer still reaches it; pressure changes what a connection starts and settles at, never what it is allowed to grow to. The allocation size and the shrink floor move together. Clamping only the allocation would leave a connection allocated below the configured size unable to shrink at all, because the shrink path both floors at that size and returns early when the buffer is already at or under it. A pool cannot recycle a buffer outside its size classes, so NetworkBufferSettings gains an explicit minimum. Deriving it from the three configured sizes, as before, gives no way to express "recycle down to 16 KB while new connections still start at 128 KB", and a clamped buffer would have been dropped rather than pooled. The per-level idle entry count derived from a byte budget is capped in the same change, since a smaller minimum size class otherwise raises it proportionally; the cap sits above the value today's defaults produce, so it is inert until the two are combined. Budget accounting is now inert when disabled rather than merely unread. The disabled budget is a shared singleton, so unbudgeted pools were accumulating a count on it that was meaningless but would have become misleading the moment anything read it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new send buffers from the network buffer budget GarnetSaeaBuffer is the single allocation point behind both GarnetTcpNetworkSender call sites, so clamping there covers the send side entirely. Send buffers never grow, so an oversized response is chunked through whatever buffer it was given, and every consumer reads the length off the pool entry rather than off the configured setting -- which is what makes the size safe to adapt. The clamp uses the separate, higher send floor so that a shrunken send buffer does not push custom raw string responses into repeated ArrayPool rentals. Adds an end-to-end test that a 1 MB value and a key larger than the send buffer both survive while the budget is pinned at its floor, and tightens the shrink assertion now that both directions adapt. Neutering the clamp fails that assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized cluster response elements The chunking audit converted the RESP session but left the cluster session on the atomic path, where an element larger than the response buffer cannot be written and SendAndReset has nothing to flush. CLUSTER GETKEYSINSLOT echoes client-supplied keys, so it kills the session for any key larger than the send buffer -- reproducible today at the 128 KB default, independent of the adaptive budget. The gossip reply echoes the serialized cluster config and grows with node count. Give the cluster session the same atomic-first helper the RESP session has: write in place, retry once on a flushed buffer, and chunk only an element that cannot fit an empty buffer, so small elements keep exactly today's path. Route both sites through it. Covered by a new fixture run at both the default send buffer and one floored by the budget; reverting either arm's fix fails it. Also extends the oversized-response fixture to run at a floored send buffer, with a guard asserting each arm actually binds the size it claims to, and corrects a stale comment about the per-level entry cap being inert -- the 16 KB adaptive floor makes it bind at the defaults. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Shrink receive buffers immediately while the budget is binding The idle hysteresis exists so a connection with bursty-but-recurring large payloads keeps its buffer through the burst, and it is raised from 16 to 256 receives to make that hold over a genuinely long quiet stretch. But a countdown that long cannot converge the aggregate when the process is actually short of memory, so shrinking now bypasses it entirely while the budget is binding. The pressure signal is the already-published target having been driven below the configured size, not a live-byte read: it costs one predictable branch on the receive path and is ordered after the read-mostly size comparisons, so an unpressured server evaluates nothing new. Wires the pressure and idle shrink counters that INFO BPSTATS already reports. The new test drives a burst and then ten small rounds -- far short of the 256-receive hysteresis -- and requires the budgeted arm to give the buffers back while the unbudgeted control still holds them. Neutering the pressure signal fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound pressure shrinking so alternating payloads do not churn Shrinking immediately under pressure was wrong. Pressure is sticky -- the published target stays below the ceiling for as long as the connections are live -- so an alternating large/small workload reallocated a pinned buffer on every large request. Measured at 640 shrinks over 640 large/small pairs, i.e. one per request, precisely when the server is most loaded. Apply a short countdown under pressure instead of none. A strictly alternating workload never trips it, because each large receive resets the countdown, while sustained small traffic still converges in eight receives rather than two hundred and fifty-six. The countdown is clamped rather than reloaded so that pressure arriving mid-countdown does not wait out whatever remained of the idle one. Same measurement now reports zero shrinks. Adds a TLS arm, since the incident is thousands of TLS connections and the decrypted transport buffer shrinks through a separate path that the plain-socket test cannot reach. Disabling the transport clamp leaves the grown buffers at their burst size and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Converge live connections onto the adapted buffer size Clamping the allocation size governs new buffers only. A connection established before the budget started binding keeps its full-size buffers: the send side hands them straight back out from its own stack, and the receive side moves them to the pool free list, where they stay pinned. Neither converges, so at incident scale the adaptation applies only to churn. Gate both return paths on the budget actually binding, using the same IsUnderPressure signal the shrink policy shipped with: - LimitedFixedBufferPool.Return drops an over-target entry instead of pooling it. Unpressured it is pooled exactly as before, so short-lived connections handling moderately large payloads keep their reuse. Send and receive adapt to separate floors, so the comparison reads the entry's own buffer type off PoolEntry.source: measuring against the receive target alone would treat a correctly sized send buffer as over-sized and make send buffers un-poolable for as long as the budget binds. Accounting is unaffected -- the drop reuses the existing non-pooled fall-through, below the unconditional live-byte and live-count release. - GarnetTcpNetworkSender.ReturnBuffer disposes an over-target send buffer rather than pushing it back onto the per-connection stack, so the next acquire allocates at the adapted size. Also clear the vacated slot in LightConcurrentStack.TryPop. Without it the array keeps the popped buffer, and through it the pinned pool entry, strongly referenced for the connection's lifetime -- so releasing a buffer freed the accounting but not the memory. Measured over ten connections established while the budget was slack, then driven into pressure by two hundred more: 2.0 MB released with both gates against 1.0 MB with the send gate neutered. Pooled bytes after the same burst: 0 against 10.5 MB with the pool gate neutered, and the free list after connection churn holds 80 KB against 32 KB when the send target is ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address adversarial review findings on the network buffer budget Five findings from two independent reviews, each reproduced before being fixed. Recompute the adapted target inside the live buffer count's own increment and decrement. It previously had a single production caller on the pool's allocate-miss path, which a pooled reuse never reaches, so the target never recovered when connections drained: after a spike, four live buffers still published a 16 KB target and reconnecting did not clear it. Clamp the TLS plaintext send buffer to the send target. It was allocated at the configured size while the pool measured it against the target on return, so under pressure every TLS connection's buffer was dropped and freshly pinned on the next connect. 64 connections now hold 11.5 MB where they held 15.7 MB. Sample the receive shrink signal before the buffer is processed rather than after. Consumption and compaction happen inside Process on the plain path, so a fully consumed large request was read as a zero-byte receive and advanced the shrink countdown: 160 shrinks over 640 alternating large/small pairs, now zero. Route 26 further response sites through the chunking helper. Set operation results and every blocking variant wrote unbounded elements atomically, which kills the session above the send buffer size at the shipped default; BLPOP loses the popped element outright. The new tests fail in both fixture arms, so this is not budget-induced. Drop the aborted-session response release added earlier in this branch. The batch's finally already returns the object on every teardown path, so 200 protocol-aborted sessions conserve the buffer count with and without it, and handing a possibly in-use buffer back to the pool is not worth an unmeasurable fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix stale transport alias, above-max release ordering, and scratch demand tracking Round 2 adversarial review found five defects, two of which were introduced by the round 1 fixes. All five reproduced before being fixed, and every new assertion is mutation-checked. Without TLS the transport receive buffer is an alias of the network buffer, refreshed only at the top of each receive. A resize left the alias pointing at the entry just returned to the pool, so a connection that grew once and then idled kept the old pinned array rooted indefinitely -- the accounting showed the release and the memory never happened. Measured at 64 connections the stale roots exceed 16 MB. Both resize paths now refresh the alias. The above-max immediate release sat behind the demand guard, so a fully consumed oversized request made target equal current and held the out-of-pool buffer until another receive arrived. This regressed C4 via the round 1 demand-capture fix. The branch now runs first and is sized from the residual rather than the pass's demand. CLUSTER SLOTS still wrote the whole serialized topology atomically, so a sufficiently fragmented cluster overran the send buffer with no oversized key involved. The new test fails at today's 128 KB default too, which is what proves it pre-existing. ScratchBufferBuilder only raised its demand high-water when a write did not fit, so a batch that kept filling an already-large buffer reported no usage at all and the shrink policy released it every time. The popped-entry collectability test could pass for the wrong reason under optimization, since the stack itself was collectable once unused. GC.KeepAlive pins it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reset the shrink countdown on the above-max early return The above-max branch returns without shrinking when the buffer is still full of unconsumed data. Leaving the countdown untouched there is the same guard-ordering class as the defect the branch reorder just fixed, so restore the reset the demand guard used to perform. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Redesign session buffer caps as countdown checkpoints The committed high-water design tracked per-command demand on the hot path and cost ~9-11 ns on Network.BasicOperations.InlinePing at batchSize=1. Replace it at both sites with a countdown checkpoint: every batch decrements a counter, and only when it reaches zero does a cold NoInlining method run. Demand is inferred from whether the buffer grew since the last checkpoint rather than tracked per command, so the hot path holds one decrement and one predictable branch and never touches the parse state. This trades bounded churn for zero hot-path tracking: reclaim takes up to two checkpoint intervals, and a session that needs the capacity every batch reallocates at most once per two intervals. Removes BufferShrinkPolicy, SessionParseState.batchHighWater and ScratchBufferBuilder.AdvanceOffset as dead code. Also re-applies the SLOWLOG chunking fix (review round 2 finding 1) for sps.GetString and entry.ClientName, tightens the parse-state retention assertion from a bare Assert.Less that passed with the shrink neutered to a magnitude bound that fails when mutated, and tightens the scratch-buffer equivalence assertion from a 2x slack bound to unbounded + 64 KB. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the unpressured common case against an unbudgeted server The central promise of the budget is that with few connections nothing changes. Two gaps in the coverage left that unpinned. UnpressuredGrownBuffersAreRetainedExactlyAsWithNoBudget runs the same grow- then-small-traffic workload with the default budget and with the budget disabled, and asserts the two hold identical bytes both immediately after growth and after the small traffic that follows. Ungating the pressure clamp in MaybeShrinkNetworkReceiveBuffer fails its retention assertion. UnpressuredConnectionChurnKeepsRecyclingGrownBuffers guards correction C3: dropping an over-target entry on return must be gated on pressure, or connection churn re-allocates pinned arrays that are recyclable today. Ungating the drop in LimitedFixedBufferPool.Return collapses pooledBytes to 128 KB and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the scratch shrink checkpoint from the batch boundary only The countdown interval is counted in batches, but it was decremented from ScratchBufferBuilder.Reset(), which several callers invoke far more often than once per batch. The Lua interpreter resets once per string while decoding a JSON document and once per response through ScratchBufferNetworkSender, so a single cjson.decode of a wide object fired many checkpoints inside one command -- releasing a buffer the very next element re-grows, which is the churn the interval exists to bound. Split the countdown into ResetAtBatchBoundary(), called only from RespServerSession's batch boundary. Plain Reset() is now free again on the per-element paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the Lua script processor's scratch checkpoint from the session The Lua interpreter writes into the scratch buffer of the script cache's inner RespServerSession, not the network session's. That inner session never reads from a socket, so after the batch-boundary change its builder was never checkpointed at all and one cjson.encode of a large value permanently enlarged every session that ran a script. Fold the Lua checkpoint into the existing cold session checkpoint, which already runs once per 64 outer batches, so this costs nothing on the hot path and is definitionally outside any script execution. Rename the countdown trio accordingly, since it is no longer parse-state-only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk the remaining unbounded response sites and cap the Lua reply buffer A full sweep of the 70 atomic write loops in libs/server/Resp and libs/cluster found six writing unbounded data: ACL LIST, ACL USERS and ACL GETUSER echo client-supplied usernames and rule descriptions, and CLUSTER SLAVES, CLUSTER REPLICAS and CLUSTER INFO write topology that grows with the cluster. All six kill the session at today's 128 KB default, which the new tests prove by failing in both fixture arms. The remaining sites write literals, numbers or fixed-width identifiers. The script cache's dummy network sender owns a second scratch buffer that the RESP fallback path for redis.call writes replies into, built uncapped and never checkpointed. It now takes the configured session cap and is checkpointed alongside the request buffer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Clear retained references on pop and on the cluster parse-state copy Three round-3 review findings, all in the same aliasing class the earlier rounds kept surfacing: a reference survives the release that was supposed to free it. SimpleStack.Pop left the vacated slot populated. Its only user stacks ScratchBufferAllocator's outgrown pinned buffers there, so every buffer a session ever outgrew stayed rooted for the life of the connection and a reset freed the accounting but not the memory. The clear is gated on IsReferenceOrContainsReferences so a struct-only instantiation is unaffected. ClusterSession took a copy of the caller's SessionParseState and left it populated after the command finished. The copy holds its own reference to the session's pinned root buffer, so a wide cluster command kept that buffer alive past the owning session's shrink. NetworkBufferBudget.Recompute validated the previous target rather than the count it derived from, so a lost exchange could publish a target derived from a stale count. The retry loop re-derives on a lost exchange and re-reads the count after a won one. This is defensive: the race was not reproduced in 20 runs of a 256-thread oversubscribed churn, and the test that ships pins the invariant rather than discriminating the fix. LatencyHistogramMemoryTests' idle arm was vacuous. GarnetServerTcp needs four bytes before it creates a RespServerSession, so an idle socket never created one and the arm measured a server with no sessions at all -- it would have passed with eager allocation restored. Sessions are now opened with a partial command and the count asserted against INFO CLIENTS, and both assertions are restated as comparisons between the arms rather than against an absolute. Restoring eager allocation now fails three tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make three assertions test what they claim, and correct three doc statements The send-buffer pooling test accepted any 64 KB on the free list, which four pooled 16 KB receive buffers also satisfy -- so it could pass with no send buffer pooled at all, which is the defect it exists to catch. It now reads the per-level entry counts the pool already reports and asserts an entry at the send size class specifically. Ignoring the entry's buffer type in the drop rule leaves it at zero entries and fails. The cluster chunking tests stopped reading at a byte count, which samples only the part of the reply the first chunk boundary has passed and cannot see a lost tail -- the half a chunking fix is most likely to get wrong. Both now pipeline a PING and read to it: RESP replies are ordered, so the sentinel cannot arrive before the whole reply is written. GETKEYSINSLOT additionally asserts the key contents rather than its length header, and CLUSTER SLOTS asserts every one of the 2,000 ranges is present. Three statements were false. GetSessionScratchBufferMaxRetainedSize says unset disables shrinking; unset returns the 64 KB default and only zero disables. MaxEntriesPerLevelCap says it is inert at the shipped defaults; the budget is on by default, so the smallest size class is the 16 KB receive floor and the derived value is 4,096, which the cap binds. defaults.conf still described the high-water design the countdown checkpoint replaced. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the reachable cluster chunking site and count sessions, not sockets Review round 4 found the cluster chunking fixture green with all three of the round-3 cluster conversions reverted, and both latency histogram tests green with the session-opening helper made a no-op. The three round-3 cluster conversions are bounded and unreachable: CLUSTER INFO is a fixed eleven-line string, CLUSTER REPLICAS elements are node lines carrying no slots, and CLUSTER BANLIST entries are a node id plus a number. The reachable caller of the same helper is CLUSTER NODES, whose node line embeds slot ranges, so a fragmented assignment overruns the send buffer at the 128 KB default with no oversized key anywhere. Covered with 4,000 alternating slots, asserting the declared bulk length against the received payload and every range. connected_clients counts accepted network handlers, not sessions, so the latency guard could not distinguish a measured session from a bare socket. CLIENT LIST enumerates the sessions themselves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Stop the Lua reply cap being undone, and re-derive an abandoned budget target Three defects on the same theme: a bound that does not bind. The Lua reply sender re-requests a response window at the full server buffer size on every use, so capping its builder below one window could not save anything -- the checkpoint released the buffer and the next redis.call immediately reallocated it larger than it started, churning a pinned array every checkpoint. The cap is now floored at one window, so it releases only the oversized replies it was added for. The existing test stopped at the shrink and could not see this. ScratchBufferBuilder rounded one byte past the request, so a request for exactly a size class allocated twice it -- and the Lua reply window, at exactly 128 KiB, is one. Every caller that must strictly grow already asks for one byte more than the current length, so exact rounding still grows them. NetworkBufferBudget.Recompute abandoned its target when its attempts ran out, leaving whatever the last contended exchange wrote, derived from a count that had since moved. Since nothing else recomputes, that value stood until the next acquire or release, and at a band boundary it could report no pressure while the budget was binding. It now re-derives from the freshest count. Contention cannot reach that path reliably, so the attempt limit is settable to force it. The ACL chunking tests stopped reading as soon as the oversized username arrived and never checked the tail; they now parse the whole array and require the pipelined PING reply to begin exactly where it ends. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the recompute attempt limit immutable Every connection reads the budget concurrently, so the attempt limit belongs in the constructor rather than in a settable field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Compare the CLUSTER NODES slot set exactly Substring matching is near-vacuous over 4,000 alternating slots: " 2" matches inside " 20" and " 2000", so almost any surviving fragment of the line satisfies it. Dropping every slot below 1,000 from the node line leaves the old assertion green and fails the new one at 3,500 slots against 4,000. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report a malformed reply as an assertion, not a timeout The hand-rolled RESP parsers in the chunking and latency tests assumed well-formed input. A truncated array, an error reply where a bulk string was expected, or a null bulk string surfaced as a fifteen-second receive timeout or an ArgumentOutOfRangeException rather than as a named assertion about the reply. Both parsers now check every bound before slicing, reject a reply whose type byte is not the expected one, and handle a null bulk string by recording it and advancing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Yield to a fresher publisher instead of overwriting it The exhaustion fallback published the target with an unconditional write, so a thread that sampled the live count before another thread published could land afterwards and replace the right answer for a fresher count with its own stale one. The remark argued the race in the wrong direction: it reasoned about being overtaken, not about overtaking. The publication is now a compare-exchange against the target that was observed alongside the count, and it does not retry. The only writer that can beat it also exchanges, so a lost exchange means a value derived from a count no older than this one is already published. The publish step takes the count and the observed target as parameters so the interleaving can be induced directly rather than raced into. Two assertions were weaker than the fixes they named: the size class test exercised the arithmetic helper but never an allocation, and the Lua reply test showed the cap survived another use without pinning it at one response window. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the cluster chunking arm small enough to chunk The CLUSTER NODES body is about 19 KB, which is below both the 128 KB default send buffer and the 64 KB the floored fixture arm actually resolved to for a single connection. Replacing the chunked write with a spinning atomic one left both arms green, so neither covered the path the test is named for. The floored arm's budget drops to 16k so its send target reaches the floor. The mutation now fails there. The default arm cannot be made to cover chunking here at all -- a single node's line is bounded by the 16,384 slots it can hold, which is well under 128 KB -- so the comment no longer claims it does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the target before the count, not after The compare-exchange was doing less than its remark claimed. It proves the target has not moved since this thread read it, which two threads that sampled the same target concurrently both satisfy regardless of whose count is fresher. Reading the target first and the count second makes the stronger statement true: a successful exchange proves no other publisher completed between the target read and the write, and the count was sampled inside that window, so it is no older than the count behind any value published earlier. The ordering is not observable from a single-threaded test, so it ships on the argument rather than on an assertion, as the other reasoning-level fixes in this work do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Abandon a contended recompute instead of publishing from a stale sample The single-attempt fallback had no correct form. An unconditional write let a stale count land after a fresher one; the compare-exchange that replaced it dropped a fresher publication when the staler one won the race; and its hysteresis band tested a target that could already have moved. Each fix traded one wrong interleaving for another. The fallback was also strictly weaker than one more iteration of the loop it backstopped: no post-write count revalidation and no retry. Remove it. Exhausting the attempts now publishes nothing, which is safe rather than resigned. Every write to the target is made by a thread that read both the previous target and the count immediately beforehand, so exhaustion leaves a value some recent count justified. Both count movers move the count before recomputing, so whichever thread invalidates a published target must itself re-derive it. A stale target left too high cannot hide behind the band either: clearing pressure requires the target to equal the ceiling, while the budget binding requires the quotient to be below it, and the band suppresses a write only when the quotient is at least the target. Replace the two tests that pinned the deleted methods with one that pins the ordering the argument rests on, and tighten two reply parsers that accepted malformed framing: any negative bulk length read as null, and a bulk string whose terminator was never checked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Remove the recompute attempt knob and harden two reply parsers Deleting the stale-publish fallback made a zero or negative attempt limit permanently stick the published target, since nothing else republishes on exhaustion. No caller passed the parameter once the two tests pinning the deleted methods were removed, so the constructor parameter is gone and the bound is a constant. A reply carrying its declared payload with no trailing CRLF blocked in Receive until the socket timed out, so the terminator assertion was unreachable. Both parsers now read to a pipelined PING sentinel and parse declared lengths as long, so an out-of-range length reports a named assertion instead of overflowing the bounds arithmetic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count sessions directly instead of parsing CLIENT LIST off the wire The hand-rolled RESP parser behind the session count produced a review finding in four consecutive rounds: an unreachable terminator check, a null-length case accepted too broadly, and twice an overflow in its bounds arithmetic. None were about the property under test. The fixture already enumerates live sessions for another test, so the count now uses that: it opens no probe connection, so it needs no self-discount, and it cannot mis-frame a reply. The remaining array parser compares its declared length by subtraction. Summing the offset, the length and the terminator wraps negative at long.MaxValue, which passes the bounds check and reaches the narrowing cast with an invalid length. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cut the per-receive and per-batch cost the buffer caps added The session buffer caps and the network budget each added work to a path that runs once per receive or once per batch, which at batchSize=1 is once per command. Three changes, none of which alter behaviour: - UpdateNetworkBuffers called MaybeShrinkNetworkReceiveBuffer on every receive, so a connection whose buffer is still at its base size paid a call and a chain of loads to discover it had nothing to do. The entry test is hoisted to the call site and the callee marked NoInlining. - BaseReceiveBufferSize walked networkPool.Budget and two properties on every read. The budget and the configured sizes are cached as readonly fields on the handler, and the clamp is one inlined call on the budget. - The scratch builder and the parse state each counted their own batches. One countdown on the session now drives both, so the batch boundary pays a single decrement regardless of how many buffers are capped, and ScratchBufferBuilder.Reset returns to exactly what it was before. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Release quiesced sessions' latency histograms A session that records latencies and then goes quiet holds its histograms for the connection's lifetime: lazy allocation defers the cost but never returns it, and the monitor's per-window Reset only zeroes the counts while keeping the array. At thousands of connections that is hundreds of MB retained for histograms recording nothing, plus a memset of all of it every sampling interval. The latency monitor already sweeps every session every window and already evaluates the exact predicate needed, so no tracking is added to the record path. After a configured number of consecutive empty windows the monitor drops that type's histograms. The reference is dropped rather than returned to the pool. The record path is deliberately unsynchronised, so a writer racing the monitor can hold the array past the release; dropping leaves that writer with the only remaining reference and its write unobservable, whereas returning would let it write into an array another session has re-rented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the histogram release threshold's hysteresis margin The component test supplies its own threshold, so the configured value was pinned by nothing: setting it to one window left the fixture green. A session issuing requests more slowly than the monitor sweeps must keep its histograms across the isolated empty windows that produces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's timing margin At a two-second request interval against a one-second sweep, an overshoot on a loaded box can manufacture the four consecutive empty windows the test asserts cannot happen. At 1.5 seconds the mutation still fires in three seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's absolute stall tolerance Scaling the sweep and the request interval together keeps the ratio the mutation needs while doubling how long the machine may stall before the test manufactures the consecutive empty windows it asserts cannot occur. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reduce the network buffer config surface from nine flags to two The branch had grown nine new command-line switches. Seven of them were tuning knobs for an adaptive mechanism that sizes itself, exposed because they were easy to expose rather than because an operator needs them. Demoted to properties on GarnetServerOptions, reachable programmatically but no longer settable at startup: --network-buffer-size --network-buffer-min-size --network-max-receive-buffer-size --network-send-buffer-min-size --network-buffer-pool-size --session-parse-state-max-retained-args --session-scratch-buffer-max-retained-size None was CONFIG SET-able, so no live mitigation is lost, and none has shipped, so no operator .conf can reference one. Kept: --network-buffer-memory-budget the process-wide ceiling this work adds, and the only knob whose right value depends on the deployment --latency-monitor-precision a genuine accuracy/memory trade-off with no correct default Renamed for direction-explicit consistency with the pre-existing NetworkSendBufferMinSize: NetworkMaxReceiveBufferSize -> NetworkReceiveBufferMaxSize NetworkBufferMinSize -> NetworkReceiveBufferMinSize NetworkBufferSize keeps its direction-agnostic name: GarnetServerBase already exposes a public NetworkBufferSize for the same concept, so renaming here would diverge from established vocabulary. Removing the defaults.conf key would have changed behaviour rather than preserved it. An unset NetworkBufferPoolSize does not mean "unbounded" -- LimitedFixedBufferPool derives a byte ceiling from its per-level entry bound instead, which at the shipped 16-entry default is ~32.5 MB rather than the 64 MB the standalone server has always run. Options.GetServerOptions() now assigns "64m" directly, so the CLI path resolves exactly as before and embedded hosts keep the derived geometry they already had. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tell a refused connection why, and count it A connection refused by --network-connection-limit was disposed with no error written and counted in no metric, so an operator hitting the limit saw an unexplained reset and had nothing to confirm the cause. Send the Redis-compatible -ERR max number of clients reached before closing, and add rejected_connections to INFO stats. The write is gated to non-TLS. A plaintext error to a peer that has only sent a ClientHello is a protocol violation, so the client reports a handshake failure and the operator is pointed at certificates instead of capacity -- worse than a clean reset. Under TLS the metric is the whole remedy. The counter is incremented only in the limit branch, not in the catch that handles sockets dying during setup, so it stays trustworthy in the incident it exists for. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the connection limit process-wide and settable as maxclients The limit was per-listener, so N endpoints gave N times the configured ceiling -- the same defect this branch fixed for buffers, on the adjacent line of GarnetServer.cs. ConnectionLimit sums the listeners' existing activeHandlerCount values rather than maintaining a second counter, so no new accounting can drift; a connection counter that ratchets fails closed, refusing everything until restart. Default is now 10000, matching Redis, and set on GarnetServerOptions rather than only defaults.conf so embedded and test hosts resolve the same value as CLI hosts. CONFIG SET maxclients pushes the new ceiling through StoreWrapper.Servers into the live accept path. Lowering it disconnects nobody, which is also how Redis treats a lowered maxclients. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the refusal message, the counter, maxclients and the shared ceiling Seven assertions. The fixture holds one persistent probe connection so INFO and CONFIG can still be issued after the server is full, and fills only the remaining slots -- a probe that connected on demand would itself be refused. metricsSamplingFreq is set because rejected_connections is published by the monitor's sampling loop and reads "0" unconditionally without it, which would have made every counter assertion vacuous. Commands go over the wire as RESP arrays: Garnet's inline parsing handles single-token commands but silently produces no reply for a multi-token one, so an inline CONFIG GET returned nothing and the first version of these tests failed for that reason rather than for anything about the limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the shared-ceiling test able to fail The second listener refusing is equally consistent with it having reached a ceiling of its own, so the test now makes it accept a connection first. Also adds the drain-and-recover assertion and asserts several connections past the old ceiling when the limit is lifted, since admitting exactly one more is what raising the limit by one would also do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Build the connection limit on net8.0, and document the maxclients semantics System.Threading.Lock is .NET 9+, so ConnectionLimit did not compile for the net8.0 target the solution also builds. A plain object monitor is what the rest of the server uses and is equivalent here -- the lock is taken only by listener registration at startup, never on the accept path. Also records the three semantics an operator will ask about: replica and cluster peer links count against the limit (as they do in Redis, whose accept path adds getClusterConnectionsCount()); lowering the limit refuses new connections rather than culling established ones; and the value is not clamped to the file descriptor limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix final newlines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover maxclients across listeners and TLS rejection accounting MaxClientsReachesTheSharedLimitFromAMultiListenerHost fills the shared ceiling from two endpoints, raises maxclients at runtime, and asserts both listeners admit again, so CONFIG SET is pinned as reaching every listener rather than the one that happened to be first. TlsRejectionsAreCountedEvenThoughTheClientCannotBeTold pins the other half of the TLS gating: a refused TLS connection is closed without a RESP error, but still increments rejected_connections, which is the only signal an operator has on that transport. The TLS fillers are real TLS clients rather than bare sockets. The handshake runs on the accept path, so a connection that never handshakes stalls the accept loop and is never admitted, which makes bare sockets useless as fillers on a TLS listener. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS accept-path stall with concurrent and partial-handshake cases Upstream now starts the handler without waiting for the handshake, so the fix this branch carried is redundant and is dropped in favour of it. The tests are not: they add a peer that stalls after a partial record header, and sixteen concurrent silent peers, neither of which the upstream regression test reaches. Each asserts on a completed RESP exchange rather than on connect, because a connect against a wedged listener is absorbed by the backlog and succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin that a failed TLS handshake releases its admission slot Starting the handshake off the accept loop moves where a handshake failure is noticed: it no longer propagates out of Start into the accept site's catch, so the slot the accept path already took is released by the handshake's own failure path instead. A leak there would let an unauthenticated peer permanently fill a connection-limited server. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Delete the non-discriminating handshake slot test Three mutations could not break it: removing the observer's Dispose, removing Dispose from the handshake failure path, and holding the peer socket open all left it green. Slot release on a failed handshake is over-determined, so the test observed nothing its subject could change. Replacing the fillers with silent sockets does fail, which is the residual the handshake timeout addresses rather than a property to pin here. Also corrects the ConnectionLimitTlsTests rationale, which cited the accept-loop stall that no longer exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound the per-session Lua script cache with an LRU cap Each cached script holds a whole Lua VM, so an unbounded per-session cache lets a session that runs many distinct scripts accumulate VMs for its lifetime. Cap the cache and evict the least recently used entry. Recency is a monotonic stamp updated in place on each hit, so a cache hit costs one store and eviction scans only at capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the script cache bound, its LRU order, and the evicted-script path Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate transaction key entries unpinned on both paths TxnKeyEntries allocated its initial array with GC.AllocateArray(pinned: true) while the growth path used a plain array, so the pinning was silently dropped at the first resize. Every use of the array is through AsSpan, indexing or Array.Copy, and no caller takes a stable address into it, so the unpinned form the growth path has always used is the correct one. Dropping the pinned initial allocation also keeps these arrays out of the pinned object heap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count and report response buffer overflow rentals ReallocateOutput rents a pooled buffer whenever a response outgrows the buffer it is being written into. That traffic was only inferable, so the amount of overflow a given send buffer size causes could not be observed directly. Adds a process-wide rental count, reported as total_output_buffer_rentals in INFO STATS. It is read straight from the counter rather than from the monitor's sampled global metrics, so unlike the neighbouring rows it stays accurate when no monitor is configured. Measured through the new counter: ordinary responses rent nothing, and an oversized response rents exactly one buffer rather than climbing a doubling ladder, which is the property that makes a smaller send buffer affordable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop trailing newlines to match the repo's final-newline convention Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report the output buffer rental count under BPSTATS rather than STATS The counter is read live from its source rather than sampled by the monitor, so under STATS it was the one row that stayed non-zero when no monitor is configured. MetricsDisabledTest asserts every STATS row reads zero in that state, which is Garnet's default, and it caught this. BPSTATS is where the rest of the buffer instrumentation on this branch already lives and is read live for the same reason, so the counter is consistent with its section there instead of being an exception inside a sampled one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count the refusals a capacity wait legitimately incurs WaitForAcceptance polls until the server has capacity, and capacity returns when the server observes the closes rather than when the client disposes. An attempt made before it does is refused for real and advances rejected_connections, so the two tests asserting an exact count were racing a teardown they do not control. Report the refusals from the wait and add them to the expectation, keeping the assertions exact rather than loosening them to a bound that a build which stopped counting would also satisfy. Only a genuine max-clients reply counts, so an unrelated socket error cannot inflate the allowance and mask a miscount. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Capture the session latency histograms once at every reader The monitor sweep nulls LatencyMetricsEntrySession.latency when a session quiesces, but every reader check-then-dereferenced the field, so nothing guaranteed the two reads saw the same value. Both RecordValue overloads run on the session hot path with no lock by design, and GarnetLatencyMetrics.Merge runs on the disposing session's thread under rwLock while the sweep's reclaim holds only disposeLock, so the two do not exclude each other. Load the field into a local once and work from that. Allocate() now returns the array so the miss path reads the field once too. The type's remark claimed the reclaim was safe because a racing writer holds the only reference to the graph; that is true only if the writer captured it, so the remark now states single-capture as the invariant to preserve. Also gate the shrink counters on the budget being enabled, matching OnBufferAcquired: every pool handed no budget shares the Disabled singleton, so counting there put a contended process-wide interlocked write on the shrink path of connections the budget does not govern. Drop two unintended UTF-8 BOMs, and correct the maxclients documentation. Garnet takes every connection on one listener and cannot tell the kinds apart at accept time, so at the limit it refuses replica and gossip links alike; Redis counts cluster bus links but accepts them on a separate port that applies no limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Name the single-capture invariant at the latency record path The two RecordValue overloads load `latency` into a local exactly once because the monitor thread can null it concurrently. The expression reads as a convenience idiom, so state why it is load-bearing at the point a future edit would touch. The invariant itself is documented on ReclaimIfQuiesced, which is the writer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * State the single-capture requirement on the latency field itself The record-path comments guard the readers that exist. An author adding a new reader passes through the field declaration instead, so the constraint belongs there too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tighten the comments added by this work Cut justifications-against-alternatives and narrative framing from the doc comments and inline comments this change introduces, keeping the constraints and invariants a reader needs in order to change the code correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Restate why Recompute needs no fallback on attempt exhaustion Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Inherit TestBase in the concurrent stack fixture Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the buffer pool's stats directly rather than over the wire Serving an INFO reply takes a send buffer and a receive buffer off the free list the assertion reads, so the probe consumed what it reported and the assertion had no margin against its own instrument. Read the listener's pool directly, and poll the size class the assertion reads rather than the byte total, returning the snapshot every assertion is then taken from. Cover the same rule against the pool directly, where nothing can take an entry off the free list between the return and the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address PR review comments on histogram release, scratch cap, and test sends Release the session latency histograms by dropping the reference rather than returning their counts arrays, so a racing recorder cannot write into an array another session has since rented. `LongHistogram.Release()` does the same for the client, whose `latency` field is readonly. `ScratchBufferAllocator.Reset()` no longer enforces the retained-capacity cap at every batch boundary; a `ShrinkCheckpoint()` on the session's coarse countdown does, mirroring `ScratchBufferBuilder`. The session and transaction allocators are both wired to it, with a guard so a WATCH spanning batches keeps its slices. Test socket sends loop until the whole request is written, and the RESP reply reader reads until the expected length has arrived. `NetworkBufferBudget` documents the budget as an adaptive target on the base buffer size rather than a ceiling on the aggregate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the scratch allocator's checkpoint interval and its outstanding-slice guard A functional WATCH test cannot observe a buffer released underneath a live slice: the pinned array is collectable but its bytes stay readable until a collection reuses them. Both invariants are asserted on the allocator directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Update the scratch allocator tests for the checkpoint release contract The allocator now keeps its current buffer across Reset and releases an over-sized one at a ShrinkCheckpoint, so the fixture asserts the retained capacity at each point and drives two checkpoints to clear the hysteresis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Floor the above-max receive shrink at the largest poolable size The residual left after a fully consumed request is zero, so sizing the replacement from it always resolved to the base. A connection issuing one oversized request per batch then re-grew through every intermediate size class on every request. Floor the replacement at the largest poolable size at both the socket and TLS transport sites; the idle hysteresis path still takes it down to the base once the traffic no longer needs the capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the client latency histogram once at every consumer Release drops the counts array while the reply-processing thread may still be recording, so the record path tolerates a released array and the copy path reads the field once instead of testing IsReturned and then dereferencing. Also make the quiesce-threshold test observe empty windows: it recorded a value and never cleared it, so the windows it asserted over were not empty and a threshold that never restarted its run still passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the released-histogram contract on the record and copy paths Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Derive the copied histogram's tracking values from the counts it carries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Correct the rationale for deriving the copied histogram's total Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS transport above-max receive replacement A TLS connection decrypts into a second receive buffer that grows through its own code path, so the above-max release there was unexercised while its plain-socket twin was pinned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Gate the above-max receive floor on budget pressure The above-max branch releases a buffer the pool cannot recycle. Flooring the replacement at the largest poolable size matches origin/main and saves a connection that needs the capacity on every request from re-growing through every size class, but it must not apply while the budget is binding: the target is derived from a buffer count, so a connection parked at the maximum costs the same single unit as one at the floor and the budget gets no feedback from it. Under pressure the replacement is now the adapted base, which the pool keeps and the budget can see. Also capture the counts array once in ClearCounts and CopyCountsInto, read the client call count off the snapshot the percentiles come from, and record the single-read contract on the field itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS half of the above-max pressure gate The plaintext and TLS receive paths gate the above-max release independently. Only the plaintext site was exercised, so the TLS gate at NetworkHandler.cs:801 could be removed with the whole fixture still green. Parameterize the test over both transports. Guard the counts array in CopyCountsInto, matching the single-read invariant documented on the field and the five sibling capture sites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop the scratch retention assertion that measures test ordering `ScratchBufferCapDoesNotChangeLargeArgumentResults` compared a per-session memory figure across two arms and failed intermittently. Two measurements show it cannot discriminate: swapping the arms moves each figure by about 130 KB per session and flips the sign of the difference, because the arm that runs second also measures the residue of the server instance the first one disposed; and releasing the buffer at every reset rather than at the checkpoint -- the churn such an assertion claims to catch -- moves it by less than that, because churn leaves garbage the forced collection reclaims rather than retention. The test keeps both arms and the reply validation they drive, which is what its name has always claimed, and the remarks state why no magnitude is asserted. `NetworkBufferBudget` documents the per-direction floors as the third reason the budget is a target rather than a ceiling: once the target reaches them, the aggregate grows with the connection count again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Sample the shrink churn either side of the batch boundary Comparing capacity across iterations cannot see the pathology the test exists to catch: a policy that released on every batch regrows before the next sample, so consecutive post-boundary capacities are equal and the churn counts as zero. Sampling before and after each boundary counts every release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Say what the latency precision number actually buys The setting is HdrHistogram significant digits: relative accuracy at every magnitude, not a fixed time unit. Stating it that way, with the memory each step costs, lets an operator pick a value without knowing HdrHistogram. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Pin that an over-sized request is accepted whole The receive buffer grows to fit and maxReceiveBufferSize bounds only what a connection retains, so a chunk far above it must still be accepted. Dribbled in small slices so the request sits partially buffered across many receive passes, where the above-max shrink runs against a live partial request. Also drop a qualifier from the latency precision help text. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Keep the byte-order marks as main has them Earlier edits flipped the BOM on seven files, which showed up as a one-line change to each file header with no content behind it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Wait for the buffer release instead of racing it A receive releases its buffer in UpdateNetworkBuffers, which runs after Process has already written the reply, so a client that has read its response can still be ahead of the release. Sampling BPSTATS once at that point read the pre-release numbers on a loaded runner and failed with the full oversized buffer still live. Poll until the counters settle. The wait uses only the out-of-band INFO connection, so it does not hand the connection under test the extra receive that would let a buffer released one pass too late off the hook: suppressing the immediate release still fails the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Fold the two receive-buffer shrink paths into one policy The network and transport receive buffers ran the same three-regime shrink policy over their own occupancy and countdown, written out twice. Extract it as TryPlanReceiveBufferShrink, which owns the countdown bookkeeping and the budget's shrink accounting, and leave each caller with nothing but the buffer swap. Rename both entry points to TryShrink*, and give the transport buffer a ShrinkTransportReceiveBuffer mirroring ShrinkNetworkReceiveBuffer. Drop the cached configuredReceiveBufferSize/configuredSendBufferSize fields and read them off networkBufferSettings, as maxReceiveBufferSize already is on the same paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Keep the configured base sizes cached on the handler BaseReceiveBufferSize is read on every receive, so reaching the configured size through networkBufferSettings puts a dependent load on that path. BDN Network.BasicOperations.InlinePing measures it at +1.1ns (89.8 -> 90.9) on net10 and +1.9ns (85.7 -> 87.6) on net8, the net8 range clearing baseline entirely. Restoring the fields returns both to baseline (89.8 / 85.8). The shrink consolidation is unaffected and stays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Capture the metrics array in ReclaimQuiescedHistograms Every other method on the session captures the metrics field once before dereferencing it. ReclaimQuiescedHistograms re-read it for the null check and again inside the loop, so it relied on the dispose lock for that safety where its peers do not. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Checkpoint the transaction scratch allocator during AOF replay Replayed transaction procedures run their prepare phase against the watch API, and every watched key is copied into the transaction scratch allocator. The replay session never reaches the network batch boundary that drives the shrink checkpoint, so the buffer one wide procedure grew stayed pinned for the lifetime of a replica. The replay path now reports a boundary per AOF record and the window is counted in TransactionManager, so the caller does not carry the cadence. The allocator shrinks only when nothing is outstanding, so a boundary reported mid-transaction is safe. Also apply two review nits: a u8 span literal for the max-clients error instead of a materialized array, and a corrected comment on the connection-limit registration lock, which was crediting the lock with reader safety that the volatile publication provides. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Release the session scratch allocator at the AOF replay boundary Replayed procedures grow both scratch allocators, not just the transaction one: the prepare phase copies watched keys into the transaction allocator, and the procedure builds its arguments from the session allocator. The session allocator is reset at the start of every procedure rather than when one ends, so a completed procedure leaves it oversized with a non-zero offset, and a shrink checkpoint declines in that state. Reset it at the replay boundary, as the network batch boundary does, and only outside a transaction, whose slices are still live. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Rename the buffer shrink hook off "checkpoint" and count replay records at the call site "Checkpoint" is load-bearing in this codebase: Tsavorite uses it for durable, versioned, recoverable snapshots, and Garnet's AOF replay handles genuine CheckpointStartCommit markers a few lines from where this hook was called. The scratch-buffer hook means something unrelated -- periodically sample a buffer's capacity and release it if it did not grow -- so it is renamed to Trim, matching ArrayPool<T>.Trim, which applies the same hysteresis. The replay hook also counted its own interval while the network one had the caller count, so TrimAfterReplayedRecord ran on every record but trimmed on one in sixty-four. The counter moves to AofReplayContext, alongside the session it governs, so both paths show the cadence at the call site and TrimReplayBuffers means "trim now". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 --------- Co-authored-by: badrishc <badrishc@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 | 1 个月前 | |
Support for Parallel Replication (#1556) * implement sequence number generator * More work on LogRecord.RecordLength * format code * Finish LogRecord.RecordLength * add checkpoint marker process coordination * add fuzzy region flag to replay context * trigger exception when remove of checkpoint barrier fails * throw exception if SignalAndWait times out * add flag to differentiate between sharded and single log AOF * AOF explorer * wait for aofSync and refreshTail tasks on Dispose * validate base replication tests v1 * Remove ObjectDb* from Sample*Txn * test fixes for disk-based replication v2 * add synchronized replay for remaining operations (i.e. FLUSHDB, STREAMING_CHECKPOINT) * sequential node dispose on teardown * ensure dispose client precedes iterator * MultiLevelPageArray stress fixes; improved LogRecord and RecordDataHeader ToString() * WIP * Fix GetObjectLogSegmentSize() * wait for aof sync driver and connection to drop before recovering * consolidate reset replica replay task group * update docs * Fix a couple comments * address data loss on synchronization * use unified synchronized operation for regulard txn replay * consistent read for scan commands * Trim ObjectIdMap arrays on ClearPage * remove unused variable * fix single log tests * fix sharded log tests * adding comments to AofReplayCoordinator * adjust ROLE command response for sharded log implementation * WIP; multi phase read consistency protocol * added basic consistent read protocol context * Add transactional consistent read context * md readmed for mlog design * add design notes * WIP; change session at state change * allow disklogRecord upserts for streamless sync * implement consistent read protocol for scan commands * add sharded diskless sync tests * eliminated interface declaration for APIs with consistent read * cleanup and refactoring * remove sharded option since we have AofBench * update tail witness task to ping replica for latest replayed sequence number information * remove unused code * wip; logical subtask replay * WIP; re-design AofHeaders * WIP; update sequence replay info * wip; v1 parallel replay within physical sublog * wip; running sharded log tests with single replay task * wip; refactor to use virtual sublog replay state and a virtual sublog idx calculation * wip; update coordinated operations to trigger when multilog is enabled * wip; add multi-replay test for single key operations * wip; cleanup * remove AofExplorer * wip; index coordination barriers with distinct key to mixing separate transactions on same session * add indexing for coordination barriers using sequence number * rename sublog count to physical sublog count * remove cproj ref * separate AofBenchType * wip; mlog description * wip; mlog description * wip; adding comments and writing dev replication website content * wip; updating documentation and description in code * wip; AofSyncTask section * wip; replay details summary and fixes * wip; renaming and updating todo items * wip; skip hashing when configured with one physical sublog and multiple replay tasks * add test for custom txn and multi-exec txn with parallel replay * refactor/rename and cleanup replica replay code * wip; adding bulk consume recovery * renaming and cleanup * simplify intra page replay and synchronization primitive * implement recover intra page replay * assert throw when configuring multi-log without FastCommit * add description of prefix consistent recovery * push log specific operations in GarnetLog instance * wip; * implement prefix consistent recovery * refactor code to use single log when multi-replay is enabled with one physical log * remove cookie generation callbacks * cleanup and documentation write-up * more documentation * refactor advance time background task * TaskCompletionSource based wait for read consistent read * fix read protocol edge cases * code comments and cleanup * add cluster command to retrieve key sequence number for debugging purposes * PR description initial draft * advance time using high precision timestamp and background task * PR details * fix formatting * revert HashUtils location * HashUtils mv * fix formatting * cleanup * single log legacy implementation updates * add docs for mlog key time * legacy checkpoint cookie serialization support * fix test delay * run aofsynctask directly with single log * append replication vector to ROLE command when using multi-log * add support for intra-cluster connection identification and fix failing management tests * set client info when clientname parameter != null * support negative values in AofAddress * add ConsistentReadContext definition into Globals * fix formatting * fix ClusterDisklessSyncResetSyncManagerCts * fix ClusterResetHardDuringDisklessReplicationAttach * fix consistent read context bugs * wip; refactor read-protocol integration * Implement read key batch consistent protocol * improve robustness of background advance time consumer * remove unused playground project * cleanup clientname setup * fix tsavorite ISessionFunctions * remove unused assembly * simplify AofAddress and check for correctness * fix typo * eliminate bytes from informational message * simplify and cleanup leader barrier * cleanup LeaderFollowerBarrier * ensure commit for multilog recovery * replace tcs with semaphore slim * fix ClusterReplicationCheckpointCleanupTest * Fix copy-paste bug sending beginAddress instead of tailAddress in disk-based replica sync (#1633) In TryReplicateDiskbasedSync, ExecuteClusterInitiateReplicaSync was sending beginAddress.Span for both the aofBeginAddress and aofTailAddress parameters. This was introduced in commit 6fb99e58 when converting from ToByteArray() to Span-based calls. The primary uses the replica's tail address to compute the AOF sync replay range. With both parameters being the begin address, the primary couldn't determine where the replica's AOF actually ended, causing the replica to never receive AOF records and remain stuck at offset 64 (kFirstValidAofAddress). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix ClusterResetHardDuring tests by using ConfigureAwait for ExceptionInjection * enforce ConfigureAwait(false) across Garnet * fix AofBench * add website description * add migration GarnetLog upgrade test * fix formatting * fix website links * add object option for replication scheme upgrade test * set default logging to Error * pre-compute enforceConsistentRead * add AOF option for cluster BDN * update global.json * consolidate enqueue calls * fix upgrade test null check * add custom stop bulk consume callback * fixing ClusterReplicationMultiRestartRecover * fixing base replication tests * use throttle to check consumer liveness during bulk consume * refactor advance time background task * ensure WorkQueueLIFO waits for process bg task to complete on dispose * add activeWorkerMonitor * cleanup and refactoring * wip; adding tracking * fix hanging ClusterReplicationCheckpointCleanupTest * fix Tsavorite formatting * fix Garnet formatting * fix breaking build * improve log message * throw the caught AsyncFlushSnapshotPage exception * ensure failover suspends replica only tasks * usingSharded when replaying with multiple tasks * add no-inlining exception throw * fix AofBench to work with AofReplayCount * wip; add variant to parallel replay a physical sublog * wip; fixing ClusterSRNoCheckpointRestartSecondary * fix defaults * classify sharded tests under same namespace * throw at TakeOverAsPrimary to fix test * log startAddress on dispose of AofSyncDriver * add better logging at ClusterAppendLog initialization * reuse parseState at replayOp * avoid cache invalidation when updating replicationOffset * ensure get slots parsing deals with unstable slots * nit; cleanup * fix formatting * better logging * eliminate monotonic updates on the enqueue hot path when tracking sequence numbers for advance time. * make aof-replay-task-count default=1 for AofBench * add ReplayNoResp * add ReplayDirect * fix Enqueue Bench * add better logging for replica receive primary AOF stream * fix InProc Bench wait for server * implemented targetted logging API for testing * enable LogPrimaryStream for ClusterSRNoCheckpointRestartSecondary * refactor cluster cmdstrings * remove unused methods * cleanup * fix logging event flag checking * update aof bench * optimistically return if not ReadWaiters * improving AofBench * Pass around keyHash and re-use it during replay * remove notes * increase thread pool for cluster tls replication * expand logging at AofSyncTask initialization * wip; cleanup replay path * use owned epoch for multilog * use owned epoch for tsavorite log and cleanup aof recover * make vector fixes for replication * fix aof gen * JIT replayOp refactor * fix range index AOF enqueue * use barrier on AofSync connection with multi-log only * adding async for connect and gossip * addressing first set of comments * addressing second set of comments * add throw cancellation * make light epoch in client compatible with tsavorite epoch implementation * simplify object RMW enqueue * separate appendLog initialization --------- Co-authored-by: TedHartMS <15467143+TedHartMS@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> | 6 个月前 | |
Remove the per-write allocations from the chunked object AOF path, and bound chunk buffering (#2182) | 26 天前 | |
Reduce allocation in random set index picking (#1242) * Reduce allocation in random set index picking * Remove method level duplication of the stackalloc threshold constant * Address PR feedback * Update RandomUtils.cs Altered one comment * format --------- Co-authored-by: Tal Zaccai <talzacc@microsoft.com> | 1 年前 | |
Large-object support in AOF, Migration, and diskless Replication (#2039) | 2 个月前 | |
Remove .NET 8 support (#2230) * Remove .NET 8 support Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve multi-framework release and benchmark infrastructure Keep release packaging and publish paths framework-aware while targeting only .NET 10. Retain benchmark job selection for future runtimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Minimize .NET 8 release-script removal Retain the original publish and archive structure, removing only .NET 8 work while keeping the framework-aware archive loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve existing release-script formatting Restore the original whitespace on unchanged .NET 10 publish commands and archive staging lines so the PR diff only shows the required .NET 8 removal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark framework options while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark README update to .NET 10 commands Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave sorted-set implementation unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave GeoHash unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while dropping .NET 8 entries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Revert publish-profile conditional path changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark config note to .NET 10 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark overview change limited to .NET 10 default Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove redundant benchmark framework exclusion guard Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Load benchmark issuer certificate without warning suppression Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 Tsavorite warning properties Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Fix .NET 10 formatting in Vector Set key accessors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 feature guards Drop conditional branches now that .NET 10 is the sole target framework. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep ref struct anti-constraints on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep alternate comparer interfaces on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep vector read batch ref struct declaration on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e --------- Co-authored-by: Tiago Napoli <tiagonapoli@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e | 17 天前 | |
Adding basic version of DUMP and RESTORE commands (#899) * Implement basic version of DUMP and RESTORE redis commands * refactor and add dump, restore to cluster slot verification test * Update comments to use 'RESP' encoding terminology * fix formating * fix tests * add acl and tests and update default config to include the skip checksum validation flag * rm accidentally commited dump.rdb file * Remove trailing whitespace in RespCommandTests.cs * fix comments * run CommandInfoUpdater and replace docs / info files * Remove trailing whitespace in Options.cs * fix RestoreACLsAsync test * fix comments * optimize RespLengthEncodingUtils * implement suggestions * fix comments * use SET_Conditional directly * rename SkipChecksumValidation * fix cluster restore test * directly write to the output buffer for non-large objects * Refactor WriteDirect call in KeyAdminCommands * Mark multiple commands as deprecated in JSON * Mark commands as deprecated in documentation --------- Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com> | 1 年前 | |
Bound per-connection memory: adaptive network buffer budget, session buffer caps, and connection admission (#2157) * Bound connection-scaled network buffer memory Per-connection network buffers were unbounded in practice: the receive buffer only shrank when it exceeded maxReceiveBufferSize, so a buffer that doubled 128K -> 256K -> 512K never shrank and every connection permanently retained its all-time peak. The TLS transport buffer had no shrink path at all. GarnetServerTcp also discarded its networkBufferSize argument and hardcoded 128 KB, so the per-connection floor was unconfigurable. - Add --network-buffer-size, --network-max-receive-buffer-size and --network-buffer-pool-size, plumbed through GarnetServerOptions into GarnetServerTcp, which now honors them. Defaults match today's behavior. - Shrink a grown receive buffer to the smallest size class that keeps 2x headroom over the buffered bytes, after 16 consecutive receives that fit. Buffers above maxReceiveBufferSize skip the hysteresis since the pool cannot recycle them. - Add the same shrink path for the TLS transport receive buffer, called only from the reader sites that already consider doubling. - Track liveBytes, peakLiveBytes, pooledBytes and a shared byte budget in LimitedFixedBufferPool, exposed through INFO BPSTATS. - Return the outstanding responseObject in DisposeNetworkSender so a session torn down mid-response does not leak its PoolEntry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Document new network buffer options and add config parsing tests Adds the three --network-buffer-* options to the configuration table, warns instead of silently correcting an inverted size pair, and covers defaults, overrides, power-of-2 rounding, the inverted pair and invalid values in GarnetServerConfigTests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bound per-session parse-state and scratch buffers A RESP session sizes SessionParseState.rootBuffer to the largest array arity the client has ever sent and keeps it for the life of the connection. One high-arity command therefore enlarges the session permanently, and the cost scales with connection count. The same ratchet exists in ScratchBufferBuilder, whose Reset() zeroes the offset but never releases the backing array. Add BufferShrinkPolicy, a shared cap-plus-hysteresis helper, and apply it at both sites. A buffer above the cap is released only after the configured number of consecutive batches that did not need the extra capacity, so a session with recurring large requests keeps its buffer. Both caps are configurable and default-on: --session-parse-state-max-retained-args (default 1024) --session-scratch-buffer-max-retained-size (default 64k) Measured end to end with a forced compacting GC, 200 sessions each issuing one 20,000-argument EXISTS followed by sustained single-key work: 327 KB/session retained before, 98 KB/session after. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover buffer shrink against transaction and watch aliasing The batch-boundary shrink releases a pinned array, so anything holding a reference into it across that boundary would dangle rather than merely read stale bytes. Two paths cross the boundary by design: a MULTI..EXEC window, which re-parses queued commands from the network buffer, and WATCH, which copies key bytes into the transaction scratch allocator so they outlive the receive buffer. Add tests that force the shrink to fire inside both windows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized response elements instead of killing the session A response element larger than the send buffer cannot be written atomically. The chunking helper for this already exists and SCAN uses it, but several commands still wrote unbounded user data through the atomic path, where TryWriteBulkString cannot make progress and SendAndReset throws, killing the session mid-response. KEYS fails today with any key larger than the 128 KB default send buffer. Give the shared WriteBulkString helper an atomic-first fallback: write in place, retry once on a flushed buffer, and only chunk an element that cannot fit an empty buffer. Small elements keep exactly today's path, so multi-element replies pay nothing. Route KEYS, COMMAND GETKEYS, COMMAND GETKEYSANDFLAGS and the pub/sub channel and pattern echoes through it. Two related defects found while covering this: ClusterSession.SendAndReset silently did nothing on an empty buffer, so a caller's retry loop spun forever rather than failing. It now throws, as the RESP session already does. MainStoreOps.GETDEL asserted its output was inline immediately after the RMW, before the spill-to-pooled-memory case is handled. Any value large enough to spill tripped it, so GETDEL of a large value failed on every debug build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Enter the chunking fallback when the send buffer is empty The fallback added for over-sized response elements flushed before retrying, but SendAndReset throws when there is nothing to send. That is exactly the over-sized case where the element is the first thing written to the response, so the chunking path was unreachable there and the session still died. Every earlier test emitted an array header first, which left the buffer non-empty and masked it. Flush only when output is actually buffered, otherwise chunk directly. CLIENT GETNAME is the reachable instance: the name is the whole response, so a name larger than the send buffer killed the session. Give the ASCII bulk-string helper the same fallback and route CLIENT GETNAME through it. Move both fallbacks to NoInlining helpers so the 49 call sites of the inlined fast path keep their previous code size. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate latency histograms on first record Every session was given twelve HdrHistograms when its connection was accepted, before it had recorded anything. At two significant digits each rents a 32 KB array from the shared pool, so a connection cost 384 KB of latency monitoring whether or not it ever issued a command. A dump of a server holding several thousand connections showed 887 MB in these arrays. Allocate the histograms for a latency type on the first value recorded into it. A session typically records into three of the six types, and an idle connection records into none, so it now pays nothing. Add --latency-monitor-precision to size the histograms. It defaults to 2, which is what the histograms were built with, so accuracy is unchanged unless it is lowered. Dropping it to 1 shrinks each histogram several-fold in exchange for reporting percentiles to 10% rather than 1% resolution. Fix three defects on the release path that lazy allocation widens the window for. LongHistogram.Return() now drops its reference to the array it hands back, so a later record throws rather than silently writing into an array the pool has re-rented to another session. The session's pending work is cancelled before its histograms are released rather than after. GarnetLatencyMetricsSession publishes its metrics as unavailable before releasing them, and the hot path reads the field once and checks it. Dropping the array reference exposed a use-after-return in GarnetClient, which copied its histogram after Dispose had already handed the array back; it now reports no metrics instead of reading a recycled array. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the latency precision to the histograms it sizes The precision knob was only covered by tests that constructed histograms directly, so a broken binding between the command line and the allocation would have gone unnoticed and silently degraded resolution. Assert the size the server reports through LATENCY HISTOGRAM, the size of the per-session histograms reached through the live sessions, and that the option parses to 2 by default and rejects values outside the 0-5 range HdrHistogram accepts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Wait for the session to record before reading its histograms A session records its latency after the reply has been written, so a test that read the histograms as soon as the reply arrived could find none allocated yet. Poll until one appears. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Publish a process-wide budget for live network buffers The buffer pool's byte ceiling bounds only its idle free list. Buffers checked out by live connections are bounded by nothing, so their footprint is connections times per-connection size, which is what grew unchecked with thousands of clients. Add NetworkBufferBudget, which divides a configured total by the live buffer count and publishes the largest base size that keeps the two consistent. It is a division rather than a control loop: buffer sizes move in factor-of-two steps, so any controller with a narrower deadband cannot converge and limit-cycles between two size classes. Growth therefore requires the quotient to reach twice the published target, matching the step it actuates. One budget is shared by every listener. Each endpoint previously built its own pool and was handed the whole ceiling, so with N endpoints the effective limit was N times the configured value. Only server listener pools participate; replication and migration keep their own unbounded behaviour so that a sync burst cannot drive live client connections down. The target is published but consumed by no allocation site yet. Landing the signal on its own makes a drift in the live buffer count observable in isolation, which matters because the target is budget divided by that count: a count that ratchets up does not degrade gracefully, it collapses the target to the floor permanently. The count rides exactly the two paths that already maintain the pool's live byte accounting and nothing else. Defaults are inert: 1 GB against a 128 KB buffer size means adaptation does not begin until 8,192 live buffers, and the target clamps at the configured size so it can only ever lower it. --network-buffer-memory-budget 0 disables it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new receive buffers from the network buffer budget The budget was published but consumed by nothing. Wire it to the receive side: a new connection's buffer starts at the published target rather than the configured size, and a grown buffer shrinks back toward that target. Only the base size is governed. Demand-driven doubling is untouched, so a connection that needs a large buffer still reaches it; pressure changes what a connection starts and settles at, never what it is allowed to grow to. The allocation size and the shrink floor move together. Clamping only the allocation would leave a connection allocated below the configured size unable to shrink at all, because the shrink path both floors at that size and returns early when the buffer is already at or under it. A pool cannot recycle a buffer outside its size classes, so NetworkBufferSettings gains an explicit minimum. Deriving it from the three configured sizes, as before, gives no way to express "recycle down to 16 KB while new connections still start at 128 KB", and a clamped buffer would have been dropped rather than pooled. The per-level idle entry count derived from a byte budget is capped in the same change, since a smaller minimum size class otherwise raises it proportionally; the cap sits above the value today's defaults produce, so it is inert until the two are combined. Budget accounting is now inert when disabled rather than merely unread. The disabled budget is a shared singleton, so unbudgeted pools were accumulating a count on it that was meaningless but would have become misleading the moment anything read it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new send buffers from the network buffer budget GarnetSaeaBuffer is the single allocation point behind both GarnetTcpNetworkSender call sites, so clamping there covers the send side entirely. Send buffers never grow, so an oversized response is chunked through whatever buffer it was given, and every consumer reads the length off the pool entry rather than off the configured setting -- which is what makes the size safe to adapt. The clamp uses the separate, higher send floor so that a shrunken send buffer does not push custom raw string responses into repeated ArrayPool rentals. Adds an end-to-end test that a 1 MB value and a key larger than the send buffer both survive while the budget is pinned at its floor, and tightens the shrink assertion now that both directions adapt. Neutering the clamp fails that assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized cluster response elements The chunking audit converted the RESP session but left the cluster session on the atomic path, where an element larger than the response buffer cannot be written and SendAndReset has nothing to flush. CLUSTER GETKEYSINSLOT echoes client-supplied keys, so it kills the session for any key larger than the send buffer -- reproducible today at the 128 KB default, independent of the adaptive budget. The gossip reply echoes the serialized cluster config and grows with node count. Give the cluster session the same atomic-first helper the RESP session has: write in place, retry once on a flushed buffer, and chunk only an element that cannot fit an empty buffer, so small elements keep exactly today's path. Route both sites through it. Covered by a new fixture run at both the default send buffer and one floored by the budget; reverting either arm's fix fails it. Also extends the oversized-response fixture to run at a floored send buffer, with a guard asserting each arm actually binds the size it claims to, and corrects a stale comment about the per-level entry cap being inert -- the 16 KB adaptive floor makes it bind at the defaults. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Shrink receive buffers immediately while the budget is binding The idle hysteresis exists so a connection with bursty-but-recurring large payloads keeps its buffer through the burst, and it is raised from 16 to 256 receives to make that hold over a genuinely long quiet stretch. But a countdown that long cannot converge the aggregate when the process is actually short of memory, so shrinking now bypasses it entirely while the budget is binding. The pressure signal is the already-published target having been driven below the configured size, not a live-byte read: it costs one predictable branch on the receive path and is ordered after the read-mostly size comparisons, so an unpressured server evaluates nothing new. Wires the pressure and idle shrink counters that INFO BPSTATS already reports. The new test drives a burst and then ten small rounds -- far short of the 256-receive hysteresis -- and requires the budgeted arm to give the buffers back while the unbudgeted control still holds them. Neutering the pressure signal fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound pressure shrinking so alternating payloads do not churn Shrinking immediately under pressure was wrong. Pressure is sticky -- the published target stays below the ceiling for as long as the connections are live -- so an alternating large/small workload reallocated a pinned buffer on every large request. Measured at 640 shrinks over 640 large/small pairs, i.e. one per request, precisely when the server is most loaded. Apply a short countdown under pressure instead of none. A strictly alternating workload never trips it, because each large receive resets the countdown, while sustained small traffic still converges in eight receives rather than two hundred and fifty-six. The countdown is clamped rather than reloaded so that pressure arriving mid-countdown does not wait out whatever remained of the idle one. Same measurement now reports zero shrinks. Adds a TLS arm, since the incident is thousands of TLS connections and the decrypted transport buffer shrinks through a separate path that the plain-socket test cannot reach. Disabling the transport clamp leaves the grown buffers at their burst size and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Converge live connections onto the adapted buffer size Clamping the allocation size governs new buffers only. A connection established before the budget started binding keeps its full-size buffers: the send side hands them straight back out from its own stack, and the receive side moves them to the pool free list, where they stay pinned. Neither converges, so at incident scale the adaptation applies only to churn. Gate both return paths on the budget actually binding, using the same IsUnderPressure signal the shrink policy shipped with: - LimitedFixedBufferPool.Return drops an over-target entry instead of pooling it. Unpressured it is pooled exactly as before, so short-lived connections handling moderately large payloads keep their reuse. Send and receive adapt to separate floors, so the comparison reads the entry's own buffer type off PoolEntry.source: measuring against the receive target alone would treat a correctly sized send buffer as over-sized and make send buffers un-poolable for as long as the budget binds. Accounting is unaffected -- the drop reuses the existing non-pooled fall-through, below the unconditional live-byte and live-count release. - GarnetTcpNetworkSender.ReturnBuffer disposes an over-target send buffer rather than pushing it back onto the per-connection stack, so the next acquire allocates at the adapted size. Also clear the vacated slot in LightConcurrentStack.TryPop. Without it the array keeps the popped buffer, and through it the pinned pool entry, strongly referenced for the connection's lifetime -- so releasing a buffer freed the accounting but not the memory. Measured over ten connections established while the budget was slack, then driven into pressure by two hundred more: 2.0 MB released with both gates against 1.0 MB with the send gate neutered. Pooled bytes after the same burst: 0 against 10.5 MB with the pool gate neutered, and the free list after connection churn holds 80 KB against 32 KB when the send target is ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address adversarial review findings on the network buffer budget Five findings from two independent reviews, each reproduced before being fixed. Recompute the adapted target inside the live buffer count's own increment and decrement. It previously had a single production caller on the pool's allocate-miss path, which a pooled reuse never reaches, so the target never recovered when connections drained: after a spike, four live buffers still published a 16 KB target and reconnecting did not clear it. Clamp the TLS plaintext send buffer to the send target. It was allocated at the configured size while the pool measured it against the target on return, so under pressure every TLS connection's buffer was dropped and freshly pinned on the next connect. 64 connections now hold 11.5 MB where they held 15.7 MB. Sample the receive shrink signal before the buffer is processed rather than after. Consumption and compaction happen inside Process on the plain path, so a fully consumed large request was read as a zero-byte receive and advanced the shrink countdown: 160 shrinks over 640 alternating large/small pairs, now zero. Route 26 further response sites through the chunking helper. Set operation results and every blocking variant wrote unbounded elements atomically, which kills the session above the send buffer size at the shipped default; BLPOP loses the popped element outright. The new tests fail in both fixture arms, so this is not budget-induced. Drop the aborted-session response release added earlier in this branch. The batch's finally already returns the object on every teardown path, so 200 protocol-aborted sessions conserve the buffer count with and without it, and handing a possibly in-use buffer back to the pool is not worth an unmeasurable fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix stale transport alias, above-max release ordering, and scratch demand tracking Round 2 adversarial review found five defects, two of which were introduced by the round 1 fixes. All five reproduced before being fixed, and every new assertion is mutation-checked. Without TLS the transport receive buffer is an alias of the network buffer, refreshed only at the top of each receive. A resize left the alias pointing at the entry just returned to the pool, so a connection that grew once and then idled kept the old pinned array rooted indefinitely -- the accounting showed the release and the memory never happened. Measured at 64 connections the stale roots exceed 16 MB. Both resize paths now refresh the alias. The above-max immediate release sat behind the demand guard, so a fully consumed oversized request made target equal current and held the out-of-pool buffer until another receive arrived. This regressed C4 via the round 1 demand-capture fix. The branch now runs first and is sized from the residual rather than the pass's demand. CLUSTER SLOTS still wrote the whole serialized topology atomically, so a sufficiently fragmented cluster overran the send buffer with no oversized key involved. The new test fails at today's 128 KB default too, which is what proves it pre-existing. ScratchBufferBuilder only raised its demand high-water when a write did not fit, so a batch that kept filling an already-large buffer reported no usage at all and the shrink policy released it every time. The popped-entry collectability test could pass for the wrong reason under optimization, since the stack itself was collectable once unused. GC.KeepAlive pins it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reset the shrink countdown on the above-max early return The above-max branch returns without shrinking when the buffer is still full of unconsumed data. Leaving the countdown untouched there is the same guard-ordering class as the defect the branch reorder just fixed, so restore the reset the demand guard used to perform. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Redesign session buffer caps as countdown checkpoints The committed high-water design tracked per-command demand on the hot path and cost ~9-11 ns on Network.BasicOperations.InlinePing at batchSize=1. Replace it at both sites with a countdown checkpoint: every batch decrements a counter, and only when it reaches zero does a cold NoInlining method run. Demand is inferred from whether the buffer grew since the last checkpoint rather than tracked per command, so the hot path holds one decrement and one predictable branch and never touches the parse state. This trades bounded churn for zero hot-path tracking: reclaim takes up to two checkpoint intervals, and a session that needs the capacity every batch reallocates at most once per two intervals. Removes BufferShrinkPolicy, SessionParseState.batchHighWater and ScratchBufferBuilder.AdvanceOffset as dead code. Also re-applies the SLOWLOG chunking fix (review round 2 finding 1) for sps.GetString and entry.ClientName, tightens the parse-state retention assertion from a bare Assert.Less that passed with the shrink neutered to a magnitude bound that fails when mutated, and tightens the scratch-buffer equivalence assertion from a 2x slack bound to unbounded + 64 KB. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the unpressured common case against an unbudgeted server The central promise of the budget is that with few connections nothing changes. Two gaps in the coverage left that unpinned. UnpressuredGrownBuffersAreRetainedExactlyAsWithNoBudget runs the same grow- then-small-traffic workload with the default budget and with the budget disabled, and asserts the two hold identical bytes both immediately after growth and after the small traffic that follows. Ungating the pressure clamp in MaybeShrinkNetworkReceiveBuffer fails its retention assertion. UnpressuredConnectionChurnKeepsRecyclingGrownBuffers guards correction C3: dropping an over-target entry on return must be gated on pressure, or connection churn re-allocates pinned arrays that are recyclable today. Ungating the drop in LimitedFixedBufferPool.Return collapses pooledBytes to 128 KB and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the scratch shrink checkpoint from the batch boundary only The countdown interval is counted in batches, but it was decremented from ScratchBufferBuilder.Reset(), which several callers invoke far more often than once per batch. The Lua interpreter resets once per string while decoding a JSON document and once per response through ScratchBufferNetworkSender, so a single cjson.decode of a wide object fired many checkpoints inside one command -- releasing a buffer the very next element re-grows, which is the churn the interval exists to bound. Split the countdown into ResetAtBatchBoundary(), called only from RespServerSession's batch boundary. Plain Reset() is now free again on the per-element paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the Lua script processor's scratch checkpoint from the session The Lua interpreter writes into the scratch buffer of the script cache's inner RespServerSession, not the network session's. That inner session never reads from a socket, so after the batch-boundary change its builder was never checkpointed at all and one cjson.encode of a large value permanently enlarged every session that ran a script. Fold the Lua checkpoint into the existing cold session checkpoint, which already runs once per 64 outer batches, so this costs nothing on the hot path and is definitionally outside any script execution. Rename the countdown trio accordingly, since it is no longer parse-state-only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk the remaining unbounded response sites and cap the Lua reply buffer A full sweep of the 70 atomic write loops in libs/server/Resp and libs/cluster found six writing unbounded data: ACL LIST, ACL USERS and ACL GETUSER echo client-supplied usernames and rule descriptions, and CLUSTER SLAVES, CLUSTER REPLICAS and CLUSTER INFO write topology that grows with the cluster. All six kill the session at today's 128 KB default, which the new tests prove by failing in both fixture arms. The remaining sites write literals, numbers or fixed-width identifiers. The script cache's dummy network sender owns a second scratch buffer that the RESP fallback path for redis.call writes replies into, built uncapped and never checkpointed. It now takes the configured session cap and is checkpointed alongside the request buffer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Clear retained references on pop and on the cluster parse-state copy Three round-3 review findings, all in the same aliasing class the earlier rounds kept surfacing: a reference survives the release that was supposed to free it. SimpleStack.Pop left the vacated slot populated. Its only user stacks ScratchBufferAllocator's outgrown pinned buffers there, so every buffer a session ever outgrew stayed rooted for the life of the connection and a reset freed the accounting but not the memory. The clear is gated on IsReferenceOrContainsReferences so a struct-only instantiation is unaffected. ClusterSession took a copy of the caller's SessionParseState and left it populated after the command finished. The copy holds its own reference to the session's pinned root buffer, so a wide cluster command kept that buffer alive past the owning session's shrink. NetworkBufferBudget.Recompute validated the previous target rather than the count it derived from, so a lost exchange could publish a target derived from a stale count. The retry loop re-derives on a lost exchange and re-reads the count after a won one. This is defensive: the race was not reproduced in 20 runs of a 256-thread oversubscribed churn, and the test that ships pins the invariant rather than discriminating the fix. LatencyHistogramMemoryTests' idle arm was vacuous. GarnetServerTcp needs four bytes before it creates a RespServerSession, so an idle socket never created one and the arm measured a server with no sessions at all -- it would have passed with eager allocation restored. Sessions are now opened with a partial command and the count asserted against INFO CLIENTS, and both assertions are restated as comparisons between the arms rather than against an absolute. Restoring eager allocation now fails three tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make three assertions test what they claim, and correct three doc statements The send-buffer pooling test accepted any 64 KB on the free list, which four pooled 16 KB receive buffers also satisfy -- so it could pass with no send buffer pooled at all, which is the defect it exists to catch. It now reads the per-level entry counts the pool already reports and asserts an entry at the send size class specifically. Ignoring the entry's buffer type in the drop rule leaves it at zero entries and fails. The cluster chunking tests stopped reading at a byte count, which samples only the part of the reply the first chunk boundary has passed and cannot see a lost tail -- the half a chunking fix is most likely to get wrong. Both now pipeline a PING and read to it: RESP replies are ordered, so the sentinel cannot arrive before the whole reply is written. GETKEYSINSLOT additionally asserts the key contents rather than its length header, and CLUSTER SLOTS asserts every one of the 2,000 ranges is present. Three statements were false. GetSessionScratchBufferMaxRetainedSize says unset disables shrinking; unset returns the 64 KB default and only zero disables. MaxEntriesPerLevelCap says it is inert at the shipped defaults; the budget is on by default, so the smallest size class is the 16 KB receive floor and the derived value is 4,096, which the cap binds. defaults.conf still described the high-water design the countdown checkpoint replaced. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the reachable cluster chunking site and count sessions, not sockets Review round 4 found the cluster chunking fixture green with all three of the round-3 cluster conversions reverted, and both latency histogram tests green with the session-opening helper made a no-op. The three round-3 cluster conversions are bounded and unreachable: CLUSTER INFO is a fixed eleven-line string, CLUSTER REPLICAS elements are node lines carrying no slots, and CLUSTER BANLIST entries are a node id plus a number. The reachable caller of the same helper is CLUSTER NODES, whose node line embeds slot ranges, so a fragmented assignment overruns the send buffer at the 128 KB default with no oversized key anywhere. Covered with 4,000 alternating slots, asserting the declared bulk length against the received payload and every range. connected_clients counts accepted network handlers, not sessions, so the latency guard could not distinguish a measured session from a bare socket. CLIENT LIST enumerates the sessions themselves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Stop the Lua reply cap being undone, and re-derive an abandoned budget target Three defects on the same theme: a bound that does not bind. The Lua reply sender re-requests a response window at the full server buffer size on every use, so capping its builder below one window could not save anything -- the checkpoint released the buffer and the next redis.call immediately reallocated it larger than it started, churning a pinned array every checkpoint. The cap is now floored at one window, so it releases only the oversized replies it was added for. The existing test stopped at the shrink and could not see this. ScratchBufferBuilder rounded one byte past the request, so a request for exactly a size class allocated twice it -- and the Lua reply window, at exactly 128 KiB, is one. Every caller that must strictly grow already asks for one byte more than the current length, so exact rounding still grows them. NetworkBufferBudget.Recompute abandoned its target when its attempts ran out, leaving whatever the last contended exchange wrote, derived from a count that had since moved. Since nothing else recomputes, that value stood until the next acquire or release, and at a band boundary it could report no pressure while the budget was binding. It now re-derives from the freshest count. Contention cannot reach that path reliably, so the attempt limit is settable to force it. The ACL chunking tests stopped reading as soon as the oversized username arrived and never checked the tail; they now parse the whole array and require the pipelined PING reply to begin exactly where it ends. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the recompute attempt limit immutable Every connection reads the budget concurrently, so the attempt limit belongs in the constructor rather than in a settable field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Compare the CLUSTER NODES slot set exactly Substring matching is near-vacuous over 4,000 alternating slots: " 2" matches inside " 20" and " 2000", so almost any surviving fragment of the line satisfies it. Dropping every slot below 1,000 from the node line leaves the old assertion green and fails the new one at 3,500 slots against 4,000. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report a malformed reply as an assertion, not a timeout The hand-rolled RESP parsers in the chunking and latency tests assumed well-formed input. A truncated array, an error reply where a bulk string was expected, or a null bulk string surfaced as a fifteen-second receive timeout or an ArgumentOutOfRangeException rather than as a named assertion about the reply. Both parsers now check every bound before slicing, reject a reply whose type byte is not the expected one, and handle a null bulk string by recording it and advancing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Yield to a fresher publisher instead of overwriting it The exhaustion fallback published the target with an unconditional write, so a thread that sampled the live count before another thread published could land afterwards and replace the right answer for a fresher count with its own stale one. The remark argued the race in the wrong direction: it reasoned about being overtaken, not about overtaking. The publication is now a compare-exchange against the target that was observed alongside the count, and it does not retry. The only writer that can beat it also exchanges, so a lost exchange means a value derived from a count no older than this one is already published. The publish step takes the count and the observed target as parameters so the interleaving can be induced directly rather than raced into. Two assertions were weaker than the fixes they named: the size class test exercised the arithmetic helper but never an allocation, and the Lua reply test showed the cap survived another use without pinning it at one response window. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the cluster chunking arm small enough to chunk The CLUSTER NODES body is about 19 KB, which is below both the 128 KB default send buffer and the 64 KB the floored fixture arm actually resolved to for a single connection. Replacing the chunked write with a spinning atomic one left both arms green, so neither covered the path the test is named for. The floored arm's budget drops to 16k so its send target reaches the floor. The mutation now fails there. The default arm cannot be made to cover chunking here at all -- a single node's line is bounded by the 16,384 slots it can hold, which is well under 128 KB -- so the comment no longer claims it does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the target before the count, not after The compare-exchange was doing less than its remark claimed. It proves the target has not moved since this thread read it, which two threads that sampled the same target concurrently both satisfy regardless of whose count is fresher. Reading the target first and the count second makes the stronger statement true: a successful exchange proves no other publisher completed between the target read and the write, and the count was sampled inside that window, so it is no older than the count behind any value published earlier. The ordering is not observable from a single-threaded test, so it ships on the argument rather than on an assertion, as the other reasoning-level fixes in this work do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Abandon a contended recompute instead of publishing from a stale sample The single-attempt fallback had no correct form. An unconditional write let a stale count land after a fresher one; the compare-exchange that replaced it dropped a fresher publication when the staler one won the race; and its hysteresis band tested a target that could already have moved. Each fix traded one wrong interleaving for another. The fallback was also strictly weaker than one more iteration of the loop it backstopped: no post-write count revalidation and no retry. Remove it. Exhausting the attempts now publishes nothing, which is safe rather than resigned. Every write to the target is made by a thread that read both the previous target and the count immediately beforehand, so exhaustion leaves a value some recent count justified. Both count movers move the count before recomputing, so whichever thread invalidates a published target must itself re-derive it. A stale target left too high cannot hide behind the band either: clearing pressure requires the target to equal the ceiling, while the budget binding requires the quotient to be below it, and the band suppresses a write only when the quotient is at least the target. Replace the two tests that pinned the deleted methods with one that pins the ordering the argument rests on, and tighten two reply parsers that accepted malformed framing: any negative bulk length read as null, and a bulk string whose terminator was never checked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Remove the recompute attempt knob and harden two reply parsers Deleting the stale-publish fallback made a zero or negative attempt limit permanently stick the published target, since nothing else republishes on exhaustion. No caller passed the parameter once the two tests pinning the deleted methods were removed, so the constructor parameter is gone and the bound is a constant. A reply carrying its declared payload with no trailing CRLF blocked in Receive until the socket timed out, so the terminator assertion was unreachable. Both parsers now read to a pipelined PING sentinel and parse declared lengths as long, so an out-of-range length reports a named assertion instead of overflowing the bounds arithmetic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count sessions directly instead of parsing CLIENT LIST off the wire The hand-rolled RESP parser behind the session count produced a review finding in four consecutive rounds: an unreachable terminator check, a null-length case accepted too broadly, and twice an overflow in its bounds arithmetic. None were about the property under test. The fixture already enumerates live sessions for another test, so the count now uses that: it opens no probe connection, so it needs no self-discount, and it cannot mis-frame a reply. The remaining array parser compares its declared length by subtraction. Summing the offset, the length and the terminator wraps negative at long.MaxValue, which passes the bounds check and reaches the narrowing cast with an invalid length. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cut the per-receive and per-batch cost the buffer caps added The session buffer caps and the network budget each added work to a path that runs once per receive or once per batch, which at batchSize=1 is once per command. Three changes, none of which alter behaviour: - UpdateNetworkBuffers called MaybeShrinkNetworkReceiveBuffer on every receive, so a connection whose buffer is still at its base size paid a call and a chain of loads to discover it had nothing to do. The entry test is hoisted to the call site and the callee marked NoInlining. - BaseReceiveBufferSize walked networkPool.Budget and two properties on every read. The budget and the configured sizes are cached as readonly fields on the handler, and the clamp is one inlined call on the budget. - The scratch builder and the parse state each counted their own batches. One countdown on the session now drives both, so the batch boundary pays a single decrement regardless of how many buffers are capped, and ScratchBufferBuilder.Reset returns to exactly what it was before. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Release quiesced sessions' latency histograms A session that records latencies and then goes quiet holds its histograms for the connection's lifetime: lazy allocation defers the cost but never returns it, and the monitor's per-window Reset only zeroes the counts while keeping the array. At thousands of connections that is hundreds of MB retained for histograms recording nothing, plus a memset of all of it every sampling interval. The latency monitor already sweeps every session every window and already evaluates the exact predicate needed, so no tracking is added to the record path. After a configured number of consecutive empty windows the monitor drops that type's histograms. The reference is dropped rather than returned to the pool. The record path is deliberately unsynchronised, so a writer racing the monitor can hold the array past the release; dropping leaves that writer with the only remaining reference and its write unobservable, whereas returning would let it write into an array another session has re-rented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the histogram release threshold's hysteresis margin The component test supplies its own threshold, so the configured value was pinned by nothing: setting it to one window left the fixture green. A session issuing requests more slowly than the monitor sweeps must keep its histograms across the isolated empty windows that produces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's timing margin At a two-second request interval against a one-second sweep, an overshoot on a loaded box can manufacture the four consecutive empty windows the test asserts cannot happen. At 1.5 seconds the mutation still fires in three seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's absolute stall tolerance Scaling the sweep and the request interval together keeps the ratio the mutation needs while doubling how long the machine may stall before the test manufactures the consecutive empty windows it asserts cannot occur. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reduce the network buffer config surface from nine flags to two The branch had grown nine new command-line switches. Seven of them were tuning knobs for an adaptive mechanism that sizes itself, exposed because they were easy to expose rather than because an operator needs them. Demoted to properties on GarnetServerOptions, reachable programmatically but no longer settable at startup: --network-buffer-size --network-buffer-min-size --network-max-receive-buffer-size --network-send-buffer-min-size --network-buffer-pool-size --session-parse-state-max-retained-args --session-scratch-buffer-max-retained-size None was CONFIG SET-able, so no live mitigation is lost, and none has shipped, so no operator .conf can reference one. Kept: --network-buffer-memory-budget the process-wide ceiling this work adds, and the only knob whose right value depends on the deployment --latency-monitor-precision a genuine accuracy/memory trade-off with no correct default Renamed for direction-explicit consistency with the pre-existing NetworkSendBufferMinSize: NetworkMaxReceiveBufferSize -> NetworkReceiveBufferMaxSize NetworkBufferMinSize -> NetworkReceiveBufferMinSize NetworkBufferSize keeps its direction-agnostic name: GarnetServerBase already exposes a public NetworkBufferSize for the same concept, so renaming here would diverge from established vocabulary. Removing the defaults.conf key would have changed behaviour rather than preserved it. An unset NetworkBufferPoolSize does not mean "unbounded" -- LimitedFixedBufferPool derives a byte ceiling from its per-level entry bound instead, which at the shipped 16-entry default is ~32.5 MB rather than the 64 MB the standalone server has always run. Options.GetServerOptions() now assigns "64m" directly, so the CLI path resolves exactly as before and embedded hosts keep the derived geometry they already had. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tell a refused connection why, and count it A connection refused by --network-connection-limit was disposed with no error written and counted in no metric, so an operator hitting the limit saw an unexplained reset and had nothing to confirm the cause. Send the Redis-compatible -ERR max number of clients reached before closing, and add rejected_connections to INFO stats. The write is gated to non-TLS. A plaintext error to a peer that has only sent a ClientHello is a protocol violation, so the client reports a handshake failure and the operator is pointed at certificates instead of capacity -- worse than a clean reset. Under TLS the metric is the whole remedy. The counter is incremented only in the limit branch, not in the catch that handles sockets dying during setup, so it stays trustworthy in the incident it exists for. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the connection limit process-wide and settable as maxclients The limit was per-listener, so N endpoints gave N times the configured ceiling -- the same defect this branch fixed for buffers, on the adjacent line of GarnetServer.cs. ConnectionLimit sums the listeners' existing activeHandlerCount values rather than maintaining a second counter, so no new accounting can drift; a connection counter that ratchets fails closed, refusing everything until restart. Default is now 10000, matching Redis, and set on GarnetServerOptions rather than only defaults.conf so embedded and test hosts resolve the same value as CLI hosts. CONFIG SET maxclients pushes the new ceiling through StoreWrapper.Servers into the live accept path. Lowering it disconnects nobody, which is also how Redis treats a lowered maxclients. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the refusal message, the counter, maxclients and the shared ceiling Seven assertions. The fixture holds one persistent probe connection so INFO and CONFIG can still be issued after the server is full, and fills only the remaining slots -- a probe that connected on demand would itself be refused. metricsSamplingFreq is set because rejected_connections is published by the monitor's sampling loop and reads "0" unconditionally without it, which would have made every counter assertion vacuous. Commands go over the wire as RESP arrays: Garnet's inline parsing handles single-token commands but silently produces no reply for a multi-token one, so an inline CONFIG GET returned nothing and the first version of these tests failed for that reason rather than for anything about the limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the shared-ceiling test able to fail The second listener refusing is equally consistent with it having reached a ceiling of its own, so the test now makes it accept a connection first. Also adds the drain-and-recover assertion and asserts several connections past the old ceiling when the limit is lifted, since admitting exactly one more is what raising the limit by one would also do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Build the connection limit on net8.0, and document the maxclients semantics System.Threading.Lock is .NET 9+, so ConnectionLimit did not compile for the net8.0 target the solution also builds. A plain object monitor is what the rest of the server uses and is equivalent here -- the lock is taken only by listener registration at startup, never on the accept path. Also records the three semantics an operator will ask about: replica and cluster peer links count against the limit (as they do in Redis, whose accept path adds getClusterConnectionsCount()); lowering the limit refuses new connections rather than culling established ones; and the value is not clamped to the file descriptor limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix final newlines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover maxclients across listeners and TLS rejection accounting MaxClientsReachesTheSharedLimitFromAMultiListenerHost fills the shared ceiling from two endpoints, raises maxclients at runtime, and asserts both listeners admit again, so CONFIG SET is pinned as reaching every listener rather than the one that happened to be first. TlsRejectionsAreCountedEvenThoughTheClientCannotBeTold pins the other half of the TLS gating: a refused TLS connection is closed without a RESP error, but still increments rejected_connections, which is the only signal an operator has on that transport. The TLS fillers are real TLS clients rather than bare sockets. The handshake runs on the accept path, so a connection that never handshakes stalls the accept loop and is never admitted, which makes bare sockets useless as fillers on a TLS listener. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS accept-path stall with concurrent and partial-handshake cases Upstream now starts the handler without waiting for the handshake, so the fix this branch carried is redundant and is dropped in favour of it. The tests are not: they add a peer that stalls after a partial record header, and sixteen concurrent silent peers, neither of which the upstream regression test reaches. Each asserts on a completed RESP exchange rather than on connect, because a connect against a wedged listener is absorbed by the backlog and succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin that a failed TLS handshake releases its admission slot Starting the handshake off the accept loop moves where a handshake failure is noticed: it no longer propagates out of Start into the accept site's catch, so the slot the accept path already took is released by the handshake's own failure path instead. A leak there would let an unauthenticated peer permanently fill a connection-limited server. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Delete the non-discriminating handshake slot test Three mutations could not break it: removing the observer's Dispose, removing Dispose from the handshake failure path, and holding the peer socket open all left it green. Slot release on a failed handshake is over-determined, so the test observed nothing its subject could change. Replacing the fillers with silent sockets does fail, which is the residual the handshake timeout addresses rather than a property to pin here. Also corrects the ConnectionLimitTlsTests rationale, which cited the accept-loop stall that no longer exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound the per-session Lua script cache with an LRU cap Each cached script holds a whole Lua VM, so an unbounded per-session cache lets a session that runs many distinct scripts accumulate VMs for its lifetime. Cap the cache and evict the least recently used entry. Recency is a monotonic stamp updated in place on each hit, so a cache hit costs one store and eviction scans only at capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the script cache bound, its LRU order, and the evicted-script path Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate transaction key entries unpinned on both paths TxnKeyEntries allocated its initial array with GC.AllocateArray(pinned: true) while the growth path used a plain array, so the pinning was silently dropped at the first resize. Every use of the array is through AsSpan, indexing or Array.Copy, and no caller takes a stable address into it, so the unpinned form the growth path has always used is the correct one. Dropping the pinned initial allocation also keeps these arrays out of the pinned object heap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count and report response buffer overflow rentals ReallocateOutput rents a pooled buffer whenever a response outgrows the buffer it is being written into. That traffic was only inferable, so the amount of overflow a given send buffer size causes could not be observed directly. Adds a process-wide rental count, reported as total_output_buffer_rentals in INFO STATS. It is read straight from the counter rather than from the monitor's sampled global metrics, so unlike the neighbouring rows it stays accurate when no monitor is configured. Measured through the new counter: ordinary responses rent nothing, and an oversized response rents exactly one buffer rather than climbing a doubling ladder, which is the property that makes a smaller send buffer affordable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop trailing newlines to match the repo's final-newline convention Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report the output buffer rental count under BPSTATS rather than STATS The counter is read live from its source rather than sampled by the monitor, so under STATS it was the one row that stayed non-zero when no monitor is configured. MetricsDisabledTest asserts every STATS row reads zero in that state, which is Garnet's default, and it caught this. BPSTATS is where the rest of the buffer instrumentation on this branch already lives and is read live for the same reason, so the counter is consistent with its section there instead of being an exception inside a sampled one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count the refusals a capacity wait legitimately incurs WaitForAcceptance polls until the server has capacity, and capacity returns when the server observes the closes rather than when the client disposes. An attempt made before it does is refused for real and advances rejected_connections, so the two tests asserting an exact count were racing a teardown they do not control. Report the refusals from the wait and add them to the expectation, keeping the assertions exact rather than loosening them to a bound that a build which stopped counting would also satisfy. Only a genuine max-clients reply counts, so an unrelated socket error cannot inflate the allowance and mask a miscount. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Capture the session latency histograms once at every reader The monitor sweep nulls LatencyMetricsEntrySession.latency when a session quiesces, but every reader check-then-dereferenced the field, so nothing guaranteed the two reads saw the same value. Both RecordValue overloads run on the session hot path with no lock by design, and GarnetLatencyMetrics.Merge runs on the disposing session's thread under rwLock while the sweep's reclaim holds only disposeLock, so the two do not exclude each other. Load the field into a local once and work from that. Allocate() now returns the array so the miss path reads the field once too. The type's remark claimed the reclaim was safe because a racing writer holds the only reference to the graph; that is true only if the writer captured it, so the remark now states single-capture as the invariant to preserve. Also gate the shrink counters on the budget being enabled, matching OnBufferAcquired: every pool handed no budget shares the Disabled singleton, so counting there put a contended process-wide interlocked write on the shrink path of connections the budget does not govern. Drop two unintended UTF-8 BOMs, and correct the maxclients documentation. Garnet takes every connection on one listener and cannot tell the kinds apart at accept time, so at the limit it refuses replica and gossip links alike; Redis counts cluster bus links but accepts them on a separate port that applies no limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Name the single-capture invariant at the latency record path The two RecordValue overloads load `latency` into a local exactly once because the monitor thread can null it concurrently. The expression reads as a convenience idiom, so state why it is load-bearing at the point a future edit would touch. The invariant itself is documented on ReclaimIfQuiesced, which is the writer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * State the single-capture requirement on the latency field itself The record-path comments guard the readers that exist. An author adding a new reader passes through the field declaration instead, so the constraint belongs there too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tighten the comments added by this work Cut justifications-against-alternatives and narrative framing from the doc comments and inline comments this change introduces, keeping the constraints and invariants a reader needs in order to change the code correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Restate why Recompute needs no fallback on attempt exhaustion Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Inherit TestBase in the concurrent stack fixture Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the buffer pool's stats directly rather than over the wire Serving an INFO reply takes a send buffer and a receive buffer off the free list the assertion reads, so the probe consumed what it reported and the assertion had no margin against its own instrument. Read the listener's pool directly, and poll the size class the assertion reads rather than the byte total, returning the snapshot every assertion is then taken from. Cover the same rule against the pool directly, where nothing can take an entry off the free list between the return and the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address PR review comments on histogram release, scratch cap, and test sends Release the session latency histograms by dropping the reference rather than returning their counts arrays, so a racing recorder cannot write into an array another session has since rented. `LongHistogram.Release()` does the same for the client, whose `latency` field is readonly. `ScratchBufferAllocator.Reset()` no longer enforces the retained-capacity cap at every batch boundary; a `ShrinkCheckpoint()` on the session's coarse countdown does, mirroring `ScratchBufferBuilder`. The session and transaction allocators are both wired to it, with a guard so a WATCH spanning batches keeps its slices. Test socket sends loop until the whole request is written, and the RESP reply reader reads until the expected length has arrived. `NetworkBufferBudget` documents the budget as an adaptive target on the base buffer size rather than a ceiling on the aggregate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the scratch allocator's checkpoint interval and its outstanding-slice guard A functional WATCH test cannot observe a buffer released underneath a live slice: the pinned array is collectable but its bytes stay readable until a collection reuses them. Both invariants are asserted on the allocator directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Update the scratch allocator tests for the checkpoint release contract The allocator now keeps its current buffer across Reset and releases an over-sized one at a ShrinkCheckpoint, so the fixture asserts the retained capacity at each point and drives two checkpoints to clear the hysteresis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Floor the above-max receive shrink at the largest poolable size The residual left after a fully consumed request is zero, so sizing the replacement from it always resolved to the base. A connection issuing one oversized request per batch then re-grew through every intermediate size class on every request. Floor the replacement at the largest poolable size at both the socket and TLS transport sites; the idle hysteresis path still takes it down to the base once the traffic no longer needs the capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the client latency histogram once at every consumer Release drops the counts array while the reply-processing thread may still be recording, so the record path tolerates a released array and the copy path reads the field once instead of testing IsReturned and then dereferencing. Also make the quiesce-threshold test observe empty windows: it recorded a value and never cleared it, so the windows it asserted over were not empty and a threshold that never restarted its run still passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the released-histogram contract on the record and copy paths Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Derive the copied histogram's tracking values from the counts it carries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Correct the rationale for deriving the copied histogram's total Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS transport above-max receive replacement A TLS connection decrypts into a second receive buffer that grows through its own code path, so the above-max release there was unexercised while its plain-socket twin was pinned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Gate the above-max receive floor on budget pressure The above-max branch releases a buffer the pool cannot recycle. Flooring the replacement at the largest poolable size matches origin/main and saves a connection that needs the capacity on every request from re-growing through every size class, but it must not apply while the budget is binding: the target is derived from a buffer count, so a connection parked at the maximum costs the same single unit as one at the floor and the budget gets no feedback from it. Under pressure the replacement is now the adapted base, which the pool keeps and the budget can see. Also capture the counts array once in ClearCounts and CopyCountsInto, read the client call count off the snapshot the percentiles come from, and record the single-read contract on the field itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS half of the above-max pressure gate The plaintext and TLS receive paths gate the above-max release independently. Only the plaintext site was exercised, so the TLS gate at NetworkHandler.cs:801 could be removed with the whole fixture still green. Parameterize the test over both transports. Guard the counts array in CopyCountsInto, matching the single-read invariant documented on the field and the five sibling capture sites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop the scratch retention assertion that measures test ordering `ScratchBufferCapDoesNotChangeLargeArgumentResults` compared a per-session memory figure across two arms and failed intermittently. Two measurements show it cannot discriminate: swapping the arms moves each figure by about 130 KB per session and flips the sign of the difference, because the arm that runs second also measures the residue of the server instance the first one disposed; and releasing the buffer at every reset rather than at the checkpoint -- the churn such an assertion claims to catch -- moves it by less than that, because churn leaves garbage the forced collection reclaims rather than retention. The test keeps both arms and the reply validation they drive, which is what its name has always claimed, and the remarks state why no magnitude is asserted. `NetworkBufferBudget` documents the per-direction floors as the third reason the budget is a target rather than a ceiling: once the target reaches them, the aggregate grows with the connection count again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Sample the shrink churn either side of the batch boundary Comparing capacity across iterations cannot see the pathology the test exists to catch: a policy that released on every batch regrows before the next sample, so consecutive post-boundary capacities are equal and the churn counts as zero. Sampling before and after each boundary counts every release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Say what the latency precision number actually buys The setting is HdrHistogram significant digits: relative accuracy at every magnitude, not a fixed time unit. Stating it that way, with the memory each step costs, lets an operator pick a value without knowing HdrHistogram. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Pin that an over-sized request is accepted whole The receive buffer grows to fit and maxReceiveBufferSize bounds only what a connection retains, so a chunk far above it must still be accepted. Dribbled in small slices so the request sits partially buffered across many receive passes, where the above-max shrink runs against a live partial request. Also drop a qualifier from the latency precision help text. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Keep the byte-order marks as main has them Earlier edits flipped the BOM on seven files, which showed up as a one-line change to each file header with no content behind it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Wait for the buffer release instead of racing it A receive releases its buffer in UpdateNetworkBuffers, which runs after Process has already written the reply, so a client that has read its response can still be ahead of the release. Sampling BPSTATS once at that point read the pre-release numbers on a loaded runner and failed with the full oversized buffer still live. Poll until the counters settle. The wait uses only the out-of-band INFO connection, so it does not hand the connection under test the extra receive that would let a buffer released one pass too late off the hook: suppressing the immediate release still fails the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Fold the two receive-buffer shrink paths into one policy The network and transport receive buffers ran the same three-regime shrink policy over their own occupancy and countdown, written out twice. Extract it as TryPlanReceiveBufferShrink, which owns the countdown bookkeeping and the budget's shrink accounting, and leave each caller with nothing but the buffer swap. Rename both entry points to TryShrink*, and give the transport buffer a ShrinkTransportReceiveBuffer mirroring ShrinkNetworkReceiveBuffer. Drop the cached configuredReceiveBufferSize/configuredSendBufferSize fields and read them off networkBufferSettings, as maxReceiveBufferSize already is on the same paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Keep the configured base sizes cached on the handler BaseReceiveBufferSize is read on every receive, so reaching the configured size through networkBufferSettings puts a dependent load on that path. BDN Network.BasicOperations.InlinePing measures it at +1.1ns (89.8 -> 90.9) on net10 and +1.9ns (85.7 -> 87.6) on net8, the net8 range clearing baseline entirely. Restoring the fields returns both to baseline (89.8 / 85.8). The shrink consolidation is unaffected and stays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Capture the metrics array in ReclaimQuiescedHistograms Every other method on the session captures the metrics field once before dereferencing it. ReclaimQuiescedHistograms re-read it for the null check and again inside the loop, so it relied on the dispose lock for that safety where its peers do not. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Checkpoint the transaction scratch allocator during AOF replay Replayed transaction procedures run their prepare phase against the watch API, and every watched key is copied into the transaction scratch allocator. The replay session never reaches the network batch boundary that drives the shrink checkpoint, so the buffer one wide procedure grew stayed pinned for the lifetime of a replica. The replay path now reports a boundary per AOF record and the window is counted in TransactionManager, so the caller does not carry the cadence. The allocator shrinks only when nothing is outstanding, so a boundary reported mid-transaction is safe. Also apply two review nits: a u8 span literal for the max-clients error instead of a materialized array, and a corrected comment on the connection-limit registration lock, which was crediting the lock with reader safety that the volatile publication provides. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Release the session scratch allocator at the AOF replay boundary Replayed procedures grow both scratch allocators, not just the transaction one: the prepare phase copies watched keys into the transaction allocator, and the procedure builds its arguments from the session allocator. The session allocator is reset at the start of every procedure rather than when one ends, so a completed procedure leaves it oversized with a non-zero offset, and a shrink checkpoint declines in that state. Reset it at the replay boundary, as the network batch boundary does, and only outside a transaction, whose slices are still live. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Rename the buffer shrink hook off "checkpoint" and count replay records at the call site "Checkpoint" is load-bearing in this codebase: Tsavorite uses it for durable, versioned, recoverable snapshots, and Garnet's AOF replay handles genuine CheckpointStartCommit markers a few lines from where this hook was called. The scratch-buffer hook means something unrelated -- periodically sample a buffer's capacity and release it if it did not grow -- so it is renamed to Trim, matching ArrayPool<T>.Trim, which applies the same hysteresis. The replay hook also counted its own interval while the network one had the caller count, so TrimAfterReplayedRecord ran on every record but trimmed on one in sixty-four. The counter moves to AofReplayContext, alongside the session it governs, so both paths show the cadence at the call site and TrimReplayBuffers means "trim now". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 --------- Co-authored-by: badrishc <badrishc@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 | 1 个月前 | |
Bump version to 2.2.4; combined RESP fixes: HINCRBY overflow, integer-parse protocol error, expired-key checks, large PEXPIRE, SET KEEPTTL (#2259) | 10 天前 | |
Support infinities in SortedSet (#1268) * Add support for positive/negative infinity in sortedset add/increment operations. * Add more tests. * Address some review comments * Handle INCRBYFLOAT so we can get rid of TryParseDouble calls, and pass on the right error where the existing value is an infinity. * Fix tests. * Follow review suggestion. * Address review comments. * Create an IncrementByFloat api to allow simple use without having to do stackalloc etc. * We can avoid back-and-forth conversions in a new API. * VS variable substitution also changed output message it shouldn't have changed. --------- Co-authored-by: Tal Zaccai <talzacc@microsoft.com> | 1 年前 | |
Fix CI BDN allocation flakiness (#1989) * [Test] Fix BDN allocation-gate flakiness for Set/JSON benchmarks Two independent causes of intermittent BDN allocation-gate failures: 1. Server GC with concurrent (background) collections corrupts BenchmarkDotNet's process-wide MemoryDiagnoser: a Gen2 GC in the measurement window makes the per-op allocation swing wildly (0 to 8x). Use blocking (non-concurrent) GC so the measurement is deterministic. Fixes the JSON ModuleJsonGetCommand spikes. 2. SAddPopSingle emptied its set every iteration (SPOP removes the last member), so the key was deleted and recreated, churning the object-store log and allocating 16MB pages that intermittently landed in the measurement window. Seed key1 with a keeper and use deterministic add/remove so the set is never emptied and the ops stay in-place (no log growth). Renamed to SAddRemSingle since it now exercises SADD/SREM, and tightened the SAddRem/SAddRemSingle thresholds to reflect the ~10x lower, stable allocation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [RESP] Reduce JSON.GET allocation, near-zero for the root path JSON.GET allocated ~536 B/op, dominated by: JSONPath parse of "$" (160), LINQ Sum + boxed WriteBulkString (176), a fresh SerializeToUtf8Bytes byte[] (144), a new List<byte[]> (32), and a path string (24). The high gen0 churn also fed the allocation-measurement variance. - Add a root ("$") fast path (GarnetJsonObject.TryGetRoot) that serializes the whole document into thread-static reusable ArrayBufferWriter/Utf8JsonWriter and writes straight to the RESP output, skipping the path string, JSONPath evaluation, the per-item byte[], and the List. - Add a non-boxing WriteBulkString(ReadOnlySpan<byte[]>) overload in RespWriteUtils/RespMemoryWriter and use it from the JSON GET reader, removing the LINQ Sum and boxed IEnumerable enumeration (helps all JSON.GET paths). ModuleJsonGetCommand: 53600 -> 2400 B (-95.5%, ~24 B/op); latency 143 -> 70 us. Non-root paths (deep/array/filter) drop 5-21% from the WriteBulkString change; their remaining cost is in the custom JSONPath SelectNodes. All 44 JSON command tests pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Mark Set AOF allocation gates warn-only The AOF variants of SAddRem/SAddRemSingle allocate identically to the None and ACL variants (net10 3200 B, net8 6400 B); the AOF enqueue path adds no per-op heap allocation. Under Server GC the MemoryDiagnoser can occasionally over-count one GC allocation-context quantum (~18 KB) when a background collection lands in the AOF measurement window, producing rare spikes (e.g. 21699 B) unrelated to the workload. Keep None/ACL hard-gated at 6400 and make the AOF variants warn-only, matching the existing treatment of the larger noisy AOF set operations in this config. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Stabilize SortedSet pop/remove BDN benchmarks ZPopMax, ZPopMin, ZMPop and ZRemRangeByScore ran against a single-member sorted set, so each iteration emptied and recreated the key, churning the object-store log (~48-113 KB per op, with rare 16 MB log-page spikes up to ~520 KB). Give each its own key seeded with a keeper member "k" scored so the benchmarked command removes the re-added "d" but never "k", keeping the set non-empty and the operation in-place. Local net10 None: ZPopMax/ZPopMin 12000 B, ZMPop 20000 B, ZRemRangeByScore 48000 B. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Right-size over-provisioned object-type BDN warn thresholds Lower the warn-only allocation thresholds for Hash, ZAddRem and the newly stabilized SortedSet pop/remove benchmarks to reflect measured allocations (~2x the stable value) instead of the previous 10-20x over-provisioning (e.g. HSetDel_None 120000->19200, HMSet_None 114000->12800, ZAddRem_None 149000->32000, ZPopMax_None 48001->24000). The _AOF thresholds for ZPopMax/ ZPopMin keep headroom (74000) for the net10 AOF MemoryDiagnoser over-count; denied-path ACL keys stay at the 6400 baseline. Validated against CI run 30191658574 results: no hard fails, no new warnings on the adjusted benchmarks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Make Set add/remove allocation gates warn-only on all params SAddRem/SAddRemSingle allocate ~3200-6400 B but, being object-store RMW benchmarks, occasionally over-count by ~1 GC allocation-context quantum (~17-18 KB) under Server GC when a background collection lands in the measurement window (observed net10 None 20755 B, net10 AOF 21699 B). The over-count can hit any param, so a tight hard gate flakes. Make the None and ACL variants warn-only to match the already-warn-only AOF variant and the object-store benchmark convention (Hash/List/SortedSet are all warn-only). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Set object-type BDN warn thresholds to expected measured values Replace the ~2x/headroom placeholders with the actual measured allocations after stabilization (the gate adds its own +10% tolerance): SortedSet ZAddRem 15200, ZMPop 20000, ZPopMax/ZPopMin 12000, ZRemRangeByScore 48000; Hash HSetDel 9600, HMSet/HMGet 6400, HSetNx/HIncrby/HStrLen 3200, HScan 776. Denied-path ACL keys stay at the 6400 baseline. These are warn-only gates, so the intermittent net10 Server-GC MemoryDiagnoser over-count now surfaces as a warning rather than being masked by an inflated threshold. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Restore SAddPopSingle as a stable SADD+SPOP BDN benchmark Revert the earlier SAddPopSingle->SAddRemSingle rename, which had turned a distinct SPOP benchmark into a near-duplicate of SAddRem (both SADD+SREM) and dropped SPOP coverage. Instead keep SADD+SPOP but make it stable: each SADD uses a distinct member (a pool of batchSize members, far larger than the set) so a re-added member has always been popped already and the set never shrinks, and key1 is seeded with several keepers so it never empties. With the key never deleted/recreated there is no object-store log churn, giving a deterministic 7200 B/op across all params and both frameworks (measured; verified stable over repeated runs). Add +spop to the benchmark ACL grant so the ACL variant runs the pop in-place rather than leaving SADD to grow the set unbounded. This preserves the SAddPopSingle chart series (no rename) and keeps SAddRem (SREM) and SAddPopSingle (SPOP) as distinct operations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [RESP] Fix JSON.GET length overflow and unbounded root-buffer retention Two issues in the JSON.GET fast path, found in review: - The response length was summed in an unchecked int, so a >2GB result wrapped negative and bypassed the RESP writer bounds check (the prior LINQ Sum was overflow-checked). Sum in long and reject results above int.MaxValue. - The thread-static root-GET ArrayBufferWriter only grows; ResetWrittenCount keeps the backing array, so one large JSON.GET permanently inflated a long-lived session thread's buffer. Release the buffers once they exceed a 64 KB retention cap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Make SAddPopSingle deterministic; tighten BDN gate coverage - SAddPopSingle: replace the 100-distinct-member SPOP batch with `SADD key a b; SPOP key`, which adds two members and pops one so the set deterministically oscillates between 2 and 1 members and never empties -- no reliance on pool-size-vs-set-size probability. Stable 10400 B (net8) / 7200 B (net10), and faster than the distinct-member version. - Config: SAddPopSingle -> 10400 (covers net8); fix two dead gate keys whose `WARN-ON-FAIL_expected_` prefix never matched the parser (LPushPop_ACL, SScan_AOF); ModuleJsonGetCommand -> warn-only 2400 (was hard 360000, which no longer detects regressions after the near-zero root-path optimization). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Correct BDN GC-job comment and note Workstation-GC follow-up The prior comment claimed Server GC keeps the allocation measurement deterministic; review showed the Server-GC allocation quantum still intermittently inflates MemoryDiagnoser (notably on .NET 10), which is why the object-store allocation gates are warn-only. Note the Workstation-GC job as the follow-up that would allow deterministic hard gates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Make ModuleJsonGetRecursive allocation gate warn-only ModuleJsonGetRecursive is a recursive-descent JSONPath query that genuinely allocates ~30-47 MB with large run-to-run variance (it passed earlier runs below the 33 MB hard-gate ceiling and exceeded it at 47 MB in another). A tight hard gate on a benchmark that variable produces false-positive CI failures, so make it warn-only. The large baseline itself comes from the per-node allocations in the custom JSONPath engine (non-root paths) -- reducing that is a separate follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Objects] JSON.GET: write single-path results directly, cutting per-node allocations The single-path JSON.GET path built a List<byte[]> with one byte[] per matched node, then concatenated them into the RESP bulk string. For a recursive wildcard ($..*) matching ~181 nodes this dominated allocation (BDN ModuleJsonGetRecursive = 2.42 MB/op locally, and up to 47 MB under Server-GC over-count on CI, causing allocation-gate flakes). Add GarnetJsonObject.TryGetToWriter: evaluate the JSONPath and serialize the matched nodes as a single JSON array straight into the thread-static buffer/writer already used by the root ("$") fast path, then write once as a RESP bulk string. Eliminates the List and the per-node byte[] arrays. Output is byte-identical (verified: 44 JsonCommandsTest cases incl. $..author multi-node, wildcard, filter, and empty-match paths). Result (net10 None, local): ModuleJsonGetRecursive 2.42 MB -> 0.52 MB (-78%, deterministic across runs) and 6.20 ms -> 4.59 ms (-26%). The residual 0.52 MB is the lazy-iterator JSONPath engine (SelectNodes/SelectMany) -- a separate follow-up. Also apply the reviewer nit: the residual formatted/multi-path length guard now checks Array.MaxLength (the real byte[] cap) instead of int.MaxValue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Right-size JSON.GET allocation gates to post-fix expected values After the JSON.GET direct-writer change, re-baseline the JsonOperations gates to the measured post-fix allocations (max of net8/net10; the gate applies +10%): - ModuleJsonGetRecursive 50000000 -> 521600 (real 521600, was 2421600 pre-fix) - ModuleJsonGetDeepPath 600000 -> 44000 (HARD -> warn; was 13x over-provisioned) - ModuleJsonGetArrayPath 370000 -> 53600 (HARD -> warn; was 7x over) - ModuleJsonGetFilterPath 770000 -> 64800 (HARD -> warn; was 12x over) - ModuleJsonGetArrayElementsPath 800 -> 800 (HARD -> warn; unchanged value) All JSON.GET path gates are warn-only, matching the object-store policy: the allocation MemoryDiagnoser over-counts under Server GC when a background GC lands in the measurement window. Data (CI runs 30221983810 clean, 30219601330 spike) shows this is proportional to per-op allocation volume: only the heavy Recursive ($..* = ~181 nodes) spikes (net10: 15.4M/47.1M vs 2.42M base); the light ops stayed dead-stable even in the spike run. Cutting Recursive's real allocation 4.6x shrinks its spike ceiling proportionally; the residual net10 spikes now warn (non-blocking) instead of failing. A deterministic hard gate needs the deferred Workstation-GC job (noted in Program.cs). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Use Workstation GC for BDN so allocation is measured accurately Root cause of the roving BDN allocation-gate flakes (FilterPath 61k->407k, Recursive->47M, and the object-store spikes): under Server GC, BDN's MemoryDiagnoser reads GC.GetTotalAllocatedBytes(precise:false), whose counter includes the unused allocation-context budgets across Server GC's per-core heaps. When a gen1/gen2 GC resets those contexts between BDN's before/after readings, the counter jumps by N-heaps x budget -- a pure measurement artifact that roves across whichever benchmark catches a context reset mid-measurement. Evidence: spiked "GC: 25 1 1 1667621296 4096" (407134 B/op) vs clean "GC: 32 0 0 250675200 4096" (61200 B/op) for the same benchmark in different runs. Fix: run the benchmarks under Workstation GC (single heap, small budget), which makes GetTotalAllocatedBytes accurate/deterministic. Verified locally: allocation is byte-identical to the clean Server-GC value (FilterPath 61200, Recursive 521600, ZAddRem 15200, SAddRem 3200, HSetDel 6400) and the spikes cannot occur. These are single-client benchmarks, so Server GC provided no latency-fidelity benefit; measured latency is equal-or-slightly-better under Workstation GC (charts are alert-only and this is an improvement, so no alert). This enables restoring tight hard allocation gates (follow-up: re-baseline gates from a Workstation-GC CI run). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Restore hard allocation gates for benchmarks stabilized by Workstation GC With Workstation GC, MemoryDiagnoser now measures allocation deterministically (CI run 30233233102: each of these benchmarks reported an identical value across all 16 rows -- 2 OS x 2 runtimes x 4 params -- with zero spikes). Convert the verified-stable, substantial benchmarks from warn-only back to hard `expected_` gates so real allocation regressions block CI again: - JSON GET: Command 2400, DeepPath 44000, ArrayPath 53600, ArrayElements 800, FilterPath 64800, Recursive 521600 - Set: SAddRem 6400, SAddPopSingle 10400 - SortedSet: ZAddRem 15200, ZMPop 20000 (None/AOF), ZPopMax/ZPopMin 12000 (None/AOF), ZRemRangeByScore 48000 (None/AOF) - Hash: HSetDel 9600; List: LPushPop 11200 (was 14400) Thresholds are the max of the net8/net10 measured values; the gate adds +10%, so the deterministic values leave headroom and cannot false-fail. Left warn-only: the ACL-denied pop paths (NOPERM), the churn benchmarks that empty+recreate a key (ZRemRangeByLex ~112k, ZRemRangeByRank ~86k -- genuine allocation variance, not a measurement artifact), and the near-zero read ops (many now correctly report "-" under Workstation GC, which the gate treats as 0). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Address PR review: tighten BDN comments - Program.cs: shorten the GC comment to a present-tense, factual statement of why Workstation GC is used (drop transient spike figures and narrative). - SetOperations.cs: SAddPopSingle comment now states the invariant it relies on (the set is never emptied and key1 is never deleted/recreated) instead of the inaccurate "oscillates between 2 and 1 members" claim (key1 is also seeded with a keeper and shared with SAddRem). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a * [Test] Convert deterministic object-store gates from warn-only to hard Now that Workstation GC makes MemoryDiagnoser deterministic, warn-only is no longer needed for benchmarks with stable allocation. Evaluated every warn-only Set/SortedSet/Hash benchmark against TWO independent Workstation-GC CI runs (30233233102, 30236831459): 62 are deterministic (within-runtime spread <=3% in both runs) and are converted to hard `expected_` gates at the measured max (the gate's +10% leaves headroom). Many old warn thresholds were 5-13x over-provisioned because they absorbed the Server-GC over-count (e.g. ZCount 150000->16800, ZRange 320000->24800) -- the new hard gates are both tight and accurate. Kept warn-only: - Genuine run-to-run variance (churn that empties+recreates a key): ZRemRangeByLex, ZRemRangeByRank, ZRangeStore, and the near-zero oscillator HIncrbyFloat. - Pure-read ops that allocate 0 (show "-"): a hard gate at 0 has no absolute headroom, so these stay warn-only to avoid false-fails on any measurement blip. RawString(LTM)/Lua/Script warn gates are left as-is (their results were not part of this evaluation). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a | 3 个月前 | |
Support for Parallel Replication (#1556) * implement sequence number generator * More work on LogRecord.RecordLength * format code * Finish LogRecord.RecordLength * add checkpoint marker process coordination * add fuzzy region flag to replay context * trigger exception when remove of checkpoint barrier fails * throw exception if SignalAndWait times out * add flag to differentiate between sharded and single log AOF * AOF explorer * wait for aofSync and refreshTail tasks on Dispose * validate base replication tests v1 * Remove ObjectDb* from Sample*Txn * test fixes for disk-based replication v2 * add synchronized replay for remaining operations (i.e. FLUSHDB, STREAMING_CHECKPOINT) * sequential node dispose on teardown * ensure dispose client precedes iterator * MultiLevelPageArray stress fixes; improved LogRecord and RecordDataHeader ToString() * WIP * Fix GetObjectLogSegmentSize() * wait for aof sync driver and connection to drop before recovering * consolidate reset replica replay task group * update docs * Fix a couple comments * address data loss on synchronization * use unified synchronized operation for regulard txn replay * consistent read for scan commands * Trim ObjectIdMap arrays on ClearPage * remove unused variable * fix single log tests * fix sharded log tests * adding comments to AofReplayCoordinator * adjust ROLE command response for sharded log implementation * WIP; multi phase read consistency protocol * added basic consistent read protocol context * Add transactional consistent read context * md readmed for mlog design * add design notes * WIP; change session at state change * allow disklogRecord upserts for streamless sync * implement consistent read protocol for scan commands * add sharded diskless sync tests * eliminated interface declaration for APIs with consistent read * cleanup and refactoring * remove sharded option since we have AofBench * update tail witness task to ping replica for latest replayed sequence number information * remove unused code * wip; logical subtask replay * WIP; re-design AofHeaders * WIP; update sequence replay info * wip; v1 parallel replay within physical sublog * wip; running sharded log tests with single replay task * wip; refactor to use virtual sublog replay state and a virtual sublog idx calculation * wip; update coordinated operations to trigger when multilog is enabled * wip; add multi-replay test for single key operations * wip; cleanup * remove AofExplorer * wip; index coordination barriers with distinct key to mixing separate transactions on same session * add indexing for coordination barriers using sequence number * rename sublog count to physical sublog count * remove cproj ref * separate AofBenchType * wip; mlog description * wip; mlog description * wip; adding comments and writing dev replication website content * wip; updating documentation and description in code * wip; AofSyncTask section * wip; replay details summary and fixes * wip; renaming and updating todo items * wip; skip hashing when configured with one physical sublog and multiple replay tasks * add test for custom txn and multi-exec txn with parallel replay * refactor/rename and cleanup replica replay code * wip; adding bulk consume recovery * renaming and cleanup * simplify intra page replay and synchronization primitive * implement recover intra page replay * assert throw when configuring multi-log without FastCommit * add description of prefix consistent recovery * push log specific operations in GarnetLog instance * wip; * implement prefix consistent recovery * refactor code to use single log when multi-replay is enabled with one physical log * remove cookie generation callbacks * cleanup and documentation write-up * more documentation * refactor advance time background task * TaskCompletionSource based wait for read consistent read * fix read protocol edge cases * code comments and cleanup * add cluster command to retrieve key sequence number for debugging purposes * PR description initial draft * advance time using high precision timestamp and background task * PR details * fix formatting * revert HashUtils location * HashUtils mv * fix formatting * cleanup * single log legacy implementation updates * add docs for mlog key time * legacy checkpoint cookie serialization support * fix test delay * run aofsynctask directly with single log * append replication vector to ROLE command when using multi-log * add support for intra-cluster connection identification and fix failing management tests * set client info when clientname parameter != null * support negative values in AofAddress * add ConsistentReadContext definition into Globals * fix formatting * fix ClusterDisklessSyncResetSyncManagerCts * fix ClusterResetHardDuringDisklessReplicationAttach * fix consistent read context bugs * wip; refactor read-protocol integration * Implement read key batch consistent protocol * improve robustness of background advance time consumer * remove unused playground project * cleanup clientname setup * fix tsavorite ISessionFunctions * remove unused assembly * simplify AofAddress and check for correctness * fix typo * eliminate bytes from informational message * simplify and cleanup leader barrier * cleanup LeaderFollowerBarrier * ensure commit for multilog recovery * replace tcs with semaphore slim * fix ClusterReplicationCheckpointCleanupTest * Fix copy-paste bug sending beginAddress instead of tailAddress in disk-based replica sync (#1633) In TryReplicateDiskbasedSync, ExecuteClusterInitiateReplicaSync was sending beginAddress.Span for both the aofBeginAddress and aofTailAddress parameters. This was introduced in commit 6fb99e58 when converting from ToByteArray() to Span-based calls. The primary uses the replica's tail address to compute the AOF sync replay range. With both parameters being the begin address, the primary couldn't determine where the replica's AOF actually ended, causing the replica to never receive AOF records and remain stuck at offset 64 (kFirstValidAofAddress). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix ClusterResetHardDuring tests by using ConfigureAwait for ExceptionInjection * enforce ConfigureAwait(false) across Garnet * fix AofBench * add website description * add migration GarnetLog upgrade test * fix formatting * fix website links * add object option for replication scheme upgrade test * set default logging to Error * pre-compute enforceConsistentRead * add AOF option for cluster BDN * update global.json * consolidate enqueue calls * fix upgrade test null check * add custom stop bulk consume callback * fixing ClusterReplicationMultiRestartRecover * fixing base replication tests * use throttle to check consumer liveness during bulk consume * refactor advance time background task * ensure WorkQueueLIFO waits for process bg task to complete on dispose * add activeWorkerMonitor * cleanup and refactoring * wip; adding tracking * fix hanging ClusterReplicationCheckpointCleanupTest * fix Tsavorite formatting * fix Garnet formatting * fix breaking build * improve log message * throw the caught AsyncFlushSnapshotPage exception * ensure failover suspends replica only tasks * usingSharded when replaying with multiple tasks * add no-inlining exception throw * fix AofBench to work with AofReplayCount * wip; add variant to parallel replay a physical sublog * wip; fixing ClusterSRNoCheckpointRestartSecondary * fix defaults * classify sharded tests under same namespace * throw at TakeOverAsPrimary to fix test * log startAddress on dispose of AofSyncDriver * add better logging at ClusterAppendLog initialization * reuse parseState at replayOp * avoid cache invalidation when updating replicationOffset * ensure get slots parsing deals with unstable slots * nit; cleanup * fix formatting * better logging * eliminate monotonic updates on the enqueue hot path when tracking sequence numbers for advance time. * make aof-replay-task-count default=1 for AofBench * add ReplayNoResp * add ReplayDirect * fix Enqueue Bench * add better logging for replica receive primary AOF stream * fix InProc Bench wait for server * implemented targetted logging API for testing * enable LogPrimaryStream for ClusterSRNoCheckpointRestartSecondary * refactor cluster cmdstrings * remove unused methods * cleanup * fix logging event flag checking * update aof bench * optimistically return if not ReadWaiters * improving AofBench * Pass around keyHash and re-use it during replay * remove notes * increase thread pool for cluster tls replication * expand logging at AofSyncTask initialization * wip; cleanup replay path * use owned epoch for multilog * use owned epoch for tsavorite log and cleanup aof recover * make vector fixes for replication * fix aof gen * JIT replayOp refactor * fix range index AOF enqueue * use barrier on AofSync connection with multi-log only * adding async for connect and gossip * addressing first set of comments * addressing second set of comments * add throw cancellation * make light epoch in client compatible with tsavorite epoch implementation * simplify object RMW enqueue * separate appendLog initialization --------- Co-authored-by: TedHartMS <15467143+TedHartMS@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> | 6 个月前 | |
Initial commit | 2 年前 | |
Bound per-connection memory: adaptive network buffer budget, session buffer caps, and connection admission (#2157) * Bound connection-scaled network buffer memory Per-connection network buffers were unbounded in practice: the receive buffer only shrank when it exceeded maxReceiveBufferSize, so a buffer that doubled 128K -> 256K -> 512K never shrank and every connection permanently retained its all-time peak. The TLS transport buffer had no shrink path at all. GarnetServerTcp also discarded its networkBufferSize argument and hardcoded 128 KB, so the per-connection floor was unconfigurable. - Add --network-buffer-size, --network-max-receive-buffer-size and --network-buffer-pool-size, plumbed through GarnetServerOptions into GarnetServerTcp, which now honors them. Defaults match today's behavior. - Shrink a grown receive buffer to the smallest size class that keeps 2x headroom over the buffered bytes, after 16 consecutive receives that fit. Buffers above maxReceiveBufferSize skip the hysteresis since the pool cannot recycle them. - Add the same shrink path for the TLS transport receive buffer, called only from the reader sites that already consider doubling. - Track liveBytes, peakLiveBytes, pooledBytes and a shared byte budget in LimitedFixedBufferPool, exposed through INFO BPSTATS. - Return the outstanding responseObject in DisposeNetworkSender so a session torn down mid-response does not leak its PoolEntry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Document new network buffer options and add config parsing tests Adds the three --network-buffer-* options to the configuration table, warns instead of silently correcting an inverted size pair, and covers defaults, overrides, power-of-2 rounding, the inverted pair and invalid values in GarnetServerConfigTests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bound per-session parse-state and scratch buffers A RESP session sizes SessionParseState.rootBuffer to the largest array arity the client has ever sent and keeps it for the life of the connection. One high-arity command therefore enlarges the session permanently, and the cost scales with connection count. The same ratchet exists in ScratchBufferBuilder, whose Reset() zeroes the offset but never releases the backing array. Add BufferShrinkPolicy, a shared cap-plus-hysteresis helper, and apply it at both sites. A buffer above the cap is released only after the configured number of consecutive batches that did not need the extra capacity, so a session with recurring large requests keeps its buffer. Both caps are configurable and default-on: --session-parse-state-max-retained-args (default 1024) --session-scratch-buffer-max-retained-size (default 64k) Measured end to end with a forced compacting GC, 200 sessions each issuing one 20,000-argument EXISTS followed by sustained single-key work: 327 KB/session retained before, 98 KB/session after. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover buffer shrink against transaction and watch aliasing The batch-boundary shrink releases a pinned array, so anything holding a reference into it across that boundary would dangle rather than merely read stale bytes. Two paths cross the boundary by design: a MULTI..EXEC window, which re-parses queued commands from the network buffer, and WATCH, which copies key bytes into the transaction scratch allocator so they outlive the receive buffer. Add tests that force the shrink to fire inside both windows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized response elements instead of killing the session A response element larger than the send buffer cannot be written atomically. The chunking helper for this already exists and SCAN uses it, but several commands still wrote unbounded user data through the atomic path, where TryWriteBulkString cannot make progress and SendAndReset throws, killing the session mid-response. KEYS fails today with any key larger than the 128 KB default send buffer. Give the shared WriteBulkString helper an atomic-first fallback: write in place, retry once on a flushed buffer, and only chunk an element that cannot fit an empty buffer. Small elements keep exactly today's path, so multi-element replies pay nothing. Route KEYS, COMMAND GETKEYS, COMMAND GETKEYSANDFLAGS and the pub/sub channel and pattern echoes through it. Two related defects found while covering this: ClusterSession.SendAndReset silently did nothing on an empty buffer, so a caller's retry loop spun forever rather than failing. It now throws, as the RESP session already does. MainStoreOps.GETDEL asserted its output was inline immediately after the RMW, before the spill-to-pooled-memory case is handled. Any value large enough to spill tripped it, so GETDEL of a large value failed on every debug build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Enter the chunking fallback when the send buffer is empty The fallback added for over-sized response elements flushed before retrying, but SendAndReset throws when there is nothing to send. That is exactly the over-sized case where the element is the first thing written to the response, so the chunking path was unreachable there and the session still died. Every earlier test emitted an array header first, which left the buffer non-empty and masked it. Flush only when output is actually buffered, otherwise chunk directly. CLIENT GETNAME is the reachable instance: the name is the whole response, so a name larger than the send buffer killed the session. Give the ASCII bulk-string helper the same fallback and route CLIENT GETNAME through it. Move both fallbacks to NoInlining helpers so the 49 call sites of the inlined fast path keep their previous code size. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate latency histograms on first record Every session was given twelve HdrHistograms when its connection was accepted, before it had recorded anything. At two significant digits each rents a 32 KB array from the shared pool, so a connection cost 384 KB of latency monitoring whether or not it ever issued a command. A dump of a server holding several thousand connections showed 887 MB in these arrays. Allocate the histograms for a latency type on the first value recorded into it. A session typically records into three of the six types, and an idle connection records into none, so it now pays nothing. Add --latency-monitor-precision to size the histograms. It defaults to 2, which is what the histograms were built with, so accuracy is unchanged unless it is lowered. Dropping it to 1 shrinks each histogram several-fold in exchange for reporting percentiles to 10% rather than 1% resolution. Fix three defects on the release path that lazy allocation widens the window for. LongHistogram.Return() now drops its reference to the array it hands back, so a later record throws rather than silently writing into an array the pool has re-rented to another session. The session's pending work is cancelled before its histograms are released rather than after. GarnetLatencyMetricsSession publishes its metrics as unavailable before releasing them, and the hot path reads the field once and checks it. Dropping the array reference exposed a use-after-return in GarnetClient, which copied its histogram after Dispose had already handed the array back; it now reports no metrics instead of reading a recycled array. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the latency precision to the histograms it sizes The precision knob was only covered by tests that constructed histograms directly, so a broken binding between the command line and the allocation would have gone unnoticed and silently degraded resolution. Assert the size the server reports through LATENCY HISTOGRAM, the size of the per-session histograms reached through the live sessions, and that the option parses to 2 by default and rejects values outside the 0-5 range HdrHistogram accepts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Wait for the session to record before reading its histograms A session records its latency after the reply has been written, so a test that read the histograms as soon as the reply arrived could find none allocated yet. Poll until one appears. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Publish a process-wide budget for live network buffers The buffer pool's byte ceiling bounds only its idle free list. Buffers checked out by live connections are bounded by nothing, so their footprint is connections times per-connection size, which is what grew unchecked with thousands of clients. Add NetworkBufferBudget, which divides a configured total by the live buffer count and publishes the largest base size that keeps the two consistent. It is a division rather than a control loop: buffer sizes move in factor-of-two steps, so any controller with a narrower deadband cannot converge and limit-cycles between two size classes. Growth therefore requires the quotient to reach twice the published target, matching the step it actuates. One budget is shared by every listener. Each endpoint previously built its own pool and was handed the whole ceiling, so with N endpoints the effective limit was N times the configured value. Only server listener pools participate; replication and migration keep their own unbounded behaviour so that a sync burst cannot drive live client connections down. The target is published but consumed by no allocation site yet. Landing the signal on its own makes a drift in the live buffer count observable in isolation, which matters because the target is budget divided by that count: a count that ratchets up does not degrade gracefully, it collapses the target to the floor permanently. The count rides exactly the two paths that already maintain the pool's live byte accounting and nothing else. Defaults are inert: 1 GB against a 128 KB buffer size means adaptation does not begin until 8,192 live buffers, and the target clamps at the configured size so it can only ever lower it. --network-buffer-memory-budget 0 disables it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new receive buffers from the network buffer budget The budget was published but consumed by nothing. Wire it to the receive side: a new connection's buffer starts at the published target rather than the configured size, and a grown buffer shrinks back toward that target. Only the base size is governed. Demand-driven doubling is untouched, so a connection that needs a large buffer still reaches it; pressure changes what a connection starts and settles at, never what it is allowed to grow to. The allocation size and the shrink floor move together. Clamping only the allocation would leave a connection allocated below the configured size unable to shrink at all, because the shrink path both floors at that size and returns early when the buffer is already at or under it. A pool cannot recycle a buffer outside its size classes, so NetworkBufferSettings gains an explicit minimum. Deriving it from the three configured sizes, as before, gives no way to express "recycle down to 16 KB while new connections still start at 128 KB", and a clamped buffer would have been dropped rather than pooled. The per-level idle entry count derived from a byte budget is capped in the same change, since a smaller minimum size class otherwise raises it proportionally; the cap sits above the value today's defaults produce, so it is inert until the two are combined. Budget accounting is now inert when disabled rather than merely unread. The disabled budget is a shared singleton, so unbudgeted pools were accumulating a count on it that was meaningless but would have become misleading the moment anything read it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Size new send buffers from the network buffer budget GarnetSaeaBuffer is the single allocation point behind both GarnetTcpNetworkSender call sites, so clamping there covers the send side entirely. Send buffers never grow, so an oversized response is chunked through whatever buffer it was given, and every consumer reads the length off the pool entry rather than off the configured setting -- which is what makes the size safe to adapt. The clamp uses the separate, higher send floor so that a shrunken send buffer does not push custom raw string responses into repeated ArrayPool rentals. Adds an end-to-end test that a 1 MB value and a key larger than the send buffer both survive while the budget is pinned at its floor, and tightens the shrink assertion now that both directions adapt. Neutering the clamp fails that assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk oversized cluster response elements The chunking audit converted the RESP session but left the cluster session on the atomic path, where an element larger than the response buffer cannot be written and SendAndReset has nothing to flush. CLUSTER GETKEYSINSLOT echoes client-supplied keys, so it kills the session for any key larger than the send buffer -- reproducible today at the 128 KB default, independent of the adaptive budget. The gossip reply echoes the serialized cluster config and grows with node count. Give the cluster session the same atomic-first helper the RESP session has: write in place, retry once on a flushed buffer, and chunk only an element that cannot fit an empty buffer, so small elements keep exactly today's path. Route both sites through it. Covered by a new fixture run at both the default send buffer and one floored by the budget; reverting either arm's fix fails it. Also extends the oversized-response fixture to run at a floored send buffer, with a guard asserting each arm actually binds the size it claims to, and corrects a stale comment about the per-level entry cap being inert -- the 16 KB adaptive floor makes it bind at the defaults. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Shrink receive buffers immediately while the budget is binding The idle hysteresis exists so a connection with bursty-but-recurring large payloads keeps its buffer through the burst, and it is raised from 16 to 256 receives to make that hold over a genuinely long quiet stretch. But a countdown that long cannot converge the aggregate when the process is actually short of memory, so shrinking now bypasses it entirely while the budget is binding. The pressure signal is the already-published target having been driven below the configured size, not a live-byte read: it costs one predictable branch on the receive path and is ordered after the read-mostly size comparisons, so an unpressured server evaluates nothing new. Wires the pressure and idle shrink counters that INFO BPSTATS already reports. The new test drives a burst and then ten small rounds -- far short of the 256-receive hysteresis -- and requires the budgeted arm to give the buffers back while the unbudgeted control still holds them. Neutering the pressure signal fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound pressure shrinking so alternating payloads do not churn Shrinking immediately under pressure was wrong. Pressure is sticky -- the published target stays below the ceiling for as long as the connections are live -- so an alternating large/small workload reallocated a pinned buffer on every large request. Measured at 640 shrinks over 640 large/small pairs, i.e. one per request, precisely when the server is most loaded. Apply a short countdown under pressure instead of none. A strictly alternating workload never trips it, because each large receive resets the countdown, while sustained small traffic still converges in eight receives rather than two hundred and fifty-six. The countdown is clamped rather than reloaded so that pressure arriving mid-countdown does not wait out whatever remained of the idle one. Same measurement now reports zero shrinks. Adds a TLS arm, since the incident is thousands of TLS connections and the decrypted transport buffer shrinks through a separate path that the plain-socket test cannot reach. Disabling the transport clamp leaves the grown buffers at their burst size and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Converge live connections onto the adapted buffer size Clamping the allocation size governs new buffers only. A connection established before the budget started binding keeps its full-size buffers: the send side hands them straight back out from its own stack, and the receive side moves them to the pool free list, where they stay pinned. Neither converges, so at incident scale the adaptation applies only to churn. Gate both return paths on the budget actually binding, using the same IsUnderPressure signal the shrink policy shipped with: - LimitedFixedBufferPool.Return drops an over-target entry instead of pooling it. Unpressured it is pooled exactly as before, so short-lived connections handling moderately large payloads keep their reuse. Send and receive adapt to separate floors, so the comparison reads the entry's own buffer type off PoolEntry.source: measuring against the receive target alone would treat a correctly sized send buffer as over-sized and make send buffers un-poolable for as long as the budget binds. Accounting is unaffected -- the drop reuses the existing non-pooled fall-through, below the unconditional live-byte and live-count release. - GarnetTcpNetworkSender.ReturnBuffer disposes an over-target send buffer rather than pushing it back onto the per-connection stack, so the next acquire allocates at the adapted size. Also clear the vacated slot in LightConcurrentStack.TryPop. Without it the array keeps the popped buffer, and through it the pinned pool entry, strongly referenced for the connection's lifetime -- so releasing a buffer freed the accounting but not the memory. Measured over ten connections established while the budget was slack, then driven into pressure by two hundred more: 2.0 MB released with both gates against 1.0 MB with the send gate neutered. Pooled bytes after the same burst: 0 against 10.5 MB with the pool gate neutered, and the free list after connection churn holds 80 KB against 32 KB when the send target is ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address adversarial review findings on the network buffer budget Five findings from two independent reviews, each reproduced before being fixed. Recompute the adapted target inside the live buffer count's own increment and decrement. It previously had a single production caller on the pool's allocate-miss path, which a pooled reuse never reaches, so the target never recovered when connections drained: after a spike, four live buffers still published a 16 KB target and reconnecting did not clear it. Clamp the TLS plaintext send buffer to the send target. It was allocated at the configured size while the pool measured it against the target on return, so under pressure every TLS connection's buffer was dropped and freshly pinned on the next connect. 64 connections now hold 11.5 MB where they held 15.7 MB. Sample the receive shrink signal before the buffer is processed rather than after. Consumption and compaction happen inside Process on the plain path, so a fully consumed large request was read as a zero-byte receive and advanced the shrink countdown: 160 shrinks over 640 alternating large/small pairs, now zero. Route 26 further response sites through the chunking helper. Set operation results and every blocking variant wrote unbounded elements atomically, which kills the session above the send buffer size at the shipped default; BLPOP loses the popped element outright. The new tests fail in both fixture arms, so this is not budget-induced. Drop the aborted-session response release added earlier in this branch. The batch's finally already returns the object on every teardown path, so 200 protocol-aborted sessions conserve the buffer count with and without it, and handing a possibly in-use buffer back to the pool is not worth an unmeasurable fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix stale transport alias, above-max release ordering, and scratch demand tracking Round 2 adversarial review found five defects, two of which were introduced by the round 1 fixes. All five reproduced before being fixed, and every new assertion is mutation-checked. Without TLS the transport receive buffer is an alias of the network buffer, refreshed only at the top of each receive. A resize left the alias pointing at the entry just returned to the pool, so a connection that grew once and then idled kept the old pinned array rooted indefinitely -- the accounting showed the release and the memory never happened. Measured at 64 connections the stale roots exceed 16 MB. Both resize paths now refresh the alias. The above-max immediate release sat behind the demand guard, so a fully consumed oversized request made target equal current and held the out-of-pool buffer until another receive arrived. This regressed C4 via the round 1 demand-capture fix. The branch now runs first and is sized from the residual rather than the pass's demand. CLUSTER SLOTS still wrote the whole serialized topology atomically, so a sufficiently fragmented cluster overran the send buffer with no oversized key involved. The new test fails at today's 128 KB default too, which is what proves it pre-existing. ScratchBufferBuilder only raised its demand high-water when a write did not fit, so a batch that kept filling an already-large buffer reported no usage at all and the shrink policy released it every time. The popped-entry collectability test could pass for the wrong reason under optimization, since the stack itself was collectable once unused. GC.KeepAlive pins it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reset the shrink countdown on the above-max early return The above-max branch returns without shrinking when the buffer is still full of unconsumed data. Leaving the countdown untouched there is the same guard-ordering class as the defect the branch reorder just fixed, so restore the reset the demand guard used to perform. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Redesign session buffer caps as countdown checkpoints The committed high-water design tracked per-command demand on the hot path and cost ~9-11 ns on Network.BasicOperations.InlinePing at batchSize=1. Replace it at both sites with a countdown checkpoint: every batch decrements a counter, and only when it reaches zero does a cold NoInlining method run. Demand is inferred from whether the buffer grew since the last checkpoint rather than tracked per command, so the hot path holds one decrement and one predictable branch and never touches the parse state. This trades bounded churn for zero hot-path tracking: reclaim takes up to two checkpoint intervals, and a session that needs the capacity every batch reallocates at most once per two intervals. Removes BufferShrinkPolicy, SessionParseState.batchHighWater and ScratchBufferBuilder.AdvanceOffset as dead code. Also re-applies the SLOWLOG chunking fix (review round 2 finding 1) for sps.GetString and entry.ClientName, tightens the parse-state retention assertion from a bare Assert.Less that passed with the shrink neutered to a magnitude bound that fails when mutated, and tightens the scratch-buffer equivalence assertion from a 2x slack bound to unbounded + 64 KB. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the unpressured common case against an unbudgeted server The central promise of the budget is that with few connections nothing changes. Two gaps in the coverage left that unpinned. UnpressuredGrownBuffersAreRetainedExactlyAsWithNoBudget runs the same grow- then-small-traffic workload with the default budget and with the budget disabled, and asserts the two hold identical bytes both immediately after growth and after the small traffic that follows. Ungating the pressure clamp in MaybeShrinkNetworkReceiveBuffer fails its retention assertion. UnpressuredConnectionChurnKeepsRecyclingGrownBuffers guards correction C3: dropping an over-target entry on return must be gated on pressure, or connection churn re-allocates pinned arrays that are recyclable today. Ungating the drop in LimitedFixedBufferPool.Return collapses pooledBytes to 128 KB and fails it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the scratch shrink checkpoint from the batch boundary only The countdown interval is counted in batches, but it was decremented from ScratchBufferBuilder.Reset(), which several callers invoke far more often than once per batch. The Lua interpreter resets once per string while decoding a JSON document and once per response through ScratchBufferNetworkSender, so a single cjson.decode of a wide object fired many checkpoints inside one command -- releasing a buffer the very next element re-grows, which is the churn the interval exists to bound. Split the countdown into ResetAtBatchBoundary(), called only from RespServerSession's batch boundary. Plain Reset() is now free again on the per-element paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drive the Lua script processor's scratch checkpoint from the session The Lua interpreter writes into the scratch buffer of the script cache's inner RespServerSession, not the network session's. That inner session never reads from a socket, so after the batch-boundary change its builder was never checkpointed at all and one cjson.encode of a large value permanently enlarged every session that ran a script. Fold the Lua checkpoint into the existing cold session checkpoint, which already runs once per 64 outer batches, so this costs nothing on the hot path and is definitionally outside any script execution. Rename the countdown trio accordingly, since it is no longer parse-state-only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Chunk the remaining unbounded response sites and cap the Lua reply buffer A full sweep of the 70 atomic write loops in libs/server/Resp and libs/cluster found six writing unbounded data: ACL LIST, ACL USERS and ACL GETUSER echo client-supplied usernames and rule descriptions, and CLUSTER SLAVES, CLUSTER REPLICAS and CLUSTER INFO write topology that grows with the cluster. All six kill the session at today's 128 KB default, which the new tests prove by failing in both fixture arms. The remaining sites write literals, numbers or fixed-width identifiers. The script cache's dummy network sender owns a second scratch buffer that the RESP fallback path for redis.call writes replies into, built uncapped and never checkpointed. It now takes the configured session cap and is checkpointed alongside the request buffer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Clear retained references on pop and on the cluster parse-state copy Three round-3 review findings, all in the same aliasing class the earlier rounds kept surfacing: a reference survives the release that was supposed to free it. SimpleStack.Pop left the vacated slot populated. Its only user stacks ScratchBufferAllocator's outgrown pinned buffers there, so every buffer a session ever outgrew stayed rooted for the life of the connection and a reset freed the accounting but not the memory. The clear is gated on IsReferenceOrContainsReferences so a struct-only instantiation is unaffected. ClusterSession took a copy of the caller's SessionParseState and left it populated after the command finished. The copy holds its own reference to the session's pinned root buffer, so a wide cluster command kept that buffer alive past the owning session's shrink. NetworkBufferBudget.Recompute validated the previous target rather than the count it derived from, so a lost exchange could publish a target derived from a stale count. The retry loop re-derives on a lost exchange and re-reads the count after a won one. This is defensive: the race was not reproduced in 20 runs of a 256-thread oversubscribed churn, and the test that ships pins the invariant rather than discriminating the fix. LatencyHistogramMemoryTests' idle arm was vacuous. GarnetServerTcp needs four bytes before it creates a RespServerSession, so an idle socket never created one and the arm measured a server with no sessions at all -- it would have passed with eager allocation restored. Sessions are now opened with a partial command and the count asserted against INFO CLIENTS, and both assertions are restated as comparisons between the arms rather than against an absolute. Restoring eager allocation now fails three tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make three assertions test what they claim, and correct three doc statements The send-buffer pooling test accepted any 64 KB on the free list, which four pooled 16 KB receive buffers also satisfy -- so it could pass with no send buffer pooled at all, which is the defect it exists to catch. It now reads the per-level entry counts the pool already reports and asserts an entry at the send size class specifically. Ignoring the entry's buffer type in the drop rule leaves it at zero entries and fails. The cluster chunking tests stopped reading at a byte count, which samples only the part of the reply the first chunk boundary has passed and cannot see a lost tail -- the half a chunking fix is most likely to get wrong. Both now pipeline a PING and read to it: RESP replies are ordered, so the sentinel cannot arrive before the whole reply is written. GETKEYSINSLOT additionally asserts the key contents rather than its length header, and CLUSTER SLOTS asserts every one of the 2,000 ranges is present. Three statements were false. GetSessionScratchBufferMaxRetainedSize says unset disables shrinking; unset returns the 64 KB default and only zero disables. MaxEntriesPerLevelCap says it is inert at the shipped defaults; the budget is on by default, so the smallest size class is the 16 KB receive floor and the derived value is 4,096, which the cap binds. defaults.conf still described the high-water design the countdown checkpoint replaced. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the reachable cluster chunking site and count sessions, not sockets Review round 4 found the cluster chunking fixture green with all three of the round-3 cluster conversions reverted, and both latency histogram tests green with the session-opening helper made a no-op. The three round-3 cluster conversions are bounded and unreachable: CLUSTER INFO is a fixed eleven-line string, CLUSTER REPLICAS elements are node lines carrying no slots, and CLUSTER BANLIST entries are a node id plus a number. The reachable caller of the same helper is CLUSTER NODES, whose node line embeds slot ranges, so a fragmented assignment overruns the send buffer at the 128 KB default with no oversized key anywhere. Covered with 4,000 alternating slots, asserting the declared bulk length against the received payload and every range. connected_clients counts accepted network handlers, not sessions, so the latency guard could not distinguish a measured session from a bare socket. CLIENT LIST enumerates the sessions themselves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Stop the Lua reply cap being undone, and re-derive an abandoned budget target Three defects on the same theme: a bound that does not bind. The Lua reply sender re-requests a response window at the full server buffer size on every use, so capping its builder below one window could not save anything -- the checkpoint released the buffer and the next redis.call immediately reallocated it larger than it started, churning a pinned array every checkpoint. The cap is now floored at one window, so it releases only the oversized replies it was added for. The existing test stopped at the shrink and could not see this. ScratchBufferBuilder rounded one byte past the request, so a request for exactly a size class allocated twice it -- and the Lua reply window, at exactly 128 KiB, is one. Every caller that must strictly grow already asks for one byte more than the current length, so exact rounding still grows them. NetworkBufferBudget.Recompute abandoned its target when its attempts ran out, leaving whatever the last contended exchange wrote, derived from a count that had since moved. Since nothing else recomputes, that value stood until the next acquire or release, and at a band boundary it could report no pressure while the budget was binding. It now re-derives from the freshest count. Contention cannot reach that path reliably, so the attempt limit is settable to force it. The ACL chunking tests stopped reading as soon as the oversized username arrived and never checked the tail; they now parse the whole array and require the pipelined PING reply to begin exactly where it ends. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the recompute attempt limit immutable Every connection reads the budget concurrently, so the attempt limit belongs in the constructor rather than in a settable field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Compare the CLUSTER NODES slot set exactly Substring matching is near-vacuous over 4,000 alternating slots: " 2" matches inside " 20" and " 2000", so almost any surviving fragment of the line satisfies it. Dropping every slot below 1,000 from the node line leaves the old assertion green and fails the new one at 3,500 slots against 4,000. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report a malformed reply as an assertion, not a timeout The hand-rolled RESP parsers in the chunking and latency tests assumed well-formed input. A truncated array, an error reply where a bulk string was expected, or a null bulk string surfaced as a fifteen-second receive timeout or an ArgumentOutOfRangeException rather than as a named assertion about the reply. Both parsers now check every bound before slicing, reject a reply whose type byte is not the expected one, and handle a null bulk string by recording it and advancing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Yield to a fresher publisher instead of overwriting it The exhaustion fallback published the target with an unconditional write, so a thread that sampled the live count before another thread published could land afterwards and replace the right answer for a fresher count with its own stale one. The remark argued the race in the wrong direction: it reasoned about being overtaken, not about overtaking. The publication is now a compare-exchange against the target that was observed alongside the count, and it does not retry. The only writer that can beat it also exchanges, so a lost exchange means a value derived from a count no older than this one is already published. The publish step takes the count and the observed target as parameters so the interleaving can be induced directly rather than raced into. Two assertions were weaker than the fixes they named: the size class test exercised the arithmetic helper but never an allocation, and the Lua reply test showed the cap survived another use without pinning it at one response window. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the cluster chunking arm small enough to chunk The CLUSTER NODES body is about 19 KB, which is below both the 128 KB default send buffer and the 64 KB the floored fixture arm actually resolved to for a single connection. Replacing the chunked write with a spinning atomic one left both arms green, so neither covered the path the test is named for. The floored arm's budget drops to 16k so its send target reaches the floor. The mutation now fails there. The default arm cannot be made to cover chunking here at all -- a single node's line is bounded by the 16,384 slots it can hold, which is well under 128 KB -- so the comment no longer claims it does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the target before the count, not after The compare-exchange was doing less than its remark claimed. It proves the target has not moved since this thread read it, which two threads that sampled the same target concurrently both satisfy regardless of whose count is fresher. Reading the target first and the count second makes the stronger statement true: a successful exchange proves no other publisher completed between the target read and the write, and the count was sampled inside that window, so it is no older than the count behind any value published earlier. The ordering is not observable from a single-threaded test, so it ships on the argument rather than on an assertion, as the other reasoning-level fixes in this work do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Abandon a contended recompute instead of publishing from a stale sample The single-attempt fallback had no correct form. An unconditional write let a stale count land after a fresher one; the compare-exchange that replaced it dropped a fresher publication when the staler one won the race; and its hysteresis band tested a target that could already have moved. Each fix traded one wrong interleaving for another. The fallback was also strictly weaker than one more iteration of the loop it backstopped: no post-write count revalidation and no retry. Remove it. Exhausting the attempts now publishes nothing, which is safe rather than resigned. Every write to the target is made by a thread that read both the previous target and the count immediately beforehand, so exhaustion leaves a value some recent count justified. Both count movers move the count before recomputing, so whichever thread invalidates a published target must itself re-derive it. A stale target left too high cannot hide behind the band either: clearing pressure requires the target to equal the ceiling, while the budget binding requires the quotient to be below it, and the band suppresses a write only when the quotient is at least the target. Replace the two tests that pinned the deleted methods with one that pins the ordering the argument rests on, and tighten two reply parsers that accepted malformed framing: any negative bulk length read as null, and a bulk string whose terminator was never checked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Remove the recompute attempt knob and harden two reply parsers Deleting the stale-publish fallback made a zero or negative attempt limit permanently stick the published target, since nothing else republishes on exhaustion. No caller passed the parameter once the two tests pinning the deleted methods were removed, so the constructor parameter is gone and the bound is a constant. A reply carrying its declared payload with no trailing CRLF blocked in Receive until the socket timed out, so the terminator assertion was unreachable. Both parsers now read to a pipelined PING sentinel and parse declared lengths as long, so an out-of-range length reports a named assertion instead of overflowing the bounds arithmetic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count sessions directly instead of parsing CLIENT LIST off the wire The hand-rolled RESP parser behind the session count produced a review finding in four consecutive rounds: an unreachable terminator check, a null-length case accepted too broadly, and twice an overflow in its bounds arithmetic. None were about the property under test. The fixture already enumerates live sessions for another test, so the count now uses that: it opens no probe connection, so it needs no self-discount, and it cannot mis-frame a reply. The remaining array parser compares its declared length by subtraction. Summing the offset, the length and the terminator wraps negative at long.MaxValue, which passes the bounds check and reaches the narrowing cast with an invalid length. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cut the per-receive and per-batch cost the buffer caps added The session buffer caps and the network budget each added work to a path that runs once per receive or once per batch, which at batchSize=1 is once per command. Three changes, none of which alter behaviour: - UpdateNetworkBuffers called MaybeShrinkNetworkReceiveBuffer on every receive, so a connection whose buffer is still at its base size paid a call and a chain of loads to discover it had nothing to do. The entry test is hoisted to the call site and the callee marked NoInlining. - BaseReceiveBufferSize walked networkPool.Budget and two properties on every read. The budget and the configured sizes are cached as readonly fields on the handler, and the clamp is one inlined call on the budget. - The scratch builder and the parse state each counted their own batches. One countdown on the session now drives both, so the batch boundary pays a single decrement regardless of how many buffers are capped, and ScratchBufferBuilder.Reset returns to exactly what it was before. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Release quiesced sessions' latency histograms A session that records latencies and then goes quiet holds its histograms for the connection's lifetime: lazy allocation defers the cost but never returns it, and the monitor's per-window Reset only zeroes the counts while keeping the array. At thousands of connections that is hundreds of MB retained for histograms recording nothing, plus a memset of all of it every sampling interval. The latency monitor already sweeps every session every window and already evaluates the exact predicate needed, so no tracking is added to the record path. After a configured number of consecutive empty windows the monitor drops that type's histograms. The reference is dropped rather than returned to the pool. The record path is deliberately unsynchronised, so a writer racing the monitor can hold the array past the release; dropping leaves that writer with the only remaining reference and its write unobservable, whereas returning would let it write into an array another session has re-rented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the histogram release threshold's hysteresis margin The component test supplies its own threshold, so the configured value was pinned by nothing: setting it to one window left the fixture green. A session issuing requests more slowly than the monitor sweeps must keep its histograms across the isolated empty windows that produces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's timing margin At a two-second request interval against a one-second sweep, an overshoot on a loaded box can manufacture the four consecutive empty windows the test asserts cannot happen. At 1.5 seconds the mutation still fires in three seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Widen the hysteresis test's absolute stall tolerance Scaling the sweep and the request interval together keeps the ratio the mutation needs while doubling how long the machine may stall before the test manufactures the consecutive empty windows it asserts cannot occur. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Reduce the network buffer config surface from nine flags to two The branch had grown nine new command-line switches. Seven of them were tuning knobs for an adaptive mechanism that sizes itself, exposed because they were easy to expose rather than because an operator needs them. Demoted to properties on GarnetServerOptions, reachable programmatically but no longer settable at startup: --network-buffer-size --network-buffer-min-size --network-max-receive-buffer-size --network-send-buffer-min-size --network-buffer-pool-size --session-parse-state-max-retained-args --session-scratch-buffer-max-retained-size None was CONFIG SET-able, so no live mitigation is lost, and none has shipped, so no operator .conf can reference one. Kept: --network-buffer-memory-budget the process-wide ceiling this work adds, and the only knob whose right value depends on the deployment --latency-monitor-precision a genuine accuracy/memory trade-off with no correct default Renamed for direction-explicit consistency with the pre-existing NetworkSendBufferMinSize: NetworkMaxReceiveBufferSize -> NetworkReceiveBufferMaxSize NetworkBufferMinSize -> NetworkReceiveBufferMinSize NetworkBufferSize keeps its direction-agnostic name: GarnetServerBase already exposes a public NetworkBufferSize for the same concept, so renaming here would diverge from established vocabulary. Removing the defaults.conf key would have changed behaviour rather than preserved it. An unset NetworkBufferPoolSize does not mean "unbounded" -- LimitedFixedBufferPool derives a byte ceiling from its per-level entry bound instead, which at the shipped 16-entry default is ~32.5 MB rather than the 64 MB the standalone server has always run. Options.GetServerOptions() now assigns "64m" directly, so the CLI path resolves exactly as before and embedded hosts keep the derived geometry they already had. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tell a refused connection why, and count it A connection refused by --network-connection-limit was disposed with no error written and counted in no metric, so an operator hitting the limit saw an unexplained reset and had nothing to confirm the cause. Send the Redis-compatible -ERR max number of clients reached before closing, and add rejected_connections to INFO stats. The write is gated to non-TLS. A plaintext error to a peer that has only sent a ClientHello is a protocol violation, so the client reports a handshake failure and the operator is pointed at certificates instead of capacity -- worse than a clean reset. Under TLS the metric is the whole remedy. The counter is incremented only in the limit branch, not in the catch that handles sockets dying during setup, so it stays trustworthy in the incident it exists for. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the connection limit process-wide and settable as maxclients The limit was per-listener, so N endpoints gave N times the configured ceiling -- the same defect this branch fixed for buffers, on the adjacent line of GarnetServer.cs. ConnectionLimit sums the listeners' existing activeHandlerCount values rather than maintaining a second counter, so no new accounting can drift; a connection counter that ratchets fails closed, refusing everything until restart. Default is now 10000, matching Redis, and set on GarnetServerOptions rather than only defaults.conf so embedded and test hosts resolve the same value as CLI hosts. CONFIG SET maxclients pushes the new ceiling through StoreWrapper.Servers into the live accept path. Lowering it disconnects nobody, which is also how Redis treats a lowered maxclients. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the refusal message, the counter, maxclients and the shared ceiling Seven assertions. The fixture holds one persistent probe connection so INFO and CONFIG can still be issued after the server is full, and fills only the remaining slots -- a probe that connected on demand would itself be refused. metricsSamplingFreq is set because rejected_connections is published by the monitor's sampling loop and reads "0" unconditionally without it, which would have made every counter assertion vacuous. Commands go over the wire as RESP arrays: Garnet's inline parsing handles single-token commands but silently produces no reply for a multi-token one, so an inline CONFIG GET returned nothing and the first version of these tests failed for that reason rather than for anything about the limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Make the shared-ceiling test able to fail The second listener refusing is equally consistent with it having reached a ceiling of its own, so the test now makes it accept a connection first. Also adds the drain-and-recover assertion and asserts several connections past the old ceiling when the limit is lifted, since admitting exactly one more is what raising the limit by one would also do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Build the connection limit on net8.0, and document the maxclients semantics System.Threading.Lock is .NET 9+, so ConnectionLimit did not compile for the net8.0 target the solution also builds. A plain object monitor is what the rest of the server uses and is equivalent here -- the lock is taken only by listener registration at startup, never on the accept path. Also records the three semantics an operator will ask about: replica and cluster peer links count against the limit (as they do in Redis, whose accept path adds getClusterConnectionsCount()); lowering the limit refuses new connections rather than culling established ones; and the value is not clamped to the file descriptor limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Fix final newlines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover maxclients across listeners and TLS rejection accounting MaxClientsReachesTheSharedLimitFromAMultiListenerHost fills the shared ceiling from two endpoints, raises maxclients at runtime, and asserts both listeners admit again, so CONFIG SET is pinned as reaching every listener rather than the one that happened to be first. TlsRejectionsAreCountedEvenThoughTheClientCannotBeTold pins the other half of the TLS gating: a refused TLS connection is closed without a RESP error, but still increments rejected_connections, which is the only signal an operator has on that transport. The TLS fillers are real TLS clients rather than bare sockets. The handshake runs on the accept path, so a connection that never handshakes stalls the accept loop and is never admitted, which makes bare sockets useless as fillers on a TLS listener. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS accept-path stall with concurrent and partial-handshake cases Upstream now starts the handler without waiting for the handshake, so the fix this branch carried is redundant and is dropped in favour of it. The tests are not: they add a peer that stalls after a partial record header, and sixteen concurrent silent peers, neither of which the upstream regression test reaches. Each asserts on a completed RESP exchange rather than on connect, because a connect against a wedged listener is absorbed by the backlog and succeeds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin that a failed TLS handshake releases its admission slot Starting the handshake off the accept loop moves where a handshake failure is noticed: it no longer propagates out of Start into the accept site's catch, so the slot the accept path already took is released by the handshake's own failure path instead. A leak there would let an unauthenticated peer permanently fill a connection-limited server. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Delete the non-discriminating handshake slot test Three mutations could not break it: removing the observer's Dispose, removing Dispose from the handshake failure path, and holding the peer socket open all left it green. Slot release on a failed handshake is over-determined, so the test observed nothing its subject could change. Replacing the fillers with silent sockets does fail, which is the residual the handshake timeout addresses rather than a property to pin here. Also corrects the ConnectionLimitTlsTests rationale, which cited the accept-loop stall that no longer exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Bound the per-session Lua script cache with an LRU cap Each cached script holds a whole Lua VM, so an unbounded per-session cache lets a session that runs many distinct scripts accumulate VMs for its lifetime. Cap the cache and evict the least recently used entry. Recency is a monotonic stamp updated in place on each hit, so a cache hit costs one store and eviction scans only at capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the script cache bound, its LRU order, and the evicted-script path Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Allocate transaction key entries unpinned on both paths TxnKeyEntries allocated its initial array with GC.AllocateArray(pinned: true) while the growth path used a plain array, so the pinning was silently dropped at the first resize. Every use of the array is through AsSpan, indexing or Array.Copy, and no caller takes a stable address into it, so the unpinned form the growth path has always used is the correct one. Dropping the pinned initial allocation also keeps these arrays out of the pinned object heap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count and report response buffer overflow rentals ReallocateOutput rents a pooled buffer whenever a response outgrows the buffer it is being written into. That traffic was only inferable, so the amount of overflow a given send buffer size causes could not be observed directly. Adds a process-wide rental count, reported as total_output_buffer_rentals in INFO STATS. It is read straight from the counter rather than from the monitor's sampled global metrics, so unlike the neighbouring rows it stays accurate when no monitor is configured. Measured through the new counter: ordinary responses rent nothing, and an oversized response rents exactly one buffer rather than climbing a doubling ladder, which is the property that makes a smaller send buffer affordable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop trailing newlines to match the repo's final-newline convention Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Report the output buffer rental count under BPSTATS rather than STATS The counter is read live from its source rather than sampled by the monitor, so under STATS it was the one row that stayed non-zero when no monitor is configured. MetricsDisabledTest asserts every STATS row reads zero in that state, which is Garnet's default, and it caught this. BPSTATS is where the rest of the buffer instrumentation on this branch already lives and is read live for the same reason, so the counter is consistent with its section there instead of being an exception inside a sampled one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Count the refusals a capacity wait legitimately incurs WaitForAcceptance polls until the server has capacity, and capacity returns when the server observes the closes rather than when the client disposes. An attempt made before it does is refused for real and advances rejected_connections, so the two tests asserting an exact count were racing a teardown they do not control. Report the refusals from the wait and add them to the expectation, keeping the assertions exact rather than loosening them to a bound that a build which stopped counting would also satisfy. Only a genuine max-clients reply counts, so an unrelated socket error cannot inflate the allowance and mask a miscount. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Capture the session latency histograms once at every reader The monitor sweep nulls LatencyMetricsEntrySession.latency when a session quiesces, but every reader check-then-dereferenced the field, so nothing guaranteed the two reads saw the same value. Both RecordValue overloads run on the session hot path with no lock by design, and GarnetLatencyMetrics.Merge runs on the disposing session's thread under rwLock while the sweep's reclaim holds only disposeLock, so the two do not exclude each other. Load the field into a local once and work from that. Allocate() now returns the array so the miss path reads the field once too. The type's remark claimed the reclaim was safe because a racing writer holds the only reference to the graph; that is true only if the writer captured it, so the remark now states single-capture as the invariant to preserve. Also gate the shrink counters on the budget being enabled, matching OnBufferAcquired: every pool handed no budget shares the Disabled singleton, so counting there put a contended process-wide interlocked write on the shrink path of connections the budget does not govern. Drop two unintended UTF-8 BOMs, and correct the maxclients documentation. Garnet takes every connection on one listener and cannot tell the kinds apart at accept time, so at the limit it refuses replica and gossip links alike; Redis counts cluster bus links but accepts them on a separate port that applies no limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Name the single-capture invariant at the latency record path The two RecordValue overloads load `latency` into a local exactly once because the monitor thread can null it concurrently. The expression reads as a convenience idiom, so state why it is load-bearing at the point a future edit would touch. The invariant itself is documented on ReclaimIfQuiesced, which is the writer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * State the single-capture requirement on the latency field itself The record-path comments guard the readers that exist. An author adding a new reader passes through the field declaration instead, so the constraint belongs there too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Tighten the comments added by this work Cut justifications-against-alternatives and narrative framing from the doc comments and inline comments this change introduces, keeping the constraints and invariants a reader needs in order to change the code correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Restate why Recompute needs no fallback on attempt exhaustion Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Inherit TestBase in the concurrent stack fixture Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the buffer pool's stats directly rather than over the wire Serving an INFO reply takes a send buffer and a receive buffer off the free list the assertion reads, so the probe consumed what it reported and the assertion had no margin against its own instrument. Read the listener's pool directly, and poll the size class the assertion reads rather than the byte total, returning the snapshot every assertion is then taken from. Cover the same rule against the pool directly, where nothing can take an entry off the free list between the return and the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Address PR review comments on histogram release, scratch cap, and test sends Release the session latency histograms by dropping the reference rather than returning their counts arrays, so a racing recorder cannot write into an array another session has since rented. `LongHistogram.Release()` does the same for the client, whose `latency` field is readonly. `ScratchBufferAllocator.Reset()` no longer enforces the retained-capacity cap at every batch boundary; a `ShrinkCheckpoint()` on the session's coarse countdown does, mirroring `ScratchBufferBuilder`. The session and transaction allocators are both wired to it, with a guard so a WATCH spanning batches keeps its slices. Test socket sends loop until the whole request is written, and the RESP reply reader reads until the expected length has arrived. `NetworkBufferBudget` documents the budget as an adaptive target on the base buffer size rather than a ceiling on the aggregate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the scratch allocator's checkpoint interval and its outstanding-slice guard A functional WATCH test cannot observe a buffer released underneath a live slice: the pinned array is collectable but its bytes stay readable until a collection reuses them. Both invariants are asserted on the allocator directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Update the scratch allocator tests for the checkpoint release contract The allocator now keeps its current buffer across Reset and releases an over-sized one at a ShrinkCheckpoint, so the fixture asserts the retained capacity at each point and drives two checkpoints to clear the hysteresis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Floor the above-max receive shrink at the largest poolable size The residual left after a fully consumed request is zero, so sizing the replacement from it always resolved to the base. A connection issuing one oversized request per batch then re-grew through every intermediate size class on every request. Floor the replacement at the largest poolable size at both the socket and TLS transport sites; the idle hysteresis path still takes it down to the base once the traffic no longer needs the capacity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Read the client latency histogram once at every consumer Release drops the counts array while the reply-processing thread may still be recording, so the record path tolerates a released array and the copy path reads the field once instead of testing IsReturned and then dereferencing. Also make the quiesce-threshold test observe empty windows: it recorded a value and never cleared it, so the windows it asserted over were not empty and a threshold that never restarted its run still passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Pin the released-histogram contract on the record and copy paths Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Derive the copied histogram's tracking values from the counts it carries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Correct the rationale for deriving the copied histogram's total Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS transport above-max receive replacement A TLS connection decrypts into a second receive buffer that grows through its own code path, so the above-max release there was unexercised while its plain-socket twin was pinned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Gate the above-max receive floor on budget pressure The above-max branch releases a buffer the pool cannot recycle. Flooring the replacement at the largest poolable size matches origin/main and saves a connection that needs the capacity on every request from re-growing through every size class, but it must not apply while the budget is binding: the target is derived from a buffer count, so a connection parked at the maximum costs the same single unit as one at the floor and the budget gets no feedback from it. Under pressure the replacement is now the adapted base, which the pool keeps and the budget can see. Also capture the counts array once in ClearCounts and CopyCountsInto, read the client call count off the snapshot the percentiles come from, and record the single-read contract on the field itself. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Cover the TLS half of the above-max pressure gate The plaintext and TLS receive paths gate the above-max release independently. Only the plaintext site was exercised, so the TLS gate at NetworkHandler.cs:801 could be removed with the whole fixture still green. Parameterize the test over both transports. Guard the counts array in CopyCountsInto, matching the single-read invariant documented on the field and the five sibling capture sites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Drop the scratch retention assertion that measures test ordering `ScratchBufferCapDoesNotChangeLargeArgumentResults` compared a per-session memory figure across two arms and failed intermittently. Two measurements show it cannot discriminate: swapping the arms moves each figure by about 130 KB per session and flips the sign of the difference, because the arm that runs second also measures the residue of the server instance the first one disposed; and releasing the buffer at every reset rather than at the checkpoint -- the churn such an assertion claims to catch -- moves it by less than that, because churn leaves garbage the forced collection reclaims rather than retention. The test keeps both arms and the reply validation they drive, which is what its name has always claimed, and the remarks state why no magnitude is asserted. `NetworkBufferBudget` documents the per-direction floors as the third reason the budget is a target rather than a ceiling: once the target reaches them, the aggregate grows with the connection count again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 * Sample the shrink churn either side of the batch boundary Comparing capacity across iterations cannot see the pathology the test exists to catch: a policy that released on every batch regrows before the next sample, so consecutive post-boundary capacities are equal and the churn counts as zero. Sampling before and after each boundary counts every release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Say what the latency precision number actually buys The setting is HdrHistogram significant digits: relative accuracy at every magnitude, not a fixed time unit. Stating it that way, with the memory each step costs, lets an operator pick a value without knowing HdrHistogram. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Pin that an over-sized request is accepted whole The receive buffer grows to fit and maxReceiveBufferSize bounds only what a connection retains, so a chunk far above it must still be accepted. Dribbled in small slices so the request sits partially buffered across many receive passes, where the above-max shrink runs against a live partial request. Also drop a qualifier from the latency precision help text. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Keep the byte-order marks as main has them Earlier edits flipped the BOM on seven files, which showed up as a one-line change to each file header with no content behind it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Wait for the buffer release instead of racing it A receive releases its buffer in UpdateNetworkBuffers, which runs after Process has already written the reply, so a client that has read its response can still be ahead of the release. Sampling BPSTATS once at that point read the pre-release numbers on a loaded runner and failed with the full oversized buffer still live. Poll until the counters settle. The wait uses only the out-of-band INFO connection, so it does not hand the connection under test the extra receive that would let a buffer released one pass too late off the hook: suppressing the immediate release still fails the assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Fold the two receive-buffer shrink paths into one policy The network and transport receive buffers ran the same three-regime shrink policy over their own occupancy and countdown, written out twice. Extract it as TryPlanReceiveBufferShrink, which owns the countdown bookkeeping and the budget's shrink accounting, and leave each caller with nothing but the buffer swap. Rename both entry points to TryShrink*, and give the transport buffer a ShrinkTransportReceiveBuffer mirroring ShrinkNetworkReceiveBuffer. Drop the cached configuredReceiveBufferSize/configuredSendBufferSize fields and read them off networkBufferSettings, as maxReceiveBufferSize already is on the same paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Keep the configured base sizes cached on the handler BaseReceiveBufferSize is read on every receive, so reaching the configured size through networkBufferSettings puts a dependent load on that path. BDN Network.BasicOperations.InlinePing measures it at +1.1ns (89.8 -> 90.9) on net10 and +1.9ns (85.7 -> 87.6) on net8, the net8 range clearing baseline entirely. Restoring the fields returns both to baseline (89.8 / 85.8). The shrink consolidation is unaffected and stays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Capture the metrics array in ReclaimQuiescedHistograms Every other method on the session captures the metrics field once before dereferencing it. ReclaimQuiescedHistograms re-read it for the null check and again inside the loop, so it relied on the dispose lock for that safety where its peers do not. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Checkpoint the transaction scratch allocator during AOF replay Replayed transaction procedures run their prepare phase against the watch API, and every watched key is copied into the transaction scratch allocator. The replay session never reaches the network batch boundary that drives the shrink checkpoint, so the buffer one wide procedure grew stayed pinned for the lifetime of a replica. The replay path now reports a boundary per AOF record and the window is counted in TransactionManager, so the caller does not carry the cadence. The allocator shrinks only when nothing is outstanding, so a boundary reported mid-transaction is safe. Also apply two review nits: a u8 span literal for the max-clients error instead of a materialized array, and a corrected comment on the connection-limit registration lock, which was crediting the lock with reader safety that the volatile publication provides. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Release the session scratch allocator at the AOF replay boundary Replayed procedures grow both scratch allocators, not just the transaction one: the prepare phase copies watched keys into the transaction allocator, and the procedure builds its arguments from the session allocator. The session allocator is reset at the start of every procedure rather than when one ends, so a completed procedure leaves it oversized with a non-zero offset, and a shrink checkpoint declines in that state. Reset it at the replay boundary, as the network batch boundary does, and only outside a transaction, whose slices are still live. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 * Rename the buffer shrink hook off "checkpoint" and count replay records at the call site "Checkpoint" is load-bearing in this codebase: Tsavorite uses it for durable, versioned, recoverable snapshots, and Garnet's AOF replay handles genuine CheckpointStartCommit markers a few lines from where this hook was called. The scratch-buffer hook means something unrelated -- periodically sample a buffer's capacity and release it if it did not grow -- so it is renamed to Trim, matching ArrayPool<T>.Trim, which applies the same hysteresis. The replay hook also counted its own interval while the network one had the caller count, so TrimAfterReplayedRecord ran on every record but trimmed on one in sixty-four. The counter moves to AofReplayContext, alongside the session it governs, so both paths show the cadence at the call site and TrimReplayBuffers means "trim now". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 --------- Co-authored-by: badrishc <badrishc@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f7ebed6a-7162-459d-849f-6a9d3a31eb30 Copilot-Session: a2361ed7-3bec-4fb8-9a6f-b4408448c160 | 1 个月前 | |
Harden TsavoriteLog commit-failure handling and fix fast-commit recovery poisoning (#2065) * Harden TsavoriteLog commit-failure handling and surface device exceptions CommitRecordBoundedGrowthTest ran log.Commit() in a tight loop on raw background threads with no exception handling. A device write failing under concurrent commit surfaces as a CommitFailureException (error code 0xFFFFFFFF), which was an unhandled throw on a background thread and terminated the whole test host. Test changes: - Parameterize CommitRecordBoundedGrowthTest with [Values(1, -1)] numThreads (-1 => ProcessorCount/2, else 1) to control commit-thread concurrency. - Capture the first background-thread exception and fail the test assertively instead of letting an unhandled throw kill the process. Production changes (surface the real fault for diagnosis): - StorageDeviceBase records the underlying write exception (RecordError / LastError); the device-to-allocator completion channel only carries a numeric error code (a non-IOException collapses to uint.MaxValue), which discarded the original type and stack. - All local device implementations record the exception in their WriteAsync catch blocks. - Thread the captured exception through CommitInfo.Exception and out as the InnerException of CommitFailureException, so an opaque 0xFFFFFFFF now carries the real device exception. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 573eaee0-3ed4-4f9d-96fc-07ea3fac9afd * Fix TsavoriteLog fast-commit recovery poisoning and CalculateReadOnlyAddress overflow When fast-commit recovery fails after setting the `HeadAddress` / `CommittedUntilAddress` = long.MaxValue "shut up safe guards" sentinels, the TsavoriteLog sync constructor swallowed the exception with a bare `catch { }`, leaving the log poisoned with a sentinel HeadAddress. The first Enqueue then routed through CalculateReadOnlyAddress(tail, long.MaxValue), whose page arithmetic overflowed to a negative address: Debug asserted; Release silently returned long.MinValue, which the `> ReadOnlyAddress` guard filtered out so ReadOnlyAddress never advanced, pages never flushed, and the next page-turning enqueue blocked forever. Root-cause fix (TsavoriteLog): - Add ResetRecoveryState(), which rolls the allocator back to a clean empty state (mirroring the existing "unable to recover using any available commit" reset), and call it on the non-tolerated failure path of RestoreLatestAsync/RestoreSpecificCommitAsync before rethrowing. - The sync constructor now logs and rethrows recovery failures instead of swallowing them, so a failed recovery fails fast rather than presenting an empty log as if recovery succeeded (data loss). This mirrors the async CreateAsync path. Garnet AOF sets TryRecoverLatest=false and so never runs this path. Defense-in-depth fix (AllocatorBase): - CalculateReadOnlyAddress clamps `headAddress >= tailAddress` to return tailAddress, so a sentinel/out-of-range headAddress can never overflow the page arithmetic regardless of source. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 573eaee0-3ed4-4f9d-96fc-07ea3fac9afd * Surface real device IO exceptions through the completion callback Follow-up to the earlier commit-failure hardening. That change stashed the failing write's exception on the device (StorageDeviceBase.LastError) and recovered it out-of-band, which is racy under concurrent flushes and only covers StorageDeviceBase-derived devices. Carry the typed exception on the completion channel itself instead, and log it for diagnosis. Production changes: - Add an `Exception ioException` parameter to DeviceIOCompletionCallback. The numeric errorCode stays the authoritative success/failure signal; the new argument only carries the typed exception for diagnosis (the code collapses to uint.MaxValue for a non-IOException, so it is a failure sentinel, not the device's real error code). - Forward the caught exception from device read/write completion callbacks (local, managed, native, random-access, sharded, tiered, memory, null, Azure); sites with no exception pass null. - Thread it through the buffered multi-write countdown (RecordError retains the first exception alongside the first error code and forwards both on final completion) and through PageAsyncFlushResult.Release. - Carry the plumbed exception straight into CommitInfo.Exception, and drop the StorageDeviceBase.LastError/RecordError slot the earlier commit added as an out-of-band workaround (now redundant since the exception rides the channel). - Log completion errors under a distinct "{exception}" placeholder via new Utility.GetCallbackExceptionDetail, which flattens AggregateException from sharded/tiered fan-out; the plain numeric "{errorCode}" placeholder is reserved for the no-exception case. Recovery-failure leak fix: - The synchronous TsavoriteLog constructor now disposes resources it already allocated (allocator, commit queue, owned epoch, default commit manager) when recovery throws, so repeated construction failures cannot leak a LightEpoch and its epoch-table slots. Cleanup is best-effort and never masks the original recovery failure. Test changes: - New LogCommitFailureTests.CommitFailureExceptionCarriesDeviceError asserts a failing write surfaces its exception through the new channel. - Add CalculateReadOnlyAddressClampsOutOfRangeHeadAddress, a deterministic regression for the fast-commit long.MaxValue "never evict" HeadAddress clamp, via a new internal AllocatorCalculateReadOnlyAddress test accessor. - Mark FastCommitRecoveryFailureFailsFastAndDoesNotPoisonLog [Explicit]: it is flaky under test-suite ordering because process-global recovery state can leak between fixtures. The deterministic clamp test above covers the same invariant. - Update SimulatedFlakyDevice and the device tests for the new callback signature. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 573eaee0-3ed4-4f9d-96fc-07ea3fac9afd * Fix flaky FastCommitRecoveryFailure test: device double-completion and async commit-metadata race SimulatedFlakyDevice fired its injected-error callback and then fell through to the underlying device, delivering a second completion for the same IO. The stray success completion re-entered AsyncGetFromDiskCallback after the error path had already nulled ctx.record, causing a background NullReferenceException (and a double semaphore release). Return after each injected-error callback so every IO completes exactly once, matching the existing already-failed-range fast path. FastCommitRecoveryFailureFailsFastAndDoesNotPoisonLog also raced fast commit's asynchronous commit-metadata write: spinWait guarantees the in-memory commit and the inline log commit record, but not that the manager's metadata file is yet visible. Phase 2a recovers with a device whose reads all fail, so that file is its only usable recovery source; if it had not landed yet, recovery found no commit, recovered an empty log, and never issued the read that must throw. Wait until the seed commit metadata is actually loadable before Phase 2a, and drop [Explicit] now that the test is deterministic. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 573eaee0-3ed4-4f9d-96fc-07ea3fac9afd --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 573eaee0-3ed4-4f9d-96fc-07ea3fac9afd | 2 个月前 | |
Remove .NET 8 support (#2230) * Remove .NET 8 support Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve multi-framework release and benchmark infrastructure Keep release packaging and publish paths framework-aware while targeting only .NET 10. Retain benchmark job selection for future runtimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Minimize .NET 8 release-script removal Retain the original publish and archive structure, removing only .NET 8 work while keeping the framework-aware archive loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Preserve existing release-script formatting Restore the original whitespace on unchanged .NET 10 publish commands and archive staging lines so the PR diff only shows the required .NET 8 removal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark framework options while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark README update to .NET 10 commands Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave sorted-set implementation unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Leave GeoHash unchanged in .NET 8 removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while dropping .NET 8 entries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Revert publish-profile conditional path changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Limit benchmark config note to .NET 10 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep benchmark overview change limited to .NET 10 default Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove redundant benchmark framework exclusion guard Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Load benchmark issuer certificate without warning suppression Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 Tsavorite warning properties Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep .NET 10 publish profiles while removing .NET 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Fix .NET 10 formatting in Vector Set key accessors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Remove obsolete .NET 9 feature guards Drop conditional branches now that .NET 10 is the sole target framework. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep ref struct anti-constraints on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep alternate comparer interfaces on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e * Keep vector read batch ref struct declaration on one line Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e --------- Co-authored-by: Tiago Napoli <tiagonapoli@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4d65adcc-753b-4191-87bc-d7df97ba762e | 17 天前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 年前 | ||
| 1 个月前 | ||
| 20 天前 | ||
| 11 天前 | ||
| 1 年前 | ||
| 5 个月前 | ||
| 1 个月前 | ||
| 9 天前 | ||
| 25 天前 | ||
| 1 年前 | ||
| 11 天前 | ||
| 6 个月前 | ||
| 1 年前 | ||
| 4 个月前 | ||
| 10 天前 | ||
| 1 年前 | ||
| 2 个月前 | ||
| 2 个月前 | ||
| 6 个月前 | ||
| 20 天前 | ||
| 1 年前 | ||
| 17 天前 | ||
| 6 个月前 | ||
| 1 个月前 | ||
| 6 个月前 | ||
| 17 天前 | ||
| 2 年前 | ||
| 10 个月前 | ||
| 4 个月前 | ||
| 2 个月前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 1 个月前 | ||
| 6 个月前 | ||
| 26 天前 | ||
| 1 年前 | ||
| 2 个月前 | ||
| 17 天前 | ||
| 1 年前 | ||
| 1 个月前 | ||
| 10 天前 | ||
| 1 年前 | ||
| 3 个月前 | ||
| 6 个月前 | ||
| 2 年前 | ||
| 1 个月前 | ||
| 2 个月前 | ||
| 17 天前 |