| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Move HttpAPIHooks from proxy to tsapibackend (#11791) * Move HttpAPIHooks from proxy to tsapibackend * private scope for tsapibackend --------- Co-authored-by: Chris McFarlen <cmcfarlen@apple.com> | 1 年前 | |
`plugin.config` to `plugin.yaml` migration. (#13070) * Add plugin.yaml as YAML alternative to plugin.config Introduce plugin.yaml, a YAML-based configuration file for global plugins that replaces the legacy line-based plugin.config format. New capabilities over plugin.config: - enabled: false to disable plugins without removing them - load_order for explicit plugin loading priority - NOTE-level startup log per plugin with load status - traffic_ctl plugin list via JSONRPC for runtime introspection - traffic_ctl config convert plugin_config for automated migration - Fallback: plugin.yaml takes precedence; plugin.config used if absent * Add inline config field to plugin.yaml Add the 'config' field to plugin.yaml entries, allowing plugin configuration to be embedded directly using a YAML block scalar (|). The literal text is written to a temporary file at startup and passed to the plugin as an argument. Only scalar values are accepted; structured YAML is rejected to preserve quoting semantics that plugins like txn_box rely on. | 4 个月前 | |
Add percentage limit for stale cache age (#13547) A fixed stale-age window can let short-lived cached responses remain usable for many times their original freshness lifetime. This makes serving stale content disproportionately risky for responses with a short max-age. This adds an optional percentage limit and applies the smaller of the percentage-based and absolute stale-age windows. The default keeps the existing behavior, while transaction overrides and supported scripting interfaces can opt in per use case. Fixes: #12252 | 13 天前 | |
Make QUIC connection ID creation explicit (#13519) Default-constructing a QUIC connection ID generates random bytes, even when the value is only a placeholder that will be overwritten. Several generation paths also randomize the same ID a second time. This patch makes empty, decoded, and newly generated IDs explicit. It deletes default construction, adds a checked CSPRNG-backed factory, and adds coverage for the connection ID representation and initialization contract. Fixes: #5504 | 5 天前 | |
Remove dead JSONRPC server_shutdown handler (#13578) Never registered nor called since it was added in #7478, and its void(YAML::Node) signature never matched the method handler shape it sat among. Shutdown already syncs the cache dir from the event thread in traffic_server.cc; doing it from an RPC thread would race the dir writers. Dropping it also removes mgmt's relative-path include of iocore's private P_CacheDir.h. | 1 天前 | |
Share HTTP accept properties (#13514) Each HTTP protocol acceptor keeps a separate copy of proxy-port properties, so adding or consuming one requires protocol-specific plumbing and can leave newer protocols without configured defaults. This patch introduces a common HTTP acceptor base that shares one immutable property set per proxy port and retains the source HttpProxyPort. Sessions track that acceptor, while transactions copy mutable outbound settings at transaction start so overrides stay isolated. Handoff paths keep their acceptors alive for the resulting session. Fixes: #3427 | 5 天前 | |
Add hidden metrics and MAX/MIN/incremental derived metric aggregation (#13505) * tsutil: make the Metrics unit tests order independent The tests asserted absolute metric ids and iterator positions, which only hold when the case runs first against an otherwise empty store. Any other test case that creates a published metric makes them fail. Assert on relative state instead so the cases can run in any order. * tsutil: add a separate storage for hidden metrics Hidden metrics are stored but never published. Using a separate Storage instance rather than a per-metric flag makes them structurally unreachable from the published store, so no metric consumer can expose them by omission. Gauge and Counter each gain createHiddenPtr overloads which return the same correctly typed pointer as createPtr, so a hidden metric is read and written with the normal typed mutators and no cast is needed at the call site. * tsutil: fail gracefully when metric storage is exhausted Storage::create() had no exhaustion check, so filling the last blob let the following bookkeeping call addBlob() and write one past the end of _blobs. Refuse the final slot instead and return the reserved bad_id, which keeps addBlob() from ever being reached in a full store and costs one slot out of 8M. The guard in addBlob() was also off by one against the access it protects, since the write is to _blobs[++_cur_blob], and being a debug_assert it was compiled out of release builds entirely. Make it a release_assert against MAX_BLOBS - 1. * traffic_ctl: add --include-hidden to metric match Hidden metrics are invisible to normal queries by design, which makes them hard to debug. Add an opt-in rec type bit, deliberately outside RECT_ALL so hidden metrics are never returned unless explicitly requested. The rec type also has to be accepted by the JSONRPC request decoder, which validates each requested type against a whitelist and rejects the whole request otherwise. No wire or schema change is needed, as rec_types is already an untyped list of ints. * tsutil: support MAX and MIN aggregation for derived metrics Derived metrics could only sum their sources. Add an op to the spec so a derived metric can also take the max or min across its sources, which is what an aggregate over instantaneous gauges needs. The accumulator is seeded from the first source rather than from zero, since a zero seed is only correct for SUM and would clamp MIN to <= 0. op defaults to SUM, so existing specs are unaffected. * tsutil: skip derived metric sources that do not resolve A source given by name or id that does not resolve was still passed to lookup(), which masks the unresolved id down to the reserved bad_id slot. The aggregate then silently included that slot's value instead of skipping the source, with no error reported. Resolve each source first and skip it if it does not resolve. This is observable under MAX and MIN, where the bad_id value can become the winning one; under SUM it happened to be hidden by bad_id holding zero. * tsutil: allow adding derived metric sources at runtime derive() only accepts a fixed initializer_list, which does not work for aggregates whose sources are discovered as the process runs. Calling it repeatedly for one derived name does not help either: it appends a separate entry per call, all targeting the same metric, so each update overwrites the others with its own subset and the last writer silently wins. add_source() accumulates sources into a single entry instead. Registering a source that is already present is a no-op, so a caller that may re-register the same source need not track that itself. * doc: document hidden and derived metrics Add a developer guide page for the metrics registry covering the hidden store, how it differs from the published one and why it is a separate store rather than a flag, and the derived metric aggregation ops including when derived values are recomputed and what that means for a sampled maximum. * Tag hidden metrics with RECT_HIDDEN_METRIC as well as RECT_PROCESS The record lookup callback rejects any record whose rec_type shares no bit with the requested mask, so tagging hidden metrics RECT_PROCESS alone made a request for only RECT_HIDDEN_METRIC fail with REQUESTED_TYPE_MISMATCH. Now that the request decoder accepts that type on its own, such a request is expressible, so set both bits. Add a unit test covering the hidden-only and include-hidden requests and confirming RECT_ALL still excludes them. * Value-initialize the synthetic records in the record lookups Both lookup functions build a RecRecord on the stack for metrics, which live outside the g_records array, and hand it to the caller's callback. The JSONRPC encoder reads version, registered, rsb_id, order and data_default unconditionally, so leaving them indeterminate lets a --format json metric query emit different values on successive runs, and reading an indeterminate bool is undefined behavior. Five sites, all with the same one word fix. Only the hidden metric loop is new in this branch; the rest have had the pattern for years. * Grow a new blob when a span ends on the blob boundary createSpan checked whether a span fit before reserving it but never re-checked afterwards, so a span ending exactly on MAX_SIZE left the offset at MAX_SIZE with no new blob allocated. The next create() then wrote one past the end of that blob's name array, and end() became an id that iterator::next() can never reach, since it wraps at ++offset == MAX_SIZE. create() has always grown as soon as it consumed the last slot; do the same here. Also refuse a span that would fill or overflow the final blob, so the new growth cannot ask addBlob() to go past the last one and trip its assert. createSpan(MAX_SIZE) always starts a fresh blob and fills it exactly, whatever the current offset, so the added test reaches the boundary deterministically. It fails without the fix. * Polish the --include-hidden surface Three small corrections to the flag added earlier in this branch: Skip slot 0 when walking the hidden store. Every Storage reserves it for the bad_id placeholder, so it exists under the same name in both stores and a query matching it returned two records differing only in value, in exactly the debugging situation the flag is for. Scope the option to 'match' with a nested program directive. As a bare option under 'traffic_ctl metric' it rendered as a peer of get, match and describe, so it read as another subcommand rather than a flag on match. This follows the 'config get --records' pattern earlier in the file. Put the flag before the positional in the CLI example usage so it agrees with that synopsis, which is also the convention the rest of the file uses. | 13 天前 | |
Add hidden metrics and MAX/MIN/incremental derived metric aggregation (#13505) * tsutil: make the Metrics unit tests order independent The tests asserted absolute metric ids and iterator positions, which only hold when the case runs first against an otherwise empty store. Any other test case that creates a published metric makes them fail. Assert on relative state instead so the cases can run in any order. * tsutil: add a separate storage for hidden metrics Hidden metrics are stored but never published. Using a separate Storage instance rather than a per-metric flag makes them structurally unreachable from the published store, so no metric consumer can expose them by omission. Gauge and Counter each gain createHiddenPtr overloads which return the same correctly typed pointer as createPtr, so a hidden metric is read and written with the normal typed mutators and no cast is needed at the call site. * tsutil: fail gracefully when metric storage is exhausted Storage::create() had no exhaustion check, so filling the last blob let the following bookkeeping call addBlob() and write one past the end of _blobs. Refuse the final slot instead and return the reserved bad_id, which keeps addBlob() from ever being reached in a full store and costs one slot out of 8M. The guard in addBlob() was also off by one against the access it protects, since the write is to _blobs[++_cur_blob], and being a debug_assert it was compiled out of release builds entirely. Make it a release_assert against MAX_BLOBS - 1. * traffic_ctl: add --include-hidden to metric match Hidden metrics are invisible to normal queries by design, which makes them hard to debug. Add an opt-in rec type bit, deliberately outside RECT_ALL so hidden metrics are never returned unless explicitly requested. The rec type also has to be accepted by the JSONRPC request decoder, which validates each requested type against a whitelist and rejects the whole request otherwise. No wire or schema change is needed, as rec_types is already an untyped list of ints. * tsutil: support MAX and MIN aggregation for derived metrics Derived metrics could only sum their sources. Add an op to the spec so a derived metric can also take the max or min across its sources, which is what an aggregate over instantaneous gauges needs. The accumulator is seeded from the first source rather than from zero, since a zero seed is only correct for SUM and would clamp MIN to <= 0. op defaults to SUM, so existing specs are unaffected. * tsutil: skip derived metric sources that do not resolve A source given by name or id that does not resolve was still passed to lookup(), which masks the unresolved id down to the reserved bad_id slot. The aggregate then silently included that slot's value instead of skipping the source, with no error reported. Resolve each source first and skip it if it does not resolve. This is observable under MAX and MIN, where the bad_id value can become the winning one; under SUM it happened to be hidden by bad_id holding zero. * tsutil: allow adding derived metric sources at runtime derive() only accepts a fixed initializer_list, which does not work for aggregates whose sources are discovered as the process runs. Calling it repeatedly for one derived name does not help either: it appends a separate entry per call, all targeting the same metric, so each update overwrites the others with its own subset and the last writer silently wins. add_source() accumulates sources into a single entry instead. Registering a source that is already present is a no-op, so a caller that may re-register the same source need not track that itself. * doc: document hidden and derived metrics Add a developer guide page for the metrics registry covering the hidden store, how it differs from the published one and why it is a separate store rather than a flag, and the derived metric aggregation ops including when derived values are recomputed and what that means for a sampled maximum. * Tag hidden metrics with RECT_HIDDEN_METRIC as well as RECT_PROCESS The record lookup callback rejects any record whose rec_type shares no bit with the requested mask, so tagging hidden metrics RECT_PROCESS alone made a request for only RECT_HIDDEN_METRIC fail with REQUESTED_TYPE_MISMATCH. Now that the request decoder accepts that type on its own, such a request is expressible, so set both bits. Add a unit test covering the hidden-only and include-hidden requests and confirming RECT_ALL still excludes them. * Value-initialize the synthetic records in the record lookups Both lookup functions build a RecRecord on the stack for metrics, which live outside the g_records array, and hand it to the caller's callback. The JSONRPC encoder reads version, registered, rsb_id, order and data_default unconditionally, so leaving them indeterminate lets a --format json metric query emit different values on successive runs, and reading an indeterminate bool is undefined behavior. Five sites, all with the same one word fix. Only the hidden metric loop is new in this branch; the rest have had the pattern for years. * Grow a new blob when a span ends on the blob boundary createSpan checked whether a span fit before reserving it but never re-checked afterwards, so a span ending exactly on MAX_SIZE left the offset at MAX_SIZE with no new blob allocated. The next create() then wrote one past the end of that blob's name array, and end() became an id that iterator::next() can never reach, since it wraps at ++offset == MAX_SIZE. create() has always grown as soon as it consumed the last slot; do the same here. Also refuse a span that would fill or overflow the final blob, so the new growth cannot ask addBlob() to go past the last one and trip its assert. createSpan(MAX_SIZE) always starts a fresh blob and fills it exactly, whatever the current offset, so the added test reaches the boundary deterministically. It fails without the fix. * Polish the --include-hidden surface Three small corrections to the flag added earlier in this branch: Skip slot 0 when walking the hidden store. Every Storage reserves it for the bad_id placeholder, so it exists under the same name in both stores and a query matching it returned two records differing only in value, in exactly the debugging situation the flag is for. Scope the option to 'match' with a nested program directive. As a bare option under 'traffic_ctl metric' it rendered as a peer of get, match and describe, so it read as another subcommand rather than a flag on match. This follows the 'config get --records' pattern earlier in the file. Put the flag before the positional in the CLI example usage so it agrees with that synopsis, which is also the convention the rest of the file uses. | 13 天前 | |
Add port descriptor destroy API (#13518) TSPortDescriptorParse allocates an HttpProxyPort that plugins cannot release, so every parsed descriptor leaks for the lifetime of Traffic Server. The API also lacks end-to-end coverage that verifies the accept callback. This patch adds TSPortDescriptorDestroy while preserving the opaque handle ABI, validates unusable descriptors during parsing, and documents the ownership contract. It updates API users and adds an AuTest that opens a dynamically selected port and observes the accept callback. Fixes: #6894 | 6 天前 | |
cache: shared-memory-backed Dir for fast restart (#13328) * cache: shared-memory-backed Dir for fast restart Cold-start cache initialization rebuilds each stripe's in-memory directory from disk on every restart, which is multi-minute on large caches. Host the directory in POSIX shared memory so the next process start attaches the existing segment in milliseconds instead of rebuilding it. Recovery stays binary and fail-safe: when the segment cannot be trusted -- crash, reboot, ABI or schema mismatch, storage change, or failed validation -- the start drops it and rebuilds through the existing disk path. Reads still validate Doc magic and key, so a stale entry is a miss and never corruption. A stripe the previous shutdown could not vouch for is marked in the control segment, never in the stripe's own header: that header aliases raw_dir, which is also the source buffer for the on-disk directory write, so a mark there could reach disk and make the next start clear the stripe instead of recovering it. Opt-in behind proxy.config.cache.shm.enabled, default 0, where it is a functional no-op. `traffic_ctl cache shm status` and `clear` inspect and drop segments out of band. The design and the full recovery matrix are in doc/developer-guide/cache-architecture/shm-fast-restart.en.rst. * traffic_ctl: run cache shm subcommands through Command_Execute The status/clear leaves passed their own `[&]() { command->execute(); }` lambda, where every other leaf in the file passes Command_Execute. Reuse it so the null guard applies and the wiring is uniform. No behavior change: `command` is assigned before args.invoke() and a null one throws, so the guard cannot fire today. * cache: fix shm shutdown test where flock is honored The untrusted-entry test simulated a restart by calling initialize() again in one process, which the concurrent-attach guard correctly refuses: the first start still holds LOCK_EX on the control fd, and the second open of the same object conflicts with it. The test only passed where flock is not honored for POSIX shm (macOS, FreeBSD) and failed on Fedora and Debian; the other Linux builds compile the feature out, so the target is not built there at all. Add CacheShm::release_for_test() to drop the process-wide state, standing in for the process exit that releases the flock in production. * cache: always write the on-disk dir at shm shutdown Skipping the write for a shm-backed stripe looked free -- the segment is already current and is attached directly next start -- but the on-disk copy is the only thing the fallback has, and recover_data() cannot always rebuild from it. handle_recover_from_data() returns without scanning the data region when the on-disk header still has sync_serial == 0, so an empty directory is accepted as-is. A stripe filled and cleanly shut down before the first periodic dir sync (60 s by default) is exactly that case: nothing had written the on-disk dir, so the next start that cannot use the segment found an empty directory and lost every object. cache_shm_dir_invalid caught it as a 502 against its deliberately absent origin, once the poked segment was correctly rejected. Only start time is what this feature set out to improve, so the shutdown write costs what it did before and the fallback stays recoverable. * cache: harden the shm trust gates and ownership guard Review of the fast-restart path found four ways a shared-memory segment could be trusted, or cleared, when it should not be. The attach gate bounded directory links but neither a live entry's offset -- which CacheVC::handleRead turns into a negative, so huge unsigned, read length -- nor the free list's structure, where an in-range cycle lets freelist_pop write a link over a live entry's tag bits. Clean shutdown cleared owner_pid while event threads were still writing, which on a platform where flock is a no-op is the only guard against a concurrent attach. And traffic_ctl swept a control segment smaller than this build's before checking for a live owner, so a newer build could unlink a running older build's segments -- the upgrade case the frozen header exists to support. The entry bound is on where an entry starts, not on its extent: dir_approx_size rounds up, so the last object in a stripe legitimately overhangs the stripe end, which is what handleRead's truncation is for. * cache: prove shm segment membership at attach A walk from the free-list head cannot see an entry that Directory::insert unlinked but never filled, so a shutdown torn in that window published a directory whose stale prev/next the next insert would write through. Require every entry to be reached exactly once, as a bucket root, an empty free-list node, or an in-use chain node, and delay the clean-shutdown mark until after the event system is down so fewer tears reach the gate at all. * Include algorithm header | 25 天前 | |
Initialize uninitialized pointer and scalar members in QUIC and tscpp classes (#13023) Add default member initializers for pointer, scalar, and enum members that Coverity flagged as uninitialized: - QUICStreamError::stream (nullptr) - QUICSentPacketInfo: packet_number, ack_eliciting, in_flight, sent_bytes, time_sent, type, pn_space - QUICSentPacketInfo::FrameInfo::_generator (nullptr) - QUICTransferProgressProviderSA::_adapter (nullptr) - InterceptPlugin::state_ (nullptr) - AsyncTimer::state_ (nullptr) | 5 个月前 | |
Test AtomicSharedPtr concurrency (#13566) Add Catch2 coverage for AtomicSharedPtr load/store/exchange operations and concurrent readers during writer swaps. Also clarify the fallback implementation comment so it no longer relies on stale library-version shorthand. | 5 天前 |