已合并
feat(worker): 下沉 QueryAndGet 支持元数据亲和直读 #2143
feat(worker): 下沉 QueryAndGet 支持元数据亲和直读 #2143
已合并
yangxiaogang14创建于 6 天前
34 个文件变更+1718-1202
M.repo_context/modules/client/client-sdk.md+9-4
@@ -583,8 +583,12 @@
583 Client-side dynamic UB activation publishes readiness with release/acquire ordering only after `UrmaManager::Init`583 Client-side dynamic UB activation publishes readiness with release/acquire ordering only after `UrmaManager::Init`
584 succeeds; transport selection remains fail-closed during initialization or after failure, without writing the584 succeeds; transport selection remains fail-closed during initialization or after failure, without writing the
585 process gflag from a heartbeat thread.585 process gflag from a heartbeat thread.
586- `MasterOCServiceImpl::QueryAndGet` logs authenticated handler entry at `INFO`, terminal handler failures at `ERROR`,586+ `QueryAndGet` is a Client-to-Worker RPC. The metadata-owner Worker reads only resident local objects inline and
587- and successful completion through `SLOW_LOG_IF_OR_VLOG` using the configured server process threshold.587+ resolves locations for misses through its existing metadata route; it does not pull, subscribe, create placeholders,
588+ or load L2 data for this fast path. Same-host requests carry the established SHM session identity, remote UB
589+ requests carry one preregistered buffer per input key. If SHM preparation fails while URMA is enabled, the client
590+ tries UB before degrading the whole owner group to TCP; when URMA is disabled, it degrades directly to TCP.
591+ Results remain positional, and absence of `data_result` means the existing replica-read phase must handle that key.
588 - when the existing client latency trace is enabled for a request, transport-layer Get contributes592 - when the existing client latency trace is enabled for a request, transport-layer Get contributes
589 `client.process.direct_route`, `client.rpc.direct_query_and_get`, `client.rpc.direct_get_data`, and593 `client.process.direct_route`, `client.rpc.direct_query_and_get`, `client.rpc.direct_get_data`, and
590 `client.process.direct_materialize` to the request `latencySummary`. The two RPC-class phases are parent-thread wall594 `client.process.direct_materialize` to the request `latencySummary`. The two RPC-class phases are parent-thread wall
@@ -596,8 +600,9 @@
596 RPC-client creation, URMA handshake/finalization, transfer, and retry preparation. If any recorded phase exceeds its600 RPC-client creation, URMA handshake/finalization, transfer, and retry preparation. If any recorded phase exceeds its
597 client process or RPC slow threshold, one `[TransportGet] Phase latency` log prints the slow phase names and all601 client process or RPC slow threshold, one `[TransportGet] Phase latency` log prints the slow phase names and all
598 recorded durations. Untraced requests do not construct the recorder, read the threshold config, or format the log.602 recorded durations. Untraced requests do not construct the recorder, read the threshold config, or format the log.
599- - `QueryAndGet` returns at most five copy locations per object. The primary address from object metadata is returned603+ - On a Worker-local miss, `QueryAndGet` uses side-effect-free `PureQueryMeta` and converts its result into a
600- first, followed by non-primary locations, so replica retry always starts with the primary copy.604+ primary-first location list, appending the selected valid non-primary replica when one exists. The established
605+ `QueryMeta` subscription, payload, and location-update behavior remains unchanged.
601 - `tests/st/client/kv_cache/kv_client_transport_get_test.cpp` covers single-key and same-owner multi-key transport606 - `tests/st/client/kv_cache/kv_client_transport_get_test.cpp` covers single-key and same-owner multi-key transport
602 reads. It disables the local cache, applies the same deterministic hash rule in the SDK and worker processes, and607 reads. It disables the local cache, applies the same deterministic hash rule in the SDK and worker processes, and
603 resolves the metadata owner through the real SDK `Routing` path before asserting TCP or UB data transport.608 resolves the metadata owner through the real SDK `Routing` path before asserting TCP or UB data transport.
M.repo_context/modules/runtime/worker-runtime.md+7-0
@@ -356,6 +356,13 @@
356 `MetadataRouteResolver`, `PlacementFacade`, `MembershipEndpointView`, immutable Snapshot, or an Object-specific356 `MetadataRouteResolver`, `PlacementFacade`, `MembershipEndpointView`, immutable Snapshot, or an Object-specific
357 endpoint policy. `WorkerOCServiceImpl` is the sole business lifecycle owner allowed to retain a non-owning Engine357 endpoint policy. `WorkerOCServiceImpl` is the sole business lifecycle owner allowed to retain a non-owning Engine
358 pointer for semantic lifecycle and cold Host queries.358 pointer for semantic lifecycle and cold Host queries.
359+ - `WorkerOCService.QueryAndGet` is the metadata-affine direct-read entrypoint. It returns resident local objects over
360+ SHM, UB, or bounded TCP payloads and uses routed `PureQueryMeta` only for local misses, so metadata redirect handling
361+ stays inside Worker without changing `QueryMeta` semantics. The service handler owns only Worker-state validation
362+ and delegates request reading, authentication, response delivery, SHM rollback, access recording, and slow logging
363+ to `WorkerQueryAndGetImpl`. The local probe must remain side-effect free: no remote pull,
364+ subscription, placeholder creation, L2 load, or cache insertion. The response has exactly one positional result per
365+ requested key; a result without `data_result` delegates data retrieval to the Client's existing replica phase.
359- Review caution:366- Review caution:
360 - topology behavior is spread across flags, `WorkerOCServer`, and `src/datasystem/cluster`, so config-only changes367 - topology behavior is spread across flags, `WorkerOCServer`, and `src/datasystem/cluster`, so config-only changes
361 may still impact worker request routing and recovery behavior.368 may still impact worker request routing and recovery behavior.
Mscripts/ds_trace_bottleneck.py+13-13
@@ -264,8 +264,8 @@ def _classify_rpc_deadline_detail(status: int, evidence: list[str]) -> dict[str,
264 re.I,264 re.I,
265 )265 )
266 )266 )
267- query_timeout = any("MasterOCService.QueryAndGet" in method for method in failed_methods) or bool(267+ query_timeout = any("WorkerOCService.QueryAndGet" in method for method in failed_methods) or bool(
268- re.search(r"MasterOCService\.QueryAndGet[^\n]*RPC deadline exceeded", joined, re.I)268+ re.search(r"WorkerOCService\.QueryAndGet[^\n]*RPC deadline exceeded", joined, re.I)
269 )269 )
270 # Method-specific failure evidence wins over the mere presence of another270 # Method-specific failure evidence wins over the mere presence of another
271 # successful RPC in the same Trace. QueryAndGet may succeed before a later271 # successful RPC in the same Trace. QueryAndGet may succeed before a later
@@ -281,17 +281,17 @@ def _classify_rpc_deadline_detail(status: int, evidence: list[str]) -> dict[str,
281 elif query_timeout:281 elif query_timeout:
282 subcategory = "QueryMeta RPC deadline"282 subcategory = "QueryMeta RPC deadline"
283 chain = f"QueryMeta RPC超时→TransportGet失败→{status}"283 chain = f"QueryMeta RPC超时→TransportGet失败→{status}"
284- failure_point = "MasterOCService.QueryAndGet未在deadline内返回"284+ failure_point = "WorkerOCService.QueryAndGet未在deadline内返回"
285 elif re.search(r"(?:WorkerWorkerOCService\.)?GetObjectRemote", joined, re.I) and not re.search(285 elif re.search(r"(?:WorkerWorkerOCService\.)?GetObjectRemote", joined, re.I) and not re.search(
286- r"MasterOCService\.QueryAndGet", joined, re.I286+ r"WorkerOCService\.QueryAndGet", joined, re.I
287 ):287 ):
288 subcategory = "Data RPC deadline"288 subcategory = "Data RPC deadline"
289 chain = f"Data RPC超时→TransportGet失败→{status}"289 chain = f"Data RPC超时→TransportGet失败→{status}"
290 failure_point = "GetObjectRemote未在deadline内返回"290 failure_point = "GetObjectRemote未在deadline内返回"
291- elif re.search(r"MasterOCService\.QueryAndGet", joined, re.I):291+ elif re.search(r"WorkerOCService\.QueryAndGet", joined, re.I):
292 subcategory = "QueryMeta RPC deadline"292 subcategory = "QueryMeta RPC deadline"
293 chain = f"QueryMeta RPC超时→TransportGet失败→{status}"293 chain = f"QueryMeta RPC超时→TransportGet失败→{status}"
294- failure_point = "MasterOCService.QueryAndGet未在deadline内返回"294+ failure_point = "WorkerOCService.QueryAndGet未在deadline内返回"
295 else:295 else:
296 subcategory = "RPC deadline·方法未细分"296 subcategory = "RPC deadline·方法未细分"
297 chain = f"RPC超时→TransportGet失败→{status}"297 chain = f"RPC超时→TransportGet失败→{status}"
@@ -1361,7 +1361,7 @@ def _query_meta_detail(row: dict) -> dict | None:
1361 entries = []1361 entries = []
1362 for text in row.get("evidence", []):1362 for text in row.get("evidence", []):
1363 method, fields = _rpc_fields(text)1363 method, fields = _rpc_fields(text)
1364- if method and "MasterOCService.QueryAndGet" in method:1364+ if method and "WorkerOCService.QueryAndGet" in method:
1365 entries.append(fields)1365 entries.append(fields)
1366 if not entries and not row.get("query_meta_ms"):1366 if not entries and not row.get("query_meta_ms"):
1367 return None1367 return None
@@ -1875,7 +1875,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict:
1875 initiator = row.get("client_observer") or "未明确"1875 initiator = row.get("client_observer") or "未明确"
1876 if initiator == "未明确":1876 if initiator == "未明确":
1877 for record in row.get("evidence_records", []):1877 for record in row.get("evidence_records", []):
1878- if "MasterOCService.QueryAndGet" in record.get("text", ""):1878+ if "WorkerOCService.QueryAndGet" in record.get("text", ""):
1879 initiator = record.get("worker") or "未明确"1879 initiator = record.get("worker") or "未明确"
1880 break1880 break
1881 initiator_groups[initiator].append(row)1881 initiator_groups[initiator].append(row)
@@ -1920,7 +1920,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict:
1920 initiator = row.get("client_observer") or "未明确"1920 initiator = row.get("client_observer") or "未明确"
1921 if initiator == "未明确":1921 if initiator == "未明确":
1922 for record in row.get("evidence_records", []):1922 for record in row.get("evidence_records", []):
1923- if "MasterOCService.QueryAndGet" in record.get("text", ""):1923+ if "WorkerOCService.QueryAndGet" in record.get("text", ""):
1924 initiator = record.get("worker") or "未明确"1924 initiator = record.get("worker") or "未明确"
1925 break1925 break
1926 if initiator != "未明确":1926 if initiator != "未明确":
@@ -1928,7 +1928,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict:
1928 query_entries = []1928 query_entries = []
1929 for text in row.get("evidence", []):1929 for text in row.get("evidence", []):
1930 method, fields = _rpc_fields(text)1930 method, fields = _rpc_fields(text)
1931- if method and "MasterOCService.QueryAndGet" in method:1931+ if method and "WorkerOCService.QueryAndGet" in method:
1932 query_entries.append(fields)1932 query_entries.append(fields)
1933 target = re.search(1933 target = re.search(
1934 r"(?:meta owner|targetAddress|target address|peer)\s*[:=]\s*([^,\s]+)",1934 r"(?:meta owner|targetAddress|target address|peer)\s*[:=]\s*([^,\s]+)",
@@ -1938,7 +1938,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict:
1938 if target:1938 if target:
1939 timeout_targets.add(target.group(1))1939 timeout_targets.add(target.group(1))
1940 if any(1940 if any(
1941- "MasterOCService.QueryAndGet" in text and re.search(r"\bresp_attachment_bytes=0\b", text)1941+ "WorkerOCService.QueryAndGet" in text and re.search(r"\bresp_attachment_bytes=0\b", text)
1942 for text in row.get("evidence", [])1942 for text in row.get("evidence", [])
1943 ):1943 ):
1944 empty_response_count += 11944 empty_response_count += 1
@@ -1972,7 +1972,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict:
1972 "dominant_second": {"second": dominant_second, "trace_count": dominant_count},1972 "dominant_second": {"second": dominant_second, "trace_count": dominant_count},
1973 "confirmed_flow": (1973 "confirmed_flow": (
1974 "Client ObjectReadFlow::Resolve → ObjectMetadataClient::QueryAndGet/QueryWithRetry → "1974 "Client ObjectReadFlow::Resolve → ObjectMetadataClient::QueryAndGet/QueryWithRetry → "
1975- "WorkerRpcClient::InvokeQueryAndGet → Meta Owner MasterOCService.QueryAndGet;"1975+ "WorkerRpcClient::InvokeQueryAndGet → Meta Owner WorkerOCService.QueryAndGet;"
1976 "超时发生在该RPC返回前,尚未进入后续独立Data Worker GetObjectRemote阶段。"1976 "超时发生在该RPC返回前,尚未进入后续独立Data Worker GetObjectRemote阶段。"
1977 ),1977 ),
1978 "likely_common_mechanism": (1978 "likely_common_mechanism": (
@@ -2237,7 +2237,7 @@ table{width:100%;table-layout:fixed}
2237CORRELATION_SECTION = r'''2237CORRELATION_SECTION = r'''
2238<section class="panel" id="query-meta-analysis">2238<section class="panel" id="query-meta-analysis">
2239<h2>4-A. QueryMeta 根因分析</h2>2239<h2>4-A. QueryMeta 根因分析</h2>
2240-<div class="notice"><b>定界口径:</b><code>MasterOCService.QueryAndGet</code> 不只是查元数据:携带 <code>data_request</code> 时,Meta Owner 会执行 <code>TryGetQueryAndGetData</code>,命中本地副本后可通过 UB/URMA 内联返回数据。细类按互斥优先级统计;TryGet/URMA 作为证据标签,不重复计入主类。失败且只有 <code>cntl_error_code=1008</code> 时,只能确认 Client 等待到截止点;缺少 server trailer 时,Meta Owner 执行、响应发送、RPC residual 与 Client 截止观察仍未闭合。</div>2240+<div class="notice"><b>定界口径:</b><code>WorkerOCService.QueryAndGet</code> 不只是查元数据:携带 <code>data_request</code> 时,Meta Owner Worker 会读取本地驻留副本,并通过 SHM、UB/URMA 或 TCP 内联返回数据;本地 miss 才向 Master 发起 metadata-only QueryMeta。细类按互斥优先级统计;本地读取/URMA 作为证据标签,不重复计入主类。失败且只有 <code>cntl_error_code=1008</code> 时,只能确认 Client 等待到截止点;缺少 server trailer 时,Meta Owner Worker 执行、响应发送、RPC residual 与 Client 截止观察仍未闭合。</div>
2241<div id="query-meta-summary" class="finding-grid"></div>2241<div id="query-meta-summary" class="finding-grid"></div>
2242<h3>QueryAndGet 超时流程定界</h3>2242<h3>QueryAndGet 超时流程定界</h3>
2243<div id="query-meta-timeout-flow" class="finding-grid"></div>2243<div id="query-meta-timeout-flow" class="finding-grid"></div>
Msrc/datasystem/client/transport/data_plane/shm_connection.cpp+17-0
@@ -413,6 +413,23 @@ Status ShmSession::BuildResult(const GetRspPb::ObjectInfoPb &info, const DataGet
413 return Status::OK();413 return Status::OK();
414}414}
415 415 
416+Status ShmSession::BuildQueryAndGetResult(const QueryAndGetShmInfoPb &info, const DataGetRequest &input,
417+ DataGetResult &result)
418+{
419+ GetRspPb::ObjectInfoPb objectInfo;
420+ objectInfo.set_store_fd(info.store_fd());
421+ objectInfo.set_offset(info.offset());
422+ objectInfo.set_data_size(info.data_size());
423+ objectInfo.set_metadata_size(info.metadata_size());
424+ objectInfo.set_mmap_size(info.mmap_size());
425+ objectInfo.set_shm_id(info.shm_id());
426+ objectInfo.set_is_seal(info.is_seal());
427+ objectInfo.set_write_mode(info.write_mode());
428+ objectInfo.set_consistency_type(info.consistency_type());
429+ objectInfo.set_cache_type(info.cache_type());
430+ return BuildResult(objectInfo, input, result);
431+}
432+ 
416Status ShmSession::MmapWriteRegion(const CreateRspPb &createRsp, const TransportRequestContext &context,433Status ShmSession::MmapWriteRegion(const CreateRspPb &createRsp, const TransportRequestContext &context,
417 uint64_t size, ObjectBufferInfo &info)434 uint64_t size, ObjectBufferInfo &info)
418{435{
Msrc/datasystem/client/transport/data_plane/shm_connection.h+10-0
@@ -82,6 +82,16 @@ public:
82 82 
83 Status BuildResult(const GetRspPb::ObjectInfoPb &info, const DataGetRequest &input, DataGetResult &result);83 Status BuildResult(const GetRspPb::ObjectInfoPb &info, const DataGetRequest &input, DataGetResult &result);
84 84 
85+ /**
86+ * @brief Materialize a QueryAndGet shared-memory result through this session.
87+ * @param[in] info Shared-memory result returned by Worker QueryAndGet.
88+ * @param[in] input Object identity and request context.
89+ * @param[out] result Materialized zero-copy data result.
90+ * @return K_OK on success; the error code otherwise.
91+ */
92+ Status BuildQueryAndGetResult(const QueryAndGetShmInfoPb &info, const DataGetRequest &input,
93+ DataGetResult &result);
94+ 
85 /** Maps the shared-memory region allocated by a routed Create into the client address space (PROT_WRITE)95 /** Maps the shared-memory region allocated by a routed Create into the client address space (PROT_WRITE)
86 * so the caller can write zero-copy, registers the worker reference, and attaches a send-side owner that96 * so the caller can write zero-copy, registers the worker reference, and attaches a send-side owner that
87 * gates Publish on session liveness (K_BUFFER_DEPRECATED) and releases the worker reference on buffer97 * gates Publish on session liveness (K_BUFFER_DEPRECATED) and releases the worker reference on buffer
Msrc/datasystem/client/transport/data_plane/shm_transporter.h+23-0
@@ -59,6 +59,29 @@ public:
59 && shmConnection_->IsAlive();59 && shmConnection_->IsAlive();
60 }60 }
61 61 
62+ /**
63+ * @brief Acquire the endpoint shared-memory session for a routed request.
64+ * @param[in] context Request authentication and tenant context.
65+ * @param[out] session Acquired endpoint session.
66+ * @return K_OK on success; the error code otherwise.
67+ */
68+ Status AcquireSession(const TransportRequestContext &context, std::shared_ptr<ShmSession> &session)
69+ {
70+ RETURN_RUNTIME_ERROR_IF_NULL(shmConnection_);
71+ return shmConnection_->Acquire(context, session);
72+ }
73+ 
74+ /**
75+ * @brief Invalidate a session that returned an unusable shared-memory result.
76+ * @param[in] session Session to invalidate.
77+ */
78+ void InvalidateSession(const std::shared_ptr<ShmSession> &session)
79+ {
80+ if (shmConnection_ != nullptr) {
81+ shmConnection_->Invalidate(session);
82+ }
83+ }
84+ 
62 Status Get(const DataGetRequest &input, DataGetResult &output) override85 Status Get(const DataGetRequest &input, DataGetResult &output) override
63 {86 {
64 RETURN_RUNTIME_ERROR_IF_NULL(rpcClient_);87 RETURN_RUNTIME_ERROR_IF_NULL(rpcClient_);
Msrc/datasystem/client/transport/metadata/object_metadata_client.cpp+232-237
@@ -14,19 +14,19 @@
14 * limitations under the License.14 * limitations under the License.
15 */15 */
16 16 
17-/** Description: Implements batched object metadata access and deadline-bounded redirect handling. */17+/** Description: Implements Worker-routed batched metadata and inline-data access. */
18 18 
19#include "datasystem/client/transport/metadata/object_metadata_client.h"19#include "datasystem/client/transport/metadata/object_metadata_client.h"
20 20 
21#include "datasystem/client/transport/object_read/object_read_types.h"21#include "datasystem/client/transport/object_read/object_read_types.h"
22 22 
23#include <cstdint>23#include <cstdint>
24-#include <deque>
25#include <limits>24#include <limits>
26-#include <unordered_map>
27#include <utility>25#include <utility>
28 26 
27+#include "datasystem/common/inject/inject_point.h"
29#include "datasystem/common/log/log.h"28#include "datasystem/common/log/log.h"
29+#include "datasystem/common/rdma/fast_transport_base.h"
30#include "datasystem/common/rpc/brpc_status_util.h"30#include "datasystem/common/rpc/brpc_status_util.h"
31#include "datasystem/common/util/rpc_util.h"31#include "datasystem/common/util/rpc_util.h"
32#include "datasystem/common/util/status_helper.h"32#include "datasystem/common/util/status_helper.h"
@@ -35,25 +35,6 @@ namespace datasystem {
35namespace client {35namespace client {
36 36 
37namespace {37namespace {
38-struct RedirectTarget {
39- HostPort address;
40- uint64_t topologyVersion = 0;
41-};
42- 
43-struct RedirectBatch {
44- HostPort address;
45- uint64_t topologyVersion = 0;
46- ObjectMetadataBatch items;
47-};
48- 
49-struct PendingMetadataBatch {
50- HostPort address;
51- uint64_t topologyVersion = 0;
52- ObjectMetadataBatch items;
53-};
54- 
55-using RedirectTargets = std::unordered_map<std::string, std::deque<RedirectTarget>>;
56- 
57bool IsMetadataOwnerRouteFailure(StatusCode code)38bool IsMetadataOwnerRouteFailure(StatusCode code)
58{39{
59 return code == K_RPC_UNAVAILABLE || code == K_RPC_DEADLINE_EXCEEDED || code == K_RPC_PEER_DEAD40 return code == K_RPC_UNAVAILABLE || code == K_RPC_DEADLINE_EXCEEDED || code == K_RPC_PEER_DEAD
@@ -73,77 +54,11 @@ Status ValidateAndResetItems(const ObjectMetadataBatch &items)
73 return Status::OK();54 return Status::OK();
74}55}
75 56 
76-Status BuildRedirectTargets(const master::QueryAndGetRspPb &response, RedirectTargets &redirectTargets)57+void CopyLocation(const QueryAndGetLocationInfoPb &source, master::ObjectLocationInfoPb &target)
77{58{
78- for (const auto &redirectInfo : response.info()) {59+ target.set_object_key(source.object_key());
79- HostPort address;60+ target.set_object_size(source.object_size());
80- RETURN_IF_NOT_OK(address.ParseString(redirectInfo.redirect_meta_address()));61+ *target.mutable_object_locations() = source.object_locations();
81- for (const auto &objectKey : redirectInfo.change_meta_ids()) {
82- redirectTargets[objectKey].push_back({ address, redirectInfo.topology_version() });
83- }
84- }
85- return Status::OK();
86-}
87- 
88-Status PartitionInitialResponse(const ObjectMetadataBatch &items, const master::QueryAndGetRspPb &response,
89- ObjectMetadataBatch &localItems,
90- std::vector<RedirectBatch> &redirectBatches,
91- std::optional<uint64_t> minimumTopologyVersion)
92-{
93- RedirectTargets redirectTargets;
94- RETURN_IF_NOT_OK(BuildRedirectTargets(response, redirectTargets));
95- 
96- localItems.clear();
97- localItems.reserve(items.size());
98- std::unordered_map<std::string, RedirectBatch *> batchesByTarget;
99- // The target map keeps pointers into redirectBatches, so prevent vector relocation while grouping.
100- redirectBatches.reserve(response.info_size());
101- batchesByTarget.reserve(response.info_size());
102- for (auto *item : items) {
103- auto target = redirectTargets.find(item->objectKey);
104- if (target == redirectTargets.end() || target->second.empty()) {
105- localItems.push_back(item);
106- continue;
107- }
108- RedirectTarget redirectTarget = std::move(target->second.front());
109- target->second.pop_front();
110- CHECK_FAIL_RETURN_STATUS(!minimumTopologyVersion.has_value()
111- || redirectTarget.topologyVersion >= *minimumTopologyVersion,
112- K_NOT_READY,
113- std::string(STALE_TRANSPORT_SNAPSHOT_MESSAGE)
114- + ": metadata redirect topology version rolled back from "
115- + std::to_string(*minimumTopologyVersion) + " to "
116- + std::to_string(redirectTarget.topologyVersion));
117- const std::string batchKey = redirectTarget.address.ToString() + "#"
118- + std::to_string(redirectTarget.topologyVersion);
119- auto inserted = batchesByTarget.emplace(batchKey, nullptr);
120- if (inserted.second) {
121- redirectBatches.push_back(
122- { std::move(redirectTarget.address), redirectTarget.topologyVersion, {} });
123- inserted.first->second = &redirectBatches.back();
124- }
125- inserted.first->second->items.push_back(item);
126- }
127- return Status::OK();
128-}
129- 
130-void SetBatchError(const ObjectMetadataBatch &items, const Status &status)
131-{
132- for (auto *item : items) {
133- item->status = status;
134- }
135-}
136- 
137-void QueueRedirectBatches(std::vector<RedirectBatch> &redirectBatches, std::deque<PendingMetadataBatch> &pending)
138-{
139- for (auto &batch : redirectBatches) {
140- if (batch.items.empty()) {
141- continue;
142- }
143- LOG(INFO) << "[TransportGet][Metadata] Follow redirect, target: " << batch.address.ToString()
144- << ", topology version: " << batch.topologyVersion << ", key count: " << batch.items.size();
145- pending.push_back({ std::move(batch.address), batch.topologyVersion, std::move(batch.items) });
146- }
147}62}
148} // namespace63} // namespace
149 64 
@@ -163,24 +78,69 @@ ObjectMetadataClient::ObjectMetadataClient(std::shared_ptr<DataPlaneManager> man
163}78}
164 79 
165Status ObjectMetadataClient::InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items,80Status ObjectMetadataClient::InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items,
81+ std::shared_ptr<const TransportReadContext> readContext,
166 InlineRequestContext &context) const82 InlineRequestContext &context) const
167{83{
168 context = InlineRequestContext{};84 context = InlineRequestContext{};
169- // One batch uses one exclusive inline mode; phase one enables it only for a single key.85+ const auto hint = advisor_ == nullptr ? TransportHint::TCP_ONLY : advisor_->GetTransportHint(address);
170- if (items.size() != 1 || advisor_ == nullptr) {86+ if (hint == TransportHint::SHM_CANDIDATE) {
87+ RETURN_IF_NOT_OK(PrepareShmInlineRequest(address, std::move(readContext), context));
88+ RETURN_OK_IF_TRUE(context.mode == InlineTransportMode::SHM);
89+ }
90+ if (hint == TransportHint::UB_CANDIDATE || IsUrmaEnabled()) {
cheyulin
cheyulincheyulin6 天前

[警告][性能] [P1] 批量 UB 在确认命中和对象大小前按 key 预留注册内存

证据:基线只允许单 key 进入 inline;这里放开批量后,AllocateUbInlineBuffers 会为每个 item 分别 Allocate(ubBufferSize_)。总量只受默认 32 MiB 的 UB MaxGetSize 限制,因此单请求可预占 32 MiB;并发 32 时约为 1 GiB,而且 100% metadata miss 也会完成全部预分配。

问题:注册内存、allocator/handle、shared ownership 和哈希节点成本与真实命中率及对象大小无关,低命中大 batch 会把元数据查询变成高成本内存路径。

影响:高并发下可能造成注册池压力、allocator/arena 争用、缓存抖动和 UB 到 TCP 的额外回退,抵消 RPC 下沉收益并恶化 P99。

建议:按 owner group 使用连续注册 buffer 并通过 offset 分片,或设置独立的 key 数与字节预算后分块;在有测量依据前至少保留批量 UB 门限。

验证:在配置 DATASYSTEM_UB_GET_DATA_SIZE_BYTES 后,以并发 1/32/128、batch 1/32/1024、命中率 0/50/100% 对比基线,记录 P50/P99、分配次数、注册字节峰值和 TCP fallback 次数。

likedislike
yangxiaogang14
5 天前 评论:
91+ return PrepareUbInlineRequest(address, items, context);
92+ }
93+ context.mode = InlineTransportMode::TCP;
94+ return Status::OK();
95+}
96+ 
97+Status ObjectMetadataClient::PrepareShmInlineRequest(const HostPort &address,
98+ std::shared_ptr<const TransportReadContext> readContext,
99+ InlineRequestContext &context) const
100+{
101+ RETURN_RUNTIME_ERROR_IF_NULL(manager_);
102+ if (readContext == nullptr) {
103+ context.DisableInlineData();
171 return Status::OK();104 return Status::OK();
172 }105 }
173- if (advisor_->GetTransportHint(address) == TransportHint::TCP_ONLY) {106+ std::shared_ptr<IDataTransporter> transporter;
174- context.mode = InlineTransportMode::TCP;107+ Status rc = manager_->GetOrCreate(address, TransportHint::SHM_CANDIDATE, transporter);
108+ auto shmTransporter = std::dynamic_pointer_cast<ShmTransporter>(transporter);
109+ if (rc.IsError() || shmTransporter == nullptr) {
110+ context.DisableInlineData();
175 return Status::OK();111 return Status::OK();
176 }112 }
177- return PrepareUbInlineRequest(address, items, context);113+ std::shared_ptr<ShmSession> session;
114+ rc = shmTransporter->AcquireSession(readContext->requestContext, session);
cheyulin
cheyulincheyulin6 天前

[警告][安全] [P1] 复用 SHM session 时未刷新当前请求的认证上下文

证据:这里 AcquireSession 复用已有 live session 后直接返回;ShmConnection::Acquire 不更新 ShmSession 或 ShmFdChannel 的 auth。普通 ShmSession::Get 每次请求都会同时刷新二者,而 QueryAndGet 的 BuildQueryAndGetResult 可能触发 GetClientFd,后者使用 ShmFdChannel 中缓存的 token,并与当前 tenantId 组合发请求。

问题:QueryAndGet 主 RPC 可以使用当前凭据成功,但冷 FD 获取仍可能使用 session 创建时或上一次普通 Get 的旧 token/tenant 上下文。

影响:凭据轮换或请求级租户变化后,同机 SHM 命中可能在结果物化阶段稳定失败并使 session 失效;认证上下文错配也触及多租户隔离边界。

建议:提供带锁的 ShmSession::RefreshAuth(context),在复用 session 后且 dispatch 前原子刷新 session 与 fdChannel 的认证上下文;或让 BuildQueryAndGetResult 显式刷新本次 context。

验证:复用同一 session 后轮换 token/AKSK,并切换 request tenant,构造需要获取新 FD 的 SHM hit,确认 GetClientFd 只使用当前凭据,且失败时有安全降级路径。

likedislike
yangxiaogang14
5 天前 评论:
115+ if (rc.IsError()) {
116+ context.DisableInlineData();
117+ return Status::OK();
118+ }
119+ context.mode = InlineTransportMode::SHM;
120+ context.shmTransporter = std::move(shmTransporter);
121+ context.shmSession = std::move(session);
122+ context.readContext = std::move(readContext);
123+ return Status::OK();
124+}
125+ 
126+Status ObjectMetadataClient::PrepareShmInlineFallback(const HostPort &address,
127+ const ObjectMetadataBatch &items,
128+ InlineRequestContext &context) const
129+{
130+ context.DisableInlineData();
131+ if (IsUrmaEnabled()) {
132+ return PrepareUbInlineRequest(address, items, context);
133+ }
134+ context.mode = InlineTransportMode::TCP;
135+ return Status::OK();
178}136}
179 137 
180Status ObjectMetadataClient::PrepareUbInlineRequest(const HostPort &address, const ObjectMetadataBatch &items,138Status ObjectMetadataClient::PrepareUbInlineRequest(const HostPort &address, const ObjectMetadataBatch &items,
181 InlineRequestContext &context) const139 InlineRequestContext &context) const
182{140{
183- if (ubBufferSize_ == 0 || ubBufferProvider_ == nullptr || ubBufferSize_ > ubBufferProvider_->MaxGetSize()) {141+ if (ubBufferSize_ == 0 || ubBufferProvider_ == nullptr || ubBufferSize_ > ubBufferProvider_->MaxGetSize()
142+ || items.size() > ubBufferProvider_->MaxGetSize() / ubBufferSize_) {
143+ context.mode = InlineTransportMode::TCP;
184 return Status::OK();144 return Status::OK();
185 }145 }
186 146 
@@ -191,10 +151,12 @@ Status ObjectMetadataClient::PrepareUbInlineRequest(const HostPort &address, con
191 if (connectionRc.IsError()) {151 if (connectionRc.IsError()) {
192 VLOG(1) << "[TransportGet][Metadata] Disable UB inline data because the connection is unavailable: "152 VLOG(1) << "[TransportGet][Metadata] Disable UB inline data because the connection is unavailable: "
193 << connectionRc.ToString();153 << connectionRc.ToString();
154+ context.mode = InlineTransportMode::TCP;
194 return Status::OK();155 return Status::OK();
195 }156 }
196 157 
197 if (AllocateUbInlineBuffers(items, context).IsError()) {158 if (AllocateUbInlineBuffers(items, context).IsError()) {
159+ context.mode = InlineTransportMode::TCP;
198 return Status::OK();160 return Status::OK();
199 }161 }
200 context.mode = InlineTransportMode::UB;162 context.mode = InlineTransportMode::UB;
@@ -229,7 +191,7 @@ Status ObjectMetadataClient::AllocateUbInlineBuffers(const ObjectMetadataBatch &
229 191 
230Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &items,192Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &items,
231 const InlineRequestContext &context,193 const InlineRequestContext &context,
232- master::QueryAndGetReqPb &request) const194+ QueryAndGetReqPb &request) const
233{195{
234 if (context.mode == InlineTransportMode::NONE) {196 if (context.mode == InlineTransportMode::NONE) {
235 return Status::OK();197 return Status::OK();
@@ -238,11 +200,17 @@ Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &ite
238 (void)request.mutable_data_request()->mutable_tcp();200 (void)request.mutable_data_request()->mutable_tcp();
239 return Status::OK();201 return Status::OK();
240 }202 }
203+ if (context.mode == InlineTransportMode::SHM) {
204+ CHECK_FAIL_RETURN_STATUS(context.shmSession != nullptr && context.shmSession->IsAlive(), K_NOT_READY,
O
OOuGongChang6 天前

[P3] SHM session 在请求构建阶段失效(K_NOT_READY)时整批失败且无 TCP/UB 回退

PrepareShmInlineRequest 成功取得 session 后,若在 AddInlineDataRequest 检查 IsAlive() 时 session 恰好失效(心跳超时窗口),直接返回 K_NOT_READY,QueryWithRetry 立即返回错误,组内所有 key 失败;而 InvokeQueryAndGet 的 lease 失败路径(invoked==false)会走 SwitchInlineRequestToTcp 降级 TCP。同一请求内两条失败路径行为不一致,窗口虽极小但失败模式生硬,建议此处同样降级 TCP 或按 miss 重试。

likedislike
yangxiaogang14
5 天前 评论:
205+ "QueryAndGet shared-memory session is unavailable");
206+ request.mutable_data_request()->mutable_shm()->set_client_id(context.shmSession->ClientId());
207+ return Status::OK();
208+ }
241 209 
242 auto *ubRequest = request.mutable_data_request()->mutable_ub();210 auto *ubRequest = request.mutable_data_request()->mutable_ub();
243 ubRequest->set_buffer_size(ubBufferSize_);211 ubRequest->set_buffer_size(ubBufferSize_);
244 ubRequest->set_urma_instance_id(context.transportInstanceId);212 ubRequest->set_urma_instance_id(context.transportInstanceId);
245- // Buffer descriptors follow object_keys order, including redirected sub-batches.213+ // Buffer descriptors follow object_keys order.
246 for (auto *item : items) {214 for (auto *item : items) {
247 auto buffer = context.ubBuffers.find(item);215 auto buffer = context.ubBuffers.find(item);
248 CHECK_FAIL_RETURN_STATUS(buffer != context.ubBuffers.end(), K_RUNTIME_ERROR,216 CHECK_FAIL_RETURN_STATUS(buffer != context.ubBuffers.end(), K_RUNTIME_ERROR,
@@ -252,132 +220,145 @@ Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &ite
252 return Status::OK();220 return Status::OK();
253}221}
254 222 
255-Status ObjectMetadataClient::InvokeQueryAndGet(const HostPort &address, master::QueryAndGetReqPb &request,223+Status ObjectMetadataClient::InvokeQueryAndGet(const HostPort &address, QueryAndGetReqPb &request,
256- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,224+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,
257- InlineRequestContext &context,225+ InlineRequestContext &context, bool &rpcDispatched)
258- std::optional<uint64_t> redirectTopologyVersion,
259- bool &rpcDispatched)
260{226{
261 rpcDispatched = false;227 rpcDispatched = false;
262 RETURN_RUNTIME_ERROR_IF_NULL(manager_);228 RETURN_RUNTIME_ERROR_IF_NULL(manager_);
263- if (context.mode == InlineTransportMode::UB) {229+ if (context.mode == InlineTransportMode::UB || context.mode == InlineTransportMode::SHM) {
264 bool invoked = false;230 bool invoked = false;
265- Status leaseRc = manager_->WithDataPlaneLease(231+ Status leaseRc = InvokeInlineQueryAndGet(address, request, response, payloads, context, invoked, rpcDispatched);
266- address, TransportHint::UB_CANDIDATE,
267- [&](const std::shared_ptr<IDataTransporter> &, const std::shared_ptr<WorkerRpcClient> &rpcClient) {
268- invoked = true;
269- return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched);
270- });
271 if (invoked) {232 if (invoked) {
272- // Once sent, RPC failures retain UB mode and follow the normal retry policy.
273 return leaseRc;233 return leaseRc;
274 }234 }
275- VLOG(1) << "[TransportGet][Metadata] UB inline connection is unavailable for " << address.ToString()235+ VLOG(1) << "[TransportGet][Metadata] Inline data plane is unavailable for " << address.ToString()
276- << ", query metadata only: " << leaseRc.ToString();236+ << ", fallback to TCP: " << leaseRc.ToString();
277- // The RPC was not sent, so permanently downgrade this request chain to metadata-only mode.237+ SwitchInlineRequestToTcp(request, payloads, context);
278- context.DisableInlineData();
279- request.clear_data_request();
280- payloads.clear();
281 }238 }
239+ return InvokeTcpQueryAndGet(address, request, response, payloads, rpcDispatched);
240+}
282 241 
242+Status ObjectMetadataClient::InvokeInlineQueryAndGet(const HostPort &address, QueryAndGetReqPb &request,
243+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,
244+ InlineRequestContext &context, bool &invoked,
245+ bool &rpcDispatched)
246+{
247+ const auto hint = context.mode == InlineTransportMode::UB ? TransportHint::UB_CANDIDATE
248+ : TransportHint::SHM_CANDIDATE;
249+ invoked = false;
250+ return manager_->WithDataPlaneLease(
251+ address, hint,
252+ [&](const std::shared_ptr<IDataTransporter> &transporter,
253+ const std::shared_ptr<WorkerRpcClient> &rpcClient) {
254+ if (context.mode == InlineTransportMode::SHM
255+ && (transporter != context.shmTransporter || !context.shmSession->IsAlive())) {
256+ RETURN_STATUS(K_NOT_READY, "QueryAndGet shared-memory session changed before dispatch");
257+ }
258+ invoked = true;
259+ return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched);
260+ });
261+}
262+ 
263+Status ObjectMetadataClient::InvokeTcpQueryAndGet(const HostPort &address, QueryAndGetReqPb &request,
264+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,
265+ bool &rpcDispatched)
266+{
283 std::shared_ptr<WorkerRpcClient> rpcClient;267 std::shared_ptr<WorkerRpcClient> rpcClient;
284- Status rc = manager_->GetOrCreateRpcClient(address, rpcClient);268+ RETURN_IF_NOT_OK(manager_->GetOrCreateRpcClient(address, rpcClient));
285- if (redirectTopologyVersion.has_value() && IsTransportSnapshotStaleLocation(rc)) {
286- VLOG(1) << "[TransportGet][Metadata] Transport snapshot lags redirected owner, query metadata only: "
287- << address.ToString() << ", redirect version: " << *redirectTopologyVersion;
288- if (metadataFailureHandler_) {
289- metadataFailureHandler_(address, rc);
290- }
291- context.DisableInlineData();
292- request.clear_data_request();
293- payloads.clear();
294- rc = manager_->GetOrCreateRedirectMetadataRpcClient(address, *redirectTopologyVersion, rpcClient);
295- }
296- RETURN_IF_NOT_OK(rc);
297 RETURN_RUNTIME_ERROR_IF_NULL(rpcClient);269 RETURN_RUNTIME_ERROR_IF_NULL(rpcClient);
298 return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched);270 return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched);
299}271}
300 272 
273+void ObjectMetadataClient::SwitchInlineRequestToTcp(QueryAndGetReqPb &request, std::vector<RpcMessage> &payloads,
274+ InlineRequestContext &context) const
275+{
276+ context.DisableInlineData();
277+ context.mode = InlineTransportMode::TCP;
278+ (void)request.mutable_data_request()->mutable_tcp();
279+ payloads.clear();
280+}
281+ 
301Status ObjectMetadataClient::QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items,282Status ObjectMetadataClient::QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items,
302- bool allowRedirect, master::QueryAndGetRspPb &response,283+ QueryAndGetRspPb &response,
303- std::vector<RpcMessage> &payloads, InlineRequestContext &context,284+ std::vector<RpcMessage> &payloads, InlineRequestContext &context)
304- std::optional<uint64_t> redirectTopologyVersion)
305{285{
306 RETURN_RUNTIME_ERROR_IF_NULL(retry_);286 RETURN_RUNTIME_ERROR_IF_NULL(retry_);
307 CHECK_FAIL_RETURN_STATUS(!items.empty(), K_INVALID, "Metadata query items are empty");287 CHECK_FAIL_RETURN_STATUS(!items.empty(), K_INVALID, "Metadata query items are empty");
308 int64_t backoffMs = 1;288 int64_t backoffMs = 1;
309 size_t attempt = 0;289 size_t attempt = 0;
310- // The context keeps prepared UB buffers reusable across RPC retries and the deadline-bounded redirect chain.290+ // The context keeps prepared data-plane state reusable across RPC retries.
311 while (true) {291 while (true) {
312 ++attempt;292 ++attempt;
313 RETURN_IF_NOT_OK(retry_->CheckDeadline());293 RETURN_IF_NOT_OK(retry_->CheckDeadline());
314- master::QueryAndGetReqPb request;294+ QueryAndGetReqPb request;
315- RETURN_IF_NOT_OK(BuildQueryRequest(items, allowRedirect, context, request));295+ RETURN_IF_NOT_OK(BuildQueryRequest(address, items, context, request));
316 response.Clear();296 response.Clear();
317 payloads.clear();297 payloads.clear();
318 VLOG(1) << "[TransportGet][Metadata] Query, meta owner: " << address.ToString()298 VLOG(1) << "[TransportGet][Metadata] Query, meta owner: " << address.ToString()
319- << ", key count: " << items.size() << ", redirect: " << allowRedirect299+ << ", key count: " << items.size() << ", attempt: " << attempt;
320- << ", attempt: " << attempt;
321 bool rpcDispatched = false;300 bool rpcDispatched = false;
322- Status rc = InvokeQueryAndGet(address, request, response, payloads, context, redirectTopologyVersion,301+ Status rc = InvokeQueryAndGet(address, request, response, payloads, context, rpcDispatched);
323- rpcDispatched);302+ RETURN_OK_IF_TRUE(rc.IsOk());
324- if (rc.IsError()) {303+ RETURN_IF_NOT_OK(PrepareQueryRetry(address, rc, rpcDispatched, context, backoffMs));
325- const bool routeFailure = IsMetadataOwnerRouteFailure(rc.GetCode());
326- if (routeFailure && metadataFailureHandler_) {
327- metadataFailureHandler_(address, rc);
328- }
329- // Tear down the shared channel only when the connection is genuinely unusable:
330- // a dead peer, a legacy peer-unreachable (K_RPC_UNAVAILABLE), or a transient
331- // blip we can prove never left the client (request definitely not sent). Pure
332- // DEADLINE/CANCELLED may have reached the master, so keep the channel and just
333- // backoff+retry — avoids rebuilding the shared channel on every slow-master
334- // timeout. K_RPC_UNAVAILABLE is torn down unconditionally to match the legacy
335- // behaviour: a peer that returned UNAVAILABLE must not be retried on the same
336- // cached connection.
337- const bool teardownWarranted = IsNonRetryableRpcError(rc)
338- || rc.GetCode() == K_RPC_UNAVAILABLE
339- || (IsRetryableRpcError(rc) && IsBrpcRequestDefinitelyNotSent(rc));
340- if (teardownWarranted && manager_ != nullptr) {
341- manager_->Teardown(address);
342- }
343- if (routeFailure) {
344- VLOG(1) << "[TransportGet][Metadata] Return stale route for outer retry, meta owner: "
345- << address.ToString() << ", dispatched: " << rpcDispatched
346- << ", status: " << rc.ToString();
347- return Status(K_NOT_READY, STALE_TRANSPORT_SNAPSHOT_MESSAGE);
348- }
349- if (!retry_->IsRetryableRpcError(rc)) {
350- VLOG(1) << "[TransportGet][Metadata] Query failed without retry, meta owner: "
351- << address.ToString() << ", status: " << rc.ToString();
352- return rc;
353- }
354- VLOG(1) << "[TransportGet][Metadata] Retrying query, meta owner: " << address.ToString()
355- << ", status: " << rc.ToString();
356- RETURN_IF_NOT_OK(retry_->Backoff(backoffMs));
357- continue;
358- }
359- if (!response.meta_is_moving()) {
360- return Status::OK();
361- }
362- VLOG(1) << "[TransportGet][Metadata] Metadata is moving, meta owner: " << address.ToString()
363- << ", key count: " << items.size();
364- RETURN_IF_NOT_OK(retry_->Backoff(backoffMs));
365 }304 }
366}305}
367 306 
368-Status ObjectMetadataClient::BuildQueryRequest(const ObjectMetadataBatch &items, bool allowRedirect,307+Status ObjectMetadataClient::PrepareQueryRetry(const HostPort &address, const Status &rc, bool rpcDispatched,
369- const InlineRequestContext &context,308+ InlineRequestContext &context, int64_t &backoffMs)
370- master::QueryAndGetReqPb &request) const309+{
310+ if (rpcDispatched && context.mode == InlineTransportMode::SHM && context.shmTransporter != nullptr) {
311+ context.shmTransporter->InvalidateSession(context.shmSession);
312+ }
313+ const bool routeFailure = IsMetadataOwnerRouteFailure(rc.GetCode());
314+ if (routeFailure && metadataFailureHandler_) {
315+ metadataFailureHandler_(address, rc);
316+ }
317+ const bool teardownWarranted = IsNonRetryableRpcError(rc) || rc.GetCode() == K_RPC_UNAVAILABLE
318+ || (IsRetryableRpcError(rc) && IsBrpcRequestDefinitelyNotSent(rc));
319+ if (teardownWarranted) {
320+ manager_->Teardown(address);
321+ }
322+ if (routeFailure) {
323+ VLOG(1) << "[TransportGet][Metadata] Return stale route for outer retry, meta owner: "
324+ << address.ToString() << ", dispatched: " << rpcDispatched << ", status: " << rc.ToString();
325+ return Status(K_NOT_READY, STALE_TRANSPORT_SNAPSHOT_MESSAGE);
326+ }
327+ if (rpcDispatched && context.mode == InlineTransportMode::SHM) {
328+ VLOG(1) << "[TransportGet][Metadata] Do not replay an ambiguous SHM QueryAndGet: " << rc.ToString();
O
OOuGongChang6 天前

[P3] SHM dispatched 失败对一切错误码(含可证明未发出的请求)一律 InvalidateSession 且不重放,恢复路径偏长

rpcDispatched 在 WorkerRpcClient::InvokeQueryAndGet 中于 DoInvoke 之前即置 true(worker_rpc_client.cpp:248),因此"签名生成失败/超时计算失败等请求实际未发出"的场景也被当作模糊结果处理:InvalidateSession + 直接返回 rc(非 routeFailure 时),只能依赖外层 STALE 有界重试(5 次 + deadline,object_client_impl.cpp)重建 session 恢复,同一次 Get 时延被显著放大。

建议对 IsBrpcRequestDefinitelyNotSent(rc) 的 SHM 失败走正常重试(请求未发出即无 AddShmUnit 副作用,重放安全),仅对真正可能已到达服务端的失败采用不重放语义。

likedislike
yangxiaogang14
5 天前 评论:
329+ return rc;
330+ }
331+ if (!retry_->IsRetryableRpcError(rc)) {
332+ VLOG(1) << "[TransportGet][Metadata] Query failed without retry, meta owner: " << address.ToString()
333+ << ", status: " << rc.ToString();
334+ return rc;
335+ }
336+ VLOG(1) << "[TransportGet][Metadata] Retrying query, meta owner: " << address.ToString()
337+ << ", status: " << rc.ToString();
338+ return retry_->Backoff(backoffMs);
339+}
340+ 
341+Status ObjectMetadataClient::BuildQueryRequest(const HostPort &address, const ObjectMetadataBatch &items,
342+ InlineRequestContext &context, QueryAndGetReqPb &request) const
371{343{
372 for (const auto *item : items) {344 for (const auto *item : items) {
373 request.add_object_keys(item->objectKey);345 request.add_object_keys(item->objectKey);
374 }346 }
375- request.set_redirect(allowRedirect);347+ if (context.mode == InlineTransportMode::SHM) {
348+ bool sessionAvailable = context.shmSession != nullptr && context.shmSession->IsAlive();
349+ INJECT_POINT_NO_RETURN("client.transport.query_and_get.shm_session_unavailable_before_build",
350+ [&sessionAvailable]() { sessionAvailable = false; });
351+ if (!sessionAvailable) {
352+ VLOG(1) << "[TransportGet][Metadata] SHM session is unavailable while building QueryAndGet; "
353+ "selecting UB or TCP fallback";
354+ RETURN_IF_NOT_OK(PrepareShmInlineFallback(address, items, context));
355+ }
356+ }
376 return AddInlineDataRequest(items, context, request);357 return AddInlineDataRequest(items, context, request);
377}358}
378 359 
379Status ObjectMetadataClient::ApplyResults(const ObjectMetadataBatch &items,360Status ObjectMetadataClient::ApplyResults(const ObjectMetadataBatch &items,
380- const master::QueryAndGetRspPb &response,361+ const QueryAndGetRspPb &response,
381 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const362 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const
382{363{
383 // Keep the count check because results are accessed positionally below.364 // Keep the count check because results are accessed positionally below.
@@ -389,16 +370,18 @@ Status ObjectMetadataClient::ApplyResults(const ObjectMetadataBatch &items,
389 return Status::OK();370 return Status::OK();
390}371}
391 372 
392-Status ObjectMetadataClient::ApplyResult(ObjectMetadataItem &item, const master::QueryAndGetResultPb &result,373+Status ObjectMetadataClient::ApplyResult(ObjectMetadataItem &item, const QueryAndGetResultPb &result,
393 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const374 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const
394{375{
395 const auto &location = result.location();376 const auto &location = result.location();
377+ CHECK_FAIL_RETURN_STATUS(location.object_key() == item.objectKey, K_RUNTIME_ERROR,
378+ "QueryAndGet result key does not match request order");
396 if (location.object_locations_size() == 0) {379 if (location.object_locations_size() == 0) {
397 item.status = Status(K_NOT_FOUND, "Object was not found");380 item.status = Status(K_NOT_FOUND, "Object was not found");
398 return Status::OK();381 return Status::OK();
399 }382 }
400 item.status = Status::OK();383 item.status = Status::OK();
401- item.location = location;384+ CopyLocation(location, item.location);
402 if (!result.has_data_result()) {385 if (!result.has_data_result()) {
403 // Absence of data_result is a per-key fast-path miss; the caller will execute phase two.386 // Absence of data_result is a per-key fast-path miss; the caller will execute phase two.
404 return Status::OK();387 return Status::OK();
@@ -411,30 +394,73 @@ Status ObjectMetadataClient::ApplyResult(ObjectMetadataItem &item, const master:
411 DataGetResult data;394 DataGetResult data;
412 data.response.mutable_error()->set_error_code(K_OK);395 data.response.mutable_error()->set_error_code(K_OK);
413 data.response.set_data_size(static_cast<int64_t>(location.object_size()));396 data.response.set_data_size(static_cast<int64_t>(location.object_size()));
414- Status rc = context.mode == InlineTransportMode::TCP397+ Status rc;
415- ? BuildTcpInlineData(result.data_result(), payloads, data)398+ if (context.mode == InlineTransportMode::TCP) {
416- : BuildUbInlineData(item, location, context, data);399+ CHECK_FAIL_RETURN_STATUS(!result.data_result().has_shm_info(), K_RUNTIME_ERROR,
400+ "TCP QueryAndGet returned shared-memory data");
401+ rc = BuildTcpInlineData(result.data_result(), location.object_size(), payloads, data);
402+ } else if (context.mode == InlineTransportMode::UB) {
403+ CHECK_FAIL_RETURN_STATUS(!result.data_result().has_shm_info(), K_RUNTIME_ERROR,
404+ "UB QueryAndGet returned shared-memory data");
405+ CHECK_FAIL_RETURN_STATUS(result.data_result().payload_indexes_size() == 0, K_RUNTIME_ERROR,
406+ "UB QueryAndGet returned TCP payload indexes");
407+ rc = BuildUbInlineData(item, item.location, context, data);
408+ } else {
409+ CHECK_FAIL_RETURN_STATUS(result.data_result().has_shm_info(), K_RUNTIME_ERROR,
410+ "SHM QueryAndGet did not return shared-memory data");
411+ CHECK_FAIL_RETURN_STATUS(result.data_result().payload_indexes_size() == 0, K_RUNTIME_ERROR,
412+ "SHM QueryAndGet returned TCP payload indexes");
413+ rc = BuildShmInlineData(item, result.data_result().shm_info(), context, data);
414+ }
415+ if (rc.IsError() && context.mode == InlineTransportMode::SHM) {
416+ VLOG(1) << "[ObjectKey " << item.objectKey
417+ << "] QueryAndGet SHM materialization fallback: " << rc.ToString();
418+ if (context.shmTransporter != nullptr) {
419+ context.shmTransporter->InvalidateSession(context.shmSession);
cheyulin
cheyulincheyulin5 天前

[严重][正确性] [P1] 单个 SHM 物化失败不能撤销同 session 的全部既有 Buffer 引用

证据:这里在任一 key 的 BuildShmInlineData 失败时调用 InvalidateSession。该调用最终执行 ShmSession::Close(true) 并异步 DisconnectClient;Worker 的 RefreshMeta 会对这个 session 的 client_id 执行 memoryRefTable_->RemoveClient,一次删除其全部 SHM 引用。此前已经返回给应用且仍在持有的 Buffer 也属于同一 client_id,而 session 关闭后其 owner 无法再正常 DecreaseReference

问题:一次冷 FD、认证或本地物化错误被提升成 session 级断开,但 session 同时还是此前所有零拷贝 Buffer 的服务端引用所有者;错误恢复跨越了仍然存活的 Buffer 生命周期。

影响:Worker 可以回收或复用旧 Buffer 对应的 SHM,而应用仍持有其映射指针,导致已成功返回的数据随后被覆盖或读到错误内容,属于内存生命周期和数据正确性风险。

建议:不要因单 key 物化失败立即 disconnect 整个 session。精确释放本次响应新增但未交付的 shm_id;若必须换代 session,应将旧 session 标记为 retired,并延迟 Worker disconnect,直到该 session 的全部 Buffer owner 都已释放。

验证:先通过同一 session 获取并保留 Buffer A,再对 Buffer B 注入 SHM materialization failure;验证 A 在失败、fallback 和新 session 建立后内容仍稳定,Worker 引用直到 A 析构才减少,并覆盖 multi-ref 与 legacy refcount 两种 Worker。

likedislike
420+ }
421+ return Status::OK();
422+ }
417 RETURN_IF_NOT_OK(rc);423 RETURN_IF_NOT_OK(rc);
418 item.inlineData.emplace(std::move(data));424 item.inlineData.emplace(std::move(data));
419 return Status::OK();425 return Status::OK();
420}426}
421 427 
422-Status ObjectMetadataClient::BuildTcpInlineData(const master::QueryAndGetDataResultPb &dataResult,428+Status ObjectMetadataClient::BuildTcpInlineData(const QueryAndGetDataResultPb &dataResult, uint64_t objectSize,
423 std::vector<RpcMessage> &payloads,429 std::vector<RpcMessage> &payloads,
424 DataGetResult &data) const430 DataGetResult &data) const
425{431{
426 data.rpcPayloads.reserve(dataResult.payload_indexes_size());432 data.rpcPayloads.reserve(dataResult.payload_indexes_size());
433+ uint64_t payloadSize = 0;
427 for (uint32_t payloadIndex : dataResult.payload_indexes()) {434 for (uint32_t payloadIndex : dataResult.payload_indexes()) {
428- // Keep only the check required before indexing RPC payload storage.
429 CHECK_FAIL_RETURN_STATUS(payloadIndex < payloads.size(), K_RUNTIME_ERROR,435 CHECK_FAIL_RETURN_STATUS(payloadIndex < payloads.size(), K_RUNTIME_ERROR,
430 "QueryAndGet payload index is out of range");436 "QueryAndGet payload index is out of range");
437+ CHECK_FAIL_RETURN_STATUS(
438+ payloadSize <= objectSize && payloads[payloadIndex].Size() <= objectSize - payloadSize,
439+ K_RUNTIME_ERROR, "QueryAndGet TCP payload exceeds object size");
440+ payloadSize += payloads[payloadIndex].Size();
431 data.rpcPayloads.emplace_back(std::move(payloads[payloadIndex]));441 data.rpcPayloads.emplace_back(std::move(payloads[payloadIndex]));
432 }442 }
443+ CHECK_FAIL_RETURN_STATUS(payloadSize == objectSize, K_RUNTIME_ERROR,
444+ "QueryAndGet TCP payload size does not match object size");
433 data.response.set_data_source(DataTransferSource::DATA_IN_PAYLOAD);445 data.response.set_data_source(DataTransferSource::DATA_IN_PAYLOAD);
434 data.kind = AccessTransportKind::TCP;446 data.kind = AccessTransportKind::TCP;
435 return Status::OK();447 return Status::OK();
436}448}
437 449 
450+Status ObjectMetadataClient::BuildShmInlineData(ObjectMetadataItem &item, const QueryAndGetShmInfoPb &shmInfo,
451+ InlineRequestContext &context, DataGetResult &data) const
452+{
453+ INJECT_POINT("client.transport.query_and_get.shm_materialization_failure");
454+ CHECK_FAIL_RETURN_STATUS(context.shmSession != nullptr && context.shmTransporter != nullptr
455+ && context.readContext != nullptr,
456+ K_RUNTIME_ERROR, "SHM QueryAndGet session context is missing");
457+ CHECK_FAIL_RETURN_STATUS(shmInfo.data_size() >= 0
458+ && static_cast<uint64_t>(shmInfo.data_size()) == item.location.object_size(),
459+ K_RUNTIME_ERROR, "SHM QueryAndGet data size does not match object size");
460+ DataGetRequest input{ item.objectKey, static_cast<uint64_t>(shmInfo.data_size()), context.readContext };
461+ return context.shmSession->BuildQueryAndGetResult(shmInfo, input, data);
462+}
463+ 
438Status ObjectMetadataClient::BuildUbInlineData(ObjectMetadataItem &item,464Status ObjectMetadataClient::BuildUbInlineData(ObjectMetadataItem &item,
439 const master::ObjectLocationInfoPb &location,465 const master::ObjectLocationInfoPb &location,
440 InlineRequestContext &context, DataGetResult &data) const466 InlineRequestContext &context, DataGetResult &data) const
@@ -457,59 +483,28 @@ Status ObjectMetadataClient::BuildUbInlineData(ObjectMetadataItem &item,
457}483}
458 484 
459Status ObjectMetadataClient::Query(const HostPort &address, const ObjectMetadataBatch &items,485Status ObjectMetadataClient::Query(const HostPort &address, const ObjectMetadataBatch &items,
460- bool enableInlineData)486+ bool enableInlineData, std::shared_ptr<const TransportReadContext> readContext)
461{487{
462 RETURN_IF_NOT_OK(ValidateAndResetItems(items));488 RETURN_IF_NOT_OK(ValidateAndResetItems(items));
463 InlineRequestContext context;489 InlineRequestContext context;
464 if (enableInlineData) {490 if (enableInlineData) {
465- RETURN_IF_NOT_OK(InitializeInlineRequest(address, items, context));491+ RETURN_IF_NOT_OK(InitializeInlineRequest(address, items, std::move(readContext), context));
466 }492 }
467 493 
468- master::QueryAndGetRspPb response;494+ QueryAndGetRspPb response;
469 std::vector<RpcMessage> payloads;495 std::vector<RpcMessage> payloads;
470- RETURN_IF_NOT_OK(QueryWithRetry(address, items, true, response, payloads, context, std::nullopt));496+ RETURN_IF_NOT_OK(QueryWithRetry(address, items, response, payloads, context));
471- ObjectMetadataBatch localItems;497+ Status rc = ApplyResults(items, response, payloads, context);
472- std::vector<RedirectBatch> redirectBatches;498+ if (rc.IsError() && context.mode == InlineTransportMode::SHM && context.shmTransporter != nullptr) {
473- RETURN_IF_NOT_OK(PartitionInitialResponse(items, response, localItems, redirectBatches, std::nullopt));499+ context.shmTransporter->InvalidateSession(context.shmSession);
474- VLOG(1) << "[TransportGet][Metadata] Query resolved, meta owner: " << address.ToString()
475- << ", local keys: " << localItems.size() << ", redirect groups: " << redirectBatches.size();
476- RETURN_IF_NOT_OK(ApplyResults(localItems, response, payloads, context));
477- if (redirectBatches.empty()) {
478- return Status::OK();
479 }500 }
480- 501+ return rc;
481- std::deque<PendingMetadataBatch> pending;
482- QueueRedirectBatches(redirectBatches, pending);
483- while (!pending.empty()) {
484- PendingMetadataBatch current = std::move(pending.front());
485- pending.pop_front();
486- response.Clear();
487- payloads.clear();
488- Status rc = QueryWithRetry(current.address, current.items, true, response, payloads, context,
489- current.topologyVersion);
490- localItems.clear();
491- redirectBatches.clear();
492- if (rc.IsOk()) {
493- rc = PartitionInitialResponse(current.items, response, localItems, redirectBatches,
494- current.topologyVersion);
495- }
496- if (rc.IsOk()) {
497- VLOG(1) << "[TransportGet][Metadata] Query resolved, meta owner: " << current.address.ToString()
498- << ", local keys: " << localItems.size() << ", redirect groups: " << redirectBatches.size();
499- rc = ApplyResults(localItems, response, payloads, context);
500- }
501- if (rc.IsError()) {
502- SetBatchError(current.items, rc);
503- continue;
504- }
505- QueueRedirectBatches(redirectBatches, pending);
506- }
507- return Status::OK();
508}502}
509 503 
510-Status ObjectMetadataClient::QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items)504+Status ObjectMetadataClient::QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items,
505+ std::shared_ptr<const TransportReadContext> readContext)
511{506{
512- return Query(address, items, true);507+ return Query(address, items, true, std::move(readContext));
513}508}
514 509 
515Status ObjectMetadataClient::QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items)510Status ObjectMetadataClient::QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items)
Msrc/datasystem/client/transport/metadata/object_metadata_client.h+48-17
@@ -31,6 +31,7 @@
31#include "datasystem/client/transport/data_plane/ub_transporter.h"31#include "datasystem/client/transport/data_plane/ub_transporter.h"
32#include "datasystem/client/transport/transport_advisor.h"32#include "datasystem/client/transport/transport_advisor.h"
33#include "datasystem/protos/master_object.pb.h"33#include "datasystem/protos/master_object.pb.h"
34+#include "datasystem/protos/object_posix.pb.h"
34 35 
35namespace datasystem {36namespace datasystem {
36namespace client {37namespace client {
@@ -55,23 +56,28 @@ public:
55 virtual ~ObjectMetadataClient() = default;56 virtual ~ObjectMetadataClient() = default;
56 57 
57 /**58 /**
58- * @brief Query a metadata-owner group and resolve redirects within the API deadline.59+ * @brief Query a metadata-owner Worker for ordered locations and optional inline data.
59 * @param[in] address Initial metadata owner.60 * @param[in] address Initial metadata owner.
60 * @param[in,out] items Ordered object metadata states belonging to address.61 * @param[in,out] items Ordered object metadata states belonging to address.
62+ * @param[in] readContext Read context required by the shared-memory transport.
61 * @return K_OK after every item has an independent result; a group-wide error otherwise.63 * @return K_OK after every item has an independent result; a group-wide error otherwise.
62 */64 */
63- virtual Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items);65+ virtual Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items,
66+ std::shared_ptr<const TransportReadContext> readContext);
64 67 
65 /** @brief Query object locations without requesting inline object data. */68 /** @brief Query object locations without requesting inline object data. */
66 virtual Status QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items);69 virtual Status QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items);
67 70 
68private:71private:
69- enum class InlineTransportMode : uint8_t { NONE = 0, TCP = 1, UB = 2 };72+ enum class InlineTransportMode : uint8_t { NONE = 0, TCP = 1, UB = 2, SHM = 3 };
70 73 
71 struct InlineRequestContext {74 struct InlineRequestContext {
72 InlineTransportMode mode = InlineTransportMode::NONE;75 InlineTransportMode mode = InlineTransportMode::NONE;
73 std::unordered_map<ObjectMetadataItem *, UbReceiveBuffer> ubBuffers;76 std::unordered_map<ObjectMetadataItem *, UbReceiveBuffer> ubBuffers;
74 std::string transportInstanceId;77 std::string transportInstanceId;
78+ std::shared_ptr<ShmTransporter> shmTransporter;
79+ std::shared_ptr<ShmSession> shmSession;
80+ std::shared_ptr<const TransportReadContext> readContext;
75 81 
76 /** @brief Disable inline transfer and release prepared receive buffers. */82 /** @brief Disable inline transfer and release prepared receive buffers. */
77 void DisableInlineData()83 void DisableInlineData()
@@ -79,20 +85,33 @@ private:
79 mode = InlineTransportMode::NONE;85 mode = InlineTransportMode::NONE;
80 ubBuffers.clear();86 ubBuffers.clear();
81 transportInstanceId.clear();87 transportInstanceId.clear();
88+ shmTransporter.reset();
89+ shmSession.reset();
90+ readContext.reset();
82 }91 }
83 };92 };
84 93 
85- Status Query(const HostPort &address, const ObjectMetadataBatch &items, bool enableInlineData);94+ Status Query(const HostPort &address, const ObjectMetadataBatch &items, bool enableInlineData,
95+ std::shared_ptr<const TransportReadContext> readContext = nullptr);
86 96 
87- Status QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items, bool allowRedirect,97+ Status QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items,
88- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,98+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,
89- InlineRequestContext &context, std::optional<uint64_t> redirectTopologyVersion);99+ InlineRequestContext &context);
90- Status BuildQueryRequest(const ObjectMetadataBatch &items, bool allowRedirect,100+ Status BuildQueryRequest(const HostPort &address, const ObjectMetadataBatch &items,
91- const InlineRequestContext &context, master::QueryAndGetReqPb &request) const;101+ InlineRequestContext &context, QueryAndGetReqPb &request) const;
92- Status InvokeQueryAndGet(const HostPort &address, master::QueryAndGetReqPb &request,102+ Status InvokeQueryAndGet(const HostPort &address, QueryAndGetReqPb &request,
93- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,103+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,
94- InlineRequestContext &context, std::optional<uint64_t> redirectTopologyVersion,104+ InlineRequestContext &context, bool &rpcDispatched);
95- bool &rpcDispatched);105+ Status InvokeInlineQueryAndGet(const HostPort &address, QueryAndGetReqPb &request,
106+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,
107+ InlineRequestContext &context, bool &invoked, bool &rpcDispatched);
108+ Status InvokeTcpQueryAndGet(const HostPort &address, QueryAndGetReqPb &request,
109+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads,
110+ bool &rpcDispatched);
111+ void SwitchInlineRequestToTcp(QueryAndGetReqPb &request, std::vector<RpcMessage> &payloads,
112+ InlineRequestContext &context) const;
113+ Status PrepareQueryRetry(const HostPort &address, const Status &rc, bool rpcDispatched,
114+ InlineRequestContext &context, int64_t &backoffMs);
96 115 
97 /**116 /**
98 * @brief Select and initialize the inline transport for one metadata-owner request.117 * @brief Select and initialize the inline transport for one metadata-owner request.
@@ -102,8 +121,17 @@ private:
102 * @return K_OK on success; the error code otherwise.121 * @return K_OK on success; the error code otherwise.
103 */122 */
104 Status InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items,123 Status InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items,
124+ std::shared_ptr<const TransportReadContext> readContext,
105 InlineRequestContext &context) const;125 InlineRequestContext &context) const;
106 126 
127+ Status PrepareShmInlineRequest(const HostPort &address,
128+ std::shared_ptr<const TransportReadContext> readContext,
129+ InlineRequestContext &context) const;
130+ 
131+ /** @brief Replace an unavailable SHM inline request with an available UB or TCP request. */
132+ Status PrepareShmInlineFallback(const HostPort &address, const ObjectMetadataBatch &items,
133+ InlineRequestContext &context) const;
134+ 
107 /**135 /**
108 * @brief Prepare UB inline transfer when the endpoint and client configuration support it.136 * @brief Prepare UB inline transfer when the endpoint and client configuration support it.
109 * @param[in] address Metadata-owner address.137 * @param[in] address Metadata-owner address.
@@ -130,7 +158,7 @@ private:
130 * @return K_OK on success; the error code otherwise.158 * @return K_OK on success; the error code otherwise.
131 */159 */
132 Status AddInlineDataRequest(const ObjectMetadataBatch &items, const InlineRequestContext &context,160 Status AddInlineDataRequest(const ObjectMetadataBatch &items, const InlineRequestContext &context,
133- master::QueryAndGetReqPb &request) const;161+ QueryAndGetReqPb &request) const;
134 162 
135 /**163 /**
136 * @brief Apply ordered metadata and inline-data results to a metadata batch.164 * @brief Apply ordered metadata and inline-data results to a metadata batch.
@@ -140,7 +168,7 @@ private:
140 * @param[in,out] context Inline-request context.168 * @param[in,out] context Inline-request context.
141 * @return K_OK on success; the error code otherwise.169 * @return K_OK on success; the error code otherwise.
142 */170 */
143- Status ApplyResults(const ObjectMetadataBatch &items, const master::QueryAndGetRspPb &response,171+ Status ApplyResults(const ObjectMetadataBatch &items, const QueryAndGetRspPb &response,
144 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const;172 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const;
145 173 
146 /**174 /**
@@ -151,7 +179,7 @@ private:
151 * @param[in,out] context Inline-request context.179 * @param[in,out] context Inline-request context.
152 * @return K_OK on success; the error code otherwise.180 * @return K_OK on success; the error code otherwise.
153 */181 */
154- Status ApplyResult(ObjectMetadataItem &item, const master::QueryAndGetResultPb &result,182+ Status ApplyResult(ObjectMetadataItem &item, const QueryAndGetResultPb &result,
155 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const;183 std::vector<RpcMessage> &payloads, InlineRequestContext &context) const;
156 184 
157 /**185 /**
@@ -161,9 +189,12 @@ private:
161 * @param[out] data Data-read result.189 * @param[out] data Data-read result.
162 * @return K_OK on success; the error code otherwise.190 * @return K_OK on success; the error code otherwise.
163 */191 */
164- Status BuildTcpInlineData(const master::QueryAndGetDataResultPb &dataResult,192+ Status BuildTcpInlineData(const QueryAndGetDataResultPb &dataResult, uint64_t objectSize,
165 std::vector<RpcMessage> &payloads, DataGetResult &data) const;193 std::vector<RpcMessage> &payloads, DataGetResult &data) const;
166 194 
195+ Status BuildShmInlineData(ObjectMetadataItem &item, const QueryAndGetShmInfoPb &shmInfo,
196+ InlineRequestContext &context, DataGetResult &data) const;
197+ 
167 /**198 /**
168 * @brief Move ownership of a prepared UB receive buffer into a data-read result.199 * @brief Move ownership of a prepared UB receive buffer into a data-read result.
169 * @param[in,out] item Object metadata item owning the buffer association.200 * @param[in,out] item Object metadata item owning the buffer association.
Msrc/datasystem/client/transport/object_read/object_read_flow.cpp+9-7
@@ -46,11 +46,13 @@ struct MetadataGroup {
46 {46 {
47 }47 }
48 48 
49- void Resolve(ObjectMetadataClient &metadata, const Status &dispatchStatus, bool metadataOnly)49+ void Resolve(ObjectMetadataClient &metadata, const Status &dispatchStatus, bool metadataOnly,
50+ const std::shared_ptr<const TransportReadContext> &context)
50 {51 {
51 Status status = dispatchStatus;52 Status status = dispatchStatus;
52 if (status.IsOk()) {53 if (status.IsOk()) {
53- status = metadataOnly ? metadata.QueryMetadata(address, items) : metadata.QueryAndGet(address, items);54+ status = metadataOnly ? metadata.QueryMetadata(address, items)
55+ : metadata.QueryAndGet(address, items, context);
54 }56 }
55 if (status.IsError()) {57 if (status.IsError()) {
56 for (auto *item : items) {58 for (auto *item : items) {
@@ -144,7 +146,7 @@ Status InitializeItems(const ObjectReadRequest &request, std::vector<ReadItem> &
144}146}
145 147 
146void QueryMetadata(ObjectMetadataClient &metadata, ThreadPool &taskPool, std::vector<ReadItem> &items,148void QueryMetadata(ObjectMetadataClient &metadata, ThreadPool &taskPool, std::vector<ReadItem> &items,
147- bool metadataOnly = false)149+ const std::shared_ptr<const TransportReadContext> &context, bool metadataOnly = false)
148{150{
149 auto groups = GroupByMetaOwner(items);151 auto groups = GroupByMetaOwner(items);
150 VLOG(1) << "[TransportGet][Flow] Query metadata, key count: " << items.size()152 VLOG(1) << "[TransportGet][Flow] Query metadata, key count: " << items.size()
@@ -156,8 +158,8 @@ void QueryMetadata(ObjectMetadataClient &metadata, ThreadPool &taskPool, std::ve
156 for (auto &group : groups) {158 for (auto &group : groups) {
157 tasks.emplace_back(&group);159 tasks.emplace_back(&group);
158 }160 }
159- RunTasks(taskPool, tasks, [&metadata, metadataOnly](MetadataGroup &group, const Status &dispatchStatus) {161+ RunTasks(taskPool, tasks, [&metadata, metadataOnly, context](MetadataGroup &group, const Status &dispatchStatus) {
160- group.Resolve(metadata, dispatchStatus, metadataOnly);162+ group.Resolve(metadata, dispatchStatus, metadataOnly, context);
161 });163 });
162 const auto resolved = std::count_if(items.begin(), items.end(), [](const ReadItem &item) {164 const auto resolved = std::count_if(items.begin(), items.end(), [](const ReadItem &item) {
163 return item.metadata.status.IsOk();165 return item.metadata.status.IsOk();
@@ -251,7 +253,7 @@ Status ObjectReadFlow::ResolveMetadata(const ObjectReadRequest &request,
251 RETURN_RUNTIME_ERROR_IF_NULL(taskPool_);253 RETURN_RUNTIME_ERROR_IF_NULL(taskPool_);
252 std::vector<ReadItem> items;254 std::vector<ReadItem> items;
253 RETURN_IF_NOT_OK(InitializeItems(request, items));255 RETURN_IF_NOT_OK(InitializeItems(request, items));
254- QueryMetadata(*metadata_, *taskPool_, items, true);256+ QueryMetadata(*metadata_, *taskPool_, items, nullptr, true);
255 size_t resultSize = 0;257 size_t resultSize = 0;
256 for (const auto &input : request.items) {258 for (const auto &input : request.items) {
257 resultSize = std::max(resultSize, input.requestIndex + 1);259 resultSize = std::max(resultSize, input.requestIndex + 1);
@@ -275,7 +277,7 @@ Status ObjectReadFlow::Run(const ObjectReadRequest &request, ObjectReadResult &r
275 std::vector<ReadItem> items;277 std::vector<ReadItem> items;
276 RETURN_IF_NOT_OK(InitializeItems(request, items));278 RETURN_IF_NOT_OK(InitializeItems(request, items));
277 AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_START);279 AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_START);
278- QueryMetadata(*metadata_, *taskPool_, items);280+ QueryMetadata(*metadata_, *taskPool_, items, request.context);
279 AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_END);281 AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_END);
280 AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_GET_DATA_START);282 AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_GET_DATA_START);
281 ReadObjects(*replicas_, items, request.context, request.traceEnabled);283 ReadObjects(*replicas_, items, request.context, request.traceEnabled);
Msrc/datasystem/client/transport/rpc/worker_rpc_client.cpp+7-12
@@ -76,18 +76,14 @@ Status WorkerRpcClient::Init()
76 auto transportStub = std::make_shared<WorkerWorkerTransportService_BrpcGenericStub>(76 auto transportStub = std::make_shared<WorkerWorkerTransportService_BrpcGenericStub>(
77 channel.get(), channelConfig_.timeout_ms);77 channel.get(), channelConfig_.timeout_ms);
78 auto dataStub = std::make_shared<WorkerWorkerOCService_BrpcGenericStub>(channel.get(), channelConfig_.timeout_ms);78 auto dataStub = std::make_shared<WorkerWorkerOCService_BrpcGenericStub>(channel.get(), channelConfig_.timeout_ms);
79- auto masterStub =
80- std::make_shared<master::MasterOCService_BrpcGenericStub>(channel.get(), channelConfig_.timeout_ms);
81 CHECK_FAIL_RETURN_STATUS(workerStub != nullptr && controlStub != nullptr && transportStub != nullptr79 CHECK_FAIL_RETURN_STATUS(workerStub != nullptr && controlStub != nullptr && transportStub != nullptr
82- && dataStub != nullptr80+ && dataStub != nullptr,
83- && masterStub != nullptr,
84 K_RUNTIME_ERROR, "Failed to create routed worker RPC stubs");81 K_RUNTIME_ERROR, "Failed to create routed worker RPC stubs");
85 channel_ = std::move(channel);82 channel_ = std::move(channel);
86 workerStub_ = std::move(workerStub);83 workerStub_ = std::move(workerStub);
87 controlStub_ = std::move(controlStub);84 controlStub_ = std::move(controlStub);
88 transportStub_ = std::move(transportStub);85 transportStub_ = std::move(transportStub);
89 dataStub_ = std::move(dataStub);86 dataStub_ = std::move(dataStub);
90- masterStub_ = std::move(masterStub);
91 connectionGeneration_ = nextConnectionGeneration_.fetch_add(1, std::memory_order_relaxed);87 connectionGeneration_ = nextConnectionGeneration_.fetch_add(1, std::memory_order_relaxed);
92 alive_.store(true, std::memory_order_release);88 alive_.store(true, std::memory_order_release);
93 return Status::OK();89 return Status::OK();
@@ -111,10 +107,10 @@ Status WorkerRpcClient::DoInvokeBatchGetObject(const RpcOptions &options, const
111 return dataStub_->BatchGetObjectRemote(options, request, response, payloads);107 return dataStub_->BatchGetObjectRemote(options, request, response, payloads);
112}108}
113 109 
114-Status WorkerRpcClient::DoInvokeQueryAndGet(const RpcOptions &options, const master::QueryAndGetReqPb &request,110+Status WorkerRpcClient::DoInvokeQueryAndGet(const RpcOptions &options, const QueryAndGetReqPb &request,
115- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads)111+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads)
116{112{
117- return masterStub_->QueryAndGet(options, request, response, payloads);113+ return controlStub_->QueryAndGet(options, request, response, payloads);
118}114}
119 115 
120Status WorkerRpcClient::DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response)116Status WorkerRpcClient::DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response)
@@ -234,14 +230,14 @@ Status WorkerRpcClient::InvokeBatchGetObject(BatchGetObjectRemoteReqPb &request,
234 return rc.IsError() ? WithRpcDiag(rc, "BatchGetObjectRemote", workerAddress_) : Status::OK();230 return rc.IsError() ? WithRpcDiag(rc, "BatchGetObjectRemote", workerAddress_) : Status::OK();
235}231}
236 232 
237-Status WorkerRpcClient::InvokeQueryAndGet(master::QueryAndGetReqPb &request, master::QueryAndGetRspPb &response,233+Status WorkerRpcClient::InvokeQueryAndGet(QueryAndGetReqPb &request, QueryAndGetRspPb &response,
238 std::vector<RpcMessage> &payloads, bool *rpcDispatched)234 std::vector<RpcMessage> &payloads, bool *rpcDispatched)
239{235{
240 if (rpcDispatched != nullptr) {236 if (rpcDispatched != nullptr) {
241 *rpcDispatched = false;237 *rpcDispatched = false;
242 }238 }
243 CHECK_FAIL_RETURN_STATUS(IsAlive(), K_RPC_UNAVAILABLE,239 CHECK_FAIL_RETURN_STATUS(IsAlive(), K_RPC_UNAVAILABLE,
244- "Routed master RPC client is not initialized");240+ "Routed worker RPC client is not initialized");
245 int32_t rpcTimeout;241 int32_t rpcTimeout;
246 RETURN_IF_NOT_OK(GetRpcTimeout(channelConfig_.timeout_ms, rpcTimeout));242 RETURN_IF_NOT_OK(GetRpcTimeout(channelConfig_.timeout_ms, rpcTimeout));
247 RETURN_IF_NOT_OK(signature_->GenerateSignature(request));243 RETURN_IF_NOT_OK(signature_->GenerateSignature(request));
@@ -503,13 +499,12 @@ bool WorkerRpcClient::IsAlive() const
503{499{
504 return alive_.load(std::memory_order_acquire) && channel_ != nullptr && workerStub_ != nullptr500 return alive_.load(std::memory_order_acquire) && channel_ != nullptr && workerStub_ != nullptr
505 && controlStub_ != nullptr501 && controlStub_ != nullptr
506- && transportStub_ != nullptr && dataStub_ != nullptr && masterStub_ != nullptr;502+ && transportStub_ != nullptr && dataStub_ != nullptr;
507}503}
508 504 
509void WorkerRpcClient::Close()505void WorkerRpcClient::Close()
510{506{
511 alive_.store(false, std::memory_order_release);507 alive_.store(false, std::memory_order_release);
512- masterStub_.reset();
513 dataStub_.reset();508 dataStub_.reset();
514 controlStub_.reset();509 controlStub_.reset();
515 workerStub_.reset();510 workerStub_.reset();
Msrc/datasystem/client/transport/rpc/worker_rpc_client.h+4-6
@@ -31,7 +31,6 @@
31#include "datasystem/common/rpc/rpc_message.h"31#include "datasystem/common/rpc/rpc_message.h"
32#include "datasystem/common/rpc/rpc_options.h"32#include "datasystem/common/rpc/rpc_options.h"
33#include "datasystem/common/util/net_util.h"33#include "datasystem/common/util/net_util.h"
34-#include "datasystem/protos/master_object.brpc.stub.pb.h"
35#include "datasystem/protos/meta_transport.pb.h"34#include "datasystem/protos/meta_transport.pb.h"
36#include "datasystem/protos/object_posix.brpc.stub.pb.h"35#include "datasystem/protos/object_posix.brpc.stub.pb.h"
37#include "datasystem/protos/share_memory.brpc.stub.pb.h"36#include "datasystem/protos/share_memory.brpc.stub.pb.h"
@@ -61,8 +60,8 @@ public:
61 virtual Status InvokeBatchGetObject(BatchGetObjectRemoteReqPb &request, BatchGetObjectRemoteRspPb &response,60 virtual Status InvokeBatchGetObject(BatchGetObjectRemoteReqPb &request, BatchGetObjectRemoteRspPb &response,
62 std::vector<RpcMessage> &payloads);61 std::vector<RpcMessage> &payloads);
63 62 
64- /** @brief Sign and query object locations and optional inline data from the master service at this endpoint. */63+ /** @brief Sign and query ordered object locations and optional inline data from this Worker. */
65- virtual Status InvokeQueryAndGet(master::QueryAndGetReqPb &request, master::QueryAndGetRspPb &response,64+ virtual Status InvokeQueryAndGet(QueryAndGetReqPb &request, QueryAndGetRspPb &response,
66 std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr);65 std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr);
67 66 
68 /** @brief Sign and invoke Exist through the cached control connection. */67 /** @brief Sign and invoke Exist through the cached control connection. */
@@ -128,8 +127,8 @@ protected:
128 virtual Status DoInvokeBatchGetObject(const RpcOptions &options, const BatchGetObjectRemoteReqPb &request,127 virtual Status DoInvokeBatchGetObject(const RpcOptions &options, const BatchGetObjectRemoteReqPb &request,
129 BatchGetObjectRemoteRspPb &response, std::vector<RpcMessage> &payloads);128 BatchGetObjectRemoteRspPb &response, std::vector<RpcMessage> &payloads);
130 129 
131- virtual Status DoInvokeQueryAndGet(const RpcOptions &options, const master::QueryAndGetReqPb &request,130+ virtual Status DoInvokeQueryAndGet(const RpcOptions &options, const QueryAndGetReqPb &request,
132- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads);131+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads);
133 132 
134 virtual Status DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response);133 virtual Status DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response);
135 134 
@@ -178,7 +177,6 @@ private:
178 std::shared_ptr<WorkerOCService_BrpcGenericStub> controlStub_;177 std::shared_ptr<WorkerOCService_BrpcGenericStub> controlStub_;
179 std::shared_ptr<WorkerWorkerTransportService_BrpcGenericStub> transportStub_;178 std::shared_ptr<WorkerWorkerTransportService_BrpcGenericStub> transportStub_;
180 std::shared_ptr<WorkerWorkerOCService_BrpcGenericStub> dataStub_;179 std::shared_ptr<WorkerWorkerOCService_BrpcGenericStub> dataStub_;
181- std::shared_ptr<master::MasterOCService_BrpcGenericStub> masterStub_;
182 std::atomic<bool> alive_{ false };180 std::atomic<bool> alive_{ false };
183 uint32_t connectionGeneration_ = 0;181 uint32_t connectionGeneration_ = 0;
184 static std::atomic<uint32_t> nextConnectionGeneration_;182 static std::atomic<uint32_t> nextConnectionGeneration_;
Msrc/datasystem/common/log/access_point.def+1-0
@@ -51,6 +51,7 @@ ACCESS_RECORDER_KEY_DEF(DS_POSIX_PUBLISH, ACCESS)
51ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTI_CREATE, ACCESS)51ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTI_CREATE, ACCESS)
52ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTIPUBLISH, ACCESS)52ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTIPUBLISH, ACCESS)
53ACCESS_RECORDER_KEY_DEF(DS_POSIX_GET, ACCESS)53ACCESS_RECORDER_KEY_DEF(DS_POSIX_GET, ACCESS)
54+ACCESS_RECORDER_KEY_DEF(DS_POSIX_QUERY_AND_GET, ACCESS)
54ACCESS_RECORDER_KEY_DEF(DS_POSIX_DELETE_ALL_COPY, ACCESS)55ACCESS_RECORDER_KEY_DEF(DS_POSIX_DELETE_ALL_COPY, ACCESS)
55ACCESS_RECORDER_KEY_DEF(DS_POSIX_GINCREASEREF, ACCESS)56ACCESS_RECORDER_KEY_DEF(DS_POSIX_GINCREASEREF, ACCESS)
56ACCESS_RECORDER_KEY_DEF(DS_POSIX_GDECREASEREF, ACCESS)57ACCESS_RECORDER_KEY_DEF(DS_POSIX_GDECREASEREF, ACCESS)
Msrc/datasystem/master/object_cache/master_oc_service_impl.cpp+0-30
@@ -486,36 +486,6 @@ Status MasterOCServiceImpl::GetObjectLocations(const GetObjectLocationsReqPb &re
486 return Status::OK();486 return Status::OK();
487}487}
488 488 
489-Status MasterOCServiceImpl::QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &resp,
490- std::vector<RpcMessage> &payloads)
491-{
492- ScopedRequestContext ctx;
493- Timer timer;
494- auto config = GetServerLatencyTraceConfig();
495- RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(req), "AK/SK failed.");
496- VLOG(1) << FormatString("[Master] Processing QueryAndGetReq, target num: %d, redirect: %d", req.object_keys_size(),
497- req.redirect());
498- std::shared_ptr<master::OCMetadataManager> ocMetadataManager;
499- RETURN_IF_NOT_OK_PRINT_ERROR_MSG(metadataManagerHolder_->GetOcMetadataManager(ocMetadataManager),
500- "GetOcMetadataManager failed");
501- Status status = ocMetadataManager->QueryAndGet(req, resp, payloads);
502- const auto totalUs = static_cast<uint64_t>(timer.ElapsedMicroSecond());
503- const double totalMs = static_cast<double>(totalUs) / US_PER_MS;
504- GetMasterTimeCost().Append("Total QueryAndGet", totalMs);
505- if (status.IsError()) {
506- LOG(ERROR) << FormatString("[Master] QueryAndGet failed, target num: %d, cost: %.3fms, status: %s",
507- req.object_keys_size(), totalMs, status.ToString());
508- return status;
509- }
510- SLOW_LOG_IF_OR_VLOG(
511- INFO, config.processSlowerThanUs > 0 && totalUs >= config.processSlowerThanUs, 1,
512- FormatString("[Master] QueryAndGet done, target num: %d, result num: %d, payload num: %zu, meta moving: %d, "
513- "cost: %.3fms, %s",
514- req.object_keys_size(), resp.results_size(), payloads.size(), resp.meta_is_moving(), totalMs,
515- GetMasterTimeCost().GetInfo()));
516- return status;
517-}
518- 
519Status MasterOCServiceImpl::DeleteAllCopyMeta(489Status MasterOCServiceImpl::DeleteAllCopyMeta(
520 std::shared_ptr<ServerUnaryWriterReader<DeleteAllCopyMetaRspPb, DeleteAllCopyMetaReqPb>> serverApi)490 std::shared_ptr<ServerUnaryWriterReader<DeleteAllCopyMetaRspPb, DeleteAllCopyMetaReqPb>> serverApi)
521{491{
Msrc/datasystem/master/object_cache/master_oc_service_impl.h+0-10
@@ -143,16 +143,6 @@ public:
143 */143 */
144 Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &resp) override;144 Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &resp) override;
145 145 
146- /**
147- * @brief Query object locations for client direct reads.
148- * @param[in] req The query request.
149- * @param[out] resp The object locations and redirect state.
150- * @param[out] payloads Optional object data returned with the metadata.
151- * @return K_OK on success; the error code otherwise.
152- */
153- Status QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &resp,
154- std::vector<RpcMessage> &payloads) override;
155- 
156 /**146 /**
157 * @brief Delete metadata and notify other workers to delete these objects.147 * @brief Delete metadata and notify other workers to delete these objects.
158 * @param[in] serverApi The ServerUnaryWriterReader object.148 * @param[in] serverApi The ServerUnaryWriterReader object.
Msrc/datasystem/master/object_cache/oc_metadata_manager.cpp+0-151
@@ -99,7 +99,6 @@ static constexpr int DEBUG_LOG_LEVEL = 1;
99static constexpr int MIN_TTL_SECOND = 0;99static constexpr int MIN_TTL_SECOND = 0;
100static constexpr int ASYNC_MIN_THREAD_NUM = 2;100static constexpr int ASYNC_MIN_THREAD_NUM = 2;
101static constexpr int ASYNC_MAX_THREAD_NUM = 5;101static constexpr int ASYNC_MAX_THREAD_NUM = 5;
102-static constexpr int QUERY_AND_GET_MAX_COPY_NUM = 5;
103static constexpr uint64_t QUERY_AND_GET_MAX_PAYLOAD_SIZE = 512 * 1024UL;102static constexpr uint64_t QUERY_AND_GET_MAX_PAYLOAD_SIZE = 512 * 1024UL;
104static const std::string OC_METADATA_MANAGER = "OCMetadataManager-";103static const std::string OC_METADATA_MANAGER = "OCMetadataManager-";
105static constexpr int MSET_PENDING_TTL_US = 60'000'000; // 60s104static constexpr int MSET_PENDING_TTL_US = 60'000'000; // 60s
@@ -2728,73 +2727,6 @@ Status OCMetadataManager::GetObjectLocations(const GetObjectLocationsReqPb &req,
2728 return Status::OK();2727 return Status::OK();
2729}2728}
2730 2729 
2731-Status OCMetadataManager::QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &rsp,
2732- std::vector<RpcMessage> &payloads)
2733-{
2734- INJECT_POINT("client.transport.query_and_get", []() { return Status::OK(); });
2735- RETURN_IF_NOT_OK(ValidateQueryAndGetDataRequest(req));
2736- std::vector<std::string> objectKeys = { req.object_keys().begin(), req.object_keys().end() };
2737- RETURN_IF_NOT_OK(FillObjectRedirectResponses(rsp, objectKeys, req.redirect()));
2738- RETURN_OK_IF_TRUE(rsp.meta_is_moving());
2739- 
2740- std::unordered_map<std::string, bool> redirectedKeys;
2741- for (const auto &redirectInfo : rsp.info()) {
2742- for (const auto &objectKey : redirectInfo.change_meta_ids()) {
2743- redirectedKeys.emplace(objectKey, true);
2744- }
2745- }
2746- 
2747- uint64_t payloadSize = 0;
2748- for (int i = 0; i < req.object_keys_size(); ++i) {
2749- const auto &objectKey = req.object_keys(i);
2750- if (redirectedKeys.find(objectKey) != redirectedKeys.end()) {
2751- continue;
2752- }
2753- 
2754- QueryAndGetResultPb *result = rsp.add_results();
2755- ObjectLocationInfoPb *location = result->mutable_location();
2756- location->set_object_key(objectKey);
2757- QueryAndGetMetaSnapshot meta;
2758- if (!FillQueryAndGetMetadata(objectKey, *location, meta)) {
2759- continue;
2760- }
2761- if (req.has_data_request()) {
2762- TryGetQueryAndGetData(req, static_cast<size_t>(i), objectKey, meta, *result, payloadSize, payloads);
2763- }
2764- }
2765- return Status::OK();
2766-}
2767- 
2768-bool OCMetadataManager::FillQueryAndGetMetadata(const std::string &objectKey, ObjectLocationInfoPb &location,
2769- QueryAndGetMetaSnapshot &meta)
2770-{
2771- TbbMetaTable::const_accessor accessor;
2772- const size_t shardIdx = GetShardIndex(objectKey);
2773- if (!metaShards_[shardIdx].table.find(accessor, objectKey)) {
2774- return false;
2775- }
2776- 
2777- const auto &objectMeta = accessor->second;
2778- const auto &primaryAddress = objectMeta.meta.primary_address();
2779- if (!primaryAddress.empty()) {
2780- location.add_object_locations(primaryAddress);
2781- }
2782- for (const auto &address : objectMeta.locations) {
2783- if (location.object_locations_size() >= QUERY_AND_GET_MAX_COPY_NUM) {
2784- break;
2785- }
2786- if (address.first != primaryAddress) {
2787- location.add_object_locations(address.first);
2788- }
2789- }
2790- meta.dataSize = objectMeta.meta.data_size();
2791- meta.version = objectMeta.meta.version();
2792- auto localCopy = objectMeta.locations.find(masterAddress_);
2793- meta.localCopyAvailable = localCopy != objectMeta.locations.end() && localCopy->second == AckState::ACK;
2794- location.set_object_size(meta.dataSize);
2795- return true;
2796-}
2797- 
2798void OCMetadataManager::GetObjRefsMatch(const std::function<bool(const std::string &)> &matchFunc,2730void OCMetadataManager::GetObjRefsMatch(const std::function<bool(const std::string &)> &matchFunc,
2799 std::unordered_set<std::string> &objKeys)2731 std::unordered_set<std::string> &objKeys)
2800{2732{
@@ -4724,89 +4656,6 @@ void OCMetadataManager::WaitInitializaiton()
4724 }4656 }
4725}4657}
4726 4658 
4727-Status OCMetadataManager::ValidateQueryAndGetDataRequest(const QueryAndGetReqPb &req) const
4728-{
4729- if (!req.has_data_request()) {
4730- return Status::OK();
4731- }
4732- const auto &dataRequest = req.data_request();
4733- CHECK_FAIL_RETURN_STATUS(dataRequest.has_tcp() || dataRequest.has_ub(), K_INVALID,
4734- "QueryAndGet data transport is not set");
4735- if (dataRequest.has_ub()) {
4736- const auto &ubRequest = dataRequest.ub();
4737- CHECK_FAIL_RETURN_STATUS(ubRequest.buffer_size() > 0 && !ubRequest.urma_instance_id().empty(), K_INVALID,
4738- "QueryAndGet UB buffer is invalid");
4739- CHECK_FAIL_RETURN_STATUS(ubRequest.buffer_infos_size() == req.object_keys_size(), K_INVALID,
4740- "QueryAndGet UB buffer count does not match object key count");
4741- }
4742- return Status::OK();
4743-}
4744- 
4745-void OCMetadataManager::TryGetQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex,
4746- const std::string &objectKey,
4747- const QueryAndGetMetaSnapshot &meta,
4748- QueryAndGetResultPb &result, uint64_t &payloadSize,
4749- std::vector<RpcMessage> &payloads)
4750-{
4751- if (localApi_ == nullptr || !meta.localCopyAvailable
4752- || notifyWorkerManager_->CheckExistAsyncWorkerOp(
4753- masterAddress_, objectKey,
4754- NotifyWorkerOpType::CACHE_INVALID | NotifyWorkerOpType::PRIMARY_COPY_INVALID)) {
4755- return;
4756- }
4757- 
4758- const bool useUb = req.data_request().has_ub();
4759- if (useUb && meta.dataSize > req.data_request().ub().buffer_size()) {
4760- return;
4761- }
4762- if (!useUb
4763- && (payloadSize > QUERY_AND_GET_MAX_PAYLOAD_SIZE
4764- || meta.dataSize > QUERY_AND_GET_MAX_PAYLOAD_SIZE - payloadSize)) {
4765- return;
4766- }
4767- 
4768- std::vector<RpcMessage> tmpPayloads;
4769- Status status = ReadLocalQueryAndGetData(req, requestIndex, objectKey, meta, tmpPayloads);
4770- if (status.IsError()) {
4771- VLOG(1) << FormatString("[ObjectKey %s] QueryAndGet local data miss: %s", objectKey, status.ToString());
4772- return;
4773- }
4774- 
4775- if (useUb) {
4776- result.mutable_data_result();
4777- return;
4778- }
4779- auto *dataResult = result.mutable_data_result();
4780- for (auto &payload : tmpPayloads) {
4781- dataResult->add_payload_indexes(static_cast<uint32_t>(payloads.size()));
4782- payloads.emplace_back(std::move(payload));
4783- }
4784- payloadSize += meta.dataSize;
4785-}
4786- 
4787-Status OCMetadataManager::ReadLocalQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex,
4788- const std::string &objectKey,
4789- const QueryAndGetMetaSnapshot &meta,
4790- std::vector<RpcMessage> &payloads)
4791-{
4792- GetObjectRemoteReqPb localReq;
4793- localReq.set_try_lock(true);
4794- localReq.set_object_key(objectKey);
4795- localReq.set_version(meta.version);
4796- localReq.set_read_offset(0);
4797- localReq.set_read_size(meta.dataSize);
4798- localReq.set_data_size(meta.dataSize);
4799- if (req.data_request().has_ub()) {
4800- const auto &ubRequest = req.data_request().ub();
4801- *localReq.mutable_urma_info() = ubRequest.buffer_infos(static_cast<int>(requestIndex));
4802- localReq.set_urma_instance_id(ubRequest.urma_instance_id());
4803- }
4804- 
4805- RETURN_RUNTIME_ERROR_IF_NULL(localApi_);
4806- RETURN_IF_NOT_OK(akSkManager_->GenerateSignature(localReq));
4807- return localApi_->GetObjectRemoteForQueryAndGet(localReq, payloads);
4808-}
4809- 
4810void OCMetadataManager::TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor,4659void OCMetadataManager::TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor,
4811 uint64_t &payloadSize, QueryMetaInfoPb &queryMeta,4660 uint64_t &payloadSize, QueryMetaInfoPb &queryMeta,
4812 std::vector<RpcMessage> &payloads)4661 std::vector<RpcMessage> &payloads)
Msrc/datasystem/master/object_cache/oc_metadata_manager.h+0-60
@@ -556,16 +556,6 @@ public:
556 */556 */
557 Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &rsp);557 Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &rsp);
558 558 
559- /**
560- * @brief Query object locations for client direct reads.
561- * @param[in] req The query request.
562- * @param[out] rsp The object locations and redirect state.
563- * @param[out] payloads Optional object data returned with the metadata.
564- * @return K_OK on success; the error code otherwise.
565- */
566- Status QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &rsp,
567- std::vector<RpcMessage> &payloads);
568- 
569 /**559 /**
570 * @brief Delete metadata and notify other workers to delete these objects synchronously.560 * @brief Delete metadata and notify other workers to delete these objects synchronously.
571 * @param[in] req The rpc request protobuf.561 * @param[in] req The rpc request protobuf.
@@ -1538,12 +1528,6 @@ private:
1538 void ReplacePrimaryObject(const ReplacePrimaryReqPb &req,1528 void ReplacePrimaryObject(const ReplacePrimaryReqPb &req,
1539 const ReplacePrimaryReqPb::ObjectInfoPb &info, ReplacePrimaryRspPb &rsp);1529 const ReplacePrimaryReqPb::ObjectInfoPb &info, ReplacePrimaryRspPb &rsp);
1540 1530 
1541- struct QueryAndGetMetaSnapshot {
1542- uint64_t dataSize = 0;
1543- int64_t version = 0;
1544- bool localCopyAvailable = false;
1545- };
1546- 
1547 friend class MasterOCServiceImpl;1531 friend class MasterOCServiceImpl;
1548 friend class OCNotifyWorkerManager;1532 friend class OCNotifyWorkerManager;
1549 friend class OCGlobalCacheDeleteManager;1533 friend class OCGlobalCacheDeleteManager;
@@ -2076,50 +2060,6 @@ private:
2076 void TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor, uint64_t &payloadSize,2060 void TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor, uint64_t &payloadSize,
2077 QueryMetaInfoPb &queryMeta, std::vector<RpcMessage> &payloads);2061 QueryMetaInfoPb &queryMeta, std::vector<RpcMessage> &payloads);
2078 2062 
2079- /**
2080- * @brief Validate the optional QueryAndGet inline-data request.
2081- * @param[in] req QueryAndGet request.
2082- * @return K_OK when the transport request is valid; the error code otherwise.
2083- */
2084- Status ValidateQueryAndGetDataRequest(const QueryAndGetReqPb &req) const;
2085- 
2086- /**
2087- * @brief Read one local object through the transport selected by QueryAndGet.
2088- * @param[in] req QueryAndGet request.
2089- * @param[in] requestIndex Object position in the original request.
2090- * @param[in] objectKey Object key.
2091- * @param[in] meta Metadata snapshot used to construct the local read.
2092- * @param[out] payloads Optional TCP payloads.
2093- * @return K_OK on success; the error code otherwise.
2094- */
2095- Status ReadLocalQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex,
2096- const std::string &objectKey, const QueryAndGetMetaSnapshot &meta,
2097- std::vector<RpcMessage> &payloads);
2098- 
2099- /**
2100- * @brief Fill object locations and snapshot fields while holding the metadata accessor.
2101- * @param[in] objectKey Object key.
2102- * @param[out] location Object locations and size.
2103- * @param[out] meta Metadata required after releasing the accessor.
2104- * @return True when the object metadata exists.
2105- */
2106- bool FillQueryAndGetMetadata(const std::string &objectKey, ObjectLocationInfoPb &location,
2107- QueryAndGetMetaSnapshot &meta);
2108- 
2109- /**
2110- * @brief Try to return one QueryAndGet object's data through the selected transport.
2111- * @param[in] req QueryAndGet request.
2112- * @param[in] requestIndex Object position in the original request.
2113- * @param[in] objectKey Object key.
2114- * @param[in] meta Metadata snapshot captured before releasing the table accessor.
2115- * @param[out] result Per-object QueryAndGet result.
2116- * @param[in,out] payloadSize Accumulated TCP payload bytes.
2117- * @param[out] payloads Response payloads.
2118- */
2119- void TryGetQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex, const std::string &objectKey,
2120- const QueryAndGetMetaSnapshot &meta, QueryAndGetResultPb &result,
2121- uint64_t &payloadSize, std::vector<RpcMessage> &payloads);
2122- 
2123 /**2063 /**
2124 * @brief Add heavy operation.2064 * @brief Add heavy operation.
2125 * @param[in] objectKey Object key.2065 * @param[in] objectKey Object key.
Msrc/datasystem/protos/master_object.proto+0-45
@@ -613,47 +613,6 @@ message GetObjectLocationsRspPb {
613 bool meta_is_moving = 3;613 bool meta_is_moving = 3;
614}614}
615 615 
616-message QueryAndGetTcpDataReqPb {}
617- 
618-message QueryAndGetUbDataReqPb {
619- repeated datasystem.UrmaRemoteAddrPb buffer_infos = 1;
620- uint64 buffer_size = 2;
621- string urma_instance_id = 3;
622-}
623- 
624-message QueryAndGetDataReqPb {
625- oneof transport {
626- QueryAndGetTcpDataReqPb tcp = 1;
627- QueryAndGetUbDataReqPb ub = 2;
628- }
629-}
630- 
631-message QueryAndGetReqPb {
632- repeated string object_keys = 1;
633- bool redirect = 2;
634- QueryAndGetDataReqPb data_request = 3;
635- 
636- // put to the end, the previous data is used to generate AK and SK signatures.
637- uint64 timestamp = 100;
638- string signature = 101;
639- string access_key = 102;
640-}
641- 
642-message QueryAndGetDataResultPb {
643- repeated uint32 payload_indexes = 1;
644-}
645- 
646-message QueryAndGetResultPb {
647- ObjectLocationInfoPb location = 1;
648- QueryAndGetDataResultPb data_result = 2;
649-}
650- 
651-message QueryAndGetRspPb {
652- repeated QueryAndGetResultPb results = 1;
653- repeated RedirectMetaInfo info = 2;
654- bool meta_is_moving = 3;
655-}
656- 
657message ReplacePrimaryReqPb {616message ReplacePrimaryReqPb {
658 message ObjectInfoPb {617 message ObjectInfoPb {
659 string object_key = 1;618 string object_key = 1;
@@ -839,10 +798,6 @@ service MasterOCService {
839 }798 }
840 rpc UpdateMeta(UpdateMetaReqPb) returns (UpdateMetaRspPb) {}799 rpc UpdateMeta(UpdateMetaReqPb) returns (UpdateMetaRspPb) {}
841 rpc GetObjectLocations(GetObjectLocationsReqPb) returns (GetObjectLocationsRspPb) {}800 rpc GetObjectLocations(GetObjectLocationsReqPb) returns (GetObjectLocationsRspPb) {}
842- rpc QueryAndGet(QueryAndGetReqPb) returns (QueryAndGetRspPb) {
843- option (datasystem.recv_payload_option) = true;
844- option (datasystem.urma_enabled_option) = true;
845- }
846 rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {}801 rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {}
O
OOuGongChang6 天前

[P1] 删除 MasterOCService.QueryAndGet 后无混版本兼容防护,滚动升级窗口内直读路径整体不可用

本 PR 删除了 MasterOCService.QueryAndGet 并把实现下沉到 WorkerOCService.QueryAndGet,但仓库内没有任何版本协商/能力检测机制,客户端也没有对"方法不存在"的快速失败或回退路径:

  • 旧客户端(routed masterStub 实际打到 meta owner worker 上内嵌的 MasterOCServiceImpl,见 worker_oc_server.cpp:933/1037)→ 新 worker:brpc 返回 ENOMETHOD(2004),brpc_status_util.h 未覆盖该错误码落入 default → K_RPC_CANCELLED;而 IsRetryableRpcError(K_RPC_CANCELLED)=true(rpc_util.h:48),客户端反复重试直至 API deadline,每次 Get 以超时类错误收场,极易误判为网络问题。
  • 新客户端 → 旧 worker:同样 ENOMETHOD → 重试至 deadline。object_metadata_client.cpp 的 QueryWithRetry/PrepareQueryRetry 只处理路由类错误,没有降级到 QueryMeta+副本读的路径(按 key 缺失 data_result 的降级只在 RPC 正常返回时生效)。

影响面:enableLocalCache=false 的直读路径在 client 与 worker/master 任一方向版本不匹配期间全部失败。PR body 声称"同版本升级约束已说明",但目前仅文档层面,代码无运行时保护。

建议至少二选一:1) 升级流程强制 client+server 严格同版本原子升级(发布说明明确);2) 对"方法不存在"类错误快速失败并给出明确提示,或对该场景回退 QueryMeta + ReplicaReader 旧链路保持兼容。

likedislike
yangxiaogang14
5 天前 评论:
847 rpc GDecreaseRef(GDecreaseReqPb) returns (GDecreaseRspPb) {802 rpc GDecreaseRef(GDecreaseReqPb) returns (GDecreaseRspPb) {
848 option (datasystem.unary_socket_option) = true;803 option (datasystem.unary_socket_option) = true;
Msrc/datasystem/protos/object_posix.proto+68-0
@@ -644,6 +644,69 @@ message GetHashRingRspPb {
644 map<string, string> host_id_map = 5; // workerAddr("host:port") -> hostId, valid when hash_ring_changed=true.644 map<string, string> host_id_map = 5; // workerAddr("host:port") -> hostId, valid when hash_ring_changed=true.
645}645}
646 646 
647+message QueryAndGetTcpDataReqPb {}
648+ 
649+message QueryAndGetUbDataReqPb {
650+ repeated UrmaRemoteAddrPb buffer_infos = 1;
651+ uint64 buffer_size = 2;
652+ string urma_instance_id = 3;
653+}
654+ 
655+message QueryAndGetShmDataReqPb {
656+ string client_id = 1;
657+}
658+ 
659+message QueryAndGetDataReqPb {
660+ oneof transport {
661+ QueryAndGetTcpDataReqPb tcp = 1;
662+ QueryAndGetUbDataReqPb ub = 2;
663+ QueryAndGetShmDataReqPb shm = 3;
664+ }
665+}
666+ 
667+message QueryAndGetReqPb {
668+ repeated string object_keys = 1;
669+ QueryAndGetDataReqPb data_request = 2;
670+ 
671+ // put to the end, the previous data is used to generate AK and SK signatures.
672+ uint64 timestamp = 100;
673+ string signature = 101;
674+ string access_key = 102;
675+}
676+ 
677+message QueryAndGetLocationInfoPb {
678+ string object_key = 1;
679+ repeated string object_locations = 2;
680+ uint64 object_size = 3;
681+}
682+ 
683+message QueryAndGetShmInfoPb {
684+ int64 store_fd = 1;
685+ int64 offset = 2;
686+ int64 data_size = 3;
687+ int64 metadata_size = 4;
688+ int64 mmap_size = 5;
689+ string shm_id = 6;
690+ bool is_seal = 7;
691+ uint32 write_mode = 8;
692+ uint32 consistency_type = 9;
693+ uint32 cache_type = 10;
694+}
695+ 
696+message QueryAndGetDataResultPb {
697+ repeated uint32 payload_indexes = 1;
698+ QueryAndGetShmInfoPb shm_info = 2;
699+}
700+ 
701+message QueryAndGetResultPb {
702+ QueryAndGetLocationInfoPb location = 1;
703+ QueryAndGetDataResultPb data_result = 2;
704+}
705+ 
706+message QueryAndGetRspPb {
707+ repeated QueryAndGetResultPb results = 1;
708+}
709+ 
647service WorkerOCService {710service WorkerOCService {
648 rpc HealthCheck (HealthCheckRequestPb) returns (HealthCheckReplyPb) {}711 rpc HealthCheck (HealthCheckRequestPb) returns (HealthCheckReplyPb) {}
649 rpc Create (CreateReqPb) returns (CreateRspPb) {}712 rpc Create (CreateReqPb) returns (CreateRspPb) {}
@@ -654,6 +717,11 @@ service WorkerOCService {
654 option (datasystem.unary_socket_option) = true;717 option (datasystem.unary_socket_option) = true;
655 option (datasystem.recv_payload_option) = true;718 option (datasystem.recv_payload_option) = true;
656 }719 }
720+ rpc QueryAndGet(QueryAndGetReqPb) returns (QueryAndGetRspPb) {
cheyulin
cheyulincheyulin5 天前

[严重][兼容性] [P1] 将新 RPC 追加到 service 末尾以保持 ZMQ 方法序号

证据:DataSystem 的 ZMQ 生成器直接用 protobuf service 中的方法下标生成 switch(meta.method_index())。基线 WorkerOCService 的既有方法下标为 0..40,但这里把 QueryAndGet 插在原 index 4 的 DecreaseReference 之前,使 DecreaseReferenceGetHashRing 的 37 个既有 RPC 全部后移一位。当前 PR 也没有协议序号不变性测试。

问题:bRPC 按方法名分发不会暴露这个问题,但 ZMQ wire protocol 使用 ordinal。新旧进程混部时,旧端发送的 index 4 会被新端当成 QueryAndGet,新端发送的 index 5 会被旧端当成下一个既有方法;影响不只限于新 RPC 是否存在。

影响:滚动升级期间,引用释放、GRef、序列号、设备对象、P2P、MultiCreate、Exist 等大量无关 Worker RPC 都可能被分发到错误 handler,造成请求解析失败、引用无法释放或业务错误,故障面远大于 QueryAndGet 本身。

建议:把 QueryAndGet 移到 WorkerOCService 最后、即 GetHashRing 之后,保证所有既有方法 ordinal 不变。增加 descriptor 契约测试,逐项断言 41 个 legacy method 的 index 未变化并断言新方法为最后一个;同时覆盖新旧 ZMQ Client/Worker 的代表性混版调用。

验证:生成 ZMQ stub/service 后检查 QueryAndGet 为 index 41,并用 old-client/new-worker、new-client/old-worker 覆盖 DecreaseReferenceGetHashRing 和新 QueryAndGet,确认既有 RPC 不会错分发。

likedislike
721+ option (datasystem.unary_socket_option) = true;
722+ option (datasystem.recv_payload_option) = true;
723+ option (datasystem.urma_enabled_option) = true;
724+ }
657 rpc DecreaseReference(DecreaseReferenceRequest) returns (DecreaseReferenceResponse) {}725 rpc DecreaseReference(DecreaseReferenceRequest) returns (DecreaseReferenceResponse) {}
658 rpc ReconcileShmRef(ReconcileShmRefReqPb) returns (ReconcileShmRefRspPb) {}726 rpc ReconcileShmRef(ReconcileShmRefReqPb) returns (ReconcileShmRefRspPb) {}
659 rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {}727 rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {}
Msrc/datasystem/worker/object_cache/BUILD.bazel+1-0
@@ -339,6 +339,7 @@ ds_cc_library(
339 "//src/datasystem/worker/object_cache/service:worker_oc_service_delete_impl_header",339 "//src/datasystem/worker/object_cache/service:worker_oc_service_delete_impl_header",
340 "//src/datasystem/worker/object_cache/service:worker_oc_service_expire_impl",340 "//src/datasystem/worker/object_cache/service:worker_oc_service_expire_impl",
341 "//src/datasystem/worker/object_cache/service:worker_oc_service_global_reference_impl",341 "//src/datasystem/worker/object_cache/service:worker_oc_service_global_reference_impl",
342+ "//src/datasystem/worker/object_cache/service:worker_oc_service_get_impl",
342 "//src/datasystem/worker/object_cache/service:worker_oc_service_migrate_impl",343 "//src/datasystem/worker/object_cache/service:worker_oc_service_migrate_impl",
343 "//src/datasystem/worker/object_cache/service:worker_oc_service_multi_publish_impl_header",344 "//src/datasystem/worker/object_cache/service:worker_oc_service_multi_publish_impl_header",
344 "//src/datasystem/worker/object_cache/service:worker_oc_service_publish_impl_header",345 "//src/datasystem/worker/object_cache/service:worker_oc_service_publish_impl_header",
Msrc/datasystem/worker/object_cache/CMakeLists.txt+2-0
@@ -45,6 +45,7 @@ set(WORKER_OC_SRCS
45 service/worker_oc_service_publish_impl.cpp45 service/worker_oc_service_publish_impl.cpp
46 service/worker_oc_service_multi_publish_impl.cpp46 service/worker_oc_service_multi_publish_impl.cpp
47 service/worker_oc_service_get_impl.cpp47 service/worker_oc_service_get_impl.cpp
48+ service/worker_query_and_get_impl.cpp
48 service/worker_oc_service_delete_impl.cpp49 service/worker_oc_service_delete_impl.cpp
49 service/worker_oc_service_clear_data_flow.cpp50 service/worker_oc_service_clear_data_flow.cpp
50 service/worker_oc_service_global_reference_impl.cpp51 service/worker_oc_service_global_reference_impl.cpp
@@ -116,6 +117,7 @@ target_link_libraries(worker_object_cache PRIVATE ${WORKER_OBJECT_CACHE_DEPEND_L
116set(WORKER_OC_HEAVY_FILES117set(WORKER_OC_HEAVY_FILES
117 worker_oc_service_impl.cpp118 worker_oc_service_impl.cpp
118 service/worker_oc_service_get_impl.cpp119 service/worker_oc_service_get_impl.cpp
120+ service/worker_query_and_get_impl.cpp
119 service/worker_oc_service_migrate_impl.cpp121 service/worker_oc_service_migrate_impl.cpp
120 worker_oc_eviction_manager.cpp122 worker_oc_eviction_manager.cpp
121)123)
Msrc/datasystem/worker/object_cache/service/BUILD.bazel+4-0
@@ -88,9 +88,11 @@ ds_cc_library(
88 srcs = [88 srcs = [
89 "worker_oc_service_batch_get_impl.cpp",89 "worker_oc_service_batch_get_impl.cpp",
90 "worker_oc_service_get_impl.cpp",90 "worker_oc_service_get_impl.cpp",
91+ "worker_query_and_get_impl.cpp",
91 ],92 ],
92 hdrs = [93 hdrs = [
93 "worker_oc_service_get_impl.h",94 "worker_oc_service_get_impl.h",
95+ "worker_query_and_get_impl.h",
94 ],96 ],
95 deps = [97 deps = [
96 ":worker_oc_service_get_impl_header",98 ":worker_oc_service_get_impl_header",
@@ -104,6 +106,8 @@ ds_cc_library(
104 "//src/datasystem/common/l2cache:l2_storage",106 "//src/datasystem/common/l2cache:l2_storage",
105 "//src/datasystem/common/log:common_log",107 "//src/datasystem/common/log:common_log",
106 "//src/datasystem/common/metrics:common_metrics",108 "//src/datasystem/common/metrics:common_metrics",
109+ "//src/datasystem/common/object_cache:object_ref_info",
110+ "//src/datasystem/common/object_cache:shm_guard",
107 "//src/datasystem/common/parallel:common_parallel",111 "//src/datasystem/common/parallel:common_parallel",
108 "//src/datasystem/common/perf:common_perf",112 "//src/datasystem/common/perf:common_perf",
109 "//src/datasystem/common/rdma:fast_transport_manager_wrapper",113 "//src/datasystem/common/rdma:fast_transport_manager_wrapper",
Msrc/datasystem/worker/object_cache/service/worker_oc_service_get_impl.cpp+106-0
@@ -874,6 +874,22 @@ Status WorkerOcServiceGetImpl::RLockGetObjectFromMem(const ReadKey &readKey, Get
874 return Status::OK();874 return Status::OK();
875}875}
876 876 
877+Status WorkerOcServiceGetImpl::TryAcquireLocalObject(const std::string &objectKey,
878+ std::unique_ptr<GetObjEntryParams> &params)
879+{
880+ params.reset();
881+ GetObjInfo info;
882+ std::set<ReadKey> misses;
883+ bool validInMemory = true;
884+ Status rc = RLockGetObjectFromMem(ReadKey(objectKey), info, misses, validInMemory);
885+ if (rc.GetCode() == K_NOT_FOUND || !validInMemory || !misses.empty()) {
886+ return Status::OK();
887+ }
888+ RETURN_IF_NOT_OK(rc);
889+ params = std::move(info.params);
890+ return Status::OK();
891+}
892+ 
877Status WorkerOcServiceGetImpl::ProcessObjectsNotExistInLocal(const std::set<ReadKey> &objectsNeedGetRemote,893Status WorkerOcServiceGetImpl::ProcessObjectsNotExistInLocal(const std::set<ReadKey> &objectsNeedGetRemote,
878 int64_t subTimeout,894 int64_t subTimeout,
879 std::unordered_set<std::string> &failedIds,895 std::unordered_set<std::string> &failedIds,
@@ -2996,6 +3012,96 @@ Status WorkerOcServiceGetImpl::GetMapOfObjectKeys(const std::vector<std::basic_s
2996 return Status::OK();3012 return Status::OK();
2997}3013}
2998 3014 
3015+Status WorkerOcServiceGetImpl::QueryObjectLocations(
3016+ const std::vector<std::string> &objectKeys,
3017+ std::unordered_map<std::string, master::ObjectLocationInfoPb> &locations)
3018+{
3019+ locations.clear();
3020+ if (objectKeys.empty()) {
3021+ return Status::OK();
3022+ }
3023+ CHECK_FAIL_RETURN_STATUS(metadataRouteResolver_ != nullptr, K_NOT_READY, "Metadata route resolver is unavailable");
3024+ auto grouped = metadataRouteResolver_->GroupOwners(objectKeys);
3025+ if (!grouped.failures.empty()) {
3026+ return grouped.failures.begin()->second;
3027+ }
3028+ std::vector<master::QueryMetaInfoPb> queryMetas;
3029+ queryMetas.reserve(objectKeys.size());
3030+ for (const auto &[masterAddress, keys] : grouped.groups) {
3031+ RETURN_IF_NOT_OK(QueryPureMetadataGroup(masterAddress, keys, queryMetas));
3032+ }
3033+ locations.reserve(objectKeys.size());
3034+ const std::unordered_set<std::string> requestedKeys(objectKeys.begin(), objectKeys.end());
3035+ for (const auto &queryMeta : queryMetas) {
3036+ const auto &meta = queryMeta.meta();
3037+ CHECK_FAIL_RETURN_STATUS(requestedKeys.find(meta.object_key()) != requestedKeys.end(), K_RUNTIME_ERROR,
3038+ "PureQueryMeta returned an unexpected object key");
3039+ master::ObjectLocationInfoPb location;
3040+ location.set_object_key(meta.object_key());
3041+ location.set_object_size(meta.data_size());
3042+ if (!meta.primary_address().empty()) {
3043+ location.add_object_locations(meta.primary_address());
3044+ }
3045+ if (!queryMeta.address().empty() && queryMeta.address() != meta.primary_address()) {
cheyulin
cheyulincheyulin6 天前

[警告][正确性] [P1] miss 响应丢失第三个及后续健康副本

证据:新实现只加入 primary_address 和 PureQueryMeta 通过 SelectObjectLocation 选出的一个 address。基线 FillQueryAndGetMetadata 会按 primary-first 返回最多 QUERY_AND_GET_MAX_COPY_NUM=5 个 location,而 ReplicaReader 会顺序遍历响应中的全部候选。

问题:三副本以上时,其余 ACK 副本不会进入响应,客户端因此无法发现并尝试它们。

影响:primary 与本次选中的 secondary 都不可读时,即使第三副本健康,请求也会失败、重试或超时,造成副本容灾能力和尾延迟相对基线退化。

建议:让 Worker 的 metadata 查询返回与旧协议等价的、primary-first 且最多 5 个的有效 ACK location 集合,并保留异步失效过滤,不要从单选 QueryMetaInfoPb.address 反推完整副本集合。

验证:构造至少三副本对象,让 primary 与首选 secondary 分别返回 peer-dead/not-found,验证单 key 与 batch 均能从第三个健康副本成功读取。

likedislike
yangxiaogang14
5 天前 评论:
3046+ location.add_object_locations(queryMeta.address());
3047+ }
3048+ locations[meta.object_key()] = std::move(location);
3049+ }
3050+ return Status::OK();
3051+}
3052+ 
3053+Status WorkerOcServiceGetImpl::QueryPureMetadataGroup(const HostPort &masterAddress,
3054+ const std::vector<std::string> &objectKeys,
3055+ std::vector<master::QueryMetaInfoPb> &queryMetas)
3056+{
3057+ auto workerMasterApi = workerMasterApiManager_->GetWorkerMasterApi(masterAddress);
3058+ CHECK_FAIL_RETURN_STATUS(workerMasterApi != nullptr, K_RUNTIME_ERROR,
3059+ "Get master API failed for PureQueryMeta");
3060+ master::PureQueryMetaReqPb request;
3061+ request.set_redirect(true);
3062+ request.set_address(localAddress_.ToString());
3063+ request.mutable_object_keys()->Add(objectKeys.begin(), objectKeys.end());
3064+ master::PureQueryMetaRspPb response;
3065+ std::function<Status(master::PureQueryMetaReqPb &, master::PureQueryMetaRspPb &)> query =
3066+ [this, &workerMasterApi](master::PureQueryMetaReqPb &req, master::PureQueryMetaRspPb &rsp) {
3067+ auto rc = workerMasterApi->PureQueryMeta(req, rsp);
3068+ ObserveMetadataRpc(workerMasterApi, rc);
3069+ return rc;
3070+ };
3071+ RETURN_IF_NOT_OK(RedirectRetryWhenMetasMoving(request, response, query));
3072+ queryMetas.insert(queryMetas.end(), response.mutable_query_metas()->begin(),
3073+ response.mutable_query_metas()->end());
3074+ return QueryPureMetadataRedirects(response.info(), queryMetas);
3075+}
3076+ 
3077+Status WorkerOcServiceGetImpl::QueryPureMetadataRedirects(
3078+ const google::protobuf::RepeatedPtrField<RedirectMetaInfo> &redirects,
3079+ std::vector<master::QueryMetaInfoPb> &queryMetas)
3080+{
3081+ for (const auto &redirect : redirects) {
3082+ HostPort masterAddress;
3083+ RETURN_IF_NOT_OK(masterAddress.ParseString(redirect.redirect_meta_address()));
3084+ auto workerMasterApi = workerMasterApiManager_->GetWorkerMasterApi(masterAddress);
3085+ CHECK_FAIL_RETURN_STATUS(workerMasterApi != nullptr, K_RUNTIME_ERROR,
3086+ "Get redirect master API failed for PureQueryMeta");
3087+ master::PureQueryMetaReqPb request;
3088+ request.set_redirect(false);
cheyulin
cheyulincheyulin6 天前

[警告][正确性] [P1] 重定向只跟随一跳会在连续迁移时漏查元数据

证据:初次 PureQueryMeta 使用 redirect=true,但这里向第一跳目标发请求时固定设置 redirect=false,且 QueryPureMetadataRedirects 不再处理第二个 response.info。基线客户端使用待处理队列迭代跟随重定向,并有 FollowsTwoRedirects 单测;本 PR 删除了该测试。

问题:元数据在路由缓存落后期间连续迁移两次时,第一跳旧 owner 无法把 key 再重定向到当前 owner,调用方会把缺失结果解释为对象不存在或读失败。

影响:扩缩容或故障恢复期间会出现本可继续路由的短时读失败,降低控制面迁移期间的可用性。

建议:在统一 deadline 内迭代处理后续 response.info,并记录 visited owner/key 以检测环路;保持每个 key 的顺序和独立错误。

验证:恢复两跳及更多跳的重定向用例,并补充环路、重复目标和 deadline 耗尽测试。

likedislike
yangxiaogang14
5 天前 评论:
3089+ request.set_address(localAddress_.ToString());
3090+ request.mutable_object_keys()->Add(redirect.change_meta_ids().begin(), redirect.change_meta_ids().end());
3091+ master::PureQueryMetaRspPb response;
3092+ std::function<Status(master::PureQueryMetaReqPb &, master::PureQueryMetaRspPb &)> query =
3093+ [this, &workerMasterApi](master::PureQueryMetaReqPb &req, master::PureQueryMetaRspPb &rsp) {
3094+ auto rc = workerMasterApi->PureQueryMeta(req, rsp);
3095+ ObserveMetadataRpc(workerMasterApi, rc);
3096+ return rc;
3097+ };
3098+ RETURN_IF_NOT_OK(RedirectRetryWhenMetasMoving(request, response, query));
3099+ queryMetas.insert(queryMetas.end(), response.mutable_query_metas()->begin(),
3100+ response.mutable_query_metas()->end());
3101+ }
3102+ return Status::OK();
3103+}
3104+ 
2999Status WorkerOcServiceGetImpl::GetObjMetaInfo(const GetObjMetaInfoReqPb &req, GetObjMetaInfoRspPb &resp)3105Status WorkerOcServiceGetImpl::GetObjMetaInfo(const GetObjMetaInfoReqPb &req, GetObjMetaInfoRspPb &resp)
3000{3106{
3001 ScopedRequestContext ctx;3107 ScopedRequestContext ctx;
Msrc/datasystem/worker/object_cache/service/worker_oc_service_get_impl.h+21-0
@@ -67,6 +67,23 @@ public:
67 */67 */
68 Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> &serverApi);68 Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> &serverApi);
69 69 
70+ /**
71+ * @brief Acquire a resident local object for a side-effect-free direct read.
72+ * @param[in] objectKey Object key to look up.
73+ * @param[out] params Resident object snapshot, or null when the object is not locally readable.
74+ * @return K_OK on hit or miss; the error code otherwise.
75+ */
76+ Status TryAcquireLocalObject(const std::string &objectKey, std::unique_ptr<GetObjEntryParams> &params);
77+ 
78+ /**
79+ * @brief Query metadata-only object locations and resolve Master redirects internally.
80+ * @param[in] objectKeys Object keys to query.
81+ * @param[out] locations Locations keyed by object key; missing keys are absent.
82+ * @return K_OK on success; the original metadata error otherwise.
83+ */
84+ Status QueryObjectLocations(const std::vector<std::string> &objectKeys,
85+ std::unordered_map<std::string, master::ObjectLocationInfoPb> &locations);
86+ 
70 /**87 /**
71 * @brief Process get request for these objects not exist in local, query object meta from master, then get88 * @brief Process get request for these objects not exist in local, query object meta from master, then get
72 * object data from remote.89 * object data from remote.
@@ -527,6 +544,10 @@ private:
527 544 
528 Status QueryExistMetadataViaPureQueryMeta(const std::vector<std::string> &objectKeys,545 Status QueryExistMetadataViaPureQueryMeta(const std::vector<std::string> &objectKeys,
529 std::vector<master::QueryMetaInfoPb> &queryMetas);546 std::vector<master::QueryMetaInfoPb> &queryMetas);
547+ Status QueryPureMetadataGroup(const HostPort &masterAddress, const std::vector<std::string> &objectKeys,
548+ std::vector<master::QueryMetaInfoPb> &queryMetas);
549+ Status QueryPureMetadataRedirects(const google::protobuf::RepeatedPtrField<RedirectMetaInfo> &redirects,
550+ std::vector<master::QueryMetaInfoPb> &queryMetas);
530 551 
531 /**552 /**
532 * @brief Query the metadata of the specified objects in the redirect master.553 * @brief Query the metadata of the specified objects in the redirect master.
Asrc/datasystem/worker/object_cache/service/worker_query_and_get_impl.cpp+438-0
@@ -0,0 +1,438 @@
1+/**
2+ * Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
3+ *
4+ * Licensed under the Apache License, Version 2.0 (the "License");
5+ * you may not use this file except in compliance with the License.
6+ * You may obtain a copy of the License at
7+ *
8+ * http://www.apache.org/licenses/LICENSE-2.0
9+ *
10+ * Unless required by applicable law or agreed to in writing, software
11+ * distributed under the License is distributed on an "AS IS" BASIS,
12+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+ * See the License for the specific language governing permissions and
14+ * limitations under the License.
15+ */
16+ 
17+/** Description: Implements Worker-side metadata-affine QueryAndGet processing. */
18+#include "datasystem/worker/object_cache/service/worker_query_and_get_impl.h"
19+ 
20+#include <chrono>
21+#include <limits>
22+#include <optional>
23+#include <unordered_map>
24+#include <utility>
25+ 
26+#include "datasystem/common/inject/inject_point.h"
27+#include "datasystem/common/log/access_recorder.h"
28+#include "datasystem/common/log/latency_phase.h"
29+#include "datasystem/common/metrics/kv_metrics.h"
30+#include "datasystem/common/object_cache/provider_ub_failure_detail.h"
31+#include "datasystem/common/object_cache/shm_guard.h"
32+#include "datasystem/common/rdma/fast_transport_manager_wrapper.h"
33+#include "datasystem/common/util/raii.h"
34+#include "datasystem/common/util/status_helper.h"
35+#include "datasystem/common/util/validator.h"
36+#include "datasystem/worker/client_manager/client_manager.h"
37+ 
38+namespace datasystem {
39+namespace object_cache {
40+namespace {
41+constexpr uint64_t QUERY_AND_GET_MAX_TCP_PAYLOAD_SIZE = 512 * 1024UL;
42+constexpr double MICROSECONDS_PER_MILLISECOND = 1000.0;
43+ 
44+uint64_t GetSteadyTimeUs()
45+{
46+ const auto now = std::chrono::steady_clock::now().time_since_epoch();
47+ return static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::microseconds>(now).count());
48+}
49+ 
50+void FillLocation(const master::ObjectLocationInfoPb &source, QueryAndGetLocationInfoPb &target)
51+{
52+ target.set_object_key(source.object_key());
53+ target.set_object_size(source.object_size());
54+ *target.mutable_object_locations() = source.object_locations();
55+}
56+} // namespace
57+ 
58+WorkerQueryAndGetImpl::WorkerQueryAndGetImpl(std::shared_ptr<WorkerOcServiceGetImpl> getProc,
59+ std::shared_ptr<SharedMemoryRefTable> memoryRefTable,
60+ std::shared_ptr<AkSkManager> akSkManager, HostPort localAddress,
61+ std::shared_ptr<PeerUbAdmission> ubAdmission)
62+ : getProc_(std::move(getProc)),
63+ memoryRefTable_(std::move(memoryRefTable)),
64+ akSkManager_(std::move(akSkManager)),
65+ localAddress_(std::move(localAddress)),
66+ ubAdmission_(std::move(ubAdmission))
67+{
68+}
69+ 
70+Status WorkerQueryAndGetImpl::QueryAndGet(
71+ std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi)
72+{
73+ RequestState state;
74+ state.startUs = GetSteadyTimeUs();
75+ state.lastCheckpointUs = state.startUs;
76+ auto access = AccessRecorder::Object(AccessRecorderKey::DS_POSIX_QUERY_AND_GET);
77+ Status rc = ReadAndAuthenticate(serverApi, state.request);
78+ state.stats.preprocessUs = RecordPhase(state);
79+ if (rc.IsOk()) {
80+ access.ObjectKeysRef(state.request.object_keys()).TransportType(GetTransportName(state.request));
81+ rc = ProcessAndDeliver(serverApi, state);
82+ }
83+ const uint64_t totalUs = state.lastCheckpointUs - state.startUs;
84+ access.DataSize(state.stats.dataSize).Result(rc).Record();
85+ LogCompletion(state, rc, totalUs);
86+ return rc;
87+}
88+ 
89+Status WorkerQueryAndGetImpl::ReadAndAuthenticate(
90+ const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi,
91+ QueryAndGetReqPb &request) const
92+{
93+ RETURN_IF_NOT_OK_PRINT_ERROR_MSG(serverApi->Read(request), "Read QueryAndGet request failed");
94+ RETURN_RUNTIME_ERROR_IF_NULL(akSkManager_);
95+ RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(request), "AK/SK failed.");
96+ return Status::OK();
97+}
98+ 
99+Status WorkerQueryAndGetImpl::ProcessAndDeliver(
100+ const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi,
101+ RequestState &state)
102+{
103+ Raii rollback([this, &state] {
104+ if (!state.delivered) {
105+ RollbackShmRefs(state);
106+ }
107+ });
108+ RETURN_IF_NOT_OK(BuildResponse(state));
109+ CollectStats(state);
110+ RETURN_IF_NOT_OK(DeliverResponse(serverApi, state));
111+ state.delivered = true;
112+ return Status::OK();
113+}
114+ 
115+Status WorkerQueryAndGetImpl::DeliverResponse(
116+ const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi,
117+ RequestState &state) const
118+{
119+ Status deliveryRc = serverApi->Write(state.response);
120+ if (deliveryRc.IsOk()) {
121+ deliveryRc = serverApi->SendPayload(state.payloads);
122+ }
123+ if (deliveryRc.IsOk()) {
124+ serverApi->SetRequestComplete();
125+ }
126+ state.stats.deliveryUs = RecordPhase(state);
127+ RETURN_IF_NOT_OK_PRINT_ERROR_MSG(deliveryRc, "Deliver QueryAndGet response failed");
128+ return Status::OK();
129+}
130+ 
131+Status WorkerQueryAndGetImpl::BuildResponse(RequestState &state)
132+{
133+ Status rc = PrepareLocalResponse(state);
134+ state.stats.localReadUs = RecordPhase(state);
135+ RETURN_IF_NOT_OK(rc);
136+ rc = FillMissLocations(state);
137+ state.stats.metadataUs = RecordPhase(state);
138+ return rc;
139+}
140+ 
141+Status WorkerQueryAndGetImpl::PrepareLocalResponse(RequestState &state)
142+{
143+ RETURN_IF_NOT_OK(ValidateRequest(state.request));
144+ RETURN_RUNTIME_ERROR_IF_NULL(getProc_);
145+ InitializeResponse(state);
146+ return EncodeLocalHits(state);
147+}
148+ 
149+uint64_t WorkerQueryAndGetImpl::RecordPhase(RequestState &state)
150+{
151+ const uint64_t nowUs = GetSteadyTimeUs();
152+ const uint64_t elapsedUs = nowUs - state.lastCheckpointUs;
153+ state.lastCheckpointUs = nowUs;
154+ return elapsedUs;
155+}
156+ 
157+void WorkerQueryAndGetImpl::InitializeResponse(RequestState &state) const
158+{
159+ state.response.Clear();
160+ state.payloads.clear();
161+ state.misses.clear();
162+ state.addedShmRefs.clear();
163+ state.tcpPayloadSize = 0;
164+ state.misses.reserve(state.request.object_keys_size());
165+ if (state.request.data_request().has_shm()) {
166+ state.addedShmRefs.reserve(state.request.object_keys_size());
167+ }
168+ state.response.mutable_results()->Reserve(state.request.object_keys_size());
169+ for (int i = 0; i < state.request.object_keys_size(); ++i) {
170+ state.response.add_results();
171+ }
172+}
173+ 
174+void WorkerQueryAndGetImpl::RollbackShmRefs(const RequestState &state) const
175+{
176+ if (!state.request.data_request().has_shm() || state.addedShmRefs.empty()) {
177+ return;
178+ }
179+ const auto clientId = ClientKey::Intern(state.request.data_request().shm().client_id());
180+ for (const auto &shmId : state.addedShmRefs) {
181+ LOG_IF_ERROR(memoryRefTable_->RemoveShmUnit(clientId, shmId),
182+ "Rollback undelivered QueryAndGet SHM reference");
183+ }
184+}
185+ 
186+void WorkerQueryAndGetImpl::CollectStats(RequestState &state) const
187+{
188+ for (const auto &result : state.response.results()) {
189+ if (result.has_data_result()) {
190+ ++state.stats.inlineHits;
191+ state.stats.dataSize += result.location().object_size();
192+ } else {
193+ ++state.stats.misses;
194+ }
195+ }
196+}
197+ 
198+void WorkerQueryAndGetImpl::LogCompletion(const RequestState &state, const Status &rc, uint64_t totalUs) const
199+{
200+ const auto &stats = state.stats;
201+ const auto config = GetServerLatencyTraceConfig();
202+ SLOW_LOG_IF_OR_VLOG(
203+ INFO, config.processSlowerThanUs > 0 && totalUs >= config.processSlowerThanUs, 1,
204+ FormatString("QueryAndGet done, keyCount: %d, inlineHits: %zu, misses: %zu, transport: %s, "
205+ "preprocess: %.3fms, localRead: %.3fms, metadata: %.3fms, delivery: %.3fms, "
206+ "total: %.3fms, status: %s",
207+ state.request.object_keys_size(), stats.inlineHits, stats.misses,
208+ GetTransportName(state.request),
209+ static_cast<double>(stats.preprocessUs) / MICROSECONDS_PER_MILLISECOND,
210+ static_cast<double>(stats.localReadUs) / MICROSECONDS_PER_MILLISECOND,
211+ static_cast<double>(stats.metadataUs) / MICROSECONDS_PER_MILLISECOND,
212+ static_cast<double>(stats.deliveryUs) / MICROSECONDS_PER_MILLISECOND,
213+ static_cast<double>(totalUs) / MICROSECONDS_PER_MILLISECOND, rc.ToString()));
214+}
215+ 
216+const char *WorkerQueryAndGetImpl::GetTransportName(const QueryAndGetReqPb &request) const
217+{
218+ if (!request.has_data_request()) {
219+ return "NONE";
220+ }
221+ if (request.data_request().has_shm()) {
222+ return "SHM";
223+ }
224+ if (request.data_request().has_ub()) {
225+ return "UB";
226+ }
227+ if (request.data_request().has_tcp()) {
228+ return "TCP";
229+ }
230+ return "UNKNOWN";
231+}
232+ 
233+Status WorkerQueryAndGetImpl::ValidateRequest(const QueryAndGetReqPb &request) const
234+{
235+ CHECK_FAIL_RETURN_STATUS(Validator::IsBatchSizeUnderLimit(request.object_keys_size()), K_INVALID,
236+ "QueryAndGet batch size is invalid");
237+ CHECK_FAIL_RETURN_STATUS(request.object_keys_size() > 0, K_INVALID, "QueryAndGet object keys are empty");
238+ for (const auto &objectKey : request.object_keys()) {
239+ CHECK_FAIL_RETURN_STATUS(!objectKey.empty(), K_INVALID, "QueryAndGet object key is empty");
240+ }
241+ if (!request.has_data_request()) {
242+ return Status::OK();
243+ }
244+ const auto &dataRequest = request.data_request();
245+ CHECK_FAIL_RETURN_STATUS(dataRequest.has_tcp() || dataRequest.has_ub() || dataRequest.has_shm(), K_INVALID,
246+ "QueryAndGet data transport is not set");
247+ if (dataRequest.has_shm()) {
248+ RETURN_RUNTIME_ERROR_IF_NULL(memoryRefTable_);
249+ CHECK_FAIL_RETURN_STATUS(!dataRequest.shm().client_id().empty(), K_INVALID,
250+ "QueryAndGet SHM client ID is empty");
251+ const auto clientId = ClientKey::Intern(dataRequest.shm().client_id());
252+ CHECK_FAIL_RETURN_STATUS(worker::ClientManager::Instance().ClientShmEnabled(clientId), K_NOT_SUPPORTED,
253+ "QueryAndGet SHM session is unavailable");
254+ }
255+ if (!dataRequest.has_ub()) {
256+ return Status::OK();
257+ }
258+ const auto &ub = dataRequest.ub();
259+ CHECK_FAIL_RETURN_STATUS(ub.buffer_size() > 0 && !ub.urma_instance_id().empty(), K_INVALID,
260+ "QueryAndGet UB buffer is invalid");
261+ CHECK_FAIL_RETURN_STATUS(ub.buffer_infos_size() == request.object_keys_size(), K_INVALID,
262+ "QueryAndGet UB buffer count does not match object key count");
263+ const auto &remote = ub.buffer_infos(0);
264+ HostPort remoteAddress(remote.request_address().host(), remote.request_address().port());
265+ const std::string connectionId = remote.client_id().empty() ? remoteAddress.ToString() : remote.client_id();
266+ RETURN_IF_NOT_OK(CheckTransportConnectionStable(connectionId, ub.urma_instance_id()));
267+ return Status::OK();
268+}
269+ 
270+Status WorkerQueryAndGetImpl::EncodeLocalHits(RequestState &state)
271+{
272+ const auto &request = state.request;
273+ for (int i = 0; i < request.object_keys_size(); ++i) {
274+ const auto &objectKey = request.object_keys(i);
275+ if (!request.has_data_request()) {
276+ state.misses.emplace_back(objectKey);
277+ continue;
278+ }
279+ std::unique_ptr<GetObjEntryParams> params;
280+ RETURN_IF_NOT_OK(getProc_->TryAcquireLocalObject(objectKey, params));
cheyulin
cheyulincheyulin6 天前

[警告][正确性] [P1] 本地直读缺少当前版本与失效中的权威栅栏

证据:新路径只要 TryAcquireLocalObject 返回本地 entry 就直接编码;该函数只检查 Worker 本地 entry 已发布、未标记 CacheInvalid 且 SHM 完整。基线 Master QueryAndGet 在直读前还要求当前 metadata 中该 location 为 ACK、没有 CACHE_INVALID 或 PRIMARY_COPY_INVALID 异步操作,并携带当前 metadata version 读取 Worker。

问题:覆盖写、删除、迁移或主副本切换已在 Master 提交、但异步失效通知尚未落到 Worker 的窗口内,旧 entry 仍可能通过本地检查并作为命中返回,且不会再查询 PureQueryMeta。

影响:KVClient::Get 可能返回已被替换或删除的旧数据,破坏对象版本一致性,也不满足 PR 所称对外语义不变。

建议:本地直读前引入与权威 metadata 的 version、ACK location 和 pending invalidation 同步的可见栅栏;无法原子证明本地 generation 仍有效时,应按 miss 走 PureQueryMeta 和 ReplicaReader。

验证:增加 overwrite、delete、primary reselection、recovery newer-version 与失效通知延迟交错的确定性用例,断言旧版本绝不会以内联结果返回。

likedislike
yangxiaogang14
5 天前 评论:
281+ if (params == nullptr) {
282+ state.misses.emplace_back(objectKey);
283+ continue;
284+ }
285+ const size_t payloadCount = state.payloads.size();
286+ const uint64_t tcpPayloadSize = state.tcpPayloadSize;
287+ bool encoded = false;
288+ Status rc = EncodeLocalHit(state, static_cast<size_t>(i), *params, encoded);
289+ if (rc.IsError() || !encoded) {
290+ state.payloads.resize(payloadCount);
291+ state.tcpPayloadSize = tcpPayloadSize;
292+ state.response.mutable_results(i)->Clear();
293+ state.misses.emplace_back(objectKey);
294+ VLOG_IF(1, rc.IsError()) << "[ObjectKey " << objectKey
295+ << "] QueryAndGet inline data fallback: " << rc.ToString();
296+ }
297+ }
298+ return Status::OK();
299+}
300+ 
301+Status WorkerQueryAndGetImpl::EncodeLocalHit(RequestState &state, size_t index,
302+ const GetObjEntryParams &params, bool &encoded)
303+{
304+ INJECT_POINT("worker.QueryAndGet.EncodeLocalHitFailure");
305+ const auto &request = state.request;
306+ auto &result = *state.response.mutable_results(static_cast<int>(index));
307+ if (request.data_request().has_tcp()) {
308+ RETURN_IF_NOT_OK(EncodeTcp(params, *result.mutable_data_result(), state, encoded));
309+ } else if (request.data_request().has_ub()) {
310+ RETURN_IF_NOT_OK(EncodeUb(request.data_request().ub(), index, params, encoded));
311+ if (encoded) {
312+ result.mutable_data_result();
313+ }
314+ } else {
315+ EncodeShm(request.data_request().shm(), params, *result.mutable_data_result(), state);
316+ encoded = true;
317+ }
318+ if (encoded) {
319+ auto *location = result.mutable_location();
320+ location->set_object_key(request.object_keys(static_cast<int>(index)));
321+ location->add_object_locations(localAddress_.ToString());
322+ location->set_object_size(params.dataSize);
323+ }
324+ return Status::OK();
325+}
326+ 
327+Status WorkerQueryAndGetImpl::EncodeTcp(const GetObjEntryParams &params, QueryAndGetDataResultPb &result,
328+ RequestState &state, bool &encoded) const
329+{
330+ encoded = false;
331+ if (state.tcpPayloadSize > QUERY_AND_GET_MAX_TCP_PAYLOAD_SIZE
332+ || params.dataSize > QUERY_AND_GET_MAX_TCP_PAYLOAD_SIZE - state.tcpPayloadSize) {
333+ return Status::OK();
334+ }
335+ ShmGuard shmGuard(params.shmUnit, params.dataSize, params.metaSize);
336+ if (WorkerOcServiceCrudCommonApi::ShmEnable()) {
337+ RETURN_IF_NOT_OK(shmGuard.TryRLatch());
338+ }
339+ const size_t firstIndex = state.payloads.size();
340+ RETURN_IF_NOT_OK(shmGuard.TransferTo(state.payloads, 0, params.dataSize));
341+ for (size_t i = firstIndex; i < state.payloads.size(); ++i) {
342+ CHECK_FAIL_RETURN_STATUS(i <= std::numeric_limits<uint32_t>::max(), K_RUNTIME_ERROR,
343+ "QueryAndGet TCP payload index exceeds protocol limits");
344+ result.add_payload_indexes(static_cast<uint32_t>(i));
345+ }
346+ state.tcpPayloadSize += params.dataSize;
347+ METRIC_ADD(metrics::KvMetricId::WORKER_TO_CLIENT_TOTAL_BYTES, params.dataSize);
348+ encoded = true;
349+ INJECT_POINT_NO_RETURN("worker.QueryAndGet.EncodeTcp");
350+ return Status::OK();
351+}
352+ 
353+Status WorkerQueryAndGetImpl::EncodeUb(const QueryAndGetUbDataReqPb &request, size_t index,
354+ const GetObjEntryParams &params, bool &encoded) const
355+{
356+ encoded = false;
357+ if (params.dataSize > request.buffer_size()) {
358+ return Status::OK();
359+ }
360+ const auto &remote = request.buffer_infos(static_cast<int>(index));
361+ ShmGuard shmGuard(params.shmUnit, params.dataSize, params.metaSize);
362+ if (WorkerOcServiceCrudCommonApi::ShmEnable()) {
363+ RETURN_IF_NOT_OK(shmGuard.TryRLatch());
364+ }
365+ const uint64_t base = reinterpret_cast<uint64_t>(params.shmUnit->GetPointer());
366+ uint64_t segmentAddress = 0;
367+ uint64_t segmentSize = 0;
368+ GetSegmentInfoFromShmUnit(params.shmUnit, base, segmentAddress, segmentSize);
369+ const uint8_t srcChipId = NumaIdToChipId(params.shmUnit->GetNumaId());
370+ const uint8_t dstChipId = remote.has_chip_id() ? static_cast<uint8_t>(remote.chip_id()) : INVALID_CHIP_ID;
371+ std::vector<uint64_t> eventKeys;
372+ UrmaWriteFailure failure;
373+ auto lateCompletionContext =
374+ ubAdmission_ == nullptr
375+ ? std::nullopt
376+ : ubAdmission_->BuildLateCompletionContext(UbOperationKind::CLIENT_GET_WRITEBACK);
377+ Status rc = UrmaWritePayload(remote, segmentAddress, segmentSize, base, 0, params.dataSize, params.metaSize,
378+ srcChipId, dstChipId, true, eventKeys, nullptr, &failure,
379+ std::move(lateCompletionContext));
380+ if (rc.IsError()) {
381+ const auto &address = remote.request_address();
382+ const HostPort failedEndpoint(address.host(), address.port());
383+ ReportLocalUbOperationFailure(ubAdmission_.get(), localAddress_, failedEndpoint,
384+ UbOperationKind::CLIENT_GET_WRITEBACK, rc, failure.providerStatus,
385+ failure.cqeStatus);
386+ return rc;
O
OOuGongChang6 天前

[P2] 内联数据编码失败按整批失败处理,与"无 data_result 即 per-key 快路径 miss"的设计不符,较旧实现存在可用性回退

PR 描述承诺"无 data_result 表示快路径未命中,由 Client 继续走现有 ReplicaReader"——容量超限路径(EncodeTcp encoded=false → miss)确实实现了 per-key 降级,但错误路径没有:

  1. 本行 EncodeUb:单个 key 的 UrmaWritePayload 失败 → return rc → 整批 QueryAndGet 报错。客户端对 UB 模式只会整批重试或失败,不会把该 key 降级为副本读。
  2. EncodeTcp(第 330 行):TryRLatch 在写锁竞争下最多重试 20×10ms=200ms 后返回 K_TRY_AGAIN(shm_guard.cpp:30-65),此处 RETURN_IF_NOT_OK 使整批失败;客户端 DeadlineRetry 对 K_TRY_AGAIN 可重试(deadline_retry.cpp:39),每次重试又整批重新编码,直至 deadline。
  3. 客户端侧 BuildUbInlineData/BuildShmInlineData 物化失败同样整批失败(object_metadata_client.cpp:385-391),即使该 key 已有有效 location 可走 ReplicaReader。

对比旧实现(被删除的 master 侧 TryGetQueryAndGetData):本地读/UB 写失败仅跳过该 key 的内联数据(VLOG 后 return),响应仍携带 location,客户端自然降级副本读,同请求其余 key 不受影响;不会把 200ms 级 latch 竞争放大为整批失败。

建议:编码/物化失败按 per-key 处理——失败 key 走 miss 分支(Clear result + 加入 misses,仅返回 location),与容量超限路径保持一致;避免 eviction/migration/set storm 期间 meta owner 上所有直读 Get 整批失败。

likedislike
yangxiaogang14
5 天前 评论:
387+ }
388+ METRIC_ADD(metrics::KvMetricId::WORKER_TO_CLIENT_TOTAL_BYTES, params.dataSize);
389+ encoded = true;
390+ INJECT_POINT_NO_RETURN("worker.QueryAndGet.EncodeUb");
391+ return Status::OK();
392+}
393+ 
394+void WorkerQueryAndGetImpl::EncodeShm(const QueryAndGetShmDataReqPb &request, const GetObjEntryParams &params,
395+ QueryAndGetDataResultPb &result, RequestState &state) const
396+{
397+ const auto clientId = ClientKey::Intern(request.client_id());
398+ auto shmUnit = params.shmUnit;
399+ memoryRefTable_->AddShmUnit(clientId, shmUnit,
400+ GetRequestContext()->reqTimeoutDuration.CalcRealRemainingTime());
401+ state.addedShmRefs.emplace_back(params.shmUnit->GetId());
402+ auto *info = result.mutable_shm_info();
403+ info->set_store_fd(params.shmUnit->GetFd());
404+ info->set_offset(static_cast<int64_t>(params.shmUnit->GetOffset()));
405+ info->set_data_size(static_cast<int64_t>(params.dataSize));
406+ info->set_metadata_size(static_cast<int64_t>(params.metaSize));
407+ info->set_mmap_size(static_cast<int64_t>(params.shmUnit->GetMmapSize()));
408+ info->set_shm_id(params.shmUnit->id.ToString());
409+ info->set_is_seal(params.isSealed);
410+ info->set_write_mode(static_cast<uint32_t>(params.objectMode.GetWriteMode()));
411+ info->set_consistency_type(static_cast<uint32_t>(params.objectMode.GetConsistencyType()));
cheyulin
cheyulincheyulin6 天前

[警告][正确性] [P2] SHM inline 响应遗漏 cache_type

证据:QueryAndGetShmInfoPb 定义了 cache_type,客户端 BuildQueryAndGetResult 会读取并写入 ExternalBufferMeta.mode;这里仅填写 write_mode 和 consistency_type,没有从 params.objectMode 填 cache_type。proto3 默认值 0 对应 MEMORY。

问题:本地驻留的 DISK cache 对象经 SHM inline 返回后会被静默解释为 MEMORY,且客户端的枚举范围校验无法发现默认值造成的语义丢失。

影响:返回 Buffer 的对象模式与普通 Get 不一致,后续依赖 cache type 的行为和诊断信息可能错误。

建议:在 SHM 结果中显式设置 params.objectMode.GetCacheType(),并确保三种 mode 字段与普通 Get 共用同一编码 helper,避免协议路径漂移。

示例:

info->set_cache_type(static_cast<uint32_t>(params.objectMode.GetCacheType()));

验证:分别创建 MEMORY 与 DISK cache 对象并命中 Worker SHM QueryAndGet,断言 materialized Buffer 保留原始 CacheType。

likedislike
yangxiaogang14
5 天前 评论:
412+ INJECT_POINT_NO_RETURN("worker.QueryAndGet.EncodeShm");
413+}
414+ 
415+Status WorkerQueryAndGetImpl::FillMissLocations(RequestState &state) const
416+{
417+ if (!state.misses.empty()) {
418+ INJECT_POINT_NO_RETURN("worker.QueryAndGet.QueryMissMetadata");
419+ }
420+ std::unordered_map<std::string, master::ObjectLocationInfoPb> locations;
421+ RETURN_IF_NOT_OK(getProc_->QueryObjectLocations(state.misses, locations));
cheyulin
cheyulincheyulin5 天前

[严重][Bug] [P1] 不要让 miss 的 Master 故障丢弃同批本地命中

证据:EncodeLocalHits 已经把本地命中的数据写入 state.response,但这里对 QueryObjectLocations 使用 RETURN_IF_NOT_OK。只要任一 miss 的嵌套 PureQueryMeta 返回 K_RPC_UNAVAILABLEK_RPC_DEADLINE_EXCEEDED 等错误,整个 Worker RPC 就直接失败,已编码的 hit 不会交付。客户端随后还会在 ObjectMetadataClient::PrepareQueryRetry 中把这些状态当作 metadata-owner Worker 的路由故障,调用 failure handler 并 teardown 健康的 owner 连接。

问题:新版把元数据归属 Worker 同时变成了本地数据快路径和 Master 查询代理,但响应仍只有一个 RPC 级错误域;内层 Master 故障因此污染整个 owner 分组,并被错误归因给外层 Worker。

影响:mixed hit/miss 批量在 Master 短时不可用时连本地已命中的对象也无法返回,还可能淘汰健康 owner 的连接或路由,扩大故障域并放大恢复时延。

建议:把嵌套元数据错误限制在 miss key:响应增加逐 key status(或等价的可区分错误结果),保留并交付已编码的 local hits;客户端只让失败 key 返回错误或重试,并保留错误来源,避免把 Master 故障传给 metadata-owner failure handler。

验证:增加 mixed local-hit + miss 用例,在 PureQueryMeta 注入 unavailable、deadline 和 peer-dead;断言 hit Buffer 正常返回、仅 miss key 失败,且 metadata owner 不被 teardown/标记失效。

likedislike
422+ for (int i = 0; i < state.request.object_keys_size(); ++i) {
423+ auto *result = state.response.mutable_results(i);
424+ if (result->has_data_result()) {
425+ continue;
426+ }
427+ const auto &objectKey = state.request.object_keys(i);
428+ result->mutable_location()->set_object_key(objectKey);
429+ auto location = locations.find(objectKey);
430+ if (location != locations.end()) {
431+ FillLocation(location->second, *result->mutable_location());
432+ }
433+ }
434+ return Status::OK();
435+}
436+ 
437+} // namespace object_cache
438+} // namespace datasystem
Asrc/datasystem/worker/object_cache/service/worker_query_and_get_impl.h+126-0
@@ -0,0 +1,126 @@
1+/**
2+ * Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
3+ *
4+ * Licensed under the Apache License, Version 2.0 (the "License");
5+ * you may not use this file except in compliance with the License.
6+ * You may obtain a copy of the License at
7+ *
8+ * http://www.apache.org/licenses/LICENSE-2.0
9+ *
10+ * Unless required by applicable law or agreed to in writing, software
11+ * distributed under the License is distributed on an "AS IS" BASIS,
12+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+ * See the License for the specific language governing permissions and
14+ * limitations under the License.
15+ */
16+ 
17+/** Description: Defines Worker-side metadata-affine QueryAndGet processing. */
18+#ifndef DATASYSTEM_WORKER_OBJECT_CACHE_SERVICE_WORKER_QUERY_AND_GET_IMPL_H
19+#define DATASYSTEM_WORKER_OBJECT_CACHE_SERVICE_WORKER_QUERY_AND_GET_IMPL_H
20+ 
21+#include <cstdint>
22+#include <memory>
23+#include <string>
24+#include <vector>
25+ 
26+#include "datasystem/common/object_cache/object_ref_info.h"
27+#include "datasystem/common/object_cache/peer_ub_admission.h"
28+#include "datasystem/common/rpc/rpc_message.h"
29+#include "datasystem/common/util/net_util.h"
30+#include "datasystem/protos/object_posix.pb.h"
31+#include "datasystem/worker/object_cache/service/worker_oc_service_get_impl.h"
32+ 
33+namespace datasystem {
34+namespace object_cache {
35+ 
36+/** @brief Reads resident Worker objects inline and returns metadata-only locations for misses. */
37+class WorkerQueryAndGetImpl {
38+public:
39+ /**
40+ * @brief Construct the Worker QueryAndGet processor.
41+ * @param[in] getProc Existing local-read and metadata-location service.
42+ * @param[in] memoryRefTable Worker shared-memory reference table.
43+ * @param[in] akSkManager Worker authentication manager.
44+ * @param[in] localAddress Local Worker address.
45+ * @param[in] ubAdmission Worker UB admission state.
46+ */
47+ WorkerQueryAndGetImpl(std::shared_ptr<WorkerOcServiceGetImpl> getProc,
48+ std::shared_ptr<SharedMemoryRefTable> memoryRefTable,
49+ std::shared_ptr<AkSkManager> akSkManager, HostPort localAddress,
50+ std::shared_ptr<PeerUbAdmission> ubAdmission);
51+ 
52+ /**
53+ * @brief Destroy the Worker QueryAndGet processor.
54+ */
55+ ~WorkerQueryAndGetImpl() = default;
56+ 
57+ /**
58+ * @brief Process one ordered QueryAndGet request.
59+ * @param[in] serverApi Server RPC reader and writer.
60+ * @return K_OK on success; the error code otherwise.
61+ */
62+ Status QueryAndGet(
63+ std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi);
64+ 
65+private:
66+ struct QueryStats {
67+ size_t inlineHits = 0;
68+ size_t misses = 0;
69+ uint64_t dataSize = 0;
70+ uint64_t preprocessUs = 0;
71+ uint64_t localReadUs = 0;
72+ uint64_t metadataUs = 0;
73+ uint64_t deliveryUs = 0;
74+ };
75+ 
76+ struct RequestState {
77+ QueryAndGetReqPb request;
78+ QueryAndGetRspPb response;
79+ std::vector<RpcMessage> payloads;
80+ std::vector<std::string> misses;
81+ std::vector<ShmKey> addedShmRefs;
82+ QueryStats stats;
83+ uint64_t startUs = 0;
84+ uint64_t lastCheckpointUs = 0;
85+ uint64_t tcpPayloadSize = 0;
86+ bool delivered = false;
87+ };
88+ 
89+ Status ReadAndAuthenticate(
90+ const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi,
91+ QueryAndGetReqPb &request) const;
92+ Status ProcessAndDeliver(
93+ const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi,
94+ RequestState &state);
95+ Status BuildResponse(RequestState &state);
96+ Status DeliverResponse(
97+ const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi,
98+ RequestState &state) const;
99+ void InitializeResponse(RequestState &state) const;
100+ Status PrepareLocalResponse(RequestState &state);
101+ static uint64_t RecordPhase(RequestState &state);
102+ void RollbackShmRefs(const RequestState &state) const;
103+ void CollectStats(RequestState &state) const;
104+ void LogCompletion(const RequestState &state, const Status &rc, uint64_t totalUs) const;
105+ const char *GetTransportName(const QueryAndGetReqPb &request) const;
106+ Status ValidateRequest(const QueryAndGetReqPb &request) const;
107+ Status EncodeLocalHits(RequestState &state);
108+ Status EncodeLocalHit(RequestState &state, size_t index, const GetObjEntryParams &params, bool &encoded);
109+ Status EncodeTcp(const GetObjEntryParams &params, QueryAndGetDataResultPb &result,
110+ RequestState &state, bool &encoded) const;
111+ Status EncodeUb(const QueryAndGetUbDataReqPb &request, size_t index, const GetObjEntryParams &params,
112+ bool &encoded) const;
113+ void EncodeShm(const QueryAndGetShmDataReqPb &request, const GetObjEntryParams &params,
114+ QueryAndGetDataResultPb &result, RequestState &state) const;
115+ Status FillMissLocations(RequestState &state) const;
116+ 
117+ std::shared_ptr<WorkerOcServiceGetImpl> getProc_;
118+ std::shared_ptr<SharedMemoryRefTable> memoryRefTable_;
119+ std::shared_ptr<AkSkManager> akSkManager_;
120+ HostPort localAddress_;
121+ std::shared_ptr<PeerUbAdmission> ubAdmission_;
122+};
123+ 
124+} // namespace object_cache
125+} // namespace datasystem
126+#endif // DATASYSTEM_WORKER_OBJECT_CACHE_SERVICE_WORKER_QUERY_AND_GET_IMPL_H
Msrc/datasystem/worker/object_cache/worker_oc_service_impl.cpp+14-0
@@ -451,6 +451,8 @@ void WorkerOCServiceImpl::InitServiceImpl()
451 getProc_ =451 getProc_ =
452 std::make_shared<WorkerOcServiceGetImpl>(param, etcdStore_, memCpyThreadPool_, threadPool_,452 std::make_shared<WorkerOcServiceGetImpl>(param, etcdStore_, memCpyThreadPool_, threadPool_,
453 akSkManager_, localAddress_, migrateRateController_, ubAdmission_);453 akSkManager_, localAddress_, migrateRateController_, ubAdmission_);
454+ queryAndGetProc_ =
455+ std::make_shared<WorkerQueryAndGetImpl>(getProc_, memoryRefTable_, akSkManager_, localAddress_, ubAdmission_);
454 456 
455 deleteProc_ =457 deleteProc_ =
456 std::make_shared<WorkerOcServiceDeleteImpl>(param, akSkManager_, localAddress_, getProc_);458 std::make_shared<WorkerOcServiceDeleteImpl>(param, akSkManager_, localAddress_, getProc_);
@@ -1655,6 +1657,18 @@ Status WorkerOCServiceImpl::Get(std::shared_ptr<::datasystem::ServerUnaryWriterR
1655 return getProc_->Get(serverApi);1657 return getProc_->Get(serverApi);
1656}1658}
1657 1659 
1660+Status WorkerOCServiceImpl::QueryAndGet(
1661+ std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> serverApi)
1662+{
1663+ ScopedRequestContext ctx;
1664+ BthreadReadGuard noRecon;
1665+ RETURN_IF_NOT_OK_PRINT_ERROR_MSG(
1666+ ValidateWorkerState(noRecon, GetRequestContext()->reqTimeoutDuration.CalcRemainingTime()),
1667+ "validate worker state failed");
1668+ RETURN_RUNTIME_ERROR_IF_NULL(queryAndGetProc_);
1669+ return queryAndGetProc_->QueryAndGet(serverApi);
1670+}
1671+ 
1658Status WorkerOCServiceImpl::RefreshMeta(const ClientKey &clientId)1672Status WorkerOCServiceImpl::RefreshMeta(const ClientKey &clientId)
1659{1673{
1660 LOG(INFO) << "[RefreshMeta] clear memory reference count for client:" << clientId;1674 LOG(INFO) << "[RefreshMeta] clear memory reference count for client:" << clientId;
Msrc/datasystem/worker/object_cache/worker_oc_service_impl.h+11-0
@@ -89,6 +89,7 @@
89#include "datasystem/worker/object_cache/service/worker_oc_service_global_reference_impl.h"89#include "datasystem/worker/object_cache/service/worker_oc_service_global_reference_impl.h"
90#include "datasystem/worker/object_cache/service/worker_oc_service_expire_impl.h"90#include "datasystem/worker/object_cache/service/worker_oc_service_expire_impl.h"
91#include "datasystem/worker/object_cache/service/worker_oc_service_clear_data_flow.h"91#include "datasystem/worker/object_cache/service/worker_oc_service_clear_data_flow.h"
92+#include "datasystem/worker/object_cache/service/worker_query_and_get_impl.h"
92#include "datasystem/worker/object_cache/slot_recovery/slot_recovery_manager.h"93#include "datasystem/worker/object_cache/slot_recovery/slot_recovery_manager.h"
93// Keep bthread headers after project RPC/log headers so brpc logging macros (CHECK_EQ etc.) are94// Keep bthread headers after project RPC/log headers so brpc logging macros (CHECK_EQ etc.) are
94// established before bthread/mutex.h (which uses but does not define them) avoid redefinition pitfalls.95// established before bthread/mutex.h (which uses but does not define them) avoid redefinition pitfalls.
@@ -359,6 +360,14 @@ public:
359 */360 */
360 Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> serverApi) override;361 Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> serverApi) override;
361 362 
363+ /**
364+ * @brief Read metadata-owner-local objects and return locations for local misses.
365+ * @param[in] serverApi Unary socket carrying the request, response, and TCP payloads.
366+ * @return K_OK on success; the error code otherwise.
367+ */
368+ Status QueryAndGet(
369+ std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> serverApi) override;
370+ 
362 /**371 /**
363 * @brief Decrease the reference count of client.372 * @brief Decrease the reference count of client.
364 * @param[in] req The rpc request protobuf.373 * @param[in] req The rpc request protobuf.
@@ -1480,6 +1489,8 @@ private:
1480 1489 
1481 std::shared_ptr<WorkerOcServiceGetImpl> getProc_{ nullptr };1490 std::shared_ptr<WorkerOcServiceGetImpl> getProc_{ nullptr };
1482 1491 
1492+ std::shared_ptr<WorkerQueryAndGetImpl> queryAndGetProc_{ nullptr };
1493+ 
1483 std::shared_ptr<PeerUbAdmission> ubAdmission_{ std::make_shared<PeerUbAdmission>() };1494 std::shared_ptr<PeerUbAdmission> ubAdmission_{ std::make_shared<PeerUbAdmission>() };
1484 1495 
1485 std::shared_ptr<WorkerOcServiceDeleteImpl> deleteProc_{ nullptr };1496 std::shared_ptr<WorkerOcServiceDeleteImpl> deleteProc_{ nullptr };
Msrc/datasystem/worker/object_cache/worker_worker_oc_api.cpp+0-7
@@ -60,13 +60,6 @@ Status WorkerLocalWorkerOCApi::GetObjectRemote(GetObjectRemoteReqPb &req, GetObj
60 return service_->GetObjectRemote(req, rsp, payload);60 return service_->GetObjectRemote(req, rsp, payload);
61}61}
62 62 
63-Status WorkerLocalWorkerOCApi::GetObjectRemoteForQueryAndGet(GetObjectRemoteReqPb &req,
64- std::vector<RpcMessage> &payload)
65-{
66- GetObjectRemoteRspPb rsp;
67- return service_->GetObjectRemote(req, rsp, payload, true);
68-}
69- 
70WorkerRemoteWorkerOCApi::WorkerRemoteWorkerOCApi(HostPort hostPort, HostPort localHostPort,63WorkerRemoteWorkerOCApi::WorkerRemoteWorkerOCApi(HostPort hostPort, HostPort localHostPort,
71 std::shared_ptr<AkSkManager> akSkManager)64 std::shared_ptr<AkSkManager> akSkManager)
72 : hostPort_(std::move(hostPort)), localHostPort_(std::move(localHostPort)), akSkManager_(std::move(akSkManager))65 : hostPort_(std::move(hostPort)), localHostPort_(std::move(localHostPort)), akSkManager_(std::move(akSkManager))
Msrc/datasystem/worker/object_cache/worker_worker_oc_api.h+0-8
@@ -100,14 +100,6 @@ public:
100 Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,100 Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,
101 std::vector<RpcMessage> &payload) override;101 std::vector<RpcMessage> &payload) override;
102 102 
103- /**
104- * @brief Try to read resident data for a local QueryAndGet fast path.
105- * @param[in] req The object-read request.
106- * @param[out] payload The optional TCP payload.
107- * @return Status of the call.
108- */
109- Status GetObjectRemoteForQueryAndGet(GetObjectRemoteReqPb &req, std::vector<RpcMessage> &payload);
110- 
111 Status GetObjectRemote(103 Status GetObjectRemote(
112 std::unique_ptr<ClientUnaryWriterReader<GetObjectRemoteReqPb, GetObjectRemoteRspPb>> *clientApi) override104 std::unique_ptr<ClientUnaryWriterReader<GetObjectRemoteReqPb, GetObjectRemoteRspPb>> *clientApi) override
113 {105 {
Msrc/datasystem/worker/object_cache/worker_worker_oc_service_impl.cpp+13-27
@@ -241,7 +241,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemote(
241}241}
242 242 
243Status WorkerWorkerOCServiceImpl::GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,243Status WorkerWorkerOCServiceImpl::GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,
244- std::vector<RpcMessage> &payload, bool isQueryAndGet)244+ std::vector<RpcMessage> &payload)
245{245{
246 // Inherit the SDK traceID from the worker thread's thread_local Trace (set by246 // Inherit the SDK traceID from the worker thread's thread_local Trace (set by
247 // WorkerEntryImpl's SetTraceContextFromMeta) into the per-request context so247 // WorkerEntryImpl's SetTraceContextFromMeta) into the per-request context so
@@ -252,14 +252,10 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemote(GetObjectRemoteReqPb &req, Get
252 ScopedRequestContext ctx;252 ScopedRequestContext ctx;
253 METRIC_TIMER(metrics::KvMetricId::WORKER_RPC_REMOTE_GET_INBOUND_LATENCY);253 METRIC_TIMER(metrics::KvMetricId::WORKER_RPC_REMOTE_GET_INBOUND_LATENCY);
254 Timer slowLogTimer;254 Timer slowLogTimer;
255- if (isQueryAndGet) {
256- RETURN_IF_NOT_OK(CheckConnectionStable(req));
257- }
258 RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(req), "AK/SK failed.");255 RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(req), "AK/SK failed.");
259 const std::string callerAddress = GetRemoteAddressForLog(req);256 const std::string callerAddress = GetRemoteAddressForLog(req);
260 std::vector<uint64_t> eventKeys;257 std::vector<uint64_t> eventKeys;
261- RETURN_IF_NOT_OK(GetObjectRemoteHandler(req, rsp, payload, true, eventKeys, nullptr, nullptr, nullptr,258+ RETURN_IF_NOT_OK(GetObjectRemoteHandler(req, rsp, payload, true, eventKeys));
262- nullptr, isQueryAndGet));
263 auto config = GetServerLatencyTraceConfig();259 auto config = GetServerLatencyTraceConfig();
264 uint64_t elapsedUs = static_cast<uint64_t>(slowLogTimer.ElapsedMicroSecond());260 uint64_t elapsedUs = static_cast<uint64_t>(slowLogTimer.ElapsedMicroSecond());
265 SLOW_LOG_IF_OR_VLOG(261 SLOW_LOG_IF_OR_VLOG(
@@ -471,7 +467,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteHandler(const GetObjectRemoteRe
471 std::vector<uint64_t> &eventKeys,467 std::vector<uint64_t> &eventKeys,
472 std::shared_ptr<AggregateMemory> batchPtr,468 std::shared_ptr<AggregateMemory> batchPtr,
473 RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus,469 RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus,
474- BatchRh2dContext *batchRh2dContext, bool isQueryAndGet)470+ BatchRh2dContext *batchRh2dContext)
475{471{
476 PerfPoint point(PerfKey::WORKER_SERVER_BATCH_GET_REMOTE_HANDLER);472 PerfPoint point(PerfKey::WORKER_SERVER_BATCH_GET_REMOTE_HANDLER);
477 const std::string &objectKey = req.object_key();473 const std::string &objectKey = req.object_key();
@@ -485,7 +481,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteHandler(const GetObjectRemoteRe
485 INJECT_POINT("worker.worker_worker_remote_get_failure");481 INJECT_POINT("worker.worker_worker_remote_get_failure");
486 }482 }
487 Status status = GetObjectRemoteImpl(req, rsp, payload, blocking, eventKeys, batchPtr, batchRootInfo,483 Status status = GetObjectRemoteImpl(req, rsp, payload, blocking, eventKeys, batchPtr, batchRootInfo,
488- fallbackStatus, batchRh2dContext, isQueryAndGet);484+ fallbackStatus, batchRh2dContext);
489 if (status.GetCode() == K_INVALID || status.GetCode() == K_NOT_FOUND) {485 if (status.GetCode() == K_INVALID || status.GetCode() == K_NOT_FOUND) {
490 status = Status(K_WORKER_PULL_OBJECT_NOT_FOUND, status.GetMsg());486 status = Status(K_WORKER_PULL_OBJECT_NOT_FOUND, status.GetMsg());
491 }487 }
@@ -579,14 +575,14 @@ Status WorkerWorkerOCServiceImpl::LoadPayloadAndFillResponse(
579 const std::string &objectKey, uint64_t offset, uint64_t size, bool blocking, std::vector<uint64_t> &eventKeys,575 const std::string &objectKey, uint64_t offset, uint64_t size, bool blocking, std::vector<uint64_t> &eventKeys,
580 const std::shared_ptr<AggregateMemory> &batchPtr, RemoteH2DRootInfoPb *batchRootInfo,576 const std::shared_ptr<AggregateMemory> &batchPtr, RemoteH2DRootInfoPb *batchRootInfo,
581 BatchRh2dContext *batchRh2dContext, Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport,577 BatchRh2dContext *batchRh2dContext, Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport,
582- bool isPipelineH2DRequest, PerfPoint &batchImplPoint, bool isQueryAndGet)578+ bool isPipelineH2DRequest, PerfPoint &batchImplPoint)
583{579{
584 PerfPoint loadDataPoint(PerfKey::WORKER_LOAD_OBJECT_DATA);580 PerfPoint loadDataPoint(PerfKey::WORKER_LOAD_OBJECT_DATA);
585 PerfPoint pointImpl(PerfKey::WORKER_REMOTE_GET_READ_KEY);581 PerfPoint pointImpl(PerfKey::WORKER_REMOTE_GET_READ_KEY);
586 ReadObjectKV objKv(ReadKey(objectKey, offset, size), entry);582 ReadObjectKV objKv(ReadKey(objectKey, offset, size), entry);
587 RETURN_IF_NOT_OK_PRINT_ERROR_MSG(objKv.CheckReadOffset(), "Read offset verify failed");583 RETURN_IF_NOT_OK_PRINT_ERROR_MSG(objKv.CheckReadOffset(), "Read offset verify failed");
588 if (entry->IsSpilled() && entry->GetShmUnit() == nullptr) {584 if (entry->IsSpilled() && entry->GetShmUnit() == nullptr) {
589- RETURN_IF_NOT_OK(LoadSpilledObjectData(objectKey, outPayload, objKv, pointImpl, isQueryAndGet));585+ RETURN_IF_NOT_OK(LoadSpilledObjectData(objectKey, outPayload, objKv, pointImpl));
590 } else {586 } else {
591 pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_SHM_UNIT);587 pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_SHM_UNIT);
592 ShmGuard shmGuard(entry->GetShmUnit(), entry->GetDataSize(), entry->GetMetadataSize());588 ShmGuard shmGuard(entry->GetShmUnit(), entry->GetDataSize(), entry->GetMetadataSize());
@@ -616,16 +612,10 @@ Status WorkerWorkerOCServiceImpl::LoadPayloadAndFillResponse(
616 fastTransportName));612 fastTransportName));
617 GetWorkerTimeCost().Append("RemoteWriteFastTransport", writeTimer.ElapsedMilliSecond());613 GetWorkerTimeCost().Append("RemoteWriteFastTransport", writeTimer.ElapsedMilliSecond());
618 }614 }
619- if (isQueryAndGet && req.has_urma_info()) {615+ RETURN_IF_NOT_OK(HandlePayloadFallback(
620- CHECK_FAIL_RETURN_STATUS(isUrmaFastTransport, K_NOT_SUPPORTED,616+ req, rsp, entry, outPayload, shmGuard, shmUnit, fastTransportStatus, fastTransportName, objectKey,
621- "QueryAndGet UB transport is unavailable");617+ isFastTransportEnabled, isUrmaFastTransport, isPipelineH2DRequest, blocking, batchPtr, fallbackStatus,
622- RETURN_IF_NOT_OK(fastTransportStatus);618+ batchRootInfo, batchRh2dContext, objKv, localSegAddress, localSegSize));
623- } else {
624- RETURN_IF_NOT_OK(HandlePayloadFallback(
625- req, rsp, entry, outPayload, shmGuard, shmUnit, fastTransportStatus, fastTransportName, objectKey,
626- isFastTransportEnabled, isUrmaFastTransport, isPipelineH2DRequest, blocking, batchPtr, fallbackStatus,
627- batchRootInfo, batchRh2dContext, objKv, localSegAddress, localSegSize));
628- }
629 pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_RESP);619 pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_RESP);
630 }620 }
631 621 
@@ -635,12 +625,8 @@ Status WorkerWorkerOCServiceImpl::LoadPayloadAndFillResponse(
635 625 
636Status WorkerWorkerOCServiceImpl::LoadSpilledObjectData(const std::string &objectKey,626Status WorkerWorkerOCServiceImpl::LoadSpilledObjectData(const std::string &objectKey,
637 std::vector<RpcMessage> &outPayload,627 std::vector<RpcMessage> &outPayload,
638- const ReadObjectKV &objKv, PerfPoint &point,628+ const ReadObjectKV &objKv, PerfPoint &point)
639- bool isQueryAndGet)
640{629{
641- if (isQueryAndGet) {
642- RETURN_STATUS(K_NOT_SUPPORTED, "QueryAndGet fast path only reads resident data");
643- }
644 point.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_FROM_DISK);630 point.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_FROM_DISK);
645 RETURN_IF_NOT_OK(631 RETURN_IF_NOT_OK(
646 WorkerOcSpill::Instance()->Get(objectKey, outPayload, objKv.GetReadSize(), objKv.GetReadOffset()));632 WorkerOcSpill::Instance()->Get(objectKey, outPayload, objKv.GetReadSize(), objKv.GetReadOffset()));
@@ -892,7 +878,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteImpl(const GetObjectRemoteReqPb
892 std::vector<uint64_t> &eventKeys,878 std::vector<uint64_t> &eventKeys,
893 std::shared_ptr<AggregateMemory> batchPtr,879 std::shared_ptr<AggregateMemory> batchPtr,
894 RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus,880 RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus,
895- BatchRh2dContext *batchRh2dContext, bool isQueryAndGet)881+ BatchRh2dContext *batchRh2dContext)
896{882{
897 // Outer scope (BatchGetObjectRemote/GetObjectRemote) already created ScopedRequestContext.883 // Outer scope (BatchGetObjectRemote/GetObjectRemote) already created ScopedRequestContext.
898 // No nested scope here: workerTimeCost is a value member not inherited by inner scopes.884 // No nested scope here: workerTimeCost is a value member not inherited by inner scopes.
@@ -940,7 +926,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteImpl(const GetObjectRemoteReqPb
940 926 
941 return LoadPayloadAndFillResponse(req, rsp, entry, outPayload, objectKey, offset, size, blocking, eventKeys,927 return LoadPayloadAndFillResponse(req, rsp, entry, outPayload, objectKey, offset, size, blocking, eventKeys,
942 batchPtr, batchRootInfo, batchRh2dContext, fallbackStatus, isFastTransportEnabled,928 batchPtr, batchRootInfo, batchRh2dContext, fallbackStatus, isFastTransportEnabled,
943- isUrmaFastTransport, isPipelineH2DRequest, batchImplPoint, isQueryAndGet);929+ isUrmaFastTransport, isPipelineH2DRequest, batchImplPoint);
944}930}
945 931 
946Status WorkerWorkerOCServiceImpl::CheckCoordinatorState(const CheckCoordinatorStateReqPb &req,932Status WorkerWorkerOCServiceImpl::CheckCoordinatorState(const CheckCoordinatorStateReqPb &req,
Msrc/datasystem/worker/object_cache/worker_worker_oc_service_impl.h+5-10
@@ -77,11 +77,9 @@ public:
77 * @param[in] req Remote get request.77 * @param[in] req Remote get request.
78 * @param[out] rsp Remote get response.78 * @param[out] rsp Remote get response.
79 * @param[out] payload Out payloads.79 * @param[out] payload Out payloads.
80- * @param[in] isQueryAndGet Whether to use the metadata-query resident-data fast path.
81 * @return Status of the call.80 * @return Status of the call.
82 */81 */
83- Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, std::vector<RpcMessage> &payload,82+ Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, std::vector<RpcMessage> &payload);
84- bool isQueryAndGet = false);
85 83 
86 /**84 /**
87 * @brief Check etcd state.85 * @brief Check etcd state.
@@ -216,14 +214,13 @@ private:
216 * @param[out] keys The new request id to wait for if not blocking.214 * @param[out] keys The new request id to wait for if not blocking.
217 * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path.215 * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path.
218 * @param[in] batchRootInfo The common root info for batched requests.216 * @param[in] batchRootInfo The common root info for batched requests.
219- * @param[in] isQueryAndGet Whether this is a QueryAndGet fast-path attempt.
220 * @return Status of the call.217 * @return Status of the call.
221 */218 */
222 Status GetObjectRemoteImpl(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,219 Status GetObjectRemoteImpl(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,
223 std::vector<RpcMessage> &outPayload, bool blocking, std::vector<uint64_t> &eventKeys,220 std::vector<RpcMessage> &outPayload, bool blocking, std::vector<uint64_t> &eventKeys,
224 std::shared_ptr<AggregateMemory> batchPtr = nullptr,221 std::shared_ptr<AggregateMemory> batchPtr = nullptr,
225 RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr,222 RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr,
226- BatchRh2dContext *batchRh2dContext = nullptr, bool isQueryAndGet = false);223+ BatchRh2dContext *batchRh2dContext = nullptr);
227 224 
228 Status LoadPayloadAndFillResponse(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, SafeObjType &entry,225 Status LoadPayloadAndFillResponse(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, SafeObjType &entry,
229 std::vector<RpcMessage> &outPayload, const std::string &objectKey,226 std::vector<RpcMessage> &outPayload, const std::string &objectKey,
@@ -231,7 +228,7 @@ private:
231 const std::shared_ptr<AggregateMemory> &batchPtr,228 const std::shared_ptr<AggregateMemory> &batchPtr,
232 RemoteH2DRootInfoPb *batchRootInfo, BatchRh2dContext *batchRh2dContext,229 RemoteH2DRootInfoPb *batchRootInfo, BatchRh2dContext *batchRh2dContext,
233 Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport,230 Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport,
234- bool isPipelineH2DRequest, PerfPoint &batchImplPoint, bool isQueryAndGet);231+ bool isPipelineH2DRequest, PerfPoint &batchImplPoint);
235 232 
236 /**233 /**
237 * @brief Load a spilled object for the regular remote-get path.234 * @brief Load a spilled object for the regular remote-get path.
@@ -239,11 +236,10 @@ private:
239 * @param[out] outPayload Payload buffers loaded from spill storage.236 * @param[out] outPayload Payload buffers loaded from spill storage.
240 * @param[in] objKv Object read range.237 * @param[in] objKv Object read range.
241 * @param[in,out] point Remote-get performance point.238 * @param[in,out] point Remote-get performance point.
242- * @param[in] isQueryAndGet Whether this is a QueryAndGet fast-path attempt.
243 * @return K_OK on success; the error code otherwise.239 * @return K_OK on success; the error code otherwise.
244 */240 */
245 Status LoadSpilledObjectData(const std::string &objectKey, std::vector<RpcMessage> &outPayload,241 Status LoadSpilledObjectData(const std::string &objectKey, std::vector<RpcMessage> &outPayload,
246- const ReadObjectKV &objKv, PerfPoint &point, bool isQueryAndGet);242+ const ReadObjectKV &objKv, PerfPoint &point);
247 243 
248 /**244 /**
249 * @brief Fill a successful remote-get response and finish its performance records.245 * @brief Fill a successful remote-get response and finish its performance records.
@@ -387,14 +383,13 @@ private:
387 * @param[out] keys The request id to wait for if not blocking.383 * @param[out] keys The request id to wait for if not blocking.
388 * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path.384 * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path.
389 * @param[in] batchRootInfo The common root info for batched requests.385 * @param[in] batchRootInfo The common root info for batched requests.
390- * @param[in] isQueryAndGet Whether this is a QueryAndGet fast-path attempt.
391 * @return Status of the call.386 * @return Status of the call.
392 */387 */
393 Status GetObjectRemoteHandler(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,388 Status GetObjectRemoteHandler(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp,
394 std::vector<RpcMessage> &payload, bool blocking, std::vector<uint64_t> &eventKeys,389 std::vector<RpcMessage> &payload, bool blocking, std::vector<uint64_t> &eventKeys,
395 std::shared_ptr<AggregateMemory> batchPtr = nullptr,390 std::shared_ptr<AggregateMemory> batchPtr = nullptr,
396 RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr,391 RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr,
397- BatchRh2dContext *batchRh2dContext = nullptr, bool isQueryAndGet = false);392+ BatchRh2dContext *batchRh2dContext = nullptr);
398 393 
399 /**394 /**
400 * @brief Complete a URMA warmup request when the target warmup object is not ready locally.395 * @brief Complete a URMA warmup request when the target warmup object is not ready locally.
Mtests/scripts/test_ds_trace_bottleneck.py+6-6
@@ -716,7 +716,7 @@ def test_query_and_get_has_time_worker_analysis_and_unclosed_failure_boundary(ru
716 "host_ip": "",716 "host_ip": "",
717 "text": (717 "text": (
718 f"{timestamp} [BRPC_RPC_FRAMEWORK_SLOW] "718 f"{timestamp} [BRPC_RPC_FRAMEWORK_SLOW] "
719- "method=datasystem.master.MasterOCService.QueryAndGet "719+ "method=datasystem.WorkerOCService.QueryAndGet "
720 "e2e_us=20150 remote_processing_us=20150 server_req_queue_us=0 "720 "e2e_us=20150 remote_processing_us=20150 server_req_queue_us=0 "
721 "server_exec_us=0 network_residual_us=0 cntl_error_code=1008 "721 "server_exec_us=0 network_residual_us=0 cntl_error_code=1008 "
722 "cntl_failed=1 resp_attachment_bytes=0"722 "cntl_failed=1 resp_attachment_bytes=0"
@@ -778,7 +778,7 @@ def test_query_and_get_has_time_worker_analysis_and_unclosed_failure_boundary(ru
778 assert "ObjectReadFlow::Resolve" in timeout_flow["confirmed_flow"]778 assert "ObjectReadFlow::Resolve" in timeout_flow["confirmed_flow"]
779 assert "不能确认" in timeout_flow["root_cause_status"]779 assert "不能确认" in timeout_flow["root_cause_status"]
780 query_events = [event for event in correlation["events"] if event["kind"] == "query_meta"]780 query_events = [event for event in correlation["events"] if event["kind"] == "query_meta"]
781- assert any(event["method"].endswith("MasterOCService.QueryAndGet") for event in query_events)781+ assert any(event["method"].endswith("WorkerOCService.QueryAndGet") for event in query_events)
782 assert all(event["component_scope"] == "Client发起QueryMeta;Meta Owner目标未观测" for event in query_events)782 assert all(event["component_scope"] == "Client发起QueryMeta;Meta Owner目标未观测" for event in query_events)
783 783 
784 html_text = mod.render_html(analysis, "QueryMeta analysis")784 html_text = mod.render_html(analysis, "QueryMeta analysis")
@@ -1430,7 +1430,7 @@ def test_failed_urma_wait_timeout_is_an_error_family_not_unsegmented_parent(
1430 "method, expected_subcategory, expected_chain, expected_scope",1430 "method, expected_subcategory, expected_chain, expected_scope",
1431 [1431 [
1432 (1432 (
1433- "datasystem.master.MasterOCService.QueryAndGet",1433+ "datasystem.WorkerOCService.QueryAndGet",
1434 "QueryMeta RPC deadline",1434 "QueryMeta RPC deadline",
1435 "QueryMeta RPC超时→TransportGet失败→1001",1435 "QueryMeta RPC超时→TransportGet失败→1001",
1436 "Client等待Meta Owner QueryAndGet超时",1436 "Client等待Meta Owner QueryAndGet超时",
@@ -1518,7 +1518,7 @@ def test_successful_query_and_get_does_not_mask_later_data_rpc_deadline(run_dir:
1518 "host_ip": "",1518 "host_ip": "",
1519 "text": (1519 "text": (
1520 "[BRPC_RPC_FRAMEWORK_SLOW] "1520 "[BRPC_RPC_FRAMEWORK_SLOW] "
1521- "method=datasystem.master.MasterOCService.QueryAndGet e2e_us=18400 "1521+ "method=datasystem.WorkerOCService.QueryAndGet e2e_us=18400 "
1522 "server_exec_us=8 network_residual_us=18390 cntl_error_code=0 cntl_failed=0"1522 "server_exec_us=8 network_residual_us=18390 cntl_error_code=0 cntl_failed=0"
1523 ),1523 ),
1524 },1524 },
@@ -1566,7 +1566,7 @@ def test_successful_query_and_get_does_not_mask_later_urma_connect_deadline(run_
1566 "host_ip": "",1566 "host_ip": "",
1567 "text": (1567 "text": (
1568 "[BRPC_RPC_FRAMEWORK_SLOW] "1568 "[BRPC_RPC_FRAMEWORK_SLOW] "
1569- "method=datasystem.master.MasterOCService.QueryAndGet e2e_us=19400 "1569+ "method=datasystem.WorkerOCService.QueryAndGet e2e_us=19400 "
1570 "server_exec_us=8 network_residual_us=19390 cntl_error_code=0 cntl_failed=0"1570 "server_exec_us=8 network_residual_us=19390 cntl_error_code=0 cntl_failed=0"
1571 ),1571 ),
1572 },1572 },
@@ -1636,7 +1636,7 @@ def test_query_meta_detail_separates_retry_rpc_residual_and_inline_urma(
1636 "host_ip": "",1636 "host_ip": "",
1637 "text": (1637 "text": (
1638 "[BRPC_RPC_FRAMEWORK_SLOW] "1638 "[BRPC_RPC_FRAMEWORK_SLOW] "
1639- "method=datasystem.master.MasterOCService.QueryAndGet "1639+ "method=datasystem.WorkerOCService.QueryAndGet "
1640 f"e2e_us={int(rpc_e2e_ms * 1000)} server_exec_us=5 "1640 f"e2e_us={int(rpc_e2e_ms * 1000)} server_exec_us=5 "
1641 f"network_residual_us={int(rpc_network_ms * 1000)} "1641 f"network_residual_us={int(rpc_network_ms * 1000)} "
1642 "cntl_error_code=0 cntl_failed=0"1642 "cntl_error_code=0 cntl_failed=0"
Mtests/st/client/kv_cache/kv_client_transport_get_test.cpp+427-141
@@ -52,12 +52,17 @@ namespace st {
52namespace {52namespace {
53constexpr uint32_t META_OWNER_INDEX = 0;53constexpr uint32_t META_OWNER_INDEX = 0;
54constexpr uint32_t TRANSPORT_CLIENT_WORKER_INDEX = 1;54constexpr uint32_t TRANSPORT_CLIENT_WORKER_INDEX = 1;
55+constexpr uint32_t DATA_WORKER_INDEX = 2;
56+constexpr uint32_t WORKER_NUM = 3;
55constexpr int32_t CLIENT_TIMEOUT_MS = 3'000;57constexpr int32_t CLIENT_TIMEOUT_MS = 3'000;
56constexpr int32_t SHM_LATCH_TIMEOUT_MS = 1'000;58constexpr int32_t SHM_LATCH_TIMEOUT_MS = 1'000;
57constexpr size_t VALUE_SIZE = 128 * 1024;59constexpr size_t VALUE_SIZE = 128 * 1024;
60+constexpr size_t MIXED_OVERSIZED_INLINE_VALUE_SIZE = VALUE_SIZE + 1;
58constexpr size_t INLINE_DATA_LIMIT = 512 * 1024;61constexpr size_t INLINE_DATA_LIMIT = 512 * 1024;
59constexpr size_t LARGE_VALUE_SIZE = 8 * 1024 * 1024;62constexpr size_t LARGE_VALUE_SIZE = 8 * 1024 * 1024;
60constexpr size_t KEY_SEARCH_LIMIT = 100'000;63constexpr size_t KEY_SEARCH_LIMIT = 100'000;
64+constexpr uint64_t MIXED_TCP_DATA_RPC_COUNT = 1;
65+constexpr uint64_t MIXED_UB_DATA_RPC_COUNT = 2;
61constexpr char REAL_ROUTE_KEY_PREFIX[] = "transport_real_route_";66constexpr char REAL_ROUTE_KEY_PREFIX[] = "transport_real_route_";
62constexpr char UB_GET_SIZE_ENV[] = "DATASYSTEM_UB_GET_DATA_SIZE_BYTES";67constexpr char UB_GET_SIZE_ENV[] = "DATASYSTEM_UB_GET_DATA_SIZE_BYTES";
63constexpr char SKIP_WARMUP_INJECT[] = "ObjectClientImpl.ClientWorkerWarmup.skip";68constexpr char SKIP_WARMUP_INJECT[] = "ObjectClientImpl.ClientWorkerWarmup.skip";
@@ -70,8 +75,15 @@ constexpr char REGISTER_SHM_CLIENT_INJECT[] = "client.transport.register_shm_cli
70constexpr char GET_CLIENT_FD_INJECT[] = "client.transport.get_client_fd";75constexpr char GET_CLIENT_FD_INJECT[] = "client.transport.get_client_fd";
71constexpr char SHM_HEARTBEAT_INJECT[] = "client.transport.shm_heartbeat";76constexpr char SHM_HEARTBEAT_INJECT[] = "client.transport.shm_heartbeat";
72constexpr char DRAIN_BEFORE_SNAPSHOT_INJECT[] = "WorkerOCServiceImpl.DrainTopologyScaleInData.beforeSnapshot";77constexpr char DRAIN_BEFORE_SNAPSHOT_INJECT[] = "WorkerOCServiceImpl.DrainTopologyScaleInData.beforeSnapshot";
73-constexpr char HASH_RING_REFRESH_BEFORE_WAIT_INJECT[] = "HashRingRefresher.RefreshLoop.beforeWait";78+constexpr char QUERY_AND_GET_TCP_HIT_INJECT[] = "worker.QueryAndGet.EncodeTcp";
74-constexpr char INLINE_READ_FAILURE_INJECT[] = "worker.worker_worker_remote_get_failure";79+constexpr char QUERY_AND_GET_UB_HIT_INJECT[] = "worker.QueryAndGet.EncodeUb";
80+constexpr char QUERY_AND_GET_SHM_HIT_INJECT[] = "worker.QueryAndGet.EncodeShm";
81+constexpr char QUERY_AND_GET_METADATA_MISS_INJECT[] = "worker.QueryAndGet.QueryMissMetadata";
82+constexpr char QUERY_AND_GET_INLINE_FAILURE_INJECT[] = "worker.QueryAndGet.EncodeLocalHitFailure";
83+constexpr char SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT[] =
84+ "client.transport.query_and_get.shm_session_unavailable_before_build";
85+constexpr char SHM_MATERIALIZATION_FAILURE_INJECT[] =
86+ "client.transport.query_and_get.shm_materialization_failure";
75constexpr char SHM_LATCH_FAIL_INJECT[] = "worker.ShmGuard.TryRLatch.Fail";87constexpr char SHM_LATCH_FAIL_INJECT[] = "worker.ShmGuard.TryRLatch.Fail";
76constexpr char PROVIDER_GET_ENTER_INJECT[] = "worker.GetObjectRemote.afterRead";88constexpr char PROVIDER_GET_ENTER_INJECT[] = "worker.GetObjectRemote.afterRead";
77constexpr char PROVIDER_BATCH_GET_ENTER_INJECT[] = "worker.BatchGetObjectRemote.afterRead";89constexpr char PROVIDER_BATCH_GET_ENTER_INJECT[] = "worker.BatchGetObjectRemote.afterRead";
@@ -87,9 +99,8 @@ constexpr char GLOBAL_UNAVAILABLE_APPLIED_INJECT[] = "client.ub_health_filter.gl
87constexpr char GLOBAL_READ_DENIED_INJECT[] = "client.ub_health_filter.global_read_denied";99constexpr char GLOBAL_READ_DENIED_INJECT[] = "client.ub_health_filter.global_read_denied";
88constexpr char SHM_HOST_ID_ENV_NAME[] = "transport_get_shm_host_id";100constexpr char SHM_HOST_ID_ENV_NAME[] = "transport_get_shm_host_id";
89constexpr char SHM_HOST_ID_VALUE[] = "transport-get-shm-host";101constexpr char SHM_HOST_ID_VALUE[] = "transport-get-shm-host";
90-constexpr uint32_t SCALE_OUT_WORKER_INDEX = 3;102+constexpr char MIXED_HOST_ID_ENV_PREFIX[] = "transport_get_mixed_host_id_";
91-constexpr uint32_t SCALE_OUT_WORKER_COUNT = 4;103+constexpr char MIXED_HOST_ID_VALUE_PREFIX[] = "transport-get-mixed-host-";
92-constexpr size_t SCALE_OUT_KEY_COUNT = 256;
93 104 
94struct TransportRpcCounts {105struct TransportRpcCounts {
95 uint64_t queryAndGet = 0;106 uint64_t queryAndGet = 0;
@@ -101,28 +112,41 @@ struct TransportRpcCounts {
101 uint64_t shmHeartbeat = 0;112 uint64_t shmHeartbeat = 0;
102};113};
103 114 
104-const char *ExpectedTransport()115+struct WorkerQueryAndGetCounts {
116+ uint64_t tcpHits = 0;
117+ uint64_t ubHits = 0;
118+ uint64_t shmHits = 0;
119+ uint64_t metadataMisses = 0;
120+};
121+ 
122+struct MixedPathCounts {
123+ TransportRpcCounts rpc;
124+ WorkerQueryAndGetCounts localOwner;
125+ WorkerQueryAndGetCounts remoteOwner;
126+};
127+ 
128+constexpr bool IsUrmaBuild()
105{129{
106#ifdef USE_URMA130#ifdef USE_URMA
107- return "UB";131+ return true;
108#else132#else
109- return "TCP";133+ return false;
110#endif134#endif
111}135}
112 136 
113-std::string SelectMetadataOwner(const ClusterTopologyPb &topology, const std::string &key)137+const char *ExpectedTransport()
114{138{
115- std::map<uint32_t, std::string> tokenWorkers;139+ return IsUrmaBuild() ? "UB" : "TCP";
116- for (const auto &worker : topology.members()) {140+}
117- for (const auto token : worker.second.tokens()) {141+ 
118- tokenWorkers.emplace(token, worker.first);142+std::string MixedHostIdEnvName(uint32_t workerIndex)
119- }143+{
120- }144+ return std::string(MIXED_HOST_ID_ENV_PREFIX) + std::to_string(workerIndex);
121- if (tokenWorkers.empty()) {145+}
122- return {};146+ 
123- }147+std::string MixedHostIdValue(uint32_t workerIndex)
124- auto owner = tokenWorkers.upper_bound(MurmurHash3_32(key));148+{
125- return (owner == tokenWorkers.end() ? tokenWorkers.begin() : owner)->second;149+ return std::string(MIXED_HOST_ID_VALUE_PREFIX) + std::to_string(workerIndex);
126}150}
127} // namespace151} // namespace
128 152 
@@ -132,7 +156,7 @@ public:
132 {156 {
133 FLAGS_v = 1;157 FLAGS_v = 1;
134 opts.numEtcd = 1;158 opts.numEtcd = 1;
135- opts.numWorkers = 3;159+ opts.numWorkers = WORKER_NUM;
136 opts.enableDistributedMaster = "true";160 opts.enableDistributedMaster = "true";
137 opts.workerGflagParams =161 opts.workerGflagParams =
138 " -shared_memory_size_mb=512 -ipc_through_shared_memory=false -arena_per_tenant=1";162 " -shared_memory_size_mb=512 -ipc_through_shared_memory=false -arena_per_tenant=1";
@@ -142,6 +166,10 @@ public:
142 opts.workerGflagParams += " -enable_urma=false";166 opts.workerGflagParams += " -enable_urma=false";
143#endif167#endif
144 opts.injectActions = "worker.batch_get_failure_for_keys:call()";168 opts.injectActions = "worker.batch_get_failure_for_keys:call()";
169+ opts.injectActions += ";" + std::string(QUERY_AND_GET_TCP_HIT_INJECT) + ":call()";
170+ opts.injectActions += ";" + std::string(QUERY_AND_GET_UB_HIT_INJECT) + ":call()";
171+ opts.injectActions += ";" + std::string(QUERY_AND_GET_SHM_HIT_INJECT) + ":call()";
172+ opts.injectActions += ";" + std::string(QUERY_AND_GET_METADATA_MISS_INJECT) + ":call()";
145 }173 }
146 174 
147 void SetUp() override175 void SetUp() override
@@ -166,7 +194,7 @@ public:
166 ASSERT_NE(etcd_, nullptr);194 ASSERT_NE(etcd_, nullptr);
167 InitTestKVClient(META_OWNER_INDEX, writer_, CLIENT_TIMEOUT_MS);195 InitTestKVClient(META_OWNER_INDEX, writer_, CLIENT_TIMEOUT_MS);
168#ifdef USE_URMA196#ifdef USE_URMA
169- SetUbGetSize(INLINE_DATA_LIMIT);197+ SetUbGetSize(UbInlineBufferSize());
170#endif198#endif
171 InitTransportClient();199 InitTransportClient();
172 }200 }
@@ -205,12 +233,23 @@ protected:
205 void InitTransportClient()233 void InitTransportClient()
206 {234 {
207 ConnectOptions options;235 ConnectOptions options;
208- InitConnectOpt(TRANSPORT_CLIENT_WORKER_INDEX, options, CLIENT_TIMEOUT_MS);236+ InitConnectOpt(TransportClientWorkerIndex(), options, CLIENT_TIMEOUT_MS);
209 options.enableLocalCache = false;237 options.enableLocalCache = false;
238+ options.dataPlacementPolicy = DataPlacementPolicy::PREFERRED_META_OWNER;
210 reader_ = std::make_shared<KVClient>(options);239 reader_ = std::make_shared<KVClient>(options);
211 DS_ASSERT_OK(reader_->Init());240 DS_ASSERT_OK(reader_->Init());
212 }241 }
213 242 
243+ virtual uint32_t TransportClientWorkerIndex() const
244+ {
245+ return TRANSPORT_CLIENT_WORKER_INDEX;
246+ }
247+ 
248+ virtual size_t UbInlineBufferSize() const
249+ {
250+ return INLINE_DATA_LIMIT;
251+ }
252+ 
214 void SetUbGetSize(size_t size)253 void SetUbGetSize(size_t size)
215 {254 {
216 ASSERT_EQ(setenv(UB_GET_SIZE_ENV, std::to_string(size).c_str(), 1), 0);255 ASSERT_EQ(setenv(UB_GET_SIZE_ENV, std::to_string(size).c_str(), 1), 0);
@@ -270,6 +309,18 @@ protected:
270 counts.shmHeartbeat = inject::GetExecuteCount(SHM_HEARTBEAT_INJECT);309 counts.shmHeartbeat = inject::GetExecuteCount(SHM_HEARTBEAT_INJECT);
271 }310 }
272 311 
312+ void GetWorkerQueryAndGetCounts(uint32_t workerIndex, WorkerQueryAndGetCounts &counts)
313+ {
314+ DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(
315+ WORKER, workerIndex, QUERY_AND_GET_TCP_HIT_INJECT, counts.tcpHits));
316+ DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(
317+ WORKER, workerIndex, QUERY_AND_GET_UB_HIT_INJECT, counts.ubHits));
318+ DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(
319+ WORKER, workerIndex, QUERY_AND_GET_SHM_HIT_INJECT, counts.shmHits));
320+ DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(
321+ WORKER, workerIndex, QUERY_AND_GET_METADATA_MISS_INJECT, counts.metadataMisses));
322+ }
323+ 
273 // Generate N distinct keys without making placement assumptions.324 // Generate N distinct keys without making placement assumptions.
274 std::vector<std::string> MakeRandomKeys(size_t count)325 std::vector<std::string> MakeRandomKeys(size_t count)
275 {326 {
@@ -550,40 +601,6 @@ private:
550 bool setupAttempted_ = false;601 bool setupAttempted_ = false;
551};602};
552 603 
553-class KVClientTransportGetScaleOutRealUrmaTest : public KVClientTransportGetTest {
554-public:
555- void SetClusterSetupOptions(ExternalClusterOptions &opts) override
556- {
557- KVClientTransportGetTest::SetClusterSetupOptions(opts);
558- opts.workerGflagParams += " -enable_lossless_data_exit_mode=true";
559- }
560- 
561- void SetUp() override
562- {
563-#if !defined(USE_URMA)
564- GTEST_SKIP() << "Real URMA scale-out redirect ST requires USE_URMA.";
565-#elif defined(USE_URMA_MOCK)
566- GTEST_SKIP() << "Real URMA scale-out redirect ST does not run with USE_URMA_MOCK.";
567-#else
568- if (std::getenv("DS_URMA_DEV_NAME") == nullptr) {
569- GTEST_SKIP() << "Real URMA scale-out redirect ST requires DS_URMA_DEV_NAME and a usable URMA device.";
570- }
571- setupAttempted_ = true;
572- KVClientTransportGetTest::SetUp();
573-#endif
574- }
575- 
576- void TearDown() override
577- {
578- if (setupAttempted_) {
579- KVClientTransportGetTest::TearDown();
580- }
581- }
582- 
583-private:
584- bool setupAttempted_ = false;
585-};
586- 
587class KVClientTransportGetWithTargetShmDisabledTest : public KVClientTransportGetTest {604class KVClientTransportGetWithTargetShmDisabledTest : public KVClientTransportGetTest {
588public:605public:
589 void SetClusterSetupOptions(ExternalClusterOptions &opts) override606 void SetClusterSetupOptions(ExternalClusterOptions &opts) override
@@ -709,6 +726,336 @@ protected:
709 }726 }
710};727};
711 728 
729+class KVClientTransportGetMixedPathTest : public KVClientTransportGetTest {
730+public:
731+ void SetClusterSetupOptions(ExternalClusterOptions &opts) override
732+ {
733+ KVClientTransportGetTest::SetClusterSetupOptions(opts);
734+ constexpr char DISABLED_SHM_OPTION[] = "-ipc_through_shared_memory=false";
735+ const auto pos = opts.workerGflagParams.find(DISABLED_SHM_OPTION);
736+ ASSERT_NE(pos, std::string::npos);
737+ opts.workerGflagParams.replace(pos, sizeof(DISABLED_SHM_OPTION) - 1, "-ipc_through_shared_memory=true");
738+ for (uint32_t i = 0; i < WORKER_NUM; ++i) {
739+ opts.workerSpecifyGflagParams[i] += " -host_id_env_name=" + MixedHostIdEnvName(i);
740+ }
741+ }
742+ 
743+ void SetUp() override
744+ {
745+ for (uint32_t i = 0; i < WORKER_NUM; ++i) {
746+ ASSERT_EQ(setenv(MixedHostIdEnvName(i).c_str(), MixedHostIdValue(i).c_str(), 1), 0);
747+ }
748+ KVClientTransportGetTest::SetUp();
749+ }
750+ 
751+ void TearDown() override
752+ {
753+ KVClientTransportGetTest::TearDown();
754+ for (uint32_t i = 0; i < WORKER_NUM; ++i) {
755+ (void)unsetenv(MixedHostIdEnvName(i).c_str());
756+ }
757+ }
758+ 
759+protected:
760+ uint32_t TransportClientWorkerIndex() const override
761+ {
762+ return META_OWNER_INDEX;
763+ }
764+ 
765+ size_t UbInlineBufferSize() const override
766+ {
767+ return VALUE_SIZE;
768+ }
769+ 
770+ void AssertMixedPathBuffers(std::vector<Optional<Buffer>> &buffers,
771+ const std::vector<std::string> &values)
772+ {
773+ ASSERT_EQ(buffers.size(), 4u);
774+ ASSERT_FALSE(buffers[0]);
775+ for (size_t i = 1; i < buffers.size(); ++i) {
776+ ASSERT_TRUE(buffers[i]);
777+ }
778+ AssertBufferEqual(*buffers[1], values[1]);
779+ AssertBufferEqual(*buffers[2], values[2]);
780+ AssertBufferEqual(*buffers[3], values[0]);
781+ }
782+ 
783+ void GetMixedPathCounts(MixedPathCounts &counts)
784+ {
785+ GetRpcCounts(counts.rpc);
786+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, counts.localOwner);
787+ GetWorkerQueryAndGetCounts(TRANSPORT_CLIENT_WORKER_INDEX, counts.remoteOwner);
788+ }
789+ 
790+ void AssertMixedPathCounts(const MixedPathCounts &before, const MixedPathCounts &after)
791+ {
792+ ASSERT_EQ(after.rpc.queryAndGet, before.rpc.queryAndGet + 2);
793+ const uint64_t expectedDataRpcCount =
794+ IsUrmaBuild() ? MIXED_UB_DATA_RPC_COUNT : MIXED_TCP_DATA_RPC_COUNT;
795+ ASSERT_EQ(after.rpc.getObjectRemote, before.rpc.getObjectRemote + expectedDataRpcCount);
796+ ASSERT_EQ(after.localOwner.shmHits, before.localOwner.shmHits + 1);
797+ ASSERT_EQ(after.localOwner.metadataMisses, before.localOwner.metadataMisses + 1);
798+ ASSERT_EQ(after.remoteOwner.tcpHits, before.remoteOwner.tcpHits + (IsUrmaBuild() ? 0 : 1));
799+ ASSERT_EQ(after.remoteOwner.ubHits, before.remoteOwner.ubHits);
800+ ASSERT_EQ(after.remoteOwner.metadataMisses, before.remoteOwner.metadataMisses + 1);
801+ }
802+};
803+ 
804+TEST_F(KVClientTransportGetWithAllWorkersShmTest, SameNodeMetadataOwnerHitUsesShmInline)
805+{
806+ std::vector<std::string> keys;
807+ GetRealHashKeysToWorker(META_OWNER_INDEX, 2, keys);
808+ ASSERT_EQ(keys.size(), 2u);
809+ const std::string value(VALUE_SIZE, 'a');
810+ DS_ASSERT_OK(writer_->Set(keys[0], value));
811+ DS_ASSERT_OK(writer_->Set(keys[1], value));
812+ 
813+ Optional<Buffer> warmup;
814+ DS_ASSERT_OK(reader_->Get(keys[0], warmup));
815+ ASSERT_TRUE(warmup);
816+ ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");
817+ 
818+ TransportRpcCounts rpcBefore;
819+ WorkerQueryAndGetCounts workerBefore;
820+ GetRpcCounts(rpcBefore);
821+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerBefore);
822+ Optional<Buffer> buffer;
823+ DS_ASSERT_OK(reader_->Get(keys[1], buffer));
824+ TransportRpcCounts rpcAfter;
825+ WorkerQueryAndGetCounts workerAfter;
826+ GetRpcCounts(rpcAfter);
827+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerAfter);
828+ 
829+ ASSERT_TRUE(buffer);
830+ AssertBufferEqual(*buffer, value);
831+ ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");
832+ ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1);
833+ ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote);
834+ ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet);
835+ ASSERT_EQ(rpcAfter.registerShmClient, rpcBefore.registerShmClient);
836+ ASSERT_EQ(rpcAfter.getClientFd, rpcBefore.getClientFd);
837+ ASSERT_EQ(workerAfter.tcpHits, workerBefore.tcpHits);
838+ ASSERT_EQ(workerAfter.ubHits, workerBefore.ubHits);
839+ ASSERT_EQ(workerAfter.shmHits, workerBefore.shmHits + 1);
840+ ASSERT_EQ(workerAfter.metadataMisses, workerBefore.metadataMisses);
841+}
842+ 
843+TEST_F(KVClientTransportGetWithAllWorkersShmTest, UnavailableShmSessionFallsBackBeforeDispatch)
844+{
845+ std::vector<std::string> keys;
846+ GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys);
847+ const std::string value(VALUE_SIZE, 's');
848+ DS_ASSERT_OK(writer_->Set(keys.front(), value));
849+ const uint64_t injectCount = inject::GetExecuteCount(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT);
850+ DS_ASSERT_OK(inject::Set(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT, "1*call()"));
851+ Raii clearInject([] { (void)inject::Clear(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT); });
852+ 
853+ Optional<Buffer> buffer;
854+ DS_ASSERT_OK(reader_->Get(keys.front(), buffer));
855+ 
856+ ASSERT_TRUE(buffer);
857+ AssertBufferEqual(*buffer, value);
858+ ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport());
859+ ASSERT_EQ(inject::GetExecuteCount(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT), injectCount + 1);
860+}
861+ 
862+TEST_F(KVClientTransportGetWithAllWorkersShmTest, ShmMaterializationFailureFallsBackPerKey)
863+{
864+ std::vector<std::string> keys;
865+ GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys);
866+ const std::string value(VALUE_SIZE, 'm');
867+ DS_ASSERT_OK(writer_->Set(keys.front(), value));
868+ DS_ASSERT_OK(inject::Set(SHM_MATERIALIZATION_FAILURE_INJECT, "1*return(K_RUNTIME_ERROR)"));
869+ Raii clearInject([] { (void)inject::Clear(SHM_MATERIALIZATION_FAILURE_INJECT); });
870+ 
871+ Optional<Buffer> buffer;
872+ DS_ASSERT_OK(reader_->Get(keys.front(), buffer));
873+ 
874+ ASSERT_TRUE(buffer);
875+ AssertBufferEqual(*buffer, value);
876+ ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");
877+}
878+ 
879+TEST_F(KVClientTransportGetTest, CrossNodeMetadataOwnerHitUsesUbInline)
880+{
881+#ifndef USE_URMA
882+ GTEST_SKIP() << "QueryAndGet UB inline ST requires USE_URMA.";
883+#else
884+ std::vector<std::string> keys;
885+ GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys);
886+ ASSERT_EQ(keys.size(), 1u);
887+ const std::string value(VALUE_SIZE, 'b');
888+ DS_ASSERT_OK(writer_->Set(keys[0], value));
889+ TransportRpcCounts rpcBefore;
890+ WorkerQueryAndGetCounts workerBefore;
891+ GetRpcCounts(rpcBefore);
892+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerBefore);
893+ 
894+ Optional<Buffer> buffer;
895+ DS_ASSERT_OK(reader_->Get(keys[0], buffer));
896+ TransportRpcCounts rpcAfter;
897+ WorkerQueryAndGetCounts workerAfter;
898+ GetRpcCounts(rpcAfter);
899+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerAfter);
900+ 
901+ ASSERT_TRUE(buffer);
902+ AssertBufferEqual(*buffer, value);
903+ ASSERT_EQ(AccessTransportTracker::ToString(), "UB");
904+ ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1);
905+ ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote);
906+ ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet);
907+ ASSERT_EQ(workerAfter.tcpHits, workerBefore.tcpHits);
908+ ASSERT_EQ(workerAfter.ubHits, workerBefore.ubHits + 1);
909+ ASSERT_EQ(workerAfter.shmHits, workerBefore.shmHits);
910+ ASSERT_EQ(workerAfter.metadataMisses, workerBefore.metadataMisses);
911+#endif
912+}
913+ 
914+TEST_F(KVClientTransportGetTest, CrossNodeMetadataOwnerHitUsesTcpInline)
915+{
916+#ifdef USE_URMA
917+ GTEST_SKIP() << "QueryAndGet TCP-only ST requires a non-URMA build.";
918+#else
919+ std::vector<std::string> keys;
920+ GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys);
921+ ASSERT_EQ(keys.size(), 1u);
922+ const std::string value(VALUE_SIZE, 'c');
923+ DS_ASSERT_OK(writer_->Set(keys[0], value));
924+ TransportRpcCounts rpcBefore;
925+ WorkerQueryAndGetCounts workerBefore;
926+ GetRpcCounts(rpcBefore);
927+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerBefore);
928+ 
929+ Optional<Buffer> buffer;
930+ DS_ASSERT_OK(reader_->Get(keys[0], buffer));
931+ TransportRpcCounts rpcAfter;
932+ WorkerQueryAndGetCounts workerAfter;
933+ GetRpcCounts(rpcAfter);
934+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerAfter);
935+ 
936+ ASSERT_TRUE(buffer);
937+ AssertBufferEqual(*buffer, value);
938+ ASSERT_EQ(AccessTransportTracker::ToString(), "TCP");
939+ ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1);
940+ ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote);
941+ ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet);
942+ ASSERT_EQ(workerAfter.tcpHits, workerBefore.tcpHits + 1);
943+ ASSERT_EQ(workerAfter.ubHits, workerBefore.ubHits);
944+ ASSERT_EQ(workerAfter.shmHits, workerBefore.shmHits);
945+ ASSERT_EQ(workerAfter.metadataMisses, workerBefore.metadataMisses);
946+#endif
947+}
948+ 
949+TEST_F(KVClientTransportGetWithAllWorkersShmTest, MetadataMissReadsSameNodeDataWorkerWithShm)
950+{
951+ std::vector<std::string> ownerKeys;
952+ std::vector<std::string> dataWorkerKeys;
953+ GetRealHashKeysToWorker(META_OWNER_INDEX, 2, ownerKeys);
954+ GetRealHashKeysToWorker(DATA_WORKER_INDEX, 1, dataWorkerKeys);
955+ ASSERT_EQ(ownerKeys.size(), 2u);
956+ ASSERT_EQ(dataWorkerKeys.size(), 1u);
957+ const std::string value(VALUE_SIZE, 'd');
958+ std::shared_ptr<KVClient> dataWriter;
959+ InitTestKVClient(DATA_WORKER_INDEX, dataWriter, CLIENT_TIMEOUT_MS);
960+ DS_ASSERT_OK(writer_->Set(ownerKeys[0], value));
961+ DS_ASSERT_OK(dataWriter->Set(dataWorkerKeys[0], value));
962+ DS_ASSERT_OK(dataWriter->Set(ownerKeys[1], value));
963+ 
964+ Optional<Buffer> warmup;
965+ DS_ASSERT_OK(reader_->Get(ownerKeys[0], warmup));
966+ DS_ASSERT_OK(reader_->Get(dataWorkerKeys[0], warmup));
967+ TransportRpcCounts rpcBefore;
968+ WorkerQueryAndGetCounts ownerBefore;
969+ GetRpcCounts(rpcBefore);
970+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerBefore);
971+ 
972+ Optional<Buffer> buffer;
973+ DS_ASSERT_OK(reader_->Get(ownerKeys[1], buffer));
974+ TransportRpcCounts rpcAfter;
975+ WorkerQueryAndGetCounts ownerAfter;
976+ GetRpcCounts(rpcAfter);
977+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerAfter);
978+ 
979+ ASSERT_TRUE(buffer);
980+ AssertBufferEqual(*buffer, value);
981+ ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");
982+ ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1);
983+ ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet + 1);
984+ ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote);
985+ ASSERT_EQ(ownerAfter.tcpHits, ownerBefore.tcpHits);
986+ ASSERT_EQ(ownerAfter.ubHits, ownerBefore.ubHits);
987+ ASSERT_EQ(ownerAfter.shmHits, ownerBefore.shmHits);
988+ ASSERT_EQ(ownerAfter.metadataMisses, ownerBefore.metadataMisses + 1);
989+}
990+ 
991+TEST_F(KVClientTransportGetTest, MetadataMissReadsCrossNodeDataWorker)
992+{
993+ std::vector<std::string> keys;
994+ GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys);
995+ ASSERT_EQ(keys.size(), 1u);
996+ const std::string value(VALUE_SIZE, 'e');
997+ std::shared_ptr<KVClient> dataWriter;
998+ InitTestKVClient(DATA_WORKER_INDEX, dataWriter, CLIENT_TIMEOUT_MS);
999+ DS_ASSERT_OK(dataWriter->Set(keys[0], value));
1000+ TransportRpcCounts rpcBefore;
1001+ WorkerQueryAndGetCounts ownerBefore;
1002+ GetRpcCounts(rpcBefore);
1003+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerBefore);
1004+ 
1005+ Optional<Buffer> buffer;
1006+ DS_ASSERT_OK(reader_->Get(keys[0], buffer));
1007+ TransportRpcCounts rpcAfter;
1008+ WorkerQueryAndGetCounts ownerAfter;
1009+ GetRpcCounts(rpcAfter);
1010+ GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerAfter);
1011+ 
1012+ ASSERT_TRUE(buffer);
1013+ AssertBufferEqual(*buffer, value);
1014+ ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport());
1015+ ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1);
1016+ ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote + 1);
1017+ ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet);
1018+ ASSERT_EQ(ownerAfter.tcpHits, ownerBefore.tcpHits);
1019+ ASSERT_EQ(ownerAfter.ubHits, ownerBefore.ubHits);
1020+ ASSERT_EQ(ownerAfter.shmHits, ownerBefore.shmHits);
1021+ ASSERT_EQ(ownerAfter.metadataMisses, ownerBefore.metadataMisses + 1);
1022+}
1023+ 
1024+TEST_F(KVClientTransportGetMixedPathTest, MultiKeyMixedPathsPreserveOrder)
1025+{
1026+ std::vector<std::string> localOwnerKeys;
1027+ std::vector<std::string> remoteOwnerKeys;
1028+ GetRealHashKeysToWorker(META_OWNER_INDEX, 3, localOwnerKeys);
1029+ GetRealHashKeysToWorker(TRANSPORT_CLIENT_WORKER_INDEX, 2, remoteOwnerKeys);
1030+ ASSERT_EQ(localOwnerKeys.size(), 3u);
1031+ ASSERT_EQ(remoteOwnerKeys.size(), 2u);
1032+ std::shared_ptr<KVClient> remoteOwnerWriter;
1033+ std::shared_ptr<KVClient> dataWriter;
1034+ InitTestKVClient(TRANSPORT_CLIENT_WORKER_INDEX, remoteOwnerWriter, CLIENT_TIMEOUT_MS);
1035+ InitTestKVClient(DATA_WORKER_INDEX, dataWriter, CLIENT_TIMEOUT_MS);
1036+ const std::vector<std::string> values = { std::string(VALUE_SIZE, 'f'), std::string(VALUE_SIZE, 'g'),
1037+ std::string(MIXED_OVERSIZED_INLINE_VALUE_SIZE, 'h') };
1038+ DS_ASSERT_OK(writer_->Set(localOwnerKeys[0], values[0]));
1039+ DS_ASSERT_OK(writer_->Set(localOwnerKeys[1], values[0]));
1040+ DS_ASSERT_OK(dataWriter->Set(localOwnerKeys[2], values[1]));
1041+ DS_ASSERT_OK(remoteOwnerWriter->Set(remoteOwnerKeys[0], values[2]));
1042+ Optional<Buffer> warmup;
1043+ DS_ASSERT_OK(reader_->Get(localOwnerKeys[0], warmup));
1044+ 
1045+ MixedPathCounts before;
1046+ GetMixedPathCounts(before);
1047+ const std::vector<std::string> keys = { remoteOwnerKeys[1], localOwnerKeys[2], remoteOwnerKeys[0],
1048+ localOwnerKeys[1] };
1049+ std::vector<Optional<Buffer>> buffers;
1050+ DS_ASSERT_OK(reader_->Get(keys, buffers));
1051+ MixedPathCounts after;
1052+ GetMixedPathCounts(after);
1053+ 
1054+ AssertMixedPathBuffers(buffers, values);
1055+ ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport());
1056+ AssertMixedPathCounts(before, after);
1057+}
1058+ 
712// Regression: when local cache is disabled, dataPlacementPolicy is a write-only setting. Even when1059// Regression: when local cache is disabled, dataPlacementPolicy is a write-only setting. Even when
713// its same-node choice differs from the metadata owner, Get must enter the metadata-owner transport flow.1060// its same-node choice differs from the metadata owner, Get must enter the metadata-owner transport flow.
714TEST_F(KVClientTransportGetWithAllWorkersShmTest, GetIgnoresSameNodeWritePlacementPolicy)1061TEST_F(KVClientTransportGetWithAllWorkersShmTest, GetIgnoresSameNodeWritePlacementPolicy)
@@ -753,8 +1100,8 @@ TEST_F(KVClientTransportGetWithAllWorkersShmTest, GetIgnoresSameNodeWritePlaceme
753 ASSERT_EQ(after.queryAndGet, before.queryAndGet + 1);1100 ASSERT_EQ(after.queryAndGet, before.queryAndGet + 1);
754 ASSERT_EQ(nonOwnerGetAfter, nonOwnerGetBefore)1101 ASSERT_EQ(nonOwnerGetAfter, nonOwnerGetBefore)
755 << "Get must not probe a same-host non-owner before metadata-owner QueryAndGet";1102 << "Get must not probe a same-host non-owner before metadata-owner QueryAndGet";
756- ASSERT_EQ(metaOwnerGetAfter, metaOwnerGetBefore + 1);1103+ ASSERT_EQ(metaOwnerGetAfter, metaOwnerGetBefore);
757- ASSERT_EQ(after.workerOcGet, before.workerOcGet + 1);1104+ ASSERT_EQ(after.workerOcGet, before.workerOcGet);
758 ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");1105 ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");
759 }1106 }
760}1107}
@@ -827,7 +1174,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass
827 GetRpcCounts(after);1174 GetRpcCounts(after);
828 1175 
829 ASSERT_EQ(after.queryAndGet, before.queryAndGet + CONCURRENT_GET_COUNT);1176 ASSERT_EQ(after.queryAndGet, before.queryAndGet + CONCURRENT_GET_COUNT);
830- ASSERT_EQ(after.workerOcGet, before.workerOcGet + CONCURRENT_GET_COUNT);1177+ ASSERT_EQ(after.workerOcGet, before.workerOcGet);
831 ASSERT_EQ(after.getObjectRemote, before.getObjectRemote);1178 ASSERT_EQ(after.getObjectRemote, before.getObjectRemote);
832 ASSERT_EQ(after.batchGetObjectRemote, before.batchGetObjectRemote);1179 ASSERT_EQ(after.batchGetObjectRemote, before.batchGetObjectRemote);
833 ASSERT_EQ(after.registerShmClient, before.registerShmClient + 1);1180 ASSERT_EQ(after.registerShmClient, before.registerShmClient + 1);
@@ -840,7 +1187,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass
840 ASSERT_TRUE(reusedBuffer);1187 ASSERT_TRUE(reusedBuffer);
841 AssertBufferEqual(*reusedBuffer, value);1188 AssertBufferEqual(*reusedBuffer, value);
842 ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");1189 ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");
843- ASSERT_EQ(reused.workerOcGet, after.workerOcGet + 1);1190+ ASSERT_EQ(reused.workerOcGet, after.workerOcGet);
844 ASSERT_EQ(reused.registerShmClient, after.registerShmClient);1191 ASSERT_EQ(reused.registerShmClient, after.registerShmClient);
845 ASSERT_EQ(reused.getClientFd, after.getClientFd);1192 ASSERT_EQ(reused.getClientFd, after.getClientFd);
846 1193 
@@ -866,7 +1213,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass
866 AssertBufferEqual(*batchBuffers[i], batchValues[i]);1213 AssertBufferEqual(*batchBuffers[i], batchValues[i]);
867 }1214 }
868 ASSERT_EQ(afterBatch.queryAndGet, beforeBatch.queryAndGet + 1);1215 ASSERT_EQ(afterBatch.queryAndGet, beforeBatch.queryAndGet + 1);
869- ASSERT_EQ(afterBatch.workerOcGet, beforeBatch.workerOcGet + 1);1216+ ASSERT_EQ(afterBatch.workerOcGet, beforeBatch.workerOcGet);
870 ASSERT_EQ(afterBatch.getObjectRemote, beforeBatch.getObjectRemote);1217 ASSERT_EQ(afterBatch.getObjectRemote, beforeBatch.getObjectRemote);
871 ASSERT_EQ(afterBatch.batchGetObjectRemote, beforeBatch.batchGetObjectRemote);1218 ASSERT_EQ(afterBatch.batchGetObjectRemote, beforeBatch.batchGetObjectRemote);
872 ASSERT_EQ(afterBatch.registerShmClient, beforeBatch.registerShmClient);1219 ASSERT_EQ(afterBatch.registerShmClient, beforeBatch.registerShmClient);
@@ -890,7 +1237,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass
890 ASSERT_TRUE(postHeartbeatBuffer);1237 ASSERT_TRUE(postHeartbeatBuffer);
891 AssertBufferEqual(*postHeartbeatBuffer, value);1238 AssertBufferEqual(*postHeartbeatBuffer, value);
892 ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");1239 ASSERT_EQ(AccessTransportTracker::ToString(), "SHM");
893- ASSERT_EQ(postHeartbeat.workerOcGet, maintained.workerOcGet + 1);1240+ ASSERT_EQ(postHeartbeat.workerOcGet, maintained.workerOcGet);
894 ASSERT_EQ(postHeartbeat.registerShmClient, maintained.registerShmClient);1241 ASSERT_EQ(postHeartbeat.registerShmClient, maintained.registerShmClient);
895 ASSERT_EQ(postHeartbeat.getClientFd, maintained.getClientFd);1242 ASSERT_EQ(postHeartbeat.getClientFd, maintained.getClientFd);
896}1243}
@@ -946,56 +1293,6 @@ TEST_F(KVClientTransportGetDrainingRealUrmaTest, DrainingTargetUsesUb)
946 ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 2);1293 ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 2);
947}1294}
948 1295 
949-TEST_F(KVClientTransportGetScaleOutRealUrmaTest, RedirectedMetadataOwnerPrecedesClientSnapshot)
950-{
951- const auto keys = MakeRandomKeys(SCALE_OUT_KEY_COUNT);
952- const std::string value(VALUE_SIZE, 'u');
953- for (const auto &key : keys) {
954- DS_ASSERT_OK(writer_->Set(key, value));
955- }
956- ClusterTopologyPb initialTopology;
957- GetClusterTopologyPb(initialTopology);
958- 
959- const uint64_t pauseBaseline = inject::GetExecuteCount(HASH_RING_REFRESH_BEFORE_WAIT_INJECT);
960- DS_ASSERT_OK(inject::Set(HASH_RING_REFRESH_BEFORE_WAIT_INJECT, "1*pause()"));
961- Raii releaseRefresh([] { (void)inject::Clear(HASH_RING_REFRESH_BEFORE_WAIT_INJECT); });
962- const auto pauseDeadline = std::chrono::steady_clock::now() + std::chrono::seconds(10);
963- while (inject::GetExecuteCount(HASH_RING_REFRESH_BEFORE_WAIT_INJECT) == pauseBaseline
964- && std::chrono::steady_clock::now() < pauseDeadline) {
965- std::this_thread::sleep_for(std::chrono::milliseconds(50));
966- }
967- ASSERT_GT(inject::GetExecuteCount(HASH_RING_REFRESH_BEFORE_WAIT_INJECT), pauseBaseline);
968- 
969- HostPort masterAddress;
970- DS_ASSERT_OK(cluster_->GetWorkerAddr(META_OWNER_INDEX, masterAddress));
971- HostPort newWorkerAddress("127.0.0.1", GetFreePort());
972- DS_ASSERT_OK(cluster_->AddNode(masterAddress, newWorkerAddress.ToString(), GetFreePort()));
973- DS_ASSERT_OK(cluster_->WaitNodeReady(WORKER, SCALE_OUT_WORKER_INDEX, 30));
974- WaitAllMembersJoinClusterTopology(SCALE_OUT_WORKER_COUNT);
975- WaitClusterTopologyChange([](const ClusterTopologyPb &topology) { return !topology.has_active_batch(); });
976- 
977- ClusterTopologyPb finalTopology;
978- GetClusterTopologyPb(finalTopology);
979- auto redirectedKey = std::find_if(keys.begin(), keys.end(), [&](const std::string &key) {
980- return SelectMetadataOwner(initialTopology, key) != newWorkerAddress.ToString()
981- && SelectMetadataOwner(finalTopology, key) == newWorkerAddress.ToString();
982- });
983- ASSERT_NE(redirectedKey, keys.end());
984- 
985- DS_ASSERT_OK(cluster_->SetInjectAction(WORKER, SCALE_OUT_WORKER_INDEX, QUERY_AND_GET_INJECT, "call()"));
986- uint64_t redirectedQueryBaseline = 0;
987- DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(WORKER, SCALE_OUT_WORKER_INDEX, QUERY_AND_GET_INJECT,
988- redirectedQueryBaseline));
989- Optional<Buffer> buffer;
990- DS_ASSERT_OK(reader_->Get(*redirectedKey, buffer));
991- ASSERT_TRUE(buffer);
992- AssertBufferEqual(*buffer, value);
993- uint64_t redirectedQueryCount = 0;
994- DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(WORKER, SCALE_OUT_WORKER_INDEX, QUERY_AND_GET_INJECT,
995- redirectedQueryCount));
996- ASSERT_GT(redirectedQueryCount, redirectedQueryBaseline);
997- ASSERT_EQ(AccessTransportTracker::ToString(), "UB");
998-}
999 1296 
1000TEST_F(KVClientTransportGetWithShmTest, PinPendingSingleAndBatchReadOnlyGetUsePageableMemory)1297TEST_F(KVClientTransportGetWithShmTest, PinPendingSingleAndBatchReadOnlyGetUsePageableMemory)
1001{1298{
@@ -1204,39 +1501,28 @@ TEST_F(KVClientTransportGetTest, NonLocalMetaOwnerFallsBack)
1204 ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 1);1501 ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 1);
1205}1502}
1206 1503 
1207-class KVClientTransportGetInlineFailureTest : public KVClientTransportGetTest {1504+TEST_F(KVClientTransportGetTest, InlineEncodeFailureFallsBackPerKey)
1208-public:
1209- void SetClusterSetupOptions(ExternalClusterOptions &opts) override
1210- {
1211- KVClientTransportGetTest::SetClusterSetupOptions(opts);
1212- opts.injectActions += ";" + std::string(INLINE_READ_FAILURE_INJECT) + ":1*return(K_RUNTIME_ERROR)";
1213- }
1214-};
1215- 
1216-TEST_F(KVClientTransportGetInlineFailureTest, InlineReadFailureFallsBack)
1217{1505{
1218 std::vector<std::string> keys;1506 std::vector<std::string> keys;
1219- GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys);1507+ GetRealHashKeysToWorker(META_OWNER_INDEX, 2, keys);
1220- ASSERT_EQ(keys.size(), 1u);1508+ const std::vector<std::string> values = { std::string(VALUE_SIZE, 'f'), std::string(VALUE_SIZE, 'g') };
1221- const std::string &key = keys.front();1509+ DS_ASSERT_OK(writer_->Set(keys[0], values[0]));
1222- const std::string value(VALUE_SIZE, 'f');1510+ DS_ASSERT_OK(writer_->Set(keys[1], values[1]));
1223- DS_ASSERT_OK(writer_->Set(key, value));1511+ DS_ASSERT_OK(cluster_->SetInjectAction(
1512+ WORKER, META_OWNER_INDEX, QUERY_AND_GET_INLINE_FAILURE_INJECT, "1*return(K_RUNTIME_ERROR)"));
1513+ Raii clearInject([this] {
1514+ (void)cluster_->ClearInjectAction(WORKER, META_OWNER_INDEX, QUERY_AND_GET_INLINE_FAILURE_INJECT);
1515+ });
1224 1516 
1225- TransportRpcCounts before;1517+ std::vector<Optional<Buffer>> buffers;
1226- GetRpcCounts(before);1518+ DS_ASSERT_OK(reader_->Get(keys, buffers));
1227 1519 
1228- Optional<Buffer> buffer;1520+ ASSERT_EQ(buffers.size(), keys.size());
1229- Status rc = reader_->Get(key, buffer);1521+ for (size_t i = 0; i < buffers.size(); ++i) {
1230- 1522+ ASSERT_TRUE(buffers[i]);
1231- TransportRpcCounts after;1523+ AssertBufferEqual(*buffers[i], values[i]);
1232- GetRpcCounts(after);1524+ }
1233- 
1234- DS_ASSERT_OK(rc);
1235- ASSERT_TRUE(buffer);
1236- AssertBufferEqual(*buffer, value);
1237 ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport());1525 ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport());
1238- ASSERT_EQ(after.queryAndGet, before.queryAndGet + 1);
1239- ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 1);
1240}1526}
1241 1527 
1242// One key's data read fails while the others succeed; overall K_OK with the failed slot empty.1528// One key's data read fails while the others succeed; overall K_OK with the failed slot empty.
Mtests/ut/client/transport_test.cpp+96-411
@@ -240,8 +240,8 @@ std::vector<std::shared_ptr<ObjectBuffer>> MakeTransportBuffers(const HostPort &
240 return buffers;240 return buffers;
241}241}
242 242 
243-master::QueryAndGetResultPb *AddLocation(master::QueryAndGetRspPb &response, const std::string &key,243+QueryAndGetResultPb *AddLocation(QueryAndGetRspPb &response, const std::string &key,
244- const HostPort &address, uint64_t size = 4)244+ const HostPort &address, uint64_t size = 4)
245{245{
246 auto *result = response.add_results();246 auto *result = response.add_results();
247 auto *location = result->mutable_location();247 auto *location = result->mutable_location();
@@ -384,7 +384,7 @@ public:
384 return batchGetObjectStatus;384 return batchGetObjectStatus;
385 }385 }
386 386 
387- Status InvokeQueryAndGet(master::QueryAndGetReqPb &request, master::QueryAndGetRspPb &response,387+ Status InvokeQueryAndGet(QueryAndGetReqPb &request, QueryAndGetRspPb &response,
388 std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr) override388 std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr) override
389 {389 {
390 if (rpcDispatched != nullptr) {390 if (rpcDispatched != nullptr) {
@@ -545,8 +545,8 @@ public:
545 std::function<Status(BatchGetObjectRemoteReqPb &, BatchGetObjectRemoteRspPb &, std::vector<RpcMessage> &)>545 std::function<Status(BatchGetObjectRemoteReqPb &, BatchGetObjectRemoteRspPb &, std::vector<RpcMessage> &)>
546 batchGetHandler;546 batchGetHandler;
547 Status queryAndGetStatus = Status::OK();547 Status queryAndGetStatus = Status::OK();
548- std::vector<master::QueryAndGetReqPb> queryAndGetRequests;548+ std::vector<QueryAndGetReqPb> queryAndGetRequests;
549- std::function<Status(const HostPort &, const master::QueryAndGetReqPb &, master::QueryAndGetRspPb &,549+ std::function<Status(const HostPort &, const QueryAndGetReqPb &, QueryAndGetRspPb &,
550 std::vector<RpcMessage> &)>550 std::vector<RpcMessage> &)>
551 queryAndGetHandler;551 queryAndGetHandler;
552 std::function<void()> onInvoke;552 std::function<void()> onInvoke;
@@ -658,7 +658,7 @@ public:
658 HeartbeatReqPb invokedShmHeartbeatRequest;658 HeartbeatReqPb invokedShmHeartbeatRequest;
659 DisconnectClientReqPb invokedShmDisconnectRequest;659 DisconnectClientReqPb invokedShmDisconnectRequest;
660 BatchGetObjectRemoteReqPb invokedBatchGetRequest;660 BatchGetObjectRemoteReqPb invokedBatchGetRequest;
661- master::QueryAndGetReqPb invokedMetadataRequest;661+ QueryAndGetReqPb invokedMetadataRequest;
662 ExistReqPb invokedExistRequest;662 ExistReqPb invokedExistRequest;
663 GetHashRingReqPb invokedHashRingRequest;663 GetHashRingReqPb invokedHashRingRequest;
664 CreateReqPb invokedCreateRequest;664 CreateReqPb invokedCreateRequest;
@@ -737,8 +737,8 @@ protected:
737 return batchGetInvokeStatus;737 return batchGetInvokeStatus;
738 }738 }
739 739 
740- Status DoInvokeQueryAndGet(const RpcOptions &options, const master::QueryAndGetReqPb &request,740+ Status DoInvokeQueryAndGet(const RpcOptions &options, const QueryAndGetReqPb &request,
741- master::QueryAndGetRspPb &, std::vector<RpcMessage> &) override741+ QueryAndGetRspPb &, std::vector<RpcMessage> &) override
742 {742 {
743 ++metadataInvokeCount;743 ++metadataInvokeCount;
744 metadataRpcTimeout = options.GetTimeout();744 metadataRpcTimeout = options.GetTimeout();
@@ -999,9 +999,6 @@ public:
999 {999 {
1000 std::lock_guard<std::mutex> lock(mutex);1000 std::lock_guard<std::mutex> lock(mutex);
1001 ++rpcBuildCount;1001 ++rpcBuildCount;
1002- if (onCreateRpcClient) {
1003- onCreateRpcClient(address);
1004- }
1005 if (!rpcBuildStatuses.empty()) {1002 if (!rpcBuildStatuses.empty()) {
1006 Status rc = rpcBuildStatuses.front();1003 Status rc = rpcBuildStatuses.front();
1007 rpcBuildStatuses.erase(rpcBuildStatuses.begin());1004 rpcBuildStatuses.erase(rpcBuildStatuses.begin());
@@ -1087,10 +1084,9 @@ public:
1087 std::vector<std::vector<Status>> transporterMSetStatuses;1084 std::vector<std::vector<Status>> transporterMSetStatuses;
1088 std::vector<bool> transporterMSetPublishAttempted;1085 std::vector<bool> transporterMSetPublishAttempted;
1089 std::vector<std::shared_ptr<FakeTransporter>> builtTransporters;1086 std::vector<std::shared_ptr<FakeTransporter>> builtTransporters;
1090- std::function<Status(const HostPort &, const master::QueryAndGetReqPb &, master::QueryAndGetRspPb &,1087+ std::function<Status(const HostPort &, const QueryAndGetReqPb &, QueryAndGetRspPb &,
1091 std::vector<RpcMessage> &)>1088 std::vector<RpcMessage> &)>
1092 queryAndGetHandler;1089 queryAndGetHandler;
1093- std::function<void(const HostPort &)> onCreateRpcClient;
1094 std::function<void(const HostPort &, FakeTransporter &)> configureTransporter;1090 std::function<void(const HostPort &, FakeTransporter &)> configureTransporter;
1095 std::mutex mutex;1091 std::mutex mutex;
1096};1092};
@@ -1101,7 +1097,8 @@ public:
1101 {1097 {
1102 }1098 }
1103 1099 
1104- Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items) override1100+ Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items,
1101+ std::shared_ptr<const TransportReadContext>) override
1105 {1102 {
1106 {1103 {
1107 std::lock_guard<std::mutex> lock(mutex);1104 std::lock_guard<std::mutex> lock(mutex);
@@ -1485,16 +1482,16 @@ TEST(WorkerRpcClientTest, SignsFinalReadRequestsBeforeRpc)
1485 EXPECT_FALSE(client.invokedDataRequest.signature().empty());1482 EXPECT_FALSE(client.invokedDataRequest.signature().empty());
1486 EXPECT_EQ(client.invokedDataRequest.urma_info().seg_va(), 123u);1483 EXPECT_EQ(client.invokedDataRequest.urma_info().seg_va(), 123u);
1487 1484 
1488- master::QueryAndGetReqPb metadataRequest;1485+ QueryAndGetReqPb metadataRequest;
1489 metadataRequest.add_object_keys("key");1486 metadataRequest.add_object_keys("key");
1490- metadataRequest.set_redirect(true);1487+ metadataRequest.mutable_data_request()->mutable_tcp();
1491- master::QueryAndGetRspPb metadataResponse;1488+ QueryAndGetRspPb metadataResponse;
1492 std::vector<RpcMessage> metadataPayloads;1489 std::vector<RpcMessage> metadataPayloads;
1493 ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk());1490 ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk());
1494 EXPECT_EQ(client.metadataInvokeCount, 1);1491 EXPECT_EQ(client.metadataInvokeCount, 1);
1495 EXPECT_EQ(client.invokedMetadataRequest.access_key(), "access-1");1492 EXPECT_EQ(client.invokedMetadataRequest.access_key(), "access-1");
1496 EXPECT_FALSE(client.invokedMetadataRequest.signature().empty());1493 EXPECT_FALSE(client.invokedMetadataRequest.signature().empty());
1497- EXPECT_TRUE(client.invokedMetadataRequest.redirect());1494+ EXPECT_TRUE(client.invokedMetadataRequest.data_request().has_tcp());
1498 1495 
1499 ExistReqPb existRequest;1496 ExistReqPb existRequest;
1500 existRequest.add_object_keys("key");1497 existRequest.add_object_keys("key");
@@ -1762,9 +1759,9 @@ TEST(WorkerRpcClientTest, BoundsRpcTimeoutByApiDeadline)
1762 EXPECT_GT(client.dataRpcTimeout, 0);1759 EXPECT_GT(client.dataRpcTimeout, 0);
1763 EXPECT_LE(client.dataRpcTimeout, 100);1760 EXPECT_LE(client.dataRpcTimeout, 100);
1764 1761 
1765- master::QueryAndGetReqPb metadataRequest;1762+ QueryAndGetReqPb metadataRequest;
1766 metadataRequest.add_object_keys("key");1763 metadataRequest.add_object_keys("key");
1767- master::QueryAndGetRspPb metadataResponse;1764+ QueryAndGetRspPb metadataResponse;
1768 std::vector<RpcMessage> metadataPayloads;1765 std::vector<RpcMessage> metadataPayloads;
1769 ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk());1766 ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk());
1770 EXPECT_EQ(client.metadataInvokeCount, 1);1767 EXPECT_EQ(client.metadataInvokeCount, 1);
@@ -1793,8 +1790,8 @@ TEST(WorkerRpcClientTest, ExpiredApiDeadlineDoesNotSendRpc)
1793 EXPECT_EQ(client.InvokeGetObject(dataRequest, dataResponse, payloads).GetCode(), K_RPC_DEADLINE_EXCEEDED);1790 EXPECT_EQ(client.InvokeGetObject(dataRequest, dataResponse, payloads).GetCode(), K_RPC_DEADLINE_EXCEEDED);
1794 EXPECT_EQ(client.getObjectInvokeCount, 0);1791 EXPECT_EQ(client.getObjectInvokeCount, 0);
1795 1792 
1796- master::QueryAndGetReqPb metadataRequest;1793+ QueryAndGetReqPb metadataRequest;
1797- master::QueryAndGetRspPb metadataResponse;1794+ QueryAndGetRspPb metadataResponse;
1798 std::vector<RpcMessage> metadataPayloads;1795 std::vector<RpcMessage> metadataPayloads;
1799 bool metadataRpcDispatched = true;1796 bool metadataRpcDispatched = true;
1800 EXPECT_EQ(1797 EXPECT_EQ(
@@ -2537,77 +2534,27 @@ TEST(DataPlaneManagerTest, ReconcileReleasesMapLockBeforeSlowDataPlaneClose)
2537 reconcileThread.join();2534 reconcileThread.join();
2538}2535}
2539 2536 
2540-TEST(ObjectMetadataClientTest, RestoresOrderAcrossPartialRedirectsAndDuplicateKeys)
2541-{
2542- ApiDeadlineGuard deadline(1000);
2543- auto manager = std::make_shared<FakeDataPlaneManager>();
2544- std::vector<HostPort> calls;
2545- manager->queryAndGetHandler = [&calls](const HostPort &address, const master::QueryAndGetReqPb &request,
2546- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2547- calls.push_back(address);
2548- if (address == MakeAddress(41)) {
2549- EXPECT_EQ(request.object_keys_size(), 4);
2550- AddLocation(response, "a", MakeAddress(51));
2551- AddLocation(response, "a", MakeAddress(52));
2552- auto *redirectC = response.add_info();
2553- redirectC->set_redirect_meta_address(MakeAddress(43).ToString());
2554- redirectC->add_change_meta_ids("c");
2555- auto *redirectB = response.add_info();
2556- redirectB->set_redirect_meta_address(MakeAddress(42).ToString());
2557- redirectB->add_change_meta_ids("b");
2558- } else if (address == MakeAddress(42)) {
2559- EXPECT_EQ(request.object_keys_size(), 1);
2560- EXPECT_EQ(request.object_keys(0), "b");
2561- EXPECT_TRUE(request.redirect());
2562- AddLocation(response, "b", MakeAddress(53));
2563- } else {
2564- EXPECT_EQ(address, MakeAddress(43));
2565- EXPECT_EQ(request.object_keys_size(), 1);
2566- EXPECT_EQ(request.object_keys(0), "c");
2567- EXPECT_TRUE(request.redirect());
2568- AddLocation(response, "c", MakeAddress(54));
2569- }
2570- return Status::OK();
2571- };
2572- ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2573- std::vector<ObjectReadItem> inputs{ { 0, "a", MakeAddress(41) }, { 1, "b", MakeAddress(41) },
2574- { 2, "a", MakeAddress(41) }, { 3, "c", MakeAddress(41) } };
2575- auto results = MakeMetadataItems(inputs);
2576- auto batch = MakeMetadataBatch(results);
2577- 
2578- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());
2579- ASSERT_EQ(results.size(), 4u);
2580- for (size_t i = 0; i < results.size(); ++i) {
2581- EXPECT_TRUE(results[i].status.IsOk());
2582- EXPECT_EQ(results[i].objectKey, inputs[i].objectKey);
2583- }
2584- ASSERT_EQ(calls.size(), 3u);
2585- EXPECT_EQ(calls[0], MakeAddress(41));
2586- EXPECT_EQ(calls[1], MakeAddress(42));
2587- EXPECT_EQ(calls[2], MakeAddress(43));
2588-}
2589- 
2590TEST(ObjectMetadataClientTest, RejectsResultCountMismatchBeforeIndexedAccess)2537TEST(ObjectMetadataClientTest, RejectsResultCountMismatchBeforeIndexedAccess)
2591{2538{
2592 ApiDeadlineGuard deadline(1000);2539 ApiDeadlineGuard deadline(1000);
2593 auto manager = std::make_shared<FakeDataPlaneManager>();2540 auto manager = std::make_shared<FakeDataPlaneManager>();
2594- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &,2541+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &,
2595- master::QueryAndGetRspPb &, std::vector<RpcMessage> &) {2542+ QueryAndGetRspPb &, std::vector<RpcMessage> &) {
2596 return Status::OK();2543 return Status::OK();
2597 };2544 };
2598 ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());2545 ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2599 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2546 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
2600 auto batch = MakeMetadataBatch(results);2547 auto batch = MakeMetadataBatch(results);
2601 2548 
2602- EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch).GetCode(), K_RUNTIME_ERROR);2549+ EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).GetCode(), K_RUNTIME_ERROR);
2603}2550}
2604 2551 
2605TEST(ObjectMetadataClientTest, EmptyLocationsFailOnlyTheirInputItem)2552TEST(ObjectMetadataClientTest, EmptyLocationsFailOnlyTheirInputItem)
2606{2553{
2607 ApiDeadlineGuard deadline(1000);2554 ApiDeadlineGuard deadline(1000);
2608 auto manager = std::make_shared<FakeDataPlaneManager>();2555 auto manager = std::make_shared<FakeDataPlaneManager>();
2609- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &,2556+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &,
2610- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {2557+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2611 response.add_results()->mutable_location()->set_object_key("missing");2558 response.add_results()->mutable_location()->set_object_key("missing");
2612 AddLocation(response, "present", MakeAddress(51));2559 AddLocation(response, "present", MakeAddress(51));
2613 return Status::OK();2560 return Status::OK();
@@ -2617,311 +2564,19 @@ TEST(ObjectMetadataClientTest, EmptyLocationsFailOnlyTheirInputItem)
2617 { 1, "present", MakeAddress(41) } });2564 { 1, "present", MakeAddress(41) } });
2618 auto batch = MakeMetadataBatch(results);2565 auto batch = MakeMetadataBatch(results);
2619 2566 
2620- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2567+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
2621 ASSERT_EQ(results.size(), 2u);2568 ASSERT_EQ(results.size(), 2u);
2622 EXPECT_EQ(results[0].status.GetCode(), K_NOT_FOUND);2569 EXPECT_EQ(results[0].status.GetCode(), K_NOT_FOUND);
2623 EXPECT_TRUE(results[1].status.IsOk());2570 EXPECT_TRUE(results[1].status.IsOk());
2624}2571}
2625 2572 
2626-TEST(ObjectMetadataClientTest, RetriesMetaMovingWithTheSameKeyGroup)
2627-{
2628- ApiDeadlineGuard deadline(1000);
2629- auto manager = std::make_shared<FakeDataPlaneManager>();
2630- int invokeCount = 0;
2631- manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &request,
2632- master::QueryAndGetRspPb &response,
2633- std::vector<RpcMessage> &) {
2634- EXPECT_EQ(request.object_keys_size(), 2);
2635- if (++invokeCount == 1) {
2636- response.set_meta_is_moving(true);
2637- } else {
2638- AddLocation(response, "a", MakeAddress(51));
2639- AddLocation(response, "b", MakeAddress(52));
2640- }
2641- return Status::OK();
2642- };
2643- ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2644- auto results = MakeMetadataItems({ { 0, "a", MakeAddress(41) }, { 1, "b", MakeAddress(41) } });
2645- auto batch = MakeMetadataBatch(results);
2646- 
2647- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());
2648- EXPECT_EQ(invokeCount, 2);
2649- ASSERT_EQ(results.size(), 2u);
2650- EXPECT_TRUE(results[0].status.IsOk());
2651- EXPECT_TRUE(results[1].status.IsOk());
2652-}
2653- 
2654-TEST(ObjectMetadataClientTest, FollowsTwoRedirects)
2655-{
2656- ApiDeadlineGuard deadline(1000);
2657- auto manager = std::make_shared<FakeDataPlaneManager>();
2658- std::vector<HostPort> calls;
2659- manager->queryAndGetHandler = [&calls](const HostPort &address, const master::QueryAndGetReqPb &request,
2660- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2661- calls.push_back(address);
2662- EXPECT_TRUE(request.redirect());
2663- if (address == MakeAddress(43)) {
2664- AddLocation(response, "key", MakeAddress(51));
2665- } else {
2666- auto *redirect = response.add_info();
2667- redirect->set_redirect_meta_address(
2668- (address == MakeAddress(41) ? MakeAddress(42) : MakeAddress(43)).ToString());
2669- redirect->add_change_meta_ids("key");
2670- }
2671- return Status::OK();
2672- };
2673- ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2674- auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
2675- auto batch = MakeMetadataBatch(results);
2676- 
2677- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());
2678- ASSERT_EQ(results.size(), 1u);
2679- EXPECT_TRUE(results[0].status.IsOk());
2680- EXPECT_EQ(calls, std::vector<HostPort>({ MakeAddress(41), MakeAddress(42), MakeAddress(43) }));
2681-}
2682- 
2683-TEST(ObjectMetadataClientTest, RedirectedOwnerAbsentFromSnapshotUsesMetadataOnlyRpcAndRequestsRefresh)
2684-{
2685- ApiDeadlineGuard deadline(1000);
2686- const HostPort currentOwner = MakeAddress(41);
2687- const HostPort redirectedOwner = MakeAddress(42);
2688- auto manager = std::make_shared<FakeDataPlaneManager>();
2689- WorkerSnapshot snapshot;
2690- snapshot.ringVersion = 10;
2691- snapshot.remoteTransportAddrs.push_back(currentOwner);
2692- ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk());
2693- 
2694- manager->queryAndGetHandler = [currentOwner, redirectedOwner](
2695- const HostPort &address, const master::QueryAndGetReqPb &request,
2696- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2697- if (address == currentOwner) {
2698- EXPECT_TRUE(request.has_data_request());
2699- EXPECT_TRUE(request.data_request().has_ub());
2700- auto *redirect = response.add_info();
2701- redirect->set_redirect_meta_address(redirectedOwner.ToString());
2702- redirect->add_change_meta_ids("key");
2703- redirect->set_topology_version(11);
2704- return Status::OK();
2705- }
2706- EXPECT_EQ(address, redirectedOwner);
2707- EXPECT_FALSE(request.has_data_request());
2708- AddLocation(response, "key", MakeAddress(51), 6);
2709- return Status::OK();
2710- };
2711- std::vector<std::pair<HostPort, Status>> failures;
2712- auto bufferProvider = std::make_shared<FakeUbBufferProvider>();
2713- ObjectMetadataClient metadata(
2714- manager, std::make_shared<DeadlineRetry>(),
2715- std::make_shared<FixedTransportAdvisor>(TransportHint::UB_CANDIDATE), bufferProvider, 16,
2716- [&failures](const HostPort &address, const Status &status) { failures.emplace_back(address, status); });
2717- auto results = MakeMetadataItems({ { 0, "key", currentOwner } });
2718- auto batch = MakeMetadataBatch(results);
2719- 
2720- ASSERT_TRUE(metadata.QueryAndGet(currentOwner, batch).IsOk());
2721- ASSERT_EQ(results.size(), 1u);
2722- EXPECT_TRUE(results[0].status.IsOk());
2723- EXPECT_FALSE(results[0].inlineData.has_value());
2724- ASSERT_EQ(failures.size(), 1u);
2725- EXPECT_EQ(failures[0].first, redirectedOwner);
2726- EXPECT_TRUE(IsTransportSnapshotStaleLocation(failures[0].second));
2727- EXPECT_EQ(manager->rpcBuildCount, 2);
2728- EXPECT_EQ(manager->transportBuildCount, 1);
2729- 
2730- std::shared_ptr<IDataTransporter> transporter;
2731- EXPECT_EQ(manager->GetOrCreate(redirectedOwner, TransportHint::UB_CANDIDATE, transporter).GetCode(), K_NOT_READY);
2732-}
2733- 
2734-TEST(ObjectMetadataClientTest, RedirectVersionNotNewerThanSnapshotRemainsRejected)
2735-{
2736- for (uint64_t redirectVersion : { 0, 9, 10 }) {
2737- ApiDeadlineGuard deadline(1000);
2738- const HostPort currentOwner = MakeAddress(41);
2739- const HostPort redirectedOwner = MakeAddress(42);
2740- auto manager = std::make_shared<FakeDataPlaneManager>();
2741- WorkerSnapshot snapshot;
2742- snapshot.ringVersion = 10;
2743- snapshot.remoteTransportAddrs.push_back(currentOwner);
2744- ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk());
2745- 
2746- int redirectedInvocations = 0;
2747- manager->queryAndGetHandler = [currentOwner, redirectedOwner, redirectVersion, &redirectedInvocations](
2748- const HostPort &address, const master::QueryAndGetReqPb &,
2749- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2750- if (address == currentOwner) {
2751- auto *redirect = response.add_info();
2752- redirect->set_redirect_meta_address(redirectedOwner.ToString());
2753- redirect->add_change_meta_ids("key");
2754- redirect->set_topology_version(redirectVersion);
2755- return Status::OK();
2756- }
2757- ++redirectedInvocations;
2758- AddLocation(response, "key", MakeAddress(51));
2759- return Status::OK();
2760- };
2761- ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2762- auto results = MakeMetadataItems({ { 0, "key", currentOwner } });
2763- 
2764- ASSERT_TRUE(metadata.QueryAndGet(currentOwner, MakeMetadataBatch(results)).IsOk());
2765- EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status));
2766- EXPECT_EQ(redirectedInvocations, 0);
2767- EXPECT_EQ(manager->rpcBuildCount, 1);
2768- }
2769-}
2770- 
2771-TEST(ObjectMetadataClientTest, NewerRedirectIsRejectedIfSnapshotAdvancesDuringRpcCreation)
2772-{
2773- ApiDeadlineGuard deadline(1000);
2774- const HostPort currentOwner = MakeAddress(41);
2775- const HostPort redirectedOwner = MakeAddress(42);
2776- auto manager = std::make_shared<FakeDataPlaneManager>();
2777- WorkerSnapshot snapshot;
2778- snapshot.ringVersion = 10;
2779- snapshot.remoteTransportAddrs.push_back(currentOwner);
2780- ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk());
2781- 
2782- int redirectedInvocations = 0;
2783- manager->queryAndGetHandler = [currentOwner, redirectedOwner, &redirectedInvocations](
2784- const HostPort &address, const master::QueryAndGetReqPb &,
2785- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2786- if (address == currentOwner) {
2787- auto *redirect = response.add_info();
2788- redirect->set_redirect_meta_address(redirectedOwner.ToString());
2789- redirect->add_change_meta_ids("key");
2790- redirect->set_topology_version(11);
2791- return Status::OK();
2792- }
2793- ++redirectedInvocations;
2794- AddLocation(response, "key", MakeAddress(51));
2795- return Status::OK();
2796- };
2797- manager->onCreateRpcClient = [manager, currentOwner, redirectedOwner](const HostPort &address) {
2798- if (address != redirectedOwner) {
2799- return;
2800- }
2801- WorkerSnapshot latest;
2802- latest.ringVersion = 12;
2803- latest.remoteTransportAddrs.push_back(currentOwner);
2804- ASSERT_TRUE(manager->UpdateWorkerSnapshot(latest).IsOk());
2805- };
2806- ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2807- auto results = MakeMetadataItems({ { 0, "key", currentOwner } });
2808- 
2809- ASSERT_TRUE(metadata.QueryAndGet(currentOwner, MakeMetadataBatch(results)).IsOk());
2810- EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status));
2811- EXPECT_EQ(redirectedInvocations, 0);
2812- EXPECT_EQ(manager->rpcBuildCount, 2);
2813-}
2814- 
2815-TEST(ObjectMetadataClientTest, RedirectChainRejectsTopologyVersionRollback)
2816-{
2817- constexpr uint64_t clientSnapshotVersion = 10;
2818- constexpr uint64_t firstRedirectVersion = 12;
2819- const std::vector<std::pair<uint64_t, bool>> cases = {
2820- { 11, false },
2821- { 12, true },
2822- { 13, true },
2823- };
2824- for (const auto &[nextRedirectVersion, shouldFollow] : cases) {
2825- ApiDeadlineGuard deadline(1000);
2826- const HostPort initialOwner = MakeAddress(41);
2827- const HostPort firstRedirectOwner = MakeAddress(42);
2828- const HostPort secondRedirectOwner = MakeAddress(43);
2829- auto manager = std::make_shared<FakeDataPlaneManager>();
2830- WorkerSnapshot snapshot;
2831- snapshot.ringVersion = clientSnapshotVersion;
2832- snapshot.remoteTransportAddrs.push_back(initialOwner);
2833- ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk());
2834- 
2835- std::vector<HostPort> calls;
2836- manager->queryAndGetHandler = [initialOwner, firstRedirectOwner, secondRedirectOwner,
2837- firstRedirectVersion, nextRedirectVersion, &calls](
2838- const HostPort &address, const master::QueryAndGetReqPb &,
2839- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2840- calls.push_back(address);
2841- if (address == secondRedirectOwner) {
2842- AddLocation(response, "key", MakeAddress(51));
2843- return Status::OK();
2844- }
2845- auto *redirect = response.add_info();
2846- redirect->set_redirect_meta_address(
2847- (address == initialOwner ? firstRedirectOwner : secondRedirectOwner).ToString());
2848- redirect->add_change_meta_ids("key");
2849- redirect->set_topology_version(
2850- address == initialOwner ? firstRedirectVersion : nextRedirectVersion);
2851- return Status::OK();
2852- };
2853- ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2854- auto results = MakeMetadataItems({ { 0, "key", initialOwner } });
2855- 
2856- ASSERT_TRUE(metadata.QueryAndGet(initialOwner, MakeMetadataBatch(results)).IsOk());
2857- if (shouldFollow) {
2858- EXPECT_TRUE(results[0].status.IsOk());
2859- EXPECT_EQ(calls, std::vector<HostPort>({ initialOwner, firstRedirectOwner, secondRedirectOwner }));
2860- EXPECT_EQ(manager->rpcBuildCount, 3);
2861- } else {
2862- EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status));
2863- EXPECT_EQ(calls, std::vector<HostPort>({ initialOwner, firstRedirectOwner }));
2864- EXPECT_EQ(manager->rpcBuildCount, 2);
2865- }
2866- }
2867-}
2868- 
2869-TEST(ObjectMetadataClientTest, InitialOwnerAbsentFromSnapshotRemainsRejected)
2870-{
2871- ApiDeadlineGuard deadline(1000);
2872- const HostPort initialOwner = MakeAddress(41);
2873- auto manager = std::make_shared<FakeDataPlaneManager>();
2874- WorkerSnapshot snapshot;
2875- snapshot.ringVersion = 10;
2876- snapshot.remoteTransportAddrs.push_back(MakeAddress(42));
2877- ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk());
2878- ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
2879- auto results = MakeMetadataItems({ { 0, "key", initialOwner } });
2880- auto batch = MakeMetadataBatch(results);
2881- 
2882- const Status rc = metadata.QueryAndGet(initialOwner, batch);
2883- 
2884- EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc));
2885- EXPECT_EQ(manager->rpcBuildCount, 0);
2886- EXPECT_EQ(manager->transportBuildCount, 0);
2887-}
2888- 
2889-TEST(ObjectMetadataClientTest, ReportsRedirectTargetAccessFailure)
2890-{
2891- ApiDeadlineGuard deadline(1000);
2892- auto manager = std::make_shared<FakeDataPlaneManager>();
2893- manager->queryAndGetHandler = [](const HostPort &address, const master::QueryAndGetReqPb &,
2894- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2895- if (address == MakeAddress(41)) {
2896- auto *redirect = response.add_info();
2897- redirect->set_redirect_meta_address(MakeAddress(42).ToString());
2898- redirect->add_change_meta_ids("key");
2899- return Status::OK();
2900- }
2901- return Status(K_RPC_PEER_DEAD, "redirect target is unavailable");
2902- };
2903- std::vector<std::pair<HostPort, Status>> failures;
2904- ObjectMetadataClient metadata(
2905- manager, std::make_shared<DeadlineRetry>(), nullptr, nullptr, 0,
2906- [&failures](const HostPort &address, const Status &status) { failures.emplace_back(address, status); });
2907- auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
2908- auto batch = MakeMetadataBatch(results);
2909- 
2910- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());
2911- ASSERT_EQ(results.size(), 1u);
2912- EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status));
2913- ASSERT_EQ(failures.size(), 1u);
2914- EXPECT_EQ(failures[0].first, MakeAddress(42));
2915- EXPECT_EQ(failures[0].second.GetCode(), K_RPC_PEER_DEAD);
2916-}
2917- 
2918TEST(ObjectMetadataClientTest, DoesNotReportDeadlineExpiredBeforeAccess)2573TEST(ObjectMetadataClientTest, DoesNotReportDeadlineExpiredBeforeAccess)
2919{2574{
2920 ApiDeadlineGuard deadline(100, InUs{});2575 ApiDeadlineGuard deadline(100, InUs{});
2921 auto manager = std::make_shared<FakeDataPlaneManager>();2576 auto manager = std::make_shared<FakeDataPlaneManager>();
2922 size_t invokeCount = 0;2577 size_t invokeCount = 0;
2923- manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &,2578+ manager->queryAndGetHandler = [&invokeCount](const HostPort &, const QueryAndGetReqPb &,
2924- master::QueryAndGetRspPb &, std::vector<RpcMessage> &) {2579+ QueryAndGetRspPb &, std::vector<RpcMessage> &) {
2925 ++invokeCount;2580 ++invokeCount;
2926 return Status::OK();2581 return Status::OK();
2927 };2582 };
@@ -2932,7 +2587,7 @@ TEST(ObjectMetadataClientTest, DoesNotReportDeadlineExpiredBeforeAccess)
2932 auto batch = MakeMetadataBatch(results);2587 auto batch = MakeMetadataBatch(results);
2933 std::this_thread::sleep_for(std::chrono::milliseconds(1));2588 std::this_thread::sleep_for(std::chrono::milliseconds(1));
2934 2589 
2935- EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch).GetCode(), K_RPC_DEADLINE_EXCEEDED);2590+ EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).GetCode(), K_RPC_DEADLINE_EXCEEDED);
2936 EXPECT_EQ(invokeCount, 0u);2591 EXPECT_EQ(invokeCount, 0u);
2937 EXPECT_EQ(failureCount, 0u);2592 EXPECT_EQ(failureCount, 0u);
2938}2593}
@@ -2949,7 +2604,7 @@ TEST(ObjectMetadataClientTest, ConnectionFailureRequestsRerouteWithoutFixedOwner
2949 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2604 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
2950 auto batch = MakeMetadataBatch(results);2605 auto batch = MakeMetadataBatch(results);
2951 2606 
2952- const auto rc = metadata.QueryAndGet(MakeAddress(41), batch);2607+ const auto rc = metadata.QueryAndGet(MakeAddress(41), batch, nullptr);
2953 2608 
2954 EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc));2609 EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc));
2955 EXPECT_EQ(manager->rpcBuildCount, 1);2610 EXPECT_EQ(manager->rpcBuildCount, 1);
@@ -2963,8 +2618,8 @@ TEST(ObjectMetadataClientTest, DispatchedDeadlineRequestsRerouteWithoutFixedOwne
2963 ApiDeadlineGuard deadline(1000);2618 ApiDeadlineGuard deadline(1000);
2964 auto manager = std::make_shared<FakeDataPlaneManager>();2619 auto manager = std::make_shared<FakeDataPlaneManager>();
2965 int invokeCount = 0;2620 int invokeCount = 0;
2966- manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &,2621+ manager->queryAndGetHandler = [&invokeCount](const HostPort &, const QueryAndGetReqPb &,
2967- master::QueryAndGetRspPb &, std::vector<RpcMessage> &) {2622+ QueryAndGetRspPb &, std::vector<RpcMessage> &) {
2968 ++invokeCount;2623 ++invokeCount;
2969 return Status(K_RPC_DEADLINE_EXCEEDED, "metadata owner deadline");2624 return Status(K_RPC_DEADLINE_EXCEEDED, "metadata owner deadline");
2970 };2625 };
@@ -2975,7 +2630,7 @@ TEST(ObjectMetadataClientTest, DispatchedDeadlineRequestsRerouteWithoutFixedOwne
2975 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2630 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
2976 auto batch = MakeMetadataBatch(results);2631 auto batch = MakeMetadataBatch(results);
2977 2632 
2978- const auto rc = metadata.QueryAndGet(MakeAddress(41), batch);2633+ const auto rc = metadata.QueryAndGet(MakeAddress(41), batch, nullptr);
2979 2634 
2980 EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc));2635 EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc));
2981 EXPECT_EQ(invokeCount, 1);2636 EXPECT_EQ(invokeCount, 1);
@@ -2989,8 +2644,8 @@ TEST(ObjectMetadataClientTest, PeerDeadTearsDownWithoutRetry)
2989 ApiDeadlineGuard deadline(1000);2644 ApiDeadlineGuard deadline(1000);
2990 auto manager = std::make_shared<FakeDataPlaneManager>();2645 auto manager = std::make_shared<FakeDataPlaneManager>();
2991 int invokeCount = 0;2646 int invokeCount = 0;
2992- manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &,2647+ manager->queryAndGetHandler = [&invokeCount](const HostPort &, const QueryAndGetReqPb &,
2993- master::QueryAndGetRspPb &, std::vector<RpcMessage> &) {2648+ QueryAndGetRspPb &, std::vector<RpcMessage> &) {
2994 ++invokeCount;2649 ++invokeCount;
2995 return Status(K_RPC_PEER_DEAD, "peer dead");2650 return Status(K_RPC_PEER_DEAD, "peer dead");
2996 };2651 };
@@ -2998,16 +2653,16 @@ TEST(ObjectMetadataClientTest, PeerDeadTearsDownWithoutRetry)
2998 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2653 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
2999 auto batch = MakeMetadataBatch(results);2654 auto batch = MakeMetadataBatch(results);
3000 2655 
3001- EXPECT_TRUE(IsTransportSnapshotStaleLocation(metadata.QueryAndGet(MakeAddress(41), batch)));2656+ EXPECT_TRUE(IsTransportSnapshotStaleLocation(metadata.QueryAndGet(MakeAddress(41), batch, nullptr)));
3002 EXPECT_EQ(invokeCount, 1);2657 EXPECT_EQ(invokeCount, 1);
3003 EXPECT_EQ(manager->rpcBuildCount, 1);2658 EXPECT_EQ(manager->rpcBuildCount, 1);
3004 2659 
3005- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &,2660+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &,
3006- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {2661+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
3007 AddLocation(response, "key", MakeAddress(51));2662 AddLocation(response, "key", MakeAddress(51));
3008 return Status::OK();2663 return Status::OK();
3009 };2664 };
3010- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2665+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3011 EXPECT_EQ(manager->rpcBuildCount, 2);2666 EXPECT_EQ(manager->rpcBuildCount, 2);
3012}2667}
3013 2668 
@@ -3015,15 +2670,15 @@ TEST(ObjectMetadataClientTest, MetadataAndDataReuseOneEndpointRpcClient)
3015{2670{
3016 ApiDeadlineGuard deadline(1000);2671 ApiDeadlineGuard deadline(1000);
3017 auto manager = std::make_shared<FakeDataPlaneManager>();2672 auto manager = std::make_shared<FakeDataPlaneManager>();
3018- manager->queryAndGetHandler = [](const HostPort &address, const master::QueryAndGetReqPb &,2673+ manager->queryAndGetHandler = [](const HostPort &address, const QueryAndGetReqPb &,
3019- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {2674+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
3020 AddLocation(response, "key", address);2675 AddLocation(response, "key", address);
3021 return Status::OK();2676 return Status::OK();
3022 };2677 };
3023 ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());2678 ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>());
3024 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2679 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3025 auto batch = MakeMetadataBatch(results);2680 auto batch = MakeMetadataBatch(results);
3026- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2681+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3027 std::shared_ptr<IDataTransporter> transporter;2682 std::shared_ptr<IDataTransporter> transporter;
3028 2683 
3029 ASSERT_TRUE(manager->GetOrCreate(MakeAddress(41), TransportHint::TCP_ONLY, transporter).IsOk());2684 ASSERT_TRUE(manager->GetOrCreate(MakeAddress(41), TransportHint::TCP_ONLY, transporter).IsOk());
@@ -3035,8 +2690,8 @@ TEST(ObjectMetadataClientTest, TcpInlineDataMovesRpcPayloadIntoMetadataResult)
3035{2690{
3036 ApiDeadlineGuard deadline(1000);2691 ApiDeadlineGuard deadline(1000);
3037 auto manager = std::make_shared<FakeDataPlaneManager>();2692 auto manager = std::make_shared<FakeDataPlaneManager>();
3038- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request,2693+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request,
3039- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) {2694+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) {
3040 EXPECT_EQ(request.object_keys_size(), 1);2695 EXPECT_EQ(request.object_keys_size(), 1);
3041 EXPECT_TRUE(request.has_data_request());2696 EXPECT_TRUE(request.has_data_request());
3042 EXPECT_TRUE(request.data_request().has_tcp());2697 EXPECT_TRUE(request.data_request().has_tcp());
@@ -3052,7 +2707,7 @@ TEST(ObjectMetadataClientTest, TcpInlineDataMovesRpcPayloadIntoMetadataResult)
3052 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2707 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3053 auto batch = MakeMetadataBatch(results);2708 auto batch = MakeMetadataBatch(results);
3054 2709 
3055- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2710+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3056 ASSERT_EQ(results.size(), 1u);2711 ASSERT_EQ(results.size(), 1u);
3057 ASSERT_TRUE(results[0].inlineData.has_value());2712 ASSERT_TRUE(results[0].inlineData.has_value());
3058 auto &data = *results[0].inlineData;2713 auto &data = *results[0].inlineData;
@@ -3069,8 +2724,8 @@ TEST(ObjectMetadataClientTest, MissingTcpInlineMarkerFallsBackToReplicaRead)
3069{2724{
3070 ApiDeadlineGuard deadline(1000);2725 ApiDeadlineGuard deadline(1000);
3071 auto manager = std::make_shared<FakeDataPlaneManager>();2726 auto manager = std::make_shared<FakeDataPlaneManager>();
3072- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request,2727+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request,
3073- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {2728+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
3074 EXPECT_TRUE(request.has_data_request());2729 EXPECT_TRUE(request.has_data_request());
3075 EXPECT_TRUE(request.data_request().has_tcp());2730 EXPECT_TRUE(request.data_request().has_tcp());
3076 AddLocation(response, "key", MakeAddress(51), 6);2731 AddLocation(response, "key", MakeAddress(51), 6);
@@ -3081,7 +2736,7 @@ TEST(ObjectMetadataClientTest, MissingTcpInlineMarkerFallsBackToReplicaRead)
3081 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2736 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3082 auto batch = MakeMetadataBatch(results);2737 auto batch = MakeMetadataBatch(results);
3083 2738 
3084- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2739+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3085 ASSERT_EQ(results.size(), 1u);2740 ASSERT_EQ(results.size(), 1u);
3086 EXPECT_TRUE(results[0].status.IsOk());2741 EXPECT_TRUE(results[0].status.IsOk());
3087 EXPECT_FALSE(results[0].inlineData.has_value());2742 EXPECT_FALSE(results[0].inlineData.has_value());
@@ -3092,8 +2747,8 @@ TEST(ObjectMetadataClientTest, RejectsInvalidTcpInlinePayloadIndex)
3092{2747{
3093 ApiDeadlineGuard deadline(1000);2748 ApiDeadlineGuard deadline(1000);
3094 auto manager = std::make_shared<FakeDataPlaneManager>();2749 auto manager = std::make_shared<FakeDataPlaneManager>();
3095- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &,2750+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &,
3096- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) {2751+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) {
3097 auto *result = AddLocation(response, "key", MakeAddress(51), 6);2752 auto *result = AddLocation(response, "key", MakeAddress(51), 6);
3098 result->mutable_data_result()->add_payload_indexes(1);2753 result->mutable_data_result()->add_payload_indexes(1);
3099 RpcMessage payload;2754 RpcMessage payload;
@@ -3106,7 +2761,7 @@ TEST(ObjectMetadataClientTest, RejectsInvalidTcpInlinePayloadIndex)
3106 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2761 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3107 auto batch = MakeMetadataBatch(results);2762 auto batch = MakeMetadataBatch(results);
3108 2763 
3109- EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch).GetCode(), K_RUNTIME_ERROR);2764+ EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).GetCode(), K_RUNTIME_ERROR);
3110}2765}
3111 2766 
3112TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuffer)2767TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuffer)
@@ -3115,8 +2770,8 @@ TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuff
3115 auto manager = std::make_shared<FakeDataPlaneManager>();2770 auto manager = std::make_shared<FakeDataPlaneManager>();
3116 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();2771 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();
3117 bufferProvider->maxGetSize = 32;2772 bufferProvider->maxGetSize = 32;
3118- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request,2773+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request,
3119- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) {2774+ QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) {
3120 EXPECT_TRUE(request.has_data_request());2775 EXPECT_TRUE(request.has_data_request());
3121 EXPECT_TRUE(request.data_request().has_ub());2776 EXPECT_TRUE(request.data_request().has_ub());
3122 const auto &ubRequest = request.data_request().ub();2777 const auto &ubRequest = request.data_request().ub();
@@ -3136,7 +2791,7 @@ TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuff
3136 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2791 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3137 auto batch = MakeMetadataBatch(results);2792 auto batch = MakeMetadataBatch(results);
3138 2793 
3139- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2794+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3140 EXPECT_EQ(bufferProvider->allocateCount, 1);2795 EXPECT_EQ(bufferProvider->allocateCount, 1);
3141 EXPECT_EQ(manager->rpcBuildCount, 1);2796 EXPECT_EQ(manager->rpcBuildCount, 1);
3142 EXPECT_EQ(manager->transportBuildCount, 1);2797 EXPECT_EQ(manager->transportBuildCount, 1);
@@ -3152,13 +2807,41 @@ TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuff
3152 EXPECT_NE(data.externalOwner, nullptr);2807 EXPECT_NE(data.externalOwner, nullptr);
3153}2808}
3154 2809 
2810+#ifdef USE_URMA
2811+TEST(ObjectMetadataClientTest, ShmPreparationFailureTriesUbBeforeTcp)
2812+{
2813+ ApiDeadlineGuard deadline(1000);
2814+ const bool enableUrma = FLAGS_enable_urma;
2815+ Raii restoreEnableUrma([enableUrma]() { FLAGS_enable_urma = enableUrma; });
2816+ FLAGS_enable_urma = true;
2817+ auto manager = std::make_shared<FakeDataPlaneManager>();
2818+ auto bufferProvider = std::make_shared<FakeUbBufferProvider>();
2819+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request,
2820+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
2821+ EXPECT_TRUE(request.data_request().has_ub());
2822+ AddLocation(response, "key", MakeAddress(51), 6);
2823+ return Status::OK();
2824+ };
2825+ ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>(),
2826+ std::make_shared<FixedTransportAdvisor>(TransportHint::SHM_CANDIDATE),
2827+ bufferProvider, 16);
2828+ auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
2829+ auto batch = MakeMetadataBatch(results);
2830+ auto readContext = std::make_shared<TransportReadContext>();
2831+ 
2832+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, readContext).IsOk());
2833+ EXPECT_EQ(manager->transportBuildCount, 2);
2834+ EXPECT_EQ(bufferProvider->allocateCount, 1);
2835+}
2836+#endif
2837+ 
3155TEST(ObjectMetadataClientTest, UbCapacityMissReleasesBufferAndFallsBack)2838TEST(ObjectMetadataClientTest, UbCapacityMissReleasesBufferAndFallsBack)
3156{2839{
3157 ApiDeadlineGuard deadline(1000);2840 ApiDeadlineGuard deadline(1000);
3158 auto manager = std::make_shared<FakeDataPlaneManager>();2841 auto manager = std::make_shared<FakeDataPlaneManager>();
3159 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();2842 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();
3160- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request,2843+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request,
3161- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {2844+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
3162 EXPECT_TRUE(request.has_data_request());2845 EXPECT_TRUE(request.has_data_request());
3163 EXPECT_TRUE(request.data_request().has_ub());2846 EXPECT_TRUE(request.data_request().has_ub());
3164 EXPECT_EQ(request.data_request().ub().buffer_size(), 16u);2847 EXPECT_EQ(request.data_request().ub().buffer_size(), 16u);
@@ -3171,20 +2854,21 @@ TEST(ObjectMetadataClientTest, UbCapacityMissReleasesBufferAndFallsBack)
3171 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2854 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3172 auto batch = MakeMetadataBatch(results);2855 auto batch = MakeMetadataBatch(results);
3173 2856 
3174- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2857+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3175 EXPECT_FALSE(results[0].inlineData.has_value());2858 EXPECT_FALSE(results[0].inlineData.has_value());
3176 EXPECT_TRUE(bufferProvider->lastOwner.expired());2859 EXPECT_TRUE(bufferProvider->lastOwner.expired());
3177}2860}
3178 2861 
3179-TEST(ObjectMetadataClientTest, UbBufferAllocationFailureQueriesMetadataOnly)2862+TEST(ObjectMetadataClientTest, UbBufferAllocationFailureFallsBackToTcp)
3180{2863{
3181 ApiDeadlineGuard deadline(1000);2864 ApiDeadlineGuard deadline(1000);
3182 auto manager = std::make_shared<FakeDataPlaneManager>();2865 auto manager = std::make_shared<FakeDataPlaneManager>();
3183 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();2866 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();
3184 bufferProvider->allocateStatus = Status(K_OUT_OF_MEMORY, "allocation failed");2867 bufferProvider->allocateStatus = Status(K_OUT_OF_MEMORY, "allocation failed");
3185- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request,2868+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request,
3186- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {2869+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
3187- EXPECT_FALSE(request.has_data_request());2870+ EXPECT_TRUE(request.has_data_request());
2871+ EXPECT_TRUE(request.data_request().has_tcp());
3188 AddLocation(response, "key", MakeAddress(51), 6);2872 AddLocation(response, "key", MakeAddress(51), 6);
3189 return Status::OK();2873 return Status::OK();
3190 };2874 };
@@ -3194,21 +2878,22 @@ TEST(ObjectMetadataClientTest, UbBufferAllocationFailureQueriesMetadataOnly)
3194 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2878 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3195 auto batch = MakeMetadataBatch(results);2879 auto batch = MakeMetadataBatch(results);
3196 2880 
3197- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2881+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3198 EXPECT_EQ(bufferProvider->allocateCount, 1);2882 EXPECT_EQ(bufferProvider->allocateCount, 1);
3199 EXPECT_EQ(manager->transportBuildCount, 1);2883 EXPECT_EQ(manager->transportBuildCount, 1);
3200 EXPECT_FALSE(results[0].inlineData.has_value());2884 EXPECT_FALSE(results[0].inlineData.has_value());
3201}2885}
3202 2886 
3203-TEST(ObjectMetadataClientTest, UbConnectionFailureQueriesMetadataOnly)2887+TEST(ObjectMetadataClientTest, UbConnectionFailureFallsBackToTcp)
3204{2888{
3205 ApiDeadlineGuard deadline(1000);2889 ApiDeadlineGuard deadline(1000);
3206 auto manager = std::make_shared<FakeDataPlaneManager>();2890 auto manager = std::make_shared<FakeDataPlaneManager>();
3207 manager->transportBuildStatuses.emplace_back(K_URMA_ERROR, "connect failed");2891 manager->transportBuildStatuses.emplace_back(K_URMA_ERROR, "connect failed");
3208 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();2892 auto bufferProvider = std::make_shared<FakeUbBufferProvider>();
3209- manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request,2893+ manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request,
3210- master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) {2894+ QueryAndGetRspPb &response, std::vector<RpcMessage> &) {
3211- EXPECT_FALSE(request.has_data_request());2895+ EXPECT_TRUE(request.has_data_request());
2896+ EXPECT_TRUE(request.data_request().has_tcp());
3212 AddLocation(response, "key", MakeAddress(51), 6);2897 AddLocation(response, "key", MakeAddress(51), 6);
3213 return Status::OK();2898 return Status::OK();
3214 };2899 };
@@ -3218,7 +2903,7 @@ TEST(ObjectMetadataClientTest, UbConnectionFailureQueriesMetadataOnly)
3218 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });2903 auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } });
3219 auto batch = MakeMetadataBatch(results);2904 auto batch = MakeMetadataBatch(results);
3220 2905 
3221- ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk());2906+ ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk());
3222 EXPECT_EQ(manager->rpcBuildCount, 1);2907 EXPECT_EQ(manager->rpcBuildCount, 1);
3223 EXPECT_EQ(manager->transportBuildCount, 1);2908 EXPECT_EQ(manager->transportBuildCount, 1);
3224 EXPECT_EQ(bufferProvider->allocateCount, 0);2909 EXPECT_EQ(bufferProvider->allocateCount, 0);