| @@ -583,8 +583,12 @@ | |||
| 583 | Client-side dynamic UB activation publishes readiness with release/acquire ordering only after `UrmaManager::Init` | 583 | Client-side dynamic UB activation publishes readiness with release/acquire ordering only after `UrmaManager::Init` |
| 584 | succeeds; transport selection remains fail-closed during initialization or after failure, without writing the | 584 | succeeds; transport selection remains fail-closed during initialization or after failure, without writing the |
| 585 | process gflag from a heartbeat thread. | 585 | process gflag from a heartbeat thread. |
| 586 | - `MasterOCServiceImpl::QueryAndGet` logs authenticated handler entry at `INFO`, terminal handler failures at `ERROR`, | 586 | + `QueryAndGet` is a Client-to-Worker RPC. The metadata-owner Worker reads only resident local objects inline and |
| 587 | - and successful completion through `SLOW_LOG_IF_OR_VLOG` using the configured server process threshold. | 587 | + resolves locations for misses through its existing metadata route; it does not pull, subscribe, create placeholders, |
| 588 | + or load L2 data for this fast path. Same-host requests carry the established SHM session identity, remote UB | ||
| 589 | + requests carry one preregistered buffer per input key. If SHM preparation fails while URMA is enabled, the client | ||
| 590 | + tries UB before degrading the whole owner group to TCP; when URMA is disabled, it degrades directly to TCP. | ||
| 591 | + Results remain positional, and absence of `data_result` means the existing replica-read phase must handle that key. | ||
| 588 | - when the existing client latency trace is enabled for a request, transport-layer Get contributes | 592 | - when the existing client latency trace is enabled for a request, transport-layer Get contributes |
| 589 | `client.process.direct_route`, `client.rpc.direct_query_and_get`, `client.rpc.direct_get_data`, and | 593 | `client.process.direct_route`, `client.rpc.direct_query_and_get`, `client.rpc.direct_get_data`, and |
| 590 | `client.process.direct_materialize` to the request `latencySummary`. The two RPC-class phases are parent-thread wall | 594 | `client.process.direct_materialize` to the request `latencySummary`. The two RPC-class phases are parent-thread wall |
| @@ -596,8 +600,9 @@ | |||
| 596 | RPC-client creation, URMA handshake/finalization, transfer, and retry preparation. If any recorded phase exceeds its | 600 | RPC-client creation, URMA handshake/finalization, transfer, and retry preparation. If any recorded phase exceeds its |
| 597 | client process or RPC slow threshold, one `[TransportGet] Phase latency` log prints the slow phase names and all | 601 | client process or RPC slow threshold, one `[TransportGet] Phase latency` log prints the slow phase names and all |
| 598 | recorded durations. Untraced requests do not construct the recorder, read the threshold config, or format the log. | 602 | recorded durations. Untraced requests do not construct the recorder, read the threshold config, or format the log. |
| 599 | - - `QueryAndGet` returns at most five copy locations per object. The primary address from object metadata is returned | 603 | + - On a Worker-local miss, `QueryAndGet` uses side-effect-free `PureQueryMeta` and converts its result into a |
| 600 | - first, followed by non-primary locations, so replica retry always starts with the primary copy. | 604 | + primary-first location list, appending the selected valid non-primary replica when one exists. The established |
| 605 | + `QueryMeta` subscription, payload, and location-update behavior remains unchanged. | ||
| 601 | - `tests/st/client/kv_cache/kv_client_transport_get_test.cpp` covers single-key and same-owner multi-key transport | 606 | - `tests/st/client/kv_cache/kv_client_transport_get_test.cpp` covers single-key and same-owner multi-key transport |
| 602 | reads. It disables the local cache, applies the same deterministic hash rule in the SDK and worker processes, and | 607 | reads. It disables the local cache, applies the same deterministic hash rule in the SDK and worker processes, and |
| 603 | resolves the metadata owner through the real SDK `Routing` path before asserting TCP or UB data transport. | 608 | resolves the metadata owner through the real SDK `Routing` path before asserting TCP or UB data transport. |
| @@ -356,6 +356,13 @@ | |||
| 356 | `MetadataRouteResolver`, `PlacementFacade`, `MembershipEndpointView`, immutable Snapshot, or an Object-specific | 356 | `MetadataRouteResolver`, `PlacementFacade`, `MembershipEndpointView`, immutable Snapshot, or an Object-specific |
| 357 | endpoint policy. `WorkerOCServiceImpl` is the sole business lifecycle owner allowed to retain a non-owning Engine | 357 | endpoint policy. `WorkerOCServiceImpl` is the sole business lifecycle owner allowed to retain a non-owning Engine |
| 358 | pointer for semantic lifecycle and cold Host queries. | 358 | pointer for semantic lifecycle and cold Host queries. |
| 359 | + - `WorkerOCService.QueryAndGet` is the metadata-affine direct-read entrypoint. It returns resident local objects over | ||
| 360 | + SHM, UB, or bounded TCP payloads and uses routed `PureQueryMeta` only for local misses, so metadata redirect handling | ||
| 361 | + stays inside Worker without changing `QueryMeta` semantics. The service handler owns only Worker-state validation | ||
| 362 | + and delegates request reading, authentication, response delivery, SHM rollback, access recording, and slow logging | ||
| 363 | + to `WorkerQueryAndGetImpl`. The local probe must remain side-effect free: no remote pull, | ||
| 364 | + subscription, placeholder creation, L2 load, or cache insertion. The response has exactly one positional result per | ||
| 365 | + requested key; a result without `data_result` delegates data retrieval to the Client's existing replica phase. | ||
| 359 | - Review caution: | 366 | - Review caution: |
| 360 | - topology behavior is spread across flags, `WorkerOCServer`, and `src/datasystem/cluster`, so config-only changes | 367 | - topology behavior is spread across flags, `WorkerOCServer`, and `src/datasystem/cluster`, so config-only changes |
| 361 | may still impact worker request routing and recovery behavior. | 368 | may still impact worker request routing and recovery behavior. |
| @@ -264,8 +264,8 @@ def _classify_rpc_deadline_detail(status: int, evidence: list[str]) -> dict[str, | |||
| 264 | re.I, | 264 | re.I, |
| 265 | ) | 265 | ) |
| 266 | ) | 266 | ) |
| 267 | - query_timeout = any("MasterOCService.QueryAndGet" in method for method in failed_methods) or bool( | 267 | + query_timeout = any("WorkerOCService.QueryAndGet" in method for method in failed_methods) or bool( |
| 268 | - re.search(r"MasterOCService\.QueryAndGet[^\n]*RPC deadline exceeded", joined, re.I) | 268 | + re.search(r"WorkerOCService\.QueryAndGet[^\n]*RPC deadline exceeded", joined, re.I) |
| 269 | ) | 269 | ) |
| 270 | # Method-specific failure evidence wins over the mere presence of another | 270 | # Method-specific failure evidence wins over the mere presence of another |
| 271 | # successful RPC in the same Trace. QueryAndGet may succeed before a later | 271 | # successful RPC in the same Trace. QueryAndGet may succeed before a later |
| @@ -281,17 +281,17 @@ def _classify_rpc_deadline_detail(status: int, evidence: list[str]) -> dict[str, | |||
| 281 | elif query_timeout: | 281 | elif query_timeout: |
| 282 | subcategory = "QueryMeta RPC deadline" | 282 | subcategory = "QueryMeta RPC deadline" |
| 283 | chain = f"QueryMeta RPC超时→TransportGet失败→{status}" | 283 | chain = f"QueryMeta RPC超时→TransportGet失败→{status}" |
| 284 | - failure_point = "MasterOCService.QueryAndGet未在deadline内返回" | 284 | + failure_point = "WorkerOCService.QueryAndGet未在deadline内返回" |
| 285 | elif re.search(r"(?:WorkerWorkerOCService\.)?GetObjectRemote", joined, re.I) and not re.search( | 285 | elif re.search(r"(?:WorkerWorkerOCService\.)?GetObjectRemote", joined, re.I) and not re.search( |
| 286 | - r"MasterOCService\.QueryAndGet", joined, re.I | 286 | + r"WorkerOCService\.QueryAndGet", joined, re.I |
| 287 | ): | 287 | ): |
| 288 | subcategory = "Data RPC deadline" | 288 | subcategory = "Data RPC deadline" |
| 289 | chain = f"Data RPC超时→TransportGet失败→{status}" | 289 | chain = f"Data RPC超时→TransportGet失败→{status}" |
| 290 | failure_point = "GetObjectRemote未在deadline内返回" | 290 | failure_point = "GetObjectRemote未在deadline内返回" |
| 291 | - elif re.search(r"MasterOCService\.QueryAndGet", joined, re.I): | 291 | + elif re.search(r"WorkerOCService\.QueryAndGet", joined, re.I): |
| 292 | subcategory = "QueryMeta RPC deadline" | 292 | subcategory = "QueryMeta RPC deadline" |
| 293 | chain = f"QueryMeta RPC超时→TransportGet失败→{status}" | 293 | chain = f"QueryMeta RPC超时→TransportGet失败→{status}" |
| 294 | - failure_point = "MasterOCService.QueryAndGet未在deadline内返回" | 294 | + failure_point = "WorkerOCService.QueryAndGet未在deadline内返回" |
| 295 | else: | 295 | else: |
| 296 | subcategory = "RPC deadline·方法未细分" | 296 | subcategory = "RPC deadline·方法未细分" |
| 297 | chain = f"RPC超时→TransportGet失败→{status}" | 297 | chain = f"RPC超时→TransportGet失败→{status}" |
| @@ -1361,7 +1361,7 @@ def _query_meta_detail(row: dict) -> dict | None: | |||
| 1361 | entries = [] | 1361 | entries = [] |
| 1362 | for text in row.get("evidence", []): | 1362 | for text in row.get("evidence", []): |
| 1363 | method, fields = _rpc_fields(text) | 1363 | method, fields = _rpc_fields(text) |
| 1364 | - if method and "MasterOCService.QueryAndGet" in method: | 1364 | + if method and "WorkerOCService.QueryAndGet" in method: |
| 1365 | entries.append(fields) | 1365 | entries.append(fields) |
| 1366 | if not entries and not row.get("query_meta_ms"): | 1366 | if not entries and not row.get("query_meta_ms"): |
| 1367 | return None | 1367 | return None |
| @@ -1875,7 +1875,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict: | |||
| 1875 | initiator = row.get("client_observer") or "未明确" | 1875 | initiator = row.get("client_observer") or "未明确" |
| 1876 | if initiator == "未明确": | 1876 | if initiator == "未明确": |
| 1877 | for record in row.get("evidence_records", []): | 1877 | for record in row.get("evidence_records", []): |
| 1878 | - if "MasterOCService.QueryAndGet" in record.get("text", ""): | 1878 | + if "WorkerOCService.QueryAndGet" in record.get("text", ""): |
| 1879 | initiator = record.get("worker") or "未明确" | 1879 | initiator = record.get("worker") or "未明确" |
| 1880 | break | 1880 | break |
| 1881 | initiator_groups[initiator].append(row) | 1881 | initiator_groups[initiator].append(row) |
| @@ -1920,7 +1920,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict: | |||
| 1920 | initiator = row.get("client_observer") or "未明确" | 1920 | initiator = row.get("client_observer") or "未明确" |
| 1921 | if initiator == "未明确": | 1921 | if initiator == "未明确": |
| 1922 | for record in row.get("evidence_records", []): | 1922 | for record in row.get("evidence_records", []): |
| 1923 | - if "MasterOCService.QueryAndGet" in record.get("text", ""): | 1923 | + if "WorkerOCService.QueryAndGet" in record.get("text", ""): |
| 1924 | initiator = record.get("worker") or "未明确" | 1924 | initiator = record.get("worker") or "未明确" |
| 1925 | break | 1925 | break |
| 1926 | if initiator != "未明确": | 1926 | if initiator != "未明确": |
| @@ -1928,7 +1928,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict: | |||
| 1928 | query_entries = [] | 1928 | query_entries = [] |
| 1929 | for text in row.get("evidence", []): | 1929 | for text in row.get("evidence", []): |
| 1930 | method, fields = _rpc_fields(text) | 1930 | method, fields = _rpc_fields(text) |
| 1931 | - if method and "MasterOCService.QueryAndGet" in method: | 1931 | + if method and "WorkerOCService.QueryAndGet" in method: |
| 1932 | query_entries.append(fields) | 1932 | query_entries.append(fields) |
| 1933 | target = re.search( | 1933 | target = re.search( |
| 1934 | r"(?:meta owner|targetAddress|target address|peer)\s*[:=]\s*([^,\s]+)", | 1934 | r"(?:meta owner|targetAddress|target address|peer)\s*[:=]\s*([^,\s]+)", |
| @@ -1938,7 +1938,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict: | |||
| 1938 | if target: | 1938 | if target: |
| 1939 | timeout_targets.add(target.group(1)) | 1939 | timeout_targets.add(target.group(1)) |
| 1940 | if any( | 1940 | if any( |
| 1941 | - "MasterOCService.QueryAndGet" in text and re.search(r"\bresp_attachment_bytes=0\b", text) | 1941 | + "WorkerOCService.QueryAndGet" in text and re.search(r"\bresp_attachment_bytes=0\b", text) |
| 1942 | for text in row.get("evidence", []) | 1942 | for text in row.get("evidence", []) |
| 1943 | ): | 1943 | ): |
| 1944 | empty_response_count += 1 | 1944 | empty_response_count += 1 |
| @@ -1972,7 +1972,7 @@ def _aggregate_query_meta(rows: list[dict]) -> dict: | |||
| 1972 | "dominant_second": {"second": dominant_second, "trace_count": dominant_count}, | 1972 | "dominant_second": {"second": dominant_second, "trace_count": dominant_count}, |
| 1973 | "confirmed_flow": ( | 1973 | "confirmed_flow": ( |
| 1974 | "Client ObjectReadFlow::Resolve → ObjectMetadataClient::QueryAndGet/QueryWithRetry → " | 1974 | "Client ObjectReadFlow::Resolve → ObjectMetadataClient::QueryAndGet/QueryWithRetry → " |
| 1975 | - "WorkerRpcClient::InvokeQueryAndGet → Meta Owner MasterOCService.QueryAndGet;" | 1975 | + "WorkerRpcClient::InvokeQueryAndGet → Meta Owner WorkerOCService.QueryAndGet;" |
| 1976 | "超时发生在该RPC返回前,尚未进入后续独立Data Worker GetObjectRemote阶段。" | 1976 | "超时发生在该RPC返回前,尚未进入后续独立Data Worker GetObjectRemote阶段。" |
| 1977 | ), | 1977 | ), |
| 1978 | "likely_common_mechanism": ( | 1978 | "likely_common_mechanism": ( |
| @@ -2237,7 +2237,7 @@ table{width:100%;table-layout:fixed} | |||
| 2237 | CORRELATION_SECTION = r''' | 2237 | CORRELATION_SECTION = r''' |
| 2238 | <section class="panel" id="query-meta-analysis"> | 2238 | <section class="panel" id="query-meta-analysis"> |
| 2239 | <h2>4-A. QueryMeta 根因分析</h2> | 2239 | <h2>4-A. QueryMeta 根因分析</h2> |
| 2240 | -<div class="notice"><b>定界口径:</b><code>MasterOCService.QueryAndGet</code> 不只是查元数据:携带 <code>data_request</code> 时,Meta Owner 会执行 <code>TryGetQueryAndGetData</code>,命中本地副本后可通过 UB/URMA 内联返回数据。细类按互斥优先级统计;TryGet/URMA 作为证据标签,不重复计入主类。失败且只有 <code>cntl_error_code=1008</code> 时,只能确认 Client 等待到截止点;缺少 server trailer 时,Meta Owner 执行、响应发送、RPC residual 与 Client 截止观察仍未闭合。</div> | 2240 | +<div class="notice"><b>定界口径:</b><code>WorkerOCService.QueryAndGet</code> 不只是查元数据:携带 <code>data_request</code> 时,Meta Owner Worker 会读取本地驻留副本,并通过 SHM、UB/URMA 或 TCP 内联返回数据;本地 miss 才向 Master 发起 metadata-only QueryMeta。细类按互斥优先级统计;本地读取/URMA 作为证据标签,不重复计入主类。失败且只有 <code>cntl_error_code=1008</code> 时,只能确认 Client 等待到截止点;缺少 server trailer 时,Meta Owner Worker 执行、响应发送、RPC residual 与 Client 截止观察仍未闭合。</div> |
| 2241 | <div id="query-meta-summary" class="finding-grid"></div> | 2241 | <div id="query-meta-summary" class="finding-grid"></div> |
| 2242 | <h3>QueryAndGet 超时流程定界</h3> | 2242 | <h3>QueryAndGet 超时流程定界</h3> |
| 2243 | <div id="query-meta-timeout-flow" class="finding-grid"></div> | 2243 | <div id="query-meta-timeout-flow" class="finding-grid"></div> |
| @@ -413,6 +413,23 @@ Status ShmSession::BuildResult(const GetRspPb::ObjectInfoPb &info, const DataGet | |||
| 413 | return Status::OK(); | 413 | return Status::OK(); |
| 414 | } | 414 | } |
| 415 | 415 | ||
| 416 | +Status ShmSession::BuildQueryAndGetResult(const QueryAndGetShmInfoPb &info, const DataGetRequest &input, | ||
| 417 | + DataGetResult &result) | ||
| 418 | +{ | ||
| 419 | + GetRspPb::ObjectInfoPb objectInfo; | ||
| 420 | + objectInfo.set_store_fd(info.store_fd()); | ||
| 421 | + objectInfo.set_offset(info.offset()); | ||
| 422 | + objectInfo.set_data_size(info.data_size()); | ||
| 423 | + objectInfo.set_metadata_size(info.metadata_size()); | ||
| 424 | + objectInfo.set_mmap_size(info.mmap_size()); | ||
| 425 | + objectInfo.set_shm_id(info.shm_id()); | ||
| 426 | + objectInfo.set_is_seal(info.is_seal()); | ||
| 427 | + objectInfo.set_write_mode(info.write_mode()); | ||
| 428 | + objectInfo.set_consistency_type(info.consistency_type()); | ||
| 429 | + objectInfo.set_cache_type(info.cache_type()); | ||
| 430 | + return BuildResult(objectInfo, input, result); | ||
| 431 | +} | ||
| 432 | + | ||
| 416 | Status ShmSession::MmapWriteRegion(const CreateRspPb &createRsp, const TransportRequestContext &context, | 433 | Status ShmSession::MmapWriteRegion(const CreateRspPb &createRsp, const TransportRequestContext &context, |
| 417 | uint64_t size, ObjectBufferInfo &info) | 434 | uint64_t size, ObjectBufferInfo &info) |
| 418 | { | 435 | { |
| @@ -82,6 +82,16 @@ public: | |||
| 82 | 82 | ||
| 83 | Status BuildResult(const GetRspPb::ObjectInfoPb &info, const DataGetRequest &input, DataGetResult &result); | 83 | Status BuildResult(const GetRspPb::ObjectInfoPb &info, const DataGetRequest &input, DataGetResult &result); |
| 84 | 84 | ||
| 85 | + /** | ||
| 86 | + * @brief Materialize a QueryAndGet shared-memory result through this session. | ||
| 87 | + * @param[in] info Shared-memory result returned by Worker QueryAndGet. | ||
| 88 | + * @param[in] input Object identity and request context. | ||
| 89 | + * @param[out] result Materialized zero-copy data result. | ||
| 90 | + * @return K_OK on success; the error code otherwise. | ||
| 91 | + */ | ||
| 92 | + Status BuildQueryAndGetResult(const QueryAndGetShmInfoPb &info, const DataGetRequest &input, | ||
| 93 | + DataGetResult &result); | ||
| 94 | + | ||
| 85 | /** Maps the shared-memory region allocated by a routed Create into the client address space (PROT_WRITE) | 95 | /** Maps the shared-memory region allocated by a routed Create into the client address space (PROT_WRITE) |
| 86 | * so the caller can write zero-copy, registers the worker reference, and attaches a send-side owner that | 96 | * so the caller can write zero-copy, registers the worker reference, and attaches a send-side owner that |
| 87 | * gates Publish on session liveness (K_BUFFER_DEPRECATED) and releases the worker reference on buffer | 97 | * gates Publish on session liveness (K_BUFFER_DEPRECATED) and releases the worker reference on buffer |
| @@ -59,6 +59,29 @@ public: | |||
| 59 | && shmConnection_->IsAlive(); | 59 | && shmConnection_->IsAlive(); |
| 60 | } | 60 | } |
| 61 | 61 | ||
| 62 | + /** | ||
| 63 | + * @brief Acquire the endpoint shared-memory session for a routed request. | ||
| 64 | + * @param[in] context Request authentication and tenant context. | ||
| 65 | + * @param[out] session Acquired endpoint session. | ||
| 66 | + * @return K_OK on success; the error code otherwise. | ||
| 67 | + */ | ||
| 68 | + Status AcquireSession(const TransportRequestContext &context, std::shared_ptr<ShmSession> &session) | ||
| 69 | + { | ||
| 70 | + RETURN_RUNTIME_ERROR_IF_NULL(shmConnection_); | ||
| 71 | + return shmConnection_->Acquire(context, session); | ||
| 72 | + } | ||
| 73 | + | ||
| 74 | + /** | ||
| 75 | + * @brief Invalidate a session that returned an unusable shared-memory result. | ||
| 76 | + * @param[in] session Session to invalidate. | ||
| 77 | + */ | ||
| 78 | + void InvalidateSession(const std::shared_ptr<ShmSession> &session) | ||
| 79 | + { | ||
| 80 | + if (shmConnection_ != nullptr) { | ||
| 81 | + shmConnection_->Invalidate(session); | ||
| 82 | + } | ||
| 83 | + } | ||
| 84 | + | ||
| 62 | Status Get(const DataGetRequest &input, DataGetResult &output) override | 85 | Status Get(const DataGetRequest &input, DataGetResult &output) override |
| 63 | { | 86 | { |
| 64 | RETURN_RUNTIME_ERROR_IF_NULL(rpcClient_); | 87 | RETURN_RUNTIME_ERROR_IF_NULL(rpcClient_); |
| @@ -14,19 +14,19 @@ | |||
| 14 | * limitations under the License. | 14 | * limitations under the License. |
| 15 | */ | 15 | */ |
| 16 | 16 | ||
| 17 | -/** Description: Implements batched object metadata access and deadline-bounded redirect handling. */ | 17 | +/** Description: Implements Worker-routed batched metadata and inline-data access. */ |
| 18 | 18 | ||
| 19 | 19 | ||
| 20 | 20 | ||
| 21 | 21 | ||
| 22 | 22 | ||
| 23 | 23 | ||
| 24 | - | ||
| 25 | 24 | ||
| 26 | - | ||
| 27 | 25 | ||
| 28 | 26 | ||
| 27 | + | ||
| 29 | 28 | ||
| 29 | + | ||
| 30 | 30 | ||
| 31 | 31 | ||
| 32 | 32 | ||
| @@ -35,25 +35,6 @@ namespace datasystem { | |||
| 35 | namespace client { | 35 | namespace client { |
| 36 | 36 | ||
| 37 | namespace { | 37 | namespace { |
| 38 | -struct RedirectTarget { | ||
| 39 | - HostPort address; | ||
| 40 | - uint64_t topologyVersion = 0; | ||
| 41 | -}; | ||
| 42 | - | ||
| 43 | -struct RedirectBatch { | ||
| 44 | - HostPort address; | ||
| 45 | - uint64_t topologyVersion = 0; | ||
| 46 | - ObjectMetadataBatch items; | ||
| 47 | -}; | ||
| 48 | - | ||
| 49 | -struct PendingMetadataBatch { | ||
| 50 | - HostPort address; | ||
| 51 | - uint64_t topologyVersion = 0; | ||
| 52 | - ObjectMetadataBatch items; | ||
| 53 | -}; | ||
| 54 | - | ||
| 55 | -using RedirectTargets = std::unordered_map<std::string, std::deque<RedirectTarget>>; | ||
| 56 | - | ||
| 57 | bool IsMetadataOwnerRouteFailure(StatusCode code) | 38 | bool IsMetadataOwnerRouteFailure(StatusCode code) |
| 58 | { | 39 | { |
| 59 | return code == K_RPC_UNAVAILABLE || code == K_RPC_DEADLINE_EXCEEDED || code == K_RPC_PEER_DEAD | 40 | return code == K_RPC_UNAVAILABLE || code == K_RPC_DEADLINE_EXCEEDED || code == K_RPC_PEER_DEAD |
| @@ -73,77 +54,11 @@ Status ValidateAndResetItems(const ObjectMetadataBatch &items) | |||
| 73 | return Status::OK(); | 54 | return Status::OK(); |
| 74 | } | 55 | } |
| 75 | 56 | ||
| 76 | -Status BuildRedirectTargets(const master::QueryAndGetRspPb &response, RedirectTargets &redirectTargets) | 57 | +void CopyLocation(const QueryAndGetLocationInfoPb &source, master::ObjectLocationInfoPb &target) |
| 77 | { | 58 | { |
| 78 | - for (const auto &redirectInfo : response.info()) { | 59 | + target.set_object_key(source.object_key()); |
| 79 | - HostPort address; | 60 | + target.set_object_size(source.object_size()); |
| 80 | - RETURN_IF_NOT_OK(address.ParseString(redirectInfo.redirect_meta_address())); | 61 | + *target.mutable_object_locations() = source.object_locations(); |
| 81 | - for (const auto &objectKey : redirectInfo.change_meta_ids()) { | ||
| 82 | - redirectTargets[objectKey].push_back({ address, redirectInfo.topology_version() }); | ||
| 83 | - } | ||
| 84 | - } | ||
| 85 | - return Status::OK(); | ||
| 86 | -} | ||
| 87 | - | ||
| 88 | -Status PartitionInitialResponse(const ObjectMetadataBatch &items, const master::QueryAndGetRspPb &response, | ||
| 89 | - ObjectMetadataBatch &localItems, | ||
| 90 | - std::vector<RedirectBatch> &redirectBatches, | ||
| 91 | - std::optional<uint64_t> minimumTopologyVersion) | ||
| 92 | -{ | ||
| 93 | - RedirectTargets redirectTargets; | ||
| 94 | - RETURN_IF_NOT_OK(BuildRedirectTargets(response, redirectTargets)); | ||
| 95 | - | ||
| 96 | - localItems.clear(); | ||
| 97 | - localItems.reserve(items.size()); | ||
| 98 | - std::unordered_map<std::string, RedirectBatch *> batchesByTarget; | ||
| 99 | - // The target map keeps pointers into redirectBatches, so prevent vector relocation while grouping. | ||
| 100 | - redirectBatches.reserve(response.info_size()); | ||
| 101 | - batchesByTarget.reserve(response.info_size()); | ||
| 102 | - for (auto *item : items) { | ||
| 103 | - auto target = redirectTargets.find(item->objectKey); | ||
| 104 | - if (target == redirectTargets.end() || target->second.empty()) { | ||
| 105 | - localItems.push_back(item); | ||
| 106 | - continue; | ||
| 107 | - } | ||
| 108 | - RedirectTarget redirectTarget = std::move(target->second.front()); | ||
| 109 | - target->second.pop_front(); | ||
| 110 | - CHECK_FAIL_RETURN_STATUS(!minimumTopologyVersion.has_value() | ||
| 111 | - || redirectTarget.topologyVersion >= *minimumTopologyVersion, | ||
| 112 | - K_NOT_READY, | ||
| 113 | - std::string(STALE_TRANSPORT_SNAPSHOT_MESSAGE) | ||
| 114 | - + ": metadata redirect topology version rolled back from " | ||
| 115 | - + std::to_string(*minimumTopologyVersion) + " to " | ||
| 116 | - + std::to_string(redirectTarget.topologyVersion)); | ||
| 117 | - const std::string batchKey = redirectTarget.address.ToString() + "#" | ||
| 118 | - + std::to_string(redirectTarget.topologyVersion); | ||
| 119 | - auto inserted = batchesByTarget.emplace(batchKey, nullptr); | ||
| 120 | - if (inserted.second) { | ||
| 121 | - redirectBatches.push_back( | ||
| 122 | - { std::move(redirectTarget.address), redirectTarget.topologyVersion, {} }); | ||
| 123 | - inserted.first->second = &redirectBatches.back(); | ||
| 124 | - } | ||
| 125 | - inserted.first->second->items.push_back(item); | ||
| 126 | - } | ||
| 127 | - return Status::OK(); | ||
| 128 | -} | ||
| 129 | - | ||
| 130 | -void SetBatchError(const ObjectMetadataBatch &items, const Status &status) | ||
| 131 | -{ | ||
| 132 | - for (auto *item : items) { | ||
| 133 | - item->status = status; | ||
| 134 | - } | ||
| 135 | -} | ||
| 136 | - | ||
| 137 | -void QueueRedirectBatches(std::vector<RedirectBatch> &redirectBatches, std::deque<PendingMetadataBatch> &pending) | ||
| 138 | -{ | ||
| 139 | - for (auto &batch : redirectBatches) { | ||
| 140 | - if (batch.items.empty()) { | ||
| 141 | - continue; | ||
| 142 | - } | ||
| 143 | - LOG(INFO) << "[TransportGet][Metadata] Follow redirect, target: " << batch.address.ToString() | ||
| 144 | - << ", topology version: " << batch.topologyVersion << ", key count: " << batch.items.size(); | ||
| 145 | - pending.push_back({ std::move(batch.address), batch.topologyVersion, std::move(batch.items) }); | ||
| 146 | - } | ||
| 147 | } | 62 | } |
| 148 | } // namespace | 63 | } // namespace |
| 149 | 64 | ||
| @@ -163,24 +78,69 @@ ObjectMetadataClient::ObjectMetadataClient(std::shared_ptr<DataPlaneManager> man | |||
| 163 | } | 78 | } |
| 164 | 79 | ||
| 165 | Status ObjectMetadataClient::InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items, | 80 | Status ObjectMetadataClient::InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items, |
| 81 | + std::shared_ptr<const TransportReadContext> readContext, | ||
| 166 | InlineRequestContext &context) const | 82 | InlineRequestContext &context) const |
| 167 | { | 83 | { |
| 168 | context = InlineRequestContext{}; | 84 | context = InlineRequestContext{}; |
| 169 | - // One batch uses one exclusive inline mode; phase one enables it only for a single key. | 85 | + const auto hint = advisor_ == nullptr ? TransportHint::TCP_ONLY : advisor_->GetTransportHint(address); |
| 170 | - if (items.size() != 1 || advisor_ == nullptr) { | 86 | + if (hint == TransportHint::SHM_CANDIDATE) { |
| 87 | + RETURN_IF_NOT_OK(PrepareShmInlineRequest(address, std::move(readContext), context)); | ||
| 88 | + RETURN_OK_IF_TRUE(context.mode == InlineTransportMode::SHM); | ||
| 89 | + } | ||
| 90 | + if (hint == TransportHint::UB_CANDIDATE || IsUrmaEnabled()) { | ||
| 91 | + return PrepareUbInlineRequest(address, items, context); | ||
| 92 | + } | ||
| 93 | + context.mode = InlineTransportMode::TCP; | ||
| 94 | + return Status::OK(); | ||
| 95 | +} | ||
| 96 | + | ||
| 97 | +Status ObjectMetadataClient::PrepareShmInlineRequest(const HostPort &address, | ||
| 98 | + std::shared_ptr<const TransportReadContext> readContext, | ||
| 99 | + InlineRequestContext &context) const | ||
| 100 | +{ | ||
| 101 | + RETURN_RUNTIME_ERROR_IF_NULL(manager_); | ||
| 102 | + if (readContext == nullptr) { | ||
| 103 | + context.DisableInlineData(); | ||
| 171 | return Status::OK(); | 104 | return Status::OK(); |
| 172 | } | 105 | } |
| 173 | - if (advisor_->GetTransportHint(address) == TransportHint::TCP_ONLY) { | 106 | + std::shared_ptr<IDataTransporter> transporter; |
| 174 | - context.mode = InlineTransportMode::TCP; | 107 | + Status rc = manager_->GetOrCreate(address, TransportHint::SHM_CANDIDATE, transporter); |
| 108 | + auto shmTransporter = std::dynamic_pointer_cast<ShmTransporter>(transporter); | ||
| 109 | + if (rc.IsError() || shmTransporter == nullptr) { | ||
| 110 | + context.DisableInlineData(); | ||
| 175 | return Status::OK(); | 111 | return Status::OK(); |
| 176 | } | 112 | } |
| 177 | - return PrepareUbInlineRequest(address, items, context); | 113 | + std::shared_ptr<ShmSession> session; |
| 114 | + rc = shmTransporter->AcquireSession(readContext->requestContext, session); | ||
[警告][安全] [P1] 复用 SHM session 时未刷新当前请求的认证上下文 证据:这里 AcquireSession 复用已有 live session 后直接返回;ShmConnection::Acquire 不更新 ShmSession 或 ShmFdChannel 的 auth。普通 ShmSession::Get 每次请求都会同时刷新二者,而 QueryAndGet 的 BuildQueryAndGetResult 可能触发 GetClientFd,后者使用 ShmFdChannel 中缓存的 token,并与当前 tenantId 组合发请求。 问题:QueryAndGet 主 RPC 可以使用当前凭据成功,但冷 FD 获取仍可能使用 session 创建时或上一次普通 Get 的旧 token/tenant 上下文。 影响:凭据轮换或请求级租户变化后,同机 SHM 命中可能在结果物化阶段稳定失败并使 session 失效;认证上下文错配也触及多租户隔离边界。 建议:提供带锁的 ShmSession::RefreshAuth(context),在复用 session 后且 dispatch 前原子刷新 session 与 fdChannel 的认证上下文;或让 BuildQueryAndGetResult 显式刷新本次 context。 验证:复用同一 session 后轮换 token/AKSK,并切换 request tenant,构造需要获取新 FD 的 SHM hit,确认 GetClientFd 只使用当前凭据,且失败时有安全降级路径。 ![]() ![]() | |||
| 115 | + if (rc.IsError()) { | ||
| 116 | + context.DisableInlineData(); | ||
| 117 | + return Status::OK(); | ||
| 118 | + } | ||
| 119 | + context.mode = InlineTransportMode::SHM; | ||
| 120 | + context.shmTransporter = std::move(shmTransporter); | ||
| 121 | + context.shmSession = std::move(session); | ||
| 122 | + context.readContext = std::move(readContext); | ||
| 123 | + return Status::OK(); | ||
| 124 | +} | ||
| 125 | + | ||
| 126 | +Status ObjectMetadataClient::PrepareShmInlineFallback(const HostPort &address, | ||
| 127 | + const ObjectMetadataBatch &items, | ||
| 128 | + InlineRequestContext &context) const | ||
| 129 | +{ | ||
| 130 | + context.DisableInlineData(); | ||
| 131 | + if (IsUrmaEnabled()) { | ||
| 132 | + return PrepareUbInlineRequest(address, items, context); | ||
| 133 | + } | ||
| 134 | + context.mode = InlineTransportMode::TCP; | ||
| 135 | + return Status::OK(); | ||
| 178 | } | 136 | } |
| 179 | 137 | ||
| 180 | Status ObjectMetadataClient::PrepareUbInlineRequest(const HostPort &address, const ObjectMetadataBatch &items, | 138 | Status ObjectMetadataClient::PrepareUbInlineRequest(const HostPort &address, const ObjectMetadataBatch &items, |
| 181 | InlineRequestContext &context) const | 139 | InlineRequestContext &context) const |
| 182 | { | 140 | { |
| 183 | - if (ubBufferSize_ == 0 || ubBufferProvider_ == nullptr || ubBufferSize_ > ubBufferProvider_->MaxGetSize()) { | 141 | + if (ubBufferSize_ == 0 || ubBufferProvider_ == nullptr || ubBufferSize_ > ubBufferProvider_->MaxGetSize() |
| 142 | + || items.size() > ubBufferProvider_->MaxGetSize() / ubBufferSize_) { | ||
| 143 | + context.mode = InlineTransportMode::TCP; | ||
| 184 | return Status::OK(); | 144 | return Status::OK(); |
| 185 | } | 145 | } |
| 186 | 146 | ||
| @@ -191,10 +151,12 @@ Status ObjectMetadataClient::PrepareUbInlineRequest(const HostPort &address, con | |||
| 191 | if (connectionRc.IsError()) { | 151 | if (connectionRc.IsError()) { |
| 192 | VLOG(1) << "[TransportGet][Metadata] Disable UB inline data because the connection is unavailable: " | 152 | VLOG(1) << "[TransportGet][Metadata] Disable UB inline data because the connection is unavailable: " |
| 193 | << connectionRc.ToString(); | 153 | << connectionRc.ToString(); |
| 154 | + context.mode = InlineTransportMode::TCP; | ||
| 194 | return Status::OK(); | 155 | return Status::OK(); |
| 195 | } | 156 | } |
| 196 | 157 | ||
| 197 | if (AllocateUbInlineBuffers(items, context).IsError()) { | 158 | if (AllocateUbInlineBuffers(items, context).IsError()) { |
| 159 | + context.mode = InlineTransportMode::TCP; | ||
| 198 | return Status::OK(); | 160 | return Status::OK(); |
| 199 | } | 161 | } |
| 200 | context.mode = InlineTransportMode::UB; | 162 | context.mode = InlineTransportMode::UB; |
| @@ -229,7 +191,7 @@ Status ObjectMetadataClient::AllocateUbInlineBuffers(const ObjectMetadataBatch & | |||
| 229 | 191 | ||
| 230 | Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &items, | 192 | Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &items, |
| 231 | const InlineRequestContext &context, | 193 | const InlineRequestContext &context, |
| 232 | - master::QueryAndGetReqPb &request) const | 194 | + QueryAndGetReqPb &request) const |
| 233 | { | 195 | { |
| 234 | if (context.mode == InlineTransportMode::NONE) { | 196 | if (context.mode == InlineTransportMode::NONE) { |
| 235 | return Status::OK(); | 197 | return Status::OK(); |
| @@ -238,11 +200,17 @@ Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &ite | |||
| 238 | (void)request.mutable_data_request()->mutable_tcp(); | 200 | (void)request.mutable_data_request()->mutable_tcp(); |
| 239 | return Status::OK(); | 201 | return Status::OK(); |
| 240 | } | 202 | } |
| 203 | + if (context.mode == InlineTransportMode::SHM) { | ||
| 204 | + CHECK_FAIL_RETURN_STATUS(context.shmSession != nullptr && context.shmSession->IsAlive(), K_NOT_READY, | ||
O [P3] SHM session 在请求构建阶段失效(K_NOT_READY)时整批失败且无 TCP/UB 回退 PrepareShmInlineRequest 成功取得 session 后,若在 AddInlineDataRequest 检查 IsAlive() 时 session 恰好失效(心跳超时窗口),直接返回 K_NOT_READY,QueryWithRetry 立即返回错误,组内所有 key 失败;而 InvokeQueryAndGet 的 lease 失败路径(invoked==false)会走 SwitchInlineRequestToTcp 降级 TCP。同一请求内两条失败路径行为不一致,窗口虽极小但失败模式生硬,建议此处同样降级 TCP 或按 miss 重试。 ![]() ![]() | |||
| 205 | + "QueryAndGet shared-memory session is unavailable"); | ||
| 206 | + request.mutable_data_request()->mutable_shm()->set_client_id(context.shmSession->ClientId()); | ||
| 207 | + return Status::OK(); | ||
| 208 | + } | ||
| 241 | 209 | ||
| 242 | auto *ubRequest = request.mutable_data_request()->mutable_ub(); | 210 | auto *ubRequest = request.mutable_data_request()->mutable_ub(); |
| 243 | ubRequest->set_buffer_size(ubBufferSize_); | 211 | ubRequest->set_buffer_size(ubBufferSize_); |
| 244 | ubRequest->set_urma_instance_id(context.transportInstanceId); | 212 | ubRequest->set_urma_instance_id(context.transportInstanceId); |
| 245 | - // Buffer descriptors follow object_keys order, including redirected sub-batches. | 213 | + // Buffer descriptors follow object_keys order. |
| 246 | for (auto *item : items) { | 214 | for (auto *item : items) { |
| 247 | auto buffer = context.ubBuffers.find(item); | 215 | auto buffer = context.ubBuffers.find(item); |
| 248 | CHECK_FAIL_RETURN_STATUS(buffer != context.ubBuffers.end(), K_RUNTIME_ERROR, | 216 | CHECK_FAIL_RETURN_STATUS(buffer != context.ubBuffers.end(), K_RUNTIME_ERROR, |
| @@ -252,132 +220,145 @@ Status ObjectMetadataClient::AddInlineDataRequest(const ObjectMetadataBatch &ite | |||
| 252 | return Status::OK(); | 220 | return Status::OK(); |
| 253 | } | 221 | } |
| 254 | 222 | ||
| 255 | -Status ObjectMetadataClient::InvokeQueryAndGet(const HostPort &address, master::QueryAndGetReqPb &request, | 223 | +Status ObjectMetadataClient::InvokeQueryAndGet(const HostPort &address, QueryAndGetReqPb &request, |
| 256 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, | 224 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, |
| 257 | - InlineRequestContext &context, | 225 | + InlineRequestContext &context, bool &rpcDispatched) |
| 258 | - std::optional<uint64_t> redirectTopologyVersion, | ||
| 259 | - bool &rpcDispatched) | ||
| 260 | { | 226 | { |
| 261 | rpcDispatched = false; | 227 | rpcDispatched = false; |
| 262 | RETURN_RUNTIME_ERROR_IF_NULL(manager_); | 228 | RETURN_RUNTIME_ERROR_IF_NULL(manager_); |
| 263 | - if (context.mode == InlineTransportMode::UB) { | 229 | + if (context.mode == InlineTransportMode::UB || context.mode == InlineTransportMode::SHM) { |
| 264 | bool invoked = false; | 230 | bool invoked = false; |
| 265 | - Status leaseRc = manager_->WithDataPlaneLease( | 231 | + Status leaseRc = InvokeInlineQueryAndGet(address, request, response, payloads, context, invoked, rpcDispatched); |
| 266 | - address, TransportHint::UB_CANDIDATE, | ||
| 267 | - [&](const std::shared_ptr<IDataTransporter> &, const std::shared_ptr<WorkerRpcClient> &rpcClient) { | ||
| 268 | - invoked = true; | ||
| 269 | - return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched); | ||
| 270 | - }); | ||
| 271 | if (invoked) { | 232 | if (invoked) { |
| 272 | - // Once sent, RPC failures retain UB mode and follow the normal retry policy. | ||
| 273 | return leaseRc; | 233 | return leaseRc; |
| 274 | } | 234 | } |
| 275 | - VLOG(1) << "[TransportGet][Metadata] UB inline connection is unavailable for " << address.ToString() | 235 | + VLOG(1) << "[TransportGet][Metadata] Inline data plane is unavailable for " << address.ToString() |
| 276 | - << ", query metadata only: " << leaseRc.ToString(); | 236 | + << ", fallback to TCP: " << leaseRc.ToString(); |
| 277 | - // The RPC was not sent, so permanently downgrade this request chain to metadata-only mode. | 237 | + SwitchInlineRequestToTcp(request, payloads, context); |
| 278 | - context.DisableInlineData(); | ||
| 279 | - request.clear_data_request(); | ||
| 280 | - payloads.clear(); | ||
| 281 | } | 238 | } |
| 239 | + return InvokeTcpQueryAndGet(address, request, response, payloads, rpcDispatched); | ||
| 240 | +} | ||
| 282 | 241 | ||
| 242 | +Status ObjectMetadataClient::InvokeInlineQueryAndGet(const HostPort &address, QueryAndGetReqPb &request, | ||
| 243 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, | ||
| 244 | + InlineRequestContext &context, bool &invoked, | ||
| 245 | + bool &rpcDispatched) | ||
| 246 | +{ | ||
| 247 | + const auto hint = context.mode == InlineTransportMode::UB ? TransportHint::UB_CANDIDATE | ||
| 248 | + : TransportHint::SHM_CANDIDATE; | ||
| 249 | + invoked = false; | ||
| 250 | + return manager_->WithDataPlaneLease( | ||
| 251 | + address, hint, | ||
| 252 | + [&](const std::shared_ptr<IDataTransporter> &transporter, | ||
| 253 | + const std::shared_ptr<WorkerRpcClient> &rpcClient) { | ||
| 254 | + if (context.mode == InlineTransportMode::SHM | ||
| 255 | + && (transporter != context.shmTransporter || !context.shmSession->IsAlive())) { | ||
| 256 | + RETURN_STATUS(K_NOT_READY, "QueryAndGet shared-memory session changed before dispatch"); | ||
| 257 | + } | ||
| 258 | + invoked = true; | ||
| 259 | + return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched); | ||
| 260 | + }); | ||
| 261 | +} | ||
| 262 | + | ||
| 263 | +Status ObjectMetadataClient::InvokeTcpQueryAndGet(const HostPort &address, QueryAndGetReqPb &request, | ||
| 264 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, | ||
| 265 | + bool &rpcDispatched) | ||
| 266 | +{ | ||
| 283 | std::shared_ptr<WorkerRpcClient> rpcClient; | 267 | std::shared_ptr<WorkerRpcClient> rpcClient; |
| 284 | - Status rc = manager_->GetOrCreateRpcClient(address, rpcClient); | 268 | + RETURN_IF_NOT_OK(manager_->GetOrCreateRpcClient(address, rpcClient)); |
| 285 | - if (redirectTopologyVersion.has_value() && IsTransportSnapshotStaleLocation(rc)) { | ||
| 286 | - VLOG(1) << "[TransportGet][Metadata] Transport snapshot lags redirected owner, query metadata only: " | ||
| 287 | - << address.ToString() << ", redirect version: " << *redirectTopologyVersion; | ||
| 288 | - if (metadataFailureHandler_) { | ||
| 289 | - metadataFailureHandler_(address, rc); | ||
| 290 | - } | ||
| 291 | - context.DisableInlineData(); | ||
| 292 | - request.clear_data_request(); | ||
| 293 | - payloads.clear(); | ||
| 294 | - rc = manager_->GetOrCreateRedirectMetadataRpcClient(address, *redirectTopologyVersion, rpcClient); | ||
| 295 | - } | ||
| 296 | - RETURN_IF_NOT_OK(rc); | ||
| 297 | RETURN_RUNTIME_ERROR_IF_NULL(rpcClient); | 269 | RETURN_RUNTIME_ERROR_IF_NULL(rpcClient); |
| 298 | return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched); | 270 | return rpcClient->InvokeQueryAndGet(request, response, payloads, &rpcDispatched); |
| 299 | } | 271 | } |
| 300 | 272 | ||
| 273 | +void ObjectMetadataClient::SwitchInlineRequestToTcp(QueryAndGetReqPb &request, std::vector<RpcMessage> &payloads, | ||
| 274 | + InlineRequestContext &context) const | ||
| 275 | +{ | ||
| 276 | + context.DisableInlineData(); | ||
| 277 | + context.mode = InlineTransportMode::TCP; | ||
| 278 | + (void)request.mutable_data_request()->mutable_tcp(); | ||
| 279 | + payloads.clear(); | ||
| 280 | +} | ||
| 281 | + | ||
| 301 | Status ObjectMetadataClient::QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items, | 282 | Status ObjectMetadataClient::QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items, |
| 302 | - bool allowRedirect, master::QueryAndGetRspPb &response, | 283 | + QueryAndGetRspPb &response, |
| 303 | - std::vector<RpcMessage> &payloads, InlineRequestContext &context, | 284 | + std::vector<RpcMessage> &payloads, InlineRequestContext &context) |
| 304 | - std::optional<uint64_t> redirectTopologyVersion) | ||
| 305 | { | 285 | { |
| 306 | RETURN_RUNTIME_ERROR_IF_NULL(retry_); | 286 | RETURN_RUNTIME_ERROR_IF_NULL(retry_); |
| 307 | CHECK_FAIL_RETURN_STATUS(!items.empty(), K_INVALID, "Metadata query items are empty"); | 287 | CHECK_FAIL_RETURN_STATUS(!items.empty(), K_INVALID, "Metadata query items are empty"); |
| 308 | int64_t backoffMs = 1; | 288 | int64_t backoffMs = 1; |
| 309 | size_t attempt = 0; | 289 | size_t attempt = 0; |
| 310 | - // The context keeps prepared UB buffers reusable across RPC retries and the deadline-bounded redirect chain. | 290 | + // The context keeps prepared data-plane state reusable across RPC retries. |
| 311 | while (true) { | 291 | while (true) { |
| 312 | ++attempt; | 292 | ++attempt; |
| 313 | RETURN_IF_NOT_OK(retry_->CheckDeadline()); | 293 | RETURN_IF_NOT_OK(retry_->CheckDeadline()); |
| 314 | - master::QueryAndGetReqPb request; | 294 | + QueryAndGetReqPb request; |
| 315 | - RETURN_IF_NOT_OK(BuildQueryRequest(items, allowRedirect, context, request)); | 295 | + RETURN_IF_NOT_OK(BuildQueryRequest(address, items, context, request)); |
| 316 | response.Clear(); | 296 | response.Clear(); |
| 317 | payloads.clear(); | 297 | payloads.clear(); |
| 318 | VLOG(1) << "[TransportGet][Metadata] Query, meta owner: " << address.ToString() | 298 | VLOG(1) << "[TransportGet][Metadata] Query, meta owner: " << address.ToString() |
| 319 | - << ", key count: " << items.size() << ", redirect: " << allowRedirect | 299 | + << ", key count: " << items.size() << ", attempt: " << attempt; |
| 320 | - << ", attempt: " << attempt; | ||
| 321 | bool rpcDispatched = false; | 300 | bool rpcDispatched = false; |
| 322 | - Status rc = InvokeQueryAndGet(address, request, response, payloads, context, redirectTopologyVersion, | 301 | + Status rc = InvokeQueryAndGet(address, request, response, payloads, context, rpcDispatched); |
| 323 | - rpcDispatched); | 302 | + RETURN_OK_IF_TRUE(rc.IsOk()); |
| 324 | - if (rc.IsError()) { | 303 | + RETURN_IF_NOT_OK(PrepareQueryRetry(address, rc, rpcDispatched, context, backoffMs)); |
| 325 | - const bool routeFailure = IsMetadataOwnerRouteFailure(rc.GetCode()); | ||
| 326 | - if (routeFailure && metadataFailureHandler_) { | ||
| 327 | - metadataFailureHandler_(address, rc); | ||
| 328 | - } | ||
| 329 | - // Tear down the shared channel only when the connection is genuinely unusable: | ||
| 330 | - // a dead peer, a legacy peer-unreachable (K_RPC_UNAVAILABLE), or a transient | ||
| 331 | - // blip we can prove never left the client (request definitely not sent). Pure | ||
| 332 | - // DEADLINE/CANCELLED may have reached the master, so keep the channel and just | ||
| 333 | - // backoff+retry — avoids rebuilding the shared channel on every slow-master | ||
| 334 | - // timeout. K_RPC_UNAVAILABLE is torn down unconditionally to match the legacy | ||
| 335 | - // behaviour: a peer that returned UNAVAILABLE must not be retried on the same | ||
| 336 | - // cached connection. | ||
| 337 | - const bool teardownWarranted = IsNonRetryableRpcError(rc) | ||
| 338 | - || rc.GetCode() == K_RPC_UNAVAILABLE | ||
| 339 | - || (IsRetryableRpcError(rc) && IsBrpcRequestDefinitelyNotSent(rc)); | ||
| 340 | - if (teardownWarranted && manager_ != nullptr) { | ||
| 341 | - manager_->Teardown(address); | ||
| 342 | - } | ||
| 343 | - if (routeFailure) { | ||
| 344 | - VLOG(1) << "[TransportGet][Metadata] Return stale route for outer retry, meta owner: " | ||
| 345 | - << address.ToString() << ", dispatched: " << rpcDispatched | ||
| 346 | - << ", status: " << rc.ToString(); | ||
| 347 | - return Status(K_NOT_READY, STALE_TRANSPORT_SNAPSHOT_MESSAGE); | ||
| 348 | - } | ||
| 349 | - if (!retry_->IsRetryableRpcError(rc)) { | ||
| 350 | - VLOG(1) << "[TransportGet][Metadata] Query failed without retry, meta owner: " | ||
| 351 | - << address.ToString() << ", status: " << rc.ToString(); | ||
| 352 | - return rc; | ||
| 353 | - } | ||
| 354 | - VLOG(1) << "[TransportGet][Metadata] Retrying query, meta owner: " << address.ToString() | ||
| 355 | - << ", status: " << rc.ToString(); | ||
| 356 | - RETURN_IF_NOT_OK(retry_->Backoff(backoffMs)); | ||
| 357 | - continue; | ||
| 358 | - } | ||
| 359 | - if (!response.meta_is_moving()) { | ||
| 360 | - return Status::OK(); | ||
| 361 | - } | ||
| 362 | - VLOG(1) << "[TransportGet][Metadata] Metadata is moving, meta owner: " << address.ToString() | ||
| 363 | - << ", key count: " << items.size(); | ||
| 364 | - RETURN_IF_NOT_OK(retry_->Backoff(backoffMs)); | ||
| 365 | } | 304 | } |
| 366 | } | 305 | } |
| 367 | 306 | ||
| 368 | -Status ObjectMetadataClient::BuildQueryRequest(const ObjectMetadataBatch &items, bool allowRedirect, | 307 | +Status ObjectMetadataClient::PrepareQueryRetry(const HostPort &address, const Status &rc, bool rpcDispatched, |
| 369 | - const InlineRequestContext &context, | 308 | + InlineRequestContext &context, int64_t &backoffMs) |
| 370 | - master::QueryAndGetReqPb &request) const | 309 | +{ |
| 310 | + if (rpcDispatched && context.mode == InlineTransportMode::SHM && context.shmTransporter != nullptr) { | ||
| 311 | + context.shmTransporter->InvalidateSession(context.shmSession); | ||
| 312 | + } | ||
| 313 | + const bool routeFailure = IsMetadataOwnerRouteFailure(rc.GetCode()); | ||
| 314 | + if (routeFailure && metadataFailureHandler_) { | ||
| 315 | + metadataFailureHandler_(address, rc); | ||
| 316 | + } | ||
| 317 | + const bool teardownWarranted = IsNonRetryableRpcError(rc) || rc.GetCode() == K_RPC_UNAVAILABLE | ||
| 318 | + || (IsRetryableRpcError(rc) && IsBrpcRequestDefinitelyNotSent(rc)); | ||
| 319 | + if (teardownWarranted) { | ||
| 320 | + manager_->Teardown(address); | ||
| 321 | + } | ||
| 322 | + if (routeFailure) { | ||
| 323 | + VLOG(1) << "[TransportGet][Metadata] Return stale route for outer retry, meta owner: " | ||
| 324 | + << address.ToString() << ", dispatched: " << rpcDispatched << ", status: " << rc.ToString(); | ||
| 325 | + return Status(K_NOT_READY, STALE_TRANSPORT_SNAPSHOT_MESSAGE); | ||
| 326 | + } | ||
| 327 | + if (rpcDispatched && context.mode == InlineTransportMode::SHM) { | ||
| 328 | + VLOG(1) << "[TransportGet][Metadata] Do not replay an ambiguous SHM QueryAndGet: " << rc.ToString(); | ||
O [P3] SHM dispatched 失败对一切错误码(含可证明未发出的请求)一律 InvalidateSession 且不重放,恢复路径偏长 rpcDispatched 在 WorkerRpcClient::InvokeQueryAndGet 中于 DoInvoke 之前即置 true(worker_rpc_client.cpp:248),因此"签名生成失败/超时计算失败等请求实际未发出"的场景也被当作模糊结果处理:InvalidateSession + 直接返回 rc(非 routeFailure 时),只能依赖外层 STALE 有界重试(5 次 + deadline,object_client_impl.cpp)重建 session 恢复,同一次 Get 时延被显著放大。 建议对 IsBrpcRequestDefinitelyNotSent(rc) 的 SHM 失败走正常重试(请求未发出即无 AddShmUnit 副作用,重放安全),仅对真正可能已到达服务端的失败采用不重放语义。 ![]() ![]() | |||
| 329 | + return rc; | ||
| 330 | + } | ||
| 331 | + if (!retry_->IsRetryableRpcError(rc)) { | ||
| 332 | + VLOG(1) << "[TransportGet][Metadata] Query failed without retry, meta owner: " << address.ToString() | ||
| 333 | + << ", status: " << rc.ToString(); | ||
| 334 | + return rc; | ||
| 335 | + } | ||
| 336 | + VLOG(1) << "[TransportGet][Metadata] Retrying query, meta owner: " << address.ToString() | ||
| 337 | + << ", status: " << rc.ToString(); | ||
| 338 | + return retry_->Backoff(backoffMs); | ||
| 339 | +} | ||
| 340 | + | ||
| 341 | +Status ObjectMetadataClient::BuildQueryRequest(const HostPort &address, const ObjectMetadataBatch &items, | ||
| 342 | + InlineRequestContext &context, QueryAndGetReqPb &request) const | ||
| 371 | { | 343 | { |
| 372 | for (const auto *item : items) { | 344 | for (const auto *item : items) { |
| 373 | request.add_object_keys(item->objectKey); | 345 | request.add_object_keys(item->objectKey); |
| 374 | } | 346 | } |
| 375 | - request.set_redirect(allowRedirect); | 347 | + if (context.mode == InlineTransportMode::SHM) { |
| 348 | + bool sessionAvailable = context.shmSession != nullptr && context.shmSession->IsAlive(); | ||
| 349 | + INJECT_POINT_NO_RETURN("client.transport.query_and_get.shm_session_unavailable_before_build", | ||
| 350 | + [&sessionAvailable]() { sessionAvailable = false; }); | ||
| 351 | + if (!sessionAvailable) { | ||
| 352 | + VLOG(1) << "[TransportGet][Metadata] SHM session is unavailable while building QueryAndGet; " | ||
| 353 | + "selecting UB or TCP fallback"; | ||
| 354 | + RETURN_IF_NOT_OK(PrepareShmInlineFallback(address, items, context)); | ||
| 355 | + } | ||
| 356 | + } | ||
| 376 | return AddInlineDataRequest(items, context, request); | 357 | return AddInlineDataRequest(items, context, request); |
| 377 | } | 358 | } |
| 378 | 359 | ||
| 379 | Status ObjectMetadataClient::ApplyResults(const ObjectMetadataBatch &items, | 360 | Status ObjectMetadataClient::ApplyResults(const ObjectMetadataBatch &items, |
| 380 | - const master::QueryAndGetRspPb &response, | 361 | + const QueryAndGetRspPb &response, |
| 381 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const | 362 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const |
| 382 | { | 363 | { |
| 383 | // Keep the count check because results are accessed positionally below. | 364 | // Keep the count check because results are accessed positionally below. |
| @@ -389,16 +370,18 @@ Status ObjectMetadataClient::ApplyResults(const ObjectMetadataBatch &items, | |||
| 389 | return Status::OK(); | 370 | return Status::OK(); |
| 390 | } | 371 | } |
| 391 | 372 | ||
| 392 | -Status ObjectMetadataClient::ApplyResult(ObjectMetadataItem &item, const master::QueryAndGetResultPb &result, | 373 | +Status ObjectMetadataClient::ApplyResult(ObjectMetadataItem &item, const QueryAndGetResultPb &result, |
| 393 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const | 374 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const |
| 394 | { | 375 | { |
| 395 | const auto &location = result.location(); | 376 | const auto &location = result.location(); |
| 377 | + CHECK_FAIL_RETURN_STATUS(location.object_key() == item.objectKey, K_RUNTIME_ERROR, | ||
| 378 | + "QueryAndGet result key does not match request order"); | ||
| 396 | if (location.object_locations_size() == 0) { | 379 | if (location.object_locations_size() == 0) { |
| 397 | item.status = Status(K_NOT_FOUND, "Object was not found"); | 380 | item.status = Status(K_NOT_FOUND, "Object was not found"); |
| 398 | return Status::OK(); | 381 | return Status::OK(); |
| 399 | } | 382 | } |
| 400 | item.status = Status::OK(); | 383 | item.status = Status::OK(); |
| 401 | - item.location = location; | 384 | + CopyLocation(location, item.location); |
| 402 | if (!result.has_data_result()) { | 385 | if (!result.has_data_result()) { |
| 403 | // Absence of data_result is a per-key fast-path miss; the caller will execute phase two. | 386 | // Absence of data_result is a per-key fast-path miss; the caller will execute phase two. |
| 404 | return Status::OK(); | 387 | return Status::OK(); |
| @@ -411,30 +394,73 @@ Status ObjectMetadataClient::ApplyResult(ObjectMetadataItem &item, const master: | |||
| 411 | DataGetResult data; | 394 | DataGetResult data; |
| 412 | data.response.mutable_error()->set_error_code(K_OK); | 395 | data.response.mutable_error()->set_error_code(K_OK); |
| 413 | data.response.set_data_size(static_cast<int64_t>(location.object_size())); | 396 | data.response.set_data_size(static_cast<int64_t>(location.object_size())); |
| 414 | - Status rc = context.mode == InlineTransportMode::TCP | 397 | + Status rc; |
| 415 | - ? BuildTcpInlineData(result.data_result(), payloads, data) | 398 | + if (context.mode == InlineTransportMode::TCP) { |
| 416 | - : BuildUbInlineData(item, location, context, data); | 399 | + CHECK_FAIL_RETURN_STATUS(!result.data_result().has_shm_info(), K_RUNTIME_ERROR, |
| 400 | + "TCP QueryAndGet returned shared-memory data"); | ||
| 401 | + rc = BuildTcpInlineData(result.data_result(), location.object_size(), payloads, data); | ||
| 402 | + } else if (context.mode == InlineTransportMode::UB) { | ||
| 403 | + CHECK_FAIL_RETURN_STATUS(!result.data_result().has_shm_info(), K_RUNTIME_ERROR, | ||
| 404 | + "UB QueryAndGet returned shared-memory data"); | ||
| 405 | + CHECK_FAIL_RETURN_STATUS(result.data_result().payload_indexes_size() == 0, K_RUNTIME_ERROR, | ||
| 406 | + "UB QueryAndGet returned TCP payload indexes"); | ||
| 407 | + rc = BuildUbInlineData(item, item.location, context, data); | ||
| 408 | + } else { | ||
| 409 | + CHECK_FAIL_RETURN_STATUS(result.data_result().has_shm_info(), K_RUNTIME_ERROR, | ||
| 410 | + "SHM QueryAndGet did not return shared-memory data"); | ||
| 411 | + CHECK_FAIL_RETURN_STATUS(result.data_result().payload_indexes_size() == 0, K_RUNTIME_ERROR, | ||
| 412 | + "SHM QueryAndGet returned TCP payload indexes"); | ||
| 413 | + rc = BuildShmInlineData(item, result.data_result().shm_info(), context, data); | ||
| 414 | + } | ||
| 415 | + if (rc.IsError() && context.mode == InlineTransportMode::SHM) { | ||
| 416 | + VLOG(1) << "[ObjectKey " << item.objectKey | ||
| 417 | + << "] QueryAndGet SHM materialization fallback: " << rc.ToString(); | ||
| 418 | + if (context.shmTransporter != nullptr) { | ||
| 419 | + context.shmTransporter->InvalidateSession(context.shmSession); | ||
[严重][正确性] [P1] 单个 SHM 物化失败不能撤销同 session 的全部既有 Buffer 引用 证据:这里在任一 key 的 问题:一次冷 FD、认证或本地物化错误被提升成 session 级断开,但 session 同时还是此前所有零拷贝 Buffer 的服务端引用所有者;错误恢复跨越了仍然存活的 Buffer 生命周期。 影响:Worker 可以回收或复用旧 Buffer 对应的 SHM,而应用仍持有其映射指针,导致已成功返回的数据随后被覆盖或读到错误内容,属于内存生命周期和数据正确性风险。 建议:不要因单 key 物化失败立即 disconnect 整个 session。精确释放本次响应新增但未交付的 验证:先通过同一 session 获取并保留 Buffer A,再对 Buffer B 注入 SHM materialization failure;验证 A 在失败、fallback 和新 session 建立后内容仍稳定,Worker 引用直到 A 析构才减少,并覆盖 multi-ref 与 legacy refcount 两种 Worker。 ![]() ![]() | |||
| 420 | + } | ||
| 421 | + return Status::OK(); | ||
| 422 | + } | ||
| 417 | RETURN_IF_NOT_OK(rc); | 423 | RETURN_IF_NOT_OK(rc); |
| 418 | item.inlineData.emplace(std::move(data)); | 424 | item.inlineData.emplace(std::move(data)); |
| 419 | return Status::OK(); | 425 | return Status::OK(); |
| 420 | } | 426 | } |
| 421 | 427 | ||
| 422 | -Status ObjectMetadataClient::BuildTcpInlineData(const master::QueryAndGetDataResultPb &dataResult, | 428 | +Status ObjectMetadataClient::BuildTcpInlineData(const QueryAndGetDataResultPb &dataResult, uint64_t objectSize, |
| 423 | std::vector<RpcMessage> &payloads, | 429 | std::vector<RpcMessage> &payloads, |
| 424 | DataGetResult &data) const | 430 | DataGetResult &data) const |
| 425 | { | 431 | { |
| 426 | data.rpcPayloads.reserve(dataResult.payload_indexes_size()); | 432 | data.rpcPayloads.reserve(dataResult.payload_indexes_size()); |
| 433 | + uint64_t payloadSize = 0; | ||
| 427 | for (uint32_t payloadIndex : dataResult.payload_indexes()) { | 434 | for (uint32_t payloadIndex : dataResult.payload_indexes()) { |
| 428 | - // Keep only the check required before indexing RPC payload storage. | ||
| 429 | CHECK_FAIL_RETURN_STATUS(payloadIndex < payloads.size(), K_RUNTIME_ERROR, | 435 | CHECK_FAIL_RETURN_STATUS(payloadIndex < payloads.size(), K_RUNTIME_ERROR, |
| 430 | "QueryAndGet payload index is out of range"); | 436 | "QueryAndGet payload index is out of range"); |
| 437 | + CHECK_FAIL_RETURN_STATUS( | ||
| 438 | + payloadSize <= objectSize && payloads[payloadIndex].Size() <= objectSize - payloadSize, | ||
| 439 | + K_RUNTIME_ERROR, "QueryAndGet TCP payload exceeds object size"); | ||
| 440 | + payloadSize += payloads[payloadIndex].Size(); | ||
| 431 | data.rpcPayloads.emplace_back(std::move(payloads[payloadIndex])); | 441 | data.rpcPayloads.emplace_back(std::move(payloads[payloadIndex])); |
| 432 | } | 442 | } |
| 443 | + CHECK_FAIL_RETURN_STATUS(payloadSize == objectSize, K_RUNTIME_ERROR, | ||
| 444 | + "QueryAndGet TCP payload size does not match object size"); | ||
| 433 | data.response.set_data_source(DataTransferSource::DATA_IN_PAYLOAD); | 445 | data.response.set_data_source(DataTransferSource::DATA_IN_PAYLOAD); |
| 434 | data.kind = AccessTransportKind::TCP; | 446 | data.kind = AccessTransportKind::TCP; |
| 435 | return Status::OK(); | 447 | return Status::OK(); |
| 436 | } | 448 | } |
| 437 | 449 | ||
| 450 | +Status ObjectMetadataClient::BuildShmInlineData(ObjectMetadataItem &item, const QueryAndGetShmInfoPb &shmInfo, | ||
| 451 | + InlineRequestContext &context, DataGetResult &data) const | ||
| 452 | +{ | ||
| 453 | + INJECT_POINT("client.transport.query_and_get.shm_materialization_failure"); | ||
| 454 | + CHECK_FAIL_RETURN_STATUS(context.shmSession != nullptr && context.shmTransporter != nullptr | ||
| 455 | + && context.readContext != nullptr, | ||
| 456 | + K_RUNTIME_ERROR, "SHM QueryAndGet session context is missing"); | ||
| 457 | + CHECK_FAIL_RETURN_STATUS(shmInfo.data_size() >= 0 | ||
| 458 | + && static_cast<uint64_t>(shmInfo.data_size()) == item.location.object_size(), | ||
| 459 | + K_RUNTIME_ERROR, "SHM QueryAndGet data size does not match object size"); | ||
| 460 | + DataGetRequest input{ item.objectKey, static_cast<uint64_t>(shmInfo.data_size()), context.readContext }; | ||
| 461 | + return context.shmSession->BuildQueryAndGetResult(shmInfo, input, data); | ||
| 462 | +} | ||
| 463 | + | ||
| 438 | Status ObjectMetadataClient::BuildUbInlineData(ObjectMetadataItem &item, | 464 | Status ObjectMetadataClient::BuildUbInlineData(ObjectMetadataItem &item, |
| 439 | const master::ObjectLocationInfoPb &location, | 465 | const master::ObjectLocationInfoPb &location, |
| 440 | InlineRequestContext &context, DataGetResult &data) const | 466 | InlineRequestContext &context, DataGetResult &data) const |
| @@ -457,59 +483,28 @@ Status ObjectMetadataClient::BuildUbInlineData(ObjectMetadataItem &item, | |||
| 457 | } | 483 | } |
| 458 | 484 | ||
| 459 | Status ObjectMetadataClient::Query(const HostPort &address, const ObjectMetadataBatch &items, | 485 | Status ObjectMetadataClient::Query(const HostPort &address, const ObjectMetadataBatch &items, |
| 460 | - bool enableInlineData) | 486 | + bool enableInlineData, std::shared_ptr<const TransportReadContext> readContext) |
| 461 | { | 487 | { |
| 462 | RETURN_IF_NOT_OK(ValidateAndResetItems(items)); | 488 | RETURN_IF_NOT_OK(ValidateAndResetItems(items)); |
| 463 | InlineRequestContext context; | 489 | InlineRequestContext context; |
| 464 | if (enableInlineData) { | 490 | if (enableInlineData) { |
| 465 | - RETURN_IF_NOT_OK(InitializeInlineRequest(address, items, context)); | 491 | + RETURN_IF_NOT_OK(InitializeInlineRequest(address, items, std::move(readContext), context)); |
| 466 | } | 492 | } |
| 467 | 493 | ||
| 468 | - master::QueryAndGetRspPb response; | 494 | + QueryAndGetRspPb response; |
| 469 | std::vector<RpcMessage> payloads; | 495 | std::vector<RpcMessage> payloads; |
| 470 | - RETURN_IF_NOT_OK(QueryWithRetry(address, items, true, response, payloads, context, std::nullopt)); | 496 | + RETURN_IF_NOT_OK(QueryWithRetry(address, items, response, payloads, context)); |
| 471 | - ObjectMetadataBatch localItems; | 497 | + Status rc = ApplyResults(items, response, payloads, context); |
| 472 | - std::vector<RedirectBatch> redirectBatches; | 498 | + if (rc.IsError() && context.mode == InlineTransportMode::SHM && context.shmTransporter != nullptr) { |
| 473 | - RETURN_IF_NOT_OK(PartitionInitialResponse(items, response, localItems, redirectBatches, std::nullopt)); | 499 | + context.shmTransporter->InvalidateSession(context.shmSession); |
| 474 | - VLOG(1) << "[TransportGet][Metadata] Query resolved, meta owner: " << address.ToString() | ||
| 475 | - << ", local keys: " << localItems.size() << ", redirect groups: " << redirectBatches.size(); | ||
| 476 | - RETURN_IF_NOT_OK(ApplyResults(localItems, response, payloads, context)); | ||
| 477 | - if (redirectBatches.empty()) { | ||
| 478 | - return Status::OK(); | ||
| 479 | } | 500 | } |
| 480 | - | 501 | + return rc; |
| 481 | - std::deque<PendingMetadataBatch> pending; | ||
| 482 | - QueueRedirectBatches(redirectBatches, pending); | ||
| 483 | - while (!pending.empty()) { | ||
| 484 | - PendingMetadataBatch current = std::move(pending.front()); | ||
| 485 | - pending.pop_front(); | ||
| 486 | - response.Clear(); | ||
| 487 | - payloads.clear(); | ||
| 488 | - Status rc = QueryWithRetry(current.address, current.items, true, response, payloads, context, | ||
| 489 | - current.topologyVersion); | ||
| 490 | - localItems.clear(); | ||
| 491 | - redirectBatches.clear(); | ||
| 492 | - if (rc.IsOk()) { | ||
| 493 | - rc = PartitionInitialResponse(current.items, response, localItems, redirectBatches, | ||
| 494 | - current.topologyVersion); | ||
| 495 | - } | ||
| 496 | - if (rc.IsOk()) { | ||
| 497 | - VLOG(1) << "[TransportGet][Metadata] Query resolved, meta owner: " << current.address.ToString() | ||
| 498 | - << ", local keys: " << localItems.size() << ", redirect groups: " << redirectBatches.size(); | ||
| 499 | - rc = ApplyResults(localItems, response, payloads, context); | ||
| 500 | - } | ||
| 501 | - if (rc.IsError()) { | ||
| 502 | - SetBatchError(current.items, rc); | ||
| 503 | - continue; | ||
| 504 | - } | ||
| 505 | - QueueRedirectBatches(redirectBatches, pending); | ||
| 506 | - } | ||
| 507 | - return Status::OK(); | ||
| 508 | } | 502 | } |
| 509 | 503 | ||
| 510 | -Status ObjectMetadataClient::QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items) | 504 | +Status ObjectMetadataClient::QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items, |
| 505 | + std::shared_ptr<const TransportReadContext> readContext) | ||
| 511 | { | 506 | { |
| 512 | - return Query(address, items, true); | 507 | + return Query(address, items, true, std::move(readContext)); |
| 513 | } | 508 | } |
| 514 | 509 | ||
| 515 | Status ObjectMetadataClient::QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items) | 510 | Status ObjectMetadataClient::QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items) |
| @@ -31,6 +31,7 @@ | |||
| 31 | 31 | ||
| 32 | 32 | ||
| 33 | 33 | ||
| 34 | + | ||
| 34 | 35 | ||
| 35 | namespace datasystem { | 36 | namespace datasystem { |
| 36 | namespace client { | 37 | namespace client { |
| @@ -55,23 +56,28 @@ public: | |||
| 55 | virtual ~ObjectMetadataClient() = default; | 56 | virtual ~ObjectMetadataClient() = default; |
| 56 | 57 | ||
| 57 | /** | 58 | /** |
| 58 | - * @brief Query a metadata-owner group and resolve redirects within the API deadline. | 59 | + * @brief Query a metadata-owner Worker for ordered locations and optional inline data. |
| 59 | * @param[in] address Initial metadata owner. | 60 | * @param[in] address Initial metadata owner. |
| 60 | * @param[in,out] items Ordered object metadata states belonging to address. | 61 | * @param[in,out] items Ordered object metadata states belonging to address. |
| 62 | + * @param[in] readContext Read context required by the shared-memory transport. | ||
| 61 | * @return K_OK after every item has an independent result; a group-wide error otherwise. | 63 | * @return K_OK after every item has an independent result; a group-wide error otherwise. |
| 62 | */ | 64 | */ |
| 63 | - virtual Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items); | 65 | + virtual Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items, |
| 66 | + std::shared_ptr<const TransportReadContext> readContext); | ||
| 64 | 67 | ||
| 65 | /** @brief Query object locations without requesting inline object data. */ | 68 | /** @brief Query object locations without requesting inline object data. */ |
| 66 | virtual Status QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items); | 69 | virtual Status QueryMetadata(const HostPort &address, const ObjectMetadataBatch &items); |
| 67 | 70 | ||
| 68 | private: | 71 | private: |
| 69 | - enum class InlineTransportMode : uint8_t { NONE = 0, TCP = 1, UB = 2 }; | 72 | + enum class InlineTransportMode : uint8_t { NONE = 0, TCP = 1, UB = 2, SHM = 3 }; |
| 70 | 73 | ||
| 71 | struct InlineRequestContext { | 74 | struct InlineRequestContext { |
| 72 | InlineTransportMode mode = InlineTransportMode::NONE; | 75 | InlineTransportMode mode = InlineTransportMode::NONE; |
| 73 | std::unordered_map<ObjectMetadataItem *, UbReceiveBuffer> ubBuffers; | 76 | std::unordered_map<ObjectMetadataItem *, UbReceiveBuffer> ubBuffers; |
| 74 | std::string transportInstanceId; | 77 | std::string transportInstanceId; |
| 78 | + std::shared_ptr<ShmTransporter> shmTransporter; | ||
| 79 | + std::shared_ptr<ShmSession> shmSession; | ||
| 80 | + std::shared_ptr<const TransportReadContext> readContext; | ||
| 75 | 81 | ||
| 76 | /** @brief Disable inline transfer and release prepared receive buffers. */ | 82 | /** @brief Disable inline transfer and release prepared receive buffers. */ |
| 77 | void DisableInlineData() | 83 | void DisableInlineData() |
| @@ -79,20 +85,33 @@ private: | |||
| 79 | mode = InlineTransportMode::NONE; | 85 | mode = InlineTransportMode::NONE; |
| 80 | ubBuffers.clear(); | 86 | ubBuffers.clear(); |
| 81 | transportInstanceId.clear(); | 87 | transportInstanceId.clear(); |
| 88 | + shmTransporter.reset(); | ||
| 89 | + shmSession.reset(); | ||
| 90 | + readContext.reset(); | ||
| 82 | } | 91 | } |
| 83 | }; | 92 | }; |
| 84 | 93 | ||
| 85 | - Status Query(const HostPort &address, const ObjectMetadataBatch &items, bool enableInlineData); | 94 | + Status Query(const HostPort &address, const ObjectMetadataBatch &items, bool enableInlineData, |
| 95 | + std::shared_ptr<const TransportReadContext> readContext = nullptr); | ||
| 86 | 96 | ||
| 87 | - Status QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items, bool allowRedirect, | 97 | + Status QueryWithRetry(const HostPort &address, const ObjectMetadataBatch &items, |
| 88 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, | 98 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, |
| 89 | - InlineRequestContext &context, std::optional<uint64_t> redirectTopologyVersion); | 99 | + InlineRequestContext &context); |
| 90 | - Status BuildQueryRequest(const ObjectMetadataBatch &items, bool allowRedirect, | 100 | + Status BuildQueryRequest(const HostPort &address, const ObjectMetadataBatch &items, |
| 91 | - const InlineRequestContext &context, master::QueryAndGetReqPb &request) const; | 101 | + InlineRequestContext &context, QueryAndGetReqPb &request) const; |
| 92 | - Status InvokeQueryAndGet(const HostPort &address, master::QueryAndGetReqPb &request, | 102 | + Status InvokeQueryAndGet(const HostPort &address, QueryAndGetReqPb &request, |
| 93 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, | 103 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, |
| 94 | - InlineRequestContext &context, std::optional<uint64_t> redirectTopologyVersion, | 104 | + InlineRequestContext &context, bool &rpcDispatched); |
| 95 | - bool &rpcDispatched); | 105 | + Status InvokeInlineQueryAndGet(const HostPort &address, QueryAndGetReqPb &request, |
| 106 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, | ||
| 107 | + InlineRequestContext &context, bool &invoked, bool &rpcDispatched); | ||
| 108 | + Status InvokeTcpQueryAndGet(const HostPort &address, QueryAndGetReqPb &request, | ||
| 109 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads, | ||
| 110 | + bool &rpcDispatched); | ||
| 111 | + void SwitchInlineRequestToTcp(QueryAndGetReqPb &request, std::vector<RpcMessage> &payloads, | ||
| 112 | + InlineRequestContext &context) const; | ||
| 113 | + Status PrepareQueryRetry(const HostPort &address, const Status &rc, bool rpcDispatched, | ||
| 114 | + InlineRequestContext &context, int64_t &backoffMs); | ||
| 96 | 115 | ||
| 97 | /** | 116 | /** |
| 98 | * @brief Select and initialize the inline transport for one metadata-owner request. | 117 | * @brief Select and initialize the inline transport for one metadata-owner request. |
| @@ -102,8 +121,17 @@ private: | |||
| 102 | * @return K_OK on success; the error code otherwise. | 121 | * @return K_OK on success; the error code otherwise. |
| 103 | */ | 122 | */ |
| 104 | Status InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items, | 123 | Status InitializeInlineRequest(const HostPort &address, const ObjectMetadataBatch &items, |
| 124 | + std::shared_ptr<const TransportReadContext> readContext, | ||
| 105 | InlineRequestContext &context) const; | 125 | InlineRequestContext &context) const; |
| 106 | 126 | ||
| 127 | + Status PrepareShmInlineRequest(const HostPort &address, | ||
| 128 | + std::shared_ptr<const TransportReadContext> readContext, | ||
| 129 | + InlineRequestContext &context) const; | ||
| 130 | + | ||
| 131 | + /** @brief Replace an unavailable SHM inline request with an available UB or TCP request. */ | ||
| 132 | + Status PrepareShmInlineFallback(const HostPort &address, const ObjectMetadataBatch &items, | ||
| 133 | + InlineRequestContext &context) const; | ||
| 134 | + | ||
| 107 | /** | 135 | /** |
| 108 | * @brief Prepare UB inline transfer when the endpoint and client configuration support it. | 136 | * @brief Prepare UB inline transfer when the endpoint and client configuration support it. |
| 109 | * @param[in] address Metadata-owner address. | 137 | * @param[in] address Metadata-owner address. |
| @@ -130,7 +158,7 @@ private: | |||
| 130 | * @return K_OK on success; the error code otherwise. | 158 | * @return K_OK on success; the error code otherwise. |
| 131 | */ | 159 | */ |
| 132 | Status AddInlineDataRequest(const ObjectMetadataBatch &items, const InlineRequestContext &context, | 160 | Status AddInlineDataRequest(const ObjectMetadataBatch &items, const InlineRequestContext &context, |
| 133 | - master::QueryAndGetReqPb &request) const; | 161 | + QueryAndGetReqPb &request) const; |
| 134 | 162 | ||
| 135 | /** | 163 | /** |
| 136 | * @brief Apply ordered metadata and inline-data results to a metadata batch. | 164 | * @brief Apply ordered metadata and inline-data results to a metadata batch. |
| @@ -140,7 +168,7 @@ private: | |||
| 140 | * @param[in,out] context Inline-request context. | 168 | * @param[in,out] context Inline-request context. |
| 141 | * @return K_OK on success; the error code otherwise. | 169 | * @return K_OK on success; the error code otherwise. |
| 142 | */ | 170 | */ |
| 143 | - Status ApplyResults(const ObjectMetadataBatch &items, const master::QueryAndGetRspPb &response, | 171 | + Status ApplyResults(const ObjectMetadataBatch &items, const QueryAndGetRspPb &response, |
| 144 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const; | 172 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const; |
| 145 | 173 | ||
| 146 | /** | 174 | /** |
| @@ -151,7 +179,7 @@ private: | |||
| 151 | * @param[in,out] context Inline-request context. | 179 | * @param[in,out] context Inline-request context. |
| 152 | * @return K_OK on success; the error code otherwise. | 180 | * @return K_OK on success; the error code otherwise. |
| 153 | */ | 181 | */ |
| 154 | - Status ApplyResult(ObjectMetadataItem &item, const master::QueryAndGetResultPb &result, | 182 | + Status ApplyResult(ObjectMetadataItem &item, const QueryAndGetResultPb &result, |
| 155 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const; | 183 | std::vector<RpcMessage> &payloads, InlineRequestContext &context) const; |
| 156 | 184 | ||
| 157 | /** | 185 | /** |
| @@ -161,9 +189,12 @@ private: | |||
| 161 | * @param[out] data Data-read result. | 189 | * @param[out] data Data-read result. |
| 162 | * @return K_OK on success; the error code otherwise. | 190 | * @return K_OK on success; the error code otherwise. |
| 163 | */ | 191 | */ |
| 164 | - Status BuildTcpInlineData(const master::QueryAndGetDataResultPb &dataResult, | 192 | + Status BuildTcpInlineData(const QueryAndGetDataResultPb &dataResult, uint64_t objectSize, |
| 165 | std::vector<RpcMessage> &payloads, DataGetResult &data) const; | 193 | std::vector<RpcMessage> &payloads, DataGetResult &data) const; |
| 166 | 194 | ||
| 195 | + Status BuildShmInlineData(ObjectMetadataItem &item, const QueryAndGetShmInfoPb &shmInfo, | ||
| 196 | + InlineRequestContext &context, DataGetResult &data) const; | ||
| 197 | + | ||
| 167 | /** | 198 | /** |
| 168 | * @brief Move ownership of a prepared UB receive buffer into a data-read result. | 199 | * @brief Move ownership of a prepared UB receive buffer into a data-read result. |
| 169 | * @param[in,out] item Object metadata item owning the buffer association. | 200 | * @param[in,out] item Object metadata item owning the buffer association. |
| @@ -46,11 +46,13 @@ struct MetadataGroup { | |||
| 46 | { | 46 | { |
| 47 | } | 47 | } |
| 48 | 48 | ||
| 49 | - void Resolve(ObjectMetadataClient &metadata, const Status &dispatchStatus, bool metadataOnly) | 49 | + void Resolve(ObjectMetadataClient &metadata, const Status &dispatchStatus, bool metadataOnly, |
| 50 | + const std::shared_ptr<const TransportReadContext> &context) | ||
| 50 | { | 51 | { |
| 51 | Status status = dispatchStatus; | 52 | Status status = dispatchStatus; |
| 52 | if (status.IsOk()) { | 53 | if (status.IsOk()) { |
| 53 | - status = metadataOnly ? metadata.QueryMetadata(address, items) : metadata.QueryAndGet(address, items); | 54 | + status = metadataOnly ? metadata.QueryMetadata(address, items) |
| 55 | + : metadata.QueryAndGet(address, items, context); | ||
| 54 | } | 56 | } |
| 55 | if (status.IsError()) { | 57 | if (status.IsError()) { |
| 56 | for (auto *item : items) { | 58 | for (auto *item : items) { |
| @@ -144,7 +146,7 @@ Status InitializeItems(const ObjectReadRequest &request, std::vector<ReadItem> & | |||
| 144 | } | 146 | } |
| 145 | 147 | ||
| 146 | void QueryMetadata(ObjectMetadataClient &metadata, ThreadPool &taskPool, std::vector<ReadItem> &items, | 148 | void QueryMetadata(ObjectMetadataClient &metadata, ThreadPool &taskPool, std::vector<ReadItem> &items, |
| 147 | - bool metadataOnly = false) | 149 | + const std::shared_ptr<const TransportReadContext> &context, bool metadataOnly = false) |
| 148 | { | 150 | { |
| 149 | auto groups = GroupByMetaOwner(items); | 151 | auto groups = GroupByMetaOwner(items); |
| 150 | VLOG(1) << "[TransportGet][Flow] Query metadata, key count: " << items.size() | 152 | VLOG(1) << "[TransportGet][Flow] Query metadata, key count: " << items.size() |
| @@ -156,8 +158,8 @@ void QueryMetadata(ObjectMetadataClient &metadata, ThreadPool &taskPool, std::ve | |||
| 156 | for (auto &group : groups) { | 158 | for (auto &group : groups) { |
| 157 | tasks.emplace_back(&group); | 159 | tasks.emplace_back(&group); |
| 158 | } | 160 | } |
| 159 | - RunTasks(taskPool, tasks, [&metadata, metadataOnly](MetadataGroup &group, const Status &dispatchStatus) { | 161 | + RunTasks(taskPool, tasks, [&metadata, metadataOnly, context](MetadataGroup &group, const Status &dispatchStatus) { |
| 160 | - group.Resolve(metadata, dispatchStatus, metadataOnly); | 162 | + group.Resolve(metadata, dispatchStatus, metadataOnly, context); |
| 161 | }); | 163 | }); |
| 162 | const auto resolved = std::count_if(items.begin(), items.end(), [](const ReadItem &item) { | 164 | const auto resolved = std::count_if(items.begin(), items.end(), [](const ReadItem &item) { |
| 163 | return item.metadata.status.IsOk(); | 165 | return item.metadata.status.IsOk(); |
| @@ -251,7 +253,7 @@ Status ObjectReadFlow::ResolveMetadata(const ObjectReadRequest &request, | |||
| 251 | RETURN_RUNTIME_ERROR_IF_NULL(taskPool_); | 253 | RETURN_RUNTIME_ERROR_IF_NULL(taskPool_); |
| 252 | std::vector<ReadItem> items; | 254 | std::vector<ReadItem> items; |
| 253 | RETURN_IF_NOT_OK(InitializeItems(request, items)); | 255 | RETURN_IF_NOT_OK(InitializeItems(request, items)); |
| 254 | - QueryMetadata(*metadata_, *taskPool_, items, true); | 256 | + QueryMetadata(*metadata_, *taskPool_, items, nullptr, true); |
| 255 | size_t resultSize = 0; | 257 | size_t resultSize = 0; |
| 256 | for (const auto &input : request.items) { | 258 | for (const auto &input : request.items) { |
| 257 | resultSize = std::max(resultSize, input.requestIndex + 1); | 259 | resultSize = std::max(resultSize, input.requestIndex + 1); |
| @@ -275,7 +277,7 @@ Status ObjectReadFlow::Run(const ObjectReadRequest &request, ObjectReadResult &r | |||
| 275 | std::vector<ReadItem> items; | 277 | std::vector<ReadItem> items; |
| 276 | RETURN_IF_NOT_OK(InitializeItems(request, items)); | 278 | RETURN_IF_NOT_OK(InitializeItems(request, items)); |
| 277 | AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_START); | 279 | AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_START); |
| 278 | - QueryMetadata(*metadata_, *taskPool_, items); | 280 | + QueryMetadata(*metadata_, *taskPool_, items, request.context); |
| 279 | AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_END); | 281 | AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_QUERY_AND_GET_END); |
| 280 | AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_GET_DATA_START); | 282 | AddLatencyTickIfEnabled(request.traceEnabled, LatencyTickKey::CLIENT_DIRECT_GET_DATA_START); |
| 281 | ReadObjects(*replicas_, items, request.context, request.traceEnabled); | 283 | ReadObjects(*replicas_, items, request.context, request.traceEnabled); |
| @@ -76,18 +76,14 @@ Status WorkerRpcClient::Init() | |||
| 76 | auto transportStub = std::make_shared<WorkerWorkerTransportService_BrpcGenericStub>( | 76 | auto transportStub = std::make_shared<WorkerWorkerTransportService_BrpcGenericStub>( |
| 77 | channel.get(), channelConfig_.timeout_ms); | 77 | channel.get(), channelConfig_.timeout_ms); |
| 78 | auto dataStub = std::make_shared<WorkerWorkerOCService_BrpcGenericStub>(channel.get(), channelConfig_.timeout_ms); | 78 | auto dataStub = std::make_shared<WorkerWorkerOCService_BrpcGenericStub>(channel.get(), channelConfig_.timeout_ms); |
| 79 | - auto masterStub = | ||
| 80 | - std::make_shared<master::MasterOCService_BrpcGenericStub>(channel.get(), channelConfig_.timeout_ms); | ||
| 81 | CHECK_FAIL_RETURN_STATUS(workerStub != nullptr && controlStub != nullptr && transportStub != nullptr | 79 | CHECK_FAIL_RETURN_STATUS(workerStub != nullptr && controlStub != nullptr && transportStub != nullptr |
| 82 | - && dataStub != nullptr | 80 | + && dataStub != nullptr, |
| 83 | - && masterStub != nullptr, | ||
| 84 | K_RUNTIME_ERROR, "Failed to create routed worker RPC stubs"); | 81 | K_RUNTIME_ERROR, "Failed to create routed worker RPC stubs"); |
| 85 | channel_ = std::move(channel); | 82 | channel_ = std::move(channel); |
| 86 | workerStub_ = std::move(workerStub); | 83 | workerStub_ = std::move(workerStub); |
| 87 | controlStub_ = std::move(controlStub); | 84 | controlStub_ = std::move(controlStub); |
| 88 | transportStub_ = std::move(transportStub); | 85 | transportStub_ = std::move(transportStub); |
| 89 | dataStub_ = std::move(dataStub); | 86 | dataStub_ = std::move(dataStub); |
| 90 | - masterStub_ = std::move(masterStub); | ||
| 91 | connectionGeneration_ = nextConnectionGeneration_.fetch_add(1, std::memory_order_relaxed); | 87 | connectionGeneration_ = nextConnectionGeneration_.fetch_add(1, std::memory_order_relaxed); |
| 92 | alive_.store(true, std::memory_order_release); | 88 | alive_.store(true, std::memory_order_release); |
| 93 | return Status::OK(); | 89 | return Status::OK(); |
| @@ -111,10 +107,10 @@ Status WorkerRpcClient::DoInvokeBatchGetObject(const RpcOptions &options, const | |||
| 111 | return dataStub_->BatchGetObjectRemote(options, request, response, payloads); | 107 | return dataStub_->BatchGetObjectRemote(options, request, response, payloads); |
| 112 | } | 108 | } |
| 113 | 109 | ||
| 114 | -Status WorkerRpcClient::DoInvokeQueryAndGet(const RpcOptions &options, const master::QueryAndGetReqPb &request, | 110 | +Status WorkerRpcClient::DoInvokeQueryAndGet(const RpcOptions &options, const QueryAndGetReqPb &request, |
| 115 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) | 111 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) |
| 116 | { | 112 | { |
| 117 | - return masterStub_->QueryAndGet(options, request, response, payloads); | 113 | + return controlStub_->QueryAndGet(options, request, response, payloads); |
| 118 | } | 114 | } |
| 119 | 115 | ||
| 120 | Status WorkerRpcClient::DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response) | 116 | Status WorkerRpcClient::DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response) |
| @@ -234,14 +230,14 @@ Status WorkerRpcClient::InvokeBatchGetObject(BatchGetObjectRemoteReqPb &request, | |||
| 234 | return rc.IsError() ? WithRpcDiag(rc, "BatchGetObjectRemote", workerAddress_) : Status::OK(); | 230 | return rc.IsError() ? WithRpcDiag(rc, "BatchGetObjectRemote", workerAddress_) : Status::OK(); |
| 235 | } | 231 | } |
| 236 | 232 | ||
| 237 | -Status WorkerRpcClient::InvokeQueryAndGet(master::QueryAndGetReqPb &request, master::QueryAndGetRspPb &response, | 233 | +Status WorkerRpcClient::InvokeQueryAndGet(QueryAndGetReqPb &request, QueryAndGetRspPb &response, |
| 238 | std::vector<RpcMessage> &payloads, bool *rpcDispatched) | 234 | std::vector<RpcMessage> &payloads, bool *rpcDispatched) |
| 239 | { | 235 | { |
| 240 | if (rpcDispatched != nullptr) { | 236 | if (rpcDispatched != nullptr) { |
| 241 | *rpcDispatched = false; | 237 | *rpcDispatched = false; |
| 242 | } | 238 | } |
| 243 | CHECK_FAIL_RETURN_STATUS(IsAlive(), K_RPC_UNAVAILABLE, | 239 | CHECK_FAIL_RETURN_STATUS(IsAlive(), K_RPC_UNAVAILABLE, |
| 244 | - "Routed master RPC client is not initialized"); | 240 | + "Routed worker RPC client is not initialized"); |
| 245 | int32_t rpcTimeout; | 241 | int32_t rpcTimeout; |
| 246 | RETURN_IF_NOT_OK(GetRpcTimeout(channelConfig_.timeout_ms, rpcTimeout)); | 242 | RETURN_IF_NOT_OK(GetRpcTimeout(channelConfig_.timeout_ms, rpcTimeout)); |
| 247 | RETURN_IF_NOT_OK(signature_->GenerateSignature(request)); | 243 | RETURN_IF_NOT_OK(signature_->GenerateSignature(request)); |
| @@ -503,13 +499,12 @@ bool WorkerRpcClient::IsAlive() const | |||
| 503 | { | 499 | { |
| 504 | return alive_.load(std::memory_order_acquire) && channel_ != nullptr && workerStub_ != nullptr | 500 | return alive_.load(std::memory_order_acquire) && channel_ != nullptr && workerStub_ != nullptr |
| 505 | && controlStub_ != nullptr | 501 | && controlStub_ != nullptr |
| 506 | - && transportStub_ != nullptr && dataStub_ != nullptr && masterStub_ != nullptr; | 502 | + && transportStub_ != nullptr && dataStub_ != nullptr; |
| 507 | } | 503 | } |
| 508 | 504 | ||
| 509 | void WorkerRpcClient::Close() | 505 | void WorkerRpcClient::Close() |
| 510 | { | 506 | { |
| 511 | alive_.store(false, std::memory_order_release); | 507 | alive_.store(false, std::memory_order_release); |
| 512 | - masterStub_.reset(); | ||
| 513 | dataStub_.reset(); | 508 | dataStub_.reset(); |
| 514 | controlStub_.reset(); | 509 | controlStub_.reset(); |
| 515 | workerStub_.reset(); | 510 | workerStub_.reset(); |
| @@ -31,7 +31,6 @@ | |||
| 31 | 31 | ||
| 32 | 32 | ||
| 33 | 33 | ||
| 34 | - | ||
| 35 | 34 | ||
| 36 | 35 | ||
| 37 | 36 | ||
| @@ -61,8 +60,8 @@ public: | |||
| 61 | virtual Status InvokeBatchGetObject(BatchGetObjectRemoteReqPb &request, BatchGetObjectRemoteRspPb &response, | 60 | virtual Status InvokeBatchGetObject(BatchGetObjectRemoteReqPb &request, BatchGetObjectRemoteRspPb &response, |
| 62 | std::vector<RpcMessage> &payloads); | 61 | std::vector<RpcMessage> &payloads); |
| 63 | 62 | ||
| 64 | - /** @brief Sign and query object locations and optional inline data from the master service at this endpoint. */ | 63 | + /** @brief Sign and query ordered object locations and optional inline data from this Worker. */ |
| 65 | - virtual Status InvokeQueryAndGet(master::QueryAndGetReqPb &request, master::QueryAndGetRspPb &response, | 64 | + virtual Status InvokeQueryAndGet(QueryAndGetReqPb &request, QueryAndGetRspPb &response, |
| 66 | std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr); | 65 | std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr); |
| 67 | 66 | ||
| 68 | /** @brief Sign and invoke Exist through the cached control connection. */ | 67 | /** @brief Sign and invoke Exist through the cached control connection. */ |
| @@ -128,8 +127,8 @@ protected: | |||
| 128 | virtual Status DoInvokeBatchGetObject(const RpcOptions &options, const BatchGetObjectRemoteReqPb &request, | 127 | virtual Status DoInvokeBatchGetObject(const RpcOptions &options, const BatchGetObjectRemoteReqPb &request, |
| 129 | BatchGetObjectRemoteRspPb &response, std::vector<RpcMessage> &payloads); | 128 | BatchGetObjectRemoteRspPb &response, std::vector<RpcMessage> &payloads); |
| 130 | 129 | ||
| 131 | - virtual Status DoInvokeQueryAndGet(const RpcOptions &options, const master::QueryAndGetReqPb &request, | 130 | + virtual Status DoInvokeQueryAndGet(const RpcOptions &options, const QueryAndGetReqPb &request, |
| 132 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads); | 131 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads); |
| 133 | 132 | ||
| 134 | virtual Status DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response); | 133 | virtual Status DoInvokeExist(const RpcOptions &options, const ExistReqPb &request, ExistRspPb &response); |
| 135 | 134 | ||
| @@ -178,7 +177,6 @@ private: | |||
| 178 | std::shared_ptr<WorkerOCService_BrpcGenericStub> controlStub_; | 177 | std::shared_ptr<WorkerOCService_BrpcGenericStub> controlStub_; |
| 179 | std::shared_ptr<WorkerWorkerTransportService_BrpcGenericStub> transportStub_; | 178 | std::shared_ptr<WorkerWorkerTransportService_BrpcGenericStub> transportStub_; |
| 180 | std::shared_ptr<WorkerWorkerOCService_BrpcGenericStub> dataStub_; | 179 | std::shared_ptr<WorkerWorkerOCService_BrpcGenericStub> dataStub_; |
| 181 | - std::shared_ptr<master::MasterOCService_BrpcGenericStub> masterStub_; | ||
| 182 | std::atomic<bool> alive_{ false }; | 180 | std::atomic<bool> alive_{ false }; |
| 183 | uint32_t connectionGeneration_ = 0; | 181 | uint32_t connectionGeneration_ = 0; |
| 184 | static std::atomic<uint32_t> nextConnectionGeneration_; | 182 | static std::atomic<uint32_t> nextConnectionGeneration_; |
| @@ -51,6 +51,7 @@ ACCESS_RECORDER_KEY_DEF(DS_POSIX_PUBLISH, ACCESS) | |||
| 51 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTI_CREATE, ACCESS) | 51 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTI_CREATE, ACCESS) |
| 52 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTIPUBLISH, ACCESS) | 52 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_MULTIPUBLISH, ACCESS) |
| 53 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_GET, ACCESS) | 53 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_GET, ACCESS) |
| 54 | +ACCESS_RECORDER_KEY_DEF(DS_POSIX_QUERY_AND_GET, ACCESS) | ||
| 54 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_DELETE_ALL_COPY, ACCESS) | 55 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_DELETE_ALL_COPY, ACCESS) |
| 55 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_GINCREASEREF, ACCESS) | 56 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_GINCREASEREF, ACCESS) |
| 56 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_GDECREASEREF, ACCESS) | 57 | ACCESS_RECORDER_KEY_DEF(DS_POSIX_GDECREASEREF, ACCESS) |
| @@ -486,36 +486,6 @@ Status MasterOCServiceImpl::GetObjectLocations(const GetObjectLocationsReqPb &re | |||
| 486 | return Status::OK(); | 486 | return Status::OK(); |
| 487 | } | 487 | } |
| 488 | 488 | ||
| 489 | -Status MasterOCServiceImpl::QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &resp, | ||
| 490 | - std::vector<RpcMessage> &payloads) | ||
| 491 | -{ | ||
| 492 | - ScopedRequestContext ctx; | ||
| 493 | - Timer timer; | ||
| 494 | - auto config = GetServerLatencyTraceConfig(); | ||
| 495 | - RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(req), "AK/SK failed."); | ||
| 496 | - VLOG(1) << FormatString("[Master] Processing QueryAndGetReq, target num: %d, redirect: %d", req.object_keys_size(), | ||
| 497 | - req.redirect()); | ||
| 498 | - std::shared_ptr<master::OCMetadataManager> ocMetadataManager; | ||
| 499 | - RETURN_IF_NOT_OK_PRINT_ERROR_MSG(metadataManagerHolder_->GetOcMetadataManager(ocMetadataManager), | ||
| 500 | - "GetOcMetadataManager failed"); | ||
| 501 | - Status status = ocMetadataManager->QueryAndGet(req, resp, payloads); | ||
| 502 | - const auto totalUs = static_cast<uint64_t>(timer.ElapsedMicroSecond()); | ||
| 503 | - const double totalMs = static_cast<double>(totalUs) / US_PER_MS; | ||
| 504 | - GetMasterTimeCost().Append("Total QueryAndGet", totalMs); | ||
| 505 | - if (status.IsError()) { | ||
| 506 | - LOG(ERROR) << FormatString("[Master] QueryAndGet failed, target num: %d, cost: %.3fms, status: %s", | ||
| 507 | - req.object_keys_size(), totalMs, status.ToString()); | ||
| 508 | - return status; | ||
| 509 | - } | ||
| 510 | - SLOW_LOG_IF_OR_VLOG( | ||
| 511 | - INFO, config.processSlowerThanUs > 0 && totalUs >= config.processSlowerThanUs, 1, | ||
| 512 | - FormatString("[Master] QueryAndGet done, target num: %d, result num: %d, payload num: %zu, meta moving: %d, " | ||
| 513 | - "cost: %.3fms, %s", | ||
| 514 | - req.object_keys_size(), resp.results_size(), payloads.size(), resp.meta_is_moving(), totalMs, | ||
| 515 | - GetMasterTimeCost().GetInfo())); | ||
| 516 | - return status; | ||
| 517 | -} | ||
| 518 | - | ||
| 519 | Status MasterOCServiceImpl::DeleteAllCopyMeta( | 489 | Status MasterOCServiceImpl::DeleteAllCopyMeta( |
| 520 | std::shared_ptr<ServerUnaryWriterReader<DeleteAllCopyMetaRspPb, DeleteAllCopyMetaReqPb>> serverApi) | 490 | std::shared_ptr<ServerUnaryWriterReader<DeleteAllCopyMetaRspPb, DeleteAllCopyMetaReqPb>> serverApi) |
| 521 | { | 491 | { |
| @@ -143,16 +143,6 @@ public: | |||
| 143 | */ | 143 | */ |
| 144 | Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &resp) override; | 144 | Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &resp) override; |
| 145 | 145 | ||
| 146 | - /** | ||
| 147 | - * @brief Query object locations for client direct reads. | ||
| 148 | - * @param[in] req The query request. | ||
| 149 | - * @param[out] resp The object locations and redirect state. | ||
| 150 | - * @param[out] payloads Optional object data returned with the metadata. | ||
| 151 | - * @return K_OK on success; the error code otherwise. | ||
| 152 | - */ | ||
| 153 | - Status QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &resp, | ||
| 154 | - std::vector<RpcMessage> &payloads) override; | ||
| 155 | - | ||
| 156 | /** | 146 | /** |
| 157 | * @brief Delete metadata and notify other workers to delete these objects. | 147 | * @brief Delete metadata and notify other workers to delete these objects. |
| 158 | * @param[in] serverApi The ServerUnaryWriterReader object. | 148 | * @param[in] serverApi The ServerUnaryWriterReader object. |
| @@ -99,7 +99,6 @@ static constexpr int DEBUG_LOG_LEVEL = 1; | |||
| 99 | static constexpr int MIN_TTL_SECOND = 0; | 99 | static constexpr int MIN_TTL_SECOND = 0; |
| 100 | static constexpr int ASYNC_MIN_THREAD_NUM = 2; | 100 | static constexpr int ASYNC_MIN_THREAD_NUM = 2; |
| 101 | static constexpr int ASYNC_MAX_THREAD_NUM = 5; | 101 | static constexpr int ASYNC_MAX_THREAD_NUM = 5; |
| 102 | -static constexpr int QUERY_AND_GET_MAX_COPY_NUM = 5; | ||
| 103 | static constexpr uint64_t QUERY_AND_GET_MAX_PAYLOAD_SIZE = 512 * 1024UL; | 102 | static constexpr uint64_t QUERY_AND_GET_MAX_PAYLOAD_SIZE = 512 * 1024UL; |
| 104 | static const std::string OC_METADATA_MANAGER = "OCMetadataManager-"; | 103 | static const std::string OC_METADATA_MANAGER = "OCMetadataManager-"; |
| 105 | static constexpr int MSET_PENDING_TTL_US = 60'000'000; // 60s | 104 | static constexpr int MSET_PENDING_TTL_US = 60'000'000; // 60s |
| @@ -2728,73 +2727,6 @@ Status OCMetadataManager::GetObjectLocations(const GetObjectLocationsReqPb &req, | |||
| 2728 | return Status::OK(); | 2727 | return Status::OK(); |
| 2729 | } | 2728 | } |
| 2730 | 2729 | ||
| 2731 | -Status OCMetadataManager::QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &rsp, | ||
| 2732 | - std::vector<RpcMessage> &payloads) | ||
| 2733 | -{ | ||
| 2734 | - INJECT_POINT("client.transport.query_and_get", []() { return Status::OK(); }); | ||
| 2735 | - RETURN_IF_NOT_OK(ValidateQueryAndGetDataRequest(req)); | ||
| 2736 | - std::vector<std::string> objectKeys = { req.object_keys().begin(), req.object_keys().end() }; | ||
| 2737 | - RETURN_IF_NOT_OK(FillObjectRedirectResponses(rsp, objectKeys, req.redirect())); | ||
| 2738 | - RETURN_OK_IF_TRUE(rsp.meta_is_moving()); | ||
| 2739 | - | ||
| 2740 | - std::unordered_map<std::string, bool> redirectedKeys; | ||
| 2741 | - for (const auto &redirectInfo : rsp.info()) { | ||
| 2742 | - for (const auto &objectKey : redirectInfo.change_meta_ids()) { | ||
| 2743 | - redirectedKeys.emplace(objectKey, true); | ||
| 2744 | - } | ||
| 2745 | - } | ||
| 2746 | - | ||
| 2747 | - uint64_t payloadSize = 0; | ||
| 2748 | - for (int i = 0; i < req.object_keys_size(); ++i) { | ||
| 2749 | - const auto &objectKey = req.object_keys(i); | ||
| 2750 | - if (redirectedKeys.find(objectKey) != redirectedKeys.end()) { | ||
| 2751 | - continue; | ||
| 2752 | - } | ||
| 2753 | - | ||
| 2754 | - QueryAndGetResultPb *result = rsp.add_results(); | ||
| 2755 | - ObjectLocationInfoPb *location = result->mutable_location(); | ||
| 2756 | - location->set_object_key(objectKey); | ||
| 2757 | - QueryAndGetMetaSnapshot meta; | ||
| 2758 | - if (!FillQueryAndGetMetadata(objectKey, *location, meta)) { | ||
| 2759 | - continue; | ||
| 2760 | - } | ||
| 2761 | - if (req.has_data_request()) { | ||
| 2762 | - TryGetQueryAndGetData(req, static_cast<size_t>(i), objectKey, meta, *result, payloadSize, payloads); | ||
| 2763 | - } | ||
| 2764 | - } | ||
| 2765 | - return Status::OK(); | ||
| 2766 | -} | ||
| 2767 | - | ||
| 2768 | -bool OCMetadataManager::FillQueryAndGetMetadata(const std::string &objectKey, ObjectLocationInfoPb &location, | ||
| 2769 | - QueryAndGetMetaSnapshot &meta) | ||
| 2770 | -{ | ||
| 2771 | - TbbMetaTable::const_accessor accessor; | ||
| 2772 | - const size_t shardIdx = GetShardIndex(objectKey); | ||
| 2773 | - if (!metaShards_[shardIdx].table.find(accessor, objectKey)) { | ||
| 2774 | - return false; | ||
| 2775 | - } | ||
| 2776 | - | ||
| 2777 | - const auto &objectMeta = accessor->second; | ||
| 2778 | - const auto &primaryAddress = objectMeta.meta.primary_address(); | ||
| 2779 | - if (!primaryAddress.empty()) { | ||
| 2780 | - location.add_object_locations(primaryAddress); | ||
| 2781 | - } | ||
| 2782 | - for (const auto &address : objectMeta.locations) { | ||
| 2783 | - if (location.object_locations_size() >= QUERY_AND_GET_MAX_COPY_NUM) { | ||
| 2784 | - break; | ||
| 2785 | - } | ||
| 2786 | - if (address.first != primaryAddress) { | ||
| 2787 | - location.add_object_locations(address.first); | ||
| 2788 | - } | ||
| 2789 | - } | ||
| 2790 | - meta.dataSize = objectMeta.meta.data_size(); | ||
| 2791 | - meta.version = objectMeta.meta.version(); | ||
| 2792 | - auto localCopy = objectMeta.locations.find(masterAddress_); | ||
| 2793 | - meta.localCopyAvailable = localCopy != objectMeta.locations.end() && localCopy->second == AckState::ACK; | ||
| 2794 | - location.set_object_size(meta.dataSize); | ||
| 2795 | - return true; | ||
| 2796 | -} | ||
| 2797 | - | ||
| 2798 | void OCMetadataManager::GetObjRefsMatch(const std::function<bool(const std::string &)> &matchFunc, | 2730 | void OCMetadataManager::GetObjRefsMatch(const std::function<bool(const std::string &)> &matchFunc, |
| 2799 | std::unordered_set<std::string> &objKeys) | 2731 | std::unordered_set<std::string> &objKeys) |
| 2800 | { | 2732 | { |
| @@ -4724,89 +4656,6 @@ void OCMetadataManager::WaitInitializaiton() | |||
| 4724 | } | 4656 | } |
| 4725 | } | 4657 | } |
| 4726 | 4658 | ||
| 4727 | -Status OCMetadataManager::ValidateQueryAndGetDataRequest(const QueryAndGetReqPb &req) const | ||
| 4728 | -{ | ||
| 4729 | - if (!req.has_data_request()) { | ||
| 4730 | - return Status::OK(); | ||
| 4731 | - } | ||
| 4732 | - const auto &dataRequest = req.data_request(); | ||
| 4733 | - CHECK_FAIL_RETURN_STATUS(dataRequest.has_tcp() || dataRequest.has_ub(), K_INVALID, | ||
| 4734 | - "QueryAndGet data transport is not set"); | ||
| 4735 | - if (dataRequest.has_ub()) { | ||
| 4736 | - const auto &ubRequest = dataRequest.ub(); | ||
| 4737 | - CHECK_FAIL_RETURN_STATUS(ubRequest.buffer_size() > 0 && !ubRequest.urma_instance_id().empty(), K_INVALID, | ||
| 4738 | - "QueryAndGet UB buffer is invalid"); | ||
| 4739 | - CHECK_FAIL_RETURN_STATUS(ubRequest.buffer_infos_size() == req.object_keys_size(), K_INVALID, | ||
| 4740 | - "QueryAndGet UB buffer count does not match object key count"); | ||
| 4741 | - } | ||
| 4742 | - return Status::OK(); | ||
| 4743 | -} | ||
| 4744 | - | ||
| 4745 | -void OCMetadataManager::TryGetQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex, | ||
| 4746 | - const std::string &objectKey, | ||
| 4747 | - const QueryAndGetMetaSnapshot &meta, | ||
| 4748 | - QueryAndGetResultPb &result, uint64_t &payloadSize, | ||
| 4749 | - std::vector<RpcMessage> &payloads) | ||
| 4750 | -{ | ||
| 4751 | - if (localApi_ == nullptr || !meta.localCopyAvailable | ||
| 4752 | - || notifyWorkerManager_->CheckExistAsyncWorkerOp( | ||
| 4753 | - masterAddress_, objectKey, | ||
| 4754 | - NotifyWorkerOpType::CACHE_INVALID | NotifyWorkerOpType::PRIMARY_COPY_INVALID)) { | ||
| 4755 | - return; | ||
| 4756 | - } | ||
| 4757 | - | ||
| 4758 | - const bool useUb = req.data_request().has_ub(); | ||
| 4759 | - if (useUb && meta.dataSize > req.data_request().ub().buffer_size()) { | ||
| 4760 | - return; | ||
| 4761 | - } | ||
| 4762 | - if (!useUb | ||
| 4763 | - && (payloadSize > QUERY_AND_GET_MAX_PAYLOAD_SIZE | ||
| 4764 | - || meta.dataSize > QUERY_AND_GET_MAX_PAYLOAD_SIZE - payloadSize)) { | ||
| 4765 | - return; | ||
| 4766 | - } | ||
| 4767 | - | ||
| 4768 | - std::vector<RpcMessage> tmpPayloads; | ||
| 4769 | - Status status = ReadLocalQueryAndGetData(req, requestIndex, objectKey, meta, tmpPayloads); | ||
| 4770 | - if (status.IsError()) { | ||
| 4771 | - VLOG(1) << FormatString("[ObjectKey %s] QueryAndGet local data miss: %s", objectKey, status.ToString()); | ||
| 4772 | - return; | ||
| 4773 | - } | ||
| 4774 | - | ||
| 4775 | - if (useUb) { | ||
| 4776 | - result.mutable_data_result(); | ||
| 4777 | - return; | ||
| 4778 | - } | ||
| 4779 | - auto *dataResult = result.mutable_data_result(); | ||
| 4780 | - for (auto &payload : tmpPayloads) { | ||
| 4781 | - dataResult->add_payload_indexes(static_cast<uint32_t>(payloads.size())); | ||
| 4782 | - payloads.emplace_back(std::move(payload)); | ||
| 4783 | - } | ||
| 4784 | - payloadSize += meta.dataSize; | ||
| 4785 | -} | ||
| 4786 | - | ||
| 4787 | -Status OCMetadataManager::ReadLocalQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex, | ||
| 4788 | - const std::string &objectKey, | ||
| 4789 | - const QueryAndGetMetaSnapshot &meta, | ||
| 4790 | - std::vector<RpcMessage> &payloads) | ||
| 4791 | -{ | ||
| 4792 | - GetObjectRemoteReqPb localReq; | ||
| 4793 | - localReq.set_try_lock(true); | ||
| 4794 | - localReq.set_object_key(objectKey); | ||
| 4795 | - localReq.set_version(meta.version); | ||
| 4796 | - localReq.set_read_offset(0); | ||
| 4797 | - localReq.set_read_size(meta.dataSize); | ||
| 4798 | - localReq.set_data_size(meta.dataSize); | ||
| 4799 | - if (req.data_request().has_ub()) { | ||
| 4800 | - const auto &ubRequest = req.data_request().ub(); | ||
| 4801 | - *localReq.mutable_urma_info() = ubRequest.buffer_infos(static_cast<int>(requestIndex)); | ||
| 4802 | - localReq.set_urma_instance_id(ubRequest.urma_instance_id()); | ||
| 4803 | - } | ||
| 4804 | - | ||
| 4805 | - RETURN_RUNTIME_ERROR_IF_NULL(localApi_); | ||
| 4806 | - RETURN_IF_NOT_OK(akSkManager_->GenerateSignature(localReq)); | ||
| 4807 | - return localApi_->GetObjectRemoteForQueryAndGet(localReq, payloads); | ||
| 4808 | -} | ||
| 4809 | - | ||
| 4810 | void OCMetadataManager::TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor, | 4659 | void OCMetadataManager::TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor, |
| 4811 | uint64_t &payloadSize, QueryMetaInfoPb &queryMeta, | 4660 | uint64_t &payloadSize, QueryMetaInfoPb &queryMeta, |
| 4812 | std::vector<RpcMessage> &payloads) | 4661 | std::vector<RpcMessage> &payloads) |
| @@ -556,16 +556,6 @@ public: | |||
| 556 | */ | 556 | */ |
| 557 | Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &rsp); | 557 | Status GetObjectLocations(const GetObjectLocationsReqPb &req, GetObjectLocationsRspPb &rsp); |
| 558 | 558 | ||
| 559 | - /** | ||
| 560 | - * @brief Query object locations for client direct reads. | ||
| 561 | - * @param[in] req The query request. | ||
| 562 | - * @param[out] rsp The object locations and redirect state. | ||
| 563 | - * @param[out] payloads Optional object data returned with the metadata. | ||
| 564 | - * @return K_OK on success; the error code otherwise. | ||
| 565 | - */ | ||
| 566 | - Status QueryAndGet(const QueryAndGetReqPb &req, QueryAndGetRspPb &rsp, | ||
| 567 | - std::vector<RpcMessage> &payloads); | ||
| 568 | - | ||
| 569 | /** | 559 | /** |
| 570 | * @brief Delete metadata and notify other workers to delete these objects synchronously. | 560 | * @brief Delete metadata and notify other workers to delete these objects synchronously. |
| 571 | * @param[in] req The rpc request protobuf. | 561 | * @param[in] req The rpc request protobuf. |
| @@ -1538,12 +1528,6 @@ private: | |||
| 1538 | void ReplacePrimaryObject(const ReplacePrimaryReqPb &req, | 1528 | void ReplacePrimaryObject(const ReplacePrimaryReqPb &req, |
| 1539 | const ReplacePrimaryReqPb::ObjectInfoPb &info, ReplacePrimaryRspPb &rsp); | 1529 | const ReplacePrimaryReqPb::ObjectInfoPb &info, ReplacePrimaryRspPb &rsp); |
| 1540 | 1530 | ||
| 1541 | - struct QueryAndGetMetaSnapshot { | ||
| 1542 | - uint64_t dataSize = 0; | ||
| 1543 | - int64_t version = 0; | ||
| 1544 | - bool localCopyAvailable = false; | ||
| 1545 | - }; | ||
| 1546 | - | ||
| 1547 | friend class MasterOCServiceImpl; | 1531 | friend class MasterOCServiceImpl; |
| 1548 | friend class OCNotifyWorkerManager; | 1532 | friend class OCNotifyWorkerManager; |
| 1549 | friend class OCGlobalCacheDeleteManager; | 1533 | friend class OCGlobalCacheDeleteManager; |
| @@ -2076,50 +2060,6 @@ private: | |||
| 2076 | void TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor, uint64_t &payloadSize, | 2060 | void TryGetObjectData(const std::string &objectKey, const TbbMetaTable::accessor &accessor, uint64_t &payloadSize, |
| 2077 | QueryMetaInfoPb &queryMeta, std::vector<RpcMessage> &payloads); | 2061 | QueryMetaInfoPb &queryMeta, std::vector<RpcMessage> &payloads); |
| 2078 | 2062 | ||
| 2079 | - /** | ||
| 2080 | - * @brief Validate the optional QueryAndGet inline-data request. | ||
| 2081 | - * @param[in] req QueryAndGet request. | ||
| 2082 | - * @return K_OK when the transport request is valid; the error code otherwise. | ||
| 2083 | - */ | ||
| 2084 | - Status ValidateQueryAndGetDataRequest(const QueryAndGetReqPb &req) const; | ||
| 2085 | - | ||
| 2086 | - /** | ||
| 2087 | - * @brief Read one local object through the transport selected by QueryAndGet. | ||
| 2088 | - * @param[in] req QueryAndGet request. | ||
| 2089 | - * @param[in] requestIndex Object position in the original request. | ||
| 2090 | - * @param[in] objectKey Object key. | ||
| 2091 | - * @param[in] meta Metadata snapshot used to construct the local read. | ||
| 2092 | - * @param[out] payloads Optional TCP payloads. | ||
| 2093 | - * @return K_OK on success; the error code otherwise. | ||
| 2094 | - */ | ||
| 2095 | - Status ReadLocalQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex, | ||
| 2096 | - const std::string &objectKey, const QueryAndGetMetaSnapshot &meta, | ||
| 2097 | - std::vector<RpcMessage> &payloads); | ||
| 2098 | - | ||
| 2099 | - /** | ||
| 2100 | - * @brief Fill object locations and snapshot fields while holding the metadata accessor. | ||
| 2101 | - * @param[in] objectKey Object key. | ||
| 2102 | - * @param[out] location Object locations and size. | ||
| 2103 | - * @param[out] meta Metadata required after releasing the accessor. | ||
| 2104 | - * @return True when the object metadata exists. | ||
| 2105 | - */ | ||
| 2106 | - bool FillQueryAndGetMetadata(const std::string &objectKey, ObjectLocationInfoPb &location, | ||
| 2107 | - QueryAndGetMetaSnapshot &meta); | ||
| 2108 | - | ||
| 2109 | - /** | ||
| 2110 | - * @brief Try to return one QueryAndGet object's data through the selected transport. | ||
| 2111 | - * @param[in] req QueryAndGet request. | ||
| 2112 | - * @param[in] requestIndex Object position in the original request. | ||
| 2113 | - * @param[in] objectKey Object key. | ||
| 2114 | - * @param[in] meta Metadata snapshot captured before releasing the table accessor. | ||
| 2115 | - * @param[out] result Per-object QueryAndGet result. | ||
| 2116 | - * @param[in,out] payloadSize Accumulated TCP payload bytes. | ||
| 2117 | - * @param[out] payloads Response payloads. | ||
| 2118 | - */ | ||
| 2119 | - void TryGetQueryAndGetData(const QueryAndGetReqPb &req, size_t requestIndex, const std::string &objectKey, | ||
| 2120 | - const QueryAndGetMetaSnapshot &meta, QueryAndGetResultPb &result, | ||
| 2121 | - uint64_t &payloadSize, std::vector<RpcMessage> &payloads); | ||
| 2122 | - | ||
| 2123 | /** | 2063 | /** |
| 2124 | * @brief Add heavy operation. | 2064 | * @brief Add heavy operation. |
| 2125 | * @param[in] objectKey Object key. | 2065 | * @param[in] objectKey Object key. |
| @@ -613,47 +613,6 @@ message GetObjectLocationsRspPb { | |||
| 613 | bool meta_is_moving = 3; | 613 | bool meta_is_moving = 3; |
| 614 | } | 614 | } |
| 615 | 615 | ||
| 616 | -message QueryAndGetTcpDataReqPb {} | ||
| 617 | - | ||
| 618 | -message QueryAndGetUbDataReqPb { | ||
| 619 | - repeated datasystem.UrmaRemoteAddrPb buffer_infos = 1; | ||
| 620 | - uint64 buffer_size = 2; | ||
| 621 | - string urma_instance_id = 3; | ||
| 622 | -} | ||
| 623 | - | ||
| 624 | -message QueryAndGetDataReqPb { | ||
| 625 | - oneof transport { | ||
| 626 | - QueryAndGetTcpDataReqPb tcp = 1; | ||
| 627 | - QueryAndGetUbDataReqPb ub = 2; | ||
| 628 | - } | ||
| 629 | -} | ||
| 630 | - | ||
| 631 | -message QueryAndGetReqPb { | ||
| 632 | - repeated string object_keys = 1; | ||
| 633 | - bool redirect = 2; | ||
| 634 | - QueryAndGetDataReqPb data_request = 3; | ||
| 635 | - | ||
| 636 | - // put to the end, the previous data is used to generate AK and SK signatures. | ||
| 637 | - uint64 timestamp = 100; | ||
| 638 | - string signature = 101; | ||
| 639 | - string access_key = 102; | ||
| 640 | -} | ||
| 641 | - | ||
| 642 | -message QueryAndGetDataResultPb { | ||
| 643 | - repeated uint32 payload_indexes = 1; | ||
| 644 | -} | ||
| 645 | - | ||
| 646 | -message QueryAndGetResultPb { | ||
| 647 | - ObjectLocationInfoPb location = 1; | ||
| 648 | - QueryAndGetDataResultPb data_result = 2; | ||
| 649 | -} | ||
| 650 | - | ||
| 651 | -message QueryAndGetRspPb { | ||
| 652 | - repeated QueryAndGetResultPb results = 1; | ||
| 653 | - repeated RedirectMetaInfo info = 2; | ||
| 654 | - bool meta_is_moving = 3; | ||
| 655 | -} | ||
| 656 | - | ||
| 657 | message ReplacePrimaryReqPb { | 616 | message ReplacePrimaryReqPb { |
| 658 | message ObjectInfoPb { | 617 | message ObjectInfoPb { |
| 659 | string object_key = 1; | 618 | string object_key = 1; |
| @@ -839,10 +798,6 @@ service MasterOCService { | |||
| 839 | } | 798 | } |
| 840 | rpc UpdateMeta(UpdateMetaReqPb) returns (UpdateMetaRspPb) {} | 799 | rpc UpdateMeta(UpdateMetaReqPb) returns (UpdateMetaRspPb) {} |
| 841 | rpc GetObjectLocations(GetObjectLocationsReqPb) returns (GetObjectLocationsRspPb) {} | 800 | rpc GetObjectLocations(GetObjectLocationsReqPb) returns (GetObjectLocationsRspPb) {} |
| 842 | - rpc QueryAndGet(QueryAndGetReqPb) returns (QueryAndGetRspPb) { | ||
| 843 | - option (datasystem.recv_payload_option) = true; | ||
| 844 | - option (datasystem.urma_enabled_option) = true; | ||
| 845 | - } | ||
| 846 | rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {} | 801 | rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {} |
O [P1] 删除 MasterOCService.QueryAndGet 后无混版本兼容防护,滚动升级窗口内直读路径整体不可用 本 PR 删除了 MasterOCService.QueryAndGet 并把实现下沉到 WorkerOCService.QueryAndGet,但仓库内没有任何版本协商/能力检测机制,客户端也没有对"方法不存在"的快速失败或回退路径:
影响面:enableLocalCache=false 的直读路径在 client 与 worker/master 任一方向版本不匹配期间全部失败。PR body 声称"同版本升级约束已说明",但目前仅文档层面,代码无运行时保护。 建议至少二选一:1) 升级流程强制 client+server 严格同版本原子升级(发布说明明确);2) 对"方法不存在"类错误快速失败并给出明确提示,或对该场景回退 QueryMeta + ReplicaReader 旧链路保持兼容。 ![]() ![]() | |||
| 847 | rpc GDecreaseRef(GDecreaseReqPb) returns (GDecreaseRspPb) { | 802 | rpc GDecreaseRef(GDecreaseReqPb) returns (GDecreaseRspPb) { |
| 848 | option (datasystem.unary_socket_option) = true; | 803 | option (datasystem.unary_socket_option) = true; |
| @@ -644,6 +644,69 @@ message GetHashRingRspPb { | |||
| 644 | map<string, string> host_id_map = 5; // workerAddr("host:port") -> hostId, valid when hash_ring_changed=true. | 644 | map<string, string> host_id_map = 5; // workerAddr("host:port") -> hostId, valid when hash_ring_changed=true. |
| 645 | } | 645 | } |
| 646 | 646 | ||
| 647 | +message QueryAndGetTcpDataReqPb {} | ||
| 648 | + | ||
| 649 | +message QueryAndGetUbDataReqPb { | ||
| 650 | + repeated UrmaRemoteAddrPb buffer_infos = 1; | ||
| 651 | + uint64 buffer_size = 2; | ||
| 652 | + string urma_instance_id = 3; | ||
| 653 | +} | ||
| 654 | + | ||
| 655 | +message QueryAndGetShmDataReqPb { | ||
| 656 | + string client_id = 1; | ||
| 657 | +} | ||
| 658 | + | ||
| 659 | +message QueryAndGetDataReqPb { | ||
| 660 | + oneof transport { | ||
| 661 | + QueryAndGetTcpDataReqPb tcp = 1; | ||
| 662 | + QueryAndGetUbDataReqPb ub = 2; | ||
| 663 | + QueryAndGetShmDataReqPb shm = 3; | ||
| 664 | + } | ||
| 665 | +} | ||
| 666 | + | ||
| 667 | +message QueryAndGetReqPb { | ||
| 668 | + repeated string object_keys = 1; | ||
| 669 | + QueryAndGetDataReqPb data_request = 2; | ||
| 670 | + | ||
| 671 | + // put to the end, the previous data is used to generate AK and SK signatures. | ||
| 672 | + uint64 timestamp = 100; | ||
| 673 | + string signature = 101; | ||
| 674 | + string access_key = 102; | ||
| 675 | +} | ||
| 676 | + | ||
| 677 | +message QueryAndGetLocationInfoPb { | ||
| 678 | + string object_key = 1; | ||
| 679 | + repeated string object_locations = 2; | ||
| 680 | + uint64 object_size = 3; | ||
| 681 | +} | ||
| 682 | + | ||
| 683 | +message QueryAndGetShmInfoPb { | ||
| 684 | + int64 store_fd = 1; | ||
| 685 | + int64 offset = 2; | ||
| 686 | + int64 data_size = 3; | ||
| 687 | + int64 metadata_size = 4; | ||
| 688 | + int64 mmap_size = 5; | ||
| 689 | + string shm_id = 6; | ||
| 690 | + bool is_seal = 7; | ||
| 691 | + uint32 write_mode = 8; | ||
| 692 | + uint32 consistency_type = 9; | ||
| 693 | + uint32 cache_type = 10; | ||
| 694 | +} | ||
| 695 | + | ||
| 696 | +message QueryAndGetDataResultPb { | ||
| 697 | + repeated uint32 payload_indexes = 1; | ||
| 698 | + QueryAndGetShmInfoPb shm_info = 2; | ||
| 699 | +} | ||
| 700 | + | ||
| 701 | +message QueryAndGetResultPb { | ||
| 702 | + QueryAndGetLocationInfoPb location = 1; | ||
| 703 | + QueryAndGetDataResultPb data_result = 2; | ||
| 704 | +} | ||
| 705 | + | ||
| 706 | +message QueryAndGetRspPb { | ||
| 707 | + repeated QueryAndGetResultPb results = 1; | ||
| 708 | +} | ||
| 709 | + | ||
| 647 | service WorkerOCService { | 710 | service WorkerOCService { |
| 648 | rpc HealthCheck (HealthCheckRequestPb) returns (HealthCheckReplyPb) {} | 711 | rpc HealthCheck (HealthCheckRequestPb) returns (HealthCheckReplyPb) {} |
| 649 | rpc Create (CreateReqPb) returns (CreateRspPb) {} | 712 | rpc Create (CreateReqPb) returns (CreateRspPb) {} |
| @@ -654,6 +717,11 @@ service WorkerOCService { | |||
| 654 | option (datasystem.unary_socket_option) = true; | 717 | option (datasystem.unary_socket_option) = true; |
| 655 | option (datasystem.recv_payload_option) = true; | 718 | option (datasystem.recv_payload_option) = true; |
| 656 | } | 719 | } |
| 720 | + rpc QueryAndGet(QueryAndGetReqPb) returns (QueryAndGetRspPb) { | ||
[严重][兼容性] [P1] 将新 RPC 追加到 service 末尾以保持 ZMQ 方法序号 证据:DataSystem 的 ZMQ 生成器直接用 protobuf service 中的方法下标生成 问题:bRPC 按方法名分发不会暴露这个问题,但 ZMQ wire protocol 使用 ordinal。新旧进程混部时,旧端发送的 index 4 会被新端当成 影响:滚动升级期间,引用释放、GRef、序列号、设备对象、P2P、MultiCreate、Exist 等大量无关 Worker RPC 都可能被分发到错误 handler,造成请求解析失败、引用无法释放或业务错误,故障面远大于 QueryAndGet 本身。 建议:把 验证:生成 ZMQ stub/service 后检查 ![]() ![]() | |||
| 721 | + option (datasystem.unary_socket_option) = true; | ||
| 722 | + option (datasystem.recv_payload_option) = true; | ||
| 723 | + option (datasystem.urma_enabled_option) = true; | ||
| 724 | + } | ||
| 657 | rpc DecreaseReference(DecreaseReferenceRequest) returns (DecreaseReferenceResponse) {} | 725 | rpc DecreaseReference(DecreaseReferenceRequest) returns (DecreaseReferenceResponse) {} |
| 658 | rpc ReconcileShmRef(ReconcileShmRefReqPb) returns (ReconcileShmRefRspPb) {} | 726 | rpc ReconcileShmRef(ReconcileShmRefReqPb) returns (ReconcileShmRefRspPb) {} |
| 659 | rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {} | 727 | rpc GIncreaseRef(GIncreaseReqPb) returns (GIncreaseRspPb) {} |
| @@ -339,6 +339,7 @@ ds_cc_library( | |||
| 339 | "//src/datasystem/worker/object_cache/service:worker_oc_service_delete_impl_header", | 339 | "//src/datasystem/worker/object_cache/service:worker_oc_service_delete_impl_header", |
| 340 | "//src/datasystem/worker/object_cache/service:worker_oc_service_expire_impl", | 340 | "//src/datasystem/worker/object_cache/service:worker_oc_service_expire_impl", |
| 341 | "//src/datasystem/worker/object_cache/service:worker_oc_service_global_reference_impl", | 341 | "//src/datasystem/worker/object_cache/service:worker_oc_service_global_reference_impl", |
| 342 | + "//src/datasystem/worker/object_cache/service:worker_oc_service_get_impl", | ||
| 342 | "//src/datasystem/worker/object_cache/service:worker_oc_service_migrate_impl", | 343 | "//src/datasystem/worker/object_cache/service:worker_oc_service_migrate_impl", |
| 343 | "//src/datasystem/worker/object_cache/service:worker_oc_service_multi_publish_impl_header", | 344 | "//src/datasystem/worker/object_cache/service:worker_oc_service_multi_publish_impl_header", |
| 344 | "//src/datasystem/worker/object_cache/service:worker_oc_service_publish_impl_header", | 345 | "//src/datasystem/worker/object_cache/service:worker_oc_service_publish_impl_header", |
| @@ -45,6 +45,7 @@ set(WORKER_OC_SRCS | |||
| 45 | service/worker_oc_service_publish_impl.cpp | 45 | service/worker_oc_service_publish_impl.cpp |
| 46 | service/worker_oc_service_multi_publish_impl.cpp | 46 | service/worker_oc_service_multi_publish_impl.cpp |
| 47 | service/worker_oc_service_get_impl.cpp | 47 | service/worker_oc_service_get_impl.cpp |
| 48 | + service/worker_query_and_get_impl.cpp | ||
| 48 | service/worker_oc_service_delete_impl.cpp | 49 | service/worker_oc_service_delete_impl.cpp |
| 49 | service/worker_oc_service_clear_data_flow.cpp | 50 | service/worker_oc_service_clear_data_flow.cpp |
| 50 | service/worker_oc_service_global_reference_impl.cpp | 51 | service/worker_oc_service_global_reference_impl.cpp |
| @@ -116,6 +117,7 @@ target_link_libraries(worker_object_cache PRIVATE ${WORKER_OBJECT_CACHE_DEPEND_L | |||
| 116 | set(WORKER_OC_HEAVY_FILES | 117 | set(WORKER_OC_HEAVY_FILES |
| 117 | worker_oc_service_impl.cpp | 118 | worker_oc_service_impl.cpp |
| 118 | service/worker_oc_service_get_impl.cpp | 119 | service/worker_oc_service_get_impl.cpp |
| 120 | + service/worker_query_and_get_impl.cpp | ||
| 119 | service/worker_oc_service_migrate_impl.cpp | 121 | service/worker_oc_service_migrate_impl.cpp |
| 120 | worker_oc_eviction_manager.cpp | 122 | worker_oc_eviction_manager.cpp |
| 121 | ) | 123 | ) |
| @@ -88,9 +88,11 @@ ds_cc_library( | |||
| 88 | srcs = [ | 88 | srcs = [ |
| 89 | "worker_oc_service_batch_get_impl.cpp", | 89 | "worker_oc_service_batch_get_impl.cpp", |
| 90 | "worker_oc_service_get_impl.cpp", | 90 | "worker_oc_service_get_impl.cpp", |
| 91 | + "worker_query_and_get_impl.cpp", | ||
| 91 | ], | 92 | ], |
| 92 | hdrs = [ | 93 | hdrs = [ |
| 93 | "worker_oc_service_get_impl.h", | 94 | "worker_oc_service_get_impl.h", |
| 95 | + "worker_query_and_get_impl.h", | ||
| 94 | ], | 96 | ], |
| 95 | deps = [ | 97 | deps = [ |
| 96 | ":worker_oc_service_get_impl_header", | 98 | ":worker_oc_service_get_impl_header", |
| @@ -104,6 +106,8 @@ ds_cc_library( | |||
| 104 | "//src/datasystem/common/l2cache:l2_storage", | 106 | "//src/datasystem/common/l2cache:l2_storage", |
| 105 | "//src/datasystem/common/log:common_log", | 107 | "//src/datasystem/common/log:common_log", |
| 106 | "//src/datasystem/common/metrics:common_metrics", | 108 | "//src/datasystem/common/metrics:common_metrics", |
| 109 | + "//src/datasystem/common/object_cache:object_ref_info", | ||
| 110 | + "//src/datasystem/common/object_cache:shm_guard", | ||
| 107 | "//src/datasystem/common/parallel:common_parallel", | 111 | "//src/datasystem/common/parallel:common_parallel", |
| 108 | "//src/datasystem/common/perf:common_perf", | 112 | "//src/datasystem/common/perf:common_perf", |
| 109 | "//src/datasystem/common/rdma:fast_transport_manager_wrapper", | 113 | "//src/datasystem/common/rdma:fast_transport_manager_wrapper", |
| @@ -874,6 +874,22 @@ Status WorkerOcServiceGetImpl::RLockGetObjectFromMem(const ReadKey &readKey, Get | |||
| 874 | return Status::OK(); | 874 | return Status::OK(); |
| 875 | } | 875 | } |
| 876 | 876 | ||
| 877 | +Status WorkerOcServiceGetImpl::TryAcquireLocalObject(const std::string &objectKey, | ||
| 878 | + std::unique_ptr<GetObjEntryParams> ¶ms) | ||
| 879 | +{ | ||
| 880 | + params.reset(); | ||
| 881 | + GetObjInfo info; | ||
| 882 | + std::set<ReadKey> misses; | ||
| 883 | + bool validInMemory = true; | ||
| 884 | + Status rc = RLockGetObjectFromMem(ReadKey(objectKey), info, misses, validInMemory); | ||
| 885 | + if (rc.GetCode() == K_NOT_FOUND || !validInMemory || !misses.empty()) { | ||
| 886 | + return Status::OK(); | ||
| 887 | + } | ||
| 888 | + RETURN_IF_NOT_OK(rc); | ||
| 889 | + params = std::move(info.params); | ||
| 890 | + return Status::OK(); | ||
| 891 | +} | ||
| 892 | + | ||
| 877 | Status WorkerOcServiceGetImpl::ProcessObjectsNotExistInLocal(const std::set<ReadKey> &objectsNeedGetRemote, | 893 | Status WorkerOcServiceGetImpl::ProcessObjectsNotExistInLocal(const std::set<ReadKey> &objectsNeedGetRemote, |
| 878 | int64_t subTimeout, | 894 | int64_t subTimeout, |
| 879 | std::unordered_set<std::string> &failedIds, | 895 | std::unordered_set<std::string> &failedIds, |
| @@ -2996,6 +3012,96 @@ Status WorkerOcServiceGetImpl::GetMapOfObjectKeys(const std::vector<std::basic_s | |||
| 2996 | return Status::OK(); | 3012 | return Status::OK(); |
| 2997 | } | 3013 | } |
| 2998 | 3014 | ||
| 3015 | +Status WorkerOcServiceGetImpl::QueryObjectLocations( | ||
| 3016 | + const std::vector<std::string> &objectKeys, | ||
| 3017 | + std::unordered_map<std::string, master::ObjectLocationInfoPb> &locations) | ||
| 3018 | +{ | ||
| 3019 | + locations.clear(); | ||
| 3020 | + if (objectKeys.empty()) { | ||
| 3021 | + return Status::OK(); | ||
| 3022 | + } | ||
| 3023 | + CHECK_FAIL_RETURN_STATUS(metadataRouteResolver_ != nullptr, K_NOT_READY, "Metadata route resolver is unavailable"); | ||
| 3024 | + auto grouped = metadataRouteResolver_->GroupOwners(objectKeys); | ||
| 3025 | + if (!grouped.failures.empty()) { | ||
| 3026 | + return grouped.failures.begin()->second; | ||
| 3027 | + } | ||
| 3028 | + std::vector<master::QueryMetaInfoPb> queryMetas; | ||
| 3029 | + queryMetas.reserve(objectKeys.size()); | ||
| 3030 | + for (const auto &[masterAddress, keys] : grouped.groups) { | ||
| 3031 | + RETURN_IF_NOT_OK(QueryPureMetadataGroup(masterAddress, keys, queryMetas)); | ||
| 3032 | + } | ||
| 3033 | + locations.reserve(objectKeys.size()); | ||
| 3034 | + const std::unordered_set<std::string> requestedKeys(objectKeys.begin(), objectKeys.end()); | ||
| 3035 | + for (const auto &queryMeta : queryMetas) { | ||
| 3036 | + const auto &meta = queryMeta.meta(); | ||
| 3037 | + CHECK_FAIL_RETURN_STATUS(requestedKeys.find(meta.object_key()) != requestedKeys.end(), K_RUNTIME_ERROR, | ||
| 3038 | + "PureQueryMeta returned an unexpected object key"); | ||
| 3039 | + master::ObjectLocationInfoPb location; | ||
| 3040 | + location.set_object_key(meta.object_key()); | ||
| 3041 | + location.set_object_size(meta.data_size()); | ||
| 3042 | + if (!meta.primary_address().empty()) { | ||
| 3043 | + location.add_object_locations(meta.primary_address()); | ||
| 3044 | + } | ||
| 3045 | + if (!queryMeta.address().empty() && queryMeta.address() != meta.primary_address()) { | ||
[警告][正确性] [P1] miss 响应丢失第三个及后续健康副本 证据:新实现只加入 primary_address 和 PureQueryMeta 通过 SelectObjectLocation 选出的一个 address。基线 FillQueryAndGetMetadata 会按 primary-first 返回最多 QUERY_AND_GET_MAX_COPY_NUM=5 个 location,而 ReplicaReader 会顺序遍历响应中的全部候选。 问题:三副本以上时,其余 ACK 副本不会进入响应,客户端因此无法发现并尝试它们。 影响:primary 与本次选中的 secondary 都不可读时,即使第三副本健康,请求也会失败、重试或超时,造成副本容灾能力和尾延迟相对基线退化。 建议:让 Worker 的 metadata 查询返回与旧协议等价的、primary-first 且最多 5 个的有效 ACK location 集合,并保留异步失效过滤,不要从单选 QueryMetaInfoPb.address 反推完整副本集合。 验证:构造至少三副本对象,让 primary 与首选 secondary 分别返回 peer-dead/not-found,验证单 key 与 batch 均能从第三个健康副本成功读取。 ![]() ![]() | |||
| 3046 | + location.add_object_locations(queryMeta.address()); | ||
| 3047 | + } | ||
| 3048 | + locations[meta.object_key()] = std::move(location); | ||
| 3049 | + } | ||
| 3050 | + return Status::OK(); | ||
| 3051 | +} | ||
| 3052 | + | ||
| 3053 | +Status WorkerOcServiceGetImpl::QueryPureMetadataGroup(const HostPort &masterAddress, | ||
| 3054 | + const std::vector<std::string> &objectKeys, | ||
| 3055 | + std::vector<master::QueryMetaInfoPb> &queryMetas) | ||
| 3056 | +{ | ||
| 3057 | + auto workerMasterApi = workerMasterApiManager_->GetWorkerMasterApi(masterAddress); | ||
| 3058 | + CHECK_FAIL_RETURN_STATUS(workerMasterApi != nullptr, K_RUNTIME_ERROR, | ||
| 3059 | + "Get master API failed for PureQueryMeta"); | ||
| 3060 | + master::PureQueryMetaReqPb request; | ||
| 3061 | + request.set_redirect(true); | ||
| 3062 | + request.set_address(localAddress_.ToString()); | ||
| 3063 | + request.mutable_object_keys()->Add(objectKeys.begin(), objectKeys.end()); | ||
| 3064 | + master::PureQueryMetaRspPb response; | ||
| 3065 | + std::function<Status(master::PureQueryMetaReqPb &, master::PureQueryMetaRspPb &)> query = | ||
| 3066 | + [this, &workerMasterApi](master::PureQueryMetaReqPb &req, master::PureQueryMetaRspPb &rsp) { | ||
| 3067 | + auto rc = workerMasterApi->PureQueryMeta(req, rsp); | ||
| 3068 | + ObserveMetadataRpc(workerMasterApi, rc); | ||
| 3069 | + return rc; | ||
| 3070 | + }; | ||
| 3071 | + RETURN_IF_NOT_OK(RedirectRetryWhenMetasMoving(request, response, query)); | ||
| 3072 | + queryMetas.insert(queryMetas.end(), response.mutable_query_metas()->begin(), | ||
| 3073 | + response.mutable_query_metas()->end()); | ||
| 3074 | + return QueryPureMetadataRedirects(response.info(), queryMetas); | ||
| 3075 | +} | ||
| 3076 | + | ||
| 3077 | +Status WorkerOcServiceGetImpl::QueryPureMetadataRedirects( | ||
| 3078 | + const google::protobuf::RepeatedPtrField<RedirectMetaInfo> &redirects, | ||
| 3079 | + std::vector<master::QueryMetaInfoPb> &queryMetas) | ||
| 3080 | +{ | ||
| 3081 | + for (const auto &redirect : redirects) { | ||
| 3082 | + HostPort masterAddress; | ||
| 3083 | + RETURN_IF_NOT_OK(masterAddress.ParseString(redirect.redirect_meta_address())); | ||
| 3084 | + auto workerMasterApi = workerMasterApiManager_->GetWorkerMasterApi(masterAddress); | ||
| 3085 | + CHECK_FAIL_RETURN_STATUS(workerMasterApi != nullptr, K_RUNTIME_ERROR, | ||
| 3086 | + "Get redirect master API failed for PureQueryMeta"); | ||
| 3087 | + master::PureQueryMetaReqPb request; | ||
| 3088 | + request.set_redirect(false); | ||
[警告][正确性] [P1] 重定向只跟随一跳会在连续迁移时漏查元数据 证据:初次 PureQueryMeta 使用 redirect=true,但这里向第一跳目标发请求时固定设置 redirect=false,且 QueryPureMetadataRedirects 不再处理第二个 response.info。基线客户端使用待处理队列迭代跟随重定向,并有 FollowsTwoRedirects 单测;本 PR 删除了该测试。 问题:元数据在路由缓存落后期间连续迁移两次时,第一跳旧 owner 无法把 key 再重定向到当前 owner,调用方会把缺失结果解释为对象不存在或读失败。 影响:扩缩容或故障恢复期间会出现本可继续路由的短时读失败,降低控制面迁移期间的可用性。 建议:在统一 deadline 内迭代处理后续 response.info,并记录 visited owner/key 以检测环路;保持每个 key 的顺序和独立错误。 验证:恢复两跳及更多跳的重定向用例,并补充环路、重复目标和 deadline 耗尽测试。 ![]() ![]() | |||
| 3089 | + request.set_address(localAddress_.ToString()); | ||
| 3090 | + request.mutable_object_keys()->Add(redirect.change_meta_ids().begin(), redirect.change_meta_ids().end()); | ||
| 3091 | + master::PureQueryMetaRspPb response; | ||
| 3092 | + std::function<Status(master::PureQueryMetaReqPb &, master::PureQueryMetaRspPb &)> query = | ||
| 3093 | + [this, &workerMasterApi](master::PureQueryMetaReqPb &req, master::PureQueryMetaRspPb &rsp) { | ||
| 3094 | + auto rc = workerMasterApi->PureQueryMeta(req, rsp); | ||
| 3095 | + ObserveMetadataRpc(workerMasterApi, rc); | ||
| 3096 | + return rc; | ||
| 3097 | + }; | ||
| 3098 | + RETURN_IF_NOT_OK(RedirectRetryWhenMetasMoving(request, response, query)); | ||
| 3099 | + queryMetas.insert(queryMetas.end(), response.mutable_query_metas()->begin(), | ||
| 3100 | + response.mutable_query_metas()->end()); | ||
| 3101 | + } | ||
| 3102 | + return Status::OK(); | ||
| 3103 | +} | ||
| 3104 | + | ||
| 2999 | Status WorkerOcServiceGetImpl::GetObjMetaInfo(const GetObjMetaInfoReqPb &req, GetObjMetaInfoRspPb &resp) | 3105 | Status WorkerOcServiceGetImpl::GetObjMetaInfo(const GetObjMetaInfoReqPb &req, GetObjMetaInfoRspPb &resp) |
| 3000 | { | 3106 | { |
| 3001 | ScopedRequestContext ctx; | 3107 | ScopedRequestContext ctx; |
| @@ -67,6 +67,23 @@ public: | |||
| 67 | */ | 67 | */ |
| 68 | Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> &serverApi); | 68 | Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> &serverApi); |
| 69 | 69 | ||
| 70 | + /** | ||
| 71 | + * @brief Acquire a resident local object for a side-effect-free direct read. | ||
| 72 | + * @param[in] objectKey Object key to look up. | ||
| 73 | + * @param[out] params Resident object snapshot, or null when the object is not locally readable. | ||
| 74 | + * @return K_OK on hit or miss; the error code otherwise. | ||
| 75 | + */ | ||
| 76 | + Status TryAcquireLocalObject(const std::string &objectKey, std::unique_ptr<GetObjEntryParams> ¶ms); | ||
| 77 | + | ||
| 78 | + /** | ||
| 79 | + * @brief Query metadata-only object locations and resolve Master redirects internally. | ||
| 80 | + * @param[in] objectKeys Object keys to query. | ||
| 81 | + * @param[out] locations Locations keyed by object key; missing keys are absent. | ||
| 82 | + * @return K_OK on success; the original metadata error otherwise. | ||
| 83 | + */ | ||
| 84 | + Status QueryObjectLocations(const std::vector<std::string> &objectKeys, | ||
| 85 | + std::unordered_map<std::string, master::ObjectLocationInfoPb> &locations); | ||
| 86 | + | ||
| 70 | /** | 87 | /** |
| 71 | * @brief Process get request for these objects not exist in local, query object meta from master, then get | 88 | * @brief Process get request for these objects not exist in local, query object meta from master, then get |
| 72 | * object data from remote. | 89 | * object data from remote. |
| @@ -527,6 +544,10 @@ private: | |||
| 527 | 544 | ||
| 528 | Status QueryExistMetadataViaPureQueryMeta(const std::vector<std::string> &objectKeys, | 545 | Status QueryExistMetadataViaPureQueryMeta(const std::vector<std::string> &objectKeys, |
| 529 | std::vector<master::QueryMetaInfoPb> &queryMetas); | 546 | std::vector<master::QueryMetaInfoPb> &queryMetas); |
| 547 | + Status QueryPureMetadataGroup(const HostPort &masterAddress, const std::vector<std::string> &objectKeys, | ||
| 548 | + std::vector<master::QueryMetaInfoPb> &queryMetas); | ||
| 549 | + Status QueryPureMetadataRedirects(const google::protobuf::RepeatedPtrField<RedirectMetaInfo> &redirects, | ||
| 550 | + std::vector<master::QueryMetaInfoPb> &queryMetas); | ||
| 530 | 551 | ||
| 531 | /** | 552 | /** |
| 532 | * @brief Query the metadata of the specified objects in the redirect master. | 553 | * @brief Query the metadata of the specified objects in the redirect master. |
| @@ -0,0 +1,438 @@ | |||
| 1 | +/** | ||
| 2 | + * Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved. | ||
| 3 | + * | ||
| 4 | + * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 5 | + * you may not use this file except in compliance with the License. | ||
| 6 | + * You may obtain a copy of the License at | ||
| 7 | + * | ||
| 8 | + * http://www.apache.org/licenses/LICENSE-2.0 | ||
| 9 | + * | ||
| 10 | + * Unless required by applicable law or agreed to in writing, software | ||
| 11 | + * distributed under the License is distributed on an "AS IS" BASIS, | ||
| 12 | + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 13 | + * See the License for the specific language governing permissions and | ||
| 14 | + * limitations under the License. | ||
| 15 | + */ | ||
| 16 | + | ||
| 17 | +/** Description: Implements Worker-side metadata-affine QueryAndGet processing. */ | ||
| 18 | + | ||
| 19 | + | ||
| 20 | + | ||
| 21 | + | ||
| 22 | + | ||
| 23 | + | ||
| 24 | + | ||
| 25 | + | ||
| 26 | + | ||
| 27 | + | ||
| 28 | + | ||
| 29 | + | ||
| 30 | + | ||
| 31 | + | ||
| 32 | + | ||
| 33 | + | ||
| 34 | + | ||
| 35 | + | ||
| 36 | + | ||
| 37 | + | ||
| 38 | +namespace datasystem { | ||
| 39 | +namespace object_cache { | ||
| 40 | +namespace { | ||
| 41 | +constexpr uint64_t QUERY_AND_GET_MAX_TCP_PAYLOAD_SIZE = 512 * 1024UL; | ||
| 42 | +constexpr double MICROSECONDS_PER_MILLISECOND = 1000.0; | ||
| 43 | + | ||
| 44 | +uint64_t GetSteadyTimeUs() | ||
| 45 | +{ | ||
| 46 | + const auto now = std::chrono::steady_clock::now().time_since_epoch(); | ||
| 47 | + return static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::microseconds>(now).count()); | ||
| 48 | +} | ||
| 49 | + | ||
| 50 | +void FillLocation(const master::ObjectLocationInfoPb &source, QueryAndGetLocationInfoPb &target) | ||
| 51 | +{ | ||
| 52 | + target.set_object_key(source.object_key()); | ||
| 53 | + target.set_object_size(source.object_size()); | ||
| 54 | + *target.mutable_object_locations() = source.object_locations(); | ||
| 55 | +} | ||
| 56 | +} // namespace | ||
| 57 | + | ||
| 58 | +WorkerQueryAndGetImpl::WorkerQueryAndGetImpl(std::shared_ptr<WorkerOcServiceGetImpl> getProc, | ||
| 59 | + std::shared_ptr<SharedMemoryRefTable> memoryRefTable, | ||
| 60 | + std::shared_ptr<AkSkManager> akSkManager, HostPort localAddress, | ||
| 61 | + std::shared_ptr<PeerUbAdmission> ubAdmission) | ||
| 62 | + : getProc_(std::move(getProc)), | ||
| 63 | + memoryRefTable_(std::move(memoryRefTable)), | ||
| 64 | + akSkManager_(std::move(akSkManager)), | ||
| 65 | + localAddress_(std::move(localAddress)), | ||
| 66 | + ubAdmission_(std::move(ubAdmission)) | ||
| 67 | +{ | ||
| 68 | +} | ||
| 69 | + | ||
| 70 | +Status WorkerQueryAndGetImpl::QueryAndGet( | ||
| 71 | + std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi) | ||
| 72 | +{ | ||
| 73 | + RequestState state; | ||
| 74 | + state.startUs = GetSteadyTimeUs(); | ||
| 75 | + state.lastCheckpointUs = state.startUs; | ||
| 76 | + auto access = AccessRecorder::Object(AccessRecorderKey::DS_POSIX_QUERY_AND_GET); | ||
| 77 | + Status rc = ReadAndAuthenticate(serverApi, state.request); | ||
| 78 | + state.stats.preprocessUs = RecordPhase(state); | ||
| 79 | + if (rc.IsOk()) { | ||
| 80 | + access.ObjectKeysRef(state.request.object_keys()).TransportType(GetTransportName(state.request)); | ||
| 81 | + rc = ProcessAndDeliver(serverApi, state); | ||
| 82 | + } | ||
| 83 | + const uint64_t totalUs = state.lastCheckpointUs - state.startUs; | ||
| 84 | + access.DataSize(state.stats.dataSize).Result(rc).Record(); | ||
| 85 | + LogCompletion(state, rc, totalUs); | ||
| 86 | + return rc; | ||
| 87 | +} | ||
| 88 | + | ||
| 89 | +Status WorkerQueryAndGetImpl::ReadAndAuthenticate( | ||
| 90 | + const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi, | ||
| 91 | + QueryAndGetReqPb &request) const | ||
| 92 | +{ | ||
| 93 | + RETURN_IF_NOT_OK_PRINT_ERROR_MSG(serverApi->Read(request), "Read QueryAndGet request failed"); | ||
| 94 | + RETURN_RUNTIME_ERROR_IF_NULL(akSkManager_); | ||
| 95 | + RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(request), "AK/SK failed."); | ||
| 96 | + return Status::OK(); | ||
| 97 | +} | ||
| 98 | + | ||
| 99 | +Status WorkerQueryAndGetImpl::ProcessAndDeliver( | ||
| 100 | + const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi, | ||
| 101 | + RequestState &state) | ||
| 102 | +{ | ||
| 103 | + Raii rollback([this, &state] { | ||
| 104 | + if (!state.delivered) { | ||
| 105 | + RollbackShmRefs(state); | ||
| 106 | + } | ||
| 107 | + }); | ||
| 108 | + RETURN_IF_NOT_OK(BuildResponse(state)); | ||
| 109 | + CollectStats(state); | ||
| 110 | + RETURN_IF_NOT_OK(DeliverResponse(serverApi, state)); | ||
| 111 | + state.delivered = true; | ||
| 112 | + return Status::OK(); | ||
| 113 | +} | ||
| 114 | + | ||
| 115 | +Status WorkerQueryAndGetImpl::DeliverResponse( | ||
| 116 | + const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi, | ||
| 117 | + RequestState &state) const | ||
| 118 | +{ | ||
| 119 | + Status deliveryRc = serverApi->Write(state.response); | ||
| 120 | + if (deliveryRc.IsOk()) { | ||
| 121 | + deliveryRc = serverApi->SendPayload(state.payloads); | ||
| 122 | + } | ||
| 123 | + if (deliveryRc.IsOk()) { | ||
| 124 | + serverApi->SetRequestComplete(); | ||
| 125 | + } | ||
| 126 | + state.stats.deliveryUs = RecordPhase(state); | ||
| 127 | + RETURN_IF_NOT_OK_PRINT_ERROR_MSG(deliveryRc, "Deliver QueryAndGet response failed"); | ||
| 128 | + return Status::OK(); | ||
| 129 | +} | ||
| 130 | + | ||
| 131 | +Status WorkerQueryAndGetImpl::BuildResponse(RequestState &state) | ||
| 132 | +{ | ||
| 133 | + Status rc = PrepareLocalResponse(state); | ||
| 134 | + state.stats.localReadUs = RecordPhase(state); | ||
| 135 | + RETURN_IF_NOT_OK(rc); | ||
| 136 | + rc = FillMissLocations(state); | ||
| 137 | + state.stats.metadataUs = RecordPhase(state); | ||
| 138 | + return rc; | ||
| 139 | +} | ||
| 140 | + | ||
| 141 | +Status WorkerQueryAndGetImpl::PrepareLocalResponse(RequestState &state) | ||
| 142 | +{ | ||
| 143 | + RETURN_IF_NOT_OK(ValidateRequest(state.request)); | ||
| 144 | + RETURN_RUNTIME_ERROR_IF_NULL(getProc_); | ||
| 145 | + InitializeResponse(state); | ||
| 146 | + return EncodeLocalHits(state); | ||
| 147 | +} | ||
| 148 | + | ||
| 149 | +uint64_t WorkerQueryAndGetImpl::RecordPhase(RequestState &state) | ||
| 150 | +{ | ||
| 151 | + const uint64_t nowUs = GetSteadyTimeUs(); | ||
| 152 | + const uint64_t elapsedUs = nowUs - state.lastCheckpointUs; | ||
| 153 | + state.lastCheckpointUs = nowUs; | ||
| 154 | + return elapsedUs; | ||
| 155 | +} | ||
| 156 | + | ||
| 157 | +void WorkerQueryAndGetImpl::InitializeResponse(RequestState &state) const | ||
| 158 | +{ | ||
| 159 | + state.response.Clear(); | ||
| 160 | + state.payloads.clear(); | ||
| 161 | + state.misses.clear(); | ||
| 162 | + state.addedShmRefs.clear(); | ||
| 163 | + state.tcpPayloadSize = 0; | ||
| 164 | + state.misses.reserve(state.request.object_keys_size()); | ||
| 165 | + if (state.request.data_request().has_shm()) { | ||
| 166 | + state.addedShmRefs.reserve(state.request.object_keys_size()); | ||
| 167 | + } | ||
| 168 | + state.response.mutable_results()->Reserve(state.request.object_keys_size()); | ||
| 169 | + for (int i = 0; i < state.request.object_keys_size(); ++i) { | ||
| 170 | + state.response.add_results(); | ||
| 171 | + } | ||
| 172 | +} | ||
| 173 | + | ||
| 174 | +void WorkerQueryAndGetImpl::RollbackShmRefs(const RequestState &state) const | ||
| 175 | +{ | ||
| 176 | + if (!state.request.data_request().has_shm() || state.addedShmRefs.empty()) { | ||
| 177 | + return; | ||
| 178 | + } | ||
| 179 | + const auto clientId = ClientKey::Intern(state.request.data_request().shm().client_id()); | ||
| 180 | + for (const auto &shmId : state.addedShmRefs) { | ||
| 181 | + LOG_IF_ERROR(memoryRefTable_->RemoveShmUnit(clientId, shmId), | ||
| 182 | + "Rollback undelivered QueryAndGet SHM reference"); | ||
| 183 | + } | ||
| 184 | +} | ||
| 185 | + | ||
| 186 | +void WorkerQueryAndGetImpl::CollectStats(RequestState &state) const | ||
| 187 | +{ | ||
| 188 | + for (const auto &result : state.response.results()) { | ||
| 189 | + if (result.has_data_result()) { | ||
| 190 | + ++state.stats.inlineHits; | ||
| 191 | + state.stats.dataSize += result.location().object_size(); | ||
| 192 | + } else { | ||
| 193 | + ++state.stats.misses; | ||
| 194 | + } | ||
| 195 | + } | ||
| 196 | +} | ||
| 197 | + | ||
| 198 | +void WorkerQueryAndGetImpl::LogCompletion(const RequestState &state, const Status &rc, uint64_t totalUs) const | ||
| 199 | +{ | ||
| 200 | + const auto &stats = state.stats; | ||
| 201 | + const auto config = GetServerLatencyTraceConfig(); | ||
| 202 | + SLOW_LOG_IF_OR_VLOG( | ||
| 203 | + INFO, config.processSlowerThanUs > 0 && totalUs >= config.processSlowerThanUs, 1, | ||
| 204 | + FormatString("QueryAndGet done, keyCount: %d, inlineHits: %zu, misses: %zu, transport: %s, " | ||
| 205 | + "preprocess: %.3fms, localRead: %.3fms, metadata: %.3fms, delivery: %.3fms, " | ||
| 206 | + "total: %.3fms, status: %s", | ||
| 207 | + state.request.object_keys_size(), stats.inlineHits, stats.misses, | ||
| 208 | + GetTransportName(state.request), | ||
| 209 | + static_cast<double>(stats.preprocessUs) / MICROSECONDS_PER_MILLISECOND, | ||
| 210 | + static_cast<double>(stats.localReadUs) / MICROSECONDS_PER_MILLISECOND, | ||
| 211 | + static_cast<double>(stats.metadataUs) / MICROSECONDS_PER_MILLISECOND, | ||
| 212 | + static_cast<double>(stats.deliveryUs) / MICROSECONDS_PER_MILLISECOND, | ||
| 213 | + static_cast<double>(totalUs) / MICROSECONDS_PER_MILLISECOND, rc.ToString())); | ||
| 214 | +} | ||
| 215 | + | ||
| 216 | +const char *WorkerQueryAndGetImpl::GetTransportName(const QueryAndGetReqPb &request) const | ||
| 217 | +{ | ||
| 218 | + if (!request.has_data_request()) { | ||
| 219 | + return "NONE"; | ||
| 220 | + } | ||
| 221 | + if (request.data_request().has_shm()) { | ||
| 222 | + return "SHM"; | ||
| 223 | + } | ||
| 224 | + if (request.data_request().has_ub()) { | ||
| 225 | + return "UB"; | ||
| 226 | + } | ||
| 227 | + if (request.data_request().has_tcp()) { | ||
| 228 | + return "TCP"; | ||
| 229 | + } | ||
| 230 | + return "UNKNOWN"; | ||
| 231 | +} | ||
| 232 | + | ||
| 233 | +Status WorkerQueryAndGetImpl::ValidateRequest(const QueryAndGetReqPb &request) const | ||
| 234 | +{ | ||
| 235 | + CHECK_FAIL_RETURN_STATUS(Validator::IsBatchSizeUnderLimit(request.object_keys_size()), K_INVALID, | ||
| 236 | + "QueryAndGet batch size is invalid"); | ||
| 237 | + CHECK_FAIL_RETURN_STATUS(request.object_keys_size() > 0, K_INVALID, "QueryAndGet object keys are empty"); | ||
| 238 | + for (const auto &objectKey : request.object_keys()) { | ||
| 239 | + CHECK_FAIL_RETURN_STATUS(!objectKey.empty(), K_INVALID, "QueryAndGet object key is empty"); | ||
| 240 | + } | ||
| 241 | + if (!request.has_data_request()) { | ||
| 242 | + return Status::OK(); | ||
| 243 | + } | ||
| 244 | + const auto &dataRequest = request.data_request(); | ||
| 245 | + CHECK_FAIL_RETURN_STATUS(dataRequest.has_tcp() || dataRequest.has_ub() || dataRequest.has_shm(), K_INVALID, | ||
| 246 | + "QueryAndGet data transport is not set"); | ||
| 247 | + if (dataRequest.has_shm()) { | ||
| 248 | + RETURN_RUNTIME_ERROR_IF_NULL(memoryRefTable_); | ||
| 249 | + CHECK_FAIL_RETURN_STATUS(!dataRequest.shm().client_id().empty(), K_INVALID, | ||
| 250 | + "QueryAndGet SHM client ID is empty"); | ||
| 251 | + const auto clientId = ClientKey::Intern(dataRequest.shm().client_id()); | ||
| 252 | + CHECK_FAIL_RETURN_STATUS(worker::ClientManager::Instance().ClientShmEnabled(clientId), K_NOT_SUPPORTED, | ||
| 253 | + "QueryAndGet SHM session is unavailable"); | ||
| 254 | + } | ||
| 255 | + if (!dataRequest.has_ub()) { | ||
| 256 | + return Status::OK(); | ||
| 257 | + } | ||
| 258 | + const auto &ub = dataRequest.ub(); | ||
| 259 | + CHECK_FAIL_RETURN_STATUS(ub.buffer_size() > 0 && !ub.urma_instance_id().empty(), K_INVALID, | ||
| 260 | + "QueryAndGet UB buffer is invalid"); | ||
| 261 | + CHECK_FAIL_RETURN_STATUS(ub.buffer_infos_size() == request.object_keys_size(), K_INVALID, | ||
| 262 | + "QueryAndGet UB buffer count does not match object key count"); | ||
| 263 | + const auto &remote = ub.buffer_infos(0); | ||
| 264 | + HostPort remoteAddress(remote.request_address().host(), remote.request_address().port()); | ||
| 265 | + const std::string connectionId = remote.client_id().empty() ? remoteAddress.ToString() : remote.client_id(); | ||
| 266 | + RETURN_IF_NOT_OK(CheckTransportConnectionStable(connectionId, ub.urma_instance_id())); | ||
| 267 | + return Status::OK(); | ||
| 268 | +} | ||
| 269 | + | ||
| 270 | +Status WorkerQueryAndGetImpl::EncodeLocalHits(RequestState &state) | ||
| 271 | +{ | ||
| 272 | + const auto &request = state.request; | ||
| 273 | + for (int i = 0; i < request.object_keys_size(); ++i) { | ||
| 274 | + const auto &objectKey = request.object_keys(i); | ||
| 275 | + if (!request.has_data_request()) { | ||
| 276 | + state.misses.emplace_back(objectKey); | ||
| 277 | + continue; | ||
| 278 | + } | ||
| 279 | + std::unique_ptr<GetObjEntryParams> params; | ||
| 280 | + RETURN_IF_NOT_OK(getProc_->TryAcquireLocalObject(objectKey, params)); | ||
[警告][正确性] [P1] 本地直读缺少当前版本与失效中的权威栅栏 证据:新路径只要 TryAcquireLocalObject 返回本地 entry 就直接编码;该函数只检查 Worker 本地 entry 已发布、未标记 CacheInvalid 且 SHM 完整。基线 Master QueryAndGet 在直读前还要求当前 metadata 中该 location 为 ACK、没有 CACHE_INVALID 或 PRIMARY_COPY_INVALID 异步操作,并携带当前 metadata version 读取 Worker。 问题:覆盖写、删除、迁移或主副本切换已在 Master 提交、但异步失效通知尚未落到 Worker 的窗口内,旧 entry 仍可能通过本地检查并作为命中返回,且不会再查询 PureQueryMeta。 影响:KVClient::Get 可能返回已被替换或删除的旧数据,破坏对象版本一致性,也不满足 PR 所称对外语义不变。 建议:本地直读前引入与权威 metadata 的 version、ACK location 和 pending invalidation 同步的可见栅栏;无法原子证明本地 generation 仍有效时,应按 miss 走 PureQueryMeta 和 ReplicaReader。 验证:增加 overwrite、delete、primary reselection、recovery newer-version 与失效通知延迟交错的确定性用例,断言旧版本绝不会以内联结果返回。 ![]() ![]() | |||
| 281 | + if (params == nullptr) { | ||
| 282 | + state.misses.emplace_back(objectKey); | ||
| 283 | + continue; | ||
| 284 | + } | ||
| 285 | + const size_t payloadCount = state.payloads.size(); | ||
| 286 | + const uint64_t tcpPayloadSize = state.tcpPayloadSize; | ||
| 287 | + bool encoded = false; | ||
| 288 | + Status rc = EncodeLocalHit(state, static_cast<size_t>(i), *params, encoded); | ||
| 289 | + if (rc.IsError() || !encoded) { | ||
| 290 | + state.payloads.resize(payloadCount); | ||
| 291 | + state.tcpPayloadSize = tcpPayloadSize; | ||
| 292 | + state.response.mutable_results(i)->Clear(); | ||
| 293 | + state.misses.emplace_back(objectKey); | ||
| 294 | + VLOG_IF(1, rc.IsError()) << "[ObjectKey " << objectKey | ||
| 295 | + << "] QueryAndGet inline data fallback: " << rc.ToString(); | ||
| 296 | + } | ||
| 297 | + } | ||
| 298 | + return Status::OK(); | ||
| 299 | +} | ||
| 300 | + | ||
| 301 | +Status WorkerQueryAndGetImpl::EncodeLocalHit(RequestState &state, size_t index, | ||
| 302 | + const GetObjEntryParams ¶ms, bool &encoded) | ||
| 303 | +{ | ||
| 304 | + INJECT_POINT("worker.QueryAndGet.EncodeLocalHitFailure"); | ||
| 305 | + const auto &request = state.request; | ||
| 306 | + auto &result = *state.response.mutable_results(static_cast<int>(index)); | ||
| 307 | + if (request.data_request().has_tcp()) { | ||
| 308 | + RETURN_IF_NOT_OK(EncodeTcp(params, *result.mutable_data_result(), state, encoded)); | ||
| 309 | + } else if (request.data_request().has_ub()) { | ||
| 310 | + RETURN_IF_NOT_OK(EncodeUb(request.data_request().ub(), index, params, encoded)); | ||
| 311 | + if (encoded) { | ||
| 312 | + result.mutable_data_result(); | ||
| 313 | + } | ||
| 314 | + } else { | ||
| 315 | + EncodeShm(request.data_request().shm(), params, *result.mutable_data_result(), state); | ||
| 316 | + encoded = true; | ||
| 317 | + } | ||
| 318 | + if (encoded) { | ||
| 319 | + auto *location = result.mutable_location(); | ||
| 320 | + location->set_object_key(request.object_keys(static_cast<int>(index))); | ||
| 321 | + location->add_object_locations(localAddress_.ToString()); | ||
| 322 | + location->set_object_size(params.dataSize); | ||
| 323 | + } | ||
| 324 | + return Status::OK(); | ||
| 325 | +} | ||
| 326 | + | ||
| 327 | +Status WorkerQueryAndGetImpl::EncodeTcp(const GetObjEntryParams ¶ms, QueryAndGetDataResultPb &result, | ||
| 328 | + RequestState &state, bool &encoded) const | ||
| 329 | +{ | ||
| 330 | + encoded = false; | ||
| 331 | + if (state.tcpPayloadSize > QUERY_AND_GET_MAX_TCP_PAYLOAD_SIZE | ||
| 332 | + || params.dataSize > QUERY_AND_GET_MAX_TCP_PAYLOAD_SIZE - state.tcpPayloadSize) { | ||
| 333 | + return Status::OK(); | ||
| 334 | + } | ||
| 335 | + ShmGuard shmGuard(params.shmUnit, params.dataSize, params.metaSize); | ||
| 336 | + if (WorkerOcServiceCrudCommonApi::ShmEnable()) { | ||
| 337 | + RETURN_IF_NOT_OK(shmGuard.TryRLatch()); | ||
| 338 | + } | ||
| 339 | + const size_t firstIndex = state.payloads.size(); | ||
| 340 | + RETURN_IF_NOT_OK(shmGuard.TransferTo(state.payloads, 0, params.dataSize)); | ||
| 341 | + for (size_t i = firstIndex; i < state.payloads.size(); ++i) { | ||
| 342 | + CHECK_FAIL_RETURN_STATUS(i <= std::numeric_limits<uint32_t>::max(), K_RUNTIME_ERROR, | ||
| 343 | + "QueryAndGet TCP payload index exceeds protocol limits"); | ||
| 344 | + result.add_payload_indexes(static_cast<uint32_t>(i)); | ||
| 345 | + } | ||
| 346 | + state.tcpPayloadSize += params.dataSize; | ||
| 347 | + METRIC_ADD(metrics::KvMetricId::WORKER_TO_CLIENT_TOTAL_BYTES, params.dataSize); | ||
| 348 | + encoded = true; | ||
| 349 | + INJECT_POINT_NO_RETURN("worker.QueryAndGet.EncodeTcp"); | ||
| 350 | + return Status::OK(); | ||
| 351 | +} | ||
| 352 | + | ||
| 353 | +Status WorkerQueryAndGetImpl::EncodeUb(const QueryAndGetUbDataReqPb &request, size_t index, | ||
| 354 | + const GetObjEntryParams ¶ms, bool &encoded) const | ||
| 355 | +{ | ||
| 356 | + encoded = false; | ||
| 357 | + if (params.dataSize > request.buffer_size()) { | ||
| 358 | + return Status::OK(); | ||
| 359 | + } | ||
| 360 | + const auto &remote = request.buffer_infos(static_cast<int>(index)); | ||
| 361 | + ShmGuard shmGuard(params.shmUnit, params.dataSize, params.metaSize); | ||
| 362 | + if (WorkerOcServiceCrudCommonApi::ShmEnable()) { | ||
| 363 | + RETURN_IF_NOT_OK(shmGuard.TryRLatch()); | ||
| 364 | + } | ||
| 365 | + const uint64_t base = reinterpret_cast<uint64_t>(params.shmUnit->GetPointer()); | ||
| 366 | + uint64_t segmentAddress = 0; | ||
| 367 | + uint64_t segmentSize = 0; | ||
| 368 | + GetSegmentInfoFromShmUnit(params.shmUnit, base, segmentAddress, segmentSize); | ||
| 369 | + const uint8_t srcChipId = NumaIdToChipId(params.shmUnit->GetNumaId()); | ||
| 370 | + const uint8_t dstChipId = remote.has_chip_id() ? static_cast<uint8_t>(remote.chip_id()) : INVALID_CHIP_ID; | ||
| 371 | + std::vector<uint64_t> eventKeys; | ||
| 372 | + UrmaWriteFailure failure; | ||
| 373 | + auto lateCompletionContext = | ||
| 374 | + ubAdmission_ == nullptr | ||
| 375 | + ? std::nullopt | ||
| 376 | + : ubAdmission_->BuildLateCompletionContext(UbOperationKind::CLIENT_GET_WRITEBACK); | ||
| 377 | + Status rc = UrmaWritePayload(remote, segmentAddress, segmentSize, base, 0, params.dataSize, params.metaSize, | ||
| 378 | + srcChipId, dstChipId, true, eventKeys, nullptr, &failure, | ||
| 379 | + std::move(lateCompletionContext)); | ||
| 380 | + if (rc.IsError()) { | ||
| 381 | + const auto &address = remote.request_address(); | ||
| 382 | + const HostPort failedEndpoint(address.host(), address.port()); | ||
| 383 | + ReportLocalUbOperationFailure(ubAdmission_.get(), localAddress_, failedEndpoint, | ||
| 384 | + UbOperationKind::CLIENT_GET_WRITEBACK, rc, failure.providerStatus, | ||
| 385 | + failure.cqeStatus); | ||
| 386 | + return rc; | ||
O [P2] 内联数据编码失败按整批失败处理,与"无 data_result 即 per-key 快路径 miss"的设计不符,较旧实现存在可用性回退 PR 描述承诺"无 data_result 表示快路径未命中,由 Client 继续走现有 ReplicaReader"——容量超限路径(EncodeTcp encoded=false → miss)确实实现了 per-key 降级,但错误路径没有:
对比旧实现(被删除的 master 侧 TryGetQueryAndGetData):本地读/UB 写失败仅跳过该 key 的内联数据(VLOG 后 return),响应仍携带 location,客户端自然降级副本读,同请求其余 key 不受影响;不会把 200ms 级 latch 竞争放大为整批失败。 建议:编码/物化失败按 per-key 处理——失败 key 走 miss 分支(Clear result + 加入 misses,仅返回 location),与容量超限路径保持一致;避免 eviction/migration/set storm 期间 meta owner 上所有直读 Get 整批失败。 ![]() ![]() | |||
| 387 | + } | ||
| 388 | + METRIC_ADD(metrics::KvMetricId::WORKER_TO_CLIENT_TOTAL_BYTES, params.dataSize); | ||
| 389 | + encoded = true; | ||
| 390 | + INJECT_POINT_NO_RETURN("worker.QueryAndGet.EncodeUb"); | ||
| 391 | + return Status::OK(); | ||
| 392 | +} | ||
| 393 | + | ||
| 394 | +void WorkerQueryAndGetImpl::EncodeShm(const QueryAndGetShmDataReqPb &request, const GetObjEntryParams ¶ms, | ||
| 395 | + QueryAndGetDataResultPb &result, RequestState &state) const | ||
| 396 | +{ | ||
| 397 | + const auto clientId = ClientKey::Intern(request.client_id()); | ||
| 398 | + auto shmUnit = params.shmUnit; | ||
| 399 | + memoryRefTable_->AddShmUnit(clientId, shmUnit, | ||
| 400 | + GetRequestContext()->reqTimeoutDuration.CalcRealRemainingTime()); | ||
| 401 | + state.addedShmRefs.emplace_back(params.shmUnit->GetId()); | ||
| 402 | + auto *info = result.mutable_shm_info(); | ||
| 403 | + info->set_store_fd(params.shmUnit->GetFd()); | ||
| 404 | + info->set_offset(static_cast<int64_t>(params.shmUnit->GetOffset())); | ||
| 405 | + info->set_data_size(static_cast<int64_t>(params.dataSize)); | ||
| 406 | + info->set_metadata_size(static_cast<int64_t>(params.metaSize)); | ||
| 407 | + info->set_mmap_size(static_cast<int64_t>(params.shmUnit->GetMmapSize())); | ||
| 408 | + info->set_shm_id(params.shmUnit->id.ToString()); | ||
| 409 | + info->set_is_seal(params.isSealed); | ||
| 410 | + info->set_write_mode(static_cast<uint32_t>(params.objectMode.GetWriteMode())); | ||
| 411 | + info->set_consistency_type(static_cast<uint32_t>(params.objectMode.GetConsistencyType())); | ||
[警告][正确性] [P2] SHM inline 响应遗漏 cache_type 证据:QueryAndGetShmInfoPb 定义了 cache_type,客户端 BuildQueryAndGetResult 会读取并写入 ExternalBufferMeta.mode;这里仅填写 write_mode 和 consistency_type,没有从 params.objectMode 填 cache_type。proto3 默认值 0 对应 MEMORY。 问题:本地驻留的 DISK cache 对象经 SHM inline 返回后会被静默解释为 MEMORY,且客户端的枚举范围校验无法发现默认值造成的语义丢失。 影响:返回 Buffer 的对象模式与普通 Get 不一致,后续依赖 cache type 的行为和诊断信息可能错误。 建议:在 SHM 结果中显式设置 params.objectMode.GetCacheType(),并确保三种 mode 字段与普通 Get 共用同一编码 helper,避免协议路径漂移。 示例:
验证:分别创建 MEMORY 与 DISK cache 对象并命中 Worker SHM QueryAndGet,断言 materialized Buffer 保留原始 CacheType。 ![]() ![]() | |||
| 412 | + INJECT_POINT_NO_RETURN("worker.QueryAndGet.EncodeShm"); | ||
| 413 | +} | ||
| 414 | + | ||
| 415 | +Status WorkerQueryAndGetImpl::FillMissLocations(RequestState &state) const | ||
| 416 | +{ | ||
| 417 | + if (!state.misses.empty()) { | ||
| 418 | + INJECT_POINT_NO_RETURN("worker.QueryAndGet.QueryMissMetadata"); | ||
| 419 | + } | ||
| 420 | + std::unordered_map<std::string, master::ObjectLocationInfoPb> locations; | ||
| 421 | + RETURN_IF_NOT_OK(getProc_->QueryObjectLocations(state.misses, locations)); | ||
[严重][Bug] [P1] 不要让 miss 的 Master 故障丢弃同批本地命中 证据: 问题:新版把元数据归属 Worker 同时变成了本地数据快路径和 Master 查询代理,但响应仍只有一个 RPC 级错误域;内层 Master 故障因此污染整个 owner 分组,并被错误归因给外层 Worker。 影响:mixed hit/miss 批量在 Master 短时不可用时连本地已命中的对象也无法返回,还可能淘汰健康 owner 的连接或路由,扩大故障域并放大恢复时延。 建议:把嵌套元数据错误限制在 miss key:响应增加逐 key status(或等价的可区分错误结果),保留并交付已编码的 local hits;客户端只让失败 key 返回错误或重试,并保留错误来源,避免把 Master 故障传给 metadata-owner failure handler。 验证:增加 mixed local-hit + miss 用例,在 ![]() ![]() | |||
| 422 | + for (int i = 0; i < state.request.object_keys_size(); ++i) { | ||
| 423 | + auto *result = state.response.mutable_results(i); | ||
| 424 | + if (result->has_data_result()) { | ||
| 425 | + continue; | ||
| 426 | + } | ||
| 427 | + const auto &objectKey = state.request.object_keys(i); | ||
| 428 | + result->mutable_location()->set_object_key(objectKey); | ||
| 429 | + auto location = locations.find(objectKey); | ||
| 430 | + if (location != locations.end()) { | ||
| 431 | + FillLocation(location->second, *result->mutable_location()); | ||
| 432 | + } | ||
| 433 | + } | ||
| 434 | + return Status::OK(); | ||
| 435 | +} | ||
| 436 | + | ||
| 437 | +} // namespace object_cache | ||
| 438 | +} // namespace datasystem | ||
| @@ -0,0 +1,126 @@ | |||
| 1 | +/** | ||
| 2 | + * Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved. | ||
| 3 | + * | ||
| 4 | + * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 5 | + * you may not use this file except in compliance with the License. | ||
| 6 | + * You may obtain a copy of the License at | ||
| 7 | + * | ||
| 8 | + * http://www.apache.org/licenses/LICENSE-2.0 | ||
| 9 | + * | ||
| 10 | + * Unless required by applicable law or agreed to in writing, software | ||
| 11 | + * distributed under the License is distributed on an "AS IS" BASIS, | ||
| 12 | + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 13 | + * See the License for the specific language governing permissions and | ||
| 14 | + * limitations under the License. | ||
| 15 | + */ | ||
| 16 | + | ||
| 17 | +/** Description: Defines Worker-side metadata-affine QueryAndGet processing. */ | ||
| 18 | + | ||
| 19 | + | ||
| 20 | + | ||
| 21 | + | ||
| 22 | + | ||
| 23 | + | ||
| 24 | + | ||
| 25 | + | ||
| 26 | + | ||
| 27 | + | ||
| 28 | + | ||
| 29 | + | ||
| 30 | + | ||
| 31 | + | ||
| 32 | + | ||
| 33 | +namespace datasystem { | ||
| 34 | +namespace object_cache { | ||
| 35 | + | ||
| 36 | +/** @brief Reads resident Worker objects inline and returns metadata-only locations for misses. */ | ||
| 37 | +class WorkerQueryAndGetImpl { | ||
| 38 | +public: | ||
| 39 | + /** | ||
| 40 | + * @brief Construct the Worker QueryAndGet processor. | ||
| 41 | + * @param[in] getProc Existing local-read and metadata-location service. | ||
| 42 | + * @param[in] memoryRefTable Worker shared-memory reference table. | ||
| 43 | + * @param[in] akSkManager Worker authentication manager. | ||
| 44 | + * @param[in] localAddress Local Worker address. | ||
| 45 | + * @param[in] ubAdmission Worker UB admission state. | ||
| 46 | + */ | ||
| 47 | + WorkerQueryAndGetImpl(std::shared_ptr<WorkerOcServiceGetImpl> getProc, | ||
| 48 | + std::shared_ptr<SharedMemoryRefTable> memoryRefTable, | ||
| 49 | + std::shared_ptr<AkSkManager> akSkManager, HostPort localAddress, | ||
| 50 | + std::shared_ptr<PeerUbAdmission> ubAdmission); | ||
| 51 | + | ||
| 52 | + /** | ||
| 53 | + * @brief Destroy the Worker QueryAndGet processor. | ||
| 54 | + */ | ||
| 55 | + ~WorkerQueryAndGetImpl() = default; | ||
| 56 | + | ||
| 57 | + /** | ||
| 58 | + * @brief Process one ordered QueryAndGet request. | ||
| 59 | + * @param[in] serverApi Server RPC reader and writer. | ||
| 60 | + * @return K_OK on success; the error code otherwise. | ||
| 61 | + */ | ||
| 62 | + Status QueryAndGet( | ||
| 63 | + std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi); | ||
| 64 | + | ||
| 65 | +private: | ||
| 66 | + struct QueryStats { | ||
| 67 | + size_t inlineHits = 0; | ||
| 68 | + size_t misses = 0; | ||
| 69 | + uint64_t dataSize = 0; | ||
| 70 | + uint64_t preprocessUs = 0; | ||
| 71 | + uint64_t localReadUs = 0; | ||
| 72 | + uint64_t metadataUs = 0; | ||
| 73 | + uint64_t deliveryUs = 0; | ||
| 74 | + }; | ||
| 75 | + | ||
| 76 | + struct RequestState { | ||
| 77 | + QueryAndGetReqPb request; | ||
| 78 | + QueryAndGetRspPb response; | ||
| 79 | + std::vector<RpcMessage> payloads; | ||
| 80 | + std::vector<std::string> misses; | ||
| 81 | + std::vector<ShmKey> addedShmRefs; | ||
| 82 | + QueryStats stats; | ||
| 83 | + uint64_t startUs = 0; | ||
| 84 | + uint64_t lastCheckpointUs = 0; | ||
| 85 | + uint64_t tcpPayloadSize = 0; | ||
| 86 | + bool delivered = false; | ||
| 87 | + }; | ||
| 88 | + | ||
| 89 | + Status ReadAndAuthenticate( | ||
| 90 | + const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi, | ||
| 91 | + QueryAndGetReqPb &request) const; | ||
| 92 | + Status ProcessAndDeliver( | ||
| 93 | + const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi, | ||
| 94 | + RequestState &state); | ||
| 95 | + Status BuildResponse(RequestState &state); | ||
| 96 | + Status DeliverResponse( | ||
| 97 | + const std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> &serverApi, | ||
| 98 | + RequestState &state) const; | ||
| 99 | + void InitializeResponse(RequestState &state) const; | ||
| 100 | + Status PrepareLocalResponse(RequestState &state); | ||
| 101 | + static uint64_t RecordPhase(RequestState &state); | ||
| 102 | + void RollbackShmRefs(const RequestState &state) const; | ||
| 103 | + void CollectStats(RequestState &state) const; | ||
| 104 | + void LogCompletion(const RequestState &state, const Status &rc, uint64_t totalUs) const; | ||
| 105 | + const char *GetTransportName(const QueryAndGetReqPb &request) const; | ||
| 106 | + Status ValidateRequest(const QueryAndGetReqPb &request) const; | ||
| 107 | + Status EncodeLocalHits(RequestState &state); | ||
| 108 | + Status EncodeLocalHit(RequestState &state, size_t index, const GetObjEntryParams ¶ms, bool &encoded); | ||
| 109 | + Status EncodeTcp(const GetObjEntryParams ¶ms, QueryAndGetDataResultPb &result, | ||
| 110 | + RequestState &state, bool &encoded) const; | ||
| 111 | + Status EncodeUb(const QueryAndGetUbDataReqPb &request, size_t index, const GetObjEntryParams ¶ms, | ||
| 112 | + bool &encoded) const; | ||
| 113 | + void EncodeShm(const QueryAndGetShmDataReqPb &request, const GetObjEntryParams ¶ms, | ||
| 114 | + QueryAndGetDataResultPb &result, RequestState &state) const; | ||
| 115 | + Status FillMissLocations(RequestState &state) const; | ||
| 116 | + | ||
| 117 | + std::shared_ptr<WorkerOcServiceGetImpl> getProc_; | ||
| 118 | + std::shared_ptr<SharedMemoryRefTable> memoryRefTable_; | ||
| 119 | + std::shared_ptr<AkSkManager> akSkManager_; | ||
| 120 | + HostPort localAddress_; | ||
| 121 | + std::shared_ptr<PeerUbAdmission> ubAdmission_; | ||
| 122 | +}; | ||
| 123 | + | ||
| 124 | +} // namespace object_cache | ||
| 125 | +} // namespace datasystem | ||
| 126 | + | ||
| @@ -451,6 +451,8 @@ void WorkerOCServiceImpl::InitServiceImpl() | |||
| 451 | getProc_ = | 451 | getProc_ = |
| 452 | std::make_shared<WorkerOcServiceGetImpl>(param, etcdStore_, memCpyThreadPool_, threadPool_, | 452 | std::make_shared<WorkerOcServiceGetImpl>(param, etcdStore_, memCpyThreadPool_, threadPool_, |
| 453 | akSkManager_, localAddress_, migrateRateController_, ubAdmission_); | 453 | akSkManager_, localAddress_, migrateRateController_, ubAdmission_); |
| 454 | + queryAndGetProc_ = | ||
| 455 | + std::make_shared<WorkerQueryAndGetImpl>(getProc_, memoryRefTable_, akSkManager_, localAddress_, ubAdmission_); | ||
| 454 | 456 | ||
| 455 | deleteProc_ = | 457 | deleteProc_ = |
| 456 | std::make_shared<WorkerOcServiceDeleteImpl>(param, akSkManager_, localAddress_, getProc_); | 458 | std::make_shared<WorkerOcServiceDeleteImpl>(param, akSkManager_, localAddress_, getProc_); |
| @@ -1655,6 +1657,18 @@ Status WorkerOCServiceImpl::Get(std::shared_ptr<::datasystem::ServerUnaryWriterR | |||
| 1655 | return getProc_->Get(serverApi); | 1657 | return getProc_->Get(serverApi); |
| 1656 | } | 1658 | } |
| 1657 | 1659 | ||
| 1660 | +Status WorkerOCServiceImpl::QueryAndGet( | ||
| 1661 | + std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> serverApi) | ||
| 1662 | +{ | ||
| 1663 | + ScopedRequestContext ctx; | ||
| 1664 | + BthreadReadGuard noRecon; | ||
| 1665 | + RETURN_IF_NOT_OK_PRINT_ERROR_MSG( | ||
| 1666 | + ValidateWorkerState(noRecon, GetRequestContext()->reqTimeoutDuration.CalcRemainingTime()), | ||
| 1667 | + "validate worker state failed"); | ||
| 1668 | + RETURN_RUNTIME_ERROR_IF_NULL(queryAndGetProc_); | ||
| 1669 | + return queryAndGetProc_->QueryAndGet(serverApi); | ||
| 1670 | +} | ||
| 1671 | + | ||
| 1658 | Status WorkerOCServiceImpl::RefreshMeta(const ClientKey &clientId) | 1672 | Status WorkerOCServiceImpl::RefreshMeta(const ClientKey &clientId) |
| 1659 | { | 1673 | { |
| 1660 | LOG(INFO) << "[RefreshMeta] clear memory reference count for client:" << clientId; | 1674 | LOG(INFO) << "[RefreshMeta] clear memory reference count for client:" << clientId; |
| @@ -89,6 +89,7 @@ | |||
| 89 | 89 | ||
| 90 | 90 | ||
| 91 | 91 | ||
| 92 | + | ||
| 92 | 93 | ||
| 93 | // Keep bthread headers after project RPC/log headers so brpc logging macros (CHECK_EQ etc.) are | 94 | // Keep bthread headers after project RPC/log headers so brpc logging macros (CHECK_EQ etc.) are |
| 94 | // established before bthread/mutex.h (which uses but does not define them) avoid redefinition pitfalls. | 95 | // established before bthread/mutex.h (which uses but does not define them) avoid redefinition pitfalls. |
| @@ -359,6 +360,14 @@ public: | |||
| 359 | */ | 360 | */ |
| 360 | Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> serverApi) override; | 361 | Status Get(std::shared_ptr<ServerUnaryWriterReader<GetRspPb, GetReqPb>> serverApi) override; |
| 361 | 362 | ||
| 363 | + /** | ||
| 364 | + * @brief Read metadata-owner-local objects and return locations for local misses. | ||
| 365 | + * @param[in] serverApi Unary socket carrying the request, response, and TCP payloads. | ||
| 366 | + * @return K_OK on success; the error code otherwise. | ||
| 367 | + */ | ||
| 368 | + Status QueryAndGet( | ||
| 369 | + std::shared_ptr<ServerUnaryWriterReader<QueryAndGetRspPb, QueryAndGetReqPb>> serverApi) override; | ||
| 370 | + | ||
| 362 | /** | 371 | /** |
| 363 | * @brief Decrease the reference count of client. | 372 | * @brief Decrease the reference count of client. |
| 364 | * @param[in] req The rpc request protobuf. | 373 | * @param[in] req The rpc request protobuf. |
| @@ -1480,6 +1489,8 @@ private: | |||
| 1480 | 1489 | ||
| 1481 | std::shared_ptr<WorkerOcServiceGetImpl> getProc_{ nullptr }; | 1490 | std::shared_ptr<WorkerOcServiceGetImpl> getProc_{ nullptr }; |
| 1482 | 1491 | ||
| 1492 | + std::shared_ptr<WorkerQueryAndGetImpl> queryAndGetProc_{ nullptr }; | ||
| 1493 | + | ||
| 1483 | std::shared_ptr<PeerUbAdmission> ubAdmission_{ std::make_shared<PeerUbAdmission>() }; | 1494 | std::shared_ptr<PeerUbAdmission> ubAdmission_{ std::make_shared<PeerUbAdmission>() }; |
| 1484 | 1495 | ||
| 1485 | std::shared_ptr<WorkerOcServiceDeleteImpl> deleteProc_{ nullptr }; | 1496 | std::shared_ptr<WorkerOcServiceDeleteImpl> deleteProc_{ nullptr }; |
| @@ -60,13 +60,6 @@ Status WorkerLocalWorkerOCApi::GetObjectRemote(GetObjectRemoteReqPb &req, GetObj | |||
| 60 | return service_->GetObjectRemote(req, rsp, payload); | 60 | return service_->GetObjectRemote(req, rsp, payload); |
| 61 | } | 61 | } |
| 62 | 62 | ||
| 63 | -Status WorkerLocalWorkerOCApi::GetObjectRemoteForQueryAndGet(GetObjectRemoteReqPb &req, | ||
| 64 | - std::vector<RpcMessage> &payload) | ||
| 65 | -{ | ||
| 66 | - GetObjectRemoteRspPb rsp; | ||
| 67 | - return service_->GetObjectRemote(req, rsp, payload, true); | ||
| 68 | -} | ||
| 69 | - | ||
| 70 | WorkerRemoteWorkerOCApi::WorkerRemoteWorkerOCApi(HostPort hostPort, HostPort localHostPort, | 63 | WorkerRemoteWorkerOCApi::WorkerRemoteWorkerOCApi(HostPort hostPort, HostPort localHostPort, |
| 71 | std::shared_ptr<AkSkManager> akSkManager) | 64 | std::shared_ptr<AkSkManager> akSkManager) |
| 72 | : hostPort_(std::move(hostPort)), localHostPort_(std::move(localHostPort)), akSkManager_(std::move(akSkManager)) | 65 | : hostPort_(std::move(hostPort)), localHostPort_(std::move(localHostPort)), akSkManager_(std::move(akSkManager)) |
| @@ -100,14 +100,6 @@ public: | |||
| 100 | Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, | 100 | Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, |
| 101 | std::vector<RpcMessage> &payload) override; | 101 | std::vector<RpcMessage> &payload) override; |
| 102 | 102 | ||
| 103 | - /** | ||
| 104 | - * @brief Try to read resident data for a local QueryAndGet fast path. | ||
| 105 | - * @param[in] req The object-read request. | ||
| 106 | - * @param[out] payload The optional TCP payload. | ||
| 107 | - * @return Status of the call. | ||
| 108 | - */ | ||
| 109 | - Status GetObjectRemoteForQueryAndGet(GetObjectRemoteReqPb &req, std::vector<RpcMessage> &payload); | ||
| 110 | - | ||
| 111 | Status GetObjectRemote( | 103 | Status GetObjectRemote( |
| 112 | std::unique_ptr<ClientUnaryWriterReader<GetObjectRemoteReqPb, GetObjectRemoteRspPb>> *clientApi) override | 104 | std::unique_ptr<ClientUnaryWriterReader<GetObjectRemoteReqPb, GetObjectRemoteRspPb>> *clientApi) override |
| 113 | { | 105 | { |
| @@ -241,7 +241,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemote( | |||
| 241 | } | 241 | } |
| 242 | 242 | ||
| 243 | Status WorkerWorkerOCServiceImpl::GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, | 243 | Status WorkerWorkerOCServiceImpl::GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, |
| 244 | - std::vector<RpcMessage> &payload, bool isQueryAndGet) | 244 | + std::vector<RpcMessage> &payload) |
| 245 | { | 245 | { |
| 246 | // Inherit the SDK traceID from the worker thread's thread_local Trace (set by | 246 | // Inherit the SDK traceID from the worker thread's thread_local Trace (set by |
| 247 | // WorkerEntryImpl's SetTraceContextFromMeta) into the per-request context so | 247 | // WorkerEntryImpl's SetTraceContextFromMeta) into the per-request context so |
| @@ -252,14 +252,10 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemote(GetObjectRemoteReqPb &req, Get | |||
| 252 | ScopedRequestContext ctx; | 252 | ScopedRequestContext ctx; |
| 253 | METRIC_TIMER(metrics::KvMetricId::WORKER_RPC_REMOTE_GET_INBOUND_LATENCY); | 253 | METRIC_TIMER(metrics::KvMetricId::WORKER_RPC_REMOTE_GET_INBOUND_LATENCY); |
| 254 | Timer slowLogTimer; | 254 | Timer slowLogTimer; |
| 255 | - if (isQueryAndGet) { | ||
| 256 | - RETURN_IF_NOT_OK(CheckConnectionStable(req)); | ||
| 257 | - } | ||
| 258 | RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(req), "AK/SK failed."); | 255 | RETURN_IF_NOT_OK_PRINT_ERROR_MSG(akSkManager_->VerifySignatureAndTimestamp(req), "AK/SK failed."); |
| 259 | const std::string callerAddress = GetRemoteAddressForLog(req); | 256 | const std::string callerAddress = GetRemoteAddressForLog(req); |
| 260 | std::vector<uint64_t> eventKeys; | 257 | std::vector<uint64_t> eventKeys; |
| 261 | - RETURN_IF_NOT_OK(GetObjectRemoteHandler(req, rsp, payload, true, eventKeys, nullptr, nullptr, nullptr, | 258 | + RETURN_IF_NOT_OK(GetObjectRemoteHandler(req, rsp, payload, true, eventKeys)); |
| 262 | - nullptr, isQueryAndGet)); | ||
| 263 | auto config = GetServerLatencyTraceConfig(); | 259 | auto config = GetServerLatencyTraceConfig(); |
| 264 | uint64_t elapsedUs = static_cast<uint64_t>(slowLogTimer.ElapsedMicroSecond()); | 260 | uint64_t elapsedUs = static_cast<uint64_t>(slowLogTimer.ElapsedMicroSecond()); |
| 265 | SLOW_LOG_IF_OR_VLOG( | 261 | SLOW_LOG_IF_OR_VLOG( |
| @@ -471,7 +467,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteHandler(const GetObjectRemoteRe | |||
| 471 | std::vector<uint64_t> &eventKeys, | 467 | std::vector<uint64_t> &eventKeys, |
| 472 | std::shared_ptr<AggregateMemory> batchPtr, | 468 | std::shared_ptr<AggregateMemory> batchPtr, |
| 473 | RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus, | 469 | RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus, |
| 474 | - BatchRh2dContext *batchRh2dContext, bool isQueryAndGet) | 470 | + BatchRh2dContext *batchRh2dContext) |
| 475 | { | 471 | { |
| 476 | PerfPoint point(PerfKey::WORKER_SERVER_BATCH_GET_REMOTE_HANDLER); | 472 | PerfPoint point(PerfKey::WORKER_SERVER_BATCH_GET_REMOTE_HANDLER); |
| 477 | const std::string &objectKey = req.object_key(); | 473 | const std::string &objectKey = req.object_key(); |
| @@ -485,7 +481,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteHandler(const GetObjectRemoteRe | |||
| 485 | INJECT_POINT("worker.worker_worker_remote_get_failure"); | 481 | INJECT_POINT("worker.worker_worker_remote_get_failure"); |
| 486 | } | 482 | } |
| 487 | Status status = GetObjectRemoteImpl(req, rsp, payload, blocking, eventKeys, batchPtr, batchRootInfo, | 483 | Status status = GetObjectRemoteImpl(req, rsp, payload, blocking, eventKeys, batchPtr, batchRootInfo, |
| 488 | - fallbackStatus, batchRh2dContext, isQueryAndGet); | 484 | + fallbackStatus, batchRh2dContext); |
| 489 | if (status.GetCode() == K_INVALID || status.GetCode() == K_NOT_FOUND) { | 485 | if (status.GetCode() == K_INVALID || status.GetCode() == K_NOT_FOUND) { |
| 490 | status = Status(K_WORKER_PULL_OBJECT_NOT_FOUND, status.GetMsg()); | 486 | status = Status(K_WORKER_PULL_OBJECT_NOT_FOUND, status.GetMsg()); |
| 491 | } | 487 | } |
| @@ -579,14 +575,14 @@ Status WorkerWorkerOCServiceImpl::LoadPayloadAndFillResponse( | |||
| 579 | const std::string &objectKey, uint64_t offset, uint64_t size, bool blocking, std::vector<uint64_t> &eventKeys, | 575 | const std::string &objectKey, uint64_t offset, uint64_t size, bool blocking, std::vector<uint64_t> &eventKeys, |
| 580 | const std::shared_ptr<AggregateMemory> &batchPtr, RemoteH2DRootInfoPb *batchRootInfo, | 576 | const std::shared_ptr<AggregateMemory> &batchPtr, RemoteH2DRootInfoPb *batchRootInfo, |
| 581 | BatchRh2dContext *batchRh2dContext, Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport, | 577 | BatchRh2dContext *batchRh2dContext, Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport, |
| 582 | - bool isPipelineH2DRequest, PerfPoint &batchImplPoint, bool isQueryAndGet) | 578 | + bool isPipelineH2DRequest, PerfPoint &batchImplPoint) |
| 583 | { | 579 | { |
| 584 | PerfPoint loadDataPoint(PerfKey::WORKER_LOAD_OBJECT_DATA); | 580 | PerfPoint loadDataPoint(PerfKey::WORKER_LOAD_OBJECT_DATA); |
| 585 | PerfPoint pointImpl(PerfKey::WORKER_REMOTE_GET_READ_KEY); | 581 | PerfPoint pointImpl(PerfKey::WORKER_REMOTE_GET_READ_KEY); |
| 586 | ReadObjectKV objKv(ReadKey(objectKey, offset, size), entry); | 582 | ReadObjectKV objKv(ReadKey(objectKey, offset, size), entry); |
| 587 | RETURN_IF_NOT_OK_PRINT_ERROR_MSG(objKv.CheckReadOffset(), "Read offset verify failed"); | 583 | RETURN_IF_NOT_OK_PRINT_ERROR_MSG(objKv.CheckReadOffset(), "Read offset verify failed"); |
| 588 | if (entry->IsSpilled() && entry->GetShmUnit() == nullptr) { | 584 | if (entry->IsSpilled() && entry->GetShmUnit() == nullptr) { |
| 589 | - RETURN_IF_NOT_OK(LoadSpilledObjectData(objectKey, outPayload, objKv, pointImpl, isQueryAndGet)); | 585 | + RETURN_IF_NOT_OK(LoadSpilledObjectData(objectKey, outPayload, objKv, pointImpl)); |
| 590 | } else { | 586 | } else { |
| 591 | pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_SHM_UNIT); | 587 | pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_SHM_UNIT); |
| 592 | ShmGuard shmGuard(entry->GetShmUnit(), entry->GetDataSize(), entry->GetMetadataSize()); | 588 | ShmGuard shmGuard(entry->GetShmUnit(), entry->GetDataSize(), entry->GetMetadataSize()); |
| @@ -616,16 +612,10 @@ Status WorkerWorkerOCServiceImpl::LoadPayloadAndFillResponse( | |||
| 616 | fastTransportName)); | 612 | fastTransportName)); |
| 617 | GetWorkerTimeCost().Append("RemoteWriteFastTransport", writeTimer.ElapsedMilliSecond()); | 613 | GetWorkerTimeCost().Append("RemoteWriteFastTransport", writeTimer.ElapsedMilliSecond()); |
| 618 | } | 614 | } |
| 619 | - if (isQueryAndGet && req.has_urma_info()) { | 615 | + RETURN_IF_NOT_OK(HandlePayloadFallback( |
| 620 | - CHECK_FAIL_RETURN_STATUS(isUrmaFastTransport, K_NOT_SUPPORTED, | 616 | + req, rsp, entry, outPayload, shmGuard, shmUnit, fastTransportStatus, fastTransportName, objectKey, |
| 621 | - "QueryAndGet UB transport is unavailable"); | 617 | + isFastTransportEnabled, isUrmaFastTransport, isPipelineH2DRequest, blocking, batchPtr, fallbackStatus, |
| 622 | - RETURN_IF_NOT_OK(fastTransportStatus); | 618 | + batchRootInfo, batchRh2dContext, objKv, localSegAddress, localSegSize)); |
| 623 | - } else { | ||
| 624 | - RETURN_IF_NOT_OK(HandlePayloadFallback( | ||
| 625 | - req, rsp, entry, outPayload, shmGuard, shmUnit, fastTransportStatus, fastTransportName, objectKey, | ||
| 626 | - isFastTransportEnabled, isUrmaFastTransport, isPipelineH2DRequest, blocking, batchPtr, fallbackStatus, | ||
| 627 | - batchRootInfo, batchRh2dContext, objKv, localSegAddress, localSegSize)); | ||
| 628 | - } | ||
| 629 | pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_RESP); | 619 | pointImpl.RecordAndReset(PerfKey::WORKER_REMOTE_GET_RESP); |
| 630 | } | 620 | } |
| 631 | 621 | ||
| @@ -635,12 +625,8 @@ Status WorkerWorkerOCServiceImpl::LoadPayloadAndFillResponse( | |||
| 635 | 625 | ||
| 636 | Status WorkerWorkerOCServiceImpl::LoadSpilledObjectData(const std::string &objectKey, | 626 | Status WorkerWorkerOCServiceImpl::LoadSpilledObjectData(const std::string &objectKey, |
| 637 | std::vector<RpcMessage> &outPayload, | 627 | std::vector<RpcMessage> &outPayload, |
| 638 | - const ReadObjectKV &objKv, PerfPoint &point, | 628 | + const ReadObjectKV &objKv, PerfPoint &point) |
| 639 | - bool isQueryAndGet) | ||
| 640 | { | 629 | { |
| 641 | - if (isQueryAndGet) { | ||
| 642 | - RETURN_STATUS(K_NOT_SUPPORTED, "QueryAndGet fast path only reads resident data"); | ||
| 643 | - } | ||
| 644 | point.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_FROM_DISK); | 630 | point.RecordAndReset(PerfKey::WORKER_REMOTE_GET_PAYLOAD_FROM_DISK); |
| 645 | RETURN_IF_NOT_OK( | 631 | RETURN_IF_NOT_OK( |
| 646 | WorkerOcSpill::Instance()->Get(objectKey, outPayload, objKv.GetReadSize(), objKv.GetReadOffset())); | 632 | WorkerOcSpill::Instance()->Get(objectKey, outPayload, objKv.GetReadSize(), objKv.GetReadOffset())); |
| @@ -892,7 +878,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteImpl(const GetObjectRemoteReqPb | |||
| 892 | std::vector<uint64_t> &eventKeys, | 878 | std::vector<uint64_t> &eventKeys, |
| 893 | std::shared_ptr<AggregateMemory> batchPtr, | 879 | std::shared_ptr<AggregateMemory> batchPtr, |
| 894 | RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus, | 880 | RemoteH2DRootInfoPb *batchRootInfo, Status *fallbackStatus, |
| 895 | - BatchRh2dContext *batchRh2dContext, bool isQueryAndGet) | 881 | + BatchRh2dContext *batchRh2dContext) |
| 896 | { | 882 | { |
| 897 | // Outer scope (BatchGetObjectRemote/GetObjectRemote) already created ScopedRequestContext. | 883 | // Outer scope (BatchGetObjectRemote/GetObjectRemote) already created ScopedRequestContext. |
| 898 | // No nested scope here: workerTimeCost is a value member not inherited by inner scopes. | 884 | // No nested scope here: workerTimeCost is a value member not inherited by inner scopes. |
| @@ -940,7 +926,7 @@ Status WorkerWorkerOCServiceImpl::GetObjectRemoteImpl(const GetObjectRemoteReqPb | |||
| 940 | 926 | ||
| 941 | return LoadPayloadAndFillResponse(req, rsp, entry, outPayload, objectKey, offset, size, blocking, eventKeys, | 927 | return LoadPayloadAndFillResponse(req, rsp, entry, outPayload, objectKey, offset, size, blocking, eventKeys, |
| 942 | batchPtr, batchRootInfo, batchRh2dContext, fallbackStatus, isFastTransportEnabled, | 928 | batchPtr, batchRootInfo, batchRh2dContext, fallbackStatus, isFastTransportEnabled, |
| 943 | - isUrmaFastTransport, isPipelineH2DRequest, batchImplPoint, isQueryAndGet); | 929 | + isUrmaFastTransport, isPipelineH2DRequest, batchImplPoint); |
| 944 | } | 930 | } |
| 945 | 931 | ||
| 946 | Status WorkerWorkerOCServiceImpl::CheckCoordinatorState(const CheckCoordinatorStateReqPb &req, | 932 | Status WorkerWorkerOCServiceImpl::CheckCoordinatorState(const CheckCoordinatorStateReqPb &req, |
| @@ -77,11 +77,9 @@ public: | |||
| 77 | * @param[in] req Remote get request. | 77 | * @param[in] req Remote get request. |
| 78 | * @param[out] rsp Remote get response. | 78 | * @param[out] rsp Remote get response. |
| 79 | * @param[out] payload Out payloads. | 79 | * @param[out] payload Out payloads. |
| 80 | - * @param[in] isQueryAndGet Whether to use the metadata-query resident-data fast path. | ||
| 81 | * @return Status of the call. | 80 | * @return Status of the call. |
| 82 | */ | 81 | */ |
| 83 | - Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, std::vector<RpcMessage> &payload, | 82 | + Status GetObjectRemote(GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, std::vector<RpcMessage> &payload); |
| 84 | - bool isQueryAndGet = false); | ||
| 85 | 83 | ||
| 86 | /** | 84 | /** |
| 87 | * @brief Check etcd state. | 85 | * @brief Check etcd state. |
| @@ -216,14 +214,13 @@ private: | |||
| 216 | * @param[out] keys The new request id to wait for if not blocking. | 214 | * @param[out] keys The new request id to wait for if not blocking. |
| 217 | * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path. | 215 | * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path. |
| 218 | * @param[in] batchRootInfo The common root info for batched requests. | 216 | * @param[in] batchRootInfo The common root info for batched requests. |
| 219 | - * @param[in] isQueryAndGet Whether this is a QueryAndGet fast-path attempt. | ||
| 220 | * @return Status of the call. | 217 | * @return Status of the call. |
| 221 | */ | 218 | */ |
| 222 | Status GetObjectRemoteImpl(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, | 219 | Status GetObjectRemoteImpl(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, |
| 223 | std::vector<RpcMessage> &outPayload, bool blocking, std::vector<uint64_t> &eventKeys, | 220 | std::vector<RpcMessage> &outPayload, bool blocking, std::vector<uint64_t> &eventKeys, |
| 224 | std::shared_ptr<AggregateMemory> batchPtr = nullptr, | 221 | std::shared_ptr<AggregateMemory> batchPtr = nullptr, |
| 225 | RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr, | 222 | RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr, |
| 226 | - BatchRh2dContext *batchRh2dContext = nullptr, bool isQueryAndGet = false); | 223 | + BatchRh2dContext *batchRh2dContext = nullptr); |
| 227 | 224 | ||
| 228 | Status LoadPayloadAndFillResponse(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, SafeObjType &entry, | 225 | Status LoadPayloadAndFillResponse(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, SafeObjType &entry, |
| 229 | std::vector<RpcMessage> &outPayload, const std::string &objectKey, | 226 | std::vector<RpcMessage> &outPayload, const std::string &objectKey, |
| @@ -231,7 +228,7 @@ private: | |||
| 231 | const std::shared_ptr<AggregateMemory> &batchPtr, | 228 | const std::shared_ptr<AggregateMemory> &batchPtr, |
| 232 | RemoteH2DRootInfoPb *batchRootInfo, BatchRh2dContext *batchRh2dContext, | 229 | RemoteH2DRootInfoPb *batchRootInfo, BatchRh2dContext *batchRh2dContext, |
| 233 | Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport, | 230 | Status *fallbackStatus, bool isFastTransportEnabled, bool isUrmaFastTransport, |
| 234 | - bool isPipelineH2DRequest, PerfPoint &batchImplPoint, bool isQueryAndGet); | 231 | + bool isPipelineH2DRequest, PerfPoint &batchImplPoint); |
| 235 | 232 | ||
| 236 | /** | 233 | /** |
| 237 | * @brief Load a spilled object for the regular remote-get path. | 234 | * @brief Load a spilled object for the regular remote-get path. |
| @@ -239,11 +236,10 @@ private: | |||
| 239 | * @param[out] outPayload Payload buffers loaded from spill storage. | 236 | * @param[out] outPayload Payload buffers loaded from spill storage. |
| 240 | * @param[in] objKv Object read range. | 237 | * @param[in] objKv Object read range. |
| 241 | * @param[in,out] point Remote-get performance point. | 238 | * @param[in,out] point Remote-get performance point. |
| 242 | - * @param[in] isQueryAndGet Whether this is a QueryAndGet fast-path attempt. | ||
| 243 | * @return K_OK on success; the error code otherwise. | 239 | * @return K_OK on success; the error code otherwise. |
| 244 | */ | 240 | */ |
| 245 | Status LoadSpilledObjectData(const std::string &objectKey, std::vector<RpcMessage> &outPayload, | 241 | Status LoadSpilledObjectData(const std::string &objectKey, std::vector<RpcMessage> &outPayload, |
| 246 | - const ReadObjectKV &objKv, PerfPoint &point, bool isQueryAndGet); | 242 | + const ReadObjectKV &objKv, PerfPoint &point); |
| 247 | 243 | ||
| 248 | /** | 244 | /** |
| 249 | * @brief Fill a successful remote-get response and finish its performance records. | 245 | * @brief Fill a successful remote-get response and finish its performance records. |
| @@ -387,14 +383,13 @@ private: | |||
| 387 | * @param[out] keys The request id to wait for if not blocking. | 383 | * @param[out] keys The request id to wait for if not blocking. |
| 388 | * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path. | 384 | * @param[in] batchPtr Batch ptr, default is nullptr means not in aggregate path. |
| 389 | * @param[in] batchRootInfo The common root info for batched requests. | 385 | * @param[in] batchRootInfo The common root info for batched requests. |
| 390 | - * @param[in] isQueryAndGet Whether this is a QueryAndGet fast-path attempt. | ||
| 391 | * @return Status of the call. | 386 | * @return Status of the call. |
| 392 | */ | 387 | */ |
| 393 | Status GetObjectRemoteHandler(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, | 388 | Status GetObjectRemoteHandler(const GetObjectRemoteReqPb &req, GetObjectRemoteRspPb &rsp, |
| 394 | std::vector<RpcMessage> &payload, bool blocking, std::vector<uint64_t> &eventKeys, | 389 | std::vector<RpcMessage> &payload, bool blocking, std::vector<uint64_t> &eventKeys, |
| 395 | std::shared_ptr<AggregateMemory> batchPtr = nullptr, | 390 | std::shared_ptr<AggregateMemory> batchPtr = nullptr, |
| 396 | RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr, | 391 | RemoteH2DRootInfoPb *batchRootInfo = nullptr, Status *fallbackStatus = nullptr, |
| 397 | - BatchRh2dContext *batchRh2dContext = nullptr, bool isQueryAndGet = false); | 392 | + BatchRh2dContext *batchRh2dContext = nullptr); |
| 398 | 393 | ||
| 399 | /** | 394 | /** |
| 400 | * @brief Complete a URMA warmup request when the target warmup object is not ready locally. | 395 | * @brief Complete a URMA warmup request when the target warmup object is not ready locally. |
| @@ -716,7 +716,7 @@ def test_query_and_get_has_time_worker_analysis_and_unclosed_failure_boundary(ru | |||
| 716 | "host_ip": "", | 716 | "host_ip": "", |
| 717 | "text": ( | 717 | "text": ( |
| 718 | f"{timestamp} [BRPC_RPC_FRAMEWORK_SLOW] " | 718 | f"{timestamp} [BRPC_RPC_FRAMEWORK_SLOW] " |
| 719 | - "method=datasystem.master.MasterOCService.QueryAndGet " | 719 | + "method=datasystem.WorkerOCService.QueryAndGet " |
| 720 | "e2e_us=20150 remote_processing_us=20150 server_req_queue_us=0 " | 720 | "e2e_us=20150 remote_processing_us=20150 server_req_queue_us=0 " |
| 721 | "server_exec_us=0 network_residual_us=0 cntl_error_code=1008 " | 721 | "server_exec_us=0 network_residual_us=0 cntl_error_code=1008 " |
| 722 | "cntl_failed=1 resp_attachment_bytes=0" | 722 | "cntl_failed=1 resp_attachment_bytes=0" |
| @@ -778,7 +778,7 @@ def test_query_and_get_has_time_worker_analysis_and_unclosed_failure_boundary(ru | |||
| 778 | assert "ObjectReadFlow::Resolve" in timeout_flow["confirmed_flow"] | 778 | assert "ObjectReadFlow::Resolve" in timeout_flow["confirmed_flow"] |
| 779 | assert "不能确认" in timeout_flow["root_cause_status"] | 779 | assert "不能确认" in timeout_flow["root_cause_status"] |
| 780 | query_events = [event for event in correlation["events"] if event["kind"] == "query_meta"] | 780 | query_events = [event for event in correlation["events"] if event["kind"] == "query_meta"] |
| 781 | - assert any(event["method"].endswith("MasterOCService.QueryAndGet") for event in query_events) | 781 | + assert any(event["method"].endswith("WorkerOCService.QueryAndGet") for event in query_events) |
| 782 | assert all(event["component_scope"] == "Client发起QueryMeta;Meta Owner目标未观测" for event in query_events) | 782 | assert all(event["component_scope"] == "Client发起QueryMeta;Meta Owner目标未观测" for event in query_events) |
| 783 | 783 | ||
| 784 | html_text = mod.render_html(analysis, "QueryMeta analysis") | 784 | html_text = mod.render_html(analysis, "QueryMeta analysis") |
| @@ -1430,7 +1430,7 @@ def test_failed_urma_wait_timeout_is_an_error_family_not_unsegmented_parent( | |||
| 1430 | "method, expected_subcategory, expected_chain, expected_scope", | 1430 | "method, expected_subcategory, expected_chain, expected_scope", |
| 1431 | [ | 1431 | [ |
| 1432 | ( | 1432 | ( |
| 1433 | - "datasystem.master.MasterOCService.QueryAndGet", | 1433 | + "datasystem.WorkerOCService.QueryAndGet", |
| 1434 | "QueryMeta RPC deadline", | 1434 | "QueryMeta RPC deadline", |
| 1435 | "QueryMeta RPC超时→TransportGet失败→1001", | 1435 | "QueryMeta RPC超时→TransportGet失败→1001", |
| 1436 | "Client等待Meta Owner QueryAndGet超时", | 1436 | "Client等待Meta Owner QueryAndGet超时", |
| @@ -1518,7 +1518,7 @@ def test_successful_query_and_get_does_not_mask_later_data_rpc_deadline(run_dir: | |||
| 1518 | "host_ip": "", | 1518 | "host_ip": "", |
| 1519 | "text": ( | 1519 | "text": ( |
| 1520 | "[BRPC_RPC_FRAMEWORK_SLOW] " | 1520 | "[BRPC_RPC_FRAMEWORK_SLOW] " |
| 1521 | - "method=datasystem.master.MasterOCService.QueryAndGet e2e_us=18400 " | 1521 | + "method=datasystem.WorkerOCService.QueryAndGet e2e_us=18400 " |
| 1522 | "server_exec_us=8 network_residual_us=18390 cntl_error_code=0 cntl_failed=0" | 1522 | "server_exec_us=8 network_residual_us=18390 cntl_error_code=0 cntl_failed=0" |
| 1523 | ), | 1523 | ), |
| 1524 | }, | 1524 | }, |
| @@ -1566,7 +1566,7 @@ def test_successful_query_and_get_does_not_mask_later_urma_connect_deadline(run_ | |||
| 1566 | "host_ip": "", | 1566 | "host_ip": "", |
| 1567 | "text": ( | 1567 | "text": ( |
| 1568 | "[BRPC_RPC_FRAMEWORK_SLOW] " | 1568 | "[BRPC_RPC_FRAMEWORK_SLOW] " |
| 1569 | - "method=datasystem.master.MasterOCService.QueryAndGet e2e_us=19400 " | 1569 | + "method=datasystem.WorkerOCService.QueryAndGet e2e_us=19400 " |
| 1570 | "server_exec_us=8 network_residual_us=19390 cntl_error_code=0 cntl_failed=0" | 1570 | "server_exec_us=8 network_residual_us=19390 cntl_error_code=0 cntl_failed=0" |
| 1571 | ), | 1571 | ), |
| 1572 | }, | 1572 | }, |
| @@ -1636,7 +1636,7 @@ def test_query_meta_detail_separates_retry_rpc_residual_and_inline_urma( | |||
| 1636 | "host_ip": "", | 1636 | "host_ip": "", |
| 1637 | "text": ( | 1637 | "text": ( |
| 1638 | "[BRPC_RPC_FRAMEWORK_SLOW] " | 1638 | "[BRPC_RPC_FRAMEWORK_SLOW] " |
| 1639 | - "method=datasystem.master.MasterOCService.QueryAndGet " | 1639 | + "method=datasystem.WorkerOCService.QueryAndGet " |
| 1640 | f"e2e_us={int(rpc_e2e_ms * 1000)} server_exec_us=5 " | 1640 | f"e2e_us={int(rpc_e2e_ms * 1000)} server_exec_us=5 " |
| 1641 | f"network_residual_us={int(rpc_network_ms * 1000)} " | 1641 | f"network_residual_us={int(rpc_network_ms * 1000)} " |
| 1642 | "cntl_error_code=0 cntl_failed=0" | 1642 | "cntl_error_code=0 cntl_failed=0" |
| @@ -52,12 +52,17 @@ namespace st { | |||
| 52 | namespace { | 52 | namespace { |
| 53 | constexpr uint32_t META_OWNER_INDEX = 0; | 53 | constexpr uint32_t META_OWNER_INDEX = 0; |
| 54 | constexpr uint32_t TRANSPORT_CLIENT_WORKER_INDEX = 1; | 54 | constexpr uint32_t TRANSPORT_CLIENT_WORKER_INDEX = 1; |
| 55 | +constexpr uint32_t DATA_WORKER_INDEX = 2; | ||
| 56 | +constexpr uint32_t WORKER_NUM = 3; | ||
| 55 | constexpr int32_t CLIENT_TIMEOUT_MS = 3'000; | 57 | constexpr int32_t CLIENT_TIMEOUT_MS = 3'000; |
| 56 | constexpr int32_t SHM_LATCH_TIMEOUT_MS = 1'000; | 58 | constexpr int32_t SHM_LATCH_TIMEOUT_MS = 1'000; |
| 57 | constexpr size_t VALUE_SIZE = 128 * 1024; | 59 | constexpr size_t VALUE_SIZE = 128 * 1024; |
| 60 | +constexpr size_t MIXED_OVERSIZED_INLINE_VALUE_SIZE = VALUE_SIZE + 1; | ||
| 58 | constexpr size_t INLINE_DATA_LIMIT = 512 * 1024; | 61 | constexpr size_t INLINE_DATA_LIMIT = 512 * 1024; |
| 59 | constexpr size_t LARGE_VALUE_SIZE = 8 * 1024 * 1024; | 62 | constexpr size_t LARGE_VALUE_SIZE = 8 * 1024 * 1024; |
| 60 | constexpr size_t KEY_SEARCH_LIMIT = 100'000; | 63 | constexpr size_t KEY_SEARCH_LIMIT = 100'000; |
| 64 | +constexpr uint64_t MIXED_TCP_DATA_RPC_COUNT = 1; | ||
| 65 | +constexpr uint64_t MIXED_UB_DATA_RPC_COUNT = 2; | ||
| 61 | constexpr char REAL_ROUTE_KEY_PREFIX[] = "transport_real_route_"; | 66 | constexpr char REAL_ROUTE_KEY_PREFIX[] = "transport_real_route_"; |
| 62 | constexpr char UB_GET_SIZE_ENV[] = "DATASYSTEM_UB_GET_DATA_SIZE_BYTES"; | 67 | constexpr char UB_GET_SIZE_ENV[] = "DATASYSTEM_UB_GET_DATA_SIZE_BYTES"; |
| 63 | constexpr char SKIP_WARMUP_INJECT[] = "ObjectClientImpl.ClientWorkerWarmup.skip"; | 68 | constexpr char SKIP_WARMUP_INJECT[] = "ObjectClientImpl.ClientWorkerWarmup.skip"; |
| @@ -70,8 +75,15 @@ constexpr char REGISTER_SHM_CLIENT_INJECT[] = "client.transport.register_shm_cli | |||
| 70 | constexpr char GET_CLIENT_FD_INJECT[] = "client.transport.get_client_fd"; | 75 | constexpr char GET_CLIENT_FD_INJECT[] = "client.transport.get_client_fd"; |
| 71 | constexpr char SHM_HEARTBEAT_INJECT[] = "client.transport.shm_heartbeat"; | 76 | constexpr char SHM_HEARTBEAT_INJECT[] = "client.transport.shm_heartbeat"; |
| 72 | constexpr char DRAIN_BEFORE_SNAPSHOT_INJECT[] = "WorkerOCServiceImpl.DrainTopologyScaleInData.beforeSnapshot"; | 77 | constexpr char DRAIN_BEFORE_SNAPSHOT_INJECT[] = "WorkerOCServiceImpl.DrainTopologyScaleInData.beforeSnapshot"; |
| 73 | -constexpr char HASH_RING_REFRESH_BEFORE_WAIT_INJECT[] = "HashRingRefresher.RefreshLoop.beforeWait"; | 78 | +constexpr char QUERY_AND_GET_TCP_HIT_INJECT[] = "worker.QueryAndGet.EncodeTcp"; |
| 74 | -constexpr char INLINE_READ_FAILURE_INJECT[] = "worker.worker_worker_remote_get_failure"; | 79 | +constexpr char QUERY_AND_GET_UB_HIT_INJECT[] = "worker.QueryAndGet.EncodeUb"; |
| 80 | +constexpr char QUERY_AND_GET_SHM_HIT_INJECT[] = "worker.QueryAndGet.EncodeShm"; | ||
| 81 | +constexpr char QUERY_AND_GET_METADATA_MISS_INJECT[] = "worker.QueryAndGet.QueryMissMetadata"; | ||
| 82 | +constexpr char QUERY_AND_GET_INLINE_FAILURE_INJECT[] = "worker.QueryAndGet.EncodeLocalHitFailure"; | ||
| 83 | +constexpr char SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT[] = | ||
| 84 | + "client.transport.query_and_get.shm_session_unavailable_before_build"; | ||
| 85 | +constexpr char SHM_MATERIALIZATION_FAILURE_INJECT[] = | ||
| 86 | + "client.transport.query_and_get.shm_materialization_failure"; | ||
| 75 | constexpr char SHM_LATCH_FAIL_INJECT[] = "worker.ShmGuard.TryRLatch.Fail"; | 87 | constexpr char SHM_LATCH_FAIL_INJECT[] = "worker.ShmGuard.TryRLatch.Fail"; |
| 76 | constexpr char PROVIDER_GET_ENTER_INJECT[] = "worker.GetObjectRemote.afterRead"; | 88 | constexpr char PROVIDER_GET_ENTER_INJECT[] = "worker.GetObjectRemote.afterRead"; |
| 77 | constexpr char PROVIDER_BATCH_GET_ENTER_INJECT[] = "worker.BatchGetObjectRemote.afterRead"; | 89 | constexpr char PROVIDER_BATCH_GET_ENTER_INJECT[] = "worker.BatchGetObjectRemote.afterRead"; |
| @@ -87,9 +99,8 @@ constexpr char GLOBAL_UNAVAILABLE_APPLIED_INJECT[] = "client.ub_health_filter.gl | |||
| 87 | constexpr char GLOBAL_READ_DENIED_INJECT[] = "client.ub_health_filter.global_read_denied"; | 99 | constexpr char GLOBAL_READ_DENIED_INJECT[] = "client.ub_health_filter.global_read_denied"; |
| 88 | constexpr char SHM_HOST_ID_ENV_NAME[] = "transport_get_shm_host_id"; | 100 | constexpr char SHM_HOST_ID_ENV_NAME[] = "transport_get_shm_host_id"; |
| 89 | constexpr char SHM_HOST_ID_VALUE[] = "transport-get-shm-host"; | 101 | constexpr char SHM_HOST_ID_VALUE[] = "transport-get-shm-host"; |
| 90 | -constexpr uint32_t SCALE_OUT_WORKER_INDEX = 3; | 102 | +constexpr char MIXED_HOST_ID_ENV_PREFIX[] = "transport_get_mixed_host_id_"; |
| 91 | -constexpr uint32_t SCALE_OUT_WORKER_COUNT = 4; | 103 | +constexpr char MIXED_HOST_ID_VALUE_PREFIX[] = "transport-get-mixed-host-"; |
| 92 | -constexpr size_t SCALE_OUT_KEY_COUNT = 256; | ||
| 93 | 104 | ||
| 94 | struct TransportRpcCounts { | 105 | struct TransportRpcCounts { |
| 95 | uint64_t queryAndGet = 0; | 106 | uint64_t queryAndGet = 0; |
| @@ -101,28 +112,41 @@ struct TransportRpcCounts { | |||
| 101 | uint64_t shmHeartbeat = 0; | 112 | uint64_t shmHeartbeat = 0; |
| 102 | }; | 113 | }; |
| 103 | 114 | ||
| 104 | -const char *ExpectedTransport() | 115 | +struct WorkerQueryAndGetCounts { |
| 116 | + uint64_t tcpHits = 0; | ||
| 117 | + uint64_t ubHits = 0; | ||
| 118 | + uint64_t shmHits = 0; | ||
| 119 | + uint64_t metadataMisses = 0; | ||
| 120 | +}; | ||
| 121 | + | ||
| 122 | +struct MixedPathCounts { | ||
| 123 | + TransportRpcCounts rpc; | ||
| 124 | + WorkerQueryAndGetCounts localOwner; | ||
| 125 | + WorkerQueryAndGetCounts remoteOwner; | ||
| 126 | +}; | ||
| 127 | + | ||
| 128 | +constexpr bool IsUrmaBuild() | ||
| 105 | { | 129 | { |
| 106 | 130 | ||
| 107 | - return "UB"; | 131 | + return true; |
| 108 | 132 | ||
| 109 | - return "TCP"; | 133 | + return false; |
| 110 | 134 | ||
| 111 | } | 135 | } |
| 112 | 136 | ||
| 113 | -std::string SelectMetadataOwner(const ClusterTopologyPb &topology, const std::string &key) | 137 | +const char *ExpectedTransport() |
| 114 | { | 138 | { |
| 115 | - std::map<uint32_t, std::string> tokenWorkers; | 139 | + return IsUrmaBuild() ? "UB" : "TCP"; |
| 116 | - for (const auto &worker : topology.members()) { | 140 | +} |
| 117 | - for (const auto token : worker.second.tokens()) { | 141 | + |
| 118 | - tokenWorkers.emplace(token, worker.first); | 142 | +std::string MixedHostIdEnvName(uint32_t workerIndex) |
| 119 | - } | 143 | +{ |
| 120 | - } | 144 | + return std::string(MIXED_HOST_ID_ENV_PREFIX) + std::to_string(workerIndex); |
| 121 | - if (tokenWorkers.empty()) { | 145 | +} |
| 122 | - return {}; | 146 | + |
| 123 | - } | 147 | +std::string MixedHostIdValue(uint32_t workerIndex) |
| 124 | - auto owner = tokenWorkers.upper_bound(MurmurHash3_32(key)); | 148 | +{ |
| 125 | - return (owner == tokenWorkers.end() ? tokenWorkers.begin() : owner)->second; | 149 | + return std::string(MIXED_HOST_ID_VALUE_PREFIX) + std::to_string(workerIndex); |
| 126 | } | 150 | } |
| 127 | } // namespace | 151 | } // namespace |
| 128 | 152 | ||
| @@ -132,7 +156,7 @@ public: | |||
| 132 | { | 156 | { |
| 133 | FLAGS_v = 1; | 157 | FLAGS_v = 1; |
| 134 | opts.numEtcd = 1; | 158 | opts.numEtcd = 1; |
| 135 | - opts.numWorkers = 3; | 159 | + opts.numWorkers = WORKER_NUM; |
| 136 | opts.enableDistributedMaster = "true"; | 160 | opts.enableDistributedMaster = "true"; |
| 137 | opts.workerGflagParams = | 161 | opts.workerGflagParams = |
| 138 | " -shared_memory_size_mb=512 -ipc_through_shared_memory=false -arena_per_tenant=1"; | 162 | " -shared_memory_size_mb=512 -ipc_through_shared_memory=false -arena_per_tenant=1"; |
| @@ -142,6 +166,10 @@ public: | |||
| 142 | opts.workerGflagParams += " -enable_urma=false"; | 166 | opts.workerGflagParams += " -enable_urma=false"; |
| 143 | 167 | ||
| 144 | opts.injectActions = "worker.batch_get_failure_for_keys:call()"; | 168 | opts.injectActions = "worker.batch_get_failure_for_keys:call()"; |
| 169 | + opts.injectActions += ";" + std::string(QUERY_AND_GET_TCP_HIT_INJECT) + ":call()"; | ||
| 170 | + opts.injectActions += ";" + std::string(QUERY_AND_GET_UB_HIT_INJECT) + ":call()"; | ||
| 171 | + opts.injectActions += ";" + std::string(QUERY_AND_GET_SHM_HIT_INJECT) + ":call()"; | ||
| 172 | + opts.injectActions += ";" + std::string(QUERY_AND_GET_METADATA_MISS_INJECT) + ":call()"; | ||
| 145 | } | 173 | } |
| 146 | 174 | ||
| 147 | void SetUp() override | 175 | void SetUp() override |
| @@ -166,7 +194,7 @@ public: | |||
| 166 | ASSERT_NE(etcd_, nullptr); | 194 | ASSERT_NE(etcd_, nullptr); |
| 167 | InitTestKVClient(META_OWNER_INDEX, writer_, CLIENT_TIMEOUT_MS); | 195 | InitTestKVClient(META_OWNER_INDEX, writer_, CLIENT_TIMEOUT_MS); |
| 168 | 196 | ||
| 169 | - SetUbGetSize(INLINE_DATA_LIMIT); | 197 | + SetUbGetSize(UbInlineBufferSize()); |
| 170 | 198 | ||
| 171 | InitTransportClient(); | 199 | InitTransportClient(); |
| 172 | } | 200 | } |
| @@ -205,12 +233,23 @@ protected: | |||
| 205 | void InitTransportClient() | 233 | void InitTransportClient() |
| 206 | { | 234 | { |
| 207 | ConnectOptions options; | 235 | ConnectOptions options; |
| 208 | - InitConnectOpt(TRANSPORT_CLIENT_WORKER_INDEX, options, CLIENT_TIMEOUT_MS); | 236 | + InitConnectOpt(TransportClientWorkerIndex(), options, CLIENT_TIMEOUT_MS); |
| 209 | options.enableLocalCache = false; | 237 | options.enableLocalCache = false; |
| 238 | + options.dataPlacementPolicy = DataPlacementPolicy::PREFERRED_META_OWNER; | ||
| 210 | reader_ = std::make_shared<KVClient>(options); | 239 | reader_ = std::make_shared<KVClient>(options); |
| 211 | DS_ASSERT_OK(reader_->Init()); | 240 | DS_ASSERT_OK(reader_->Init()); |
| 212 | } | 241 | } |
| 213 | 242 | ||
| 243 | + virtual uint32_t TransportClientWorkerIndex() const | ||
| 244 | + { | ||
| 245 | + return TRANSPORT_CLIENT_WORKER_INDEX; | ||
| 246 | + } | ||
| 247 | + | ||
| 248 | + virtual size_t UbInlineBufferSize() const | ||
| 249 | + { | ||
| 250 | + return INLINE_DATA_LIMIT; | ||
| 251 | + } | ||
| 252 | + | ||
| 214 | void SetUbGetSize(size_t size) | 253 | void SetUbGetSize(size_t size) |
| 215 | { | 254 | { |
| 216 | ASSERT_EQ(setenv(UB_GET_SIZE_ENV, std::to_string(size).c_str(), 1), 0); | 255 | ASSERT_EQ(setenv(UB_GET_SIZE_ENV, std::to_string(size).c_str(), 1), 0); |
| @@ -270,6 +309,18 @@ protected: | |||
| 270 | counts.shmHeartbeat = inject::GetExecuteCount(SHM_HEARTBEAT_INJECT); | 309 | counts.shmHeartbeat = inject::GetExecuteCount(SHM_HEARTBEAT_INJECT); |
| 271 | } | 310 | } |
| 272 | 311 | ||
| 312 | + void GetWorkerQueryAndGetCounts(uint32_t workerIndex, WorkerQueryAndGetCounts &counts) | ||
| 313 | + { | ||
| 314 | + DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount( | ||
| 315 | + WORKER, workerIndex, QUERY_AND_GET_TCP_HIT_INJECT, counts.tcpHits)); | ||
| 316 | + DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount( | ||
| 317 | + WORKER, workerIndex, QUERY_AND_GET_UB_HIT_INJECT, counts.ubHits)); | ||
| 318 | + DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount( | ||
| 319 | + WORKER, workerIndex, QUERY_AND_GET_SHM_HIT_INJECT, counts.shmHits)); | ||
| 320 | + DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount( | ||
| 321 | + WORKER, workerIndex, QUERY_AND_GET_METADATA_MISS_INJECT, counts.metadataMisses)); | ||
| 322 | + } | ||
| 323 | + | ||
| 273 | // Generate N distinct keys without making placement assumptions. | 324 | // Generate N distinct keys without making placement assumptions. |
| 274 | std::vector<std::string> MakeRandomKeys(size_t count) | 325 | std::vector<std::string> MakeRandomKeys(size_t count) |
| 275 | { | 326 | { |
| @@ -550,40 +601,6 @@ private: | |||
| 550 | bool setupAttempted_ = false; | 601 | bool setupAttempted_ = false; |
| 551 | }; | 602 | }; |
| 552 | 603 | ||
| 553 | -class KVClientTransportGetScaleOutRealUrmaTest : public KVClientTransportGetTest { | ||
| 554 | -public: | ||
| 555 | - void SetClusterSetupOptions(ExternalClusterOptions &opts) override | ||
| 556 | - { | ||
| 557 | - KVClientTransportGetTest::SetClusterSetupOptions(opts); | ||
| 558 | - opts.workerGflagParams += " -enable_lossless_data_exit_mode=true"; | ||
| 559 | - } | ||
| 560 | - | ||
| 561 | - void SetUp() override | ||
| 562 | - { | ||
| 563 | - | ||
| 564 | - GTEST_SKIP() << "Real URMA scale-out redirect ST requires USE_URMA."; | ||
| 565 | - | ||
| 566 | - GTEST_SKIP() << "Real URMA scale-out redirect ST does not run with USE_URMA_MOCK."; | ||
| 567 | - | ||
| 568 | - if (std::getenv("DS_URMA_DEV_NAME") == nullptr) { | ||
| 569 | - GTEST_SKIP() << "Real URMA scale-out redirect ST requires DS_URMA_DEV_NAME and a usable URMA device."; | ||
| 570 | - } | ||
| 571 | - setupAttempted_ = true; | ||
| 572 | - KVClientTransportGetTest::SetUp(); | ||
| 573 | - | ||
| 574 | - } | ||
| 575 | - | ||
| 576 | - void TearDown() override | ||
| 577 | - { | ||
| 578 | - if (setupAttempted_) { | ||
| 579 | - KVClientTransportGetTest::TearDown(); | ||
| 580 | - } | ||
| 581 | - } | ||
| 582 | - | ||
| 583 | -private: | ||
| 584 | - bool setupAttempted_ = false; | ||
| 585 | -}; | ||
| 586 | - | ||
| 587 | class KVClientTransportGetWithTargetShmDisabledTest : public KVClientTransportGetTest { | 604 | class KVClientTransportGetWithTargetShmDisabledTest : public KVClientTransportGetTest { |
| 588 | public: | 605 | public: |
| 589 | void SetClusterSetupOptions(ExternalClusterOptions &opts) override | 606 | void SetClusterSetupOptions(ExternalClusterOptions &opts) override |
| @@ -709,6 +726,336 @@ protected: | |||
| 709 | } | 726 | } |
| 710 | }; | 727 | }; |
| 711 | 728 | ||
| 729 | +class KVClientTransportGetMixedPathTest : public KVClientTransportGetTest { | ||
| 730 | +public: | ||
| 731 | + void SetClusterSetupOptions(ExternalClusterOptions &opts) override | ||
| 732 | + { | ||
| 733 | + KVClientTransportGetTest::SetClusterSetupOptions(opts); | ||
| 734 | + constexpr char DISABLED_SHM_OPTION[] = "-ipc_through_shared_memory=false"; | ||
| 735 | + const auto pos = opts.workerGflagParams.find(DISABLED_SHM_OPTION); | ||
| 736 | + ASSERT_NE(pos, std::string::npos); | ||
| 737 | + opts.workerGflagParams.replace(pos, sizeof(DISABLED_SHM_OPTION) - 1, "-ipc_through_shared_memory=true"); | ||
| 738 | + for (uint32_t i = 0; i < WORKER_NUM; ++i) { | ||
| 739 | + opts.workerSpecifyGflagParams[i] += " -host_id_env_name=" + MixedHostIdEnvName(i); | ||
| 740 | + } | ||
| 741 | + } | ||
| 742 | + | ||
| 743 | + void SetUp() override | ||
| 744 | + { | ||
| 745 | + for (uint32_t i = 0; i < WORKER_NUM; ++i) { | ||
| 746 | + ASSERT_EQ(setenv(MixedHostIdEnvName(i).c_str(), MixedHostIdValue(i).c_str(), 1), 0); | ||
| 747 | + } | ||
| 748 | + KVClientTransportGetTest::SetUp(); | ||
| 749 | + } | ||
| 750 | + | ||
| 751 | + void TearDown() override | ||
| 752 | + { | ||
| 753 | + KVClientTransportGetTest::TearDown(); | ||
| 754 | + for (uint32_t i = 0; i < WORKER_NUM; ++i) { | ||
| 755 | + (void)unsetenv(MixedHostIdEnvName(i).c_str()); | ||
| 756 | + } | ||
| 757 | + } | ||
| 758 | + | ||
| 759 | +protected: | ||
| 760 | + uint32_t TransportClientWorkerIndex() const override | ||
| 761 | + { | ||
| 762 | + return META_OWNER_INDEX; | ||
| 763 | + } | ||
| 764 | + | ||
| 765 | + size_t UbInlineBufferSize() const override | ||
| 766 | + { | ||
| 767 | + return VALUE_SIZE; | ||
| 768 | + } | ||
| 769 | + | ||
| 770 | + void AssertMixedPathBuffers(std::vector<Optional<Buffer>> &buffers, | ||
| 771 | + const std::vector<std::string> &values) | ||
| 772 | + { | ||
| 773 | + ASSERT_EQ(buffers.size(), 4u); | ||
| 774 | + ASSERT_FALSE(buffers[0]); | ||
| 775 | + for (size_t i = 1; i < buffers.size(); ++i) { | ||
| 776 | + ASSERT_TRUE(buffers[i]); | ||
| 777 | + } | ||
| 778 | + AssertBufferEqual(*buffers[1], values[1]); | ||
| 779 | + AssertBufferEqual(*buffers[2], values[2]); | ||
| 780 | + AssertBufferEqual(*buffers[3], values[0]); | ||
| 781 | + } | ||
| 782 | + | ||
| 783 | + void GetMixedPathCounts(MixedPathCounts &counts) | ||
| 784 | + { | ||
| 785 | + GetRpcCounts(counts.rpc); | ||
| 786 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, counts.localOwner); | ||
| 787 | + GetWorkerQueryAndGetCounts(TRANSPORT_CLIENT_WORKER_INDEX, counts.remoteOwner); | ||
| 788 | + } | ||
| 789 | + | ||
| 790 | + void AssertMixedPathCounts(const MixedPathCounts &before, const MixedPathCounts &after) | ||
| 791 | + { | ||
| 792 | + ASSERT_EQ(after.rpc.queryAndGet, before.rpc.queryAndGet + 2); | ||
| 793 | + const uint64_t expectedDataRpcCount = | ||
| 794 | + IsUrmaBuild() ? MIXED_UB_DATA_RPC_COUNT : MIXED_TCP_DATA_RPC_COUNT; | ||
| 795 | + ASSERT_EQ(after.rpc.getObjectRemote, before.rpc.getObjectRemote + expectedDataRpcCount); | ||
| 796 | + ASSERT_EQ(after.localOwner.shmHits, before.localOwner.shmHits + 1); | ||
| 797 | + ASSERT_EQ(after.localOwner.metadataMisses, before.localOwner.metadataMisses + 1); | ||
| 798 | + ASSERT_EQ(after.remoteOwner.tcpHits, before.remoteOwner.tcpHits + (IsUrmaBuild() ? 0 : 1)); | ||
| 799 | + ASSERT_EQ(after.remoteOwner.ubHits, before.remoteOwner.ubHits); | ||
| 800 | + ASSERT_EQ(after.remoteOwner.metadataMisses, before.remoteOwner.metadataMisses + 1); | ||
| 801 | + } | ||
| 802 | +}; | ||
| 803 | + | ||
| 804 | +TEST_F(KVClientTransportGetWithAllWorkersShmTest, SameNodeMetadataOwnerHitUsesShmInline) | ||
| 805 | +{ | ||
| 806 | + std::vector<std::string> keys; | ||
| 807 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 2, keys); | ||
| 808 | + ASSERT_EQ(keys.size(), 2u); | ||
| 809 | + const std::string value(VALUE_SIZE, 'a'); | ||
| 810 | + DS_ASSERT_OK(writer_->Set(keys[0], value)); | ||
| 811 | + DS_ASSERT_OK(writer_->Set(keys[1], value)); | ||
| 812 | + | ||
| 813 | + Optional<Buffer> warmup; | ||
| 814 | + DS_ASSERT_OK(reader_->Get(keys[0], warmup)); | ||
| 815 | + ASSERT_TRUE(warmup); | ||
| 816 | + ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); | ||
| 817 | + | ||
| 818 | + TransportRpcCounts rpcBefore; | ||
| 819 | + WorkerQueryAndGetCounts workerBefore; | ||
| 820 | + GetRpcCounts(rpcBefore); | ||
| 821 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerBefore); | ||
| 822 | + Optional<Buffer> buffer; | ||
| 823 | + DS_ASSERT_OK(reader_->Get(keys[1], buffer)); | ||
| 824 | + TransportRpcCounts rpcAfter; | ||
| 825 | + WorkerQueryAndGetCounts workerAfter; | ||
| 826 | + GetRpcCounts(rpcAfter); | ||
| 827 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerAfter); | ||
| 828 | + | ||
| 829 | + ASSERT_TRUE(buffer); | ||
| 830 | + AssertBufferEqual(*buffer, value); | ||
| 831 | + ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); | ||
| 832 | + ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1); | ||
| 833 | + ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote); | ||
| 834 | + ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet); | ||
| 835 | + ASSERT_EQ(rpcAfter.registerShmClient, rpcBefore.registerShmClient); | ||
| 836 | + ASSERT_EQ(rpcAfter.getClientFd, rpcBefore.getClientFd); | ||
| 837 | + ASSERT_EQ(workerAfter.tcpHits, workerBefore.tcpHits); | ||
| 838 | + ASSERT_EQ(workerAfter.ubHits, workerBefore.ubHits); | ||
| 839 | + ASSERT_EQ(workerAfter.shmHits, workerBefore.shmHits + 1); | ||
| 840 | + ASSERT_EQ(workerAfter.metadataMisses, workerBefore.metadataMisses); | ||
| 841 | +} | ||
| 842 | + | ||
| 843 | +TEST_F(KVClientTransportGetWithAllWorkersShmTest, UnavailableShmSessionFallsBackBeforeDispatch) | ||
| 844 | +{ | ||
| 845 | + std::vector<std::string> keys; | ||
| 846 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys); | ||
| 847 | + const std::string value(VALUE_SIZE, 's'); | ||
| 848 | + DS_ASSERT_OK(writer_->Set(keys.front(), value)); | ||
| 849 | + const uint64_t injectCount = inject::GetExecuteCount(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT); | ||
| 850 | + DS_ASSERT_OK(inject::Set(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT, "1*call()")); | ||
| 851 | + Raii clearInject([] { (void)inject::Clear(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT); }); | ||
| 852 | + | ||
| 853 | + Optional<Buffer> buffer; | ||
| 854 | + DS_ASSERT_OK(reader_->Get(keys.front(), buffer)); | ||
| 855 | + | ||
| 856 | + ASSERT_TRUE(buffer); | ||
| 857 | + AssertBufferEqual(*buffer, value); | ||
| 858 | + ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport()); | ||
| 859 | + ASSERT_EQ(inject::GetExecuteCount(SHM_SESSION_UNAVAILABLE_BEFORE_BUILD_INJECT), injectCount + 1); | ||
| 860 | +} | ||
| 861 | + | ||
| 862 | +TEST_F(KVClientTransportGetWithAllWorkersShmTest, ShmMaterializationFailureFallsBackPerKey) | ||
| 863 | +{ | ||
| 864 | + std::vector<std::string> keys; | ||
| 865 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys); | ||
| 866 | + const std::string value(VALUE_SIZE, 'm'); | ||
| 867 | + DS_ASSERT_OK(writer_->Set(keys.front(), value)); | ||
| 868 | + DS_ASSERT_OK(inject::Set(SHM_MATERIALIZATION_FAILURE_INJECT, "1*return(K_RUNTIME_ERROR)")); | ||
| 869 | + Raii clearInject([] { (void)inject::Clear(SHM_MATERIALIZATION_FAILURE_INJECT); }); | ||
| 870 | + | ||
| 871 | + Optional<Buffer> buffer; | ||
| 872 | + DS_ASSERT_OK(reader_->Get(keys.front(), buffer)); | ||
| 873 | + | ||
| 874 | + ASSERT_TRUE(buffer); | ||
| 875 | + AssertBufferEqual(*buffer, value); | ||
| 876 | + ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); | ||
| 877 | +} | ||
| 878 | + | ||
| 879 | +TEST_F(KVClientTransportGetTest, CrossNodeMetadataOwnerHitUsesUbInline) | ||
| 880 | +{ | ||
| 881 | + | ||
| 882 | + GTEST_SKIP() << "QueryAndGet UB inline ST requires USE_URMA."; | ||
| 883 | + | ||
| 884 | + std::vector<std::string> keys; | ||
| 885 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys); | ||
| 886 | + ASSERT_EQ(keys.size(), 1u); | ||
| 887 | + const std::string value(VALUE_SIZE, 'b'); | ||
| 888 | + DS_ASSERT_OK(writer_->Set(keys[0], value)); | ||
| 889 | + TransportRpcCounts rpcBefore; | ||
| 890 | + WorkerQueryAndGetCounts workerBefore; | ||
| 891 | + GetRpcCounts(rpcBefore); | ||
| 892 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerBefore); | ||
| 893 | + | ||
| 894 | + Optional<Buffer> buffer; | ||
| 895 | + DS_ASSERT_OK(reader_->Get(keys[0], buffer)); | ||
| 896 | + TransportRpcCounts rpcAfter; | ||
| 897 | + WorkerQueryAndGetCounts workerAfter; | ||
| 898 | + GetRpcCounts(rpcAfter); | ||
| 899 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerAfter); | ||
| 900 | + | ||
| 901 | + ASSERT_TRUE(buffer); | ||
| 902 | + AssertBufferEqual(*buffer, value); | ||
| 903 | + ASSERT_EQ(AccessTransportTracker::ToString(), "UB"); | ||
| 904 | + ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1); | ||
| 905 | + ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote); | ||
| 906 | + ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet); | ||
| 907 | + ASSERT_EQ(workerAfter.tcpHits, workerBefore.tcpHits); | ||
| 908 | + ASSERT_EQ(workerAfter.ubHits, workerBefore.ubHits + 1); | ||
| 909 | + ASSERT_EQ(workerAfter.shmHits, workerBefore.shmHits); | ||
| 910 | + ASSERT_EQ(workerAfter.metadataMisses, workerBefore.metadataMisses); | ||
| 911 | + | ||
| 912 | +} | ||
| 913 | + | ||
| 914 | +TEST_F(KVClientTransportGetTest, CrossNodeMetadataOwnerHitUsesTcpInline) | ||
| 915 | +{ | ||
| 916 | + | ||
| 917 | + GTEST_SKIP() << "QueryAndGet TCP-only ST requires a non-URMA build."; | ||
| 918 | + | ||
| 919 | + std::vector<std::string> keys; | ||
| 920 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys); | ||
| 921 | + ASSERT_EQ(keys.size(), 1u); | ||
| 922 | + const std::string value(VALUE_SIZE, 'c'); | ||
| 923 | + DS_ASSERT_OK(writer_->Set(keys[0], value)); | ||
| 924 | + TransportRpcCounts rpcBefore; | ||
| 925 | + WorkerQueryAndGetCounts workerBefore; | ||
| 926 | + GetRpcCounts(rpcBefore); | ||
| 927 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerBefore); | ||
| 928 | + | ||
| 929 | + Optional<Buffer> buffer; | ||
| 930 | + DS_ASSERT_OK(reader_->Get(keys[0], buffer)); | ||
| 931 | + TransportRpcCounts rpcAfter; | ||
| 932 | + WorkerQueryAndGetCounts workerAfter; | ||
| 933 | + GetRpcCounts(rpcAfter); | ||
| 934 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, workerAfter); | ||
| 935 | + | ||
| 936 | + ASSERT_TRUE(buffer); | ||
| 937 | + AssertBufferEqual(*buffer, value); | ||
| 938 | + ASSERT_EQ(AccessTransportTracker::ToString(), "TCP"); | ||
| 939 | + ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1); | ||
| 940 | + ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote); | ||
| 941 | + ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet); | ||
| 942 | + ASSERT_EQ(workerAfter.tcpHits, workerBefore.tcpHits + 1); | ||
| 943 | + ASSERT_EQ(workerAfter.ubHits, workerBefore.ubHits); | ||
| 944 | + ASSERT_EQ(workerAfter.shmHits, workerBefore.shmHits); | ||
| 945 | + ASSERT_EQ(workerAfter.metadataMisses, workerBefore.metadataMisses); | ||
| 946 | + | ||
| 947 | +} | ||
| 948 | + | ||
| 949 | +TEST_F(KVClientTransportGetWithAllWorkersShmTest, MetadataMissReadsSameNodeDataWorkerWithShm) | ||
| 950 | +{ | ||
| 951 | + std::vector<std::string> ownerKeys; | ||
| 952 | + std::vector<std::string> dataWorkerKeys; | ||
| 953 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 2, ownerKeys); | ||
| 954 | + GetRealHashKeysToWorker(DATA_WORKER_INDEX, 1, dataWorkerKeys); | ||
| 955 | + ASSERT_EQ(ownerKeys.size(), 2u); | ||
| 956 | + ASSERT_EQ(dataWorkerKeys.size(), 1u); | ||
| 957 | + const std::string value(VALUE_SIZE, 'd'); | ||
| 958 | + std::shared_ptr<KVClient> dataWriter; | ||
| 959 | + InitTestKVClient(DATA_WORKER_INDEX, dataWriter, CLIENT_TIMEOUT_MS); | ||
| 960 | + DS_ASSERT_OK(writer_->Set(ownerKeys[0], value)); | ||
| 961 | + DS_ASSERT_OK(dataWriter->Set(dataWorkerKeys[0], value)); | ||
| 962 | + DS_ASSERT_OK(dataWriter->Set(ownerKeys[1], value)); | ||
| 963 | + | ||
| 964 | + Optional<Buffer> warmup; | ||
| 965 | + DS_ASSERT_OK(reader_->Get(ownerKeys[0], warmup)); | ||
| 966 | + DS_ASSERT_OK(reader_->Get(dataWorkerKeys[0], warmup)); | ||
| 967 | + TransportRpcCounts rpcBefore; | ||
| 968 | + WorkerQueryAndGetCounts ownerBefore; | ||
| 969 | + GetRpcCounts(rpcBefore); | ||
| 970 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerBefore); | ||
| 971 | + | ||
| 972 | + Optional<Buffer> buffer; | ||
| 973 | + DS_ASSERT_OK(reader_->Get(ownerKeys[1], buffer)); | ||
| 974 | + TransportRpcCounts rpcAfter; | ||
| 975 | + WorkerQueryAndGetCounts ownerAfter; | ||
| 976 | + GetRpcCounts(rpcAfter); | ||
| 977 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerAfter); | ||
| 978 | + | ||
| 979 | + ASSERT_TRUE(buffer); | ||
| 980 | + AssertBufferEqual(*buffer, value); | ||
| 981 | + ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); | ||
| 982 | + ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1); | ||
| 983 | + ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet + 1); | ||
| 984 | + ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote); | ||
| 985 | + ASSERT_EQ(ownerAfter.tcpHits, ownerBefore.tcpHits); | ||
| 986 | + ASSERT_EQ(ownerAfter.ubHits, ownerBefore.ubHits); | ||
| 987 | + ASSERT_EQ(ownerAfter.shmHits, ownerBefore.shmHits); | ||
| 988 | + ASSERT_EQ(ownerAfter.metadataMisses, ownerBefore.metadataMisses + 1); | ||
| 989 | +} | ||
| 990 | + | ||
| 991 | +TEST_F(KVClientTransportGetTest, MetadataMissReadsCrossNodeDataWorker) | ||
| 992 | +{ | ||
| 993 | + std::vector<std::string> keys; | ||
| 994 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys); | ||
| 995 | + ASSERT_EQ(keys.size(), 1u); | ||
| 996 | + const std::string value(VALUE_SIZE, 'e'); | ||
| 997 | + std::shared_ptr<KVClient> dataWriter; | ||
| 998 | + InitTestKVClient(DATA_WORKER_INDEX, dataWriter, CLIENT_TIMEOUT_MS); | ||
| 999 | + DS_ASSERT_OK(dataWriter->Set(keys[0], value)); | ||
| 1000 | + TransportRpcCounts rpcBefore; | ||
| 1001 | + WorkerQueryAndGetCounts ownerBefore; | ||
| 1002 | + GetRpcCounts(rpcBefore); | ||
| 1003 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerBefore); | ||
| 1004 | + | ||
| 1005 | + Optional<Buffer> buffer; | ||
| 1006 | + DS_ASSERT_OK(reader_->Get(keys[0], buffer)); | ||
| 1007 | + TransportRpcCounts rpcAfter; | ||
| 1008 | + WorkerQueryAndGetCounts ownerAfter; | ||
| 1009 | + GetRpcCounts(rpcAfter); | ||
| 1010 | + GetWorkerQueryAndGetCounts(META_OWNER_INDEX, ownerAfter); | ||
| 1011 | + | ||
| 1012 | + ASSERT_TRUE(buffer); | ||
| 1013 | + AssertBufferEqual(*buffer, value); | ||
| 1014 | + ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport()); | ||
| 1015 | + ASSERT_EQ(rpcAfter.queryAndGet, rpcBefore.queryAndGet + 1); | ||
| 1016 | + ASSERT_EQ(rpcAfter.getObjectRemote, rpcBefore.getObjectRemote + 1); | ||
| 1017 | + ASSERT_EQ(rpcAfter.workerOcGet, rpcBefore.workerOcGet); | ||
| 1018 | + ASSERT_EQ(ownerAfter.tcpHits, ownerBefore.tcpHits); | ||
| 1019 | + ASSERT_EQ(ownerAfter.ubHits, ownerBefore.ubHits); | ||
| 1020 | + ASSERT_EQ(ownerAfter.shmHits, ownerBefore.shmHits); | ||
| 1021 | + ASSERT_EQ(ownerAfter.metadataMisses, ownerBefore.metadataMisses + 1); | ||
| 1022 | +} | ||
| 1023 | + | ||
| 1024 | +TEST_F(KVClientTransportGetMixedPathTest, MultiKeyMixedPathsPreserveOrder) | ||
| 1025 | +{ | ||
| 1026 | + std::vector<std::string> localOwnerKeys; | ||
| 1027 | + std::vector<std::string> remoteOwnerKeys; | ||
| 1028 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 3, localOwnerKeys); | ||
| 1029 | + GetRealHashKeysToWorker(TRANSPORT_CLIENT_WORKER_INDEX, 2, remoteOwnerKeys); | ||
| 1030 | + ASSERT_EQ(localOwnerKeys.size(), 3u); | ||
| 1031 | + ASSERT_EQ(remoteOwnerKeys.size(), 2u); | ||
| 1032 | + std::shared_ptr<KVClient> remoteOwnerWriter; | ||
| 1033 | + std::shared_ptr<KVClient> dataWriter; | ||
| 1034 | + InitTestKVClient(TRANSPORT_CLIENT_WORKER_INDEX, remoteOwnerWriter, CLIENT_TIMEOUT_MS); | ||
| 1035 | + InitTestKVClient(DATA_WORKER_INDEX, dataWriter, CLIENT_TIMEOUT_MS); | ||
| 1036 | + const std::vector<std::string> values = { std::string(VALUE_SIZE, 'f'), std::string(VALUE_SIZE, 'g'), | ||
| 1037 | + std::string(MIXED_OVERSIZED_INLINE_VALUE_SIZE, 'h') }; | ||
| 1038 | + DS_ASSERT_OK(writer_->Set(localOwnerKeys[0], values[0])); | ||
| 1039 | + DS_ASSERT_OK(writer_->Set(localOwnerKeys[1], values[0])); | ||
| 1040 | + DS_ASSERT_OK(dataWriter->Set(localOwnerKeys[2], values[1])); | ||
| 1041 | + DS_ASSERT_OK(remoteOwnerWriter->Set(remoteOwnerKeys[0], values[2])); | ||
| 1042 | + Optional<Buffer> warmup; | ||
| 1043 | + DS_ASSERT_OK(reader_->Get(localOwnerKeys[0], warmup)); | ||
| 1044 | + | ||
| 1045 | + MixedPathCounts before; | ||
| 1046 | + GetMixedPathCounts(before); | ||
| 1047 | + const std::vector<std::string> keys = { remoteOwnerKeys[1], localOwnerKeys[2], remoteOwnerKeys[0], | ||
| 1048 | + localOwnerKeys[1] }; | ||
| 1049 | + std::vector<Optional<Buffer>> buffers; | ||
| 1050 | + DS_ASSERT_OK(reader_->Get(keys, buffers)); | ||
| 1051 | + MixedPathCounts after; | ||
| 1052 | + GetMixedPathCounts(after); | ||
| 1053 | + | ||
| 1054 | + AssertMixedPathBuffers(buffers, values); | ||
| 1055 | + ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport()); | ||
| 1056 | + AssertMixedPathCounts(before, after); | ||
| 1057 | +} | ||
| 1058 | + | ||
| 712 | // Regression: when local cache is disabled, dataPlacementPolicy is a write-only setting. Even when | 1059 | // Regression: when local cache is disabled, dataPlacementPolicy is a write-only setting. Even when |
| 713 | // its same-node choice differs from the metadata owner, Get must enter the metadata-owner transport flow. | 1060 | // its same-node choice differs from the metadata owner, Get must enter the metadata-owner transport flow. |
| 714 | TEST_F(KVClientTransportGetWithAllWorkersShmTest, GetIgnoresSameNodeWritePlacementPolicy) | 1061 | TEST_F(KVClientTransportGetWithAllWorkersShmTest, GetIgnoresSameNodeWritePlacementPolicy) |
| @@ -753,8 +1100,8 @@ TEST_F(KVClientTransportGetWithAllWorkersShmTest, GetIgnoresSameNodeWritePlaceme | |||
| 753 | ASSERT_EQ(after.queryAndGet, before.queryAndGet + 1); | 1100 | ASSERT_EQ(after.queryAndGet, before.queryAndGet + 1); |
| 754 | ASSERT_EQ(nonOwnerGetAfter, nonOwnerGetBefore) | 1101 | ASSERT_EQ(nonOwnerGetAfter, nonOwnerGetBefore) |
| 755 | << "Get must not probe a same-host non-owner before metadata-owner QueryAndGet"; | 1102 | << "Get must not probe a same-host non-owner before metadata-owner QueryAndGet"; |
| 756 | - ASSERT_EQ(metaOwnerGetAfter, metaOwnerGetBefore + 1); | 1103 | + ASSERT_EQ(metaOwnerGetAfter, metaOwnerGetBefore); |
| 757 | - ASSERT_EQ(after.workerOcGet, before.workerOcGet + 1); | 1104 | + ASSERT_EQ(after.workerOcGet, before.workerOcGet); |
| 758 | ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); | 1105 | ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); |
| 759 | } | 1106 | } |
| 760 | } | 1107 | } |
| @@ -827,7 +1174,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass | |||
| 827 | GetRpcCounts(after); | 1174 | GetRpcCounts(after); |
| 828 | 1175 | ||
| 829 | ASSERT_EQ(after.queryAndGet, before.queryAndGet + CONCURRENT_GET_COUNT); | 1176 | ASSERT_EQ(after.queryAndGet, before.queryAndGet + CONCURRENT_GET_COUNT); |
| 830 | - ASSERT_EQ(after.workerOcGet, before.workerOcGet + CONCURRENT_GET_COUNT); | 1177 | + ASSERT_EQ(after.workerOcGet, before.workerOcGet); |
| 831 | ASSERT_EQ(after.getObjectRemote, before.getObjectRemote); | 1178 | ASSERT_EQ(after.getObjectRemote, before.getObjectRemote); |
| 832 | ASSERT_EQ(after.batchGetObjectRemote, before.batchGetObjectRemote); | 1179 | ASSERT_EQ(after.batchGetObjectRemote, before.batchGetObjectRemote); |
| 833 | ASSERT_EQ(after.registerShmClient, before.registerShmClient + 1); | 1180 | ASSERT_EQ(after.registerShmClient, before.registerShmClient + 1); |
| @@ -840,7 +1187,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass | |||
| 840 | ASSERT_TRUE(reusedBuffer); | 1187 | ASSERT_TRUE(reusedBuffer); |
| 841 | AssertBufferEqual(*reusedBuffer, value); | 1188 | AssertBufferEqual(*reusedBuffer, value); |
| 842 | ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); | 1189 | ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); |
| 843 | - ASSERT_EQ(reused.workerOcGet, after.workerOcGet + 1); | 1190 | + ASSERT_EQ(reused.workerOcGet, after.workerOcGet); |
| 844 | ASSERT_EQ(reused.registerShmClient, after.registerShmClient); | 1191 | ASSERT_EQ(reused.registerShmClient, after.registerShmClient); |
| 845 | ASSERT_EQ(reused.getClientFd, after.getClientFd); | 1192 | ASSERT_EQ(reused.getClientFd, after.getClientFd); |
| 846 | 1193 | ||
| @@ -866,7 +1213,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass | |||
| 866 | AssertBufferEqual(*batchBuffers[i], batchValues[i]); | 1213 | AssertBufferEqual(*batchBuffers[i], batchValues[i]); |
| 867 | } | 1214 | } |
| 868 | ASSERT_EQ(afterBatch.queryAndGet, beforeBatch.queryAndGet + 1); | 1215 | ASSERT_EQ(afterBatch.queryAndGet, beforeBatch.queryAndGet + 1); |
| 869 | - ASSERT_EQ(afterBatch.workerOcGet, beforeBatch.workerOcGet + 1); | 1216 | + ASSERT_EQ(afterBatch.workerOcGet, beforeBatch.workerOcGet); |
| 870 | ASSERT_EQ(afterBatch.getObjectRemote, beforeBatch.getObjectRemote); | 1217 | ASSERT_EQ(afterBatch.getObjectRemote, beforeBatch.getObjectRemote); |
| 871 | ASSERT_EQ(afterBatch.batchGetObjectRemote, beforeBatch.batchGetObjectRemote); | 1218 | ASSERT_EQ(afterBatch.batchGetObjectRemote, beforeBatch.batchGetObjectRemote); |
| 872 | ASSERT_EQ(afterBatch.registerShmClient, beforeBatch.registerShmClient); | 1219 | ASSERT_EQ(afterBatch.registerShmClient, beforeBatch.registerShmClient); |
| @@ -890,7 +1237,7 @@ TEST_F(KVClientTransportGetWithShmTest, NonBoundSameHostWorkerUsesWorkerOcFdPass | |||
| 890 | ASSERT_TRUE(postHeartbeatBuffer); | 1237 | ASSERT_TRUE(postHeartbeatBuffer); |
| 891 | AssertBufferEqual(*postHeartbeatBuffer, value); | 1238 | AssertBufferEqual(*postHeartbeatBuffer, value); |
| 892 | ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); | 1239 | ASSERT_EQ(AccessTransportTracker::ToString(), "SHM"); |
| 893 | - ASSERT_EQ(postHeartbeat.workerOcGet, maintained.workerOcGet + 1); | 1240 | + ASSERT_EQ(postHeartbeat.workerOcGet, maintained.workerOcGet); |
| 894 | ASSERT_EQ(postHeartbeat.registerShmClient, maintained.registerShmClient); | 1241 | ASSERT_EQ(postHeartbeat.registerShmClient, maintained.registerShmClient); |
| 895 | ASSERT_EQ(postHeartbeat.getClientFd, maintained.getClientFd); | 1242 | ASSERT_EQ(postHeartbeat.getClientFd, maintained.getClientFd); |
| 896 | } | 1243 | } |
| @@ -946,56 +1293,6 @@ TEST_F(KVClientTransportGetDrainingRealUrmaTest, DrainingTargetUsesUb) | |||
| 946 | ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 2); | 1293 | ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 2); |
| 947 | } | 1294 | } |
| 948 | 1295 | ||
| 949 | -TEST_F(KVClientTransportGetScaleOutRealUrmaTest, RedirectedMetadataOwnerPrecedesClientSnapshot) | ||
| 950 | -{ | ||
| 951 | - const auto keys = MakeRandomKeys(SCALE_OUT_KEY_COUNT); | ||
| 952 | - const std::string value(VALUE_SIZE, 'u'); | ||
| 953 | - for (const auto &key : keys) { | ||
| 954 | - DS_ASSERT_OK(writer_->Set(key, value)); | ||
| 955 | - } | ||
| 956 | - ClusterTopologyPb initialTopology; | ||
| 957 | - GetClusterTopologyPb(initialTopology); | ||
| 958 | - | ||
| 959 | - const uint64_t pauseBaseline = inject::GetExecuteCount(HASH_RING_REFRESH_BEFORE_WAIT_INJECT); | ||
| 960 | - DS_ASSERT_OK(inject::Set(HASH_RING_REFRESH_BEFORE_WAIT_INJECT, "1*pause()")); | ||
| 961 | - Raii releaseRefresh([] { (void)inject::Clear(HASH_RING_REFRESH_BEFORE_WAIT_INJECT); }); | ||
| 962 | - const auto pauseDeadline = std::chrono::steady_clock::now() + std::chrono::seconds(10); | ||
| 963 | - while (inject::GetExecuteCount(HASH_RING_REFRESH_BEFORE_WAIT_INJECT) == pauseBaseline | ||
| 964 | - && std::chrono::steady_clock::now() < pauseDeadline) { | ||
| 965 | - std::this_thread::sleep_for(std::chrono::milliseconds(50)); | ||
| 966 | - } | ||
| 967 | - ASSERT_GT(inject::GetExecuteCount(HASH_RING_REFRESH_BEFORE_WAIT_INJECT), pauseBaseline); | ||
| 968 | - | ||
| 969 | - HostPort masterAddress; | ||
| 970 | - DS_ASSERT_OK(cluster_->GetWorkerAddr(META_OWNER_INDEX, masterAddress)); | ||
| 971 | - HostPort newWorkerAddress("127.0.0.1", GetFreePort()); | ||
| 972 | - DS_ASSERT_OK(cluster_->AddNode(masterAddress, newWorkerAddress.ToString(), GetFreePort())); | ||
| 973 | - DS_ASSERT_OK(cluster_->WaitNodeReady(WORKER, SCALE_OUT_WORKER_INDEX, 30)); | ||
| 974 | - WaitAllMembersJoinClusterTopology(SCALE_OUT_WORKER_COUNT); | ||
| 975 | - WaitClusterTopologyChange([](const ClusterTopologyPb &topology) { return !topology.has_active_batch(); }); | ||
| 976 | - | ||
| 977 | - ClusterTopologyPb finalTopology; | ||
| 978 | - GetClusterTopologyPb(finalTopology); | ||
| 979 | - auto redirectedKey = std::find_if(keys.begin(), keys.end(), [&](const std::string &key) { | ||
| 980 | - return SelectMetadataOwner(initialTopology, key) != newWorkerAddress.ToString() | ||
| 981 | - && SelectMetadataOwner(finalTopology, key) == newWorkerAddress.ToString(); | ||
| 982 | - }); | ||
| 983 | - ASSERT_NE(redirectedKey, keys.end()); | ||
| 984 | - | ||
| 985 | - DS_ASSERT_OK(cluster_->SetInjectAction(WORKER, SCALE_OUT_WORKER_INDEX, QUERY_AND_GET_INJECT, "call()")); | ||
| 986 | - uint64_t redirectedQueryBaseline = 0; | ||
| 987 | - DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(WORKER, SCALE_OUT_WORKER_INDEX, QUERY_AND_GET_INJECT, | ||
| 988 | - redirectedQueryBaseline)); | ||
| 989 | - Optional<Buffer> buffer; | ||
| 990 | - DS_ASSERT_OK(reader_->Get(*redirectedKey, buffer)); | ||
| 991 | - ASSERT_TRUE(buffer); | ||
| 992 | - AssertBufferEqual(*buffer, value); | ||
| 993 | - uint64_t redirectedQueryCount = 0; | ||
| 994 | - DS_ASSERT_OK(cluster_->GetInjectActionExecuteCount(WORKER, SCALE_OUT_WORKER_INDEX, QUERY_AND_GET_INJECT, | ||
| 995 | - redirectedQueryCount)); | ||
| 996 | - ASSERT_GT(redirectedQueryCount, redirectedQueryBaseline); | ||
| 997 | - ASSERT_EQ(AccessTransportTracker::ToString(), "UB"); | ||
| 998 | -} | ||
| 999 | 1296 | ||
| 1000 | TEST_F(KVClientTransportGetWithShmTest, PinPendingSingleAndBatchReadOnlyGetUsePageableMemory) | 1297 | TEST_F(KVClientTransportGetWithShmTest, PinPendingSingleAndBatchReadOnlyGetUsePageableMemory) |
| 1001 | { | 1298 | { |
| @@ -1204,39 +1501,28 @@ TEST_F(KVClientTransportGetTest, NonLocalMetaOwnerFallsBack) | |||
| 1204 | ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 1); | 1501 | ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 1); |
| 1205 | } | 1502 | } |
| 1206 | 1503 | ||
| 1207 | -class KVClientTransportGetInlineFailureTest : public KVClientTransportGetTest { | 1504 | +TEST_F(KVClientTransportGetTest, InlineEncodeFailureFallsBackPerKey) |
| 1208 | -public: | ||
| 1209 | - void SetClusterSetupOptions(ExternalClusterOptions &opts) override | ||
| 1210 | - { | ||
| 1211 | - KVClientTransportGetTest::SetClusterSetupOptions(opts); | ||
| 1212 | - opts.injectActions += ";" + std::string(INLINE_READ_FAILURE_INJECT) + ":1*return(K_RUNTIME_ERROR)"; | ||
| 1213 | - } | ||
| 1214 | -}; | ||
| 1215 | - | ||
| 1216 | -TEST_F(KVClientTransportGetInlineFailureTest, InlineReadFailureFallsBack) | ||
| 1217 | { | 1505 | { |
| 1218 | std::vector<std::string> keys; | 1506 | std::vector<std::string> keys; |
| 1219 | - GetRealHashKeysToWorker(META_OWNER_INDEX, 1, keys); | 1507 | + GetRealHashKeysToWorker(META_OWNER_INDEX, 2, keys); |
| 1220 | - ASSERT_EQ(keys.size(), 1u); | 1508 | + const std::vector<std::string> values = { std::string(VALUE_SIZE, 'f'), std::string(VALUE_SIZE, 'g') }; |
| 1221 | - const std::string &key = keys.front(); | 1509 | + DS_ASSERT_OK(writer_->Set(keys[0], values[0])); |
| 1222 | - const std::string value(VALUE_SIZE, 'f'); | 1510 | + DS_ASSERT_OK(writer_->Set(keys[1], values[1])); |
| 1223 | - DS_ASSERT_OK(writer_->Set(key, value)); | 1511 | + DS_ASSERT_OK(cluster_->SetInjectAction( |
| 1512 | + WORKER, META_OWNER_INDEX, QUERY_AND_GET_INLINE_FAILURE_INJECT, "1*return(K_RUNTIME_ERROR)")); | ||
| 1513 | + Raii clearInject([this] { | ||
| 1514 | + (void)cluster_->ClearInjectAction(WORKER, META_OWNER_INDEX, QUERY_AND_GET_INLINE_FAILURE_INJECT); | ||
| 1515 | + }); | ||
| 1224 | 1516 | ||
| 1225 | - TransportRpcCounts before; | 1517 | + std::vector<Optional<Buffer>> buffers; |
| 1226 | - GetRpcCounts(before); | 1518 | + DS_ASSERT_OK(reader_->Get(keys, buffers)); |
| 1227 | 1519 | ||
| 1228 | - Optional<Buffer> buffer; | 1520 | + ASSERT_EQ(buffers.size(), keys.size()); |
| 1229 | - Status rc = reader_->Get(key, buffer); | 1521 | + for (size_t i = 0; i < buffers.size(); ++i) { |
| 1230 | - | 1522 | + ASSERT_TRUE(buffers[i]); |
| 1231 | - TransportRpcCounts after; | 1523 | + AssertBufferEqual(*buffers[i], values[i]); |
| 1232 | - GetRpcCounts(after); | 1524 | + } |
| 1233 | - | ||
| 1234 | - DS_ASSERT_OK(rc); | ||
| 1235 | - ASSERT_TRUE(buffer); | ||
| 1236 | - AssertBufferEqual(*buffer, value); | ||
| 1237 | ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport()); | 1525 | ASSERT_EQ(AccessTransportTracker::ToString(), ExpectedTransport()); |
| 1238 | - ASSERT_EQ(after.queryAndGet, before.queryAndGet + 1); | ||
| 1239 | - ASSERT_EQ(after.getObjectRemote, before.getObjectRemote + 1); | ||
| 1240 | } | 1526 | } |
| 1241 | 1527 | ||
| 1242 | // One key's data read fails while the others succeed; overall K_OK with the failed slot empty. | 1528 | // One key's data read fails while the others succeed; overall K_OK with the failed slot empty. |
| @@ -240,8 +240,8 @@ std::vector<std::shared_ptr<ObjectBuffer>> MakeTransportBuffers(const HostPort & | |||
| 240 | return buffers; | 240 | return buffers; |
| 241 | } | 241 | } |
| 242 | 242 | ||
| 243 | -master::QueryAndGetResultPb *AddLocation(master::QueryAndGetRspPb &response, const std::string &key, | 243 | +QueryAndGetResultPb *AddLocation(QueryAndGetRspPb &response, const std::string &key, |
| 244 | - const HostPort &address, uint64_t size = 4) | 244 | + const HostPort &address, uint64_t size = 4) |
| 245 | { | 245 | { |
| 246 | auto *result = response.add_results(); | 246 | auto *result = response.add_results(); |
| 247 | auto *location = result->mutable_location(); | 247 | auto *location = result->mutable_location(); |
| @@ -384,7 +384,7 @@ public: | |||
| 384 | return batchGetObjectStatus; | 384 | return batchGetObjectStatus; |
| 385 | } | 385 | } |
| 386 | 386 | ||
| 387 | - Status InvokeQueryAndGet(master::QueryAndGetReqPb &request, master::QueryAndGetRspPb &response, | 387 | + Status InvokeQueryAndGet(QueryAndGetReqPb &request, QueryAndGetRspPb &response, |
| 388 | std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr) override | 388 | std::vector<RpcMessage> &payloads, bool *rpcDispatched = nullptr) override |
| 389 | { | 389 | { |
| 390 | if (rpcDispatched != nullptr) { | 390 | if (rpcDispatched != nullptr) { |
| @@ -545,8 +545,8 @@ public: | |||
| 545 | std::function<Status(BatchGetObjectRemoteReqPb &, BatchGetObjectRemoteRspPb &, std::vector<RpcMessage> &)> | 545 | std::function<Status(BatchGetObjectRemoteReqPb &, BatchGetObjectRemoteRspPb &, std::vector<RpcMessage> &)> |
| 546 | batchGetHandler; | 546 | batchGetHandler; |
| 547 | Status queryAndGetStatus = Status::OK(); | 547 | Status queryAndGetStatus = Status::OK(); |
| 548 | - std::vector<master::QueryAndGetReqPb> queryAndGetRequests; | 548 | + std::vector<QueryAndGetReqPb> queryAndGetRequests; |
| 549 | - std::function<Status(const HostPort &, const master::QueryAndGetReqPb &, master::QueryAndGetRspPb &, | 549 | + std::function<Status(const HostPort &, const QueryAndGetReqPb &, QueryAndGetRspPb &, |
| 550 | std::vector<RpcMessage> &)> | 550 | std::vector<RpcMessage> &)> |
| 551 | queryAndGetHandler; | 551 | queryAndGetHandler; |
| 552 | std::function<void()> onInvoke; | 552 | std::function<void()> onInvoke; |
| @@ -658,7 +658,7 @@ public: | |||
| 658 | HeartbeatReqPb invokedShmHeartbeatRequest; | 658 | HeartbeatReqPb invokedShmHeartbeatRequest; |
| 659 | DisconnectClientReqPb invokedShmDisconnectRequest; | 659 | DisconnectClientReqPb invokedShmDisconnectRequest; |
| 660 | BatchGetObjectRemoteReqPb invokedBatchGetRequest; | 660 | BatchGetObjectRemoteReqPb invokedBatchGetRequest; |
| 661 | - master::QueryAndGetReqPb invokedMetadataRequest; | 661 | + QueryAndGetReqPb invokedMetadataRequest; |
| 662 | ExistReqPb invokedExistRequest; | 662 | ExistReqPb invokedExistRequest; |
| 663 | GetHashRingReqPb invokedHashRingRequest; | 663 | GetHashRingReqPb invokedHashRingRequest; |
| 664 | CreateReqPb invokedCreateRequest; | 664 | CreateReqPb invokedCreateRequest; |
| @@ -737,8 +737,8 @@ protected: | |||
| 737 | return batchGetInvokeStatus; | 737 | return batchGetInvokeStatus; |
| 738 | } | 738 | } |
| 739 | 739 | ||
| 740 | - Status DoInvokeQueryAndGet(const RpcOptions &options, const master::QueryAndGetReqPb &request, | 740 | + Status DoInvokeQueryAndGet(const RpcOptions &options, const QueryAndGetReqPb &request, |
| 741 | - master::QueryAndGetRspPb &, std::vector<RpcMessage> &) override | 741 | + QueryAndGetRspPb &, std::vector<RpcMessage> &) override |
| 742 | { | 742 | { |
| 743 | ++metadataInvokeCount; | 743 | ++metadataInvokeCount; |
| 744 | metadataRpcTimeout = options.GetTimeout(); | 744 | metadataRpcTimeout = options.GetTimeout(); |
| @@ -999,9 +999,6 @@ public: | |||
| 999 | { | 999 | { |
| 1000 | std::lock_guard<std::mutex> lock(mutex); | 1000 | std::lock_guard<std::mutex> lock(mutex); |
| 1001 | ++rpcBuildCount; | 1001 | ++rpcBuildCount; |
| 1002 | - if (onCreateRpcClient) { | ||
| 1003 | - onCreateRpcClient(address); | ||
| 1004 | - } | ||
| 1005 | if (!rpcBuildStatuses.empty()) { | 1002 | if (!rpcBuildStatuses.empty()) { |
| 1006 | Status rc = rpcBuildStatuses.front(); | 1003 | Status rc = rpcBuildStatuses.front(); |
| 1007 | rpcBuildStatuses.erase(rpcBuildStatuses.begin()); | 1004 | rpcBuildStatuses.erase(rpcBuildStatuses.begin()); |
| @@ -1087,10 +1084,9 @@ public: | |||
| 1087 | std::vector<std::vector<Status>> transporterMSetStatuses; | 1084 | std::vector<std::vector<Status>> transporterMSetStatuses; |
| 1088 | std::vector<bool> transporterMSetPublishAttempted; | 1085 | std::vector<bool> transporterMSetPublishAttempted; |
| 1089 | std::vector<std::shared_ptr<FakeTransporter>> builtTransporters; | 1086 | std::vector<std::shared_ptr<FakeTransporter>> builtTransporters; |
| 1090 | - std::function<Status(const HostPort &, const master::QueryAndGetReqPb &, master::QueryAndGetRspPb &, | 1087 | + std::function<Status(const HostPort &, const QueryAndGetReqPb &, QueryAndGetRspPb &, |
| 1091 | std::vector<RpcMessage> &)> | 1088 | std::vector<RpcMessage> &)> |
| 1092 | queryAndGetHandler; | 1089 | queryAndGetHandler; |
| 1093 | - std::function<void(const HostPort &)> onCreateRpcClient; | ||
| 1094 | std::function<void(const HostPort &, FakeTransporter &)> configureTransporter; | 1090 | std::function<void(const HostPort &, FakeTransporter &)> configureTransporter; |
| 1095 | std::mutex mutex; | 1091 | std::mutex mutex; |
| 1096 | }; | 1092 | }; |
| @@ -1101,7 +1097,8 @@ public: | |||
| 1101 | { | 1097 | { |
| 1102 | } | 1098 | } |
| 1103 | 1099 | ||
| 1104 | - Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items) override | 1100 | + Status QueryAndGet(const HostPort &address, const ObjectMetadataBatch &items, |
| 1101 | + std::shared_ptr<const TransportReadContext>) override | ||
| 1105 | { | 1102 | { |
| 1106 | { | 1103 | { |
| 1107 | std::lock_guard<std::mutex> lock(mutex); | 1104 | std::lock_guard<std::mutex> lock(mutex); |
| @@ -1485,16 +1482,16 @@ TEST(WorkerRpcClientTest, SignsFinalReadRequestsBeforeRpc) | |||
| 1485 | EXPECT_FALSE(client.invokedDataRequest.signature().empty()); | 1482 | EXPECT_FALSE(client.invokedDataRequest.signature().empty()); |
| 1486 | EXPECT_EQ(client.invokedDataRequest.urma_info().seg_va(), 123u); | 1483 | EXPECT_EQ(client.invokedDataRequest.urma_info().seg_va(), 123u); |
| 1487 | 1484 | ||
| 1488 | - master::QueryAndGetReqPb metadataRequest; | 1485 | + QueryAndGetReqPb metadataRequest; |
| 1489 | metadataRequest.add_object_keys("key"); | 1486 | metadataRequest.add_object_keys("key"); |
| 1490 | - metadataRequest.set_redirect(true); | 1487 | + metadataRequest.mutable_data_request()->mutable_tcp(); |
| 1491 | - master::QueryAndGetRspPb metadataResponse; | 1488 | + QueryAndGetRspPb metadataResponse; |
| 1492 | std::vector<RpcMessage> metadataPayloads; | 1489 | std::vector<RpcMessage> metadataPayloads; |
| 1493 | ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk()); | 1490 | ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk()); |
| 1494 | EXPECT_EQ(client.metadataInvokeCount, 1); | 1491 | EXPECT_EQ(client.metadataInvokeCount, 1); |
| 1495 | EXPECT_EQ(client.invokedMetadataRequest.access_key(), "access-1"); | 1492 | EXPECT_EQ(client.invokedMetadataRequest.access_key(), "access-1"); |
| 1496 | EXPECT_FALSE(client.invokedMetadataRequest.signature().empty()); | 1493 | EXPECT_FALSE(client.invokedMetadataRequest.signature().empty()); |
| 1497 | - EXPECT_TRUE(client.invokedMetadataRequest.redirect()); | 1494 | + EXPECT_TRUE(client.invokedMetadataRequest.data_request().has_tcp()); |
| 1498 | 1495 | ||
| 1499 | ExistReqPb existRequest; | 1496 | ExistReqPb existRequest; |
| 1500 | existRequest.add_object_keys("key"); | 1497 | existRequest.add_object_keys("key"); |
| @@ -1762,9 +1759,9 @@ TEST(WorkerRpcClientTest, BoundsRpcTimeoutByApiDeadline) | |||
| 1762 | EXPECT_GT(client.dataRpcTimeout, 0); | 1759 | EXPECT_GT(client.dataRpcTimeout, 0); |
| 1763 | EXPECT_LE(client.dataRpcTimeout, 100); | 1760 | EXPECT_LE(client.dataRpcTimeout, 100); |
| 1764 | 1761 | ||
| 1765 | - master::QueryAndGetReqPb metadataRequest; | 1762 | + QueryAndGetReqPb metadataRequest; |
| 1766 | metadataRequest.add_object_keys("key"); | 1763 | metadataRequest.add_object_keys("key"); |
| 1767 | - master::QueryAndGetRspPb metadataResponse; | 1764 | + QueryAndGetRspPb metadataResponse; |
| 1768 | std::vector<RpcMessage> metadataPayloads; | 1765 | std::vector<RpcMessage> metadataPayloads; |
| 1769 | ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk()); | 1766 | ASSERT_TRUE(client.InvokeQueryAndGet(metadataRequest, metadataResponse, metadataPayloads).IsOk()); |
| 1770 | EXPECT_EQ(client.metadataInvokeCount, 1); | 1767 | EXPECT_EQ(client.metadataInvokeCount, 1); |
| @@ -1793,8 +1790,8 @@ TEST(WorkerRpcClientTest, ExpiredApiDeadlineDoesNotSendRpc) | |||
| 1793 | EXPECT_EQ(client.InvokeGetObject(dataRequest, dataResponse, payloads).GetCode(), K_RPC_DEADLINE_EXCEEDED); | 1790 | EXPECT_EQ(client.InvokeGetObject(dataRequest, dataResponse, payloads).GetCode(), K_RPC_DEADLINE_EXCEEDED); |
| 1794 | EXPECT_EQ(client.getObjectInvokeCount, 0); | 1791 | EXPECT_EQ(client.getObjectInvokeCount, 0); |
| 1795 | 1792 | ||
| 1796 | - master::QueryAndGetReqPb metadataRequest; | 1793 | + QueryAndGetReqPb metadataRequest; |
| 1797 | - master::QueryAndGetRspPb metadataResponse; | 1794 | + QueryAndGetRspPb metadataResponse; |
| 1798 | std::vector<RpcMessage> metadataPayloads; | 1795 | std::vector<RpcMessage> metadataPayloads; |
| 1799 | bool metadataRpcDispatched = true; | 1796 | bool metadataRpcDispatched = true; |
| 1800 | EXPECT_EQ( | 1797 | EXPECT_EQ( |
| @@ -2537,77 +2534,27 @@ TEST(DataPlaneManagerTest, ReconcileReleasesMapLockBeforeSlowDataPlaneClose) | |||
| 2537 | reconcileThread.join(); | 2534 | reconcileThread.join(); |
| 2538 | } | 2535 | } |
| 2539 | 2536 | ||
| 2540 | -TEST(ObjectMetadataClientTest, RestoresOrderAcrossPartialRedirectsAndDuplicateKeys) | ||
| 2541 | -{ | ||
| 2542 | - ApiDeadlineGuard deadline(1000); | ||
| 2543 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2544 | - std::vector<HostPort> calls; | ||
| 2545 | - manager->queryAndGetHandler = [&calls](const HostPort &address, const master::QueryAndGetReqPb &request, | ||
| 2546 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2547 | - calls.push_back(address); | ||
| 2548 | - if (address == MakeAddress(41)) { | ||
| 2549 | - EXPECT_EQ(request.object_keys_size(), 4); | ||
| 2550 | - AddLocation(response, "a", MakeAddress(51)); | ||
| 2551 | - AddLocation(response, "a", MakeAddress(52)); | ||
| 2552 | - auto *redirectC = response.add_info(); | ||
| 2553 | - redirectC->set_redirect_meta_address(MakeAddress(43).ToString()); | ||
| 2554 | - redirectC->add_change_meta_ids("c"); | ||
| 2555 | - auto *redirectB = response.add_info(); | ||
| 2556 | - redirectB->set_redirect_meta_address(MakeAddress(42).ToString()); | ||
| 2557 | - redirectB->add_change_meta_ids("b"); | ||
| 2558 | - } else if (address == MakeAddress(42)) { | ||
| 2559 | - EXPECT_EQ(request.object_keys_size(), 1); | ||
| 2560 | - EXPECT_EQ(request.object_keys(0), "b"); | ||
| 2561 | - EXPECT_TRUE(request.redirect()); | ||
| 2562 | - AddLocation(response, "b", MakeAddress(53)); | ||
| 2563 | - } else { | ||
| 2564 | - EXPECT_EQ(address, MakeAddress(43)); | ||
| 2565 | - EXPECT_EQ(request.object_keys_size(), 1); | ||
| 2566 | - EXPECT_EQ(request.object_keys(0), "c"); | ||
| 2567 | - EXPECT_TRUE(request.redirect()); | ||
| 2568 | - AddLocation(response, "c", MakeAddress(54)); | ||
| 2569 | - } | ||
| 2570 | - return Status::OK(); | ||
| 2571 | - }; | ||
| 2572 | - ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | ||
| 2573 | - std::vector<ObjectReadItem> inputs{ { 0, "a", MakeAddress(41) }, { 1, "b", MakeAddress(41) }, | ||
| 2574 | - { 2, "a", MakeAddress(41) }, { 3, "c", MakeAddress(41) } }; | ||
| 2575 | - auto results = MakeMetadataItems(inputs); | ||
| 2576 | - auto batch = MakeMetadataBatch(results); | ||
| 2577 | - | ||
| 2578 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | ||
| 2579 | - ASSERT_EQ(results.size(), 4u); | ||
| 2580 | - for (size_t i = 0; i < results.size(); ++i) { | ||
| 2581 | - EXPECT_TRUE(results[i].status.IsOk()); | ||
| 2582 | - EXPECT_EQ(results[i].objectKey, inputs[i].objectKey); | ||
| 2583 | - } | ||
| 2584 | - ASSERT_EQ(calls.size(), 3u); | ||
| 2585 | - EXPECT_EQ(calls[0], MakeAddress(41)); | ||
| 2586 | - EXPECT_EQ(calls[1], MakeAddress(42)); | ||
| 2587 | - EXPECT_EQ(calls[2], MakeAddress(43)); | ||
| 2588 | -} | ||
| 2589 | - | ||
| 2590 | TEST(ObjectMetadataClientTest, RejectsResultCountMismatchBeforeIndexedAccess) | 2537 | TEST(ObjectMetadataClientTest, RejectsResultCountMismatchBeforeIndexedAccess) |
| 2591 | { | 2538 | { |
| 2592 | ApiDeadlineGuard deadline(1000); | 2539 | ApiDeadlineGuard deadline(1000); |
| 2593 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2540 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 2594 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &, | 2541 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &, |
| 2595 | - master::QueryAndGetRspPb &, std::vector<RpcMessage> &) { | 2542 | + QueryAndGetRspPb &, std::vector<RpcMessage> &) { |
| 2596 | return Status::OK(); | 2543 | return Status::OK(); |
| 2597 | }; | 2544 | }; |
| 2598 | ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | 2545 | ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); |
| 2599 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2546 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 2600 | auto batch = MakeMetadataBatch(results); | 2547 | auto batch = MakeMetadataBatch(results); |
| 2601 | 2548 | ||
| 2602 | - EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch).GetCode(), K_RUNTIME_ERROR); | 2549 | + EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).GetCode(), K_RUNTIME_ERROR); |
| 2603 | } | 2550 | } |
| 2604 | 2551 | ||
| 2605 | TEST(ObjectMetadataClientTest, EmptyLocationsFailOnlyTheirInputItem) | 2552 | TEST(ObjectMetadataClientTest, EmptyLocationsFailOnlyTheirInputItem) |
| 2606 | { | 2553 | { |
| 2607 | ApiDeadlineGuard deadline(1000); | 2554 | ApiDeadlineGuard deadline(1000); |
| 2608 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2555 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 2609 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &, | 2556 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &, |
| 2610 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | 2557 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { |
| 2611 | response.add_results()->mutable_location()->set_object_key("missing"); | 2558 | response.add_results()->mutable_location()->set_object_key("missing"); |
| 2612 | AddLocation(response, "present", MakeAddress(51)); | 2559 | AddLocation(response, "present", MakeAddress(51)); |
| 2613 | return Status::OK(); | 2560 | return Status::OK(); |
| @@ -2617,311 +2564,19 @@ TEST(ObjectMetadataClientTest, EmptyLocationsFailOnlyTheirInputItem) | |||
| 2617 | { 1, "present", MakeAddress(41) } }); | 2564 | { 1, "present", MakeAddress(41) } }); |
| 2618 | auto batch = MakeMetadataBatch(results); | 2565 | auto batch = MakeMetadataBatch(results); |
| 2619 | 2566 | ||
| 2620 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2567 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 2621 | ASSERT_EQ(results.size(), 2u); | 2568 | ASSERT_EQ(results.size(), 2u); |
| 2622 | EXPECT_EQ(results[0].status.GetCode(), K_NOT_FOUND); | 2569 | EXPECT_EQ(results[0].status.GetCode(), K_NOT_FOUND); |
| 2623 | EXPECT_TRUE(results[1].status.IsOk()); | 2570 | EXPECT_TRUE(results[1].status.IsOk()); |
| 2624 | } | 2571 | } |
| 2625 | 2572 | ||
| 2626 | -TEST(ObjectMetadataClientTest, RetriesMetaMovingWithTheSameKeyGroup) | ||
| 2627 | -{ | ||
| 2628 | - ApiDeadlineGuard deadline(1000); | ||
| 2629 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2630 | - int invokeCount = 0; | ||
| 2631 | - manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &request, | ||
| 2632 | - master::QueryAndGetRspPb &response, | ||
| 2633 | - std::vector<RpcMessage> &) { | ||
| 2634 | - EXPECT_EQ(request.object_keys_size(), 2); | ||
| 2635 | - if (++invokeCount == 1) { | ||
| 2636 | - response.set_meta_is_moving(true); | ||
| 2637 | - } else { | ||
| 2638 | - AddLocation(response, "a", MakeAddress(51)); | ||
| 2639 | - AddLocation(response, "b", MakeAddress(52)); | ||
| 2640 | - } | ||
| 2641 | - return Status::OK(); | ||
| 2642 | - }; | ||
| 2643 | - ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | ||
| 2644 | - auto results = MakeMetadataItems({ { 0, "a", MakeAddress(41) }, { 1, "b", MakeAddress(41) } }); | ||
| 2645 | - auto batch = MakeMetadataBatch(results); | ||
| 2646 | - | ||
| 2647 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | ||
| 2648 | - EXPECT_EQ(invokeCount, 2); | ||
| 2649 | - ASSERT_EQ(results.size(), 2u); | ||
| 2650 | - EXPECT_TRUE(results[0].status.IsOk()); | ||
| 2651 | - EXPECT_TRUE(results[1].status.IsOk()); | ||
| 2652 | -} | ||
| 2653 | - | ||
| 2654 | -TEST(ObjectMetadataClientTest, FollowsTwoRedirects) | ||
| 2655 | -{ | ||
| 2656 | - ApiDeadlineGuard deadline(1000); | ||
| 2657 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2658 | - std::vector<HostPort> calls; | ||
| 2659 | - manager->queryAndGetHandler = [&calls](const HostPort &address, const master::QueryAndGetReqPb &request, | ||
| 2660 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2661 | - calls.push_back(address); | ||
| 2662 | - EXPECT_TRUE(request.redirect()); | ||
| 2663 | - if (address == MakeAddress(43)) { | ||
| 2664 | - AddLocation(response, "key", MakeAddress(51)); | ||
| 2665 | - } else { | ||
| 2666 | - auto *redirect = response.add_info(); | ||
| 2667 | - redirect->set_redirect_meta_address( | ||
| 2668 | - (address == MakeAddress(41) ? MakeAddress(42) : MakeAddress(43)).ToString()); | ||
| 2669 | - redirect->add_change_meta_ids("key"); | ||
| 2670 | - } | ||
| 2671 | - return Status::OK(); | ||
| 2672 | - }; | ||
| 2673 | - ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | ||
| 2674 | - auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | ||
| 2675 | - auto batch = MakeMetadataBatch(results); | ||
| 2676 | - | ||
| 2677 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | ||
| 2678 | - ASSERT_EQ(results.size(), 1u); | ||
| 2679 | - EXPECT_TRUE(results[0].status.IsOk()); | ||
| 2680 | - EXPECT_EQ(calls, std::vector<HostPort>({ MakeAddress(41), MakeAddress(42), MakeAddress(43) })); | ||
| 2681 | -} | ||
| 2682 | - | ||
| 2683 | -TEST(ObjectMetadataClientTest, RedirectedOwnerAbsentFromSnapshotUsesMetadataOnlyRpcAndRequestsRefresh) | ||
| 2684 | -{ | ||
| 2685 | - ApiDeadlineGuard deadline(1000); | ||
| 2686 | - const HostPort currentOwner = MakeAddress(41); | ||
| 2687 | - const HostPort redirectedOwner = MakeAddress(42); | ||
| 2688 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2689 | - WorkerSnapshot snapshot; | ||
| 2690 | - snapshot.ringVersion = 10; | ||
| 2691 | - snapshot.remoteTransportAddrs.push_back(currentOwner); | ||
| 2692 | - ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk()); | ||
| 2693 | - | ||
| 2694 | - manager->queryAndGetHandler = [currentOwner, redirectedOwner]( | ||
| 2695 | - const HostPort &address, const master::QueryAndGetReqPb &request, | ||
| 2696 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2697 | - if (address == currentOwner) { | ||
| 2698 | - EXPECT_TRUE(request.has_data_request()); | ||
| 2699 | - EXPECT_TRUE(request.data_request().has_ub()); | ||
| 2700 | - auto *redirect = response.add_info(); | ||
| 2701 | - redirect->set_redirect_meta_address(redirectedOwner.ToString()); | ||
| 2702 | - redirect->add_change_meta_ids("key"); | ||
| 2703 | - redirect->set_topology_version(11); | ||
| 2704 | - return Status::OK(); | ||
| 2705 | - } | ||
| 2706 | - EXPECT_EQ(address, redirectedOwner); | ||
| 2707 | - EXPECT_FALSE(request.has_data_request()); | ||
| 2708 | - AddLocation(response, "key", MakeAddress(51), 6); | ||
| 2709 | - return Status::OK(); | ||
| 2710 | - }; | ||
| 2711 | - std::vector<std::pair<HostPort, Status>> failures; | ||
| 2712 | - auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); | ||
| 2713 | - ObjectMetadataClient metadata( | ||
| 2714 | - manager, std::make_shared<DeadlineRetry>(), | ||
| 2715 | - std::make_shared<FixedTransportAdvisor>(TransportHint::UB_CANDIDATE), bufferProvider, 16, | ||
| 2716 | - [&failures](const HostPort &address, const Status &status) { failures.emplace_back(address, status); }); | ||
| 2717 | - auto results = MakeMetadataItems({ { 0, "key", currentOwner } }); | ||
| 2718 | - auto batch = MakeMetadataBatch(results); | ||
| 2719 | - | ||
| 2720 | - ASSERT_TRUE(metadata.QueryAndGet(currentOwner, batch).IsOk()); | ||
| 2721 | - ASSERT_EQ(results.size(), 1u); | ||
| 2722 | - EXPECT_TRUE(results[0].status.IsOk()); | ||
| 2723 | - EXPECT_FALSE(results[0].inlineData.has_value()); | ||
| 2724 | - ASSERT_EQ(failures.size(), 1u); | ||
| 2725 | - EXPECT_EQ(failures[0].first, redirectedOwner); | ||
| 2726 | - EXPECT_TRUE(IsTransportSnapshotStaleLocation(failures[0].second)); | ||
| 2727 | - EXPECT_EQ(manager->rpcBuildCount, 2); | ||
| 2728 | - EXPECT_EQ(manager->transportBuildCount, 1); | ||
| 2729 | - | ||
| 2730 | - std::shared_ptr<IDataTransporter> transporter; | ||
| 2731 | - EXPECT_EQ(manager->GetOrCreate(redirectedOwner, TransportHint::UB_CANDIDATE, transporter).GetCode(), K_NOT_READY); | ||
| 2732 | -} | ||
| 2733 | - | ||
| 2734 | -TEST(ObjectMetadataClientTest, RedirectVersionNotNewerThanSnapshotRemainsRejected) | ||
| 2735 | -{ | ||
| 2736 | - for (uint64_t redirectVersion : { 0, 9, 10 }) { | ||
| 2737 | - ApiDeadlineGuard deadline(1000); | ||
| 2738 | - const HostPort currentOwner = MakeAddress(41); | ||
| 2739 | - const HostPort redirectedOwner = MakeAddress(42); | ||
| 2740 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2741 | - WorkerSnapshot snapshot; | ||
| 2742 | - snapshot.ringVersion = 10; | ||
| 2743 | - snapshot.remoteTransportAddrs.push_back(currentOwner); | ||
| 2744 | - ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk()); | ||
| 2745 | - | ||
| 2746 | - int redirectedInvocations = 0; | ||
| 2747 | - manager->queryAndGetHandler = [currentOwner, redirectedOwner, redirectVersion, &redirectedInvocations]( | ||
| 2748 | - const HostPort &address, const master::QueryAndGetReqPb &, | ||
| 2749 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2750 | - if (address == currentOwner) { | ||
| 2751 | - auto *redirect = response.add_info(); | ||
| 2752 | - redirect->set_redirect_meta_address(redirectedOwner.ToString()); | ||
| 2753 | - redirect->add_change_meta_ids("key"); | ||
| 2754 | - redirect->set_topology_version(redirectVersion); | ||
| 2755 | - return Status::OK(); | ||
| 2756 | - } | ||
| 2757 | - ++redirectedInvocations; | ||
| 2758 | - AddLocation(response, "key", MakeAddress(51)); | ||
| 2759 | - return Status::OK(); | ||
| 2760 | - }; | ||
| 2761 | - ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | ||
| 2762 | - auto results = MakeMetadataItems({ { 0, "key", currentOwner } }); | ||
| 2763 | - | ||
| 2764 | - ASSERT_TRUE(metadata.QueryAndGet(currentOwner, MakeMetadataBatch(results)).IsOk()); | ||
| 2765 | - EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status)); | ||
| 2766 | - EXPECT_EQ(redirectedInvocations, 0); | ||
| 2767 | - EXPECT_EQ(manager->rpcBuildCount, 1); | ||
| 2768 | - } | ||
| 2769 | -} | ||
| 2770 | - | ||
| 2771 | -TEST(ObjectMetadataClientTest, NewerRedirectIsRejectedIfSnapshotAdvancesDuringRpcCreation) | ||
| 2772 | -{ | ||
| 2773 | - ApiDeadlineGuard deadline(1000); | ||
| 2774 | - const HostPort currentOwner = MakeAddress(41); | ||
| 2775 | - const HostPort redirectedOwner = MakeAddress(42); | ||
| 2776 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2777 | - WorkerSnapshot snapshot; | ||
| 2778 | - snapshot.ringVersion = 10; | ||
| 2779 | - snapshot.remoteTransportAddrs.push_back(currentOwner); | ||
| 2780 | - ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk()); | ||
| 2781 | - | ||
| 2782 | - int redirectedInvocations = 0; | ||
| 2783 | - manager->queryAndGetHandler = [currentOwner, redirectedOwner, &redirectedInvocations]( | ||
| 2784 | - const HostPort &address, const master::QueryAndGetReqPb &, | ||
| 2785 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2786 | - if (address == currentOwner) { | ||
| 2787 | - auto *redirect = response.add_info(); | ||
| 2788 | - redirect->set_redirect_meta_address(redirectedOwner.ToString()); | ||
| 2789 | - redirect->add_change_meta_ids("key"); | ||
| 2790 | - redirect->set_topology_version(11); | ||
| 2791 | - return Status::OK(); | ||
| 2792 | - } | ||
| 2793 | - ++redirectedInvocations; | ||
| 2794 | - AddLocation(response, "key", MakeAddress(51)); | ||
| 2795 | - return Status::OK(); | ||
| 2796 | - }; | ||
| 2797 | - manager->onCreateRpcClient = [manager, currentOwner, redirectedOwner](const HostPort &address) { | ||
| 2798 | - if (address != redirectedOwner) { | ||
| 2799 | - return; | ||
| 2800 | - } | ||
| 2801 | - WorkerSnapshot latest; | ||
| 2802 | - latest.ringVersion = 12; | ||
| 2803 | - latest.remoteTransportAddrs.push_back(currentOwner); | ||
| 2804 | - ASSERT_TRUE(manager->UpdateWorkerSnapshot(latest).IsOk()); | ||
| 2805 | - }; | ||
| 2806 | - ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | ||
| 2807 | - auto results = MakeMetadataItems({ { 0, "key", currentOwner } }); | ||
| 2808 | - | ||
| 2809 | - ASSERT_TRUE(metadata.QueryAndGet(currentOwner, MakeMetadataBatch(results)).IsOk()); | ||
| 2810 | - EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status)); | ||
| 2811 | - EXPECT_EQ(redirectedInvocations, 0); | ||
| 2812 | - EXPECT_EQ(manager->rpcBuildCount, 2); | ||
| 2813 | -} | ||
| 2814 | - | ||
| 2815 | -TEST(ObjectMetadataClientTest, RedirectChainRejectsTopologyVersionRollback) | ||
| 2816 | -{ | ||
| 2817 | - constexpr uint64_t clientSnapshotVersion = 10; | ||
| 2818 | - constexpr uint64_t firstRedirectVersion = 12; | ||
| 2819 | - const std::vector<std::pair<uint64_t, bool>> cases = { | ||
| 2820 | - { 11, false }, | ||
| 2821 | - { 12, true }, | ||
| 2822 | - { 13, true }, | ||
| 2823 | - }; | ||
| 2824 | - for (const auto &[nextRedirectVersion, shouldFollow] : cases) { | ||
| 2825 | - ApiDeadlineGuard deadline(1000); | ||
| 2826 | - const HostPort initialOwner = MakeAddress(41); | ||
| 2827 | - const HostPort firstRedirectOwner = MakeAddress(42); | ||
| 2828 | - const HostPort secondRedirectOwner = MakeAddress(43); | ||
| 2829 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2830 | - WorkerSnapshot snapshot; | ||
| 2831 | - snapshot.ringVersion = clientSnapshotVersion; | ||
| 2832 | - snapshot.remoteTransportAddrs.push_back(initialOwner); | ||
| 2833 | - ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk()); | ||
| 2834 | - | ||
| 2835 | - std::vector<HostPort> calls; | ||
| 2836 | - manager->queryAndGetHandler = [initialOwner, firstRedirectOwner, secondRedirectOwner, | ||
| 2837 | - firstRedirectVersion, nextRedirectVersion, &calls]( | ||
| 2838 | - const HostPort &address, const master::QueryAndGetReqPb &, | ||
| 2839 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2840 | - calls.push_back(address); | ||
| 2841 | - if (address == secondRedirectOwner) { | ||
| 2842 | - AddLocation(response, "key", MakeAddress(51)); | ||
| 2843 | - return Status::OK(); | ||
| 2844 | - } | ||
| 2845 | - auto *redirect = response.add_info(); | ||
| 2846 | - redirect->set_redirect_meta_address( | ||
| 2847 | - (address == initialOwner ? firstRedirectOwner : secondRedirectOwner).ToString()); | ||
| 2848 | - redirect->add_change_meta_ids("key"); | ||
| 2849 | - redirect->set_topology_version( | ||
| 2850 | - address == initialOwner ? firstRedirectVersion : nextRedirectVersion); | ||
| 2851 | - return Status::OK(); | ||
| 2852 | - }; | ||
| 2853 | - ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | ||
| 2854 | - auto results = MakeMetadataItems({ { 0, "key", initialOwner } }); | ||
| 2855 | - | ||
| 2856 | - ASSERT_TRUE(metadata.QueryAndGet(initialOwner, MakeMetadataBatch(results)).IsOk()); | ||
| 2857 | - if (shouldFollow) { | ||
| 2858 | - EXPECT_TRUE(results[0].status.IsOk()); | ||
| 2859 | - EXPECT_EQ(calls, std::vector<HostPort>({ initialOwner, firstRedirectOwner, secondRedirectOwner })); | ||
| 2860 | - EXPECT_EQ(manager->rpcBuildCount, 3); | ||
| 2861 | - } else { | ||
| 2862 | - EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status)); | ||
| 2863 | - EXPECT_EQ(calls, std::vector<HostPort>({ initialOwner, firstRedirectOwner })); | ||
| 2864 | - EXPECT_EQ(manager->rpcBuildCount, 2); | ||
| 2865 | - } | ||
| 2866 | - } | ||
| 2867 | -} | ||
| 2868 | - | ||
| 2869 | -TEST(ObjectMetadataClientTest, InitialOwnerAbsentFromSnapshotRemainsRejected) | ||
| 2870 | -{ | ||
| 2871 | - ApiDeadlineGuard deadline(1000); | ||
| 2872 | - const HostPort initialOwner = MakeAddress(41); | ||
| 2873 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2874 | - WorkerSnapshot snapshot; | ||
| 2875 | - snapshot.ringVersion = 10; | ||
| 2876 | - snapshot.remoteTransportAddrs.push_back(MakeAddress(42)); | ||
| 2877 | - ASSERT_TRUE(manager->UpdateWorkerSnapshot(snapshot).IsOk()); | ||
| 2878 | - ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | ||
| 2879 | - auto results = MakeMetadataItems({ { 0, "key", initialOwner } }); | ||
| 2880 | - auto batch = MakeMetadataBatch(results); | ||
| 2881 | - | ||
| 2882 | - const Status rc = metadata.QueryAndGet(initialOwner, batch); | ||
| 2883 | - | ||
| 2884 | - EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc)); | ||
| 2885 | - EXPECT_EQ(manager->rpcBuildCount, 0); | ||
| 2886 | - EXPECT_EQ(manager->transportBuildCount, 0); | ||
| 2887 | -} | ||
| 2888 | - | ||
| 2889 | -TEST(ObjectMetadataClientTest, ReportsRedirectTargetAccessFailure) | ||
| 2890 | -{ | ||
| 2891 | - ApiDeadlineGuard deadline(1000); | ||
| 2892 | - auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2893 | - manager->queryAndGetHandler = [](const HostPort &address, const master::QueryAndGetReqPb &, | ||
| 2894 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2895 | - if (address == MakeAddress(41)) { | ||
| 2896 | - auto *redirect = response.add_info(); | ||
| 2897 | - redirect->set_redirect_meta_address(MakeAddress(42).ToString()); | ||
| 2898 | - redirect->add_change_meta_ids("key"); | ||
| 2899 | - return Status::OK(); | ||
| 2900 | - } | ||
| 2901 | - return Status(K_RPC_PEER_DEAD, "redirect target is unavailable"); | ||
| 2902 | - }; | ||
| 2903 | - std::vector<std::pair<HostPort, Status>> failures; | ||
| 2904 | - ObjectMetadataClient metadata( | ||
| 2905 | - manager, std::make_shared<DeadlineRetry>(), nullptr, nullptr, 0, | ||
| 2906 | - [&failures](const HostPort &address, const Status &status) { failures.emplace_back(address, status); }); | ||
| 2907 | - auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | ||
| 2908 | - auto batch = MakeMetadataBatch(results); | ||
| 2909 | - | ||
| 2910 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | ||
| 2911 | - ASSERT_EQ(results.size(), 1u); | ||
| 2912 | - EXPECT_TRUE(IsTransportSnapshotStaleLocation(results[0].status)); | ||
| 2913 | - ASSERT_EQ(failures.size(), 1u); | ||
| 2914 | - EXPECT_EQ(failures[0].first, MakeAddress(42)); | ||
| 2915 | - EXPECT_EQ(failures[0].second.GetCode(), K_RPC_PEER_DEAD); | ||
| 2916 | -} | ||
| 2917 | - | ||
| 2918 | TEST(ObjectMetadataClientTest, DoesNotReportDeadlineExpiredBeforeAccess) | 2573 | TEST(ObjectMetadataClientTest, DoesNotReportDeadlineExpiredBeforeAccess) |
| 2919 | { | 2574 | { |
| 2920 | ApiDeadlineGuard deadline(100, InUs{}); | 2575 | ApiDeadlineGuard deadline(100, InUs{}); |
| 2921 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2576 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 2922 | size_t invokeCount = 0; | 2577 | size_t invokeCount = 0; |
| 2923 | - manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &, | 2578 | + manager->queryAndGetHandler = [&invokeCount](const HostPort &, const QueryAndGetReqPb &, |
| 2924 | - master::QueryAndGetRspPb &, std::vector<RpcMessage> &) { | 2579 | + QueryAndGetRspPb &, std::vector<RpcMessage> &) { |
| 2925 | ++invokeCount; | 2580 | ++invokeCount; |
| 2926 | return Status::OK(); | 2581 | return Status::OK(); |
| 2927 | }; | 2582 | }; |
| @@ -2932,7 +2587,7 @@ TEST(ObjectMetadataClientTest, DoesNotReportDeadlineExpiredBeforeAccess) | |||
| 2932 | auto batch = MakeMetadataBatch(results); | 2587 | auto batch = MakeMetadataBatch(results); |
| 2933 | std::this_thread::sleep_for(std::chrono::milliseconds(1)); | 2588 | std::this_thread::sleep_for(std::chrono::milliseconds(1)); |
| 2934 | 2589 | ||
| 2935 | - EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch).GetCode(), K_RPC_DEADLINE_EXCEEDED); | 2590 | + EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).GetCode(), K_RPC_DEADLINE_EXCEEDED); |
| 2936 | EXPECT_EQ(invokeCount, 0u); | 2591 | EXPECT_EQ(invokeCount, 0u); |
| 2937 | EXPECT_EQ(failureCount, 0u); | 2592 | EXPECT_EQ(failureCount, 0u); |
| 2938 | } | 2593 | } |
| @@ -2949,7 +2604,7 @@ TEST(ObjectMetadataClientTest, ConnectionFailureRequestsRerouteWithoutFixedOwner | |||
| 2949 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2604 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 2950 | auto batch = MakeMetadataBatch(results); | 2605 | auto batch = MakeMetadataBatch(results); |
| 2951 | 2606 | ||
| 2952 | - const auto rc = metadata.QueryAndGet(MakeAddress(41), batch); | 2607 | + const auto rc = metadata.QueryAndGet(MakeAddress(41), batch, nullptr); |
| 2953 | 2608 | ||
| 2954 | EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc)); | 2609 | EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc)); |
| 2955 | EXPECT_EQ(manager->rpcBuildCount, 1); | 2610 | EXPECT_EQ(manager->rpcBuildCount, 1); |
| @@ -2963,8 +2618,8 @@ TEST(ObjectMetadataClientTest, DispatchedDeadlineRequestsRerouteWithoutFixedOwne | |||
| 2963 | ApiDeadlineGuard deadline(1000); | 2618 | ApiDeadlineGuard deadline(1000); |
| 2964 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2619 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 2965 | int invokeCount = 0; | 2620 | int invokeCount = 0; |
| 2966 | - manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &, | 2621 | + manager->queryAndGetHandler = [&invokeCount](const HostPort &, const QueryAndGetReqPb &, |
| 2967 | - master::QueryAndGetRspPb &, std::vector<RpcMessage> &) { | 2622 | + QueryAndGetRspPb &, std::vector<RpcMessage> &) { |
| 2968 | ++invokeCount; | 2623 | ++invokeCount; |
| 2969 | return Status(K_RPC_DEADLINE_EXCEEDED, "metadata owner deadline"); | 2624 | return Status(K_RPC_DEADLINE_EXCEEDED, "metadata owner deadline"); |
| 2970 | }; | 2625 | }; |
| @@ -2975,7 +2630,7 @@ TEST(ObjectMetadataClientTest, DispatchedDeadlineRequestsRerouteWithoutFixedOwne | |||
| 2975 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2630 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 2976 | auto batch = MakeMetadataBatch(results); | 2631 | auto batch = MakeMetadataBatch(results); |
| 2977 | 2632 | ||
| 2978 | - const auto rc = metadata.QueryAndGet(MakeAddress(41), batch); | 2633 | + const auto rc = metadata.QueryAndGet(MakeAddress(41), batch, nullptr); |
| 2979 | 2634 | ||
| 2980 | EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc)); | 2635 | EXPECT_TRUE(IsTransportSnapshotStaleLocation(rc)); |
| 2981 | EXPECT_EQ(invokeCount, 1); | 2636 | EXPECT_EQ(invokeCount, 1); |
| @@ -2989,8 +2644,8 @@ TEST(ObjectMetadataClientTest, PeerDeadTearsDownWithoutRetry) | |||
| 2989 | ApiDeadlineGuard deadline(1000); | 2644 | ApiDeadlineGuard deadline(1000); |
| 2990 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2645 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 2991 | int invokeCount = 0; | 2646 | int invokeCount = 0; |
| 2992 | - manager->queryAndGetHandler = [&invokeCount](const HostPort &, const master::QueryAndGetReqPb &, | 2647 | + manager->queryAndGetHandler = [&invokeCount](const HostPort &, const QueryAndGetReqPb &, |
| 2993 | - master::QueryAndGetRspPb &, std::vector<RpcMessage> &) { | 2648 | + QueryAndGetRspPb &, std::vector<RpcMessage> &) { |
| 2994 | ++invokeCount; | 2649 | ++invokeCount; |
| 2995 | return Status(K_RPC_PEER_DEAD, "peer dead"); | 2650 | return Status(K_RPC_PEER_DEAD, "peer dead"); |
| 2996 | }; | 2651 | }; |
| @@ -2998,16 +2653,16 @@ TEST(ObjectMetadataClientTest, PeerDeadTearsDownWithoutRetry) | |||
| 2998 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2653 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 2999 | auto batch = MakeMetadataBatch(results); | 2654 | auto batch = MakeMetadataBatch(results); |
| 3000 | 2655 | ||
| 3001 | - EXPECT_TRUE(IsTransportSnapshotStaleLocation(metadata.QueryAndGet(MakeAddress(41), batch))); | 2656 | + EXPECT_TRUE(IsTransportSnapshotStaleLocation(metadata.QueryAndGet(MakeAddress(41), batch, nullptr))); |
| 3002 | EXPECT_EQ(invokeCount, 1); | 2657 | EXPECT_EQ(invokeCount, 1); |
| 3003 | EXPECT_EQ(manager->rpcBuildCount, 1); | 2658 | EXPECT_EQ(manager->rpcBuildCount, 1); |
| 3004 | 2659 | ||
| 3005 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &, | 2660 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &, |
| 3006 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | 2661 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { |
| 3007 | AddLocation(response, "key", MakeAddress(51)); | 2662 | AddLocation(response, "key", MakeAddress(51)); |
| 3008 | return Status::OK(); | 2663 | return Status::OK(); |
| 3009 | }; | 2664 | }; |
| 3010 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2665 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3011 | EXPECT_EQ(manager->rpcBuildCount, 2); | 2666 | EXPECT_EQ(manager->rpcBuildCount, 2); |
| 3012 | } | 2667 | } |
| 3013 | 2668 | ||
| @@ -3015,15 +2670,15 @@ TEST(ObjectMetadataClientTest, MetadataAndDataReuseOneEndpointRpcClient) | |||
| 3015 | { | 2670 | { |
| 3016 | ApiDeadlineGuard deadline(1000); | 2671 | ApiDeadlineGuard deadline(1000); |
| 3017 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2672 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3018 | - manager->queryAndGetHandler = [](const HostPort &address, const master::QueryAndGetReqPb &, | 2673 | + manager->queryAndGetHandler = [](const HostPort &address, const QueryAndGetReqPb &, |
| 3019 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | 2674 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { |
| 3020 | AddLocation(response, "key", address); | 2675 | AddLocation(response, "key", address); |
| 3021 | return Status::OK(); | 2676 | return Status::OK(); |
| 3022 | }; | 2677 | }; |
| 3023 | ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); | 2678 | ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>()); |
| 3024 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2679 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3025 | auto batch = MakeMetadataBatch(results); | 2680 | auto batch = MakeMetadataBatch(results); |
| 3026 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2681 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3027 | std::shared_ptr<IDataTransporter> transporter; | 2682 | std::shared_ptr<IDataTransporter> transporter; |
| 3028 | 2683 | ||
| 3029 | ASSERT_TRUE(manager->GetOrCreate(MakeAddress(41), TransportHint::TCP_ONLY, transporter).IsOk()); | 2684 | ASSERT_TRUE(manager->GetOrCreate(MakeAddress(41), TransportHint::TCP_ONLY, transporter).IsOk()); |
| @@ -3035,8 +2690,8 @@ TEST(ObjectMetadataClientTest, TcpInlineDataMovesRpcPayloadIntoMetadataResult) | |||
| 3035 | { | 2690 | { |
| 3036 | ApiDeadlineGuard deadline(1000); | 2691 | ApiDeadlineGuard deadline(1000); |
| 3037 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2692 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3038 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request, | 2693 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request, |
| 3039 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) { | 2694 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) { |
| 3040 | EXPECT_EQ(request.object_keys_size(), 1); | 2695 | EXPECT_EQ(request.object_keys_size(), 1); |
| 3041 | EXPECT_TRUE(request.has_data_request()); | 2696 | EXPECT_TRUE(request.has_data_request()); |
| 3042 | EXPECT_TRUE(request.data_request().has_tcp()); | 2697 | EXPECT_TRUE(request.data_request().has_tcp()); |
| @@ -3052,7 +2707,7 @@ TEST(ObjectMetadataClientTest, TcpInlineDataMovesRpcPayloadIntoMetadataResult) | |||
| 3052 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2707 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3053 | auto batch = MakeMetadataBatch(results); | 2708 | auto batch = MakeMetadataBatch(results); |
| 3054 | 2709 | ||
| 3055 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2710 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3056 | ASSERT_EQ(results.size(), 1u); | 2711 | ASSERT_EQ(results.size(), 1u); |
| 3057 | ASSERT_TRUE(results[0].inlineData.has_value()); | 2712 | ASSERT_TRUE(results[0].inlineData.has_value()); |
| 3058 | auto &data = *results[0].inlineData; | 2713 | auto &data = *results[0].inlineData; |
| @@ -3069,8 +2724,8 @@ TEST(ObjectMetadataClientTest, MissingTcpInlineMarkerFallsBackToReplicaRead) | |||
| 3069 | { | 2724 | { |
| 3070 | ApiDeadlineGuard deadline(1000); | 2725 | ApiDeadlineGuard deadline(1000); |
| 3071 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2726 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3072 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request, | 2727 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request, |
| 3073 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | 2728 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { |
| 3074 | EXPECT_TRUE(request.has_data_request()); | 2729 | EXPECT_TRUE(request.has_data_request()); |
| 3075 | EXPECT_TRUE(request.data_request().has_tcp()); | 2730 | EXPECT_TRUE(request.data_request().has_tcp()); |
| 3076 | AddLocation(response, "key", MakeAddress(51), 6); | 2731 | AddLocation(response, "key", MakeAddress(51), 6); |
| @@ -3081,7 +2736,7 @@ TEST(ObjectMetadataClientTest, MissingTcpInlineMarkerFallsBackToReplicaRead) | |||
| 3081 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2736 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3082 | auto batch = MakeMetadataBatch(results); | 2737 | auto batch = MakeMetadataBatch(results); |
| 3083 | 2738 | ||
| 3084 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2739 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3085 | ASSERT_EQ(results.size(), 1u); | 2740 | ASSERT_EQ(results.size(), 1u); |
| 3086 | EXPECT_TRUE(results[0].status.IsOk()); | 2741 | EXPECT_TRUE(results[0].status.IsOk()); |
| 3087 | EXPECT_FALSE(results[0].inlineData.has_value()); | 2742 | EXPECT_FALSE(results[0].inlineData.has_value()); |
| @@ -3092,8 +2747,8 @@ TEST(ObjectMetadataClientTest, RejectsInvalidTcpInlinePayloadIndex) | |||
| 3092 | { | 2747 | { |
| 3093 | ApiDeadlineGuard deadline(1000); | 2748 | ApiDeadlineGuard deadline(1000); |
| 3094 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2749 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3095 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &, | 2750 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &, |
| 3096 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) { | 2751 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) { |
| 3097 | auto *result = AddLocation(response, "key", MakeAddress(51), 6); | 2752 | auto *result = AddLocation(response, "key", MakeAddress(51), 6); |
| 3098 | result->mutable_data_result()->add_payload_indexes(1); | 2753 | result->mutable_data_result()->add_payload_indexes(1); |
| 3099 | RpcMessage payload; | 2754 | RpcMessage payload; |
| @@ -3106,7 +2761,7 @@ TEST(ObjectMetadataClientTest, RejectsInvalidTcpInlinePayloadIndex) | |||
| 3106 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2761 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3107 | auto batch = MakeMetadataBatch(results); | 2762 | auto batch = MakeMetadataBatch(results); |
| 3108 | 2763 | ||
| 3109 | - EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch).GetCode(), K_RUNTIME_ERROR); | 2764 | + EXPECT_EQ(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).GetCode(), K_RUNTIME_ERROR); |
| 3110 | } | 2765 | } |
| 3111 | 2766 | ||
| 3112 | TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuffer) | 2767 | TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuffer) |
| @@ -3115,8 +2770,8 @@ TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuff | |||
| 3115 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2770 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3116 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); | 2771 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); |
| 3117 | bufferProvider->maxGetSize = 32; | 2772 | bufferProvider->maxGetSize = 32; |
| 3118 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request, | 2773 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request, |
| 3119 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) { | 2774 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &payloads) { |
| 3120 | EXPECT_TRUE(request.has_data_request()); | 2775 | EXPECT_TRUE(request.has_data_request()); |
| 3121 | EXPECT_TRUE(request.data_request().has_ub()); | 2776 | EXPECT_TRUE(request.data_request().has_ub()); |
| 3122 | const auto &ubRequest = request.data_request().ub(); | 2777 | const auto &ubRequest = request.data_request().ub(); |
| @@ -3136,7 +2791,7 @@ TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuff | |||
| 3136 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2791 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3137 | auto batch = MakeMetadataBatch(results); | 2792 | auto batch = MakeMetadataBatch(results); |
| 3138 | 2793 | ||
| 3139 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2794 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3140 | EXPECT_EQ(bufferProvider->allocateCount, 1); | 2795 | EXPECT_EQ(bufferProvider->allocateCount, 1); |
| 3141 | EXPECT_EQ(manager->rpcBuildCount, 1); | 2796 | EXPECT_EQ(manager->rpcBuildCount, 1); |
| 3142 | EXPECT_EQ(manager->transportBuildCount, 1); | 2797 | EXPECT_EQ(manager->transportBuildCount, 1); |
| @@ -3152,13 +2807,41 @@ TEST(ObjectMetadataClientTest, UbInlineDataUsesConfiguredCapacityAndExternalBuff | |||
| 3152 | EXPECT_NE(data.externalOwner, nullptr); | 2807 | EXPECT_NE(data.externalOwner, nullptr); |
| 3153 | } | 2808 | } |
| 3154 | 2809 | ||
| 2810 | + | ||
| 2811 | +TEST(ObjectMetadataClientTest, ShmPreparationFailureTriesUbBeforeTcp) | ||
| 2812 | +{ | ||
| 2813 | + ApiDeadlineGuard deadline(1000); | ||
| 2814 | + const bool enableUrma = FLAGS_enable_urma; | ||
| 2815 | + Raii restoreEnableUrma([enableUrma]() { FLAGS_enable_urma = enableUrma; }); | ||
| 2816 | + FLAGS_enable_urma = true; | ||
| 2817 | + auto manager = std::make_shared<FakeDataPlaneManager>(); | ||
| 2818 | + auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); | ||
| 2819 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request, | ||
| 2820 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | ||
| 2821 | + EXPECT_TRUE(request.data_request().has_ub()); | ||
| 2822 | + AddLocation(response, "key", MakeAddress(51), 6); | ||
| 2823 | + return Status::OK(); | ||
| 2824 | + }; | ||
| 2825 | + ObjectMetadataClient metadata(manager, std::make_shared<DeadlineRetry>(), | ||
| 2826 | + std::make_shared<FixedTransportAdvisor>(TransportHint::SHM_CANDIDATE), | ||
| 2827 | + bufferProvider, 16); | ||
| 2828 | + auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | ||
| 2829 | + auto batch = MakeMetadataBatch(results); | ||
| 2830 | + auto readContext = std::make_shared<TransportReadContext>(); | ||
| 2831 | + | ||
| 2832 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, readContext).IsOk()); | ||
| 2833 | + EXPECT_EQ(manager->transportBuildCount, 2); | ||
| 2834 | + EXPECT_EQ(bufferProvider->allocateCount, 1); | ||
| 2835 | +} | ||
| 2836 | + | ||
| 2837 | + | ||
| 3155 | TEST(ObjectMetadataClientTest, UbCapacityMissReleasesBufferAndFallsBack) | 2838 | TEST(ObjectMetadataClientTest, UbCapacityMissReleasesBufferAndFallsBack) |
| 3156 | { | 2839 | { |
| 3157 | ApiDeadlineGuard deadline(1000); | 2840 | ApiDeadlineGuard deadline(1000); |
| 3158 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2841 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3159 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); | 2842 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); |
| 3160 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request, | 2843 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request, |
| 3161 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | 2844 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { |
| 3162 | EXPECT_TRUE(request.has_data_request()); | 2845 | EXPECT_TRUE(request.has_data_request()); |
| 3163 | EXPECT_TRUE(request.data_request().has_ub()); | 2846 | EXPECT_TRUE(request.data_request().has_ub()); |
| 3164 | EXPECT_EQ(request.data_request().ub().buffer_size(), 16u); | 2847 | EXPECT_EQ(request.data_request().ub().buffer_size(), 16u); |
| @@ -3171,20 +2854,21 @@ TEST(ObjectMetadataClientTest, UbCapacityMissReleasesBufferAndFallsBack) | |||
| 3171 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2854 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3172 | auto batch = MakeMetadataBatch(results); | 2855 | auto batch = MakeMetadataBatch(results); |
| 3173 | 2856 | ||
| 3174 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2857 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3175 | EXPECT_FALSE(results[0].inlineData.has_value()); | 2858 | EXPECT_FALSE(results[0].inlineData.has_value()); |
| 3176 | EXPECT_TRUE(bufferProvider->lastOwner.expired()); | 2859 | EXPECT_TRUE(bufferProvider->lastOwner.expired()); |
| 3177 | } | 2860 | } |
| 3178 | 2861 | ||
| 3179 | -TEST(ObjectMetadataClientTest, UbBufferAllocationFailureQueriesMetadataOnly) | 2862 | +TEST(ObjectMetadataClientTest, UbBufferAllocationFailureFallsBackToTcp) |
| 3180 | { | 2863 | { |
| 3181 | ApiDeadlineGuard deadline(1000); | 2864 | ApiDeadlineGuard deadline(1000); |
| 3182 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2865 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3183 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); | 2866 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); |
| 3184 | bufferProvider->allocateStatus = Status(K_OUT_OF_MEMORY, "allocation failed"); | 2867 | bufferProvider->allocateStatus = Status(K_OUT_OF_MEMORY, "allocation failed"); |
| 3185 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request, | 2868 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request, |
| 3186 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | 2869 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { |
| 3187 | - EXPECT_FALSE(request.has_data_request()); | 2870 | + EXPECT_TRUE(request.has_data_request()); |
| 2871 | + EXPECT_TRUE(request.data_request().has_tcp()); | ||
| 3188 | AddLocation(response, "key", MakeAddress(51), 6); | 2872 | AddLocation(response, "key", MakeAddress(51), 6); |
| 3189 | return Status::OK(); | 2873 | return Status::OK(); |
| 3190 | }; | 2874 | }; |
| @@ -3194,21 +2878,22 @@ TEST(ObjectMetadataClientTest, UbBufferAllocationFailureQueriesMetadataOnly) | |||
| 3194 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2878 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3195 | auto batch = MakeMetadataBatch(results); | 2879 | auto batch = MakeMetadataBatch(results); |
| 3196 | 2880 | ||
| 3197 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2881 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3198 | EXPECT_EQ(bufferProvider->allocateCount, 1); | 2882 | EXPECT_EQ(bufferProvider->allocateCount, 1); |
| 3199 | EXPECT_EQ(manager->transportBuildCount, 1); | 2883 | EXPECT_EQ(manager->transportBuildCount, 1); |
| 3200 | EXPECT_FALSE(results[0].inlineData.has_value()); | 2884 | EXPECT_FALSE(results[0].inlineData.has_value()); |
| 3201 | } | 2885 | } |
| 3202 | 2886 | ||
| 3203 | -TEST(ObjectMetadataClientTest, UbConnectionFailureQueriesMetadataOnly) | 2887 | +TEST(ObjectMetadataClientTest, UbConnectionFailureFallsBackToTcp) |
| 3204 | { | 2888 | { |
| 3205 | ApiDeadlineGuard deadline(1000); | 2889 | ApiDeadlineGuard deadline(1000); |
| 3206 | auto manager = std::make_shared<FakeDataPlaneManager>(); | 2890 | auto manager = std::make_shared<FakeDataPlaneManager>(); |
| 3207 | manager->transportBuildStatuses.emplace_back(K_URMA_ERROR, "connect failed"); | 2891 | manager->transportBuildStatuses.emplace_back(K_URMA_ERROR, "connect failed"); |
| 3208 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); | 2892 | auto bufferProvider = std::make_shared<FakeUbBufferProvider>(); |
| 3209 | - manager->queryAndGetHandler = [](const HostPort &, const master::QueryAndGetReqPb &request, | 2893 | + manager->queryAndGetHandler = [](const HostPort &, const QueryAndGetReqPb &request, |
| 3210 | - master::QueryAndGetRspPb &response, std::vector<RpcMessage> &) { | 2894 | + QueryAndGetRspPb &response, std::vector<RpcMessage> &) { |
| 3211 | - EXPECT_FALSE(request.has_data_request()); | 2895 | + EXPECT_TRUE(request.has_data_request()); |
| 2896 | + EXPECT_TRUE(request.data_request().has_tcp()); | ||
| 3212 | AddLocation(response, "key", MakeAddress(51), 6); | 2897 | AddLocation(response, "key", MakeAddress(51), 6); |
| 3213 | return Status::OK(); | 2898 | return Status::OK(); |
| 3214 | }; | 2899 | }; |
| @@ -3218,7 +2903,7 @@ TEST(ObjectMetadataClientTest, UbConnectionFailureQueriesMetadataOnly) | |||
| 3218 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); | 2903 | auto results = MakeMetadataItems({ { 0, "key", MakeAddress(41) } }); |
| 3219 | auto batch = MakeMetadataBatch(results); | 2904 | auto batch = MakeMetadataBatch(results); |
| 3220 | 2905 | ||
| 3221 | - ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch).IsOk()); | 2906 | + ASSERT_TRUE(metadata.QueryAndGet(MakeAddress(41), batch, nullptr).IsOk()); |
| 3222 | EXPECT_EQ(manager->rpcBuildCount, 1); | 2907 | EXPECT_EQ(manager->rpcBuildCount, 1); |
| 3223 | EXPECT_EQ(manager->transportBuildCount, 1); | 2908 | EXPECT_EQ(manager->transportBuildCount, 1); |
| 3224 | EXPECT_EQ(bufferProvider->allocateCount, 0); | 2909 | EXPECT_EQ(bufferProvider->allocateCount, 0); |


[警告][性能] [P1] 批量 UB 在确认命中和对象大小前按 key 预留注册内存
证据:基线只允许单 key 进入 inline;这里放开批量后,AllocateUbInlineBuffers 会为每个 item 分别 Allocate(ubBufferSize_)。总量只受默认 32 MiB 的 UB MaxGetSize 限制,因此单请求可预占 32 MiB;并发 32 时约为 1 GiB,而且 100% metadata miss 也会完成全部预分配。
问题:注册内存、allocator/handle、shared ownership 和哈希节点成本与真实命中率及对象大小无关,低命中大 batch 会把元数据查询变成高成本内存路径。
影响:高并发下可能造成注册池压力、allocator/arena 争用、缓存抖动和 UB 到 TCP 的额外回退,抵消 RPC 下沉收益并恶化 P99。
建议:按 owner group 使用连续注册 buffer 并通过 offset 分片,或设置独立的 key 数与字节预算后分块;在有测量依据前至少保留批量 UB 门限。
验证:在配置 DATASYSTEM_UB_GET_DATA_SIZE_BYTES 后,以并发 1/32/128、batch 1/32/1024、命中率 0/50/100% 对比基线,记录 P50/P99、分配次数、注册字节峰值和 TCP fallback 次数。