master

Lliudongliangfeat(log): add request-stage latency trace and slow-log optimization

2094a711创建于 18 小时前历史提交

文件	最后提交记录	最后更新时间
spdlog	feat(log): add request-stage latency trace and slow-log optimization	18 小时前
BUILD.bazel	feat(log): add request-stage latency trace and slow-log optimization	18 小时前
access_transport_tracker_test.cpp	fix: 修复 client access 日志传输类型记录不正确 用 thread-local AccessTransportTracker 替代 g_isThroughUb，在调用方 决策点显式记录 UB/TCP/SHM 传输方式，并从 SendBufferViaUb 移除隐式 Record，避免 Publish UB 成功误记 SHM、Get meta 校验失败或回退 TCP 时仍记录 kUb 的问题。	11 天前
failure_handler_test.cpp	Bazel support for tests	2 个月前
hard_disk_exporter_test.cpp	fix: gate conditional resource logs cherry-pick merge request: fix: gate conditional resource logs 7ec2fd224cebc1d9206e0947bf657addf7bbd031 This cherry-pick merge request !980	1 个月前
latency_phase_test.cpp	feat(log): add request-stage latency trace and slow-log optimization	18 小时前
log_access_facade_test.cpp	fix(log): address Issue #610 review findings — perf test baseline, SDK sampled-out optimization, repo context update - #1: Split fast deterministic tests into log_sampler_fast_test (default path); add tags=['manual'] to log_performance_test; add StartLoggingWithoutSampler() and LEVEL1_NoSamplerBaselineQps4000 for true no-sampler baseline; convert LEVEL1_SamplerPassThroughMacroOverheadQps4000 to A/B comparison (no-sampler vs sampler-through); enhance PrintPureLogResult with samplerMode, samplerOnly, accessRate, diagnosticRate, achievedQps, p50/p95/p99 - #2: Java SDK — delete JavaKvAccessFields struct; pass ObjectAccessRecorder& into impl; use ObjectKeyOwned(key) for single-key setters (copy, key lifetime safe); use ObjectKeyProvider for batch-key setters (move after business use); sampled-out setter no-op via ShouldRecordInternal() early return Python SDK — RAII lambda keys capture changed from value to reference (&keys); ReadSpecifyOffsetData changed to ObjectKeyProvider with deferred evaluation - #3: repo_context docs — ShouldRecordAccess() marked as internal API only; log_rate_limit/LogRateLimiter marked as historical/removed by PR 1064; DATASYSTEM_LOG_RATE_LIMIT marked as removed; current behavior described with request_sample_rate/access_sample_rate/diagnostic_sample_rate and LogSampleConfigPb; access guard references replaced with facade descriptions Fixes: #610	6 天前
log_performance_test.cpp	feat: deprecate KVClient::MSetTx and remove 2PC protocol Keep KVClient::MSetTx and Python KVClient.msettx as deprecated SDK compatibility entry points that return K_RUNTIME_ERROR / RuntimeError with a deprecated API message. Remove the transactional MSetTx 2PC implementation paths while preserving the non-transactional MSet flow. Drop obsolete proto compatibility reservations, restore worker-to-master RPC diagnostics, clean stale pending metadata state, and update docs, bindings, build files, and tests.	1 天前
log_sampler_access_test.cpp	feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop)	8 天前
log_sampler_config_test.cpp	feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop)	8 天前
log_sampler_fast_test.cpp	fix(log): address Issue #610 review findings — perf test baseline, SDK sampled-out optimization, repo context update - #1: Split fast deterministic tests into log_sampler_fast_test (default path); add tags=['manual'] to log_performance_test; add StartLoggingWithoutSampler() and LEVEL1_NoSamplerBaselineQps4000 for true no-sampler baseline; convert LEVEL1_SamplerPassThroughMacroOverheadQps4000 to A/B comparison (no-sampler vs sampler-through); enhance PrintPureLogResult with samplerMode, samplerOnly, accessRate, diagnosticRate, achievedQps, p50/p95/p99 - #2: Java SDK — delete JavaKvAccessFields struct; pass ObjectAccessRecorder& into impl; use ObjectKeyOwned(key) for single-key setters (copy, key lifetime safe); use ObjectKeyProvider for batch-key setters (move after business use); sampled-out setter no-op via ShouldRecordInternal() early return Python SDK — RAII lambda keys capture changed from value to reference (&keys); ReadSpecifyOffsetData changed to ObjectKeyProvider with deferred evaluation - #3: repo_context docs — ShouldRecordAccess() marked as internal API only; log_rate_limit/LogRateLimiter marked as historical/removed by PR 1064; DATASYSTEM_LOG_RATE_LIMIT marked as removed; current behavior described with request_sample_rate/access_sample_rate/diagnostic_sample_rate and LogSampleConfigPb; access guard references replaced with facade descriptions Fixes: #610	6 天前
log_sampler_integration_test.cpp	feat(log): add request-stage latency trace and slow-log optimization	18 小时前
log_sampler_test.cpp	feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop)	8 天前
logging_test.cpp	!1113 feat(config): optimize dynamic configuration updates From: @young_stack Reviewed-by: @yaohaolin See merge request: openeuler/yuanrong-datasystem!1113	20 小时前
operation_logger_test.cpp	feat(config): optimize dynamic configuration updates Add DS_DEFINE_*_dynamic flag metadata, operation audit log, UpdateConfig APIs for KVClient/Worker, empty-path monitor skip, and user-visible dynamic_modifiable markers in worker_config.json and dscli.md.	1 天前
trace_test.cpp	fix log rate limit	1 个月前
trace_tick_test.cpp	feat(log): add request-stage latency trace and slow-log optimization	18 小时前