| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop) | 12 天前 | |
feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop) | 12 天前 | |
feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop) | 12 天前 | |
feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop) | 12 天前 | |
feat(config): optimize dynamic configuration updates Add DS_DEFINE_*_dynamic flag metadata, operation audit log, UpdateConfig APIs for KVClient/Worker, empty-path monitor skip, and user-visible dynamic_modifiable markers in worker_config.json and dscli.md. | 4 天前 | |
feat(log): implement unified random log sampling system Replace LogRateLimiter's first-N rate limiting with LogSampler's unified random hash-threshold sampling across request/access/diagnostic dimensions. Key changes: - Introduce LogSampler core component with per-request sampling granularity - Implement LogSampleState RPC propagation for cross-process consistency - Add AccessRecorderGuard RAII pattern to skip construction when dropped - Support request_sample_rate/access_sample_rate/diagnostic_sample_rate (0.0-1.0) - Ensure FATAL/CHECK logs are never dropped regardless of sampling - Remove log_rate_limit gflag and LogRateLimiter class entirely - Reserve proto field 24 in RegisterClientRspPb (old log_rate_limit removed) Performance guarantees: - Hot paths (ShouldCreateLogMessage/ShouldCreatePlogMessage) have zero overhead when request is sampled out: no allocations, locks, CAS, RNG, or clock reads - Reject decision gates diagnostic/access log creation early Documentation: - Update docs/source_zh_cn/appendix/log_guide.md with sampling section - Remove obsolete DATASYSTEM_LOG_RATE_LIMIT from client_env_guide.md - Update .repo_context/ with LogSampler design (log-sampler-design.md) Tests: - UT: log_sampler_test, log_sampler_access_test, log_sampler_config_test, log_sampler_integration_test (cover LS-001 to LS-016 externally observable scenarios, 6 retained tests for external features, 14 internal-implementation tests removed) - ST: log_sampler_st_test (cover LS-002b/012/013 cross-process behavior) - Performance: log_performance_test benchmarks (LS-010/011) Breaking changes: - log_rate_limit gflag removed (reserved field 24 in proto) - DATASYSTEM_LOG_RATE_LIMIT env var removed (client receives config via RPC) - EnsureRequestSampleDecision() removed (use ShouldRecordAccess + Record backstop) | 12 天前 | |
fix cli scp commands for ipv6 | 4 个月前 | |
Hi ALL, stream is available now! | 7 个月前 | |
fix[log]: use local time for spdlog timestamps Signed-off-by: yangsonglin13 <yangsonglin13@huawei.com> | 3 个月前 | |
Hi ALL, stream is available now! | 7 个月前 | |
Hi all, this is yuanrong datasystem | 8 个月前 | |
Hi all, this is yuanrong datasystem | 8 个月前 | |
fix: 修复同步pr的问题 修复同步pr后provider.cpp中引入重复头文件的问题 | 1 个月前 | |
The hostpod field in logs shall print POD_IP with priority. | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 12 天前 | ||
| 12 天前 | ||
| 12 天前 | ||
| 12 天前 | ||
| 4 天前 | ||
| 12 天前 | ||
| 4 个月前 | ||
| 7 个月前 | ||
| 3 个月前 | ||
| 7 个月前 | ||
| 8 个月前 | ||
| 8 个月前 | ||
| 1 个月前 | ||
| 1 个月前 |