Star5
67
代码
代码
Issues
Pull Requests15
项目讨论
Wiki
分析
Star5
67
已开启
bthread local storage开发 #168
songrui创建于 7月17日
bthread local storage开发 #168
已开启
songrui创建于 7月17日
从songrui42/brpc-testmaster-test合入到fanzhaonan/brpcmaster
共 5 个文件变更+184-154
Mdocs/cn/bazel_brpc.md+39-46
@@ -32,71 +32,64 @@ ubsocket通过环境变量进行配置，在bRPC与ubsocket集成的过程中，
32 32 
33| 名称 | 含义 | 取值范围 | 默认值 | 必填 |33| 名称 | 含义 | 取值范围 | 默认值 | 必填 |
34|--|--|--|--|--|34|--|--|--|--|--|
35-| ubsocket_trans_mode | 通信协议 |ub，ib | ub | 否 |
36-|ubsocket_dev_name|设备名称|根据实际场景填写设备名称；例如，udma2或者bonding_dev_0|主动获得当前环境bonding设备名称，如bonding_dev_xx|否|
37-|ubsocket_eid_idx|使用普通设备的eid编号|ub协议下，通过urma_admin show命令查询获得|0|否|
38-|ubsocket_src_eid|使用bonding设备的eid|ub协议下，通过urma_admin show命令查询获得|主动获得当前环境bonding设备的eid|否|
39-|ubsocket_log_level|日志级别|error，warn，notice，info，debug|info|否|
40|ubsocket_tx_depth|发送队列深度|最小值是64，设置上限由实际机器环境决定（根据命令urma_admin show --whole中max_jfc_depth与max_jfs_depth两者的最小值）|1024|否|35|ubsocket_tx_depth|发送队列深度|最小值是64，设置上限由实际机器环境决定（根据命令urma_admin show --whole中max_jfc_depth与max_jfs_depth两者的最小值）|1024|否|
41|ubsocket_rx_depth|接受队列深度|最小值是64，设置上限由实际机器环境决定（根据命令urma_admin show --whole中max_jfc_depth与max_jfr_depth两者的最小值）|1024|否|36|ubsocket_rx_depth|接受队列深度|最小值是64，设置上限由实际机器环境决定（根据命令urma_admin show --whole中max_jfc_depth与max_jfr_depth两者的最小值）|1024|否|
42|ubsocket_readv_unlimited|是否打开readv上报限制|false，true|true|否|37|ubsocket_readv_unlimited|是否打开readv上报限制|false，true|true|否|
43-|ubsocket_block_type|内存池的最小分片|tiny：4k，default：8k，small：16k，medium：32k，large：64k|default|否|38+|ubsocket_block_type|内存池的最小分片|default：4k(CTP)/8k(TP)，large：64k|default|否|
44|ubsocket_pool_initial_size|IO内存总大小（MB）|正整数|200|否|39|ubsocket_pool_initial_size|IO内存总大小（MB）|正整数|200|否|
45|ubsocket_pool_max_size|单bRPC进程UB通信内存占用弹性扩容最大值，单位MB| [ubsocket_pool_initial_size + 64, 6144], 单次最小扩容大小为 64M，因此 ubsocket_pool_max_size - ubsocket_pool_initial_size >= 64M |2048|否|40|ubsocket_pool_max_size|单bRPC进程UB通信内存占用弹性扩容最大值，单位MB| [ubsocket_pool_initial_size + 64, 6144], 单次最小扩容大小为 64M，因此 ubsocket_pool_max_size - ubsocket_pool_initial_size >= 64M |2048|否|
46-|ubsocket_buf_pool_depth|单bRPC进程线程内存池深度|应用按需配置|12000|否|41+|ubsocket_buf_pool_depth|单bRPC进程线程内存池深度|应用按需配置|24576|否|
47-|`ubsocket_tiny_pool_enable`|是否启用UMQ tiny pool|false, true|true|否|42+|ubsocket_tiny_pool_enable|是否启用UMQ tiny pool|false, true|true|否|
48-|`ubsocket_tiny_pool_block_size`|UMQ tiny pool块大小|512, 1024, 2048, 4096, 8192|1024|否|43+|ubsocket_tiny_pool_block_size|UMQ tiny pool块大小|512, 1024, 2048, 4096, 8192|1024|否|
49-|`ubsocket_tiny_buf_pool_depth`|UMQ tiny pool块数量|正整数|8192|否|44+|ubsocket_tiny_buf_pool_depth|UMQ tiny pool块数量|正整数|8192|否|
50|ubsocket_schedule_policy|设置多平面负载分担策略|affinity_priority, affinity，rr|affinity_priority|否|45|ubsocket_schedule_policy|设置多平面负载分担策略|affinity_priority, affinity，rr|affinity_priority|否|
51|ubsocket_auto_fallback_tcp| 协议不匹配时是否自动降级为TCP|false：协议不匹配时不降级为TCP true：协议不匹配时自动降级为TCP|true|否|46|ubsocket_auto_fallback_tcp| 协议不匹配时是否自动降级为TCP|false：协议不匹配时不降级为TCP true：协议不匹配时自动降级为TCP|true|否|
52-|ubsocket_trace_enable | 是否打开trace统计|false, true | true |否|47+|ubsocket_monitor_enable | 是否打开维测数据统计|false, true | true |否|
53|ubsocket_backup_link_enable | 是否打开切换主备链路功能|false, true | true |否|48|ubsocket_backup_link_enable | 是否打开切换主备链路功能|false, true | true |否|
54-|ubsocket_trace_time | 控制维测数据输出间隔（单位s）| [1, 300] | 10 |否|49+|ubsocket_share_jfr_loop_poll_enable|设置是否开启共享JFR循环轮询模式|false, true|true|否|
55-|ubsocket_trace_file_path | 控制维测数据输出路径，路径长度范围在1到512bytes| [1, 512] | /tmp/ubsocket/log |否|50+|ubsocket_monitor_interval | 控制维测数据输出间隔（单位s）| [1, 300] | 10 |否|
56-|ubsocket_trace_file_size | 控制维测数据文件大小（MB）| [1, 300] | 10 |否|51+|ubsocket_monitor_file_path | 控制维测数据输出路径，路径长度范围在1到512bytes| [1, 512] | /tmp/ubsocket/log |否|
57-|ubsocket_stats_cli | 是否打开cli服务|false, true | true |否|52+|ubsocket_monitor_file_size | 控制维测数据文件大小（MB）| [1, 300] | 10 |否|
58-|ubsocket_enable_share_jfr| 设置是否开启共享JFR|false：不开启共享jfr<br>true：开启共享jfr|true|否|53+|ubsocket_cli_enable | 是否打开cli服务|false, true | true |否|
59-|ubsocket_share_jfr_rx_queue_depth|设置开启共享JFR后，每个Socket链接接收缓存队列深度|最小值是64，设置上限由实际机器环境决定|1024|否|54+|ubsocket_share_jfr_enable| 设置是否开启共享JFR|false：不开启共享jfr<br>true：开启共享jfr|true|否|
60-|ubsocket_share_jfr_rx_o3_queue_depth|仅RM_CTP模式下可用，设置开启共享JFR后，每个Socket链接接收时，乱序缓存队列深度|默认值是256，上限为接收缓存队列深度|256|否|
61|ubsocket_link_priority|设置URMA流量SL优先级|[-1, 15],-1表示不设置| 4 |否|55|ubsocket_link_priority|设置URMA流量SL优先级|[-1, 15],-1表示不设置| 4 |否|
62-|`ubsocket_block_type`|内存池的最小分片|tiny: 4k, default：8k，small：16k，medium：32k，large：64k| tiny | 否 |56+|ubsocket_ub_trans_mode | ub协议模式 |`RC_TP`，`RM_TP`, `RM_CTP`, `RC_CTP` | `RM_CTP` | 否 |
63-|`ubsocket_ub_trans_mode` | ub协议模式 |`RC_TP`，`RM_TP`, `RM_CTP`, `RC_CTP` | `RM_CTP` | 否 |57+|ubsocket_enable | 是否启用 ubsocket 加速 | false, true | false | 否 |
64-|`ubsocket_enable` | 是否启用 ubsocket 加速 | false, true | false | 否 |58+|ubsocket_degrade_enable | 是否允许 ubsocket 在 UB 错误时降级成 TCP | false, true | true | 否 |
65-|`ubsocket_degrade` | 是否允许 ubsocket 在 UB 错误时降级成 TCP | false, true | true | 否 |59+|ubsocket_async_accept_enable | 是否允许 ubsocket 在server做accept时做异步accept | false, true | false | 否 |
66-|`ubsocket_async_accept` | 是否允许 ubsocket 在server做accept时做异步accept | false, true | false | 否 |60+|ubsocket_async_accept_thread_num | ubsocket后台线程池的线程数 | [1,64] | 1 | 否 |
67-|`ubsocket_thread_pool_size` | ubsocket后台线程池的线程数 | [1,64] | 1 | 否 |61+|ubsocket_probe_enable | ubsocket开启探测包功能 | false,true | false | 否 |
68-|`ubsocket_async_epoll_wait` | 是否开启 ubsocket 在 epoll_wait时异步处理 | false, true | false | 否 |62+|ubsocket_probe_interval_ms | ubsocket 探测间隔时间（毫秒） | [1, 360000] | 1000 | 否 |
69-|`ubsocket_probe_enable` | ubsocket开启探测包功能 | false,true | false | 否 |63+|ubsocket_probe_batch_size | ubsocket 每批次探测的 Socket 数量 | [1, 500] | 10 | 否 |
70-|`ubsocket_probe_time_ms` | ubsocket 探测间隔时间（毫秒） | [1, 360000] | 1000 | 否 |64+|ubsocket_ub_handshake_mode | UB通信建链握手模式 | tfo, ub_sock_opt | ub_sock_opt | 否 |
71-|`ubsocket_probe_batch` | ubsocket 每批次探测的 Socket 数量 | [1, 500] | 10 | 否 |65+|ubsocket_flow_control_enable | 是否开启流控 | false, true | true | 否 |
72-|`ubsocket_ub_epoll_enable` | ubsocket 设置是否使用UB Epoll实现处理 | false: 使用原生Epoll，true: 使用UB Epoll | true | 否 |66+|ubsocket_prof_enable | 是否打开profiling打点功能 | false,true | false | 否 |
73-|`ubsocket_ub_handshake_mode` | UB通信建链握手模式 | tfo, ub_sock_opt | ub_sock_opt | 否 |67+|ubsocket_prof_mode | 是否打开profiling百分位统计功能，fast：关闭，ext：打开 | [fast, ext] | fast | 否 |
74-|`ubsocket_flow_control_enable` | 是否开启流控 | false, true | true | 否 |68+|ubsocket_prof_dump_interval_min | 每次统计profiling打点数据时间间隔（单位s）） | [1, 5] | 1 | 否 |
75-|`ubsocket_prof_enable` | 是否打开profiling打点功能 | false,true | false | 否 |69+|ubsocket_prof_dump_file_path | profiling打点数据输出路径 | [1, 512] | /tmp/ubsocket/profiling | 否 |
76-|`ubsocket_prof_mode` | 是否打开profiling百分位统计功能，fast：关闭，ext：打开 | [fast, ext] | fast | 否 |70+|ubsocket_jetty_type | jetty连接复用方式 | single,pool | pool | 否 |
77-|`ubsocket_prof_dump_interval_min` | 每次统计profiling打点数据时间间隔（单位s）） | [1, 5] | 1 | 否 |71+|ubsocket_jetty_pool_size | jetty连接池大小，jetty连接池大小需要占用fd，因此建议ulimit -n 65535 | [1, 1000] | 800 | 否 |
78-|`ubsocket_prof_dump_path` | profiling打点数据输出路径 | [1, 512] | /tmp/ubsocket/profiling | 否 |72+|ubsocket_initial_credit | 首次流控请求所申请的信用个数, 单个 WR 消耗 1 个信用 | [1, 1024] | 16 | 否 |
79-|`ubsocket_tp_type` | jetty连接复用方式 | single,pool | pool | 否 |73+|ubsocket_max_credit_per_request | 流控信用被耗尽后会在前一次的申请个数基础上翻倍申请<br>例如第1次如果申请 16, 被耗尽后会申请 256 信令。但是最多不会超过此值 | [1, 1024] | 256 | 否 |
80-|`ubsocket_tp_pool_size` | jetty连接池大小，jetty连接池大小需要占用fd，因此建议ulimit -n 65535 | [1, 1000] | 800 | 否 |74+|ubsocket_min_reserved_credit | 单个连接最小保留的的流控信用。如果连接所持有的流控信用小于此值，这些信用不会因连接无活动而归还给上层 | [1, 1024] | 2 | 否 |
81-|`ubsocket_initial_credit` | 首次流控请求所申请的信用个数, 单个 WR 消耗 1 个信用 | [1, 1024] | 128 | 否 |75+|ubsocket_split_trace_enable | 是否开启链路级打点工具 | false, true | false |否|
82-|`ubsocket_max_credit_per_request` | 流控信用被耗尽后会在前一次的申请个数基础上翻倍申请<br>例如第1次如果申请 128, 被耗尽后会申请 256 信令。但是最多不会超过此值 | [1, 1024] | 1024 | 否 |76+|ubsocket_split_trace_buf_cap | 链路级打点工具日志打印数组的大小 | [16384, 10240000] | 65535 |否|
83-|`ubsocket_min_reserved_credit` | 单个连接最小保留的的流控信用。如果连接所持有的流控信用小于此值，这些信用不会因连接无活动而归还给上层 | [1, 1024] | 2 | 否 |77+|ubsocket_split_trace_drain_interval_ms | 链路级打点工具日志打印的间隔 | [1, 10000] | 10 |否|
84-|`ubsocket_split_trace_enable` | 是否开启链路级打点工具 | false, true | true |否|78+|ubsocket_split_trace_level | 链路级打点工具级别,默认会打印所有的点位，可以设置只打印ubsocket或者umq的点位信息 | "all", "ubsocket", "umq" | all |否|
85-|`ubsocket_split_trace_buf_cap` | 链路级打点工具日志打印数组的大小 | [16384, 10240000] | 65535 |否|
86-|`ubsocket_split_trace_drain_interval_ms` | 链路级打点工具日志打印的间隔 | [1, 10000] | 10 |否|
87-|`ubsocket_split_trace_level` | 链路级打点工具级别,默认会打印所有的点位，可以设置只打印ubsocket或者umq的点位信息 | "all", "ubsocket", "umq" | all |否|
88 79 
89> 注意：80> 注意：
90>81>
91-> 1. 为最大程度的兼容bPRC及gflags的使用习惯。新增的这些gflags配置项，均在bRPC的源码中指定了默认值。bRPC集成ubsocket的场景中，ubsocket自身的环境变量不再生效，以gflags的默认值或用户指定的gflags值为准。82+> 1. 为最大程度的兼容bPRC及gflags的使用习惯。新增的这些gflags配置项，均在bRPC的源码中指定了默认值。bRPC集成ubsocket的场景中，ubsocket自身的环境变量不再生效，以gflags的默认值或用户指定的gflags值为准；且在该场景下，建议将`ubsocket_enable`设置为`true`。
92> 2. 如果设置ub协议模式`ubsocket_ub_trans_mode`为`CTP`模式，内存池分片`ubsocket_block_type`则应设置为tiny，同时小内存池分片`ubsocket_tiny_pool_block_size`最大设置为`4096`，不能超出分片大小。83> 2. 如果设置ub协议模式`ubsocket_ub_trans_mode`为`CTP`模式，内存池分片`ubsocket_block_type`则应设置为tiny，同时小内存池分片`ubsocket_tiny_pool_block_size`最大设置为`4096`，不能超出分片大小。
93> 3. brpc保留`ubiobuf_tiny_pool_threshold`用于选择tiny pool，小包可由多个tiny block承载。brpc normal UB block申请失败时使用brpc侧malloc作为escape block，发送前由`UBIOBuf::normalize()`拷贝回已注册内存。UMQ tiny TLS深度不再作为brpc对外参数暴露。84> 3. brpc保留`ubiobuf_tiny_pool_threshold`用于选择tiny pool，小包可由多个tiny block承载。brpc normal UB block申请失败时使用brpc侧malloc作为escape block，发送前由`UBIOBuf::normalize()`拷贝回已注册内存。UMQ tiny TLS深度不再作为brpc对外参数暴露。
94> 4. `ubiobuf_tiny_pool_threshold`参数设定值应当小于等于`ubsocket_tiny_pool_block_size`，否则会导致额外tiny block分片，增大性能开销。85> 4. `ubiobuf_tiny_pool_threshold`参数设定值应当小于等于`ubsocket_tiny_pool_block_size`，否则会导致额外tiny block分片，增大性能开销。
95> 5. `ubsocket_ub_handshake_mode`参数不同值的使用限制如下:86> 5. `ubsocket_ub_handshake_mode`参数不同值的使用限制如下:
96 > 1. tfo: 需要运行OS使能TFO选项，参考[TFO选项开启配置](#451-tfo选项开启配置可选)87 > 1. tfo: 需要运行OS使能TFO选项，参考[TFO选项开启配置](#451-tfo选项开启配置可选)
97 > 2. ub_sock_opt: 需要内核支持，查询指令：`cat /boot/{当前内核版本} | grep "CONFIG_UB_SOCKET_HANDSHAKE=y"`88 > 2. ub_sock_opt: 需要内核支持，查询指令：`cat /boot/{当前内核版本} | grep "CONFIG_UB_SOCKET_HANDSHAKE=y"`
98-> 6. `ubsocket_tp_pool_size`参数设定值初始化时需要占用fd，因此需要确保有足够的fd，建议ulimit -n 65535。89+> 6. `ubsocket_jetty_pool_size`参数设定值初始化时需要占用fd，因此需要确保有足够的fd，建议ulimit -n 65535。
99> 7. `ubsocket_pool_initial_size` 最小值参考（单位 MB）：((ubsocket_rx_depth * 4 + ubsocket_buf_pool_depth) * (ubsocket_block_type分片 + 128) + (ubsocket_tiny_pool_block_size + 128) * ubsocket_tiny_buf_pool_depth) / 1024 / 1024。90> 7. `ubsocket_pool_initial_size` 最小值参考（单位 MB）：((ubsocket_rx_depth * 4 + ubsocket_buf_pool_depth) * (ubsocket_block_type分片 + 128) + (ubsocket_tiny_pool_block_size + 128) * ubsocket_tiny_buf_pool_depth) / 1024 / 1024。
91+> 8. 开启debug日志命令 `--v=1`。
92+> 9. 若想要使能链路级打点工具`ubsocket_split_trace_enable`为`true`,则需要先在bazel编译时加上`--copt="-DUBS_SPLIT_TRACE_ENABLE_COMPILE"`。
100### 3.2 配置项93### 3.2 配置项
101在bRPC与ubsocket集成过程中，会增加配置项保证相关资源在使用ubsocket时正确初始化，配置项一般需要client和server侧同时配置生效。94在bRPC与ubsocket集成过程中，会增加配置项保证相关资源在使用ubsocket时正确初始化，配置项一般需要client和server侧同时配置生效。
102 95 
Msrc/brpc/channel.cpp+6-2
@@ -463,7 +463,9 @@ void Channel::CallMethod(const google::protobuf::MethodDescriptor* method,
463#ifdef BRPC_WITH_URMA463#ifdef BRPC_WITH_URMA
464 uint64_t timestamp = butil::cpuwide_time_ns();464 uint64_t timestamp = butil::cpuwide_time_ns();
465 g_brpc_ubs_step_latency[BRPC_CLIENT_CALL] = timestamp;465 g_brpc_ubs_step_latency[BRPC_CLIENT_CALL] = timestamp;
466- bthread_setspecific(ubsocket_trace_call_timestamp, reinterpret_cast<void*>(timestamp));466+ if (FLAGS_ubsocket_enable) {
467+ bthread_setspecific(ubsocket_trace_call_timestamp, reinterpret_cast<void*>(timestamp));
468+ }
467#endif469#endif
468 const int64_t start_send_real_us = butil::gettimeofday_us();470 const int64_t start_send_real_us = butil::gettimeofday_us();
469 Controller* cntl = static_cast<Controller*>(controller_base);471 Controller* cntl = static_cast<Controller*>(controller_base);
@@ -496,7 +498,9 @@ void Channel::CallMethod(const google::protobuf::MethodDescriptor* method,
496 }498 }
497 const CallId correlation_id = cntl->call_id();499 const CallId correlation_id = cntl->call_id();
498#ifdef BRPC_WITH_URMA500#ifdef BRPC_WITH_URMA
499- bthread_setspecific(ubsocket_trace_rpcid_key, reinterpret_cast<void*>(correlation_id.value));501+ if (FLAGS_ubsocket_enable) {
502+ bthread_setspecific(ubsocket_trace_rpcid_key, reinterpret_cast<void*>(correlation_id.value));
503+ }
500#endif504#endif
501 const int rc = bthread_id_lock_and_reset_range(505 const int rc = bthread_id_lock_and_reset_range(
502 correlation_id, NULL, 2 + cntl->max_retry());506 correlation_id, NULL, 2 + cntl->max_retry());
Msrc/brpc/ubsocket_initializer.cpp+125-94
@@ -21,9 +21,11 @@
21#include <cstdint>21#include <cstdint>
22#include <new>22#include <new>
23#include <sys/epoll.h>23#include <sys/epoll.h>
24+#include <pthread.h>
24 25 
25#include <gflags/gflags.h>26#include <gflags/gflags.h>
26#include <string>27#include <string>
28+#include <cstdlib>
27#include "brpc/event_dispatcher.h"29#include "brpc/event_dispatcher.h"
28#include "butil/reloadable_flags.h"30#include "butil/reloadable_flags.h"
29#include "brpc/log.h"31#include "brpc/log.h"
@@ -39,61 +41,51 @@ namespace brpc {
39///////////////////////////////////////////////////////////////////////////////41///////////////////////////////////////////////////////////////////////////////
40// Parses gflags parameters and applies the configuration to UBSocket.42// Parses gflags parameters and applies the configuration to UBSocket.
41///////////////////////////////////////////////////////////////////////////////43///////////////////////////////////////////////////////////////////////////////
42-DEFINE_string(ubsocket_trans_mode, "ub", "Transport mode for ubsocket (e.g., 'ub', 'ib')");
43-DEFINE_string(ubsocket_dev_name, "", "Device name for ubsocket (e.g., 'udma2', 'bonding_dev_0')");
44-DEFINE_string(ubsocket_dev_ip, "", "Device ip for ubsocket");
45-DEFINE_string(ubsocket_eid_idx, "", "Normal device eid idx for ubsocket, necessary while using ub. Obtained by querying with the 'urma_admin show' command");
46-DEFINE_string(ubsocket_src_eid, "", "Bonding device eid idx for ubsocket, necessary while using ub. Obtained by querying with the 'urma_admin show' command");
47-DEFINE_string(ubsocket_log_level, "info", "Log level for ubsocket (e.g., 'error', 'warn', 'notice', 'info', 'debug')");
48DEFINE_string(ubsocket_tx_depth, "1024", "Send queue depth, the minimum value is 2. The upper limit of the setting is determined by the actual machine environment, based on the value of 'max_jfc_depth' in the command 'urma_admin show --whole'.");44DEFINE_string(ubsocket_tx_depth, "1024", "Send queue depth, the minimum value is 2. The upper limit of the setting is determined by the actual machine environment, based on the value of 'max_jfc_depth' in the command 'urma_admin show --whole'.");
49-DEFINE_string(ubsocket_rx_depth, "1024", "Receive queue depth, the minimum value is 2. The upper limit of the setting is determined by the actual machine environment, based on the value of 'max_jfc_depth' in the command 'urma_admin show --whole'.");45+DEFINE_string(ubsocket_rx_depth, "2048", "Receive queue depth, the minimum value is 2. The upper limit of the setting is determined by the actual machine environment, based on the value of 'max_jfc_depth' in the command 'urma_admin show --whole'.");
50-DEFINE_string(ubsocket_block_type, "tiny", "Minimum fragment of the memory pool for ubsocket (e.g., 'tiny'(4K), 'default'(8k), 'small'(16k), 'medium'(32k), 'large'(64k))");46+DEFINE_string(ubsocket_block_type, "default", "Minimum fragment of the memory pool for ubsocket (e.g., 'default'(4k(CTP)/8k(TP)), 'large'(64k))");
51DEFINE_string(ubsocket_pool_initial_size, "200", "Total size of IO memory for ubsocket, in MB");47DEFINE_string(ubsocket_pool_initial_size, "200", "Total size of IO memory for ubsocket, in MB");
52DEFINE_string(ubsocket_pool_max_size, "2048", "Max size of ubsocket pool, in MB");48DEFINE_string(ubsocket_pool_max_size, "2048", "Max size of ubsocket pool, in MB");
53-DEFINE_string(ubsocket_buf_pool_depth, "12000", "Depth of ubsocket buffer pool");49+DEFINE_string(ubsocket_buf_pool_depth, "24576", "Depth of ubsocket buffer pool");
54DEFINE_bool(ubsocket_tiny_pool_enable, true, "Whether to enable the UMQ tiny pool for UBIOBuf");50DEFINE_bool(ubsocket_tiny_pool_enable, true, "Whether to enable the UMQ tiny pool for UBIOBuf");
55DEFINE_uint32(ubsocket_tiny_pool_block_size, 1024, "UMQ tiny pool block size (e.g., 512, 1024, 2048, 4096, 8192)");51DEFINE_uint32(ubsocket_tiny_pool_block_size, 1024, "UMQ tiny pool block size (e.g., 512, 1024, 2048, 4096, 8192)");
56DEFINE_string(ubsocket_tiny_buf_pool_depth, "8192", "UMQ tiny pool block count");52DEFINE_string(ubsocket_tiny_buf_pool_depth, "8192", "UMQ tiny pool block count");
57DEFINE_string(ubsocket_schedule_policy, "affinity_priority", "Set the multi-plane load balancing policy (e.g., 'affinity_priority', 'affinity', 'rr')");53DEFINE_string(ubsocket_schedule_policy, "affinity_priority", "Set the multi-plane load balancing policy (e.g., 'affinity_priority', 'affinity', 'rr')");
58DEFINE_string(ubsocket_readv_unlimited, "true", "Whether to enable the readv reporting limit for ubsocket (e.g., 'false', 'true')");54DEFINE_string(ubsocket_readv_unlimited, "true", "Whether to enable the readv reporting limit for ubsocket (e.g., 'false', 'true')");
59DEFINE_string(ubsocket_use_polling, "false", "Whether to enable message processing polling for ubsocket (e.g., 'false', 'true')");55DEFINE_string(ubsocket_use_polling, "false", "Whether to enable message processing polling for ubsocket (e.g., 'false', 'true')");
60-DEFINE_string(ubsocket_adpt_stats, "false", "Count statistics for ubsocket (e.g., 'false', 'true')");56+DEFINE_string(ubsocket_share_jfr_enable, "true", "Whether to enable share jfr (e.g., 'false', 'true')");
61-DEFINE_string(ubsocket_auto_fallback_tcp, "true", "Whether to automatically downgrade TCP when the protocols do not match (e.g., 'false', 'true')");57+DEFINE_string(ubsocket_share_jfr_loop_poll_enable, "true", "Whether to enable share jfr loop poll (e.g., 'false', 'true')");
62-DEFINE_string(ubsocket_enable_share_jfr, "true", "Whether to enable share jfr (e.g., 'false', 'true')");58+DEFINE_string(ubsocket_monitor_enable, "true", "Enable ubsocket monitor statistics (e.g., 'false', 'true')");
63-DEFINE_string(ubsocket_share_jfr_rx_queue_depth, "1024", "Share jfr receive queue depth, the minimum value is 64. The upper limit of the setting is determined by the actual machine environment.");
64-DEFINE_string(ubsocket_share_jfr_rx_o3_queue_depth, "256", "Share jfr receive out of order queue depth (only available when ubsocket_ub_trans_mode is \"RM_CTP\"), the default value is 256. The upper limit of the setting is equal to ubsocket_share_jfr_rx_queue_depth.");
65-DEFINE_string(ubsocket_trace_enable, "true", "Enable ubsocket trace statistics (e.g., 'false', 'true')");
66DEFINE_string(ubsocket_backup_link_enable, "true", "Enable ubsocket backup link (e.g., 'false', 'true')");59DEFINE_string(ubsocket_backup_link_enable, "true", "Enable ubsocket backup link (e.g., 'false', 'true')");
67-DEFINE_string(ubsocket_trace_time, "10", "Set monitoring ubsocket data output interval, the minimum value is 1, the maximum value is 300");60+DEFINE_string(ubsocket_monitor_interval, "10", "Set monitoring ubsocket data output interval, the minimum value is 1, the maximum value is 300");
68-DEFINE_string(ubsocket_trace_file_path, "/tmp/ubsocket/log", "Set monitoring ubsocket data output path (e.g., '/tmp/ubsocket/log')");61+DEFINE_string(ubsocket_monitor_file_path, "/tmp/ubsocket/log", "Set monitoring ubsocket data output path (e.g., '/tmp/ubsocket/log')");
69-DEFINE_string(ubsocket_trace_file_size, "10", "Set monitoring ubsocket data file size, the minimum value is 1, the maximum value is 300");62+DEFINE_string(ubsocket_monitor_file_size, "10", "Set monitoring ubsocket data file size, the minimum value is 1, the maximum value is 300");
70-DEFINE_string(ubsocket_stats_cli, "true", "Enable ubsocket cli service (e.g., 'false', 'true')");63+DEFINE_string(ubsocket_cli_enable, "true", "Enable ubsocket cli service (e.g., 'false', 'true')");
71DEFINE_string(ubsocket_ub_trans_mode, "RM_CTP", "Protocol mode for ubsocket (e.g., 'RC_TP', 'RM_TP', 'RM_CTP', 'RC_CTP')");64DEFINE_string(ubsocket_ub_trans_mode, "RM_CTP", "Protocol mode for ubsocket (e.g., 'RC_TP', 'RM_TP', 'RM_CTP', 'RC_CTP')");
72-DEFINE_string(ubsocket_initial_credit, "128", "The initial credits requested for a new send operation.");65+DEFINE_string(ubsocket_initial_credit, "16", "The initial credits requested for a new send operation.");
73-DEFINE_string(ubsocket_max_credit_per_request, "1024", "Upper bound for credit in one request.");66+DEFINE_string(ubsocket_max_credit_per_request, "256", "Upper bound for credit in one request.");
74DEFINE_string(ubsocket_min_reserved_credit, "2", "Minimum reserved credit, if the held credit <= min_reserved_credit, the credit will not be returned.");67DEFINE_string(ubsocket_min_reserved_credit, "2", "Minimum reserved credit, if the held credit <= min_reserved_credit, the credit will not be returned.");
75DEFINE_string(ubsocket_link_priority, "4", "Set urma flow service level priority, range from -1 to 15,-1 is not set");68DEFINE_string(ubsocket_link_priority, "4", "Set urma flow service level priority, range from -1 to 15,-1 is not set");
76-DEFINE_string(ubsocket_degrade, "true", "Allow degradation to TCP when UB fails; default: true");69+DEFINE_string(ubsocket_degrade_enable, "true", "Allow degradation to TCP when UB fails; default: true");
77-DEFINE_string(ubsocket_async_accept, "false", "Allow do accept async; default: false");70+DEFINE_string(ubsocket_async_accept_enable, "false", "Allow do accept async; default: false");
78-DEFINE_string(ubsocket_thread_pool_size, "1", "the number of threads in ubsocket thread pool; default: 0");71+DEFINE_string(ubsocket_async_accept_thread_num, "1", "the number of threads in ubsocket thread pool; default: 0");
79-DEFINE_string(ubsocket_async_epoll_wait, "false", "Allow do epoll wait async; default: false");
80DEFINE_string(ubsocket_probe_enable, "false", "Enable ubsocket probe (e.g., 'false', 'true')");72DEFINE_string(ubsocket_probe_enable, "false", "Enable ubsocket probe (e.g., 'false', 'true')");
81-DEFINE_string(ubsocket_probe_time_ms, "1000", "ubsocket probe interval time, the minimum value is 1, the maximum value is 360000");73+DEFINE_string(ubsocket_probe_interval_ms, "1000", "ubsocket probe interval time, the minimum value is 1, the maximum value is 360000");
82-DEFINE_string(ubsocket_probe_batch, "10", "ubsocket number of sock to probe per batch, the minimum value is 1, the maximum value is 500");74+DEFINE_string(ubsocket_probe_batch_size, "10", "ubsocket number of sock to probe per batch, the minimum value is 1, the maximum value is 500");
83-DEFINE_string(ubsocket_ub_epoll_enable, "true", "Whether to enable ub epoll; default: false (optional: false, true)");
84DEFINE_string(ubsocket_prof_enable, "false", "Enable ubsocket profiling (e.g., 'false', 'true')");75DEFINE_string(ubsocket_prof_enable, "false", "Enable ubsocket profiling (e.g., 'false', 'true')");
85DEFINE_string(ubsocket_prof_mode, "fast", "Set ubsocket profiling mode; default: fast (optional: fast, ext)");76DEFINE_string(ubsocket_prof_mode, "fast", "Set ubsocket profiling mode; default: fast (optional: fast, ext)");
86DEFINE_string(ubsocket_prof_dump_interval_min, "1", "Set dump ubsocket profiling data output interval(minute), the minimum value is 1, the maximum value is 5");77DEFINE_string(ubsocket_prof_dump_interval_min, "1", "Set dump ubsocket profiling data output interval(minute), the minimum value is 1, the maximum value is 5");
87-DEFINE_string(ubsocket_prof_dump_path, "/tmp/ubsocket/profiling", "Set dump ubsocket profiling data output path (e.g., '/tmp/ubsocket/profiling')");78+DEFINE_string(ubsocket_prof_dump_file_path, "/tmp/ubsocket/profiling", "Set dump ubsocket profiling data output path (e.g., '/tmp/ubsocket/profiling')");
88DEFINE_string(ubsocket_ub_handshake_mode, "ub_sock_opt", "Handshake mode for UB connection; default: ub_sock_opt (optional: tfo, ub_sock_opt)");79DEFINE_string(ubsocket_ub_handshake_mode, "ub_sock_opt", "Handshake mode for UB connection; default: ub_sock_opt (optional: tfo, ub_sock_opt)");
89DEFINE_string(ubsocket_flow_control_enable, "true", "Whether to enable flow control; default: true (optional: false, true)");80DEFINE_string(ubsocket_flow_control_enable, "true", "Whether to enable flow control; default: true (optional: false, true)");
90DEFINE_string(ubsocket_split_trace_enable, "false", "Enable ubsocket split trace (e.g., 'false', 'true')");81DEFINE_string(ubsocket_split_trace_enable, "false", "Enable ubsocket split trace (e.g., 'false', 'true')");
91DEFINE_string(ubsocket_split_trace_buf_cap, "65535", "Set ubsocket split trace buf capacity; default: 65535, the minimum value is 16384, the maximum value is 10240000");82DEFINE_string(ubsocket_split_trace_buf_cap, "65535", "Set ubsocket split trace buf capacity; default: 65535, the minimum value is 16384, the maximum value is 10240000");
92DEFINE_string(ubsocket_split_trace_drain_interval_ms, "10", "Set ubsocket split trace buf log drain interval(ms), the minimum value is 1, the maximum value is 10000");83DEFINE_string(ubsocket_split_trace_drain_interval_ms, "10", "Set ubsocket split trace buf log drain interval(ms), the minimum value is 1, the maximum value is 10000");
93DEFINE_string(ubsocket_split_trace_level, "all", "Set ubsocket split trace level, default: all, can set to 'ubsocket' 'umq' or 'ubsocket,umq'");84DEFINE_string(ubsocket_split_trace_level, "all", "Set ubsocket split trace level, default: all, can set to 'ubsocket' 'umq' or 'ubsocket,umq'");
94-DEFINE_string(ubsocket_tp_type, "pool", "Ubsocket jetty tranport type; default: single (optional: single, pool)");85+DEFINE_string(ubsocket_jetty_type, "pool", "Ubsocket jetty tranport type; default: pool (optional: single, pool)");
95-DEFINE_string(ubsocket_tp_pool_size, "800", "Ubsocket jetty tranport pool size; the minimum value is 1, the maximum value is 1000");86+DEFINE_string(ubsocket_jetty_pool_size, "800", "Ubsocket jetty tranport pool size; the minimum value is 1, the maximum value is 1000");
96DEFINE_bool(ubsocket_reg_poller, false, "Whether to register brpc poller");87DEFINE_bool(ubsocket_reg_poller, false, "Whether to register brpc poller");
88+DEFINE_bool(ubsocket_tls_ops, false, "Register bthread-local TLS ops for ubsocket; TLS values then follow bthreads across worker threads (default: pthread-local)");
97 89 
98static bool validate_ubsocket_tiny_pool_block_size(const char*, uint32_t value)90static bool validate_ubsocket_tiny_pool_block_size(const char*, uint32_t value)
99{91{
@@ -233,24 +225,6 @@ static void brpc_poller_remove_consumer(void *consumer, int)
233}225}
234 226 
235static void SetUBSocketEnv() {227static void SetUBSocketEnv() {
236- if (!FLAGS_ubsocket_trans_mode.empty()) {
237- ::setenv("UBSOCKET_TRANS_MODE", FLAGS_ubsocket_trans_mode.c_str(), 1);
238- }
239- if (!FLAGS_ubsocket_dev_name.empty()) {
240- ::setenv("UBSOCKET_DEV_NAME", FLAGS_ubsocket_dev_name.c_str(), 1);
241- }
242- if (!FLAGS_ubsocket_dev_ip.empty()) {
243- ::setenv("UBSOCKET_DEV_IP", FLAGS_ubsocket_dev_ip.c_str(), 1);
244- }
245- if (!FLAGS_ubsocket_eid_idx.empty()) {
246- ::setenv("UBSOCKET_EID_IDX", FLAGS_ubsocket_eid_idx.c_str(), 1);
247- }
248- if (!FLAGS_ubsocket_src_eid.empty()) {
249- ::setenv("UBSOCKET_SRC_EID", FLAGS_ubsocket_src_eid.c_str(), 1);
250- }
251- if (!FLAGS_ubsocket_log_level.empty()) {
252- ::setenv("UBSOCKET_LOG_LEVEL", FLAGS_ubsocket_log_level.c_str(), 1);
253- }
254 if (!FLAGS_ubsocket_tx_depth.empty()) {228 if (!FLAGS_ubsocket_tx_depth.empty()) {
255 ::setenv("UBSOCKET_TX_DEPTH", FLAGS_ubsocket_tx_depth.c_str(), 1);229 ::setenv("UBSOCKET_TX_DEPTH", FLAGS_ubsocket_tx_depth.c_str(), 1);
256 }230 }
@@ -284,38 +258,29 @@ static void SetUBSocketEnv() {
284 if (!FLAGS_ubsocket_use_polling.empty()) {258 if (!FLAGS_ubsocket_use_polling.empty()) {
285 ::setenv("UBSOCKET_USE_POLLING", FLAGS_ubsocket_use_polling.c_str(), 1);259 ::setenv("UBSOCKET_USE_POLLING", FLAGS_ubsocket_use_polling.c_str(), 1);
286 }260 }
287- if (!FLAGS_ubsocket_adpt_stats.empty()) {261+ if (!FLAGS_ubsocket_share_jfr_enable.empty()) {
288- ::setenv("UBSOCKET_STATS_CLI", FLAGS_ubsocket_adpt_stats.c_str(), 1);262+ ::setenv("UBSOCKET_SHARE_JFR_ENABLE", FLAGS_ubsocket_share_jfr_enable.c_str(), 1);
289 }263 }
290- if (!FLAGS_ubsocket_auto_fallback_tcp.empty()) {264+ if (!FLAGS_ubsocket_share_jfr_loop_poll_enable.empty()) {
291- ::setenv("UBSOCKET_AUTO_FALLBACK_TCP", FLAGS_ubsocket_auto_fallback_tcp.c_str(), 1);265+ ::setenv("UBSOCKET_SHARE_JFR_LOOP_POLL_ENABLED", FLAGS_ubsocket_share_jfr_loop_poll_enable.c_str(), 1);
292 }266 }
293- if (!FLAGS_ubsocket_enable_share_jfr.empty()) {267+ if (!FLAGS_ubsocket_monitor_enable.empty()) {
294- ::setenv("UBSOCKET_ENABLE_SHARE_JFR", FLAGS_ubsocket_enable_share_jfr.c_str(), 1);268+ ::setenv("UBSOCKET_MONITOR_ENABLE", FLAGS_ubsocket_monitor_enable.c_str(), 1);
295- }
296- if (!FLAGS_ubsocket_share_jfr_rx_queue_depth.empty()) {
297- ::setenv("UBSOCKET_SHARE_JFR_RX_QUEUE_DEPTH", FLAGS_ubsocket_share_jfr_rx_queue_depth.c_str(), 1);
298- }
299- if (!FLAGS_ubsocket_share_jfr_rx_o3_queue_depth.empty()) {
300- ::setenv("UBSOCKET_SHARE_JFR_RX_O3_QUEUE_DEPTH", FLAGS_ubsocket_share_jfr_rx_o3_queue_depth.c_str(), 1);
301- }
302- if (!FLAGS_ubsocket_trace_enable.empty()) {
303- ::setenv("UBSOCKET_TRACE_ENABLE", FLAGS_ubsocket_trace_enable.c_str(), 1);
304 }269 }
305 if (!FLAGS_ubsocket_backup_link_enable.empty()) {270 if (!FLAGS_ubsocket_backup_link_enable.empty()) {
306 ::setenv("UBSOCKET_BACKUP_LINK_ENABLE", FLAGS_ubsocket_backup_link_enable.c_str(), 1);271 ::setenv("UBSOCKET_BACKUP_LINK_ENABLE", FLAGS_ubsocket_backup_link_enable.c_str(), 1);
307 }272 }
308- if (!FLAGS_ubsocket_trace_time.empty()) {273+ if (!FLAGS_ubsocket_monitor_interval.empty()) {
309- ::setenv("UBSOCKET_TRACE_TIME", FLAGS_ubsocket_trace_time.c_str(), 1);274+ ::setenv("UBSOCKET_MONITOR_INTERVAL", FLAGS_ubsocket_monitor_interval.c_str(), 1);
310 }275 }
311- if (!FLAGS_ubsocket_trace_file_path.empty()) {276+ if (!FLAGS_ubsocket_monitor_file_path.empty()) {
312- ::setenv("UBSOCKET_TRACE_FILE_PATH", FLAGS_ubsocket_trace_file_path.c_str(), 1);277+ ::setenv("UBSOCKET_MONITOR_FILE_PATH", FLAGS_ubsocket_monitor_file_path.c_str(), 1);
313 }278 }
314- if (!FLAGS_ubsocket_trace_file_size.empty()) {279+ if (!FLAGS_ubsocket_monitor_file_size.empty()) {
315- ::setenv("UBSOCKET_TRACE_FILE_SIZE", FLAGS_ubsocket_trace_file_size.c_str(), 1);280+ ::setenv("UBSOCKET_MONITOR_FILE_SIZE", FLAGS_ubsocket_monitor_file_size.c_str(), 1);
316 }281 }
317- if (!FLAGS_ubsocket_stats_cli.empty()) {282+ if (!FLAGS_ubsocket_cli_enable.empty()) {
318- ::setenv("UBSOCKET_STATS_CLI", FLAGS_ubsocket_stats_cli.c_str(), 1);283+ ::setenv("UBSOCKET_CLI_ENABLE", FLAGS_ubsocket_cli_enable.c_str(), 1);
319 }284 }
320 if (!FLAGS_ubsocket_ub_trans_mode.empty()) {285 if (!FLAGS_ubsocket_ub_trans_mode.empty()) {
321 ::setenv("UBSOCKET_UB_TRANS_MODE", FLAGS_ubsocket_ub_trans_mode.c_str(), 1);286 ::setenv("UBSOCKET_UB_TRANS_MODE", FLAGS_ubsocket_ub_trans_mode.c_str(), 1);
@@ -332,30 +297,24 @@ static void SetUBSocketEnv() {
332 if (!FLAGS_ubsocket_link_priority.empty()) {297 if (!FLAGS_ubsocket_link_priority.empty()) {
333 ::setenv("UBSOCKET_LINK_PRIORITY", FLAGS_ubsocket_link_priority.c_str(), 1);298 ::setenv("UBSOCKET_LINK_PRIORITY", FLAGS_ubsocket_link_priority.c_str(), 1);
334 }299 }
335- if (!FLAGS_ubsocket_degrade.empty()) {300+ if (!FLAGS_ubsocket_degrade_enable.empty()) {
336- ::setenv("UBSOCKET_DEGRADE", FLAGS_ubsocket_degrade.c_str(), 1);301+ ::setenv("UBSOCKET_DEGRADE_ENABLE", FLAGS_ubsocket_degrade_enable.c_str(), 1);
337 }302 }
338- if (!FLAGS_ubsocket_async_accept.empty()) {303+ if (!FLAGS_ubsocket_async_accept_enable.empty()) {
339- ::setenv("UBSOCKET_ASYNC_ACCEPT", FLAGS_ubsocket_async_accept.c_str(), 1);304+ ::setenv("UBSOCKET_ASYNC_ACCEPT_ENABLE", FLAGS_ubsocket_async_accept_enable.c_str(), 1);
340 }305 }
341- if (!FLAGS_ubsocket_thread_pool_size.empty()) {306+ if (!FLAGS_ubsocket_async_accept_thread_num.empty()) {
342- ::setenv("UBSOCKET_THREAD_POOL_SIZE", FLAGS_ubsocket_thread_pool_size.c_str(), 1);307+ ::setenv("UBSOCKET_ASYNC_ACCEPT_THREAD_NUM", FLAGS_ubsocket_async_accept_thread_num.c_str(), 1);
343- }
344- if (!FLAGS_ubsocket_async_epoll_wait.empty()) {
345- ::setenv("UBSOCKET_ASYNC_EPOLL_WAIT", FLAGS_ubsocket_async_epoll_wait.c_str(), 1);
346 }308 }
347 if (!FLAGS_ubsocket_probe_enable.empty()) {309 if (!FLAGS_ubsocket_probe_enable.empty()) {
348 ::setenv("UBSOCKET_PROBE_ENABLE", FLAGS_ubsocket_probe_enable.c_str(), 1);310 ::setenv("UBSOCKET_PROBE_ENABLE", FLAGS_ubsocket_probe_enable.c_str(), 1);
349 }311 }
350- if (!FLAGS_ubsocket_probe_time_ms.empty()) {312+ if (!FLAGS_ubsocket_probe_interval_ms.empty()) {
351- ::setenv("UBSOCKET_PROBE_TIME_MS", FLAGS_ubsocket_probe_time_ms.c_str(), 1);313+ ::setenv("UBSOCKET_PROBE_INTERVAL_MS", FLAGS_ubsocket_probe_interval_ms.c_str(), 1);
352 }314 }
353- if (!FLAGS_ubsocket_probe_batch.empty()) {315+ if (!FLAGS_ubsocket_probe_batch_size.empty()) {
354- ::setenv("UBSOCKET_PROBE_BATCH", FLAGS_ubsocket_probe_batch.c_str(), 1);316+ ::setenv("UBSOCKET_PROBE_BATCH_SIZE", FLAGS_ubsocket_probe_batch_size.c_str(), 1);
355 }317 }
356- if (!FLAGS_ubsocket_ub_epoll_enable.empty()) {
357- ::setenv("UBSOCKET_UB_EPOLL_ENABLE", FLAGS_ubsocket_ub_epoll_enable.c_str(), 1);
358- }
359 if (!FLAGS_ubsocket_flow_control_enable.empty()) {318 if (!FLAGS_ubsocket_flow_control_enable.empty()) {
360 ::setenv("UBSOCKET_FLOW_CONTROL_ENABLE", FLAGS_ubsocket_flow_control_enable.c_str(), 1);319 ::setenv("UBSOCKET_FLOW_CONTROL_ENABLE", FLAGS_ubsocket_flow_control_enable.c_str(), 1);
361 }320 }
@@ -383,14 +342,14 @@ static void SetUBSocketEnv() {
383 if (!FLAGS_ubsocket_split_trace_level.empty()) {342 if (!FLAGS_ubsocket_split_trace_level.empty()) {
384 ::setenv("UBSOCKET_SPLIT_TRACE_LEVEL", FLAGS_ubsocket_split_trace_level.c_str(), 1);343 ::setenv("UBSOCKET_SPLIT_TRACE_LEVEL", FLAGS_ubsocket_split_trace_level.c_str(), 1);
385 }344 }
386- if (!FLAGS_ubsocket_prof_dump_path.empty()) {345+ if (!FLAGS_ubsocket_prof_dump_file_path.empty()) {
387- ::setenv("UBSOCKET_PROF_DUMP_PATH", FLAGS_ubsocket_prof_dump_path.c_str(), 1);346+ ::setenv("UBSOCKET_PROF_DUMP_FILE_PATH", FLAGS_ubsocket_prof_dump_file_path.c_str(), 1);
388 }347 }
389- if (!FLAGS_ubsocket_tp_type.empty()) {348+ if (!FLAGS_ubsocket_jetty_type.empty()) {
390- ::setenv("UBSOCKET_TP_TYPE", FLAGS_ubsocket_tp_type.c_str(), 1);349+ ::setenv("UBSOCKET_JETTY_TYPE", FLAGS_ubsocket_jetty_type.c_str(), 1);
391 }350 }
392- if (!FLAGS_ubsocket_tp_pool_size.empty()) {351+ if (!FLAGS_ubsocket_jetty_pool_size.empty()) {
393- ::setenv("UBSOCKET_TP_POOL_SIZE", FLAGS_ubsocket_tp_pool_size.c_str(), 1);352+ ::setenv("UBSOCKET_JETTY_POOL_SIZE", FLAGS_ubsocket_jetty_pool_size.c_str(), 1);
394 }353 }
395 ::setenv("UBSOCKET_USE_UB_FORCE", "false", 1);354 ::setenv("UBSOCKET_USE_UB_FORCE", "false", 1);
396}355}
@@ -580,6 +539,58 @@ static void* brpc_get_call_timestamp()
580 return bthread_getspecific(ubsocket_trace_call_timestamp);539 return bthread_getspecific(ubsocket_trace_call_timestamp);
581}540}
582 541 
542+ // Initialize TLS for UBSocket. When registered (--ubsocket_tls_ops=true) the
543+ // values are bthread-local: they follow the bthread across worker pthread
544+ // migration and are reclaimed by the key destructor on bthread exit. Calls
545+ // from plain pthreads (e.g. ubsocket internal poller threads) also work --
546+ // bthread keys fall back to the calling thread's own keytable there.
547+static u_tls_key_t *brpc_tls_create(void (*destructor)(void* data))
548+{
549+ bthread_key_t *bkey = (bthread_key_t *)calloc(1, sizeof(bthread_key_t));
550+ if (bkey == NULL) {
551+ LOG(ERROR) << "Error to execute tls_create: calloc failed";
552+ return NULL;
553+ }
554+ if (bthread_key_create(bkey, destructor) != 0) {
555+ LOG(ERROR) << "Error to execute tls_create: bthread_key_create failed";
556+ free(bkey);
557+ return NULL;
558+ }
559+ return (u_tls_key_t *)bkey;
560+}
561+ 
562+static int brpc_tls_destroy(u_tls_key_t *key)
563+{
564+ if (key == nullptr) {
565+ LOG(ERROR) << "Error to execute tls_destroy for key is NULL";
566+ return -1;
567+ }
568+ bthread_key_t bkey = *((bthread_key_t *)key);
569+ (void)bthread_key_delete(bkey);
570+ free(key);
571+ return 0;
572+}
573+ 
574+static int brpc_tls_set(u_tls_key_t *key, const void* value)
575+{
576+ if (key == nullptr) {
577+ LOG(ERROR) << "Error to execute tls_set for key is NULL";
578+ return -1;
579+ }
580+ bthread_key_t bkey = *((bthread_key_t *)key);
581+ return bthread_setspecific(bkey, const_cast<void*>(value));
582+}
583+ 
584+static void* brpc_tls_get(u_tls_key_t *key)
585+{
586+ if (key == nullptr) {
587+ LOG(ERROR) << "Error to execute tls_get for key is NULL";
588+ return nullptr;
589+ }
590+ bthread_key_t bkey = *((bthread_key_t *)key);
591+ return bthread_getspecific(bkey);
592+}
593+ 
583u_external_lock_ops_t brpc_external_lock_ops = {594u_external_lock_ops_t brpc_external_lock_ops = {
584 .create = brpc_external_lock_create,595 .create = brpc_external_lock_create,
585 .destroy = brpc_external_lock_destroy,596 .destroy = brpc_external_lock_destroy,
@@ -615,6 +626,13 @@ u_external_poller_ops_t brpc_poller_ops = {
615 .add_consumer = brpc_poller_add_consumer,626 .add_consumer = brpc_poller_add_consumer,
616 .remove_consumer = brpc_poller_remove_consumer627 .remove_consumer = brpc_poller_remove_consumer
617};628};
629+ 
630+u_external_tls_ops_t brpc_tls_ops = {
631+ .create = brpc_tls_create,
632+ .destroy = brpc_tls_destroy,
633+ .set = brpc_tls_set,
634+ .get = brpc_tls_get,
635+};
618///////////////////////////////////////////////////////////////////////////////636///////////////////////////////////////////////////////////////////////////////
619// Register brpc log for UBSocket.637// Register brpc log for UBSocket.
620///////////////////////////////////////////////////////////////////////////////638///////////////////////////////////////////////////////////////////////////////
@@ -679,6 +697,16 @@ int InitializeUBSocket()
679 return -1;697 return -1;
680 }698 }
681 699 
700+ // Create the bthread-local keys used to pass rpc id / call timestamp into
701+ // ubsocket trace (consumed via brpc_rpc_id_ops). Must be real keys from
702+ // bthread_key_create -- hardcoded {index, version} pairs alias slots handed
703+ // out to other modules and break silently once versions are bumped.
704+ if (bthread_key_create(&ubsocket_trace_rpcid_key, nullptr) != 0 ||
705+ bthread_key_create(&ubsocket_trace_call_timestamp, nullptr) != 0) {
706+ LOG(ERROR) << "Fail to create ubsocket trace bthread keys";
707+ return -1;
708+ }
709+ 
682 SetUBSocketEnv();710 SetUBSocketEnv();
683 711 
684 // 1. 获取基础日志级别712 // 1. 获取基础日志级别
@@ -723,6 +751,9 @@ int InitializeUBSocket()
723 options.rw_lock_ops = &brpc_rw_lock_ops;751 options.rw_lock_ops = &brpc_rw_lock_ops;
724 options.sem_ops = &brpc_semaphore_ops;752 options.sem_ops = &brpc_semaphore_ops;
725 options.rpc_id_ops = &brpc_rpc_id_ops;753 options.rpc_id_ops = &brpc_rpc_id_ops;
754+ if (FLAGS_ubsocket_tls_ops) {
755+ options.tls_ops = &brpc_tls_ops;
756+ }
726 if (FLAGS_ubsocket_reg_poller) {757 if (FLAGS_ubsocket_reg_poller) {
727 options.poller_ops = &brpc_poller_ops;758 options.poller_ops = &brpc_poller_ops;
728 }759 }
Msrc/bthread/bthread.cpp+6-2
@@ -32,8 +32,12 @@
32#include "bthread/bthread.h"32#include "bthread/bthread.h"
33 33 
34#ifdef BRPC_WITH_URMA34#ifdef BRPC_WITH_URMA
35-bthread_key_t ubsocket_trace_rpcid_key{0, 0};35+// Created via bthread_key_create in brpc::InitializeUBSocket(). Initialized to
36-bthread_key_t ubsocket_trace_call_timestamp{1, 0};36+// an out-of-range sentinel so that accidental use before creation fails fast
37+// in KeyTable bounds check instead of aliasing slot 0/1 with legally created
38+// keys (version 0 is the "never created" sentinel of bthread_key_create2).
39+bthread_key_t ubsocket_trace_rpcid_key{(uint32_t)-1, 0};
40+bthread_key_t ubsocket_trace_call_timestamp{(uint32_t)-1, 0};
37#endif41#endif
38namespace bthread {42namespace bthread {
39 43 
Msrc/butil/ub/ubiobuf.cpp+8-10
@@ -29,6 +29,7 @@ namespace brpc {
29 DECLARE_string(ubsocket_block_type);29 DECLARE_string(ubsocket_block_type);
30 DECLARE_bool(ubsocket_tiny_pool_enable);30 DECLARE_bool(ubsocket_tiny_pool_enable);
31 DECLARE_uint32(ubsocket_tiny_pool_block_size);31 DECLARE_uint32(ubsocket_tiny_pool_block_size);
32+ DECLARE_string(ubsocket_ub_trans_mode);
32} // namespace brpc33} // namespace brpc
33 34 
34namespace butil {35namespace butil {
@@ -877,18 +878,15 @@ bool UBIOBuf::has_ub_block(const IOBuf* buf) {
877}878}
878 879 
879size_t UBIOBuf::get_block_size() {880size_t UBIOBuf::get_block_size() {
880- if (brpc::FLAGS_ubsocket_block_type == "tiny") {881+ if (brpc::FLAGS_ubsocket_block_type == "large") {
881- return 4UL * 1024;
882- } else if (brpc::FLAGS_ubsocket_block_type == "default") {
883- return UBIOBuf::DEFAULT_BLOCK_SIZE;
884- } else if (brpc::FLAGS_ubsocket_block_type == "small") {
885- return 16UL * 1024;
886- } else if (brpc::FLAGS_ubsocket_block_type == "medium") {
887- return 32UL * 1024;
888- } else if (brpc::FLAGS_ubsocket_block_type == "large") {
889 return 64UL * 1024;882 return 64UL * 1024;
890 }883 }
891- LOG(WARNING) << "Unknown ubsocket_block_type: " << brpc::FLAGS_ubsocket_block_type << ", use the default IOBuf BLOCK_SIZE";884+ if (brpc::FLAGS_ubsocket_block_type != "default") {
885+ LOG(WARNING) << "Unknown ubsocket_block_type: " << brpc::FLAGS_ubsocket_block_type << ", use the default IOBuf BLOCK_SIZE";
886+ }
887+ if (brpc::FLAGS_ubsocket_ub_trans_mode == "RM_CTP" || brpc::FLAGS_ubsocket_ub_trans_mode == "RC_CTP") {
888+ return 4UL * 1024;
889+ }
892 return UBIOBuf::DEFAULT_BLOCK_SIZE;890 return UBIOBuf::DEFAULT_BLOCK_SIZE;
893}891}
894 892