/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "session_id" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "request_id" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "priority" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "payload" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
2026-05-07 19:44:24 - INFO - openjiuwen_runtime.management.session.timer - timer.py:20 - init - Timer initialized
2026-05-07 19:44:24 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:76 - init - ServiceManager 已 init: sc=10 min_idle=0 max=1 service_ttl(in_use 全空→idle 等待秒数; 入 idle 后超 min 立即回收, 不二次等待)=10
2026-05-07 19:44:24 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:95 - start - ServiceManager 已启动, 预拉热完成, 当前实例数=0
2026-05-07 19:44:24 - INFO - openjiuwen_runtime.management.session.access - access.py:90 - init - Access 已初始化: user_q=1000 sys_q=100 image=swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1 session_max=1 session_ttl=0 service_concurrency=10 min_idle=0 max=1 port=18092 path= ws_tls=False service_ttl=10
sessions=[('A', 10, 1, 1), ('B', 20, 1, 1), ('C', 30, 1, 1)]
sum_conc=3 service_concurrency=10
max_ttl=30s service_ttl=10s buffer=30s → 全部完成后等待 70s
建议另开终端: kubectl get pods -n default -w | findstr jiuwenclaw
[19:44:24.470] >>> 开始执行模式：SERIAL，共 3 个 Session，3 条消息
[19:44:24.470] === START session sid=A ttl=10s msgs=1 conc=1 ===
[19:44:24.470 + 0.00s] >>> SEND sid=A req#0 rid=req_A_0_56907c
2026-05-07 19:44:24 - INFO - openjiuwen_runtime.management.session.k8s_service_handler - k8s_service_handler.py:211 - deploy - K8s 创建 Pod: name=jiuwenclaw-l7pl910vm9-b6zfg namespace=default image=swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.k8s_service_handler - k8s_service_handler.py:245 - deploy - Pod ready: name=jiuwenclaw-l7pl910vm9-b6zfg, pod_ip=192.168.6.144, node=dggphis259576
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.service_handler - service_handler.py:224 - deploy - ServiceHandler deploy 完成: service_id=1605e727-4626-41b2-b050-133b32214183 pod/资源已就绪
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:146 - on_pod_ready - WSS 基址已就绪: service_id=1605e727-4626-41b2-b050-133b32214183 url=ws://192.168.6.144:18092
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:188 - _ensure_connected - WSS 正在连接: ws://192.168.6.144:18092
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:204 - _ensure_connected - WSS 已连接并启动接收循环: ws://192.168.6.144:18092
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:148 - on_pod_ready - WSS 已在 deploy 后完成预建链: service_id=1605e727-4626-41b2-b050-133b32214183
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:656 - _pick_or_create - 新建实例并入 in_use: service_id=1605e727-4626-41b2-b050-133b32214183 gen=0 当前总数=1
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:342 - _handle_user_request - session 已绑定到实例: session_id=A -> service_id=1605e727-4626-41b2-b050-133b32214183
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.service_handler - service_handler.py:125 - handle_message - 新 session 子处理器: service_id=1605e727-4626-41b2-b050-133b32214183 session_id=A max_parallel=1
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:277 - send - WSS 业务上行: service_id=1605e727-4626-41b2-b050-133b32214183 request_id=req_A_0_56907c
2026-05-07 19:44:50 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:324 - _decode_ws_message - WSS 下行 JSON
2026-05-07 19:45:40 - ERROR - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:220 - _recv_loop - WSS 接收已断开: sent 1011 (internal error) keepalive ping timeout; no close frame received
TimeoutError: timed out while closing connection
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/runtime/agent-runtime/management/openjiuwen_runtime/management/session/ws_client_channel.py", line 218, in _recv_loop
raw = await w.recv()
^^^^^^^^^^^^^^
File "/root/runtime/agent-runtime/management/.venv/lib/python3.14/site-packages/websockets/asyncio/connection.py", line 324, in recv
raise self.protocol.close_exc from self.recv_exc
websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
^C2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.timer - timer.py:29 - start_timer - 计时器已启动: key=sess:A ttl=10秒, 活动计时器数=1
2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:386 - _arm_session_timer - 已 arm session TTL 计时: session_id=A ttl=10
2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.dual_queue - dual_queue.py:35 - mark_closed - 双队列已标记关闭, 不再接受入队
2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:107 - stop - ServiceManager 正在停止: 已标记队列关闭, 在途用户路由任务数=0
2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.timer - timer.py:68 - stop_all - 停止全部计时器, 数量=1
2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.timer - timer.py:71 - stop_all - 全部计时器已停止
2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:170 - close - WSServiceMessageChannel 已关闭: last_service_id=1605e727-4626-41b2-b050-133b32214183
2026-05-07 19:46:36 - INFO - openjiuwen_runtime.management.session.k8s_service_handler - k8s_service_handler.py:324 - delete - Deleting pod: name=jiuwenclaw-l7pl910vm9-b6zfg, namespace=default
^CTraceback (most recent call last):
File "/root/runtime/agent-runtime/management/tests/system_tests/management_session/main_k8s_multi_session.py", line 567, in
main()
~~~~^^
File "/root/runtime/agent-runtime/management/tests/system_tests/management_session/main_k8s_multi_session.py", line 563, in main
raise SystemExit(asyncio.run(_amain()))
~~~~~~~~~~~^^^^^^^^^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/runners.py", line 204, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/runners.py", line 127, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/base_events.py", line 706, in run_until_complete
self.run_forever()
~~~~~~~~~~~~~~~~^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/base_events.py", line 677, in run_forever
self.run_once()
~~~~~~~~~~~~~~^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/base_events.py", line 2019, in run_once
event_list = self.selector.select(timeout)
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/selectors.py", line 452, in select
fd_event_list = self.selector.poll(timeout, max_ev)
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/runners.py", line 166, in on_sigint
raise KeyboardInterrupt()
KeyboardInterrupt
[root@dggphis259584 management]# pwd
/root/runtime/agent-runtime/management
[root@dggphis259584 management]# ^C
[root@dggphis259584 management]# uv run python tests/system_tests/management_session/main_k8s_multi_session.py --image swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1 --namespace default --name-prefix jiuwenclaw --container-port 18092 --ws-tls 0 --min-idle 0 --max-services 1 --service-concurrency 10 --service-ttl 10 --session-run-mode serial --message-timeout 300 --env MODEL_PROVIDER=OpenAI --env MODEL_NAME=Qwen/Qwen3-32B --env API_BASE=https://dashscope.aliyuncs.com/compatible-mode/v1 --env API_KEY=sk-ba717df7ac2d418bb4c4c762dd844c36 --session "id=A,ttl=10,msgs=1,conc=1" --session "id=B,ttl=20,msgs=1,conc=1" --session "id=C,ttl=30,msgs=1,conc=1" --message-template-json '{"chat_id":"chat{sid}","bot_id":"bot{sid}","user_id":"u{sid}","session_id":"sess{sid}","channel_id":"web","req_method":"chat.send","params":{"session_id":"sess{sid}","content":"hello","mode":"agent","query":"1+1等于几"},"is_stream":true,"metadata":{"query":{},"method":"chat.send"}}'
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "session_id" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "request_id" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "priority" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "payload" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.timer - timer.py:20 - init - Timer initialized
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:76 - init - ServiceManager 已 init: sc=10 min_idle=0 max=1 service_ttl(in_use 全空→idle 等待秒数; 入 idle 后超 min 立即回收, 不二次等待)=10
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:95 - start - ServiceManager 已启动, 预拉热完成, 当前实例数=0
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.access - access.py:90 - init - Access 已初始化: user_q=1000 sys_q=100 image=swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1 session_max=1 session_ttl=0 service_concurrency=10 min_idle=0 max=1 port=18092 path= ws_tls=False service_ttl=10
main()
~~~~^^
File "/root/runtime/agent-runtime/management/tests/system_tests/management_session/main_k8s_multi_session.py", line 563, in main
raise SystemExit(asyncio.run(_amain()))
~~~~~~~~~~~^^^^^^^^^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/runners.py", line 204, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/runners.py", line 127, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/base_events.py", line 706, in run_until_complete
self.run_forever()
~~~~~~~~~~~~~~~~^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/base_events.py", line 677, in run_forever
self.run_once()
~~~~~~~~~~~~~~^^
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/base_events.py", line 2019, in run_once
event_list = self.selector.select(timeout)
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/selectors.py", line 452, in select
fd_event_list = self.selector.poll(timeout, max_ev)
File "/root/.local/share/uv/python/cpython-3.14.4-linux-x86_64-gnu/lib/python3.14/asyncio/runners.py", line 166, in on_sigint
raise KeyboardInterrupt()
KeyboardInterrupt
[root@dggphis259584 management]# pwd
/root/runtime/agent-runtime/management
[root@dggphis259584 management]# ^C
[root@dggphis259584 management]# uv run python tests/system_tests/management_session/main_k8s_multi_session.py --image swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1 --namespace default --name-prefix jiuwenclaw --container-port 18092 --ws-tls 0 --min-idle 0 --max-services 1 --service-concurrency 10 --service-ttl 10 --session-run-mode serial --message-timeout 300 --env MODEL_PROVIDER=OpenAI --env MODEL_NAME=Qwen/Qwen3-32B --env API_BASE=https://dashscope.aliyuncs.com/compatible-mode/v1 --env API_KEY=sk-ba717df7ac2d418bb4c4c762dd844c36 --session "id=A,ttl=10,msgs=1,conc=1" --session "id=B,ttl=20,msgs=1,conc=1" --session "id=C,ttl=30,msgs=1,conc=1" --message-template-json '{"chat_id":"chat{sid}","bot_id":"bot{sid}","user_id":"u{sid}","session_id":"sess{sid}","channel_id":"web","req_method":"chat.send","params":{"session_id":"sess{sid}","content":"hello","mode":"agent","query":"1+1等于几"},"is_stream":true,"metadata":{"query":{},"method":"chat.send"}}'
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "session_id" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "request_id" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "priority" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
/root/runtime/agent-runtime/management/openjiuwen_runtime/management/orchestrator/models.py:35: UserWarning: Field name "payload" in "Message" shadows an attribute in parent "IMessage"
class Message(BaseModel, IMessage):
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.timer - timer.py:20 - init - Timer initialized
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:76 - init - ServiceManager 已 init: sc=10 min_idle=0 max=1 service_ttl(in_use 全空→idle 等待秒数; 入 idle 后超 min 立即回收, 不二次等待)=10
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:95 - start - ServiceManager 已启动, 预拉热完成, 当前实例数=0
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.access - access.py:90 - init - Access 已初始化: user_q=1000 sys_q=100 image=swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1 session_max=1 session_ttl=0 service_concurrency=10 min_idle=0 max=1 port=18092 path= ws_tls=False service_ttl=10
sessions=[('A', 10, 1, 1), ('B', 20, 1, 1), ('C', 30, 1, 1)]
sum_conc=3 service_concurrency=10
max_ttl=30s service_ttl=10s buffer=30s → 全部完成后等待 70s
建议另开终端: kubectl get pods -n default -w | findstr jiuwenclaw
[19:47:05.889] >>> 开始执行模式：SERIAL，共 3 个 Session，3 条消息
[19:47:05.889] === START session sid=A ttl=10s msgs=1 conc=1 ===
[19:47:05.889 + 0.00s] >>> SEND sid=A req#0 rid=req_A_0_1e30ce
2026-05-07 19:47:05 - INFO - openjiuwen_runtime.management.session.k8s_service_handler - k8s_service_handler.py:211 - deploy - K8s 创建 Pod: name=jiuwenclaw-lxh7zeweam-onrcc namespace=default image=swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.k8s_service_handler - k8s_service_handler.py:245 - deploy - Pod ready: name=jiuwenclaw-lxh7zeweam-onrcc, pod_ip=192.168.10.69, node=dggphis259572
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.service_handler - service_handler.py:224 - deploy - ServiceHandler deploy 完成: service_id=3bb137d2-2728-4bbc-a102-e155909c8a36 pod/资源已就绪
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:146 - on_pod_ready - WSS 基址已就绪: service_id=3bb137d2-2728-4bbc-a102-e155909c8a36 url=ws://192.168.10.69:18092
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:188 - _ensure_connected - WSS 正在连接: ws://192.168.10.69:18092
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:204 - _ensure_connected - WSS 已连接并启动接收循环: ws://192.168.10.69:18092
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:148 - on_pod_ready - WSS 已在 deploy 后完成预建链: service_id=3bb137d2-2728-4bbc-a102-e155909c8a36
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:656 - _pick_or_create - 新建实例并入 in_use: service_id=3bb137d2-2728-4bbc-a102-e155909c8a36 gen=0 当前总数=1
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.service_manager - service_manager.py:342 - _handle_user_request - session 已绑定到实例: session_id=A -> service_id=3bb137d2-2728-4bbc-a102-e155909c8a36
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.service_handler - service_handler.py:125 - handle_message - 新 session 子处理器: service_id=3bb137d2-2728-4bbc-a102-e155909c8a36 session_id=A max_parallel=1
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:277 - send - WSS 业务上行: service_id=3bb137d2-2728-4bbc-a102-e155909c8a36 request_id=req_A_0_1e30ce
2026-05-07 19:47:21 - INFO - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:324 - _decode_ws_message - WSS 下行 JSON
2026-05-07 19:48:11 - ERROR - openjiuwen_runtime.management.session.ws_client_channel - ws_client_channel.py:220 - _recv_loop - WSS 接收已断开: sent 1011 (internal error) keepalive ping timeout; no close frame received
TimeoutError: timed out while closing connection
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/runtime/agent-runtime/management/openjiuwen_runtime/management/session/ws_client_channel.py", line 218, in _recv_loop
raw = await w.recv()
^^^^^^^^^^^^^^
File "/root/runtime/agent-runtime/management/.venv/lib/python3.14/site-packages/websockets/asyncio/connection.py", line 324, in recv
raise self.protocol.close_exc from self.recv_exc
websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
kubectl port-forward pod/jiuwenclaw-h9iwwixusv-jhc8d 8888:18092
uv run python tests/system_tests/management_session/main_k8s_multi_session.py --image swr.cn-north-4.myhuaweicloud.com/openjiuwen/jiuwenclaw-agentserver-amd64:0.0.1 --namespace default --name-prefix jiuwenclaw --container-port 18092 --ws-tls 0 --min-idle 0 --max-services 1 --service-concurrency 10 --service-ttl 10 --session-run-mode serial --message-timeout 300 --env MODEL_PROVIDER=OpenAI --env MODEL_NAME=Qwen/Qwen3-32B --env API_BASE=https://dashscope.aliyuncs.com/compatible-mode/v1 --env API_KEY=sk- --session "id=A,ttl=10,msgs=1,conc=1" --session "id=B,ttl=20,msgs=1,conc=1" --session "id=C,ttl=30,msgs=1,conc=1" --message-template-json '{"chat_id":"chat_{sid}","bot_id":"bot_{sid}","user_id":"u_{sid}","session_id":"sess_{sid}","channel_id":"web","req_method":"chat.send","params":{"session_id":"sess_{sid}","content":"hello","mode":"agent","query":"1+1等于几"},"is_stream":true,"metadata":{"query":{},"method":"chat.send"}}'