文件最后提交记录最后更新时间
inference_service: support RKNN distributed runtime Support RKNN artifacts in the distributed inference flow so the board runtime can load .rknn models while keeping LeRobot checkpoint metadata for edge-side preprocessing and postprocessing. Align the live dispatcher defaults, board packaging, launch docs, and reliable controller command QoS with the measured RKNN deployment path. Defer optional backend imports so RKNN and CPU paths remain importable and testable without Ascend runtime packages installed. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 23 天前
camera_isp: tighten close detection and fix focus/gain bugs Address PR105 review feedback: * calibrator close: only treat WND_PROP_VISIBLE < 0 as destroyed so minimise / transient unmap no longer self-exits the calibrator. X-click is still caught via _trackbar_panel_closed() (the most reliable signal under GTK). * run() finally: join the AUTO/search worker (timeout 2s) before destroying windows, so background v4l2 writes don't outlive the GUI. * v4l2_ctl.apply_params now returns (ok, msg, handled_keys) so the calibrator can route unrecognised keys (e.g. focus) to the ROS fallback instead of silently dropping them. * evaluate_gain_step_b takes gain_caps and clamps the halved gain via gain_caps.minimum (was incorrectly using brightness_caps.minimum, which forced final_gain=64 on UVCs whose brightness floor was 64). Signed-off-by: grangerxsp <xingshiping@huawei.com> 26 天前
fix(moveit): refactor moveit_gateway for shared 5-DOF IK strategy Extract _move_with_strategies helper to deduplicate the IK planning logic used by both MoveToPose and MoveRelativeEE service handlers. Applies a consistent 3-attempt cascade (relaxed → strict → fallback) for 5-DOF arm planning across all motion entry points. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: liuweihong <liuweihong8@huawei.com> 21 天前
inference_service: support RKNN distributed runtime Support RKNN artifacts in the distributed inference flow so the board runtime can load .rknn models while keeping LeRobot checkpoint metadata for edge-side preprocessing and postprocessing. Align the live dispatcher defaults, board packaging, launch docs, and reliable controller command QoS with the measured RKNN deployment path. Defer optional backend imports so RKNN and CPU paths remain importable and testable without Ascend runtime packages installed. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 23 天前
feat(navi): add robot_navigation package - Add robot_navigation package with Nav2 integration, EKF localization (robot_localization + RTAB-Map), and cmd_vel bridge for omni-base control - Add voice navigation via FunASR: keyword matching, destination resolution, and nav_stop topic for voice-driven navigation cancellation - Add launch builders for localization, nav2, static TF, cmd_vel, and voice_nav - Add nav2_params (omniMotionModel, DWB controller), EKF config, RViz config, and FunASR install/start scripts - Wire nav2_goal_client to /voice_asr/nav_stop for voice-cancel support - Fix min_y_velocity_threshold (0.001) for omni-base lateral velocity Signed-off-by: yanhan <yanhan31@huawei.com> 27 天前
feat(navi): replace FunASR with local sherpa-onnx + add tests Voice control refactoring: - Add voice_control node bridging voice_asr_service (sherpa-onnx) to nav2_goal_client via keyword matching, replacing external FunASR WebSocket client that required a separate server deployment - Add hotword registration: voice_control auto-pushes keywords to voice_asr_node at startup via SetHotwords service (ibrobot_msgs) - Remove funasr_client_node, FunASR install/start scripts, and python3-websockets dependency - Clean up lekiwi_navi.yaml: replace stale funasr config with voice_asr section; update model path to relative path - Update voice_nav.py launch builder for voice_control node - Update README with new architecture and ROS interface docs lekiwi_hardware refactoring: - Extract lekiwi_conversions.hpp (header-only) with 6 pure conversion functions (ticks_to_radians, radians_to_ticks, steps_to_rad_s, rad_s_to_steps, decode_motor_register, encode_homing_offset) - Refactor lekiwi_system_hardware.cpp to use extracted functions, removing inline conversion logic - Add 55 gtest cases covering all conversion functions and round-trip consistency (test_lekiwi_conversions.cpp) Test coverage: - Add pytest tests for voice_control (40 cases: keyword matching, destination resolution, stop handling, hotword extraction, e2e) - Add pytest tests for cmd_vel_bridge (IK/FK, velocity scaling, quaternion, joint feedback, odometry integration) - Add pytest tests for nav2_goal_client (goal dispatch, cancellation, task description caching, timeout handling) Also fix asr_inference_module to correctly load encoder/decoder ONNX files for streaming paraformer model. Signed-off-by: yanhan <yanhan31@huawei.com> 17 天前
inference_service: support RKNN distributed runtime Support RKNN artifacts in the distributed inference flow so the board runtime can load .rknn models while keeping LeRobot checkpoint metadata for edge-side preprocessing and postprocessing. Align the live dispatcher defaults, board packaging, launch docs, and reliable controller command QoS with the measured RKNN deployment path. Defer optional backend imports so RKNN and CPU paths remain importable and testable without Ascend runtime packages installed. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 23 天前
feat(navi): replace FunASR with local sherpa-onnx + add tests Voice control refactoring: - Add voice_control node bridging voice_asr_service (sherpa-onnx) to nav2_goal_client via keyword matching, replacing external FunASR WebSocket client that required a separate server deployment - Add hotword registration: voice_control auto-pushes keywords to voice_asr_node at startup via SetHotwords service (ibrobot_msgs) - Remove funasr_client_node, FunASR install/start scripts, and python3-websockets dependency - Clean up lekiwi_navi.yaml: replace stale funasr config with voice_asr section; update model path to relative path - Update voice_nav.py launch builder for voice_control node - Update README with new architecture and ROS interface docs lekiwi_hardware refactoring: - Extract lekiwi_conversions.hpp (header-only) with 6 pure conversion functions (ticks_to_radians, radians_to_ticks, steps_to_rad_s, rad_s_to_steps, decode_motor_register, encode_homing_offset) - Refactor lekiwi_system_hardware.cpp to use extracted functions, removing inline conversion logic - Add 55 gtest cases covering all conversion functions and round-trip consistency (test_lekiwi_conversions.cpp) Test coverage: - Add pytest tests for voice_control (40 cases: keyword matching, destination resolution, stop handling, hotword extraction, e2e) - Add pytest tests for cmd_vel_bridge (IK/FK, velocity scaling, quaternion, joint feedback, odometry integration) - Add pytest tests for nav2_goal_client (goal dispatch, cancellation, task description caching, timeout handling) Also fix asr_inference_module to correctly load encoder/decoder ONNX files for streaming paraformer model. Signed-off-by: yanhan <yanhan31@huawei.com> 17 天前
feat(sim): align Gazebo/MuJoCo cameras, assets and lighting Add camera_presets.py with platform-specific default camera poses so that a single YAML config (use_default_transform: true) works for both Gazebo and MuJoCo without manual coordinate conversion. Update pick_banana Gazebo world template to match layout.yaml canonical values (table scale 0.156, banana x=0, curtain yaw flip, ground z). Tune MuJoCo lighting: reduce headlight diffuse, use single shadow- casting directional light, add low-specular materials for table and robot to eliminate top-camera overexposure. Add Gazebo material overrides in so101_gazebo.xacro so the robot arm renders with correct colors instead of default grey. Signed-off-by: $(git config user.name) <$(git config user.email)> Signed-off-by: grangerxsp <xingshiping@huawei.com> 2 个月前
fix(moveit): refactor moveit_gateway for shared 5-DOF IK strategy Extract _move_with_strategies helper to deduplicate the IK planning logic used by both MoveToPose and MoveRelativeEE service handlers. Applies a consistent 3-attempt cascade (relaxed → strict → fallback) for 5-DOF arm planning across all motion entry points. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: liuweihong <liuweihong8@huawei.com> 21 天前
feat(navi): replace FunASR with local sherpa-onnx + add tests Voice control refactoring: - Add voice_control node bridging voice_asr_service (sherpa-onnx) to nav2_goal_client via keyword matching, replacing external FunASR WebSocket client that required a separate server deployment - Add hotword registration: voice_control auto-pushes keywords to voice_asr_node at startup via SetHotwords service (ibrobot_msgs) - Remove funasr_client_node, FunASR install/start scripts, and python3-websockets dependency - Clean up lekiwi_navi.yaml: replace stale funasr config with voice_asr section; update model path to relative path - Update voice_nav.py launch builder for voice_control node - Update README with new architecture and ROS interface docs lekiwi_hardware refactoring: - Extract lekiwi_conversions.hpp (header-only) with 6 pure conversion functions (ticks_to_radians, radians_to_ticks, steps_to_rad_s, rad_s_to_steps, decode_motor_register, encode_homing_offset) - Refactor lekiwi_system_hardware.cpp to use extracted functions, removing inline conversion logic - Add 55 gtest cases covering all conversion functions and round-trip consistency (test_lekiwi_conversions.cpp) Test coverage: - Add pytest tests for voice_control (40 cases: keyword matching, destination resolution, stop handling, hotword extraction, e2e) - Add pytest tests for cmd_vel_bridge (IK/FK, velocity scaling, quaternion, joint feedback, odometry integration) - Add pytest tests for nav2_goal_client (goal dispatch, cancellation, task description caching, timeout handling) Also fix asr_inference_module to correctly load encoder/decoder ONNX files for streaming paraformer model. Signed-off-by: yanhan <yanhan31@huawei.com> 17 天前
scripts: refactor setup.sh into modular components Split the monolithic setup.sh into a modular architecture to improve maintainability and testability. Key changes: - Created scripts/setup/ containing discrete modules for submodules, python_venv, rosdep, and environment verification. - Introduced Template Method pattern for platform-specific hooks (platform_ros_setup_path, platform_handle_missing_ros) to eliminate platform if/else branches in the core script. - Separated Python requirements into dedicated files (base, dev-tools, hardware, platform-specific). - Added explicit dependency checking and robust git-lfs handling. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 27 天前
feat(sim): align Gazebo/MuJoCo cameras, assets and lighting Add camera_presets.py with platform-specific default camera poses so that a single YAML config (use_default_transform: true) works for both Gazebo and MuJoCo without manual coordinate conversion. Update pick_banana Gazebo world template to match layout.yaml canonical values (table scale 0.156, banana x=0, curtain yaw flip, ground z). Tune MuJoCo lighting: reduce headlight diffuse, use single shadow- casting directional light, add low-specular materials for table and robot to eliminate top-camera overexposure. Add Gazebo material overrides in so101_gazebo.xacro so the robot arm renders with correct colors instead of default grey. Signed-off-by: $(git config user.name) <$(git config user.email)> Signed-off-by: grangerxsp <xingshiping@huawei.com> 2 个月前
build(so101_hardware): use gitee mirror for FTServo_Linux SDK Using the Gitee mirror prevents timeouts and slow downloads when FetchContent pulls the feetech servo SDK during the build process in environments with restricted access. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 1 个月前
feat: add task_dispatch framework for unified task-level execution Add a unified task execution framework (task_dispatch) that bridges task-level planners (visual_grasp, VoxPoser, etc.) and the existing MoveIt motion execution layer (moveit_gateway + ros2_control). Architecture overview: Planners → ExecuteTaskPlan action → TaskExecutorNode TaskExecutorNode dispatches: MOVE_TO_POSE → /moveit_gateway/move_to_pose service GRIPPER → gripper trajectory controller WAIT → timed delay New packages: - task_dispatch: TaskExecutorNode with ExecuteTaskPlan action server New message types (ibrobot_msgs): - msg/TaskStep.msg: single step in a task plan (pose/gripper/wait) - action/ExecuteTaskPlan.action: execute a sequence of TaskSteps - srv/MoveToPose.srv: synchronous move-to-pose via MoveIt gateway Modified packages: - robot_moveit: moveit_gateway gains MoveToPose service + motion_status - robot_config: launch_builders/task_execution.py auto-launches task_executor for moveit-based control modes - robot.launch.py: section 11.5 integrates task executor launch This framework is the counterpart to action_dispatch: action_dispatch = joint-level streaming (ACT/VLA, 100Hz) task_dispatch = task-level sequencing (MoveIt planners, event-driven) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> 1 个月前
feat(dataset_tools): harden Rerun recording visualizer runtime Ensure the Rerun recording visualizer runs from the workspace venv and avoids user-site package conflicts that caused mixed rerun imports. Update the build and setup scripts to install the required rerun and compatibility dependencies in the workspace environment. Improve rerun_viewer shutdown behavior so a spawned viewer exits with the ROS node. Centralize image decoding in tensormsg, make feedback topics namespace-aware, and document the callback closure binding pattern. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 1 个月前
feat(navi): replace FunASR with local sherpa-onnx + add tests Voice control refactoring: - Add voice_control node bridging voice_asr_service (sherpa-onnx) to nav2_goal_client via keyword matching, replacing external FunASR WebSocket client that required a separate server deployment - Add hotword registration: voice_control auto-pushes keywords to voice_asr_node at startup via SetHotwords service (ibrobot_msgs) - Remove funasr_client_node, FunASR install/start scripts, and python3-websockets dependency - Clean up lekiwi_navi.yaml: replace stale funasr config with voice_asr section; update model path to relative path - Update voice_nav.py launch builder for voice_control node - Update README with new architecture and ROS interface docs lekiwi_hardware refactoring: - Extract lekiwi_conversions.hpp (header-only) with 6 pure conversion functions (ticks_to_radians, radians_to_ticks, steps_to_rad_s, rad_s_to_steps, decode_motor_register, encode_homing_offset) - Refactor lekiwi_system_hardware.cpp to use extracted functions, removing inline conversion logic - Add 55 gtest cases covering all conversion functions and round-trip consistency (test_lekiwi_conversions.cpp) Test coverage: - Add pytest tests for voice_control (40 cases: keyword matching, destination resolution, stop handling, hotword extraction, e2e) - Add pytest tests for cmd_vel_bridge (IK/FK, velocity scaling, quaternion, joint feedback, odometry integration) - Add pytest tests for nav2_goal_client (goal dispatch, cancellation, task description caching, timeout handling) Also fix asr_inference_module to correctly load encoder/decoder ONNX files for streaming paraformer model. Signed-off-by: yanhan <yanhan31@huawei.com> 17 天前
chore: initial monorepo migration for IB_Robot Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 个月前
deps: add omni wheel controller submodule Add the upstream omni_wheel_controller repository as a submodule. This keeps the LeKiwi controller dependency separate from the main robot_config integration changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 1 个月前
feat: Add pymoveit2 as submodule - Add pymoveit2 submodule at src/pymoveit2 path - Point to official AndrejOrsula/pymoveit2 repository - Enables MoveIt2 Python interface for robot control 2 个月前
feat: integrate OpenClaw social control and add rosclaw submodule Enable remote natural language control of IB-Robot via OpenClaw AI agent and RosClaw bridge. Supports Gazebo simulation and real SO-101 arm. - Add RosClaw as a git submodule in 'src/rosclaw'. - Implement 'scripts/start_rosclaw.sh' for independent startup. - Update 'README.md' with announcement, demo videos, and setup guides. - Add 'docs/ib_robot_social_skill.md' for LLM skill definition. - Update system architecture diagram. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 个月前
chore: initial monorepo migration for IB_Robot Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 个月前
voice_asr_service: add ROS2 ASR node Add a sherpa-onnx based ROS2 ASR package so the robot can capture microphone audio, transcribe files, and expose hotword updates through services and topics. Register the new RecognizeFile and SetHotwords interfaces and extend the build dependency bootstrap with the required ASR and audio Python packages. Signed-off-by: liuweihong <liuweihong8@huawei.com> 2 个月前
README.md

IB-Robot 核心源码库 (Source Code)

本目录包含了 IB-Robot 框架的所有核心 ROS 2 功能包。通过契约驱动 (Contract-driven) 的架构设计,实现了高层 AI 推理与底层机器人控制的深度解耦与无缝集成。


核心设计理念:配置驱动中心 (Configuration-as-Data)

IB-Robot 采用了 robot_config 作为全系统的“唯一真理源 (Single Source of Truth)”。

在这种架构下,开发者的关注点从繁琐的节点连接转移到了高层的规格定义:

  • 本体定义: 所有的关节限位、控制器映射、传感器外参均在 YAML 中声明。
  • 自动合成: robot_config 会根据 YAML 动态合成 AI 模型所需的通讯契约(Contract),消除人工对齐误差。
  • 能力插拔: 仅需通过启动参数,即可一键在“纯硬件调试”、“MoveIt 规划”、“端到端 AI 推理”与可选的语音 ASR 输入之间切换。

系统架构与数据流

graph TD
    %% 配置中心
    Config[robot_config] -- 1. 分发规格/契约 --> Init[系统初始化]
    
    %% 执行层
    Init --> InfSrv[inference_service]
    Init --> Dispatch[action_dispatch]
    Init --> Bridge[tensormsg]
    
    %% 数据闭环
    subgraph "决策与调度 (Decision & Dispatch)"
        InfSrv -- 预测动作块/张量 --> Dispatch
        Dispatch -- 高频插值指令 --> Bridge
    end
    
    subgraph "感知与执行 (Perception & Execution)"
        Bridge -- 标准控制接口 --> Hardware{ros2_control}
        Hardware -- 实机驱动 --> Real[so101_hardware]
        Hardware -- 仿真驱动 --> Sim[Gazebo / Ignition]
        Sensor[相机/状态反馈] -- 原始数据 --> Bridge
    end
    
    Bridge -- 契约对齐的张量 --> InfSrv

功能包深度解析

1. 📂 robot_config — 系统总控与规格中心

系统的“大脑”与决策入口。

  • 统一入口: 提供 robot.launch.py 脚本,协调感知、推理、调度各层的启动顺序。
  • 契约合成器: 内置 contract_builder,自动为 ACT/Pi0 等模型生成输入输出映射。
  • 模块化构建: 采用 launch_builders模式,将复杂的启动逻辑拆解为控制、仿真、感知、执行四大模块。

2. 📂 inference_service — 模型推理服务端

一个高性能、可扩展的模型部署后端。

  • 多模型适配: 统一封装了 ACT、Diffusion Policy、Pi0.5 及 SmolVLA 等主流具身模型。
  • 异步拉取: 采用 Action 通讯机制,支持按需触发推理,有效节省计算资源。
  • 硬件透明: 自动识别并利用 CUDA 进行加速,处理多路高分辨率图像输入。

3. 📂 action_dispatch — 动作调度与安全小脑

负责将高层张量转化为机器人可执行的连贯动作。

  • Action Chunking: 管理长序列动作块,内置线性插值逻辑,确保关节运动平滑无抖动。
  • 双模支持: 同时支持 model_inference(高频话题)和 moveit_planning(轨迹动作)两种执行模式。
  • 水位线监控: 实时监控动作队列状态,在数据中断时提供 Hold/Stop 等安全降级策略。

4. 📂 tensormsg — LeRobot ↔ ROS 2 协议枢纽

(拟更名为 tensormsg)

  • 实时序列化: 实现 ROS 2 消息与 NumPy/Torch 张量之间的高性能转换。
  • 时戳对齐: 采用 asof 采样策略,确保多传感器观测数据在时间轴上精确对齐。

5. 📂 robot_moveit — 运动规划集成

  • 避障与规划: 提供 SO-101 机械臂的 MoveIt 2 配置,支持 OMPL 和 Pilz 规划器。
  • 交互控制: 预配置了 RViz 交互式标记点,支持手动拖拽规划。

6. 📂 robot_description — 模型资产库

  • 统一描述: 维护全系统的 URDF、SRDF 和 STL 网格文件。
  • 仿真适配: 预置了 Gazebo 传感器插件和 ros2_control 硬件仿真接口。

7. 📂 so101_hardware — 物理硬件驱动

  • 底层驱动: 深度集成飞特 (Feetech) 舵机 SDK,支持高频状态反馈与位置控制。
  • 资源隔离: 作为纯粹的 C++ 驱动包,与上游 ML 依赖完全隔离。

8. 📂 voice_asr_service — 语音识别接入层

  • 统一入口: voice_asr_node 提供麦克风实时识别和音频文件识别两种入口。
  • 配置收敛: 业务参数由 robot_config 中的 robot.voice_asr 注入;包内 launch 仅保留调试用途。
  • 稳健失败: 模型初始化失败时会拒绝识别请求并返回明确错误,而不是在服务回调中直接崩溃。

快速开发命令

请确保在操作前已在根目录执行过 source .shrc_local

任务 命令示例
全量编译 ./scripts/build.sh
启动 MoveIt 调试 ros2 launch robot_config robot.launch.py control_mode:=moveit_planning use_sim:=true
启动 AI 推理 ros2 launch robot_config robot.launch.py control_mode:=model_inference use_sim:=true with_inference:=true
配置一致性检查 python3 scripts/validate_config.py

许可证 (License)

本项目源码遵循 Apache License 2.0