文件最后提交记录最后更新时间
docs: sync CN/EN READMEs and update repo structure - Update repository structure tree in both root READMEs: remove non-existent usb_cam, add 7 new packages (task_dispatch, lekiwi_description, lekiwi_hardware, hardware_mock, omni_wheel_controller, robot_navigation, attention_viz) - Rewrite README.en.md to fully align with Chinese version: fix wrong .shrc_local instructions, add distributed inference, dataset recording, OpenHarmony board guide, missing parameters (execution_mode, cloud_local, record, record_mode, record_visualizer), troubleshooting #3, user guide link - Add missing inference_reset_service and policy_reset_timeout_sec parameters to action_dispatch English README - Add missing Tool Documentation section to so101_hardware English README Signed-off-by: BreezeWu <xqw@isrc.iscas.ac.cn> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 天前
attention_viz: add ACT attention debugging package 1. Add AttentionWeights messages, hooks, renderers, launch files, docs, and tests. 2. Support file/realtime heatmaps and interactive mask conversion. Signed-off-by: Shi Xin <shixin21@h-partners.com> 9 天前
dataset_tools: add direct camera source to ISP calibrator Allow camera_isp_calibrator to read frames directly from an OpenCV VideoCapture source when --camera_index is provided. This keeps the ROS image-topic path as the default while enabling camera-only ISP calibration without robot.launch.py. Request the reference image resolution on direct OpenCV captures so USB cameras open at the same size used by the calibrator. Map direct integer sources back to /dev/videoN for V4L2 control writes, derive a fallback override name when --camera is omitted, and document both modes. Apply numeric V4L2 values before toggling auto controls and cover the direct-source path with unit tests. Closes: https://gitcode.com/openeuler/IB_Robot/issues/59 Signed-off-by: Shi Xin <shixin21@h-partners.com> 6 天前
feat: robot.launch.py adapt & README * Add README.md for hardware_mock * Adapt the general launchfile: robot.launch.py Signed-off-by: YidaHao <haoyida@huawei.com> 9 天前
attention_viz: add ACT attention debugging package 1. Add AttentionWeights messages, hooks, renderers, launch files, docs, and tests. 2. Support file/realtime heatmaps and interactive mask conversion. Signed-off-by: Shi Xin <shixin21@h-partners.com> 9 天前
inference_service: honor runtime policy device Route LeRobot config loading through a runtime policy copy so launch device settings win. Keep source models unchanged and clean temporary policy copies after upstream loaders use them. Add startup diagnostics for model config device, runtime backend, and tensor device. Cover missing device fields and compiled backend config handling in regressions. Signed-off-by: Shi Xin <shixin21@h-partners.com> 6 天前
feat(navi): add robot_navigation package - Add robot_navigation package with Nav2 integration, EKF localization (robot_localization + RTAB-Map), and cmd_vel bridge for omni-base control - Add voice navigation via FunASR: keyword matching, destination resolution, and nav_stop topic for voice-driven navigation cancellation - Add launch builders for localization, nav2, static TF, cmd_vel, and voice_nav - Add nav2_params (omniMotionModel, DWB controller), EKF config, RViz config, and FunASR install/start scripts - Wire nav2_goal_client to /voice_asr/nav_stop for voice-cancel support - Fix min_y_velocity_threshold (0.001) for omni-base lateral velocity Signed-off-by: yanhan <yanhan31@huawei.com> 27 天前
feat(navi): replace FunASR with local sherpa-onnx + add tests Voice control refactoring: - Add voice_control node bridging voice_asr_service (sherpa-onnx) to nav2_goal_client via keyword matching, replacing external FunASR WebSocket client that required a separate server deployment - Add hotword registration: voice_control auto-pushes keywords to voice_asr_node at startup via SetHotwords service (ibrobot_msgs) - Remove funasr_client_node, FunASR install/start scripts, and python3-websockets dependency - Clean up lekiwi_navi.yaml: replace stale funasr config with voice_asr section; update model path to relative path - Update voice_nav.py launch builder for voice_control node - Update README with new architecture and ROS interface docs lekiwi_hardware refactoring: - Extract lekiwi_conversions.hpp (header-only) with 6 pure conversion functions (ticks_to_radians, radians_to_ticks, steps_to_rad_s, rad_s_to_steps, decode_motor_register, encode_homing_offset) - Refactor lekiwi_system_hardware.cpp to use extracted functions, removing inline conversion logic - Add 55 gtest cases covering all conversion functions and round-trip consistency (test_lekiwi_conversions.cpp) Test coverage: - Add pytest tests for voice_control (40 cases: keyword matching, destination resolution, stop handling, hotword extraction, e2e) - Add pytest tests for cmd_vel_bridge (IK/FK, velocity scaling, quaternion, joint feedback, odometry integration) - Add pytest tests for nav2_goal_client (goal dispatch, cancellation, task description caching, timeout handling) Also fix asr_inference_module to correctly load encoder/decoder ONNX files for streaming paraformer model. Signed-off-by: yanhan <yanhan31@huawei.com> 17 天前
fix(export_atc): fix export failed * Caused by a previous bugfix for overriding "is_ascend_om_enabled" * And is no longer needed after refector of inference_service Signed-off-by: YidaHao <haoyida@huawei.com> 6 天前
sim_models: add pick_banana MuJoCo scene with CoACD collision meshes Add a complete pick-and-place simulation scene for the SO101 arm grasping a banana on a plate, with CoACD-based collision geometry throughout. Changes: - scene_compiler: compile XML to .mjb with SDF non-convex collision enabled for gripper finger meshes (needsdf=True); required so mujoco_ros2_control preserves the octree data via mj_loadModel - so101.xml.template: replace 3-box gripper collision proxies with CoACD 16-geom mesh collision (8 pieces per finger); add gripper_col default class for unified friction tuning; increase joint damping 0.5->1.5, add armature; retune actuator gains (kp 500->998.22, joint-6 kp 200->748.66 + kv 5.5); raise noslip_iterations 5->20 for stable gripper-banana contact - pick_banana.xml.template: new MuJoCo scene with CoACD banana (4 hulls) and plate (8 hulls); banana_col default class for centralized friction tuning - pick_banana.world.template: fix table collision plane offset (z=0.002->z=0) that caused all objects to float 2 mm above the surface in Gazebo; replace cylinder plate with CoACD STL; update banana mass/inertia to 30 g - requirements/setup-constraints.txt: pin mujoco==3.4.0 to match bundled libmujoco.so ABI in mujoco_ros2_control - Mesh assets: add banana_col_0..3.stl, plate_col_0..7.stl, wrist_roll_follower_col_0..7.stl, moving_jaw_col_0..7.stl (CoACD outputs) Signed-off-by: grangerxsp <xingshiping@huawei.com> 2 天前
review(sim): address PR #193 code review feedback - scene_compiler: wrap mujoco compile pipeline in try/except, fallback to .xml on any compile error; add logging.warning for diagnostics - pick_banana.world.template: add comment explaining STL over OBJ for Gazebo visual mesh (avoids OGRE2 GL3Plus buffer double-lock assertion) - setup.py: add per-type comments explaining role of each mesh glob - material.mtl: add missing trailing newline (POSIX compliance) - so101.xml.template: add comments on joints 1-4 explaining intentional removal of collision box proxies - requirements: delete dead setup-constraints.txt; move mujoco==3.4.0 pin with rationale comment to requirements/ubuntu-22.04.txt where python_venv.sh actually consumes it on ubuntu-22.04 1 天前
fix(moveit): refactor moveit_gateway for shared 5-DOF IK strategy Extract _move_with_strategies helper to deduplicate the IK planning logic used by both MoveToPose and MoveRelativeEE service handlers. Applies a consistent 3-attempt cascade (relaxed → strict → fallback) for 5-DOF arm planning across all motion entry points. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: liuweihong <liuweihong8@huawei.com> 22 天前
feat(navi): replace FunASR with local sherpa-onnx + add tests Voice control refactoring: - Add voice_control node bridging voice_asr_service (sherpa-onnx) to nav2_goal_client via keyword matching, replacing external FunASR WebSocket client that required a separate server deployment - Add hotword registration: voice_control auto-pushes keywords to voice_asr_node at startup via SetHotwords service (ibrobot_msgs) - Remove funasr_client_node, FunASR install/start scripts, and python3-websockets dependency - Clean up lekiwi_navi.yaml: replace stale funasr config with voice_asr section; update model path to relative path - Update voice_nav.py launch builder for voice_control node - Update README with new architecture and ROS interface docs lekiwi_hardware refactoring: - Extract lekiwi_conversions.hpp (header-only) with 6 pure conversion functions (ticks_to_radians, radians_to_ticks, steps_to_rad_s, rad_s_to_steps, decode_motor_register, encode_homing_offset) - Refactor lekiwi_system_hardware.cpp to use extracted functions, removing inline conversion logic - Add 55 gtest cases covering all conversion functions and round-trip consistency (test_lekiwi_conversions.cpp) Test coverage: - Add pytest tests for voice_control (40 cases: keyword matching, destination resolution, stop handling, hotword extraction, e2e) - Add pytest tests for cmd_vel_bridge (IK/FK, velocity scaling, quaternion, joint feedback, odometry integration) - Add pytest tests for nav2_goal_client (goal dispatch, cancellation, task description caching, timeout handling) Also fix asr_inference_module to correctly load encoder/decoder ONNX files for streaming paraformer model. Signed-off-by: yanhan <yanhan31@huawei.com> 17 天前
scripts: refactor setup.sh into modular components Split the monolithic setup.sh into a modular architecture to improve maintainability and testability. Key changes: - Created scripts/setup/ containing discrete modules for submodules, python_venv, rosdep, and environment verification. - Introduced Template Method pattern for platform-specific hooks (platform_ros_setup_path, platform_handle_missing_ros) to eliminate platform if/else branches in the core script. - Separated Python requirements into dedicated files (base, dev-tools, hardware, platform-specific). - Added explicit dependency checking and robust git-lfs handling. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 27 天前
review(sim): address PR #193 code review feedback - scene_compiler: wrap mujoco compile pipeline in try/except, fallback to .xml on any compile error; add logging.warning for diagnostics - pick_banana.world.template: add comment explaining STL over OBJ for Gazebo visual mesh (avoids OGRE2 GL3Plus buffer double-lock assertion) - setup.py: add per-type comments explaining role of each mesh glob - material.mtl: add missing trailing newline (POSIX compliance) - so101.xml.template: add comments on joints 1-4 explaining intentional removal of collision box proxies - requirements: delete dead setup-constraints.txt; move mujoco==3.4.0 pin with rationale comment to requirements/ubuntu-22.04.txt where python_venv.sh actually consumes it on ubuntu-22.04 1 天前
docs: sync CN/EN READMEs and update repo structure - Update repository structure tree in both root READMEs: remove non-existent usb_cam, add 7 new packages (task_dispatch, lekiwi_description, lekiwi_hardware, hardware_mock, omni_wheel_controller, robot_navigation, attention_viz) - Rewrite README.en.md to fully align with Chinese version: fix wrong .shrc_local instructions, add distributed inference, dataset recording, OpenHarmony board guide, missing parameters (execution_mode, cloud_local, record, record_mode, record_visualizer), troubleshooting #3, user guide link - Add missing inference_reset_service and policy_reset_timeout_sec parameters to action_dispatch English README - Add missing Tool Documentation section to so101_hardware English README Signed-off-by: BreezeWu <xqw@isrc.iscas.ac.cn> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 天前
feat: add task_dispatch framework for unified task-level execution Add a unified task execution framework (task_dispatch) that bridges task-level planners (visual_grasp, VoxPoser, etc.) and the existing MoveIt motion execution layer (moveit_gateway + ros2_control). Architecture overview: Planners → ExecuteTaskPlan action → TaskExecutorNode TaskExecutorNode dispatches: MOVE_TO_POSE → /moveit_gateway/move_to_pose service GRIPPER → gripper trajectory controller WAIT → timed delay New packages: - task_dispatch: TaskExecutorNode with ExecuteTaskPlan action server New message types (ibrobot_msgs): - msg/TaskStep.msg: single step in a task plan (pose/gripper/wait) - action/ExecuteTaskPlan.action: execute a sequence of TaskSteps - srv/MoveToPose.srv: synchronous move-to-pose via MoveIt gateway Modified packages: - robot_moveit: moveit_gateway gains MoveToPose service + motion_status - robot_config: launch_builders/task_execution.py auto-launches task_executor for moveit-based control modes - robot.launch.py: section 11.5 integrates task executor launch This framework is the counterpart to action_dispatch: action_dispatch = joint-level streaming (ACT/VLA, 100Hz) task_dispatch = task-level sequencing (MoveIt planners, event-driven) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> 1 个月前
feat(dataset_tools): harden Rerun recording visualizer runtime Ensure the Rerun recording visualizer runs from the workspace venv and avoids user-site package conflicts that caused mixed rerun imports. Update the build and setup scripts to install the required rerun and compatibility dependencies in the workspace environment. Improve rerun_viewer shutdown behavior so a spawned viewer exits with the ROS node. Centralize image decoding in tensormsg, make feedback topics namespace-aware, and document the callback closure binding pattern. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 1 个月前
feat(navi): replace FunASR with local sherpa-onnx + add tests Voice control refactoring: - Add voice_control node bridging voice_asr_service (sherpa-onnx) to nav2_goal_client via keyword matching, replacing external FunASR WebSocket client that required a separate server deployment - Add hotword registration: voice_control auto-pushes keywords to voice_asr_node at startup via SetHotwords service (ibrobot_msgs) - Remove funasr_client_node, FunASR install/start scripts, and python3-websockets dependency - Clean up lekiwi_navi.yaml: replace stale funasr config with voice_asr section; update model path to relative path - Update voice_nav.py launch builder for voice_control node - Update README with new architecture and ROS interface docs lekiwi_hardware refactoring: - Extract lekiwi_conversions.hpp (header-only) with 6 pure conversion functions (ticks_to_radians, radians_to_ticks, steps_to_rad_s, rad_s_to_steps, decode_motor_register, encode_homing_offset) - Refactor lekiwi_system_hardware.cpp to use extracted functions, removing inline conversion logic - Add 55 gtest cases covering all conversion functions and round-trip consistency (test_lekiwi_conversions.cpp) Test coverage: - Add pytest tests for voice_control (40 cases: keyword matching, destination resolution, stop handling, hotword extraction, e2e) - Add pytest tests for cmd_vel_bridge (IK/FK, velocity scaling, quaternion, joint feedback, odometry integration) - Add pytest tests for nav2_goal_client (goal dispatch, cancellation, task description caching, timeout handling) Also fix asr_inference_module to correctly load encoder/decoder ONNX files for streaming paraformer model. Signed-off-by: yanhan <yanhan31@huawei.com> 17 天前
chore: initial monorepo migration for IB_Robot Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 个月前
deps: add omni wheel controller submodule Add the upstream omni_wheel_controller repository as a submodule. This keeps the LeKiwi controller dependency separate from the main robot_config integration changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 1 个月前
feat: Add pymoveit2 as submodule - Add pymoveit2 submodule at src/pymoveit2 path - Point to official AndrejOrsula/pymoveit2 repository - Enables MoveIt2 Python interface for robot control 2 个月前
feat: integrate OpenClaw social control and add rosclaw submodule Enable remote natural language control of IB-Robot via OpenClaw AI agent and RosClaw bridge. Supports Gazebo simulation and real SO-101 arm. - Add RosClaw as a git submodule in 'src/rosclaw'. - Implement 'scripts/start_rosclaw.sh' for independent startup. - Update 'README.md' with announcement, demo videos, and setup guides. - Add 'docs/ib_robot_social_skill.md' for LLM skill definition. - Update system architecture diagram. Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 个月前
chore: initial monorepo migration for IB_Robot Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 个月前
docs: sync CN/EN READMEs and update repo structure - Update repository structure tree in both root READMEs: remove non-existent usb_cam, add 7 new packages (task_dispatch, lekiwi_description, lekiwi_hardware, hardware_mock, omni_wheel_controller, robot_navigation, attention_viz) - Rewrite README.en.md to fully align with Chinese version: fix wrong .shrc_local instructions, add distributed inference, dataset recording, OpenHarmony board guide, missing parameters (execution_mode, cloud_local, record, record_mode, record_visualizer), troubleshooting #3, user guide link - Add missing inference_reset_service and policy_reset_timeout_sec parameters to action_dispatch English README - Add missing Tool Documentation section to so101_hardware English README Signed-off-by: BreezeWu <xqw@isrc.iscas.ac.cn> Signed-off-by: XiaoqiangWu <wuxiaoqiang.rtos@huawei.com> 2 天前
README.md

IB-Robot 核心源码库 (Source Code)

本目录包含了 IB-Robot 框架的所有核心 ROS 2 功能包。通过契约驱动 (Contract-driven) 的架构设计,实现了高层 AI 推理与底层机器人控制的深度解耦与无缝集成。


核心设计理念:配置驱动中心 (Configuration-as-Data)

IB-Robot 采用了 robot_config 作为全系统的“唯一真理源 (Single Source of Truth)”。

在这种架构下,开发者的关注点从繁琐的节点连接转移到了高层的规格定义:

  • 本体定义: 所有的关节限位、控制器映射、传感器外参均在 YAML 中声明。
  • 自动合成: robot_config 会根据 YAML 动态合成 AI 模型所需的通讯契约(Contract),消除人工对齐误差。
  • 能力插拔: 仅需通过启动参数,即可一键在“纯硬件调试”、“MoveIt 规划”、“端到端 AI 推理”与可选的语音 ASR 输入之间切换。

系统架构与数据流

graph TD
    %% 配置中心
    Config[robot_config] -- 1. 分发规格/契约 --> Init[系统初始化]
    
    %% 执行层
    Init --> InfSrv[inference_service]
    Init --> Dispatch[action_dispatch]
    Init --> Bridge[tensormsg]
    
    %% 数据闭环
    subgraph "决策与调度 (Decision & Dispatch)"
        InfSrv -- 预测动作块/张量 --> Dispatch
        Dispatch -- 高频插值指令 --> Bridge
    end
    
    subgraph "感知与执行 (Perception & Execution)"
        Bridge -- 标准控制接口 --> Hardware{ros2_control}
        Hardware -- 实机驱动 --> Real[so101_hardware]
        Hardware -- 仿真驱动 --> Sim[Gazebo / Ignition]
        Sensor[相机/状态反馈] -- 原始数据 --> Bridge
    end
    
    Bridge -- 契约对齐的张量 --> InfSrv

功能包深度解析

1. 📂 robot_config — 系统总控与规格中心

系统的“大脑”与决策入口。

  • 统一入口: 提供 robot.launch.py 脚本,协调感知、推理、调度各层的启动顺序。
  • 契约合成器: 内置 contract_builder,自动为 ACT/Pi0 等模型生成输入输出映射。
  • 模块化构建: 采用 launch_builders模式,将复杂的启动逻辑拆解为控制、仿真、感知、执行四大模块。

2. 📂 inference_service — 模型推理服务端

一个高性能、可扩展的模型部署后端。

  • 多模型适配: 统一封装了 ACT、Diffusion Policy、Pi0.5 及 SmolVLA 等主流具身模型。
  • 异步拉取: 采用 Action 通讯机制,支持按需触发推理,有效节省计算资源。
  • 硬件透明: 自动识别并利用 CUDA 进行加速,处理多路高分辨率图像输入。

3. 📂 action_dispatch — 动作调度与安全小脑

负责将高层张量转化为机器人可执行的连贯动作。

  • Action Chunking: 管理长序列动作块,内置线性插值逻辑,确保关节运动平滑无抖动。
  • 双模支持: 同时支持 model_inference(高频话题)和 moveit_planning(轨迹动作)两种执行模式。
  • 水位线监控: 实时监控动作队列状态,在数据中断时提供 Hold/Stop 等安全降级策略。

4. 📂 tensormsg — LeRobot ↔ ROS 2 协议枢纽

(拟更名为 tensormsg)

  • 实时序列化: 实现 ROS 2 消息与 NumPy/Torch 张量之间的高性能转换。
  • 时戳对齐: 采用 asof 采样策略,确保多传感器观测数据在时间轴上精确对齐。

5. 📂 robot_moveit — 运动规划集成

  • 避障与规划: 提供 SO-101 机械臂的 MoveIt 2 配置,支持 OMPL 和 Pilz 规划器。
  • 交互控制: 预配置了 RViz 交互式标记点,支持手动拖拽规划。

6. 📂 robot_description — 模型资产库

  • 统一描述: 维护全系统的 URDF、SRDF 和 STL 网格文件。
  • 仿真适配: 预置了 Gazebo 传感器插件和 ros2_control 硬件仿真接口。

7. 📂 so101_hardware — 物理硬件驱动

  • 底层驱动: 深度集成飞特 (Feetech) 舵机 SDK,支持高频状态反馈与位置控制。
  • 资源隔离: 作为纯粹的 C++ 驱动包,与上游 ML 依赖完全隔离。

8. 📂 voice_asr_service — 语音识别接入层

  • 统一入口: voice_asr_node 提供麦克风实时识别和音频文件识别两种入口。
  • 配置收敛: 业务参数由 robot_config 中的 robot.voice_asr 注入;包内 launch 仅保留调试用途。
  • 稳健失败: 模型初始化失败时会拒绝识别请求并返回明确错误,而不是在服务回调中直接崩溃。

快速开发命令

请确保在操作前已在根目录执行过 source .shrc_local

任务 命令示例
全量编译 ./scripts/build.sh --clean
启动 MoveIt 调试 ros2 launch robot_config robot.launch.py control_mode:=moveit_planning use_sim:=true
启动 AI 推理 ros2 launch robot_config robot.launch.py control_mode:=model_inference use_sim:=true with_inference:=true
配置一致性检查 python3 scripts/validate_config.py

许可证 (License)

本项目源码遵循 Apache License 2.0