🚀 Background Description

需求描述：
构建面向多 agent 团队（swarm agent）的自动评测与优化闭环。系统支持从任务出发自动准备数据集，调度 Team Skill 与成员 Expert Harness 执行评测，收集 case 结果、轨迹和评分，并基于失败分析自动判断问题属于团队协作层还是成员能力层。针对成员层问题，流程需要能够自动定位问题成员，规划并执行teamskill / harness 优化，并在优化后通过候选评测确认效果。

需求价值
该能力可以降低多 Agent 团队调优成本，将原本依赖人工分析、修改和反复评测的流程自动化。通过统一的评测、诊断、优化、验证和经验沉淀链路，可以持续提升 Team Skill 与成员 Harness 的稳定性和任务表现。同时，member harness 优化引入 candidate gate 后，系统能够避免无效或负向优化直接进入后续链路，提高自动优化闭环的可信度。

验收标准
系统能够基于指定 task 或 dataset 完成一次端到端优化流程，产出评测结果、分析结果、优化结果和 checkpoint。
member harness 优化后，系统能够使用同一批 case 对候选版本重新评测，并仅在候选分数优于原版本时接受该优化。
优化后的 harness refs 能够被后续 evaluator 正确加载并参与评测。
epoch 结束后能够完成全量评测，记录 best 状态，并沉淀优化经验和失败样例。
Here is the English translation:

Requirements Description:

Build an automated evaluation and optimization closed-loop for multi-agent teams (swarm agents). The system should support automatic dataset preparation from tasks, scheduling Team Skill and member Expert Harness for evaluation execution, collecting case results, traces, and scores, and automatically diagnosing whether failures stem from team collaboration layer or member capability layer based on failure analysis. For member-level issues, the workflow must automatically identify the problematic member, plan and execute Team Skill / harness optimization, and verify effectiveness through candidate evaluation after optimization.

Value Proposition

This capability reduces multi-agent team tuning costs by automating processes that previously relied on manual analysis, modification, and iterative evaluation. Through a unified pipeline of evaluation, diagnosis, optimization, validation, and experience accumulation, it continuously improves the stability and task performance of Team Skills and member Harnesses. Additionally, by introducing a candidate gate for member harness optimization, the system prevents ineffective or negative optimizations from entering subsequent stages, enhancing the reliability of the automated optimization closed-loop.

Acceptance Criteria

• The system can complete an end-to-end optimization process based on a specified task or dataset, producing evaluation results, analysis results, optimization results, and checkpoints.
• After member harness optimization, the system can re-evaluate the candidate version using the same batch of cases, and only accept the optimization if the candidate score outperforms the original version.
• Optimized harness refs can be correctly loaded by subsequent evaluators and participate in evaluation.
• After an epoch completes, the system can perform full evaluation, record the best state, and accumulate optimization experience and failure examples.

Design Ideas

None

Involved Public APIs

Description of Relevance to Other Modules

Test Design and Test Plan

None

Additional Information

Thanks for your contribution 🎉!