SWE-Milestone: Evaluating AI Agents on Continuous Software Evolution
Published in ICML 2026, 2026
SWE-Milestone (formerly EvoClaw) evaluates AI agents on continuous software evolution. Its DeepCommit pipeline reconstructs verifiable development milestones from repository histories, allowing the benchmark to assess how agents maintain system integrity and manage accumulating errors across a sequence of development tasks.
| Paper | Project website | Code |
Recommended citation: Gangda Deng, Zhaoling Chen, Zhongming Yu, Haoyang Fan, Yuhong Liu, **Yuxin Yang**, Dhruv Parikh, Rajgopal Kannan, Le Cong, Mengdi Wang, Qian Zhang, Viktor Prasanna, Xiangru Tang, Xingyao Wang. "SWE-Milestone: Evaluating AI Agents on Continuous Software Evolution." International Conference on Machine Learning (ICML), 2026.
Download Paper