Even the best AI coding agent fully completes only about one task in four when given real work, according to a new benchmark built from genuine user requests by researchers from Alibaba and several Asian universities.

Co-Design Bench, published as an arXiv paper on 26 October, draws 273 tasks from 37,463 deduplicated change requests across 26,314 real coding sessions. The agents must rework existing web applications, sometimes from images, audio recordings, videos or documents, and are graded against 2,331 hidden acceptance criteria written by experts.

Of the ten models tested, OpenAI's GPT-6 Astra came first, meeting about 69.9 percent of each task's requirements on average, but fully completing only 26.4 percent of tasks. Claude Opus 5 was next at 18.3 percent fully completed, while xAI's Grok 4.6 finished last at 6.6 percent. The gap between partial and full completion is the point: an agent can handle most of a task and still leave details broken.

The test was designed to expose how far benchmark scores overstate real productivity. Older tests such as WebCompass-edit are now saturated, with models scoring above 99 percent. Co-Design Bench instead deliberately picked hard cases where users had to rephrase requests or roll back changes. The choice of agent harness alone swung results by up to 18.9 points. The benchmark has limits: it tests one fixed brief per task, so it says nothing about how agents cope when a user pushes back midway through.