Clarity for AI buildersSponsored by AI Routing
adoption

Anthropic’s Fable 5.2 reaches 41.3% completion on freelance-work benchmark, The Decoder reports

Anthropic’s Fable 5.2 reaches 41.3% completion on freelance-work benchmark, The Decoder reports
Anthropic’s Fable 5.2 completed 41.3% of jobs in a freelance-work benchmark, according to The Decoder. The report describes a sharp rise in performance on the RLI benchmark, which tests agents on complete professional assignments rather than isolated skills. The leading system at the benchmark’s launch about a year earlier, Manus, completed 2.5% of tasks. Fable 5 reached about 16% in July; four months later, Fable 5.2 scored 41.3%. Its direct predecessor, Fable 5.1, scored just under 18%. On the leaderboard described by The Decoder, Anthropic’s Opus 5.5 followed at 21.25%, with OpenAI’s GPT 6 Astra at 20.83%. Google’s Gemini Argon 4 placed fifth at 17.08%. These are reported benchmark results, not evidence that the same completion rates apply across freelance work generally. What the benchmark measures According to The Decoder’s account of the RLI research paper, the dataset contains 240 completed projects from 358 verified Upwork professionals, with a combined value of about $144,000. Assignments span video editing, CAD design, game development and data analysis. The report puts average human working time at just under 29 hours per project. Agents receive a virtual workspace with professional applications including Blender, GIMP and Audacity, alongside specialized tools for generating images, audio and video. Trained human experts open submitted files in the relevant software and compare the results with the human freelancer’s work. That evaluation method matters because the test goes beyond whether an agent can produce a convincing preview. Opening and inspecting the deliverables can reveal whether the underlying files work and whether the assignment is actually complete. The reported score therefore concerns successful project delivery under the benchmark’s conditions, rather than the quality of a text answer alone. Gains remain uneven The Decoder attributes to Scale Labs the finding that Fable 5.2’s biggest gains came in audio and music editing, architecture and interior design, and 3D modeling. Nearly 50 of its successful completions were jobs that no agent in the test had previously solved. Industrial product design and animation remained particularly difficult. The overall result thus combines substantial progress with a clear remaining limit: at 41.3%, successful completions still account for fewer than half of the benchmark’s jobs. The category differences also mean that the headline rate should not be treated as a uniform capability across every kind of professional assignment. The report lists amateur-quality work, incomplete results, broken files and contradictory designs among typical failures. It also says CAIS’s attempts to automate evaluation with AI have so far failed: AI reviewers struggled to operate the professional software and awarded overly generous ratings to superficially polished but technically flawed submissions. Those evaluation difficulties reinforce the distinction between generating an output and verifying a usable deliverable. The reported improvement is meaningful within this test, but the same account shows why expert inspection remains important when assessing whether an agent has actually finished a job.