Model Requirements

Not every agent needs the same model. The right choice depends on the complexity of the tasks the agent performs, and picking per-agent (rather than one model for everything) is the biggest lever on both result quality and running cost.

This page gives the guidance; Model Evaluation gives the data behind it.

Task complexity drives model size

Agent work falls into rough tiers. The heavier the tier, the more capable a model it needs to finish reliably.

Tier	Typical work	What it demands of the model
Light	List/query data, summarize a single object, explain the data model, render read-only widgets	Follow instructions, call the right read tool, emit the requested widget. Small/fast models cope well.
Medium	Single-object, multi-field update proposals; tagging; classification-store edits	Hold a multi-field change-set together, respect field shapes, converge in one proposal. Mid-tier models cope; weaker ones drop or mis-shape fields.
Heavy	Multi-step document/landing-page authoring, multi-object proposals, web research, image (vision) tagging, target-group personalization	Long-horizon planning, tool orchestration without looping, correct composition, vision, and knowing when to stop. Only stronger models finish these consistently.

The recurring failure modes on the heavy tier are instructive - weaker models tend to:

• emit the wrong widget type (a markdown/data table where an object-preview is required);
• skip web research even when a fetch tool is available;
• hallucinate in vision tasks (wrong colour/brand on an image);
• never converge - loop on the same tool calls or "approve a minimal update first" without finishing; and
• produce an empty document scaffold with no real content proposal.

Stronger models avoid these; that gap, more than raw benchmark scores, is what separates a model that completes agent work from one that merely starts it.

General findings from our evaluation

From our evaluation runs in June 2026 (full data and caveats in Model Evaluation):

• Best overall (18/18): claude-sonnet-4.6 is the only model that cleared the entire suite, including the hardest document- and object-creation tasks. The catch is cost - several times more expensive per run than the efficient leaders - so it earns its place on the heaviest agents rather than as a blanket default.
• Best all-round success (17/18): claude-haiku-4.5, gpt-5.4-mini, qwen3p6-plus, and gemini-3.5-flash each passed 17 of 18, missing only a single hard task.
• Best efficiency: claude-haiku-4.5 is the standout - it tops the efficiency ranking (fewest tokens, least wall-clock time, strong caching) while sitting one task off the top of the accuracy ranking. That balance is why it is the shipped default.
• Solid (16/18): kimi-k2p6 (very low input use) and deepseek-v3.2 (accurate but the slowest and caches poorly, so comparatively expensive).
• Mid-tier (14/18): minimax-m2.7 and Qwen3.6-35B-A3B - generally correct content but recurring widget-type, vision, or convergence slips.
• Weak on heavy tasks (≤13/18): the Gemma variants, gpt-oss-120b, and qwen3-32b - these frequently fail to materialize a proposal at all on document/creation tasks, and several do not cache, making them both less reliable and more expensive.

Recommendation. Keep the shipped default (GitHub Copilot + Claude Haiku 4.5) for general and data agents - it is fast, cheap, and well-cached. For document-authoring and research agents, point at a stronger model - claude-sonnet-4.6 was the only model to pass every task in our suite, or pick another higher-tier model from the Copilot catalog or a BYOK provider. Then validate against your own tasks with the eval tool - these findings are spot-checks, and your workload may stress models differently.