Inference

"Inference" is the large-language-model call behind every agent turn. The bundle is provider-agnostic: agents are defined once, and which model actually runs them - and who pays for it - is a configuration choice. This section explains the available options, how to match a model to a task, and what our own evaluation runs found.

Terminology is defined in Core Concepts. The field-by-field configuration schema lives in Configuration → Inference Providers; this section is the conceptual companion to it.

Out of the box

A fresh installation ships with a working default: every agent runs through GitHub Copilot on Claude Haiku 4.5. The only value a developer has to supply is the AGENT_SERVER_GH_TOKEN environment variable (a GitHub PAT with the Copilot Requests permission). The bundle provides an AgentServerEnvVarDefinition; once it is registered in a Pimcore install profile, the Pimcore installer prompts for this token (as an optional secret) during installation.

# Shipped default (config/pimcore/config.yaml) - override from your application config.
pimcore_agent:
    inference:
        default_provider: copilot
        providers:
            copilot:
                driver: copilot
                auth_mode: github
                token: '${AGENT_SERVER_GH_TOKEN}'
                default_model: claude-haiku-4.5

Everything below is about going beyond that default - using other models, other providers, or your own keys.

Where to go

I want to…	Read
Understand which model fits which task, and how to size for cost	Model Requirements
See how we evaluated models and the measured results	Model Evaluation
Configure a named provider (fields, per-model limits, examples)	Configuration → Inference Providers
Use a strict OpenAI-compatible endpoint (Cerebras, Ollama, vLLM, …)	Configuration → Provider Compatibility
Assign a provider/model to a specific agent	Configuration → Agents
Set the token secrets	Configuration → Environment Variables

Three ways to run inference

All providers share one driver (copilot, the SDK that powers the adapter). What differs is how you authenticate (auth_mode) and where the request goes.

1. GitHub Copilot - billed through GitHub

auth_mode: github. Inference is routed through the GitHub Copilot catalog and billed against the GitHub Copilot subscription tied to the token. The model list is fetched live from Copilot, so you can pick any model the catalog exposes (Anthropic, OpenAI, and Google families, among others) without listing models yourself. This is the simplest option and the shipped default - no per-model limits to maintain, one secret to set.

2. Bring Your Own Key (BYOK) - billed by the provider

auth_mode: byok. You supply an API key for a specific provider and are billed by that provider directly. Three provider shapes are supported via the provider field:

• anthropic - Claude models against https://api.anthropic.com.
• openai - OpenAI against https://api.openai.com/v1.
• openai (compatible) - any endpoint that speaks the OpenAI wire format: Hugging Face Inference, Cerebras, Groq-style routers, self-hosted servers, and so on. Some strict endpoints need the compatibility shim (compat: openai-strict / cerebras).

In BYOK mode base_url is required and the model list comes entirely from your available_models block, where you also declare per-model context-window and token limits.

3. Cloud service vs. self-hosted

BYOK is orthogonal to where the model runs. The same configuration can point at:

• a managed cloud service - Anthropic, OpenAI, Hugging Face Inference Providers, Cerebras, etc. (set base_url to the vendor endpoint); or
• a model you host yourself - Ollama, vLLM, or Text Generation Inference behind your own network (set base_url to your server, usually with compat: openai-strict).

Self-hosting removes per-token vendor cost and keeps data in your infrastructure, at the price of running and scaling the inference server. Managed services trade cost-per-token for zero operational overhead and instant access to frontier models.

Per-agent provider and model

Inference is not all-or-nothing. Each agent may name its own provider and model (see Configuration → Agents); agents that name neither fall back to default_provider and that provider's default_model. This lets you match the model to the job - a small, cheap model for a data-lookup agent and a stronger model for a document-authoring or research agent - which is the single biggest lever on both quality and cost. Model Requirements covers how to make that call, and Model Evaluation backs it with measured data.