Agent Framework

The agent-server is built on GitHub's Copilot SDK (@github/copilot-sdk). The SDK owns LLM sessions, tool calling, skill loading, and response streaming. The agent-server wraps it in a thin adapter layer and adds everything Pimcore-specific on top: a registry of agents, cookie-forwarded MCP calls, session persistence, HITL proposals, and rich chat widgets.

Adapters

An adapter is the layer between the agent-server and the LLM backend. Two ship today.

Copilot adapter

The production path. Responsible for:

• Lazy initialisation - the SDK client starts on first use, not at server boot.
• One SDK session per chat session - an "agent" is a config applied to a session, not a separate session. The SDK session ID stays stable across agent switches.
• Same-agent resume - client.resumeSession() reattaches to an existing SDK session after a server restart (cold resume) or returns the cached wrapper (warm reuse). Conversation history is preserved by the SDK.
• Agent-switch resume - the SDK CLI ignores mcpServers on resume (they are locked at session.create time). So when the target agent actually changes, the adapter destroys the SDK session via deleteSession and recreates it with the new config, reusing the same session ID. Conversation context is carried across via a bounded prior-conversation block injected into the new system message from the persistent session store. End-user behaviour of agent switching is covered in Features → Agents and Switching.
• Model validation - at startup and on every reload, the adapter checks whether configured models are available from the provider. Mismatches log warnings; runtime validation blocks session creation if the selected model is unavailable. Validation is skipped in BYOK mode until a key is present.
• Title generation - after the first exchange, the adapter generates a 3–5 word conversation title using the resolved provider's title_model, falling back to the provider's default_model when unset.

Dummy adapter

Scripted responses, no LLM call. Three modes:

Mode	Behaviour
echo	Echoes the user's message.
scripted	Keyword-matched canned responses (search results, widget emissions, thinking blocks).
random	Randomised event sequences for stress tests.

Activate with AGENT_SERVER_ADAPTER=dummy and optionally AGENT_SERVER_DUMMY_MODE=scripted. No API key needed.

LLM authentication modes

Authentication to the LLM backend is configured per provider in the pimcore_agent.inference Symfony config tree, not as environment variables. Each provider has an auth_mode field:

GitHub mode (auth_mode: github)

Inference is routed through the GitHub Copilot catalog using a Personal Access Token. Requires a Copilot-enabled account. The token field in the provider block holds a ${VAR} reference to a GitHub PAT with the "Copilot Requests" permission.

BYOK mode (auth_mode: byok)

Uses a direct provider API key. You control model, provider, billing, and endpoint. Any OpenAI-compatible provider can be used. The token field in the provider block holds a ${VAR} reference to the API key; the provider field names the SDK type (openai, anthropic, or azure).

Both default_model and title_model on a BYOK provider must be models that the configured provider serves.

Anthropic BYOK - no streaming deltas. The Copilot SDK CLI does not emit assistant.message_delta events for Anthropic (copilot-sdk#637). Responses arrive as a single assistant.message. The agent-server synthesises deltas (text-complete → text-delta) so the UI still streams visually, but the wall-clock latency is end-of-response, not word-by-word.

Anthropic BYOK - incomplete token accounting. assistant.usage events don't include cacheReadTokens / cacheWriteTokens for Anthropic BYOK. The Input tokens counter in the UI only reflects non-cached input tokens, so sessions with heavy cache reuse show a number significantly lower than the figure in the Anthropic console. Tracked in copilot-cli#1152. GitHub mode is unaffected.

See Configuration → Inference Providers for the full schema and provider setup examples. See Architecture → Authentication → Per-provider LLM authentication for how the resolved credentials are applied at session-creation time.

Model configuration

Each agent selects its model via the model: field in its YAML. The model is resolved within the agent's chosen provider - for BYOK providers it must appear in available_models; for GitHub providers it must be in the Copilot catalog. If the agent omits model:, the provider's default_model is used. There is no hardcoded global default - if neither is set the provider chooses its own model and the server logs a warning. Typical usage is to pair a fast model (data lookup agent) with a more capable one (complex editing agent); mixing providers across agents is supported.

The reload endpoint validates every agent's model against the provider's model list and returns a warnings array; unknown models block session creation at runtime. See Configuration System for reload paths.

Performance instrumentation

Every completed request logs a timing summary at info level:

{
  "msg": "Request timing summary",
  "data": {
    "totalMs": 179111,
    "timeToFirstEventMs": 6,
    "modelMs": 10579,
    "totalToolMs": 86643,
    "toolCallCount": 21,
    "askUserPausedMs": 81889,
    "slowTools": [{"tool": "ask_user", "ms": 78027}]
  }
}

Field	Meaning
timeToFirstEventMs	Time until the first SDK event arrives (provider latency).
modelMs	totalMs minus tool execution minus user-wait time (pure LLM thinking).
totalToolMs	Aggregate wall-clock spent in tool calls.
slowTools	Tools that exceeded 500 ms, sorted by duration (top 10).

Individual tool calls are logged at debug (<1 s), info (1–3 s), or warn (>3 s). Every SSE event carries a ts (epoch ms) field so clients can do timing analysis too.