Inference Service

The bundle does not ship, load, or download any model. Every configured model points at an HTTP inference endpoint — any service, self-hosted or third-party, that fulfils the contract below. This page covers what works out of the box; to support an endpoint whose request or response shape differs, see Customization.

Getting and serving a model

Three hosting options, all equally supported by the shipped provider:

Option	When it fits
A managed embeddings API	Fastest to start. Check that it serves the model you picked, and that sending your content off-site is acceptable — the composed object text and image thumbnails leave your infrastructure.
A self-hosted OpenAI-compatible server	Several open-source inference servers expose exactly this API for a model id from a public model hub.
A custom service	When you need your own preprocessing, or an image tower a generic server does not expose.

Model weights themselves come from a model hub (Hugging Face and comparable registries); the server you run fetches them, not Pimcore.

info

Pimcore provides a shared Hugging Face Space (pimcore/local-inference-service) that fulfils the HTTP contract below for the default models. It exists for development and testing purposes only and is not intended for production use — there are no availability, capacity, or model-stability guarantees, cold starts can take minutes, and everything you embed (composed object text, image thumbnails, search queries) is transmitted to a public, shared endpoint. It can be used at your own risk; for production, serve a model yourself using one of the hosting options above.

Base URL https://pimcore-local-inference-service.hf.space, endpoints per the contract below:

Endpoint	Modality	Serves (default models)
POST /text-embedding	text	nomic-ai/nomic-embed-text-v2-moe, google/siglip2-base-patch16-256 (text tower)
POST /image-embedding	image	google/siglip2-base-patch16-256 (vision tower)

The Space publishes its full API at /openapi.json. The first request per model may cold-start the Space and download weights — allow minutes, then retry. The Getting Started example config points at these endpoints.

Three operational tips:

• Pin the model revision. A model name is not a version. If the weights behind a name change, the vectors change with them — silently, because the bundle's identity key is computed from the input, not the weights. When that happens (a pinned upgrade, or a provider-side change you learn about afterward), bump that model's serving_version, which exists precisely for this.
• Pre-cache the weights into your image layer or a persistent volume. Otherwise, the first request after every deployment blocks on a multi-gigabyte download, and cold-start timeouts look like inference failures.
• Keep the runtime offline. A request naming an arbitrary model must not be able to trigger a download — both for predictable start-up and because many models load with remote code enabled. Cache the models you trust, then disable fetching.

note

Using a third-party API is a deliberate trade: you avoid running GPUs, and in exchange your content is transmitted to that provider and its model versioning is outside your control. Configure auth_token from an environment variable, prefer https, and treat a provider-side model upgrade as a serving_version bump.

HTTP contract

The default provider (provider: default) POSTs to the model's endpoint URL (http/https only):

{ "model": "nomic-ai/nomic-embed-text-v2-moe", "input": ["text one", "text two"] }

and expects the OpenAI-compatible embeddings shape back:

{ "data": [ { "index": 0, "embedding": [0.01, ...] }, { "index": 1, "embedding": [...] } ] }

• Batching: up to 96 texts or 16 images per request (image inputs are base64-encoded thumbnails, so their request bodies are large).
• Retries: transient failures (network, 429, 5xx) are retried with backoff; a permanent failure degrades the affected elements and is repaired by the next backfill run.
• Partial results: if the service cannot embed one input, it should omit that index rather than fail the batch — the bundle then degrades only that element.
• Auth: a configured auth_token is sent as Authorization: Bearer <token>. Endpoints with a different auth scheme (e.g. Azure's api-key header) or extra header requirements use the model's options.headers, which override same-named defaults — see custom request headers. Extra request body parameters can be added via options.params.

A different request/response shape is supported by writing your own provider — see Writing a Custom Embedding Provider.

Serving invariants

These are correctness requirements, not suggestions — each one was the root cause of a real, hard-to-see quality failure during development:

1. Return unit-length vectors. Every embedding must be L2-normalized, from every code path of the service. The kNN fields score in an L2/cosine space, and unnormalized vectors rank partly by their length: the symptom is uniformly tiny, near-identical scores with irrelevant top results — nothing errors, the search just gets quietly wrong.
2. Preprocess deterministically, and version the preprocessing. Stored vectors are identified by the text as sent. Any serving-side change — a tokenizer fix, text canonicalization, image resizing — changes vectors without changing that text, leaving every stored vector silently stale. Whenever such a change ships, bump that model's serving_version in the bundle config; it is part of the vector identity key exactly for this.
3. Keep the dimension fixed per model. dimension is required in the model config, and the bundle refuses to index vectors of the wrong size instead of poisoning the index.
4. Do not add instruction prefixes server-side. The bundle prepends the configured prefixes.document / prefixes.query before sending; a service adding its own would double them.

Cross-modal models (text → image search)

Image search uses a two-tower model: a vision tower embeds the indexed images, a text tower embeds the incoming queries, and both meet in one latent space. Configure them as two models sharing a shared_space, paired in the usage:

models:
    my-vision-model:
        modality: 'image'
        shared_space: 'siglip2-base'
        # ...
    my-vision-text-model:
        modality: 'text'
        shared_space: 'siglip2-base'
        # ...
usages:
    image_search:
        model: my-vision-model
        query_model: my-vision-text-model

The registry rejects a pairing whose spaces differ or are unset — matching dimensions alone would let two unrelated models produce confidently wrong results. Both towers must be served with the same preprocessing discipline (invariant 2 applies to the pair as a whole).