Configuration
All options live under pimcore_backend_power_tools.embeddings. Where a default was a measured decision rather
than a convention, the numbers behind it are in
Defaults and Benchmarks.
Example
The configuration the benchmark was performed on, reduced to the models it actually used — one multilingual text model, and the two towers of a cross-modal image model. Every option left at its default is omitted, so nothing here is redundant:
pimcore_backend_power_tools:
embeddings:
enabled: true # master switch; off by default
assets:
enabled: true # embed image assets; off by default
objects:
classes: # ONLY these classes are embedded; empty = none
- Car
- News
models:
nomic-text-v2: # free id, referenced by `usages` below
modality: text # embeds text (objects + asset text)
endpoint: 'https://your-inference-host/text-embedding'
model: 'nomic-ai/nomic-embed-text-v2-moe' # multilingual, Apache-2.0
dimension: 768 # required; sizes the kNN field, validates responses
prefixes: # instruction prefixes this model is trained to expect
document: 'search_document: ' # prepended when embedding indexed content
query: 'search_query: ' # prepended when embedding a search query
timeout: 1800 # rose from 120s when the benchmark ran on CPU-only inference
siglip2-text: # TEXT tower: embeds the incoming image-search query
modality: text
endpoint: 'https://your-inference-host/text-embedding'
model: 'google/siglip2-base-patch16-256' # multilingual, Apache-2.0
dimension: 768
timeout: 600
shared_space: 'siglip2-base' # must match the vision tower exactly
siglip2-vision: # VISION tower: embeds the indexed images
modality: image # input is a base64 thumbnail
endpoint: 'https://your-inference-host/image-embedding'
model: 'google/siglip2-base-patch16-256' # multilingual, Apache-2.0
dimension: 768
timeout: 600
shared_space: 'siglip2-base' # same space as siglip2-text
usages: # which model serves which capability
semantic_text_search: nomic-text-v2 # shorthand: same model for documents and queries
object_dedup: nomic-text-v2
image_search:
model: siglip2-vision # embedded the indexed images
query_model: siglip2-text # embeds the text query (cross-modal)
image_dedup: siglip2-vision
excluded_fields: # field keys never embedded, at any nesting level
- internalNotes
Top level
| Option | Default | Description |
|---|---|---|
enabled | false | Master switch for all embedding behaviour (mapping, generation, indexing, kNN querying). Off by default: embeddings add index fields and call an external service, so they must be opted into — set it to true and configure at least one model and usage. |
excluded_fields | [] | Field keys never embedded, by both production indexing and the evaluation tool, so the two cannot disagree about an element's text. Changing it needs a reindex. |
store_embedded_text | false | Debug aid: also store the composed element text in an emb_<model>_source sibling field — without the model's instruction prefix, which generation prepends afterwards. Text inputs only; image inputs are never stored. Costs storage — keep off in production. |
evaluation_mode | false | When enabled, every text representation is mapped, generated and stored per element so the Evaluation Tool can compare them. Off, only the primary representation is stored. Changing it needs index recreation + reindex. |
cache_version | v1 | Folded into every vector's identity key; bump to invalidate all stored vectors at once. For a single model's serving-side change, prefer that model's serving_version. |
cache_database | '' | Optional schema name on the same MySQL/MariaDB server that hosts the embedding vector cache table. Empty (the default) keeps the table in Pimcore's main schema. Must match ^[A-Za-z0-9_]*$. |
cache_table_storage_engine | '' | Storage engine for the embedding vector cache table. Empty (the default) auto-detects one at table-creation time — see storage engine. An explicit value is verified to exist on the server and bypasses auto-detection, including its replication guard. Must match ^[A-Za-z0-9_]*$. |
cache_databasequalifies the cache table name on every read/write — it is injected straight into the repository.cache_table_storage_engineonly takes effect at table-creation time, via the Installer (fresh install) or the bundle's migrations (existing installation) — see Provisioning the table. Neither everALTERs an existing table onto a different engine.
assets and objects — the element scope
Which elements are embedded? Deny by default on both sides: enabling embeddings alone embeds nothing, because sending a catalogue to an inference service should be a decision, not a side effect.
| Option | Default | Description |
|---|---|---|
assets.enabled | false | Embed assets (image vector plus their text representation). Only image assets can produce input when this is true. |
objects.classes | [] | Data object class names to embed, e.g. [Car, News]. Empty means no data object is embedded. Changing this needs an index recreation + reindex. |
The image thumbnail
An image model embeds a Pimcore image thumbnail of the asset, never the original file, and names it
with thumbnail. Please define a thumbnail based on the dimensions your molde requires for the best results:
pimcore:
assets:
image:
thumbnails:
definitions:
embeddings_siglip2:
name: embeddings_siglip2
description: 'Image input for embedding generation (SigLIP 2 base, 256x256)'
format: JPEG
quality: 90
items:
- method: cover
arguments:
width: 256
height: 256
- Match the model's native input.
siglip2-base-patch16-256takes 256×256; feeding it 224 makes the service upscale and lose detail. A larger thumbnail is downscaled correctly, so err upward. - Per model. Two image models can use two definitions — useful when one wants 256 and another 384.
- Editing it re-embeds. The definition's pixel-affecting settings are part of the identity key, so a resize exactly invalidates that model's images. See Caching.
models
One entry per embedding model, keyed by a free id:
| Option | Default | Description |
|---|---|---|
modality | (required) | text or image. |
endpoint | (required) | Base URL of the HTTP inference service. Not part of the identity key — see model. |
model | (required) | Model name the inference service should load. Not part of the identity key: swapping it under the same config id does not invalidate stored vectors — bump serving_version alongside it (see Caching). |
provider | default | Which registered embedding provider talks to the endpoint. Not part of the identity key — see model. |
dimension | (required) | Vector dimension the model returns. It sizes the kNN field mapping — immutable once the index exists — and is the expectation every provider response is validated against, so a misconfigured endpoint is rejected instead of poisoning the index. Must be ≥ 1. Not part of the identity key — changing it re-embeds nothing by itself. |
timeout | 120 | Request timeout in seconds — embedding batches on CPU-only services can be slow. |
thumbnail | (required for image models) | Pimcore image thumbnail configuration used as the model input. |
prefixes.document / prefixes.query | '' | Instruction prefixes some models are trained to expect (e.g. search_document: / search_query: for Nomic; E5 uses passage: / query: ). The bundle prepends them before the request, so the endpoint must not add its own. Only prefixes.document is part of the stored-vector identity key, so editing it re-embeds that model's text vectors; prefixes.query only re-keys the query cache and re-embeds nothing. |
serving_version | '' | Version of the inference service's preprocessing for this model, folded into the identity key. Bump when serving-side preparation changes (tokenizer fix, normalization) — such changes are invisible to the text-based key, so stored vectors would otherwise stay silently stale. Empty keeps keys byte-identical to before the option existed. |
shared_space | null | Name of the latent space the model embeds into. Two models may only be combined in one search (cross-modal, e.g. text query → image field) when both declare the same space — equal dimensions are not sufficient and would return confident nonsense. |
auth_token | null | Optional bearer token for the endpoint; supports env vars. |
options | [] | Free-form provider-specific settings, passed through to the provider untouched. The default provider reads options.params (extra request body params) and options.headers (custom HTTP headers — see below). Not part of the identity key: if a change here alters how the service embeds (a pooling mode, a truncation flag, a version header selecting different preprocessing), stored vectors go silently stale — bump serving_version alongside it. |
quantization.mode | on_disk | OpenSearch kNN storage mode; on_disk trades a little latency for far less graph memory. Ignored on Elasticsearch. |
quantization.compression_level | 32x | 1x (float32) … 32x (binary, the default). Changing it needs a reindex (no re-embed — vectors carry over via index recreation). |
Custom request headers
Endpoints that don't authenticate with a bearer token (or need extra routing/tenant headers) can be
served without a custom provider via options.headers. Configured headers are added to every request
and replace a same-named default, compared case-insensitively — so an Authorization entry wins
over the auth_token bearer default. Like auth_token, values support env vars; never put a literal
secret in config:
models:
azure-text:
modality: 'text'
endpoint: 'https://my-resource.openai.azure.com/openai/deployments/embed/embeddings?api-version=2024-02-01'
model: 'text-embedding-3-small'
dimension: 1536
options:
headers:
api-key: '%env(AZURE_OPENAI_KEY)%' # Azure authenticates via api-key, not Bearer
Headers are transport details, not part of the vector identity key. If a header changes what the
service computes (e.g. an API-version header selecting different preprocessing), bump
serving_version alongside it. Header values may carry credentials — the bundle never logs them, and
neither should your own tooling.
usages
What each search capability embeds with. The default four usages are semantic_text_search,
object_dedup, image_search and image_dedup.
usages:
semantic_text_search: my-text-model # shorthand: documents AND queries use this model
image_search:
model: my-vision-model # embedded the indexed images
query_model: my-vision-text-model # embeds the incoming text query (cross-modal)
query_model defaults to model; when it differs, both must declare the same shared_space.
chunking
| Option | Default | Description |
|---|---|---|
max_input_chars | 20000 (min 2000) | Cost/scope guard: composed text beyond this is truncated before chunking (truncation is logged), bounding how many embedding requests one element can cause. Raise it for long-form content. |
cache_eviction
Controls bpt:embeddings:reindex --evict (see Caching):
| Option | Default | Description |
|---|---|---|
max_age_days | 90 (min 1) | Cache rows untouched for longer than this are pruned first. |
max_rows | 1000000 (min 1) | If the table is still over this row count after age-based pruning, the least-recently-used rows are removed down to the cap. |
Vector storage, reuse and index recreation
Vectors live in the search index documents and in a durable MySQL/MariaDB cache table, and are
reused via version-aware identity keys, so re-indexing unchanged content costs no inference calls.
cache_version, serving_version, the cache table, and the ways to force a re-embed are all
covered in Caching.
How the defaults were chosen
The measured reasoning behind the default model, text representation, chunking and compression choices lives in Defaults and Benchmarks.