Defaults and Benchmarks
This page explains why the defaults are what they are, with the numbers that decided each one. It exists so you can judge whether a default fits your data.
Every measurement here comes from the demo dataset: one product category (Car), in one language
(English), around 300 objects, and 338 image assets, with several near-identical variants per car
model.
Use the Evaluation Tool to analyze and reproduce any of this on your own content before treating a default as the right choice for you.
Summary
| Decision | Default | Reason |
|---|---|---|
| Text model | nomic-ai/nomic-embed-text-v2-moe (768-d) | Multilingual and Apache-2.0, and it beats the English-only predecessor on English too — so the language support costs nothing. |
| Image model | google/siglip2-base-patch16-256 (768-d), two towers, one shared_space | The only multilingual image family among the candidates, in its cheap base variant — the English-only alternative cannot serve non-English queries at all. |
| Text representation | values-only (keyless) | Field names spend the model's attention on schema words nobody queries. |
| Chunk size / overlap | 1000 / 150 characters, per model | A convention — see below. |
| Image thumbnail | 256×256 cover, JPEG q90 | Matches siglip2-base-patch16-256's native input; a smaller thumbnail is upscaled and throws detail away. |
| Compression | on_disk + 32x binary | ~13× less graph memory at no accuracy cost on this dataset. |
| Duplicate detection | content hash + vector + MinHash | A vector score alone over-merges; the exact hash is the only fully reliable signal. |
Models
The candidates
The defaults are the two multilingual models in this table. That is the whole story: multilingual support became a hard requirement, and among the candidates that have it, these were the cheapest with a commercially usable licence.
| Model | Role | Multilingual | Dim | Rel. speed | Weights | Licence |
|---|---|---|---|---|---|---|
nomic-embed-text-v2-moe (default) | text | yes (~100 langs) | 768 | ~1.2× † | 1.77 GB | Apache-2.0 |
siglip2-base-patch16-256 (default) | image + query tower | yes | 768 | ~1.4× img, ~1.5× text † | 1.40 GB | Apache-2.0 |
nomic-embed-text-v1.5 | text | no | 768 | 1.0× (text baseline) | 0.51 GB | Apache-2.0 |
nomic-embed-vision-v1.5 | image | no | 768 | 1.0× (image baseline) | 0.35 GB | Apache-2.0 |
clip-vit-base-patch16 | image | no | 512 | ~0.9× img † | 0.56 GB | MIT |
Qwen3-Embedding-0.6B | text | yes | 1024 | 4.3× slower | 1.11 GB | Apache-2.0 |
siglip2-so400m-patch14-384 | image | yes | 1152 | 16.4× slower | 4.23 GB | Apache-2.0 |
Weights is the model's safetensors size, measured from the served model cache. It is comparable
across rows but is not runtime memory.
Rel. speed is per modality — text models timed on text, image models on images, each against the
Nomic model of the same modality. For multilingual support Qwen3 was several× slower,
so400m tens of× slower.
Candidates excluded before benchmarking
| Model | Multilingual | Why it never reached the run |
|---|---|---|
jina-clip-v2, jina-embeddings-v3 | yes | CC-BY-NC-4.0 — non-commercial; unusable in a shipped product without a paid licence. |
jina-embeddings-v4 | yes | Qwen Research Licence (commercial on request); 3.8B params and 2048-d inflate GPU and index cost. |
Cohere Embed v4 | yes | Commercial API only, not self-hostable — contradicts the local-inference default and adds per-token cost. |
BGE-M3, multilingual-e5-large, LAION XLM-R CLIP | yes | MIT and viable for text, but BGE-M3 has no image tower, so an image model would still be needed alongside. |
Image thumbnail
The image model embeds a Pimcore thumbnail, not the original file, and its size is a real quality decision rather than a formality:
| Setting | Recommended | Why |
|---|---|---|
| Dimensions | 256×256 (cover) | The native input of siglip2-base-patch16-256. |
| Format / quality | JPEG, 90 | A lossless format costs bandwidth per request without changing what the model sees; below ~85, compression artefacts start reaching the vectors. |
| Method | cover | Fills the square by cropping rather than letterboxing — padding bars are content the model would otherwise embed. |
Err upward rather than downward when unsure: a larger thumbnail is downscaled correctly by the service, while a smaller one is interpolated up and cannot recover detail that was never there.
Each image model names one via thumbnail, so the size is always a
choice you made. Its pixel-affecting settings are fingerprinted into the identity key, so changing the
definition re-embeds exactly that model's images.
Text data format
The composed text can name its fields or not. In the beginning there were eight different candidates compared from which three best performing candidates were selected for the further benchmark. Candidates were compared on identical data, queries, and index settings:
Known-item retrieval — one query per object, "did the right object come back in the top 10":
| Format | Whole-document hit@10 | Chunked hit@10 |
|---|---|---|
| values-only | 0.975 | 1.000 |
| key-value | 0.961 | 0.989 |
| position-labeled | 0.936 | 0.993 |
Curated relevance labels — precision within the first page, scored against handwritten label files (24 general + 7 attribute queries):
| Format | P@10 (general) | P@10 (attribute) |
|---|---|---|
| values-only | 0.47 | 0.27 |
| key-value | 0.42 | 0.21 |
| position-labeled | 0.42 | 0.24 |
Both metrics put values-only first, and the reason is consistent: human scaffolding dilutes the signal. Field names ("cylinders: 4" instead of "4") and full sentences ("The manufacturer is…") spend the model's limited attention on schema words no user ever types.
Chunking
| Parameter | Default | Basis |
|---|---|---|
max_text_chars | 1500 per model | (512 − 6 prefix − 10 %) × 3.5 chars/token ≈ 1,590, rounded down: the default model's window for European languages. English (4 chars/token) has room to spare, Russian (3) sits at the edge, denser scripts need a lower value — see the sizing table in Configuration. The former fixed 2,000-character threshold was ≈500 English tokens and overran the window for German and dense scripts. |
chunk_chars | 1000 per model | 250 tokens × 3.5–4 chars/token, a standard retrieval size; passage retrieval is sharpest between ~200 and ~500 tokens and the curve is flat there for most models. Never swept on this corpus. |
| Overlap | 15 % of chunk_chars | Convention, so a sentence straddling a cut survives whole in one chunk. |
| Fusion across chunks | maximum | Locked for the first version; max did beat mean for the default format. |
max_input_chars | 500000 (global) | A cost guard, bounding how many requests one element can trigger — not a quality knob. Raised from 20000 when document assets entered the scope: a mid-sized PDF or workbook is 30–40k characters of text, and the nested kNN probe showed a long document takes one result slot however many chunks it has, so a higher cap cannot crowd other elements out. |
Budgets are characters, not tokens, on purpose: provider, endpoint and model are replaceable and none exposes a tokenizer, so the operator converts the model window to characters for their content language (see Configuration).
Why to chunk at all? On the same run, chunking improved 7 of 8 candidate formats (up to +0.225 recall) and took values-only from 0.975 to a perfect 1.000 hit@10.
Document and text extraction limits
How much text is read out of an asset is governed by the single Max input characters cap, plus a few fixed guards that protect the indexing worker's memory. What each one is, per source, is listed in Configuration; this section is why those numbers.
Why one knob, in characters. Rows and characters only line up for one row width: a 20,000-row cap would cut a narrow ID list at 400,000 characters, below the default cap, while a 256-column sheet reaches the cap after about 40 rows. Stopping at the character cap is the same rule for every shape. The flip side is inherent to row-major text: a very wide sheet yields only its first few dozen rows, and the sheets after the one that filled the cap are not read at all.
Why the memory guards are what they are. PhpSpreadsheet parses a workbook's shared-string table and each sheet's XML in full before any read filter runs, at several times the XML size in memory, so a highly compressible upload has to be refused by its uncompressed size up front: 32 MB keeps that parse near 200 MB. The read filter cannot see cell values, so it cannot apply the character cap while the file is parsed; it bounds what is loaded instead. PhpSpreadsheet holds roughly 1 KB per loaded cell — a 283-row demo workbook peaked at 20 MB — so 200,000 cells keep the loaded workbook near 200 MB as well, inside a 512 MB PHP memory limit, and the column cap stops a single vast row from spending that budget on cells no row-major text would ever include.
What one element costs, per text field, at three caps:
| Max input characters | Chunks | Nested vectors in evaluation mode (3 formats) | Inference requests (96 texts each) | Stored _source (768-dim floats) |
|---|---|---|---|---|
| 100,000 | ~118 | ~354 | 2 | ~2 MB |
| 500,000 | ~588 | ~1,764 | 7 | ~9 MB |
| 1,000,000 | ~1,176 | ~3,528 | 13 | ~18 MB |
The default of 500,000 is the practical upper bound: it keeps a document's _source in the tens of
megabytes even with evaluation mode on, and stays well clear of OpenSearch's 10,000 nested objects per
document.
Compression
The default is on_disk + 32x binary quantization — the Binary, disk-backed (recommended)
preset in the Studio editor's Quantization select — because vector storage dominates cost at scale.
Measured on real vectors:
| Strategy | Dedup recall@1 | Search recall@10 | Latency | Graph memory |
|---|---|---|---|---|
| float32 | 1.000 | 1.000 | 12.2 ms | 1734 KB |
| int8 | 1.000 | 0.986 | 11.8 ms | — |
| int4 | 0.860 ✗ | 0.804 ✗ | 15.4 ms | 185 KB |
| binary (32x) | 1.000 | 1.000 | 27.6 ms | 133 KB |
551 documents, nomic_text; pass bars were dedup recall@1 ≥ 0.99 and search recall@10 ≥ 0.95.
int4 fails both — it is not a supported option for that reason.
Storage at scale, 768 dimensions:
| Level | Per vector | @ 1M vectors |
|---|---|---|
1x float32 | 3.0 KB | ~3.15 GB |
4x int8 | ~0.75 KB | ~0.9 GB |
32x binary | ~0.1 KB | ~92 MB floor, ~242 MB |
Binary is the default because it passed every quality bar on real vectors while using roughly 13× less graph memory than float32. It also brings some downsides:
- Longer query latency due to rescoring and disk storage.
- It relies on rescore oversampling (×3.0), which keeps a full-precision copy on disk.
Quantization is per-model, and changing it needs only a reindex — no re-embedding — so it is inexpensive to
test alternatives. For uncompressed vectors, pick the Full precision (float32) preset in Studio,
or set quantization: { mode: in_memory, compression_level: '1x' } directly in yaml.
Reproducing this
All of it comes from the Evaluation Tool, which runs against a real index with curated label files. The tool can be used to reproduce any of the above measurements on your own data and to compare alternative models, formats, and compression strategies. It is the recommended way to know whether a default is right for your content.