Progress and Cancellation

A re-embed does not run synchronously. Save & Re-embed and Save & Full Re-embed return as soon as the work is queued; the actual vector generation is still behind the workers. Neither transport draining is proof that a run has finished either. This page covers the two things that give a real answer: the status banner in Pimcore Studio, and the endpoints behind it.

Following a run in Studio

While a run is active, a banner above the tabs of System > Embedding Configuration reports its progress, and a Cancel re-embed button appears in the toolbar.

The banner reports, from left to right, how many eligible elements have been processed out of the total, how many failed, and whether the run is an incremental Re-embed (changed items) or a Full re-embed. Once all work is processed it switches to confirming completion until the run is formally closed.

When no run is active, the editor instead shows the last run's result: when it finished, how many elements are still missing a vector and how many failed.

Two warnings can appear instead of a number:

• Index mapping not ready for: … — the live index does not carry that model's vector field yet. A re-embed updates the mappings itself, so this normally resolves on its own within the same run.
• Progress cannot be measured until the index mappings are ready — the same cause, before any count is possible.

When the run finishes, Pimcore sends a notification to the user who started it.

The status endpoint

GET /configurations/{configurationId}/status

PIMCORE_ADMIN-gated, like the other embedding-configuration endpoints. This is what the banner polls, and what to use for your own monitoring.

The three numbers

Progress is reported as three numbers, never collapsed into one percentage:

• Processed — eligible elements that already carry a current vector.
• Failed / backfilled — backfilled (top level), the count of elements the failure-backfill policy caught since the active run started.
• Remaining — staleSinceRun per model while a run is active, missingVectors when none is.

Per model and element type:

Field	Meaning
modelId, elementType	Which model and which element type (object or asset) this entry covers.
totalEligible	Total elements in scope for this model. A DB-side count, stable even while the index is still filling.
missingVectors	Eligible elements with no vector at all.
staleSinceRun	Eligible elements that are missing or were last painted before the active run started; 0 when no run is active. This is the number that answers "how much is left for the run in progress".
mappingReady	false when the live index mapping does not have this model's vector field yet. No engine count is attempted in that case, so an unmapped nested.exists probe can never 500 the endpoint.

Queues

queues reports the three pipeline stages a run drains through, plus the failure transport:

Field	Stage
gdiIndexQueue	GDI index-update queue: elements enqueued for (re)compose.
gdiTransport	GDI messenger transport: in-flight compose messages that can still spawn generation messages.
embeddingQueue	The embedding generation queue (pimcore_backend_power_tools_embedding_queue).
embeddingFailed	Retry-exhausted messages parked in pimcore_backend_power_tools_embedding_failed. Informational only.

-1 means not countable, not zero. Some transports — RabbitMQ in particular — cannot report an exact depth for every slot, and a -1 must never be read as "drained". Queue counts are also shared-queue numbers (organic cache-miss traffic mixes in) and batch-granular, not element-granular. Treat them as "pending work"; staleSinceRun is the progress number.

The _painted stamp

Coverage alone cannot show progress on a full re-run over elements that already have vectors: the fields already exist, so missingVectors reads 0 for the whole run. Every vector write therefore also stamps a sibling field, emb_<modelId>[__<format>]_painted, with the Unix time of the write. staleSinceRun counts elements that are missing or whose stamp predates the active run's startedAt, so the same cheap index count covers every scenario: an initial run, a newly added class, and a full re-run.

Completion

A run is complete when, in the same observation:

• All three gating queues (gdiIndexQueue, gdiTransport, embeddingQueue) read 0. A -1 on any of them fails safe to "not complete".
• Every model entry reports mappingReady: true. A field whose mapping is not ready contributes a staleSinceRun of 0 simply because it is never counted, so that zero is unverifiable rather than drained — it fails safe to "not complete" for the same reason a -1 queue depth does.
• staleSinceRun <= backfilled summed across models — not ==. Backfill markers are only cleared by the CLI backfill drain, never by a later successful paint, so an element that failed once and healed on retry stays counted on the marker side while leaving the stale side.

Cancelling a run

Press Cancel re-embed in the toolbar and confirm. Use it when a run was started with the wrong configuration and should stop before it burns more inference calls. The underlying endpoint is:

DELETE /configurations/{configurationId}/reembed

What it does:

1. Cancels the running Generic Execution Engine job run, if any.
2. Purges the embedding generation queue — but only on the doctrine transport. The purge sits behind EmbeddingQueuePurgerInterface; an AMQP or other transport needs its own adapter. Purging is safe because everything in this dedicated transport is redundant by design: anything genuinely still needed is re-detected by the reconcile maintenance task or on the next composition.
3. Flips the run row to cancelled, so the completion maintenance task never sends a "finished" notification for a run that was stopped on purpose.
4. Clears this run's backfill markers, so the aborted run leaves no half-tracked failure records behind. The next run re-derives everything from scratch.

The GDI index queue is never purged. In incremental mode it holds UPDATE rows that are indistinguishable from organic index updates; purging it would silently drop real structural index changes with no healing path. Cancel therefore only stops dispatch and purges the embedding transport. Already-enqueued GDI rows still compose normally — harmless and idempotent — and may trickle a few more generation messages afterwards.

Cancel stops the run within one batch, not instantly. A worker mid-inference finishes its current batch, which takes seconds to a minute, before the purge takes effect on anything still queued behind it. Elements already processed keep their embeddings, and a new run can be started immediately afterwards.