Progress and Cancellation

A re-embed (the Studio Re-embed button or bpt:embeddings:reindex) does not run synchronously — it dispatches work and returns immediately after JobRun finished. The actual vector generation is still queued behind it. This page covers the two things that give you a real answer: the status endpoint and the cancel endpoint.

The status endpoint

GET /configurations/{configurationId}/status

PIMCORE_ADMIN-gated, same as the Re-embed endpoint.

The three numbers

Progress is reported as three numbers, never collapsed into one percentage:

• Processed — eligible elements that already carry a current vector.
• Failed / backfilled — backfilled (top-level), the count of elements the failure-backfill policy caught since the active run started.
• Remaining — staleSinceRun per model while a run is active (missingVectors when none is), the elements still to go.

Per model/element-type entry:

Field	Meaning
modelId, elementType	Which model and which element type (object or asset) this entry covers.
totalEligible	Total elements in scope for this model (DB-side count, stable even while the index itself is still filling).
missingVectors	Eligible elements with no vector at all.
staleSinceRun	Eligible elements that are missing or were last painted before the active run started — 0 when no run is active. This is the number that answers "how much is left for the run in progress".
mappingReady	false when the live index mapping doesn't have this model's vector field yet (new class, mapping not yet updated). No engine count is attempted in that case — the endpoint never lets an unmapped nested.exists probe 500.

Queues

queues reports the three pipeline stages a run drains through, plus the failure transport:

Field	Stage
gdiIndexQueue	GDI index-update queue (elements enqueued for (re)compose).
gdiTransport	GDI messenger transport (in-flight compose messages that can still spawn generation messages).
embeddingQueue	The embedding generation queue (pimcore_backend_power_tools_embedding_queue).
embeddingFailed	Retry-exhausted messages parked in pimcore_backend_power_tools_embedding_failed. Informational only — see Completion below.

-1 means not countable, not zero. Some transports (RabbitMQ in particular) cannot report an exact depth for every slot; a -1 must never be read as "drained". Queue counts are also shared-queue numbers — organic cache-miss traffic mixes in — and batch-, not element-granular. Treat them as "pending work", not as run progress; staleSinceRun is the progress number.

The _painted stamp

Coverage alone (missingVectors) cannot show progress on a full re-run over elements that already have vectors — the fields already exist, so "missing" reads 0% for the whole run. To fix this, every vector writing also stamps a sibling field, emb_<modelId>[__<format>]_painted (Unix seconds), on both incremental and full-mode paints. staleSinceRun then counts elements that are missing or whose paint stamp predates the active run's startedAt — the same cheap index count covers every scenario (initial run, new class added, full re-run).

Completion

A run is complete when, in the same observation:

• All three gating queues (gdiIndexQueue, gdiTransport, embeddingQueue) read 0. A -1 on any of them fails safe to "not complete" — it is never read as drained.
• staleSinceRun <= backfilled (summed across models), not ==. Backfill markers are only cleared by the CLI backfill-drain, never by a later successful paint, so an element that failed once and healed on retry stays counted on the marker side while leaving the stale side.

Cancelling a run

DELETE /configurations/{configurationId}/reembed

The same permission gate as the other embedding-configuration endpoints. Use it when a run was started with the wrong configuration and needs to stop before it burns more inference calls.

What it does:

1. Cancels the running Generic Execution Engine job run, if any.
2. Purges the embedding generation queue (pimcore_backend_power_tools_embedding_queue) — but only on the doctrine transport. The purge is wrapped behind EmbeddingQueuePurgerInterface; an AMQP (or other) transport needs its own adapter to be purged the same way. Purging is safe because everything in this dedicated transport is redundant by design — anything genuinely still needed is re-detected by the reconcile maintenance task or on the next composition.
3. Flips the run row to cancelled, so the completion maintenance task never sends a "finished" notification for a run that was stopped on purpose.
4. Clears this run's backfill markers (marked_at >= run.startedAt) so the aborted run doesn't leave half-tracked failure records behind; the next run re-derives everything from scratch.

The GDI index queue is never purged. In incremental mode it holds UPDATE rows that are indistinguishable from organic index updates — purging it would silently drop real structural index changes with no healing path. Cancel therefore only stops dispatch and purges the embedding transport; already-enqueued GDI rows still compose normally (harmless, idempotent) and may still trickle a few more generation messages afterward.

Cancel stops the run within one batch, not instantly. A worker mid-inference finishes its current batch (seconds to a minute) before the purge takes effect on anything still queued behind it.