Hugging Face Fine-tune Models

The following action steps use the Pimcore Fine-Tuning Service to fine-tune AI models on data managed within Pimcore.

Supported tasks are image and text classification. Both rely on data filtered by the Filter Data Objects or Filter Assets action steps, and each consist of two steps: preparing data and starting training. Add the Cleanup Tmp Files action step at the end of the automation action to remove the temporary export file afterwards.

Training runs on a Pimcore Fine-Tuning Service instance and is monitored by the start training action step. The Fine-Tuning Service can be hosted on-premises or in a Hugging Face space; see its README for hosting details. Use GPU instances for training; the required GPU size depends on your training data size and the model being fine-tuned.

Once a training job finishes, the fine-tuned model is uploaded to the Hugging Face Hub, and other Copilot action steps can then classify newly added data with it.

Image Classification

Image classification is based on tags assigned to the image assets.

Preparing Data

The Hugging Face Prepare Training Asset Classification step:

• Reads the filtered assets from the job run context. Only image assets are processed; everything else is logged and skipped.
• Extracts the classification from each asset's tags, based on the parent_tag_path setting. It uses the first leaf tag it finds (a tag with no children); assets with no matching classification tag are skipped.
• Calculates thumbnails of the assets based on the thumbnail setting. Use the same thumbnail configuration later when applying the fine-tuned model, so training and inference see identically scaled images.
• Packs all thumbnails, in a folder structure that reflects the classification, into a zip file named huggingface-training-export/JOBRUN_ID.zip in the temp folder.
• Adds the zip file to the clean-up list for later removal by the Cleanup Tmp Files action step.

Configuration options:

• thumbnail (string, optional): Name of an existing Pimcore thumbnail configuration to use for the exported images. Defaults to an inline 300px-wide JPEG thumbnail if not set.
• parent_tag_path (string, required): Root path of the tag tree used for classification. The path is matched as a plain string prefix with no path-boundary check, so /Colors also matches tags under /Colors Archive.

Starting Training

The Hugging Face Start Training Asset Classification step:

• Gets the training file from the job run context.
• Uploads the training file and starts training on the configured Pimcore Fine-Tuning Service instance.
• Waits for training to finish.

Only one training can run on a Fine-Tuning Service instance at a time. If another training is already in progress, the step waits about two minutes and then fails.

Configuration options:

• project_name (string, required): Project name, also used as the name of the resulting model.
• base_url (string, required): URL of the Pimcore Fine-Tuning Service instance.
• access_token (string, required): Access token for the Pimcore Fine-Tuning Service instance. Must match the AUTHENTICATION_TOKEN configured on that instance.
• source_model (string, required): Model to use as the base for fine-tuning.
• epocs (int, optional): Number of epochs for training. Default: 10.
• learning_rate (float, optional): Learning rate for training. Default: 5e-5.

Text Classification

Text classification is based on data fields assigned to the data objects. The value to classify on is generated from a Twig template.

Preparing Data

The Hugging Face Prepare Training Text Classification step:

• Reads the filtered data objects from the job run context.
• Extracts the classification from each data object using the target_field setting.
• Generates the value to classify on using the value_template setting.
• Packs all rows into a CSV file named huggingface-training-export/JOBRUN_ID.csv in the temp folder.
• Adds the CSV file to the clean-up list for later removal by the Cleanup Tmp Files action step.

Configuration options:

• value_template (string, required): Twig template that generates the value to classify on.
• target_field (string, required): Name of the data object field that holds the classification.

Starting Training

The Hugging Face Start Training Text Classification step:

• Gets the training file from the job run context.
• Starts training on the configured Pimcore Fine-Tuning Service instance.
• Waits for training to finish.

Configuration options:

• project_name (string, required): Project name, also used as the name of the resulting model.
• base_url (string, required): URL of the Pimcore Fine-Tuning Service instance.
• access_token (string, required): Access token for the Pimcore Fine-Tuning Service instance. Must match the AUTHENTICATION_TOKEN configured on that instance.
• source_model (string, required): Model to use as the base for fine-tuning.
• epocs (int, optional): Number of epochs for training. Default: 10.
• learning_rate (float, optional): Learning rate for training. Default: 5e-5.

Starting and Stopping Hugging Face Spaces

If your Fine-Tuning Service instance runs in a Hugging Face space, use the Hugging Face Space Start and Hugging Face Space Stop action steps to start it before training and pause it afterwards, so it only runs (and incurs cost) while training is active. Hugging Face Space Start restarts the space if it reports PAUSED or SLEEPING, then polls every 20 seconds until it reports RUNNING. There is no timeout, so a space stuck in a build or error state blocks the step indefinitely.

Both steps share the same configuration options:

• organization (string, required): Organization the space belongs to.
• space (string, required): Name of the space.

Sample Training Action Configuration

A typical automation action for asset classification fine-tuning consists of the following steps:

1. Filter Assets
2. Hugging Face Prepare Training Asset Classification
3. (Optional) Hugging Face Space Start
4. Hugging Face Start Training Asset Classification
5. (Optional) Hugging Face Space Stop
6. Cleanup Tmp Files