House Keeping

For certain processes within the portal engine it is needed to store temporary files in the file system. The portal engine should do quite a good job in cleaning them automatically.

This section gives a short overview over those files and how long they are stored.

var/tmp/portal-engine/json-tmp

Used to store JSON tmp files for the JSON download format. Will be deleted as soon as the JSON got downloaded or added to the zip.

var/tmp/portal-engine/csv-tmp

Used to store CSV tmp files for the CSV download format. Will be deleted as soon as the CSV got downloaded or added to the zip.

var/tmp/portal-engine/zip-tmp

Used to generate download zip files. For each download zip a TmpStore entry will be created too which stores some meta information.

The generated download zip files ready for downloading will be stored in /var/tmp/portal-engine/downloads. For more details see below.

The download zip file and TmpStore entry will be deleted as soon as the final zip file got downloaded.

If the final download will not be downloaded by the user or something breaks in the zip generation process, the download batch task and the related tmp files will be deleted after a configurable amount of time by a maintenance task.

pimcore_portal_engine:
    batch_task_queue:
        cleanup:
            cleanup_uncompleted_tasks_after_hours: 24
            cleanup_finished_tasks_after_hours: 72

Additionally, zip upload files will be stored in this directory. They are cleaned as soon as the upload got finished and in addition to that a maintenance task checks for these files too and deletes them after 24 hours.

/var/tmp/portal-engine/downloads

Generating download zip files is done asynchronously via Symfony messaging queue. Thus, it is necessary, to place generation results in a shared location that they are accessible by other nodes, e.g. for delivering the download to the webserver.

While var/tmp/portal-engine/json-tmp, var/tmp/portal-engine/csv-tmp and var/tmp/portal-engine/zip-tmp are local tmp directories that are not shared between nodes (and only contain tmp files which need to be accessible only in their own node), /var/tmp/portal-engine/downloads needs to be a shared tmp folder.

It is used as storage location for download zips after they are generated by a message handler and are ready for download, or also as starting point for the next message handler when number of files exceeds the chunk size of one message.

/var/tmp/portal-engine/downloads is accessed via flysystem and thus its actual storage location can be easily configured via flysystem configuration:

flysystem:
    storages:
        pimcore.portalEngine.download.storage:
            adapter: 'local'
            visibility: private
            options:
                directory: '%kernel.project_dir%/var/tmp/portal-engine/downloads'