Skip to content

2. Understanding and Configuring Apptainer Cache and Temporary Directories¶

Why This Matters¶

By default, Apptainer will attempt to use your home directory for all operations, creating a hidden directory ~/.apptainer for this purpose. Since home directories are limited to 20GB, we recommend redirecting these operations to the nobackup filesystem to avoid quota issues.

What Are These Directories?¶

APPTAINER_CACHEDIR (Local Cache Directory)¶

This is where Apptainer stores downloaded container images, layers, and other reusable components. When you pull a container image from a registry (like Docker Hub or Sylabs Cloud), Apptainer downloads and caches the image layers here. The cache helps speed up subsequent operations by avoiding redundant downloads. Over time, this directory can grow significantly, especially if you work with multiple large container images.

APPTAINER_TMPDIR (Temporary Directory)¶

This directory is used for temporary files during container build and execution operations. Apptainer uses this space for:

  • Unpacking and assembling container layers
  • Storing temporary files during container builds
  • Managing overlay filesystems when running containers
  • Other transient operations that require scratch space

These temporary files are typically cleaned up after operations complete, but the directory can require substantial space during active operations, particularly when building large containers.

Setting Up Your Environment¶

To configure Apptainer to use filesystem other than your home directory, set the following environment variables before running Apptainer.

We recommend using scratch file system (/gpfs3/well/kir/scratch/group) for both cache and temporary.

code

  1. create a dedicated directory for yourself on scrach ( make sure to replace group with your research group) 
    mkdir -p /gpfs3/well/kir/scratch/group/$USER_apptainer_cache

    export APPTAINER_CACHEDIR="/gpfs3/well/kir/scratch/group/$USER_apptainer_cache"
export APPTAINER_TMPDIR=${APPTAINER_CACHEDIR}

Making Changes Permanent

We recommend adding these environment variables to your ~/.bashrc file so they're automatically set for all future sessions. This ensures consistent behavior and prevents accidentally filling up your home directory quota.

echo 'export APPTAINER_CACHEDIR="/gpfs3/well/kir/scratch/group/$USER_apptainer_cache"' >> ~/.bashrc
echo 'export APPTAINER_TMPDIR=${APPTAINER_CACHEDIR}' >> ~/.bashrc

Alternatively you can use symlink for to create a soft link to the scratch

code

# Create apptainer cache folder
mkdir -p /gpfs3/well/kir/scratch/group/$USER_apptainer_cache

# symlinking
ln -s /gpfs3/well/kir/scratch/group/$USER_apptainer .apptainer

🧹 Cleaning the Apptainer image cache¶

We can remove container images from the cache using the apptainer cache clean command. Running the command without any options will display a warning and ask you to confirm that you want to remove everything from your cache.

$  apptainer cache clean
This will delete everything in your cache (containers from all sources and OCI blobs).
Hint: You can see exactly what would be deleted by canceling and using the --dry-run option.
Do you want to continue? [y/N] y
INFO:    Removing blob cache entry: blobs
INFO:    Removing blob cache entry: index.json
INFO:    Removing blob cache entry: oci-layout
INFO:    No cached files to remove at /gpfs3/well/kir-scratch/sansom/mat611-apptainer-cache/cache/library
INFO:    Removing oci-tmp cache entry: 21fa01d221dc3926f97e47a50ad2ad9550680d10f504583e5069777b30983832
INFO:    No cached files to remove at /gpfs3/well/kir-scratch/sansom/mat611-apptainer-cache/cache/shub
INFO:    No cached files to remove at /gpfs3/well/kir-scratch/sansom/mat611-apptainer-cache/cache/oras
INFO:    No cached files to remove at /gpfs3/well/kir-scratch/sansom/mat611-apptainer-cache/cache/net
  • You can also remove specific container images or all container images of a particular type. Look at the output of apptainer cache clean --help for more information.