4. Apptainer Image Pull and Build¶

This section covers different methods for creating Apptainer container images. You can build images from Docker registries, local Docker images, or Apptainer definition files.

TOC

Pull from Docker Registry
Build from Existing Local Docker Image
Build from Apptainer Definition File
- Definition File Sections Explained
- Advantages of Definition Files

4.1 Pull from Docker Registry¶

Docker containers (primarily stored in https://hub.docker.com/) can be pulled as Apptainer container images. For example, if we want to pull a specific container from Docker Hub:

Search and find the container registry in https://hub.docker.com.
- For this example, the image we are after is https://hub.docker.com/r/staphb/trimmomatic/tags
Copy the version you prefer with the blue Copy button next to the corresponding tag. We will use latest as an example
- It will be in the form of docker pull staphb/trimmomatic:latest
- In order to pull this as an Apptainer image, remove the word pull and create a URL as docker://staphb/trimmomatic:latest
Then pull it with apptainer pull <nameforthelocalimage>.sif docker://staphb/trimmomatic:latest
- <nameforthelocalimage>.sif is the name of the file to be saved once pulled. File extension can be anything but it is ideal to use .sif making it easier to identify the container image

code

apptainer pull trimmomatic_0.40.sif docker://staphb/trimmomatic:latest

Example output:

❯  apptainer pull trimmomatic_0.40.sif docker://staphb/trimmomatic:latest
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
INFO:    Fetching OCI image...
28.2MiB / 28.2MiB [======================================================================================================] 100 % 17.0 MiB/s 0s
179.6KiB / 179.6KiB [====================================================================================================] 100 % 17.0 MiB/s 0s
72.0MiB / 72.0MiB [======================================================================================================] 100 % 17.0 MiB/s 0s
INFO:    Extracting OCI image...
INFO:    Inserting Apptainer configuration...
INFO:    Creating SIF file...

4.2 Build from Existing Local Docker Image¶

Apptainer supports local Docker image file conversion. This is useful when you have locally built Docker images that are:

Built locally on your personal computer or workstation
Not published to any public registry (Docker Hub, etc.)
Proprietary or contain sensitive information that cannot be shared publicly
Custom-built for specific research workflows

In these cases, you can convert Docker images to Apptainer SIF format for use on HPC systems like BMRC.

Step 1: Export Docker Image to tar Archive¶

On your local machine where Docker is installed, export your Docker image to a tar file:

code

# List your local Docker images
docker images

Example output:

REPOSITORY          TAG       IMAGE ID       CREATED        SIZE
myapp               latest    a1b2c3d4e5f6   2 days ago     1.2GB
custom-python       3.11      b2c3d4e5f6a7   1 week ago     800MB

# Export the Docker image to a tar file
docker save -o myapp.tar myapp:latest

For multiple images or specific image IDs:

# Using image ID
docker save -o myimage.tar a1b2c3d4e5f6

# Multiple images in one tar file
docker save -o multiple_images.tar image1:tag1 image2:tag2

File Size Considerations

Docker tar archives can be large (often several GB). Make sure you have sufficient disk space both locally and on the destination filesystem.

Step 2: Transfer tar File to BMRC¶

Transfer the tar file to your BMRC account:

code

# Using scp (from your local machine)
scp myapp.tar username@cluster2.bmrc.ox.ac.uk:/path/to/destination/

# Or using rsync for large files (recommended)
rsync -avP myapp.tar username@cluster2.bmrc.ox.ac.uk:/path/to/destination/

Step 3: Build Apptainer SIF from tar Archive¶

On BMRC, convert the tar archive to Apptainer SIF format:

code

# Build SIF from Docker tar archive
apptainer build myapp.sif docker-archive://myapp.tar

Example output:

INFO:    Starting build...
INFO:    Extracting Docker image from archive...
INFO:    Creating SIF file...
INFO:    Build complete: myapp.sif

Verifying Your Converted Image¶

After conversion, verify that your Apptainer image works correctly:

code

# Inspect the container metadata
apptainer inspect myapp.sif

# Test by entering a shell
apptainer shell myapp.sif

# Run a test command
apptainer exec myapp.sif <your-command>

4.3 Build from Apptainer Definition File¶

Apptainer definition files (.def files) are the native way to build Apptainer containers. They provide more control and features specific to HPC environments.

Example Definition File¶

Here's an equivalent Apptainer definition file with the same content as the Dockerfile example:

code

cat << EOF > myimage.def
BootStrap: docker
From: ubuntu:20.04

%post
    # Update and install basic tools
    apt-get -y update
    apt-get install -y wget curl
    apt-get clean
    rm -rf /var/lib/apt/lists/*

%environment
    export LANG=C.UTF-8
    export LC_ALL=C.UTF-8

%runscript
    echo "Container started in /workspace"
    cd /workspace
    exec /bin/bash "$@"

%labels
    Author your.email@example.com
    Version v1.0.0

%help
    This is a basic Ubuntu 20.04 container with wget and curl installed.
    Use: apptainer shell myimage.sif
EOF

Build from Definition File¶

Although BMRC does not have a dedicated sandbox to build containers at the moment, fakeroot is enabled in both the login nodes and compute nodes allowing researchers to build containers as needed.

Build a container on a BMRC login node

Build command follows the format of apptainer build --force --fakeroot NAME_OF_THE_CONTAINER.sif name_of_the.def

Important: If you have $APPTAINER_BIND set globally, unset it with unset APPTAINER_BIND for apptainer build commands. Otherwise the build commands will fail as bind mounts set via this environment variable are NOT supported during build operations.

The build process executes in a special, isolated environment and does NOT honor the APPTAINER_BIND or APPTAINER_BINDPATH environment variables.
Only directories specified with the --bind argument are recognized and mounted during the build's %post or %test sections, if the destination mount points already exist in the bootstrap image

unset APPTAINER_BIND

apptainer build --force --fakeroot myimage.sif myimage.def

Build a container on BMRC via Slurm job

Since some container builds can consume a significant amount of CPUs and memory, our recommendation is to perform these builds via a Slurm job (either interactive or batch queue). Unfortunately, BMRC compute nodes do not have www access by default. Please contact the KIR Research Computing Manager for assistance configuring your environment to allow compute nodes www access.

Definition File Sections Explained¶

BootStrap: Specifies the bootstrap agent (docker, library, localimage, etc.)
From: The base image to start from
%post: Commands to execute during build (like RUN in Dockerfile)
%environment: Environment variables set at runtime
%runscript: Commands executed when container is run with apptainer run
%labels: Metadata labels for the container
%help: Help text displayed with apptainer run-help

Advantages of Definition Files¶

Reproducibility: Definition files are text-based and version-controllable
HPC-specific features: Better integration with HPC environments
Sections: Clear separation of build-time and runtime configurations
Documentation: Built-in help and label sections