4. Procedure for using Singularity¶
In order to use the application with Singularity, you need to acquire or create the necessary images. Here, we will follow the flow of creating an image by the user.
There are three main ways to create an image.
Get the original image. or reuse and convert other images.
Expand the file system that is the source of the image, manually install the application, etc., and convert it to an image file.
Let Singularity do all the work using the definition file that describes the procedure for creating the image.
The manual method is convenient for debugging and simple correction when the worker is familiar with environment construction and the work content is simple, or when a problem occurs during creation. However, if you do not save the work contents separately, it may not be possible to make an exact reproducution again. Rather than manually creating an image each time, creating a definition file, managing it in a repository, etc. so that it can be executed as a batch job enables high reproducibility and effective use of resources.
Singularity has two ways to create an image using a definition file, one is to use the local environment and the other is to use a cloud service. This document is limited to a description how to do it on local environment, such as FUGAKU or other ARM environment.
Singularity allows you to reuse existing container images to create your own. You can use not only Singularity images, but also Docker, OS yum/apt repositories, etc. In most cases, you will be making changes based on some base image. In this document, we will use AlmaLinux and Ubuntu distributed by Docker hub, as well as the Universal Base Image (UBI) supplied by Redhat for containers as a base.
4.1. FUGAKU Singularity environment¶
In FUGAKU, the login node is x86_64 architecture, so you need to work on the compute node to create an image for ARM architecture. First, let’s do some manual work as an interactive job to see how Singularity works. After April 2024, the previously required setting jobenv=singularity is no longer required.Check the version of singularity that you use. 4.1.2-2 as a default or 3.11-7 are available as follows.
$ pjsub --interact -L "rscunit=rscunit_ft01,rscgrp=int,node=1,elapse=0:30:00"
[INFO] PJM 0000 pjsub Job XXXXXXX submitted.
[INFO] PJM 0081 ..connected.
[INFO] PJM 0082 pjsub Interactive job XXXXXXX started.
[a0XXXX@b31-3208c ~]$ singularity --version
SingularityPRO version 4.1.2-2.el8
[a0XXXX@b31-3200c ~]$ /opt/singularitypro311/bin/singularity --version
SingularityPRO version 3.11-7.el8
[a0XXXX@b31-3200c ~]$ /opt/singularitypro41/singularity --version
SingularityPRO version 4.1.2-2.el8
In the above example, the working time is 30 minutes. After this, working contents other than those saved in the home directory etc. may be lost.
4.2. Image creation via the command line¶
When retrieving from docker hub or a repository that holds images in the same format. You can do it as follows.
$ singularity pull centos_latest.sif docker://centos:latest
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Getting image source signatures
Copying blob 333cbcae3fb8 done
Copying config 424ef6ddfd done
Writing manifest to image destination
Storing signatures
<<..snip..>>
INFO: Creating SIF file...
INFO: Build complete:
$ singularity pull ubuntu2004.sif docker://ubuntu:20.04
$ singularity pull ubi8-python38.sif docker://registry.access.redhat.com/ubi8/python-38
Generate an SIF file from docker compatible repositories. Once acquired, the docker format image is saved under ~/.singularity/cache, and only the part with some update will be downloaded from the next time onward, so it takes only a short time to complete. It is also possible to omit the image name (file name), in which case the file name generated from the repository name will be used. Also, although the architecture is not specified, Singularity tries to get the same architecture of the local machine, so it automatically gets AARCH64 image as recognized from the compute node. According to the user manual, even version 3.11, it is supposed to be possible to specify with the –arch option, but the source of download is limited to the 'library', and the architecture cannot be specified for other type of repository such as 'docker'.From the version 4.1 or later, you can do that as 'singularity pull –arch=arm64 docker://…' on the login node.
The main part of a SIF image is based on the SquashFS format. SquashFS is a direct mount of the tar.gz file. Hence, it is readonly. You cannot modify it after creation directly. There are two ways to integrate applications with runtime environment into the container image. One is using a definition file described later. Another is to extract required files into directory on local file system and then convert them to SIF after. The overlay feature, which overlays the writable areas of an image without directly modifying it, can be used to make changes that are only effective at runtime or to replace specific directories or files in a single SIF image.
The image expanded to the file system is called sandbox and is created by adding the –sandbox option to the build command. The save destination is a directory, and the image is expanded in it. In that directory, all files originally owned by root are owned by the user oneself, and the contents can be edited interactively without starting the container.
$ singularity build --sandbox ubuntu2004 docker://ubuntu:20.04
INFO: Starting build...
Getting image source signatures
Copying blob 32d7611b468c skipped: already exists
Copying blob e5be16fdc306 skipped: already exists
Copying blob a361e87bde5e [--------------------------------------] 0.0b / 0.0b
Copying config ff84d45488 done
Writing manifest to image destination
Storing signatures
2021/03/31 12:00:00 info unpack layer: sha256:32d7611b468cbc07986302f1a8e74d92e30a1f11cdfa8bc2900aedda2758d050
<<..snip..>>
INFO: Creating sandbox directory...
INFO: Build complete: ubuntu2004
$ cd ubuntu2004
$ touch foobar
$ ls -l
In fact, the usage so far makes no difference between the pull and build commands except with the –sandbox option. You can create an SIF file from the image extracted to sandbox and vice versa with the build command.
$ singularity build from_sandbox.sif ubuntu2004
$ singularity build --sandbox from_SIF ubuntu2004.sif
Note that the sandbox directory itself is of course writable, but booting the container from the sandbox image is read-only, just like the SIF image. This means that you cannot install the application on the system in the image by booting from the sandbox. Specify the –writable option if you want to boot in writable mode (not available for SIF images). However, you do not have root privileges, and you cannot use su etc. in the container to obtain root privileges. Therefore, package management commands such as dnf, yum, rpm, dpkg, etc. that require root privileges are enabled with –fakeroot option. However, it is possible to copy files from outside the container into sandbox and then convert them to a SIF image.
$ singularity exec from_sandbox.sif ls -l /foobar
Next, let’s start the container using the created image.
4.3. Launching a container from an image¶
One of the ways to start a container is to use singularity’s shell command to start a shell inside the container. Let’s compare it with the information of the host OS.
$ uname -a
Linux c27-0200c 4.18.0-513.5.1.el8_9.aarch64 #1 SMP Fri Sep 29 05:50:39 EDT 2023 aarch64 aarch64 aarch64 GNU/Linux
$ whoami
a0XXXX
$cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="8.9 (Ootpa)"
<<..snip..>>
$ singularity shell ubuntu2004.sif
Singularity> uname -a
Linux c27-0200c 4.18.0-513.5.1.el8_9.aarch64 #1 SMP Fri Sep 29 05:50:39 EDT 2023 aarch64 aarch64 aarch64 GNU/Linux
Singularity> whoami
a0XXXX
Singularity> cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
<<..snip..>>
Even though the OS of “FUGAKU” is Redhat Enterprise Linux 8.9, the kernel and users are still Ubuntu 20.04.2 LTS after the container is started. You can’t use any Redhat commands on the host OS here, only the Ubuntu commands in the image. However, since the kernel remains the same, applications that strongly depend on the function or version of the kernel may interfere with operation.
Then run ls, you can see the home directory as it is, as shown below. however, you can only see your own home directory. Try running ls -al /home, ls -al ~/.. to see this. You can see that the home directories of users other than yourself are not visible.
Singularity> ls -l
total 440172
drwxr-xr-x 1 a0XXXX rccs-aot 4096 Mar 31 12:30 ubuntu2004
-rwxr-xr-x 1 a0XXXX rccs-aot 26320896 Mar 31 12:00 ubuntu2004.sif
Singularity> ls -l ..
drwx------ 1 a0XXXX rccs-aot 4096 Mar 31 12:30 a0XXXX
If you also do cat /etc/passwd or cat /etc/group, you can see that only your own entry has been added. Therefore, if you want to access another user’s home directory from inside the container, you need to devise something.
Singularity> cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
<<..snip..>>
_apt:x:100:65534::/nonexistent:/usr/sbin/nologin
a0XXXX:x:1XXXX:1XXXX::/home/rccs-aot/a0XXXX:/bin/bash
As you can see by looking at the files in the sandbox directly outside the container, this user addition is done at container startup and is not included in the image.
Also, since the process space is also shared, the result of the ps command are not different between that of inside and outside the container, and the kill command can send signals to each other.
Next, try running the command directly inside the container without launching a shell. Here, the exec command of Singularity is used to specify the image and start it. The image can be a SIF file, a sandbox image, or an image repository. Let’s also check if the image you just converted from sandbox has the file you created.
$ singularity exec ubuntu2004.sif cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
<<..snip..>>
$ singularity exec from_sandbox.sif ls -l /foobar
-rwxr-xr-x 1 root root 0 Mar 31 12:45 /foobar
$ singularity exec ubuntu2020 cat /etc/os-release
$ singularity exec docker://centos:latest cat /etc/passwd
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
<<..snip..>>
The command was able to be executed in the environment inside the container. Also, inside of the container from converted SIF image, you can see the file created by the user is owned by root.
If a repository is specified, the image is converted to a SIF image as shown in above and then executed. When you do the same again, it compares the images in the cache and that in repository. If there are any updates, only the parts with update are downloaded and converted to a SIF again. If there are no updates, the created SIF is used as is.This makes the startup slower than using the SIF directly, but it is useful if you want to always use the latest version or if you do not want to manage SIF images by file name. Basically, using a SIF image is recommended if you want to fix the contents of the image or minimize the boot overhead, etc. Running from sandbox or Docker image directly should be used with attention to overhead and pros or cons.
The home directory is also shared inside the container, so let’s use the files there from inside the container. Display the beginning of the SIF image file with the command in the container.
$ singularity exec ubuntu2004.sif head -n 1 ubuntu2004.sif
#!/usr/bin/env run-singularity
It was able to access the files in your home directory from within the container. If you have other files, try them too.
Another thing to know here is that the SIF image is a script. The second and subsequent lines are binary, but this part contains the image body in a compressed file system called SquashFS. Therefore, the size of the SIF file is much smaller than the total amount of files inside. As a result, SIF images are easier to cache and make a lower load on shared storage, especially metadata access, than deploying small files in your home directory. Therefore, applications launch faster.
Being a script, the SIF image can be executed directly;
$ ./ubuntu2004.sif
Singularity>
A prompt will appear, and you can operate it as it is. In other words, launching a simply created SIF image directly is the same as using the shell command. Also, if you give the command as an option, it is the same as executing the exec command.
$ ./ubuntu2004.sif cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
<<..snip..>>
By using Singularity, you can use various environment as the application execution environment, independent from the environment prepared in the system.