Calculation of the storage space utilization

About

The information contained in this document provides a thorough understanding of the method we use to calculate the storage space utilization (usage) on per-user and per-project basis on Discoverer. Here we extend the information already provided in Home folder (/home/username), Personal scratch and storage folder (/discofs/username), and Per-project scratch and storage folder.

Storage units

Base unit

The amount of storage reported by the Linux system diagnostic tools is displayed as a number multiple of 1 KiB blocks, where 1 KiB = 1024 bytes. For instance, if a tool reports the space occupied by the files of an user is 12345 that means the reported amount of space is 12345 KiB.

Disk space amount abbreviations

On the Discoverer accounting database and in the documentation here, we count the disk usage correctly in bytes. That means we use the following disk space amount abbreviations:

1 KiB = 1024 bytes

1 MiB = 1024 KiB = 1024 * 1024 = 1048576 bytes

1 GiB = 1024 MiB = 1024 * 1024 * 1024 = 1073741824 bytes

1 TiB = 1024 GiB = 1024 * 1024 * 1024 * 1024 = 1099511627776 bytes

That means if you request two terrabytes of disk space for your project on Discoverer, you will receive 1024 * 1024 * 1024 * 1024 * 2 = 2199023255552 bytes of disk space.

Warning

We do not use measures like KB, MB, GB, and TB to show how much storage space your project utilizes because they count the storage space as a multiple of 1000 bytes.

Quotas

There are two types of quotas that limit the amount of storage resources the users are allowed to utilize:

  • block quota: defines the maximum amount of storage (in 1 KiB blocks) that can be utilized by the user’s files or folders
  • inode quota: defines the maximum number of files and folders each user can create

The following file systems available to Discoverer HPC users are quota-enabled:

/home
/valhalla

/home

Warning

At this point we may not provide information about the quota on the /home filesystem because of problems with the NFS quota remote checking.

The quota on the /home filesystem is handled in the classic way the Linux OS manages quotas set on locally mounted Ext4 type file systems. Both per-user and per-group quota are enabled on /home, but currently only user quota is in use. The document Home folder (/home/username) describes the way to check the storage amount used on /home. Here we discuss in details the per-user limit enforcing on /home.

Important

The accounting precisions followed from here onwards implies that only the folder /home/username can contain files and folders owned by the user “username” on /home.

To display the current amount of storage used by the current user, as well as the quota set, execute in terminal:

quota

The result will be similar to this one:

Disk quotas for user username (uid 2001):
Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
/home            4 2097152 3145728               2  100000  150000

It consists of nine (9) columns. The first column lists the mounting point of the file system with quota enabled on it. Next are columns 2-5, which show the quota properties related to the storage size control. The columns 6-9 provide information about the restriction set on the number of files (inode control).

Columns 2-5:

blocks: the number of 1 KiB blocks already allocated to store the user’s files and folders on /home quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output limit: (hard quota) once blocks == limit it is impossible to create new files or folders, nor can any content be added to the existing files grace: when usage is over the soft limit (quota) but under the hard limit (limit), this column shows the remaining grace time (e.g. 6d). When usage is below the soft limit, the column shows - because no grace is in effect. See the subsection “Grace period on /valhalla” below for details and how to check the configured default.

Columns 6-9:

Important

Linux counts the folders as files.

files: the number of files owned by the user that are already stored on /home quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output limit: (hard quota) once blocks == limit no new files or folders can be created grace: when usage is over the soft limit (quota) but under the hard limit (limit), this column shows the remaining grace time (e.g. 6d). When usage is below the soft limit, the column shows - because no grace is in effect. See the subsection “Grace period on /valhalla” below for details and how to check the configured default.

Note that the column files holds the sum of the number of files and folders currently owned by the user.

/valhalla

Note that /valhalla hosts the project folders. The backend is a LustreFS file system for project-specific quotas on LustreFS, so you can check the project quota using the numerical identification number (ID) that corresponds to the project.

Note

There is not per-user or per-group quota set on /valhalla. The only quota supported on /valhalla is the project one.

Warning

To be able to execute the command lines suggested below in Discoverer+ GPU cluster, you must start interactive Bash session in the cluster:

srun -p common -N 1 -n 1 --account=<your_slurm_project_name> --qos=2cpu-single-host --mem-per-cpu=512 --pty bash

There is no need to do that on Discoverer CPU cluster. There you can execute the commands below directly on the login node.

To get the numerical identification number (ID) of the project, execute in terminal:

lfs project -d /valhalla/projects/project_id

For example the execition of this check:

lfs project -d /valhalla/projects/ehpc-reg-2025d0-000

will produce the following output:

816 P /valhalla/projects/ehpc-reg-2025d0-000

Therefore, the numerical identification number (ID) of the project is 816.

Now you just need to pass the numerical identification number (ID) of the project to the lfs quota command to display the current amount of storage used by the project:

lfs quota -p 816 /valhalla

will produce the following output:

Disk quotas for prj 816 (pid 816):
     Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
      /valhalla 67040956  53687091200 54009213747       -  545997  49000000 50000000       -

It consists of nine (9) columns. The first column lists the mounting point of the file system with quota enabled on it. Next are columns 2-5, which show the quota properties related to the storage size control. The columns 6-9 provide information about the restriction set on the number of files (inode control).

Columns 2-5:

  • blocks: the number of 1 KiB blocks already allocated to store the user’s files and folders on /valhalla
  • quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output
  • limit: (hard quota) once blocks == limit it is impossible to create new files or folders, nor can any content be added to the existing files
  • grace: when usage is over the soft limit (quota) but under the hard limit (limit), this column shows the remaining grace time (e.g. 6d). When usage is below the soft limit, the column shows - because no grace is in effect. See the information about the grace period below and how to check the configured default.

Columns 6-9:

  • files: the number of files owned by the user that are already stored on /valhalla
  • quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output
  • limit: (hard quota) once blocks == limit no new files or folders can be created
  • grace: when usage is over the soft limit (quota) but under the hard limit (limit), this column shows the remaining grace time (e.g. 6d). When usage is below the soft limit, the column shows - because no grace is in effect. See the information about the grace period below and how to check the configured default.

Important

Linux counts the folders as files

Note that the column files holds the sum of the number of files and folders currently owned by the user.

The grace column in lfs quota -p <id> /valhalla does not show the configured default grace period. It shows only the remaining grace time when the project is already over its soft limit (quota). When usage is below the soft limit, no grace period is in effect, so the column shows -. Once usage exceeds the soft limit but stays below the hard limit (limit), the grace period starts and the column shows the countdown (e.g. 6d, 2d). When usage reaches the hard limit, writes are blocked immediately regardless of grace.

To see the configured default grace period for project quotas on /valhalla, run:

lfs quota -t -p /valhalla/

Example output:

Block grace time: 1w; Inode grace time: 1w

The default is typically one week (1w) for both block and inode unless it has been changed by the administrator. This default applies to all projects on /valhalla; the same grace period is used whenever any project exceeds its soft limit.