Calculation of the storage space utilization¶
About¶
The information contained in this document provides a thorough understanding of the method we use to calculate the storage space utilization (usage) on per-user and per-project basis on Discoverer. Here we extend the information already provided in Home folder (/home/username), Personal scratch and storage folder (/discofs/username), and Per-project scratch and storage folder (/discofs/projectname).
Storage units¶
Base unit¶
The amount of storage reported by the Linux system diagnostic tools is displayed as a number multiple of 1 KiB blocks, where 1 KiB = 1024 bytes. For instance, if a tool reports the space occupied by the files of an user is 12345 that means the reported amount of space is 12345 KiB.
Disk space amount abbreviations¶
On the Discoverer accounting database and in the documentation here, we count the disk usage correctly in bytes. That means we use the following disk space amount abbreviations:
1 KiB = 1024 bytes
1 MiB = 1024 KiB = 1024 * 1024 = 1048576 bytes
1 GiB = 1024 MiB = 1024 * 1024 * 1024 = 1073741824 bytes
1 TiB = 1024 GiB = 1024 * 1024 * 1024 * 1024 = 1099511627776 bytes
That means if you request two terrabytes of disk space for your project on Discoverer, you will receive 1024 * 1024 * 1024 * 1024 * 2 = 2199023255552 bytes of disk space.
Warning
We do not use measures like KB, MB, GB, and TB to show how much storage space your project utilizes because they count the storage space as a multiple of 1000 bytes.
Quotas¶
There are two types of quotas that limit the amount of storage resources the users are allowed to utilize:
- block quota: defines the maximum amount of storage (in 1 KiB blocks) that can be utilized by the user’s files or folders
- inode quota: defines the maximum number of files and folders each user can create
The following file systems available to Discoverer HPC users are quota-enabled:
/home
/discofs
/home¶
The quota on the /home filesystem is handled in the classic way the Linux OS manages quotas set on locally mounted Ext4 type file systems. Both per-user and per-group quota are enabled on /home, but currently only user quota is in use. The document Home folder (/home/username) describes the way to check the storage amount used on /home. Here we discuss in details the per-user limit enforcing on /home.
Warning
The accounting precisions followed from here onwards implies that only the folder /home/username can contain files and folders owned by the user “username” on /home.
To display the current amount of storage used by the current user, as well as the quota set, execute in terminal:
quota
The result will be similar to this one:
Disk quotas for user username (uid 2001):
Filesystem blocks quota limit grace files quota limit grace
/home 4 2097152 3145728 2 100000 150000
It consists of nine (9) columns. The first column lists the mounting point of the file system with quota enabled on it. Next are columns 2-5, which show the quota properties related to the storage size control. The columns 6-9 provide information about the restriction set on the number of files (inode control).
Columns 2-5:
blocks: the number of 1 KiB blocks already allocated to store the user’s files and folders on /home
quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output
limit: (hard quota) once blocks == limit it is impossible to create new files or folders, nor can any content be added to the existing files
grace: the period of time during which users can continue exceeding their soft limit (soft quota), while still maintaining compliance with the specified limit (hard quota), is 7 days (a number under “grace” appears only when blocks >= quota)
Columns 6-9:
Important
Linux counts the folders as files.
files: the number of files owned by the user that are already stored on /home
quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output
limit: (hard quota) once blocks == limit no new files or folders can be created
grace: the period of time during which users can continue exceeding their soft limit (soft quota), while still maintaining compliance with the specified limit (hard quota), is 7 days (a number under “grace” appears only when blocks >= quota)
Note that the column files holds the sum of the number of files and folders currently owned by the user.
/discofs¶
Warning
Keep in mind that both the user scratch folder and the project folder are on /discofs file system.
The quota on the /discofs filesystem is handled by the Lustre file system driver. Both per-user and per-group quota are enabled on /discofs, but currently only the group quota is in use. The documents Personal scratch and storage folder (/discofs/username) and Personal scratch and storage folder (/discofs/username) show how to check the quota on /discofs.
The major question we want to address here is about the use of group-based quota to control the storage resource provided by /discofs file system. In other words, why user-based quota works on /home, but it is not effective on /discofs.
The /discofs hosts both the user scratch folder and the project folder. It is imperative that we have the capability to set quota to both projects and scratch folders autonomously on the same files system, whereupon by default the users may create files owned by their username in both destinations. If we count the storage usage based on who owns the files, it won’t be possible to tell which quota the files created by the user should contribute to. This is the reason we use the POSIX group ownership as an identifier to count the storage utilization on /discofs. That way, the group ownership of each file created within the project’s folder is automatically assigned to the project’s POSIX group, while the main POSIX group of the user is always assigned to the files created in the scratch folder.
To display the current amount of storage used by username or projectname POSIX groups, as well as the quota set, execute in terminal:
lfs quota -g username /discofs
and
lfs quota -g projectname /discofs
The result looks like:
Disk quotas for grp projectname (gid 1129):
Filesystem kbytes quota limit grace files quota limit grace
/discofs 923055456 107374182400 108447924224 - 54997 110000000 120000000 -
It consists of nine (9) columns. The first column lists the mounting point of the file system with quota enabled on it. Next are columns 2-5, which show the quota properties related to the storage size control. The columns 6-9 provide information about the restriction set on the number of files (inode control).
Columns 2-5:
blocks: the number of 1 KiB blocks already allocated to store the user’s files and folders on /discofs
quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output
limit: (hard quota) once blocks == limit it is impossible to create new files or folders, nor can any content be added to the existing files
grace: the period of time during which users can continue exceeding their soft limit (soft quota), while still maintaining compliance with the specified limit (hard quota), is 7 days (a number under “grace” appears only when blocks >= quota)
Columns 6-9:
Important
Linux counts the folders as files.
files: the number of files owned by the user that are already stored on /discofs
quota: (soft quota) once blocks >= quota a warning message will be displayed in the standard output
limit: (hard quota) once blocks == limit no new files or folders can be created
grace: the period of time during which users can continue exceeding their soft limit (soft quota), while still maintaining compliance with the specified limit (hard quota), is 7 days (a number under “grace” appears only when blocks >= quota)
Note that the column files holds the sum of the number of files and folders currently owned by the user.
Getting help¶
See Getting help