Libdeflate

About

Libdeflate is a library for fast, whole-buffer DEFLATE-based compression and decompression. It depends on some of the Zlib methods and functions.

Supported versions

To check which Libdeflate versions and build types are currently supported on Discoverer, execute on the login node:

module avail libdeflate

Loading

module avail libdeflate/latest

Gzip and gunzip utilities

We provide highly productive gzip and gunzip utilities that are linked to the optimized libdeflate versions installed in our software repository. For most tasks, they seem to perform faster than the gzip and gunzip utilities that come with the Linux distribution installed on our compute nodes. If it is necessary to rely on those optimized tools, first load the appropriate libdeflate environment modules, as previously mentioned, and then execute the executables under their respective names:

libdeflate-gzip

or

libdeflate-gunzip

Warning

When large archives should be processed, use the Slurm batch job to execute the gzip and gunzip utilities.

Build recipes and tests

For those who are interested in the process of compiling and optimizing the code of Libdeflate we follow, there is public access to our build recipes at:

https://gitlab.discoverer.bg/vkolev/recipes/-/tree/main/libdeflate/

There are also logs that were collected during the compilation and testing processes.

Getting help

See Getting help