Zstandard/zstd¶
Fast real-time compression with better ratios than zlib
Overview¶
Zstandard (zstd) is a fast lossless compression algorithm designed for:
- Real-time compression scenarios
- Better compression ratios than zlib
- Fast entropy stage through Huff0 and FSE library
Note
We provide Zstandard installation that is faster and newer than the one provided by the system-wide package manager (e.g. /usr/lib64/libzstd.so brought to the system by the Red Hat Enterprise Linux package manager). Therefore, we recommend using our installation instead of the system-wide one (see below).
Available versions¶
To view available zstd versions:
$ module avail zstd
Build recipes and configuration details are maintained in our GitLab repository:
Compiler support¶
The following compiler-specific builds are available:
Supported builds:
$ module avail zstd/*/*llvm # LLVM build (recommended, will be supported)
$ module avail zstd/*/*gcc # GCC build (recommended for NVIDIA HPC SDK users, retiring soon and becoming not supported on Discoverer or Discoverer+ clusters)
Legacy builds:
$ module avail zstd/*/*intel # Intel oneAPI build (deprecated, will be retired)
$ module avail zstd/*/*aocc # AMD AOCC build (deprecated, will be retired)
Linking your code against zstd installation¶
When compiling a programming code that uses zstd throught its API, load the appropriate module and link with -lzstd:
$ module load zstd/<version>
$ <compiler or linker> <flags> <source_files or object files> -lzstd
Note
We support both dynamic and static linking here.
Replacing the system-wide zstd installation¶
To use the zstd.so library from our installation instead of relying on the system-wide installation:
$ module load zstd/<version>
$ ./your_program # will automatically use libzstd.so from zstd installation
This way your executable will use the zstd library from our installation instead of the system-wide one (e.g. /usr/lib64/libzstd.so brought to the system by the Red Hat Enterprise Linux distribution).