Zstandard/zstd

Fast real-time compression with better ratios than zlib

Overview

Zstandard (zstd) is a fast lossless compression algorithm designed for:

  • Real-time compression scenarios
  • Better compression ratios than zlib
  • Fast entropy stage through Huff0 and FSE library

Note

We provide Zstandard installation that is faster and newer than the one provided by the system-wide package manager (e.g. /usr/lib64/libzstd.so brought to the system by the Red Hat Enterprise Linux package manager). Therefore, we recommend using our installation instead of the system-wide one (see below).

Available versions

To view available zstd versions:

$ module avail zstd

Build recipes and configuration details are maintained in our GitLab repository:

Compiler support

The following compiler-specific builds are available:

Supported builds:

$ module avail zstd/*/*llvm    # LLVM build (recommended, will be supported)
$ module avail zstd/*/*gcc     # GCC build (recommended for NVIDIA HPC SDK users, retiring soon and becoming not supported on Discoverer or Discoverer+ clusters)

Legacy builds:

$ module avail zstd/*/*intel   # Intel oneAPI build (deprecated, will be retired)
$ module avail zstd/*/*aocc    # AMD AOCC build (deprecated, will be retired)

Linking your code against zstd installation

When compiling a programming code that uses zstd throught its API, load the appropriate module and link with -lzstd:

$ module load zstd/<version>
$ <compiler or linker> <flags> <source_files or object files> -lzstd

Note

We support both dynamic and static linking here.

Replacing the system-wide zstd installation

To use the zstd.so library from our installation instead of relying on the system-wide installation:

$ module load zstd/<version>
$ ./your_program   # will automatically use libzstd.so from zstd installation

This way your executable will use the zstd library from our installation instead of the system-wide one (e.g. /usr/lib64/libzstd.so brought to the system by the Red Hat Enterprise Linux distribution).

Getting Help

For additional assistance: