OpenFOAM Foundation (CPU)

This page documents the OpenFOAM Foundation builds on Discoverer (integer major versions such as OpenFOAM 11 and OpenFOAM 13). Those packages are distinct from the OpenFOAM.com (OpenCFD / ESI) line. For OpenFOAM.com modules (v2312, v2606, and similar), see OpenFOAM.com (CPU).

About this lineage

OpenFOAM (Open Field Operation And Manipulation) is a C++ toolbox for continuum mechanics, used most often for computational fluid dynamics. The Foundation distribution provides finite-volume discretisation on unstructured meshes, solvers and utilities, turbulence and multiphase models, mesh tools, and parallel execution through MPI.

On Discoverer, Foundation installs use:

  • environment modules under openfoam/

  • system OpenMPI with the shim name openmpi-system

  • a chosen label size (WM_LABEL_SIZE 32 or 64), reflected in the module name as int32 or int64

Foundation and OpenFOAM.com share ancestry but are not drop-in replacements. Do not mix binaries, case dictionaries, or modules from one line with the other without checking compatibility.

Versions available

Supported versions

Note

The versions of OpenFOAM installed in the software repository are built and supported by the Discoverer HPC team.

To list Foundation modules on the login node:

module avail openfoam

Module names follow the pattern:

openfoam/<major>/<major>-<compiler>-openmpi-int32
openfoam/<major>/<major>-<compiler>-openmpi-int64

Examples for OpenFOAM 13:

module load openfoam/13/13-gcc-openmpi-int32
module load openfoam/13/13-gcc-openmpi-int64
module load openfoam/13/13-llvm-openmpi-int32
module load openfoam/13/13-llvm-openmpi-int64

After loading a module, confirm the environment:

echo "$WM_PROJECT_VERSION" "$WM_OPTIONS" "$WM_LABEL_SIZE"
which foamDictionary

Build recipes used by the Discoverer HPC team are published at:

https://gitlab.discoverer.bg/vkolev/recipes/-/tree/main/openfoam

Compiler notes

Tests on OpenFOAM 10 and 11 showed that Intel oneAPI compilers (icx/icpx or icc/icpc) did not speed up the produced executables compared with GCC or LLVM for the same purpose. Prefer LLVM Compiler Infrastructure (version 16 or higher) or GCC for Foundation builds on this site. LLVM modules still use Clang for C/C++; a GCC module may be loaded only for Fortran / runtime support.

User-supported versions

Users may bring or compile their own OpenFOAM builds, but those builds are not supported by the Discoverer HPC team. Public recipes (see the link above) may be useful as a starting point.

Running OpenFOAM

Warning

You MUST NOT execute OpenFOAM calculations directly on the login node (login.discoverer.bg). Run calculations only as Slurm jobs.

Warning

Write results only inside your Personal scratch and storage folder. Do not use your Home folder (/home/username) for result data under any circumstances.

Example: wave tutorial (OpenFOAM 13)

The following example runs the Foundation wave tutorial as a Slurm batch job. On OpenFOAM 13 the case lives under tutorials/incompressibleVoF/wave (the older multiphase/interFoam/laminar/wave path does not apply to this release). The case decomposeParDict requests 6 subdomains.

First, load a Foundation module and copy the tutorial to scratch:

module load openfoam/13/13-llvm-openmpi-int32
mkdir -p /discofs/$USER/run_openfoam
cd /discofs/$USER/run_openfoam
cp -pr $FOAM_TUTORIALS/incompressibleVoF/wave .

Create /discofs/$USER/run_openfoam/wave/run_openfoam_mpi.batch with content similar to:

#!/bin/bash

#SBATCH --partition=cn
#SBATCH --job-name=openfoam_wave
#SBATCH --time=04:00:00

#SBATCH --account=your_slurm_account_name
#SBATCH --qos=your_slurm_qos_name

#SBATCH --nodes           1
#SBATCH --ntasks-per-node 6
#SBATCH --ntasks-per-core 1

#SBATCH -o slurm.%j.out
#SBATCH -e slurm.%j.err

module purge
module load openfoam/13/13-llvm-openmpi-int32

. $WM_PROJECT_DIR/bin/tools/RunFunctions

cd $SLURM_SUBMIT_DIR

runApplication blockMesh
runApplication extrudeMesh
runApplication refineMesh
runApplication setWaves
runApplication decomposePar

runParallel foamRun

runApplication reconstructPar

Warning

For this tutorial, keep --ntasks-per-node equal to numberOfSubdomains in system/decomposeParDict (here: 6). For other cases, always match the Slurm task count to the decomposition defined for that case.

Submit the job:

cd /discofs/$USER/run_openfoam/wave
sbatch run_openfoam_mpi.batch

Slurm writes standard output to slurm.%j.out and standard error to slurm.%j.err in the submit directory (%j is the job ID). Tail slurm.%j.out to follow progress.

Getting help

See Getting help.