OpenFOAM.com (CPU) ================== .. toctree:: :maxdepth: 1 :caption: Contents: This page documents the OpenFOAM.com (OpenCFD / ESI) builds on Discoverer (year-month tags such as ``v2312`` and ``v2606``). Those packages are distinct from the OpenFOAM Foundation line. For Foundation modules (integer majors such as OpenFOAM 11 and 13), see :doc:`openfoam`. About this lineage ------------------ OpenFOAM.com is the OpenCFD / ESI distribution of OpenFOAM (Open Field Operation And Manipulation), a C++ toolbox for continuum mechanics used most often for computational fluid dynamics. Like the Foundation line, it provides finite-volume discretisation on unstructured meshes, solvers and utilities, turbulence and multiphase models, mesh tools, and parallel execution through MPI. On Discoverer, OpenFOAM.com installs use: * environment modules under ``openfoam.com/`` * system OpenMPI with the shim name ``sys-openmpi`` * a chosen label size (``WM_LABEL_SIZE`` 32 or 64), reflected in the module name as ``int32`` or ``int64`` * ThirdParty components that commonly include FFTW (``makeFFTW``), and may attempt optional extras such as CGAL/boost or ADIOS2 Foundation and OpenFOAM.com share ancestry but are not drop-in replacements. Do not mix binaries, case dictionaries, or modules from one line with the other without checking compatibility. Comparison with Foundation OpenFOAM ----------------------------------- .. list-table:: :header-rows: 1 :widths: 25 35 40 * - Topic - OpenFOAM Foundation - OpenFOAM.com * - Version tags - Integer majors (``11``, ``13``, …) - Year-month tags (``v2312``, ``v2606``, …) * - Site module tree - ``openfoam/`` - ``openfoam.com/`` * - System OpenMPI shim - ``openmpi-system`` - ``sys-openmpi`` * - Typical extra ThirdParty - Scotch, Kahip, Zoltan (as built) - FFTW via ``makeFFTW``; optional ADIOS2, CGAL, … Versions available ------------------ Supported versions .................. .. note:: The versions of `OpenFOAM.com`_ installed in the software repository are built and supported by the Discoverer HPC team. To list OpenFOAM.com modules on the login node: .. code-block:: bash module avail openfoam.com Module names follow the pattern: .. code-block:: bash openfoam.com//--openmpi-int32 openfoam.com//--openmpi-int64 Examples for OpenFOAM.com v2606: .. code-block:: bash module load openfoam.com/26/2606-gcc-openmpi-int32 module load openfoam.com/26/2606-gcc-openmpi-int64 module load openfoam.com/26/2606-llvm-openmpi-int32 module load openfoam.com/26/2606-llvm-openmpi-int64 Older ``v2312`` modules live under ``openfoam.com/23/``. After loading a module, confirm the environment: .. code-block:: bash echo "$WM_PROJECT_VERSION" "$WM_OPTIONS" "$WM_LABEL_SIZE" which foamDictionary Build recipes used by the Discoverer HPC team are published at: https://gitlab.discoverer.bg/vkolev/recipes/-/tree/main/openfoam Optional ADIOS2 ............... ADIOS2 backs the optional OpenFOAM.com ``adiosFoam`` / ``adiosWrite`` path (experimental ADIOS BP I/O and coupling). Ordinary CFD solvers, mesh tools, decomposition, and standard field I/O do not require ADIOS2. Site builds may leave ADIOS2 skipped; that does not prevent normal use of OpenFOAM.com. Enable ADIOS2 only if you need that I/O path (a working CMake on ``PATH`` is required to build it from ThirdParty). User-supported versions ....................... Users may bring or compile their own OpenFOAM.com builds, but those builds are not supported by the Discoverer HPC team. Public recipes (see the link above) may be useful as a starting point. Running OpenFOAM.com -------------------- .. warning:: You MUST NOT execute OpenFOAM calculations directly on the login node (login.discoverer.bg). Run calculations only as Slurm jobs. .. warning:: Write results only inside your :doc:`scratchfolder`. Do not use your :doc:`homefolder` for result data under any circumstances. Example: ``damBreak`` tutorial (v2606) ...................................... The following example runs the OpenFOAM.com ``damBreak`` tutorial as a Slurm batch job. The case ``tutorials/multiphase/interFoam/RAS/damBreak/damBreak`` uses ``interFoam`` and requests 4 subdomains in ``decomposeParDict``. First, load an OpenFOAM.com module and copy the tutorial to scratch: .. code:: bash module load openfoam.com/26/2606-gcc-openmpi-int32 mkdir -p /discofs/$USER/run_openfoam_com cd /discofs/$USER/run_openfoam_com cp -pr $FOAM_TUTORIALS/multiphase/interFoam/RAS/damBreak/damBreak . Create ``/discofs/$USER/run_openfoam_com/damBreak/run_openfoam_mpi.batch`` with content similar to: .. code:: bash #!/bin/bash #SBATCH --partition=cn #SBATCH --job-name=openfoam_dambreak #SBATCH --time=04:00:00 #SBATCH --account=your_slurm_account_name #SBATCH --qos=your_slurm_qos_name #SBATCH --nodes 1 #SBATCH --ntasks-per-node 4 #SBATCH --ntasks-per-core 1 #SBATCH -o slurm.%j.out #SBATCH -e slurm.%j.err module purge module load openfoam.com/26/2606-gcc-openmpi-int32 . $WM_PROJECT_DIR/bin/tools/RunFunctions cd $SLURM_SUBMIT_DIR restore0Dir runApplication blockMesh runApplication setFields runApplication decomposePar runParallel $(getApplication) runApplication reconstructPar .. warning:: For this tutorial, keep ``--ntasks-per-node`` equal to ``numberOfSubdomains`` in ``system/decomposeParDict`` (here: 4). For other cases, always match the Slurm task count to the decomposition defined for that case. Submit the job: .. code:: bash cd /discofs/$USER/run_openfoam_com/damBreak sbatch run_openfoam_mpi.batch Slurm writes standard output to ``slurm.%j.out`` and standard error to ``slurm.%j.err`` in the submit directory (``%j`` is the job ID). Tail ``slurm.%j.out`` to follow progress. Getting help ------------ See :doc:`help`. Related pages ------------- * :doc:`openfoam` — OpenFOAM Foundation builds on Discoverer * OpenFOAM.com: https://www.openfoam.com/ * OpenFOAM Foundation (related but separate): https://openfoam.org/ * ADIOS project: https://www.adios.software/ .. _`OpenFOAM.com`: https://www.openfoam.com/