StarCCM

Prerequisites

For versions older than 8.02.008-R8, these instructions does not work.

Procuring the software

Installing

Follow the installation instructions that comes with StarCCM itself.

We recommend installing in a directory under your project storage.

If you wish to use the graphical frontends for adjusting StarCCM projects we recommend that you install and use the software when connected via the ThinLinc graphical interface to our clusters. You may need to load a Java module (Java/11 or Java/1.8).

There are several environment variables that may be needed when installing the software:

• CHECK_DISK_SPACE=OFF

• IATEMPDIR, setting it to an existing directory with 10GB+ free space

There are some command line options to the installer that avoids installing things we don't want and avoids the GUI installer:

• -i console

• -DINSTALLFLEX=false

• -DADDSYSTEMPATH=false

• -DINSTALLDIR=/proj/nobackup/yourproj/you/starccm

All versions

The submitfile should always contain the following:

hostlist --expand --append=':' --append-slurm-tasks=$SLURM_TASKS_PER_NODE $SLURM_JOB_NODELIST  > machinefile

Version 8 and 9

The submitfile should contain the following:

module add GCC/5.4.0-2.26 OpenMPI/1.6.5
export OMPI_MCA_btl_tcp_if_exclude=lo,ib0

Use these arguments to starccm+

-machinefile machinefile -mpi openmpi -np $SLURM_NTASKS -cpubind off

Submitfile example

This is an example of the important parts of a working submitfile.

#!/bin/bash
#SBATCH -A <your-project-id>
#SBATCH -n 28
#SBATCH --ntasks-per-node=14
#SBATCH -t 2:00:00

hostlist --expand --append=':' --append-slurm-tasks=$SLURM_TASKS_PER_NODE $SLURM_JOB_NODELIST  > machinefile

module add GCC/5.4.0-2.26 OpenMPI/1.6.5
export OMPI_MCA_btl_tcp_if_exclude=lo,ib0

STARDIR=<path-to-your-starccm-installation>

$STARDIR/star/bin/starccm+ -machinefile machinefile -mpi openmpi -np $SLURM_NTASKS -cpubind off -nbuserdir <path-to-your-starccm-user-dir> <any-other-starccm-parameters>

If whole nodes are allocated one can use other cpubind arguments, see your starccm documentation for options.

Version 11

Version 11 (at least 11.06.011-R8) is built with a too old GCC compiler to be able to use one of the site installed OpenMPI versions, therefore one must use the Intel MPI packaged with Star-CCM+ itself. There is also a problem with the default fabric choosen by that MPI version and the TCP fabric must be explicitly selected.

The important flags are therefore:

-machinefile machinefile -mpi intel -mpiflags -slurm -np $SLURM_NTASKS -fabric TCP

Submit file example

#!/bin/bash
#SBATCH -A <your-project-id>
#SBATCH -J star-ccm-v11
#SBATCH -n 56
#SBATCH -t 1:00:00

hostlist --expand --append=':' --append-slurm-tasks=$SLURM_TASKS_PER_NODE $SLURM_JOB_NODELIST  > machinefile

STARDIR=<path-to-your-starccm-installation>

$STARDIR/star/bin/starccm+ -machinefile machinefile -mpi intel -mpiflags -slurm -np $SLURM_NTASKS -fabric TCP -nbuserdir <path-to-your-starccm-user-dir> <any-other-starccm-parameters>