Changeset 4796 for BOL

Timestamp:

Feb 2, 2024, 1:06:35 PM (11 months ago)

Author:

asima

Message:

The usual combination 5MPI * 8OMP per node crashes now with messages :

srun: warning: can't honor --ntasks-per-node set to 5 ... Ignoring --ntasks-per-node.
run: error: Unable to create step for job ...: More processors requested than permitted

(strangely enough, it still runs for 4MPI * 10OMP per node)

Solution adopted (and some comments added) : comment out "--ntasks-per-node=..."

For the record, other solutions that keep "--ntasks-per-node=..." :
--> add "--nodes=..." (=ntasks/ntasks-per-node)
--> or, replace "nthreads=8 ; export OMP_NUM_THREADS=$nthreads"

by "export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK"

(Not easy to have a job for HPC that could work, one day, on a PC as well...)

Location:

BOL/LMDZ_Setup

Files:

• script_SIMU (modified) (2 diffs)
• setup.sh (modified) (1 diff)

BOL/LMDZ_Setup/script_SIMU

@@ -5 +5 @@
 # Nombre de processus MPI :
 #SBATCH --ntasks=8
-# number of MPI processes per node :
-#SBATCH --ntasks-per-node=5
+##### number of MPI processes per node : 40(procs/node on Jean-Zay) / cpus-per-task (ex : =5 for 8 OMP)
+####SBATCH --ntasks-per-node=5    # if specified, also add "#SBATCH --nodes= ..."  with nodes=ntasks/(ntasks-per-node)
 # nombre de threads OpenMP
-#SBATCH --cpus-per-task=5
+#SBATCH --cpus-per-task=8
 # de Slurm "multithread" fait bien reference a l'hyperthreading.
 #SBATCH --hint=nomultithread       # 1 thread par coeur physique (pas d'hyperthreading)
@@ -20 +20 @@
 set -ex
 
-
+# Number of MPI processes :
 ntasks=8
-nthreads=4
-# number of OpenMP threads:
-export OMP_NUM_THREADS=$nthreads
+# number of OpenMP threads  
+nthreads=8 
+export OMP_NUM_THREADS=$nthreads # for Jean-Zay it would be recommendend to use : 
+#       export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
 # private memory for each thread
 export OMP_STACKSIZE=800M

BOL/LMDZ_Setup/setup.sh

@@ -456 +456 @@
 # Choix du nombre de processeurs
 # NOTES : 
-# omp=8 by default, but we need 
+# omp=8 by default (for Jean-Zay must be a divisor of 40 procs/node), but we need 
 #   omp=1 for SPLA (only MPI parallelisation)
 #   omp=2 for veget=CMIP6 beacause of a bug in ORCHIDEE/src_xml/xios_orchidee.f90