Changeset 3852

Timestamp:

Jul 16, 2025, 4:10:15 PM (6 weeks ago)

Author:

jmauxion

Message:

Mars PCM:
Improve the omp_chunk computation to fix a MPI request error arising for some combinations of llm/omp_size:

• Make sure that the omp_chunk is not too large so that the last thread do nothing.
• Check and raise a warning if the chunk is so small that the loop distribution require a second cycle over the threads (which is not optimum).

JM

Location:

trunk

Files:

• LMDZ.COMMON/libf/dyn3dpar/parallel_lmdz.F90 (modified) (1 diff)
• LMDZ.MARS/changelog.txt (modified) (1 diff)

trunk/LMDZ.COMMON/libf/dyn3dpar/parallel_lmdz.F90

@@ -191 +191 @@
       omp_chunk=(llm+1)/omp_size
       IF (MOD(llm+1,omp_size)/=0) omp_chunk=omp_chunk+1
+      
+      ! Jonah: for some combinations of llm/omp_size, the chunk
+      ! is such that at least one thread is given no task inside
+      ! "DO SCHEDULE" (e.g when filling MPI buffers to send/recv).
+      ! This leads these "idle" threads to trigger MPI_Waitall
+      ! while no instruction send/recv was emitted, resulting in a request error. 
+      ! Fix:
+      do while (((omp_chunk*(omp_size-1))>=(llm+1)).and.(omp_chunk>=1))
+        omp_chunk=omp_chunk-1
+      end do
+      ! Jonah: connected to the above, one must check that the chunk is
+      ! still big enough to prevent cyclic distribution of tasks
+      ! (i.e. first thread is distributed two packs of iterations due to
+      ! the last thread being unable to complete the loop), which is not optimum 
+      if ((omp_chunk*omp_size)<(llm+1)) then
+         print *, 'Warning: the current OMP chunk is too small and would trigger ', &
+                  'cycling scheduling. You should consider reducing the number of OMP threads.'
+      endif
       CALL getin('omp_chunk',omp_chunk)
 !$OMP END MASTER

trunk/LMDZ.MARS/changelog.txt

@@ -4913 +4913 @@
 == 15/07/2025 == JBC
 Simplification and generalization of the "display_netcdf.py" script which can now handle more cases of shapes/dimensions in the variables + using 'pcolormesh' instead of 'imshow' to better stick to the original data.
+
+== 16/07/2025 == JM
+Improve the omp_chunk computation to fix a MPI request error arising for some combinations of llm/omp_size:
+- Make sure that the omp_chunk is not too large so that the last thread do nothing.
+- Check and raise a warning if the chunk is so small that the loop distribution require a second cycle over the threads (which is not optimum).
+