Changeset 3495
- Timestamp:
- Nov 7, 2024, 10:27:18 AM (6 weeks ago)
- Location:
- trunk/LMDZ.COMMON/libf/evolution
- Files:
-
- 6 edited
-
changelog.txt (modified) (1 diff)
-
deftank/PCMrun.job (modified) (2 diffs)
-
deftank/PEMrun.job (modified) (2 diffs)
-
deftank/README (modified) (2 diffs)
-
deftank/launchPEM.sh (modified) (5 diffs)
-
deftank/lib_launchPEM.sh (modified) (13 diffs)
Legend:
- Unmodified
- Added
- Removed
-
trunk/LMDZ.COMMON/libf/evolution/changelog.txt
r3493 r3495 466 466 - Deletion of the wrapper subroutine; 467 467 - Making the initialization, variables management and arguments of the main subroutine for the dynamic computation of ice table to be more suitable. 468 469 == 07/11/2024 == JBC 470 Modifications related to the launching script: 471 - There is actually no launching difference between the 1D and 3D models. It is more about how and where you want to execute. So now, the user can choose between two launching modes with the parameter "mode" (0 = "processing scripts"; any other values = "submitting jobs"). The former option is usually used to process the script on a local machine while the latter is used to submit jobs on supercomputer; 472 - The execution command line in the job scripts that should be modified by the user according to the set-up is now given as an argument at the beginning to be more identifiable and adaptable; 473 - Making the job scripts more robust to detect a successful end. -
trunk/LMDZ.COMMON/libf/evolution/deftank/PCMrun.job
r3416 r3495 23 23 # Name of executable for the PCM: 24 24 exePCM="gcm_64x48x32_phymars_para.e" 25 26 # Execution command: 27 exe_cmd="srun --cpu-bind=threads --label -c${OMP_NUM_THREADS:=1}" 25 28 ######################################################################## 26 29 … … 32 35 read i_myear n_myear convert_years iPCM iPEM nPCM nPCM_ini < info_PEM.txt 33 36 cp run_PCM.def run.def 34 srun --cpu-bind=threads --label -c${OMP_NUM_THREADS:=1} $exePCM > out_runPCM${iPCM} 2>&1 35 if [ ! -f "restartfi.nc" ] || ! (tail -n 1 out_runPCM${iPCM} | grep -iq "everything is cool!"); then # Check if it ended abnormally37 eval "$exe_cmd $exePCM > out_runPCM${iPCM} 2>&1" 38 if [ ! -f "restartfi.nc" ] || ! (tail -n 100 out_runPCM${iPCM} | grep -iq "everything is cool!"); then # Check if it ended abnormally 36 39 echo "Error: the run PCM $iPCM crashed!" 37 40 echo "Be careful: there may be dependent jobs remaining in the SLURM queue with status 'DependencyNeverSatisfied'! You can cancel them by executing the script \"kill_launchPEM.sh\"." -
trunk/LMDZ.COMMON/libf/evolution/deftank/PEMrun.job
r3417 r3495 22 22 # Name of executable for reshaping PCM data with XIOS: 23 23 exeReshape="reshape_XIOS_output_64x48x32_phymars_seq.e" 24 25 # Argument for the PEM execution (for SLURM: "$SLURM_JOB_ID" | for PBD/TORQUE: "$PBS_JOBID" | "" when the script is not run as a job): 26 arg_pem="$SLURM_JOB_ID" 24 27 ######################################################################## 25 28 … … 39 42 echo "Run PEM $iPEM is starting." 40 43 cp run_PEM.def run.def 41 ./$exePEM $SLURM_JOB_ID > out_runPEM${iPEM} 2>&1 42 if [ ! -f "restartfi.nc" ] || ! (tail -n 1 out_runPEM${iPEM} | grep -iq "so far, so good!"); then # Check if it ended abnormally44 eval "./$exePEM $arg_pem > out_runPEM${iPEM} 2>&1" 45 if [ ! -f "restartfi.nc" ] || ! (tail -n 100 out_runPEM${iPEM} | grep -iq "so far, so good!"); then # Check if it ended abnormally 43 46 echo "Error: the run PEM $iPEM crashed!" 44 47 exit 1 -
trunk/LMDZ.COMMON/libf/evolution/deftank/README
r3460 r3495 7 7 (ii) nPCM_ini -> the number of initial PCM runs (at least 2); 8 8 (iii) nPCM -> the number of PCM runs between each PEM run (usually 2); 9 (iv) dim -> the dimension of the model (1 for 1D, any other values stand for 3D).9 (iv) mode -> the launching mode (0 = "processing scripts"; any other values = "submitting jobs"). The former option is usually used to process the script on a local machine while the latter is used to submit jobs on supercomputer. 10 10 The script can take an argument: 11 11 - If there is no argument, then the script initiates a PEM simulation from scratch. 12 12 - If the argument is 're', then the script relaunches an existing PEM simulation. It will ask for parameters to know the starting point that you want to. 13 T he script works only with the job scheduler SLURM to submit chained jobs.13 To submit chained jobs, the script works with the job schedulers SLURM and PBS/TORQUE. 14 14 15 15 # liblaunchPEM.sh: … … 17 17 18 18 # PCMrun.job: 19 Bash script file to submit a PCM job (with SLURM or PBS/TORQUE). The name of the PCM executable file should be adapted. The headers correspond to the ADASTRA supercomputer and should be changed for other machines and job schedulers. In case of 1D, the headers are naturally omitted.19 Bash script file to submit a PCM job (with SLURM or PBS/TORQUE). The headers correspond to the ADASTRA supercomputer and should be changed for other machines and job schedulers. In case of "processiong scripts" launching mode, the headers are naturally omitted. 20 20 The path to source the arch file should be adapted to the machine. 21 The execution line should also be adapted according to the set-up. 21 The name of the PCM executable file should be adapted. 22 The execution command should also be adapted according to the set-up. 22 23 23 24 # PEMrun.job: 24 Bash script file to submit PEM job (with SLURM or PBS/TORQUE). The name of the PEM executable file and Reshaping executable file should be adapted. The headers correspond to the ADASTRA supercomputer and should be changed for other machines and job schedulers. In case of 1D, the headers are naturally omitted.25 Bash script file to submit PEM job (with SLURM or PBS/TORQUE).The headers correspond to the ADASTRA supercomputer and should be changed for other machines and job schedulers. In case of "processiong scripts" launching mode, the headers are naturally omitted. 25 26 The path to source the arch file should be adapted to the machine. 26 The execution line should also be adapted according to the set-up.27 The PEM executable can have an optional argument to specify the SLURM job ID in order to detect the job time limit and deal withit.27 The name of the PEM executable file and Reshaping executable file should be adapted. 28 The PEM executable can have an optional argument which should be specified according to the set-up. This the job ID to make the PEM detect the job time limit. 28 29 29 30 # run_PEM.def -
trunk/LMDZ.COMMON/libf/evolution/deftank/launchPEM.sh
r3419 r3495 17 17 #n_earth_years=300 18 18 19 # Set the number of initial PCM runs :19 # Set the number of initial PCM runs (>= 2): 20 20 nPCM_ini=3 21 21 22 # Set the number of PCM runs between each PEM run :22 # Set the number of PCM runs between each PEM run (>= 2): 23 23 nPCM=2 24 24 25 # Set the dimension of the model (1 = "1D"; other values = "3D"):26 dim=3 25 # Set the launching mode (0 = "processing scripts"; any other values = "submitting jobs"). The former option is usually used to process the script on a local machine while the latter is used to submit jobs on supercomputer: 26 mode=1 27 27 ######################################################################## 28 28 … … 48 48 checklaunch 49 49 initlaunch 50 cyclelaunch $ dim$nPCM_ini50 cyclelaunch $mode $nPCM_ini 51 51 52 52 else … … 57 57 echo "This is a new cycle for the PEM simulation." 58 58 date 59 if [ $ dim -ne 1]; then59 if [ $mode -ne 0 ]; then 60 60 job_scheduler 61 61 fi 62 62 read i_myear n_myear convert_years iPCM iPEM nPCM nPCM_ini < info_PEM.txt 63 cyclelaunch $ dim$nPCM63 cyclelaunch $mode $nPCM 64 64 65 65 # Starting a relaunch … … 122 122 fi 123 123 if [ $relaunch = "PCM" ]; then 124 relaunchPCM $ dim124 relaunchPCM $mode 125 125 else 126 relaunchPEM $ dim126 relaunchPEM $mode 127 127 fi 128 128 … … 133 133 echo "This is a continuation of the previous PEM run." 134 134 date 135 submitPEM $ dim135 submitPEM $mode 136 136 137 137 # Default case: error -
trunk/LMDZ.COMMON/libf/evolution/deftank/lib_launchPEM.sh
r3446 r3495 32 32 submit_job="sbatch --parsable" 33 33 submit_dependjob="sbatch --parsable --dependency" 34 sed -i 's/\$PBS_JOBID/\$SLURM_JOB_ID/g' PEMrun.job35 34 elif command -v qstat &> /dev/null; then 36 35 echo "PBS/TORQUE is installed on $machine." … … 39 38 submit_job="qsub" 40 39 submit_dependjob="qsub -W depend" 41 sed -i 's/\$SLURM_JOB_ID/\$PBS_JOBID/g' PEMrun.job42 40 else 43 41 echo "Error: neither SLURM nor TORQUE/PBS is installed on $machine!" … … 119 117 mkdir diags 120 118 fi 121 if [ $ dim -ne 1]; then119 if [ $mode -ne 0 ]; then 122 120 job_scheduler 123 121 fi … … 161 159 162 160 # To submit the PCM runs 163 # arg1: model dimension161 # arg1: launching mode 164 162 # arg2: number of PCM runs to launch 165 163 # arg3: local number of the PCM run from which to start (optional) … … 172 170 if [ $i_myear -lt $n_myear ]; then 173 171 echo "Run PCM $iPCM: call $ii/$2..." 174 if [ $1 -eq 1 ]; then # 1D model172 if [ $1 -eq 0 ]; then # Mode: processing scripts 175 173 sed -i "s/^k=[0-9]\+$/k=$(echo "3 - $nPCM_ini" | bc)/" PCMrun.job 176 174 ./PCMrun.job … … 178 176 errlaunch 179 177 fi 180 else # 3D model178 else # Mode: launching jobs 181 179 cp PCMrun.job PCMrun${iPCM}.job 182 180 sed -i -E "s/($name_job[^0-9]*[0-9]*[^0-9]*)[0-9]+$/\1${iPCM}/" PCMrun${iPCM}.job … … 197 195 if [ $i_myear -lt $n_myear ]; then 198 196 echo "Run PCM $iPCM: call $i/$2..." 199 if [ $1 -eq 1 ]; then # 1D model197 if [ $1 -eq 0 ]; then # Mode: processing scripts 200 198 sed -i "s/^k=[0-9]\+$/k=$(echo "$i + 2 - $nPCM_ini" | bc)/" PCMrun.job 201 199 ./PCMrun.job … … 203 201 errlaunch 204 202 fi 205 else # 3D model203 else # Mode: launching jobs 206 204 cp PCMrun.job PCMrun${iPCM}.job 207 205 sed -i -E "s/($name_job[^0-9]*[0-9]*[^0-9]*)[0-9]+$/\1${iPCM}/" PCMrun${iPCM}.job … … 219 217 220 218 # To submit the PEM run 221 # arg1: model dimension219 # arg1: launching mode 222 220 submitPEM() { 223 221 if [ $i_myear -lt $n_myear ]; then 224 222 echo "Run PEM $iPEM" 225 if [ $1 -eq 1 ]; then # 1D model223 if [ $1 -eq 0 ]; then # Mode: processing scripts 226 224 ./PEMrun.job 227 225 if [ $? -ne 0 ]; then 228 226 errlaunch 229 227 fi 230 else # 3D model228 else # Mode: launching jobs 231 229 sed -i -E "s/($name_job[^0-9]*[0-9]*[^0-9]*)[0-9]+$/\1${iPEM}/" PEMrun.job 232 230 jobID=$(eval "$submit_job PEMrun.job") … … 242 240 243 241 # To make one cycle of PCM and PEM runs 244 # arg1: model dimension242 # arg1: launching mode 245 243 # arg2: number of PCM runs to launch 246 244 # arg3: local number of the PCM run from which to start (optional) … … 252 250 if [ $i_myear -lt $n_myear ]; then 253 251 echo "Run PEM $iPEM" 254 if [ $1 -eq 1 ]; then # 1D model252 if [ $1 -eq 0 ]; then # Mode: processing scripts 255 253 ./PEMrun.job 256 254 if [ $? -ne 0 ]; then 257 255 errlaunch 258 256 fi 259 else # 3D model257 else # Mode: launching jobs 260 258 sed -i -E "s/($name_job[^0-9]*[0-9]*[^0-9]*)[0-9]+$/\1${iPEM}/" PEMrun.job 261 259 jobID=$(eval "$submit_dependjob=afterok:${jobID} PEMrun.job") … … 293 291 294 292 # To relaunch from PCM run 295 # arg1: model dimension293 # arg1: launching mode 296 294 relaunchPCM() { 297 295 iPCM=$(($irelaunch + 1)) … … 361 359 362 360 # To relaunch from PEM run 363 # arg1: model dimension361 # arg1: launching mode 364 362 relaunchPEM() { 365 363 iPEM=$(echo "$irelaunch + 1" | bc)
Note: See TracChangeset
for help on using the changeset viewer.
