source: dynamico_lmdz/aquaplanet/CMIP5a/RUN_AQUAPLANET_DYNAMICO_LMDZ5/README @ 4042

Overview
========

This directory contains :
* .def files for DYNAMICO and LMDZ5
* .xml files for XIOS
* scripts

You can safely modify all *.def and *.xml or replace them with your own files, EXCEPT :
* iodef.xml
* icosa.xml
* context_lmdz.xml
* start.def

The above files CAN NOT be used 'as is' because they contain keywords that must be replaced by acceptable values. This substitution is made by the scripts described below (see "Procedure"). The keywords are :
    CE0L         : true if the simulation is a 'CE0L' run that creates the limit.nc file
    FIRST        : true if the simulation is the first run in a series (after a 'CE0L' run)
    NEXT         : true if the simulation is a restart of a run of type 'FIRST' or 'NEXT'
    DOM_OUT      : dom_glo if CE0L, dom_regular otherwise
    USING_SERVER : false if CE0L, true otherwise
    ETAT0        : start_file if NEXT, held_suarez otherwise
    IFLAG_PHYS   : 103 if CE0L, 1 otherwise
    READ_START   : no if CE0L, yes otherwise

For 'CE0L' XIOS is run in 'attached' mode. In server mode, xios_server.exe would hang after the dynamico_lmdz.exe MPI processes have aborted. 
For 'START' and 'NEXT', XIOS is run in server mode for flexibility. Especially it allows to restart with a different number of processors, which is not possible in attached mode.

Notice that the definitions of start.def have the highest priority since start.def is the last *.def file included in run.def .

Procedure using prepare.sh
==========================

0 - First of create a small text file 'config.sh' defining a few environment variables
  ARCH : This will select appropriate machine-specific parts of the scripts, especially when generating the shell script to be submitted to the batch scheduler
  ESM_ROOT : the directory containing the ICOSA_LMDZ directory
> export ARCH=X64_CURIE
> export ESM_ROOT=$PWD/../..

1 - Edit the .def and .xml files if you want settings that differ from the basis that is proposed here. DO NOT TOUCH the files mentioned above.

2 - In order to create a run of type 'ce0l' :
> ./prepare ce0l MYDIR0 STORE0
This will create STORE0 and MYDIR0 if necessary and populate MYDIR0 with .xml and .def files, as well as a batch job 'job.sh' to be submitted (on CURIE) as :
ccc_msub job.sh .
After successful completion, STORE0 contains all *.nc files produced by the run, especially limit.nc

3 - In order to create a one-year run of type 'first'
> ./prepare first STORE0 MYDIR1 STORE1
This will create STORE1 and MYDIR1 if necessary and populate MYDIR1 with .xml and .def files, as well as a batch job 'job.sh'. This batch job copies input files from STORE0. Submit MYDIR1/job.sh . After successful completion, STORE1 contains all *.nc files produced by the run, especially start.nc

4 - In order to create a one-year run of type NEXT, type
./prepare first STORE0 STOREPREV MYDIR2 STORE2
This will create STORE2 and MYDIR2 if necessary and populate MYDIR2 with .xml and .def files, as well as a batch job 'job.sh'. This batch job copies input files from STORE0 and STOREPREV. Submit MYDIR2/job.sh .

Procedure using example.sh
==========================
The above procedure can be made more automatic using the script example.sh (itself calling prepare.sh).

0 - create a directory 'EXP' for your experiment where you wish, probably not in the source tree. 

1 - Create a subdirectory 'EXP/storedir' or simlink 'storedir' to an empty directory elsewhere. This is where simulations will store their input/output files.

2 - cd to 'EXP', and run 'fullpath/example.sh init' where fullpath is the full path to the directory containing this README and example.sh

3 - optionnally, edit config.sh and/or *.xml or *.def files .

4 - run './example.sh create' . IMPORTANT : DO NOT RUN 'fullpath/example.sh create'. You should now see directories EXP/ce0l, EXP/storedir/ce0l,  EXP/yearXX and EXP/storedir/yearXX . 

5 - on Curie, run './example.sh submit'. Otherwise, edit the scripts */jobs.sh and submit them manually to your batch scheduler.