source: trunk/LMDZ.COMMON/libf/evolution/deftank/README

   *    .          .   +     .    *        .  +      .    .       .
            +         _______  ________  ____    ____      *           +
  +   .        *     |_   __ \|_   __  ||_   \  /   _|          .       *
           .     .     | |__) | | |_ \_|  |   \/   |  *        *      .
        .              |  ___/  |  _| _   | |\  /| |      .        .
 .  *          *      _| |_    _| |__/ | _| |_\/_| |_                  *
             +       |_____|  |________||_____||_____|   +     .
   .      *          .   *       .   +       *          .        +      .


This folder contains all the files needed to run and work with the Planetary Evolution Model (PEM).

Information:
------------
For further explanations about the PEM, see following website: https://lmdz-forge.lmd.jussieu.fr/mediawiki/Planets/index.php/PEM_(Planetary_Evolution_Model).

Installation:
-------------
The PEM can be downloaded alongside the "LMDZ.COMMON" repository of your trunk. The Fortran code is in the following directory: "trunk/LMDZ.COMMON/libf/evolution/". The PEM main program is "pem.F90".

Compilation:
------------
To compile the PEM, in "LMDZ.COMMON", do: ./makelmdz_fcm -arch [local] -p [planet] -d [dimensions] -j 8 pem
Options with example:
    1) [local]     : root name of arch files, assuming that they have been set up for your configuration;
    2) [planet]    : mars to use the PEM dedicated to the Mars planet;
    3) [dimensions]: 64x48x54 to define the grid you want to use  (longitude x latitude x atmospheric layers).
To run the PEM, you need a PCM working with XIOS and consistent options. To compile it, in "LMDZ.COMMON", do: ./makelmdz_fcm -arch [local] -p [planet] -parallel mpi_omp -io xios -d [dimensions] -j 8 gcm
After compilation, the executable file can be found in the "bin" sub-directory.

Usage:
------
To run a PEM simulation, do: ./pem_workflow.sh [options]
Options:
    1) None: to start a simulation from scratch;
    2) 're': to resume a simulation from a starting point (interactive prompt).

The Bash file ''pem_workflow.sh'' is the master script to launch the PEM chained simulation. It checks if necessary files and required options for your simulation are ok.

Requirements:
-------------
To run the PEM, you can create a folder in which you need the following files:
    > your executable files for the PCM and the PEM with consistent options;
    > the xml files for XIOS which can be found in the PCM deftank folder: "iodef.xml", "context_lmdz_physics.xml", "file_def_physics_mars.xml" and "field_def_physics_mars.xml";
    > the def files you want to run the PCM: "run.def", "callphys.def", "traceur.def", etc.
      /!\ Do not forget to rename the PCM "run.def" into "run_pcm.def";
    > the starting files you want to run the PCM: "startfi.nc" and "start.nc"/"start1D.txt";
    > the necessary PEM files: "pem_workflow.sh", "pem_workflow_lib.sh", "pcm_run.job", "pem_run.job", "run_pem.def" and "obl_ecc_lsp.asc";
    > the optional PEM files: "diagevol.def" to define the PEM variables to be ouputted and "startevol.nc" to set the initial state of the PEM.

The PEM files can be found in the deftank folder.

Before a simulation, you have to set up some parameters/options in:
    > "pem_workflow.sh" (see below for details);
    > "pcm_run.job" (see below for details);
    > "pem_run.job" (see below for details);
    > def files, especially for "run_pem.def", "run_pcm.def" and "callphys.def" which must be consistent. In particular, "run_pem.def" needs to include some definitions for the physics in file "run_pcm.def" with "INCLUDEDEF=run_pcm.def".
In addition, the user has to provide a "startfi.nc" whose orbital parameters are consistent with the initial date set in "run_pem.def". The script "ini_pem_orbit.sh" can do it automatically reading "obl_ecc_lsp.asc".

Outputs:
--------
The PEM simulation generates the following files:
    > the usual outputs of the PCM: "restartfi.nc", "restart.nc"/"restart1D.txt", "diagfi.nc", etc;
    > the XIOS outputs of the PCM: "Xoutdaily4pem*.nc"/"Xoutyearly4pem*.nc";
    > the outputs of the chained simulation: "pem_workflow.log", "pem_workflow.sts" and possibly "kill_pem_workflow.sh";
    > the usual outputs of the PEM: "restartevol.nc", "restartfi.nc", "restart.nc"/"restart1D.txt" and "diagevol.nc".
During the simulation, the PCM/PEM run files are renamed conveniently and stored in the sub-directories "logs" (log files), "starts" (starting files) and "diags" (diagnostic files).
If you run a simulation by submitting jobs, the script "kill_pem_workflow.sh" is automatically generated. It can be used to kill the jobs related to your chained simulation in the queue of the job scheduler.

Deftank files:
--------------
# pem_workflow.sh:
  Bash script file to launch the chained simulation of PEM and PCM runs.
  The user has to specify:
      > 'n_planetary_years' or 'n_earth_years' -> the number of Planetary/Earth years to be simulated in total (> 0);
      > 'n_pcm_runs_ini' -> the number of initial PCM years (>= 2);
      > 'n_pcm_runs' -> the number of PCM years between each PEM run (>= 2, usually 2);
      > 'exec_mode' -> the execution mode (0 = "processing scripts"; any other values = "submitting jobs"). The former option is usually used to process the script on a local machine while the latter is used to submit jobs on a supercomputer with SLURM or PBS/TORQUE.
  The script can take an argument:
      1) None: to start a simulation from scratch;
      2) 're': to resume a simulation from a starting point (interactive prompt).

# pem_workflow_lib.sh:
  Library of bash functions used by the PEM workflow script "pem_workflow.sh".

# pcm_run.job:
  Bash script file to submit a PCM job (with SLURM or PBS/TORQUE).
  The user has to specify:
      > The headers correspond to the ADASTRA supercomputer and should be changed for other machines and job schedulers. In case of "processing scripts" execution mode, the headers are naturally omitted.
      > The path to source the arch file should be adapted to the machine.
      > 'pcm_exe', the name of the PCM executable file should be adapted.
      > 'exec_cmd', the execution command should also be adapted according to the set-up.

# pem_run.job:
  Bash script file to submit PEM job (with SLURM or PBS/TORQUE).
  The user has to specify:
      > The headers correspond to the ADASTRA supercomputer and should be changed for other machines and job schedulers. In case of "processing scripts" execution mode, the headers are naturally omitted.
      > The path to source the arch file should be adapted to the machine.
      > 'pem_exe', the name of the PEM executable file should be adapted.
      > 'pem_arg', the PEM executable can have an optional argument which should be specified according to the set-up ("--auto-exit" for SLURM and PBS/TORQUE | "" when the script is not run as a job).

# run_pem.def
  All the possible parameters to define a PEM run (read in "config.F90").
  It needs to include some definitions for the physics in file "run_pcm.def" with "INCLUDEDEF=run_pcm.def".

# obl_ecc_lsp.asc [default], obl_ecc_lsp_pos.asc [future years]
  Files describing orbital parameters for Mars from Laskar's data. It contains the Earth year (kyr) and the corresponding obliquity (°), eccentricity and Ls of perihelion (°).

# modify_startfi_orbit.sh:
  Bash script file to modify orbital parameters in the file "startfi.nc". They should be consistent with Laskar's data in "obl_ecc_lsp.asc" at the same date if orbital parameters are meant to vary during the simulation. See also "ini_pem_orbit.sh".

# modify_startfi_var.sh:
  Bash script file to modify the value of a variable in a file "startfi.nc".

# ini_pem_orbit.sh:
  Bash script file to set the orbital parameters of a file "startfi.nc" from Laskar's data contained in "obl_ecc_lsp.asc" according to the initial date 'pem_ini_earth_date' defined in "run_pem.def". See also "modify_startfi_orbit.sh".

# concat_pem.py:
  Python script file to concatenate the NetCDF files of the PEM along the dimension 'Time' into one NetCDF file. 'Time' is re-indexed by increment.

# clean.sh:
  Bash script file to clean the folder after a PEM simulation and reset the initial starting files to prepare a new simulation.

# multiple_exec.sh:
  Bash script file to execute multiple scripts in subdirectories. It is useful to launch multiple simulations at once.

# visu_layering.py:
  Python script file to output the stratification data from the "startevol.nc" files.

# visu_layering_evol.py:
  Python script file to output the stratification data over time from the "startevol.nc" files.