******************************************************************** * * * LMDZ_Setup is a software suite for installing and running LMDZ * * * ******************************************************************** Authors: Frédéric Hourdin, Adriana Sima, Amaury Baral, Laurent Fairhead (and the LMDZ team) Last modification : 2025/01/07 Complementary information and FAQ (in French) : https://docs.google.com/document/d/1OLZG6e-86NiXuv5-aALxKIh-QPkp4BdCwWtiBFot-6c =================================================================== 0. Target platforms =================================================================== Default gfortran/bash installation on local computers. Specific installation on certain computing centers: spirit: IPSL cluster adastra: Cines supercomputer jean-zay: idris supercomputer irene: TGCC supercomputer WORK TO BE DONE Machine configuration dependency is managed by lmdz_env.sh =================================================================== I. What this working environment does and does not =================================================================== It proposes a light automatic setup of LMDZ long chained climate simulations including installation of the model itself. It consists in running LMDZ coupled to Orchidee for land surface (optional) but with imposed sea surface temperature. The basic default configuration makes use of the IPSL-CM6A grid configuration and tuning, and runs a multi annual simulation on "climatological" amip sea surface temperature (with a mean annual cycle) using a calendar with 360 days. Optionally : the configuration includes the ability to use interannually varying SST and to activate "nudgning" by reanalysis. In both cases, the calendar is then a real one. When nudging is activated, the simulation must be run on a monthly basis, while otherwise, it can be either monthly or yearly. Since it automatically generates its own initial files, it can be run with zoom configurations by only changing the number of grid points ("resol" in main.sh) and the DEF/gcm.def file (see bellow). Aerosols can be : - not taken into account - read for the year 2000 (weighted average over 1999-2001 cf Lurton et al 2020), and instantaneous forcing with respect to 1850 can be computed as well. - computed interactively the SPLA model (SimPLe Aerosol, activated with option aerosols=spla in setup.sh). Emissions of dust and sea salt are then computed interactively. A configuration with isotopes is also available. =================================================================== II. Files and scripts =================================================================== ---------------------------------------------------------------- EXPERT on super computing centers : If the login groupe and computing allocation groupe are different : choose on which STORE space you work (login vs computing allocation); edit lmdz_env.sh, and set "groupe", "SCRATCHD" etc. accordingly. ---------------------------------------------------------------- main.sh : a script written to pass options to the main script setup.sh and configure one or series of simulations. setup.sh : sets up one simulation by editing the "job" script_SIMU When run with -init 1 options, creates INIT and LIMIT directories Run ./setup.sh -h for options RECOMMENDATION: do not run it directly, but use main.sh to drive it script_SIMU : script that is the basis for the job to run the simulations Automatically edited by setup.sh reb.sh : to rebuild splitted outputfiles (parallelism with IOISPL) automatically run by script_SIMU era2gcm_tuto.sh : to interpolate reanalysis onto the GCM grid, for nudging. To be run manually after the first use of main.sh season.sh : computing seasonal means and daily time series interp_aerosols.sh : interpolating aerosols on the GCM grid. Automatically run by setup.sh post_season.sh : post processing a series of simulations with season.sh test_ERAfiles_JeanZay.sh : script to test the availability of ERA* files sur Jean-Zay In case of nudging, one may use it before running era2gcm.sh DEF: .def files for simulations Some of them are changed by setup.sh or script_SIMU depending on chosen options Some others need to be chosen and customized BEFORE RUNNING main.sh - DEF/config.def (versions *.def_* available): output files, frequencies, levels (see lines phys_out_file*) - DEF/gcm.def (versions *.def_* available): grid (and grid-related) parameters - DEF/guide.def : nudging parameters ============================================================ III. Downloading input data ============================================================ Installation requires downloading data from from http://http://lmdz.lmd.jussieu.fr/pub/3DInputData Downloading is managed by a series of functions found in lmdz_env.sh: wget_pub cp_from_pub ln_from_pub get_input_files One specificity of LMDZ_Setup is to enable installation on any account on any machine, downloading all data from wget. All data are gathered on a directory $LMDZpub so that the downloading is done once for all for a given account. The LMDZsub path can be changed in lmdz_env.sh The correponding amount of data is very small compared to the amount of data generated. So I think we're very parsimonious in our use of IT resources. The counterpart is that LMDZ_Setup doesn't manage a lot of different configurations in terms of boundary conditions, forcing, etc. libiGCM installations should be privileged then. ============================================================ IV. Known problems ============================================================ Running zoomed or coarse grid simulations May cause problems with routing and aerosols Suggesting to run without aerosols, aerosols="n", in this case ============================================================ V. Updates (reverse chronological order) ============================================================ 2025-01 (FH) : retravail de l'accès au données sur http://lmdz... reactivation options sequential/debug. Configuration SPLA muti machines 2024-11 (AB) : Travail de portage multi-machine par Amaury Barral. Précédemment sur jean-zay uniquement. Porté sur adastra et machines locales. 2023-07 (AS-FH-LF) : mise sous svn du repertoire TEST_PROD, renommé en "LMDZ_config" puis "LMDZ_Setup" 2023-06 (AS) : Ajout de l'option "veget=7994", pour ORCHIDEE trunk correspondant à la config libIGCM LMDZOR_6.4work Le routage (méthode = simple) est désactivé par défaut ; il peut être activé comme décrit dans la section "ORCHIDEE-routage". Le routage n’a pas été testé avec zoom. 2023-05 (AS) : optimisation du functionnement de l'option isotopes : tout géré par phys=lmdiso 2023-04 (S Nguyen) : introducing the option of running with isotopes) 2021-10 (AS): introducing the option of using the new snow scheme INLANDSIS (setup.sh) Option not active yet, E Vignon will take care of that. 2021-05 : New version with SPLA 2020 (AS): introducing the possibility to run with SPLA aerosol model 24/03/2020 (FH): corrections for use with AMIP SSTs and monthly freq ============================================================ VI. TBD ============================================================ Better managment of DEF files Inclusion of Dynamico ============================================================ Appendix ============================================================ Notes concernant Orchidee ------------------------- Plusieurs versions disponibles via la clé veget veget=CMIP6 : version CMIP6 veget=XXX : XXX numéro de relase svn d'orchidee XXX+:1 On rapatrie automatiquement des champs nécessaires pour les dernières versions d'orchidee PFTmap_IPCC_2000.nc cartepente2d_15min.nc routing.nc routing_simple.nc lai2D.nc alb_bg_modisopt_2D_ESA_v2.nc reftemp.nc soils_param.nc woodharvest_2000.nc PFTmap_15PFT.v1_2000.nc soil_bulk_and_ph.nc Les deriners fichiers sont nécessaires seulement pour les version azotées. ndep_nhx.nc ndep_noy.nc nfert_cropland.nc nfert_pasture.nc nmanure_cropland.nc nmanure_pasture.nc bnf.nc" Elles sont cependant rapatriées systématiquement par souci de simplicité. Notes concernant la confiugration SPLA -------------------------------------- Pour l'instant on copie là-dedans de chez Binta les fichiers a la res zoomNaf; plus tard on y recupererea des fichiers a haute resolution reguliere depuis http:/LMDZ, a regrider ensuite par un script interp_fichiers_spla.sh (comme pour aerosols="clim") Les fichiers (regrides, sauf SOILSPEC.data utilise tel quel) seront mis dans $MAINDIR/INPUT_SPLA (equivalent de INPUT_DUST) The Path below is for JEANZAY. See with the team how to adapt to your cluster. #for file in donnees_lisa.nc SOILSPEC.data \ # For Chimere dust emissions # cly.dat $( for i in $(seq 1 12 ) ; do echo dust$i.nc ; done ) wth.dat \ # For Inca dust emissions # carbon_emissions.nc sulphur_emissions_antro.nc sulphur_emissions_nat.nc \ # For non dust emissions #Cas particulier des fichiers mensuels dust.nc :A DECIDER : #C'est entre le cas des aerosols.clim= 1 seul fichier, annuel, # et le cas des vents guidage, pré-interpolés avec era2gcm pour toute la periode, dans MAINDIR/GUIDE. # On pourrait (a)demander de precalculer dust.nc aussi, dans MAINDIR/INPUT_SPLA - avec un script à adapter de Jeronimo. Rien a mettre dans SPLA_Init alors. # Ou (b) fournir dans SPLA_Init les 12 mois d'un dust.nc climato (an 2006 pour nous ici) à la res EC, et faire juste le regrid vers MAINDIR/INPUT_SPLA A la fin on doit avoir dans SPLA_Init les fichiers initiaux dans INITIAL plus les repertoires PERIOD00MM contenant dust.nc Cela doit se retrouver dans script_SIMU qui les copie dans le repertoire de run sur $SCRATCH