source: trunk/MESOSCALE_DEV/MANUAL/SRC/preproc.tex @ 220

\chapter{Preprocessing utilities}\label{zepreproc}

\vk
In this chapter, we describe the installation and use of the preprocessing tools to define the domain of simulation, calculate an initial atmospheric state and prepare the boundary conditions for the chosen simulation season and time of day. This corresponds to steps 1,2,3 as defined in section~\ref{steps}. These operations would eventually allow you to run your own simulations at the specific season and region you are interested in, with a complete ability to modify any of the parameters in \ttt{namelist.input}, including the ones labelled with~\ttt{(p1)}, \ttt{(p2)} or \ttt{(p3)}.

\mk
\section{Installing the preprocessing utilities}

\sk
The compilation operations indicated here need to be done only once on a given system. 

\sk
\subsection{Prerequisites}

\sk
First and foremost, since the preprocessing utilities could generate (or involve) files of quite significant sizes, it is necessary to define a directory where these files would be stored. Such a directory (e.g. \ttt{/bigdisk/user}) must be linked with the name \ttt{\$TMPDIR} as follows. In addition, three directories \ttt{GCMINI}, \ttt{WPSFEED}, \ttt{WRFFEED} have to be defined in \ttt{\$LMDMOD/TMPDIR} as indicated below.

\begin{verbatim}
ln -sf /bigdisk/user $LMDMOD/TMPDIR
mkdir $LMDMOD/TMPDIR/GCMINI
mkdir $LMDMOD/TMPDIR/WPSFEED
mkdir $LMDMOD/TMPDIR/WRFFEED
\end{verbatim}

\sk
A second prerequisite to the installation of the preprocessing tools is that the LMD Martian Mesoscale Model was compiled at least once. If this is not the case, please compile the model with the \ttt{makemeso} command described in section~\ref{sc:makemeso}. The compilation process created an installation directory adapted to your particular choice of compiler$+$machine (what we named \ttt{DIRCOMP} for illustration in section~\ref{sc:makemeso}, which could be for instance \ttt{g95\_32\_single}). The preprocessing tools will also be installed in this directory. Please type the following commands:

\begin{verbatim}
cd $LMDMOD/LMD_MM_MARS/g95_32_single/   ## or any of your install directory
ln -sf ../SRC/SCRIPTS/prepare_ini .
./prepare_ini
\end{verbatim}

\sk
\subsection{Compiling preprocessing utilities}

\sk
The script \ttt{prepare\_ini} plays for the preprocessing tools a similar role as the \ttt{copy\_model} with the model sources: files are simply linked to their actual location in the \ttt{SRC} folder. Once you have executed \ttt{prepare\_ini}, please check that two folders were generated: \ttt{PREP\_MARS} and \ttt{WPS}. In the \ttt{PREP\_MARS} directory, please compile the programs \ttt{create\_readmeteo.exe} and \ttt{readmeteo.exe}, using the compiler mentionned in the name of the current installation directory. In the \ttt{WPS} directory, please compile the programs \ttt{geogrid.exe} and \ttt{metgrid.exe}.

\begin{verbatim}
echo $PWD
cd PREP_MARS/
./compile [or] ./compile_g95     ## the first script compiles with pgf90 
                                 ## the second script compiles with g95 
                                 ## scripts can be easily adapted to, e.g., ifort
ls -lt create_readmeteo.exe readmeteo.exe
cd ..
cd WPS/   
./configure     ## select your compiler + 'NO GRIB2' option
./compile
ls -lt geogrid.exe metgrid.exe
\end{verbatim}

\sk
Apart from the executables just compiled, the preprocessing utilities include \ttt{real.exe}, which was compiled by the \ttt{makemeso} script along with the mesoscale model executable \ttt{wrf.exe}\footnote{Even though the name of the executable reads e.g. \ttt{real\_x61\_y61\_z61\_d1\_t1\_p1.exe}, such program is not related to the specific \ttt{makemeso} parameters -- contrary to the \ttt{wrf.exe} executable. We just found that renaming the (possibly similar if the model sources were not modified) \ttt{real.exe} was a practical way not to confuse between executables compiled at different moments.}. \ttt{real.exe} should be copied or linked in the simulation directory (e.g. \ttt{TESTCASE} for the Arsia Mons test case) to be at the same level than \ttt{namelist.input}.

\sk
\subsection{Preparing input static data}

\sk
All the static data (topography, thermal inertia, albedo) needed to initialize the model are included in the \ttt{\$LMDMOD/LMD\_MM\_MARS/WPS\_GEOG} directory. By default, only coarse-resolution datasets\footnote{ Corresponding to the fields stored in the file \ttt{surface.nc} known by LMD-MGCM users: \url{http://web.lmd.jussieu.fr/~forget/datagcm/datafile/surface.nc} } are available, but the directory also contains sources and scripts to install finer resolution datasets: 32 and/or 64 pixel-per-degree (ppd) MOLA topography (\ttt{mola\_topo32} and \ttt{mola\_topo64}), 8 ppd MGS/Thermal Emission Spectrometer (TES) albedo (\ttt{albedo\_TES}), 20 ppd TES thermal inertia (\ttt{thermal\_TES}). The role of the \ttt{build\_static} script is to automatically download these datasets from the web (namely PDS archives) and convert them to an acceptable format for a future use by the preprocessing utilities:

\begin{verbatim}
cd $LMDMOD/LMD_MM_MARS
./build_static
\end{verbatim}

\sk
\begin{finger}
\item Please install the \ttt{octave} free software\footnote{ Available at \url{http://www.gnu.org/software/octave} } on your system to execute the \ttt{build\_static} script\footnote{ Another solution is to browse into each of the directories within \ttt{WPS\_GEOG/res}, download the data with the shell scripts and execute the \ttt{.m} scripts with either \ttt{octave} or the commercial software \ttt{matlab} (just replace \ttt{\#} by \ttt{\%}). }. 
\item Building the MOLA 64ppd database can be quite long; hence this is not performed by default by the \ttt{build\_static} script. If you would like to build this database, please remove the \ttt{exit} command in the script, just above the commands related to the MOLA 64ppd.
\item If you do not manage to execute the \ttt{build\_static} script, ready-to-use datafiles can be found in the link \url{ftp://ftp.lmd.jussieu.fr/pub/aslmd} and must be extracted in \ttt{\$MMM/WPS\_GEOG}.
\item The resulting \ttt{WPS\_GEOG} can reach a size of several hundreds of Mo. You might move such a folder in a place with more disk space available and define a link named~\ttt{WPS\_GEOG} in \ttt{\$MMM}.
\end{finger}

\sk
\subsection{Compiling the GCM for initial and boundary conditions}

\sk
The LMD Martian GCM is supposed to be run to compute meteorological fields that will be used as initial and boundary conditions each one or two Martian hours to the limited-area LMD Martian Mesoscale Model. Hence the LMD Martian GCM must be compiled in your system (see the LMD-MGCM user manual for further details \url{http://web.lmd.jussieu.fr/~forget/datagcm/user_manual.pdf}). If you did not get the model using the \ttt{svn} method, please request us to send you an archive containing the LMD-MGCM named \ttt{LMDZ.MARS.meso.tar.gz}, which you have to extract in the \ttt{\$LMDMOD} directory. If you got the model using \ttt{svn}, you do not have to request this file. In the \ttt{\$LMDMOD/LMDZ.MARS} directory, a script named \ttt{compile} can be found and must be used \emph{on the system you plan to run the mesoscale model on} to compile the GCM. The \ttt{compile} script is actually just a wrapper for the \ttt{makegcm} script which compile the GCM for you; the default \ttt{makegcm} script only works with Portland Group Fortran compiler \ttt{pgf90} but scripts allowing to compile the model using other Fortran compilers (including \ttt{g95} or \ttt{ifort}) are available upon request. The following commands must be used and should yield the compilation of two executables \ttt{newstart.e} and \ttt{gcm.e}:

\begin{verbatim}
cd $LMDMOD/LMDZ.MARS
./compile
\end{verbatim}

\sk
The other necessary operation to prepare the LMD-MGCM for step~1 is to store a set of initial states for the LMD-MGCM to start with -- based on previous typical LMD-MGCM runs having reached equilibrium after ten years of integration. A reference database can be found in the following online big archive~\url{ftp://ftp.lmd.jussieu.fr/pub/aslmd/STARTBASE_64_48_32_t2.tar.gz}. This archive must be extracted somewhere on a disk that would be accessible by the system you plan to run the mesoscale model on; the absolute link of the \ttt{STARTBASE\_64\_48\_32\_t2} directory on your disk must be reported in the beginning of the script~\ttt{\$LMDMOD/LMDZ.MARS/myGCM/launch\_gcm} (variable \ttt{startbase}). If those operations went well, please try the command line~\ttt{echo 22 | launch\_gcm} which should launch the GCM integrations on your system.

\mk
\section{Running the preprocessing utilities}

\sk
\subsection{General overview}

\sk
When you run a simulation with \ttt{wrf.exe} (e.g. section \ref{sc:arsia}), the program attempts to read the initial state in \ttt{wrfinput\_d01} and the domain boundary conditions in \ttt{wrfbdy\_d01}. The whole chain of data conversion and interpolation needed to generate those files is summarized in the diagram on Figure~\ref{preproc}. Three distinct preprocessing steps are necessary to generate the final files (steps are numbered 1,2,3 as in section~\ref{steps}). Figure~\ref{preproc} helps to better understand the labels \ttt{(p1)}, \ttt{(p2)}, \ttt{(p3)} used to describe \ttt{namelist.input} parameters in chapter~\ref{zeparam}. For instance: 
\begin{finger}
\item changing the season of simulation implies to re-run the LMD Mars GCM for this specific season to prepare initial and boundary conditions for the mesoscale model. Hence e.g. \ttt{start\_month} is labelled with \ttt{(p1)} because changing this in \ttt{namelist.input} requires a complete reprocessing from step~$1$ to step~$3$ to successfully launch the simulation.
\item changing the number of horizontal grid points for the mesoscale domain implies to interpolate the static and GCM fields to the new domain, while no new computations on the GCM side are needed. Hence e.g. \ttt{e\_we} is labelled with \ttt{(p2)} because changing this in \ttt{namelist.input} requires a reprocessing from step~$2$ to step~$3$ to successfully launch the simulation (and also for this specific parameter recompiling with \ttt{makemeso}).
\item changing the position of model top implies to interpolate initial and boundary conditions to the new vertical levels, while no horizontal re-interpolations are needed. Hence e.g. \ttt{p\_top\_requested} is labelled with \ttt{(p3)} because changing this requires a reprocessing of step~$3$.
\item changing the timestep for dynamical integration does not require any change in initial and bouondary conditions. Hence e.g. \ttt{time\_step} is not labelled with \ttt{(p1)}, \ttt{(p2)} or \ttt{(p3)}.
\end{finger}

\begin{center}
\begin{figure}[p] 
\includegraphics[width=0.99\textwidth]{diagramme.pdf} 
\caption{\label{preproc} The details of preprocessing steps and their related software and inputs/ouputs}
\end{figure}
\end{center}

\end{document}

\mk
\subsubsection{Meteorological data}

\ttt{launch\_gcm}

\mk
The preprocessing tools generate initial and boundary conditions
from the \ttt{diagfi.nc} outputs of LMD-MGCM simulations.
%
If you would like to run a mesoscale simulation at a given
season, you need to first run a GCM simulation and output
the meteorological fields at the considered season.
%
For optimal forcing at the boundaries, we advise you
to write the meteorological fields to the 
\ttt{diagfi.nc} file at least each two hours.
%
Please also make sure that the following fields
are stored in the NETCDF \ttt{diagfi.nc} file:

\footnotesize
\codesource{contents_diagfi}

\normalsize
\begin{finger}
\item If the fields 
\ttt{emis}, 
\ttt{co2ice}, 
\ttt{q01}, 
\ttt{q02}, 
\ttt{tsoil} 
are missing in the \ttt{diagfi.nc} file, 
they are replaced by respective default 
values $0.95$, $0$, $0$, $0$, tsurf.
\end{finger}

\mk
\marge An example of input meteorological file 
\ttt{diagfi.nc} file can be downloaded
at \url{http://web.lmd.jussieu.fr/~aslmd/LMD_MM_MARS/diagfi.nc.tar.gz}.
%
Please deflate the archive and copy the \ttt{diagfi.nc} file
in \ttt{\$LMDMOD/TMPDIR/GCMINI}.
%
Such a file can then be used to define the initial
and boundary conditions, and we will go 
through the three preprocessing steps.

\mk
\subsection{Preprocessing steps} 

\mk
\subsubsection{Step 1: Converting GCM data}

\mk
The programs in the \ttt{PREP\_MARS} directory
convert the data from the NETCDF \ttt{diagfi.nc}
file into separated binary datafiles for each
date contained in \ttt{diagfi.nc}, according to
the formatting needed by the 
preprocessing programs at step 2.
%
These programs can be executed by the following
commands: 
\begin{verbatim}
cd $LMDMOD/LMD_MM_MARS/your_install_dir/PREP\_MARS
echo 1 | ./create_readmeteo.exe     # drop the "echo 1 |" if you want control
./readmeteo.exe < readmeteo.def
\end{verbatim}
%
\marge If every went well with the conversion,
the directory \ttt{\$LMDMOD/TMPDIR/WPSFEED}
should contain files named \ttt{LMD:}.

\mk
\subsubsection{2: Interpolation on the regional domain}

\mk
In the \ttt{WPS} directory, the \ttt{geogrid.exe} program allows 
you to define the mesoscale simulation domain
to horizontally interpolate the topography, 
thermal inertia and albedo fields at the domain
resolution and to calculate useful fields
such as topographical slopes.%\pagebreak

\mk
\marge Please execute the commands:
%
\begin{verbatim}
cd $LMDMOD/LMD_MM_MARS/your_install_dir/WPS
ln -sf ../../TESTCASE/namelist.wps .   # test case
./geogrid.exe
\end{verbatim}
%
\marge The result of \ttt{geogrid.exe} 
-- and thus the definition of the mesoscale
domain -- can be checked in the NETCDF
file \ttt{geo\_em.d01.nc}.
%
A quick check can be performed using the command line
\begin{verbatim}
ncview geo_em.d01.nc
\end{verbatim} 
\marge if \ttt{ncview} is installed, or the \ttt{IDL}
script \ttt{out\_geo.pro}
\begin{verbatim}
idl
IDL> out_geo, field1='TOPO'
IDL> out_geo, field1='TI'
IDL> SPAWN, 'ghostview geo_em.d01_HGT_M.ps &'
IDL> SPAWN, 'ghostview geo_em.d01_THERMAL_INERTIA.ps &'
IDL> exit
\end{verbatim}
\marge if the demo version of \ttt{IDL} is installed.
%
Of course if your favorite graphical tool supports
the NETCDF standard, you might use it to check the
domain definition in \ttt{geo\_em.d01.nc}.

\mk
\marge If you are unhappy with the results or
you want to change 
the location of the mesoscale domain on the planet, 
the horizontal resolution,
the number of grid points \ldots,
please modify the parameter
file \ttt{namelist.wps} and execute again \ttt{geogrid.exe}.
%
Here are the contents of \ttt{namelist.wps}:
%
\codesource{namelist.wps_TEST} 

\begin{finger}
%
\item No input meteorological data 
are actually needed to execute \ttt{geogrid.exe}.
%
\item More details about the database and
more options of interpolation could be
found in the file \ttt{geogrid/GEOGRID.TBL}.
%
\item Defining several domains yields
distinct files 
\ttt{geo\_em.d01.nc},
\ttt{geo\_em.d02.nc},
\ttt{geo\_em.d03.nc}\ldots
\end{finger}

\mk
\marge Once the \ttt{geo\_em} file(s) are generated,
the \ttt{metgrid.exe} program performs
a similar horizontal interpolation 
of the meteorological fields to the mesoscale 
domain as the one performed by \ttt{geogrid.exe} 
for the surface data.
%
Then the program writes the results in
\ttt{met\_em} files and also collects
the static fields and domain parameters
included in the \ttt{geo\_em} file(s)
%
Please type the following commands:
\begin{verbatim}
cd $LMDMOD/LMD_MM_MARS/your_install_dir/WPS
./metgrid.exe
\end{verbatim}
%
\marge If every went well,
the directory \ttt{\$LMDMOD/TMPDIR/WRFFEED}
should contain the \ttt{met\_em.*} files.

\mk
\subsubsection{Step 3: Vertical interpolation on mesoscale levels}

\mk
\marge The last step is to execute \ttt{real.exe}
to perform the interpolation from the vertical
levels of the GCM to the vertical levels 
defined in the mesoscale model.
%
This program also prepares the final initial
state for the simulation in files called
\ttt{wrfinput} and the boundary conditions
in files called \ttt{wrfbdy}.

\mk
\marge To successfully execute \ttt{real.exe}, 
you need the \ttt{met\_em.*} files
and the \ttt{namelist.input} file
to be in the same directory as \ttt{real.exe}.
%
Parameters in \ttt{namelist.input} 
controlling the behavior of the vertical interpolation 
are those labelled with \ttt{(p3)} in the detailed
list introduced in the previous chapter.

\mk
\marge Please type the following commands
to prepare files for the Arsia Mons test case
(or your personal test case if you changed
the parameters in \ttt{namelist.wps}):
\begin{verbatim}
cd $LMDMOD/TESTCASE
ln -sf $LMDMOD/WRFFEED/met_em* .
./real.exe
\end{verbatim}

\mk
\marge The final message of the \ttt{real.exe}
should claim the success of the processes and you
are now ready to launch the integrations
of the LMD Martian Mesoscale Model again
with the \ttt{wrf.exe} command as in section
\ref{sc:arsia}.

\begin{finger}
\item When you modify either 
\ttt{namelist.wps} or \ttt{namelist.input},
make sure that the common parameters
are exactly similar in both files
(especially when running nested simulations)
otherwise either \ttt{real.exe} or \ttt{wrf.exe}
command will exit with an error message.
\end{finger}
%\pagebreak