\newpage
%

\chapter{Compiling, running and debugging}
\label{sec_compilationrunning}

\section{Compiling OASIS3-MCT}
\label{subsec_compile}

Compiling OASIS3-MCT libraries can be done in directory {\tt
  oasis3-MCT/util/make\_dir} with Makefile \\ {\tt TopMakefileOasis3}
which must be completed with a header file {\tt make.{\it
    your\_platform}} specific to the compiling platform used and
specified in {\tt oasis3-MCT/util/make\_dir/make.inc}.  One of the
header files distributed with the release can by used as a template.
The root of the OASIS3-MCT tree can be anywhere and must be set in the
variable {\tt COUPLE} in the {\tt make.{\it your\_platform}} file.

The coupled models must be compiled with the same library as the one 
used to compile OASIS3-MCT.

The following commands are available:

\begin{itemize}
\item {\tt make -f TopMakefileOasis3} or {\tt make oasis3\_psmile -f
    TopMakefileOasis3} (for backward compatibility):

  compiles all OASIS3-MCT libraries {\it mct}, {\it scrip} and {\it
    psmile};

\item {\tt make realclean -f TopMakefileOasis3}:

  removes all OASIS3-MCT compiled sources and librairies.

\end{itemize}

Log and error messages from compilation are saved in {\tt /util/make\_dir} in the files
{\tt COMP.log} and {\tt COMP.err} .

During compilation, a new compiling directory, defined by variable
{\tt ARCHDIR} is created.  After successful compilation, resulting
libraries are found in the compiling directory in {\tt /lib} and
object and module files in {\tt /build}. 

If module {\tt mod\_oasis} is used in the
models, it is enough to include the path of the psmile objects and
modules ({\tt \$ARCHDIR/build/lib/psmile.MPI1}) for the compilation and to use the psmile library {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} when
linking. If module {\tt mod\_prism} is used in the models, it is necessary to
include the path of the psmile and of the mct objects and modules for
the compilation (i.e. {\tt \$ARCHDIR/build/lib/psmile.MPI1} and {\tt /mct} and to use both the psmile and mct libraries {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} and {\tt libmct.a} and {\tt libmpeu.a} when linking.

Exchange of coupling fields in single and double precision is now supported directly through the interface 
(see section \ref{subsubsec_Declaration}), even if all real variables are now treated in double precision internally.
For double precision coupling field, there is no need anymore to promote REAL variables to double precision at compilation.

\section{CPP keys}
\label{subsec_cpp}

The following CPP keys are coded in OASIS3-MCT and can be specified in
{\tt CPPDEF} in {\tt make.{\it your\_platform}} file.

\begin{itemize}

\item {\tt TREAT\_OVERLAY}: To ensure, in {\tt SCRIPR/CONSERV}
  remapping (see section \ref{subsec_interp}), that if two cells of
  the source grid overlay, at least the one with the greater numerical
  index is masked (they also can be both masked); this is mandatory
  for this remapping. For example, if the grid line with {\it i=1} overlaps
  the grid line with {\it i=imax}, it is the latter that must be masked;
  when this is not the case with the mask defined in {\it masks.nc},
  this CPP key forces these rules to be respected.

\item {\tt balance}: Add a MPI\_Wtime() function before and after
  mct\_isend (MPI put) and mct\_recv (MPI get) to calculate the time
  of the send and receive of a coupling field. This option can be used
  to produce timestamps in OASIS3-MCT debug files. During a post-processing
  phase, this information can be used to perform an analysis of the
  coupled components load (un)balance; specific tools have been
  developed to do this and will be released with a further version of
  OASIS3-MCT. {\bf This option is temporarily not recommended as it was observed that
  it was increasing the simulation time of coupled models on
  the PRACE computer MareNostrum.}

\end{itemize}

\section{Running OASIS3-MCT}
\label{subsec_running}

Examples of running environement are provided with the sources. For
more details, see the instructions on OASIS web site at
{\tt https://verc.enes.org/oasis/oasis-dedicated-user-support-1/} {\tt user-toys/tutorial-and-test\_interpolation-of-oasis3-mct-1}.
.

\subsection{The tutorial toy model}
\label{subsec_tutorial}

A practical example on how to run OASIS3-MCT and running it in a
coupled system is provided in {\tt oasis3-mct/examples/tutorial}. See also the document {\tt tutorial\_oasis3-mct.pdf} there in for
more details,

\subsection{The test\_interpolation environment}
\label{subsec_testinterpolation}

This environment available with the sources in {\tt
  oasis3-mct/examples/test\_interpolation} allows the user to test the
quality of the interpolations and transformations between his source
and target grids by calculating the error of interpolation on the
target grid. For more details, see also the document {\tt test\_interpolation\_oasis3-mct.pdf} there in.

\section{Debugging}
\label{subsec_debug}

\subsection{Debug files}
If you experience problems while running your coupled model with
OASIS3-MCT, you can obtain more information on what is happening by
increasing the {\tt \$NLOGPRT} value in your {\it namcouple} (see also section
\ref{subsec_namcouplefirst}).

Different outputs are written depending on {\tt \$NLOGPRT} value:
\begin{itemize}
\item {0} : one file debug.root.xx is open by the master process of
  each model and one file debug\_notroot.xx is open for all the other
  processes of each model to write only error information if an error
  occurs.
\item {1} : one file debug.root.xx is open by the master process of
  each model to write information equivalent to level 10 (see below);
  one file debug\_notroot.xx is open for all the other processes of
  each model to write only error information if an error occurs.
\item {2} : one file debug.xx.xxxxxx is open by each process of each
  model to write normal production diagnostics.
\item {5} : as for 2 with in addition some initial debug info.
\item {10} : as for 5 with in addition the routine calling tree.
\item {12} : as for 10 with in addition some routine calling notes.
\item {15} : as for 12 with even more debug diagnostics.
\item {20} : as for 15 with in addition some extra runtime analysis.
\item {30} : full debug information.
\end{itemize}

\subsection{Time statistics files}
\label{timestat}

The variable TIMER\_Debug, defined in the {\it namcouple} (second
number on the line below \$NLOGPRT keyword), is used to obtain time
statistics over all the processors for each routine.

Different output are written (in files named {\tt *.timers\_xxxx})
depending on TIMER\_Debug value :

\begin{itemize}
\item {TIMER\_Debug=0} : nothing is calculated, nothing is written.
\item {TIMER\_Debug=1} : the times are calculated and written in a
  single file by the process 0 as well as the min and the max times
  over all the processes.
\item {TIMER\_Debug=2} : the times are calculated and each process
  writes its own file ; process 0 also writes the min and the max
  times over all the processes in its file.
\item {TIMER\_Debug=3} : the times are calculated and each process
  writes its own file ; process 0 also writes in its file the min
  and the max times over all processes and also writes in its file
  all the results for each process.
\end{itemize}

The time given for each timer is calculated by the difference between
calls to {\tt oasis\_timer\_start()} and {\tt oasis\_timer\_stop()}
and is the accumulated time over the entire run. Here is an overview
of the meaning of the different timers as implemented by default.

\begin{itemize}
\item {'total after init'} : total time of the simulation, implemented
  in {\tt mod\_oasis\_method}, i.e. between the end of {\tt
    oasis\_init\_comp} and the call to {\tt mpi\_barrier} and {\tt
    mpi\_finalize} in routine {\tt oasis\_terminate}.
\item {'map definition'} : time spent in {\tt mct\_gsmap\_init} in
  routine {\tt oasis\_def\_partition}; this routine defines the
  patterns of communication between the source and target processes.
\item {'cpl\_setup'} :time spent in {\tt oasis\_coupler\_setup}, which
  sets up additional coupling aspects related to {\tt
    oasis\_method\_enddef}.
\item {'cpl\_smatrd'} : time spent in {\tt
    oasis\_coupler\_smatreaddnc} in {\tt mod\_oasis\_coupler} (routine
  {\tt oasis\_coupler\_setup}); this routine performs a distributed
  read of a NetCDF SCRIP file and returns weights in a distributed
  SparseMatrix, and calls {\tt mct\_sMatP\_Init} which initialises the
  sparse matrix vector multiplication.
\item {'oasis\_advance\_init()'} : time spent in {\tt
    oasis\_advance\_init}, which handles reading of initial coupling
  restart and communication of data for fields with positive lags.
\item {'grcv\_00x'} : time spent in the reception of field x in {\tt
    mct\_recv} (including communication and possible waiting time
  linked to unbalance of components).
\item {'wout\_00x'} : time spent in the I/O for field x in routine
  {\tt oasis\_advance\_run}.
\item {'gcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run}
  in copying the field x just received in a local array.
\item {'pcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run}
  in copying the local field x in the array to send (i.e. with local
  transformation besides division for averaging).
\item {'pavg\_00x'} : time spent in routine {\tt oasis\_advance\_run}
  to calculate the average of field x (if done).
\item {'pmap\_00x'/'gmap\_00x'} : time spent in routine {\tt
    oasis\_advance\_run} for the matrix vector multiplication for
  field x on the source/target processes.
\item {'psnd\_00x'} : time spent in routine {\tt oasis\_advance\_run}
  for sending field x (i.e. including call to {\tt mct\_waitsend} and
  {\tt mct\_isend}).
\end{itemize}