[4775] | 1 | \newpage |
---|
| 2 | % |
---|
| 3 | |
---|
| 4 | \chapter{Compiling, running and debugging} |
---|
| 5 | \label{sec_compilationrunning} |
---|
| 6 | |
---|
| 7 | \section{Compiling OASIS3-MCT} |
---|
| 8 | \label{subsec_compile} |
---|
| 9 | |
---|
| 10 | Compiling OASIS3-MCT libraries can be done in directory {\tt |
---|
| 11 | oasis3-MCT/util/make\_dir} with Makefile \\ {\tt TopMakefileOasis3} |
---|
| 12 | which must be completed with a header file {\tt make.{\it |
---|
| 13 | your\_platform}} specific to the compiling platform used and |
---|
| 14 | specified in {\tt oasis3-MCT/util/make\_dir/make.inc}. One of the |
---|
| 15 | header files distributed with the release can by used as a template. |
---|
| 16 | The root of the OASIS3-MCT tree can be anywhere and must be set in the |
---|
| 17 | variable {\tt COUPLE} in the {\tt make.{\it your\_platform}} file. |
---|
| 18 | |
---|
| 19 | The coupled models must be compiled with the same library as the one |
---|
| 20 | used to compile OASIS3-MCT. |
---|
| 21 | |
---|
| 22 | The following commands are available: |
---|
| 23 | |
---|
| 24 | \begin{itemize} |
---|
| 25 | \item {\tt make -f TopMakefileOasis3} or {\tt make oasis3\_psmile -f |
---|
| 26 | TopMakefileOasis3} (for backward compatibility): |
---|
| 27 | |
---|
| 28 | compiles all OASIS3-MCT libraries {\it mct}, {\it scrip} and {\it |
---|
| 29 | psmile}; |
---|
| 30 | |
---|
| 31 | \item {\tt make realclean -f TopMakefileOasis3}: |
---|
| 32 | |
---|
| 33 | removes all OASIS3-MCT compiled sources and librairies. |
---|
| 34 | |
---|
| 35 | \end{itemize} |
---|
| 36 | |
---|
| 37 | Log and error messages from compilation are saved in {\tt /util/make\_dir} in the files |
---|
| 38 | {\tt COMP.log} and {\tt COMP.err} . |
---|
| 39 | |
---|
| 40 | During compilation, a new compiling directory, defined by variable |
---|
| 41 | {\tt ARCHDIR} is created. After successful compilation, resulting |
---|
| 42 | libraries are found in the compiling directory in {\tt /lib} and |
---|
| 43 | object and module files in {\tt /build}. |
---|
| 44 | |
---|
| 45 | If module {\tt mod\_oasis} is used in the |
---|
| 46 | models, it is enough to include the path of the psmile objects and |
---|
| 47 | modules ({\tt \$ARCHDIR/build/lib/psmile.MPI1}) for the compilation and to use the psmile library {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} when |
---|
| 48 | linking. If module {\tt mod\_prism} is used in the models, it is necessary to |
---|
| 49 | include the path of the psmile and of the mct objects and modules for |
---|
| 50 | the compilation (i.e. {\tt \$ARCHDIR/build/lib/psmile.MPI1} and {\tt /mct} and to use both the psmile and mct libraries {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} and {\tt libmct.a} and {\tt libmpeu.a} when linking. |
---|
| 51 | |
---|
| 52 | Exchange of coupling fields in single and double precision is now supported directly through the interface |
---|
| 53 | (see section \ref{subsubsec_Declaration}), even if all real variables are now treated in double precision internally. |
---|
| 54 | For double precision coupling field, there is no need anymore to promote REAL variables to double precision at compilation. |
---|
| 55 | |
---|
| 56 | \section{CPP keys} |
---|
| 57 | \label{subsec_cpp} |
---|
| 58 | |
---|
| 59 | The following CPP keys are coded in OASIS3-MCT and can be specified in |
---|
| 60 | {\tt CPPDEF} in {\tt make.{\it your\_platform}} file. |
---|
| 61 | |
---|
| 62 | \begin{itemize} |
---|
| 63 | |
---|
| 64 | \item {\tt TREAT\_OVERLAY}: To ensure, in {\tt SCRIPR/CONSERV} |
---|
| 65 | remapping (see section \ref{subsec_interp}), that if two cells of |
---|
| 66 | the source grid overlay, at least the one with the greater numerical |
---|
| 67 | index is masked (they also can be both masked); this is mandatory |
---|
| 68 | for this remapping. For example, if the grid line with {\it i=1} overlaps |
---|
| 69 | the grid line with {\it i=imax}, it is the latter that must be masked; |
---|
| 70 | when this is not the case with the mask defined in {\it masks.nc}, |
---|
| 71 | this CPP key forces these rules to be respected. |
---|
| 72 | |
---|
| 73 | \item {\tt balance}: Add a MPI\_Wtime() function before and after |
---|
| 74 | mct\_isend (MPI put) and mct\_recv (MPI get) to calculate the time |
---|
| 75 | of the send and receive of a coupling field. This option can be used |
---|
| 76 | to produce timestamps in OASIS3-MCT debug files. During a post-processing |
---|
| 77 | phase, this information can be used to perform an analysis of the |
---|
| 78 | coupled components load (un)balance; specific tools have been |
---|
| 79 | developed to do this and will be released with a further version of |
---|
| 80 | OASIS3-MCT. {\bf This option is temporarily not recommended as it was observed that |
---|
| 81 | it was increasing the simulation time of coupled models on |
---|
| 82 | the PRACE computer MareNostrum.} |
---|
| 83 | |
---|
| 84 | \end{itemize} |
---|
| 85 | |
---|
| 86 | \section{Running OASIS3-MCT} |
---|
| 87 | \label{subsec_running} |
---|
| 88 | |
---|
| 89 | Examples of running environement are provided with the sources. For |
---|
| 90 | more details, see the instructions on OASIS web site at |
---|
| 91 | {\tt https://verc.enes.org/oasis/oasis-dedicated-user-support-1/} {\tt user-toys/tutorial-and-test\_interpolation-of-oasis3-mct-1}. |
---|
| 92 | . |
---|
| 93 | |
---|
| 94 | \subsection{The tutorial toy model} |
---|
| 95 | \label{subsec_tutorial} |
---|
| 96 | |
---|
| 97 | A practical example on how to run OASIS3-MCT and running it in a |
---|
| 98 | coupled system is provided in {\tt oasis3-mct/examples/tutorial}. See also the document {\tt tutorial\_oasis3-mct.pdf} there in for |
---|
| 99 | more details, |
---|
| 100 | |
---|
| 101 | \subsection{The test\_interpolation environment} |
---|
| 102 | \label{subsec_testinterpolation} |
---|
| 103 | |
---|
| 104 | This environment available with the sources in {\tt |
---|
| 105 | oasis3-mct/examples/test\_interpolation} allows the user to test the |
---|
| 106 | quality of the interpolations and transformations between his source |
---|
| 107 | and target grids by calculating the error of interpolation on the |
---|
| 108 | target grid. For more details, see also the document {\tt test\_interpolation\_oasis3-mct.pdf} there in. |
---|
| 109 | |
---|
| 110 | \section{Debugging} |
---|
| 111 | \label{subsec_debug} |
---|
| 112 | |
---|
| 113 | \subsection{Debug files} |
---|
| 114 | If you experience problems while running your coupled model with |
---|
| 115 | OASIS3-MCT, you can obtain more information on what is happening by |
---|
| 116 | increasing the {\tt \$NLOGPRT} value in your {\it namcouple} (see also section |
---|
| 117 | \ref{subsec_namcouplefirst}). |
---|
| 118 | |
---|
| 119 | Different outputs are written depending on {\tt \$NLOGPRT} value: |
---|
| 120 | \begin{itemize} |
---|
| 121 | \item {0} : one file debug.root.xx is open by the master process of |
---|
| 122 | each model and one file debug\_notroot.xx is open for all the other |
---|
| 123 | processes of each model to write only error information if an error |
---|
| 124 | occurs. |
---|
| 125 | \item {1} : one file debug.root.xx is open by the master process of |
---|
| 126 | each model to write information equivalent to level 10 (see below); |
---|
| 127 | one file debug\_notroot.xx is open for all the other processes of |
---|
| 128 | each model to write only error information if an error occurs. |
---|
| 129 | \item {2} : one file debug.xx.xxxxxx is open by each process of each |
---|
| 130 | model to write normal production diagnostics. |
---|
| 131 | \item {5} : as for 2 with in addition some initial debug info. |
---|
| 132 | \item {10} : as for 5 with in addition the routine calling tree. |
---|
| 133 | \item {12} : as for 10 with in addition some routine calling notes. |
---|
| 134 | \item {15} : as for 12 with even more debug diagnostics. |
---|
| 135 | \item {20} : as for 15 with in addition some extra runtime analysis. |
---|
| 136 | \item {30} : full debug information. |
---|
| 137 | \end{itemize} |
---|
| 138 | |
---|
| 139 | \subsection{Time statistics files} |
---|
| 140 | \label{timestat} |
---|
| 141 | |
---|
| 142 | The variable TIMER\_Debug, defined in the {\it namcouple} (second |
---|
| 143 | number on the line below \$NLOGPRT keyword), is used to obtain time |
---|
| 144 | statistics over all the processors for each routine. |
---|
| 145 | |
---|
| 146 | Different output are written (in files named {\tt *.timers\_xxxx}) |
---|
| 147 | depending on TIMER\_Debug value : |
---|
| 148 | |
---|
| 149 | \begin{itemize} |
---|
| 150 | \item {TIMER\_Debug=0} : nothing is calculated, nothing is written. |
---|
| 151 | \item {TIMER\_Debug=1} : the times are calculated and written in a |
---|
| 152 | single file by the process 0 as well as the min and the max times |
---|
| 153 | over all the processes. |
---|
| 154 | \item {TIMER\_Debug=2} : the times are calculated and each process |
---|
| 155 | writes its own file ; process 0 also writes the min and the max |
---|
| 156 | times over all the processes in its file. |
---|
| 157 | \item {TIMER\_Debug=3} : the times are calculated and each process |
---|
| 158 | writes its own file ; process 0 also writes in its file the min |
---|
| 159 | and the max times over all processes and also writes in its file |
---|
| 160 | all the results for each process. |
---|
| 161 | \end{itemize} |
---|
| 162 | |
---|
| 163 | The time given for each timer is calculated by the difference between |
---|
| 164 | calls to {\tt oasis\_timer\_start()} and {\tt oasis\_timer\_stop()} |
---|
| 165 | and is the accumulated time over the entire run. Here is an overview |
---|
| 166 | of the meaning of the different timers as implemented by default. |
---|
| 167 | |
---|
| 168 | \begin{itemize} |
---|
| 169 | \item {'total after init'} : total time of the simulation, implemented |
---|
| 170 | in {\tt mod\_oasis\_method}, i.e. between the end of {\tt |
---|
| 171 | oasis\_init\_comp} and the call to {\tt mpi\_barrier} and {\tt |
---|
| 172 | mpi\_finalize} in routine {\tt oasis\_terminate}. |
---|
| 173 | \item {'map definition'} : time spent in {\tt mct\_gsmap\_init} in |
---|
| 174 | routine {\tt oasis\_def\_partition}; this routine defines the |
---|
| 175 | patterns of communication between the source and target processes. |
---|
| 176 | \item {'cpl\_setup'} :time spent in {\tt oasis\_coupler\_setup}, which |
---|
| 177 | sets up additional coupling aspects related to {\tt |
---|
| 178 | oasis\_method\_enddef}. |
---|
| 179 | \item {'cpl\_smatrd'} : time spent in {\tt |
---|
| 180 | oasis\_coupler\_smatreaddnc} in {\tt mod\_oasis\_coupler} (routine |
---|
| 181 | {\tt oasis\_coupler\_setup}); this routine performs a distributed |
---|
| 182 | read of a NetCDF SCRIP file and returns weights in a distributed |
---|
| 183 | SparseMatrix, and calls {\tt mct\_sMatP\_Init} which initialises the |
---|
| 184 | sparse matrix vector multiplication. |
---|
| 185 | \item {'oasis\_advance\_init()'} : time spent in {\tt |
---|
| 186 | oasis\_advance\_init}, which handles reading of initial coupling |
---|
| 187 | restart and communication of data for fields with positive lags. |
---|
| 188 | \item {'grcv\_00x'} : time spent in the reception of field x in {\tt |
---|
| 189 | mct\_recv} (including communication and possible waiting time |
---|
| 190 | linked to unbalance of components). |
---|
| 191 | \item {'wout\_00x'} : time spent in the I/O for field x in routine |
---|
| 192 | {\tt oasis\_advance\_run}. |
---|
| 193 | \item {'gcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
| 194 | in copying the field x just received in a local array. |
---|
| 195 | \item {'pcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
| 196 | in copying the local field x in the array to send (i.e. with local |
---|
| 197 | transformation besides division for averaging). |
---|
| 198 | \item {'pavg\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
| 199 | to calculate the average of field x (if done). |
---|
| 200 | \item {'pmap\_00x'/'gmap\_00x'} : time spent in routine {\tt |
---|
| 201 | oasis\_advance\_run} for the matrix vector multiplication for |
---|
| 202 | field x on the source/target processes. |
---|
| 203 | \item {'psnd\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
| 204 | for sending field x (i.e. including call to {\tt mct\_waitsend} and |
---|
| 205 | {\tt mct\_isend}). |
---|
| 206 | \end{itemize} |
---|
| 207 | |
---|