1 | \newpage |
---|
2 | % |
---|
3 | |
---|
4 | \chapter{Compiling, running and debugging} |
---|
5 | \label{sec_compilationrunning} |
---|
6 | |
---|
7 | \section{Compiling OASIS3-MCT} |
---|
8 | \label{subsec_compile} |
---|
9 | |
---|
10 | Compiling OASIS3-MCT libraries can be done in directory {\tt |
---|
11 | oasis3-MCT/util/make\_dir} with Makefile \\ {\tt TopMakefileOasis3} |
---|
12 | which must be completed with a header file {\tt make.{\it |
---|
13 | your\_platform}} specific to the compiling platform used and |
---|
14 | specified in {\tt oasis3-MCT/util/make\_dir/make.inc}. One of the |
---|
15 | header files distributed with the release can by used as a template. |
---|
16 | The root of the OASIS3-MCT tree can be anywhere and must be set in the |
---|
17 | variable {\tt COUPLE} in the {\tt make.{\it your\_platform}} file. |
---|
18 | |
---|
19 | The coupled models must be compiled with the same library as the one |
---|
20 | used to compile OASIS3-MCT. |
---|
21 | |
---|
22 | The following commands are available: |
---|
23 | |
---|
24 | \begin{itemize} |
---|
25 | \item {\tt make -f TopMakefileOasis3} or {\tt make oasis3\_psmile -f |
---|
26 | TopMakefileOasis3} (for backward compatibility): |
---|
27 | |
---|
28 | compiles all OASIS3-MCT libraries {\it mct}, {\it scrip} and {\it |
---|
29 | psmile}; |
---|
30 | |
---|
31 | \item {\tt make realclean -f TopMakefileOasis3}: |
---|
32 | |
---|
33 | removes all OASIS3-MCT compiled sources and librairies. |
---|
34 | |
---|
35 | \end{itemize} |
---|
36 | |
---|
37 | Log and error messages from compilation are saved in {\tt /util/make\_dir} in the files |
---|
38 | {\tt COMP.log} and {\tt COMP.err} . |
---|
39 | |
---|
40 | During compilation, a new compiling directory, defined by variable |
---|
41 | {\tt ARCHDIR} is created. After successful compilation, resulting |
---|
42 | libraries are found in the compiling directory in {\tt /lib} and |
---|
43 | object and module files in {\tt /build}. |
---|
44 | |
---|
45 | If module {\tt mod\_oasis} is used in the |
---|
46 | models, it is enough to include the path of the psmile objects and |
---|
47 | modules ({\tt \$ARCHDIR/build/lib/psmile.MPI1}) for the compilation and to use the psmile library {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} when |
---|
48 | linking. If module {\tt mod\_prism} is used in the models, it is necessary to |
---|
49 | include the path of the psmile and of the mct objects and modules for |
---|
50 | the compilation (i.e. {\tt \$ARCHDIR/build/lib/psmile.MPI1} and {\tt /mct} and to use both the psmile and mct libraries {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} and {\tt libmct.a} and {\tt libmpeu.a} when linking. |
---|
51 | |
---|
52 | Exchange of coupling fields in single and double precision is now supported directly through the interface |
---|
53 | (see section \ref{subsubsec_Declaration}), even if all real variables are now treated in double precision internally. |
---|
54 | For double precision coupling field, there is no need anymore to promote REAL variables to double precision at compilation. |
---|
55 | |
---|
56 | \section{CPP keys} |
---|
57 | \label{subsec_cpp} |
---|
58 | |
---|
59 | The following CPP keys are coded in OASIS3-MCT and can be specified in |
---|
60 | {\tt CPPDEF} in {\tt make.{\it your\_platform}} file. |
---|
61 | |
---|
62 | \begin{itemize} |
---|
63 | |
---|
64 | \item {\tt TREAT\_OVERLAY}: To ensure, in {\tt SCRIPR/CONSERV} |
---|
65 | remapping (see section \ref{subsec_interp}), that if two cells of |
---|
66 | the source grid overlay, at least the one with the greater numerical |
---|
67 | index is masked (they also can be both masked); this is mandatory |
---|
68 | for this remapping. For example, if the grid line with {\it i=1} overlaps |
---|
69 | the grid line with {\it i=imax}, it is the latter that must be masked; |
---|
70 | when this is not the case with the mask defined in {\it masks.nc}, |
---|
71 | this CPP key forces these rules to be respected. |
---|
72 | |
---|
73 | \item {\tt balance}: Add a MPI\_Wtime() function before and after |
---|
74 | mct\_isend (MPI put) and mct\_recv (MPI get) to calculate the time |
---|
75 | of the send and receive of a coupling field. This option can be used |
---|
76 | to produce timestamps in OASIS3-MCT debug files. During a post-processing |
---|
77 | phase, this information can be used to perform an analysis of the |
---|
78 | coupled components load (un)balance; specific tools have been |
---|
79 | developed to do this and will be released with a further version of |
---|
80 | OASIS3-MCT. {\bf This option is temporarily not recommended as it was observed that |
---|
81 | it was increasing the simulation time of coupled models on |
---|
82 | the PRACE computer MareNostrum.} |
---|
83 | |
---|
84 | \end{itemize} |
---|
85 | |
---|
86 | \section{Running OASIS3-MCT} |
---|
87 | \label{subsec_running} |
---|
88 | |
---|
89 | Examples of running environement are provided with the sources. For |
---|
90 | more details, see the instructions on OASIS web site at |
---|
91 | {\tt https://verc.enes.org/oasis/oasis-dedicated-user-support-1/} {\tt user-toys/tutorial-and-test\_interpolation-of-oasis3-mct-1}. |
---|
92 | . |
---|
93 | |
---|
94 | \subsection{The tutorial toy model} |
---|
95 | \label{subsec_tutorial} |
---|
96 | |
---|
97 | A practical example on how to run OASIS3-MCT and running it in a |
---|
98 | coupled system is provided in {\tt oasis3-mct/examples/tutorial}. See also the document {\tt tutorial\_oasis3-mct.pdf} there in for |
---|
99 | more details, |
---|
100 | |
---|
101 | \subsection{The test\_interpolation environment} |
---|
102 | \label{subsec_testinterpolation} |
---|
103 | |
---|
104 | This environment available with the sources in {\tt |
---|
105 | oasis3-mct/examples/test\_interpolation} allows the user to test the |
---|
106 | quality of the interpolations and transformations between his source |
---|
107 | and target grids by calculating the error of interpolation on the |
---|
108 | target grid. For more details, see also the document {\tt test\_interpolation\_oasis3-mct.pdf} there in. |
---|
109 | |
---|
110 | \section{Debugging} |
---|
111 | \label{subsec_debug} |
---|
112 | |
---|
113 | \subsection{Debug files} |
---|
114 | If you experience problems while running your coupled model with |
---|
115 | OASIS3-MCT, you can obtain more information on what is happening by |
---|
116 | increasing the {\tt \$NLOGPRT} value in your {\it namcouple} (see also section |
---|
117 | \ref{subsec_namcouplefirst}). |
---|
118 | |
---|
119 | Different outputs are written depending on {\tt \$NLOGPRT} value: |
---|
120 | \begin{itemize} |
---|
121 | \item {0} : one file debug.root.xx is open by the master process of |
---|
122 | each model and one file debug\_notroot.xx is open for all the other |
---|
123 | processes of each model to write only error information if an error |
---|
124 | occurs. |
---|
125 | \item {1} : one file debug.root.xx is open by the master process of |
---|
126 | each model to write information equivalent to level 10 (see below); |
---|
127 | one file debug\_notroot.xx is open for all the other processes of |
---|
128 | each model to write only error information if an error occurs. |
---|
129 | \item {2} : one file debug.xx.xxxxxx is open by each process of each |
---|
130 | model to write normal production diagnostics. |
---|
131 | \item {5} : as for 2 with in addition some initial debug info. |
---|
132 | \item {10} : as for 5 with in addition the routine calling tree. |
---|
133 | \item {12} : as for 10 with in addition some routine calling notes. |
---|
134 | \item {15} : as for 12 with even more debug diagnostics. |
---|
135 | \item {20} : as for 15 with in addition some extra runtime analysis. |
---|
136 | \item {30} : full debug information. |
---|
137 | \end{itemize} |
---|
138 | |
---|
139 | \subsection{Time statistics files} |
---|
140 | \label{timestat} |
---|
141 | |
---|
142 | The variable TIMER\_Debug, defined in the {\it namcouple} (second |
---|
143 | number on the line below \$NLOGPRT keyword), is used to obtain time |
---|
144 | statistics over all the processors for each routine. |
---|
145 | |
---|
146 | Different output are written (in files named {\tt *.timers\_xxxx}) |
---|
147 | depending on TIMER\_Debug value : |
---|
148 | |
---|
149 | \begin{itemize} |
---|
150 | \item {TIMER\_Debug=0} : nothing is calculated, nothing is written. |
---|
151 | \item {TIMER\_Debug=1} : the times are calculated and written in a |
---|
152 | single file by the process 0 as well as the min and the max times |
---|
153 | over all the processes. |
---|
154 | \item {TIMER\_Debug=2} : the times are calculated and each process |
---|
155 | writes its own file ; process 0 also writes the min and the max |
---|
156 | times over all the processes in its file. |
---|
157 | \item {TIMER\_Debug=3} : the times are calculated and each process |
---|
158 | writes its own file ; process 0 also writes in its file the min |
---|
159 | and the max times over all processes and also writes in its file |
---|
160 | all the results for each process. |
---|
161 | \end{itemize} |
---|
162 | |
---|
163 | The time given for each timer is calculated by the difference between |
---|
164 | calls to {\tt oasis\_timer\_start()} and {\tt oasis\_timer\_stop()} |
---|
165 | and is the accumulated time over the entire run. Here is an overview |
---|
166 | of the meaning of the different timers as implemented by default. |
---|
167 | |
---|
168 | \begin{itemize} |
---|
169 | \item {'total after init'} : total time of the simulation, implemented |
---|
170 | in {\tt mod\_oasis\_method}, i.e. between the end of {\tt |
---|
171 | oasis\_init\_comp} and the call to {\tt mpi\_barrier} and {\tt |
---|
172 | mpi\_finalize} in routine {\tt oasis\_terminate}. |
---|
173 | \item {'map definition'} : time spent in {\tt mct\_gsmap\_init} in |
---|
174 | routine {\tt oasis\_def\_partition}; this routine defines the |
---|
175 | patterns of communication between the source and target processes. |
---|
176 | \item {'cpl\_setup'} :time spent in {\tt oasis\_coupler\_setup}, which |
---|
177 | sets up additional coupling aspects related to {\tt |
---|
178 | oasis\_method\_enddef}. |
---|
179 | \item {'cpl\_smatrd'} : time spent in {\tt |
---|
180 | oasis\_coupler\_smatreaddnc} in {\tt mod\_oasis\_coupler} (routine |
---|
181 | {\tt oasis\_coupler\_setup}); this routine performs a distributed |
---|
182 | read of a NetCDF SCRIP file and returns weights in a distributed |
---|
183 | SparseMatrix, and calls {\tt mct\_sMatP\_Init} which initialises the |
---|
184 | sparse matrix vector multiplication. |
---|
185 | \item {'oasis\_advance\_init()'} : time spent in {\tt |
---|
186 | oasis\_advance\_init}, which handles reading of initial coupling |
---|
187 | restart and communication of data for fields with positive lags. |
---|
188 | \item {'grcv\_00x'} : time spent in the reception of field x in {\tt |
---|
189 | mct\_recv} (including communication and possible waiting time |
---|
190 | linked to unbalance of components). |
---|
191 | \item {'wout\_00x'} : time spent in the I/O for field x in routine |
---|
192 | {\tt oasis\_advance\_run}. |
---|
193 | \item {'gcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
194 | in copying the field x just received in a local array. |
---|
195 | \item {'pcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
196 | in copying the local field x in the array to send (i.e. with local |
---|
197 | transformation besides division for averaging). |
---|
198 | \item {'pavg\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
199 | to calculate the average of field x (if done). |
---|
200 | \item {'pmap\_00x'/'gmap\_00x'} : time spent in routine {\tt |
---|
201 | oasis\_advance\_run} for the matrix vector multiplication for |
---|
202 | field x on the source/target processes. |
---|
203 | \item {'psnd\_00x'} : time spent in routine {\tt oasis\_advance\_run} |
---|
204 | for sending field x (i.e. including call to {\tt mct\_waitsend} and |
---|
205 | {\tt mct\_isend}). |
---|
206 | \end{itemize} |
---|
207 | |
---|