1 | \newpage |
---|
2 | \chapter{Interfacing a component model with OASIS3-MCT} |
---|
3 | \label{sec_modelinterfacing} |
---|
4 | |
---|
5 | At run-time, OASIS3-MCT allows exchanging coupling data between two |
---|
6 | components as well as interpolation and transformation of these |
---|
7 | coupling fields. Different communication techniques have been |
---|
8 | historically developed in OASIS. With OASIS3-MCT, communication is |
---|
9 | performed by MCT based on Message Passing Interface 1 (MPI1) protocol. |
---|
10 | |
---|
11 | For a practical example using the OASIS3-MCT library, see the sources |
---|
12 | in {\tt examples/tutorial} and explanations in the {\tt |
---|
13 | test\_interpolation\_oasis3-mct.pdf} document therein. |
---|
14 | |
---|
15 | To communicate with another component model or to perform I/O actions, |
---|
16 | a component model needs to be interfaced with the OASIS3-MCT library, |
---|
17 | which sources can be found in {\tt oasis3-mct/lib/psmile} |
---|
18 | directory. The OASIS3-MCT library supports: |
---|
19 | |
---|
20 | \begin{itemize} |
---|
21 | \item parallel communication between parallel component models, |
---|
22 | \item coupling a component on a subset of it's processeses only, |
---|
23 | \item automatic sending and receiving actions at appropriate times |
---|
24 | following user's choice indicated in the {\it namcouple}, |
---|
25 | \item time integration or accumulation of the coupling fields, |
---|
26 | \item some transformations such as mapping (interpolation) between |
---|
27 | grids, |
---|
28 | \item I/O actions from/to files. |
---|
29 | \end{itemize} |
---|
30 | |
---|
31 | To adapt a component model to OASIS3-MCT, specific calls of the |
---|
32 | following classes have to be implemented in the code: |
---|
33 | |
---|
34 | \begin{enumerate} |
---|
35 | \item Initialisation (section \ref{subsubsec_Initialisation}) |
---|
36 | \item Grid data file definition (section \ref{subsubsec_griddef}) |
---|
37 | \item Partition definition (section \ref{subsubsec_Partition}) |
---|
38 | \item Coupling-I/O field declaration (section |
---|
39 | \ref{subsubsec_Declaration}) |
---|
40 | \item End of definition phase (section |
---|
41 | \ref{subsubsec_Endofdefinition}) |
---|
42 | \item Coupling-I/O field sending and receiving (section |
---|
43 | \ref{subsubsec_sendingreceiving}) |
---|
44 | \item Termination (section \ref{subsubsec_Termination}) |
---|
45 | \item Optional auxiliary routines (section |
---|
46 | \ref{subsubsec_auxroutines}) |
---|
47 | \end{enumerate} |
---|
48 | |
---|
49 | Finally, in section \ref{subsubsec_Algoritms}, different coupling |
---|
50 | algorithms are illustrated and details on how to reproduce them with |
---|
51 | OASIS3-MCT are provided. More information on the {\tt LAG} and {\tt |
---|
52 | SEQ} indices are also given in that section. |
---|
53 | |
---|
54 | \section{Use of OASIS3-MCT library} |
---|
55 | \label{subsubsec_Use} |
---|
56 | % {Use} |
---|
57 | |
---|
58 | To use OASIS3-MCT library, a user needs to add in his code: |
---|
59 | |
---|
60 | \begin{itemize} |
---|
61 | |
---|
62 | \item {\tt USE mod\_oasis} |
---|
63 | |
---|
64 | ** OR ** |
---|
65 | |
---|
66 | \item {\tt USE mod\_prism} |
---|
67 | |
---|
68 | \end{itemize} |
---|
69 | |
---|
70 | Both use statements are valid and use of just one or the other is |
---|
71 | recommended in a particular component model. A single use statement |
---|
72 | now provides all the methods that required multiple use statements in |
---|
73 | previous OASIS3 versions. The methods, datatypes, and capabilities |
---|
74 | are identical for both the {\tt mod\_prism} or {\tt mod\_oasis} |
---|
75 | interfaces. The only difference is the name of the interface. The |
---|
76 | interface in module {\tt mod\_prism} is provided for backwards |
---|
77 | compatability with prior versions of OASIS3. Use of module {\tt |
---|
78 | mod\_oasis} is now recommended provides access to a set of updated |
---|
79 | routine names that will continue to evolve in the future, always |
---|
80 | ensuring backward compatibility. In the following sections, both the |
---|
81 | {\tt mod\_prism} and {\tt mod\_oasis} interface names are defined and |
---|
82 | a single description of the interface arguments is provided. |
---|
83 | |
---|
84 | \section{Initialisation} |
---|
85 | \label{subsubsec_Initialisation} |
---|
86 | % {Initialisation} |
---|
87 | |
---|
88 | \subsection{Coupling initialisation} |
---|
89 | |
---|
90 | \begin{itemize} |
---|
91 | |
---|
92 | \item {\tt CALL oasis\_init\_comp (compid, model\_name, ierror)} |
---|
93 | \item {\tt CALL prism\_init\_comp\_proto (compid, model\_name, |
---|
94 | ierror)} |
---|
95 | |
---|
96 | \begin{itemize} |
---|
97 | \item {\tt compid [INTEGER; OUT]}: component model ID |
---|
98 | \item {\tt model\_name [CHARACTER*6; IN]}: component model name (as |
---|
99 | in {\em namcouple} under {\tt \$NBMODEL}) |
---|
100 | \item {\tt ierror [INTEGER; OUT]}: returned error code. |
---|
101 | \end{itemize} |
---|
102 | |
---|
103 | This routine must called by all component processes to initialise |
---|
104 | the coupling.\footnote{The model may call MPI\_Init explicitly, but |
---|
105 | if so, has to call it before calling {\tt oasis\_init\_comp}; in |
---|
106 | this case, the model also has to call MPI\_Finalize explicitly, |
---|
107 | but only after calling {\tt oasis\_terminate}.} |
---|
108 | \end{itemize} |
---|
109 | |
---|
110 | \subsection{Communicator for internal parallelisation} |
---|
111 | \label{subsec_MPI1} |
---|
112 | |
---|
113 | \begin{itemize} |
---|
114 | \item {\tt CALL oasis\_get\_localcomm (local\_comm, ierror )} |
---|
115 | \item {\tt CALL prism\_get\_localcomm\_proto (local\_comm, ierror )} |
---|
116 | |
---|
117 | \begin{itemize} |
---|
118 | \item {\tt local\_comm [INTEGER; OUT]}: value of local communicator |
---|
119 | \item {\tt ierror [INTEGER; OUT]}: returned error code. |
---|
120 | \end{itemize} |
---|
121 | |
---|
122 | If needed, this routine may be called by the component processes to |
---|
123 | get the value of a local communicator to be used by the component |
---|
124 | for its internal parallelisation. |
---|
125 | |
---|
126 | This may be needed as all component models started in a pseudo-MPMD |
---|
127 | mode with MPI1 share automatically the same MPI\_COMM\_WORLD |
---|
128 | communicator. Another communicator has to be used for the internal |
---|
129 | parallelisation of each component. OASIS3-MCT creates this local |
---|
130 | communicator based on the name given to {\tt oasis\_init\_comp} |
---|
131 | routine; its value is returned as the first argument of the routine, |
---|
132 | {\tt local\_comm}. |
---|
133 | |
---|
134 | % With CLIM-MPI2, OASIS3 executable spawns the component model |
---|
135 | % executables at the beginning of the run; the components keep their |
---|
136 | % internal parallelisation context unchanged with respect to their |
---|
137 | % standalone mode. In this case, calling the |
---|
138 | % prism\_get\_localcomm\_proto routine is useless but if called, the |
---|
139 | % communicator MPI\_COMM\_WORLD will be returned as local |
---|
140 | % communicator. |
---|
141 | \end{itemize} |
---|
142 | |
---|
143 | \subsection{Coupling through a subset of the component model |
---|
144 | processes} |
---|
145 | \label{subsec_subset} |
---|
146 | |
---|
147 | If only a subset of the component processes participate in the |
---|
148 | coupling, a communicator gathering only these processes must be |
---|
149 | defined, with {\tt oasis/prism\_create\_couplcomm} or \newline {\tt |
---|
150 | oasis/prism\_set\_couplcomm}. These routines should be called after |
---|
151 | the {\tt oasis\_init\_comp} but before the grid, partition, or |
---|
152 | coupling field declaration. |
---|
153 | |
---|
154 | If such communicator does not exist in the code, the component |
---|
155 | processes have to create it with: |
---|
156 | |
---|
157 | \begin{itemize} |
---|
158 | \item {\tt CALL oasis\_create\_couplcomm(icpl, local\_comm, |
---|
159 | coupl\_comm, kinfo)} |
---|
160 | \item {\tt CALL prism\_create\_couplcomm(icpl, local\_comm, |
---|
161 | coupl\_comm, kinfo)} |
---|
162 | \begin{itemize} |
---|
163 | \item {\tt icpl [INTEGER; IN]}: coupling process flag |
---|
164 | \item {\tt local\_comm [INTEGER; IN]}: MPI communicator with all |
---|
165 | processes of the component |
---|
166 | \item {\tt coupl\_comm [INTEGER; OUT]}: returned MPI communicator |
---|
167 | gathering only component processes participating in the coupling |
---|
168 | \item {\tt kinfo [INTEGER; OUT; OPTIONAL]}: returned error code |
---|
169 | \end{itemize} |
---|
170 | |
---|
171 | This routine creates a coupling communicator for a subset of |
---|
172 | processes. It must be called by all component processes with {\tt |
---|
173 | icpl=1} for processes participating in the coupling and with {\tt |
---|
174 | icpl =} {\tt MPI\_UNDEFINED} for the others. Argument {\tt |
---|
175 | local\_comm} is the MPI communicator associated with all processes |
---|
176 | of the component. The new coupling communicator is returned in {\tt |
---|
177 | coupl\_comm}. |
---|
178 | \end{itemize} |
---|
179 | |
---|
180 | If this communicator already exist in the code, the component should |
---|
181 | simply provide it to OASIS3-MCT with: |
---|
182 | |
---|
183 | \begin{itemize} |
---|
184 | \item {\tt CALL oasis\_set\_couplcomm(coupl\_comm, kinfo)} |
---|
185 | \item {\tt CALL prism\_set\_couplcomm(coupl\_comm, kinfo)} |
---|
186 | \begin{itemize} |
---|
187 | \item {\tt coupl\_comm [INTEGER; IN]}: MPI communicator gathering |
---|
188 | only component processes participating in the coupling |
---|
189 | \item {\tt kinfo [INTEGER; OUT; OPTIONAL]}: returned error code |
---|
190 | \end{itemize} |
---|
191 | |
---|
192 | This routine allows to provide a local coupling communicator to |
---|
193 | OASIS3-MCT, given that it already exists in the code. The value of |
---|
194 | {\tt coupl\_comm} must be the value of this local coupling |
---|
195 | communicator for the processes participating to the coupling and it |
---|
196 | must be {\tt MPI\_COMM\_NULL} for processes not involved in the |
---|
197 | coupling. |
---|
198 | \end{itemize} |
---|
199 | |
---|
200 | |
---|
201 | All OASIS3-MCT interface routines, besides the grid definition (see |
---|
202 | section \ref{subsubsec_griddef}) and the {\tt oasis\_put} and {\tt |
---|
203 | oasis\_get} per se (see section \ref{subsubsec_sendingreceiving}), |
---|
204 | are collective and must be called by all processes of an executable. |
---|
205 | % \footnote{In fact, the {\tt oasis\_def\_var/prism\_def\_var\_proto} |
---|
206 | % must be %called by at least the root process of the coupler |
---|
207 | % communicator but can be called by all processes.}. |
---|
208 | In particular, the {\tt oasis\_def\_partition} (see section |
---|
209 | \ref{subsubsec_Partition}) must be called by all processes, but with |
---|
210 | {\tt ig\_paral(:)=0} for the processes not involved in the coupling; |
---|
211 | also, these processes must call the same number of {\tt |
---|
212 | oasis\_def\_var} than the ones participating to the coupling, even |
---|
213 | if the associated field symbolic names and partition ID (see {\tt |
---|
214 | name} and {\tt il\_part\_id} arguments in section |
---|
215 | \ref{subsubsec_Declaration}) can then be anything. |
---|
216 | |
---|
217 | |
---|
218 | \subsection{Separate executable not coupling at all} |
---|
219 | \label{subsec_nocpl} |
---|
220 | |
---|
221 | For a separate executable part of a coupled system managed by |
---|
222 | OASIS3-MCT but for which all processes are not involved in the |
---|
223 | coupling (such as a separate I/O server), the only calls that should |
---|
224 | be made are : |
---|
225 | |
---|
226 | \begin{verbatim} |
---|
227 | CALL oasis_init_comp (comp_id, comp_name, ierror ) |
---|
228 | CALL oasis_get_localcomm ( localComm, ierror ) ! optional |
---|
229 | CALL oasis_enddef ( ... ) |
---|
230 | CALL oasis_terminate ( ... ) |
---|
231 | \end{verbatim} |
---|
232 | |
---|
233 | \section{Grid data file definition} |
---|
234 | \label{subsubsec_griddef} |
---|
235 | |
---|
236 | With OASIS3-MCT, the grid data files {\em grids.nc, masks.nc} and {\em |
---|
237 | areas.nc} are required only for certain operations (see also section |
---|
238 | \ref{subsec_griddata}), i.e. {\em grids.nc}, and {\em masks.nc} for |
---|
239 | {\tt SCRIPR} (see section \ref{subsec_interp}) and {\em masks.nc} and |
---|
240 | {\em areas.nc} for {\tt CONSERV} (see section |
---|
241 | \ref{subsec_cooking}). These grid data files can be created by the |
---|
242 | user before the run or can be written directly at run time by the {\bf |
---|
243 | master process of each component model} with the following routines. |
---|
244 | These routines can be used by the component models to {\bf add grid} |
---|
245 | fields to the grid files but {\bf not to overwrite} them . These |
---|
246 | routines have to be called just after {\tt oasis\_init\_comp}. |
---|
247 | |
---|
248 | \begin{itemize} |
---|
249 | |
---|
250 | \item {\tt CALL oasis\_start\_grids\_writing (flag)} or |
---|
251 | \item {\tt CALL prism\_start\_grids\_writing (flag)} |
---|
252 | \begin{itemize} |
---|
253 | \item {\tt flag [INTEGER; OUT]}: not used |
---|
254 | \end{itemize} |
---|
255 | Obsolete in OASIS3-MCT; exists however for upward compatibility. |
---|
256 | |
---|
257 | \vspace{0.2cm} |
---|
258 | \item {\tt CALL oasis\_write\_grid (cgrid, nx, ny, lon, lat)} or |
---|
259 | \item {\tt CALL prism\_write\_grid (cgrid, nx, ny, lon, lat)} |
---|
260 | |
---|
261 | \begin{itemize} |
---|
262 | \item {\tt cgrid [CHARACTER*4; IN]}: grid name prefix (see |
---|
263 | \ref{subsec_namcouplesecond}) |
---|
264 | \item {\tt nx [INTEGER; IN]} : first grid dimension (x) |
---|
265 | \item {\tt ny [INTEGER; IN]} : second grid dimension (y) |
---|
266 | \item {\tt lon [REAL, DIMENSION(nx,ny); IN)} : single or double real array of longitudes |
---|
267 | (degrees East) |
---|
268 | \item {\tt lat [REAL, DIMENSION(nx,ny); IN)} : single or double real array of latitudes |
---|
269 | (degrees North) |
---|
270 | \end{itemize} |
---|
271 | |
---|
272 | Writes the model grid longitudes and latitudes. Longitudes must be |
---|
273 | given in degrees East in the interval -360.0 to 720.0. Latitudes |
---|
274 | must be given in degrees North in the interval -90.0 to 90.0. Note |
---|
275 | that if some grid points overlap, it is recommended to define those |
---|
276 | points with the same number (e.g. 90.0 for both, not 450.0 for one |
---|
277 | and 90.0 for the other) to ensure automatic detection of overlap by |
---|
278 | OASIS3-MCT (which is essential to have a correct conservative |
---|
279 | remapping \texttt{SCRIPR/CONSERV}, see section \ref{subsec_interp}). |
---|
280 | |
---|
281 | \vspace{0.2cm} |
---|
282 | \item {\tt CALL oasis\_write\_corner (cgrid, nx, ny, nc, clon, clat)} |
---|
283 | or |
---|
284 | \item {\tt CALL prism\_write\_corner (cgrid, nx, ny, nc, clon, clat)} |
---|
285 | |
---|
286 | \begin{itemize} |
---|
287 | \item {\tt cgrid [CHARACTER*4; IN]}: grid name prefix |
---|
288 | \item {\tt nx [INTEGER; IN]} : first grid dimension (x) |
---|
289 | \item {\tt ny [INTEGER; IN]} : second grid dimension (y) |
---|
290 | \item {\tt nc [INTEGER; IN]} : number of corners per grid cell |
---|
291 | (always 4 in the version) |
---|
292 | \item {\tt lon [REAL, DIMENSION (nx,ny,nc);IN]} : single or double real array of corner |
---|
293 | longitudes (in degrees\_East) |
---|
294 | \item {\tt lat [REAL, DIMENSION (nx,ny,nc);IN]} : single or double real array of corner |
---|
295 | latitudes (in degrees\_North) |
---|
296 | \end{itemize} |
---|
297 | |
---|
298 | Writes the grid cell corner longitudes and latitudes |
---|
299 | (counterclockwise sense). Longitudes must be given in degrees East |
---|
300 | in the interval -360.0 to 720.0. Latitudes must be given in degrees |
---|
301 | North in the interval -90.0 to 90.0. Note also that cells larger |
---|
302 | than 180.0 degrees in longitude are not supported. Writing of |
---|
303 | corners is optional as corner information is needed only for {\tt |
---|
304 | SCRIPR/CONSERV} (see section \ref{subsec_interp}). If called, |
---|
305 | needs to be called after {\tt oasis/prism\_write\_grid}. |
---|
306 | |
---|
307 | \vspace{0.2cm} |
---|
308 | \item {\tt CALL oasis\_write\_mask (cgrid, nx, ny, mask)} or |
---|
309 | \item {\tt CALL prism\_write\_mask (cgrid, nx, ny, mask)} |
---|
310 | |
---|
311 | \begin{itemize} |
---|
312 | \item {\tt cgrid [CHARACTER*4; IN]}: grid name prefix |
---|
313 | \item {\tt nx [INTEGER; IN]} : first grid dimension (x) |
---|
314 | \item {\tt ny [INTEGER; IN]} : second grid dimension (y) |
---|
315 | \item {\tt mask [INTEGER, DIMENSION(nx,ny) ;IN]} : mask array (be |
---|
316 | careful about the OASIS historical convention (!): 0 = not masked, |
---|
317 | 1 = masked) |
---|
318 | \end{itemize} |
---|
319 | Writes the model grid mask. |
---|
320 | |
---|
321 | \vspace{0.2cm} |
---|
322 | \item {\tt CALL oasis\_write\_area (cgrid, nx, ny, area)} or |
---|
323 | \item {\tt CALL prism\_write\_area (cgrid, nx, ny, area)} |
---|
324 | |
---|
325 | \begin{itemize} |
---|
326 | \item {\tt cgrid [CHARACTER*4; IN]}: grid name prefix |
---|
327 | \item {\tt nx [INTEGER; IN]} : first grid dimension (x) |
---|
328 | \item {\tt ny [INTEGER; IN]} : second grid dimension (y) |
---|
329 | \item {\tt area [REAL, DIMENSION(nx,ny); IN]} : single or double real array of grid cell |
---|
330 | areas |
---|
331 | \end{itemize} |
---|
332 | Writes of the model grid cell areas. Needed only for {\tt CONSERV} |
---|
333 | operation (see section \ref{subsec_cooking}). |
---|
334 | |
---|
335 | \vspace{0.2cm} |
---|
336 | \item {\tt CALL prism\_terminate\_grids\_writing()} |
---|
337 | \item {\tt CALL oasis\_terminate\_grids\_writing()} |
---|
338 | |
---|
339 | Terminates grids writing (required if some of the above grid writing |
---|
340 | routines are called). |
---|
341 | |
---|
342 | \end{itemize} |
---|
343 | |
---|
344 | The creation of the different files is effective in the routine |
---|
345 | oasis\_enddef. |
---|
346 | |
---|
347 | \section{Partition definition} |
---|
348 | \label{subsubsec_Partition} |
---|
349 | |
---|
350 | |
---|
351 | The coupling fields sent or received by a component model are usually |
---|
352 | scattered among the different component processes. With OASIS3-MCT, |
---|
353 | all processes exchanging coupling data have to define their local |
---|
354 | partition in the global index space. |
---|
355 | |
---|
356 | If only a subset of the processes actually exchange coupling data, the |
---|
357 | processes not implied in the coupling have to call this routine to |
---|
358 | describe a null partition (i.e. with {\tt ig\_paral(:)=0}). |
---|
359 | |
---|
360 | If the sum of the partitions defined over all the processes do not |
---|
361 | cover the global grid, the total dimension of the grid (total number |
---|
362 | of points) must be given as an optional argument (see {\tt isize} |
---|
363 | below). |
---|
364 | |
---|
365 | \begin{itemize} |
---|
366 | |
---|
367 | \vspace{0.2cm} |
---|
368 | \item {\tt CALL oasis\_def\_partition (il\_part\_id, ig\_paral, |
---|
369 | ierror, isize)} or |
---|
370 | \item {\tt CALL prism\_def\_partition\_proto (il\_part\_id, ig\_paral, |
---|
371 | ierror, isize)} |
---|
372 | |
---|
373 | \begin{itemize} |
---|
374 | \item {\tt il\_part\_id [INTEGER; OUT]}: partition ID |
---|
375 | \item {\tt ig\_paral [INTEGER, DIMENSION(:), IN]}: vector of |
---|
376 | integers describing the local partition in the global index space; |
---|
377 | has a different expression depending on the type of the partition; |
---|
378 | in OASIS3-MCT, 5 types of partition are supported: Serial (no |
---|
379 | partition), Apple, Box, Orange, and Points (see below). |
---|
380 | \item {\tt ierror [INTEGER; OUT]}: returned error code. |
---|
381 | \item {\tt isize [INTEGER, OPTIONAL, IN]}: Optional argument needed |
---|
382 | if the coupling data is exchanged for only a subdomain of the |
---|
383 | global grid; in this case, isize must give the total dimension of |
---|
384 | the grid (total number of points). |
---|
385 | \end{itemize} |
---|
386 | \end{itemize} |
---|
387 | |
---|
388 | \subsection{Serial (no partition)} |
---|
389 | |
---|
390 | This is the choice for a monoprocess model. In this case, we have {\tt |
---|
391 | ig\_paral(1:3)}: |
---|
392 | \begin{itemize} |
---|
393 | \item {\tt ig\_paral(1)} = 0 (indicates a Serial ``partition'') |
---|
394 | \item {\tt ig\_paral(2)} = 0 |
---|
395 | \item {\tt ig\_paral(3)} = the total grid size. |
---|
396 | \end{itemize} |
---|
397 | |
---|
398 | \subsection{Apple partition} |
---|
399 | |
---|
400 | Each partition is a segment of the global domain, described by its |
---|
401 | global offset and its local size. In this case, we have {\tt |
---|
402 | ig\_paral(1:3)}: |
---|
403 | \begin{itemize} |
---|
404 | \item {\tt ig\_paral(1)} = 1 (indicates an Apple partition) |
---|
405 | \item {\tt ig\_paral(2)} = the segment global offset |
---|
406 | \item {\tt ig\_paral(3)} = the segment local size |
---|
407 | \end{itemize} |
---|
408 | |
---|
409 | Figure \ref{apple_partition} illustrates an Apple partition over 3 |
---|
410 | processes. |
---|
411 | \begin{figure} |
---|
412 | \includegraphics[scale=.6]{figures/apple_new} |
---|
413 | \caption{Apple partition. It is assumed here that the index start at |
---|
414 | 0 in the upper left corner.} |
---|
415 | \label{apple_partition} |
---|
416 | \end{figure} |
---|
417 | |
---|
418 | |
---|
419 | \subsection{Box partition} |
---|
420 | |
---|
421 | Each partition is a rectangular region of the global domain, described |
---|
422 | by the global offset of its upper left corner, and its local extents |
---|
423 | in the X and Y dimensions. The global extent in the X dimension must |
---|
424 | also be given. In this case, we have {\tt ig\_paral(1:5)}: |
---|
425 | \begin{itemize} |
---|
426 | \item {\tt ig\_paral(1)} = 2 (indicates a Box partition) |
---|
427 | \item {\tt ig\_paral(2)} = the upper left corner global offset |
---|
428 | \item {\tt ig\_paral(3)} = the local extent in x |
---|
429 | \item {\tt ig\_paral(4)} = the local extent in y |
---|
430 | % \footnote{The maximum value of the local extent in y is presently |
---|
431 | % 338; it can be increased by modifying the value of {\tt |
---|
432 | % Clim\_MaxSegments} in {\tt oasis3/lib/clim/src/mod\_clim.F90} |
---|
433 | % and in {\tt oasis3/lib/psmile/src/mod\_prism\_proto.F90} and by |
---|
434 | % recompiling OASIS3 and the PSMILe library.} |
---|
435 | \item {\tt ig\_paral(5)} = the global extent in x. |
---|
436 | \end{itemize} |
---|
437 | |
---|
438 | Figure \ref{box_partition} illustrates a Box partition over 3 |
---|
439 | processes. |
---|
440 | |
---|
441 | \begin{figure} |
---|
442 | \includegraphics[scale=.6]{figures/box_new} |
---|
443 | \caption{Box partition. It is assumed here that the index start at 0 |
---|
444 | in the upper left corner.} |
---|
445 | \label{box_partition} |
---|
446 | \end{figure} |
---|
447 | |
---|
448 | \subsection{Orange partition} |
---|
449 | |
---|
450 | Each partition is an ensemble of segments of the global domain. Each |
---|
451 | segment is described by its global offset and its local extent. In |
---|
452 | this case, we have {\tt ig\_paral(1:N)} where {\tt N = 2 + 2*number of |
---|
453 | segments} |
---|
454 | % \footnote{As for the Box partition, the maximum number of segments |
---|
455 | % is presently 338; it can be increased by modifying the value of |
---|
456 | % {\tt Clim\_MaxSegments}}. |
---|
457 | |
---|
458 | \begin{itemize} |
---|
459 | \item {\tt ig\_paral(1)} = 3 (indicates a Orange partition) |
---|
460 | \item {\tt ig\_paral(2)} = the total number of segments for the |
---|
461 | partition (limited to 200 presently, see note for ig\_paral(4) for |
---|
462 | Box partition above) |
---|
463 | \item {\tt ig\_paral(3)} = the first segment global offset |
---|
464 | \item {\tt ig\_paral(4)} = the first segment local extent |
---|
465 | \item {\tt ig\_paral(5)} = the second segment global offset |
---|
466 | \item {\tt ig\_paral(6)} = the second segment local extent |
---|
467 | \item ... |
---|
468 | \item {\tt ig\_paral(N-1)} = the last segment global offset |
---|
469 | \item {\tt ig\_paral(N)} = the last segment local extent |
---|
470 | \end{itemize} |
---|
471 | |
---|
472 | Figure \ref{orange_partition} illustrates an Orange partition with 3 |
---|
473 | segments for one process. The other process partitions are not |
---|
474 | illustrated. |
---|
475 | |
---|
476 | \begin{figure} |
---|
477 | \includegraphics[scale=.6]{figures/orange_new} |
---|
478 | \caption{Orange partition for one process. It is assumed here that |
---|
479 | the index start at 0 in the upper left corner.} |
---|
480 | \label{orange_partition} |
---|
481 | \end{figure} |
---|
482 | |
---|
483 | \subsection{Points partition} |
---|
484 | |
---|
485 | This partition is a list of global indices associated with each |
---|
486 | processor. The index naming |
---|
487 | is arbitrary but must be consistent between all processors involved |
---|
488 | in the partition description. In |
---|
489 | this case, we have {\tt ig\_paral(1:N)} where {\tt N = 2 + number of |
---|
490 | points} |
---|
491 | % \footnote{As for the Box partition, the maximum number of segments |
---|
492 | % is presently 338; it can be increased by modifying the value of |
---|
493 | % {\tt Clim\_MaxSegments}}. |
---|
494 | |
---|
495 | \begin{itemize} |
---|
496 | \item {\tt ig\_paral(1)} = 4 (indicates a Points partition) |
---|
497 | \item {\tt ig\_paral(2)} = currently not used |
---|
498 | \item {\tt ig\_paral(3)} = the first global index |
---|
499 | \item {\tt ig\_paral(4)} = the second global index |
---|
500 | \item ... |
---|
501 | \item {\tt ig\_paral(N)} = the last global index |
---|
502 | \end{itemize} |
---|
503 | |
---|
504 | % {Partition definition} |
---|
505 | |
---|
506 | \section{Coupling field declaration} |
---|
507 | \label{subsubsec_Declaration} |
---|
508 | |
---|
509 | All component processes declares the fields sent or received by the |
---|
510 | component during the simulation. |
---|
511 | |
---|
512 | If some processes of a model are not involved in the coupling, they |
---|
513 | still have to call the same number of {\tt oasis\_def\_var} than the |
---|
514 | ones participating to the coupling, even if the associated field |
---|
515 | symbolic names and partition ID (see {\tt name} and {\tt il\_part\_id} |
---|
516 | below) can then be anything. |
---|
517 | |
---|
518 | \begin{itemize} |
---|
519 | |
---|
520 | \item {\tt CALL oasis\_def\_var (var\_id, name, il\_part\_id, |
---|
521 | var\_nodims, kinout, \newline var\_actual\_shape, var\_type, |
---|
522 | ierror)} or |
---|
523 | |
---|
524 | \item {\tt CALL prism\_def\_var\_proto(var\_id, name, il\_part\_id, |
---|
525 | var\_nodims, kinout, var\_actual\_shape, var\_type, ierror)} |
---|
526 | |
---|
527 | \begin{itemize} |
---|
528 | \item {\tt var\_id [INTEGER; OUT]}: coupling field ID\footnote{If |
---|
529 | the field does not appear in the {\it namcouple}, the returned |
---|
530 | field ID will be equal to -1 and corresponding {\it oasis\_put} |
---|
531 | and {\it oasis\_get} cannot be called in the code}. |
---|
532 | \item {\tt name [CHARACTER*128; IN]}: field symbolic name (as in the |
---|
533 | {\it namcouple}) |
---|
534 | \item {\tt il\_part\_id [INTEGER; IN]}: partition ID (see section |
---|
535 | \ref{subsubsec_Partition}) |
---|
536 | \item {\tt var\_nodims [INTEGER, DIMENSION(2); IN]}: var\_nodims(1) |
---|
537 | is the rank of field array (1 or 2); var\_nodims(2) is the number |
---|
538 | of bundles (always 1 in the current OASIS3-MCT version). |
---|
539 | \item {\tt kinout [INTEGER; IN]}: {\tt OASIS\_In} or {\tt PRISM\_In} |
---|
540 | (i.e. = 15) for fields received by the model; {\tt OASIS\_Out}, |
---|
541 | {\tt PRISM\_Out} (i.e. = 14) for fields sent by the model |
---|
542 | \footnote{Parameters OASIS\_In, PRISM\_In, OASIS\_Out, PRISM\_Out |
---|
543 | are defined in |
---|
544 | oasis3-mct/lib/psmile/src/mod\_oasis\_parameters.F90}. |
---|
545 | \item {\tt var\_actual\_shape [INTEGER, DIMENSION(2*var\_nodims(1)); |
---|
546 | IN]}: vector of integers giving the minimum and maximum index |
---|
547 | for each dimension of the coupling field array; for the current |
---|
548 | OASIS3-MCT version, the minimum index has to be 1 and the maximum |
---|
549 | index has to be the extent of the dimension. |
---|
550 | \item {\tt var\_type [INTEGER; IN]}: type of coupling field array; |
---|
551 | put {\tt OASIS\_Real} or {\tt PRISM\_Real} (i.e. = 4) for single |
---|
552 | or double precision real arrays. All coupling data is treated as |
---|
553 | double precision in the coupling layer, but conversion to or from |
---|
554 | single precision data is supported in the interface. |
---|
555 | \item {\tt ierror [INTEGER; OUT]}: returned error code. |
---|
556 | \end{itemize} |
---|
557 | \end{itemize} |
---|
558 | % {coupling field declaration} |
---|
559 | |
---|
560 | \section{End of definition phase} |
---|
561 | \label{subsubsec_Endofdefinition} |
---|
562 | All component processes close the definition phase. |
---|
563 | \begin{itemize} |
---|
564 | \item {\tt CALL oasis\_enddef (ierror)} |
---|
565 | \item {\tt CALL prism\_enddef\_proto(ierror)} |
---|
566 | \begin{itemize} |
---|
567 | \item ierror [INTEGER; OUT]: returned error code. |
---|
568 | \end{itemize} |
---|
569 | \end{itemize} |
---|
570 | |
---|
571 | % {End of definition phase} |
---|
572 | |
---|
573 | \section{Sending ``put'' and receiving ``get'' actions} |
---|
574 | \label{subsubsec_sendingreceiving} |
---|
575 | |
---|
576 | \subsection{Sending a coupling (or I/O) field} |
---|
577 | \label{prismput} |
---|
578 | |
---|
579 | In the model time step loop, each process sends its part of the |
---|
580 | coupling (or I/O) field. |
---|
581 | |
---|
582 | \begin{itemize} |
---|
583 | \item {\tt CALL oasis\_put (var\_id, date, fld1, info, fld2, fld3, |
---|
584 | fld4, fld5)} |
---|
585 | \item {\tt CALL prism\_put\_proto(var\_id, date, fld1, info, fld2, |
---|
586 | fld3, fld4, fld5)} |
---|
587 | \begin{itemize} |
---|
588 | \item {\tt var\_id [INTEGER; IN]}: field ID (from corresponding {\tt |
---|
589 | oasis\_def\_var}, see section \ref{subsubsec_Declaration}) |
---|
590 | \item {\tt date [INTEGER; IN]}: number of seconds (or any other time |
---|
591 | units as long as the same are used in all models and in the {\it |
---|
592 | namcouple}) at the time of the call (by convention at the |
---|
593 | beginning of the timestep) |
---|
594 | \item {\tt fld1 [REAL, IN]}: coupling (or I/O) field array; can be |
---|
595 | 1D or 2D |
---|
596 | \item {\tt info [INTEGER; OUT]}: returned info code: |
---|
597 | \begin{itemize} |
---|
598 | \item OASIS\_Sent(=4) if the field was sent to another model |
---|
599 | \item OASIS\_LocTrans (=5) if the field was only used in a time |
---|
600 | transformation (not sent, not output) |
---|
601 | \item OASIS\_ToRest (=6) if the field was written to a restart |
---|
602 | file only |
---|
603 | \item OASIS\_Output (=7) if the field was written to an output |
---|
604 | file only |
---|
605 | \item OASIS\_SentOut (=8) if the field was both written to an |
---|
606 | output file and sent to another model (directly or via OASIS3 |
---|
607 | main process) |
---|
608 | \item OASIS\_ToRestOut (=9) if the field was written both to a |
---|
609 | restart file and to an output file. |
---|
610 | \item OASIS\_Waitforallingroup (=14) if the field was not sent because it is in a group. |
---|
611 | It will be sent only when the oasis\_put of the last field in the group will be called. |
---|
612 | \item OASIS\_Ok (=0) otherwise and no error occurred. |
---|
613 | \end{itemize} |
---|
614 | \item {\tt fld2 [REAL, IN, OPTIONAL]}: optional $2^{nd}$ coupling |
---|
615 | field array; can be 1D or 2D |
---|
616 | \item {\tt fld3 [REAL, IN, OPTIONAL]}: optional $3^{rd}$ coupling |
---|
617 | field array; can be 1D or 2D |
---|
618 | \item {\tt fld4 [REAL, IN, OPTIONAL]}: optional $4^{th}$ coupling |
---|
619 | field array; can be 1D or 2D |
---|
620 | \item {\tt fld5 [REAL, IN, OPTIONAL]}: optional $5^{th}$ coupling |
---|
621 | field array; can be 1D or 2D |
---|
622 | \end{itemize} |
---|
623 | \end{itemize} |
---|
624 | |
---|
625 | To ensure a proper use of the {\tt oasis\_put}, one has to take care |
---|
626 | of the following aspects: |
---|
627 | |
---|
628 | \begin{itemize} |
---|
629 | |
---|
630 | \item A $2^{nd}$, $3^{rd}$, $4^{th}$ and $5^{th}$ source field can be |
---|
631 | passed as optional arguments. If so, the $2^{nd}$, $3^{rd}$, |
---|
632 | $4^{th}$ and $5^{th}$ set of weights present in the remapping file |
---|
633 | will be applied, respectively (see section \ref{subsec_mapdata} for |
---|
634 | the weight file format). This will be used for example for the {\tt |
---|
635 | SCRIPR/BICUBIC} remapping for which a $1^{st}$, $2^{nd}$, |
---|
636 | $3^{rd}$, $4^{th}$ set of weights should be respectively applied to |
---|
637 | the field value, its gradient in the first dimension, its gradient |
---|
638 | in the second dimension, and its cross-gradient. With |
---|
639 | OASIS3-MCT\_2.0 and following versions, bicubic and higher order |
---|
640 | remapping are therefore supported given that the higher order fields |
---|
641 | are provided at each time step as {\tt oasis\_put} arguments. Note |
---|
642 | that if {\tt fld3} is passed, {\tt fld2} must also be passed; if |
---|
643 | {\tt fld4} is passed, {\tt fld3} and {\tt fld2} must also be passed; |
---|
644 | and if {\tt fld5} is passed, {\tt fld4}, {\tt fld3} and {\tt fld2} |
---|
645 | must also be passed. However, having a $2^{nd}$, $3^{rd}$, $4^{th}$ |
---|
646 | and $5^{th}$ set of weights in the remapping file but not passing a |
---|
647 | $2^{nd}$, $3^{rd}$, $4^{th}$ and $5^{th}$ source field as argument |
---|
648 | will not lead to an error; the appropriate set of weights will |
---|
649 | simply be applied to the fields provided as arguments. |
---|
650 | \item Trying to send with {\tt oasis\_put} a field declared with a |
---|
651 | {\tt oasis\_def\_var} but not defined in the configuration file {\it |
---|
652 | namcouple} will lead to an abort. In this case, the field ID |
---|
653 | returned by the {\tt oasis\_def\_var} is equal to -1 and the |
---|
654 | corresponding {\tt oasis\_put} should not be called. |
---|
655 | \item This routine may be called by the model at each timestep. The |
---|
656 | convention for the {\tt date} argument is to indicate the time at |
---|
657 | the beginning of the timestep. The sending is actually performed |
---|
658 | only if the time obtained by adding the field lag ({\tt LAG} in the |
---|
659 | {\em namcouple}) to the {\tt date} corresponds to a time at which it |
---|
660 | should be activated, given the coupling or I/O period indicated by |
---|
661 | the user in the {\it namcouple} (see section \ref{sec_namcouple}). |
---|
662 | \item If the convention for {\tt date} is followed, the first coupling |
---|
663 | of a run should occur at time=0 and the final coupling should occur |
---|
664 | at time = runtime - cpl\_period, where runtime is the total time of |
---|
665 | the run and cpl\_period is the coupling period. |
---|
666 | \item The total run time should match an integer number of coupling |
---|
667 | periods. |
---|
668 | \item If a coupling field has a positive lag, the coupling field that |
---|
669 | matches the {\tt oasis\_get} at time=0 will come from a coupling |
---|
670 | restart file written by the last active {\tt oasis\_put} of the |
---|
671 | previous run (see section \ref{subsubsec_Algoritms}). For a coupling |
---|
672 | field with a positive lag, the coupling restart file (see section |
---|
673 | \ref{subsec_restartdata}) is automatically overwritten by the {\tt |
---|
674 | oasis\_put} when the time (i.e. {\tt date+LAG}) equals to the |
---|
675 | total runtime. |
---|
676 | % \item A field will not be sent at all if its coupling (or I/O) |
---|
677 | % period indicated in the {\it namcouple} is greater than the |
---|
678 | % total run time. |
---|
679 | \item If a local time transformation is indicated for the field by the |
---|
680 | user in the {\it namcouple} (INSTANT, AVERAGE, ACCUMUL, T\_MIN or |
---|
681 | T\_MAX, see section \ref{sec_transformations}), it is automatically |
---|
682 | performed and the resulting field is finally sent at the coupling or |
---|
683 | I/O frequency. For non-instant transformations, partially |
---|
684 | transformed fields will be written to the restart file at the end of |
---|
685 | the run for use on the next model startup. |
---|
686 | \item A coupling field sent by a source component model can be |
---|
687 | associated with more than one target field and model, if specified |
---|
688 | as so with different entries in the {\it namcouple} configuration |
---|
689 | file. In that case, the source model needs to send the field only |
---|
690 | once and the corresponding data will arrive at multiple targets as |
---|
691 | specified in the {\it namcouple}. Different coupling frequencies and |
---|
692 | transformations are allowed for different coupling exchanges of the |
---|
693 | same field. If coupling restart files are required (either if a {\tt |
---|
694 | LAG} or if a {\tt LOCTRANS} transformation is specified), it is |
---|
695 | mandatory to specify different files for the different fields. |
---|
696 | |
---|
697 | \end{itemize} |
---|
698 | |
---|
699 | \subsection{Receiving a coupling (or I/O) field} |
---|
700 | |
---|
701 | In the model time stepping loop, each process receives its part of the |
---|
702 | coupling field. |
---|
703 | |
---|
704 | \begin{itemize} |
---|
705 | |
---|
706 | \item {\tt CALL oasis\_get (var\_id, date, fld, info)} |
---|
707 | \item {\tt CALL prism\_get\_proto(var\_id, date, fld, info)} |
---|
708 | \begin{itemize} |
---|
709 | \item {\tt var\_id [INTEGER; IN]}: field ID (from the corresponding |
---|
710 | {\tt oasis\_def\_var}) |
---|
711 | \item {\tt date [INTEGER; IN]}: number of seconds (or any other time |
---|
712 | units as long as the same are used in all models and in the {\it |
---|
713 | namcouple}) at the time of the call (by convention at the |
---|
714 | beginning of the timestep) |
---|
715 | \item {\tt fld [REAL, OUT]}: I/O or coupling field array |
---|
716 | \item {\tt info [INTEGER; OUT]}: returned info code: |
---|
717 | \begin{itemize} |
---|
718 | \item OASIS\_Recvd(=3) if the field was received from another |
---|
719 | model |
---|
720 | \item OASIS\_FromRest (=10) if the field was read from a restart |
---|
721 | file only |
---|
722 | \item OASIS\_Input (=11) if the field was read from an input file |
---|
723 | only |
---|
724 | \item OASIS\_RecvOut (=12) if the field was both received from |
---|
725 | another model and written to an output file |
---|
726 | \item OASIS\_FromRestOut (=13) if the field was both read from a |
---|
727 | restart file and written to an output file |
---|
728 | \item OASIS\_Waitforallingroup (=14) if the field was not sent because it is in a group. |
---|
729 | It will be sent only when the oasis\_put of the last field in the group will be called. |
---|
730 | \item OASIS\_Ok (=0) otherwise and no error occurred. |
---|
731 | \end{itemize} |
---|
732 | \end{itemize} |
---|
733 | \end{itemize} |
---|
734 | |
---|
735 | This routine may be called by the model at each timestep. The {\tt |
---|
736 | date} argument is automatically analysed and the receiving action is |
---|
737 | actually performed only if {\tt date} corresponds to a time for which |
---|
738 | it should be activated, given the period indicated by the user in the |
---|
739 | {\it namcouple}. An exchange at the beginning of the run at time=0 is |
---|
740 | expected. |
---|
741 | |
---|
742 | Trying to receive with {\tt oasis\_get} a field declared with a {\tt |
---|
743 | oasis\_def\_var} but not defined in the configuration file {\it |
---|
744 | namcouple} will lead to an abort. In this case, the field ID |
---|
745 | returned by the {\tt oasis\_def\_var} is equal to -1 and the |
---|
746 | corresponding {\tt oasis\_get} should not be called. |
---|
747 | |
---|
748 | \section{Termination} |
---|
749 | \label{subsubsec_Termination} |
---|
750 | |
---|
751 | \begin{itemize} |
---|
752 | |
---|
753 | \item {\tt CALL oasis\_terminate (ierror)} |
---|
754 | \item {\tt CALL prism\_terminate\_proto(ierror)} |
---|
755 | \begin{itemize} |
---|
756 | \item {\tt ierror [INTEGER; OUT]}: returned error code. |
---|
757 | \end{itemize} |
---|
758 | All processes of the component model must terminate the coupling by |
---|
759 | calling this routine\footnote{If the process called {\tt MPI\_Init} |
---|
760 | (before calling {\tt oasis\_init\_comp} or {\tt |
---|
761 | prism\_init\_comp\_proto}), it must also call {\tt |
---|
762 | MPI\_Finalize} explicitly, but only after calling {\tt |
---|
763 | oasis\_terminate\_proto} or {\tt prism\_terminate\_proto}.} |
---|
764 | (normal termination). |
---|
765 | |
---|
766 | \end{itemize} |
---|
767 | |
---|
768 | % {Termination} |
---|
769 | |
---|
770 | \section{Auxiliary routines} |
---|
771 | \label{subsubsec_auxroutines} |
---|
772 | |
---|
773 | Not all auxiliary routines that were in OASIS3.3 are currently |
---|
774 | available. |
---|
775 | |
---|
776 | \begin{itemize} |
---|
777 | \item {\tt CALL oasis\_abort (compid, routine\_name, abort\_message)} |
---|
778 | \item {\tt CALL prism\_abort\_proto(compid, routine\_name, |
---|
779 | abort\_message)} |
---|
780 | \begin{itemize} |
---|
781 | \item {\tt compid [INTEGER; IN]}: component model ID (from {\tt |
---|
782 | oasis\_init\_comp} or \newline {\tt prism\_init\_comp\_proto}) |
---|
783 | \item {\tt routine\_name[CHARACTER*; IN]}: name of calling routine |
---|
784 | \item {\tt abort\_message[CHARACTER*; IN]}: message to be written |
---|
785 | out. |
---|
786 | \end{itemize} |
---|
787 | |
---|
788 | If a process needs to abort voluntarily, it should do so by calling |
---|
789 | {\tt oasis\_abort}. This will ensure a proper termination of all |
---|
790 | processes in the coupled model communicator. This routine writes the |
---|
791 | name of the calling model, the name of the calling routine, and the |
---|
792 | message to the process debug file (see {\tt \$NLOGPRT} in section |
---|
793 | \ref{subsec_namcouplefirst}). This routine cannot be called before |
---|
794 | {\tt oasis\_init\_comp}. |
---|
795 | |
---|
796 | \vspace{0.2cm} |
---|
797 | \item {\tt CALL oasis\_get\_debug(debug\_value)} |
---|
798 | \item {\tt CALL prism\_get\_debug(debug\_value)} |
---|
799 | \begin{itemize} |
---|
800 | \item {\tt debug\_value [INTEGER; OUT]}: debug value |
---|
801 | \end{itemize} |
---|
802 | |
---|
803 | This routine may be called at any time to retrieve the current |
---|
804 | OASIS3-MCT internal debug level (see {\tt \$NLOGPRT} in section |
---|
805 | \ref{subsec_namcouplefirst}). This is useful if the user wants to |
---|
806 | return the original debug value after changing it. |
---|
807 | |
---|
808 | \vspace{0.2cm} |
---|
809 | \item {\tt CALL oasis\_set\_debug(debug\_value)} |
---|
810 | \item {\tt CALL prism\_set\_debug(debug\_value)} |
---|
811 | \begin{itemize} |
---|
812 | \item {\tt debug\_value [INTEGER; IN]}: debug value |
---|
813 | \end{itemize} |
---|
814 | |
---|
815 | This routine may be called at any time to change the debug level in |
---|
816 | OASIS3-MCT. This method allows users to vary the debug level at |
---|
817 | different points in the model integration. |
---|
818 | |
---|
819 | \vspace{0.2cm} |
---|
820 | \item {\tt CALL oasis\_get\_intercomm(new\_comm, cdnam, kinfo)} |
---|
821 | \item {\tt CALL prism\_get\_intercomm(new\_comm, cdnam, kinfo)} |
---|
822 | \begin{itemize} |
---|
823 | \item {\tt new\_comm [INTEGER; OUT]}: mpi intercomm communicator |
---|
824 | \item {\tt cdnam [CHARACTER*; IN]}: other model name |
---|
825 | \item {\tt kinfo [INTEGER; OUT; OPTIONAL]}: returned error code |
---|
826 | \end{itemize} |
---|
827 | |
---|
828 | This routine sets up an MPI intercomm communicator between the root |
---|
829 | processors of two components, the local component and the component |
---|
830 | associated with {\tt cdnam}. This call is collective across the |
---|
831 | tasks of the two components but other components are not involved. |
---|
832 | |
---|
833 | \vspace{0.2cm} |
---|
834 | \item {\tt CALL oasis\_get\_intracomm(new\_comm, cdnam, kinfo)} |
---|
835 | \item {\tt CALL prism\_get\_intracomm(new\_comm, cdnam, kinfo)} |
---|
836 | \begin{itemize} |
---|
837 | \item {\tt new\_comm [INTEGER; OUT]}: mpi intracomm communicator |
---|
838 | \item {\tt cdnam [CHARACTER*; IN]}: other model name |
---|
839 | \item {\tt kinfo [INTEGER; OUT; OPTIONAL]}: returned error code |
---|
840 | \end{itemize} |
---|
841 | |
---|
842 | This routine sets up an MPI intracomm communicator between the root |
---|
843 | processors of two components, the local component and the component |
---|
844 | associated with cdnam. This call is collective across the tasks of |
---|
845 | the two components but other components are not involved. |
---|
846 | |
---|
847 | \item {\tt CALL oasis\_put\_inquire(var\_id, date, kinfo)} |
---|
848 | \item {\tt CALL prism\_put\_inquire\_proto(var\_id, date, kinfo)} |
---|
849 | \begin{itemize} |
---|
850 | \item {\tt var\_id [INTEGER; IN]}: field ID (from |
---|
851 | corresponding oasis\_def\_var) |
---|
852 | \item {\tt date [INTEGER; IN]}: number of seconds in the run at the |
---|
853 | time of the call |
---|
854 | \item {\tt kinfo [INTEGER; OUT]}: returned info code |
---|
855 | \begin{itemize} |
---|
856 | \item OASIS\_Sent(=4) if the field was sent to another model |
---|
857 | \item OASIS\_LocTrans (=5) if the field was only used in a time |
---|
858 | transformation (not sent, not output) |
---|
859 | \item OASIS\_ToRest (=6) if the field was written to a restart |
---|
860 | file only |
---|
861 | \item OASIS\_Output (=7) if the field was written to an output |
---|
862 | file only |
---|
863 | \item OASIS\_SentOut (=8) if the field was both written to an |
---|
864 | output file and sent to another model (directly or via OASIS3 |
---|
865 | main process) |
---|
866 | \item OASIS\_ToRestOut (=9) if the field was written both to a |
---|
867 | restart file and to an output file. |
---|
868 | \item OASIS\_Waitforallingroup (=14) if the field was not sent because it is in a group. |
---|
869 | It will be sent only when the oasis\_put of the last field in the group will be called. |
---|
870 | \item OASIS\_Ok (=0) otherwise and no error occurred. |
---|
871 | \end{itemize} |
---|
872 | \end{itemize} |
---|
873 | |
---|
874 | This routine may be called at any time to |
---|
875 | inquire what would happen to the corresponding field (i.e. with same |
---|
876 | {\tt var\_id} and at same {\tt date}) below the corresponding {\tt |
---|
877 | oasis\_put}. |
---|
878 | |
---|
879 | \item {\tt CALL oasis\_get\_ncpl(var\_id, ncpl, kinfo)} |
---|
880 | \item {\tt CALL prism\_get\_ncpl\_proto(var\_id, ncpl, kinfo)} |
---|
881 | \begin{itemize} |
---|
882 | \item {\tt var\_id [INTEGER; IN]}: field ID (from |
---|
883 | corresponding oasis\_def\_var) |
---|
884 | \item {\tt ncpl [INTEGER; OUT]}: number of couplings in which the field |
---|
885 | is involved (i.e. when a field is sent to multiple targets) |
---|
886 | \item {\tt kinfo [INTEGER; OUT]}: returned info code |
---|
887 | \end{itemize} |
---|
888 | |
---|
889 | This routine returns the number of coupling in which the field var\_id is |
---|
890 | involved. This number is needed to get the coupling frequencies with the |
---|
891 | routine oasis\_get\_freqs, see below. |
---|
892 | |
---|
893 | \item {\tt CALL oasis\_get\_freqs(var\_id, ncpl, cpl\_freqs, kinfo)} |
---|
894 | \item {\tt CALL prims\_get\_freqs\_proto(var\_id, ncpl, cpl\_freqs, kinfo)} |
---|
895 | \begin{itemize} |
---|
896 | \item {\tt var\_id [INTEGER; IN]}: field ID (from |
---|
897 | corresponding oasis\_def\_var) |
---|
898 | \item {\tt ncpl [INTEGER; IN]}: number of couplings in which the field |
---|
899 | is involved (i.e. when a field is sent to multiple targets) |
---|
900 | \item {\tt cpl\_freqs [INTEGER; DIMENSION(ncpl); OUT]}: periods of coupling |
---|
901 | (in number of seconds) of field var\_id. There is one coupling period for |
---|
902 | each coupling in which the field is involved |
---|
903 | \item {\tt kinfo [INTEGER; OUT]}: returned info code |
---|
904 | \end{itemize} |
---|
905 | |
---|
906 | This routine can be used to retrieve the coupling period(s) of field with |
---|
907 | corresponding {\tt var\_id}, as defined in the {\it namcouple} |
---|
908 | |
---|
909 | \end{itemize} |
---|
910 | |
---|
911 | \section{Coupling algorithms - SEQ and LAG concepts} |
---|
912 | \label{subsubsec_Algoritms} |
---|
913 | |
---|
914 | Using the OASIS3-MCT coupling library, the user has full flexibility |
---|
915 | to reproduce different coupling algorithms. In the components, the |
---|
916 | sending and receiving routines, respectively {\tt oasis\_put} and {\tt |
---|
917 | oasis\_get}, can be called at each model timestep, with the |
---|
918 | appropriate {\tt date} argument giving the actual time (at the |
---|
919 | beginning of the timestep), expressed in number of seconds since the |
---|
920 | start of the run or in any other time units, as long as the same are |
---|
921 | used in all models and in the {\it namcouple} (see section |
---|
922 | \ref{prismput}). This {\tt date} argument is automatically analysed by |
---|
923 | the coupling library and depending on the coupling period and the lag |
---|
924 | (LAG) chosen by the user for each coupling field in the configuration |
---|
925 | file {\it namcouple}, different coupling algorithms can be reproduced |
---|
926 | without modifying the component model codes themselves. |
---|
927 | |
---|
928 | With OASIS3-MCT, the SEQ index is no longer needed in the {\it |
---|
929 | namcouple} input to specify the sequencing order of different |
---|
930 | coupling fields. The sequence (SEQ) index in the {\it namcouple} file |
---|
931 | now provides the coupling layer with an ability to detect a deadlock |
---|
932 | before it happens and exit. |
---|
933 | |
---|
934 | The lag concept and indices are explained in more detail below and |
---|
935 | some examples are provided. |
---|
936 | |
---|
937 | \vspace{-0.3cm} |
---|
938 | \subsection{The lag concept} |
---|
939 | \label{subsub_lag} |
---|
940 | |
---|
941 | The lag (LAG) value tells the coupler to modify the time at which that |
---|
942 | data is sent (put) by the amount of the lag. The lag must be |
---|
943 | expressed in the time unit used (that must be the same in the models |
---|
944 | and in the {\it namcouple}, see section \ref{prismput}) and can be |
---|
945 | positive or negative but should never be larger (in absolute |
---|
946 | magnitude) than the coupling period of any field due to problems with |
---|
947 | restartability and dead-locking. When a component model calls a {\tt |
---|
948 | oasis\_put}, the value of the lag is automatically added to the |
---|
949 | value of the {\tt date} argument and the ``put'' is actually performed |
---|
950 | when the sum {\tt date+lag} is a coupling time; in the target |
---|
951 | component, this ``put'' will match a {\tt oasis\_get} for which the |
---|
952 | {\tt date} argument is the same coupling time. |
---|
953 | % A positive lag indicates the put will occur sooner than zero lag and |
---|
954 | % a negative lag tells the coupler to send the data later than zero |
---|
955 | % lag. |
---|
956 | The lag only shifts the time data is sent. It cannot be used to shift |
---|
957 | the time data is received yet. |
---|
958 | |
---|
959 | When the lag is positive, a restart file must be available to initiate |
---|
960 | the coupling and in those cases, the restart file is then updated at |
---|
961 | the end of the run. A positive lag acts like a send occurred before |
---|
962 | the model started. In fact, for a field with positive lag, the source |
---|
963 | component model automatically reads the field in the restart file |
---|
964 | during the coupling initialization phase (below the {\tt |
---|
965 | oasis\_enddef}) and send the data to match the {\tt oasis\_get} |
---|
966 | performed at time=0 in the target component model. The final coupling |
---|
967 | data on the source side will then be automatically written to the |
---|
968 | restart file for use in the next run. |
---|
969 | |
---|
970 | When there is a lag, the first and last instance of the source field |
---|
971 | in the debug netCDF file (EXPOUT fields, see section |
---|
972 | \ref{subsec_namcouplesecond}) always correspond respectively to the |
---|
973 | field read from and written to the restart file. |
---|
974 | |
---|
975 | \begin{enumerate} |
---|
976 | |
---|
977 | \item LAG concept first example |
---|
978 | |
---|
979 | A first coupling algorithm, exploiting the LAG concept, is |
---|
980 | illustrated on figure \ref{fig:lag_concept_1}. |
---|
981 | |
---|
982 | On the 4 figures in this section, short black arrows correspond to |
---|
983 | {\tt oasis\_put} or {\tt oasis\_get} called in the component model |
---|
984 | that do not lead to any ``put" or receiving action; long black |
---|
985 | arrows correspond to {\tt oasis\_put} or {\tt oasis\_get} called in |
---|
986 | the component models that do actually lead to a ``put" or ``get'' |
---|
987 | action; long red arrows correspond to {\tt oasis\_put} or {\tt |
---|
988 | oasis\_get} called in the component models that lead to a reading |
---|
989 | or writing of the coupling field from or to a coupling restart file. |
---|
990 | |
---|
991 | \begin{figure} |
---|
992 | \includegraphics[scale=.6]{figures/fig_lag_concept_1} |
---|
993 | \caption{LAG concept first example} |
---|
994 | \label{fig:lag_concept_1} |
---|
995 | \end{figure} |
---|
996 | |
---|
997 | During a coupling timestep, model A receives $F_2$ and then sends |
---|
998 | $F_1$; its timestep length is 4. During a coupling timestep, model B |
---|
999 | receives $F_1$ and then sends $F_2$; its timestep length is 6. |
---|
1000 | $F_1$ and $F_2$ coupling periods are respectively 12 and 24. If |
---|
1001 | $F_1$/$F_2$ ``put" action by model A/B was used at a coupling |
---|
1002 | timestep to match the model B/A ``get" action, a deadlock would |
---|
1003 | occur as both models would be initially waiting on a ``get" |
---|
1004 | action. To prevent this, $F_1$ and $F_2$ produced at the timestep |
---|
1005 | before have to be used to match respectively the model B and model A |
---|
1006 | ``get" actions. |
---|
1007 | |
---|
1008 | This implies that a lag of respectively 4 and 6 seconds must be |
---|
1009 | defined for $F_1$ and $F_2$. For $F_1$, the {\tt oasis\_put} |
---|
1010 | performed at time 8 and 20 by model A will then lead to ``put" |
---|
1011 | actions (as 8 + 4 = 12 and 20 + 4 = 24 which are coupling periods) |
---|
1012 | that match the ``get" actions performed at times 12 and 24 below the |
---|
1013 | {\tt oasis\_get} called by model B. For $F_2$, the {\tt oasis\_put} |
---|
1014 | performed at time 18 by model B then leads to a ``put" action (as 18 |
---|
1015 | + 6 = 24 which is a coupling period) that matches the ``get" action |
---|
1016 | performed at time 24 below the {\tt oasis\_get} called by model A. |
---|
1017 | |
---|
1018 | At the beginning of the run, as their LAG index is greater than 0, |
---|
1019 | the first {\tt oasis\_get} of $F_1$ and $F_2$ will automatically be |
---|
1020 | fulfilled with fields read from their respective coupling restart |
---|
1021 | files. The user therefore has to create those coupling restart files |
---|
1022 | before the first run in the experiment. At the end of the run, $F_1$ |
---|
1023 | having a lag greater than 0, is automatically written to its |
---|
1024 | coupling restart file below the last $F_1$ {\tt oasis\_put} as the |
---|
1025 | {\tt date+lag} equals the total run time. The analogue is true for |
---|
1026 | $F_2$. These values will automatically be read in at the beginning |
---|
1027 | of the next run below the respective {\tt oasis\_get}. |
---|
1028 | |
---|
1029 | \item LAG concept second example |
---|
1030 | |
---|
1031 | A second coupling algorithm exploiting the LAG concept is |
---|
1032 | illustrated on figure \ref{fig:lag_concept_2}. During its timestep, |
---|
1033 | model A receives $F_2$, sends $F_3$ and then $F_1$; its timestep |
---|
1034 | length is 6. During its timestep, model B receives $F_1$, receives |
---|
1035 | $F_3$ and then sends $F_2$; its timestep length is also 6. $F_1$, |
---|
1036 | $F_2$ and $F_3$ coupling periods are both supposed to be equal to |
---|
1037 | 12. |
---|
1038 | |
---|
1039 | \begin{figure} |
---|
1040 | \includegraphics[scale=.6]{figures/fig_lag_concept_2} |
---|
1041 | \caption{LAG concept second example} |
---|
1042 | \label{fig:lag_concept_2} |
---|
1043 | \end{figure} |
---|
1044 | |
---|
1045 | For $F_1$ and $F_2$ the situation is similar to the first |
---|
1046 | example. If $F_1$/$F_2$ ``put" action by model A/B was used at a |
---|
1047 | coupling timestep to match the model B/A ``get" action, a deadlock |
---|
1048 | would occur as both models would be waiting on a ``get" action. To |
---|
1049 | prevent this, $F_1$ and $F_2$ produced at the timestep before have |
---|
1050 | to be used to match the model A and model B ``get" actions, which |
---|
1051 | means that a lag of 6 must be defined for both $F_1$ and $F_2$. For |
---|
1052 | both coupling fields, the {\tt oasis\_put} performed at times 6 and |
---|
1053 | 18 by the source model then lead to ``put" actions (as 6 + 6 = 12 |
---|
1054 | and 18 + 6 = 24 which are coupling periods) that match the ``get" |
---|
1055 | action performed at time 12 and 24 below the {\tt oasis\_get} called |
---|
1056 | by the target model. |
---|
1057 | |
---|
1058 | For $F_3$, sent by model A and received by model B, no lag needs to |
---|
1059 | be defined: the coupling field produced by model A at the coupling |
---|
1060 | timestep can be ``consumed'' by model B without causing a deadlock |
---|
1061 | situation. |
---|
1062 | |
---|
1063 | As in the first example, the {\tt oasis\_get} performed at the |
---|
1064 | beginning of the run for $F_1$ and $F_2$, will automatically receive |
---|
1065 | data read from their coupling restart files, and the last {\tt |
---|
1066 | oasis\_put} performed at the end of the run automatically write |
---|
1067 | them to their coupling restart file. For $F_3$, no coupling restart |
---|
1068 | file is needed nor used as at each coupling period the coupling |
---|
1069 | field produced by model A can be directly ``consumed'' by model B. |
---|
1070 | |
---|
1071 | We see here how the introduction of appropriate LAG indices results |
---|
1072 | in receiving in the target model the coupling fields produced by the |
---|
1073 | source model the timestep before; this is, in some coupling |
---|
1074 | configurations, essential to avoid deadlock situations. |
---|
1075 | |
---|
1076 | \end{enumerate} |
---|
1077 | |
---|
1078 | \vspace{-0.3cm} |
---|
1079 | \subsection{The sequence concept} |
---|
1080 | \label{subsec_sec} |
---|
1081 | |
---|
1082 | The order of coupling operations in the system is determined solely by |
---|
1083 | the order of calls to send (put) and receive (get) data in the models |
---|
1084 | in conjunction with the setting of the lag in the {\it namcouple}. |
---|
1085 | Data that is received (get) is always blocking while data that is sent |
---|
1086 | (put) is non-blocking with respect to the model making that call. It |
---|
1087 | is possible to deadlock the system if the relative orders of puts and |
---|
1088 | gets in different models are not compatible. |
---|
1089 | |
---|
1090 | With OASIS3-MCT, the sequence (SEQ) index in the {\it namcouple} file |
---|
1091 | now provides the coupling layer with an ability to detect a deadlock |
---|
1092 | before it happens and exit. It does this by tracking the order of get |
---|
1093 | and put calls in models compared to the SEQ specified in the {\it |
---|
1094 | namcouple}. If there are any inconsistencies, the model will abort |
---|
1095 | gracefully with a useable error message before the system deadlocks. |
---|
1096 | If there are any coupling dependencies in the system, use of the SEQ |
---|
1097 | index is recommended for diagnosis but has no impact on the ultimate |
---|
1098 | solution and is NOT required. |
---|
1099 | |
---|
1100 | Take the following two examples. In both examples, there are two |
---|
1101 | models, each ``put" a field to the other at every coupling period |
---|
1102 | without any lags. In the first case, there is no dependency and each |
---|
1103 | model sends (puts) the data first and then receives the data second at |
---|
1104 | each timestep. |
---|
1105 | |
---|
1106 | \begin{verbatim} |
---|
1107 | model1 model2 |
---|
1108 | ------ ------ |
---|
1109 | put(fld1) put(fld2) |
---|
1110 | get(fld2) get(fld1) |
---|
1111 | \end{verbatim} |
---|
1112 | |
---|
1113 | The put from model1 for fld1 is received by the get in model2 and the |
---|
1114 | put from model2 for fld2 is received by the get in model1. In this |
---|
1115 | case, there is no sequencing dependency and the value of SEQ must be |
---|
1116 | identical (or unset) in the {\it namcouple} description of the fld1 |
---|
1117 | and fld2 coupling. If SEQ is set to 1 for fld1 and 2 for fld2 in this |
---|
1118 | case, then the model will abort because at runtime, the coupling will |
---|
1119 | detect, in model 2, that fld2 was sent before fld1 was received which |
---|
1120 | is out of sequence as defined by the SEQ settings. |
---|
1121 | |
---|
1122 | In the next example, there is a dependency in the sequencing. |
---|
1123 | |
---|
1124 | \begin{verbatim} |
---|
1125 | model1 model2 |
---|
1126 | ------ ------ |
---|
1127 | put(fld1) get(fld1) |
---|
1128 | fld2=g(fld1) |
---|
1129 | get(fld2) put(fld2) |
---|
1130 | \end{verbatim} |
---|
1131 | |
---|
1132 | In model2, fld2 depends on fld1. If this dependency is known, then |
---|
1133 | there is a benefit in using SEQ=1 for fld1 and SEQ=2 for fld2. At |
---|
1134 | runtime, if the sequencings of both model1 and model2 do not match the |
---|
1135 | above diagram, the model will abort gracefully. For instance, if |
---|
1136 | model2 has the dependency shown above but model1 does not have |
---|
1137 | consistent ordering of the put and get as required by model2, then if |
---|
1138 | SEQ is unused, the model will deadlock and hang. If SEQ is set |
---|
1139 | properly, the coupling layer will detect that the required sequence |
---|
1140 | has not been followed and will exit gracefully with an error message. |
---|
1141 | |
---|
1142 | Again, the SEQ namecouple setting is only diagnostic and is not |
---|
1143 | required. |
---|
1144 | |
---|
1145 | |
---|
1146 | |
---|