# source:NEMO/branches/2018/dev_r10164_HPC09_ESIWACE_PREP_MERGE/doc/latex/NEMO/subfiles/chap_misc.tex@10419

Last change on this file since 10419 was 10419, checked in by smasson, 22 months ago

dev_r10164_HPC09_ESIWACE_PREP_MERGE: merge with trunk@10418, see #2133

File size: 15.4 KB
Line
1\documentclass[../main/NEMO_manual]{subfiles}
2
3\begin{document}
4% ================================================================
5% Chapter --- Miscellaneous Topics
6% ================================================================
7\chapter{Miscellaneous Topics}
8\label{chap:MISC}
9
10\minitoc
11
12\newpage
13
14% ================================================================
15% Representation of Unresolved Straits
16% ================================================================
17\section{Representation of unresolved straits}
18\label{sec:MISC_strait}
19
20In climate modeling, it often occurs that a crucial connections between water masses is broken as
21the grid mesh is too coarse to resolve narrow straits.
22For example, coarse grid spacing typically closes off the Mediterranean from the Atlantic at
23the Strait of Gibraltar.
24In this case, it is important for climate models to include the effects of salty water entering the Atlantic from
25the Mediterranean.
26Likewise, it is important for the Mediterranean to replenish its supply of water from the Atlantic to
27balance the net evaporation occurring over the Mediterranean region.
28This problem occurs even in eddy permitting simulations.
29For example, in ORCA 1/4\deg several straits of the Indonesian archipelago (Ombai, Lombok...)
30are much narrow than even a single ocean grid-point.
31
32We describe briefly here the three methods that can be used in \NEMO to handle such improperly resolved straits.
33The first two consist of opening the strait by hand while ensuring that the mass exchanges through
34the strait are not too large by either artificially reducing the surface of the strait grid-cells or,
35locally increasing the lateral friction.
36In the third one, the strait is closed but exchanges of mass, heat and salt across the land are allowed.
37Note that such modifications are so specific to a given configuration that no attempt has been made to
38set them in a generic way.
39However, examples of how they can be set up is given in the ORCA 2\deg and 0.5\deg configurations.
40For example, for details of implementation in ORCA2, search: \texttt{IF( cp\_cfg == "orca" .AND. jp\_cfg == 2 )}
41
42% -------------------------------------------------------------------------------------------------------------
44% -------------------------------------------------------------------------------------------------------------
46\label{subsec:MISC_strait_hand}
47
48$\bullet$ reduced scale factor in the cross-strait direction to a value in better agreement with
49the true mean width of the strait (\autoref{fig:MISC_strait_hand}).
50This technique is sometime called "partially open face" or "partially closed cells".
51The key issue here is only to reduce the faces of $T$-cell
52($i.e.$ change the value of the horizontal scale factors at $u$- or $v$-point) but not the volume of the $T$-cell.
53Indeed, reducing the volume of strait $T$-cell can easily produce a numerical instability at
54that grid point that would require a reduction of the model time step.
55The changes associated with strait management are done in \mdl{domhgr},
56just after the definition or reading of the horizontal scale factors.
57
58$\bullet$ increase of the viscous boundary layer thickness by local increase of the fmask value at the coast
59(\autoref{fig:MISC_strait_hand}).
60This is done in \mdl{dommsk} together with the setting of the coastal value of fmask (see  \autoref{sec:LBC_coast}).
61
62%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
63\begin{figure}[!tbp]
64  \begin{center}
65    \includegraphics[width=0.80\textwidth]{Fig_Gibraltar}
66    \includegraphics[width=0.80\textwidth]{Fig_Gibraltar2}
67    \caption{
68      \protect\label{fig:MISC_strait_hand}
69      Example of the Gibraltar strait defined in a $1^{\circ} \times 1^{\circ}$ mesh.
70      \textit{Top}: using partially open cells.
71      The meridional scale factor at $v$-point is reduced on both sides of the strait to account for
72      the real width of the strait (about 20 km).
73      Note that the scale factors of the strait $T$-point remains unchanged.
74      \textit{Bottom}: using viscous boundary layers.
75      The four fmask parameters along the strait coastlines are set to a value larger than 4,
76      $i.e.$ "strong" no-slip case (see \autoref{fig:LBC_shlat}) creating a large viscous boundary layer that
77      allows a reduced transport through the strait.
78    }
79  \end{center}
80\end{figure}
81%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
82
83
84% ================================================================
85% Closed seas
86% ================================================================
87\section{Closed seas (\protect\mdl{closea})}
88\label{sec:MISC_closea}
89
90\colorbox{yellow}{Add here a short description of the way closed seas are managed}
91
92
93% ================================================================
94% Sub-Domain Functionality
95% ================================================================
96\section{Sub-domain functionality}
97\label{sec:MISC_zoom}
98
99\subsection{Simple subsetting of input files via NetCDF attributes}
100
101The extended grids for use with the under-shelf ice cavities will result in redundant rows around Antarctica if
102the ice cavities are not active.
103A simple mechanism for subsetting input files associated with the extended domains has been implemented to
104avoid the need to maintain different sets of input fields for use with or without active ice cavities.
105The existing 'zoom' options are overly complex for this task and marked for deletion anyway.
106This alternative subsetting operates for the j-direction only and works by optionally looking for and
107using a global file attribute (named: \np{open\_ocean\_jstart}) to determine the starting j-row for input.
108The use of this option is best explained with an example:
109consider an ORCA1 configuration using the extended grid bathymetry and coordinate files:
110\vspace{-10pt}
111\ifile{eORCA1\_bathymetry\_v2}
112\ifile{eORCA1\_coordinates}
113\noindent These files define a horizontal domain of 362x332.
114Assuming the first row with open ocean wet points in the non-isf bathymetry for this set is row 42
115(Fortran indexing) then the formally correct setting for \np{open\_ocean\_jstart} is 41.
116Using this value as the first row to be read will result in a 362x292 domain which is the same size as
117the original ORCA1 domain.
118Thus the extended coordinates and bathymetry files can be used with all the original input files for ORCA1 if
119the ice cavities are not active (\np{ln\_isfcav = .false.}).
120Full instructions for achieving this are:
121
122\noindent Add the new attribute to any input files requiring a j-row offset, i.e:
123\vspace{-10pt}
124\begin{cmds}
125ncatted  -a open_ocean_jstart,global,a,d,41 eORCA1_coordinates.nc
126ncatted  -a open_ocean_jstart,global,a,d,41 eORCA1_bathymetry_v2.nc
127\end{cmds}
128
129\noindent Add the logical switch to \ngn{namcfg} in the configuration namelist and set true:
130%--------------------------------------------namcfg--------------------------------------------------------
131
132\nlst{namcfg}
133%--------------------------------------------------------------------------------------------------------------
134
135\noindent Note the j-size of the global domain is the (extended j-size minus \np{open\_ocean\_jstart} + 1 ) and
136this must match the size of all datasets other than bathymetry and coordinates currently.
137However the option can be extended to any global, 2D and 3D, netcdf, input field by adding the:
138\vspace{-10pt}
139\begin{forlines}
140lrowattr=ln_use_jattr
141\end{forlines}
142optional argument to the appropriate \np{iom\_get} call and the \np{open\_ocean\_jstart} attribute to
143the corresponding input files.
144It remains the users responsibility to set \np{jpjdta} and \np{jpjglo} values in
145the \np{namelist\_cfg} file according to their needs.
146
147%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
148\begin{figure}[!ht]
149  \begin{center}
150    \includegraphics[width=0.90\textwidth]{Fig_LBC_zoom}
151    \caption{
152      \protect\label{fig:LBC_zoom}
153      Position of a model domain compared to the data input domain when the zoom functionality is used.
154    }
155  \end{center}
156\end{figure}
157%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
158
159
160% ================================================================
161% Accuracy and Reproducibility
162% ================================================================
163\section{Accuracy and reproducibility (\protect\mdl{lib\_fortran})}
164\label{sec:MISC_fortran}
165
166\subsection{Issues with intrinsinc SIGN function (\protect\key{nosignedzero})}
167\label{subsec:MISC_sign}
168
169The SIGN(A, B) is the \textsc {Fortran} intrinsic function delivers the magnitude of A with the sign of B.
170For example, SIGN(-3.0,2.0) has the value 3.0.
171The problematic case is when the second argument is zero, because, on platforms that support IEEE arithmetic,
172zero is actually a signed number.
173There is a positive zero and a negative zero.
174
175In \textsc{Fortran}~90, the processor was required always to deliver a positive result for SIGN(A, B) if B was zero.
176Nevertheless, in \textsc{Fortran}~95, the processor is allowed to do the correct thing and deliver ABS(A) when
177B is a positive zero and -ABS(A) when B is a negative zero.
178This change in the specification becomes apparent only when B is of type real, and is zero,
179and the processor is capable of distinguishing between positive and negative zero,
180and B is negative real zero.
181Then SIGN delivers a negative result where, under \textsc{Fortran}~90 rules, it used to return a positive result.
182This change may be especially sensitive for the ice model,
183so we overwrite the intrinsinc function with our own function simply performing :   \\
184\verb?   IF( B >= 0.e0 ) THEN   ;   SIGN(A,B) = ABS(A)  ?    \\
185\verb?   ELSE                   ;   SIGN(A,B) =-ABS(A)     ?  \\
186\verb?   ENDIF    ? \\
187This feature can be found in \mdl{lib\_fortran} module and is effective when \key{nosignedzero} is defined.
188We use a CPP key as the overwritting of a intrinsic function can present performance issues with
189some computers/compilers.
190
191
192\subsection{MPP reproducibility}
193\label{subsec:MISC_glosum}
194
195The numerical reproducibility of simulations on distributed memory parallel computers is a critical issue.
196In particular, within NEMO global summation of distributed arrays is most susceptible to rounding errors,
197and their propagation and accumulation cause uncertainty in final simulation reproducibility on
198different numbers of processors.
199To avoid so, based on \citet{He_Ding_JSC01} review of different technics,
200we use a so called self-compensated summation method.
201The idea is to estimate the roundoff error, store it in a buffer, and then add it back in the next addition.
202
203Suppose we need to calculate $b = a_1 + a_2 + a_3$.
204The following algorithm will allow to split the sum in two
205($sum_1 = a_{1} + a_{2}$ and $b = sum_2 = sum_1 + a_3$) with exactly the same rounding errors as
206the sum performed all at once.
207\begin{align*}
208   sum_1 \ \  &= a_1 + a_2 \\
209   error_1     &= a_2 + ( a_1 - sum_1 ) \\
210   sum_2 \ \  &= sum_1 + a_3 + error_1 \\
211   error_2     &= a_3 + error_1 + ( sum_1 - sum_2 ) \\
212   b \qquad \ &= sum_2 \\
213\end{align*}
214An example of this feature can be found in \mdl{lib\_fortran} module.
215It is systematicallt used in glob\_sum function (summation over the entire basin excluding duplicated rows and
216columns due to cyclic or north fold boundary condition as well as overlap MPP areas).
217The self-compensated summation method should be used in all summation in i- and/or j-direction.
218See \mdl{closea} module for an example.
219Note also that this implementation may be sensitive to the optimization level.
220
221\subsection{MPP scalability}
222\label{subsec:MISC_mppsca}
223
224The default method of communicating values across the north-fold in distributed memory applications (\key{mpp\_mpi})
225uses a \textsc{MPI\_ALLGATHER} function to exchange values from each processing region in
226the northern row with every other processing region in the northern row.
227This enables a global width array containing the top 4 rows to be collated on every northern row processor and then
228folded with a simple algorithm.
229Although conceptually simple, this "All to All" communication will hamper performance scalability for
230large numbers of northern row processors.
231From version 3.4 onwards an alternative method is available which only performs direct "Peer to Peer" communications
232between each processor and its immediate "neighbours" across the fold line.
233This is achieved by using the default \textsc{MPI\_ALLGATHER} method during initialisation to
234help identify the "active" neighbours.
235Stored lists of these neighbours are then used in all subsequent north-fold exchanges to
236restrict exchanges to those between associated regions.
237The collated global width array for each region is thus only partially filled but is guaranteed to
238be set at all the locations actually required by each individual for the fold operation.
239This alternative method should give identical results to the default \textsc{ALLGATHER} method and
240is recommended for large values of \np{jpni}.
241The new method is activated by setting \np{ln\_nnogather} to be true ({\bf nammpp}).
242The reproducibility of results using the two methods should be confirmed for each new,
243non-reference configuration.
244
245% ================================================================
246% Model optimisation, Control Print and Benchmark
247% ================================================================
248\section{Model optimisation, control print and benchmark}
249\label{sec:MISC_opt}
250%--------------------------------------------namctl-------------------------------------------------------
251
252\nlst{namctl}
253%--------------------------------------------------------------------------------------------------------------
254
255 \gmcomment{why not make these bullets into subsections?}
256Options are defined through the  \ngn{namctl} namelist variables.
257
258$\bullet$ Vector optimisation:
259
260\key{vectopt\_loop} enables the internal loops to collapse.
261This is very a very efficient way to increase the length of vector calculations and thus
262to speed up the model on vector computers.
263
264% Add here also one word on NPROMA technique that has been found useless, since compiler have made significant progress during the last decade.
265
266% Add also one word on NEC specific optimisation (Novercheck option for example)
267
268$\bullet$ Control print %: describe here 4 things:
269
2701- \np{ln\_ctl}: compute and print the trends averaged over the interior domain in all TRA, DYN, LDF and
271ZDF modules.
272This option is very helpful when diagnosing the origin of an undesired change in model results.
273
2742- also \np{ln\_ctl} but using the nictl and njctl namelist parameters to check the source of differences between
275mono and multi processor runs.
276
277%%gm   to be removed both here and in the code
2783- last digit comparison (\np{nn\_bit\_cmp}).
279In an MPP simulation, the computation of a sum over the whole domain is performed as the summation over
280all processors of each of their sums over their interior domains.
281This double sum never gives exactly the same result as a single sum over the whole domain,
282due to truncation differences.
283The "bit comparison" option has been introduced in order to be able to check that
284mono-processor and multi-processor runs give exactly the same results.
285% THIS is to be updated with the mpp_sum_glo  introduced in v3.3
286% nn_bit_cmp  today only check that the nn_cla = 0 (no cross land advection)
287%%gm end
288
289$\bullet$  Benchmark (\np{nn\_bench}).
290This option defines a benchmark run based on a GYRE configuration (see \autoref{sec:CFG_gyre}) in which
291the resolution remains the same whatever the domain size.
292This allows a very large model domain to be used, just by changing the domain size (\jp{jpiglo}, \jp{jpjglo}) and
293without adjusting either the time-step or the physical parameterisations.
294
295% ================================================================
296\biblio
297
298\end{document}
Note: See TracBrowser for help on using the repository browser.