# source:branches/2017/dev_merge_2017/DOC/texfiles/chapters/chap_misc.tex@9373

Last change on this file since 9373 was 9373, checked in by nicolasmartin, 2 years ago

Global reorganisation of DOC directory: files/folders renaming

File size: 15.5 KB
Line
1\documentclass[NEMO_book]{subfiles}
2\begin{document}
3% ================================================================
4% Chapter --- Miscellaneous Topics
5% ================================================================
6\chapter{Miscellaneous Topics}
7\label{MISC}
8\minitoc
9
10\newpage
11$\$\newline    % force a new ligne
12
13% ================================================================
14% Representation of Unresolved Straits
15% ================================================================
16\section{Representation of Unresolved Straits}
17\label{MISC_strait}
18
19In climate modeling, it often occurs that a crucial connections between water masses
20is broken as the grid mesh is too coarse to resolve narrow straits. For example, coarse
21grid spacing typically closes off the Mediterranean from the Atlantic at the Strait of
22Gibraltar. In this case, it is important for climate models to include the effects of salty
23water entering the Atlantic from the Mediterranean. Likewise, it is important for the
24Mediterranean to replenish its supply of water from the Atlantic to balance the net
25evaporation occurring over the Mediterranean region. This problem occurs even in
26eddy permitting simulations. For example, in ORCA 1/4\deg several straits of the Indonesian
27archipelago (Ombai, Lombok...) are much narrow than even a single ocean grid-point.
28
29We describe briefly here the three methods that can be used in \NEMO to handle
30such improperly resolved straits. The first two consist of opening the strait by hand
31while ensuring that the mass exchanges through the strait are not too large by
32either artificially reducing the surface of the strait grid-cells or, locally increasing
33the lateral friction. In the third one, the strait is closed but exchanges of mass,
34heat and salt across the land are allowed.
35Note that such modifications are so specific to a given configuration that no attempt
36has been made to set them in a generic way. However, examples of how
37they can be set up is given in the ORCA 2\deg and 0.5\deg configurations. For example,
38for details of implementation in ORCA2, search:
39\texttt{ IF( cp\_cfg == "orca" .AND. jp\_cfg == 2 ) }
40
41% -------------------------------------------------------------------------------------------------------------
43% -------------------------------------------------------------------------------------------------------------
45\label{MISC_strait_hand}
46
47$\bullet$ reduced scale factor in the cross-strait direction to a value in better agreement
48with the true mean width of the strait. (Fig.~\ref{Fig_MISC_strait_hand}).
49This technique is sometime called "partially open face" or "partially closed cells".
50The key issue here is only to reduce the faces of $T$-cell ($i.e.$ change the value
51of the horizontal scale factors at $u$- or $v$-point) but not the volume of the $T$-cell.
52Indeed, reducing the volume of strait $T$-cell can easily produce a numerical
53instability at that grid point that would require a reduction of the model time step.
54The changes associated with strait management are done in \mdl{domhgr},
55just after the definition or reading of the horizontal scale factors.
56
57$\bullet$ increase of the viscous boundary layer thickness by local increase of the
58fmask value at the coast (Fig.~\ref{Fig_MISC_strait_hand}). This is done in
59\mdl{dommsk} together with the setting of the coastal value of fmask
60(see Section \ref{LBC_coast})
61
62%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
63\begin{figure}[!tbp]     \begin{center}
64\includegraphics[width=0.80\textwidth]{Fig_Gibraltar}
65\includegraphics[width=0.80\textwidth]{Fig_Gibraltar2}
66\caption{   \protect\label{Fig_MISC_strait_hand}
67Example of the Gibraltar strait defined in a $1^{\circ} \times 1^{\circ}$ mesh.
68\textit{Top}: using partially open cells. The meridional scale factor at $v$-point
69is reduced on both sides of the strait to account for the real width of the strait
70(about 20 km). Note that the scale factors of the strait $T$-point remains unchanged.
71\textit{Bottom}: using viscous boundary layers. The four fmask parameters
72along the strait coastlines are set to a value larger than 4, $i.e.$ "strong" no-slip
73case (see Fig.\ref{Fig_LBC_shlat}) creating a large viscous boundary layer
74that allows a reduced transport through the strait.}
75\end{center}   \end{figure}
76%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
77
78
79% ================================================================
80% Closed seas
81% ================================================================
82\section{Closed seas (\protect\mdl{closea})}
83\label{MISC_closea}
84
85\colorbox{yellow}{Add here a short description of the way closed seas are managed}
86
87
88% ================================================================
89% Sub-Domain Functionality
90% ================================================================
91\section{Sub-Domain Functionality}
92\label{MISC_zoom}
93
94\subsection{Simple subsetting of input files via netCDF attributes}
95
96The extended grids for use with the under-shelf ice cavities will result in redundant rows
97around Antarctica if the ice cavities are not active. A simple mechanism for subsetting
98input files associated with the extended domains has been implemented to avoid the need to
99maintain different sets of input fields for use with or without active ice cavities. The
100existing 'zoom' options are overly complex for this task and marked for deletion anyway.
101This alternative subsetting operates for the j-direction only and works by optionally
102looking for and using a global file attribute (named: \np{open\_ocean\_jstart}) to
103determine the starting j-row for input. The use of this option is best explained with an
104example: Consider an ORCA1 configuration using the extended grid bathymetry and coordinate
105files:
106\vspace{-10pt}
107\begin{alltt}
108\tiny
109\begin{verbatim}
110eORCA1_bathymetry_v2.nc
111eORCA1_coordinates.nc
112\end{verbatim}
113\end{alltt}
114\noindent These files define a horizontal domain of 362x332. Assuming the first row with
115open ocean wet points in the non-isf bathymetry for this set is row 42 (Fortran indexing)
116then the formally correct setting for \np{open\_ocean\_jstart} is 41. Using this value as the
117first row to be read will result in a 362x292 domain which is the same size as the original
118ORCA1 domain. Thus the extended coordinates and bathymetry files can be used with all the
119original input files for ORCA1 if the ice cavities are not active (\np{ln\_isfcav =
120.false.}). Full instructions for achieving this are:
121
122\noindent Add the new attribute to any input files requiring a j-row offset, i.e:
123\vspace{-10pt}
124\begin{alltt}
125\tiny
126\begin{verbatim}
127ncatted  -a open_ocean_jstart,global,a,d,41 eORCA1_coordinates.nc
128ncatted  -a open_ocean_jstart,global,a,d,41 eORCA1_bathymetry_v2.nc
129\end{verbatim}
130\end{alltt}
131
132\noindent Add the logical switch to \ngn{namcfg} in the configuration namelist and set true:
133%--------------------------------------------namcfg--------------------------------------------------------
134\namdisplay{namcfg}
135%--------------------------------------------------------------------------------------------------------------
136
137\noindent Note the j-size of the global domain is the (extended j-size minus
138\np{open\_ocean\_jstart} + 1 ) and this must match the size of all datasets other than
139bathymetry and coordinates currently. However the option can be extended to any global, 2D
140and 3D, netcdf, input field by adding the:
141\vspace{-10pt}
142\begin{alltt}
143\tiny
144\begin{verbatim}
145lrowattr=ln_use_jattr
146\end{verbatim}
147\end{alltt}
148optional argument to the appropriate \np{iom\_get} call and the \np{open\_ocean\_jstart} attribute to the corresponding input files. It remains the users responsibility to set \np{jpjdta} and \np{jpjglo} values in the \np{namelist\_cfg} file according to their needs.
149
150%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
151\begin{figure}[!ht]    \begin{center}
152\includegraphics[width=0.90\textwidth]{Fig_LBC_zoom}
153\caption{   \protect\label{Fig_LBC_zoom}
154Position of a model domain compared to the data input domain when the zoom functionality is used.}
155\end{center}   \end{figure}
156%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
157
158
159% ================================================================
160% Accuracy and Reproducibility
161% ================================================================
162\section{Accuracy and Reproducibility (\protect\mdl{lib\_fortran})}
163\label{MISC_fortran}
164
165\subsection{Issues with intrinsinc SIGN function (\protect\key{nosignedzero})}
166\label{MISC_sign}
167
168The SIGN(A, B) is the \textsc {Fortran} intrinsic function delivers the magnitude
169of A with the sign of B. For example, SIGN(-3.0,2.0) has the value 3.0.
170The problematic case is when the second argument is zero, because, on platforms
171that support IEEE arithmetic, zero is actually a signed number.
172There is a positive zero and a negative zero.
173
174In \textsc{Fortran}~90, the processor was required always to deliver a positive result for SIGN(A, B)
175if B was zero. Nevertheless, in \textsc{Fortran}~95, the processor is allowed to do the correct thing
176and deliver ABS(A) when B is a positive zero and -ABS(A) when B is a negative zero.
177This change in the specification becomes apparent only when B is of type real, and is zero,
178and the processor is capable of distinguishing between positive and negative zero,
179and B is negative real zero. Then SIGN delivers a negative result where, under \textsc{Fortran}~90
180rules,  it used to return a positive result.
181This change may be especially sensitive for the ice model, so we overwrite the intrinsinc
182function with our own function simply performing :   \\
183\verb?   IF( B >= 0.e0 ) THEN   ;   SIGN(A,B) = ABS(A)  ?    \\
184\verb?   ELSE                   ;   SIGN(A,B) =-ABS(A)     ?  \\
185\verb?   ENDIF    ? \\
186This feature can be found in \mdl{lib\_fortran} module and is effective when \key{nosignedzero}
187is defined. We use a CPP key as the overwritting of a intrinsic function can present
188performance issues with some computers/compilers.
189
190
191\subsection{MPP reproducibility}
192\label{MISC_glosum}
193
194The numerical reproducibility of simulations on distributed memory parallel computers
195is a critical issue. In particular, within NEMO global summation of distributed arrays
196is most susceptible to rounding errors, and their propagation and accumulation cause
197uncertainty in final simulation reproducibility on different numbers of processors.
198To avoid so, based on \citet{He_Ding_JSC01} review of different technics,
199we use a so called self-compensated summation method. The idea is to estimate
200the roundoff error, store it in a buffer, and then add it back in the next addition.
201
202Suppose we need to calculate $b = a_1 + a_2 + a_3$. The following algorithm
203will allow to split the sum in two ($sum_1 = a_{1} + a_{2}$ and $b = sum_2 = sum_1 + a_3$)
204with exactly the same rounding errors as the sum performed all at once.
205\begin{align*}
206   sum_1 \ \  &= a_1 + a_2 \\
207   error_1     &= a_2 + ( a_1 - sum_1 ) \\
208   sum_2 \ \  &= sum_1 + a_3 + error_1 \\
209   error_2     &= a_3 + error_1 + ( sum_1 - sum_2 ) \\
210   b \qquad \ &= sum_2 \\
211\end{align*}
212An example of this feature can be found in \mdl{lib\_fortran} module.
213It is systematicallt used in glob\_sum function (summation over the entire basin excluding
214duplicated rows and columns due to cyclic or north fold boundary condition as well as
215overlap MPP areas). The self-compensated summation method should be used in all summation
216in i- and/or j-direction. See closea.F90 module for an example.
217Note also that this implementation may be sensitive to the optimization level.
218
219\subsection{MPP scalability}
220\label{MISC_mppsca}
221
222The default method of communicating values across the north-fold in distributed memory applications
223(\key{mpp\_mpi}) uses a \textsc{MPI\_ALLGATHER} function to exchange values from each processing
224region in the northern row with every other processing region in the northern row. This enables a
225global width array containing the top 4 rows to be collated on every northern row processor and then
226folded with a simple algorithm. Although conceptually simple, this "All to All" communication will
227hamper performance scalability for large numbers of northern row processors. From version 3.4
228onwards an alternative method is available which only performs direct "Peer to Peer" communications
229between each processor and its immediate "neighbours" across the fold line. This is achieved by
230using the default \textsc{MPI\_ALLGATHER} method during initialisation to help identify the "active"
231neighbours. Stored lists of these neighbours are then used in all subsequent north-fold exchanges to
232restrict exchanges to those between associated regions. The collated global width array for each
233region is thus only partially filled but is guaranteed to be set at all the locations actually
234required by each individual for the fold operation. This alternative method should give identical
235results to the default \textsc{ALLGATHER} method and is recommended for large values of \np{jpni}.
236The new method is activated by setting \np{ln\_nnogather} to be true ({\bf nammpp}). The
237reproducibility of results using the two methods should be confirmed for each new, non-reference
238configuration.
239
240% ================================================================
241% Model optimisation, Control Print and Benchmark
242% ================================================================
243\section{Model Optimisation, Control Print and Benchmark}
244\label{MISC_opt}
245%--------------------------------------------namctl-------------------------------------------------------
246\namdisplay{namctl}
247%--------------------------------------------------------------------------------------------------------------
248
249 \gmcomment{why not make these bullets into subsections?}
250Options are defined through the  \ngn{namctl} namelist variables.
251
252$\bullet$ Vector optimisation:
253
254\key{vectopt\_loop} enables the internal loops to collapse. This is very
255a very efficient way to increase the length of vector calculations and thus
256to speed up the model on vector computers.
257
258% Add here also one word on NPROMA technique that has been found useless, since compiler have made significant progress during the last decade.
259
260% Add also one word on NEC specific optimisation (Novercheck option for example)
261
262$\bullet$ Control print %: describe here 4 things:
263
2641- \np{ln\_ctl} : compute and print the trends averaged over the interior domain
265in all TRA, DYN, LDF and ZDF modules. This option is very helpful when
266diagnosing the origin of an undesired change in model results.
267
2682- also \np{ln\_ctl} but using the nictl and njctl namelist parameters to check
269the source of differences between mono and multi processor runs.
270
271%%gm   to be removed both here and in the code
2723- last digit comparison (\np{nn\_bit\_cmp}). In an MPP simulation, the computation of
273a sum over the whole domain is performed as the summation over all processors of
274each of their sums over their interior domains. This double sum never gives exactly
275the same result as a single sum over the whole domain, due to truncation differences.
276The "bit comparison" option has been introduced in order to be able to check that
277mono-processor and multi-processor runs give exactly the same results.
278%THIS is to be updated with the mpp_sum_glo  introduced in v3.3
279% nn_bit_cmp  today only check that the nn_cla = 0 (no cross land advection)
280%%gm end
281
282$\bullet$  Benchmark (\np{nn\_bench}). This option defines a benchmark run based on
283a GYRE configuration (see \S\ref{CFG_gyre}) in which the resolution remains the same
284whatever the domain size. This allows a very large model domain to be used, just by
285changing the domain size (\jp{jpiglo}, \jp{jpjglo}) and without adjusting either the time-step
286or the physical parameterisations.
287
288% ================================================================
289\end{document}
290
291
292
293
Note: See TracBrowser for help on using the repository browser.