[707] | 1 | % ================================================================ |
---|
| 2 | % Chapter Ñ Miscellaneous Topics |
---|
| 3 | % ================================================================ |
---|
[2282] | 4 | \chapter{Miscellaneous Topics} |
---|
[707] | 5 | \label{MISC} |
---|
| 6 | \minitoc |
---|
| 7 | |
---|
[2282] | 8 | \newpage |
---|
| 9 | $\ $\newline % force a new ligne |
---|
| 10 | |
---|
[707] | 11 | % ================================================================ |
---|
| 12 | % Representation of Unresolved Straits |
---|
| 13 | % ================================================================ |
---|
| 14 | \section{Representation of Unresolved Straits} |
---|
| 15 | \label{MISC_strait} |
---|
| 16 | |
---|
[2282] | 17 | In climate modeling, it often occurs that a crucial connections between water masses |
---|
| 18 | is broken as the grid mesh is too coarse to resolve narrow straits. For example, coarse |
---|
| 19 | grid spacing typically closes off the Mediterranean from the Atlantic at the Strait of |
---|
| 20 | Gibraltar. In this case, it is important for climate models to include the effects of salty |
---|
| 21 | water entering the Atlantic from the Mediterranean. Likewise, it is important for the |
---|
| 22 | Mediterranean to replenish its supply of water from the Atlantic to balance the net |
---|
| 23 | evaporation occurring over the Mediterranean region. This problem occurs even in |
---|
[2349] | 24 | eddy permitting simulations. For example, in ORCA 1/4\deg several straits of the Indonesian |
---|
[2282] | 25 | archipelago (Ombai, Lombok...) are much narrow than even a single ocean grid-point. |
---|
[707] | 26 | |
---|
[2282] | 27 | We describe briefly here the three methods that can be used in \NEMO to handle |
---|
| 28 | such improperly resolved straits. The first two consist of opening the strait by hand |
---|
| 29 | while ensuring that the mass exchanges through the strait are not too large by |
---|
| 30 | either artificially reducing the surface of the strait grid-cells or, locally increasing |
---|
| 31 | the lateral friction. In the third one, the strait is closed but exchanges of mass, |
---|
| 32 | heat and salt across the land are allowed. |
---|
| 33 | Note that such modifications are so specific to a given configuration that no attempt |
---|
| 34 | has been made to set them in a generic way. However, examples of how |
---|
[2349] | 35 | they can be set up is given in the ORCA 2\deg and 0.5\deg configurations (search for |
---|
| 36 | \key{orca\_r2} or \key{orca\_r05} in the code). |
---|
[707] | 37 | |
---|
| 38 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 39 | % Hand made geometry changes |
---|
| 40 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 41 | \subsection{Hand made geometry changes} |
---|
| 42 | \label{MISC_strait_hand} |
---|
| 43 | |
---|
[2282] | 44 | $\bullet$ reduced scale factor in the cross-strait direction to a value in better agreement |
---|
| 45 | with the true mean width of the strait. (Fig.~\ref{Fig_MISC_strait_hand}). |
---|
| 46 | This technique is sometime called "partially open face" or "partially closed cells". |
---|
| 47 | The key issue here is only to reduce the faces of $T$-cell ($i.e.$ change the value |
---|
| 48 | of the horizontal scale factors at $u$- or $v$-point) but not the volume of the $T$-cell. |
---|
| 49 | Indeed, reducing the volume of strait $T$-cell can easily produce a numerical |
---|
| 50 | instability at that grid point that would require a reduction of the model time step. |
---|
| 51 | The changes associated with strait management are done in \mdl{domhgr}, |
---|
| 52 | just after the definition or reading of the horizontal scale factors. |
---|
[707] | 53 | |
---|
[2282] | 54 | $\bullet$ increase of the viscous boundary layer thickness by local increase of the |
---|
| 55 | fmask value at the coast (Fig.~\ref{Fig_MISC_strait_hand}). This is done in |
---|
| 56 | \mdl{dommsk} together with the setting of the coastal value of fmask |
---|
| 57 | (see Section \ref{LBC_coast}) |
---|
[994] | 58 | |
---|
[707] | 59 | %>>>>>>>>>>>>>>>>>>>>>>>>>>>> |
---|
[2376] | 60 | \begin{figure}[!tbp] \begin{center} |
---|
[998] | 61 | \includegraphics[width=0.80\textwidth]{./TexFiles/Figures/Fig_Gibraltar.pdf} |
---|
| 62 | \includegraphics[width=0.80\textwidth]{./TexFiles/Figures/Fig_Gibraltar2.pdf} |
---|
[2376] | 63 | \caption{ \label{Fig_MISC_strait_hand} |
---|
| 64 | Example of the Gibraltar strait defined in a $1\deg \times 1\deg$ mesh. |
---|
[994] | 65 | \textit{Top}: using partially open cells. The meridional scale factor at $v$-point |
---|
| 66 | is reduced on both sides of the strait to account for the real width of the strait |
---|
[1224] | 67 | (about 20 km). Note that the scale factors of the strait $T$-point remains unchanged. |
---|
| 68 | \textit{Bottom}: using viscous boundary layers. The four fmask parameters |
---|
[994] | 69 | along the strait coastlines are set to a value larger than 4, $i.e.$ "strong" no-slip |
---|
| 70 | case (see Fig.\ref{Fig_LBC_shlat}) creating a large viscous boundary layer |
---|
| 71 | that allows a reduced transport through the strait.} |
---|
[707] | 72 | \end{center} \end{figure} |
---|
| 73 | %>>>>>>>>>>>>>>>>>>>>>>>>>>>> |
---|
| 74 | |
---|
| 75 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 76 | % Cross Land Advection |
---|
| 77 | % ------------------------------------------------------------------------------------------------------------- |
---|
[2282] | 78 | \subsection{Cross Land Advection (\mdl{tracla})} |
---|
[707] | 79 | \label{MISC_strait_cla} |
---|
[1225] | 80 | %--------------------------------------------namcla-------------------------------------------------------- |
---|
| 81 | \namdisplay{namcla} |
---|
[707] | 82 | %-------------------------------------------------------------------------------------------------------------- |
---|
| 83 | |
---|
| 84 | \colorbox{yellow}{Add a short description of CLA staff here or in lateral boundary condition chapter?} |
---|
| 85 | |
---|
[2282] | 86 | %The problem is resolved here by allowing the mixing of tracers and mass/volume between non-adjacent water columns at nominated regions within the model. Momentum is not mixed. The scheme conserves total tracer content, and total volume (the latter in $z*$- or $s*$-coordinate), and maintains compatibility between the tracer and mass/volume budgets. |
---|
| 87 | |
---|
[707] | 88 | % ================================================================ |
---|
| 89 | % Closed seas |
---|
| 90 | % ================================================================ |
---|
[2282] | 91 | \section{Closed seas (\mdl{closea})} |
---|
[707] | 92 | \label{MISC_closea} |
---|
| 93 | |
---|
[2282] | 94 | \colorbox{yellow}{Add here a short description of the way closed seas are managed} |
---|
[707] | 95 | |
---|
[2282] | 96 | |
---|
[707] | 97 | % ================================================================ |
---|
| 98 | % Sub-Domain Functionality (\textit{nizoom, njzoom}, namelist parameters) |
---|
| 99 | % ================================================================ |
---|
| 100 | \section{Sub-Domain Functionality (\jp{jpizoom}, \jp{jpjzoom})} |
---|
| 101 | \label{MISC_zoom} |
---|
| 102 | |
---|
[994] | 103 | The sub-domain functionality, also improperly called the zoom option |
---|
| 104 | (improperly because it is not associated with a change in model resolution) |
---|
| 105 | is a quite simple function that allows a simulation over a sub-domain of an |
---|
| 106 | already defined configuration ($i.e.$ without defining a new mesh, initial |
---|
| 107 | state and forcings). This option can be useful for testing the user settings |
---|
| 108 | of surface boundary conditions, or the initial ocean state of a huge ocean |
---|
| 109 | model configuration while having a small computer memory requirement. |
---|
| 110 | It can also be used to easily test specific physics in a sub-domain (for example, |
---|
[2282] | 111 | see \citep{Madec_al_JPO96} for a test of the coupling used in the global ocean |
---|
[994] | 112 | version of OPA between sea-ice and ocean model over the Arctic or Antarctic |
---|
| 113 | ocean, using a sub-domain). In the standard model, this option does not |
---|
| 114 | include any specific treatment for the ocean boundaries of the sub-domain: |
---|
| 115 | they are considered as artificial vertical walls. Nevertheless, it is quite easy |
---|
| 116 | to add a restoring term toward a climatology in the vicinity of such boundaries |
---|
| 117 | (see \S\ref{TRA_dmp}). |
---|
[707] | 118 | |
---|
| 119 | In order to easily define a sub-domain over which the computation can be |
---|
| 120 | performed, the dimension of all input arrays (ocean mesh, bathymetry, |
---|
[994] | 121 | forcing, initial state, ...) are defined as \jp{jpidta}, \jp{jpjdta} and \jp{jpkdta} |
---|
| 122 | (\mdl{par\_oce} module), while the computational domain is defined through |
---|
| 123 | \jp{jpiglo}, \jp{jpjglo} and \jp{jpk} (\mdl{par\_oce} module). When running the |
---|
| 124 | model over the whole domain, the user sets \jp{jpiglo}=\jp{jpidta} \jp{jpjglo}=\jp{jpjdta} |
---|
| 125 | and \jp{jpk}=\jp{jpkdta}. When running the model over a sub-domain, the user |
---|
| 126 | has to provide the size of the sub-domain, (\jp{jpiglo}, \jp{jpjglo}, \jp{jpkglo}), |
---|
| 127 | and the indices of the south western corner as \jp{jpizoom} and \jp{jpjzoom} in |
---|
| 128 | the \mdl{par\_oce} module (Fig.~\ref{Fig_LBC_zoom}). |
---|
[707] | 129 | |
---|
[994] | 130 | Note that a third set of dimensions exist, \jp{jpi}, \jp{jpj} and \jp{jpk} which is |
---|
| 131 | actually used to perform the computation. It is set by default to \jp{jpi}=\jp{jpjglo} |
---|
| 132 | and \jp{jpj}=\jp{jpjglo}, except for massively parallel computing where the |
---|
| 133 | computational domain is laid out on local processor memories following a 2D |
---|
| 134 | horizontal splitting. % (see {\S}IV.2-c) ref to the section to be updated |
---|
[707] | 135 | |
---|
| 136 | %>>>>>>>>>>>>>>>>>>>>>>>>>>>> |
---|
[2376] | 137 | \begin{figure}[!ht] \begin{center} |
---|
[998] | 138 | \includegraphics[width=0.90\textwidth]{./TexFiles/Figures/Fig_LBC_zoom.pdf} |
---|
[2376] | 139 | \caption{ \label{Fig_LBC_zoom} |
---|
| 140 | Position of a model domain compared to the data input domain when the zoom functionality is used.} |
---|
[707] | 141 | \end{center} \end{figure} |
---|
| 142 | %>>>>>>>>>>>>>>>>>>>>>>>>>>>> |
---|
| 143 | |
---|
| 144 | |
---|
| 145 | % ================================================================ |
---|
| 146 | % Accelerating the Convergence |
---|
| 147 | % ================================================================ |
---|
| 148 | \section{Accelerating the Convergence (\np{nn\_acc} = 1)} |
---|
| 149 | \label{MISC_acc} |
---|
| 150 | %--------------------------------------------namdom------------------------------------------------------- |
---|
| 151 | \namdisplay{namdom} |
---|
| 152 | %-------------------------------------------------------------------------------------------------------------- |
---|
| 153 | |
---|
[2282] | 154 | Searching an equilibrium state with an global ocean model requires a very long time |
---|
| 155 | integration period (a few thousand years for a global model). Due to the size of |
---|
| 156 | the time step required for numerical stability (less than a few hours), |
---|
| 157 | this usually requires a large elapsed time. In order to overcome this problem, |
---|
| 158 | \citet{Bryan1984} introduces a technique that is intended to accelerate |
---|
[994] | 159 | the spin up to equilibrium. It uses a larger time step in |
---|
[2282] | 160 | the tracer evolution equations than in the momentum evolution |
---|
[707] | 161 | equations. It does not affect the equilibrium solution but modifies the |
---|
| 162 | trajectory to reach it. |
---|
| 163 | |
---|
[994] | 164 | The acceleration of convergence option is used when \np{nn\_acc}=1. In that case, |
---|
[2282] | 165 | $\rdt=rn\_rdt$ is the time step of dynamics while $\widetilde{\rdt}=rdttra$ is the |
---|
| 166 | tracer time-step. the former is set from the \np{rn\_rdt} namelist parameter while the latter |
---|
| 167 | is computed using a hyperbolic tangent profile and the following namelist parameters : |
---|
| 168 | \np{rn\_rdtmin}, \np{rn\_rdtmax} and \np{rn\_rdth}. Those three parameters correspond |
---|
| 169 | to the surface value the deep ocean value and the depth at which the transition occurs, respectively. |
---|
| 170 | The set of prognostic equations to solve becomes: |
---|
[707] | 171 | \begin{equation} \label{Eq_acc} |
---|
| 172 | \begin{split} |
---|
| 173 | \frac{\partial \textbf{U}_h }{\partial t} |
---|
[2282] | 174 | &\equiv \frac{\textbf{U}_h ^{t+1}-\textbf{U}_h^{t-1} }{2\rdt} = \ldots \\ |
---|
| 175 | \frac{\partial T}{\partial t} &\equiv \frac{T^{t+1}-T^{t-1}}{2 \widetilde{\rdt}} = \ldots \\ |
---|
| 176 | \frac{\partial S}{\partial t} &\equiv \frac{S^{t+1} -S^{t-1}}{2 \widetilde{\rdt}} = \ldots \\ |
---|
[707] | 177 | \end{split} |
---|
| 178 | \end{equation} |
---|
| 179 | |
---|
[994] | 180 | \citet{Bryan1984} has examined the consequences of this distorted physics. |
---|
| 181 | Free waves have a slower phase speed, their meridional structure is slightly |
---|
[707] | 182 | modified, and the growth rate of baroclinically unstable waves is reduced |
---|
[994] | 183 | but with a wider range of instability. This technique is efficient for |
---|
| 184 | searching for an equilibrium state in coarse resolution models. However its |
---|
[707] | 185 | application is not suitable for many oceanic problems: it cannot be used for |
---|
| 186 | transient or time evolving problems (in particular, it is very questionable |
---|
[994] | 187 | to use this technique when there is a seasonal cycle in the forcing fields), |
---|
[707] | 188 | and it cannot be used in high-resolution models where baroclinically |
---|
| 189 | unstable processes are important. Moreover, the vertical variation of |
---|
[2282] | 190 | $\widetilde{ \rdt}$ implies that the heat and salt contents are no longer |
---|
[707] | 191 | conserved due to the vertical coupling of the ocean level through both |
---|
[2282] | 192 | advection and diffusion. Therefore \np{rn\_rdtmin} = \np{rn\_rdtmax} should be |
---|
| 193 | a more clever choice. |
---|
[707] | 194 | |
---|
| 195 | % ================================================================ |
---|
| 196 | % Model optimisation, Control Print and Benchmark |
---|
| 197 | % ================================================================ |
---|
[994] | 198 | \section{Model Optimisation, Control Print and Benchmark} |
---|
[707] | 199 | \label{MISC_opt} |
---|
[1225] | 200 | %--------------------------------------------namctl------------------------------------------------------- |
---|
| 201 | \namdisplay{namctl} |
---|
[707] | 202 | %-------------------------------------------------------------------------------------------------------------- |
---|
| 203 | |
---|
[994] | 204 | % \gmcomment{why not make these bullets into subsections?} |
---|
[707] | 205 | |
---|
[994] | 206 | |
---|
[2349] | 207 | $\bullet$ Vector optimisation: |
---|
[707] | 208 | |
---|
[2282] | 209 | \key{vectopt\_loop} enables the internal loops to collapse. This is very |
---|
| 210 | a very efficient way to increase the length of vector calculations and thus |
---|
| 211 | to speed up the model on vector computers. |
---|
[707] | 212 | |
---|
[994] | 213 | % Add here also one word on NPROMA technique that has been found useless, since compiler have made significant progress during the last decade. |
---|
[707] | 214 | |
---|
[994] | 215 | % Add also one word on NEC specific optimisation (Novercheck option for example) |
---|
[707] | 216 | |
---|
[994] | 217 | $\bullet$ Control print %: describe here 4 things: |
---|
[707] | 218 | |
---|
[994] | 219 | 1- \np{ln\_ctl} : compute and print the trends averaged over the interior domain |
---|
| 220 | in all TRA, DYN, LDF and ZDF modules. This option is very helpful when |
---|
| 221 | diagnosing the origin of an undesired change in model results. |
---|
[707] | 222 | |
---|
[994] | 223 | 2- also \np{ln\_ctl} but using the nictl and njctl namelist parameters to check |
---|
| 224 | the source of differences between mono and multi processor runs. |
---|
[707] | 225 | |
---|
[994] | 226 | 3- \key{esopa} (to be rename key\_nemo) : which is another option for model |
---|
| 227 | management. When defined, this key forces the activation of all options and |
---|
| 228 | CPP keys. For example, all tracer and momentum advection schemes are called! |
---|
[2282] | 229 | Therefore the model results have no physical meaning. |
---|
[994] | 230 | However, this option forces both the compiler and the model to run through |
---|
| 231 | all the \textsc{Fortran} lines of the model. This allows the user to check for obvious |
---|
| 232 | compilation or execution errors with all CPP options, and errors in namelist options. |
---|
[707] | 233 | |
---|
[2282] | 234 | 4- last digit comparison (\np{nn\_bit\_cmp}). In an MPP simulation, the computation of |
---|
[994] | 235 | a sum over the whole domain is performed as the summation over all processors of |
---|
| 236 | each of their sums over their interior domains. This double sum never gives exactly |
---|
| 237 | the same result as a single sum over the whole domain, due to truncation differences. |
---|
| 238 | The "bit comparison" option has been introduced in order to be able to check that |
---|
| 239 | mono-processor and multi-processor runs give exactly the same results. |
---|
[2376] | 240 | %THIS is to be updated with the mpp_sum_glo introduced in v3.3 |
---|
| 241 | % nn_bit_cmp today only check that the nn_cla = 0 (no cross land advection) |
---|
[707] | 242 | |
---|
[2282] | 243 | $\bullet$ Benchmark (\np{nn\_bench}). This option defines a benchmark run based on |
---|
[2376] | 244 | a GYRE configuration (see \S\ref{CFG_gyre}) in which the resolution remains the same |
---|
| 245 | whatever the domain size. This allows a very large model domain to be used, just by |
---|
| 246 | changing the domain size (\jp{jpiglo}, \jp{jpjglo}) and without adjusting either the time-step |
---|
| 247 | or the physical parameterisations. |
---|
[707] | 248 | |
---|
| 249 | |
---|
| 250 | % ================================================================ |
---|
| 251 | % Elliptic solvers (SOL) |
---|
| 252 | % ================================================================ |
---|
[2282] | 253 | \section{Elliptic solvers (SOL)} |
---|
[707] | 254 | \label{MISC_sol} |
---|
| 255 | %--------------------------------------------namdom------------------------------------------------------- |
---|
| 256 | \namdisplay{namsol} |
---|
| 257 | %-------------------------------------------------------------------------------------------------------------- |
---|
| 258 | |
---|
[2282] | 259 | When the filtered sea surface height option is used, the surface pressure gradient is |
---|
| 260 | computed in \mdl{dynspg\_flt}. The force added in the momentum equation is solved implicitely. |
---|
| 261 | It is thus solution of an elliptic equation \eqref{Eq_PE_flt} for which two solvers are available: |
---|
| 262 | a Successive-Over-Relaxation scheme (SOR) and a preconditioned conjugate gradient |
---|
| 263 | scheme(PCG) \citep{Madec_al_OM88, Madec_PhD90}. The solver is selected trough the |
---|
| 264 | the value of \np{nn\_solv} (namelist parameter). |
---|
| 265 | |
---|
[994] | 266 | The PCG is a very efficient method for solving elliptic equations on vector computers. |
---|
| 267 | It is a fast and rather easy method to use; which are attractive features for a large |
---|
| 268 | number of ocean situations (variable bottom topography, complex coastal geometry, |
---|
| 269 | variable grid spacing, islands, open or cyclic boundaries, etc ...). It does not require |
---|
| 270 | a search for an optimal parameter as in the SOR method. However, the SOR has |
---|
| 271 | been retained because it is a linear solver, which is a very useful property when |
---|
[2282] | 272 | using the adjoint model of \NEMO. |
---|
[707] | 273 | |
---|
[2282] | 274 | At each time step, the time derivative of the sea surface height at time step $t+1$ |
---|
| 275 | (or equivalently the divergence of the \textit{after} barotropic transport) that appears |
---|
| 276 | in the filtering forced is the solution of the elliptic equation obtained from the horizontal |
---|
| 277 | divergence of the vertical summation of \eqref{Eq_PE_flt}. |
---|
| 278 | Introducing the following coefficients: |
---|
| 279 | \begin{equation} \label{Eq_sol_matrix} |
---|
[707] | 280 | \begin{aligned} |
---|
[2282] | 281 | &c_{i,j}^{NS} &&= {2 \rdt }^2 \; \frac{H_v (i,j) \; e_{1v} (i,j)}{e_{2v}(i,j)} \\ |
---|
| 282 | &c_{i,j}^{EW} &&= {2 \rdt }^2 \; \frac{H_u (i,j) \; e_{2u} (i,j)}{e_{1u}(i,j)} \\ |
---|
| 283 | &b_{i,j} &&= \delta_i \left[ e_{2u}M_u \right] - \delta_j \left[ e_{1v}M_v \right]\ , \\ |
---|
[707] | 284 | \end{aligned} |
---|
| 285 | \end{equation} |
---|
[994] | 286 | the five-point finite difference equation \eqref{Eq_psi_total} can be rewritten as: |
---|
[2282] | 287 | \begin{equation} \label{Eq_solmat} |
---|
| 288 | \begin{split} |
---|
| 289 | c_{i+1,j}^{NS} D_{i+1,j} + \; c_{i,j+1}^{EW} D_{i,j+1} |
---|
| 290 | + c_{i,j} ^{NS} D_{i-1,j} + \; c_{i,j} ^{EW} D_{i,j-1} & \\ |
---|
| 291 | - \left(c_{i+1,j}^{NS} + c_{i,j+1}^{EW} + c_{i,j}^{NS} + c_{i,j}^{EW} \right) D_{i,j} &= b_{i,j} |
---|
| 292 | \end{split} |
---|
| 293 | \end{equation} |
---|
[996] | 294 | \eqref{Eq_solmat} is a linear symmetric system of equations. All the elements of |
---|
| 295 | the corresponding matrix \textbf{A} vanish except those of five diagonals. With |
---|
[707] | 296 | the natural ordering of the grid points (i.e. from west to east and from |
---|
| 297 | south to north), the structure of \textbf{A} is block-tridiagonal with |
---|
| 298 | tridiagonal or diagonal blocks. \textbf{A} is a positive-definite symmetric |
---|
| 299 | matrix of size $(jpi \cdot jpj)^2$, and \textbf{B}, the right hand side of |
---|
| 300 | \eqref{Eq_solmat}, is a vector. |
---|
| 301 | |
---|
[2282] | 302 | Note that in the linear free surface case, the depth that appears in \eqref{Eq_sol_matrix} |
---|
| 303 | does not vary with time, and thus the matrix can be computed once for all. In non-linear free surface |
---|
| 304 | (\key{vvl} defined) the matrix have to be updated at each time step. |
---|
| 305 | |
---|
[707] | 306 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 307 | % Successive Over Relaxation |
---|
| 308 | % ------------------------------------------------------------------------------------------------------------- |
---|
[2282] | 309 | \subsection{Successive Over Relaxation (\np{nn\_solv}=2, \mdl{solsor})} |
---|
[707] | 310 | \label{MISC_solsor} |
---|
| 311 | |
---|
[2282] | 312 | Let us introduce the four cardinal coefficients: |
---|
| 313 | \begin{align*} |
---|
| 314 | a_{i,j}^S &= c_{i,j }^{NS}/d_{i,j} &\qquad a_{i,j}^W &= c_{i,j}^{EW}/d_{i,j} \\ |
---|
| 315 | a_{i,j}^E &= c_{i,j+1}^{EW}/d_{i,j} &\qquad a_{i,j}^N &= c_{i+1,j}^{NS}/d_{i,j} |
---|
| 316 | \end{align*} |
---|
| 317 | where $d_{i,j} = c_{i,j}^{NS}+ c_{i+1,j}^{NS} + c_{i,j}^{EW} + c_{i,j+1}^{EW}$ |
---|
| 318 | (i.e. the diagonal of the matrix). \eqref{Eq_solmat} can be rewritten as: |
---|
| 319 | \begin{equation} \label{Eq_solmat_p} |
---|
| 320 | \begin{split} |
---|
| 321 | a_{i,j}^{N} D_{i+1,j} +\,a_{i,j}^{E} D_{i,j+1} +\, a_{i,j}^{S} D_{i-1,j} +\,a_{i,j}^{W} D_{i,j-1} - D_{i,j} = \tilde{b}_{i,j} |
---|
| 322 | \end{split} |
---|
| 323 | \end{equation} |
---|
| 324 | with $\tilde b_{i,j} = b_{i,j}/d_{i,j}$. \eqref{Eq_solmat_p} is the equation actually solved |
---|
| 325 | with the SOR method. This method used is an iterative one. Its algorithm can be |
---|
| 326 | summarised as follows (see \citet{Haltiner1980} for a further discussion): |
---|
[707] | 327 | |
---|
[994] | 328 | initialisation (evaluate a first guess from previous time step computations) |
---|
[707] | 329 | \begin{equation} |
---|
[2282] | 330 | D_{i,j}^0 = 2 \, D_{i,j}^t - D_{i,j}^{t-1} |
---|
[707] | 331 | \end{equation} |
---|
[994] | 332 | iteration $n$, from $n=0$ until convergence, do : |
---|
[2282] | 333 | \begin{equation} \label{Eq_sor_algo} |
---|
| 334 | \begin{split} |
---|
| 335 | R_{i,j}^n = &a_{i,j}^{N} D_{i+1,j}^n +\,a_{i,j}^{E} D_{i,j+1} ^n |
---|
| 336 | +\, a_{i,j}^{S} D_{i-1,j} ^{n+1}+\,a_{i,j}^{W} D_{i,j-1} ^{n+1} |
---|
| 337 | - D_{i,j}^n - \tilde{b}_{i,j} \\ |
---|
| 338 | D_{i,j} ^{n+1} = &D_{i,j} ^{n} + \omega \;R_{i,j}^n |
---|
| 339 | \end{split} |
---|
[707] | 340 | \end{equation} |
---|
[994] | 341 | where \textit{$\omega $ }satisfies $1\leq \omega \leq 2$. An optimal value exists for |
---|
| 342 | \textit{$\omega$} which significantly accelerates the convergence, but it has to be |
---|
| 343 | adjusted empirically for each model domain (except for a uniform grid where an |
---|
| 344 | analytical expression for \textit{$\omega$} can be found \citep{Richtmyer1967}). |
---|
[2282] | 345 | The value of $\omega$ is set using \np{rn\_sor}, a \textbf{namelist} parameter. |
---|
[994] | 346 | The convergence test is of the form: |
---|
[707] | 347 | \begin{equation} |
---|
[2282] | 348 | \delta = \frac{\sum\limits_{i,j}{R_{i,j}^n}{R_{i,j}^n}} |
---|
| 349 | {\sum\limits_{i,j}{ \tilde{b}_{i,j}^n}{\tilde{b}_{i,j}^n}} \leq \epsilon |
---|
[707] | 350 | \end{equation} |
---|
[994] | 351 | where $\epsilon$ is the absolute precision that is required. It is recommended |
---|
[2282] | 352 | that a value smaller or equal to $10^{-6}$ is used for $\epsilon$ since larger |
---|
| 353 | values may lead to numerically induced basin scale barotropic oscillations. |
---|
| 354 | The precision is specified by setting \np{rn\_eps} (\textbf{namelist} parameter). |
---|
| 355 | In addition, two other tests are used to halt the iterative algorithm. They involve |
---|
| 356 | the number of iterations and the modulus of the right hand side. If the former |
---|
| 357 | exceeds a specified value, \np{nn\_max} (\textbf{namelist} parameter), |
---|
| 358 | or the latter is greater than $10^{15}$, the whole model computation is stopped |
---|
| 359 | and the last computed time step fields are saved in a abort.nc NetCDF file. |
---|
| 360 | In both cases, this usually indicates that there is something wrong in the model |
---|
| 361 | configuration (an error in the mesh, the initial state, the input forcing, |
---|
[994] | 362 | or the magnitude of the time step or of the mixing coefficients). A typical value of |
---|
[2282] | 363 | $nn\_max$ is a few hundred when $\epsilon = 10^{-6}$, increasing to a few |
---|
[994] | 364 | thousand when $\epsilon = 10^{-12}$. |
---|
| 365 | The vectorization of the SOR algorithm is not straightforward. The scheme |
---|
| 366 | contains two linear recurrences on $i$ and $j$. This inhibits the vectorisation. |
---|
[2282] | 367 | \eqref{Eq_sor_algo} can be been rewritten as: |
---|
| 368 | \begin{equation} |
---|
| 369 | \begin{split} |
---|
[707] | 370 | R_{i,j}^n |
---|
[2282] | 371 | = &a_{i,j}^{N} D_{i+1,j}^n +\,a_{i,j}^{E} D_{i,j+1} ^n |
---|
| 372 | +\,a_{i,j}^{S} D_{i-1,j} ^{n}+\,_{i,j}^{W} D_{i,j-1} ^{n} - D_{i,j}^n - \tilde{b}_{i,j} \\ |
---|
| 373 | R_{i,j}^n = &R_{i,j}^n - \omega \;a_{i,j}^{S}\; R_{i,j-1}^n \\ |
---|
| 374 | R_{i,j}^n = &R_{i,j}^n - \omega \;a_{i,j}^{W}\; R_{i-1,j}^n |
---|
| 375 | \end{split} |
---|
[707] | 376 | \end{equation} |
---|
[2282] | 377 | This technique slightly increases the number of iteration required to reach the convergence, |
---|
| 378 | but this is largely compensated by the gain obtained by the suppression of the recurrences. |
---|
[707] | 379 | |
---|
[2282] | 380 | Another technique have been chosen, the so-called red-black SOR. It consist in solving successively |
---|
| 381 | \eqref{Eq_sor_algo} for odd and even grid points. It also slightly reduced the convergence rate |
---|
| 382 | but allows the vectorisation. In addition, and this is the reason why it has been chosen, it is able to handle the north fold boundary condition used in ORCA configuration ($i.e.$ tri-polar global ocean mesh). |
---|
[707] | 383 | |
---|
[2282] | 384 | The SOR method is very flexible and can be used under a wide range of conditions, |
---|
| 385 | including irregular boundaries, interior boundary points, etc. Proofs of convergence, etc. |
---|
| 386 | may be found in the standard numerical methods texts for partial differential equations. |
---|
[707] | 387 | |
---|
| 388 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 389 | % Preconditioned Conjugate Gradient |
---|
| 390 | % ------------------------------------------------------------------------------------------------------------- |
---|
[2282] | 391 | \subsection{Preconditioned Conjugate Gradient (\np{nn\_solv}=1, \mdl{solpcg}) } |
---|
[707] | 392 | \label{MISC_solpcg} |
---|
| 393 | |
---|
| 394 | \textbf{A} is a definite positive symmetric matrix, thus solving the linear |
---|
[994] | 395 | system \eqref{Eq_solmat} is equivalent to the minimisation of a quadratic |
---|
[707] | 396 | functional: |
---|
| 397 | \begin{equation*} |
---|
| 398 | \textbf{Ax} = \textbf{b} \leftrightarrow \textbf{x} =\text{inf}_{y} \,\phi (\textbf{y}) |
---|
| 399 | \quad , \qquad |
---|
| 400 | \phi (\textbf{y}) = 1/2 \langle \textbf{Ay},\textbf{y}\rangle - \langle \textbf{b},\textbf{y} \rangle |
---|
| 401 | \end{equation*} |
---|
| 402 | where $\langle , \rangle$ is the canonical dot product. The idea of the |
---|
[994] | 403 | conjugate gradient method is to search for the solution in the following |
---|
| 404 | iterative way: assuming that $\textbf{x}^n$ has been obtained, $\textbf{x}^{n+1}$ |
---|
| 405 | is found from $\textbf {x}^{n+1}={\textbf {x}}^n+\alpha^n{\textbf {d}}^n$ which satisfies: |
---|
[707] | 406 | \begin{equation*} |
---|
[994] | 407 | {\textbf{ x}}^{n+1}=\text{inf} _{{\textbf{ y}}={\textbf{ x}}^n+\alpha^n \,{\textbf{ d}}^n} \,\phi ({\textbf{ y}})\;\;\Leftrightarrow \;\;\frac{d\phi }{d\alpha}=0 |
---|
[707] | 408 | \end{equation*} |
---|
[1224] | 409 | and expressing $\phi (\textbf{y})$ as a function of \textit{$\alpha $}, we obtain the |
---|
| 410 | value that minimises the functional: |
---|
[707] | 411 | \begin{equation*} |
---|
| 412 | \alpha ^n = \langle{ \textbf{r}^n , \textbf{r}^n} \rangle / \langle {\textbf{ A d}^n, \textbf{d}^n} \rangle |
---|
| 413 | \end{equation*} |
---|
[994] | 414 | where $\textbf{r}^n = \textbf{b}-\textbf{A x}^n = \textbf{A} (\textbf{x}-\textbf{x}^n)$ |
---|
| 415 | is the error at rank $n$. The descent vector $\textbf{d}^n$ s chosen to be dependent |
---|
| 416 | on the error: $\textbf{d}^n = \textbf{r}^n + \beta^n \,\textbf{d}^{n-1}$. $\beta ^n$ |
---|
| 417 | is searched such that the descent vectors form an orthogonal basis for the dot |
---|
| 418 | product linked to \textbf{A}. Expressing the condition |
---|
[707] | 419 | $\langle \textbf{A d}^n, \textbf{d}^{n-1} \rangle = 0$ the value of $\beta ^n$ is found: |
---|
[1224] | 420 | $\beta ^n = \langle{ \textbf{r}^n , \textbf{r}^n} \rangle / \langle {\textbf{r}^{n-1}, \textbf{r}^{n-1}} \rangle$. |
---|
| 421 | As a result, the errors $ \textbf{r}^n$ form an orthogonal |
---|
[994] | 422 | base for the canonic dot product while the descent vectors $\textbf{d}^n$ form |
---|
[707] | 423 | an orthogonal base for the dot product linked to \textbf{A}. The resulting |
---|
| 424 | algorithm is thus the following one: |
---|
| 425 | |
---|
| 426 | initialisation : |
---|
| 427 | \begin{equation*} |
---|
[2282] | 428 | \begin{split} |
---|
| 429 | \textbf{x}^0 &= D_{i,j}^0 = 2 D_{i,j}^t - D_{i,j}^{t-1} \quad, \text{the initial guess } \\ |
---|
| 430 | \textbf{r}^0 &= \textbf{d}^0 = \textbf{b} - \textbf{A x}^0 \\ |
---|
| 431 | \gamma_0 &= \langle{ \textbf{r}^0 , \textbf{r}^0} \rangle |
---|
| 432 | \end{split} |
---|
[707] | 433 | \end{equation*} |
---|
| 434 | |
---|
| 435 | iteration $n,$ from $n=0$ until convergence, do : |
---|
| 436 | \begin{equation} |
---|
| 437 | \begin{split} |
---|
| 438 | \text{z}^n& = \textbf{A d}^n \\ |
---|
[2282] | 439 | \alpha_n &= \gamma_n / \langle{ \textbf{z}^n , \textbf{d}^n} \rangle \\ |
---|
[707] | 440 | \textbf{x}^{n+1} &= \textbf{x}^n + \alpha_n \,\textbf{d}^n \\ |
---|
| 441 | \textbf{r}^{n+1} &= \textbf{r}^n - \alpha_n \,\textbf{z}^n \\ |
---|
| 442 | \gamma_{n+1} &= \langle{ \textbf{r}^{n+1} , \textbf{r}^{n+1}} \rangle \\ |
---|
| 443 | \beta_{n+1} &= \gamma_{n+1}/\gamma_{n} \\ |
---|
| 444 | \textbf{d}^{n+1} &= \textbf{r}^{n+1} + \beta_{n+1}\; \textbf{d}^{n}\\ |
---|
| 445 | \end{split} |
---|
| 446 | \end{equation} |
---|
| 447 | |
---|
| 448 | |
---|
| 449 | The convergence test is: |
---|
| 450 | \begin{equation} |
---|
| 451 | \delta = \gamma_{n}\; / \langle{ \textbf{b} , \textbf{b}} \rangle \leq \epsilon |
---|
| 452 | \end{equation} |
---|
| 453 | where $\epsilon $ is the absolute precision that is required. As for the SOR algorithm, |
---|
[2282] | 454 | the whole model computation is stopped when the number of iterations, \np{nn\_max}, or |
---|
[994] | 455 | the modulus of the right hand side of the convergence equation exceeds a |
---|
| 456 | specified value (see \S\ref{MISC_solsor} for a further discussion). The required |
---|
| 457 | precision and the maximum number of iterations allowed are specified by setting |
---|
[2282] | 458 | \np{rn\_eps} and \np{nn\_max} (\textbf{namelist} parameters). |
---|
[707] | 459 | |
---|
[994] | 460 | It can be demonstrated that the above algorithm is optimal, provides the exact |
---|
[707] | 461 | solution in a number of iterations equal to the size of the matrix, and that |
---|
[994] | 462 | the convergence rate is faster as the matrix is closer to the identity matrix, |
---|
[2282] | 463 | $i.e.$ its eigenvalues are closer to 1. Therefore, it is more efficient to solve |
---|
| 464 | a better conditioned system which has the same solution. For that purpose, |
---|
| 465 | we introduce a preconditioning matrix \textbf{Q} which is an approximation |
---|
| 466 | of \textbf{A} but much easier to invert than \textbf{A}, and solve the system: |
---|
[994] | 467 | \begin{equation} \label{Eq_pmat} |
---|
[707] | 468 | \textbf{Q}^{-1} \textbf{A x} = \textbf{Q}^{-1} \textbf{b} |
---|
| 469 | \end{equation} |
---|
| 470 | |
---|
[994] | 471 | The same algorithm can be used to solve \eqref{Eq_pmat} if instead of the |
---|
| 472 | canonical dot product the following one is used: |
---|
[1224] | 473 | ${\langle{ \textbf{a} , \textbf{b}} \rangle}_Q = \langle{ \textbf{a} , \textbf{Q b}} \rangle$, and |
---|
| 474 | if $\textbf{\~{b}} = \textbf{Q}^{-1}\;\textbf{b}$ and $\textbf{\~{A}} = \textbf{Q}^{-1}\;\textbf{A}$ |
---|
[2282] | 475 | are substituted to \textbf{b} and \textbf{A} \citep{Madec_al_OM88}. |
---|
[1224] | 476 | In \NEMO, \textbf{Q} is chosen as the diagonal of \textbf{ A}, i.e. the simplest form for |
---|
| 477 | \textbf{Q} so that it can be easily inverted. In this case, the discrete formulation of |
---|
[994] | 478 | \eqref{Eq_pmat} is in fact given by \eqref{Eq_solmat_p} and thus the matrix and |
---|
| 479 | right hand side are computed independently from the solver used. |
---|
[707] | 480 | |
---|
| 481 | % ================================================================ |
---|
| 482 | % Diagnostics |
---|
| 483 | % ================================================================ |
---|
[2349] | 484 | \section{Diagnostics (DIA, IOM, TRD, FLO)} |
---|
[707] | 485 | \label{MISC_diag} |
---|
| 486 | |
---|
| 487 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 488 | % Standard Model Output |
---|
| 489 | % ------------------------------------------------------------------------------------------------------------- |
---|
[2364] | 490 | \subsection{Model Output (default or \key{iomput} or \key{dimgout} or \key{netcdf4})} |
---|
[707] | 491 | \label{MISC_iom} |
---|
| 492 | |
---|
| 493 | %to be updated with Seb documentation on the IO |
---|
[2282] | 494 | |
---|
[707] | 495 | The model outputs are of three types: the restart file, the output listing, |
---|
| 496 | and the output file(s). The restart file is used internally by the code when |
---|
| 497 | the user wants to start the model with initial conditions defined by a |
---|
[994] | 498 | previous simulation. It contains all the information that is necessary in |
---|
| 499 | order for there to be no changes in the model results (even at the computer |
---|
| 500 | precision) between a run performed with several restarts and the same run |
---|
| 501 | performed in one step. It should be noted that this requires that the restart file |
---|
| 502 | contain two consecutive time steps for all the prognostic variables, and |
---|
| 503 | that it is saved in the same binary format as the one used by the computer |
---|
| 504 | that is to read it (in particular, 32 bits binary IEEE format must not be used for |
---|
| 505 | this file). The output listing and file(s) are predefined but should be checked |
---|
[707] | 506 | and eventually adapted to the user's needs. The output listing is stored in |
---|
[994] | 507 | the $ocean.output$ file. The information is printed from within the code on the |
---|
| 508 | logical unit $numout$. To locate these prints, use the UNIX command |
---|
[2282] | 509 | "\textit{grep -i numout}" in the source code directory. |
---|
[707] | 510 | |
---|
[2282] | 511 | In the standard configuration, the user will find the model results in |
---|
| 512 | NetCDF files containing mean values (or instantaneous values if |
---|
| 513 | \key{diainstant} is defined) for every time-step where output is demanded. |
---|
| 514 | These outputs are defined in the \mdl{diawri} module. |
---|
| 515 | When defining \key{dimgout}, the output are written in DIMG format, |
---|
| 516 | an IEEE output format. |
---|
[707] | 517 | |
---|
[2364] | 518 | Since version 3.3, support for NetCDF4 chunking and (loss-less) compression has |
---|
| 519 | been included. These options build on the standard NetCDF output and allow |
---|
| 520 | the user control over the size of the chunks via namelist settings. Chunking |
---|
| 521 | and compression can lead to significant reductions in file sizes for a small |
---|
| 522 | runtime overhead. For a fuller discussion on chunking and other performance |
---|
| 523 | issues the reader is referred to the NetCDF4 documentation: |
---|
| 524 | http://www.unidata.ucar.edu/software/netcdf/docs/netcdf.html\#Chunking |
---|
| 525 | |
---|
| 526 | The new features are only available when the code has been linked with a |
---|
| 527 | NetCDF4 library (version 4.1 onwards, recommended) which has been built |
---|
| 528 | with HDF5 support (version 1.8.4 onwards, recommended). Datasets created |
---|
| 529 | with chunking and compression are not backwards compatible with NetCDF3 |
---|
| 530 | "classic" format but most analysis codes can be relinked simply with the |
---|
| 531 | new libraries and will then read both NetCDF3 and NetCDF4 files. NEMO |
---|
| 532 | executables linked with NetCDF4 libraries can be made to produce NetCDF3 |
---|
| 533 | files by setting the \np{ln\_nc4zip} logical to false in the \np{namnc4} |
---|
| 534 | namelist: |
---|
| 535 | |
---|
| 536 | %------------------------------------------namnc4---------------------------------------------------- |
---|
| 537 | \namdisplay{namnc4} |
---|
| 538 | %------------------------------------------------------------------------------------------------------------- |
---|
| 539 | |
---|
| 540 | If \key{netcdf4} has not been defined, these namelist parameters are not read. |
---|
| 541 | In this case, \np{ln\_nc4zip} is set false and dummy routines for a few |
---|
| 542 | NetCDF4-specific functions are defined. These functions will not be used but |
---|
| 543 | need to be included so that compilation is possible with NetCDF3 libraries. |
---|
| 544 | |
---|
| 545 | When using NetCDF4 libraries, \key{netcdf4} should be defined even if the |
---|
| 546 | intention is to create only NetCDF3-compatible files. This is necessary to |
---|
| 547 | avoid duplication between the dummy routines and the actual routines present |
---|
| 548 | in the library. Most compilers will fail at compile time when faced with |
---|
| 549 | such duplication. Thus when linking with NetCDF4 libraries the user must |
---|
| 550 | define \key{netcdf4} and control the type of NetCDF file produced via the |
---|
| 551 | namelist parameter. |
---|
| 552 | |
---|
| 553 | Chunking and compression is applied only to 4D fields and there is no |
---|
| 554 | advantage in chunking across more than one time dimension since previously |
---|
| 555 | written chunks would have to be read back and decompressed before being |
---|
| 556 | added to. Therefore, user control over chunk sizes is provided only for the |
---|
| 557 | three space dimensions. The user sets an approximate number of chunks along |
---|
| 558 | each spatial axis. The actual size of the chunks will depend on global domain |
---|
| 559 | size for mono-processors or, more likely, the local processor domain size for |
---|
| 560 | distributed processing. The derived values are subject to practical minimum |
---|
| 561 | values (to avoid wastefully small chunk sizes) and cannot be greater than the |
---|
| 562 | domain size in any dimension. The algorithm used is: |
---|
| 563 | |
---|
| 564 | \begin{alltt} {{\tiny |
---|
| 565 | \begin{verbatim} |
---|
| 566 | ichunksz(1) = MIN( idomain_size,MAX( (idomain_size-1)/nn_nchunks_i + 1 ,16 ) ) |
---|
| 567 | ichunksz(2) = MIN( jdomain_size,MAX( (jdomain_size-1)/nn_nchunks_j + 1 ,16 ) ) |
---|
| 568 | ichunksz(3) = MIN( kdomain_size,MAX( (kdomain_size-1)/nn_nchunks_k + 1 , 1 ) ) |
---|
| 569 | ichunksz(4) = 1 |
---|
| 570 | \end{verbatim} |
---|
| 571 | }}\end{alltt} |
---|
| 572 | |
---|
| 573 | \noindent As an example, setting: |
---|
| 574 | \vspace{-20pt} |
---|
| 575 | \begin{alltt} {{\tiny |
---|
| 576 | \begin{verbatim} |
---|
| 577 | nn_nchunks_i=4, nn_nchunks_j=4 and nn_nchunks_k=31 |
---|
| 578 | \end{verbatim} |
---|
| 579 | }}\end{alltt} \vspace{-10pt} |
---|
| 580 | |
---|
| 581 | \noindent for a standard ORCA2\_LIM configuration gives chunksizes of {\small\tt 46x38x1} |
---|
| 582 | respectively in the mono-processor case (i.e. global domain of {\small\tt 182x149x31}). |
---|
| 583 | An illustration of the potential space savings that NetCDF4 chunking and compression |
---|
| 584 | provides is given in table \ref{Tab_NC4} which compares the results of two short |
---|
| 585 | runs of the ORCA2\_LIM reference configuration with a 4x2 mpi partitioning. Note |
---|
| 586 | the variation in the compression ratio achieved which reflects chiefly the dry to wet |
---|
| 587 | volume ratio of each processing region. |
---|
| 588 | |
---|
[2376] | 589 | %------------------------------------------TABLE---------------------------------------------------- |
---|
| 590 | \begin{table} \begin{tabular}{lrrr} |
---|
[2364] | 591 | Filename & NetCDF3 & NetCDF4 & Reduction\\ |
---|
| 592 | &filesize & filesize & \% \\ |
---|
| 593 | &(KB) & (KB) & \\ |
---|
| 594 | ORCA2\_restart\_0000.nc & 16420 & 8860 & 47\%\\ |
---|
| 595 | ORCA2\_restart\_0001.nc & 16064 & 11456 & 29\%\\ |
---|
| 596 | ORCA2\_restart\_0002.nc & 16064 & 9744 & 40\%\\ |
---|
| 597 | ORCA2\_restart\_0003.nc & 16420 & 9404 & 43\%\\ |
---|
| 598 | ORCA2\_restart\_0004.nc & 16200 & 5844 & 64\%\\ |
---|
| 599 | ORCA2\_restart\_0005.nc & 15848 & 8172 & 49\%\\ |
---|
| 600 | ORCA2\_restart\_0006.nc & 15848 & 8012 & 50\%\\ |
---|
| 601 | ORCA2\_restart\_0007.nc & 16200 & 5148 & 69\%\\ |
---|
| 602 | ORCA2\_2d\_grid\_T\_0000.nc & 2200 & 1504 & 32\%\\ |
---|
| 603 | ORCA2\_2d\_grid\_T\_0001.nc & 2200 & 1748 & 21\%\\ |
---|
| 604 | ORCA2\_2d\_grid\_T\_0002.nc & 2200 & 1592 & 28\%\\ |
---|
| 605 | ORCA2\_2d\_grid\_T\_0003.nc & 2200 & 1540 & 30\%\\ |
---|
| 606 | ORCA2\_2d\_grid\_T\_0004.nc & 2200 & 1204 & 46\%\\ |
---|
| 607 | ORCA2\_2d\_grid\_T\_0005.nc & 2200 & 1444 & 35\%\\ |
---|
| 608 | ORCA2\_2d\_grid\_T\_0006.nc & 2200 & 1428 & 36\%\\ |
---|
| 609 | ORCA2\_2d\_grid\_T\_0007.nc & 2200 & 1148 & 48\%\\ |
---|
| 610 | ... & ... & ... & .. \\ |
---|
| 611 | ORCA2\_2d\_grid\_W\_0000.nc & 4416 & 2240 & 50\%\\ |
---|
| 612 | ORCA2\_2d\_grid\_W\_0001.nc & 4416 & 2924 & 34\%\\ |
---|
| 613 | ORCA2\_2d\_grid\_W\_0002.nc & 4416 & 2512 & 44\%\\ |
---|
| 614 | ORCA2\_2d\_grid\_W\_0003.nc & 4416 & 2368 & 47\%\\ |
---|
| 615 | ORCA2\_2d\_grid\_W\_0004.nc & 4416 & 1432 & 68\%\\ |
---|
| 616 | ORCA2\_2d\_grid\_W\_0005.nc & 4416 & 1972 & 56\%\\ |
---|
| 617 | ORCA2\_2d\_grid\_W\_0006.nc & 4416 & 2028 & 55\%\\ |
---|
| 618 | ORCA2\_2d\_grid\_W\_0007.nc & 4416 & 1368 & 70\%\\ |
---|
| 619 | \end{tabular} |
---|
[2376] | 620 | \caption{ \label{Tab_NC4} |
---|
| 621 | Filesize comparison between NetCDF3 and NetCDF4 with chunking and compression} |
---|
[2364] | 622 | \end{table} |
---|
[2376] | 623 | %---------------------------------------------------------------------------------------------------- |
---|
[2364] | 624 | |
---|
[2282] | 625 | Since version 3.2, an I/O server has been added which provides more |
---|
| 626 | flexibility in the choice of the fields to be output as well as how the |
---|
| 627 | writing work is distributed over the processors in massively parallel |
---|
[2364] | 628 | computing. It is activated when \key{iomput} is defined. |
---|
[707] | 629 | |
---|
[2364] | 630 | When \key{iomput} is activated with \key{netcdf4} chunking and |
---|
| 631 | compression parameters for fields produced via \np{iom\_put} calls are |
---|
| 632 | set via an equivalent and identically named namelist to \np{namnc4} in |
---|
| 633 | \np{xmlio\_server.def}. Typically this namelist serves the mean files |
---|
| 634 | whilst the \np{ namnc4} in the main namelist file continues to serve the |
---|
| 635 | restart files. This duplication is unfortunate but appropriate since, if |
---|
| 636 | using io\_servers, the domain sizes of the individual files produced by the |
---|
| 637 | io\_server processes may be different to those produced by the invidual |
---|
| 638 | processing regions and different chunking choices may be desired. |
---|
| 639 | { |
---|
| 640 | |
---|
[707] | 641 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 642 | % Tracer/Dynamics Trends |
---|
| 643 | % ------------------------------------------------------------------------------------------------------------- |
---|
[2349] | 644 | \subsection[Tracer/Dynamics Trends (TRD)] |
---|
| 645 | {Tracer/Dynamics Trends (\key{trdmld}, \key{trdtra}, \key{trddyn}, \key{trdmld\_trc})} |
---|
[707] | 646 | \label{MISC_tratrd} |
---|
| 647 | |
---|
[2349] | 648 | %------------------------------------------namtrd---------------------------------------------------- |
---|
| 649 | \namdisplay{namtrd} |
---|
| 650 | %------------------------------------------------------------------------------------------------------------- |
---|
| 651 | |
---|
| 652 | When \key{trddyn} and/or \key{trddyn} CPP variables are defined, each |
---|
[994] | 653 | trend of the dynamics and/or temperature and salinity time evolution equations |
---|
| 654 | is stored in three-dimensional arrays just after their computation ($i.e.$ at the end |
---|
[2349] | 655 | of each $dyn\cdots.F90$ and/or $tra\cdots.F90$ routines). These trends are then |
---|
| 656 | used in \mdl{trdmod} (see TRD directory) every \textit{nn\_trd } time-steps. |
---|
[707] | 657 | |
---|
[2349] | 658 | What is done depends on the CPP keys defined: |
---|
| 659 | \begin{description} |
---|
| 660 | \item[\key{trddyn}, \key{trdtra}] : a check of the basin averaged properties of the momentum |
---|
| 661 | and/or tracer equations is performed ; |
---|
| 662 | \item[\key{trdvor}] : a vertical summation of the moment tendencies is performed, |
---|
| 663 | then the curl is computed to obtain the barotropic vorticity tendencies which are output ; |
---|
| 664 | \item[\key{trdmld}] : output of the tracer tendencies averaged vertically |
---|
| 665 | either over the mixed layer (\np{nn\_ctls}=0), |
---|
| 666 | or over a fixed number of model levels (\np{nn\_ctls}$>$1 provides the number of level), |
---|
| 667 | or over a spatially varying but temporally fixed number of levels (typically the base |
---|
| 668 | of the winter mixed layer) read in \ifile{ctlsurf\_idx} (\np{nn\_ctls}=1). |
---|
| 669 | \end{description} |
---|
[707] | 670 | |
---|
[2349] | 671 | The units in the output file can be changed using the \np{nn\_ucf} namelist parameter. |
---|
| 672 | For example, in case of salinity tendency the units are given by PSU/s/\np{nn\_ucf}. |
---|
| 673 | Setting \np{nn\_ucf}=86400 provides the tendencies in PSU/d. |
---|
| 674 | |
---|
| 675 | When \key{trdmld} is defined, two time averaging procedure are proposed. |
---|
| 676 | Setting \np{ln\_trdmld\_instant} to \textit{true}, a simple time averaging is performed, |
---|
| 677 | so that the resulting tendency is the contribution to the change of a quantity between |
---|
| 678 | the two instantaneous values taken at the extremities of the time averaging period. |
---|
| 679 | Setting \np{ln\_trdmld\_instant} to \textit{false}, a double time averaging is performed, |
---|
| 680 | so that the resulting tendency is the contribution to the change of a quantity between |
---|
| 681 | two \textit{time mean} values. The later option requires the use of an extra file, \ifile{restart\_mld} |
---|
| 682 | (\np{ln\_trdmld\_restart}=true), to restart a run. |
---|
| 683 | |
---|
| 684 | |
---|
| 685 | Note that the mixed layer tendency diagnostic can also be used on biogeochemical models |
---|
| 686 | via Êthe \key{trdtrc} and \key{trdmld\_trc} CPP keys. |
---|
| 687 | |
---|
[707] | 688 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 689 | % On-line Floats trajectories |
---|
| 690 | % ------------------------------------------------------------------------------------------------------------- |
---|
[2349] | 691 | \subsection{On-line Floats trajectories (FLO) (\key{floats}} |
---|
[707] | 692 | \label{FLO} |
---|
| 693 | %--------------------------------------------namflo------------------------------------------------------- |
---|
| 694 | \namdisplay{namflo} |
---|
| 695 | %-------------------------------------------------------------------------------------------------------------- |
---|
| 696 | |
---|
[2349] | 697 | The on-line computation of floats advected either by the three dimensional velocity |
---|
[1224] | 698 | field or constraint to remain at a given depth ($w = 0$ in the computation) have been |
---|
[2349] | 699 | introduced in the system during the CLIPPER project. The algorithm used is based |
---|
| 700 | either on the work of \cite{Blanke_Raynaud_JPO97} (default option), or on a $4^th$ |
---|
| 701 | Runge-Hutta algorithm (\np{ln\_flork4}=true). Note that the \cite{Blanke_Raynaud_JPO97} |
---|
| 702 | algorithm have the advantage of providing trajectories which are consistent with the |
---|
| 703 | numeric of the code, so that the trajectories never intercept the bathymetry. |
---|
[707] | 704 | |
---|
[2349] | 705 | See also the web site describing the off-line use of this marvellous diagnostic tool |
---|
| 706 | (http://stockage.univ-brest.fr/~grima/Ariane/). |
---|
| 707 | |
---|
[707] | 708 | % ------------------------------------------------------------------------------------------------------------- |
---|
| 709 | % Other Diagnostics |
---|
| 710 | % ------------------------------------------------------------------------------------------------------------- |
---|
[2349] | 711 | \subsection{Other Diagnostics (\key{diahth}, \key{diaar5})} |
---|
[707] | 712 | \label{MISC_diag_others} |
---|
| 713 | |
---|
| 714 | |
---|
[994] | 715 | Aside from the standard model variables, other diagnostics can be computed |
---|
[2349] | 716 | on-line. The available ready-to-add diagnostics routines can be found in directory DIA. |
---|
| 717 | Among the available diagnostics the following ones are obtained when defining |
---|
| 718 | the \key{diahth} CPP key: |
---|
[707] | 719 | |
---|
[2349] | 720 | - the mixed layer depth (based on a density criterion, \citet{de_Boyer_Montegut_al_JGR04}) (\mdl{diahth}) |
---|
[707] | 721 | |
---|
[2349] | 722 | - the turbocline depth (based on a turbulent mixing coefficient criterion) (\mdl{diahth}) |
---|
[707] | 723 | |
---|
[2349] | 724 | - the depth of the 20\deg C isotherm (\mdl{diahth}) |
---|
[707] | 725 | |
---|
| 726 | - the depth of the thermocline (maximum of the vertical temperature gradient) (\mdl{diahth}) |
---|
| 727 | |
---|
[2349] | 728 | The poleward heat and salt transports, their advective and diffusive component, and |
---|
| 729 | the meriodional stream function can be computed on-line in \mdl{diaptr} by setting |
---|
| 730 | \np{ln\_diaptr} to true (see the \textit{namptr} namelist below). |
---|
| 731 | When \np{ln\_subbas}~=~true, transports and stream function are computed |
---|
| 732 | for the Atlantic, Indian, Pacific and Indo-Pacific Oceans (defined north of 30\deg S) |
---|
| 733 | as well as for the World Ocean. The sub-basin decomposition requires an input file |
---|
| 734 | (\ifile{subbasins}) which contains three 2D mask arrays, the Indo-Pacific mask |
---|
| 735 | been deduced from the sum of the Indian and Pacific mask (Fig~\ref{Fig_mask_subasins}). |
---|
[707] | 736 | |
---|
[2349] | 737 | %------------------------------------------namptr---------------------------------------------------- |
---|
| 738 | \namdisplay{namptr} |
---|
| 739 | %------------------------------------------------------------------------------------------------------------- |
---|
| 740 | %>>>>>>>>>>>>>>>>>>>>>>>>>>>> |
---|
[2376] | 741 | \begin{figure}[!t] \begin{center} |
---|
[2349] | 742 | \includegraphics[width=1.0\textwidth]{./TexFiles/Figures/Fig_mask_subasins.pdf} |
---|
[2376] | 743 | \caption{ \label{Fig_mask_subasins} |
---|
| 744 | Decomposition of the World Ocean (here ORCA2) into sub-basin used in to compute |
---|
[2349] | 745 | the heat and salt transports as well as the meridional stream-function: Atlantic basin (red), |
---|
| 746 | Pacific basin (green), Indian basin (bleue), Indo-Pacific basin (bleue+green). |
---|
| 747 | Note that semi-enclosed seas (Red, Med and Baltic seas) as well as Hudson Bay |
---|
| 748 | are removed from the sub-basin. Note also that the Arctic Ocean has been split |
---|
| 749 | into Atlantic and Pacific basins along the North fold line. } |
---|
| 750 | \end{center} \end{figure} |
---|
| 751 | %>>>>>>>>>>>>>>>>>>>>>>>>>>>> |
---|
| 752 | |
---|
[2282] | 753 | In addition, a series of diagnostics has been added in the \mdl{diaar5}. |
---|
| 754 | They corresponds to outputs that are required for AR5 simulations |
---|
| 755 | (see Section \ref{MISC_steric} below for one of them). |
---|
| 756 | Activating those outputs requires to define the \key{diaar5} CPP key. |
---|
| 757 | |
---|
| 758 | |
---|
| 759 | |
---|
| 760 | % ================================================================ |
---|
| 761 | % Steric effect in sea surface height |
---|
| 762 | % ================================================================ |
---|
| 763 | \section{Steric effect in sea surface height} |
---|
| 764 | \label{MISC_steric} |
---|
| 765 | |
---|
| 766 | |
---|
| 767 | Changes in steric sea level are caused when changes in the density of the water |
---|
| 768 | column imply an expansion or contraction of the column. It is essentially produced |
---|
| 769 | through surface heating/cooling and to a lesser extent through non-linear effects of |
---|
| 770 | the equation of state (cabbeling, thermobaricity...). |
---|
| 771 | Non-Boussinesq models contain all ocean effects within the ocean acting |
---|
| 772 | on the sea level. In particular, they include the steric effect. In contrast, |
---|
| 773 | Boussinesq models, such as \NEMO, conserve volume, rather than mass, |
---|
| 774 | and so do not properly represent expansion or contraction. The steric effect is |
---|
| 775 | therefore not explicitely represented. |
---|
| 776 | This approximation does not represent a serious error with respect to the flow field |
---|
| 777 | calculated by the model \citep{Greatbatch_JGR94}, but extra attention is required |
---|
| 778 | when investigating sea level, as steric changes are an important |
---|
| 779 | contribution to local changes in sea level on seasonal and climatic time scales. |
---|
| 780 | This is especially true for investigation into sea level rise due to global warming. |
---|
| 781 | |
---|
| 782 | Fortunately, the steric contribution to the sea level consists of a spatially uniform |
---|
| 783 | component that can be diagnosed by considering the mass budget of the world |
---|
| 784 | ocean \citep{Greatbatch_JGR94}. |
---|
| 785 | In order to better understand how global mean sea level evolves and thus how |
---|
| 786 | the steric sea level can be diagnosed, we compare, in the following, the |
---|
| 787 | non-Boussinesq and Boussinesq cases. |
---|
| 788 | |
---|
| 789 | Let denote |
---|
| 790 | $\mathcal{M}$ the total mass of liquid seawater ($\mathcal{M}=\int_D \rho dv$), |
---|
| 791 | $\mathcal{V}$ the total volume of seawater ($\mathcal{V}=\int_D dv$), |
---|
| 792 | $\mathcal{A}$ the total surface of the ocean ($\mathcal{A}=\int_S ds$), |
---|
| 793 | $\bar{\rho}$ the global mean seawater (\textit{in situ}) density ($\bar{\rho}= 1/\mathcal{V} \int_D \rho \,dv$), and |
---|
| 794 | $\bar{\eta}$ the global mean sea level ($\bar{\eta}=1/\mathcal{A}\int_S \eta \,ds$). |
---|
| 795 | |
---|
| 796 | A non-Boussinesq fluid conserves mass. It satisfies the following relations: |
---|
| 797 | \begin{equation} \label{Eq_MV_nBq} |
---|
| 798 | \begin{split} |
---|
| 799 | \mathcal{M} &= \mathcal{V} \;\bar{\rho} \\ |
---|
| 800 | \mathcal{V} &= \mathcal{A} \;\bar{\eta} |
---|
| 801 | \end{split} |
---|
| 802 | \end{equation} |
---|
| 803 | Temporal changes in total mass is obtained from the density conservation equation : |
---|
| 804 | \begin{equation} \label{Eq_Co_nBq} |
---|
| 805 | \frac{1}{e_3} \partial_t ( e_3\,\rho) + \nabla( \rho \, \textbf{U} ) = \left. \frac{\textit{emp}}{e_3}\right|_\textit{surface} |
---|
| 806 | \end{equation} |
---|
| 807 | where $\rho$ is the \textit{in situ} density, and \textit{emp} the surface mass |
---|
| 808 | exchanges with the other media of the Earth system (atmosphere, sea-ice, land). |
---|
| 809 | Its global averaged leads to the total mass change |
---|
| 810 | \begin{equation} \label{Eq_Mass_nBq} |
---|
| 811 | \partial_t \mathcal{M} = \mathcal{A} \;\overline{\textit{emp}} |
---|
| 812 | \end{equation} |
---|
| 813 | where $\overline{\textit{emp}}=\int_S \textit{emp}\,ds$ is the net mass flux |
---|
| 814 | through the ocean surface. |
---|
| 815 | Bringing \eqref{Eq_Mass_nBq} and the time derivative of \eqref{Eq_MV_nBq} |
---|
| 816 | together leads to the evolution equation of the mean sea level |
---|
| 817 | \begin{equation} \label{Eq_ssh_nBq} |
---|
| 818 | \partial_t \bar{\eta} = \frac{\overline{\textit{emp}}}{ \bar{\rho}} |
---|
| 819 | - \frac{\mathcal{V}}{\mathcal{A}} \;\frac{\partial_t \bar{\rho} }{\bar{\rho}} |
---|
| 820 | \end{equation} |
---|
| 821 | The first term in equation \eqref{Eq_ssh_nBq} alters sea level by adding or |
---|
| 822 | subtracting mass from the ocean. |
---|
| 823 | The second term arises from temporal changes in the global mean |
---|
| 824 | density; $i.e.$ from steric effects. |
---|
| 825 | |
---|
| 826 | In a Boussinesq fluid, $\rho$ is replaced by $\rho_o$ in all the equation except when $\rho$ |
---|
| 827 | appears multiplied by the gravity ($i.e.$ in the hydrostatic balance of the primitive Equations). |
---|
| 828 | In particular, the mass conservation equation, \eqref{Eq_Co_nBq}, degenerates into |
---|
| 829 | the incompressibility equation: |
---|
| 830 | \begin{equation} \label{Eq_Co_Bq} |
---|
| 831 | \frac{1}{e_3} \partial_t ( e_3 ) + \nabla( \textbf{U} ) = \left. \frac{\textit{emp}}{\rho_o \,e_3}\right|_ \textit{surface} |
---|
| 832 | \end{equation} |
---|
| 833 | and the global average of this equation now gives the temporal change of the total volume, |
---|
| 834 | \begin{equation} \label{Eq_V_Bq} |
---|
| 835 | \partial_t \mathcal{V} = \mathcal{A} \;\frac{\overline{\textit{emp}}}{\rho_o} |
---|
| 836 | \end{equation} |
---|
| 837 | Only the volume is conserved, not mass, or, more precisely, the mass which is conserved is the |
---|
| 838 | Boussinesq mass, $\mathcal{M}_o = \rho_o \mathcal{V}$. The total volume (or equivalently |
---|
| 839 | the global mean sea level) is altered only by net volume fluxes across the ocean surface, |
---|
| 840 | not by changes in mean mass of the ocean: the steric effect is missing in a Boussinesq fluid. |
---|
| 841 | |
---|
| 842 | Nevertheless, following \citep{Greatbatch_JGR94}, the steric effect on the volume can be |
---|
| 843 | diagnosed by considering the mass budget of the ocean. |
---|
| 844 | The apparent changes in $\mathcal{M}$, mass of the ocean, which are not induced by surface |
---|
| 845 | mass flux must be compensated by a spatially uniform change in the mean sea level due to |
---|
| 846 | expansion/contraction of the ocean \citep{Greatbatch_JGR94}. In others words, the Boussinesq |
---|
| 847 | mass, $\mathcal{M}_o$, can be related to $\mathcal{M}$, the total mass of the ocean seen |
---|
| 848 | by the Boussinesq model, via the steric contribution to the sea level, $\eta_s$, a spatially |
---|
| 849 | uniform variable, as follows: |
---|
| 850 | \begin{equation} \label{Eq_M_Bq} |
---|
| 851 | \mathcal{M}_o = \mathcal{M} + \rho_o \,\eta_s \,\mathcal{A} |
---|
| 852 | \end{equation} |
---|
| 853 | Any change in $\mathcal{M}$ which cannot be explained by the net mass flux through |
---|
| 854 | the ocean surface is converted into a mean change in sea level. Introducing the total density |
---|
| 855 | anomaly, $\mathcal{D}= \int_D d_a \,dv$, where $d_a= (\rho -\rho_o ) / \rho_o$ |
---|
| 856 | is the density anomaly used in \NEMO (cf. \S\ref{TRA_eos}) in \eqref{Eq_M_Bq} |
---|
| 857 | leads to a very simple form for the steric height: |
---|
| 858 | \begin{equation} \label{Eq_steric_Bq} |
---|
| 859 | \eta_s = - \frac{1}{\mathcal{A}} \mathcal{D} |
---|
| 860 | \end{equation} |
---|
| 861 | |
---|
| 862 | The above formulation of the steric height of a Boussinesq ocean requires four remarks. |
---|
| 863 | First, one can be tempted to define $\rho_o$ as the initial value of $\mathcal{M}/\mathcal{V}$, |
---|
| 864 | $i.e.$ set $\mathcal{D}_{t=0}=0$, so that the initial steric height is zero. We do not |
---|
| 865 | recommend that. Indeed, in this case $\rho_o$ depends on the initial state of the ocean. |
---|
| 866 | Since $\rho_o$ has a direct effect on the dynamics of the ocean (it appears in the pressure |
---|
| 867 | gradient term of the momentum equation) it is definitively not a good idea when |
---|
| 868 | inter-comparing experiments. |
---|
| 869 | We better recommend to fixe once for all $\rho_o$ to $1035\;Kg\,m^{-3}$. This value is a |
---|
| 870 | sensible choice for the reference density used in a Boussinesq ocean climate model since, |
---|
| 871 | with the exception of only a small percentage of the ocean, density in the World Ocean |
---|
| 872 | varies by no more than 2$\%$ from this value (\cite{Gill1982}, page 47). |
---|
| 873 | |
---|
| 874 | Second, we have assumed here that the total ocean surface, $\mathcal{A}$, does not |
---|
| 875 | change when the sea level is changing as it is the case in all global ocean GCMs |
---|
| 876 | (wetting and drying of grid point is not allowed). |
---|
| 877 | |
---|
| 878 | Third, the discretisation of \eqref{Eq_steric_Bq} depends on the type of free surface |
---|
| 879 | which is considered. In the non linear free surface case, $i.e.$ \key{vvl} defined, it is |
---|
| 880 | given by |
---|
| 881 | \begin{equation} \label{Eq_discrete_steric_Bq} |
---|
| 882 | \eta_s = - \frac{ \sum_{i,\,j,\,k} d_a\; e_{1t} e_{2t} e_{3t} } |
---|
| 883 | { \sum_{i,\,j,\,k} e_{1t} e_{2t} e_{3t} } |
---|
| 884 | \end{equation} |
---|
| 885 | whereas in the linear free surface, the volume above the \textit{z=0} surface must be explicitly taken |
---|
| 886 | into account to better approximate the total ocean mass and thus the steric sea level: |
---|
| 887 | \begin{equation} \label{Eq_discrete_steric_Bq} |
---|
| 888 | \eta_s = - \frac{ \sum_{i,\,j,\,k} d_a\; e_{1t}e_{2t}e_{3t} + \sum_{i,\,j} d_a\; e_{1t}e_{2t} \eta } |
---|
| 889 | {\sum_{i,\,j,\,k} e_{1t}e_{2t}e_{3t} + \sum_{i,\,j} e_{1t}e_{2t} \eta } |
---|
| 890 | \end{equation} |
---|
| 891 | |
---|
| 892 | The fourth and last remark concerns the effective sea level and the presence of sea-ice. |
---|
| 893 | In the real ocean, sea ice (and snow above it) depresses the liquid seawater through |
---|
| 894 | its mass loading. This depression is a result of the mass of sea ice/snow system acting |
---|
| 895 | on the liquid ocean. There is, however, no dynamical effect associated with these depressions |
---|
| 896 | in the liquid ocean sea level, so that there are no associated ocean currents. Hence, the |
---|
| 897 | dynamically relevant sea level is the effective sea level, $i.e.$ the sea level as if sea ice |
---|
| 898 | (and snow) were converted to liquid seawater \citep{Campin_al_OM08}. However, |
---|
| 899 | in the current version of \NEMO the sea-ice is levitating above the ocean without |
---|
| 900 | mass exchanges between ice and ocean. Therefore the model effective sea level |
---|
| 901 | is always given by $\eta + \eta_s$, whether or not there is sea ice present. |
---|
| 902 | |
---|
| 903 | In AR5 outputs, the thermosteric sea level is demanded. It is steric sea level due to |
---|
| 904 | changes in ocean density arising just from changes in temperature. It is given by: |
---|
| 905 | \begin{equation} \label{Eq_thermosteric_Bq} |
---|
| 906 | \eta_s = - \frac{1}{\mathcal{A}} \int_D d_a(T,S_o,p_o) \,dv |
---|
| 907 | \end{equation} |
---|
| 908 | where $S_o$ and $p_o$ are the initial salinity and pressure, respectively. |
---|
| 909 | |
---|
| 910 | Both steric and thermosteric sea level are computed in \mdl{diaar5} which needs |
---|
| 911 | the \key{diaar5} defined to be called. |
---|
| 912 | |
---|
[2349] | 913 | |
---|
| 914 | % ================================================================ |
---|