Changeset 3976 for trunk/DOC/TexFiles/Chapters/Chap_DIA.tex

Timestamp:

2013-07-16T11:28:29+02:00 (11 years ago)

Author:

acc

Message:

Refinement of XIOS documentation (#1122). More still to come soonso pdf file is not yet updated. Meanwhile, see ticket wiki page for a PDF of the modified chapter.

File:

• trunk/DOC/TexFiles/Chapters/Chap_DIA.tex (modified) (26 diffs)

trunk/DOC/TexFiles/Chapters/Chap_DIA.tex

@@ -1 +1 @@
 % ================================================================
-% Chapter Ñ I/O & Diagnostics
+% Chapter I/O & Diagnostics
 % ================================================================
 \chapter{Ouput and Diagnostics (IOM, DIA, TRD, FLO)}
@@ -16 +16 @@
 
 The model outputs are of three types: the restart file, the output listing, 
-and the output file(s). The restart file is used internally by the code when 
+and the diagnostic output file(s). The restart file is used internally by the code when 
 the user wants to start the model with initial conditions defined by a 
 previous simulation. It contains all the information that is necessary in 
@@ -25 +25 @@
 that it is saved in the same binary format as the one used by the computer 
 that is to read it (in particular, 32 bits binary IEEE format must not be used for 
-this file). The output listing and file(s) are predefined but should be checked 
+this file). 
+
+The output listing and file(s) are predefined but should be checked 
 and eventually adapted to the user's needs. The output listing is stored in 
 the $ocean.output$ file. The information is printed from within the code on the 
@@ -31 +33 @@
 "\textit{grep -i numout}" in the source code directory.
 
-By default, outpout files are written in NetCDF format but an IEEE output format, called DIMG, can be choosen when defining \key{dimgout}. Since version 3.2, when defining \key{iomput}, an I/O server has been added which provides more flexibility in the choice of the fields to be outputted as well as how the writing work is distributed over the processors in massively parallel computing. The complete description of the use of this I/O server is presented in next section. If neither \key{iomput} nor \key{dimgout} are defined, NEMO is producing NetCDF with the old IOIPSL library which has been kept for compatibility and its easy installation, but it is quite inefficient on parrallel machines. If \key{iomput} is not defined, output files are defined in the \mdl{diawri} module and containing mean (or instantaneous if \key{diainstant} is defined) values over a period of nn\_write time-step (namelist parameter). 
+By default, diagnostic output files are written in NetCDF format but an IEEE binary output format, called DIMG, can be choosen by defining \key{dimgout}. 
+
+Since version 3.2, when defining \key{iomput}, an I/O server has been added which provides more flexibility in the choice of the fields to be written as well as how the writing work is distributed over the processors in massively parallel computing. The complete description of the use of this I/O server is presented in the next section. 
+
+By default, if neither \key{iomput} nor \key{dimgout} are defined, NEMO produces NetCDF with the old IOIPSL library which has been kept for compatibility and its easy installation. However, the IOIPSL library is quite inefficient on parallel machines and, since version 3.2, many diagnostic options have been added presuming the use of \key{iomput}. The usefulness of the default IOIPSL-based option is expected to reduce with each new release. If \key{iomput} is not defined, output files and content are defined in the \mdl{diawri} module and contain mean (or instantaneous if \key{diainstant} is defined) values over a regular period of nn\_write time-steps (namelist parameter). 
 
 %\gmcomment{                    % start of gmcomment
@@ -42 +48 @@
 
 
-Since version 3.2, iomput is the NEMO output interface. It was designed to be simple to use, flexible and efficient. The two main purposes of iomput are: \\
-(1) the complete and flexible control of the output files through an external xml file defined by the user \\
-(2) to achieve high performance outputs through the distribution (or not) of all tasks related to output files on dedicated processes. \\
-The first functionality allows the user to specify, without touching anything into the code, the way he want to output data: \\
-- choice of output frequencies that can be different for each file (including real months and years) \\
-- choice of file contents: decide which data will be written in which file (the same data can be outputted in different files)  \\
-- possibility to split output files at a choosen frequency \\
-- possibility to extract a vertical or an horizontal subdomain  \\
-- choice of the temporal operation to perform: average, accumulate, instantaneous, min, max and once  \\
-- extremely large choice of data available   \\
-- redefine variables name and long\_name  \\
-In addition, iomput allows the user to output any variable (scalar, 2D or 3D) in the code in a very easy way. All details of iomput functionalities are listed in the following subsections. Example of the iodef.xml files that control the outputs can be found here: NEMOGCM/CONFIG/ORCA2\_LIM/EXP00/iodef*.xml
-
-The second functionality targets outputs performances when running on a very large number of processes. First, iomput provides the possibility to dedicate N specific processes (in addition to NEMO processes) to write the outputs, where N is big enough (and defined by the user) to suppress the bottle neck associated with the the writing of the output files. Since version 3.5, this interface depends on an external code called \href{http://forge.ipsl.jussieu.fr/ioserver}{XIOS}. This new IO server takes advantage of the new functionalitiy of NetCDF4 that allows the user to write files in parallel and therefore to bypass the rebuilding phase. Note that writting in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput trough xios are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3.
-
-\subsection{Basic knowledge}
-
+Since version 3.2, iomput is the NEMO output interface of choice. It has been designed to be simple to use, flexible and efficient. The two main purposes of iomput are: 
+\begin{enumerate}
+\item The complete and flexible control of the output files through external XML files adapted by the user from standard templates. 
+\item To achieve high performance and scalable output through the optional distribution of all diagnostic output related tasks to dedicated processes. 
+\end{enumerate}
+The first functionality allows the user to specify, without code changes or recompilation, aspects of the diagnostic output stream, such as:
+\begin{itemize}
+\item The choice of output frequencies that can be different for each file (including real months and years).
+\item The choice of file contents; includes complete flexibility over which data are written in which files (the same data can be written in different files). 
+\item The possibility to split output files at a choosen frequency.
+\item The possibility to extract a vertical or an horizontal subdomain.
+\item The choice of the temporal operation to perform, e.g.: average, accumulate, instantaneous, min, max and once.
+\item Control over metadata via a large XML "database" of possible output fields.
+\end{itemize}
+In addition, iomput allows the user to add the output of any new variable (scalar, 2D or 3D) in the code in a very easy way. All details of iomput functionalities are listed in the following subsections. Examples of the XML files that control the outputs can be found in:
+\begin{alltt}
+\begin{verbatim}
+  NEMOGCM/CONFIG/ORCA2_LIM/EXP00/iodef.xml
+  NEMOGCM/CONFIG/SHARED/field_def.xml
+  and
+  NEMOGCM/CONFIG/SHARED/domain_def.xml.
+\end{verbatim}
+\end{alltt}
+
+The second functionality targets output performance when running in parallel (\key{mpp\_mpi}). Iomput provides the possibility to specify N dedicated I/O processes (in addition to the NEMO processes) to collect and write the outputs. With an appropriate choice of N by the user, the bottleneck associated with the writing of the output files can be greatly reduced. 
+
+Since version 3.5, the iom\_put interface depends on an external code called \href{http://forge.ipsl.jussieu.fr/ioserver}{XIOS}. This new IO server can take advantage of the parallel I/O functionality of NetCDF4 to create a single output file and therefore to bypass the rebuilding phase. Note that writing in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput through XIOS are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3.
+
+Even if not using the parallel I/O functionality of NetCDF4, using N dedicated I/O servers, where N is typically much less than the number of NEMO processors, will reduce the number of output files created. This can greatly reduce the post-processing burden usually associated with using large numbers of NEMO processors. Note that for smaller configurations, the rebuilding phase can be avoided, even without a parallel-enabled NetCDF4 library, simply by employing only one dedicated I/O server.
+
+\subsection{XIOS: the IO\_SERVER}
+
+\subsubsection{Attached or detached mode?}
+
+Iomput is based on \href{http://forge.ipsl.jussieu.fr/ioserver/wiki}{XIOS}, the io\_server developed by Yann Meurdesoif from IPSL. The behaviour of the io subsystem is controlled by settings in the external XML files listed above. Key settings in the iodef.xml file are {\tt using\_server} and the {\tt type} tag associated with each defined file. The {\tt using\_server} setting determines whether or not the server will be used in ''attached mode'' (as a library) [{\tt false}] or in ''detached mode'' (as an external executable on N additional, dedicated cpus) [{\tt true}]. The ''attached mode'' is simpler to use but much less efficient for massively parallel applications. The type of each file can be either ''multiple\_file'' or ''one\_file''.
+
+In attached mode and if the type of file is ''multiple\_file'', then each NEMO process will also act as an IO server and produce its own set of output files. Superficially, this emulates the standard behaviour in previous versions, However, the subdomain written out by each process does not correspond to the {\tt jpi x jpj x jpk} domain actually computed by the process (although it may if {\tt jpni=1}). Instead each process will have collected and written out a number of complete longitudinal strips. If the ''one\_file'' option is chosen then all processes will collect their longitudinal strips and write (in parallel) to a single output file. 
+
+In detached mode and if the type of file is ''multiple\_file'', then each stand-alone XIOS process will collect data for a range of complete longitudinal strips and write to its own set of output files. If the ''one\_file'' option is chosen then all XIOS processes will collect their longitudinal strips and write (in parallel) to a single output file. Note running in detached mode requires launching a Multiple Process Multiple Data (MPMD) parallel job. The following subsection provides a typical example but the syntax will vary in different MPP environments.
+
+\subsubsection{Number of cpu used by XIOS in detached mode}
+
+The number of cores used by the XIOS is specified when launching the model. The number of cores dedicated to XIOS should be from ~1/10 to ~1/50 of the number or cores dedicated to NEMO. Some manufacturers suggest using O($\sqrt{N}$) dedicated IO processors for N processors but this is a general recommendation and not specific to NEMO. It is difficult to provide precise recommendations because the optimal choice will depend on the particular hardware properties of the target system (parallel filesystem performance, available memory, memory bandwidth etc.) and the volume and frequency of data to be created. Here is an example of 2 cpus for the io\_server and 62 cpu for nemo using mpirun:
+
+\texttt{ mpirun -np 62 ./nemo.exe : -np 2 ./xios\_server.exe }
+
+\subsubsection{Control of XIOS: the XIOS context in iodef.xml}
+
+As well as the {\tt using\_server} flag, other controls on the use of XIOS are set in the XIOS context in iodef.xml. See the XML basics section below for more details on XML syntax and rules.
+
+\begin{tabular}{|p{4cm}|p{6.0cm}|p{2.0cm}|}
+   \hline
+   variable name & 
+   description & 
+   example \\ 
+   \hline   
+   \hline
+   buffer\_size & 
+   buffer size used by XIOS to send data from NEMO to XIOS. Larger is more efficient. Note that needed/used buffer sizes are summarized at the end of the job & 
+   25000000 \\ 
+   \hline   
+   buffer\_server\_factor\_size & 
+   ratio between NEMO and XIOS buffer size. Should be 2. & 
+   2 \\ 
+   \hline
+   info\_level & 
+   verbosity level (0 to 100) & 
+   0 \\ 
+   \hline
+   using\_server & 
+   activate attached(false) or detached(true) mode & 
+   true \\ 
+   \hline
+   using\_oasis & 
+   XIOS is used with OASIS(true) or not (false) & 
+   false \\ 
+   \hline
+   oasis\_codes\_id & 
+   when using oasis, define the identifier of NEMO in the namcouple. Note that the identifier of XIOS is xios.x & 
+   oceanx \\ 
+   \hline   
+\end{tabular}
+
+
+\subsection{Practical issues}
+
+\subsubsection{Installation}
+
+As mentioned, XIOS is supported separately and must be downloaded and compiled before it can be used with NEMO. See the installation guide on the \href{http://forge.ipsl.jussieu.fr/ioserver/wiki}{XIOS} wiki for help and guidance. NEMO will need to link to the compiled XIOS library. The 
+\href{http://www.nemo-ocean.eu/Using-NEMO/User-Guides/Basics/XIOS-IO-server-installation-and-use}{XIOS with NEMO} guide provides an example illustration of how this can be achieved.
+
+\subsubsection{Add your own outputs}
+
+It is very easy to add your own outputs with iomput. Many standard fields and diagnostics are already prepared (i.e., steps 1 to 3 below have been done) and simply need to be activated by including the required output in a file definition in iodef.xml (step 4). To add new output variables, all 4 of the following steps must be taken.
+\begin{description}
+\item[1.] in NEMO code, add a \\
+\texttt{      CALL iom\_put( 'identifier', array ) } \\
+where you want to output a 2D or 3D array.
+
+\item[2.] If necessary, add \\
+\texttt{   USE iom\ \ \ \ \ \ \ \ \ \ \ \ ! I/O manager library }  \\
+to the list of used modules in the upper part of your module. 
+
+\item[3.] in the field\_def.xml file, add the definition of your variable using the same identifier you used in the f90 code (see subsequent sections for a details of the XML syntax and rules). For example:
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize
+\begin{verbatim}
+   <field_definition>
+      <!-- T grid -->
+
+     <field_group id="grid_T" grid_ref="grid_T_3D">
+      ...
+      <field id="identifier" long_name="blabla" ... />   
+      ...
+   </field_definition> 
+\end{verbatim}
+}}\end{alltt} 
+Note your definition must be added to the field\_group whose reference grid is consistent with the size of the array passed to iomput. The grid\_ref attribute refers to definitions set in iodef.xml which, in turn, reference grids and axes either defined in the code (iom\_set\_domain\_attr and iom\_set\_axis\_attr in iom.F90) or defined in the domain\_def.xml file. E.g.:
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize
+\begin{verbatim}
+     <grid id="grid_T_3D" domain_ref="grid_T" axis_ref="deptht"/>
+\end{verbatim}
+}}\end{alltt} 
+Note, if your array is computed within the surface module each nn\_fsbc time\_step, 
+add the field definition within the field\_group defined with the id ''SBC'': $<$field\_group id=''SBC''...$>$ which has been defined with the correct frequency of operations (iom\_set\_field\_attr in iom.F90)
+
+\item[4.] add your field in one of the output files defined in iodef.xml (again see subsequent sections for syntax and rules)   \\
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize
+\begin{verbatim}
+   <file id="file1" .../>   
+      ...
+      <field field_ref="identifier" />   
+      ...
+   </file>   
+\end{verbatim}
+}}\end{alltt} 
+
+\end{description}
+\subsection{XML fundamentals}
 
 \subsubsection{ XML basic rules}
@@ -72 +202 @@
 
 The XML file used in XIOS is structured by 7 families of tags: context, axis, domain, grid, field, file and variable. Each tag family has hierarchy of three flavors (except for context):
-\begin{description}
-\item[root]: declaration of the root element that can contain element groups or elements, for example : $<$file\_definition ...$/>$ \\
-\item[group]: declaration of a group element that can contain element groups or elements, for example : $<$file\_group ...$/>$ \\
-\item[element]: declaration of an element that can contain elements, for example : $<$file ...$/>$  \\
-\end{description}
+\\
+\begin{tabular}{|p{3.0cm}|p{4.5cm}|p{4.5cm}|}
+   \hline
+   flavor &
+   description &
+   example \\
+   \hline
+   \hline
+   root &
+   declaration of the root element that can contain element groups or elements &
+   {\scriptsize \verb? < file_definition ... >?} \\
+   \hline
+   group &
+   declaration of a group element that can contain element groups or elements &
+   {\scriptsize \verb? < file_group ... >?} \\
+   \hline
+   element &
+   declaration of an element that can contain elements &
+   {\scriptsize \verb? < file ... >?} \\
+   \hline
+\end{tabular}
+\\
 
 Each element may have several attributes. Some attributes are mandatory, other are optional but have a default value and other are are completely optional. Id is a special attribute used to identify an element or a group of elements. It must be unique for a kind of element. It is optional, but no reference to the corresponding element can be done if it is not defined.
 
-The XML file is split into context tags that are used to isolate IO definition from different codes or different parts of a code. No interference is possible between 2 different contexts. Each context has its own calendar and an associated timestep. In NEMO, we used the following contexts (that can be defined in any order):
-\begin{description}
-\item[contex xios]: context containing informations for XIOS \\
-\verb?   <context id="xios" ... ? 
-\item[context nemo]: contex containing IO informations for NEMO (mother grid when using AGRIF) \\
-\verb?   <context id="nemo" ... ? 
-\item[context 1\_nemo]: contex containing IO informations for NEMO child grid 1 (when using AGRIF) \\
-\verb?   <context id="1_nemo" ... ?  
-\item[context n\_nemo]: contex containing IO informations for NEMO child grid n (when using AGRIF) \\
-\verb?   <context id="n_nemo" ... ?
-\end{description}
-
-Each context tag related to NEMO (mother or child grids) is divided into 5 parts (that can be defined in any order):
-\begin{description}
-\item[field definition]: define all variables that can potentially be outputted \\
-\verb?   <field_definition ... ?
-\item[file definition]: define the netcdf files to be created and the variables they will contain \\
-\verb?   <file_definition ... ? 
-\item[axis definitions]: define vertical axis \\
-\verb?   <axis_definition ... ?
-\item[domain definitions]: define the horizontal grids \\
-\verb?   <domain_definition ... ?
-\item[grid definitions]: define the 2D and 3D grids (association of an axis and a domain) \\
-\verb?   <grid_definition ... ? 
-\end{description}
-
-the xios context contains only 1 tag:
-\begin{description}
-\item[variable definition]: define variables needed by xios. This can be seen as a kind of namelist for xios. \\
-\verb?   <variable_definition ... ? 
-\end{description}
-
-The XML file can be split in different parts to improve its readability and facilitate its use. The inclusing of XML files into the main XML file can be done through the attribute src: \\
-\verb?   <context src="./nemo_def.xml" /> ? 
-In NEMO, by default, the field and domain définition is done in 2 séparate files: \\
-NEMOGCM/CONFIG/SHARED/field\_def.xml and \\
-NEMOGCM/CONFIG/SHARED/domain\_def.xml that are included in the main iodef.xml file through the following commands: \\
-\verb?   <field_definition src="./field_def.xml" /> ? \\
-\verb?   <domain_definition src="./domain_def.xml" /> ? 
+The XML file is split into context tags that are used to isolate IO definition from different codes or different parts of a code. No interference is possible between 2 different contexts. Each context has its own calendar and an associated timestep. In NEMO, we used the following contexts (that can be defined in any order):\\
+\\
+\begin{tabular}{|p{3.0cm}|p{4.5cm}|p{4.5cm}|}
+   \hline
+   context &
+   description &
+   example \\
+   \hline
+   \hline
+   context xios &
+   context containing information for XIOS &
+   {\scriptsize \verb? <context id="xios" ...  ?} \\
+   \hline
+   context nemo &
+   context containing IO information for NEMO (mother grid when using AGRIF) &
+   {\scriptsize \verb? <context id="nemo" ... ?} \\
+   \hline
+   context 1\_nemo &
+   context containing IO information for NEMO child grid 1 (when using AGRIF) &
+   {\scriptsize \verb? <context id="1_nemo" ...  ?} \\
+   \hline
+   context n\_nemo &
+   context containing IO information for NEMO child grid n (when using AGRIF) &
+   {\scriptsize \verb? <context id="n_nemo" ...  ?} \\
+   \hline
+\end{tabular}
+\\
+
+\noindent The xios context contains only 1 tag:
+\\
+\begin{tabular}{|p{3.0cm}|p{4.5cm}|p{4.5cm}|}
+   \hline
+   context tag &
+   description &
+   example \\
+   \hline
+   \hline
+   variable\_definition &
+   define variables needed by XIOS. This can be seen as a kind of namelist for XIOS. &
+   {\scriptsize \verb? <variable_definition ... ?} \\
+   \hline
+\end{tabular}
+\\
+
+\noindent Each context tag related to NEMO (mother or child grids) is divided into 5 parts (that can be defined in any order):\\
+\\
+\begin{tabular}{|p{3.0cm}|p{4.5cm}|p{4.5cm}|}
+   \hline
+   context tag &
+   description &
+   example \\
+   \hline
+   \hline
+   field\_definition &
+   define all variables that can potentially be outputted &
+   {\scriptsize \verb? <field_definition ... ?} \\
+   \hline
+   file\_definition &
+   define the netcdf files to be created and the variables they will contain &
+   {\scriptsize \verb? <file_definition ... ?} \\
+   \hline
+   axis\_definition &
+   define vertical axis &
+   {\scriptsize \verb? <axis_definition ... ?} \\
+   \hline
+   domain\_definition &
+   define the horizontal grids &
+   {\scriptsize \verb? <domain_definition ... ?} \\
+   \hline
+   grid\_definition &
+   define the 2D and 3D grids (association of an axis and a domain) &
+   {\scriptsize \verb? <grid_definition ... ?} \\
+   \hline
+\end{tabular}
+\\
+
+\subsubsection{Nesting XML files}
+
+The XML file can be split in different parts to improve its readability and facilitate its use. The inclusion of XML files into the main XML file can be done through the attribute src: \\
+{\scriptsize \verb? <context src="./nemo_def.xml" /> ?}\\
+ 
+\noindent In NEMO, by default, the field and domain definition is done in 2 separate files:
+{\scriptsize \tt
+\begin{verbatim}
+NEMOGCM/CONFIG/SHARED/field_def.xml
+and
+NEMOGCM/CONFIG/SHARED/domain_def.xml 
+\end{verbatim}
+}
+\noindent that are included in the main iodef.xml file through the following commands: \\
+{\scriptsize \verb? <field_definition src="./field_def.xml" /> ? \\
+\verb? <domain_definition src="./domain_def.xml" /> ? }
 
 
 \subsubsection{Use of inheritance}
 
-XML extensively uses the concept of inheritance. XML has a based tree structure with a parent-child oriented relation: all children inherit attributes from parent, but an attribute defined in a child replace the inherited attribute value. Note that the special attribute ''id'' is never inherited.  \\
+XML extensively uses the concept of inheritance. XML has a tree based structure with a parent-child oriented relation: all children inherit attributes from parent, but an attribute defined in a child replace the inherited attribute value. Note that the special attribute ''id'' is never inherited.  \\
 \\
-example 1: Direct inheritance. \\
+example 1: Direct inheritance.
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize    
 \begin{verbatim}
    <field_definition operation="average" >
-      <field id="sst"                    />   <!-- averaged      sst --> 
-      <field id="sss" operation="instant"/>   <!-- instantaneous sss --> 
+     <field id="sst"                    />   <!-- averaged      sst --> 
+     <field id="sss" operation="instant"/>   <!-- instantaneous sss --> 
    </field_definition> 
 \end{verbatim}
@@ -140 +341 @@
 for example output instantaneous values instead of average values. \\
 \\
-example 2: Inheritance by reference. \\
+example 2: Inheritance by reference.
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
    <field_definition>
-      <field id="sst" long_name="sea surface temperature" />   
-      <field id="sss" long_name="sea surface salinity"    />  
+     <field id="sst" long_name="sea surface temperature" />   
+     <field id="sss" long_name="sea surface salinity"    />  
    </field_definition>      
 
    <file_definition>
-      <file id="myfile" output_freq="1d" />   
-            <field field_ref="sst"                            />  <!-- default def -->
-            <field field_ref="sss" long_name="my description" />  <!-- overwrite   -->
-      </file>   
+     <file id="myfile" output_freq="1d" />   
+       <field field_ref="sst"                            />  <!-- default def -->
+       <field field_ref="sss" long_name="my description" />  <!-- overwrite   -->
+     </file>   
    </file_definition> 
 \end{verbatim}
 }}\end{alltt} 
-Inherite (and overwrite, if needed) the attributes of a tag you are refering to.
-
-\subsubsection{Use of Group}
-
-Groups can be used fort 2 purposes. \\
-
-First, the group can be used to define common attributes to be shared by the elements of the group through the inheritance. In the following example, we define a group of field that will share a common grid ''grid\_T\_2D''. Note that for the field ''toce'', we overwrite the grid definition inherited from the group by ''grid\_T\_3D''.
+Inherit (and overwrite, if needed) the attributes of a tag you are refering to.
+
+\subsubsection{Use of Groups}
+
+Groups can be used for 2 purposes. Firstly, the group can be used to define common attributes to be shared by the elements of the group through the inheritance. In the following example, we define a group of field that will share a common grid ''grid\_T\_2D''. Note that for the field ''toce'', we overwrite the grid definition inherited from the group by ''grid\_T\_3D''.
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
    <field_group id="grid_T" grid_ref="grid_T_2D">
-      <field id="toce" long_name="temperature"             unit="degC" grid_ref="grid_T_3D"/>
-      <field id="sst"  long_name="sea surface temperature" unit="degC"                     />
-      <field id="sss"  long_name="sea surface salinity"    unit="psu"                      />
-      <field id="ssh"  long_name="sea surface height"      unit="m"                        />
+    <field id="toce" long_name="temperature"             unit="degC" grid_ref="grid_T_3D"/>
+    <field id="sst"  long_name="sea surface temperature" unit="degC"                     />
+    <field id="sss"  long_name="sea surface salinity"    unit="psu"                      />
+    <field id="ssh"  long_name="sea surface height"      unit="m"                        />
          ...
 \end{verbatim}
 }}\end{alltt} 
 
-Second, the group can be used to replace a list of elements. Several examples of groups of fields are proposed at the end of the file \\
-NEMOGCM/CONFIG/SHARED/field\_def.xml. For example, a short list of usual variables related to the U grid:
+Secondly, the group can be used to replace a list of elements. Several examples of groups of fields are proposed at the end of the file {\tt CONFIG/SHARED/field\_def.xml}. For example, a short list of the usual variables related to the U grid:
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
    <field_group id="groupU" >
-      <field field_ref="uoce"  />
-      <field field_ref="suoce" />
-      <field field_ref="utau"  />
+    <field field_ref="uoce"  />
+    <field field_ref="suoce" />
+    <field field_ref="utau"  />
    </field_group>
 \end{verbatim}
 }}\end{alltt} 
-that can be directly include in a file through the following syntaxe:
+that can be directly include in a file through the following syntax:
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
    <file id="myfile_U" output_freq="1d" />   
-      <field_group group_ref="groupU"/>  
-      <field field_ref="uocetr_eff"  />  <!-- add another field -->
+    <field_group group_ref="groupU"/>  
+    <field field_ref="uocetr_eff"  />  <!-- add another field -->
    </file>   
 \end{verbatim}
@@ -197 +399 @@
 \subsection{Detailed functionalities }
 
-The file NEMOGCM/CONFIG/ORCA2\_LIM/iodef\_demo.xml provides several examples of the use of the new functionalities offered by the XML interface of XIOS. 
+The file {\tt NEMOGCM/CONFIG/ORCA2\_LIM/iodef\_demo.xml} provides several examples of the use of the new functionalities offered by the XML interface of XIOS. 
 
 \subsubsection{Define horizontal subdomains}
-Horizontal subdomains are defined through the attributs zoom\_ibegin, zoom\_jbegin, zoom\_ni, zoom\_nj of the tag family domain. It must therefore be done in the domain part of the XML file. For example, in NEMOGCM/CONFIG/SHARED/domain\_def.xml, we provide the following example of a definition of a 5 by 5 box with the bottom left corner at point (10,10).
+Horizontal subdomains are defined through the attributs zoom\_ibegin, zoom\_jbegin, zoom\_ni, zoom\_nj of the tag family domain. It must therefore be done in the domain part of the XML file. For example, in {\tt CONFIG/SHARED/domain\_def.xml}, we provide the following example of a definition of a 5 by 5 box with the bottom left corner at point (10,10).
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
    <domain_group id="grid_T">
-      <domain id="myzoom" zoom_ibegin="10" zoom_jbegin="10" zoom_ni="5" zoom_nj="5" />
+    <domain id="myzoom" zoom_ibegin="10" zoom_jbegin="10" zoom_ni="5" zoom_nj="5" />
 \end{verbatim}
 }}\end{alltt} 
 The use of this subdomain is done through the redefinition of the attribute domain\_ref of the tag family field. For example:
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
@@ -216 +420 @@
 }}\end{alltt} 
 Moorings are seen as an extrem case corresponding to a 1 by 1 subdomain. The Equatorial section, the TAO, RAMA and PIRATA moorings are alredy registered in the code and can therefore be outputted without taking care of their (i,j) position in the grid. These predefined domains can be activated by the use of specific domain\_ref: ''EqT'', ''EqU'' or ''EqW'' for the equatorial sections and the mooring position for TAO, RAMA and PIRATA followed by ''T'' (for example: ''8s137eT'', ''1.5s80.5eT'' ...)
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
@@ -227 +432 @@
 \subsubsection{Define vertical zooms}
 Vertical zooms are defined through the attributs zoom\_begin and zoom\_end of the tag family axis. It must therefore be done in the axis part of the XML file. For example, in NEMOGCM/CONFIG/ORCA2\_LIM/iodef\_demo.xml, we provide the following example:
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
@@ -235 +441 @@
 }}\end{alltt} 
 The use of this vertical zoom is done through the redefinition of the attribute axis\_ref of the tag family field. For example:
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
@@ -246 +453 @@
 
 The output file names are defined by the attributs ''name'' and ''name\_suffix'' of the tag family file. for example:
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize
 \begin{verbatim}
@@ -258 +466 @@
 \end{verbatim}
 }}\end{alltt} 
-However it is also often very convienent to define the file name with the name of the experience, the output file frequency and the date of the beginning and the end of the simulation (which are informations stored either in the namelist or in the XML file). To do so, we added the following rule: if the id of the tag file is ''fileN''(where N = 1 to 99) or one of the predefined section or mooring (see next subsection), the following part of the name and the name\_suffix (that can be inherited) will be automatically replaced by: \\
+However it is often very convienent to define the file name with the name of the experience, the output file frequency and the date of the beginning and the end of the simulation (which are informations stored either in the namelist or in the XML file). To do so, we added the following rule: if the id of the tag file is ''fileN''(where N = 1 to 99) or one of the predefined section or mooring (see next subsection), the following part of the name and the name\_suffix (that can be inherited) will be automatically replaced by:\\
 \\
 \begin{tabular}{|p{4cm}|p{8cm}|}
    \hline
-   \centering part of the name automatically to be replaced &
-   by \\
+   \centering placeholder string & automatically  replaced by \\
    \hline
    \hline
@@ -284 +491 @@
    ending date of the simulation (from nn\_date0 and nn\_itend in the namelist). \verb?yyyymmdd_hh:mm:ss? format \\
    \hline
-\end{tabular}
+\end{tabular}\\
 \\
 
-For example, 
-
-\begin{alltt}  {{\scriptsize
+\noindent For example, 
+{{\scriptsize
 \begin{verbatim}
    <file id="myfile_hzoom" name="myfile_@expname@_@startdate@_freq@freq@" output_freq="1d" >
 \end{verbatim}
-}}\end{alltt} 
-
-With, in the namelist:
-
-\begin{alltt}  {{\scriptsize
+}}
+\noindent with the namelist:
+{{\scriptsize
 \begin{verbatim}
    cn_exp      =  "ORCA2"
@@ -303 +507 @@
    ln_rstart   = .false.
 \end{verbatim}
-}}\end{alltt} 
-
-will give the following file name radical:
-
-\begin{alltt}  {{\scriptsize
+}}
+\noindent will give the following file name radical:
+{{\scriptsize
 \begin{verbatim}
    myfile_ORCA2_19891231_freq1d 
 \end{verbatim}
-}}\end{alltt} 
-
+}}
 
 \subsubsection{Other controls of the xml attributes from NEMO}
 
-The values of some attributes are automatically defined by NEMO (and any definition given in the xml file is overwritten). By convention, these attributes are defined to ''auto'' (for string) or ''0000'' (for integer) in the xml file (but this is not necessary). 
-
-Here is the list of these attributes: \\
+The values of some attributes are defined by subroutine calls within NEMO (calls to iom\_set\_domain\_attr, iom\_set\_axis\_attr and iom\_set\_field\_attr in iom.F90). Any definition given in the xml file will be overwritten. By convention, these attributes are defined to ''auto'' (for string) or ''0000'' (for integer) in the xml file (but this is not necessary). 
+
+Here is the list of these attributes:\\
 \\
 \begin{tabular}{|l|c|c|c|}
@@ -343 +544 @@
 
 
+\subsection{XML reference tables}
+
 \subsubsection{Tag list}
 
-
-\begin{tabular}{|p{2cm}|p{2.5cm}|p{3.5cm}|p{2cm}|p{2cm}|}
+\begin{longtable}{|p{2.2cm}|p{2.5cm}|p{3.5cm}|p{2.2cm}|p{1.6cm}|}
    \hline
    tag name & 
@@ -352 +554 @@
    accepted attribute & 
    child of &
-   parent of \\
-   \hline   
+   parent of \endhead
    \hline   
    simulation & 
@@ -362 +563 @@
    \hline   
    context &
-   encapsulates parts of the xml file dédicated to different codes or different parts of a code &
+   encapsulates parts of the xml file dedicated to different codes or different parts of a code &
    id (''xios'', ''nemo'' or ''n\_nemo'' for the nth AGRIF zoom), src, time\_origin &
    simulation &
-   all root tags: ...\_definition \\
+   all root tags: ... \_definition \\
    \hline   
    \hline   
@@ -400 +601 @@
    \hline   
    file & 
-   defile the contentof a file to be outputted &
+   define the contents of a file to be outputted &
    enabled, description, id, min\_digits, name, name\_suffix, output\_freq, output\_level, split\_format, split\_freq, sync\_freq, type, src &
    file\_definition, file\_group & 
    field \\
-   \hline   
-\end{tabular}
-\begin{tabular}{|p{2cm}|p{2.5cm}|p{3.5cm}|p{2cm}|p{2cm}|}
-   \hline
-   tag name & 
-   description & 
-   accepted attribute & 
-   child of &
-   parent of \\
-   \hline   
    \hline   
    axis\_definition & 
@@ -434 +625 @@
    \hline   
    \hline   
-   domain\_definition & 
+   domain\_\-definition & 
    define all the horizontal domains potentially used by the variables &
    src &
    context & 
-   domain\_group, domain \\
+   domain\_\-group, domain \\
    \hline   
    domain\_group & 
    encapsulates a group of horizontal domains &
    id, lon\_name, src, zoom\_ibegin, zoom\_jbegin, zoom\_ni, zoom\_nj &
-   domain\_definition, domain\_group & 
-   domain\_group, domain \\
+   domain\_\-definition, domain\_group & 
+   domain\_\-group, domain \\
    \hline   
    domain & 
    define an horizontal domain &
    id, lon\_name, src, zoom\_ibegin, zoom\_jbegin, zoom\_ni, zoom\_nj &
-   domain\_definition, domain\_group & 
+   domain\_\-definition, domain\_group & 
    none \\
    \hline   
@@ -471 +662 @@
    none \\
    \hline   
-\end{tabular}
+\end{longtable}
 
 
 \subsubsection{Attributes list}
 
-\begin{tabular}{|p{2cm}|p{4cm}|p{4cm}|p{2cm}|}
-   \hline
+\begin{longtable}{|p{2.2cm}|p{4cm}|p{3.8cm}|p{2cm}|}
+   \hline
+   attribute name & 
+   description & 
+   example & 
+   accepted by \endhead
+   \hline   
+   axis\_ref & 
+   refers to the id of a vertical axis & 
+   axis\_ref="deptht" & 
+   field, grid families \\ 
+   \hline   
+   enabled & 
+   switch on/off the output of a field or a file & 
+   enabled=".TRUE." & 
+   field, file families \\ 
+   \hline   
+   default\_value & 
+   missing\_value definition & 
+   default\_value="1.e20" & 
+   field family \\ 
+   \hline   
+   description & 
+   just for information, not used & 
+   description="ocean T grid variables" & 
+   all tags \\ 
+   \hline   
+   domain\_ref & 
+   refers to the id of a domain & 
+   domain\_ref="grid\_T" & 
+   field or grid families \\ 
+   \hline   
+   field\_ref & 
+   id of the field we want to add in a file & 
+   field\_ref="toce" & 
+   field \\ 
+   \hline   
+   grid\_ref & 
+   refers to the id of a grid & 
+   grid\_ref="grid\_T\_2D" & 
+   field family \\ 
+   \hline   
+   group\_ref & 
+   refer to a group of variables & 
+   group\_ref="mooring" & 
+   field\_group \\ 
+   \hline   
+   id & 
+   allow to identify a tag & 
+   id="nemo" &
+   accepted by all tags except simulation \\ 
+   \hline   
+   level & 
+   output priority of a field: 0 (high) to 10 (low)& 
+   level="1" & 
+   field family \\ 
+   \hline   
+   long\_name & 
+   define the long\_name attribute in the NetCDF file & 
+   long\_name="Vertical T levels" & 
+   field \\ 
+   \hline   
+   min\_digits & 
+   specify the minimum of digits used in the core number in the name of the NetCDF file & 
+   min\_digits="4" & 
+   file family \\ 
+   \hline   
+   name & 
+   name of a variable or a file. If the name of a file is undefined, its id is used as a name & 
+   name="tos" & 
+   field or file families \\ 
+   \hline   
+   name\_suffix & 
+   suffix to be inserted after the name and before the cpu number and the ''.nc'' termination of a file & 
+   name\_suffix="\_myzoom" & 
+   file family \\ 
+   \hline   
    attribute name & 
    description & 
@@ -484 +750 @@
    \hline   
    \hline   
-   axis\_ref & 
-   refers to the id of a vertical axis & 
-   axis\_ref="deptht" & 
-   field, grid families \\ 
-   \hline   
-   enabled & 
-   switch on/off the output of a field or a file & 
-   enabled=".TRUE." & 
-   field, file families \\ 
-   \hline   
-   default\_value & 
-   missing\_value definition & 
-   default\_value="1.e20" & 
+   operation & 
+   type of temporal operation: average, accumulate, instantaneous, min, max and once & 
+   operation="average" & 
    field family \\ 
    \hline   
-   description & 
-   just for information, not used & 
-   description="ocean T grid variables" & 
-   all tags \\ 
-   \hline   
-   domain\_ref & 
-   refers to the id of a domain & 
-   domain\_ref="grid\_T" & 
-   field or grid families \\ 
-   \hline   
-   field\_ref= & 
-   id of the field we want to add in a file & 
-   field\_ref="toce" & 
+   output\_freq & 
+   operation frequency. units can be ts (timestep), y, mo, d, h, mi, s. & 
+   output\_freq="1d12h" & 
+   field family \\ 
+   \hline   
+   output\_level & 
+   output priority of variables in a file: 0 (high) to 10 (low). All variables listed in the file with a level smaller or equal to output\_level will be output. Other variables won't be output even if they are listed in the file. &  
+   output\_level="10"& 
+   file family \\ 
+   \hline   
+   positive & 
+   convention used for the orientation of vertival axis (positive downward in \NEMO). & 
+   positive="down" & 
+   axis family \\ 
+   \hline   
+   prec & 
+   output precision: real 4 or real 8 & 
+   prec="4" & 
+   field family \\ 
+   \hline   
+   split\_format & 
+   date format used in the name of splitted output files. can be spécified using the following syntaxe: \%y, \%mo, \%d, \%h \%mi and \%s & 
+   split\_format= "\%yy\%mom\%dd" & 
+   file family \\ 
+   \hline   
+   split\_freq & 
+   split output files frequency. units can be ts (timestep), y, mo, d, h, mi, s. & 
+   split\_freq="1mo" & 
+   file family \\ 
+   \hline   
+   src & 
+   allow to include a file & 
+   src="./field\_def.xml" & 
+   accepted by all tags except simulation \\ 
+   \hline   
+   standard\_name & 
+   define the standard\_name attribute in the NetCDF file & 
+   standard\_name= "Eastward\_Sea\_Ice\_Transport" & 
    field \\ 
    \hline   
-   grid\_ref & 
-   refers to the id of a grid & 
-   grid\_ref="grid\_T\_2D" & 
-   field family \\ 
-   \hline   
-   group\_ref & 
-   refer to a group of variables & 
-   group\_ref="mooring" & 
-   field\_group \\ 
-   \hline   
-   id & 
-   allow to identify a tag & 
-   id="nemo" &
-   accepted by all tags except simulation \\ 
-   \hline   
-   level & 
-   output priority of a field: 0 (high) to 10 (low)& 
-   level="1" & 
-   field family \\ 
-   \hline   
-   long\_name & 
-   define the long\_name attribute in the NetCDF file & 
-   long\_name="Vertical T levels" & 
-   field \\ 
-   \hline   
-   min\_digits & 
-   specify the minimum of digits used in the core number in the name of the NetCDF file & 
-   min\_digits="4" & 
+   sync\_freq & 
+   NetCDF file synchronization frequency (update of the time\_counter). units can be ts (timestep), y, mo, d, h, mi, s. & 
+   sync\_freq="10d" & 
    file family \\ 
    \hline   
-   name & 
-   name of a variable or a file. If the name of a file is undefined, its id is used as a name & 
-   name="tos" & 
-   field or file families \\ 
-   \hline   
-   name\_suffix & 
-   suffix to be inserted after the name and before the cpu number and the ''.nc'' termination of a file & 
-   name\_suffix="\_myzoom" & 
-   file family \\ 
-   \hline   
-\end{tabular}
-\begin{tabular}{|p{2cm}|p{4cm}|p{4cm}|p{2cm}|}
-   \hline
    attribute name & 
    description & 
@@ -563 +806 @@
    \hline   
    \hline   
-   operation & 
-   type of temporal operation: average, accumulate, instantaneous, min, max and once & 
-   operation="average" & 
-   field family \\ 
-   \hline   
-   output\_freq & 
-   operation frequency. units can be ts (timestep), y, mo, d, h, mi, s. & 
-   output\_freq="1d12h" & 
-   field family \\ 
-   \hline   
-   output\_level & 
-   output priority of variables in a file: 0 (high) to 10 (low). All variables listed in the file with a level smaller or equal to output\_level will be output. Other variables won't be output even if they are listed in the file. &  
-   output\_level="10"& 
-   file family \\ 
-   \hline   
-   positive & 
-   convention used for the orientation of vertival axis (positive downward in \NEMO). & 
-   positive="down" & 
-   axis family \\ 
-   \hline   
-   prec & 
-   output precision: real 4 or real 8 & 
-   prec="4" & 
-   field family \\ 
-   \hline   
-   split\_format & 
-   date format used in the name of splitted output files. can be spécified using the following syntaxe: \%y, \%mo, \%d, \%h \%mi and \%s & 
-   split\_format="\%yy\%mom\%dd" & 
-   file family \\ 
-   \hline   
-   split\_freq & 
-   split output files frequency. units can be ts (timestep), y, mo, d, h, mi, s. & 
-   split\_freq="1mo" & 
-   file family \\ 
-   \hline   
-   src & 
-   allow to include a file & 
-   src="./field\_def.xml" & 
-   accepted by all tags except simulation \\ 
-   \hline   
-   standard\_name & 
-   define the standard\_name attribute in the NetCDF file & 
-   standard\_name="Eastward\_Sea\_Ice\_Transport" & 
-   field \\ 
-   \hline   
-   sync\_freq & 
-   NetCDF file synchronization frequency (update of the time\_counter). units can be ts (timestep), y, mo, d, h, mi, s. & 
-   sync\_freq="10d" & 
-   file family \\ 
-   \hline   
-\end{tabular}
-\begin{tabular}{|p{2cm}|p{4cm}|p{4cm}|p{2cm}|}
-   \hline
-   attribute name & 
-   description & 
-   example & 
-   accepted by \\ 
-   \hline   
-   \hline   
    time\_origin & 
    specify the origin of the time counter & 
@@ -628 +812 @@
    \hline   
    type (1)& 
-   specify if the output files must be splitted (multiple\_file) or not (one\_file) & 
+   specify if the output files must be split (multiple\_file) or not (one\_file) & 
    type="multiple\_file" & 
    file familly \\ 
@@ -662 +846 @@
    domain family \\ 
    \hline   
-\end{tabular}
-
-\subsection{XIOS: the IO\_SERVER}
-
-\subsubsection{Attached or detached mode?}
-
-Iomput is based on \href{http://forge.ipsl.jussieu.fr/ioserver/wiki}{XIOS}, the io\_server developed by Yann Meurdesoif from IPSL. This server can be used in ''attached mode'' (as a library) or in ''detached mode'' (as an external executable on n cpus). The ''attached mode'' is simpler to use but much less efficient. If the type of file is ''multiple\_file'', then in attached(detached) mode, each NEMO(XIOS) process will output its own subdomain: if NEMO(XIOS) is runnning on N cores, the ouput files will be splitted into N files. If the type of file is ''one\_file'', the output files will be directly recombined into one unique file either in ''detached mode'' or ''attached mode''.  
-
-\subsubsection{Control of xios: the xios context in iodef.xml}
-
-The control of the use of xios is done through the xios context in iodef.xml.
-
-\begin{tabular}{|p{3cm}|p{6.5cm}|p{2.5cm}|}
-   \hline
-   variable name & 
-   description & 
-   example \\ 
-   \hline   
-   \hline
-   buffer\_size & 
-   buffer size used by XIOS to send data from NEMO to XIOS. Larger is more efficient. Note that needed/used buffer sizes are summarized at the end of the job & 
-   25000000 \\ 
-   \hline   
-   buffer\_server\_factor\_size & 
-   ratio between NEMO and XIOS buffer size. Should be 2. & 
-   2 \\ 
-   \hline
-   info\_level & 
-   verbosity level (0 to 100) & 
-   0 \\ 
-   \hline
-   using\_server & 
-   activate attached(false) or detached(true) mode & 
-   true \\ 
-   \hline
-   using\_oasis & 
-   xios is used with OASIS(true) or not (false) & 
-   false \\ 
-   \hline
-   oasis\_codes\_id & 
-   when using oasis, define the identifier of NEMO in the namcouple. Not that the identifier of XIOS is xios.x & 
-   oceanx \\ 
-   \hline   
-\end{tabular}
-
-\subsubsection{Number of cpu used by XIOS in detached mode}
-
-The number of cores used by the xios is specified only when launching the model. The number or cores dedicated to XIOS should be from ~1/10 to ~1/50 of the number or cores dedicated to NEMO (according of the amount of data to be created). Here is an example of 2 cpus for the io\_server and 62 cpu for opa using mpirun:
-
-\texttt{ mpirun -np 2 ./nemo.exe : -np 62 ./xios\_server.exe }
-
-\subsection{Practical issues}
-
-\subsubsection{Add your own outputs}
-
-It is very easy to add you own outputs with iomput. 4 points must be followed.
-\begin{description}
-\item[1-] in NEMO code, add a \\
-\texttt{      CALL iom\_put( 'identifier', array ) } \\
-where you want to output a 2D or 3D array.
-
-\item[2-] don't forget to add \\
-\texttt{   USE iom            ! I/O manager library }  \\
-in the list of used modules in the upper part of your module. 
-
-\item[3-] in the file\_definition part of the xml file, add the definition of your variable using the same identifier you used in the f90 code.
-\vspace{-20pt}
-\begin{alltt}  {{\scriptsize
-\begin{verbatim}
-   <field_definition>
-      ...
-      <field id="identifier" long_name="blabla" ... />   
-      ...
-   </field_definition> 
-\end{verbatim}
-}}\end{alltt} 
-attributes axis\_ref and grid\_ref must be consistent with the size of the array to pass to iomput.
-if your array is computed within the surface module each nn\_fsbc time\_step, 
-add the field definition within the group defined with the id ''SBC'': $<$group id=''SBC''...$>$
-
-\item[4-] add your field in one of the output files   \\
-\vspace{-20pt}
-\begin{alltt}  {{\scriptsize
-\begin{verbatim}
-   <file id="file1" .../>   
-      ...
-      <field ref="identifier" />   
-      ...
-   </file>   
-\end{verbatim}
-}}\end{alltt} 
-
-\end{description}
+\end{longtable}
+
 
 
@@ -809 +902 @@
 domain size in any dimension. The algorithm used is:
 
+\vspace{-20pt}
 \begin{alltt}  {{\scriptsize 
 \begin{verbatim}