Changeset 1526 for XIOS/dev

Timestamp:

06/08/18 09:12:00 (6 years ago)

Author:

oabramkina

Message:

Updating user's manual: UGRID and server-two.

Location:

XIOS/dev/XIOS_DEV_CMIP6/doc

Files:

• XIOS_user_guide.lyx (modified) (5 diffs)
• XIOS_user_guide.pdf (modified) (previous)
• inputs/images/Server2.pdf (added)

XIOS/dev/XIOS_DEV_CMIP6/doc/XIOS_user_guide.lyx

@@ -1863 +1863 @@
 
 \begin_layout Section
+UGRID
+\end_layout
+
+\begin_layout Standard
+In addition to the CF conventions, it is also possible to output data using
+ 
+\begin_inset CommandInset href
+LatexCommand href
+name "UGRID"
+target "https://ugrid-conventions.github.io/ugrid-conventions/"
+
+\end_inset
+
+ metadata conventions developed for unstructured meshes.
+ It allows users to store the topology of an underlying unstructured mesh.
+ Currently XIOS supports 2D unstructured meshes of any shape (triangular,
+ quadrilateral, etc) and their mixture.
+ 
+\end_layout
+
+\begin_layout Standard
+A 2D mesh can be described by a set of nodes, edges and/or faces.
+ XIOS allows one to define data on any of these three types of elements.
+ XIOS will generate a full list of connectivity attributes proposed by the
+ UGRID conventions.
+ For example in case of a mesh comprised of faces the following connectivity
+ parameters will be the calculated:
+\end_layout
+
+\begin_layout Standard
+
+\family typewriter
+edge_node_connectivity
+\end_layout
+
+\begin_layout Standard
+
+\family typewriter
+face_node_connectivity 
+\end_layout
+
+\begin_layout Standard
+
+\family typewriter
+edge_nodes_connectivity 
+\end_layout
+
+\begin_layout Standard
+
+\family typewriter
+face_nodes_connectivity
+\end_layout
+
+\begin_layout Standard
+
+\family typewriter
+face_edges_connectivity
+\end_layout
+
+\begin_layout Standard
+
+\family typewriter
+edge_face_connectivity
+\end_layout
+
+\begin_layout Standard
+
+\family typewriter
+face_face_connectivity
+\end_layout
+
+\begin_layout Standard
+In order to select UGRID output format, one has to set file attribute 
+\series bold
+convention
+\series default
+ to 
+\series bold
+\shape italic
+"UGRID"
+\series default
+\shape default
+ (its default value is 
+\series bold
+\shape italic
+
+\begin_inset Quotes eld
+\end_inset
+
+CF
+\begin_inset Quotes erd
+\end_inset
+
+
+\series default
+\shape default
+).
+ Domain attribute 
+\series bold
+nvertex
+\series default
+ is mandatory for UGRID.
+ It servers for identifying one of three types of mesh elements on which
+ data can be defined: nodes (nvertex=1), edges (nvertex=2), and faces (nvertex
+\begin_inset Formula $\geq$
+\end_inset
+
+3).
+ In order to write fields on the same mesh but on its different elements,
+ one has to assign the same domain name to each of the domains.
+ Example given below illustrates this point for three fields defined on
+ the same mesh but on its different elements: nodes, edges, and faces.
+\end_layout
+
+\begin_layout Standard
+\begin_inset listings
+lstparams "language=XML,breaklines=true,tabsize=2,frame=tb,postbreak={\raisebox{0ex}[0ex][0ex]{\ensuremath{\rcurvearrowse\space}}}"
+inline false
+status open
+
+\begin_layout Plain Layout
+
+<file_definition> 
+\end_layout
+
+\begin_layout Plain Layout
+
+        <file id="output_UGRID" convention="UGRID"> 
+\end_layout
+
+\begin_layout Plain Layout
+
+                <field id="varOnNodes" ...
+ domain_ref="node"/> 
+\end_layout
+
+\begin_layout Plain Layout
+
+                <field id="varOnEdges" ...
+ domain_ref="edge"/>
+\end_layout
+
+\begin_layout Plain Layout
+
+                <field id="varOnFaces" ...
+ domain_ref="face"/> 
+\end_layout
+
+\begin_layout Plain Layout
+
+        </file>
+\end_layout
+
+\begin_layout Plain Layout
+
+</file_definition> 
+\end_layout
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\begin_layout Plain Layout
+
+<domain_definition> 
+\end_layout
+
+\begin_layout Plain Layout
+
+        <domain id="node" name="mesh2D" nvertex="1"/>
+\end_layout
+
+\begin_layout Plain Layout
+
+        <domain id="edge" name="mesh2D" nvertex="2"/> 
+\end_layout
+
+\begin_layout Plain Layout
+
+        <domain id="face" name="mesh2D" nvertex="4"/> 
+\end_layout
+
+\begin_layout Plain Layout
+
+</domain_definition> 
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
 How to use file splitting
 \end_layout
@@ -4574 +4767 @@
  Since it depends directly on HDF5, this feature works only when the NetCDF-4
  format is used.
- Unfortunately, HDF5 does not support compression (yet) for parallel output
- so you have to use only one server or to engage the 
+ Since HDF5 does not (yet) support compression for parallel output, one
+ has to use two server-level functionality (see Sec.
+ 
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "sec:Launching-secondary-server"
+
+\end_inset
+
+) or to engage the 
 \series bold
 \emph on
@@ -4800 +5001 @@
 
 \begin_layout Chapter
-XIOS options
+XIOS parameterization
 \end_layout
 
@@ -4885 +5086 @@
 
 
+\end_layout
+
+\begin_layout Section
+Launching secondary server
+\begin_inset CommandInset label
+LatexCommand label
+name "sec:Launching-secondary-server"
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+To improve I/O performance and to be able to use HDF5 compression with the
+ 
+\series bold
+\emph on
+
+\begin_inset Quotes eld
+\end_inset
+
+multiple_file
+\begin_inset Quotes erd
+\end_inset
+
+
+\series default
+\emph default
+ mode, it is possible to separate servers into two levels: intermediaries
+ (level one) and writers (level two).
+ A single MPI communicator will be created for the intermediaries, while
+ multiple communicators will be created for the writers according to the
+ number of 
+\begin_inset Quotes eld
+\end_inset
+
+pools
+\begin_inset Quotes erd
+\end_inset
+
+ which can be set by a user.
+ Level-one servers will receive data from clients and will redistribute
+ it to be sent to pools of level-two servers whilst level-two servers will
+ do the I/O (important: for now level-two servers only do writing data).
+ Secondary servers can be launched by means of three parameters:
+\end_layout
+
+\begin_layout Itemize
+
+\series bold
+using_server2
+\series default
+ (type: 
+\series bold
+bool
+\series default
+) activates the secondary server
+\end_layout
+
+\begin_layout Itemize
+
+\series bold
+ratio_server2
+\series default
+ (type: 
+\series bold
+int
+\series default
+) defines the percentage of servers that will be dedicated to level two.
+ The parameter can take value from 0 to 100 with the default value of 50%.
+ In case if the requested number of level-two servers is not valid (for
+ example, zero or equal to the total number of servers), XIOS will run in
+ its classical server mode with one server level.
+\end_layout
+
+\begin_layout Itemize
+
+\series bold
+number_pools_server2 
+\series default
+(type: 
+\series bold
+int
+\series default
+) sets the number of server-two pools (i.e.
+ MPI communicators on level two).
+ By default the number of pools is equal to the number of level-two servers,
+ thus permitting one process per communicator.
+\end_layout
+
+\begin_layout Standard
+Shown in Fig.
+ 
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "Fig:server2"
+
+\end_inset
+
+ is the two-level server structure for the following definitions:
+\end_layout
+
+\begin_layout Standard
+\begin_inset listings
+lstparams "breaklines=true,frame=tb,language=XML,postbreak={\raisebox{0ex}[0ex][0ex]{\ensuremath{\rcurvearrowse\space}}},tabsize=2"
+inline false
+status open
+
+\begin_layout Plain Layout
+
+<context id="xios">     
+\end_layout
+
+\begin_layout Plain Layout
+
+...
+       
+\end_layout
+
+\begin_layout Plain Layout
+
+        <variable id="using_server2" type="bool">true</variable>
+\end_layout
+
+\begin_layout Plain Layout
+
+        <variable id="ratio_server2" type="int">75</variable>         
+\end_layout
+
+\begin_layout Plain Layout
+
+        <variable id="number_pools_server2" type="int">3</variable>         
+\end_layout
+
+\begin_layout Plain Layout
+
+...
+\end_layout
+
+\begin_layout Plain Layout
+
+</context> 
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Float figure
+placement H
+wide false
+sideways false
+status open
+
+\begin_layout Plain Layout
+\begin_inset Graphics
+        filename inputs/images/Server2.pdf
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Plain Layout
+\begin_inset Caption Standard
+
+\begin_layout Plain Layout
+Two levels of servers for the total number of servers of 8 and ratio_server2=75%.
+ The number of level-two servers is 
+\begin_inset Formula $8\times\text{ratio\_server2}=6$
+\end_inset
+
+ and, thus, the remaining 2 servers are of level one.
+ 
+\end_layout
+
+\end_inset
+
+
+\begin_inset CommandInset label
+LatexCommand label
+name "Fig:server2"
+
+\end_inset
+
+
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+Note that with one server per pool, the I/O is actually sequential and thus
+ the use of HDF5 compression is possible.
+ 
+\end_layout
+
+\begin_layout Standard
+By default file distribution among server-two pools is optimized for bandwidth.
+ An alternative way of distributing files is possible in order to minimize
+ memory consumption by level-two servers.
+ For this, two additional parameters should be specified:
+\end_layout
+
+\begin_layout Itemize
+
+\series bold
+server2_dist_file_memory
+\series default
+ (type: 
+\series bold
+bool
+\series default
+) activates memory optimization.
+\end_layout
+
+\begin_layout Itemize
+
+\series bold
+server2_dist_file_memory_ratio
+\series default
+ (type: 
+\series bold
+double
+\series default
+) (optional) takes value from 0 (memory optimization) to 1 (bandwidth optimizati
+on).
+ The default value is 0.5.
 \end_layout
 
@@ -4981 +5415 @@
 int
 \series default
-) defines the mimimum buffer size in bytes (8192 by default).
+) defines the minimum buffer size in bytes (8192 by default).
  This value will be used by XIOS only for buffers whose detected size is
- smaller than the user defined mimimum size.
+ smaller than the user defined minimum size.
 \end_layout