Changeset 3940 for trunk/DOC/TexFiles/Chapters/Chap_DIA.tex
- Timestamp:
- 2013-06-26T10:10:12+02:00 (11 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/DOC/TexFiles/Chapters/Chap_DIA.tex
r3764 r3940 31 31 "\textit{grep -i numout}" in the source code directory. 32 32 33 In the standard configuration, the user will find the model results in 34 NetCDF files containing mean values (or instantaneous values if 35 \key{diainstant} is defined) for every time-step where output is demanded. 36 These outputs are defined in the \mdl{diawri} module. 37 When defining \key{dimgout}, the output are written in DIMG format, 38 an IEEE output format. 39 40 Since version 3.2, an I/O server has been added which provides more 41 flexibility in the choice of the fields to be output as well as how the 42 writing work is distributed over the processors in massively parallel 43 computing. It is presented in next section. 44 33 By default, outpout files are written in NetCDF format but an IEEE output format, called DIMG, can be choosen when defining \key{dimgout}. Since version 3.2, when defining \key{iomput}, an I/O server has been added which provides more flexibility in the choice of the fields to be outputted as well as how the writing work is distributed over the processors in massively parallel computing. The complete description of the use of this I/O server is presented in next section. If neither \key{iomput} nor \key{dimgout} are defined, NEMO is producing NetCDF with the old IOIPSL library which has been kept for compatibility and its easy installation, but it is quite inefficient on parrallel machines. If \key{iomput} is not defined, output files are defined in the \mdl{diawri} module and containing mean (or instantaneous if \key{diainstant} is defined) values over a period of nn\_write time-step (namelist parameter). 45 34 46 35 %\gmcomment{ % start of gmcomment … … 53 42 54 43 55 Since version 3.2, iom\_put is the NEMO output interface. It was designed to be simple to use, 56 flexible and efficient. Two main functionalities are covered by iom\_put: 57 (1) the control of the output files through an external xml file defined by the user ; 58 (2) the distribution (or not) of all task related to output files on dedicated processors. 59 The first functionality allows the user to specify, without touching anything into the code, 60 the way he want to output data: \\ 44 Since version 3.2, iomput is the NEMO output interface. It was designed to be simple to use, flexible and efficient. The two main purposes of iomput are: \\ 45 (1) the complete and flexible control of the output files through an external xml file defined by the user \\ 46 (2) to achieve high performance outputs through the distribution (or not) of all tasks related to output files on dedicated processes. \\ 47 The first functionality allows the user to specify, without touching anything into the code, the way he want to output data: \\ 61 48 - choice of output frequencies that can be different for each file (including real months and years) \\ 62 - choice of file contents: decide which data will be written in which file (the same data can be 63 outputted in different files)\\64 - possibility to extract a subdomain (for example all TAO-PIRATA-RAMA moorings are already defined)\\65 - choice of the temporal operation to perform: mean, instantaneous, min, max\\49 - choice of file contents: decide which data will be written in which file (the same data can be outputted in different files) \\ 50 - possibility to split output files at a choosen frequency \\ 51 - possibility to extract a vertical or an horizontal subdomain \\ 52 - choice of the temporal operation to perform: average, accumulate, instantaneous, min, max and once \\ 66 53 - extremely large choice of data available \\ 67 54 - redefine variables name and long\_name \\ 68 In addition, iom\_put allows the user to output any variable (scalar, 2D or 3D) in the code 69 in a very easy way. All details of iom\_put functionalities are listed in the following subsections. 70 An example of the iodef.xml file that control the outputs can be found here: 71 NEMOGCM/CONFIG/ORCA2\_LIM/EXP00/iodef.xml 72 73 The second functionality targets outputs performances when running on a very large number of processes. 74 The idea is to dedicate N specific processes to write the outputs, where N is defined by the user. 75 In the current version, this functionality is technically working however, its performance are usually poor 76 (for known reasons). Users can therefore test this functionality but they must be aware that expected 77 performance improvement will not be achieved before the release 3.4. 78 An example of xmlio\_server.def NEMOGCM/CONFIG/ORCA2\_LIM/EXP00/xmlio\_server.def 79 55 In addition, iomput allows the user to output any variable (scalar, 2D or 3D) in the code in a very easy way. All details of iomput functionalities are listed in the following subsections. Example of the iodef.xml files that control the outputs can be found here: NEMOGCM/CONFIG/ORCA2\_LIM/EXP00/iodef*.xml 56 57 The second functionality targets outputs performances when running on a very large number of processes. First, iomput provides the possibility to dedicate N specific processes (in addition to NEMO processes) to write the outputs, where N is big enough (and defined by the user) to suppress the bottle neck associated with the the writing of the output files. Since version 3.5, this interface depends on an external code called \href{http://forge.ipsl.jussieu.fr/ioserver}{XIOS}. This new IO server takes advantage of the new functionalitiy of NetCDF4 that allows the user to write files in parallel and therefore to bypass the rebuilding phase. Note that writting in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput trough xios are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3. 80 58 81 59 \subsection{Basic knowledge} … … 93 71 \subsubsection{Structure of the xml file used in NEMO} 94 72 95 The xml file is split into 3 parts:73 The XML file used in XIOS is structured by 7 families of tags: context, axis, domain, grid, field, file and variable. Each tag family has hierarchy of three flavors (except for context): 96 74 \begin{description} 97 \item[field definition]: define all variables that can be output \\ 98 all lines in between the following two tags\\ 99 \verb? <field\_definition ...> ? \\ 100 \verb? </field\_definition ...> ? 75 \item[root]: declaration of the root element that can contain element groups or elements, for example : $<$file\_definition ...$/>$ \\ 76 \item[group]: declaration of a group element that can contain element groups or elements, for example : $<$file\_group ...$/>$ \\ 77 \item[element]: declaration of an element that can contain elements, for example : $<$file ...$/>$ \\ 78 \end{description} 79 80 Each element may have several attributes. Some attributes are mandatory, other are optional but have a default value and other are are completely optional. Id is a special attribute used to identify an element or a group of elements. It must be unique for a kind of element. It is optional, but no reference to the corresponding element can be done if it is not defined. 81 82 The XML file is split into context tags that are used to isolate IO definition from different codes or different parts of a code. No interference is possible between 2 different contexts. Each context has its own calendar and an associated timestep. In NEMO, we used the following contexts (that can be defined in any order): 83 \begin{description} 84 \item[contex xios]: context containing informations for XIOS \\ 85 \verb? <context id="xios" ... ? 86 \item[context nemo]: contex containing IO informations for NEMO (mother grid when using AGRIF) \\ 87 \verb? <context id="nemo" ... ? 88 \item[context 1\_nemo]: contex containing IO informations for NEMO child grid 1 (when using AGRIF) \\ 89 \verb? <context id="1_nemo" ... ? 90 \item[context n\_nemo]: contex containing IO informations for NEMO child grid n (when using AGRIF) \\ 91 \verb? <context id="n_nemo" ... ? 92 \end{description} 93 94 Each context tag related to NEMO (mother or child grids) is divided into 5 parts (that can be defined in any order): 95 \begin{description} 96 \item[field definition]: define all variables that can potentially be outputted \\ 97 \verb? <field_definition ... ? 101 98 \item[file definition]: define the netcdf files to be created and the variables they will contain \\ 102 all lines in between the following two tags \\ 103 \verb? <field\_definition> ? \\ 104 \verb? </field\_definition> ? 105 \item[axis and grid definitions]: define the horizontal and vertical grids \\ 106 all lines in between the following two set of two tags\\ 107 \verb? <axis\_definition ...> ? \\ 108 \verb? </axis\_definition ...> ? 109 and \\ 110 \verb? <grid\_definition ...> ? \\ 111 \verb? </grid\_definition ...> ? 99 \verb? <file_definition ... ? 100 \item[axis definitions]: define vertical axis \\ 101 \verb? <axis_definition ... ? 102 \item[domain definitions]: define the horizontal grids \\ 103 \verb? <domain_definition ... ? 104 \item[grid definitions]: define the 2D and 3D grids (association of an axis and a domain) \\ 105 \verb? <grid_definition ... ? 112 106 \end{description} 113 107 114 \subsubsection{Inheritance and group } 115 116 Xml extensively uses the concept of inheritance. \\ 108 the xios context contains only 1 tag: 109 \begin{description} 110 \item[variable definition]: define variables needed by xios. This can be seen as a kind of namelist for xios. \\ 111 \verb? <variable_definition ... ? 112 \end{description} 113 114 The XML file can be split in different parts to improve its readability and facilitate its use. The inclusing of XML files into the main XML file can be done through the attribute src: \\ 115 \verb? <context src="./nemo_def.xml" /> ? 116 In NEMO, by default, the field and domain définition is done in 2 séparate files: \\ 117 NEMOGCM/CONFIG/SHARED/field\_def.xml and \\ 118 NEMOGCM/CONFIG/SHARED/domain\_def.xml that are included in the main iodef.xml file through the following commands: \\ 119 \verb? <field_definition src="./field_def.xml" /> ? \\ 120 \verb? <domain_definition src="./domain_def.xml" /> ? 121 122 123 \subsubsection{Use of inheritance} 124 125 XML extensively uses the concept of inheritance. XML has a based tree structure with a parent-child oriented relation: all children inherit attributes from parent, but an attribute defined in a child replace the inherited attribute value. Note that the special attribute ''id'' is never inherited. \\ 117 126 \\ 118 example 1: \\ 119 \vspace{-30pt} 127 example 1: Direct inheritance. \\ 120 128 \begin{alltt} {{\scriptsize 121 129 \begin{verbatim} 122 <field_definition operation="ave (X)" >130 <field_definition operation="average" > 123 131 <field id="sst" /> <!-- averaged sst --> 124 <field id="sss" operation="inst (X)"/> <!-- instantaneous sss -->132 <field id="sss" operation="instant"/> <!-- instantaneous sss --> 125 133 </field_definition> 126 134 \end{verbatim} 127 135 }}\end{alltt} 128 136 129 The field ''sst'' which is part (or a child) of the field\_definition will inherit the value ''ave (X)''130 of the attribute ''operation'' from its parent ''field definition''. Note that a child can overwrite137 The field ''sst'' which is part (or a child) of the field\_definition will inherit the value ''average'' 138 of the attribute ''operation'' from its parent. Note that a child can overwrite 131 139 the attribute definition inherited from its parents. In the example above, the field ''sss'' will 132 therefore output instantaneous values instead of average values. 133 134 example 2: Use (or overwrite) attributes value of a field when listing the variables included in a file 135 \vspace{-20pt} 140 for example output instantaneous values instead of average values. \\ 141 \\ 142 example 2: Inheritance by reference. \\ 136 143 \begin{alltt} {{\scriptsize 137 144 \begin{verbatim} 138 145 <field_definition> 139 <field id="sst" description="sea surface temperature" />140 <field id="sss" description="sea surface salinity" />146 <field id="sst" long_name="sea surface temperature" /> 147 <field id="sss" long_name="sea surface salinity" /> 141 148 </field_definition> 142 149 143 150 <file_definition> 144 <file id=" file_1" />145 <field ref="sst"/> <!-- default def -->146 <field ref="sss" description="my description" /> <!-- overwrite -->151 <file id="myfile" output_freq="1d" /> 152 <field field_ref="sst" /> <!-- default def --> 153 <field field_ref="sss" long_name="my description" /> <!-- overwrite --> 147 154 </file> 148 155 </file_definition> 149 156 \end{verbatim} 150 157 }}\end{alltt} 151 152 With the help of the inheritance, the concept of group allow to define a set of attributes 153 for several fields or files. 154 155 example 3, group of fields: define a group ''T\_grid\_variables'' identified with the name 156 ''grid\_T''. By default variables of this group have no vertical axis but, following inheritance 157 rules, ''axis\_ref'' can be redefined for the field ''toce'' that is a 3D variable. 158 \vspace{-30pt} 159 \begin{alltt} {{\scriptsize 160 \begin{verbatim} 161 <field_definition> 162 <group id="grid_T" axis_ref="none" grid_ref="T_grid_variables"> 163 <field id="sst"/> 164 <field id="sss"/> 165 <field id="toce" axis_ref="deptht"/> <!-- overwrite axis def --> 166 </group> 167 </field_definition> 168 \end{verbatim} 169 }}\end{alltt} 170 171 example 4, group of files: define a group of file with the attribute output\_freq equal to 432000 (5 days) 172 \vspace{-30pt} 173 \begin{alltt} {{\scriptsize 174 \begin{verbatim} 175 <file_definition> 176 <group id="5d" output_freq="432000"> <!-- 5d files --> 177 <file id="5d_grid_T" name="auto"> <!-- T grid file --> 158 Inherite (and overwrite, if needed) the attributes of a tag you are refering to. 159 160 \subsubsection{Use of Group} 161 162 Groups can be used fort 2 purposes. \\ 163 164 First, the group can be used to define common attributes to be shared by the elements of the group through the inheritance. In the following example, we define a group of field that will share a common grid ''grid\_T\_2D''. Note that for the field ''toce'', we overwrite the grid definition inherited from the group by ''grid\_T\_3D''. 165 \begin{alltt} {{\scriptsize 166 \begin{verbatim} 167 <field_group id="grid_T" grid_ref="grid_T_2D"> 168 <field id="toce" long_name="temperature" unit="degC" grid_ref="grid_T_3D"/> 169 <field id="sst" long_name="sea surface temperature" unit="degC" /> 170 <field id="sss" long_name="sea surface salinity" unit="psu" /> 171 <field id="ssh" long_name="sea surface height" unit="m" /> 178 172 ... 179 </file> 180 <file id="5d_grid_U" name="auto"> <!-- U grid file --> 173 \end{verbatim} 174 }}\end{alltt} 175 176 Second, the group can be used to replace a list of elements. Several examples of groups of fields are proposed at the end of the file \\ 177 NEMOGCM/CONFIG/SHARED/field\_def.xml. For example, a short list of usual variables related to the U grid: 178 \begin{alltt} {{\scriptsize 179 \begin{verbatim} 180 <field_group id="groupU" > 181 <field field_ref="uoce" /> 182 <field field_ref="suoce" /> 183 <field field_ref="utau" /> 184 </field_group> 185 \end{verbatim} 186 }}\end{alltt} 187 that can be directly include in a file through the following syntaxe: 188 \begin{alltt} {{\scriptsize 189 \begin{verbatim} 190 <file id="myfile_U" output_freq="1d" /> 191 <field_group group_ref="groupU"/> 192 <field field_ref="uocetr_eff" /> <!-- add another field --> 193 </file> 194 \end{verbatim} 195 }}\end{alltt} 196 197 \subsection{Detailed functionalities } 198 199 The file NEMOGCM/CONFIG/ORCA2\_LIM/iodef\_demo.xml provides several examples of the use of the new functionalities offered by the XML interface of XIOS. 200 201 \subsubsection{Define horizontal subdomains} 202 Horizontal subdomains are defined through the attributs zoom\_ibegin, zoom\_jbegin, zoom\_ni, zoom\_nj of the tag family domain. It must therefore be done in the domain part of the XML file. For example, in NEMOGCM/CONFIG/SHARED/domain\_def.xml, we provide the following example of a definition of a 5 by 5 box with the bottom left corner at point (10,10). 203 \begin{alltt} {{\scriptsize 204 \begin{verbatim} 205 <domain_group id="grid_T"> 206 <domain id="myzoom" zoom_ibegin="10" zoom_jbegin="10" zoom_ni="5" zoom_nj="5" /> 207 \end{verbatim} 208 }}\end{alltt} 209 The use of this subdomain is done through the redefinition of the attribute domain\_ref of the tag family field. For example: 210 \begin{alltt} {{\scriptsize 211 \begin{verbatim} 212 <file id="myfile_vzoom" output_freq="1d" > 213 <field field_ref="toce" domain_ref="myzoom"/> 214 </file> 215 \end{verbatim} 216 }}\end{alltt} 217 Moorings are seen as an extrem case corresponding to a 1 by 1 subdomain. The Equatorial section, the TAO, RAMA and PIRATA moorings are alredy registered in the code and can therefore be outputted without taking care of their (i,j) position in the grid. These predefined domains can be activated by the use of specific domain\_ref: ''EqT'', ''EqU'' or ''EqW'' for the equatorial sections and the mooring position for TAO, RAMA and PIRATA followed by ''T'' (for example: ''8s137eT'', ''1.5s80.5eT'' ...) 218 \begin{alltt} {{\scriptsize 219 \begin{verbatim} 220 <file id="myfile_vzoom" output_freq="1d" > 221 <field field_ref="toce" domain_ref="0n180wT"/> 222 </file> 223 \end{verbatim} 224 }}\end{alltt} 225 Note that if the domain decomposition used in XIOS cuts the subdomain in several parts and if you use the ''multiple\_file'' type for your output files, you will endup with several files you will need to rebuild using unprovided tools (like ncpdq and ncrcat, \href{http://nco.sourceforge.net/nco.html#Concatenation}{see nco manual}). We are therefore advising to use the ''one\_file'' type in this case. 226 227 \subsubsection{Define vertical zooms} 228 Vertical zooms are defined through the attributs zoom\_begin and zoom\_end of the tag family axis. It must therefore be done in the axis part of the XML file. For example, in NEMOGCM/CONFIG/ORCA2\_LIM/iodef\_demo.xml, we provide the following example: 229 \begin{alltt} {{\scriptsize 230 \begin{verbatim} 231 <axis_group id="deptht" long_name="Vertical T levels" unit="m" positive="down" > 232 <axis id="deptht" /> 233 <axis id="deptht_myzoom" zoom_begin="1" zoom_end="10" /> 234 \end{verbatim} 235 }}\end{alltt} 236 The use of this vertical zoom is done through the redefinition of the attribute axis\_ref of the tag family field. For example: 237 \begin{alltt} {{\scriptsize 238 \begin{verbatim} 239 <file id="myfile_hzoom" output_freq="1d" > 240 <field field_ref="toce" axis_ref="deptht_myzoom"/> 241 </file> 242 \end{verbatim} 243 }}\end{alltt} 244 245 \subsubsection{Control of the output file names} 246 247 The output file names are defined by the attributs ''name'' and ''name\_suffix'' of the tag family file. for example: 248 \begin{alltt} {{\scriptsize 249 \begin{verbatim} 250 <file_group id="1d" output_freq="1d" name="myfile_1d" > 251 <file id="myfileA" name_suffix="_AAA" > <!-- will create file "myfile_1d_AAA" --> 181 252 ... 182 </file> 183 </group> 184 </file_definition> 185 \end{verbatim} 186 }}\end{alltt} 187 188 \subsubsection{Control of the xml attributes from NEMO} 189 190 The values of some attributes are automatically defined by NEMO (and any definition 191 given in the xml file is overwritten). By convention, these attributes are defined to ''auto'' 192 (for string) or ''0000'' (for integer) in the xml file (but this is not necessary). 253 </file> 254 <file id="myfileB" name_suffix="_BBB" > <!-- will create file "myfile_1d_BBB" --> 255 ... 256 </file> 257 </file_group> 258 \end{verbatim} 259 }}\end{alltt} 260 However it is also often very convienent to define the file name with the name of the experience, the output file frequency and the date of the beginning and the end of the simulation (which are informations stored either in the namelist or in the XML file). To do so, we added the following rule: if the id of the tag file is ''fileN''(where N = 1 to 99) or one of the predefined section or mooring (see next subsection), the following part of the name and the name\_suffix (that can be inherited) will be automatically replaced by: \\ 261 \\ 262 \begin{tabular}{|p{4cm}|p{8cm}|} 263 \hline 264 \centering part of the name automatically to be replaced & 265 by \\ 266 \hline 267 \hline 268 \centering @expname@ & 269 the experience name (from cn\_exp in the namelist) \\ 270 \hline 271 \centering @freq@ & 272 output frequency (from attribute output\_freq) \\ 273 \hline 274 \centering @startdate@ & 275 starting date of the simulation (from nn\_date0 in the restart or the namelist). \verb?yyyymmdd? format \\ 276 \hline 277 \centering @startdatefull@ & 278 starting date of the simulation (from nn\_date0 in the restart or the namelist). \verb?yyyymmdd_hh:mm:ss? format \\ 279 \hline 280 \centering @enddate@ & 281 ending date of the simulation (from nn\_date0 and nn\_itend in the namelist). \verb?yyyymmdd? format \\ 282 \hline 283 \centering @enddatefull@ & 284 ending date of the simulation (from nn\_date0 and nn\_itend in the namelist). \verb?yyyymmdd_hh:mm:ss? format \\ 285 \hline 286 \end{tabular} 287 \\ 288 289 For example, 290 291 \begin{alltt} {{\scriptsize 292 \begin{verbatim} 293 <file id="myfile_hzoom" name="myfile_@expname@_@startdate@_freq@freq@" output_freq="1d" > 294 \end{verbatim} 295 }}\end{alltt} 296 297 With, in the namelist: 298 299 \begin{alltt} {{\scriptsize 300 \begin{verbatim} 301 cn_exp = "ORCA2" 302 nn_date0 = 19891231 303 ln_rstart = .false. 304 \end{verbatim} 305 }}\end{alltt} 306 307 will give the following file name radical: 308 309 \begin{alltt} {{\scriptsize 310 \begin{verbatim} 311 myfile_ORCA2_19891231_freq1d 312 \end{verbatim} 313 }}\end{alltt} 314 315 316 \subsubsection{Other controls of the xml attributes from NEMO} 317 318 The values of some attributes are automatically defined by NEMO (and any definition given in the xml file is overwritten). By convention, these attributes are defined to ''auto'' (for string) or ''0000'' (for integer) in the xml file (but this is not necessary). 193 319 194 320 Here is the list of these attributes: \\ … … 202 328 \multicolumn{2}{|c|}{field\_definition} & freq\_op & \np{rn\_rdt} \\ 203 329 \hline 204 \multicolumn{2}{|c|}{SBC} & freq\_op & \np{rn\_rdt} $\times$ \np{nn\_fsbc} \\ 205 \hline 206 1h, 2h, 3h, 4h, 6h, 12h & \_grid\_T, \_grid\_U, & name & filename defined by \\ 207 1d, 3d, 5d & \_grid\_V, \_grid\_W, & & a call to rou{dia\_nam} \\ 208 1m, 2m, 3m, 4m, 6m & \_icemod, \_ptrc\_T, & & following NEMO \\ 209 1y, 2y, 5y, 10y & \_diad\_T, \_scalar & & nomenclature \\ 330 \multicolumn{2}{|c|}{SBC} & freq\_op & \np{rn\_rdt} $\times$ \np{nn\_fsbc} \\ 331 \hline 332 \multicolumn{2}{|c|}{ptrc\_T} & freq\_op & \np{rn\_rdt} $\times$ \np{nn\_dttrc} \\ 333 \hline 334 \multicolumn{2}{|c|}{diad\_T} & freq\_op & \np{rn\_rdt} $\times$ \np{nn\_dttrc} \\ 210 335 \hline 211 336 \multicolumn{2}{|c|}{EqT, EqU, EqW} & jbegin, ni, & according to the grid \\ 212 337 \multicolumn{2}{|c|}{ } & name\_suffix & \\ 213 338 \hline 214 \multicolumn{2}{|c|}{TAO, RAMA and PIRATA moorings} & ibegin, jbegin,& according to the grid \\339 \multicolumn{2}{|c|}{TAO, RAMA and PIRATA moorings} & zoom\_ibegin, zoom\_jbegin, & according to the grid \\ 215 340 \multicolumn{2}{|c|}{ } & name\_suffix & \\ 216 341 \hline … … 218 343 219 344 220 \subsection{ Detailed functionalities }221 222 345 \subsubsection{Tag list} 223 346 224 \begin{description} 225 226 \item[context]: define the model using the xml file. Id is the only attribute accepted. 227 Its value must be ''nemo'' or ''n\_nemo'' for the nth AGRIF zoom. Child of simulation tag. 228 229 \item[field]: define the field to be output. Accepted attributes are axis\_ref, description, enable, 230 freq\_op, grid\_ref, id (if child of field\_definition), level, operation, name, ref (if child of file), 231 unit, zoom\_ref. Child of field\_definition, file or group of fields tag. 232 233 \item[field\_definition]: definition of the part of the xml file corresponding to the field definition. 234 Accept the same attributes as field tag. Child of context tag. 235 236 \item[group]: define a group of file or field. Accept the same attributes as file or field. 237 238 \item[file]: define the output file's characteristics. Accepted attributes are description, enable, 239 output\_freq, output\_level, id, name, name\_suffix. Child of file\_definition or group of files tag. 240 241 \item[file\_definition]: definition of the part of the xml file corresponding to the file definition. 242 Accept the same attributes as file tag. Child of context tag. 243 244 \item[axis]: definition of the vertical axis. Accepted attributes are description, id, positive, size, unit. 245 Child of axis\_definition tag. 246 247 \item[axis\_definition]: definition of the part of the xml file corresponding to the vertical axis definition. 248 Accept the same attributes as axis tag. Child of context tag 249 250 \item[grid]: definition of the horizontal grid. Accepted attributes are description and id. 251 Child of axis\_definition tag. 252 253 \item[grid\_definition]: definition of the part of the xml file corresponding to the horizontal grid definition. 254 Accept the same attributes as grid tag. Child of context tag 255 256 \item[zoom]: definition of a subdomain of an horizontal grid. Accepted attributes are description, id, 257 i/jbegin, ni/j. Child of grid tag. 258 259 \end{description} 347 348 \begin{tabular}{|p{2cm}|p{2.5cm}|p{3.5cm}|p{2cm}|p{2cm}|} 349 \hline 350 tag name & 351 description & 352 accepted attribute & 353 child of & 354 parent of \\ 355 \hline 356 \hline 357 simulation & 358 this tag is the root tag which encapsulates all the content of the xml file & 359 none & 360 none & 361 context \\ 362 \hline 363 context & 364 encapsulates parts of the xml file dédicated to different codes or different parts of a code & 365 id (''xios'', ''nemo'' or ''n\_nemo'' for the nth AGRIF zoom), src, time\_origin & 366 simulation & 367 all root tags: ...\_definition \\ 368 \hline 369 \hline 370 field\_definition & 371 encapsulates the definition of all the fields that can potentially be outputted & 372 axis\_ref, default\_value, domain\_ref, enabled, grid\_ref, level, operation, prec, src & 373 context & 374 field or field\_group \\ 375 \hline 376 field\_group & 377 encapsulates a group of fields & 378 axis\_ref, default\_value, domain\_ref, enabled, group\_ref, grid\_ref, id, level, operation, prec, src & 379 field\_definition, field\_group, file & 380 field or field\_group \\ 381 \hline 382 field & 383 define a specific field & 384 axis\_ref, default\_value, domain\_ref, enabled, field\_ref, grid\_ref, id, level, long\_name, name, operation, prec, standard\_name, unit & 385 field\_definition, field\_group, file & 386 none \\ 387 \hline 388 \hline 389 file\_definition & 390 encapsulates the definition of all the files that will be outputted & 391 enabled, min\_digits, name, name\_suffix, output\_level, split\_format, split\_freq, sync\_freq, type, src & 392 context & 393 file or file\_group \\ 394 \hline 395 file\_group & 396 encapsulates a group of files that will be outputted & 397 enabled, description, id, min\_digits, name, name\_suffix, output\_freq, output\_level, split\_format, split\_freq, sync\_freq, type, src & 398 file\_definition, file\_group & 399 file or file\_group \\ 400 \hline 401 file & 402 defile the contentof a file to be outputted & 403 enabled, description, id, min\_digits, name, name\_suffix, output\_freq, output\_level, split\_format, split\_freq, sync\_freq, type, src & 404 file\_definition, file\_group & 405 field \\ 406 \hline 407 \end{tabular} 408 \begin{tabular}{|p{2cm}|p{2.5cm}|p{3.5cm}|p{2cm}|p{2cm}|} 409 \hline 410 tag name & 411 description & 412 accepted attribute & 413 child of & 414 parent of \\ 415 \hline 416 \hline 417 axis\_definition & 418 define all the vertical axis potentially used by the variables & 419 src & 420 context & 421 axis\_group, axis \\ 422 \hline 423 axis\_group & 424 encapsulates a group of vertical axis & 425 id, lon\_name, positive, src, standard\_name, unit, zoom\_begin, zoom\_end, zoom\_size & 426 axis\_definition, axis\_group & 427 axis\_group, axis \\ 428 \hline 429 axis & 430 define a vertical axis & 431 id, lon\_name, positive, src, standard\_name, unit, zoom\_begin, zoom\_end, zoom\_size & 432 axis\_definition, axis\_group & 433 none \\ 434 \hline 435 \hline 436 domain\_definition & 437 define all the horizontal domains potentially used by the variables & 438 src & 439 context & 440 domain\_group, domain \\ 441 \hline 442 domain\_group & 443 encapsulates a group of horizontal domains & 444 id, lon\_name, src, zoom\_ibegin, zoom\_jbegin, zoom\_ni, zoom\_nj & 445 domain\_definition, domain\_group & 446 domain\_group, domain \\ 447 \hline 448 domain & 449 define an horizontal domain & 450 id, lon\_name, src, zoom\_ibegin, zoom\_jbegin, zoom\_ni, zoom\_nj & 451 domain\_definition, domain\_group & 452 none \\ 453 \hline 454 \hline 455 grid\_definition & 456 define all the grid (association of a domain and/or an axis) potentially used by the variables & 457 src & 458 context & 459 grid\_group, grid \\ 460 \hline 461 grid\_group & 462 encapsulates a group of grids & 463 id, domain\_ref, axis\_ref & 464 grid\_definition, grid\_group & 465 grid\_group, grid \\ 466 \hline 467 grid & 468 define a grid & 469 id, domain\_ref, axis\_ref & 470 grid\_definition, grid\_group & 471 none \\ 472 \hline 473 \end{tabular} 260 474 261 475 262 476 \subsubsection{Attributes list} 263 477 264 Applied to a tag or a group of tags. 265 266 % table to be added ? 267 Another table, perhaps? 268 269 %%%% 270 271 Attribute 272 Applied to? 273 Definition 274 Comment 275 axis\_ref 276 field 277 String defining the vertical axis of the variable. It refers to the id of the vertical axis defined in the axis tag. 278 Use ''non'' if the variable has no vertical axis 279 280 %%%%%% 281 282 \begin{description} 283 284 \item[axis\_ref]: field attribute. String defining the vertical axis of the variable. 285 It refers to the id of the vertical axis defined in the axis tag. 286 Use ''none'' if the variable has no vertical axis 287 288 \item[description]: this attribute can be applied to all tags but it is used only with the field tag. 289 In this case, the value of description will be used to define, in the output netcdf file, 290 the attributes long\_name and standard\_name of the variable. 291 292 \item[enabled]: field and file attribute. Logical to switch on/off the output of a field or a file. 293 294 \item[freq\_op]: field attribute (automatically defined, see part 1.4 (''control of the xml attributes'')). 295 An integer defining the frequency in seconds at which NEMO is calling iom\_put for this variable. 296 It corresponds to the model time step (rn\_rdt in the namelist) except for the variables computed 297 at the frequency of the surface boundary condition (rn\_rdt ? nn\_fsbc in the namelist). 298 299 \item[grid\_ref]: field attribute. String defining the horizontal grid of the variable. 300 It refers to the id of the grid tag. 301 302 \item[ibegin]: zoom attribute. Integer defining the zoom starting point along x direction. 303 Automatically defined for TAO/RAMA/PIRATA moorings (see part 1.4). 304 305 \item[id]: exists for all tag. This is a string defining the name to a specific tag that will be used 306 later to refer to this tag. Tags of the same category must have different ids. 307 308 \item[jbegin]: zoom attribute. Integer defining the zoom starting point along y direction. 309 Automatically defined for TAO/RAMA/PIRATA moorings and equatorial section (see part 1.4). 310 311 \item[level]: field attribute. Integer from 0 to 10 defining the output priority of a field. 312 See output\_level attribute definition 313 314 \item[operation]: field attribute. String defining the type of temporal operation to perform on a variable. 315 Possible choices are ''ave(X)'' for temporal mean, ''inst(X)'' for instantaneous, ''t\_min(X)'' for temporal min 316 and ''t\_max(X)'' for temporal max. 317 318 \item[output\_freq]: file attribute. Integer defining the operation frequency in seconds. 319 For example 86400 for daily mean. 320 321 \item[output\_level]: file attribute. Integer from 0 to 10 defining the output priority of variables in a file: 322 all variables listed in the file with a level smaller or equal to output\_level will be output. 323 Other variables won't be output even if they are listed in the file. 324 325 \item[positive]: axis attribute (always .FALSE.). Logical defining the vertical axis convention used 326 in \NEMO (positive downward). Define the attribute positive of the variable in the netcdf output file. 327 328 \item[prec]: field attribute. Integer defining the output precision. 329 Not implemented, we always output real4 arrays. 330 331 \item[name]: field or file attribute. String defining the name of a variable or a file. 332 If the name of a file is undefined, its id is used as a name. 2 files must have different names. 333 Files with specific ids will have their name automatically defined (see part 1.4). 334 Note that is name will be automatically completed by the cpu number (if needed) and ''.nc'' 335 336 \item[name\_suffix]: file attribute. String defining a suffix to be inserted after the name 337 and before the cpu number and the ''.nc'' termination. Files with specific ids have an 338 automatic definition of their suffix (see part 1.4). 339 340 \item[ni]: zoom attribute. Integer defining the zoom extent along x direction. 341 Automatically defined for equatorial sections (see part 1.4). 342 343 \item[nj]: zoom attribute. Integer defining the zoom extent along x direction. 344 345 \item[ref]: field attribute. String referring to the id of the field we want to add in a file. 346 347 \item[size]: axis attribute. use unknown... 348 349 \item[unit]: field attribute. String defining the unit of a variable and the associated 350 attribute in the netcdf output file. 351 352 \item[zoom\_ref]: field attribute. String defining the subdomain of data on which 353 the file should be written (to ouput data only in a limited area). 354 It refers to the id of a zoom defined in the zoom tag. 355 \end{description} 356 357 358 \subsection{IO\_SERVER} 478 \begin{tabular}{|p{2cm}|p{4cm}|p{4cm}|p{2cm}|} 479 \hline 480 attribute name & 481 description & 482 example & 483 accepted by \\ 484 \hline 485 \hline 486 axis\_ref & 487 refers to the id of a vertical axis & 488 axis\_ref="deptht" & 489 field, grid families \\ 490 \hline 491 enabled & 492 switch on/off the output of a field or a file & 493 enabled=".TRUE." & 494 field, file families \\ 495 \hline 496 default\_value & 497 missing\_value definition & 498 default\_value="1.e20" & 499 field family \\ 500 \hline 501 description & 502 just for information, not used & 503 description="ocean T grid variables" & 504 all tags \\ 505 \hline 506 domain\_ref & 507 refers to the id of a domain & 508 domain\_ref="grid\_T" & 509 field or grid families \\ 510 \hline 511 field\_ref= & 512 id of the field we want to add in a file & 513 field\_ref="toce" & 514 field \\ 515 \hline 516 grid\_ref & 517 refers to the id of a grid & 518 grid\_ref="grid\_T\_2D" & 519 field family \\ 520 \hline 521 group\_ref & 522 refer to a group of variables & 523 group\_ref="mooring" & 524 field\_group \\ 525 \hline 526 id & 527 allow to identify a tag & 528 id="nemo" & 529 accepted by all tags except simulation \\ 530 \hline 531 level & 532 output priority of a field: 0 (high) to 10 (low)& 533 level="1" & 534 field family \\ 535 \hline 536 long\_name & 537 define the long\_name attribute in the NetCDF file & 538 long\_name="Vertical T levels" & 539 field \\ 540 \hline 541 min\_digits & 542 specify the minimum of digits used in the core number in the name of the NetCDF file & 543 min\_digits="4" & 544 file family \\ 545 \hline 546 name & 547 name of a variable or a file. If the name of a file is undefined, its id is used as a name & 548 name="tos" & 549 field or file families \\ 550 \hline 551 name\_suffix & 552 suffix to be inserted after the name and before the cpu number and the ''.nc'' termination of a file & 553 name\_suffix="\_myzoom" & 554 file family \\ 555 \hline 556 \end{tabular} 557 \begin{tabular}{|p{2cm}|p{4cm}|p{4cm}|p{2cm}|} 558 \hline 559 attribute name & 560 description & 561 example & 562 accepted by \\ 563 \hline 564 \hline 565 operation & 566 type of temporal operation: average, accumulate, instantaneous, min, max and once & 567 operation="average" & 568 field family \\ 569 \hline 570 output\_freq & 571 operation frequency. units can be ts (timestep), y, mo, d, h, mi, s. & 572 output\_freq="1d12h" & 573 field family \\ 574 \hline 575 output\_level & 576 output priority of variables in a file: 0 (high) to 10 (low). All variables listed in the file with a level smaller or equal to output\_level will be output. Other variables won't be output even if they are listed in the file. & 577 output\_level="10"& 578 file family \\ 579 \hline 580 positive & 581 convention used for the orientation of vertival axis (positive downward in \NEMO). & 582 positive="down" & 583 axis family \\ 584 \hline 585 prec & 586 output precision: real 4 or real 8 & 587 prec="4" & 588 field family \\ 589 \hline 590 split\_format & 591 date format used in the name of splitted output files. can be spécified using the following syntaxe: \%y, \%mo, \%d, \%h \%mi and \%s & 592 split\_format="\%yy\%mom\%dd" & 593 file family \\ 594 \hline 595 split\_freq & 596 split output files frequency. units can be ts (timestep), y, mo, d, h, mi, s. & 597 split\_freq="1mo" & 598 file family \\ 599 \hline 600 src & 601 allow to include a file & 602 src="./field\_def.xml" & 603 accepted by all tags except simulation \\ 604 \hline 605 standard\_name & 606 define the standard\_name attribute in the NetCDF file & 607 standard\_name="Eastward\_Sea\_Ice\_Transport" & 608 field \\ 609 \hline 610 sync\_freq & 611 NetCDF file synchronization frequency (update of the time\_counter). units can be ts (timestep), y, mo, d, h, mi, s. & 612 sync\_freq="10d" & 613 file family \\ 614 \hline 615 \end{tabular} 616 \begin{tabular}{|p{2cm}|p{4cm}|p{4cm}|p{2cm}|} 617 \hline 618 attribute name & 619 description & 620 example & 621 accepted by \\ 622 \hline 623 \hline 624 time\_origin & 625 specify the origin of the time counter & 626 time\_origin="1900-01-01 00:00:00"& 627 context \\ 628 \hline 629 type (1)& 630 specify if the output files must be splitted (multiple\_file) or not (one\_file) & 631 type="multiple\_file" & 632 file familly \\ 633 \hline 634 type (2)& 635 define the type of a variable tag & 636 type="boolean" & 637 variable \\ 638 \hline 639 unit & 640 unit of a variable or the vertical axis & 641 unit="m" & 642 field and axis families \\ 643 \hline 644 zoom\_ibegin & 645 starting point along x direction of the zoom. Automatically defined for TAO/RAMA/PIRATA moorings & 646 zoom\_ibegin="1" & 647 domain family \\ 648 \hline 649 zoom\_jbegin & 650 starting point along y direction of the zoom. Automatically defined for TAO/RAMA/PIRATA moorings & 651 zoom\_jbegin="1" & 652 domain family \\ 653 \hline 654 zoom\_ni & 655 zoom extent along x direction & 656 zoom\_ni="1" & 657 domain family \\ 658 \hline 659 zoom\_nj & 660 zoom extent along y direction & 661 zoom\_nj="1" & 662 domain family \\ 663 \hline 664 \end{tabular} 665 666 \subsection{XIOS: the IO\_SERVER} 359 667 360 668 \subsubsection{Attached or detached mode?} 361 669 362 Iom\_put is based on the io\_server developed by Yann Meurdesoif from IPSL 363 (see \href{http://forge.ipsl.jussieu.fr/ioserver/browser}{here} for the source code or 364 see its copy in NEMOGCM/EXTERNAL directory). 365 This server can be used in ''attached mode'' (as a library) or in ''detached mode'' 366 (as an external executable on n cpus). In attached mode, each cpu of NEMO will output 367 its own subdomain. In detached mode, the io\_server will gather data from NEMO 368 and output them split over n files with n the number of cpu dedicated to the io\_server. 369 370 \subsubsection{Control the io\_server: the namelist file xmlio\_server.def} 371 372 % 373 %Again, a small table might be more readable? 374 %Name 375 %Type 376 %Description 377 %Comment 378 %Using_server 379 %Logical 380 %Switch to use the server in attached or detached mode 381 %(.TRUE. corresponding to detached mode). 382 383 The control of the use of the io\_server is done through the namelist file of the io\_server 384 called xmlio\_server.def. 385 386 \textbf{using\_server}: logical, switch to use the server in attached or detached mode 387 (.TRUE. corresponding to detached mode). 388 389 \textbf{using\_oasis}: logical, set to .TRUE. if NEMO is used in coupled mode. 390 391 \textbf{client\_id} = ''oceanx'' : character, used only in coupled mode. 392 Specify the id used in OASIS to refer to NEMO. The same id must be used to refer to NEMO 393 in the \$NBMODEL part of OASIS namcouple in the call of prim\_init\_comp\_proto in cpl\_oasis3f90 394 395 \textbf{server\_id} = ''ionemo'' : character, used only in coupled mode. 396 Specify the id used in OASIS to refer to the IO\_SERVER when used in detached mode. 397 Use the same id to refer to the io\_server in the \$NBMODEL part of OASIS namcouple. 398 399 \textbf{global\_mpi\_buffer\_size}: integer; define the size in Mb of the MPI buffer used by the io\_server. 400 401 \subsubsection{Number of cpu used by the io\_server in detached mode} 402 403 The number of cpu used by the io\_server is specified only when launching the model. 404 Here is an example of 2 cpus for the io\_server and 6 cpu for opa using mpirun: 405 406 \texttt{ -p 2 -e ./ioserver} 407 408 \texttt{ -p 6 -e ./opa } 409 670 Iomput is based on \href{http://forge.ipsl.jussieu.fr/ioserver/wiki}{XIOS}, the io\_server developed by Yann Meurdesoif from IPSL. This server can be used in ''attached mode'' (as a library) or in ''detached mode'' (as an external executable on n cpus). The ''attached mode'' is simpler to use but much less efficient. If the type of file is ''multiple\_file'', then in attached(detached) mode, each NEMO(XIOS) process will output its own subdomain: if NEMO(XIOS) is runnning on N cores, the ouput files will be splitted into N files. If the type of file is ''one\_file'', the output files will be directly recombined into one unique file either in ''detached mode'' or ''attached mode''. 671 672 \subsubsection{Control of xios: the xios context in iodef.xml} 673 674 The control of the use of xios is done through the xios context in iodef.xml. 675 676 \begin{tabular}{|p{3cm}|p{6.5cm}|p{2.5cm}|} 677 \hline 678 variable name & 679 description & 680 example \\ 681 \hline 682 \hline 683 buffer\_size & 684 buffer size used by XIOS to send data from NEMO to XIOS. Larger is more efficient. Note that needed/used buffer sizes are summarized at the end of the job & 685 25000000 \\ 686 \hline 687 buffer\_server\_factor\_size & 688 ratio between NEMO and XIOS buffer size. Should be 2. & 689 2 \\ 690 \hline 691 info\_level & 692 verbosity level (0 to 100) & 693 0 \\ 694 \hline 695 using\_server & 696 activate attached(false) or detached(true) mode & 697 true \\ 698 \hline 699 using\_oasis & 700 xios is used with OASIS(true) or not (false) & 701 false \\ 702 \hline 703 oasis\_codes\_id & 704 when using oasis, define the identifier of NEMO in the namcouple. Not that the identifier of XIOS is xios.x & 705 oceanx \\ 706 \hline 707 \end{tabular} 708 709 \subsubsection{Number of cpu used by XIOS in detached mode} 710 711 The number of cores used by the xios is specified only when launching the model. The number or cores dedicated to XIOS should be from ~1/10 to ~1/50 of the number or cores dedicated to NEMO (according of the amount of data to be created). Here is an example of 2 cpus for the io\_server and 62 cpu for opa using mpirun: 712 713 \texttt{ mpirun -np 2 ./nemo.exe : -np 62 ./xios\_server.exe } 410 714 411 715 \subsection{Practical issues} … … 413 717 \subsubsection{Add your own outputs} 414 718 415 It is very easy to add you own outputs with iom \_put. 4 points must be followed.719 It is very easy to add you own outputs with iomput. 4 points must be followed. 416 720 \begin{description} 417 721 \item[1-] in NEMO code, add a \\ … … 429 733 <field_definition> 430 734 ... 431 <field id="identifier" description="blabla"/>735 <field id="identifier" long_name="blabla" ... /> 432 736 ... 433 737 </field_definition> 434 738 \end{verbatim} 435 739 }}\end{alltt} 436 attributes axis\_ref and grid\_ref must be consistent with the size of the array to pass to iom \_put.740 attributes axis\_ref and grid\_ref must be consistent with the size of the array to pass to iomput. 437 741 if your array is computed within the surface module each nn\_fsbc time\_step, 438 742 add the field definition within the group defined with the id ''SBC'': $<$group id=''SBC''...$>$ … … 442 746 \begin{alltt} {{\scriptsize 443 747 \begin{verbatim} 444 <file id="file _1" .../>748 <file id="file1" .../> 445 749 ... 446 750 <field ref="identifier" /> … … 451 755 452 756 \end{description} 453 454 \subsubsection{Several time axes in the output file}455 456 If your output file contains variables with different operations (see operation definition),457 IOIPSL will create one specific time axis for each operation. Note that inst(X) will have458 a time axis corresponding to the end each output period whereas all other operators459 will have a time axis centred in the middle of the output periods.460 461 \subsubsection{Error/bug messages from IOIPSL}462 463 If you get the following error in the standard output file:464 \vspace{-20pt}465 \begin{alltt} {{\scriptsize466 \begin{verbatim}467 FATAL ERROR FROM ROUTINE flio_dom_set468 --> too many domains simultaneously defined469 --> please unset useless domains470 --> by calling flio_dom_unset471 \end{verbatim}472 }}\end{alltt}473 474 You must increase the value of dom\_max\_nb in fliocom.f90 (multiply it by 10 for example).475 476 If you mix, in the same file, variables with different freq\_op (see definition above),477 like for example variables from the surface module with other variables,478 IOIPSL will print in the standard output file warning messages saying there may be a bug.479 \vspace{-20pt}480 \begin{alltt} {{\scriptsize481 \begin{verbatim}482 WARNING FROM ROUTINE histvar_seq483 --> There were 10 errors in the learned sequence of variables484 --> for file 4485 --> This looks like a bug, please report it.486 \end{verbatim}487 }}\end{alltt}488 489 Don't worry, there is no bug, everything is properly working!490 491 % } %end \gmcomment492 757 493 758
Note: See TracChangeset
for help on using the changeset viewer.