Changeset 5600 for branches/2014/dev_r4650_UKMO14.12_STAND_ALONE_OBSOPER/DOC/TexFiles/Chapters/Chap_DIA.tex

Timestamp:

2015-07-15T17:46:12+02:00 (9 years ago)

Author:

andrewryan

Message:

merged in latest version of trunk alongside changes to SAO_SRC to be compatible with latest OBS

File:

• branches/2014/dev_r4650_UKMO14.12_STAND_ALONE_OBSOPER/DOC/TexFiles/Chapters/Chap_DIA.tex (modified) (3 diffs)

branches/2014/dev_r4650_UKMO14.12_STAND_ALONE_OBSOPER/DOC/TexFiles/Chapters/Chap_DIA.tex

@@ -74 +74 @@
 The second functionality targets output performance when running in parallel (\key{mpp\_mpi}). Iomput provides the possibility to specify N dedicated I/O processes (in addition to the NEMO processes) to collect and write the outputs. With an appropriate choice of N by the user, the bottleneck associated with the writing of the output files can be greatly reduced. 
 
-Since version 3.5, the iom\_put interface depends on an external code called \href{http://forge.ipsl.jussieu.fr/ioserver}{XIOS}. This new IO server can take advantage of the parallel I/O functionality of NetCDF4 to create a single output file and therefore to bypass the rebuilding phase. Note that writing in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput through XIOS are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3.
+In version 3.6, the iom\_put interface depends on an external code called \href{https://forge.ipsl.jussieu.fr/ioserver/browser/XIOS/branchs/xios-1.0}{XIOS-1.0} (use of revision 618 or higher is required). This new IO server can take advantage of the parallel I/O functionality of NetCDF4 to create a single output file and therefore to bypass the rebuilding phase. Note that writing in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput through XIOS are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3.
 
 Even if not using the parallel I/O functionality of NetCDF4, using N dedicated I/O servers, where N is typically much less than the number of NEMO processors, will reduce the number of output files created. This can greatly reduce the post-processing burden usually associated with using large numbers of NEMO processors. Note that for smaller configurations, the rebuilding phase can be avoided, even without a parallel-enabled NetCDF4 library, simply by employing only one dedicated I/O server.
@@ -543 +543 @@
 \end{tabular}
 
+\subsubsection{Advanced use of XIOS functionalities}
 
 \subsection{XML reference tables}
+\label{IOM_xmlref}
+
+(1) Simple computation: directly define the computation when refering to the variable in the file definition.
+
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+ <field field\_ref="sst"  name="tosK"  unit="degK" > sst + 273.15 </field>
+ <field field\_ref="taum" name="taum2" unit="N2/m4" long\_name="square of wind stress module" > taum * taum </field>
+ <field field\_ref="qt"   name="stupid\_check" > qt - qsr - qns </field>
+\end{verbatim}
+}}\end{alltt} 
+
+(2) Simple computation: define a new variable and use it in the file definition.
+
+in field\_definition:
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+ <field id="sst2" long\_name="square of sea surface temperature" unit="degC2" >  sst * sst </field >
+\end{verbatim}
+}}\end{alltt} 
+in file\_definition:
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+ <field field\_ref="sst2" > sst2 </field>
+\end{verbatim}
+}}\end{alltt} 
+Note that in this case, the following syntaxe $<$field field\_ref="sst2" /$>$ is not working as sst2 won't be evaluated.
+
+(3) Change of variable precision:
+
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+     <!-- force to keep real 8 -->
+ <field field\_ref="sst" name="tos\_r8" prec="8" />
+      <!-- integer 2  with add\_offset and scale\_factor attributes -->
+ <field field\_ref="sss" name="sos\_i2" prec="2" add\_offset="20." scale\_factor="1.e-3" />
+\end{verbatim}
+}}\end{alltt} 
+Note that, then the code is crashing, writting real4 variables forces a numerical convection from real8 to real4 which will create an internal error in NetCDF and will avoid the creation of the output files. Forcing double precision outputs with prec="8" (for example in the field\_definition) will avoid this problem.
+
+(4) add user defined attributes:
+
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+      <file\_group id="1d" output\_freq="1d" output\_level="10" enabled=".TRUE."> <!-- 1d files --> 
+   <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" >
+     <field field\_ref="sst" name="tos" >
+       <variable id="my\_attribute1" type="string"  > blabla </variable>
+       <variable id="my\_attribute2" type="integer" > 3      </variable>
+       <variable id="my\_attribute3" type="float"   > 5.0    </variable>
+     </field>
+     <variable id="my\_global\_attribute" type="string" > blabla\_global </variable>
+       </file>
+     </file\_group> 
+\end{verbatim}
+}}\end{alltt} 
+
+(5) use of the ``@'' function: example 1, weighted temporal average
+
+ - define a new variable in field\_definition
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+ <field id="toce\_e3t" long\_name="temperature * e3t" unit="degC*m" grid\_ref="grid\_T\_3D" > toce * e3t </field >
+\end{verbatim}
+}}\end{alltt}
+ - use it when defining your file.  
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+<file\_group id="5d" output\_freq="5d"  output\_level="10" enabled=".TRUE." >  <!-- 5d files -->  
+ <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" >
+  <field field\_ref="toce" operation="instant" freq\_op="5d" > @toce\_e3t / @e3t </field>
+ </file>
+</file\_group> 
+\end{verbatim}
+}}\end{alltt}
+The freq\_op="5d" attribute is used to define the operation frequency of the ``@'' function: here 5 day. The temporal operation done by the ``@'' is the one defined in the field definition: here we use the default, average. So, in the above case, @toce\_e3t will do the 5-day mean of toce*e3t. Operation="instant" refers to the temporal operation to be performed on the field''@toce\_e3t / @e3t'': here the temporal average is alreday done by the ``@'' function so we just use instant to do the ratio of the 2 mean values. field\_ref="toce" means that attributes not explicitely defined, are inherited from toce field. Note that in this case, freq\_op must be equal to the file output\_freq.
+
+(6) use of the ``@'' function: example 2, monthly SSH standard deviation
+
+ - define a new variable in field\_definition
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+ <field id="ssh2" long\_name="square of sea surface temperature" unit="degC2" >  ssh * ssh </field >
+\end{verbatim}
+}}\end{alltt} 
+ - use it when defining your file.  
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+<file\_group id="1m" output\_freq="1m"  output\_level="10" enabled=".TRUE." >  <!-- 1m files -->  
+ <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" >
+  <field field\_ref="ssh" name="sshstd" long\_name="sea\_surface\_temperature\_standard\_deviation" operation="instant" freq\_op="1m" > sqrt( @ssh2 - @ssh * @ssh ) </field>
+ </file>
+</file\_group> 
+\end{verbatim}
+}}\end{alltt}
+The freq\_op="1m" attribute is used to define the operation frequency of the ``@'' function: here 1 month. The temporal operation done by the ``@'' is the one defined in the field definition: here we use the default, average. So, in the above case, @ssh2 will do the monthly mean of ssh*ssh. Operation="instant" refers to the temporal operation to be performed on the field ''sqrt( @ssh2 - @ssh * @ssh )'': here the temporal average is alreday done by the ``@'' function so we just use instant. field\_ref="ssh" means that attributes not explicitely defined, are inherited from ssh field. Note that in this case, freq\_op must be equal to the file output\_freq.
+
+(7) use of the ``@'' function: example 3, monthly average of SST diurnal cycle
+
+ - define 2 new variables in field\_definition
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+ <field id="sstmax" field\_ref="sst" long\_name="max of sea surface temperature" operation="maximum" />
+ <field id="sstmin" field\_ref="sst" long\_name="min of sea surface temperature" operation="minimum" />
+\end{verbatim}
+}}\end{alltt} 
+ - use these 2 new variables when defining your file.  
+\vspace{-20pt}
+\begin{alltt}  {{\scriptsize    
+\begin{verbatim}
+<file\_group id="1m" output\_freq="1m"  output\_level="10" enabled=".TRUE." >  <!-- 1m files -->  
+ <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" >
+  <field field\_ref="sst" name="sstdcy" long\_name="amplitude of sst diurnal cycle" operation="average" freq\_op="1d" > @sstmax - @sstmin </field>
+ </file>
+</file\_group> 
+\end{verbatim}
+}}\end{alltt}
+The freq\_op="1d" attribute is used to define the operation frequency of the ``@'' function: here 1 day. The temporal operation done by the ``@'' is the one defined in the field definition: here maximum for sstmax and minimum for sstmin. So, in the above case, @sstmax will do the daily max and @sstmin the daily min. Operation="average" refers to the temporal operation to be performed on the field ``@sstmax - @sstmin'': here monthly mean (of daily max - daily min of the sst). field\_ref="sst" means that attributes not explicitely defined, are inherited from sst field.
+
+
 
 \subsubsection{Tag list}
@@ -849 +980 @@
 \end{longtable}
 
+\subsection{CF metadata standard compliance}
+
+Output from the XIOS-1.0 IO server is compliant with \href{http://cfconventions.org/Data/cf-conventions/cf-conventions-1.5/build/cf-conventions.html}{version 1.5} of the CF metadata standard. Therefore while a user may wish to add their own metadata to the output files (as demonstrated in example 4 of section \ref{IOM_xmlref}) the metadata should, for the most part, comply with the CF-1.5 standard.
+
+Some metadata that may significantly increase the file size (horizontal cell areas and vertices) are controlled by the namelist parameter \np{ln\_cfmeta} in the \ngn{namrun} namelist. This must be set to true if these metadata are to be included in the output files.