Ignore:
Timestamp:
2015-07-15T17:46:12+02:00 (5 years ago)
Author:
andrewryan
Message:

merged in latest version of trunk alongside changes to SAO_SRC to be compatible with latest OBS

File:
1 edited

Legend:

Unmodified
Added
Removed
  • branches/2014/dev_r4650_UKMO14.12_STAND_ALONE_OBSOPER/DOC/TexFiles/Chapters/Chap_DIA.tex

    r5034 r5600  
    7474The second functionality targets output performance when running in parallel (\key{mpp\_mpi}). Iomput provides the possibility to specify N dedicated I/O processes (in addition to the NEMO processes) to collect and write the outputs. With an appropriate choice of N by the user, the bottleneck associated with the writing of the output files can be greatly reduced.  
    7575 
    76 Since version 3.5, the iom\_put interface depends on an external code called \href{http://forge.ipsl.jussieu.fr/ioserver}{XIOS}. This new IO server can take advantage of the parallel I/O functionality of NetCDF4 to create a single output file and therefore to bypass the rebuilding phase. Note that writing in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput through XIOS are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3. 
     76In version 3.6, the iom\_put interface depends on an external code called \href{https://forge.ipsl.jussieu.fr/ioserver/browser/XIOS/branchs/xios-1.0}{XIOS-1.0} (use of revision 618 or higher is required). This new IO server can take advantage of the parallel I/O functionality of NetCDF4 to create a single output file and therefore to bypass the rebuilding phase. Note that writing in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput through XIOS are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3. 
    7777 
    7878Even if not using the parallel I/O functionality of NetCDF4, using N dedicated I/O servers, where N is typically much less than the number of NEMO processors, will reduce the number of output files created. This can greatly reduce the post-processing burden usually associated with using large numbers of NEMO processors. Note that for smaller configurations, the rebuilding phase can be avoided, even without a parallel-enabled NetCDF4 library, simply by employing only one dedicated I/O server. 
     
    543543\end{tabular} 
    544544 
     545\subsubsection{Advanced use of XIOS functionalities} 
    545546 
    546547\subsection{XML reference tables} 
     548\label{IOM_xmlref} 
     549 
     550(1) Simple computation: directly define the computation when refering to the variable in the file definition. 
     551 
     552\vspace{-20pt} 
     553\begin{alltt}  {{\scriptsize     
     554\begin{verbatim} 
     555 <field field\_ref="sst"  name="tosK"  unit="degK" > sst + 273.15 </field> 
     556 <field field\_ref="taum" name="taum2" unit="N2/m4" long\_name="square of wind stress module" > taum * taum </field> 
     557 <field field\_ref="qt"   name="stupid\_check" > qt - qsr - qns </field> 
     558\end{verbatim} 
     559}}\end{alltt}  
     560 
     561(2) Simple computation: define a new variable and use it in the file definition. 
     562 
     563in field\_definition: 
     564\vspace{-20pt} 
     565\begin{alltt}  {{\scriptsize     
     566\begin{verbatim} 
     567 <field id="sst2" long\_name="square of sea surface temperature" unit="degC2" >  sst * sst </field > 
     568\end{verbatim} 
     569}}\end{alltt}  
     570in file\_definition: 
     571\vspace{-20pt} 
     572\begin{alltt}  {{\scriptsize     
     573\begin{verbatim} 
     574 <field field\_ref="sst2" > sst2 </field> 
     575\end{verbatim} 
     576}}\end{alltt}  
     577Note that in this case, the following syntaxe $<$field field\_ref="sst2" /$>$ is not working as sst2 won't be evaluated. 
     578 
     579(3) Change of variable precision: 
     580 
     581\vspace{-20pt} 
     582\begin{alltt}  {{\scriptsize     
     583\begin{verbatim} 
     584     <!-- force to keep real 8 --> 
     585 <field field\_ref="sst" name="tos\_r8" prec="8" /> 
     586      <!-- integer 2  with add\_offset and scale\_factor attributes --> 
     587 <field field\_ref="sss" name="sos\_i2" prec="2" add\_offset="20." scale\_factor="1.e-3" /> 
     588\end{verbatim} 
     589}}\end{alltt}  
     590Note that, then the code is crashing, writting real4 variables forces a numerical convection from real8 to real4 which will create an internal error in NetCDF and will avoid the creation of the output files. Forcing double precision outputs with prec="8" (for example in the field\_definition) will avoid this problem. 
     591 
     592(4) add user defined attributes: 
     593 
     594\vspace{-20pt} 
     595\begin{alltt}  {{\scriptsize     
     596\begin{verbatim} 
     597      <file\_group id="1d" output\_freq="1d" output\_level="10" enabled=".TRUE."> <!-- 1d files -->  
     598   <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     599     <field field\_ref="sst" name="tos" > 
     600       <variable id="my\_attribute1" type="string"  > blabla </variable> 
     601       <variable id="my\_attribute2" type="integer" > 3      </variable> 
     602       <variable id="my\_attribute3" type="float"   > 5.0    </variable> 
     603     </field> 
     604     <variable id="my\_global\_attribute" type="string" > blabla\_global </variable> 
     605       </file> 
     606     </file\_group>  
     607\end{verbatim} 
     608}}\end{alltt}  
     609 
     610(5) use of the ``@'' function: example 1, weighted temporal average 
     611 
     612 - define a new variable in field\_definition 
     613\vspace{-20pt} 
     614\begin{alltt}  {{\scriptsize     
     615\begin{verbatim} 
     616 <field id="toce\_e3t" long\_name="temperature * e3t" unit="degC*m" grid\_ref="grid\_T\_3D" > toce * e3t </field > 
     617\end{verbatim} 
     618}}\end{alltt} 
     619 - use it when defining your file.   
     620\vspace{-20pt} 
     621\begin{alltt}  {{\scriptsize     
     622\begin{verbatim} 
     623<file\_group id="5d" output\_freq="5d"  output\_level="10" enabled=".TRUE." >  <!-- 5d files -->   
     624 <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     625  <field field\_ref="toce" operation="instant" freq\_op="5d" > @toce\_e3t / @e3t </field> 
     626 </file> 
     627</file\_group>  
     628\end{verbatim} 
     629}}\end{alltt} 
     630The freq\_op="5d" attribute is used to define the operation frequency of the ``@'' function: here 5 day. The temporal operation done by the ``@'' is the one defined in the field definition: here we use the default, average. So, in the above case, @toce\_e3t will do the 5-day mean of toce*e3t. Operation="instant" refers to the temporal operation to be performed on the field''@toce\_e3t / @e3t'': here the temporal average is alreday done by the ``@'' function so we just use instant to do the ratio of the 2 mean values. field\_ref="toce" means that attributes not explicitely defined, are inherited from toce field. Note that in this case, freq\_op must be equal to the file output\_freq. 
     631 
     632(6) use of the ``@'' function: example 2, monthly SSH standard deviation 
     633 
     634 - define a new variable in field\_definition 
     635\vspace{-20pt} 
     636\begin{alltt}  {{\scriptsize     
     637\begin{verbatim} 
     638 <field id="ssh2" long\_name="square of sea surface temperature" unit="degC2" >  ssh * ssh </field > 
     639\end{verbatim} 
     640}}\end{alltt}  
     641 - use it when defining your file.   
     642\vspace{-20pt} 
     643\begin{alltt}  {{\scriptsize     
     644\begin{verbatim} 
     645<file\_group id="1m" output\_freq="1m"  output\_level="10" enabled=".TRUE." >  <!-- 1m files -->   
     646 <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     647  <field field\_ref="ssh" name="sshstd" long\_name="sea\_surface\_temperature\_standard\_deviation" operation="instant" freq\_op="1m" > sqrt( @ssh2 - @ssh * @ssh ) </field> 
     648 </file> 
     649</file\_group>  
     650\end{verbatim} 
     651}}\end{alltt} 
     652The freq\_op="1m" attribute is used to define the operation frequency of the ``@'' function: here 1 month. The temporal operation done by the ``@'' is the one defined in the field definition: here we use the default, average. So, in the above case, @ssh2 will do the monthly mean of ssh*ssh. Operation="instant" refers to the temporal operation to be performed on the field ''sqrt( @ssh2 - @ssh * @ssh )'': here the temporal average is alreday done by the ``@'' function so we just use instant. field\_ref="ssh" means that attributes not explicitely defined, are inherited from ssh field. Note that in this case, freq\_op must be equal to the file output\_freq. 
     653 
     654(7) use of the ``@'' function: example 3, monthly average of SST diurnal cycle 
     655 
     656 - define 2 new variables in field\_definition 
     657\vspace{-20pt} 
     658\begin{alltt}  {{\scriptsize     
     659\begin{verbatim} 
     660 <field id="sstmax" field\_ref="sst" long\_name="max of sea surface temperature" operation="maximum" /> 
     661 <field id="sstmin" field\_ref="sst" long\_name="min of sea surface temperature" operation="minimum" /> 
     662\end{verbatim} 
     663}}\end{alltt}  
     664 - use these 2 new variables when defining your file.   
     665\vspace{-20pt} 
     666\begin{alltt}  {{\scriptsize     
     667\begin{verbatim} 
     668<file\_group id="1m" output\_freq="1m"  output\_level="10" enabled=".TRUE." >  <!-- 1m files -->   
     669 <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     670  <field field\_ref="sst" name="sstdcy" long\_name="amplitude of sst diurnal cycle" operation="average" freq\_op="1d" > @sstmax - @sstmin </field> 
     671 </file> 
     672</file\_group>  
     673\end{verbatim} 
     674}}\end{alltt} 
     675The freq\_op="1d" attribute is used to define the operation frequency of the ``@'' function: here 1 day. The temporal operation done by the ``@'' is the one defined in the field definition: here maximum for sstmax and minimum for sstmin. So, in the above case, @sstmax will do the daily max and @sstmin the daily min. Operation="average" refers to the temporal operation to be performed on the field ``@sstmax - @sstmin'': here monthly mean (of daily max - daily min of the sst). field\_ref="sst" means that attributes not explicitely defined, are inherited from sst field. 
     676 
     677 
    547678 
    548679\subsubsection{Tag list} 
     
    849980\end{longtable} 
    850981 
     982\subsection{CF metadata standard compliance} 
     983 
     984Output from the XIOS-1.0 IO server is compliant with \href{http://cfconventions.org/Data/cf-conventions/cf-conventions-1.5/build/cf-conventions.html}{version 1.5} of the CF metadata standard. Therefore while a user may wish to add their own metadata to the output files (as demonstrated in example 4 of section \ref{IOM_xmlref}) the metadata should, for the most part, comply with the CF-1.5 standard. 
     985 
     986Some metadata that may significantly increase the file size (horizontal cell areas and vertices) are controlled by the namelist parameter \np{ln\_cfmeta} in the \ngn{namrun} namelist. This must be set to true if these metadata are to be included in the output files. 
    851987 
    852988 
Note: See TracChangeset for help on using the changeset viewer.