Changeset 5107 for trunk/DOC/TexFiles


Ignore:
Timestamp:
2015-02-26T18:18:47+01:00 (7 years ago)
Author:
smasson
Message:

update IO for VVL compatibility, see ticket #1474

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/DOC/TexFiles/Chapters/Chap_DIA.tex

    r5003 r5107  
    7474The second functionality targets output performance when running in parallel (\key{mpp\_mpi}). Iomput provides the possibility to specify N dedicated I/O processes (in addition to the NEMO processes) to collect and write the outputs. With an appropriate choice of N by the user, the bottleneck associated with the writing of the output files can be greatly reduced.  
    7575 
    76 Since version 3.5, the iom\_put interface depends on an external code called \href{http://forge.ipsl.jussieu.fr/ioserver}{XIOS}. This new IO server can take advantage of the parallel I/O functionality of NetCDF4 to create a single output file and therefore to bypass the rebuilding phase. Note that writing in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput through XIOS are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3. 
     76In version 3.6, the iom\_put interface depends on an external code called \href{https://forge.ipsl.jussieu.fr/ioserver/browser/XIOS/branchs/xios-1.0}{XIOS-1.0}. This new IO server can take advantage of the parallel I/O functionality of NetCDF4 to create a single output file and therefore to bypass the rebuilding phase. Note that writing in parallel into the same NetCDF files requires that your NetCDF4 library is linked to an HDF5 library that has been correctly compiled (i.e. with the configure option $--$enable-parallel). Note that the files created by iomput through XIOS are incompatible with NetCDF3. All post-processsing and visualization tools must therefore be compatible with NetCDF4 and not only NetCDF3. 
    7777 
    7878Even if not using the parallel I/O functionality of NetCDF4, using N dedicated I/O servers, where N is typically much less than the number of NEMO processors, will reduce the number of output files created. This can greatly reduce the post-processing burden usually associated with using large numbers of NEMO processors. Note that for smaller configurations, the rebuilding phase can be avoided, even without a parallel-enabled NetCDF4 library, simply by employing only one dedicated I/O server. 
     
    543543\end{tabular} 
    544544 
     545\subsubsection{Advanced use of XIOS functionalities} 
    545546 
    546547\subsection{XML reference tables} 
     548 
     549(1) Simple computation: directly define the computation when refering to the variable in the file definition. 
     550 
     551\vspace{-20pt} 
     552\begin{alltt}  {{\scriptsize     
     553\begin{verbatim} 
     554 <field field\_ref="sst"  name="tosK"  unit="degK" > sst + 273.15 </field> 
     555 <field field\_ref="taum" name="taum2" unit="N2/m4" long\_name="square of wind stress module" > taum * taum </field> 
     556 <field field\_ref="qt"   name="stupid\_check" > qt - qsr - qns </field> 
     557\end{verbatim} 
     558}}\end{alltt}  
     559 
     560(2) Simple computation: define a new variable and use it in the file definition. 
     561 
     562in field\_definition: 
     563\vspace{-20pt} 
     564\begin{alltt}  {{\scriptsize     
     565\begin{verbatim} 
     566 <field id="sst2" long\_name="square of sea surface temperature" unit="degC2" >  sst * sst </field > 
     567\end{verbatim} 
     568}}\end{alltt}  
     569in file\_definition: 
     570\vspace{-20pt} 
     571\begin{alltt}  {{\scriptsize     
     572\begin{verbatim} 
     573 <field field\_ref="sst2" > sst2 </field> 
     574\end{verbatim} 
     575}}\end{alltt}  
     576Note that in this case, the following syntaxe $<$field field\_ref="sst2" /$>$ is not working as sst2 won't be evaluated. 
     577 
     578(3) Change of variable precision: 
     579 
     580\vspace{-20pt} 
     581\begin{alltt}  {{\scriptsize     
     582\begin{verbatim} 
     583     <!-- force to keep real 8 --> 
     584 <field field\_ref="sst" name="tos\_r8" prec="8" /> 
     585      <!-- integer 2  with add\_offset and scale\_factor attributes --> 
     586 <field field\_ref="sss" name="sos\_i2" prec="2" add\_offset="20." scale\_factor="1.e-3" /> 
     587\end{verbatim} 
     588}}\end{alltt}  
     589Note that, then the code is crashing, writting real4 variables forces a numerical convection from real8 to real4 which will create an internal error in NetCDF and will avoid the creation of the output files. Forcing double precision outputs with prec="8" (for example in the field\_definition) will avoid this problem. 
     590 
     591(4) add user defined attributes: 
     592 
     593\vspace{-20pt} 
     594\begin{alltt}  {{\scriptsize     
     595\begin{verbatim} 
     596      <file\_group id="1d" output\_freq="1d" output\_level="10" enabled=".TRUE."> <!-- 1d files -->  
     597   <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     598     <field field\_ref="sst" name="tos" > 
     599       <variable id="my\_attribute1" type="string"  > blabla </variable> 
     600       <variable id="my\_attribute2" type="integer" > 3      </variable> 
     601       <variable id="my\_attribute3" type="float"   > 5.0    </variable> 
     602     </field> 
     603     <variable id="my\_global\_attribute" type="string" > blabla\_global </variable> 
     604       </file> 
     605     </file\_group>  
     606\end{verbatim} 
     607}}\end{alltt}  
     608 
     609(5) use of the ``@'' function: example 1, weighted temporal average 
     610 
     611 - define a new variable in field\_definition 
     612\vspace{-20pt} 
     613\begin{alltt}  {{\scriptsize     
     614\begin{verbatim} 
     615 <field id="toce\_e3t" long\_name="temperature * e3t" unit="degC*m" grid\_ref="grid\_T\_3D" > toce * e3t </field > 
     616\end{verbatim} 
     617}}\end{alltt} 
     618 - use it when defining your file.   
     619\vspace{-20pt} 
     620\begin{alltt}  {{\scriptsize     
     621\begin{verbatim} 
     622<file\_group id="5d" output\_freq="5d"  output\_level="10" enabled=".TRUE." >  <!-- 5d files -->   
     623 <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     624  <field field\_ref="toce" operation="instant" freq\_op="5d" > @toce\_e3t / @e3t </field> 
     625 </file> 
     626</file\_group>  
     627\end{verbatim} 
     628}}\end{alltt} 
     629The freq\_op="5d" attribute is used to define the operation frequency of the ``@'' function: here 5 day. The temporal operation done by the ``@'' is the one defined in the field definition: here we use the default, average. So, in the above case, @toce\_e3t will do the 5-day mean of toce*e3t. Operation="instant" refers to the temporal operation to be performed on the field''@toce\_e3t / @e3t'': here the temporal average is alreday done by the ``@'' function so we just use instant to do the ratio of the 2 mean values. field\_ref="toce" means that attributes not explicitely defined, are inherited from toce field. Note that in this case, freq\_op must be equal to the file output\_freq. 
     630 
     631(6) use of the ``@'' function: example 2, monthly SSH standard deviation 
     632 
     633 - define a new variable in field\_definition 
     634\vspace{-20pt} 
     635\begin{alltt}  {{\scriptsize     
     636\begin{verbatim} 
     637 <field id="ssh2" long\_name="square of sea surface temperature" unit="degC2" >  ssh * ssh </field > 
     638\end{verbatim} 
     639}}\end{alltt}  
     640 - use it when defining your file.   
     641\vspace{-20pt} 
     642\begin{alltt}  {{\scriptsize     
     643\begin{verbatim} 
     644<file\_group id="1m" output\_freq="1m"  output\_level="10" enabled=".TRUE." >  <!-- 1m files -->   
     645 <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     646  <field field\_ref="ssh" name="sshstd" long\_name="sea\_surface\_temperature\_standard\_deviation" operation="instant" freq\_op="1m" > sqrt( @ssh2 - @ssh * @ssh ) </field> 
     647 </file> 
     648</file\_group>  
     649\end{verbatim} 
     650}}\end{alltt} 
     651The freq\_op="1m" attribute is used to define the operation frequency of the ``@'' function: here 1 month. The temporal operation done by the ``@'' is the one defined in the field definition: here we use the default, average. So, in the above case, @ssh2 will do the monthly mean of ssh*ssh. Operation="instant" refers to the temporal operation to be performed on the field ''sqrt( @ssh2 - @ssh * @ssh )'': here the temporal average is alreday done by the ``@'' function so we just use instant. field\_ref="ssh" means that attributes not explicitely defined, are inherited from ssh field. Note that in this case, freq\_op must be equal to the file output\_freq. 
     652 
     653(7) use of the ``@'' function: example 3, monthly average of SST diurnal cycle 
     654 
     655 - define 2 new variables in field\_definition 
     656\vspace{-20pt} 
     657\begin{alltt}  {{\scriptsize     
     658\begin{verbatim} 
     659 <field id="sstmax" field\_ref="sst" long\_name="max of sea surface temperature" operation="maximum" /> 
     660 <field id="sstmin" field\_ref="sst" long\_name="min of sea surface temperature" operation="minimum" /> 
     661\end{verbatim} 
     662}}\end{alltt}  
     663 - use these 2 new variables when defining your file.   
     664\vspace{-20pt} 
     665\begin{alltt}  {{\scriptsize     
     666\begin{verbatim} 
     667<file\_group id="1m" output\_freq="1m"  output\_level="10" enabled=".TRUE." >  <!-- 1m files -->   
     668 <file id="file1" name\_suffix="\_grid\_T" description="ocean T grid variables" > 
     669  <field field\_ref="sst" name="sstdcy" long\_name="amplitude of sst diurnal cycle" operation="average" freq\_op="1d" > @sstmax - @sstmin </field> 
     670 </file> 
     671</file\_group>  
     672\end{verbatim} 
     673}}\end{alltt} 
     674The freq\_op="1d" attribute is used to define the operation frequency of the ``@'' function: here 1 day. The temporal operation done by the ``@'' is the one defined in the field definition: here maximum for sstmax and minimum for sstmin. So, in the above case, @sstmax will do the daily max and @sstmin the daily min. Operation="average" refers to the temporal operation to be performed on the field ``@sstmax - @sstmin'': here monthly mean (of daily max - daily min of the sst). field\_ref="sst" means that attributes not explicitely defined, are inherited from sst field. 
     675 
     676 
    547677 
    548678\subsubsection{Tag list} 
Note: See TracChangeset for help on using the changeset viewer.