XIOS training 2021

March 15^th - 19^th, 2021 & April 12^th - 16^th, 2021

LSCE/IPSL, France

This training is funded and organized by the ​IS-ENES3 project, which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 824084

What is XIOS

The climate simulation models launched on super-calculators produce a significant amount of data. At this scale, the management of output files and post-treatment on data become the performance bottleneck for those climate models. In order to generate efficiently the output from models while taking advantage of the calculate resources, Institut Pierre Simon Laplace has launched the XIOS project. XIOS is a library dedicated to generate efficiently and easily the parallel input/output of climate models. It uses a client-server methodology in which clients (or models) will simulate the physics and send data asynchronously to servers. It is the XIOS servers who are in charge of writing data to files. In this way, we can smooth the I/O peaks by sending to servers a constant data flux all along the simulations and the I/O is overlapped with calculates. What's more, XIOS provides multiple functionalities which allow an "in situ" data post-treatment. For example, we can read the data from models, operate transformations on those data, combine data from different source models, and send results to multiple output files. All these operations are defined from an external XML file with a very compact and flexible way. Any modifications in this XML file will not require re-compilation of XIOS or models. XIOS is written in C++ and uses MPI for asynchronous communications between clients and servers. NETCDF, HDF5, and MPIIO are also used in XIOS for efficient file writing.

What is dr2xml

DR2XML is a python package useful for climente models whose I/O are handled by XIOS. It has been originally designed to facilitate the configuration of XIOS (automatic generation of XML files) so as to fully satisfy the CMIP6 DataRequest?. With DR2XML, the NetCDF files output by the model are directly CMOR3 compliant. Some extended functionalities were also developed so that DR2XML enables the user to set up his own Data Request. Consequently DR2XML can be used to customize the official CMIP6 DR or even to work off the CMIP6 framework, but benefiting from the CF-NetCDF, CMOR, and CMIP6 standards (file naming convention, file structure, variable definitions, file and variable attributes, etc.).

Program

• Day 1 & 2 :
  • Introduction on XIOS
    • Context of the project XIOS
    • How XIOS is useful for the climat simulation models
    • How to use XIOS in a model (Fortran interface)
  • Introduction of XML
    • XML syntax
    • XIOS component with XML (calendar, file, field, grid, ...)
  • Hands-on exercise
    • Install and compile XIOS
    • Pluge XIOS to a model
    • Hello World with XIOS
    • Visualization of the file generated by XIOS
• Day 3 & 4
  • Advanced tutorial of XIOS
    • Data transformation (temporal, spatial, ...)
    • Filters in XIOS
    • Active the workflow graph in XIOS
    • Client-server mode of XIOS
    • 2-level server mode of XIOS
    • How buffer works in XIOS
    • How to understand the XIOS performance report and parametrize XIOS
    • How to debug with XIOS (exception, stack, ...)
  • Hands-on exercise (interleaved with lecture)
• Day 5 :
  • Tutorial of dr2xml
    • Introduction of Data Request
    • Introduction of dr2xml:
    • Hands-on exercise of dr2xml
      • Get started with configuration files: settings, ping_file, home_data_request.
      • Visualize contents of generated XML files.
      • Production of netCDF files, conformality with CMIP6 norms.
    Q&A on XIOS and dr2xml

Slides of the training is now available

*XIOS Tutorial : Slides used for the training session

*Hands-on notes : Some notes for the hands-on exercises

Registration for this training

Registration is closed for both sessions.

Additional Information

Prerequisite :

• install virtualbox (​https://www.virtualbox.org/) on your PC
• we use Gather Town (​https://gather.town/) for lectures and hands-on exercises.

All useful information can be found here : ​https://docs.google.com/document/d/1zH5ZJ-J6lesaqiMNBHu8t-CkJ-i8Zx2ZkiBsYh3T9YE/edit?usp=sharing

If you have any questions about this training, please contact Yushan Wang (yushan.wang@…) of XIOS team.