HPC-07(2017WP) Write restart file using XIOS

[andmirek]

Context

Implementation of XIOS restart write interface in trunk (see ticket #1882 for version 3.6)

Proposal

Write an interface to use XIOS to write restart file (core variables only,the ones written to numror)

[andmirek]

Branch dev_r8600_xios_write: ​http://forge.ipsl.jussieu.fr/nemo/log/branches/2017/dev_r8600_xios_write created.

[andmirek]

To enable write functionality nn_wxios must be set to value larger than 0. nn_wxios = 0 - uses NEMO interface to write restart; nn_write = 1 - XIOS writes restart to a single file; nn_write = 2 - XIOS writes restart to a multiple files (one per server).

No changes in iodef.xml are required, all definitions are in NEMO.

For AGRIF configuration setting for AGRIF is ignored and NEMO value is used.

[andmirek]

comments by Tim Graham:
1)) The purpose of writing restart files via XIOS. I guess the main purpose is preparation for being able to read them with XIOS but I think it’s important to clarify this. You could also comment on the advantage of being able to write to a single restart file instead of having to rebuild it first.

On a HPC system writing multiple files at the same time may impact I/O performance. This situation may happen during writing restart file. In NEMO all processors write the restart file at the end of the simulation at the same time. With XIOS number of files written at the same time may be significantly reduced; either to the number of XIOS servers or to a single file if necessary. Because writing to multiple files may be faster than to a single file rebuild of the restart will still be necessary when performance is major concern (and there is a need to use single file restart, for instance when change in processors decomposition is needed). Rebuild may be easier and fasted when there are only few sub-domains to rebuild. In this ticket only data written to numror is considered. NEMO restart file is the biggest file and for high resolution configuration (ORCA12) the rebuild is time consuming process, especially when netcdf4 is used.

2) Is it necessary to have the lxios argument to iom_rstput. Why not just check the variable locally withing the iom_rstput subroutine instead of passing it as an argument every time.

iom_rstput is a general subroutine and not all output (files) have to be written by XIOS. I think the implementation with argument is more general.

3) The NEMO coding conventions state that the variables should be l_xios and l_wxios (assuming that they are logicals). If they are integers then they should start with n (i.e. n_xios).

done in r8662

4) There are a lot of places where it looks like lines of code have changed but it is just spaces have been replaced by tabs (or vice versa). This is a pain at the merge party so it would be good to try to revert these changes if possible.

done in r8661

[andmirek]

Results from tests performed to evaluate XIOS write functionality for Met Office GO6 configuration, for 2 model resolutions: ORCA25 and ORCA12. NEMO 3.6 was used for tests. Runs were for 1 day.

NEMO - default NEMO functionality for reading single file restart
XIOS - XIOS read interface for reading single file restart

For XIOS data was written to ONE file and MULTIPLE files

Results: eORCA12:

	Execution time [s]	Execution time [s]
CPU (decomposition)	XIOS ONE/MULTIPLE (36 servers)	NEMO
1602 (50x50)	1513/1195	1119
2262 (60x60)	1556/1185	1066
3027 (70x70)	1871/1644	1062

Results: eORCA25:

	Execution time [s]	Execution time [s]
CPU (decomposition)	XIOS ONE/MULTIPLE (6 servers)	NEMO
120 (10x12)	645/613	560
440 (20x24)	308/396	246
1440 (40x36)	284/260	173

[andmirek]

In 8990:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[nicolasmartin]

Modification of summary for standardisation

[mathiot]

In 9367:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 9405:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 9535:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 9536:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found