Opened 2 years ago

Closed 2 years ago

Last modified 21 months ago

#1962 closed Task (fixed)

HPC-07(2017WP) Write restart file using XIOS

Reported by: andmirek Owned by: andmirek
Priority: high Milestone: 2018 release-4.0
Component: OCE Version: trunk
Severity: major Keywords:
Cc: Review: failed
MP ready?: no
Progress: In the trunk. Currently tested in MO GO8 (eORCA25) configuration.

Description

Context

Implementation of XIOS restart write interface in trunk (see ticket #1882 for version 3.6)

Proposal

Write an interface to use XIOS to write restart file (core variables only,the ones written to numror)

Commit History (30)

ChangesetAuthorTimeChangeLog
9536andmirek2018-05-01T13:37:48+02:00

#1953 and #1962 stop immediately if XIOS restart read/write functionality is used without key_iomput

9535andmirek2018-05-01T13:22:33+02:00

#1953 and #1962 disable XIOS related calls when key_iomput is not set

9405andmirek2018-03-15T17:29:06+01:00

#1953 and #1962: Initialize lwxios and ln_xios_read

9367mathiot2018-02-28T17:23:20+01:00

Add restart read/write via XIOS capability (#1953 and #1962 and twiki: 2017WP/Met_Office-1_Mirek_XIOSread). WARNING: need to upgrade XIOS to r1296 to compile

8990andmirek2017-12-12T15:56:38+01:00

#1953 and #1962: fix ORCA2_OFF_PISCES test

8858andmirek2017-11-30T15:50:38+01:00

#1953 and #1962 tidy up prints for previous commit

8857andmirek2017-11-30T15:24:15+01:00

#1953 and #1962 handle together setting XIOS restart read and write flags

8836andmirek2017-11-28T15:45:20+01:00

#1953 and #1962 Techinal corrections

8821andmirek2017-11-27T14:46:12+01:00

#1953 and #1962 disable restart reading with XIOS for SAS

8816andmirek2017-11-27T12:01:56+01:00

#1953 and #1962 XIOS read/write in SAS

8812andmirek2017-11-24T17:34:50+01:00

#1953 and #1962 enable restart read/write with XIOS in SAS

8802andmirek2017-11-23T17:48:06+01:00

#1953 and #1962 lrxios in SAS

8801andmirek2017-11-23T16:52:17+01:00

#1953 and #1962 merge dev_r8600_xios_read_write r8793

8800andmirek2017-11-23T16:48:37+01:00

#1953 and #1962 merged dev_r8600_xios_read beanch r8668

8799andmirek2017-11-23T16:45:20+01:00

#1953 and #1962 version 2 of merged XIOS read/write branch

8793andmirek2017-11-23T13:02:23+01:00

#1953 and #1962 change lxios_read to lrxios to be consistent with write branch

8792andmirek2017-11-23T12:29:39+01:00

#1953 and #1962 rename rst_fields to rst_rfields

8786andmirek2017-11-22T14:39:57+01:00

#1953 and #1962 remove print statement

8785andmirek2017-11-22T13:34:53+01:00

#1953 and #1962 change in subroutine name and way temperature first level is read

8774andmirek2017-11-21T17:54:04+01:00

#1953 and #1962 remove duplicated set_rst_context

8770andmirek2017-11-21T17:21:22+01:00

#1953 and #1962 merged (and some modifications) with write branch

8763andmirek2017-11-21T11:28:18+01:00

#1962 correct name of XIOS write context

8760andmirek2017-11-21T10:44:53+01:00

#1962 bug fix

8759andmirek2017-11-21T09:57:42+01:00

#1953 and #1962 XIOS read/write branch

8662andmirek2017-10-26T13:15:54+02:00

#1962 change variable names to follow NEMO coding convention

8661andmirek2017-10-26T12:06:21+02:00

#1962 put back tabs, spaces and lines removed when implementiong XIOS write functionality

8644andmirek2017-10-20T10:04:07+02:00

ticket #1962 xios write functionality works

8630andmirek2017-10-17T14:31:49+02:00

#1962 merge with branches/UKMO/dev_r7573_xios_write (doesn't woork)

8622andmirek2017-10-12T18:38:26+02:00

ticket #1962 - write restart file with XIOS

8620andmirek2017-10-12T16:18:23+02:00

ticket #1962 - write restart file with XIOS

Change History (17)

comment:1 Changed 2 years ago by andmirek

  • Milestone set to 2017 WP
  • Owner set to andmirek
  • Status changed from new to assigned

comment:3 Changed 2 years ago by andmirek

To enable write functionality nn_wxios must be set to value larger than 0 (namelist namrun). nn_wxios = 0 - uses NEMO interface to write restart; nn_write = 1 - XIOS writes restart to a single file; nn_write = 2 - XIOS writes restart to a multiple files (one per server).

No changes in iodef.xml are required, all definitions are in NEMO.

For AGRIF configuration setting for AGRIF is ignored and NEMO value is used.

Last edited 2 years ago by andmirek (previous) (diff)

comment:4 Changed 2 years ago by clevy

  • Type changed from Enhancement to Task

comment:5 Changed 2 years ago by andmirek

  • Type changed from Task to Enhancement

comments by Tim Graham:
1)) The purpose of writing restart files via XIOS. I guess the main purpose is preparation for being able to read them with XIOS but I think it’s important to clarify this. You could also comment on the advantage of being able to write to a single restart file instead of having to rebuild it first.

On a HPC system writing multiple files at the same time may impact I/O performance. This situation may happen during writing restart file. In NEMO all processors write the restart file at the end of the simulation at the same time. With XIOS number of files written at the same time may be significantly reduced; either to the number of XIOS servers or to a single file if necessary. Because writing to multiple files may be faster than to a single file rebuild of the restart will still be necessary when performance is major concern (and there is a need to use single file restart, for instance when change in processors decomposition is needed). Rebuild may be easier and fasted when there are only few sub-domains to rebuild. In this ticket only data written to numror is considered. NEMO restart file is the biggest file and for high resolution configuration (ORCA12) the rebuild is time consuming process, especially when netcdf4 is used.

2) Is it necessary to have the lxios argument to iom_rstput. Why not just check the variable locally withing the iom_rstput subroutine instead of passing it as an argument every time.

iom_rstput is a general subroutine and not all output (files) have to be written by XIOS. I think the implementation with argument is more general.

3) The NEMO coding conventions state that the variables should be l_xios and l_wxios (assuming that they are logicals). If they are integers then they should start with n (i.e. n_xios).

done in r8662

4) There are a lot of places where it looks like lines of code have changed but it is just spaces have been replaced by tabs (or vice versa). This is a pain at the merge party so it would be good to try to revert these changes if possible.

done in r8661

Last edited 2 years ago by nemo (previous) (diff)

comment:6 Changed 2 years ago by andmirek

  • Type changed from Enhancement to Task

comment:7 Changed 2 years ago by andmirek

Results from tests performed to evaluate XIOS write functionality for Met Office GO6 configuration, for 2 model resolutions: ORCA25 and ORCA12. NEMO 3.6 was used for tests. Runs were for 1 day.

NEMO - default NEMO functionality for reading single file restart
XIOS - XIOS read interface for reading single file restart

For XIOS data was written to ONE file and MULTIPLE files

Results: eORCA12:

Execution time [s]

Execution time [s]

CPU (decomposition)

XIOS ONE/MULTIPLE (36 servers)

NEMO

1602 (50x50)

1513/1195

1119

2262 (60x60)

1556/1185

1066

3027 (70x70)

1871/1644

1062

Results: eORCA25:

Execution time [s]

Execution time [s]

CPU (decomposition)

XIOS ONE/MULTIPLE (6 servers)

NEMO

120 (10x12)

645/613

560

440 (20x24)

308/396

246

1440 (40x36)

284/260

173

comment:8 Changed 2 years ago by andmirek

In 8990:

#1953 and #1962: fix ORCA2_OFF_PISCES test

comment:9 Changed 2 years ago by mathiot

  • Milestone changed from 2017 WP to 2018 release-4.0

comment:10 Changed 2 years ago by nicolasmartin

  • Summary changed from Write restart file using XIOS to Met_Office-1(2017WP) Write restart file using XIOS

Modification of summary for standardisation

Last edited 2 years ago by nicolasmartin (previous) (diff)

comment:11 Changed 2 years ago by clevy

  • Summary changed from Met_Office-1(2017WP) Write restart file using XIOS to HPC-07(2017WP) Write restart file using XIOS

comment:12 Changed 2 years ago by mathiot

In 9367:

Add restart read/write via XIOS capability (#1953 and #1962 and twiki: 2017WP/Met_Office-1_Mirek_XIOSread). WARNING: need to upgrade XIOS to r1296 to compile

comment:13 Changed 2 years ago by mathiot

  • Resolution set to fixed
  • Status changed from assigned to closed

comment:14 Changed 2 years ago by andmirek

In 9405:

#1953 and #1962: Initialize lwxios and ln_xios_read

comment:15 Changed 2 years ago by andmirek

In 9535:

#1953 and #1962 disable XIOS related calls when key_iomput is not set

comment:16 Changed 2 years ago by andmirek

In 9536:

#1953 and #1962 stop immediately if XIOS restart read/write functionality is used without key_iomput

comment:17 Changed 21 months ago by andmirek

  • Progress modified (diff)
Note: See TracTickets for help on using tickets.