restart read write with XIOS

[andmirek]

Extend use of XIOS to handle restart with XIOS

Workplan action

Wikipage: wiki:2019WP/HPC-11_Mirek_Andrejczuk_IO_with_XIOS

Development branch: ​svn+ssh://andmirek@forge.ipsl.jussieu.fr/ipsl/forge/projets/nemo/svn/NEMO/branches/2020/dev_12905_xios_restart

Test branch : ​svn+ssh://andmirek@forge.ipsl.jussieu.fr/ipsl/forge/projets/nemo/svn/NEMO/branches/2020/test_12905_xios_restart

[andmirek]

In 12913:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

branch
​svn+ssh://andmirek@forge.ipsl.jussieu.fr/ipsl/forge/projets/nemo/svn/NEMO/branches/2020/dev_12905_xios_restart
created

[andmirek]

In 12914:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12950:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12951:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12957:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

restart in trdmxl_trc_rst.F90 doesn't follow convention in other parts of NEMO. As a result the approach used to write restart with XIOS can't be used. Reading with XIOS still can be implemented.

[andmirek]

In 12961:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12969:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12977:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12997:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12998:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 12999:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13000:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13001:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

the code was changed this test is not valid

[andmirek]

In 13022:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13023:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13033:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13034:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

Current code is : NEMO/branches/2020/test_12905_xios_restart @ r13034 ( last change @ r13034 )

SETTE validation report generated for :

NEMO/branches/2020/test_12905_xios_restart @ r13034 (last changed revision)

on XC40_METO_IFORT arch file

!!---------------1st pass------------------!!

!----restart----!

WGYRE_PISCES_ST run.stat restartability passed : 13034
WGYRE_PISCES_ST tracer.stat restartability passed : 13034
WORCA2_ICE_PISCES_ST run.stat restartability passed : 13034
WORCA2_ICE_PISCES_ST tracer.stat restartability passed : 13034
WORCA2_OFF_PISCES_ST tracer.stat restartability passed : 13034
WAMM12_ST run.stat restartability passed : 13034
WORCA2_SAS_ICE_ST run.stat restartability passed : 13034
WAGRIF_DEMO_ST run.stat restartability passed : 13034
WSPITZ12_ST run.stat restartability passed : 13034
WISOMIP_ST run.stat restartability passed : 13034
WOVERFLOW_ST run.stat restartability passed : 13034
WLOCK_EXCHANGE_ST run.stat restartability passed : 13034
WVORTEX_ST run.stat restartability passed : 13034
WICE_AGRIF_ST run.stat restartability passed : 13034

!----repro----!

WGYRE_PISCES_ST run.stat reproducibility passed : 13034
WGYRE_PISCES_ST tracer.stat reproducibility passed : 13034
WORCA2_ICE_PISCES_ST run.stat reproducibility passed : 13034
WORCA2_ICE_PISCES_ST tracer.stat reproducibility passed : 13034
WORCA2_OFF_PISCES_ST tracer.stat reproducibility passed : 13034
WAMM12_ST run.stat reproducibility passed : 13034
WORCA2_SAS_ICE_ST run.stat reproducibility passed : 13034
WORCA2_ICE_OBS_ST run.stat reproducibility passed : 13034
WAGRIF_DEMO_ST run.stat reproducibility passed : 13034
WSPITZ12_ST run.stat reproducibility passed : 13034
WISOMIP_ST run.stat reproducibility passed : 13034
WVORTEX_ST run.stat reproducibility passed : 13034
WICE_AGRIF_ST run.stat reproducibility passed : 13034

!----agrif check----!

ORCA2 AGRIF vs ORCA2 NOAGRIF run.stat unchanged - passed : 13034 13034

!----result comparison check----!

check result differences between :
VALID directory : /home/d02/mandrej/nemo_validation_RRW/NEMO_VALIDATION at rev 13034
and
REFERENCE directory : /home/d02/mandrej/nemo_validation/NEMO_VALIDATION at rev 12913

WGYRE_PISCES_ST run.stat files are identical
WGYRE_PISCES_ST tracer.stat files are identical
WORCA2_ICE_PISCES_ST run.stat files are identical
WORCA2_ICE_PISCES_ST tracer.stat files are identical
WORCA2_OFF_PISCES_ST tracer.stat files are identical
WAMM12_ST run.stat files are identical
WISOMIP_ST run.stat files are identical
WORCA2_SAS_ICE_ST run.stat files are identical
WAGRIF_DEMO_ST run.stat files are identical
WSPITZ12_ST run.stat files are identical
WISOMIP_ST run.stat files are identical
WVORTEX_ST run.stat files are identical
WICE_AGRIF_ST run.stat files are identical

[andmirek]

In 13727:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13728:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13750:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13751:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13754:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13755:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13756:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13763:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13764:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13765:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[smasson]

Branch Review

I am happy to see that you get read of the long hard coded lists of restart variables and that you now follow what is done for restarts written without xios.
I think the code structure is now close to be ok.
However, to me, the version I reviewed (branches/2020/dev_12905_xios_restart@13750) is still containing many “small” things (some of them dating from before the branch creation) that should be improved/cleaned.
To me, this branch does not correspond yet to the standards required for a finalized branch ready to be merged.
As you we see bellow in the details comments, I propose modifications that would simplify and clarify your code, keep the number of modified routines as small as possible (basically only the iom routines).
I definitely think that the less you modify things, the better it is.

My main issue is the multiplication of variables and routines calls that seams to be useless or could be avoided (as far I as understand it).
The chosen variables names are often not easy to understand.
To me this makes the things more complicated than they should be.

The starting point is these 2 namelist parameters:

   ln_xios_read = .false.  !  use XIOS to read restart file (only for a single file restart)
   nn_wxios = 0      !  use XIOS to write restart file 0 - no, 1 - single file output, 2 - multiple file output

• I agree with the choice of adding 2 parameters.
• I know the history and understand the motivations but It think It would be better if they had similar names. One day we could also use xios to read restarts in multiple files. The name should also include the idea of restart.

=> Why not using something like: nn_rst_rxios and nn_rst_wxios ?

I am concern by the variables you next add in iom_def, in_out_manager and a few other places. I don’t understand their need. This makes the code harder to follow.

• nxioso: i don’t understand the need of this variable. For me it is just a duplication of nn_wxios
• lwxios is just the same as nn_wxios > 0
• lrxios is just the same as ln_xios_read.AND.ln_rstart. Why not forcing ln_xios_read = .false. when ln_rstart = .false. : ln_xios_read = ln_xios_read .and. ln_rstart = .true.
• lxios_sini: i don’t understand this variable. We read only “one_file” restarts so why having this additional switch?
• lroxios, lrixios, lrtxios, lrsxios: i don’t understand the need of these variables. All component are writing their restart the same way according to ln_xios_read. Why do you need specific variables for each component?
• you added the ldxios optional argument in the calls to iom_get and to iom_rstput (changing a few hundred lines of code!). I don’t understand why you need to do this and change all theses lines.

=> for iom_get: to define llxios, you can: (1) you test ln_xios_read, (2) you compare kiomid to numror, numrir, etc…
=> for iom_rstput: to define llx, you can: (1) you test nn_wxios, (2) you compare kiomid to numrow, numriw, etc…

Every time you read/write restarts, you swap xios context to select the proper restart context. I am wondering if this could not be done in iom_get and iom_rstput interfaces. As I just said above, using the namelist parameters and comparing kiomid to numror, numrir, etc… you now what you want to do so you could determine at this point if you have to switch context or not.
This would allow to remove the calls to iom_swap that you had in many places.
=> Suppressing the ldxios parameter and the calls to iom_swap would, I guess, suppress (almost?) all the modifications you done outside of iom routines. This would greatly reduce the impact of this development on the code, simplifying its implementation, its understanding and its maintenance.

Regarding the Context names, I agree, these are needed.
Originally we had only cxios_context. Now that we have 9 contexts, we need to differentiate their names more easily.
I like your idea to start with cr or cw. Next I propose to add who is doing what: all, oce, ice, top, sed. Next the type of file: out or rst and ending by cxt instead of xios_context which is too long
This would give the following variables names:

cw_allout_cxt
cw_ocerst_cxt, cr_ocerst_cxt
cw_icerst_cxt, cr_icerst_cxt
cw_toprst_cxt, cr_toprst_cxt
cw_sedrst_cxt, cr_sedrst_cxt

Other points (not to be neglected)

• very strange indentation in many iom routines. non-respect of “NEMO coding style"
• using nf90_* functions in iom is not clean. This should be done only in iom_nf90. cn_var and dimsz of the file_descriptor should be used.
• following this idea, idfp should be suppressed and you should directly use numror
• same idea: why do you use fname in iom_set_vars_active? You should directly use iom_file(numror)%name.
• in iom_set_vars_active, meta is a local variable that is not used anywhere else. Why is it declared in iom_def.F90 and not simply locally?
• There id many calls to the TRIM function that are useless. There is no need to use TRIM when comparing 2 character varables. I also thing there is no need to use TRIM when passing a character as an argument to XIOS routines.
• It several places, there are variables declared but not used. I noticed for example

  CHARACTER(lc)  ::   clpath
   INTEGER :: i
   CHARACTER(len=1024) :: fname
   CHARACTER(len=lc)   :: axis_ref
  

[andmirek]

In 13871:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13932:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13933:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

As of 01/12/2020 I can't run sette test. The error

Config file (bld): /data/d02/mandrej/NEMO_2020_DEVELOPMENT/dev_12905_xios_restart/mk/cpp.fcm
ERROR: /data/d02/mandrej/NEMO_2020_DEVELOPMENT/dev_12905_xios_restart/mk/bldxag.cfg: LINE 17:
       /data/d02/mandrej/NEMO_2020_DEVELOPMENT/dev_12905_xios_restart/ext/PPR/src: source does not exist or is not readable.
ERROR: /data/d02/mandrej/NEMO_2020_DEVELOPMENT/dev_12905_xios_restart/mk/bldxag.cfg: LINE 39:
       bld::pp::ppr_1d: invalid sub-package in declaration.
ERROR: /data/d02/mandrej/NEMO_2020_DEVELOPMENT/dev_12905_xios_restart/mk/bldxag.cfg: LINE 42:
       bld::tool::fppflags::ppr_1d: invalid sub-package in declaration.

[andmirek]

In 13934:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13940:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13941:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13964:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found

[andmirek]

In 13970:

Error: Failed to load processor CommitTicketReference

No macro or processor named 'CommitTicketReference' found