Namelist Integrity checks

Misplaced or mis-spelt variables in configuration namelist files can be a source of frustration when porting configurations used with older releases of NEMO to more modern versions. The code will report these occurrences and stop during start-up; but only one instance is reported each time. Finding them all this way can be a time-consuming business. Part 1 of this page details a simple Python script which can be used to find mis-matches between a configuration namelist and its reference counterpart before attempting to run the model.

For most users, this should be sufficient but developers of new features have the added uncertainty of not knowing if the reference namelist correctly contains all the namelist options available in the code. Part 2 of this page suggests further tools which can can construct a 'pseudo reference namelist' by parsing the fortran code base. The actual reference namelist and the 'pseudo' version can then be used with the integrity checker from part 1. By using the integrity checker twice, with the roles of the two namelists swapped, it is possible to confirm that:

• The actual reference namelist is compatible with the code base
• The actual reference namelist contains examples of all the namelist variables defined in the code base.

All the scripts mentioned are attached to this page.

Part 1 Integrity checker

If we assume the following conventions for NEMO namelists:

• namelists are started with &name_of_namelist appearing as the first non-whitespace on a line (and not followed by anything other than an optional comment)
• Only one variable is set on each line
• namelists are closed with / appearing as the first non-whitespace on a line

then the attached nemo_nml_check.py​ script will:

• read supplied namelist files
• construct lists of namelist blocks and their contents (the latter being a list of lists)
• compare the contents lists for matching named blocks to confirm membership
• report any mis-matches together with the line number in the configuration namelist

By default, the script will compare namelist_cfg and namelist_ref from the current directory. Alternative pairs can be named using the -c and -r command-line options:

python nemo_nml_check.py --help
usage: nemo_nml_check.py [-h] [--cfg [CFGFILE]]  [--ref [REFFILE]]

Check for consistency between cfg namelists and ref versions e.g. 
python nemo_nml_check.py --cfg <cfg namelist> --ref <ref namelist>

optional arguments:
  -h, --help            show this help message and exit
  --cfg [CFGFILE], -c [CFGFILE]  configuration namelist
  --ref [REFFILE], -r [REFFILE]  reference namelist

A typical example of an error found between the default pairing is:

python nemo_nml_check.py
No match for:  rn_rdt  in  namdom  (at line no.  39 )

and a quick check in the namelist_ref file will confirm that this is because rn_rdt has been renamed as rn_Dt.

Part 2 Constructing a pseudo-reference namelist

Parsing the source code to extract information to build a pseudo reference namelist is simpler if we assume adherence to some coding conventions:

• The namelist opening declaration: namelist / name_of_namelist / occurs on one line
• The name_of_namelist string contains only alphanumeric characters and underscores
• subsequent continuation lines have a leading ampersand and a trailing ampersand is placed on the preceding line (comments are allowed after the trailing ampersand)

With these (seemingly safe) assumptions the attached Perl script: nemo_list_nmls.pl​ can be used to extract namelist templates from the source. For example:

perl nemo_list_nmls.pl OCE/nemogcm.F90
&namctl
  ln_ctl =
  sn_cfctl =
  nn_print =
  nn_ictls =
  nn_ictle =
  nn_isplt =
  nn_jsplt =
  nn_jctls =
  nn_jctle =
  ln_timing =
  ln_diacfl =
/
&namcfg
  ln_read_cfg =
  cn_domcfg =
  ln_closea =
  ln_write_cfg =
  cn_domcfg_out =
  ln_use_jattr =
/

Combined with a few lines of bash, this can be used to construct a pseudo-namelist_ref:

for f in `find ./OCE -name '*.[Fh]90'`
do
  echo "! "$f >> pseudo_nml_ref
  perl nemo_list_nmls.pl $f >> pseudo_nml_ref
done

(this does assume scanning OCE is sufficient; how about OFF, SAS etc.?)

Part 3 Checking reference namelist and source compatibility

The pseudo-namelist_ref produced this way should contain all the variables that can be set via namelists (at least for the 'blue ocean'). Treating the actual namelist_ref as a configuration namelist and checking its integrity against the pseudo version should confirm the validity of everything in the reference namelist.

Update:There is at least one namelist declared in the code with multiple declarations, namely:

      NAMELIST/nambdy_dta/ cn_dir, bn_tem, bn_sal, bn_u3d, bn_v3d, bn_ssh, bn_u2d, bn_v2d
      NAMELIST/nambdy_dta/ bn_a_i, bn_h_i, bn_h_s, bn_t_i, bn_t_s, bn_tsu, bn_s_i, bn_aip, bn_hip
      NAMELIST/nambdy_dta/ rn_ice_tem, rn_ice_sal, rn_ice_age, rn_ice_apnd, rn_ice_hpnd
      NAMELIST/nambdy_dta/ ln_full_vel, ln_zinterp

The Perl script has been adapted to handle this but each entry will appear in the pseudo file as a separate block. These blocks should be combined by hand into a single block before running the integrity checks.

Surprisingly a few variables are flagged even for a fresh copy of r4.0-HEAD (Rev: 13029):

python nemo_nml_check.py -c namelist_ref -r pseudo_nml_ref 
No match for:  sn_wnum  in  namsbc_wave  (at line no.  497 )
No match for:  sn_tauwoc  in  namsbc_wave  (at line no.  498 )
No match for:  sn_tauwx  in  namsbc_wave  (at line no.  499 )
No match for:  sn_tauwy  in  namsbc_wave  (at line no.  500 )
No match for:  clname(1)  in  nam_tide  (at line no.  581 )
No match for:  tname(1)  in  nam_diaharm  (at line no.  1188 )
No match for:  tname(2)  in  nam_diaharm  (at line no.  1189 )
No match for:  sn_cfctl%l_config  in  namctl  (at line no.  1322 )
No match for:  sn_cfctl%l_runstat  in  namctl  (at line no.  1323 )
No match for:  sn_cfctl%l_trcstat  in  namctl  (at line no.  1324 )
No match for:  sn_cfctl%l_oceout  in  namctl  (at line no.  1325 )
No match for:  sn_cfctl%l_layout  in  namctl  (at line no.  1326 )
No match for:  sn_cfctl%l_mppout  in  namctl  (at line no.  1327 )
No match for:  sn_cfctl%l_mpptop  in  namctl  (at line no.  1328 )
No match for:  sn_cfctl%procmin  in  namctl  (at line no.  1329 )
No match for:  sn_cfctl%procmax  in  namctl  (at line no.  1330 )
No match for:  sn_cfctl%procincr  in  namctl  (at line no.  1331 )
No match for:  sn_cfctl%ptimincr  in  namctl  (at line no.  1332 )

It is probably the case that the structure variables flagged (sn_*) are false positives. This is because only the structure name is declared in the namelist but individual members of the structure may be (and are) set in the namelist_ref. Nonetheless, there are a few normal variables to investigate and this does illustrate the potential usefulness of this tool.

Secondly, reversing the roles of the input namelists allows a check on whether or not all the available namelist variables have a default setting in namelist_ref:

python nemo_nml_check.py -r namelist_ref -c pseudo_nml_ref 
No match for:  ln_rnf_icb  in  namsbc_rnf  (at line no.  169 )
No match for:  sn_i_rnf  in  namsbc_rnf  (at line no.  172 )
No match for:  clname  in  nam_tide  (at line no.  195 )
No match for:  ln_time_average_weight  in  namberg  (at line no.  582 )
No match for:  tname  in  nam_diaharm  (at line no.  814 )
No match for:  sn_cfctl  in  namctl  (at line no.  949 )
No match for:  ln_sto_ldf  in  namsto  (at line no.  1056 )
No match for:  rn_ldf_std  in  namsto  (at line no.  1057 )
No match for:  rn_ldf_tcor  in  namsto  (at line no.  1058 )
No match for:  ln_sto_hpg  in  namsto  (at line no.  1059 )
No match for:  rn_hpg_std  in  namsto  (at line no.  1060 )
No match for:  rn_hpg_tcor  in  namsto  (at line no.  1061 )
No match for:  ln_sto_pstar  in  namsto  (at line no.  1062 )
No match for:  rn_pstar_std  in  namsto  (at line no.  1063 )
No match for:  rn_pstar_tcor  in  namsto  (at line no.  1064 )
No match for:  nn_pstar_flt  in  namsto  (at line no.  1065 )
No match for:  nn_pstar_ord  in  namsto  (at line no.  1066 )
No match for:  ln_sto_trd  in  namsto  (at line no.  1067 )
No match for:  rn_trd_std  in  namsto  (at line no.  1068 )
No match for:  rn_trd_tcor  in  namsto  (at line no.  1069 )
No match for:  ln_sto_trc  in  namsto  (at line no.  1078 )
No match for:  nn_sto_trc  in  namsto  (at line no.  1079 )
No match for:  rn_trc_stdxy  in  namsto  (at line no.  1080 )
No match for:  rn_trc_stdz  in  namsto  (at line no.  1081 )
No match for:  rn_trc_tcor  in  namsto  (at line no.  1082 )
No match for:  nn_trc_ord  in  namsto  (at line no.  1083 )
No match for:  nn_trc_flt  in  namsto  (at line no.  1084 )
No match for:  rn_trc_lim  in  namsto  (at line no.  1085 )

The same caveats apply but there are a few issues warranting further investigation.

Full marks for Clem though since repeating this process for ICE:

for f in `find ./ICE -name '*.[Fh]90'`
do   
   echo "! "$f >> pseudo_nml_ice_ref
   perl nemo_list_nmls.pl $f >> pseudo_nml_ice_ref
done
python nemo_nml_check.py -c ../cfgs/SHARED/namelist_ice_ref -r pseudo_nml_ice_ref
python nemo_nml_check.py -r ../cfgs/SHARED/namelist_ice_ref -c pseudo_nml_ice_ref

finds no mismatches

For TOP we have to be more selective about which directories to search when building the pseudo file:

for f in `find -f  ./TOP ./TOP/TRP  -name '*.[Fh]90' -depth 1`
do
  echo "! "$f >> pseudo_nml_top_ref
  perl nemo_list_nmls.pl $f >> pseudo_nml_top_ref
done

and the following potential issues are flagged:

python nemo_nml_check.py -c ../cfgs/SHARED/namelist_top_ref -r pseudo_nml_top_ref
No match for:  sn_trcdta(1)  in  namtrc_dta  (at line no.  61 )
No match for:  ln_trdmld_trc_restart  in  namtrc_trd  (at line no.  124 )
No match for:  ln_trdmld_trc_instant  in  namtrc_trd  (at line no.  125 )
No match for:  ln_trdtrc( 1)  in  namtrc_trd  (at line no.  126 )
No match for:  ln_trdtrc( 2)  in  namtrc_trd  (at line no.  127 )
No match for:  ln_trdtrc(23)  in  namtrc_trd  (at line no.  128 )


python nemo_nml_check.py -r ../cfgs/SHARED/namelist_top_ref -c pseudo_nml_top_ref
No match for:  sn_trcobc  in  namtrc_bc  (at line no.  4 )
No match for:  rn_trofac  in  namtrc_bc  (at line no.  5 )
No match for:  sn_trcsbc  in  namtrc_bc  (at line no.  7 )
No match for:  rn_trsfac  in  namtrc_bc  (at line no.  8 )
No match for:  sn_trccbc  in  namtrc_bc  (at line no.  10 )
No match for:  rn_trcfac  in  namtrc_bc  (at line no.  11 )
No match for:  sn_tri_tracer  in  namtrc_ice  (at line no.  25 )
No match for:  sn_tracer  in  namtrc  (at line no.  48 )
No match for:  ln_trdmxl_trc_restart  in  namtrc_trd  (at line no.  59 )
No match for:  ln_trdmxl_trc_instant  in  namtrc_trd  (at line no.  60 )
No match for:  cn_trdrst_trc_in  in  namtrc_trd  (at line no.  61 )
No match for:  cn_trdrst_trc_out  in  namtrc_trd  (at line no.  62 )
No match for:  ln_trdtrc  in  namtrc_trd  (at line no.  63 )
No match for:  sn_trcdta  in  namtrc_dta  (at line no.  67 )
No match for:  rn_trfac  in  namtrc_dta  (at line no.  69 )

and finally for PISCES:

for f in `find ./TOP/PISCES  -name '*.[Fh]90'`
do   
  echo "! "$f >> pseudo_nml_pisces_ref
  perl nemo_list_nmls.pl $f >> pseudo_nml_pisces_ref
done

python nemo_nml_check.py -c ../cfgs/SHARED/namelist_pisces_ref -r pseudo_nml_pisces_ref
No match for:  mprat2  in  namp5zmort  (at line no.  199 )
No match for:  xpref2p  in  namp5zmes  (at line no.  233 )

python nemo_nml_check.py -r ../cfgs/SHARED/namelist_pisces_ref -c pseudo_nml_pisces_ref
No match for:  mpratd  in  namp5zmort  (at line no.  357 )
No match for:  xpref2n  in  namp5zmes  (at line no.  398 )