setting up and running NVTK on new target

[nemo_user]

Hello. I've just finished installing NVTK at NOCS on a new target machine.
I hope you don't mind, but I have a number of problems and comments about the procedure to report,
which may be of interest to others installing on a new target machine, even if they cannot be
incorporated into the source copy.
This uses NVTK code from revision 1164.

NVTK
The first mistake I made was not reading the documentation carefully enough.

Because NVTK came with my checked out copy of the subversion trunk, I assumed it could be installed from its default location. This lead to all sorts of problems with paths in scripts.

NVTK/fait_AA_make

Our korn shell does not understand the \c in echo statements, so I have to alias echo to print. Parallel makes do not work for us so that I have to remove the -j option in the gmake step.

NVTK/INSTALL/ins_nvtk.ksh

The "tree" command isn't available on our machine and so I replaced it with "find -type d"

NVTK/INSTALL/JOBS/lance_batch.ksh

Some of the user specified variables can involve full paths (e.g. LAUN, LLJOBS and LSUB), which are then substituted into other files with a sed substitute command using "/" as a delimiter. I replaced "/" where necessary, but wouldn't it be better to consistently use something other than "/"?

NVTK/INSTALL/JOBS/job_<configuration>.ksh

Our machine does not automatically create and use the TMPDIR facility for running jobs, so I added code to create a temporary directory, assign it to a variable called TMPDIR and remove it at the end.

NVTK/use_cfg

There would seem to be an error in this since ins_nvtk.ksh calls use_cfg with the "-n" option, but the reference configuration (r_cfg) is selected on the basis of the "-r' option.
The result is that all configurations use ORCA2-LIM files which causes problems later in the process.

NVTK/Makefile

Again parallel makes are used.

Running NVTK

A run with no new code runs fine. When I introduce new code into the 2TEST directory I have a small problem. All my new routines are just modified versions of their originals. However this includes extra module dependencies (USE statements). fait_AA_make is run from ins_nvtk.ksh before the symbolic links to the 2TEST code are created. The fait_AA_make in the WORK Makefile only takes place if there are new files, so in my case the WORK AA_make is not recreated, and the make step fails because the order in which module files are created is wrong. The way I've got round this is to create a dummy empty module in the 2TEST directory: it's existence forces the fait_AA_make in the WORK makefile to run. But unfortunately this means that each configuration build fails in turn at the src_file_list target (with no explanatory messages, since they are wallowed up by log files) and the make of the master makefile has to be restarted. This isn't ideal since, unless I edit the LISTE_CONF variable in the master makefile each time, I'm forced to repeat the tests a number of times. Is there another way to do this, or am I doing something wrong?

In the "NEMO VALIDATION" report, there doesn't seem to be anything to identify what new routines have been tested. Could this be added to the output? This would clearly distinguish between a reference run and a code-testing run.

Finally, for new code to be accepted for inclusion in NEMO, what documentation is required? Are a set of patch files to a specified revision with the accompanying NEMO VALIDATION reports sufficient? Which configurations and tests are mandatory? Who do I send them to?

Steven Alderson
sga@…

[nemo_user]

A couple of further points:

• When creating a reference set (to use with REF_TAGV), the directory structure changes (e.g. NEMO_VALID/WGYRE becomes GYRE/$REF_TAGV)
  It would be easier if the reference tree just mirrored the original, but with the REF_TAGV at the top (e.g. $(REF_TAGV)/WGYRE), so that NEMO_VALID just gets moved into $REFDAT with new name $REF_TAGV

• The input tar file for ORCA2 is hard wired into the scripts
  Since the only use of INPUTD seems to be to find this one file, wouldn't it be better to have INPUTD point to the file instead? This allows developers to change the input files and give the tar file a sensible name rather than having to call it ORCA2_LIM_nemo_v3.tar

Steven

[ctlod]

Replying to nemo_user:

Hello. I've just finished installing NVTK at NOCS on a new target machine.
I hope you don't mind, but I have a number of problems and comments about the procedure to report,
which may be of interest to others installing on a new target machine, even if they cannot be
incorporated into the source copy.
This uses NVTK code from revision 1164.

Thank you for these comments.

NVTK

The first mistake I made was not reading the documentation carefully enough.
Because NVTK came with my checked out copy of the subversion trunk, I assumed it could be
installed from its default location. This lead to all sorts of problems with paths in scripts.

I hope it is clearer now on the ​http://forge.ipsl.jussieu.fr/nemo/wiki/NVTKpage NVTK page.

NVTK/fait_AA_make

Our korn shell does not understand the \c in echo statements, so I have to alias echo to print.
Parallel makes do not work for us so that I have to remove the -j option in the gmake step.

Shell scripts portability problem ..., difficult to solve in a proper way.

NVTK/INSTALL/ins_nvtk.ksh

The "tree" command isn't available on our machine and so I replaced it with "find -type d"

idem

NVTK/INSTALL/JOBS/lance_batch.ksh

Some of the user specified variables can involve full paths (e.g. LAUN, LLJOBS and LSUB),
which are then substituted into other files with a sed substitute command using "/" as a delimiter.
I replaced "/" where necessary, but wouldn't it be better to consistently use something other
than "/"?

Yes, we could for instance replace "/" with "%", it has been already and partially done in few jobs

NVTK/INSTALL/JOBS/job_<configuration>.ksh

Our machine does not automatically create and use the TMPDIR facility for running jobs, so
I added code to create a temporary directory, assign it to a variable called TMPDIR and remove it
at the end.

We could maybe add one line which will depend on the platform as below:
#-T- platform cd ${TMPDIR}
The drawback is that we have to add it in each jobs.

NVTK/use_cfg

There would seem to be an error in this since ins_nvtk.ksh calls use_cfg with the "-n" option,
but the reference configuration (r_cfg) is selected on the basis of the "-r' option.
The result is that all configurations use ORCA2-LIM files which causes problems later in the process.

Indeed, ORCA2_LIM files are used only if the configuration name directory to build doesn't exist under ../modipsl/config directory. For instance, if TOTO is a new configuration (i.e. it doesn't exist under ../modipsl/config directory) and you use use_cfg -n TOTO without option "-r" then ORCA2_LIM files will be duplicated under the TOTO directory otherwise no files will be copied.
When using ins_nvtk.ksh script, all configurations ORCA2_LIM, GYRE, GYRE_LOBSTER directories exist so no need to use the -n option

NVTK/Makefile

Again parallel makes are used.

Running NVTK

A run with no new code runs fine.
When I introduce new code into the 2TEST directory I have a small problem.
All my new routines are just modified versions of their originals. However this includes extra
module dependencies (USE statements). fait_AA_make is run from ins_nvtk.ksh before the
symbolic links to the 2TEST code are created. The fait_AA_make in the WORK Makefile only takes
place if there are new files, so in my case the WORK AA_make is not recreated, and the make step
fails because the order in which module files are created is wrong.
The way I've got round this is to create a dummy empty module in the 2TEST directory: it's existence
forces the fait_AA_make in the WORK makefile to run. But unfortunately this means that each
configuration build fails in turn at the src_file_list target (with no explanatory messages, since
they are swallowed up by log files) and the make of the master makefile has to be restarted.
This isn't ideal since, unless I edit the LISTE_CONF variable in the master makefile each time, I'm
forced to repeat the tests a number of times. Is there another way to do this, or am I doing
something wrong?

yes, this is a black point of NVTK. A simple way to correct this will be to add a variable (MAKE_DEP) of LOGICAL type in the ./config/NVTK/Makefile script, and the following lines:
( cd ../$@/WORK ; if [ -n "$$MYREP_LIST" ] ; then ln -sf ../MY_SRC/*.[Ffh]90 . ; cd .. ; ../NVTK/fait_AA_make ; fi ; )
So if "USE statements" or new cpp keys have been added they will be taken into account for the compilation process in just setting MAKE_DEP=yes

In the "NEMO VALIDATION" report, there doesn't seem to be anything to identify what new routines
have been tested. Could this be added to the output? This would clearly distinguish between a
reference run and a code-testing run.

Yes, I agree with you. We'll add this list in the assessment report; we could also add the revision number used

Finally, for new code to be accepted for inclusion in NEMO, what documentation is required?

At least comments (description of steps performed and a reference to an article if one) in the header of a new routine are clearly required. If it is a new numerical algortihm AND/OR a new physical parameterization a paper documentation following the "format" of the current NEMO documentation should be provided. The best will be to send it at the same time than the code .....

Are a set of patch files to a specified revision with the accompanying NEMO VALIDATION reports sufficient?

Yes

Which configurations and tests are mandatory?

Currently, ORCA2_LIM, GYRE and GYRE_LOBSTER are the current configurations which should be tested, in any case, when using NVTK, the standard configurations to be tested are already set in the NVTK/Makefile script through the LISTE_CONF variable. This list should increase.
Concerning the tests, 2 things:
# firstly: your routines still ensure the reproductibility and restartability of standard configurations
# secondly: ensure that when using your modifications, results are reproductible and restartable

Who do I send them to?

To the NEMO team nemo_st@… or the person of the team you interacted with.

Steven Alderson
sga@…

[ctlod]

Among modifications partly done (see Changeset [1301]) and suggested below, main improvements are:

• control of the dependencies file build, usefull when adding a new cpp key or the "USE module" statement (in using gmake -s dep under NVTK directory before launching the compilation process with command gmake)
  

• allow to perform several experiences for a given configuration at the same time, this possibility is mainly dedicated to developpers
  

[nicolasmartin]

Milestone NEMO Validation: SETTE (previously NVTK) deleted