What is still missing in SETTE ?

[mathiot]

Context

after last commit on SETTE, SETTE is better and more reliable, but it is still not perfect, here a list of idea to be discuss to improve SETTE further. Feel free to comment or update the list. The order do not reflect the amount of work or any kind of priority.

Proposal

• Find a way to have only one SETTE directory to manage various branches (at this stage, you need one SETTE directory per branch)

• Clean sette_reference_config.sh (former sette.sh) and sette_test_cases.sh (~1500 lines).
• Ease the process to include new configs for personal testing (ideally one NEWCFG repo in cfgs/., one input file NEWCFG_input.cfg and ./sette.st -t NEWCFG and that's it)
• Add an input_CFG.cfg namelist variable used by SETTE related to the configuration setup as 'set_namelist namelist_pisces_cfg ln_solub .false.' for exemple, processor decomposition, time step you want to run ... ; if namelist_cfg of sette configuration different to the reference configuration, we could add the namelist_cfg file into the tar file. These points should allow to build a function in sette_*_config.sh and ease previous points.

• Decrease the number of configurations to test (not the amount of options tested) by merging some configuration (ORCA2_ICE_PISCES + ORCA2_ICE_OBS + SAS for example).
• Decrease the number of run (use the LONG run as reference also for the reproducibility). Instead of 4 run: LONG/SHORT and REPRO_XX/REPRO_YY, could 3 run be enough: REF (ie the long run), RST (the short run)and REPRO (one of the REPRO_XX runs)
• Compilation of SETTE in parallel.

• Is sette_test_cases.sh is the script to run all the test cases of a specific config (based on the list of namelist_*_cfg) in addition of the standard SETTE tests? Should it be in an other script ?

• Do we need a 2nd pass for the report ?
• Check timing of SETTE runs in the report? Memory usage?
• Enable the configurability of the revision-number suffix used to indicate local modification of the working copy
• Save the svn status or even svn diff related to the src/ and cfgs/CFG directory in the directory RRRRx/WCFG/. .

• Is someone has a list of all the options tested in SETTE compare to the amount of option available in the namelist ?

• Remove variable ADD_NOSIGNEDZERO and the associated mechanism to add preprocessor key key_nosignedzero from SETTE (see ticket:2302#comment:1)

DONE:

• Need to check consistency between CFG_ST and CFG directory (in case CFG change during a revision, change will not propagate to CFG_ST using SETTE, you need to clean_config CFG_ST before and recompile from scratch). => ticket #2304