[11140] | 1 | ====================== |
---|
| 2 | usage of SETTE package |
---|
| 3 | ====================== |
---|
[3520] | 4 | |
---|
[11140] | 5 | INSTALLATION |
---|
| 6 | ============ |
---|
| 7 | * checkout the code as follow: |
---|
| 8 | .. code-block:: console |
---|
[10819] | 9 | |
---|
[11140] | 10 | $ svn checkout http://forge.ipsl.jussieu.fr/nemo/svn/utils/CI/sette sette |
---|
| 11 | |
---|
| 12 | * The sette directory have to be located in the main directory (at the same level as ``src/`` or ``cfg/``). |
---|
| 13 | |
---|
| 14 | MASTER SCRIPT: sette.sh |
---|
| 15 | ======================= |
---|
| 16 | * ``sette.sh`` is a simple wrapper that runs tests on the reference configurations, test cases and generates the report at the end. |
---|
[11153] | 17 | * ``sette.sh`` is able n option, to take a list of configurations in argument (sette.sh -t "CFG1 CFG2 CFG3"). |
---|
[10819] | 18 | - The list of available configurations is available in running ./sette_list_avail_cfg.sh. |
---|
| 19 | - The default for sette is to run all of the reference and test configurations. |
---|
[11140] | 20 | * Users need to set up the ``param.cfg`` file correctly and download input files before running the script. |
---|
[10687] | 21 | |
---|
[10732] | 22 | SECONDARY SCRIPT |
---|
[11140] | 23 | ================ |
---|
[11153] | 24 | * ``sette_reference-configuration.sh`` runs sette tests on the reference configurations. |
---|
| 25 | * ``sette_test-cases.sh`` runs sette tests on test cases. |
---|
| 26 | * ``sette_list_avail_rev.sh`` generates a listing on all the revisions available in the validation directory for each configuration. |
---|
[11140] | 27 | * ``sette_list_avail_cfg.sh`` generates a listing of all the available configurations. |
---|
[11153] | 28 | * ``sette_rpt.sh`` generates the report. |
---|
[10732] | 29 | |
---|
[10819] | 30 | USER SETUP (recommended) |
---|
[11140] | 31 | ======================== |
---|
| 32 | * in ``param.cfg`` : |
---|
| 33 | - variables (NEMO_VALIDATION_REF, NEMO_REV_REF) have to be specified in param.cfg |
---|
| 34 | - variables SETTE_(COMPILER, USING_XIOS, USING_MPMD, JOB_PREFIX_MPMD, JOB_PREFIX_NOMPMD, BATCH_CMD, BATCH_STAT, BATCH_NAME, FORCING_DIR, SVN_CMD, NEMO_VALIDATION_DIR, ADD_NOSIGNEDZERO) |
---|
| 35 | can be exported from your shell startup files. It is advise to setup the default value in your startup file if it doesn't fit your need. |
---|
| 36 | If you prefer not doing so, you need to update the default initialisation |
---|
[10819] | 37 | - variable NEMO_VALIDATION_DIR is sette by default in MAIN_DIR/NEMO_VALIDATION (this is to avoid overwritting results when running sette on different branches). |
---|
[11140] | 38 | - description of variables listed in param.cfg (SETTE_): |
---|
| 39 | |
---|
| 40 | # reference version for bit reproducibility results |
---|
| 41 | | NEMO_VALIDATION_REF : reference directory |
---|
| 42 | | NEMO_REV_REF : reference revision |
---|
| 43 | |
---|
| 44 | # compiler information |
---|
| 45 | | COMPILER : compiler among those available in NEMOGCM/ARCH |
---|
| 46 | |
---|
| 47 | # XIOS management |
---|
[14240] | 48 | | USING_XIOS : flag to control the activation of key_xios. |
---|
| 49 | "yes" to compile using key_xios and link to the external XIOS library. |
---|
| 50 | "no" to compile without key_xios and link to the old IOIPSL library. |
---|
[11140] | 51 | | USING_MPMD : flag to control the use of stand-alone IO servers |
---|
| 52 | (requires USING_XIOS="yes"). |
---|
| 53 | "yes" to run in MPMD (detached) mode with stand-alone IO servers. |
---|
| 54 | "no" to run in SPMD (attached) mode without separate IO servers. |
---|
[13943] | 55 | # MPI communication management |
---|
| 56 | | USING_MPI3 : flag to control the activation of key_mpi3 |
---|
| 57 | "yes" to use the MPI3 neighbourhood collectives for halo exchange |
---|
| 58 | "no" to use standard point-to-point communications for halo exchange |
---|
| 59 | # loop fusion management |
---|
| 60 | | USING_LOOP_FUSION : flag to control the activation of key_loop_fusion |
---|
| 61 | "yes" to use the loop fusion adv routines when halo = 2 |
---|
| 62 | "no" to use standard adv routine |
---|
| 63 | |
---|
[11140] | 64 | # generique batch scrip prefix name if MPMD set to true/false |
---|
| 65 | | JOB_PREFIX_MPMD |
---|
| 66 | | JOB_PREFIX_NOMPMD |
---|
| 67 | |
---|
| 68 | # batch command needed |
---|
| 69 | | BATCH_CMD : command for job submission in batch mode |
---|
| 70 | | BATCH_STAT : command to check job status |
---|
| 71 | | BATCH_NAME : generic sette job name (as it appears with $BATCH_STAT command) |
---|
| 72 | |
---|
| 73 | # file storing |
---|
| 74 | | FORCING_DIR : directory where is stored input.tar file (same name in input_CONFIG_NAME.cfg) |
---|
| 75 | reference configuration input tar file could be found here : https://zenodo.org/record/1472245 |
---|
| 76 | | NEMO_VALIDATION_DIR : directory where is stored restarts, run.stat, tracer.stat and ocean.output files for each configuration |
---|
| 77 | ( NEMO_VALIDATION_DIR/WCONFIG_NAME/WCOMPILER_NAME/TEST_NAME/REVISION_NUMBER(or DATE) ) |
---|
| 78 | | INPUT_DIR : directory where is stored input files (DO NOT CHANGE IT) |
---|
| 79 | |
---|
| 80 | # misc. |
---|
| 81 | | SVN_CMD : svn command use to do svn info (default svn). Could be useful if you are using git svn |
---|
| 82 | : Reference directory for result comparison will be NEMO_VALIDATION_REF/COMPILER/NEMO_REV_REF |
---|
| 83 | | ADD_NODIGNEDZERO : set "yes" if you need key_nosignedzero to run nemo |
---|
[10687] | 84 | |
---|
[11140] | 85 | USAGE of main scripts |
---|
| 86 | ===================== |
---|
| 87 | * ``sette.sh`` : it is a simple wrapper to run tests on the reference configurations, test-cases and generate the report at the end. |
---|
[10819] | 88 | - if no argument is given, sette run all the reference configurations and test cases |
---|
| 89 | - if a list of argument is provided (sette.sh -t "CFG1 CFG2 CFG3"), sette will only run these configurations. |
---|
| 90 | The list of available configurations is available in running ./sette_list_avail_cfg.sh. |
---|
| 91 | - user need to set up the param.cfg file correctly and download input file before running the script. |
---|
[12569] | 92 | - user can enforce synchronisation (-s option) of the existing CFG_ST with the REF configuration (EXPREF and MY_SRC). |
---|
| 93 | - user can enforce cleaning of the CFG_ST configuration (use of makenemo -n CFG_ST clean) (-c option) |
---|
[11140] | 94 | * ``sette_rpt.sh`` : it generates the sette report. |
---|
[10819] | 95 | - if no argument is given, the report will be generated on the last changed revision. |
---|
| 96 | - if an argument is given (revision number) the report will be generated for this revision only |
---|
| 97 | - if 'old' is given as argument the former behavior is applied (the latest revision is check whatever the current revision |
---|
| 98 | - it is possible to retreive all the available revision test using sette_list_avail_rev.sh |
---|
[11153] | 99 | - XXXXX+ means sette results for revision XXXXX contain local modification in src/cfgs/test |
---|
[10819] | 100 | - it is NOT possible to run sette_rpt.sh for a single configuration. |
---|
[11140] | 101 | * ``sette_list_avail_rev.sh`` : generate a listing on all the revisions available in the validation directory for each configuration. |
---|
[10819] | 102 | - no argument needed |
---|
[11153] | 103 | - XXXXX+ means sette results for revision XXXXX contain local modification in src/cfgs/test |
---|
[11140] | 104 | * ``sette_list_avail_cfg.sh`` : generate a listing of all the available configurations. |
---|
[10819] | 105 | - no argument needed |
---|
[3520] | 106 | |
---|
[11140] | 107 | NOTES |
---|
| 108 | ===== |
---|
| 109 | * compilation issues: |
---|
| 110 | - in case of error you can remove your NEW_CONF directory and all files doing : |
---|
| 111 | |
---|
| 112 | :: |
---|
| 113 | |
---|
| 114 | $ ./makenemo -n MY_CONFIG clean_config |
---|
| 115 | |
---|
[10819] | 116 | - if you want recompile nemo but before you want to clean compiled code do : |
---|
[11140] | 117 | |
---|
| 118 | :: |
---|
| 119 | |
---|
[10819] | 120 | ./makenemo clean |
---|
[11140] | 121 | |
---|
[10819] | 122 | - if you have already compiled you can re-run all sette.sh and compilation part will be by-passed. |
---|
[3520] | 123 | |
---|
[11140] | 124 | * verbose sette output: |
---|
| 125 | - if you want a completly verbose makenemo you can uncomment `set -x` in makenemo script |
---|
| 126 | and then launch `./sette.sh 2>&1 | tee out.sette` . This creates out.sette file in ${SETTE_DIR} |
---|
[3520] | 127 | |
---|
[11140] | 128 | TO ADD NEW CONFIGURATION |
---|
| 129 | ========================= |
---|
| 130 | 1. creates a new ``input_NEW_CONFIG.cfg`` if you need tar file (if you use same tar file of GYRE, ORCA2_LIM or ORCA2_LIM_PISCES you can use it) |
---|
| 131 | 2. add a bloc in one of the ``sette_reference-configuration.sh`` or ``sette_test-cases.sh`` script |
---|
| 132 | 3. add your configuration to the list in ``param.cfg`` |
---|
| 133 | |
---|
| 134 | TO ADD NEW MACHINE |
---|
| 135 | =================== |
---|
| 136 | 1. add ``arch-compiler.fcm`` in ``NEMOGCM/ARCH`` directory |
---|
[3520] | 137 | 2. makenemo -m new_compiler (see makenemo -h for help) |
---|
[10750] | 138 | 3. add new batch-file |
---|
[3520] | 139 | |
---|
[11140] | 140 | TO ADD NEW BATCH-FILE |
---|
| 141 | ====================== |
---|
| 142 | 1. see in ``SETTE/BATCH_TEMPLATE`` directory existing examples |
---|
| 143 | 2. create you own batch file like: ``batch-${COMPILER}`` file |
---|
| 144 | (note: sette_test-cases.sh and ``sette_cfg-ref.sh`` will copy it in job_batch_template if you run tests in INTERACT_FLAG="no") |
---|
[10819] | 145 | |
---|
[11140] | 146 | EXTRA SETTING POSSIBLE |
---|
| 147 | ====================== |
---|
| 148 | Among the setting that can be modified by the user in sette_cfg-ref.sh and sette_test-cases.sh script, |
---|
| 149 | there are: sequential/parrallel (default), interacive or not (default) and mpi (default) or not. |
---|
| 150 | - | BATCH_COMMAND_PAR is the command for job submission in batch mode parallel (specified in param.cfg). |
---|
| 151 | - | BATCH_COMMAND_SEQ is the command for job submission in batch mode sequential (NB_PROC = 1). |
---|
| 152 | the default value is the BATCH_COMMAND_PAR value. |
---|
| 153 | - | INTERACT_FLAG : "yes" if you want to run in interactive mode. |
---|
| 154 | "no" if you want to run in batch mode (default). |
---|
| 155 | - | MPIRUN_FLAG : "yes" if you want to run in parallel (MPI) (default). |
---|
| 156 | "no" if you want to run in sequential mode (NB_PROC = 1). |
---|