[3520] | 1 | usage of SETTE package: |
---|
| 2 | |
---|
| 3 | principal script is sette.sh (usage: ./sette.sh) |
---|
| 4 | |
---|
| 5 | VERY IMPORTANT : for a verbose debug do: |
---|
| 6 | ./sette.sh 2>&1 | tee out.sette |
---|
| 7 | this create out.sette file in ${SETTE_DIR} |
---|
| 8 | |
---|
| 9 | USER MODIFICATIONS |
---|
| 10 | in sette.sh script : |
---|
| 11 | COMPILER : compiler among those available in NEMOGCM/ARCH, needed for makenemo (see below) |
---|
| 12 | can also be pass as an argument, e.g. ./sette.sh ifort_osx |
---|
| 13 | BATCH_COMMAND_PAR : command for job submission in batch mode parallel (MMIÂ) |
---|
| 14 | BATCH_COMMAND_SEQ : command for job submission in batch mode sequential (NB_PROC = 1) |
---|
| 15 | INTERACT_FLAG : "yes" if you want to run in interactive mode |
---|
| 16 | "no" if you want to run in batch mode |
---|
| 17 | MPIRUN_FLAG : "yes" if you want to run in parallel (MPI) |
---|
| 18 | "no" if you want to run in sequential mode (NB_PROC = 1) |
---|
| 19 | |
---|
| 20 | makenemo -m : machine (compilator); see ./makenemo -m help (in NEMOGCM/CONFIG directory) |
---|
| 21 | -n : config name (in examples config_name_nproci_nprocj), see ./makenemo -n help |
---|
| 22 | -r : reference configuration (if you don't give it you have to choise directories to install) |
---|
| 23 | add_key : to add list of keys (for example to test reproductibility: add_key mpp_rep) |
---|
| 24 | del_key : to del list of keys |
---|
| 25 | and for more details and options of makenemo see ./makenemo -h |
---|
| 26 | |
---|
| 27 | set_namelist : to set namelist values |
---|
| 28 | (example : to set 75 time steps for one test : namelist nn_itend 75) |
---|
| 29 | ( : to change cn_icerst_in name : namelist_ice cn_icerst_in \"test_00101231_restart_icemod\") |
---|
| 30 | ( : to activate control print : namelist ln_ctl false ) |
---|
| 31 | prepare_exe_dir.sh creates execution directory for each test |
---|
| 32 | : NOTE for EACH JOB EXECUTION it is needed to set TEST_NAME variable |
---|
| 33 | TEST_NAME variable gives name to execution directory (created in prepare_exe_dir.sh) |
---|
| 34 | IT is mandatory to have different TEST_NAME for each job run if not existing files are re-written (for example namelist) |
---|
| 35 | |
---|
| 36 | prepare_job.sh generates the script to be executed |
---|
| 37 | input: INPUT_FILE_CONFIG_NAME NUMBER_PROC TEST_NAME MPIRUN_FLAG JOB_FILE |
---|
| 38 | |
---|
| 39 | fcm_job.sh needs input: TOTAL_NUMBER_OF_PROCS_used JOB_FILE_NAME INTERACT_FLAG MPIRUN_FLAG |
---|
| 40 | (for example if you run with 4 procs JOB_FILE batch parallel |
---|
| 41 | and only run short test ===> ./fcm_job 4 SHORT no yes ) |
---|
| 42 | |
---|
| 43 | in param.cfg : |
---|
| 44 | FORCING_DIR : directory in which will be found input.tar file (same name in input_CONFIG_NAME.cfg) |
---|
| 45 | INPUT_DIR : directory in which store input files (tar file) |
---|
| 46 | TMPDIR : temporary directory NEEDED ONLY FOR IBM machines (put EXP00 directory) |
---|
| 47 | NEMO_VALIDATION_DIR : directory in which create NEMO_VALIDATION tree, and store restart, solver.stat, tracer.stat and ocean.output files in |
---|
| 48 | tree NEMO_VALIDATION_DIR/WCONFIG_NAME/WCOMPILER_NAME/TEST_NAME/REVISION_NUMBER(or DATE) |
---|
| 49 | in fcm_job.sh : |
---|
| 50 | IMPORTANT : control that name of tar file in FORCING directory |
---|
| 51 | is the same of that in input_REF_CONFIG.cfg, if not change one. |
---|
| 52 | |
---|
| 53 | change in "RUN OPA" if [ ${INTERACT_FLAG} == "yes" ] mpirun command line (some examples are already written) |
---|
| 54 | NOTE: if INTERACT_FLAG="no" it is needed a batch_file (see below to create a new one) |
---|
| 55 | |
---|
| 56 | NOTE: |
---|
| 57 | in case of error you can remove you NEW_CONF directory and all files doing : |
---|
| 58 | ./makenemo -n MY_CONFIG clean_config |
---|
| 59 | |
---|
| 60 | if you want recompile nemo but before you want to clean compiled code do : |
---|
| 61 | ./makenemo clean |
---|
| 62 | |
---|
| 63 | and if you have already compiled you can re-run all sette.sh and compilation part will be by-passed. |
---|
| 64 | |
---|
| 65 | NOTE 2: |
---|
| 66 | if you want a completly verbose makenemo you can uncomment "set -x" in makenemo script |
---|
| 67 | and then launch ./sette.sh 2>&1 | tee out.sette |
---|
| 68 | |
---|
| 69 | TO ADD NEW CONFIGURATION: |
---|
| 70 | 1. makenemo -n NEW_CONFIG (see makenemo -h for help) OR |
---|
| 71 | 2. makenemo -n NEW_CONFIG -r REF_CONFIG (if it is based on another reference configuration) |
---|
| 72 | 3. for fcm_job.sh creates a new input_NEW_CONFIG.cfg if you need tar file (if you use same tar file of GYRE, ORCA2_LIM or ORCA2_LIM_PISCES you can use it) |
---|
| 73 | |
---|
| 74 | TO ADD NEW MACHINE: |
---|
| 75 | 1. add arch-compiler.fcm in NEMOGCM/ARCH directory |
---|
| 76 | 2. makenemo -m new_compiler (see makenemo -h for help) |
---|
| 77 | |
---|
| 78 | TO ADD NEW BATCH-FILE: |
---|
| 79 | 1. see in SETTE/BATCH_TEMPLATE directory existing examples |
---|
| 80 | 2. create you own batch file like: batch-${COMPILER} |
---|
| 81 | (note: sette.sh will copy it in job_batch_template if you run tests in INTERACT_FLAG="no") |
---|
[5095] | 82 | |
---|
| 83 | # Variables to be checked by user: |
---|
| 84 | # |
---|
| 85 | # COMPILER : name of compiler as defined in NEMOGCM/ARCH directory |
---|
| 86 | # BATCH_COMMAND_PAR : name of the command for submitting parallel batch jobs |
---|
| 87 | # BATCH_COMMAND_SEQ : name of the command for submitting sequential batch jobs |
---|
| 88 | # INTERACT_FLAG : flag to run in interactive mode "yes" |
---|
| 89 | # to run in batch mode "no" |
---|
| 90 | # MPIRUN_FLAG : flag to run in parallel (MPI) "yes" |
---|
| 91 | # to run in sequential mode (NB_PROC = 1) "no" |
---|
| 92 | # USING_XIOS : flag to control the activation of key_iomput |
---|
| 93 | # "yes" to compile using key_iomput and link to the external XIOS library |
---|
| 94 | # "no" to compile without key_iomput and link to the old IOIPSL library |
---|
| 95 | # USING_MPMD : flag to control the use of stand-alone IO servers |
---|
| 96 | # requires USING_XIOS="yes" |
---|
| 97 | # "yes" to run in MPMD (detached) mode with stand-alone IO servers |
---|
| 98 | # "no" to run in SPMD (attached) mode without separate IO servers |
---|
| 99 | # NUM_XIOSERVERS : number of stand-alone IO servers to employ |
---|
| 100 | # set to zero if USING_MPMD="no" |
---|
| 101 | # |
---|
| 102 | # Principal script is sette.sh, that calls |
---|
| 103 | # |
---|
| 104 | # makenemo : to create successive exectuables in ${CONFIG_NAME}/BLD/bin/nemo.exe |
---|
| 105 | # and links to opa in ${CONFIG_NAME}/EXP00) |
---|
| 106 | # |
---|
| 107 | # param.cfg : sets and loads following directories: |
---|
| 108 | # |
---|
| 109 | # FORCING_DIR : is the directory for forcing files (tarfile) |
---|
| 110 | # INPUT_DIR : is the directory for input files storing |
---|
| 111 | # TMPDIR : is the temporary directory (if needed) |
---|
| 112 | # NEMO_VALIDATION_DIR : is the validation directory |
---|
| 113 | # |
---|
| 114 | # (NOTE: this file is the same for all configrations to be tested with sette) |
---|
| 115 | # |
---|
| 116 | # all_functions.sh : loads functions used by sette (note: new functions can be added here) |
---|
| 117 | # set_namelist : function declared in all_functions that sets namelist parameters |
---|
| 118 | # post_test_tidyup : creates validation storage directory and copies required output files |
---|
| 119 | # (solver.stat and ocean.output) in it after execution of test. |
---|
| 120 | # |
---|
| 121 | # VALIDATION tree is: |
---|
| 122 | # |
---|
| 123 | # NEMO_VALIDATION_DIR/WCONFIG_NAME/WCOMPILER_NAME/TEST_NAME/REVISION_NUMBER(or DATE) |
---|
| 124 | # |
---|
| 125 | # prepare_exe_dir.sh : defines and creates directory where the test is executed |
---|
| 126 | # execution directory takes name of TEST_NAME defined for every test |
---|
| 127 | # in sette.sh. (each test in executed in its own directory) |
---|
| 128 | # |
---|
| 129 | # prepare_job.sh : to generate the script run_job.sh |
---|
| 130 | # |
---|
| 131 | # fcm_job.sh : run in batch (INTERACT_FLAG="no") or interactive (INTERACT_FLAG="yes") |
---|
| 132 | # see sette.sh and BATCH_TEMPLATE directory |
---|
| 133 | # |
---|
| 134 | # NOTE: jobs requiring initial or forcing data need to have an input_CONFIG.cfg in which |
---|
| 135 | # can be found paths to the input tar file) |
---|
| 136 | # NOTE: if job is not launched for any reason you have the executable ready in ${EXE_DIR} |
---|
| 137 | # directory |
---|
| 138 | # NOTE: the changed namelists are left in ${EXE_DIR} directory whereas original namelists |
---|
| 139 | # remain in ${NEW_CONF}/EXP00 |
---|
| 140 | # |
---|
| 141 | # NOTE: a log file, output.sette, is created in ${SETTE_DIR} with the echoes of |
---|
| 142 | # executed commands |
---|
| 143 | # |
---|
| 144 | # NOTE: if sette.sh is stopped in output.sette there is written the last command |
---|
| 145 | # executed by sette.sh |
---|