Changeset 11140


Ignore:
Timestamp:
2019-06-19T17:38:50+02:00 (16 months ago)
Author:
mathiot
Message:

sette rpt changes: FAILED line in red, indication when the run start to diverge, move old/unused script to OLD, update README to README.rst and add extra parameters in param.cfg

Location:
NEMO/branches/2019/fix_sette_ticket2239
Files:
1 added
4 edited
4 moved

Legend:

Unmodified
Added
Removed
  • NEMO/branches/2019/fix_sette_ticket2239/README.rst

    r10820 r11140  
    1 usage of SETTE package: 
     1====================== 
     2usage of SETTE package 
     3====================== 
    24 
    3 INSTALLATION: 
    4 The sette directory have to be located in the main directory (at the same level as src/ or cfg/). 
     5INSTALLATION 
     6============ 
     7* checkout the code as follow: 
     8.. code-block:: console 
    59 
    6 PRINCIPAL SCRIPT: sette.sh  
    7  * sette.sh is a simple wrapper that runs tests on the reference configurations, test cases and generates the report at the end. 
    8  * sette.sh is able to take a list of configurations in argument (sette.sh -t "CFG1 CFG2 CFG3").  
     10        $ svn checkout http://forge.ipsl.jussieu.fr/nemo/svn/utils/CI/sette sette 
     11 
     12* The sette directory have to be located in the main directory (at the same level as ``src/`` or ``cfg/``). 
     13 
     14MASTER SCRIPT: sette.sh  
     15======================= 
     16* ``sette.sh`` is a simple wrapper that runs tests on the reference configurations, test cases and generates the report at the end. 
     17* ``sette.sh`` is able to take a list of configurations in argument (sette.sh -t "CFG1 CFG2 CFG3").  
    918       - The list of available configurations is available in running ./sette_list_avail_cfg.sh. 
    1019       - The default for sette is to run all of the reference and test configurations. 
    11  * Users need to set up the param.cfg file correctly and download input files before running the script. 
     20* Users need to set up the ``param.cfg`` file correctly and download input files before running the script. 
    1221 
    1322SECONDARY SCRIPT 
    14  * sette_reference-configuration.sh runs sette tests on the reference configurations 
    15  * sette_test-cases.sh              runs sette tests on test cases 
    16  * sette_list_avail_rev.sh          generates a listing on all the revisions available in the validation directory for each configuration. 
    17  * sette_list_avail_cfg.sh          generates a listing of all the available configurations. 
    18  * sette_rpt.sh                     generates the report 
     23================ 
     24* ``sette_reference-configuration.sh`` runs sette tests on the reference configurations 
     25* ``sette_test-cases.sh``              runs sette tests on test cases 
     26* ``sette_list_avail_rev.sh``          generates a listing on all the revisions available in the validation directory for each configuration. 
     27* ``sette_list_avail_cfg.sh``          generates a listing of all the available configurations. 
     28* ``sette_rpt.sh``                     generates the report 
    1929 
    2030USER SETUP (recommended) 
    21  * in param.cfg : 
    22       - variables (COMPILER, NEMO_VALIDATION_REF, NEMO_REV_REF) have to be specified in param.cfg 
    23       - variables (BATCH_CMD, BATCH_STAT, BATCH_NAME, FORCING_DIR, SVN_CMD, ADD_NOSIGNEDZERO) 
    24            can be exported from your shell startup files. It is advise to setup the default value in your startup file if it doesn't fit your need. 
    25            If you prefer not doing so, you need to update the default initialisation 
     31======================== 
     32* in ``param.cfg`` : 
     33      - variables (NEMO_VALIDATION_REF, NEMO_REV_REF) have to be specified in param.cfg 
     34      - variables SETTE_(COMPILER, USING_XIOS, USING_MPMD, JOB_PREFIX_MPMD, JOB_PREFIX_NOMPMD, BATCH_CMD, BATCH_STAT, BATCH_NAME, FORCING_DIR, SVN_CMD, NEMO_VALIDATION_DIR, ADD_NOSIGNEDZERO) 
     35        can be exported from your shell startup files. It is advise to setup the default value in your startup file if it doesn't fit your need. 
     36        If you prefer not doing so, you need to update the default initialisation 
    2637      - variable NEMO_VALIDATION_DIR is sette by default in MAIN_DIR/NEMO_VALIDATION (this is to avoid overwritting results when running sette on different branches). 
    27       - description of variables listed in param.cfg: 
    28                COMPILER            : compiler among those available in NEMOGCM/ARCH 
    29                # 
    30                # reference version for bit reproducibility results 
    31                NEMO_VALIDATION_REF : reference directory 
    32                NEMO_REV_REF        : reference revision 
    33                # 
    34                # batch command needed 
    35                BATCH_CMD           : command for job submission in batch mode 
    36                BATCH_STAT          : command to check job status 
    37                BATCH_NAME          : generic sette job name (as it appears with $BATCH_STAT command) 
    38                # 
    39                # file storing 
    40                FORCING_DIR         : directory where is stored input.tar file (same name in input_CONFIG_NAME.cfg) 
    41                                      reference configuration input tar file could be found here : https://zenodo.org/record/1472245 
    42                NEMO_VALIDATION_DIR : directory where is stored restarts, run.stat, tracer.stat and ocean.output files for each configuration 
    43                                      ( NEMO_VALIDATION_DIR/WCONFIG_NAME/WCOMPILER_NAME/TEST_NAME/REVISION_NUMBER(or DATE) ) 
    44                INPUT_DIR           : directory where is stored input files (DO NOT CHANGE IT) 
    45                # 
    46                # misc. 
    47                SVN_CMD             : svn command use to do svn info (default svn). Could be useful if you are using git svn 
    48                                    : Reference directory for result comparison will be NEMO_VALIDATION_REF/COMPILER/NEMO_REV_REF 
    49                ADD_NODIGNEDZERO    : set "yes" if you need key_nosignedzero to run nemo 
     38      - description of variables listed in param.cfg (SETTE_): 
     39                 
     40        # reference version for bit reproducibility results 
     41          |  NEMO_VALIDATION_REF : reference directory 
     42          |  NEMO_REV_REF        : reference revision 
     43                 
     44        # compiler information 
     45          |  COMPILER            : compiler among those available in NEMOGCM/ARCH 
     46                 
     47        # XIOS management 
     48          |  USING_XIOS        : flag to control the activation of key_iomput.  
     49                                "yes" to compile using key_iomput and link to the external XIOS library. 
     50                                "no"  to compile without key_iomput and link to the old IOIPSL library. 
     51          |  USING_MPMD        : flag to control the use of stand-alone IO servers  
     52                                (requires USING_XIOS="yes"). 
     53                                "yes" to run in MPMD (detached) mode with stand-alone IO servers. 
     54                                "no"  to run in SPMD (attached) mode without separate IO servers.  
     55        # generique batch scrip prefix name if MPMD set to true/false 
     56           | JOB_PREFIX_MPMD 
     57           | JOB_PREFIX_NOMPMD 
     58                 
     59        # batch command needed 
     60           | BATCH_CMD           : command for job submission in batch mode 
     61           | BATCH_STAT          : command to check job status 
     62           | BATCH_NAME          : generic sette job name (as it appears with $BATCH_STAT command) 
     63                 
     64        # file storing 
     65           | FORCING_DIR         : directory where is stored input.tar file (same name in input_CONFIG_NAME.cfg) 
     66                                   reference configuration input tar file could be found here : https://zenodo.org/record/1472245 
     67           | NEMO_VALIDATION_DIR : directory where is stored restarts, run.stat, tracer.stat and ocean.output files for each configuration 
     68                                   ( NEMO_VALIDATION_DIR/WCONFIG_NAME/WCOMPILER_NAME/TEST_NAME/REVISION_NUMBER(or DATE) ) 
     69           | INPUT_DIR           : directory where is stored input files (DO NOT CHANGE IT) 
     70                 
     71        # misc. 
     72           | SVN_CMD             : svn command use to do svn info (default svn). Could be useful if you are using git svn 
     73                                 : Reference directory for result comparison will be NEMO_VALIDATION_REF/COMPILER/NEMO_REV_REF 
     74           | ADD_NODIGNEDZERO    : set "yes" if you need key_nosignedzero to run nemo 
    5075 
    51 USAGE of main scripts: 
    52  * sette.sh                : it is a simple wrapper to run tests on the reference configurations, test-cases and generate the report at the end. 
     76USAGE of main scripts 
     77===================== 
     78* ``sette.sh``                : it is a simple wrapper to run tests on the reference configurations, test-cases and generate the report at the end. 
    5379          - if no argument is given, sette run all the reference configurations and test cases  
    5480          - if a list of argument is provided (sette.sh -t "CFG1 CFG2 CFG3"), sette will only run these configurations.  
    5581             The list of available configurations is available in running ./sette_list_avail_cfg.sh. 
    5682          - user need to set up the param.cfg file correctly and download input file before running the script. 
    57  * sette_rpt.sh            : it generates the sette report. 
     83* ``sette_rpt.sh``            : it generates the sette report. 
    5884          - if no argument is given, the report will be generated on the last changed revision. 
    5985          - if an argument is given (revision number) the report will be generated for this revision only 
     
    6187          - it is possible to retreive all the available revision test using sette_list_avail_rev.sh 
    6288          - it is NOT possible to run sette_rpt.sh for a single configuration. 
    63  * sette_list_avail_rev.sh : generate a listing on all the revisions available in the validation directory for each configuration. 
     89* ``sette_list_avail_rev.sh`` : generate a listing on all the revisions available in the validation directory for each configuration. 
    6490          - no argument needed 
    65  * sette_list_avail_cfg.sh : generate a listing of all the available configurations. 
     91* ``sette_list_avail_cfg.sh`` : generate a listing of all the available configurations. 
    6692          - no argument needed 
    6793 
    68 NOTES: 
    69  * compilation issues: 
    70    - in case of error you can remove you NEW_CONF directory and all files doing : 
    71      ./makenemo -n MY_CONFIG clean_config 
     94NOTES 
     95===== 
     96* compilation issues: 
     97   - in case of error you can remove your NEW_CONF directory and all files doing : 
     98 
     99     :: 
     100 
     101     $ ./makenemo -n MY_CONFIG clean_config 
     102 
    72103   - if you want recompile nemo but before you want to clean compiled code do : 
     104 
     105     :: 
     106 
    73107     ./makenemo clean 
     108 
    74109   - if you have already compiled you can re-run all sette.sh and compilation part will be by-passed. 
    75  * verbose sette output: 
    76    - if you want a completly verbose makenemo you can uncomment "set -x" in makenemo script 
    77      and then launch ./sette.sh 2>&1 | tee out.sette . This creates out.sette file in ${SETTE_DIR} 
    78110 
    79 TO ADD NEW CONFIGURATION: 
    80 1. makenemo -n NEW_CONFIG (see makenemo -h for help) OR 
    81 2. makenemo -n NEW_CONFIG -r REF_CONFIG (if it is based on another reference configuration) 
    82 3. for fcm_job.sh creates a new input_NEW_CONFIG.cfg if you need tar file (if you use same tar file of GYRE, ORCA2_LIM or ORCA2_LIM_PISCES you can use it) 
    83 4. add a bloc in one of the sette_reference-configuration.sh or sette_test-cases.sh script  
    84 5. add your configuration to the list in param.cfg 
     111* verbose sette output: 
     112   - if you want a completly verbose makenemo you can uncomment `set -x` in makenemo script 
     113     and then launch `./sette.sh 2>&1 | tee out.sette` . This creates out.sette file in ${SETTE_DIR} 
    85114 
    86 TO ADD NEW MACHINE: 
    87 1. add arch-compiler.fcm in NEMOGCM/ARCH directory 
     115TO ADD NEW CONFIGURATION 
     116========================= 
     1171. creates a new ``input_NEW_CONFIG.cfg`` if you need tar file (if you use same tar file of GYRE, ORCA2_LIM or ORCA2_LIM_PISCES you can use it) 
     1182. add a bloc in one of the ``sette_reference-configuration.sh`` or ``sette_test-cases.sh`` script  
     1193. add your configuration to the list in ``param.cfg`` 
     120 
     121TO ADD NEW MACHINE 
     122=================== 
     1231. add ``arch-compiler.fcm`` in ``NEMOGCM/ARCH`` directory 
    881242. makenemo -m new_compiler  (see makenemo -h for help) 
    891253. add new batch-file 
    90126 
    91 TO ADD NEW BATCH-FILE: 
    92 1. see in SETTE/BATCH_TEMPLATE directory existing examples 
    93 2. create you own batch file like: batch-${COMPILER} 
    94    (note: sette_test-cases.sh and sette_cfg-ref.sh will copy it in job_batch_template if you run tests in INTERACT_FLAG="no") 
     127TO ADD NEW BATCH-FILE 
     128====================== 
     1291. see in ``SETTE/BATCH_TEMPLATE`` directory existing examples 
     1302. create you own batch file like: ``batch-${COMPILER}`` file 
     131   (note: sette_test-cases.sh and ``sette_cfg-ref.sh`` will copy it in job_batch_template if you run tests in INTERACT_FLAG="no") 
    95132 
    96  
    97  
    98  
    99  
    100 USER MODIFICATIONS (sequential/parrallel (default), interacive or not (default) and mpi (default) or not ) 
    101  * in sette_cfg-ref.sh and sette_test-cases.sh script : 
    102      - BATCH_COMMAND_PAR is the command for job submission in batch mode parallel (MMIÂ) specified in param.cfg 
    103      - BATCH_COMMAND_SEQ is the command for job submission in batch mode sequential (NB_PROC = 1) 
    104                                  the default value is the BATCH_COMMAND_PAR value 
    105           - INTERACT_FLAG : "yes" if you want to run in interactive mode  
    106                             "no"  if you want to run in batch mode (default)      
    107      - MPIRUN_FLAG   : "yes" if you want to run in parallel (MPI) (default) 
    108                   "no"  if you want to run in sequential mode (NB_PROC = 1) 
    109  
    110 USAGE of internal function/script 
    111  * makenemo -m : machine (compilator); see ./makenemo -m help (in NEMOGCM/CONFIG directory) 
    112             -n : config name (in examples config_name_nproci_nprocj), see ./makenemo -n help 
    113             -r : reference configuration (if you don't give it you have to choise directories to install) 
    114        add_key : to add list of keys 
    115        del_key : to del list of keys 
    116        and for more details and options of makenemo see ./makenemo -h 
    117  * set_namelist       : to set namelist values 
    118                           (example : to set 75 time steps for one test : namelist nn_itend 75) 
    119                           (        : to change cn_icerst_in name : namelist_ice cn_icerst_in  \"test_00101231_restart_icemod\") 
    120                           (        : to activate control print : namelist ln_ctl false ) 
    121  * prepare_exe_dir.sh : creates execution directory for each test 
    122                           - for EACH JOB EXECUTION it is needed to set TEST_NAME variable 
    123                           - TEST_NAME variable gives name to execution directory (created in prepare_exe_dir.sh) 
    124                           - It is mandatory to have different TEST_NAME for each job run if not existing files are re-written (for example namelist) 
    125  * prepare_job.sh     : generates the script to be executed 
    126                           - input arguments are INPUT_FILE_CONFIG_NAME NUMBER_PROC TEST_NAME MPIRUN_FLAG JOB_FILE 
    127  * fcm_job.sh         : input arguments are TOTAL_NUMBER_OF_PROCS_used JOB_FILE_NAME INTERACT_FLAG MPIRUN_FLAG 
    128                         (for example if you run with 4 procs JOB_FILE batch parallel 
    129                         and only run short test ===> ./fcm_job 4 SHORT no    yes   ) 
    130                           - IMPORTANT control that name of tar file in FORCING directory 
    131                             is the same of that in input_REF_CONFIG.cfg, if not change one. 
    132                           - change in "RUN OPA" if [ ${INTERACT_FLAG} == "yes" ] mpirun command line (some examples are already written) 
    133                           - if INTERACT_FLAG="no" it is needed a batch_file (see below to create a new one) 
     133EXTRA SETTING POSSIBLE 
     134====================== 
     135Among the setting that can be modified by the user in sette_cfg-ref.sh and sette_test-cases.sh script, 
     136there are: sequential/parrallel (default), interacive or not (default) and mpi (default) or not. 
     137  - | BATCH_COMMAND_PAR is the command for job submission in batch mode parallel (specified in param.cfg). 
     138  - | BATCH_COMMAND_SEQ is the command for job submission in batch mode sequential (NB_PROC = 1). 
     139                     the default value is the BATCH_COMMAND_PAR value. 
     140  - | INTERACT_FLAG : "yes" if you want to run in interactive mode. 
     141                    "no"  if you want to run in batch mode (default).      
     142  - | MPIRUN_FLAG   : "yes" if you want to run in parallel (MPI) (default). 
     143                    "no"  if you want to run in sequential mode (NB_PROC = 1). 
  • NEMO/branches/2019/fix_sette_ticket2239/param.cfg

    r10819 r11140  
    11# Computer/user dependant variable needed to run sette 
    2 # 
    3 # Compiler among those in NEMOGCM/ARCH 
    4 COMPILER=X64_ADA 
    52# 
    63# REFERENCE run.stat tracer.stat you want to compare with 
     
    118# ------------------------------------------------------------------------------------------ 
    129# IMPORTANT: 
    13 # variables (BATCH_CMD, BATCH_STAT, BATCH_NAME, FORCING_DIR, SVN_CMD, ADD_NOSIGNEDZERO) 
     10# variables (COMPILER, USING_XIOS, USING_MPMD, BATCH_CMD, BATCH_STAT, BATCH_NAME, FORCING_DIR, SVN_CMD, ADD_NOSIGNEDZERO) 
    1411# can be exported from your shell startup files.  
    1512# If it is the case, nothing to do 
    1613# otherwise, you need to update the default initialisation 
    1714# 
    18 # ADVISE: setup the default value in your startup file if it doesn't fit your need. 
     15# ADVISE: setup the default value in your startup file if it doesn't fit your need for at least 
     16# SETTE_COMPILER, SETTE_BATCH_CMD, SETTE_BATCH_STAT, SETTE_BATCH_NAME, SETTE_FORCING_DIR,  
     17# 
     18# ------------------------------------------------------------------------------------------ 
     19# COMPILER setting 
     20# Compiler among those in NEMOGCM/ARCH 
     21COMPILER=${SETTE_COMPILER:-XXXXXXXX} 
     22# ------------------------------------------------------------------------------------------ 
    1923# 
    2024# BATCH command 
     
    2529# generic sette job name (as it appears with $BATCH_STAT command) 
    2630BATCH_NAME=${SETTE_BATCH_NAME:-sette} 
     31# ------------------------------------------------------------------------------------------ 
    2732# 
    28 # 
    29 # FILES STORING 
     33# FILES STORING paths 
    3034# forcing files storing 
    3135FORCING_DIR=${SETTE_FORCING_DIR:-$WORKDIR/FORCING} 
    3236# validation files storing 
    33 NEMO_VALIDATION_DIR=${WORKDIR:-$MAIN_DIR}/NEMO_VALIDATION 
     37NEMO_VALIDATION_DIR=${SETTE_NEMO_VALIDATION_DIR:-$MAIN_DIR}/NEMO_VALIDATION 
    3438# input files storing (namelist, iodef ...) (DO NOT CHANGE) 
    3539INPUT_DIR=${CONFIG_DIR}/${NEW_CONF}/EXP00 
     40# ------------------------------------------------------------------------------------------ 
    3641# 
     42# RUN set up 
     43# USING_XIOS        : flag to control the activation of key_iomput 
     44#                      "yes" to compile using key_iomput and link to the external XIOS library 
     45#                      "no"  to compile without key_iomput and link to the old IOIPSL library 
     46USING_XIOS=${SETTE_XIOS:-"yes"} 
     47# USING_MPMD        : flag to control the use of stand-alone IO servers 
     48#                     requires USING_XIOS="yes" 
     49#                      "yes" to run in MPMD (detached) mode with stand-alone IO servers 
     50#                      "no"  to run in SPMD (attached) mode without separate IO servers  
     51# MPMD 
     52USING_MPMD=${SETTE_MPMD:-"no"} 
     53# generique batch scrip prefix name if MPMD set to true/false 
     54JOB_PREFIX_MPMD=${SETTE_JOB_PREFIX_MPMD:-batch-mpmd} 
     55JOB_PREFIX_NOMPMD=${SETTE_JOB_PREFIX_NOMPMD:-batch} 
     56# ------------------------------------------------------------------------------------------ 
    3757# 
    3858# MISCELENIOUS 
     
    4464#- only for IBM 
    4565#TMPDIR=${CONFIG_DIR}/${NEW_CONF}/EXP00 
    46 # 
     66# ------------------------------------------------------------------------------------------ 
    4767# 
    4868# LIST OF CONFIGURATIONS 
    4969# to be updated if you added an new configuration in sette_test-cases.sh or sette_reference-configuration.sh 
    5070TEST_CONFIGS=${SETTE_TEST_CONFIGS:-"ORCA2_ICE_PISCES ORCA2_OFF_PISCES AMM12 AGRIF SPITZ12 GYRE_PISCES SAS ORCA2_ICE_OBS ICE_AGRIF OVERFLOW LOCK_EXCHANGE VORTEX ISOMIP"} 
     71# ------------------------------------------------------------------------------------------ 
  • NEMO/branches/2019/fix_sette_ticket2239/sette_reference-configurations.sh

    r10755 r11140  
    101101export INTERACT_FLAG="no" 
    102102export MPIRUN_FLAG="yes" 
    103 export USING_XIOS="yes" 
    104103export USING_ICEBERGS="yes" 
    105104# 
     
    118117# Settings which control the use of stand alone servers (only relevant if using xios) 
    119118# 
    120 export USING_MPMD="no" 
    121119export NUM_XIOSERVERS=4 
    122 export JOB_PREFIX=batch-mpmd 
     120export JOB_PREFIX=${JOB_PREFIX_MPMD} 
    123121# 
    124122if [ ${USING_MPMD} == "no" ]  
    125123 then 
    126124   export NUM_XIOSERVERS=0 
    127    export JOB_PREFIX=batch 
     125   export JOB_PREFIX=${JOB_PREFIX_NOMPMD} 
    128126fi 
    129127# 
  • NEMO/branches/2019/fix_sette_ticket2239/sette_rpt.sh

    r10769 r11140  
    2020} 
    2121 
    22 function restfile() { 
    23 # Rebuild ice restart for SAS CONFIG, and restartability checks. Expects LONG and SHORT run directories. 
    24 # For Stand Alone Surface configuration ocean is not running, just run ice model; so no outputs ocean files. 
    25 # Compares LONG rebuild restart ice file with equivalent entry from the SHORT rebuild restart ice file. 
    26 # 
    27 # check nco module loaded, and load it if not 
    28 if [ ! $( echo $LOADEDMODULES | grep cdo ) ]; then module load cdo >& /dev/null ; fi 
    29 # 
    30   vdir=$1 
    31   nam=$2 
    32   pass=$3 
    33  
    34 # get $dorv 
    35   get_dorv 
    36 # 
    37 # check if directory is here 
    38   if [ ! -d $vdir/$nam/$mach/$dorv ]; then 
    39     printf "%-27s %s %s\n" $nam  " directory                  MISSING : " $dorv 
    40     return 
    41   fi 
    42  
    43   if [ -d $vdir/$nam/$mach/$dorv/ ]; then 
    44     rep1=`ls -1rt $vdir/$nam/$mach/$dorv/ | tail -2l | head -1 ` 
    45     rep2=`ls -1rt $vdir/$nam/$mach/$dorv/ | tail -1l` 
    46     cd ${SAS_RESTART_DIR}/LONG 
    47     #SF add here compilation of rebuild_tools to rebuild restart files, and add comparison of restart files 
    48     cd ${TOOLS_DIR} 
    49     ./maketools -n REBUILD_NEMO -m ${mach} > /dev/null 2>&1 
    50     cd ${TOOLS_DIR}/REBUILD_NEMO 
    51     #SF echo "REBUILD LONG restart SAS files, without standard output" 
    52     ./rebuild_nemo -t 4 ../../cfgs/ORCA2_SAS_ICE_ST/LONG/SAS_00000240_restart_ice  $NPROC > /dev/null 2>&1 
    53     #SF echo "REBUILD SHORT restart SAS files, without standard output" 
    54     ./rebuild_nemo -t 4 ../../cfgs/ORCA2_SAS_ICE_ST/SHORT/SAS_00000240_restart_ice $NPROC >&- 
    55     cd ${SAS_RESTART_DIR}/LONG 
    56     #SF echo "COPY rebuild restart files" 
    57     cp SAS_00000240_restart_ice.nc $vdir/$nam/$mach/$dorv/LONG/. 
    58     cp ../SHORT/SAS_00000240_restart_ice.nc $vdir/$nam/$mach/$dorv/SHORT/. 
    59  
    60     f1o=$vdir/$nam/$mach/$dorv/LONG/SAS_00000240_restart_ice.nc 
    61     f2o=$vdir/$nam/$mach/$dorv/SHORT/SAS_00000240_restart_ice.nc 
    62     if  [ ! -f $f1o ] &&  [ ! -f $f2o ] ; then 
    63       printf "%-27s %s\n" $nam " REBUILD SAS restart ice DOES NOT exists; incomplete test"; 
    64       return; 
    65     fi 
    66     # 
    67     done_oce=0 
    68     # 
    69   if  [  -f $f1o ] && [  -f $f2o ]; then 
    70 ## Compare the two netcdf files 
    71     cdo diffn $f1o $f2o > cdo_diff.out 2> /dev/null 
    72 ## Identical if cdo_diff.out exists but has zero size 
    73     if [ ! -s cdo_diff.out ]; then 
    74        difi=0 
    75     else 
    76 ## Identical if first character of $dif ==0 
    77        dif=$( grep -om1 '[0-9]* of [0-9]* records differ' cdo_diff.out ) 
    78 # difi contains the first character of summary of cdo dif. if = 0, then 0 record differ between the 2 files     
    79        if [ -n "$dif" ]; then 
    80            difi=`echo $dif | cut -c -1` 
    81        fi 
    82     fi 
    83     \rm cdo_diff.out  
    84  
    85     if [ $difi == 0 ]; then    
    86        if [ $pass == 0 ]; then 
    87          printf "%-27s %s %s\n" $nam  " ice restarts are IDENTICAL  passed : " $dorv 
    88        fi 
    89     else 
    90        printf "%-27s %s %s\n" $nam  " ice restarts are DIFFERENT  FAILED : " $dorv  
    91         # 
    92    # Offer view of differences on the second pass 
    93    # 
    94         if [ $pass == 1 ]; then 
    95           echo "<return> to view restart_ice.nc differences" 
    96           read y 
    97           cdo -diffv $f1o $f2o 
    98           done_oce=1 
    99           #echo "<return> to continue" 
    100           #read y 
    101         fi 
    102     fi 
    103   else 
    104       printf "%-27s %s\n" $nam " incomplete test"; 
    105       return; 
    106   fi 
    107 # 
    108 fi 
     22function get_ktdiff() { 
     23  ktdiff=`diff ${1} ${2} | head -2 | grep it | awk '{ print $4 }'` 
    10924} 
    11025 
     
    15974        fi 
    16075      else 
    161         printf "%-27s %s %s\n" $nam  " run.stat    restartability  FAILED : " $dorv  
     76        get_ktdiff f1.tmp$$ $f2s 
     77        printf "\e[38;5;196m%-27s %s %s %s %-5s %s\e[0m\n" $nam  " run.stat    restartability  FAILED : " $dorv " (results are different after " $ktdiff " time steps)" 
    16278# 
    16379# Offer view of differences on the second pass 
     
    188104        fi 
    189105      else 
    190         printf "%-27s %s %s\n" $nam  " tracer.stat restartability  FAILED : " $dorv  
     106        get_ktdiff f1.tmp$$ $f2t 
     107        printf "\e[38;5;196m%-27s %s %s %s %-5s %s\e[0m\n" $nam  " tracer.stat    restartability  FAILED : " $dorv " (results are different after " $ktdiff " time steps)" 
    191108# 
    192109# Offer view of differences on the second pass 
     
    263180        fi 
    264181      else 
    265         printf "%-27s %s %s\n" $nam  " run.stat    reproducibility FAILED : " $dorv  
     182        get_ktdiff $f1s $f2s 
     183        printf "\e[38;5;196m%-27s %s %s %s %-5s %s\e[0m\n" $nam  " run.stat    reproducibility FAILED : " $dorv " (results are different after " $ktdiff " time steps)" 
    266184# 
    267185# Offer view of differences on the second pass 
     
    289207        fi 
    290208      else 
    291         printf "%-27s %s %s\n" $nam  " tracer.stat reproducibility FAILED : " $dorv 
     209        get_ktdiff $f1t $f2t 
     210        printf "\e[38;5;196m%-27s %s %s %s %-5s %s\e[0m\n" $nam  " tracer.stat    reproducibility FAILED : " $dorv " (results are different after " $ktdiff " time steps)" 
    292211# 
    293212# Offer view of differences on the second pass 
     
    426345          nerr=`grep 'E R R O R' $f1o | wc -l` 
    427346          if [[ $nerr > 0 ]]; then 
    428              printf "%-27s %s %s %s\n" $nam " run                         FAILED : " $dorv " ( E R R O R in ocean.output) "  
     347             printf "\e[38;5;196m%-27s %s %s %s\e[0m\n" $nam " run                         FAILED : " $dorv " ( E R R O R in ocean.output) "  
    429348             if [ $pass == 1 ]; then 
    430349                echo "<return> to view end of ocean.output" 
     
    479398        fi 
    480399      else 
    481         printf "%-5s %s %-5s %s %s %s\n" $rep1 "AGRIF vs" $rep2 "NOAGRIF run.stat    changed  -     FAILED : " $dorv1 $dorv2 
     400        get_ktdiff $f1s $f2s 
     401        printf "\e[38;5;196m%-5s %s %-5s %s %s %s %s %-5s %s\e[0m\n" $rep1 "AGRIF vs" $rep2 "NOAGRIF run.stat    changed  -     FAILED : " $dorv1 $dorv2 " (results are different after " $ktdiff " time steps)" 
    482402# 
    483403# Offer view of differences on the second pass 
     
    559479# 
    560480 
    561 # Rebuild and restartability test for SAS 
    562 # clem: not needed anymore 
    563 # for restart_file in WORCA2_SAS_ICE_ST 
    564 # do 
    565 #   restfile $NEMO_VALID $restart_file $pass 
    566 # done 
    567 # 
    568481# Restartability test 
    569482 echo "" 
  • NEMO/branches/2019/fix_sette_ticket2239/sette_test-cases.sh

    r10755 r11140  
    6060#                      (run.stat and ocean.output) in it after execution of test. 
    6161# 
    62 #  clean_valid_dir    : rename ocean.output/run.stat and tracer.stat to avoid checking them in the report  
    63 #                       ( not doing it could lead to false positive ) 
    64 # 
    6562#  VALIDATION tree is: 
    6663# 
     
    7067#                       execution directory takes name of TEST_NAME defined for every test  
    7168#                       in sette_test-cases.sh. (each test in executed in its own directory) 
     69# 
     70#  set_valid_dir       : rename ocean.output/run.stat and tracer.stat to avoid checking them in the report  
     71# 
     72#  clean_valid_dir    : rename ocean.output/run.stat and tracer.stat to avoid checking them in the report  
     73#                       ( not doing it could lead to false positive ) 
    7274# 
    7375#  prepare_job.sh     : to generate the script run_job.sh 
     
    101103export INTERACT_FLAG="no" 
    102104export MPIRUN_FLAG="yes" 
    103 export USING_XIOS="yes" 
    104105# 
    105106export DEL_KEYS="key_iomput" 
     
    117118# Settings which control the use of stand alone servers (only relevant if using xios) 
    118119# 
    119 export USING_MPMD="no" 
    120120export NUM_XIOSERVERS=4 
    121 export JOB_PREFIX=batch-mpmd 
     121export JOB_PREFIX=${JOB_PREFIX_MPMD} 
    122122# 
    123123if [ ${USING_MPMD} == "no" ]  
    124124 then 
    125125   export NUM_XIOSERVERS=0 
    126    export JOB_PREFIX=batch 
     126   export JOB_PREFIX=${JOB_PREFIX_NOMPMD} 
    127127fi 
    128128# 
Note: See TracChangeset for help on using the changeset viewer.