| 1 | |
| 2 | This wiki relates to the ticket on NEMO reproducibility fails with land domains exclusion (#1704) |
| 3 | |
| 4 | '''Author''': Tomas Lovato |
| 5 | |
| 6 | '''Institution''': CMCC |
| 7 | |
| 8 | '''Proposed actions''', after the confirmation from other ST members: |
| 9 | - revise the code and the log of mppini subroutines |
| 10 | - evaluate the modified tool for offline mpp decomposition |
| 11 | |
| 12 | '''Additional details not included in the ticket''' |
| 13 | |
| 14 | I tried to use configuration 4 (ORCA2_LIM_PISCES) to test the reproducibility for a run with a total of 88 PEs (80 ocean+8 land) and one with only the ocean domains (80 PEs). |
| 15 | Here below a diff file of changes used in the test. |
| 16 | |
| 17 | |
| 18 | {{{ |
| 19 | @@ -385,7 +386,7 @@ if [ ${config} -eq 4 ] ; then |
| 20 | . ./all_functions.sh |
| 21 | . ./prepare_exe_dir.sh |
| 22 | JOB_FILE=${EXE_DIR}/run_job.sh |
| 23 | - NPROC=16 |
| 24 | + NPROC=88 |
| 25 | if [ -f ${JOB_FILE} ] ; then \rm ${JOB_FILE} ; fi |
| 26 | cd ${EXE_DIR} |
| 27 | set_namelist namelist_cfg nn_it000 1 |
| 28 | @@ -393,9 +394,9 @@ if [ ${config} -eq 4 ] ; then |
| 29 | set_namelist namelist_cfg nn_fwb 0 |
| 30 | set_namelist namelist_cfg ln_ctl .false. |
| 31 | set_namelist namelist_cfg ln_clobber .true. |
| 32 | - set_namelist namelist_cfg jpni 4 |
| 33 | - set_namelist namelist_cfg jpnj 4 |
| 34 | - set_namelist namelist_cfg jpnij 16 |
| 35 | + set_namelist namelist_cfg jpni 8 |
| 36 | + set_namelist namelist_cfg jpnj 11 |
| 37 | + set_namelist namelist_cfg jpnij 88 |
| 38 | set_namelist namelist_cfg nn_solv 2 |
| 39 | set_namelist namelist_top_cfg ln_trcdta .false. |
| 40 | set_namelist namelist_top_cfg ln_diatrc .false. |
| 41 | @@ -425,16 +426,16 @@ if [ ${config} -eq 4 ] ; then |
| 42 | export TEST_NAME="REPRO_2_8" |
| 43 | . ./prepare_exe_dir.sh |
| 44 | JOB_FILE=${EXE_DIR}/run_job.sh |
| 45 | - NPROC=16 |
| 46 | + NPROC=80 |
| 47 | if [ -f ${JOB_FILE} ] ; then \rm ${JOB_FILE} ; fi |
| 48 | cd ${EXE_DIR} |
| 49 | set_namelist namelist_cfg nn_it000 1 |
| 50 | set_namelist namelist_cfg nn_itend 75 |
| 51 | set_namelist namelist_cfg ln_clobber .true. |
| 52 | set_namelist namelist_cfg nn_fwb 0 |
| 53 | - set_namelist namelist_cfg jpni 2 |
| 54 | - set_namelist namelist_cfg jpnj 8 |
| 55 | - set_namelist namelist_cfg jpnij 16 |
| 56 | + set_namelist namelist_cfg jpni 8 |
| 57 | + set_namelist namelist_cfg jpnj 11 |
| 58 | + set_namelist namelist_cfg jpnij 80 |
| 59 | set_namelist namelist_cfg nn_solv 2 |
| 60 | set_namelist namelist_top_cfg ln_trcdta .false. |
| 61 | set_namelist namelist_top_cfg ln_diatrc .false. |
| 62 | |
| 63 | |
| 64 | }}} |
| 65 | |
| 66 | The test for reproducibility failed. |
| 67 | |
| 68 | I made the following change in the mppini_2.h subroutine to modify the |
| 69 | exclusion criteria, which now consider a domain as land when all points |
| 70 | within the region (inner+overlap) are zero (I also modified the offline |
| 71 | decomposition tool accordingly): |
| 72 | |
| 73 | {{{ |
| 74 | isurf = 0 |
| 75 | - DO jj = 1+jprecj, ilj-jprecj |
| 76 | - DO ji = 1+jpreci, ili-jpreci |
| 77 | + DO jj = 1, ilj |
| 78 | + DO ji = 1, ili |
| 79 | IF( imask(ji+iimppt(ii,ij)-1, jj+ijmppt(ii,ij)-1) == 1) |
| 80 | isurf = isurf+1 |
| 81 | END DO |
| 82 | END DO |
| 83 | }}} |
| 84 | |
| 85 | |
| 86 | After these modifications, I rerun SETTE using the configuration 4 test |
| 87 | always using 88 PEs, but this time the decomposition was 81 ocean + 7 |
| 88 | land. |
| 89 | The reproducibility test passed. |
| 90 | |
| 91 | |
| 92 | |
| 93 | |