Changes between Version 17 and Version 18 of 2021WP/HPC-02_Daley_Tiling

Timestamp:

2021-06-04T16:25:49+02:00 (3 years ago)

Author:

hadcv

Comment:

--

2021WP/HPC-02_Daley_Tiling

@@ -35 +35 @@
 [http://forge.ipsl.jussieu.fr/nemo/browser/NEMO/branches/2021/dev_r14393_HPC-03_Mele_Comm_Cleanup?rev=14776 dev_r14393_HPC-03_Mele_Comm_Cleanup@14776] has been merged into the branch.
  * [http://forge.ipsl.jussieu.fr/nemo/changeset?sfp_email=&sfph_mail=&reponame=&new=14805%40NEMO%2Fbranches%2F2021%2Fdev_r14273_HPC-02_Daley_Tiling/src/OCE&old=14776%40NEMO%2Fbranches%2F2021%2Fdev_r14393_HPC-03_Mele_Comm_Cleanup%2Fsrc%2FOCE Difference@14805 vs dev_r14393_HPC-03_Mele_Comm_Cleanup]
+
+The following bug fixes were applied to the trunk post-merge:
+ * [14840]- Add `ln_tile` to ORCA2_ICE_PISCES/namelist_cfg 
+ * [14845]- Fix diagnostics preventing ORCA2_ICE_PISCES running with `nn_hls = 2` and tiling
+ * [14857]- Fixes in MY_SRC for `nn_hls = 2`/tiling and traadv_fct.F90 for `nn_hls = 1`
+ * [14882]- Fix diagnostics preventing ORCA2_ICE_PISCES running with `nn_hls = 2` and tiling; r14845 missing pieces
+ * [14903]- Fix bug with A1Di/A1Dj/A2D macros, update standard tiling namelists
 
 ==== Changes to tiling framework
@@ -296 +303 @@
  * `nn_hls = 2` (`USING_EXTRA_HALO="yes"`) and `ln_tile = .true.` (using default 10i x 10j tile sizes)
 
-and are compared with results from the [http://forge.ipsl.jussieu.fr/nemo/browser/NEMO/trunk?rev=14820 trunk@14820] with `nn_hls = 1`.
-
-The Intel compiler (ifort 18.0.5 20180823) is used with XIOS ([http://forge.ipsl.jussieu.fr/ioserver/browser/XIOS/trunk?rev=2131 r2131 of the trunk]) in detached mode.
+and are compared with results from the [http://forge.ipsl.jussieu.fr/nemo/browser/NEMO/trunk?rev=14820 trunk@14820] with `nn_hls = 1` and the same settings for `NOT_USING_QCO`/`USING_ICEBERGS`.
+
+The Intel compiler (ifort 18.0.5 20180823, `XC40_METO_IFORT` arch file) is used with XIOS ([http://forge.ipsl.jussieu.fr/ioserver/browser/XIOS/trunk?rev=2131 r2131 of the trunk]) in detached mode.
 
 All tests (including SWG) pass, but it should be noted that the `USING_EXTRA_HALO` option is only used by ORCA2_ICE_PISCES. 
@@ -339 +346 @@
 === SETTE (post merge)
 
+The SETTE tests have been repeated with the [http://forge.ipsl.jussieu.fr/nemo/browser/NEMO/trunk?rev=14922 trunk@14922] in order to include bug fixes that allow all SETTE tests to be run with `nn_hls = 2` and tiling.
+
+The tests are the same as detailed above except:
+
+ * The [http://forge.ipsl.jussieu.fr/nemo/browser/NEMO/trunk?rev=14922 trunk@14922] is used (but still compared with results from the [http://forge.ipsl.jussieu.fr/nemo/browser/NEMO/trunk?rev=14820 trunk@14820])
+ * [http://forge.ipsl.jussieu.fr/nemo/browser/utils/CI/sette?rev=14844 SETTE@14844] is used
+ * Additional tests with `key_loop_fusion` have been performed
+ * `nn_hls = 2` is set directly in namelist_ref, instead of via `USING_EXTRA_HALO`, in order to run all SETTE tests with the extended haloes (and tiling)
+ * The default tile size in namelist_ref is 99999i x 10j (to ensure there is always only 1 tile in i)
+ * Icebergs are not activated
+
+All SETTE tests pass and give the same results as the [http://forge.ipsl.jussieu.fr/nemo/browser/NEMO/trunk?rev=14820 trunk@14820], except AGRIF_DEMO which differs after 17 timesteps for all `nn_hls = 2` tests.
+This is thought to be because one of the AGRIF domains in this configuration is not large enough for `nn_hls = 2`.
+
+==== Regular checks
+
+All checks are the same as before, but the run time/memory changes are significant in some cases. 
+These are reported here for increases in time/memory larger than 10% that are present in both REPRO experiments of a configuration:
+
+  * QCO, `nn_hls == 1`
+    * No significant changes
+  * QCO, `nn_hls == 2`
+    * GYRE_PISCES: time + 13-18%, memory + 13-18%
+  * QCO, `nn_hls == 2` and `ln_tile = .true.`
+    * AMM12: memory + 18%
+    * WED025: memory + 17%
+  * QCO, loop fusion and `nn_hls == 2`
+    * AMM12: time + 20%
+  * QCO, loop fusion, `nn_hls == 2` and `ln_tile = .true.`
+    * AGRIF_DEMO: time + 11-15%
+    * AMM12: memory + 17-20%
+    * WED025: memory + 19%
+
+  * non-QCO, `nn_hls == 1`
+    * No significant changes
+  * non-QCO, `nn_hls == 2`
+    * No significant changes
+  * non-QCO, `nn_hls == 2` and `ln_tile = .true.`
+    * AGRIF_DEMO: memory + 13%
+    * AMM12: memory + 18-20%
+    * GYRE_PISCES: time + 11-24%
+    * ORCA2_ICE_OBS: memory + 12-16%
+    * WED025: memory + 15-16%
+  * non-QCO, loop fusion and `nn_hls == 2`
+    * ORCA2_ICE_OBS: time + 11-17%
+  * non-QCO, loop fusion, `nn_hls == 2` and `ln_tile = .true.`
+    * AGRIF_DEMO: memory + 11-12%
+    * AMM12: memory + 21-23%
+    * WED025: memory + 17-19%
+
+The time increases do not seem consistent enough to indicate a systematic issue. 
+However, there is evidence to suggest that tiling increases the memory cost of AGRIF_DEMO (11-13%), AMM12 (17-23%) and WED025 (15-19%).
+This is partly due to the use of `nn_hls = 2`, which increases the domain size, but in AMM12 & WED025 this is only responsible for up to 7% of the increased memory cost.
 
 === Development testing