In coupled mode, the total non-solar heat flux is imposed by the atmosphere and should remain untouched. However, in the ice model we need to remove heat flux associated to E-P from the total flux. We did it but the sign was wrong. This line must be changed:

      ! --- total non solar flux --- !
      zqns_tot(:,:) = zqns_tot(:,:) + zqemp_ice(:,:) + zqemp_oce(:,:)

to

      ! --- total non solar flux --- !
      zqns_tot(:,:) = zqns_tot(:,:) - zqemp_ice(:,:) - zqemp_oce(:,:)

When running SETTE with the default settings all tests of NEOM 3.6 are passed. I setup a specific test to verify the reproducibility of the code with land domain exclusion, as this feature is very useful in high resolution configurations.

I tried to use configuration 4 (ORCA2_LIM_PISCES) to test the reproducibility for a run with a total of 88 PEs (80 ocean+8 land) and one with only the ocean domains (80 PEs). See the wiki of this ticket for the diff file of changes used in the test.

The test for reproducibility failed.

After a while digging in the code, I realised that the problem is due to the land domain exclusion criterion. In fact, the exclusion of land domains when all points of the internal domain are zeros is not taking into account for the need to preserve the ocean points that lies in the overlapping region of these domains. These point are necessary to maintain the coherence of the model between the full domain decomposition and the reduced one without land, since they enter in the definition, e.g., boundary conditions through the MPI data exchange.

I made the following change in the mppini_2.h subroutine to modify the exclusion criteria, which now consider a domain as land when all points within the region (inner+overlap) are zero (I also modified the offline decomposition tool accordingly):

isurf = 0

• DO jj = 1+jprecj, ilj-jprecj
• DO ji = 1+jpreci, ili-jpreci

+ DO jj = 1, ilj + DO ji = 1, ili

IF( imask(ji+iimppt(ii,ij)-1, jj+ijmppt(ii,ij)-1) == 1) isurf = isurf+1

END DO

END DO

After these modifications, I rerun SETTE using the configuration 4 test always using 88 PEs, but this time the decomposition was 81 ocean + 7 land.

The reproducibility test passed.

If confirmed by other members of the ST, this issue applies to both 3.6 and the trunk ( and also to NEMO 3.4) I think it is also necessary to uniform the log information produced by both mppini and mppini2.

I attach to this issue a substantially revised version of the offline tool which provides a more synthetic output of domain decompositions structure and metrics.

Context

Code improvements and revisions required for FOAM assimilation system in GO6

Analysis

Initial attempts to increment ssh directly as a change to the surface eta resulted in the introduction of fresh-water blotches into the ocean, presumably due to increased volume added with no heat/content change.

Fix

Chris Harris has devised a scheme to implement the ssh change though out the water column through changes to e3t. Associated changes to heat and salt content have also been added.