New URL for NEMO forge!   http://forge.nemo-ocean.eu

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
#2724 (rewrite northpole folding in the nogather case) – NEMO

Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#2724 closed Defect (fixed)

rewrite northpole folding in the nogather case

Reported by: smasson Owned by: systeam
Priority: low Milestone:
Component: LBC Version: trunk
Severity: minor Keywords:
Cc:

Description

Context

There is a list of points which worries me about the nogather folding:

  • The code is quite complicated to understand...
  • We are sending halos values (which is usually not what should be done in MPI exchanges!) -> We assume that the east-west communications were done before applying the folding
  • T, U, V and F grids have the same neighbours, which is strange as the communication patterns are not the same for these grids (there are potential shifts of +/-1). I think we have no error as we send the halos.
  • I am not sure if it is really working when nn_hls > 1
  • There is a duplication of code between lbc_nfd_nogather_generic.h90 and lbc_nfd_generic.h90


Analysis

  • lbc_nfd_generic.h90 was already cleaned to make it compatible with nn_hls > 1 and to make sure we don't use halos values. It can be called independently of the east-west periodicity. We should take advantage of this!
  • The idea is to use the "gather" folding during mppini to know which neighbour is sending which data and therefore build a mapping of the communications associated with the folding. This will allow us to skip lbc_nfd_nogather_generic. This strategy is possible as the folding is symmetrical: sending data to neighbour N <-> receiving data from neighbour N.

Beware as the folding is also filling the e-w halos, the number of point exchanged between 2 neighbourgs can be different.

Recommendation

  • Limite the size and the number of allocated buffers
  • Use mpiwaitall and mpi_Irecv.
  • We could also code a "collective neighbourhood" version for the folding.
  • change the name of ln_nnogather

Commit History (2)

ChangesetAuthorTimeChangeLog
15270smasson2021-09-17T16:27:55+02:00

trunk: forget some cleaning (remove dom_glo), #2724

15267smasson2021-09-17T11:04:34+02:00

trunk: new nogather nolding, #2724

Change History (5)

comment:1 Changed 3 years ago by smasson

In 15267:

Error: Failed to load processor CommitTicketReference
No macro or processor named 'CommitTicketReference' found

comment:2 Changed 3 years ago by smasson

  • Resolution set to fixed
  • Status changed from new to closed

[15267] passes all sette tests with nn_hls = 1 or 2 (./sette.sh -A -C -t -F -c -v ACFt and ./sette.sh -A -C -t -e -F -c -v ACeFt ) and gives the same results as [15266]

comment:3 Changed 3 years ago by acc

Nice one Seb. I can confirm it passes and matches on my system too. Just to be clear: you didn't change the name of ln_nnogather after all?

comment:4 Changed 3 years ago by smasson

no I did not change it (yet...).
I was a little bit afraid of the consequences (-> all the changes to do in sette)

comment:5 Changed 3 years ago by smasson

In 15270:

Error: Failed to load processor CommitTicketReference
No macro or processor named 'CommitTicketReference' found
Note: See TracTickets for help on using tickets.