Opened 20 months ago

Last modified 6 weeks ago

#2011 assigned Task


Reported by: mocavero Owned by: mocavero
Priority: high Milestone: 2019 WP
Component: OCE Version: trunk
Severity: minor Keywords:
Cc: Review:
MP ready?:
Progress: Some tests on a significative kernel have been executed (using a mini-app approach). The integration of the neighbourhood collective communications in the trunk is ongoing. The branch will be ready for merge party end 2019

Description (last modified by francesca)


MPI-3 provides new neighbourhood collective operations (i.e. MPI_Neighbor_allgather and MPI_Neighbor_alltoall) that allow to perform halo exchange with a single MPI communication call when a 5-point stencil is used.

Collective communications will be tested on the NEMO code in order to evaluate the code performance compared with the traditional point-to-point halo exchange currently implemented in NEMO.

The replacement of point-to-point communication with new collective ones will be designed and implemented taking care of the results accuracy.

Implementation plan

The work, started in 2018, is described in the following:

Step 1: extraction of a mini-app to be used as test case. The advection kernel has been considered as test case and a mini-app has been implemented. The parallel application performs the MUSCL advection scheme and the dimension of the subdomain as well as the number of parallel processes can be set by the user (done)

Step 2: integration of the new MPI-3 collective communications in the mini-app and performance comparison with the standard MPI-2 point-to-point communications. The evaluation of the proof of concept will be performed by changing the subdomain size. Performance analysis will be executed on systems available at CMCC. However, tests on other systems (available at Consortium partners sites) are welcome (ongoing)

Step 3: the collective communications will be integrated in the NEMO code. The use of collective communications could be optional and the choice between point-to-point and collective communications will be demanded to the user (through a dedicated namelist parameter), also depending on the architecture where the code will run. The initialisation of the cartesian topology will be integrated in the mppini module, while the new version of lbc_lnk (perform a single MPI-3 collective call) will be added in the lib_mpp module. No changes are required in the NEMO routines where the lbc_lnk is called.

The proposed changes do not impact on NEMO usabilty. Reference manual will not be changed.

Commit History (1)


Create HPC-12 branch - ticket #2011

Change History (8)

comment:1 Changed 20 months ago by mocavero

  • Owner set to mocavero
  • Status changed from new to assigned

comment:2 Changed 12 months ago by francesca

  • Description modified (diff)
  • Milestone changed from 2018 WP to 2019 WP
  • Owner changed from mocavero to francesca
  • Progress modified (diff)

comment:3 Changed 12 months ago by nicolasmartin

  • Summary changed from HPC-04_Mocavero_mpi3 to HPC-04(2018WP)_Mocavero_mpi3

comment:4 Changed 9 months ago by nemo

  • Priority changed from low to high

comment:5 Changed 5 months ago by francesca

  • Description modified (diff)

comment:6 Changed 5 months ago by francesca

  • Owner changed from francesca to mocavero

comment:7 Changed 6 weeks ago by mocavero

In 11496:

Create HPC-12 branch - ticket #2011

comment:8 Changed 6 weeks ago by francesca

  • Progress modified (diff)
Note: See TracTickets for help on using tickets.