New URL for NEMO forge!   http://forge.nemo-ocean.eu

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
#1912 (parallel running of rebuild_nemo) – NEMO

Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#1912 closed Enhancement (wontfix)

parallel running of rebuild_nemo

Reported by: clivefujitsu Owned by: nemo
Priority: low Milestone:
Component: OCE Version: trunk
Severity: Keywords:
Cc:

Description

The system is currently designed to run rebuild_nemo in serial, one file at a time on a single process. This is fine if one only has a few files to combine but we have a requirement to create restart files at frequent intervals and the time it takes to combine them all can be longer than it takes to run the model.

So, I have developed a method whereby I can initiate a rebuild_nemo simultaneously across all the processes that are available in a job thus effectively combining files in parallel. However, the problem I encountered was that each occurrence of rebuild_nemo creates a namelist file and this resulted in conflicts between processes.

To solve this I changed rebuild_nemo so that the namelist file created by each instance of rebuild_nemo is a unique name (or at least unique within the job).

I can send my revised rebuild_nemo source code and script to someone in the systems team to incorporate it into the trunk if desired.

Cheers
Clive

Commit History (0)

(No commits)

Change History (2)

comment:1 Changed 7 years ago by clevy

  • Resolution set to wontfix
  • Status changed from new to closed

Using XIOS2 in one file mode now usuallly bypasses the need for rebuild_nemo.
For any other cae, this message should rather be added in the NEMO forum on the trac wiki pages.
I'll close this ticket for now

comment:2 Changed 7 years ago by timgraham

I'm not sure I would agree that rebuild_nemo isn't needed anymore: it is needed frequently for rebuilding restart files so that the domain decomposition can be changed (e.g. if data assimilation steps are run on different domain decompositions to the forecast run).

Having said that I think this ticket is a duplicate of #1848 which also includes several other modifications to rebuild_nemo in a branch which could be committed to the trunk if the rest of the systems team agree to this.

Note: See TracTickets for help on using tickets.