#871 closed Enhancement (fixed)
Add faster rebuild to NEMO/TOOLS respository
| Reported by: | edblockley | Owned by: | rblod |
|---|---|---|---|
| Priority: | low | Milestone: | Unscheduled |
| Component: | OCE | Version: | v3.3 |
| Severity: | Keywords: | Miscellaneous OPA Tools rebuild v3.3 | |
| Cc: |
Description
At the Met Office we have created a new executable for rebuilding NEMO diagnostic and restart files.
This code goes considerably faster than the existing IOIPSL code at the expense of a larger memory requirement. The code is further optimised using OMP compiler directives.
This rebuild code can rebuild an orca2 physics restart file decomposed over on 30 processors in under 4 seconds.
Commit History (6)
| Changeset | Author | Time | ChangeLog |
|---|---|---|---|
| 3025 | edblockley | 2011-10-28T17:00:02+02:00 | 6th commit for rebuild branch; Rewirkign the code to allow time chunk splitting for 4D variables giving the user the option to save memory at the expense of run time. see ticket:#871 |
| 3019 | edblockley | 2011-10-28T10:55:06+02:00 | 5th commit for rebuild branch; Fixing a small bug in the rebuild to ensure that the rebuild dimension no.s are properly set when the names are specified in the namelist. Also changing indimlens to SHARED rather than PRIVATE for OMP loops to satisfy certain compilers. see ticket:#871 |
| 2898 | edblockley | 2011-10-07T17:02:16+02:00 | 4th commit for rebuild branch; Moving rebuild_nemo code from TOOLS/REBUILD to TOOLS/REBUILD_NEMO (NB. this could be renamed REBUILD at a later date if that directory were to be removed) . see ticket:#871 |
| 2897 | edblockley | 2011-10-07T17:00:54+02:00 | 3rd commit for rebuild branch; Adding arch-xlf_aix.fcm for xlf compiler, arch-xlf_pwr6.fcm for xlf compiler with IBM Power6 extra options as well as arch-ifort_linux_omp.fcm for ifort compiler with omp enabled. see ticket:#871 |
| 2896 | edblockley | 2011-10-07T16:41:12+02:00 | 2nd commit for rebuild branch; Renbaming arch-xlf_aix.fcm as arch-mpxlf_aix.fcm because it actually uses the mpxlf compiler (for MPI). see ticket:#871 |
| 2895 | edblockley | 2011-10-07T16:36:41+02:00 | 1st commit for rebuild branch including the rebuild_nemo.f90 code and accompanying shell script. see ticket:#871 |
Change History (11)
comment:1 Changed 11 years ago by edblockley
- Status changed from new to assigned
comment:2 Changed 11 years ago by edblockley
comment:3 Changed 11 years ago by edblockley
The rebuild_nemo code has now been added to the relevant branch.
initially this has been put in https://forge.ipsl.jussieu.fr/nemo/browser/branches/2011/dev_r2802_UKMET3_rebuild/NEMOGCM/TOOLS/REBUILD_NEMO
However this directory could be renamed REBUILD if the plan was to remove the IOIPSL code?
I have tried to add some more compiler settings to https://forge.ipsl.jussieu.fr/nemo/browser/branches/2011/dev_r2802_UKMET3_rebuild/NEMOGCM/ARCH which includes renaming the file arch-xlf_aix.fcm to arch-mpxlf_aix.fcm (as it actually uses the mpxlf90_r compiler) and uploading a new version of arch-xlf_aix.fcm which truly uses xlf (as is needed for building this non-MPI tool).
comment:4 in reply to: ↑ description Changed 11 years ago by edblockley
A wiki page for this ticket has been created at https://forge.ipsl.jussieu.fr/nemo/wiki/ticket/871
comment:5 Changed 11 years ago by edblockley
- Owner changed from edblockley to rblod
- Status changed from assigned to new
Rachid, please could you review these changes and let me know your thoughts.
The code has been thoroughly tested here at the Met Office and we run many, many instances of it each day in our operational suite. Further to this I have tested it on various other diagnostic and restart files and all tests have proved positive.
One thing I would say is that I am unsure about the changes in ARCH and whether or not anything more is required to get this tool built on other architectures.
comment:6 Changed 11 years ago by edblockley
The code has been modified to allow the user to trade a longer run-time for lower memory requirements.
This is done by using a 'time-chunking' functionality which splits 4D variables up into chunks over the record (4th) dimension (usually time).
comment:7 Changed 10 years ago by edblockley
- Resolution set to fixed
- Status changed from new to closed
comment:8 Changed 5 years ago by nemo
- Keywords Tools added; tools removed
comment:9 Changed 5 years ago by nemo
- Keywords Misc. added
comment:10 Changed 5 years ago by nemo
- Keywords Misc. removed
comment:11 Changed 14 months ago by nemo
- Keywords Miscellaneous OPA v3.3 added
Branch https://forge.ipsl.jussieu.fr/nemo/browser/branches/2011/dev_r2802_UKMET3_rebuild created at changeset:2894