Changes between Version 1 and Version 2 of Working Groups/HPC/Mins_2015_11_05

2016-03-08T18:46:02+01:00 (5 years ago)


  • Working Groups/HPC/Mins_2015_11_05

    v1 v2  
    1 = Minutes = 
     1= Minutes Notes from NEMO HPC working group meeting: 5 Nov 2015 = 
     3Attending: Mike Bell, Miguel Castrillo, Silvia Mocavero, Tim Graham, Julien le Sommer, Oriol Mula-Valls, Martin Schreiber,  Kim Serradell 
     5Appologies Mondher Chekki, Claire Levy, Marie-Alice Foujols.  
     81.      Agree who will own actions identified in Sebastien Masson’s task list (and Gurvan’s additional action(s) from last meeting) 
     9Silvia (CMCC) is willing to take on task 1 (suppress global communications). She will do it in two steps: a) an analysis leading to a proposed approach b) implementation of approach agreed with Systems Team.  
     11Miguel’s team (BSC) is willing to do the part of task 2 extending the mpp_lnk functionality to a 4D array. Its implementation throughout the code would need to be done by another team.  
     13Tim (Met Office) is willing to do task 3   (suppress useless communication).  
     15No one volunteered for tasks 4 and 5 – larger halos. These were agreed to be important.  
     17Action: Mike and Julien to look for suitable people to take on tasks 4 and 5, the second half of task 2, and the task proposed by Gurvan in previous meeting. (item 5 below is relevant to this)   
     192.      Discuss which benchmark configurations to use for future tests  (see notes from previous meeting)  
     20Standard benchmark configurations would enable comparison of performance on different machines by the group members. The system team receives requests for such configurations by external users; benchmark configurations also need to be given to vendors. So there are good motivations for shared benchmark configurations. The costs of setting them up were thought to be modest and NEMO version 3.6 could be a good choice of version initially. However a large number of configurations could be of interest: ORCA025 without sea-ice, ORCA025 with LIM3; ORCA12; GYRE with resolution equivalent to ORCA12. We probably want to choose only 2 configurations to get started.  Miguel will set up a GoogleDrive page where each group can say which configurations they would prefer to use as benchmarks. A decision will be made at the next meeting.    
     22Action Miguel to set up GoogleDrive page. Sylvia, Miguel, Tim to fill in their views.  
     243.      Discussion of existing evidence about bottlenecks  (Silvia’s roofline analysis) and present performance on a single node 
     25Roofline analysis is a useful first step in analysing the code efficiency within a single node. Figures 6 and 7 of Silvia’s paper suggest that the efficiency of advection within NEMO on a single processor is poor and could be significantly improved. Evidence of performance of other subroutines, configurations and machines would be valuable.  
     27It would be useful to exchange information on how roofline diagrams are generated – to help ensure we are all using comparable and reliable methods  
     29Action : Tim and Sylvia to discuss what specification of this would be useful and then iterate with Martin, Oriol and Miguel.  
     324.      Discuss how to develop a longer term strategy  
     33We are starting to implement a short-term plan to improve MPI communications between nodes. The benchmark configuration and single node bottle-neck analysis is a short-term plan to improve understanding of issues and the performance on a single node.  
     35Julien described two actions within an AGRIF consolidiation plan which could have a major impact on performance. In particular AGRIF could enable the sea-ice or biogeochemistry components of the model to run on a coarser grid than the dynamics.    
     37Psyclone has potential as a pre-processing tool to enable NEMO to work efficiently on a wide range of HPCs. The G-Ocean project explored its application to a 2D shallow-water version of  NEMO. Psyclone would be applied to NEMO using metadata which identify the field-type of each of the arrays (e.g. ua as a field defined at u-velocity gridpoints). The  main NEMO subroutines would be converted into “kernels” essentially by stripping out the i,j,k loops. Psyclone applies a series of user-directed transformations to the code (such as MPI domain decomposition, OpenMP directives, etc) for its implementation on a particular HPC. Psyclone can be implemented in stages, one subroutine at a time, and allows the vanilla code (present version) to be re-built. It was agreed that a presentation describing the Psyclone concepts and how it could be applied to NEMO would be useful. The presentation should include discussion of the steps to implementation, the effort involved and the longer term maintenance issues.  
     39Action: Mike to arrange a suitable presentation on Psyclone at the next meeting.  
     41AGRIF and Psyclone are both pre-processors and likely to conflict with each other 
     43Martin has started a document describing the targets for HPC performance, the present performance and options for improvement. Martin, Sylvia, Tim and Mike intend to tidy this up a bit next week.  
     45Action: Martin to circulate his document before 15 November.  
     485.      Copernicus call within H2020 (Julien)  
     49Julien described the outline proposal he has developed with Mike and Paolo Oddo to respond to the Horizon 2020 Space  2016-2017 (see item 5iii in  
     51 ) 
     53Julien has submitted his document to the NEMO steering committee. It proposes that about 2 FTE effort for 3 years is devoted to HPC work. The work will need to be done in the period 2017-2019 and have an impact on the CMEMS service before end of 2020.  The deadline for responses to the call is 3 March 2016. 
     55The document is not specific about what tasks will be done. We should discuss ideas for tasks at our next meeting     
     57Action: Julien to circulate document to group members (done)  
     596.      Next meeting 
     60We should discuss benchmark configurations, the strategy document, Psyclone and ideas for the Copernicus call.  
     62The last week of November or the first week of December would be good timing. 30 Nov – 2 Dec are not suitable dates.  
     64Action: Mike to set up a doodle call.