Version 9 (modified by smueller, 13 months ago) (diff)

ENHANCE-05_SimonM-Harmonic_Analysis

Last edition: 12/09/19 19:18:03 by smueller

The PI is responsible to closely follow the progress of the action, and especially to contact NEMO project manager if the delay on preview (or review) are longer than the 2 weeks expected.

  1. Summary
  2. Preview
  3. Tests
  4. Review

Summary

The harmonic-analysis diagnostics available in the current reference code is limited to two-dimensional fields (surface only), is activated via a preprocessor key, uses unconventional namelist parameter names, uses a mixture of dynamic and static allocation for large arrays, and appears to be computationally inefficient. Further, while being based on multiple linear regression, the current implementation does not provide for regressions on harmonic components other than tidal constituents.

This action will replace the current tidal harmonic-analysis diagnostics with a generic implementation for multiple linear regression analysis that can be utilised for both tidal harmonic and non-tidal regression analyses. This implementation will provide harmonic-analysis diagnostics enhancements previously tested in a pre-4.0beta NEMO version by N. Bruneau: the analysis of three-dimensional fields, analysis across model restarts, and improved computational efficiency.

In contrast to both the existing harmonic analysis diagnostics in the reference NEMO code and the enhanced pre-4.0beta version by N. Bruneau, the new implementation will make extensive use of XIOS and an off-line tool. This approach should make it possible to simplify the regression analysis-related Fortran module in the core NEMO code, to relocate the regression-analysis configuration to XIOS configuration files, and to enable the selection of any model field handled by XIOS for analysis.

See ticket #2175.

Preview

Current implementation of tidal harmonic-analysis diagnostics and available enhancements

Tidal harmonic analysis of model fields based on multiple linear least-squares regression is available in the current trunk version of NEMO in module diaharm (source:/NEMO/trunk/src/OCE/DIA/diaharm.F90@10835). It is restricted to the analysis of sea-surface height and barotropic velocity fields, to a hard-coded maximum number of time slices within uninterrupted model runs, and its memory footprint during the whole analysis period can be very large. The current implementation has aspects that are deprecated (use of a pre-processor key, non-conventional namelist variable names, and statical allocation of potentially oversized arrays) and appears to be computationally inefficient. Further, it lacks sought-after features, such as the analysis of a wider range of model fields (including three-dimensional fields) and the possibility to span the analysis time period across restarted model runs, which have become available in more efficient alternative implementations of the same analysis method by N. Bruneau (pers. comm.) and E. O'Dea (https://forge.ipsl.jussieu.fr/nemo/browser/branches/UKMO/dev_5518_tid_analysis_restart) in previous NEMO versions. In addition, the harmonic analysis diagnostics of module diaharm remains deactivated, and thus its compilability untested, during the standard tests of the SETTE test suite.

Proposed replacement of the tidal harmonic-analysis diagnostics by a generic formulation for multiple linear regression analysis

Rather than adapting and upgrading an existing implementation of harmonic-analysis diagnostics (the current implementation in the trunk version or an existing alternative implementation) for the latest reference version of NEMO, the development of generic multiple linear least-squares regression analysis diagnostics for NEMO as outlined below in the form of a new module diamlr (source:/NEMO/trunk/src/OCE/DIA/diamlr.F90), additional XIOS configuration, and an off-line tool is proposed. This new diagnostics would readily allow for tidal harmonic analysis, and module diaharm (source:/NEMO/trunk/src/OCE/DIA/diaharm.F90) could be removed. In addition to tidal harmonic analysis the new development would facilitate non-tidal applications, such as the identification of the seasonal cycle or linear trends in model fields.

The use of XIOS for regression diagnostics

In general, least-squares linear regression can be formulated in terms of scalar products of all possible pairings between the dependent variable, |y>, and the regressors, |xm> (the index identifies the regressor), where each vector component represents the corresponding value at each of the time steps included in the analysis. In particular, during the model run, it suffices to accumulate the scalar products <xm|y> and <xm|xn> by summing up the respective products for each time step; at the end of the analysis interval, which does not need to be known in advance, the regression analysis can be finalised using pre-computed scalar products.

The computation of the regressors formulated as functions of time, the computation of the scalar products, and the output of the scalar products in model runs with enabled linear regression analysis can be delegated to the I/O server XIOS. This would have three main benefits:

  • as the fields of dependent variables selected for regression analysis are typically already available in XIOS processes for regular model output, the NEMO processes would no longer have to access the large fields selected for analysis directly;
  • the potentially large additional amounts of temporary storage required for the computation of the scalar products would be provided by XIOS processes, and so the memory footprint of the NEMO processes would hardly be affected when enabling linear regression analysis; and
  • output files of partial scalar products for partial model runs can readily be generated by XIOS and re-combined later (through addition) in order to prepare sets of scalar products for various analysis intervals, including analysis intervals that span across model restarts.

Tests

Once the development is done, the PI should complete the tests section below and after ask the reviewers to start their review.

This part should contain the detailed results of SETTE tests (restartability and reproducibility for each of the reference configuration) and detailed results of restartability and reproducibility when the option is activated on specified configurations used for this test

Regular checks:

  • Can this change be shown to produce expected impact (option activated)?
  • Can this change be shown to have a null impact (option not activated)?
  • Results of the required bit comparability tests been run: are there no differences when activating the development?
  • If some differences appear, is reason for the change valid/understood?
  • If some differences appear, is the impact as expected on model configurations?
  • Is this change expected to preserve all diagnostics?
  • If no, is reason for the change valid/understood?
  • Are there significant changes in run time/memory?

Review

A successful review is needed to schedule the merge of this development into the future NEMO release during next Merge Party (usually in November).

Assessments:

  • Is the proposed methodology now implemented?
  • Are the code changes in agreement with the flowchart defined at preview step?
  • Are the code changes in agreement with list of routines and variables as proposed at preview step?
    If, not, are the discrepancies acceptable?
  • Is the in-line documentation accurate and sufficient?
  • Do the code changes comply with NEMO coding standards?
  • Is the development documented with sufficient details for others to understand the impact of the change?
  • Is the project literature (manual, guide, web, …) now updated or completed following the proposed summary in preview section?

Finding:

Is the review fully successful? If not, please indicate what is still missing


Once review is successful, the development must be scheduled for merge during next Merge Party Meeting.