New URL for NEMO forge!   http://forge.nemo-ocean.eu

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
ticket/0846 – NEMO
wiki:ticket/0846

Version 14 (modified by momme, 12 years ago) (diff)

--

Last edited Timestamp?

Author : Marcello Vichi

ticket : #846

Branch : dev_r2784_CMCC1_topbfm


Description

Branch to develop a common interface for other biogeochemical models not included in the TOP directory. The idea is to provide an as much as possible seamless interface for more complex biogeochemical models that do not fit into the TOP MY_TRC framework. This will consider the development done by CMCC, MetOffice? (Karen Edwards) and NCOF-PML (Momme Butenschoen) integrating with the existing TOP interface.

Current state

Currently, BGC models are resolved within the SMS part (trcsms.F0) and that the transport (trc_trp) and output (trc_wri, trc_dia and trc_rst) is done identically for each model , assuming that all the memory layout is provided by NEMO through the tra and trn arrays and the number of passive tracers jptra. This is a good solution for models that are developed within NEMO but does not fit well with more complex biogeochemical models that are also used with other OGCMs. This is the case for the BFM, which has a wide growing community of users and therefore has an independent development with a specific coding structure. For models like the BFM (CMCC and INGV) and ERSEM (MetOffice? and NERC) the common solution (independently developed) was to add an alternative set of routines for stepping and output

This is an excerpt from a modified trcstp.F90 for the BFM case

...
#if defined key_bfm
   !---------------------------------------------
   ! Check the main BFM flag
   !---------------------------------------------

         CALL trc_bfm( kt )           ! main call to BFM

         CALL trc_trp_bfm( kt )       ! transport of BFM tracers

         CALL trc_dia_bfm( kt )       ! diagnostic output for BFM

#else

...

         tra(:,:,:,:) = 0.e0
         !
         IF( kt == nit000 .AND. lk_trdmld_trc  )  &
            &                      CALL trd_mld_trc_init        ! trends: Mixed-layer
                                   CALL trc_rst_opn( kt )       ! Open tracer restart file 
         IF( lk_iomput ) THEN  ;   CALL trc_wri( kt )           ! output of passive tracers
         ELSE                  ;   CALL trc_dia( kt )
         ENDIF
                                   CALL trc_sms( kt )           ! tracers: sink and source
                                   CALL trc_trp( kt )           ! transport of passive tracers
         IF( kt == nit000 )     CALL iom_close( numrtr )     ! close input  passive tracers restart file
         IF( lrst_trc )            CALL trc_rst_wri( kt )       ! write tracer restart file
         IF( lk_trdmld_trc  )      CALL trd_mld_trc( kt )       ! trends: Mixed-layer

Actually MetOffice? is currently using a hybrid approach mixing the standard SMS approach with a specific key.

Strategy

To accommodate this kind of models it is required to add flexibility to the code, like allowing the possibility to not use the output/restart facilities of NEMO and to have different combinations of transport schemes. The major issue here is the loop over the number of tracers, which is not possible within the my_trc framework and therefore requires another tracer stepping scheme. It is indeed possible to use trn(tra) with more complex models (Karen is doing this with ERSEM, while NERC/PML is not). However, this is not likely with the BFM as it allows to define the number of variables dynamically and it builds its own structure separately with specific input/output routines. Another issue is that it is important to keep the sedimentation rates separated from vertical advection as MPDATA schemes are not mass conserving. I think we can work out a common solution within the framework of my_trc, although I would propose to have an extended my_trc that allows more flexibility. Whether to achieve this through additional logical flags or by means of macros is a matter of discussion. I guess we all agree that it makes no sense to have a different key for each new model, though we also need to realize that we'll have to accommodate two different coupling strategies, one for SMS models and one for "ecosystem" models (which may also incorporate benthic systems like the ones we use now).

One (very) simple solution is to have only an additional key (like key_obgcm, other biogeochemical model) to accommodate any BGC model that does not use at all the NEMO memory structure and facilities. This would only involve two changes in the NEMO code, during the initialization step and during the time stepping

nemogcm.F90:
-------------
#if defined key_top
  #if defined key_obgcm
      !                                     ! Other external biogeochemical model
                            CALL     trc_init_obgc
  #else
      !                                     ! Passive tracers
                            CALL     trc_init
  #endif
#endif

step.F90:
---------
#if defined key_top
  #if defined key_obgcm
      !>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
      ! Passive Tracer Model
      !<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
                         CALL trc_stp_obgc( kstp )         ! time-stepping
#else
      !>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
      ! Other Biogeochemical Model
      !<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
                         CALL trc_stp( kstp )         ! time-stepping
  #endif
#endif

The routines trc_init_obgc and trc_stp_obgc are wrappers that have to be provided by the external model in its own directory structure and linked during compilation. We can also put an EXTERNAL statement in the code to clarify this.

The strategy is open for discussion because it is not urgent matter in the 2011 work plan, but it is important that we find a common way forward that will maximize all development efforts and the research carried on by the community of ecosystem modelling.

Alternatively to allow for switching between models at run-time without the need for recompilation, the follwoing approach can be taken:

par_trc.F90:

   IMPLICIT NONE
   PUBLIC

   ! Passive tracers : Total size
   ! ---------------               ! total number of passive tracers, of
2d and 3d output and trend arrays
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
# if defined key_PMLersem
   INTEGER, PUBLIC, PARAMETER :: jp_pmlersem = 56
   INTEGER, PUBLIC, PARAMETER ::   jptra    =  jp_pmlersem +
jp_lobster    + jp_pisces     + jp_cfc     + jp_c14b     + jp_my_trc
   INTEGER, PUBLIC, PARAMETER :: jp_pmlersem0=1, jp_pmlersem1=jp_pmlersem
# else
! /\
! /\
!===============================================
   INTEGER, PUBLIC, PARAMETER ::   jptra    =  jp_lobster    +
jp_pisces     + jp_cfc     + jp_c14b     + jp_my_trc
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
# endif
! /\
! /\
!===============================================
   INTEGER, PUBLIC, PARAMETER ::   jpdia2d  =  jp_lobster_2d +
jp_pisces_2d  + jp_cfc_2d  + jp_c14b_2d  + jp_my_trc_2d

trcini.F90:

      ENDIF
     
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
# if defined key_PMLersem
      CALL trc_ini_PMLersem       ! MY_TRC  tracers
# else
      WRITE(numout,*) '          ERSEM not used'
# endif
! /\
! /\
!===============================================

      IF( lk_my_trc  ) THEN   ;   CALL trc_ini_my_trc       ! MY_TRC  tracer

trc_oce.F90:

#if defined key_top && defined key_pisces
   !!----------------------------------------------------------------------
   !!   'key_top'   &   'key_pisces'                       PISCES
bio-model         
   !!----------------------------------------------------------------------
   LOGICAL, PUBLIC, PARAMETER ::   lk_qsr_bio = .TRUE.   !: bio-model
light absorption flag
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2008
! \/
! \/
#elif defined key_top && defined key_PMLersem
   !! light for NEMO-ERSEM
   LOGICAL, PUBLIC, PARAMETER ::   lk_qsr_bio = .TRUE.
! /\
! /\
!===============================================
#else
   !!----------------------------------------------------------------------
   !! Default option                          No bio-model light absorption

trcrad.F90:

      IF( lk_my_trc  )   CALL trc_rad_sms( kt, trb, trn, jp_myt0 ,
jp_myt1               )  ! MY_TRC model
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
# if defined key_PMLersem
      write(6,*) jp_pmlersem0,jp_pmlersem1
      CALL trc_rad_sms( kt, trb, trn, jp_pmlersem0 , jp_pmlersem1) 
# endif
! /\
! /\
!===============================================


trcsms.F90:

      IF( lk_my_trc  )   CALL trc_sms_my_trc ( kt )    ! MY_TRC  tracers

!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
# if defined key_PMLersem
      CALL trc_PMLersem ( kt )
# endif
! /\
! /\
!===============================================

trcstp.F90:

                             CALL trc_sms( kt )           ! tracers:
sink and source
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
#if defined key_PMLersem     
                             CALL trc_trp_PMLersem( kt )
#else
! /\
! /\
!===============================================
                             CALL trc_trp( kt )           ! transport of
passive tracers
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
#endif
! /\
! /\
!===============================================
      IF( kt == nittrc000 )  CALL iom_close( numrtr )     ! close input 
passive tracers restart file
      IF( lrst_trc )         CALL trc_rst_wri( kt )       ! write tracer
restart file
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
# if defined key_PMLersem
                             CALL trc_dia_PMLersem( kt )   ! diagnostics
# else
! /\
! /\
!===============================================
      IF( lk_iomput ) THEN
                             CALL trc_wri( kt )           ! output of
passive tracers
      ELSE
                             CALL trc_dia( kt )   ! diagnostics
      ENDIF
!===============================================
! Modification to NEMO trunk for ERSEM coupling
! M.Butenschoen, PML - March 2009
! \/
! \/
# endif
! /\
! /\
!===============================================
      IF( lk_trdmld_trc  )   CALL trd_mld_trc( kt )     ! trends:
Mixed-layer

========= Christian's comments =========

The NEMO architecture ( with the use of FCM to build dependancies ), allow to rewrite a routine without needing to change anything. In that case, we can use the MY_TRC framework to manage every type of biogeochemical model.

In fact, when compiling with makenemo ( for example : ./makenemo -n ORCA2_LIM_MY_TRC -r ORCA2_LIM_PISCES del_key "key_iomput key_pisces" add_key "key_my_trc"), a sub-directory where one can copy and modify all the routines having to be changed, is created : NEMOGCM/CONFIG/ORCA2_LIM_MY_TRC/MY_SRC. One can then copy the listed routines and modify them according to our convenience

  • TOP_SRC/trcini.F90
  • TOP_SRC/trcstp.F90
  • TOP_SRC/trcdia.F90
  • TOP_SRC/TRP/trctrp.F90
  • TOP_SRC/MY_TRC/*

To go a little further - this is another proposition - we can add a new namelist parameter ln_nemo_io which is true if we want to use NEMO I/O manager and false if not. So we can modify the 2 modules trcini.F90 and trcstp.F90 as follow :

trcini.F90

      IF( lk_lobster ) THEN   ;   CALL trc_ini_lobster      ! LOBSTER bio-model
      ELSE                    ;   IF(lwp) WRITE(numout,*) '          LOBSTER not used'
      ENDIF

      IF( lk_pisces  ) THEN   ;   CALL trc_ini_pisces       ! PISCES  bio-model
      ELSE                    ;   IF(lwp) WRITE(numout,*) '          PISCES not used'
      ENDIF

      IF( lk_cfc     ) THEN   ;   CALL trc_ini_cfc          ! CFC     tracers
      ELSE                    ;   IF(lwp) WRITE(numout,*) '          CFC not used'
      ENDIF

      IF( lk_c14b    ) THEN   ;   CALL trc_ini_c14b         ! C14 bomb  tracer
      ELSE                    ;   IF(lwp) WRITE(numout,*) '          C14 not used'
      ENDIF

      IF( lk_my_trc  ) THEN   ;   CALL trc_ini_my_trc       ! MY_TRC  tracers
      ELSE                    ;   IF(lwp) WRITE(numout,*) '          MY_TRC not used'
      ENDIF

      IF( ln_nemo_io ) THEN               ! Uses NEMO I/O and restart manager 
         !
         IF( ln_rsttr ) THEN
           !
           IF( lk_offline )  neuler = 1   ! Set time-step indicator at nit000 (leap-frog)
           CALL trc_rst_read              ! restart from a file
           !
         ELSE
           IF( lk_offline )  THEN
              neuler = 0                  ! Set time-step indicator at nit000 (euler)
              CALL day_init               ! set calendar
           ENDIF
#if defined key_dtatrc
              CALL trc_dta( nit000 )      ! Initialization of tracer from a file that may also be used for damping
              DO jn = 1, jptra
              IF( lutini(jn) )   trn(:,:,:,jn) = trdta(:,:,:,jn) * tmask(:,:,:)   ! initialisation from file if required
           END DO
#endif
           trb(:,:,:,:) = trn(:,:,:,:)
           ! 
         ENDIF
         !
      ENDIF

      tra(:,:,:,:) = 0._wp


trcstp.F90

      IF( MOD( kt - 1 , nn_dttrc ) == 0 ) THEN      ! only every nn_dttrc time step
         !
         IF(ln_ctl) THEN
            WRITE(charout,FMT="('kt =', I4,'  d/m/y =',I2,I2,I4)") kt, nday, nmonth, nyear
            CALL prt_ctl_trc_info(charout)
         ENDIF
         !
         tra(:,:,:,:) = 0.e0
         !
         IF( ln_nemo_io ) THEN                                  ! Uses NEMO I/O and restart manager 
            !
            IF( kt == nit000 .AND. lk_trdmld_trc  )  &
               &                    CALL trd_mld_trc_init        ! trends: Mixed-layer
                                    CALL trc_rst_opn( kt )       ! Open tracer restart file 
         ENDIF
         IF( lk_iomput ) THEN  ;    CALL trc_wri( kt )           ! output of passive tracers
         ELSE                  ;    CALL trc_dia( kt )
         ENDIF
                                    CALL trc_sms( kt )       ! tracers: sink and source
                                    CALL trc_trp( kt )       ! transport of passive tracers 

         IF( ln_nemo_io ) THEN                                  ! Uses NEMO I/O and restart manager 
            IF( kt == nit000 )      CALL iom_close( numrtr )     ! close input  passive tracers restart file
            IF( lrst_trc )          CALL trc_rst_wri( kt )       ! write tracer restart file
            IF( lk_trdmld_trc  )    CALL trd_mld_trc( kt )       ! trends: Mixed-layer
         ENDIF
         !
      ENDIF

and then, just copy & modify in MY_SRC directory the listed modules to manage transport and outputs

  • TOP_SRC/trcdia.F90
  • TOP_SRC/TRP/trctrp.F90
  • TOP_SRC/MY_TRC/*

========= Marcello's comments ========= Christian's comment is remarking the importance of FCM as "the tool" to incorporate other models into the system with little modification of the source code. I definitely agree with this point. The way we handle the coupling with the BFM now is to have a directory called nemo within the BFM tree that contains all needed routines. In our case, This directory is exactly what should go into MY_SRC. Extending on this method, the solution would be to totally substitute the main calling routines (trcstp and trctrp) with modified copies that will be maintained by each group in the tree of the biological model. In that case, I think we should be able to fit into the MY_TRC layout easily. As far as I can understand, FCM should consider the content opf MY_SRC directory in place of the files with the same names in the WORK directory. My only concern is whether FCM can eventually put the content of an external directory into MY_SRC, without the need to do it manually.

Momme's comments on FCM use and my_trc interface

How to compile external "trc" code with NEMO using FCM, modifying only the fcm file of the local configuration. (Leaving the repository code untouched.)

Example of fcm configuration file:

# NEMO keys:
 bld::tool::fppkeys key_trabbl key_vectopt_loop key_orca_r2 key_lim3 key_dynspg_flt key_diaeiv key_ldfslp key_traldf_c2d key_traldf_eiv key_dynldf_c3d key_dtatem key_dtasal key_tradmp key_zdftke key_zdfddm key_zdftmx key_iomput key_nproci=1 key_nprocj=1 key_top key_PMLersem
#NEMO-ERSEM and ERSEM keys:
 bld::tool::fppkeys %bld::tool::fppkeys PPNCDFERSEM PPNCDFERSEM IRON MASSTRACER
#include FCM configuration with source locations: 
 inc /common/work/NEMO/NemoErsem/nemoersem.fcm

nemoersem.fcm:

#Source locations for NemoErsem and ERSEM code:
 src::pmlersem::nemoersem       /common/work/NEMO/NemoErsem/NEMOv3.3
 src::pmlersem::ersem       /common/work/NEMO/ERSEM/trunk
#Preprocess NemoErsem and ERSEM code:
 bld::pp::pmlersem     1   

However, to use the my_tracer facility with ERSEM, I still had to introduce some changes to the src trunk, that at the current stage are unavoidable to me.

Modifications to NEMO source tree: I still can't see a feasable solution without introducing a key_PMLersem key to the NEMO source, essentially for three reason:

1) lk_qsr_bio is active only with key_pisces switched on, so if I want bioshading, I have to use PISCES. Not good.

2) I need to map the ERSEM rates on trn before transport/diffusion and I need to map the NEMO states back to the ERSEM after transport/diffusion, so I need a wrapper around the trctrp call. I can of course overwrite trctrp with a copy that simply adds the mapping at the beginning end and the end, but then I have to introduce changes in the trunk to trctrp manually to my copy, that's not really what version control is about, I prefer to add a key that calls my wrapper instead which maps and calls the original trctrp.

3) I don't use the NEMO output for the ERSEM variables, so I need to swap the trcdia. Again I could deal with this by overwritting, but then again NEMO calls sometimes trc_dia sometimes trc_wri, so I'd have to use two names for an identical routine, that calls the ERSEM internal output, again the PMLERSEM key is the cleaner option.

All this is based on the NEMO src of the last tag (3.3), I haven't checked back against the trunk.

lk_qsr_bio -> TRUE requires key_pisces, should be switchable in relation to user requirements for my_tracer:
!=============================================== 
! Modification to NEMO trunk for ERSEM coupling 
! M.Butenschoen, PML - March 2008 
! \/ 
! \/ 
#if defined key_top
! /\ 
! /\ 
!=============================================== 

mapping of ERSEM states to trn:

!=============================================== ! Modification to NEMO trunk for ERSEM coupling ! M.Butenschoen, PML - March 2009 ! \/ ! \/ #if defined key_PMLersem

  CALL trc_trp_PMLersem( kt )

#else ! /\ ! /\ !===============================================

  CALL trc_trp( kt )           ! transport of passive tracers

!=============================================== ! Modification to NEMO trunk for ERSEM coupling ! M.Butenschoen, PML - March 2009 ! \/ ! \/ #endif ! /\ ! /\ !=============================================== }}} avoid NEMO output of trc: {{{!=============================================== ! Modification to NEMO trunk for ERSEM coupling ! M.Butenschoen, PML - March 2009 ! \/ ! \/ #if defined key_PMLersem

  CALL trc_dia_PMLErsem( kt )

#else

  IF( lk_iomput ) THEN  ;   CALL trc_wri( kt )           ! output of passive tracers ELSE                  ;   CALL trc_dia( kt ) ENDIF

!=============================================== ! Modification to NEMO trunk for ERSEM coupling ! M.Butenschoen, PML - March 2009

  \/

! \/ #endif

Side Note: don't know if that's already flagged somewhere: I couldn't compile the original test case ORCA2_LIM using the last NEMO release, tag 3.3, it has key_lim2 defined, but limrhg is needed by limdyn_2, but only available in SRC_LIM3!

Status

Outcomes of the videoconference held on September 17th. Participants: Marcello Vichi, Christian Ethe', Karen Edwards, Momme Butenschoen. Christian's proposal of using FCM is seen as the way forward to minimize the impacts on the current NEMO code and allow all the developers of external ecosystem models to carry on their own development still maintaining the possibility to couple with NEMO in a seamless way. A powerpoint presentation is attached to this page, explaining the status and the possible solution. The underlying idea is to explore the method used at MetOffice? where different components are first extracted (using FCM extract.cfg configuration script) and then built. Currently, FCM is used within NEMO just to perform the build phase (build.cfg) as the extraction is not needed being the code part of the downloaded NEMO version. A possible strategy to explore is to include an additional flag for the makenemo script that should look for a user-provided build.cfg file with all the information on the external biogeochemical model. This model code will then be copied at the same level of the WORK directory. The proper versions of trcini.F90 and trcstp.F0 will instead be put in the MY_SRC directory. All files present in MY_SRC directory are compiled in place of the ones in the WORK directory. The current development branch will thus be used as a testing ground for the usage of FCM as interface to . It is not expected that this development will be part of the next release, but we believe this strategy will facilitate any future inclusion of other external models.


Testing


System Changes


Resources


IPR issues

Has the code been wholly (100%) produced by NEMO developers staff working exclusively on NEMO? ''' YES '''

Attachments (1)

Download all attachments as: .zip