| Version 9 (modified by acc, 4 years ago) (diff) |
|---|
ENHANCE-04_AndrewC-reporting
Last edition: Wikinfo(changed_ts)? by Wikinfo(changed_by)?
The PI is responsible to closely follow the progress of the action, and especially to contact NEMO project manager if the delay on preview (or review) are longer than the 2 weeks expected.
- Summary
- Preview
- Sebastien's comments on the first draft, many of which have already been incorporated into version2 above:
- Tests
- Review
Summary
Investigate ways of improving the code's reporting facilities. Currently, errors away from the lead process are not fully reported and the ln_ctl mechanism produces an overwhelming volume of output for large processor counts. Options are needed to produce more selective output (either by output type or processor range). This is an un-started task from the 2018WP (formerly ROBUST-06_AndrewC-reporting) that has been carried forward to 2019. #2167
Preview
(version 3 following initial preview by Sebastien ( see his comments below) and tidying up of the existing nn_verbose_level settings to control the icebergs.stat files)
One of the features of NEMO 4.0 is the large reduction in global communications at the cost of suppressing some global diagnostics which tend to be useful only during SETTE testing, debugging or new configuration development. Such diagnostics can be re-activated using the ln_ctl namelist variable but this is a rather blunt instrument in that it activates all extra output for all processing elements. This list can include:
run.stat
run.stat.nc
tracer.stat
ocean.output_XXXX
layout.dat_XXXX
mpp.output_XXXX
mpp.top.output_XXX <---- correct this to 4-digits for consistency
EMPave.dat_XXXX <---- These are all identical, suppress writing to write-master only
icebergs.stat_XXXX <---- These should already be controlled by the nn_verbose_level variable; tidy up
and ensure silent running with nn_verbose_level=0
and possibly others depending on runtime options. One of the 2019 NEMO workplan entries (ENHANCE-04_AndrewC-reporting) plans to introduce more control over the choice of output created but it may be best to introduce the basic structure prior to the v4.0 release. Below is a summary of the overall plans and what can be implemented immediately if there is consensus.
At this stage, the plan is to leave ln_ctl as an all-or-nothing switch but to add the basic structure for finer control. For the v4.0 release, this new structure will only be capable of activating the global stats files (run.stat, run.stat.nc and tracer.stat) independently of ln_ctl but the placeholders will be there for extended control over other outputs. Minimal changes to achieve this will affect:
1. OCE/IOM/in_out_manager.F90 cfgs/SHARED/namelist_ref 2. OCE/nemogcm.F90 OFF/nemogcm.F90 SAO/nemogcm.F90 SAS/nemogcm.F90 3. OCE/stpctl.F90 4. TOP/prtctl_trc.F90 TOP/trcini.F90 TOP/trcstp.F90 5. OCE/LBC/mppini.F90 OCE/SBC/sbcfwb.F90 6. Tidy up icb modules so that the existing nn_verbose_level parameter can be used to turn off icebergs.stat files 7. sette/sette.sh
To reiterate, these changes allow run.stat and tracer.stat to be produced even when ln_ctl is .false.. They also introduce additional controls such as updating run.stat, tracer.stat and time.step at integer multiples of time step rather than every time step and for restricting the production of some types of output (e.g. layout.dat) to a subset of processing regions. There are good arguments for separating out the multiple uses that ln_ctl has grown to support (see Sebastien's comments on its history) but I propose these changes , as a quick, pre-release addition.
1. OCE/IOM/in_out_manager.F90 cfgs/SHARED/namelist_ref
These changes introduce a derived-type structure into the in_out_manager module and populates the reference namelist with default values.
There are 5 basic sets of parameters in this structure:
* l_config Activates use of the settings in the rest of the structure (specifically for when ln_ctl is false) * l_runstat, l_trcstat Activates production of global stats files. Only a single file of each of these is ever produced. * l_oceout, l_layout Normal operation is to produce a single version of each of these. If true then a version for each area is produced * l_mppout, l_mpptrc Suppressed if false, otherwise produce a version for each area. * procmin, procmax, procincr Allow subsetting of areas when producing output in the previous two categories. Default values will ensure all areas report. * ptimincr Timestep increment for outputting of time step status information less frequently (affects run.stat, tracer.stat and time.step only)
-
OCE/IOM/in_out_manager.F90
99 99 !! output monitoring 100 100 !!---------------------------------------------------------------------- 101 101 LOGICAL :: ln_ctl !: run control for debugging 102 TYPE :: sn_ctl !: optional use structure for finer control over output selection 103 LOGICAL :: l_config = .FALSE. !: activate/deactivate finer control 104 ! Note if l_config is True then ln_ctl is ignored. 105 ! Otherwise setting ln_ctl True is equivalent to setting 106 ! all the following logicals in this structure True 107 LOGICAL :: l_runstat = .FALSE. !: Produce/do not produce run.stat file (T/F) 108 LOGICAL :: l_trcstat = .FALSE. !: Produce/do not produce tracer.stat file (T/F) 109 LOGICAL :: l_oceout = .FALSE. !: Produce all ocean.outputs (T) or just one (F) 110 LOGICAL :: l_layout = .FALSE. !: Produce all layout.dat files (T) or just one (F) 111 LOGICAL :: l_mppout = .FALSE. !: Produce/do not produce mpp.output_XXXX files (T/F) 112 LOGICAL :: l_mpptop = .FALSE. !: Produce/do not produce mpp.top.output_XXXX files (T/F) 113 ! Optional subsetting of processor report files 114 ! Default settings of 0/1000000/1 should ensure all areas report. 115 ! Set to a more restrictive range to select specific areas 116 INTEGER :: procmin = 0 !: Minimum narea to output 117 INTEGER :: procmax = 1000000 !: Maximum narea to output 118 INTEGER :: procincr = 1 !: narea increment to output 119 INTEGER :: ptimincr = 1 !: timestep increment to output (time.step and run.stat) 120 END TYPE 121 TYPE (sn_ctl) :: sn_cfctl !: run control structure for selective output 102 122 LOGICAL :: ln_timing !: run control for timing 103 123 LOGICAL :: ln_diacfl !: flag whether to create CFL diagnostics 104 124 INTEGER :: nn_print !: level of print (0 no print) -
../cfgs/SHARED/namelist_ref
1303 1303 !----------------------------------------------------------------------- 1304 1304 &namctl ! Control prints (default: OFF) 1305 1305 !----------------------------------------------------------------------- 1306 ln_ctl = .false. ! trends control print (expensive!) 1306 ln_ctl = .FALSE. ! Toggle all report printing on/off (T/F); Ignored if sn_cfctl%l_config is T 1307 sn_cfctl%l_config = .TRUE. ! IF .true. then control which reports are written with the following 1308 sn_cfctl%l_runstat = .FALSE. ! switches and which areas produce reports with the proc integer settings. 1309 sn_cfctl%l_trcstat = .FALSE. ! The default settings for the proc integers should ensure 1310 sn_cfctl%l_oceout = .FALSE. ! that all areas report. 1311 sn_cfctl%l_layout = .FALSE. ! 1312 sn_cfctl%l_mppout = .FALSE. ! 1313 sn_cfctl%l_mpptop = .FALSE. ! 1314 sn_cfctl%procmin = 0 ! Minimum area number for reporting [default:0] 1315 sn_cfctl%procmax = 1000000 ! Maximum area number for reporting [default:1000000] 1316 sn_cfctl%procincr = 1 ! Increment for optional subsetting of areas [default:1] 1317 sn_cfctl%ptimincr = 1 ! Timestep increment for writing time step progress info 1307 1318 nn_print = 0 ! level of print (0 no extra print) 1308 1319 nn_ictls = 0 ! start i indice of control sum (use to compare mono versus 1309 1320 nn_ictle = 0 ! end i indice of control sum multi processor runs
2. OCE/nemogcm.F90, OFF/nemogcm.F90, SAO/nemogcm.F90, SAS/nemogcm.F90
The new structure is read and reported in nemogcm.F90. Changes for all variants are similar OCE/nemogcm.F90 is shown here:
-
OCE/nemogcm.F90
256 256 INTEGER :: ios, ilocal_comm ! local integers 257 257 CHARACTER(len=120), DIMENSION(60) :: cltxt, cltxt2, clnam 258 258 !! 259 NAMELIST/namctl/ ln_ctl , nn_print, nn_ictls, nn_ictle, &260 & nn_isplt , nn_jsplt, nn_jctls, nn_jctle, &259 NAMELIST/namctl/ ln_ctl , sn_cfctl, nn_print, nn_ictls, nn_ictle, & 260 & nn_isplt , nn_jsplt, nn_jctls, nn_jctle, & 261 261 & ln_timing, ln_diacfl 262 262 NAMELIST/namcfg/ ln_read_cfg, cn_domcfg, ln_closea, ln_write_cfg, cn_domcfg_out, ln_use_jattr 263 263 !!---------------------------------------------------------------------- … … 327 327 328 328 narea = narea + 1 ! mynode return the rank of proc (0 --> jpnij -1 ) 329 329 330 IF( sn_cfctl%l_config ) THEN 331 ! Activate finer control of report outputs 332 ! optionally switch off output from selected areas (note this only 333 ! applies to output which does not involve global communications) 334 IF( ( narea < sn_cfctl%procmin .OR. narea > sn_cfctl%procmax ) .OR. & 335 & ( MOD( narea - sn_cfctl%procmin, sn_cfctl%procincr ) /= 0 ) ) & 336 & CALL nemo_set_cfctl( sn_cfctl, .FALSE., .FALSE. ) 337 ELSE 338 ! Use ln_ctl to turn on or off all options. 339 CALL nemo_set_cfctl( sn_cfctl, ln_ctl, .TRUE. ) 340 ENDIF 341 330 342 lwm = (narea == 1) ! control of output namelists 331 343 lwp = (narea == 1) .OR. ln_ctl ! control of all listing output print 332 344 … … 503 515 WRITE(numout,*) '~~~~~~~~' 504 516 WRITE(numout,*) ' Namelist namctl' 505 517 WRITE(numout,*) ' run control (for debugging) ln_ctl = ', ln_ctl 518 WRITE(numout,*) ' finer control over o/p sn_cfctl%l_config = ', sn_cfctl%l_config 519 WRITE(numout,*) ' sn_cfctl%l_runstat = ', sn_cfctl%l_runstat 520 WRITE(numout,*) ' sn_cfctl%l_trcstat = ', sn_cfctl%l_trcstat 521 WRITE(numout,*) ' sn_cfctl%l_oceout = ', sn_cfctl%l_oceout 522 WRITE(numout,*) ' sn_cfctl%l_layout = ', sn_cfctl%l_layout 523 WRITE(numout,*) ' sn_cfctl%l_mppout = ', sn_cfctl%l_mppout 524 WRITE(numout,*) ' sn_cfctl%l_mpptop = ', sn_cfctl%l_mpptop 525 WRITE(numout,*) ' sn_cfctl%procmin = ', sn_cfctl%procmin 526 WRITE(numout,*) ' sn_cfctl%procmax = ', sn_cfctl%procmax 527 WRITE(numout,*) ' sn_cfctl%procincr = ', sn_cfctl%procincr 528 WRITE(numout,*) ' sn_cfctl%ptimincr = ', sn_cfctl%ptimincr 506 529 WRITE(numout,*) ' level of print nn_print = ', nn_print 507 530 WRITE(numout,*) ' Start i indice for SUM control nn_ictls = ', nn_ictls 508 531 WRITE(numout,*) ' End i indice for SUM control nn_ictle = ', nn_ictle … … 649 672 ! 650 673 END SUBROUTINE nemo_alloc 651 674 675 SUBROUTINE nemo_set_cfctl(sn_cfctl, setto, for_all ) 676 !!---------------------------------------------------------------------- 677 !! *** ROUTINE nemo_set_cfctl *** 678 !! 679 !! ** Purpose : Set elements of the output control structure to setto. 680 !! for_all should be .false. unless all areas are to be 681 !! treated identically. 682 !! 683 !! ** Method : Note this routine can be used to switch on/off some 684 !! types of output for selected areas but any output types 685 !! that involve global communications (e.g. mpp_max, glob_sum) 686 !! should be protected from selective switching by the 687 !! for_all argument 688 !!---------------------------------------------------------------------- 689 LOGICAL :: setto, for_all 690 TYPE (sn_ctl) :: sn_cfctl 691 !!---------------------------------------------------------------------- 692 IF( for_all ) THEN 693 sn_cfctl%l_runstat = setto 694 sn_cfctl%l_trcstat = setto 695 ENDIF 696 sn_cfctl%l_oceout = setto 697 sn_cfctl%l_layout = setto 698 sn_cfctl%l_mppout = setto 699 sn_cfctl%l_mpptop = setto 700 END SUBROUTINE nemo_set_cfctl 701 652 702 !!====================================================================== 653 703 END MODULE nemogcm
Eventually, setting sn_cfctl%l_config true will force ln_ctl to be false but the current implementation is incomplete and can only be used the activate run.stat, tracer.stat and multiple layout.dats independently of ln_ctl. ln_ctl, therefore, remains as the overriding control for all outputs.
3. OCE/stpctl.F90
The logic changes required in stpctl.F90 to control the production of run.stat are relatively simple. Three new local logicals are introduced to make the purpose of the logical constructs clearer. These determine whether or not to collect the global maximums (ll_colruns) , whether or not to handle the actual writing (ll_wrtruns) and whether or not to update the time.step file (ll_wrtstp).
-
OCE/stpctl.F90
66 66 INTEGER, DIMENSION(3) :: iu, is1, is2 ! min/max loc indices 67 67 REAL(wp) :: zzz ! local real 68 68 REAL(wp), DIMENSION(9) :: zmax 69 LOGICAL :: ll_wrtstp, ll_colruns, ll_wrtruns 69 70 CHARACTER(len=20) :: clname 70 71 !!---------------------------------------------------------------------- 71 72 ! 73 ll_wrtstp = ( MOD( kt, sn_cfctl%ptimincr ) == 0 ) .OR. ( kt == nitend ) 74 ll_colruns = ll_wrtstp .AND. ( ln_ctl .OR. sn_cfctl%l_runstat ) 75 ll_wrtruns = ll_colruns .AND. lwm 72 76 IF( kt == nit000 .AND. lwp ) THEN 73 77 WRITE(numout,*) 74 78 WRITE(numout,*) 'stp_ctl : time-stepping control' 75 79 WRITE(numout,*) '~~~~~~~' 76 80 ! ! open time.step file 77 81 IF( lwm ) CALL ctl_opn( numstp, 'time.step', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, lwp, narea ) 78 ! ! open run.stat file 79 IF( ln_ctl .AND. lwm ) THEN 82 ! ! open run.stat file(s) at start whatever 83 ! ! the value of sn_cfctl%ptimincr 84 IF( lwm .AND. ( ln_ctl .OR. sn_cfctl%l_runstat ) ) THEN 80 85 CALL ctl_opn( numrun, 'run.stat', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, lwp, narea ) 81 86 clname = 'run.stat.nc' 82 87 IF( .NOT. Agrif_Root() ) clname = TRIM(Agrif_CFixed())//"_"//TRIM(clname) … … 98 103 ENDIF 99 104 IF( kt == nit000 ) lsomeoce = COUNT( ssmask(:,:) == 1._wp ) > 0 100 105 ! 101 IF(lwm ) THEN!== current time step ==! ("time.step" file)106 IF(lwm .AND. ll_wrtstp) THEN !== current time step ==! ("time.step" file) 102 107 WRITE ( numstp, '(1x, i8)' ) kt 103 108 REWIND( numstp ) 104 109 ENDIF … … 120 125 zmax(9) = MAXVAL( Cu_adv(:,:,:) , mask = tmask(:,:,:) == 1._wp ) ! cell Courant no. max 121 126 ENDIF 122 127 ! 123 IF( l n_ctl) THEN128 IF( ll_colruns ) THEN 124 129 CALL mpp_max( "stpctl", zmax ) ! max over the global domain 125 130 nstop = NINT( zmax(7) ) ! nstop indicator sheared among all local domains 126 131 ENDIF 127 132 ! !== run statistics ==! ("run.stat" files) 128 IF( l n_ctl .AND. lwm) THEN133 IF( ll_wrtruns ) THEN 129 134 WRITE(numrun,9500) kt, zmax(1), zmax(2), -zmax(3), zmax(4) 130 135 istatus = NF90_PUT_VAR( idrun, idssh, (/ zmax(1)/), (/kt/), (/1/) ) 131 136 istatus = NF90_PUT_VAR( idrun, idu, (/ zmax(2)/), (/kt/), (/1/) )
4. TOP/prtctl_trc.F90 TOP/trcini.F90 TOP/trcstp.F90
Changes to control the production of tracer.stat follow similar lines with the introduction of a lltrcstat local logical. Note also changes to prtctl_trc.F90 to make mpp.top.output filenames compatible with other similar filenames (i.e. use I4.4 for area number). This one is a little off message because the control for this TOP output has been read in OCE/nemogcm.F90. I think this makes sense and moving it to the TOP name list just for the sake of keeping OCE and TOP fully independent seems unnecessary; TBD.
-
TOP/prtctl_trc.F90
209 209 IF( lk_mpp ) THEN 210 210 sind = narea 211 211 eind = narea 212 clb_name = "('mpp.top.output_',I 3.3)"212 clb_name = "('mpp.top.output_',I4.4)" 213 213 cl_run = 'MULTI processor run' 214 214 ! use indices for each area computed by mpp_init subroutine 215 215 nlditl(1:jpnij) = nldit(:) … … 228 228 ELSE 229 229 sind = 1 230 230 eind = ijsplt 231 clb_name = "('mono.top.output_',I 3.3)"231 clb_name = "('mono.top.output_',I4.4)" 232 232 cl_run = 'MONO processor run ' 233 233 ! compute indices for each area as done in mpp_init subroutine 234 234 CALL sub_dom -
TOP/trcini.F90
71 71 CALL trc_ini_trp ! passive tracers transport 72 72 CALL trc_ice_ini ! Tracers in sea ice 73 73 ! 74 IF(lwm) CALL ctl_opn( numstr, 'tracer.stat', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, lwp , narea ) 74 IF( lwm .AND. ( ln_ctl .OR. (sn_cfctl%l_config .AND. sn_cfctl%l_trcstat) ) ) THEN 75 CALL ctl_opn( numstr, 'tracer.stat', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, lwp , narea ) 76 ENDIF 75 77 ! 76 78 CALL trc_ini_state ! passive tracers initialisation : from a restart or from clim 77 79 IF( nn_dttrc /= 1 ) & -
TOP/trcstp.F90
56 56 ! 57 57 INTEGER :: jk, jn ! dummy loop indices 58 58 REAL(wp):: ztrai ! local scalar 59 LOGICAL :: ll_trcstat ! local logical 59 60 CHARACTER (len=25) :: charout ! 60 61 !!------------------------------------------------------------------- 61 62 ! … … 67 68 r2dttrc = 2. * rdttrc ! = 2 rdttrc (leapfrog) 68 69 ENDIF 69 70 ! 71 ll_trcstat = ( ln_ctl .OR. sn_cfctl%l_trcstat ) .AND. & 72 & ( ( MOD( kt, sn_cfctl%ptimincr ) == 0 ) .OR. ( kt == nitend ) ) 70 73 IF( kt == nittrc000 .AND. lk_trdmxl_trc ) CALL trd_mxl_trc_init ! trends: Mixed-layer 71 74 ! 72 75 IF( .NOT.ln_linssh ) THEN ! update ocean volume due to ssh temporal evolution … … 108 111 ! 109 112 ENDIF 110 113 ! 111 IF (l n_ctl) THEN114 IF (ll_trcstat) THEN 112 115 ztrai = 0._wp ! content of all tracers 113 116 DO jn = 1, jptra 114 117 ztrai = ztrai + glob_sum( 'trcstp', trn(:,:,:,jn) * cvol(:,:,:) )
5. OCE/LBC/mppini.F90
Control over the creation of multiple layout.dat files is easy to implement and so has been done as an example of how output in this category will be handled in future. For this category, the area subsetting can be used to restrict which areas produce files. The standard layout.dat file, produced by narea = 1 is always produced. Also, OCE/SBC/sbcfwb.F90 has been fixed so that only one EMPave.dat file is ever read or produced. Ultimately, only one processor should read this file and MPI_BCAST the values to all others as a more scalable solution (for name lists too?).
-
OCE/LBC/mppini.F90
150 150 INTEGER :: ierr, ios ! 151 151 INTEGER :: inbi, inbj, iimax, ijmax, icnt1, icnt2 152 152 LOGICAL :: llbest 153 LOGICAL :: llwrtlay 153 154 INTEGER, ALLOCATABLE, DIMENSION(:) :: iin, ii_nono, ii_noea ! 1D workspace 154 155 INTEGER, ALLOCATABLE, DIMENSION(:) :: ijn, ii_noso, ii_nowe ! - - 155 156 INTEGER, ALLOCATABLE, DIMENSION(:,:) :: iimppt, ilci, ibondi, ipproc ! 2D workspace … … 166 167 & ln_vol, nn_volctl, nn_rimwidth, nb_jpk_bdy 167 168 !!---------------------------------------------------------------------- 168 169 170 llwrtlay = lwp .OR. ln_ctl .OR. ( sn_cfctl%l_config .AND. sn_cfctl%l_layout ) 169 171 ! do we need to take into account bdy_msk? 170 172 REWIND( numnam_ref ) ! Namelist nambdy in reference namelist : BDY 171 173 READ ( numnam_ref, nambdy, IOSTAT = ios, ERR = 903) … … 553 555 END DO 554 556 555 557 ! Save processor layout in ascii file 556 IF (l wp) THEN558 IF (llwrtlay) THEN 557 559 CALL ctl_opn( inum, 'layout.dat', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, .FALSE., narea ) 558 560 WRITE(inum,'(a)') ' jpnij jpimax jpjmax jpk jpiglo jpjglo'//& 559 561 & ' ( local: narea jpi jpj )' … … 614 616 WRITE(numout,*) 615 617 WRITE(numout,*) ' ==>>> North fold boundary prepared for jpni >1' 616 618 ! additional prints in layout.dat 619 ENDIF 620 IF (llwrtlay) THEN 617 621 WRITE(inum,*) 618 622 WRITE(inum,*) 619 623 WRITE(inum,*) 'number of subdomains located along the north fold : ', ndim_rank_north … … 628 632 ! 629 633 IF( ln_nnogather ) THEN 630 634 CALL mpp_init_nfdcom ! northfold neighbour lists 631 IF (l wp) THEN635 IF (llwrtlay) THEN 632 636 WRITE(inum,*) 633 637 WRITE(inum,*) 634 638 WRITE(inum,*) 'north fold exchanges with explicit point-to-point messaging :' … … 639 643 ENDIF 640 644 ENDIF 641 645 ! 642 IF (l wp) CLOSE(inum)646 IF (llwrtlay) CLOSE(inum) 643 647 ! 644 648 DEALLOCATE(iin, ijn, ii_nono, ii_noea, ii_noso, ii_nowe, & 645 649 & iimppt, ijmppt, ibondi, ibondj, ipproc, ipolj, & -
OCE/SBC/sbcfwb.F90
143 143 qns(:,:) = qns(:,:) - zcoef * sst_m(:,:) * tmask(:,:,1) ! account for change to the heat budget due to fw correction 144 144 ENDIF 145 145 ! 146 IF( kt == nitend .AND. lw p ) THEN ! save fwfold value in a file146 IF( kt == nitend .AND. lwm ) THEN ! save fwfold value in a file (only one required) 147 147 CALL ctl_opn( inum, 'EMPave.dat', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, .FALSE., narea ) 148 148 WRITE( inum, "(24X,I8,2ES24.16)" ) nyear, a_fwb_b, a_fwb 149 149 CLOSE( inum )
6. Tidy up icb modules so that the existing nn_verbose_level parameter can be used to turn off icebergs.stat files
The icb modules already contain their own mechanism for controlling the verbosity of the icebergs.stat file reporting. This is done via the nn_verbose_level name list parameter which is set to a default value of 1. Output is minimal for nn_verbose_level=0 but is not switched off completely. Thus the icebergs.stat files are always created even when not required. This is particularly annoying at larger processor counts. Only a little work is required to ensure that nn_verbose_level=0 runs silently and does not create the unwanted files:
Changes required in: OCE/ICB/icbdia.F90 OCE/ICB/icbdyn.F90 OCE/ICB/icbini.F90 OCE/ICB/icblbc.F90 OCE/ICB/icbstp.F90 OCE/ICB/icbutl.F90
-
OCE/ICB/icbdia.F90
326 326 CALL report_consistant( 'bot interface','kg','sent',calving_out_net, & 327 327 & 'returned',calving_ret_net) 328 328 ENDIF 329 WRITE( numicb, '("calved by class = ",i6,20(",",i6))') (nbergs_calved_by_class(ik),ik=1,nclasses) 330 IF( nspeeding_tickets > 0 ) WRITE( numicb, '("speeding tickets issued = ",i6)') nspeeding_tickets 329 IF (nn_verbose_level > 0) THEN 330 WRITE( numicb, '("calved by class = ",i6,20(",",i6))') (nbergs_calved_by_class(ik),ik=1,nclasses) 331 IF( nspeeding_tickets > 0 ) WRITE( numicb, '("speeding tickets issued = ",i6)') nspeeding_tickets 332 ENDIF 331 333 ! 332 334 nbergs_start = nbergs_end 333 335 stored_start = stored_end … … 436 438 IF( kt == nit000 ) THEN 437 439 stored_start = SUM( berg_grid%stored_ice(:,:,:) ) 438 440 CALL mpp_sum( 'icbdia', stored_start ) 439 WRITE(numicb,'(a,es13.6,a)') 'icb_dia_income: initial stored mass=',stored_start,' kg'440 441 ! 441 442 stored_heat_start = SUM( berg_grid%stored_heat(:,:) ) 442 443 CALL mpp_sum( 'icbdia', stored_heat_start ) 443 WRITE(numicb,'(a,es13.6,a)') 'icb_dia_income: initial stored heat=',stored_heat_start,' J' 444 IF (nn_verbose_level > 0) THEN 445 WRITE(numicb,'(a,es13.6,a)') 'icb_dia_income: initial stored mass=',stored_start,' kg' 446 WRITE(numicb,'(a,es13.6,a)') 'icb_dia_income: initial stored heat=',stored_heat_start,' J' 447 ENDIF 444 448 ENDIF 445 449 ! 446 450 calving_rcv_net = calving_rcv_net + SUM( berg_grid%calving(:,:) ) * berg_dt … … 514 518 INTEGER, INTENT(in), OPTIONAL :: kbergs 515 519 !!---------------------------------------------------------------------- 516 520 ! 521 IF (nn_verbose_level == 0) RETURN 517 522 IF( PRESENT(kbergs) ) THEN 518 523 WRITE(numicb,100) cd_budgetstr // ' state:', & 519 524 & cd_startstr // ' start', pstartval, cd_budgetunits, & … … 538 543 REAL(wp), INTENT(in) :: pstartval, pendval 539 544 !!---------------------------------------------------------------------- 540 545 ! 546 IF (nn_verbose_level == 0) RETURN 541 547 WRITE(numicb,200) cd_budgetstr // ' check:', & 542 548 & cd_startstr, pstartval, cd_budgetunits, & 543 549 & cd_endstr, pendval, cd_budgetunits, & … … 557 563 REAL(wp) :: zval 558 564 !!---------------------------------------------------------------------- 559 565 ! 566 IF (nn_verbose_level == 0) RETURN 560 567 zval = ( ( pendval - pstartval ) - ( pinval - poutval ) ) / & 561 568 & MAX( 1.e-30, MAX( ABS( pendval - pstartval ) , ABS( pinval - poutval ) ) ) 562 569 ! … … 577 584 INTEGER , INTENT(in) :: pstartval, pendval 578 585 !!---------------------------------------------------------------------- 579 586 ! 587 IF (nn_verbose_level == 0) RETURN 580 588 WRITE(numicb,100) cd_budgetstr // ' state:', & 581 589 & cd_startstr // ' start', pstartval, & 582 590 & cd_endstr // ' end', pendval, & … … 594 602 INTEGER, INTENT(in) :: pinval, poutval, pstartval, pendval 595 603 !!---------------------------------------------------------------------- 596 604 ! 605 IF (nn_verbose_level == 0) RETURN 597 606 WRITE(numicb,200) cd_budgetstr // ' budget:', & 598 607 & cd_instr // ' in', pinval, & 599 608 & cd_outstr // ' out', poutval, & -
OCE/ICB/icbdyn.F90
370 370 ENDIF 371 371 ENDIF 372 372 ! ! check the speed and acceleration limits 373 IF( ABS( zuveln ) > pp_vel_lim .OR. ABS( zvveln ) > pp_vel_lim ) & 374 WRITE(numicb,'("pe=",i3,x,a)') narea,'Dump triggered by excessive velocity' 375 IF( ABS( pax ) > pp_accel_lim .OR. ABS( pay ) > pp_accel_lim ) & 376 WRITE(numicb,'("pe=",i3,x,a)') narea,'Dump triggered by excessive acceleration' 373 IF (nn_verbose_level > 0) THEN 374 IF( ABS( zuveln ) > pp_vel_lim .OR. ABS( zvveln ) > pp_vel_lim ) & 375 WRITE(numicb,'("pe=",i3,x,a)') narea,'Dump triggered by excessive velocity' 376 IF( ABS( pax ) > pp_accel_lim .OR. ABS( pay ) > pp_accel_lim ) & 377 WRITE(numicb,'("pe=",i3,x,a)') narea,'Dump triggered by excessive acceleration' 378 ENDIF 377 379 ! 378 380 END SUBROUTINE icb_accel 379 381 -
OCE/ICB/icbini.F90
77 77 ! ! open ascii output file or files for iceberg status information 78 78 ! ! note that we choose to do this on all processors since we cannot 79 79 ! ! predict where icebergs will be ahead of time 80 CALL ctl_opn( numicb, 'icebergs.stat', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, lwp, narea ) 80 IF( nn_verbose_level > 0) THEN 81 CALL ctl_opn( numicb, 'icebergs.stat', 'REPLACE', 'FORMATTED', 'SEQUENTIAL', -1, numout, lwp, narea ) 82 ENDIF 81 83 82 84 ! set parameters (mostly from namelist) 83 85 ! … … 240 242 ENDIF 241 243 CALL iom_close( inum ) ! close file 242 244 ! 243 WRITE(numicb,*) 244 WRITE(numicb,*) ' calving read in a file' 245 IF( nn_verbose_level > 0) THEN 246 WRITE(numicb,*) 247 WRITE(numicb,*) ' calving read in a file' 248 ENDIF 245 249 ALLOCATE( sf_icb(1), STAT=istat1 ) ! Create sf_icb structure (calving) 246 250 ALLOCATE( sf_icb(1)%fnow(jpi,jpj,1), STAT=istat2 ) 247 251 ALLOCATE( sf_icb(1)%fdta(jpi,jpj,1,2), STAT=istat3 ) … … 335 339 ! 336 340 ibergs = icb_utl_count() 337 341 CALL mpp_sum('icbini', ibergs) 338 WRITE(numicb,'(a,i6,a)') 'diamonds, icb_ini_gen: ',ibergs,' were generated' 342 IF( nn_verbose_level > 0) THEN 343 WRITE(numicb,'(a,i6,a)') 'diamonds, icb_ini_gen: ',ibergs,' were generated' 344 ENDIF 339 345 ! 340 346 END SUBROUTINE icb_ini_gen 341 347 -
OCE/ICB/icblbc.F90
639 639 640 640 zsbergs(0) = narea 641 641 zsbergs(1) = nicbfldnsend(jn) 642 !IF ( nicbfldnsend(jn) .GT. 0 ) write(numicb,*) 'ICB sending ',nicbfldnsend(jn),' to ', ifldproc642 !IF ( nicbfldnsend(jn) .GT. 0 .AND. nn_verbose_level > 0 ) write(numicb,*) 'ICB sending ',nicbfldnsend(jn),' to ', ifldproc 643 643 CALL mppsend( 21, zsbergs(0:1), 2, ifldproc-1, nicbfldreq(jn)) 644 644 ENDIF 645 645 ! … … 655 655 DO jjn = 1,jpni 656 656 IF( nicbfldproc(jjn) .eq. INT(znbergs(1)) ) EXIT 657 657 END DO 658 IF( jjn .GT. jpni ) write(numicb,*) 'ICB ERROR'658 IF( jjn .GT. jpni .AND. nn_verbose_level > 0 ) write(numicb,*) 'ICB ERROR' 659 659 nicbfldexpect(jjn) = INT( znbergs(2) ) 660 !IF ( nicbfldexpect(jjn) .GT. 0 ) write(numicb,*) 'ICB expecting ',nicbfldexpect(jjn),' from ', nicbfldproc(jjn)661 ! CALL FLUSH(numicb)660 !IF ( nicbfldexpect(jjn) .GT. 0 .AND. nn_verbose_level > 0 ) write(numicb,*) 'ICB expecting ',nicbfldexpect(jjn),' from ', nicbfldproc(jjn) 661 !IF (nn_verbose_level > 0) CALL FLUSH(numicb) 662 662 ENDIF 663 663 ! 664 664 END DO … … 910 910 DEALLOCATE(old) 911 911 ENDIF 912 912 old => new 913 !WRITE( numicb,*) 'icb_increase_ibuffer',narea,' increased to',inew_size913 !IF (nn_verbose_level > 0) WRITE( numicb,*) 'icb_increase_ibuffer',narea,' increased to',inew_size 914 914 ENDIF 915 915 ! 916 916 END SUBROUTINE icb_increase_ibuffer -
OCE/ICB/icbstp.F90
162 162 163 163 IF(lwp) WRITE(numout,'(a,i6)') 'icebergs: icb_end complete', narea 164 164 ! 165 CALL flush( numicb ) 166 CLOSE( numicb ) 165 IF( nn_verbose_level > 0 ) THEN 166 CALL flush( numicb ) 167 CLOSE( numicb ) 168 ENDIF 167 169 ! 168 170 END SUBROUTINE icb_end 169 171 -
OCE/ICB/icbutl.F90
624 624 INTEGER :: kt ! timestep number 625 625 !!---------------------------------------------------------------------- 626 626 ! 627 IF (nn_verbose_level == 0) RETURN 627 628 pt => berg%current_point 628 629 WRITE(numicb, 9200) kt, berg%number(1), & 629 630 pt%xi, pt%yj, pt%lon, pt%lat, pt%uvel, pt%vvel, & … … 648 649 TYPE(iceberg), POINTER :: this 649 650 !!---------------------------------------------------------------------- 650 651 ! 652 IF (nn_verbose_level == 0) RETURN 651 653 this => first_berg 652 654 IF( ASSOCIATED(this) ) THEN 653 655 WRITE(numicb,'(a," pe=(",i3,")")' ) cd_label, narea
7. sette/sette.sh
Finally, once implemented, these changes allow SETTE tests to be run without creating the full set of multiple output files. It is simply a case of replacing all ln_ctl settings like so:
-
sette.sh
287 288 set_namelist namelist_cfg nn_stock 495 288 289 set_namelist namelist_cfg jpni 4 289 290 set_namelist namelist_cfg jpnj 8 290 set_namelist namelist_cfg ln_ctl .true. 291 set_namelist namelist_cfg ln_ctl .false. 292 set_namelist namelist_cfg sn_cfctl%l_config .true. 293 set_namelist namelist_cfg sn_cfctl%l_runstat .true. 294 set_namelist namelist_cfg sn_cfctl%l_trcstat .true. 295 set_namelist namelist_cfg nn_verbose_level 0 291 296 set_namelist namelist_cfg ln_use_calving .true. 292 297 set_namelist namelist_cfg ln_wave .true. 293 298 set_namelist namelist_cfg ln_cdgw .true.
Sebastien's comments on the first draft, many of which have already been incorporated into version2 above:
Maybe you know this story, but it case you don’t know… :-)
1) A long time ago - in a galaxy far far away - ... ln_ctl was there (since ever?, at least in rev3) and was a control print of the trend over the first core.
2) At rev258 https://forge.ipsl.jussieu.fr/nemo/changeset/258 the concept was modified and extended to be able to debug mpi reproducibility. prtctl was added but, to my knowledge, never documented. The idea is to compare the mean value of every trend in the code over (1) the local mpi domain and (2) the same domain but in a 1 core simulation (or n core as soon as the domain you want to test is included in your mpi subdomain). You compare you mpp.output_xxxx and mono.output_xxxx files and the first difference tells you which core and which trend is the first to diverge.
3) Next, as prtctl was dealing with mpp.output_xxxx and mono.output_xxxx files, we again extended the use of ln_ctl to control the creation of any file trough the definition of lwp: https://forge.ipsl.jussieu.fr/nemo/changeset/1579 This was a bad idea as, with 1 variable (ln_ctl), we control 2 different concepts : (1) mpp debug which requires the production of mpp.output_xxxx and mono.output_xxxx files and (2) the control of all other outputs files (ocean.output, run.stat, layout.dat…)
4) For Performance issues, with Eric, we wanted to be able to remove the globsum in stpctl. At some point we thought that we should introduce a new namelist variable something like in_fast or ln_prod or ln_debug or nn_debug which would allow us to reduce the printed informations and use faster code (for example without globsum in stpctl). After some discussions, we thought that our idea was not clear enough to be introduced in the code before the release in December. We therefore decided to postpone this and simply use ln_ctl to switch on/off the globsum (because if ln_ctl = T, you don’t care of the performances so you can do globsum in stpctl). Once you switch off the globsum, the data in run.stat are useless, so we also switch off run.stat files.
So Today, ln_ctl is mixing different functionalities coming from theses 4 layers of developments. I think we should take advantage of your development to clean-up this mess!
One proposition (to be discussed) could be to split ln_ctl into 3 functionalities:
- mpp debug associated to prtctl. That could be renamed, for example in prttrd or trdprt for "trend print". Or mppdbg for mpp debug? this part requires the creation of the mpp.output_xxxx and mono.output_xxxx files which, to me, should be controlled only by the activation or not of prtctl
- something (a logical, an integer, a structure) related to the balance between verbosity-control-debug and performance. This would control/replace/be linked with the creation of run.stat files, nn_print, ln_timing maybe also layout.dat
- use your sn_ctl to control other files
Other minor points :
- EMPave.dat : I had a quick look but it seems that this file is the same for each core (contains only the year and global mean). So we should not offer the possibility to create EMPave.dat_xxxx files. Either all cores read the same file (as we do for the nameliste) or only core 0 read it and use mpi_broadcast to send the informations to all other cores (as, I think, we should also do for the name lists). First part done, will consider use of mpi_bcast in 2019
- We have files created by oce, si3 and top. Maybe, It is strange if a part of OCE, control the creation of top files like tracer.stat, no? To be discussed
- instead of having to test sn_cfctl%l_config everywhere as for example in "sn_cfctl%l_config .AND. sn_cfctl%l_trcstat”, I would, in nemo_set_cfctl, do sn_cfctl%l_trcstat = setto .AND. sn_cfctl%l_config Actually the test of l_config isn't needed at the lower levels, removed
- there is also time.step file Not sure we ever want to switch this off but it can now be updated less frequently according to ptimincr
- do we really need the sn_ctl structure? why no a simple list of logical and integer? To be discussed
- I would like to “promote” the use of ln_timing, except in production mode, so people become used to look at this file to see the main bottlenecks in computational coast and communications… Think this is a separate issue unless multiple files are produced?
