Opened 4 years ago

Closed 12 months ago

#277 closed enhancement (fixed)

1 + 1 >< 2

Reported by: luyssaert Owned by: luyssaert
Priority: minor Milestone: IPSLCM6.v2
Component: Model architecture Version:
Keywords: Cc:

Description (last modified by jgipsl)

two identical installations of the revision 2978 (branch ORCHIDEE-CN-CAN) do not result in identical output for SRF. The SRF output differed by 10-5 to 10-8 for the C-fluxes for the 4 pixels and 10-6 to 10-9 for the single pixel run. This appears as a minor issue but it suggests that the 1+1 problem has not been solved. Interestingly GPP in stomate is identical. GPP in SRF is among the variables with shows small differences.

Change History (19)

comment:1 Changed 4 years ago by jgipsl

  • Description modified (diff)

comment:2 Changed 4 years ago by jgipsl

Note that the in the description of the ticket, it is not clear if the simulations have the same PeriodLenth. If they are exactly the same, the model is not reproductible which is even worth than the problem 1+1><2

Tests have also been done with ORCHIDEE-CN, revision 4105 (done by Palmira Messina):

  • 2 identical simulations for 1 year reproduces exactly the same results, 1=1. The model is reproducible.
  • 2 simulations for 1 year, only different for PeriodLength=1M and PeriodLength=1Y do not give exactly the same results. 1+1><2

comment:3 Changed 2 years ago by luyssaert

  • Resolution set to wontfix
  • Status changed from new to closed

Further testing with r5554 showed a similar issue as that described above.
(1) Setting a run of 1Y and then restarting the model for another 1Y gives exactly the same result as running the model for 2Y at once.
(2) Running the model for 12M results in small differences compared to running the model for 1Y at once.

Since CN was merged into the trunk this ticket most likely duplicates #471. Given that this is not a blocking issue for the current use of ORCHIDEE-CN-CAN we will wait until the issue has been fixed in the trunk

comment:4 Changed 2 years ago by luyssaert

  • Resolution wontfix deleted
  • Status changed from closed to reopened

comment:5 Changed 2 years ago by mmcgrath

  • Owner changed from somebody to mmcgrath
  • Status changed from reopened to accepted

Since I am looking at the same issue in the trunk, I may have an idea of what is causing this afterwards. Therefore, I can take this one.

First priority is to fix the trunk for tagging, and then I will look at this after.

comment:6 Changed 2 years ago by mmcgrath

Fixed in the trunk in r5733.

The trunk fix is included in r5826 of CAN. Still need to confirm if there are any other restartability problems with CAN.

comment:7 follow-up: Changed 23 months ago by luyssaert

Looks like there is a problem with how albedo is intiniatilized. JG will look into that issue. Note that the first test 1Y + 1Y = 2Y only tests whether the config.card works properly as both approaches do exactly the same.

comment:8 in reply to: ↑ 7 Changed 23 months ago by jgipsl

Replying to luyssaert:

Looks like there is a problem with how albedo is intiniatilized. JG will look into that issue. Note that the first test 1Y + 1Y = 2Y only tests whether the config.card works properly as both approaches do exactly the same.

Variables were added to restart file in the albedo module in changeset [5918]. No test for 1+1=2 has been done.

comment:9 Changed 23 months ago by jgipsl

Note : height and lai calculated in slowproc_initialize might be an issue of non convergence. Because of veget_max which is used. To be checked if 1+1=2 is not found.

comment:10 Changed 22 months ago by jgipsl

Ticket #554 must be resolved before this ticket can be resolved.

comment:11 Changed 17 months ago by luyssaert

  • Owner changed from mmcgrath to alanso
  • Status changed from accepted to assigned

comment:12 Changed 17 months ago by alanso

r6247 is a partial bugfix for #277 that was tested for r6213. It fixes the 1+1 problem for a smaller region by adding two variables to the restart files (harvest_pool and daylight_count). Not all run run.def settings were tested for, only the standard FG2, thus not litter_raking, land cove change, species change, wind_throw and FM=1,3,4.

However, globally the 1+1 >< 2 still persists.

Next step, test the bugfix from Albert in the IOIPSL trunk.

comment:13 Changed 17 months ago by alanso

  • Resolution set to fixed
  • Status changed from assigned to closed

r6272 fixes the 1+1><2 problem. Two variables needed to be added to the sechiba restart files. Moreover, in the hydrol_arch swc_adj was initialized to min_sechiba when l_first_sechiba is true. It seems l_first_sechiba is true every time a new periodlength is started, which resulted in deviations between simulations with different periodlengths.

Thus, 1+1=2 globally for the standard settings of FG2 (i.e. no LCC, no species change, no litter raking, no wind throw and not tested for FM=1,3,4).

The IOIPLS bugfix did not impact my previous test cases, but it is included in the newest revision (4746) of IOIPSL.

comment:14 Changed 14 months ago by alanso

  • Priority changed from major to critical
  • Resolution fixed deleted
  • Status changed from closed to reopened

Test with different period lengths for r6425 showed that the 1+1><2 problem has been re-introduced, therefore the ticket has been reopened.

comment:15 Changed 13 months ago by luyssaert

  • Owner changed from alanso to luyssaert
  • Status changed from reopened to assigned

comment:16 Changed 13 months ago by luyssaert

r6542 compiled in production mode

Simplified test case (1 pixel, no land cover change)

31x1D = 1M

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

12x1M = 1Y

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

SPINUP_ANALYTIC_FG1

31x1D = 1M

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

12x1M = 1Y

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

OOL_SEC_STO_FG2

31x1D = 1M

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

12x1M = 1Y

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

OOL_SEC_STO_FG4

31x1D = 1M

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

12x1M = 1Y

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

OOL_SEC_STO_FG5

31x1D = 1M

  • SRF ae_ns, assim_param, cc_biomass_c, cc_biomass_n, cgrnd, cgrnd_snow, circ_class_n, dgrnd, dgrnd_snow, drysoil_frac, evap_bare_lim, evap_bare_lim_ns, evapora, evapot, evapot_corr, fluxlat, fluxsens, gtemp, humrel, ksoil, laieff_isotrop, lambda_snow, mc_out, moistc, moistcl, ptn, ptn_pftmean, qsintveg, qsurf, roughheight, snow, snow_age, snow_beg, snow_nobio, snow_nobio_age, snowdz, snowgrain, snowheat, snowrho, snowtemp, soilcap, soilflx, temp_sol, tot_watsoil_beg, tot_watveg_beg, tsolrad, us, veget, vegstress, water2infilt, z0h, z0m
  • SBG Many, many variables
  • OOL differences in eair_old, fluxsens, peqBcoef, petBcoef, qair_old, rau_old, vevapp, z0

12x1M = 1Y

  • SRF Many differences
  • SBG Many differences
  • OOL Many differences

r6522 compiled in debug mode

SPINUP_ANALYTIC_FG1

1D+1D = 2D

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

OOL_SEC_STO_FG2

1D+1D = 2D

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK

OOL_SEC_STO_FG4

1D+1D = 2D

  • SRF Ninput_year (see ticket #621)
  • SBG OK
  • OOL OK
Last edited 13 months ago by luyssaert (previous) (diff)

comment:17 Changed 13 months ago by luyssaert

  • Priority changed from critical to minor

For FG5 restartability needs to be implemented and confirmed

comment:18 Changed 12 months ago by luyssaert

For FG5 1D+1D differs from 2D. FG5 uses daily forcing files. If just these files are replaced by 6-hourly files 1D+1D equals 2D. It seems that the code that enables using daily forcing is not restartable. The problem with the FG5 restartability can likely be traced to readdim2.f90 because FG5 should use interpol_daily when forced by daily forcing files whereas FG5 forced by 6-hourly files does not. Note that for the moment FG5 is NOT the problem, it is just the test case.

Some first insights from the code (there is basically no documentation)

  • L1199 reads "IF ((last_read == 0) .OR. ( rw==(1./split)) ) THEN". This block of code reads the forcing data if we have never read them before (first time step in the execution) or in the middle of the day.
  • L1275. Here are either with reading forcing data at the first time step or reading mid-day. If it is the first time step in the day (itau_split).
  • Block around L1282 -> executed at the first time step of the day. Should a restart be read?

comment:19 Changed 12 months ago by luyssaert

  • Resolution set to fixed
  • Status changed from assigned to closed

All ORCHIDEE-CN-CAN specific problems with restartability have been solved. A new ticket was created for the remaining problem (#682) which is related to dim2driver.f90 and bears no relationship with the CN-CAN developments.

Note: See TracTickets for help on using tickets.