Opened 8 years ago

Closed 8 years ago

#93 closed defect (fixed)

NF90_64BIT_OFFSET flag required for global stomate_Cforcing file

Reported by: maignan Owned by: jgipsl
Priority: major Milestone: ORCHIDEE 2.0
Component: Biogeochemical processes Version: orchidee_1_9_6
Keywords: Cc:

Description

We may now have a netcdf file that is larger than 2 Gb, but each variable cannot be above 2 Gb for the netcdf 3.6 classic version (see for example here:http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Large-File-Support.html).
In the stomate_Cforcing file we have this huge variable:

float soilcarbon_input(time_step, vegtype, carbtype, points)

Its size is thus for a global WFDEI yearly simulation:
4*365*13*3*67209 = 3.82688e+09 # 3.8Gb

So we have to use the 64-bit offset flag when creating the stomate_Cforcing file in stomate.f90, see for example here: http://www.unidata.ucar.edu/software/netcdf/faq-lfs.html#Large File Support8

In Fortran-90, use the NF90_64BIT_OFFSET flag when you call nf_create(), as in:
iret = nf90_create(path="foo.nc",
cmode=or(nf90_clobber,nf90_64bit_offset),
ncid=ncFileID)

Change History (6)

comment:1 Changed 8 years ago by maignan

By Nicolas Viovy:
Good point!
But does ORCHIDEE generate only one forcing and Cforcing file even in parallel?
I didn't imagined that the problem of Ben could come from this as I already did a lot of simulations with cruncep with a similar number of points, but in my case it generates one forcing file per proc.
This would be probably something to do also in the standard version as it is probably very inefficient for ORCHIDEE to deal with such huge files!

comment:2 Changed 8 years ago by maignan

By Benjamin Poulter:
An update for global runs, which are now working successfully:
stomate_restart.nc also needs to be created with the 64 bit capacity because the file is > 2GB.
When the ncdf is created in 64 bit mode, then this makes the restclo warning go away.

comment:3 Changed 8 years ago by jgipsl

  • Resolution set to fixed
  • Status changed from new to closed

NF90_64BIT_OFFSET is now added in

comment:4 Changed 8 years ago by jgipsl

  • Resolution fixed deleted
  • Status changed from closed to reopened
  • Type changed from task to defect

Adding NF90_64BIT_OFFSET makes problems at ADA. Model stops because not enough memory.

comment:5 Changed 8 years ago by jgipsl

  • Owner changed from somebody to jgipsl
  • Status changed from reopened to accepted

comment:6 Changed 8 years ago by jgipsl

  • Resolution set to fixed
  • Status changed from accepted to closed

The problem was not due to NF90_64BIT_OFFSET option.

The whole option for writing the Stomate_Cforcing file is memory demanding. At Ada, the default limit of 3.5gb per processes is not enough. It is possible to ask for more memory. See here HowTo/IDRIS

Note: See TracTickets for help on using tickets.