Opened 11 years ago

Closed 11 years ago

Last modified 2 years ago

#372 closed Bug (invalid)

problem with netcdf syncing

Reported by: momme Owned by: nemo
Priority: normal Milestone:
Component: env Version: release-3.1
Severity: Keywords: NetCDF* outputs
Cc:

Description

With the code as it is from the repository I end up with corrupted netcdf files that clearly had some data written to them (size >1GB), but are inaccessable (header intact, but no data). The problem is avoided by uncommenting the call to NF90_SYNC in histcom.F90 line 2181.
Platform: Fedora Linux 64-bit, sinlge process
Compiler: ifort
Netcdf: 3.6.3
Configuration: ORCA 2dgr, 1D and 3D global

Commit History (3)

ChangesetAuthorTimeChangeLog
9287anaguiar2018-01-30T12:47:33+01:00

ticket GMED#372,NEMO#741: Branch that enables varying weight of snow precipitation forcing over sea ice

9233anaguiar2018-01-15T15:39:14+01:00

ticket GMED#372,NEMO#741: Half snow precipitation forcing

9231anaguiar2018-01-15T15:30:17+01:00

ticket GMED #372: Half snow precipitation forcing

Change History (14)

comment:1 Changed 11 years ago by smasson

I don't really understand. Do you mean that the time dimension has a size of 0?

Have a look to http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-f90/NF90_005fSYNC.html :

The function NF90_SYNC offers a way to synchronize the disk copy of a netCDF dataset with in-memory buffers. There are two reasons you might want to synchronize after writes:

 - To minimize data loss in case of abnormal termination, or
 - To make data available to other processes for reading immediately after it is written. But note that a process that already had the dataset open for reading would not see the number of records increase when the writing process calls NF90_SYNC; to accomplish this, the reading process must call NF90_SYNC.

i think you are in the first case…

Data is automatically synchronized to disk when a netCDF dataset is closed, or whenever you leave define mode.

⇒ I think the model stopped before the end of the run and didn't get through the call to NF90_CLOSE.

So I think calling NF90_SYNC is not really solving your problem.

comment:2 Changed 11 years ago by smasson

  • Resolution set to invalid
  • Status changed from new to closed

There is some additional information from NetCDF FAQ web page.
Unfortunately, it appears that the archived support message has been lost, but Russ Rew from netCDF support sent me the information:

You can use an editor which allows you to change binary files, such as emacs (use for example the hexl mode), to correct the 'number of records' in the file. This is a bigendian 4 byte integer which begins at the 4th byte in the file.

This is what the first eight bytes would look like in emacs if you had zero records:

CDF^A^@^@^@^@

and this is what it would like if you had 6 records (^F is ascii 6)

CDF^A^@^@^@^F

See Appendix B of the netcdf user's guide for more info.


comment:3 Changed 4 years ago by nicolasmartin

  • Keywords outputs added; output removed

comment:4 Changed 4 years ago by nicolasmartin

  • Keywords nemo_v3_1* added

comment:5 Changed 4 years ago by nicolasmartin

  • Keywords netCDF added; netcdf removed

comment:6 Changed 4 years ago by nicolasmartin

  • Keywords syncing added; sync removed

comment:7 Changed 4 years ago by nicolasmartin

  • Keywords NetCDF added; netCDF removed

comment:8 Changed 4 years ago by nicolasmartin

  • Keywords histcom removed

comment:9 Changed 4 years ago by nicolasmartin

  • Keywords NetCDF* added; NetCDF removed

comment:10 Changed 3 years ago by nemo

  • Keywords syncing removed

comment:11 Changed 3 years ago by nemo

  • Keywords nemo_v3_1* removed

comment:12 Changed 2 years ago by anaguiar

In 9231:

ticket GMED #372: Half snow precipitation forcing

comment:13 Changed 2 years ago by anaguiar

In 9233:

ticket GMED#372,NEMO#741: Half snow precipitation forcing

comment:14 Changed 2 years ago by anaguiar

In 9287:

ticket GMED#372,NEMO#741: Branch that enables varying weight of snow precipitation forcing over sea ice

Note: See TracTickets for help on using tickets.