Opened 6 years ago

Closed 5 years ago

#342 closed task (fixed)

Adapt libIGCM to new file system of irene

Reported by: acosce Owned by: somebody
Priority: major Milestone:
Component: system Version:
Keywords: Cc:

Description

since the last maintenance users can have several workdir/storedir/scratchdir/homedir. This choice was made by project PI, and to be coherent with this choice, the new way to work will be : when we are using hour of a project, we write automatically on the file system of this specific project. We need to conserv a solution to write on an other file system.

Proposition : use a switch dfldatadir/own dfldatadir/project to position Environment variables on the project filesystem
Note : it's possible that dfldatadir was already on a project, so we need to determine this before the switch
Note : we need to check if option archive in config.card works

We don't change sys_curie

Change History (3)

comment:1 Changed 6 years ago by jgipsl

See here a proposition where libIGCM changes the dfldatadir.

Change

# Use CMIP6 storage space when using project id gch0316
[[ "X${PROJECT}" = "Xgch0316" && ! $(module list --terse 2>&1 | grep dfldatadir/gencmip6) ]] && module switch dfldatadir dfldatadir/gencmip6

into

# Load dfldatadir depending on the project used for submission
if [ "X${PROJECT}" = "Xgch0316" ] ; then
  # For project gch0316 use dfldatadir gencmip6
  module switch dfldatadir dfldatadir/gencmip6
elif [ X"$( echo ${PROJECT} | grep cmip6 )" != "X" ] ; then
  # For project all cmip6 project use dfldatadir gencmip6
  module switch dfldatadir dfldatadir/gencmip6
else
  # For all other projects use corresponding dfldatadir
  module switch dfldatadir dfldatadir/${PROJECT}
fi

Later on, an optional variable in config.card could be used to specify another dfldatadir if for example a user runs on one project but needs temporary to save output on another project.

comment:2 Changed 6 years ago by jgipsl

Commit done [1461]. The special cases for gch0316 and cmip6 have been removed. Instead a new variable DataProject can be set in UserChoices section in config.card. See more details in the changeset log for the commit.

I think it should be recommended to always set this variable in config.card to ensure the good functioning of post-treatement jobs if they are launched manually. If not, the user needs to be sure to make the corresponding module load of dlfdatadir in the terminal. We could think about adding this in ins_job...

Note that module purge do not affect the module dfldatadir.

comment:3 Changed 5 years ago by sdipsl

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.