Version 285 (modified by flavoni, 12 years ago) (diff)

Last edited Timestamp?

Using libIGCM library with ORCA2_LIM


  • Document the using of libIGCM for NEMO, in configuration ORCA2_LIM

Quick start guide:

  • Define the following alias to use SVN :
     alias svn_ano='svn co modipsl'
  • Create and go into working directory :
     mkdir TRY ; cd TRY
  • Extract modipsl :
  • Extract NEMO :
     cd modipsl/util 
     ./model NEMO_IGCM 
  • Choosing ORCA2_LIM configuration :
     cd ../modeles/UTIL
     ./fait_config ORCA2_LIM
  • Install the makefiles :
     cd ../../util 
  • Compile :
     cd ../config/ORCA2_LIM 
  • Submit Job :
     ( Editing at this point modipsl/config/ORCA2_LIM/IGCM00/config.card )
     '''change RESOL file in Job_Name:  RESOL=ORCA2'''
     cd IGCM00 
     qsub Job_NL1  

Set up a simulation:

  • First of all see official documentation

  • All parameters in config.card file

In modipsl/config/ORCA2_LIM/IGCM00/config.card file you can set all parameters to manage your experience:

How to manage restart files ( in the same simulation )

in config.card you have to set duration of your experience and setting PeriodLenght parameter, so you create restart files.

( for example : if you have and experience of 10 years and a PeriodLenght of 1 year, you will have 10 restart files )

#— Début et fin de Job
Date Begin?=2001-01-01
Date End?=2010-12-31
#— 1Y, 1M, 5D, 1D
Period Length?=1Y

Note : If you want to use restart data from another simulation ( already run not in this moment ) you have to manage "Restarts" part of config.card

#D— Restarts -

How to manage write output frequency

in config.card you have to set WriteFrequency of your experience

( for example : if you want output files every 1 month you have to put for Ocean

#D— OCE -
Write Frequency?="1M"

and if you want output files every 1 month you have to put for ICE

#D— ICE -
Write Frequency?="1M"


How to manage calendar type

in config.card you have to set CalendarType like 360d or leap or noleap

( for example : if you want 360 days you have to be careful of

Date End?=2010-12-30

because every month has 30 days

If you put for example Date End?=2005-12-31, you will have another run, of duration like your PeriodLenght )

How to manage rdt ( i.e. timestep )

in COMP/nemo.driver you have to change parameter : ORCA_NPDT_JOUR=15

( 15 corresponds to 86400s/5760s; it does mean that in one day you have 15 iterations; and it does mean that timestep is 5760)

How control all parameters in modipsl/config/ORCA2_LIM/IGCM00/PARAM/namelist

libIGCM change namelist's values but not in namelist's file, so it is possible to control all changes of namelist's parameters looking at :


How to activate Monitoring

in config.card file

#D- If you want to produce time series, this flag determines #D- frequency of post-processing submission Time Series Frequency?=NONE

you have to put you experience lenght. If you do an experience during 10 years you have to put:

Time Series Frequency?=10Y

How to do dynamic Monitoring (25/02/2009)

  • Copy original monitoring files in SUBMIT_DIR :
     cp ~/KEEPED_FILES/monitoring01_nemo_ORCA2.cfg $SUBMIT_DIR/.
     cp ~/KEEPED_FILES/monitoring01_lim2_ORCA2.cfg $SUBMIT_DIR/.
  • Copy monitoring.job in libIGCM :
     cp ~/KEEPED_FILES/monitoring.job modipsl/libIGCM/.
  • Control Time Series? in COMP/nemo.card and COMP/lim2.card:
     and if you add or remove Time Serie in nemo.card or lim2.card => 
     it is necessary to add or remove correspondig variable in $SUBMIT_DIR/monitoring01_nemo[lim2]_ORCA2.cfg

How to activate Atlas

in config.card file

#D- If you want to produce seasonal average, this flag determines #D- the period of this average Seasonal Frequency?=NONE

you have to put you experience lenght. If you do an experience during 10 years you have to put:

Seasonal Frequency?=10Y

How to manage frequency of post processing

( Note: in this moment is not possible to have atlas and monitoring of frequency different from 1M, if we want this we've to see it with Patrick Brockman!! )

in COMP/nemo.card you have to change in [Output Files?] chapter:

Post_1M_grid_T (U,V,W)

and you have to change [Post_1M_grid_T(U,V,W)] chapter.

and in in COMP/lim2.card you have to change in [Output Files?] chapiter:

List= (${PREFIX_NWRITE}_${DATE_OPA}, ${R_OUT_ICE_NWRITE}/${PREFIX}_${WF1}, Post_1M_icemod)

and you have to change [Post_1M_icemod] chapter.

  • How to make a parallel run

first of all add a cpp key:

and then see:

  • How to have mail of end of Job, sur ccrt's machines

on mercure: in Job_Name: #PBS -M name@mail

on platine: In Job_Name: #BSUB -u name@mail

  • How to activate particular key cpp

(if not for example we have this error:


Run only post processing:

  • If you have already run a job and you need only to run post processing

you can do it in 3 ways:


in particular:


for example:

If you want to post process a Job already run for 10 years form 2001 to 2010:
in run.card

  • modify OldPrefix
    where (aa-1)mmdd is last day of last month of one year before you simulation's start
  • modify CumulPeriod
    is 10 years that you've already run
  • modify PeriodState
  • modify TimeSeriesRunnging
    no because Time series are depending for each other, so if you want to start a time series it starts ONLY when a previous one is finished, so if you put TimeSeriesRunnging=y (i.e. yes) it can't never start
  • modify TimeSeriesCompleted
    nothing because we want to do time series for all 10 years ( If not you have to put last time series already realised )

in Job_Name:

  • modify cputim_job
    #PBS -l cputim_job=4:00:00  # Limite temps 
  • modify DRYRUN
  • modify RUN_DIR_PATH
     Create, if necessary, a test directory (for example in you workdir)
     Decomment RUN_DIR_PATH=/$workdir/TESTPOST         
  • modify SUBMIT_DIR
     Decomment and put right submit directory (the one in which you are changed you run.card and you Job) 

List of possible errors in modipsl/config/ORCA2_LIM/IGCM00/Script_Output_JobName :

  • Error : TERM = (vt100) ${HOME:-.}/.profile: .bashrc: not found
  • Error : ksh: D—————————————————-==: not found

it can depends on .profile file.

To solve it
.profile file it can be like:

«# .profile (idris)
# Fichier d environnement a charger sur la frontale TX7 unix LINUX
# Vous pouvez mettre a jour le fichier .profile_tx
if [ `hostname!` = "brodie" -a -f $HOME/.profile_tx ]; then
. $HOME/.profile_tx
# Fichier d environnement a charger sur les noeuds SX8 unix SUPER-UX
# Vous pouvez mettre a jour le fichier .profile_sx
elif [ `hostname!` != "brodie" -a -f $HOME/.profile_sx ]; then
. $HOME/.profile_sx
fi »

  • Error : cat: cannot open ………/modipsl/config/ORCA2_LIM/IGCM00/../.resol :

it is due to modipsl/libIGCM/AA_job file.

To solve it
change in this way:

#— Resolution FLAG for oasis input
#RESOL=$(cat ${SUBMIT_DIR}/../.resol)

NOTE: Be careful because this is ok for ORCA2, but if we use ORCA0.5 ore ORCA0.25 is NOT TRUE!!!!'''

go in modipsl/config/ORCA2_LIM/IGCM00 directory

cancel run.card.init and Job_Name files

and then do an ins_job again.

(does not still fixed in libIGCM_v1_2)

  • Error : Input files and BC for NEMO DO NOT EXIST (i.e.,,,, etc…)

IGCM_sys_Get, ERROR : regular file …../OCE/ORCA2_LIM/ DOES NOT EXIST .

To solve it
In this moment we've still to download on!reee512/NEMO/ORCA2_LIM_nemo_v3.tar file, and to untar it into the directory:


and where ${R_BC} is a variable that can be changed in modipsl/config/ORCA2_LIM/IGCM00/config.card file.

  • Error : IGCM_debug_Exit : IGCM_config_Initialize Error PeriodState : Fatal
To solve it
change line in modipsl/config/ORCA2_LIM/IGCM00/run.card


or rm modipsl/config/ORCA2_LIM/IGCM00/run.card

  • ls: ERROR: Cannot access stack: No such file or directory

in Script_Outputxxx:

( is it a problem with filemane? )

IGCM_sys_Put_Out : out_nemo /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N365d_5d/Out/N365d_5d_20030101_20031231_out_nemo
IGCM_sys_MkdirArchive : /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N365d_5d/Out
IGCM_sys_Chmod : 444 out_nemo
—Debug1—> OCE
—Debug1—> ICE
Files that will be deleted before next period-run :
ls: ERROR: Cannot access stack: No such file or directory

(no ok still with libIGCM_v1_2)

List of possible remarks and suggestions :

  • be careful at group (for example for a gensi one)

if we use a common account on CCRT we have to be sure that everygroup can have access to it, this is important for chmod 444, do another test!!!!!!

  • Every time that we change modipsl/config/ORCA2_LIM/IGCM00/config.card

we have to cancel modipsl/config/ORCA2_LIM/IGCM00/run.card.init

cancel modipsl/config/ORCA2_LIM/IGCM00/Job_Name

and run ./modipsl/util/ns_job script

  • To perform a second simulation after a run with PeriodState=Fatal

cancel modipsl/config/ORCA2_LIM/IGCM00/stack_error, modipsl/config/ORCA2_LIM/IGCM00/Script_Output_JobName

but do not cancel modipsl/config/ORCA2_LIM/IGCM00/run.card.init file, if not run.card file is not re-created.

( File run.card.init is created with ins_job script )

  • Does it exists a caption file for run.card.init options?

Or Is there a reference file?
In to run.card.init file who can we undesrtand syntax of? :



RebuildFrequency : how much has it? = PeriodLenght???

  • test tags/libIGCM_v1_2

Why defaul of config.card is Write Frequency?="5D" instead of 1M?

We've seen that with 1M monitoring and atlas are not possible.

  • test tags/libIGCM_v1_2

Is it still true?: To use libIGCM trunk version it is necessary to add in config.card in POST Session:


  • Change in nemo.card or in nemo.driver variable: ORCA_NPDT_JOUR and OPA_NPDT_JOURS

Done :

  • run on CCRT machines ( mercure )

Tests for Calendar Type, Output frequency and Restarts:

Simulation environment

Simulation duration = 5 year

PeriodLength = 1Y

timestep: rdt = 5760 equivalent to ORCA_NPDT_JOUR = 15

Output Frequency Output Frequency Output Frequency
5d 1M 1Y
Calendar Type 360d ok ok ok
Calendar Type 365d ok ko ok
Calendar Type leap ok ko to do

ko = there are only 11 output ( point how NEMO do averadge for every month )

NOTE: control in:

  • why in /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N360d_1m/OCE/Debug/*_ocean.output

Namelist namrun

job number no = 0

is EVER = 0????

Control if file name is correct!!!! (my job is N365d_1m)

Job Name versions calendar type
ONLY nemo_v3, ioipsl_v2_1_4, libIGCM_v1_2
option only(x) in diawri.F90
ONLYio4 nemo_v3, ioipsl_v2_1_4, libIGCM_v1_2 360d
dynamic nemo_v3, ioipsl_v2_1_4, libIGCM_v1_2
with monitorin01_nemo_ORCA2.cfg and monitornig01_lim2_ORCA2.cfg in SUBMIT_DIR: monitoring dynamic
io7nemo3 nemo_v3, ioipsl_v2_1_7, libIGCM trunk 360d
v3360d nemo_v3, ioipsl_v2_1_8, libIGCM_v1_2 360d
v31bok nemo_v3_1_beta, ioipsl_v2_1_8, libIGCM_v1_2 360d

To do list :

  • simulation=100Y ( ongoing with tags/libIGCM_v1_2 )
    and sortiee=1Y
  • monitoring : ongoing on mercure/brodie for experience=10Y, PeriodLenght=1Y, WriteFrequency=1M

'ok': monitoring dynamic ( reading nemo.card ⇒ doing monitoring )

NOTE: control mesure unity for monitoring_nemo_ORCA2.cfg for soicecov ( [0,1]: is it ok? )

  • atlas : ongoing on mercure/brodie for experience=10Y, PeriodLenght=1Y, WriteFrequency=1M

NOTE: changed COMP/nemo.card and set Output Files? and Post_1M_grid_T[U,V,W]
NOTE: changed COMP/lim2.card and set Output Files? and Post_1M_grid_T[U,V,W]

  • test mpi

NOTE files to change (or changed) for NEMO :

and modipsl/modeles/NEMO/WORK/domain.F90

  • modipsl/modeles/UTIL/fait_AA_make

difference with the old one:

in #- Compilation options —-

Committed: shade 10000 (for mercure)

  • modipsl/libIGCM/AA_job

difference with the old one:
#RESOL=$(cat ${SUBMIT_DIR}/../.resol)

  • modipsl/modeles/NEMO/WORK/domain.F90

difference with the old one:

in line 171

WRITE(numout,*) ' leap year calendar (0/1/30) nleapy = ', nleapy

  • modipsl/config/ORCA2_LIM/IGCM00/COMP/nemo.driver

difference with the old one:

added nleapy variable that corresponds to nbisex on opa.driver

# nleapy leap year calendar (0/1) (30 for 360d)

PAT_NLEAPY=$( supergrep nleapy ${SUBMIT_DIR}/PARAM/namelist )

in lines 141-151

case ${config_UserChoices_CalendarType} in

leap | | Gregorian)


in line 158

-e "s%${PAT_NLEAPY}% nleapy=${ORCA_NLEAPY}%" \

Appendix A - Dods directory:

  • to remove a directory from dods :

On Mercure to CCRT DOds directory is :


cd ${SAVE_DIR}/dods
dods_rm public/${LOGIN}/${TAG_NAME}/directory

Simona Flavoni last update: Timestamp?

Attachments (1)

Download all attachments as: .zip