Version 266 (modified by flavoni, 12 years ago) (diff)

Last edited Timestamp?

Using libIGCM library with ORCA2_LIM


Purposes:


  • Document the using of libIGCM for NEMO, in configuration ORCA2_LIM



Quick start guide:


  • Define the following alias to use SVN :
     alias svn_ano='svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl'
    
  • Create and go into working directory :
     mkdir TRY ; cd TRY
    
  • Extract modipsl :
     svn_ano 
    
  • Extract NEMO :
     cd modipsl/util 
     ./model NEMO_IGCM 
    
  • Choosing ORCA2_LIM configuration :
     cd ../modeles/UTIL
     ./fait_config ORCA2_LIM
    
  • Install the makefiles :
     cd ../../util 
     ./ins_make
    
  • Compile :
     cd ../config/ORCA2_LIM 
     gmake
    
  • Submit Job :
     ( Editing at this point modipsl/config/ORCA2_LIM/IGCM00/config.card )
    
     ../../util/ins_job
     cd IGCM00 
     qsub Job_NL1  
    



Set up a simulation:


  • First of all see official documentation

http://forge.ipsl.jussieu.fr/libigcm/wiki/

  • All parameters in config.card file

In modipsl/config/ORCA2_LIM/IGCM00/config.card file you can set all parameters to manage your experience:

How to manage restart files ( in the same simulation )

in config.card you have to set duration of your experience and setting PeriodLenght parameter, so you create restart files.

( for example : if you have and experience of 10 years and a PeriodLenght of 1 year, you will have 10 restart files )

#— Début et fin de Job
#— "YYYY-MM-DD"
Date Begin?=2001-01-01
Date End?=2010-12-31
#============================
#— 1Y, 1M, 5D, 1D
Period Length?=1Y

Note : If you want to use restart data from another simulation ( already run not in this moment ) you have to manage "Restarts" part of config.card

#================
#D— Restarts -
[Restarts]

How to manage write output frequency

in config.card you have to set WriteFrequency of your experience

( for example : if you want output files every 1 month you have to put for Ocean

#===================
#D— OCE -
[OCE]
Write Frequency?="1M"

and if you want output files every 1 month you have to put for ICE

#===================
#D— ICE -
[ICE]
Write Frequency?="1M"

)

How to manage calendar type

in config.card you have to set CalendarType like 360d or leap or noleap

( for example : if you want 360 days you have to be careful of

Date End?=2010-12-30

because every month has 30 days

If you put for example Date End?=2005-12-31, you will have another run, of duration like your PeriodLenght )

How to manage rdt ( i.e. timestep )

in COMP/nemo.driver you have to change parameter : ORCA_NPDT_JOUR=15

( 15 corresponds to 86400s/5760s; it does mean that in one day you have 15 iterations; and it does mean that timestep is 5760)

How control all parameters in modipsl/config/ORCA2_LIM/IGCM00/PARAM/namelist

libIGCM change namelist's values but not in namelist's file, so it is possible to control all changes of namelist's parameters looking at :

{OUTPUT_DIR}/IGCM_OUT/ORCA2_LIM/N360d_5d/OCE/Debug/!JobName_date_date_ocean.output

How to activate Monitoring

in config.card file

#D- If you want to produce time series, this flag determines #D- frequency of post-processing submission Time Series Frequency?=NONE

you have to put you experience lenght. If you do an experience during 10 years you have to put:

Time Series Frequency?=10Y

How to do dynamic Monitoring (25/02/2009)

  • Copy original monitoring files in SUBMIT_DIR :
     cp ~/KEEPED_FILES/monitoring01_nemo_ORCA2.cfg $SUBMIT_DIR/.
     cp ~/KEEPED_FILES/monitoring01_lim2_ORCA2.cfg $SUBMIT_DIR/.
    
  • Copy monitoring.job in libIGCM :
     cp ~/KEEPED_FILES/monitoring.job modipsl/libIGCM/.
    
  • Control Time Series? in COMP/nemo.card and COMP/lim2.card:
     and if you add or remove somthin in nemo.card or lim2.card => it is necessary to add or remove in $SUBMIT_DIR/monitoring01_nemo[lim2]_ORCA2.cfg}}}
    
     '''How to activate Atlas'''
    
     in config.card file 
    
     #D- If you want to produce seasonal average, this flag determines
     #D- the period of this average
     SeasonalFrequency=NONE
     
     you have to put you experience lenght. If you do an experience during 10 years you have to put:[[BR]]
    
     SeasonalFrequency=10Y
    
      
     '''How to manage frequency of post processing''' 
    
     ( Note: in this moment is not possible to have atlas and monitoring of frequency different from '''1M''', if we want this we've to see it with Patrick Brockman!! )
    
     in COMP/nemo.card you have to change in [OutputFiles] chapter:
    
     Post_'''1M'''_grid_T (U,V,W)
    
     and you have to change [Post_'''1M'''_grid_T(U,V,W)] chapter.
    
     and in in COMP/lim2.card you have to change in [OutputFiles] chapiter:
     
     List=   (${PREFIX_NWRITE}_${DATE_OPA}_icemod.nc, ${R_OUT_ICE_NWRITE}/${PREFIX}_${WF1}_icemod.nc, Post_'''1M'''_icemod)
     
      and you have to change [Post_'''1M'''_icemod] chapter.
    
     * '''How to make a parallel run '''
    
     first of all add a cpp key: 
     
     http://www.nemo-ocean.eu/index.php//Using-NEMO/User-Guides/Advanced/How-to-add-modify-new-modules-or-new-cpp-keys
    
     and then see: http://www.nemo-ocean.eu/index.php//Using-NEMO/User-Guides/Advanced/How-to-set-up-one-simulation#eztoc1210_5
    
     * '''How to have mail of end of Job, sur ccrt's machines'''
    
     on mercure: in Job_Name: #PBS -M name@mail
     
     on platine: In Job_Name: #BSUB -u name@mail
    
     * '''How to activate particular key cpp '''
    
     (if not for example we have this error:
    
     WARNING : IGCM_sys_Put_Out xxx_trends.nc DOES NOT EXIST [[BR]]
     WARNING : IGCM_sys_Put_Out xxx_diagap.nc DOES NOT EXIST [[BR]]
     WARNING : IGCM_sys_Put_Out output.abort.nc DOES NOT EXIST )
     
    [[BR]]
    [[BR]]
    
    == Run only post processing: ==
    
    [[BR]]
       
        * '''If you have already run a job and you need only to run post processing'''
    
     you can do it in 3 ways:
    
     see: http://forge.ipsl.jussieu.fr/libigcm/wiki/libIGCM/DocUtilisateur/FAQ
    
     in particular: http://forge.ipsl.jussieu.fr/libigcm/wiki/libIGCM/DocUtilisateur/FAQ#Commentrelancerlespost-traitements%C3%A0partirdelafrontale
    
     and 
     
     http://forge.ipsl.jussieu.fr/libigcm/wiki/libIGCM/DocUtilisateur/FAQ#Commentrelancerlespost-traitements%C3%A0partirdelamachinedecalcul
    
     for example ...................... FINIRE
     
     in Job_Name you have to change:
     
      #PBS -l cputim_job=4:00:00  # Limite temps [[BR]]
      DRYRUN=3
      
    
    [[BR]]
    [[BR]]
    
    == List of possible errors in modipsl/config/ORCA2_LIM/IGCM00/Script_Output_''!JobName'' : ==
    
    [[BR]]
    
        * '''Error : TERM = (vt100) ${HOME:-.}/.profile: .bashrc:  not found'''
        * '''Error : ksh: D-----------------------------------==:  not found'''
    
    
     it can depends on '''.profile''' file.
    
     To solve it::
     .profile file it can be like:
    
     «# .profile  (idris) [[BR]]
     # Fichier d environnement a charger sur la frontale TX7 unix LINUX [[BR]]
     # Vous pouvez mettre a jour le fichier .profile_tx [[BR]]
     if [ !`hostname!` = "brodie" -a -f $HOME/.profile_tx ]; then [[BR]]
     . $HOME/.profile_tx [[BR]]
     # Fichier d environnement a charger sur les noeuds SX8 unix SUPER-UX [[BR]]
     # Vous pouvez mettre a jour le fichier .profile_sx [[BR]]
     elif [ !`hostname!` != "brodie" -a -f $HOME/.profile_sx ]; then [[BR]]
     . $HOME/.profile_sx [[BR]]
     else [[BR]]
     echo [[BR]]
     fi »
    
        * '''Error :  cat: cannot open ........./modipsl/config/ORCA2_LIM/IGCM00/../.resol ''' :
    
    
     it is due to modipsl/libIGCM/AA_job file.
    
     To solve it::
     change in this way:
    
     #-- Resolution FLAG for oasis input [[BR]]
     #RESOL=$(cat ${SUBMIT_DIR}/../.resol) [[BR]]
     RESOL=ORCA2
    
     '''NOTE: Be careful because this is ok for ORCA2, but if we use ORCA0.5 ore ORCA0.25 is NOT TRUE!!!!!'''
    
     
     go in modipsl/config/ORCA2_LIM/IGCM00 directory 
    
     cancel '''run.card.init''' and '''Job_''Name''' files
     
     and then do an '''ins_job''' again.
    
     (does not still fixed in libIGCM_v1_'''2''')
    
    
        * '''Error : Input files and BC for NEMO DO NOT EXIST (i.e. sst_data.nc, sss_data.nc, geothermal_heating.nc, bathy_level.nc, etc...)'''
    
     IGCM_sys_Get, ERROR : regular file ...../OCE/ORCA2_LIM/bathy_level.nc DOES NOT EXIST .
    
     To solve it::
     In this moment we've still to download on gaya.idris.fr:~/!reee512/NEMO/ORCA2_LIM_nemo_v3.tar file, and to untar it into the directory:
    
     ${R_BC}/OCE/ORCA2_LIM
    
     and where  ${R_BC} is a variable that can be changed in modipsl/config/ORCA2_LIM/IGCM00/config.card file.
    
        * '''Error : IGCM_debug_Exit :  IGCM_config_Initialize Error !PeriodState :  Fatal '''
    
     To solve it::  
     change line in modipsl/config/ORCA2_LIM/IGCM00/run.card
    
     !PeriodState=Fatal [[BR]]
     in  [[BR]]
     !PeriodState=
    
     or rm modipsl/config/ORCA2_LIM/IGCM00/run.card
    
     * '''ls: ERROR: Cannot access stack: No such file or directory'''
     
     in Script_Outputxxx: 
     
     ( '''is it a problem with filemane'''? )
     
     IGCM_sys_Put_Out : out_nemo /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N365d_5d/Out/N365d_5d_20030101_20031231_out_nemo [[BR]]
     IGCM_sys_MkdirArchive : /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N365d_5d/Out [[BR]]
     IGCM_sys_Chmod : 444 out_nemo [[BR]]
     --Debug1--> OCE [[BR]]
     --Debug1--> ICE [[BR]]
     Files that will be deleted before next period-run : [[BR]]
     ls: ERROR: Cannot access stack: No such file or directory [[BR]]
     -r--r--r--   N365d_5d_00016425_restart.nc [[BR]]
     -r--r--r--   N365d_5d_00016425_restart_ice.nc [[BR]]
    
     (no ok still with libIGCM_v1_'''2''')
    
     
    [[BR]]
    [[BR]]
    
    == List of possible remarks and suggestions : ==
    
    [[BR]]
     
        * '''be careful at group (for example for a gensi one)'''
    
     if we use a common account on CCRT we have to be sure that everygroup can have access to it, this is important for chmod 444, do another test!!!!!!
       
       
        * '''Every time that we change modipsl/config/ORCA2_LIM/IGCM00/config.card'''
    
     we have to cancel modipsl/config/ORCA2_LIM/IGCM00/run.card.init
    
     cancel modipsl/config/ORCA2_LIM/IGCM00/Job_''Name'' 
    
     and run ./modipsl/util/ns_job script 
    
        * '''To perform a second simulation after a run with !PeriodState=Fatal''' 
    
     cancel modipsl/config/ORCA2_LIM/IGCM00/stack_error, modipsl/config/ORCA2_LIM/IGCM00/Script_Output_''!JobName''
    
     but do not cancel modipsl/config/ORCA2_LIM/IGCM00/run.card.init file, if not run.card file is not re-created.
    
     ( File run.card.init is created with ins_job script )
    
     * '''Does it exists a caption file for run.card.init options?''' 
    
     Or Is there a reference file?[[BR]]
     In to run.card.init file who can we undesrtand syntax of? :
    
     !TimeSeriesRunning=n [[BR]]
     !TimeSeriesCompleted= [[BR]]
    
     !SeasonalRunning=n  [[BR]]
    
     * '''test tags/libIGCM_v1_2'''
    
     Is it still true?:  To use libIGCM trunk version it is necessary to add in config.card in POST Session:
     
     !RebuildFromArchive=NONE
    
     * '''Change in nemo.card or in nemo.driver variable: ORCA_NPDT_JOUR and OPA_NPDT_JOURS'''
    
    [[BR]]
    [[BR]]
    
    == Done : ==
    
    [[BR]]
     
     * '''run on CCRT machines ( mercure )''' 
    
     Tests for Calendar Type, Output frequency and Restarts: [[BR]]
    
    '''Simulation environment''' 
    
    Simulation duration = '''5 year '''
    
    !PeriodLength = '''1Y'''
    
    timestep: rdt = '''5760''' equivalent to ORCA_NPDT_JOUR = '''15'''
    
    ||  || || '''Output Frequency'''  || '''Output Frequency''' || '''Output Frequency''' ||
    ||  || || 5d || 1M  || 1Y ||
    || '''Calendar Type    '''|| 360d || '''ok''' || '''ok''' || '''ok''' ||
    || '''Calendar Type    '''|| 365d || '''ok''' || '''ko''' || '''ok''' || 
    || '''Calendar Type    '''|| leap || '''ok''' || '''ko''' || to do ||
    
    ko = there are only 11 output ( point how NEMO do averadge for every month )
    
    '''NOTE: control in:'''
      
    - why in /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N360d_1m/OCE/Debug/*_ocean.output 
    
    Namelist namrun
               job number                      no        =   '''0'''
    
    is EVER = 0????
    
    '''N365d_1m_20011231_restart.nc'''
    
    Control if file name is correct!!!! (my job is N365d_1m)
    
    * Test result:[[BR]]
     http://dods.extra.cea.fr/data/flavoni/ORCA2_LIM/ [[BR]]
    
     ||'''Job Name''' || '''versions''' || '''calendar type''' ||
     || ONLY       || nemo_v3, ioipsl_v2_1_4, libIGCM_v1_2 [[BR]] option only(x) in diawri.F90  || 360d           ||
     || ONLYio4    || nemo_v3, ioipsl_v2_1_4, libIGCM_v1_2 || 360d             ||
     || dynamic    || nemo_v3, ioipsl_v2_1_4, libIGCM_v1_2 [[BR]] with monitorin01_nemo_ORCA2.cfg and monitornig01_lim2_ORCA2.cfg in SUBMIT_DIR: monitoring dynamic || 360d               ||
     || v3360d  || nemo_v3, ioipsl_v2_1_4, libIGCM_v1_2||360d             ||
     
    
    [[BR]]
    [[BR]]
    
    == To do list : ==
    
    [[BR]]
    
     * simulation=100Y ( ongoing with tags/libIGCM_v1_2 ) [[BR]]
     periodlenght=10y[[BR]]
     and sortiee=1Y[[BR]]
    
     * '''monitoring''' : ongoing on mercure/brodie for experience=10Y, !PeriodLenght=1Y, !WriteFrequency=1M
     
     'ok': monitoring dynamic ( reading nemo.card => doing monitoring )
    
     '''NOTE: control mesure unity for monitoring_nemo_ORCA2.cfg for soicecov ( ![0,1]: is it ok? )'''
    
     * '''atlas''' : ongoing on mercure/brodie for experience=10Y, !PeriodLenght=1Y, !WriteFrequency=1M
    
     NOTE: changed COMP/nemo.card and set OutputFiles and Post_''1M''_grid_T[U,V,W] [[BR]]
     NOTE: changed COMP/lim2.card and set OutputFiles and Post_''1M''_grid_T[U,V,W] [[BR]]
     * test mpi
     
    [[BR]]
    [[BR]]
    
    == NOTE files to change (or changed) for NEMO : ==
    
    [[BR]]
    
     * for 360d CalendarType changed:
    
     config/ORCA2_LIM/IGCM00/COMP/'''nemo.driver'''  [[BR]]
     and modipsl/modeles/NEMO/WORK/'''domain.F90'''
     
     * modipsl/modeles/UTIL/'''fait_AA_make'''
     
     difference with the old one: 
    
     in #- Compilation options ---
    
     Committed: shade 10000 (for mercure)
    
     * modipsl/libIGCM/'''AA_job'''
    
     difference with the old one: [[BR]]
     #RESOL=$(cat ${SUBMIT_DIR}/../.resol) [[BR]]
     '''RESOL=ORCA2'''
    
     * modipsl/modeles/NEMO/WORK/'''domain.F90'''
    
     difference with the old one:
    
     in line 171
    
     WRITE(numout,*) '           leap year calendar (0/1/'''30''')     nleapy    = ', nleapy
    
     * modipsl/config/ORCA2_LIM/IGCM00/COMP/'''nemo.driver'''
    
     difference with the old one:
     
     added nleapy variable that corresponds to nbisex on opa.driver
    
     # nleapy leap year calendar (0/1) (30 for 360d)
    
     PAT_NLEAPY=$( supergrep nleapy    ${SUBMIT_DIR}/PARAM/namelist )
    
     in lines 141-151
    
     case ${config_UserChoices_CalendarType} in [[BR]]
                   leap | | Gregorian)[[BR]]
                   ORCA_NLEAPY=1;;[[BR]]
                   noleap)[[BR]]
                   ORCA_NLEAPY=0;;[[BR]]
                   360d)[[BR]]
                   ORCA_NLEAPY=30;;[[BR]]
           esac[[BR]]
    
     in line 158
    
     -e "s%${PAT_NLEAPY}%       nleapy=${ORCA_NLEAPY}%"                    \
    
     
    [[BR]]
    [[BR]]
    
     Simona Flavoni last update:  [[Timestamp]]
    

Attachments (1)

Download all attachments as: .zip