TOC(heading=Using libIGCM library to run 50 years of ORCA2_LIM3_PISCES nemo_v4_0_alpha, depth=2, libIGCM_nemo_v_3_6/config.card_description)?

Using libIGCM library with ORCA2_LIM3_PISCES with nemo_v3_4_0_alpha (= dev_merge_2016) January 2017

Last edition: Wikinfo(changed_ts)? by Wikinfo(changed_by)?

Purposes

Through LibIGCM give an example of 50 years run of ORCA2_LIM3_PISCES configuration, with :

• Climatological and Interannual (years from 1948 to 2009) forcing files COREII
• outputs:
  • ocean: 1D 5D 1M 1Y
  • ice: 1M
  • biology: 1D 1M 1Y
• RESTARTS every year
• on Idris ADA machine

Official documentation of Libigcm is :

​https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/Doc

Quick start guide

https://forge.ipsl.jussieu.fr/nemo/wiki/Users/ReferenceConfigurations/ORCA2_LIM3_PISCES

• Create and go into working directory :

  cd $WORKDIR  ;  mkdir test_O2L3PIS_50years_merge2016 ; cd  test_O2L3PIS_50years_merge2016
  

• Define the following alias to use SVN :

  alias svn_ano='svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl'
  

• Extract modipsl (modipsl with LibIGCM)

  svn_ano
  

• Change mod.def: the model "NEMO_v5":

  cd modipsl/util 
  vi mod.def
  

  edit mod.def file at NEMO_v5 configuration: change the original with these lines:

  #---- NEMO  configurations
  
  #-H- NEMO_v5  NEMOGCM  in forced mode   dev_merge_2016  Compilation using FCM
  #-H- NEMO_v5  libIGCM tag libIGCM_v2.8.2
  #-M- NEMO_v5  simona.flavoni@locean.upmc.fr
  #-C- NEMO_v5  tags/libIGCM_v2.8.2                                  HEAD  10 libIGCM               .
  #-C- NEMO_v5  branches/2016/dev_merge_2016/NEMOGCM                                       HEAD  7  .                       modeles
  #-C- NEMO_v5  CONFIG/UNIFORM/v6/NEMO_v6                                                  HEAD  8  NEMO_v6                 config
  

• Extract the model "NEMO_v5":

  cd modipsl/util   
  ./model NEMO_v5
  

• To compile and run NEMO is needed to download and compile xios:
  • Download XIOS (branch 1.0 revision 703)
    (The documentation of xios is available here)
  • Compile XIOS (on ADA machine):

    cd $WORKDIR    
    svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/branchs/xios-1.0 XIOS -r 703
    cd XIOS
    ./make_xios --avail
    ./make_xios --arch X64_ADA --full --prod --job 8
    

• Compile ORCA2_LIM3_PISCES configuration (Using ./makenemo -h to see help ( to see list of compilers available, to see options,...))

  cd ../modeles/NEMOGCM/CONFIG
  ./makenemo -n ORCA2_LIM3_PISCES -m X64_ADA  -j8 
  #-j8 compile with 8 proc for compilation phase
  #-j0 option create only configuration directory but does not compile it 
  

• Copy executable nemo.exe (and xios_server.exe if needed in detached mode) in modipsl :

  cd ../../../bin
  ln -s ../modeles/NEMOGCM/CONFIG/ORCA2_LIM3_PISCES/BLD/bin/nemo.exe nemo.exe
  ln -s $WORKDIR/XIOS/bin/xios_server.exe xios_server.exe
  

  NOTA: YOU CAN WANT TO USE MORE THAN ONE EXECUTABLE: so you can do, for example,

  ln -s ../modeles/NEMOGCM/CONFIG/ORCA2_LIM3_PISCES/BLD/bin/nemo.exe nemo_orca2_lim3_pisces.exe
  ln -s ../modeles/NEMOGCM/CONFIG/ORCA2_LIM2_PISCES/BLD/bin/nemo.exe nemo_orca2_lim2_pisces.exe
  

  JUST BE CAREFUL to use SAME executable name in config.card

• This is the structure of config in modipsl:

   modipsl/config/NEMO_v6/
  	            GENERAL
                      EXPERIMENTS/
                           ORCA2_LIM3_PISCES/
                                           clim/
                                             config.card
                                             COMP/
                                                 lim3.card
                                                 pisces.card
                                                 opa9.card
  

• To create submission directory you have to copy config.card file in config/NEMO-v6 directory:

  cd ../config/NEMO_v6
  cp EXPERIMENTS/ORCA2_LIM3_PISCES/clim/config.card . 
  

• Editing at this point modipsl/config/NEMO_v6/config.card (according to iodef.xml file)
  See detailed page for more informations: page description of config.card?

IMPORTANT NOTE : in config.card BE CAREFUL TO THESE LINES:
Parameters Files

in opa.card

List= (${MODIPSL}/modeles/NEMOGCM/CONFIG/SHARED/namelist_ref             , namelist_ref       ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_${RESOL_OCE}${EXPTYPE}_cfg, namelist_cfg       ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/iodef.xml                          , iodef.xml          ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/domain_def.xml                     , domain_def.xml     ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/field_def.xml                      , field_def.xml      )

in lim3.card

      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_lim3_ref                  , namelist_ice_ref   ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_lim3_cfg                  , namelist_ice_cfg   )

in pisces.card

      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_top_ref                   , namelist_top_ref   ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_top_${RESOL_OCE}_cfg      , namelist_top_cfg   ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_pisces_ref                , namelist_pisces_ref), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_pisces_${RESOL_OCE}_cfg   , namelist_pisces_cfg)

here you're coping iodef.xml,field_def.xml and domain_def.xml for XIOS1

vi config.card  
# Modify  AT LEAST JobName=eOR2L3P

• Create Job : (using ins_job in modsipl/lib IGCM directory)

../../libIGCM/ins_job
ls modipsl/config/NEMO_v6
   GENERAL
        EXPERIMENTS
        eOR2L3P  #new submission directory, config card is now HERE

• LOOK AT all parameters of the Job you've created :

NOTE : VERY VERY IMPORTANT: increase size of memory:

• by default on ada is 3.5Gb ==> increase it to 5Gb (maximum is 20,... but 20 s really not necessary; for more information "news class" command on ada:

# @ as_limit = 5gb

• by default is 1 hour (@ wall_clock_limit = 1:00:00), but for ORCA2-LIM3-PISCES is about ???? 1:45 on 32 cpus for 1 year ==>increas it to @ wall_clock_limit = 1:30:00

  cd eOR2L3P
  vi Job_eOR2L3P
   # @ wall_clock_limit = 2:00:00
  

• Submit Job :

  llsubmit Job_eOR2L3P
  

If you want to switch from climatological to interannual see page description of config.card