Version 21 (modified by flavoni, 2 years ago) (diff)

Using libIGCM library to run 50 years of ORCA2_LIM3_PISCES nemo_v4_0_alpha

    Error: Page libIGCM_nemo_v_3_6/config.card_description does not exist

Using libIGCM library with ORCA2_LIM3_PISCES with nemo_v3_4_0_alpha (= dev_merge_2016) January 2017

Last edition: 09/10/18 17:59:44 by nicolasmartin

Purposes

Through LibIGCM give an example of 50 years run of ORCA2_LIM3_PISCES configuration, with :

  • Climatological and Interannual (years from 1948 to 2009) forcing files COREII
  • outputs:
    • ocean: 1D 5D 1M 1Y
    • ice: 1M
    • biology: 1D 1M 1Y
  • RESTARTS every year
  • on Idris ADA machine

Official documentation of Libigcm is :

https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/Doc

Quick start guide

https://forge.ipsl.jussieu.fr/nemo/wiki/Users/ReferenceConfigurations/ORCA2_LIM3_PISCES

  • Create and go into working directory :
    cd $WORKDIR  ;  mkdir test_O2L3PIS_50years_merge2016 ; cd  test_O2L3PIS_50years_merge2016
    
  • Define the following alias to use SVN :
    alias svn_ano='svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl'
    
  • Extract modipsl (modipsl with LibIGCM)
    svn_ano
    
  • Change mod.def: the model "NEMO_v5":
    cd modipsl/util 
    vi mod.def
    
    edit mod.def file at NEMO_v5 configuration: change the original with these lines:
    #---- NEMO  configurations
    
    #-H- NEMO_v5  NEMOGCM  in forced mode   dev_merge_2016  Compilation using FCM
    #-H- NEMO_v5  libIGCM tag libIGCM_v2.8.2
    #-M- NEMO_v5  simona.flavoni@locean.upmc.fr
    #-C- NEMO_v5  tags/libIGCM_v2.8.2                                  HEAD  10 libIGCM               .
    #-C- NEMO_v5  branches/2016/dev_merge_2016/NEMOGCM                                       HEAD  7  .                       modeles
    #-C- NEMO_v5  CONFIG/UNIFORM/v6/NEMO_v6                                                  HEAD  8  NEMO_v6                 config
    
  • Extract the model "NEMO_v5":
    cd modipsl/util   
    ./model NEMO_v5
    
  • To compile and run NEMO is needed to download and compile xios:
    • Download XIOS (branch 1.0 revision 703)
      (The documentation of xios is available here)
    • Compile XIOS (on ADA machine):
      cd $WORKDIR    
      svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/branchs/xios-1.0 XIOS -r 703
      cd XIOS
      ./make_xios --avail
      ./make_xios --arch X64_ADA --full --prod --job 8
      
  • Compile ORCA2_LIM3_PISCES configuration (Using ./makenemo -h to see help ( to see list of compilers available, to see options,…))
    cd ../modeles/NEMOGCM/CONFIG
    ./makenemo -n ORCA2_LIM3_PISCES -m X64_ADA  -j8 
    #-j8 compile with 8 proc for compilation phase
    #-j0 option create only configuration directory but does not compile it 
    
  • Copy executable nemo.exe (and xios_server.exe if needed in detached mode) in modipsl :
    cd ../../../bin
    ln -s ../modeles/NEMOGCM/CONFIG/ORCA2_LIM3_PISCES/BLD/bin/nemo.exe nemo.exe
    ln -s $WORKDIR/XIOS/bin/xios_server.exe xios_server.exe
    
    NOTA: YOU CAN WANT TO USE MORE THAN ONE EXECUTABLE: so you can do, for example,
    ln -s ../modeles/NEMOGCM/CONFIG/ORCA2_LIM3_PISCES/BLD/bin/nemo.exe nemo_orca2_lim3_pisces.exe
    ln -s ../modeles/NEMOGCM/CONFIG/ORCA2_LIM2_PISCES/BLD/bin/nemo.exe nemo_orca2_lim2_pisces.exe
    
    JUST BE CAREFUL to use SAME executable name in config.card
  • This is the structure of config in modipsl:
     modipsl/config/NEMO_v6/
    	            GENERAL
                        EXPERIMENTS/
                             ORCA2_LIM3_PISCES/
                                             clim/
                                               config.card
                                               COMP/
                                                   lim3.card
                                                   pisces.card
                                                   opa9.card
    
  • To create submission directory you have to copy config.card file in config/NEMO-v6 directory:
    cd ../config/NEMO_v6
    cp EXPERIMENTS/ORCA2_LIM3_PISCES/clim/config.card . 
    
  • Editing at this point modipsl/config/NEMO_v6/config.card (according to iodef.xml file)
    See detailed page for more informations: page description of config.card?

IMPORTANT NOTE : in config.card BE CAREFUL TO THESE LINES:
Parameters Files

in opa.card

List= (${MODIPSL}/modeles/NEMOGCM/CONFIG/SHARED/namelist_ref             , namelist_ref       ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_${RESOL_OCE}${EXPTYPE}_cfg, namelist_cfg       ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/iodef.xml                          , iodef.xml          ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/domain_def.xml                     , domain_def.xml     ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/field_def.xml                      , field_def.xml      )

in lim3.card

      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_lim3_ref                  , namelist_ice_ref   ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_lim3_cfg                  , namelist_ice_cfg   )

in pisces.card

      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_top_ref                   , namelist_top_ref   ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_top_${RESOL_OCE}_cfg      , namelist_top_cfg   ), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_pisces_ref                , namelist_pisces_ref), \
      (${SUBMIT_DIR}/PARAM/v4_0_alpha/namelist_pisces_${RESOL_OCE}_cfg   , namelist_pisces_cfg)

here you're coping iodef.xml,field_def.xml and domain_def.xml for XIOS1

vi config.card  
# Modify  AT LEAST JobName=eOR2L3P
  • Create Job : (using ins_job in modsipl/lib IGCM directory)
../../libIGCM/ins_job
ls modipsl/config/NEMO_v6
   GENERAL
        EXPERIMENTS
        eOR2L3P  #new submission directory, config card is now HERE
  • LOOK AT all parameters of the Job you've created :

NOTE : VERY VERY IMPORTANT: increase size of memory:

  • by default on ada is 3.5Gb =⇒ increase it to 5Gb (maximum is 20,… but 20 s really not necessary; for more information "news class" command on ada:

# @ as_limit = 5gb

  • by default is 1 hour (@ wall_clock_limit = 1:00:00), but for ORCA2-LIM3-PISCES is about ???? 1:45 on 32 cpus for 1 year =⇒increas it to @ wall_clock_limit = 1:30:00
    cd eOR2L3P
    vi Job_eOR2L3P
     # @ wall_clock_limit = 2:00:00
    
  • Submit Job :
    llsubmit Job_eOR2L3P
    

If you want to switch from climatological to interannual see page description of config.card