wiki:Doc/Config/LMDZOR_v6.1

Version 67 (modified by jgipsl, 2 years ago) (diff)

--

The LMDZOR_v6.1 configuration

LMDZOR_v6.1 is a configuration close the the coupled configuration IPSLCM6-LR used for CMIP6 simulations. The configuration exists in several versions LMDZOR_v6.1.x corresponding to the coupled configuration with the same suffix.

Jean-Zay/IDRIS computing center: Only version LMDZOR_v6.1.10 and higher can be used. See the section launching below to know how to modify the number of OMP before launching at Jean-Zay.

Person in charge: Josefine Ghattas


1. Description

LMDZOR_v6.1 configuration contains the atmospheric model LMDZ and the land surface model ORCHIDEE. This configuration exist with several versions depending on the version of the models used. All versions with suffix _v6.1.x corresponds to the coupled model IPSLCM6.1.x-LR used for CMIP6. LMDZOR_v6.2 and LMDZOR_v6.3 correspond to different working version with newer sources.

1.1. LMDZOR_v6.1

Different versions LMDZOR_v6.1.x corresponds to the coupled model IPSLCM6.1.x-LR with the same suffix number. These versions are used for CMIP6 simulations. See here the different available versions, the latest version is often recommended:

Name in modipsl Creation date LMDZ ORCHIDEE XIOS Comments
LMDZOR_v6.1.8 10th of January 2019 IPSLCM6.0.15 rev 3427 ORCHIDEE_2_0 rev 5661 xios-2.5 rev 1550 Mostly updates in diagnostics compared to LMDZOR_v6.1.5. Version used for scenario simulations.
LMDZOR_v6.1.5 29th of June 2018 IPSLCM6.0.15 rev 3353 ORCHIDEE_2_0 rev 5280 xios-2.5 rev 1550
LMDZOR_v6.1.2 23th of May 2018 IPSLCM6.0.15 rev 3315 ORCHIDEE_2_0 rev 5112 XIOS_DEV_CMIP6 rev 1475 First version used for CMIP6 simulations

In the table below you can see the sources used in different intermediate version used during the development of the model IPSLCM6 used for CMIP6 project. At that time, in modipsl, the target LMDZOR_v6 was used to extract the model and the different versions of the components were updated. This means if you extracted modipsl the 2 August 2017 and installed LMDZOR_v6 or if you did the same the 21 of December 2017, you did not get the same model. Since the 23 of May 2018, the target LMDZOR_v6 has been removed and newer version are added such as LMDZOR_v6.1.2, LMDZOR_v6.1.5 and LMDZOR_v6.1.8. They all correspond to the coupled model with the same suffix.

See below different successive versions with the name LMDZOR_v6:

Creation date Corresponding coupled model LMDZ ORCHIDEE XIOS IOIPSL libIGCM LMDZOR_v6 configuration files ORCHIDEE_OL configuration files modipsl Comments
21 feb 2018 IPSLCM6.0.15-LR IPSLCM6.0.15 rev 3224 trunk rev 5004 XIOS_DEV_CMIP6 rev 1428 tag v2_2_4 trunk rev 1424 rev 3593 trunk rev 5016 trunk rev 3597 New physics NPv6.0.15_T as default and NPv6.0.15_S as option in LMDZ
21 dec 2017 IPSLCM6.0.14-LR IPSLCM6.0.14 rev 3140 trunk rev 4857 XIOS_DEV_CMIP6 rev 1309 tag v2_2_4 trunk rev 1418 rev 3532 trunk rev 4857 trunk rev 3535 New physics NPv6.0.14ttop as default in LMDZ, new PFTmap files and some new parameters for ORCHIDEE, new tag IOIPSL
4 dec 2017 IPSLCM6.0.13-LR IPSLCM6.0.13 rev 3085 trunk rev 4695 XIOS_DEV_CMIP6 rev 1309 trunk rev 3374 trunk rev 1415 rev 3460 trunk rev 4695(set up still as 6.0.12) trunk rev 3495
20 sept 2017 IPSLCM6.0.12-LR IPSLCM6.0.12 rev 2982 trunk rev 4563 trunk rev 1210tag v2_2_2 trunk libIGCM 1411 rev 3376 trunk rev 4488 trunk rev 3377 New physics NPv6.0.12 in LMDZ, no changes in setup ORCHIDEE. New aerosols(except for amip) and ozone
2 august 2017 IPSLCM6.0.11-LR IPSLCM6.0.11 rev 2932 trunk rev 4470 trunk rev 1043tag v2_2_2 trunk libIGCM 1406 rev 3340 trunk rev 4488 trunk rev 3343 New physics NPv6.0.11 in LMDZ, soil freezing and zobler soil classification in ORCHIDEE
17 mai 2017 IPSLCM6.0.10-LR IPSLCM6.0.10 rev 2883 trunk rev 4261 trunk rev 1043tag v2_2_2 trunk libIGCM 1390 rev 3255 trunk rev 4339 trunk rev 3256 New physics NPv6.0.10, soil resistance activated in ORCHIDEE
IPSLCM6.0.8-LR IPSLCM6.0.8 rev 2791 trunk rev 4067 trunk rev 1043tag v2_2_2 trunk libIGCM 1379 rev 3141 trunk rev 4112 trunk rev 3150 New physics NPv6.0.8, new vegetation forcing file, new albedo background file, new options in Orchidee for stress and albedo
IPSLCM6.0.7-LR testing rev 2729 trunk rev 3934 trunk rev 965 tag v2_2_2 trunk libIGCM 1373 rev 3075 trunk rev 3947 trunk rev 3078 New physics NPv6.0.7, new 2D ozone forcing files
IPSLCM6.0.6-LR testing rev 2720 trunk rev 3823 trunk rev 965 tag v2_2_2 tag libIGCM_v2.8.2 rev 3052 trunk rev 3945 trunk rev 3048 New default options all as IPSLCM6.0.6: resolution 144x142x79, NPv5.80bz0, CWRR
IPSLCM6.0.5-LR testing rev 2595 trunk rev 3623 trunk rev 858 tag v2_2_2 trunk rev 1333 rev 2958 trunk rev 3761 trunk rev 2990 Default options as CMIP5 (*). This version of LMDZ can not be used with XIOS/trunk 965.
IPSLCM6.0.4-LR testing rev 2546 trunk rev 3525 trunk rev 858 tag v2_2_2 trunk rev 1330 rev 2938 trunk rev 3551 trunk rev 2917 Default options as CMIP5 (*)

(*) Default options for CMIP5 are : resolution 96x96x39, old physics in LMDZ and 2-layer hydrology in ORCHIDEE. These options can be changed.

See below in the section Quick installation how to extract an old version.

1.2. Main differences between LMDZOR_v6 and LMDZOR_v5, LMDZOR_v5.2

LMDZOR_v6 :

  • is adapted to the versions of LMDZ and ORCHIDEE used for CMIP6.
  • is only using XIOS to write diagnostics. All diagnostics are configured in the xml files read by XIOS. The use of XIOS in server mode (default set up) removes the need for rebuild.
  • is compiling the hybrid parallelization mode (mixed MPI-OPenMP) which means that more cores can be used during run time to reduce the elapsed computing time.
  • has a clearer method for changing parameter files : the drivers can only change parameters set equal to AUTO in the parameter files
    • variables marked as _AUTO_ in the parameter files can be changed by the users. They can not be changed again by the scripts (the drivers). For example this is the case for all variables concerning the output. The user can set the output level as she wants directly into the PARAM/file_def_...xml files and it can not be overwritten by the drivers.
    • variables marked as _AUTOBLOCKER_ must not be changed. If they are changed anyhow, the simulation will stop with an error message because the drivers are not allowed to make hidden modifications.

1.3. Compiling

Default compilation is 144x142x79 but other resolutions can be compiled. To compile 96x95x39 grid do as follow :

gmake LMD9695-L39

By default the compiling is done for hybrid MPI and OMP parallelism. Though it is not necessary to activate OMP threads during run time, its activation is possible.

1.4. Launching

The default set up is using 71 MPI * 8 OMP for the gcm.e and 1 MPI for the xios server. Following is set in config.card :

[Executable]
ATM= (gcm.e, lmdz.x, 71MPI, 8OMP)
SRF= ("", "")
SBG= ("", "")
IOS= (xios_server.exe, xios.x, 1MPI)

Using the resolution 96x95 it is not possible to use more than 31MP for the gcm.e. Following config.card setting is recommended for the resolution 96x95x39:

#D-- Executable -
[Executable]
ATM= (gcm.e, lmdz.x, 31MPI, 4OMP)
SRF= ("", "")
SBG= ("", "")
IOS= (xios_server.exe, xios.x, 1MPI)

1.4.1. Launching at IDRIS-JeanZay

Because of JeanZay architecture (computing nodes based on sockets of 20 cores), we advice you to use 2, 4, 5, 10 or 20 OpenMP threads for LMDZ-ORCHIDEE component. By default, the number of OpenMP threads defined in config.card is 8. To change that, you have to edit and modify config.card as follows :

[Executable]
#D- For each component, Real name of executable, Name of executable for oasis
ATM= (gcm.e, lmdz.x, 71MPI, 10OMP)

Do not forget to install a new Job (command ins_job) in order to take into account these modifications.

1.5. XIOS and output control

XIOS is the new component used for output writing. It consists of a library linked to the model and an external server executable (xios_server.exe). It can be launched in attached mode or in server mode. The default set up in LMDZOR_v6 is using the server mode with 1 MPI for the server. Two executables are launched in the default set up: gcm.e and xios_server.exe. No reconstruction is needed for the output files. The rebuild tool detects automatically that recosntruction is not needed.

LMDZOR_v6 is only using XIOS for writing diagnostic output files. The output level is managed by xml files. The files beginning with file_def....xml are controlling the variables and the files that are written. These files are stored in the model source directories, look into lmdz.card and orchidee.card to have the exact information.

A basic level of output control is done from config.card with the variables WriteFrequency. Finer control of the output must be done directly in the file_def_....xml files. Where it is set _AUTO_, it can be replaced manually if wanted.

For example in LMDZ/DefLists/file_def_histday_lmdz.xml :

        <file id="histday" name="histday" output_freq="1d" output_level="5" enabled="_AUTO_">

can be changed into

        <file id="histday" name="histday" output_freq="1800s" output_level="2" enabled=".TRUE.">

1.6. Parameter files

Most parameter file are found in PARAM/ directory as for the v5 configurations. But some files are now also copied from the model directories. This is the case for some of the xml files. In lmdz.card and orchidee.card you can see where the files are taken from.

In the parameter files, some of the variables will be modified after the job is submitted. This is done by the comp.driver: lmdz.driver, orchidee.driver, stomate.driver and xios.driver. All the variables that might be modified are set equal to _AUTO_ or _AUTOBLOCKER_. No variable can be modified without having this key word. The values are set according to the options from lmdz.card, orchidee.card, stomate.card and config.card.

A variable which is set to _AUTO_ can also be changed manually in the corresponding file in the PARAM directory. For this case, the driver can not change it again. Some variables are set equal to _AUTOBLOCKER_. They cannot be changed. If they are changed anyhow, the job will exit with an error message.

New physics in LMDZ:
Different versions of physics for LMDZ exist in PARAM directory. The choice is done from lmdz.card by setting the variable LMDZ_Physics. The physiq.def and gcm.def files are suffixed by the version of the physics.

CWRR 11-layer hydrology in ORCHIDEE:
Change between CWRR 11-layer(default) and Choisnel 2-layer in orchidee.card by setting DefSuffix=CWRR or DefSuffix=Choi. Note that this change implies to copy orchidee.def_CWRR (default) or orchidee.def_Choi. The differences between these files are more than just the hydrology. The choice DefSuffix=Choi are close to the version for CMIP5 and DefSuffix=CWRR is the current best choice for CMIP6.

1.7. COSP and NMC output

Default compiling is done with COSP. Options to activate COSP are available in lmdz.card but running with COSP output has not been tested.

NMC monthly output are activated by default.

2. Quick installation

# Install modipsl
svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl
cd modipsl/util

# Before next command, modify the file mod.def if you want to extract an old version (*)

# Download and compile configuration LMDZOR_v6.1.10
./model LMDZOR_v6.1.10
cd ../config/LMDZOR_v6
gmake      [or gmake LMD9695-L39]

# Experiment setup
cp EXPERIMENT/LMDZOR/clim_pdControl/config.card .     # => modify at least JobName=MyJobname, DateBegin, DateEnd in config.card
../../libIGCM/ins_job
cd MyJobname

# Modify default setup: 
# for example change physics in lmdz.card, or hydrology in orchidee.card
ccc_msub Job_MyJobname / llsubmit Job_MyJobname

2.1. Extract a different version

Modify the file util/mod.def if you do not want the default set up. This must be done at step (*) above, before the command ./model LMDZOR_v6. Look for the lines starting with #-C- LMDZOR_v6. Each of these lines describes the extraction of one component which is included in the configuration LMDZOR_v6. Change the revision number to correspond to the revision you need. See the section Different versions in the begging of this page for reference versions.

3. Simulations

4. How to reproduce a coupled simulation with the forced model ?

You have run a simulation with IPSLCM6 and you would like to launch the same with LMDZOR_v6 with limit conditions (SST and seaice) coming from the same or another IPSLCM6 simulation. In practice this is needed to estimate radiative forcing from MyIPSLCM6 experiment using SST and sea-ice from the same simulation or from a reference simulation.

For this you need to follow all these steps :

  • Create limit.nc using sst / sic and land-sea mask from your IPSLCM6 simulation, as described here
  • Create your new experiment directory
    cd modipsl/config/IPSLCM6
    cp -r MyIPSLCM6 MyLmdzor6
    cd MyLmdzor6
    rm -rf Script* run.card Job* Debug
    
  • modify config.card :
    • modify the JobName as you want
    • modify the TagName to LMDZOR
    • modify the Restart section to use Restart from MyIPSLCM6
    • remove or comment out all lines for the ocean component (OCE, MBG) and the coupler (CPL) in ListOfCompondents and Executable section :
      #========================================================================
      #D-- ListOfComponents -
      [ListOfComponents]
      #D- For each component, Name of component, Tag of component
      ATM= (lmdz, LMDZ4-dev)
      SRF= (orchidee, ORCHIDEE_1_9_5)
      SBG= (stomate, ORCHIDEE_1_9_5)
      #OCE= (opa9, CMIP5_IPSL)
      #ICE= (lim2, CMIP5_IPSL)
      #MBG= (pisces, CMIP5_IPSL)
      #CPL= (oasis, OASIS3-MCT)
      IOS= (xios, XIOS)
      
      #========================================================================
      #D-- Executable -
      [Executable]
      #D- For each component, Real name of executable, Name of executable for oasis
      ATM= (gcm.e, lmdz.x, 32MPI, 4OMP)
      SRF= ("" ,"" )
      SBG= ("" ,"" )
      #OCE= (opa, opa.xx, 31MPI)
      #ICE= ("" ,"" )
      #MBG= ("" ,"" )
      #CPL= ("", "" )
      IOS= (xios_server.exe, xios.x, 1MPI)
      
      
  • Add the path of your new limit.nc files in [BoundaryFiles] Section of lmdz.card
  • (If you are working with IPSLCM5A you need to put "type_ocean=force" in PARAM/run.def)
  • create your new job
    ../../../libIGCM/ins_job
    

5. How to make a simulation like CMIP5 using LMDZOR_v6?

See here how use the model LMDZOR_v6.1.8 with options as used for CMIP5.

  • Install and compile LMDZOR_v6.1.8 as usual
  • For ORCHIDEE
    • Use the parameter file orchidee.def_Choi as orchidee.def. In this file, Choisnel hydrology is set (HYDROL_CWRR = n) but also all other options to obtain a set up as used for CMIP5. For example following is set in orchidee.def_Choi: OK_FREEZE=n, ALB_BG_MODIS=n,OK_EXPLICITSNOW=n, DO_WOOD_HARVEST=n, ROUGH_DYN=n... To use this file, set in orchidee.card:
      DefSuffix=Choi
      
    • To have the same input vegetation map, then also modify
      • orchidee.def_Choi to read 13 pfts, change to have following:
        NVM=13
        PFT_TO_MTC=1,2,3,4,5,6,7,8,9,10,11,12,13
        
      • orchidee.card to read the vegetation map file from CMIP5. Change following:
        ${R_IN}/SRF/PFTMAPS/CMIP6/ESA-LUH2v2/historical/15PFT.v1/PFTmap_${year}.nc
        into
        ${R_IN}/SRF/PFTMAPS/CMIP5/PFTmap_1850to2005_AR5_LUHa.rc2/PFTmap_IPCC_${year}.nc
        
    • Note that the files woodharvest.nc, reftemp.nc, alb_bg.nc and cartepente2d_15min.nc will not be read using orchidee.def_Choi. They can be removed or left in orchidee.card.
  • For LMDZ:
    • Activate "old" physics by changing in lmdz.card:
      LMDZ_Physics=AP
      
    • Adapt input files...