{{{
#!html
<h1>The IPSLCM5 configuration </h1>
}}}
----
[[PageOutline(1-2,Table of contents,,numbered)]]

Persons in charge: Arnaud Caubel and Marie-Alice Foujols
----

# The IPSLCM5_v5 configuration #

IPSLCM5_v5 is '''the reference configuration of the coupled IPSL model''' for the CMIP5 simulations including atmosphere, land, ocean, sea ice and carbon cycle. This configuration includes :
 * the model components:
  * LMDZ, the atmospheric model ; 
  * NEMO, the ocean model including sea ice (LIM2) and marine biogeochemistry (PISCES) ; 
  * ORCHIDEE, the land model ;  
  * the OASIS coupler ;
 * the tools :
  * IOIPSL, the library dealing with the IPSL model I/O ;
  * XMLIOSERVER, the I/O server ; 
  * scripts to run the model and to perform post processing and the ksh functions library libIGCM

[[Image(IPSLCM5_v5.jpg, 360px)]]

# The components' origin #

The ocean configuration is ORCA2 : global ocean with a tripolar grid with one South pole, one North pole above Siberia and one North pole above northern America. The resolution is 2°. In the tropical region, the latitudinal resolution decreases to 1/2°. There are 31 vertical levels.

As a reminder, here are the versions of the components and tools.
{{{
vi modipsl/util/mod.def
#-H- IPSLCM5_v5  IPSLCM5_v5 coupled configuration
#-H- IPSLCM5_v5  CMIP5 version 08/12/2011
#-H- IPSLCM5_v5  with 5 NEMO sub-domains for vargas or titane by default
#-H- IPSLCM5_v5  NEMO svn branches/CMIP5_IPSL 3205
#-H- IPSLCM5_v5  XMLF90 svn trunk revision 297
#-H- IPSLCM5_v5  XMLIO_SERVER svn trunk revision 297
#-H- IPSLCM5_v5  IOIPSL/src svn tags/v2_2_1
#-H- IPSLCM5_v5  LMDZ5 trunk revision 1628
#-H- IPSLCM5_v5  ORCHIDEE version orchidee_1_9_5
#-H- IPSLCM5_v5  OASIS3 rev 1677 trunk
#-H- IPSLCM5_v5  IPSLCM5_v5 svn
#-H- IPSLCM5_v5  libIGCM trunk
#-M- IPSLCM5_v5  arnaud.caubel@lsce.ipsl.fr
#-C- IPSLCM5_v5  IOIPSL/tags/v2_2_1/src        HEAD         8  IOIPSL/src     modeles
#-C- IPSLCM5_v5  tags/ORCHIDEE_1_9_5/ORCHIDEE  HEAD        14  ORCHIDEE       modeles
#-C- IPSLCM5_v5  CPL/oasis3/trunk              1677         8  prism          .
#-C- IPSLCM5_v5  LMDZ5/trunk                   1628        11  LMDZ           modeles
#-C- IPSLCM5_v5  CONFIG/UNIFORM/v5/IPSLCM5_v5  HEAD         8  IPSLCM5_v5     config
#-C- IPSLCM5_v5  tags/libIGCM_v2.0_rc2               HEAD        10  libIGCM        .
#-C- IPSLCM5_v5  branches/CMIP5_IPSL/NEMO      3205         7  .              modeles
#-C- IPSLCM5_v5  branches/CMIP5_IPSL/UTIL      3405         7  .              modeles
#-C- IPSLCM5_v5  XMLF90                         297        12  .              modeles
#-C- IPSLCM5_v5  XMLIO_SERVER/trunk             297        12  XMLIO_SERVER   modeles
}}}

# Resolutions and configurations #
Three configurations are setup and used for CMIP5. They all have 39 vertical levels for the atmosphere component. The horizontal resolution of the atmosphere and the atmospheric physics differ: 
 * The IPSLCM5A configurations use the LMDZ4 physics (close to the one used for CMIP3). These configurations are setup and evaluated for 2 resolutions :
   * ORCA2xLMD9695-L39. The resolution of LMDZ is 96x95 (3,75° in longitude and 1,875° in latitude) with 39 vertical levels.
   * ORCA2xLMD144142-L39. The resolution of LMDZ is 144x142 (2,5° in longitude and 1,25° in latitude) with 39 vertical levels. Note that this resolution is numerically unstable. 
 * The IPSLCM5B configuration uses the LMDZ5 physics. It includes thermals and cold pools. This configuration is not entirely setup and it must not be used without the help of the LMD experts. Its usual resolution is ORCA2xLMD9695-L39.

Other resolutions exist as benchmarks but they are not setup.

## IPSLCM5A-LR ##

By default the coupled version is IPSLCM5A-LR.

## IPSLCM5A-MR ##

 * You must choose the resolution when compiling. To choose MR resolutions: {{{ gmake ORCA2xLMD144142-L39 }}}
 * change !TagName : `TagName=IPSLCM5A-MR` in `config.card` 
 * specify pmagic=-0.01 to be setup like IPSLCM5A-MR in CMIP5. Change `PARAM/config.def_preind`,  `PARAM/config.def_annuel`or `PARAM/config.def_actuel`
 * use a larger number of processors. If you use 5 processors for NEMO, increase this number to 53 processors. Change `JobNumProcTot=53` in `config.card` 
 * reduce the length of the time series. Use a  [http://forge.ipsl.jussieu.fr/igcmg/changeset/1961/CMIP5/historical/v5.historicalCMR4/COMP historical MR CMIP5] simulation as an example.
 * Be careful, the MR configuration is not stable. It can stop due to a numerical instability leading to temperature values that are too small at certain gridpoints. In those cases, restart the last period by choosing a Matsuno-type timestep. Change `ByPass_hgardfou_mats=n`in `COMP/lmdz.card`.

'''Summary of the difference between IPSLCM5A-LR and IPSLCM5A-MR'''

|| File || IPSLCM5A-LR || IPSLCM5A-MR|| Comments ||
|| PARAM/config.def_xxx || pmagic=0.0 || pmagic || Choose the pmagic parameter ||
|| PARAM/gcm.def ||gcm.def_96x95x39|| gcm.def_144x142x39 || PARAM file used ||
|| PARAM/namcouple || namcouple_ORCA2xLMD9695 || namcouple_ORCA2xLMD144142 || PARAM file used ||
|| COMP/*.card || || ChunckJobxD || reduce the TS length ||

## IPSLCM5B ##

 * Change !TagName : `TagName=IPSLCM5B`in `config.card`
 * Change `LMDZ_Physics=NPv3.1`in `COMP/lmdz.card`
 * The physics parameter are taken in the `PARAM/physiq.def_L39_NPv3.1` file

'''Summary of the differences between IPSLCM5A-LR and IPSLCM5B-LR'''

|| File || IPSLCM5A-LR || IPSLCM5A-MR|| Comments ||
|| PARAM/physiq.def|| PARAM/physiq.def_L39_AP || PARAM/physiq.def_L39_NPv3.1 || PARAM file used ||

# The experiments #

The EXPERIMENTS directory contains the benchmark experiments. They are stored in 3 subdirectories: LMDZ, LMDZOR and IPSLCM5.

Reminder: it is possible to perform LMDZ-type (atmosphere only) or LMDZOR-like (atmosphere and land) experiments with executables of this configuration. See [wiki:DocHconfigBlmdzor LMDZOR]

The experiments associated to IPSLCM5A are the following:
 * EXP00  : a present-day control experiment (pdControl).
 * piControl : a preindustrial control experiment (piControl). The outputs correspond to the CMIP5 requirements.
 * historical  : a historical experiment (1850-2005) with all necessary forcings.
 * Chistorical  : a historical experiment (1850-2005) with interactive CO2. It is in its final stage but not yet ready.
 * !CpiControl  : a preindustrial control experiment with interactive CO2. It is in its final stage but not yet ready.

## EXP00 ##

## piControl ##

### Main differences between piControl and EXP00 ###

|| File || EXP00 || piControl || Comments ||
|| {{{COMP/lim2.card}}} || NONE || Post_1D_icemod || No post processing for the 1D_icemod file (no mask, no TS) ||
|| {{{COMP/lmdz.card}}} || !ConfType=actuel || !ConfType=preind || Present-day or preindustrial forcings ||
|| PARAM/config.def || config.def_actuel || config.def_preind || PARAM file used||
|| || !OutLevel=low || !OutLevel=medium || Low or medium output level ||
|| PARAM/output.def || output.def_low || output.def_medium || PARAM file used ||
|| || LMDZ_NMC_daily=n || LMDZ_NMC_daily=y || daily NMC outputs||
|| || LMDZ_NMC_hf=n || LMDZ_NMC_hf=y || 6-h NMC outputs  ||
|| ||  || aerosols_11YearsClim_1995.nc|| ||
|| || climoz_LMDZ_1855_v2.nc || climoz_LMDZ_1995_v2.nc|| ||
|| {{{COMP/orchidee.card}}} || PFTmap_IPCC_2000.nc || PFTmap_IPCC_1850.nc || Vegetation map used ||
|| {{{COMP/pisces.card}}} || DBIO_ENABLE=n || DBIO_ENABLE=y || ||
|| {{{COMP/*}}} || ChunckJobxD=OFF || || deactivation of TS ||
|| {{{config.card}}} || !WriteFrequency="1M 1D" || !WriteFrequency="1M 1D HF" || No HF for ATM  ||
|| || !WriteFrequency="1M" ||!WriteFrequency="1M HF" || No HF for SRF ||


## historical ##

### Main differences between piControl and historical ###

|| File || piControl || historical || Comments ||
|| {{{COMP/lim2.card}}} || NONE || Post_1D_icemod || No post processing for the 1D_icemod file (no mask, no TS) ||
|| {{{COMP/lmdz.card}}} || !ConfType=preind || !ConfType=annuel || Preindustrial or annual forcings ||
|| || !OutLevel=medium || !OutLevel=high || Low or medium output level ||
|| || LMDZ_COSP_OK=n || LMDZ_COSP_OK=y || COSP activated||
|| || || LMDZ_COSP_daily1979=y || activated COSP daily outputs starting from 1979 ||
|| || aerosols_11YearsClim_1995.nc || aerosols_11YearsClim_${year}.nc, aerosols${year}.nc, 1:12: || annual aerosols: starting from the 1st period and every 12 period (month) [min]:[modulo:][max] ||
|| || || climoz_LMDZ_${year}.nc,          climoz_LMDZ.nc,     1:12: || Annual climoz  ||
|| || || CO2_1765_2005.txt || list of annual CO2 values ||
|| || || CH4_1765_2005.txt || list of annual CH4 values ||
|| || || N2O_1765_2005.txt || list of annual N2O values||
|| || || CFC11_1765_2005.txt || list of annual CFC11 values ||
|| || || CFC12_1765_2005.txt || list of annual CFC12 values ||
|| {{{COMP/opa9.card }}} || OUTPUT_LEVEL=2 || OUTPUT_LEVEL=3 || larger number of ocean outputs ||
|| {{{COMP/orchidee.card}}} || PFTmap_IPCC_1850.nc || PFTmap_1850to2005_AR5_LUHa.rc2/PFTmap_IPCC_${year_p1}.nc, PFTmap.nc, 12:12: || Vegetation map used ||
|| || ||  VEGET_UPDATE=1Y || update frequency of the vegetation map ||
|| || ||  LAND_COVER_CHANGE=y || update of the vegetation map  || 
|| {{{COMP/*}}} || || ChunckJobxD || activation of TS (OFF/xxY) ||
|| || || TimeSeriesVarsxD || Larger number of variables ||
|| {{{config.card}}} || !WriteFrequency="1M 1D HF" || !WriteFrequency="1M 1D HF HF3h HF3hm" || HF3h and HF3hm for ATM  ||

## Paléo. configurations ##

IPSLCM5A-LR has been used at LSCE for glacial climates (21 ka BP, contact Masa Kageyama or Olivier Marti), mid-Holocene (4kaBP, 6ka BP and 9ka BP, contact Olivier Marti or Pascale Braconnot) and for other ancient climates configurations.

## Chistorical ##

Equivalent to historical with interactive CO2 (contact Patricia Cadule or Laurent Bopp). The model is forced with estimations of fossil emissions during the historical period. The CO2 atmospheric concentration results from the equilibrium between prescribed emissions (fossil), carbon absorption by the ocean (PISCES) and the terrestrial biosphere (ORCHIDEE).

## !CpiControl ##

Equivalent to piControl with interactive CO2 (contact Patricia Cadule or Laurent Bopp). Control simulation.

# The initial states #

You must take the time to identify the initial states used by the model.
 * By default, the EXP00 and pdControl simulations have climatological initial states.
 * By default, the historical simulation is not performed because the initial states are not provided.
 * The carbon reservoirs equilibrium is a necessary and essential step to create the initial state for the biogeochemical cycles of the terrestrial biosphere (ORCHIDEE) and of the ocean (PISCES). This initial state of the cycles must be performed with an IPSLCM5 equilibrated climate corresponding to the period for which you wish to start the simulation.

## Which achieved simulations can you use as initial state? ##

A few files allowing you to start your own simulations have been copied at TGCC and at IDRIS :
 * they are stored in the shared directories, at TGCC  : /ccc/store/cont003/dsm/p86ipsl/IGCM_RESTART/IPSLCM5A/... and at IDRIS : /u/rech/psl/rpsl035/IGCM_RESTART/... and with the same names as in each IGCM_OUT directories
 * they are synchronized between TGCC and IDRIS. You can ask to have more restart files copied by writing to the platform-users _@_ipsl.jussieu.fr list
 * For IPSLCM5A, a restart date is 9 files and 700 MO.

Table of simulations and dates stored (extract) :
|| Simulations || Date || Original machine ||
|| piControl2 || 1849-12-31 || SX9 ||
|| piControl2 || 1859-12-31 || SX9 ||
|| piControl2 || 1869-12-31 || SX9 ||
|| piControl2 || 1985-12-31 || SX9 ||
|| !CpiControl01 || 1849-12-31 || SX9 ||
|| v3.historical1 || 2005-12-31 || SX9 ||
|| v3.historical2 || 2005-12-31 || SX9 ||
|| v3.historical3 || 2005-12-31 || SX9 ||
|| v3.historical4 || 2005-12-31 || SX9 ||


# The output levels #

## Be careful: the configurations can produce a very large number of outputs ##
We recommend you to adjust the output levels based on your scientific goal. An estimate of the volume and number of files produced is given below.

# What is the amount of outputs produced by those 3 experiments? #

## Outputs parameterizations ##

|| || '''EXP00''' || '''piControl''' || '''historical''' ||
|| {{{COMP/lmdz.card}}} || !OutLevel=low || !OutLevel=medium || !Outlevel=high ||

### Table of the amount of outputs for 10 years (in Go) ###

|| || '''low''' || || || || '''medium''' || || || ||'''high''' |||||||| Restart ||
|| || MO || DA || HF || TS || MO || DA || HF || TS ||MO || DA || HF || TS || ||
|| ATM || 15 || 11 || || 13 || 10 || 20 || 38 || 50 ||23 || 108 (+210 COSP) || 150 (+840 COSP) || 213 || 6 ||
|| CPL || 1 || 27 || || || 1 || 26 || ||  || 1 || 26 || ||  || 0,4 ||
|| OCE || 8 || 1 || || || 9 || 1 || || 10 || 9 || 51 || || 61 || 16 ||
|| ICE || 0,4 || || || || 0,4 || || || 0,4 || 0,4 ||2 || ||2 || 1 ||
|| MBG || 15 || || || || 20 || || || 0,7 ||20 || || || 24 || 40 ||
|| SRF || 1 || || || || 2  || || 10  || 4 || 2 || || || 0,6 || 5 ||
|| SBG || 5 || || || || 5 || || || 1,2 ||5 || || || 2 || 15 ||
|| '''Total''' || || || || || 47,4 || 47 || 48 || 66,2 || 61,4 || 187 || 150 || 302 || 84 ||
|| '''Grand Total''' || || || || ||  ||  ||  || '''210''' ||  ||  ||  || '''400''' (610 with COSP DA, 1450 with COSP HF) || ||

[[BR]]

## Dimension and number of files for a historical simulation: 156 years, !OutLevel=high, without ATLAS and MONITORING directories ##

The pack period is configured in {{{config.card}}}. A larger pack period allows you to reduce the number of files but in case your simulation stops, you must restart the whole period.

|| Resolution || || Pack for 120 months (10 years) || Pack for 60 months (5 years) ||
|| 96x95x39 || file number || 3852 || 4339 ||
|| || Restart file number || 1404 || 1404 ||
|| || Restart file space || 88 Go || 88 Go ||
|| || the largest || 65 Go || 32 Go ||
|| || medium || 3,5 Go || 3 Go ||
|| || Total space || 13 To || 13 To ||
|| 144x142x39 || file number || 3852 || 4339 ||
|| || Restart file number || 1404 || 1404 ||
|| || the largest || 148 Go || 73 Go ||
|| || medium || 7 Go || 6 Go ||
|| || Total space || 25 To || 25 To ||
|| || Restart file space || 100 Go || 100 Go ||

## Dimension and number of files for a historical simulation, for the ATLAS and MONITORING directories ##

|| Directory || File number || Size in GO ||
|| MONITORING	|| 399	|| 0,01 ||
|| ATLAS || 19395 || 4,60 ||

# Ressources on usual machines #

## Information obtained in 2012 for the computing time requested for 2013 ##
Reminder: [wiki:PerformancesIPSLCM5A The ressources needed for IPSL configurations including the IPSLCM5 coupled model configuration on the usual computing centers.]

## TGCC ##

### Bull Curie ###

The basic configuration uses 32 computing cores: 1 for the OASIS coupler, 5 for NEMO, and 26 for LMDZ. The configuration is regularly used and evaluated on this machine: [[http://webservices.ipsl.jussieu.fr/trusting/]]

### Bull Titane (this computing machine will be stopped in June 2013) ###

## IDRIS ##

### IBM : Ada ###

# Command summary #
Here are the commands you need to know if you want to retrieve and compile the IPSLCM5_v5 model and if you want to setup and run an EXP00/pdControl experiment:
{{{
#!sh
mkdir MONREPERTOIRE ; cd MONREPERTOIRE
svn_ano # svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl
cd modipsl/util
./model IPSLCM5_v5
cd ../config/IPSLCM5_v5
gmake # par défaut ORCA2xLMD9695-L39
cp EXPERIMENTS/IPSLCM5/EXP00/config.card .
vi config.card # modify JobName (at least) : MYJOBNAME
../../util/ins_job
cd MYJOBNAME
vi Job_MYJOBNAME # modify PeriodNb, adjust the time, ...
llsubmit Job_MYJOBNAME # IDRIS
ccc_msub  Job_MYJOBNAME # TGCC
}}}

# Evaluation #

Remember to evaluate the files produced by this simulation.