ENSEMBLE: fluxnet sites ​http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation

Figure shows the sequence of simulations the FLUXNET configuration will produce in a grey box. Prognostic nutrient cycles have to be activated in a three steps. First, a FLUXNET simulation with imposed N and P tissue concentration is done (C-only). Second, a simulation with full-dynamics N cycle but prescribed tissue P concentration (CN) -or - a simulation with full dynamic N & P cycles (CNP). One can skip the second step in the scheme and directly initialize the full flexible CNP stoichiometry.

---

---

WARNING: The FLUXNET configuration is very rigid compared to the other configurations. You should follow exactly what is written below. The strategy should be to minimize any personalization of the paths or settings.

---

---

0.0 in case you do not use the ORCHIDEE-CN-P code

The forcing of many FLUXNET sites has a duration of less than 7 years. Per default the variable spinup_period in ORCHIDEE is increased to period larger than 6. This automatic correction doesn't work with the FLUXNET configuration. It will prevent the analytical spinup in the FOR-X simulation. Therefore you need to deactivate the automatic correction in src_stomate/stomate.f90:

        ELSE IF (spinup_period <= 6) THEN
           ! Adjust to bigger period. The period must be a multiple of the original period.
-          WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be adjusted.'
-          spinup_period = spinup_period*(INT(9/spinup_period)+1)
+      !DSG    WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be adjusted.'
+          WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be not adjusted.'
+      !DSG:    spinup_period = spinup_period*(INT(9/spinup_period)+1)

then, re-compile the model.

---

0.1 deactivate BigBrother? in libIGCM when running on curie

You should deactivate the BigBrother? on curie. The model runs also with BigBrother? but thee curie staff asked us not to use it.

You have to set the following

BigBrother=false

in all of the listed files here(*):

modipsl/libIGCM/AA_job

modipsl/config/ORCHIDEE_OL/SPINUP/Job_Spinup

modipsl/config/ORCHIDEE_OL/SPINUP/SUBJOB/OOL_SEC_STO/Job_AUTO

as well as in the Job_x you are going to create in the following (Don't worry: I will mention to deactivate BigBrother? later again).

(*) in case Job_* files listed do not exist you have to do the following:

cd modipsl/config/ORCHIDEE_OL/SPINUP
../../../libIGCM/ins_job

1.1 create new experiment folder

cd ../../config/ORCHIDEE_OL/
cp -fr ENSEMBLE/ ENSEMBLE_CNP/
cd ENSEMBLE_CNP/

1.2 tell libIGCM about the new input files

For now, I store all files needed on the IMBALANCE-P disk on obelix (you find the path below). Please ensure you are using the newest revision.

The configuration which are used are stored in ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/ so, do as in 4.1.2: change the path of soil_param.nc and add the new files

../SPINUP/SUBJOB/OOL_SEC_STO/COMP/sechiba.card:

(/home/surface3/common/CNP_files/revXX/soils_param.nc, .), \
(/home/users/dgoll/ORC_data/N_deposition/LMDzINCA_DEP_2000.nc, .), \

../SPINUP/SUBJOB/OOL_SEC_STO/COMP/stomate.card:

ListNonDel= (${R_IN}/SRF/reftemp.nc, .), \
(/home/surface3/common/CNP_files/revXX/USDA_SoilSuborder.nc, .), \
(/home/surface3/common/CNP_files/revXX/lithology.nc, .), \
(/home/users/dgoll/ORC_data/N_deposition/LMDzINCA_DEP_2000.nc, .), \
(/home/surface3/common/CNP_files/revXX/deposition.nc, .)

REMARK: make sure there are no blanks behind the "\"!

REMARK: Although the file with N deposition is used in sechiba we add the file also to the stomate.card. This is needed to ensure it is copied when we restart from a simulation with impose_cn. In that case, the N deposition variable will not be present in the sechiba_restart_out.nc but the libIGCM assumes all sechiba variables are present.

WARNING: the deposition file is a static field at the moment; it does not include N deposition either. At the moment N deposition is set to P deposition in the ORCHIDEE code. Anyway N deposition is not accounted for in the calculations for the N & P cycle for now.

exchange the spinup.driver file in ../SPINUP/COMP/ with the new modified driver from NV.

cp /home/surface3/common/CNP_files/revXX/spinup.driver ../SPINUP/COMP/

copy a run.def with parameter values for the ORCHIDEE-CAN components. Make sure you are using the newest version here:

cp /home/surface3/common/CNP_files/revXX/run.def PARAM/run.def

1.3 adjust the libIGCM

1.3.1 PARAM/run.def

make sure the variables to turn on/off the nutrients cycles are in PARAM/run.def:

# switch for N cycle
STOMATE_OK_NCYCLE=y
# switch for P cycle
STOMATE_OK_PCYCLE=y
# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=y
# switch to either impose N content of vegetation (to surpass P limitation):
STOMATE_IMPOSE_NP=y
# do not fake the initial height (DOFOCO parameter)
LD_FAKE_HEIGHT=n

Nammonium_FILE=LMDzINCA_DEP_2000.nc
Nnitrate_FILE=LMDzINCA_DEP_2000.nc
Nnitrate_VAR = NOY
Nammonium_VAR = NHX


Nfert_FILE=NONE
Nbnf_FILE=NONE
NINPUT_UPDATE=-1

if you want to debug the model, set printlev to 4 for full debug information in PARAM/run.def:

PRINTLEV=4

1.3.2 PARAM/orchidee.default (optional)

you can add defaults for the nutrient switches into PARAM/orchidee.default

# STOMATE_OK_NCYLE ([FLAG]) :  Activate the nitrogen cycle {OK_STOMATE}
STOMATE_OK_NCYCLE = n

# STOMATE_OK_PCYLE ([FLAG]) :  Activate the phosphorus cycle {OK_STOMATE}
STOMATE_OK_PCYCLE = n

# STOMATE_MASS_CONSERVATION ([FLAG]) : activate mass conservation checks which force stop if violated {OK_STOMATE}
STOMATE_MASS_CONSERVATION = n

# STOMATE_DSG_DEBUG ([FLAG]) : activate checks for stoichiometry and negative pools {OK_STOMATE}
STOMATE_DSG_DEBUG = n

# LD_FAKE_HEIGHT ([FLAG]) : fakes height of vegetation (ORCHIDEE-CAN related) {OK_STOMATE}
LD_FAKE_HEIGHT = n

REMARKS: Do not enable XIOS, it is per default disabled in "PARAM/orchidee.default".

1.3.3 fluxnet.card

WARNING: Do not enable river routing in site scale simulation, is per default disabled in "fluxnet.card"

WARNING: make sure that the variable SPINUP_ANALYTIC is not present in any section of fluxnet.card (per default SPINUP_ANALYTIC can be found twice in the fluxnet.card. Remove both entries. )

# in case this variable is set to "n" or "y" the configuration will run but the analytic spinup is either constantly on/off.
#SPINUP_ANALYTIC=n

Modify the configuration of the different phases in the FLUXNET simulations. These variables were originally used for the outdated spinup method using FORCESOIL. Thus the variable names are misleading. Depending on the simulations (C-only, CN, CNP) the variables have to set differently.

[SPINUP]
# Initialisation for spin-up :
# orchidee with sechiba alone (!!! if ok_stomate == n !!!)
duree_nostomate=0
#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
duree_inistomate=250
# teststomate (only if duree_inistomate > 0); DSG: don't touch this one
duree_offlineini=0

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=3
# ... for duree_sechiba years
duree_sechiba=15
# teststomate; DSG: don't touch this one
duree_stomate=0
# forcesoil; DSG: don't touch this one
duree_carbonsol=1

# Final run after the analytical spinup
# This last parameter must be non-zero.
duree_final=250

1.3.1 C-only simulations

All parameter settings above are for a C-only

1.3.2 CN simulations

Here I specify the modification you have to the do the setting of the C-only simulation to start a CN simulation. You need to have restart files from a C-only simulation.

modify in fluxnet.card:

#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
# this must be zero for use with restart files
duree_inistomate=0 

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=5
# ... for duree_sechiba years
duree_sechiba=50

modify PARAM/run.def:

# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=n

#this switch adds nutrients when immobilisation demands exceed supply 
# to avoid crashing the model). This problem of overuse happens regularly during spinups.
# a mass conserving solution would be better, but I have no solution at the moment.
SPINUP_CNP=y

Specify the path to the restart files for SECHIBA and STOMATE of the C-only FLUXNET simulations (you must have performed before); modify config.card:

#D-- SRF options passed to SuRFace (SECHIBA) subjobs
[SRF]
WriteFrequency="1800s"
Restart=y
#-- Last day of the experience used as restart
RestartDate=1997-12-31
# Define restart simulation name
RestartJobName=<JobName of C-only>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of CN>/ensemble/<JobName of C-only>
# Old component name for restart (if empty, use new name)
OldName=

#D-- Surface BioGeochemistry ( STOMATE ) subjobs
[SBG]
WriteFrequency="1M"
Restart=y
#-- Last day of the experience used as restart
RestartDate= 1900-12-31
# Define restart simulation name
RestartJobName= <JobName of C-only>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of C-only>/ensemble/<JobName of C-only>
# Old component name for restart (if empty, use new name)
OldName=

<JobName?> and <SpaceName?> can be found in the config.card of the C-only simulation.

copy the modified Job_ENSEMBLE:

cp /home/surface3/common/CNP_files/revXX/Job_ENSEMBLE_restart Job_ENSEMBLE

1.3.2 CNP simulations

Here I specify the modification you have to the do the setting of the C-only simulation to start a CNP simulation. You need to have restart files from a CN(!) simulation. modify in fluxnet.card:

#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
duree_inistomate=0 

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=5
# ... for duree_sechiba years
duree_sechiba=50

modify PARAM/run.def:

# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=n
STOMATE_IMPOSE_NP=n

#this switch adds nutrients when immobilisation demands exceed supply 
# to avoid crashing the model). This problem of overuse happens regularly during spinups
# a better solution would be good, but I have no solution at the moment.
SPINUP_CNP=y

Specify the path to the restart files for SECHIBA and STOMATE of the CN FLUXNET simulations (you must have performed before); modify config.card:

#D-- SRF options passed to SuRFace (SECHIBA) subjobs
[SRF]
WriteFrequency="1800s"
Restart= y # NVres Y
#-- Last day of the experience used as restart
RestartDate=1997-12-31
# Define restart simulation name
RestartJobName= <JobName of CN>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of CN>/ensemble/<JobName of CN>
# Old component name for restart (if empty, use new name)
OldName=

#D-- Surface BioGeochemistry ( STOMATE ) subjobs
[SBG]
WriteFrequency="1M"
Restart= y # NVres Y
#-- Last day of the experience used as restart
RestartDate= 1900-12-31
# Define restart simulation name
RestartJobName= <ID of CN>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of CN>/ensemble/<JobName of CN>
# Old component name for restart (if empty, use new name)
OldName=

<JobName?> and <SpaceName?> can be found in the config.card of the CN simulation.

copy the modified Job_ENSEMBLE:

cp /home/surface3/common/CNP_files/revXX/Job_ENSEMBLE_restart Job_ENSEMBLE

1.4 adjust the job as usual to your specific needs

1. modify config.card following ​http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation

give information on Job Name, Tag Name, Dates and Restart.

Make sure the path ARCHIVE is set to the same path in config.card ...

ARCHIVE=/home/surface3/$LOGIN

... and in the ../SPINUP/SUBJOB/OOL_SEC_STO/config.card

ARCHIVE=/home/surface3/$LOGIN

REMARK: it seems you are not allowed to put the ARCHIVE on obelix to /home/scratch01/$LOGIN.

2. modify run.def following ​http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation

3. modify fluxnet.card following ​http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation

1.5 create job

../../../libIGCM/ins_job

1.6 adjust job

on curie You must deactivate BigBrother? by setting the existing entry from true to false:

BigBrother=false

on obelix Change the queue to longp to increase the wallclock time (the default queue mediump is too short for this setting and you will exceed wall clock time). WARNING: In case you have set level_hist=10 (fluxnet.card) you need to take the extra long queue:

#PBS -q longp

If you want to debug on obelix: set the variable RUN_DIR_PATH to disk with enough space (the default path is not accessible for the user)

# for example on scratch01
RUN_DIR_PATH=/home/scratch01/dgoll/RUN_DIR

ACHTUNG: On curie, do not set RUN_DIR_PATH in the Job_*. If you do, the configuration will crash. You find the default path to the RUN_DIR_PATH in the section "3.2.8 check simulations".

1.7 start simulations

ksh
nohup ./Job_ENSEMBLE fluxnet > out_Job 2>&1 &

1.8 check simulations

to see the decomposition of the spatial domain among the processors check in the run dir "Load_balance_orchidee.dat" on curie The RUN_DIR is in

/ccc/scratch/cont003/dsm/<userid>/RUN_DIR

to check the status of the job; type

ccc_mstat -u <userid>

to delete a job; type

ccc_mdel <jobid>

1.8.1 what to do when a simulation crash

You cannot use clean_month and clean_year. You should remove the sites that crashed from the IGCM_OUT path and rerun the same experiment with a fluxnet.card containing only the sites which crashed.

1.9 post processing

ensure the Jobname and the path to the reference simulation is set in fluxnet.card; in case there is no reference set it to NONE

# History file of former ORCHIDEE runs (Reference) to compare with the current simulations
reference_base_path=/home/scratch01/dgoll/IGCM_OUT/OL2/DEVT/ensemble
reference_config=DSG04EnsCN

start the skript

ksh
nohup ./Job_Post_FLUXNET fluxnet > out_Post 2>&1 &

you will find the graphics in the experiment folder in the IGCM_OUT path. There is a pdf in the folder starting with "fluxnet_taylor_diff".

OPTIONAL FEATURES

specify the name of the exectuable of orchidee

If you want to use different version of the orchidee_ol which you store in modipsl/bin you have to do the following: add the following line to the [Executable] section in ../SPINUP/config.card

OOL=(orchidee_ol, orchidee_ol)

add the following line to Job_ENSEMBLE; make sure it is exactly at this spot (before # Give path for ...):

IGCM_card_WriteOption ${New_SUBMIT_DIR}/config.card Executable OOL ${config_Executable_OOL}

# Give path for all skeleton SubJob scripts to SPINUP directory
IGCM_card_WriteOption ${New_SUBMIT_DIR}/COMP/spinup.card UserChoices SubJobPath ${Skeleton_Path}

At the moment I cannot figure out how to set the variable ${config_Executable_OOL} according to the config.card; thus I hardcode the name in Job_ENSEMBLE for the moment:

IGCM_card_WriteOption ${New_SUBMIT_DIR}/config.card Executable OOL "(my_orchidee_ol, orchidee_ol)"

PROBLEM: although my_orchidee_ol is not present in modipsl/bin a orchidee_ol is present in the RUN_DIR.

output for the spinup simulations

In case you want to check if the spinup works well, increase output level of the spinup simulations & prevent libIGCM from removing these files. The files can be found in the respective site folders in the experiment folder.

modify in fluxnet.card and ensemble.card:

# level of the history files for each iteration of the SPINUP
level_hist=10
# level of the history files for the final iteration of the SPINUP
level_hist_final=10

# DEBUG mode for SPINUP 
#
# This mode keep all SPINUP directory in ARCHIVE
# If disable, all ARCHIVE is automaticly cleaned.
DEBUG_SPIN=y
# If you don't want to keep old spinup steps, but last one
CONSERVE=y

You find the files in the folder "JobName?"STOI in the IGCM_OUT path for the STOI simulation. Exchange "STOI" with "ORC-1", "FIN" etc for the simulations you want to check.