= ENSEMBLE: fluxnet sites [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation] =
[[Image(ORC-CNP- report 201603A.png)]]
Figure shows the sequence of simulations the FLUXNET configuration will produce in a grey box. Prognostic nutrient cycles have to be activated in a three steps. First, a FLUXNET simulation with imposed N and P tissue concentration is done (C-only). Second, a simulation with full-dynamics N cycle but prescribed tissue P concentration (CN). Third, a simulation with full dynamic N & P cycles (CNP).

== 1.1 create new experiment folder ==
{{{
cd ../../config/ORCHIDEE_OL/
cp -fr ENSEMBLE/ ENSEMBLE_CNP/
cd ENSEMBLE_CNP/
}}}

== 1.2 tell libIGCM about the new input files ==
For now, I store all files needed on the IMBALANCE-P disk on obelix (you find the path below). Please ensure you are using the newest revision (rev00 is not fully tested).

The configuration which are used are stored in 
../SPINUP/SUBJOB/OOL_SEC_STO/COMP/
so, do as in 4.1.2: change the path of soil_param.nc and add the new files 

sechiba.card:
{{{
(/home/surface3/common/CNP_files/rev00/soils_param.nc, .), \
}}}
stomate.card:
{{{
ListNonDel= (${R_IN}/SRF/reftemp.nc, .) \	 
(/home/surface3/common/CNP_files/rev00/USDA_SoilSuborder.nc, .), \
(/home/surface3/common/CNP_files/rev00/lithology.nc, .), \
(/home/surface3/common/CNP_files/rev00/deposition.nc, .)
}}}
WARNING: the deposition file is a static field at the moment; it does not include N deposition either. At the moment N deposition is set to P deposition in the ORCHIDEE code. Anyway N deposition is not accounted for in the calculations for the N & P cycle for now.

exchange the spinup.driver file in ../SPINUP/COMP/ with the new modified driver from NV.
{{{
cp /home/surface3/common/CNP_files/rev00/spinup.driver ../SPINUP/COMP/
}}}


copy a run.def with all parameters needed for the ORCHIDEE-CAN parts of the model to PARAM, for example this one:
{{{
cp /home/surface3/common/CNP_files/rev00/run.def_NV2 PARAM/run.def
}}}

== 1.3 adjust the libIGCM  ==
add the variables to turn on/off the nutrients cycles  to PARAM/run.def:
{{{
# switch for N cycle
STOMATE_OK_NCYCLE=y
# switch for P cycle
STOMATE_OK_PCYCLE=y
# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=y
# switch to either impose N content of vegetation (to surpass P limitation):
STOMATE_IMPOSE_NP=y
# do not fake the initial height (DOFOCO parameter)
LD_FAKE_HEIGHT=n

Nammonium_FILE =NONE
Nnitrate_FILE =NONE
Nfert_FILE=NONE
Nbnf_FILE=NONE
NINPUT_UPDATE=-1
}}}
if you want to debug the model, set printlev to 4 for full debug information in PARAM/run.def:
{{{
PRINTLEV=4
}}}

you can add defaults for the nutrient switches into PARAM/orchidee.default
{{{
# STOMATE_OK_NCYLE ([FLAG]) :  Activate the nitrogen cycle {OK_STOMATE}
STOMATE_OK_NCYCLE = n

# STOMATE_OK_PCYLE ([FLAG]) :  Activate the phosphorus cycle {OK_STOMATE}
STOMATE_OK_PCYCLE = n

# STOMATE_MASS_CONSERVATION ([FLAG]) : activate mass conservation checks which force stop if violated {OK_STOMATE}
STOMATE_MASS_CONSERVATION = n

# STOMATE_DSG_DEBUG ([FLAG]) : activate checks for stoichiometry and negative pools {OK_STOMATE}
STOMATE_DSG_DEBUG = n

# LD_FAKE_HEIGHT ([FLAG]) : fakes height of vegetation (ORCHIDEE-CAN related) {OK_STOMATE}
LD_FAKE_HEIGHT = n

}}}

make sure that the variable SPINUP_ANALYTIC is set to "n":
{{{
SPINUP_ANALYTIC=n
}}}

REMARKS:
Do not enable XIOS, it is per default disabled in "PARAM/orchidee.default".
Do not enable river routing in site scale simulation, is per default disabled in "fluxnet.card"


Modify the configuration of the different phases in the FLUXNET simulations. These variables were originally used for the outdated spinup method using FORCESOIL.
Thus the variable names are misleading. Depending on the simulations (C-only, CN, CNP) the variables have to set differently.

modify them in ensemble.card and fluxnet.card:
{{{
[SPINUP]
# Initialisation for spin-up :
# orchidee with sechiba alone (!!! if ok_stomate == n !!!)
duree_nostomate=0
#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
duree_inistomate=1000
# teststomate (only if duree_inistomate > 0); DSG: don't touch this one
duree_offlineini=0

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=3
# ... for duree_sechiba years
duree_sechiba=50
# teststomate; DSG: don't touch this one
duree_stomate=0
# forcesoil; DSG: don't touch this one
duree_carbonsol=1

# Final run after the analytical spinup
# This last parameter must be non-zero.
duree_final=100
}}}



=== 1.3.1 C-only simulations ===
All parameter settings above are for a C-only

=== 1.3.2 CN simulations ===
Here I specify the modification you have to the do the setting of the C-only simulation to start a CN simulation.
You need to have restart files from a C-only simulation.

modify in fluxnet.card:
{{{
#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
duree_inistomate=0 
}}}

modify PARAM/run.def:
{{{
# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=n
}}}

Specify the path to the restart files for SECHIBA and STOMATE of the C-only FLUXNET simulations (you must have performed before);
modify config.card:
{{{
#D-- SRF options passed to SuRFace (SECHIBA) subjobs
[SRF]
WriteFrequency="1800s"
Restart=y
#-- Last day of the experience used as restart
RestartDate=1997-12-31
# Define restart simulation name
RestartJobName=SPINUP # NVres set
RestartPath=${ARCHIVE}/${LOGIN}/IGCM_OUT/${config_UserChoices_TagName}  # NVres set path /home/scratch01/dgoll/IGCM_OUT/OL2/DEVT/ensemble
# Old component name for restart (if empty, use new name)
OldName=

#D-- Surface BioGeochemistry ( STOMATE ) subjobs
[SBG]
WriteFrequency="1M"
Restart=y
#-- Last day of the experience used as restart
RestartDate= 1900-12-31
# Define restart simulation name
RestartJobName= SPINUP # NVres set
RestartPath=${ARCHIVE}/${LOGIN}/IGCM_OUT/${config_UserChoices_TagName} # NVres set
# Old component name for restart (if empty, use new name)
OldName=
}}}

copy the modified Job_ENSEMBLE:
{{{
cp /home/surface3/common/CNP_files/rev00/Job_ENSEMBLE_restart Job_ENSEMBLE
}}}

=== 1.3.2 CNP simulations ===
Here I specify the modification you have to the do the setting of the C-only simulation to start a CN simulation.
You need to have restart files from a CN simulation.
modify in fluxnet.card:
{{{
#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
duree_inistomate=0 
}}}

modify PARAM/run.def:
{{{
# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=n
STOMATE_IMPOSE_NP=n
}}}

Specify the path to the restart files for SECHIBA and STOMATE of the CN FLUXNET simulations (you must have performed before);
modify config.card:
{{{
#D-- SRF options passed to SuRFace (SECHIBA) subjobs
[SRF]
WriteFrequency="1800s"
Restart=n # NVres Y
#-- Last day of the experience used as restart
RestartDate=1997-12-31
# Define restart simulation name
RestartJobName=SPINUP # NVres set
RestartPath=${ARCHIVE}/${LOGIN}/IGCM_OUT/${config_UserChoices_TagName}  # NVres set path /home/scratch01/dgoll/IGCM_OUT/OL2/DEVT/ensemble
# Old component name for restart (if empty, use new name)
OldName=

#D-- Surface BioGeochemistry ( STOMATE ) subjobs
[SBG]
WriteFrequency="1M"
Restart= n # NVres Y
#-- Last day of the experience used as restart
RestartDate= 1900-12-31
# Define restart simulation name
RestartJobName= SPINUP # NVres set
RestartPath=${ARCHIVE}/${LOGIN}/IGCM_OUT/${config_UserChoices_TagName} # NVres set
# Old component name for restart (if empty, use new name)
OldName=
}}}

copy the modified Job_ENSEMBLE:
{{{
cp /home/surface3/common/CNP_files/rev00/Job_ENSEMBLE_restart Job_ENSEMBLE
}}}

== 1.4 adjust the job as usual to your specific needs ==
1. modify config.card following [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation]
give information on Job Name, Tag Name, Dates and Restart. Make sure the path ARCHIVE is correct for the machine you are using. You must set the ARCHIVE on curie or your simulation will crash.
for example in the IMBALANCE-P storage; add to config.card:
{{{
ARCHIVE=/home/surface3/dgoll 
}}} 
2. modify run.def following [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation]

3. modify fluxnet.card following [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation]

In case you want to check if the spinup works well, increase output level of the spinup simulations & prevent libIGCM from removing these files.
The files can be found in the respective site folders in the experiment folder.

modify in fluxnet.card and ensemble.card:
{{{
# level of the history files for each iteration of the SPINUP
level_hist=10
# level of the history files for the final iteration of the SPINUP
level_hist_final=10

# DEBUG mode for SPINUP 
#
# This mode keep all SPINUP directory in ARCHIVE
# If disable, all ARCHIVE is automaticly cleaned.
DEBUG_SPIN=y
# If you don't want to keep old spinup steps, but last one
CONSERVE=y
}}}
You find the files in the folder "JobName"STOI in the IGCM_OUT path for the STOI simulation. Exchange "STOI" with "ORC-1", "FIN" etc for the simulations you want to check.

== 1.5 create job ==
{{{
../../../libIGCM/ins_job
}}}

== 1.6 adjust job  ==
'''on obelix'''
change the queue to longp to increase the allocated time (the default queue mediump is too short and you will exceed wall clock time).
{{{
#PBS -q longp
}}}

If you want to debug on obelix: set the variable RUN_DIR_PATH to disk with enough space (the default path is not accessible for the user)
{{{
# for example on scratch01
RUN_DIR_PATH=/home/scratch01/dgoll/RUN_DIR
}}}
ACHTUNG: On curie, do not set RUN_DIR_PATH in the Job_*. If you do, the configuration will crash. You find the default path to the RUN_DIR_PATH in the section "3.2.8 check simulations".


== 1.7 start simulations ==
{{{
ksh
nohup ./Job_ENSEMBLE fluxnet > out_Job 2>&1 &
}}}

== 1.8 check simulations ==
to see the decomposition of the spatial domain among the processors check in the run dir "Load_balance_orchidee.dat"
'''on curie'''
The RUN_DIR is in 
{{{
/ccc/scratch/cont003/dsm/<userid>/RUN_DIR
}}}

to check the status of the job; type
{{{
ccc_mstat -u <userid>
}}}
to delete a job; type
{{{
ccc_mdel <jobid>
}}}
=== 1.8.1 what to do when a simulation crash ===
You cannot use clean_month and clean_year. You should remove the sites that crashed from the IGCM_OUT path
and rerun the same experiment with a fluxnet.card containing only the sites which crashed.


== 1.9 post processing ==
ensure the Jobname and the path to the reference simulation is set in fluxnet.card; in case there is no reference set it to NONE
{{{
# History file of former ORCHIDEE runs (Reference) to compare with the current simulations
reference_base_path=/home/scratch01/dgoll/IGCM_OUT/OL2/DEVT/ensemble
reference_config=DSG04EnsCN
}}}
start the skript
{{{
ksh
nohup ./Job_Post_FLUXNET fluxnet > out_Post 2>&1 &
}}}
you will find the graphics in the experiment folder in the IGCM_OUT path. There is a pdf in the folder starting with "fluxnet_taylor_diff".