wiki:DevelopmentActivities/ORCHIDEE-CNP/howtoFLUXNET

Version 69 (modified by dgoll, 7 years ago) (diff)

--

ENSEMBLE: fluxnet sites http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation

Sequence of simulations needed for FLUXNET sites (C, CN, CNP) Figure shows the sequence of simulations the FLUXNET configuration will produce in a grey box. Prognostic nutrient cycles have to be activated in a three steps. First, a FLUXNET simulation with imposed N and P tissue concentration is done (C-only). Second, a simulation with full-dynamics N cycle but prescribed tissue P concentration (CN) -or - a simulation with full dynamic N & P cycles (CNP). One can skip the second step in the scheme and directly initialize the full flexible CNP stoichiometry.



WARNING: The FLUXNET configuration is very rigid compared to the other configurations. You should follow exactly what is written below. The strategy should be to minimize any personalization of the paths or settings.



0.0 in case you do not use the ORCHIDEE-CN-P code

The forcing of many FLUXNET sites has a duration of less than 7 years. Per default the variable spinup_period in ORCHIDEE is increased to period larger than 6. This automatic correction doesn't work with the FLUXNET configuration. It will prevent the analytical spinup in the FOR-X simulation. Therefore you need to deactivate the automatic correction in src_stomate/stomate.f90:

        ELSE IF (spinup_period <= 6) THEN
           ! Adjust to bigger period. The period must be a multiple of the original period.
-          WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be adjusted.'
-          spinup_period = spinup_period*(INT(9/spinup_period)+1)
+      !DSG    WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be adjusted.'
+          WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be not adjusted.'
+      !DSG:    spinup_period = spinup_period*(INT(9/spinup_period)+1)

then, re-compile the model.


0.1 deactivate BigBrother? in libIGCM when running on curie

You should deactivate the BigBrother? on curie. The model runs also with BigBrother? but thee curie staff asked us not to use it.

You have to set it to false in modipsl/libIGCM/AA_job

BigBrother=false

Then all jobs you are going to create will automatically be without BigBrother?.

In case you already run the libIGCM command to create a job (*) at least once (FLUXNET or not), you need to also set

BigBrother=false

in: modipsl/config/ORCHIDEE_OL/SPINUP/Job_Spinup

modipsl/config/ORCHIDEE_OL/SPINUP/SUBJOB/OOL_SEC_STO/Job_AUTO

(*) this is what I am talking about:

cd modipsl/config/ORCHIDEE_OL/SPINUP
../../../libIGCM/ins_job

1.1 create new experiment folder

cd ../../config/ORCHIDEE_OL/
cp -fr ENSEMBLE/ ENSEMBLE_CNP/
cd ENSEMBLE_CNP/

1.2 tell libIGCM about the new input files

For now, I store all files needed on the IMBALANCE-P disk on obelix (you find the path below). Please ensure you are using the newest revision.

The configuration which are used are stored in ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/ so, do as in 4.1.2: change the path of soil_param.nc and add the new files

../SPINUP/SUBJOB/OOL_SEC_STO/COMP/sechiba.card:

(/home/surface3/common/CNP_files/revXX/soils_param.nc, .), \
(/home/users/dgoll/ORC_data/N_deposition/LMDzINCA_DEP_2000.nc, .), \

../SPINUP/SUBJOB/OOL_SEC_STO/COMP/stomate.card:

ListNonDel= (${R_IN}/SRF/reftemp.nc, .), \
(/home/surface3/common/CNP_files/revXX/USDA_SoilSuborder.nc, .), \
(/home/surface3/common/CNP_files/revXX/lithology.nc, .), \
(/home/users/dgoll/ORC_data/N_deposition/LMDzINCA_DEP_2000.nc, .), \
(/home/surface3/common/CNP_files/deposition/deposition_2000.nc, .)

REMARK: make sure there are no blanks behind the "\"!

REMARK: Although the file with N deposition is used in sechiba we add the file also to the stomate.card. This is needed to ensure it is copied when we restart from a simulation with impose_cn. In that case, the N deposition variable will not be present in the sechiba_restart_out.nc but the libIGCM assumes all sechiba variables are present.

WARNING: the deposition file is a static field at the moment; it does not include N deposition either. At the moment N deposition is set to P deposition in the ORCHIDEE code. Anyway N deposition is not accounted for in the calculations for the N & P cycle for now.

1.3 adjust the libIGCM

1.3.0 exchange the spinup.driver

exchange the spinup.driver file in ../SPINUP/COMP/ with the new modified driver from NV.

cp /home/surface3/common/CNP_files/revXX/spinup.driver ../SPINUP/COMP/

the new driver will use analytical spinup steps instead of forcsoil steps which a) will not work with nutrients and b) are anyway outdated.

1.3.1 PARAM/run.def

copy a run.def with parameter values for the ORCHIDEE-CAN components. Make sure you are using the newest version here:

cp /home/surface3/common/CNP_files/revXX/run.def PARAM/run.def

make sure the variables to turn on/off the nutrients cycles are in PARAM/run.def:

# switch for N cycle
STOMATE_OK_NCYCLE=y
# switch for P cycle
STOMATE_OK_PCYCLE=y
# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=y
# switch to either impose N content of vegetation (to surpass P limitation):
STOMATE_IMPOSE_NP=y
# do not fake the initial height (DOFOCO parameter)
LD_FAKE_HEIGHT=n

Nammonium_FILE=LMDzINCA_DEP_2000.nc
Nnitrate_FILE=LMDzINCA_DEP_2000.nc
Nnitrate_VAR = NOY
Nammonium_VAR = NHX


Nfert_FILE=NONE
Nbnf_FILE=NONE
NINPUT_UPDATE=-1

if you want to debug the model, set printlev to 4 for full debug information in PARAM/run.def:

PRINTLEV=4

1.3.2 PARAM/orchidee.default (optional)

you can add defaults for the nutrient switches into PARAM/orchidee.default

# STOMATE_OK_NCYLE ([FLAG]) :  Activate the nitrogen cycle {OK_STOMATE}
STOMATE_OK_NCYCLE = n

# STOMATE_OK_PCYLE ([FLAG]) :  Activate the phosphorus cycle {OK_STOMATE}
STOMATE_OK_PCYCLE = n

# STOMATE_MASS_CONSERVATION ([FLAG]) : activate mass conservation checks which force stop if violated {OK_STOMATE}
STOMATE_MASS_CONSERVATION = n

# STOMATE_DSG_DEBUG ([FLAG]) : activate checks for stoichiometry and negative pools {OK_STOMATE}
STOMATE_DSG_DEBUG = n

# LD_FAKE_HEIGHT ([FLAG]) : fakes height of vegetation (ORCHIDEE-CAN related) {OK_STOMATE}
LD_FAKE_HEIGHT = n

REMARKS: Do not enable XIOS as the current version of ORCHIDEE-CN-P does not support XIOS (I am not sure if ENSEMBLE can handle XIOS), it is per default disabled in "PARAM/orchidee.default".

1.3.3 fluxnet.card

WARNING: Do not enable river routing in site scale simulation, is per default disabled in "fluxnet.card"

WARNING: make sure that the variable SPINUP_ANALYTIC is not present in any section of fluxnet.card (per default SPINUP_ANALYTIC can be found twice in the fluxnet.card. Remove both entries. )

# in case this variable is set to "n" or "y" the configuration will run but the analytic spinup is either constantly on/off.
#SPINUP_ANALYTIC=n

Modify the configuration of the different phases in the FLUXNET simulations. These variables were originally used for the outdated spinup method using FORCESOIL. Thus the variable names are misleading. Depending on the simulations (C-only, CN, CNP) the variables have to set differently.

[SPINUP]
# Initialisation for spin-up :
# orchidee with sechiba alone (!!! if ok_stomate == n !!!)
duree_nostomate=0
#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
duree_inistomate=250
# teststomate (only if duree_inistomate > 0); DSG: don't touch this one
duree_offlineini=0

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=3
# ... for duree_sechiba years
duree_sechiba=15
# teststomate; DSG: don't touch this one
duree_stomate=0
# forcesoil; DSG: don't touch this one
duree_carbonsol=1

# Final run after the analytical spinup
# This last parameter must be non-zero.
duree_final=250

1.3.1 C-only simulations

All parameter settings above are for a C-only

1.3.2 CN simulations

Here I specify the modification you have to the do the setting of the C-only simulation to start a CN simulation. You need to have restart files from a C-only simulation.

modify in fluxnet.card:

#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
# this must be zero for use with restart files
duree_inistomate=0 

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=5
# ... for duree_sechiba years
duree_sechiba=50

modify PARAM/run.def:

# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=n

#this switch adds nutrients when immobilisation demands exceed supply 
# to avoid crashing the model). This problem of overuse happens regularly during spinups.
# a mass conserving solution would be better, but I have no solution at the moment.
SPINUP_CNP=y

Specify the path to the restart files for SECHIBA and STOMATE of the C-only FLUXNET simulations (you must have performed before); modify config.card:

#D-- SRF options passed to SuRFace (SECHIBA) subjobs
[SRF]
WriteFrequency="1800s"
Restart=y
#-- Last day of the experience used as restart
RestartDate=1997-12-31
# Define restart simulation name
RestartJobName=<JobName of C-only>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of CN>/ensemble/<JobName of C-only>
# Old component name for restart (if empty, use new name)
OldName=

#D-- Surface BioGeochemistry ( STOMATE ) subjobs
[SBG]
WriteFrequency="1M"
Restart=y
#-- Last day of the experience used as restart
RestartDate= 1900-12-31
# Define restart simulation name
RestartJobName= <JobName of C-only>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of C-only>/ensemble/<JobName of C-only>
# Old component name for restart (if empty, use new name)
OldName=

<JobName?> and <SpaceName?> can be found in the config.card of the C-only simulation.

copy the modified Job_ENSEMBLE:

cp /home/surface3/common/CNP_files/revXX/Job_ENSEMBLE_restart Job_ENSEMBLE

1.3.2 CNP simulations

Here I specify the modification you have to the do the setting of the C-only simulation to start a CNP simulation. You need to have restart files from a CN or a C simulation (see steps before). modify in fluxnet.card:

#  normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years
duree_inistomate=0 

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=5
# ... for duree_sechiba years
duree_sechiba=50

modify PARAM/run.def:

# switch to either impose N content of vegetation (to surpass N limitation):
STOMATE_IMPOSE_CN=n
STOMATE_IMPOSE_NP=n

#this switch adds nutrients when immobilisation demands exceed supply 
# to avoid crashing the model). This problem of overuse happens regularly during spinups
# a better solution would be good, but I have no solution at the moment.
SPINUP_CNP=y

Specify the path to the restart files for SECHIBA and STOMATE of the CN FLUXNET simulations (you must have performed before); modify config.card:

#D-- SRF options passed to SuRFace (SECHIBA) subjobs
[SRF]
WriteFrequency="1800s"
Restart= y # NVres Y
#-- Last day of the experience used as restart
RestartDate=1997-12-31
# Define restart simulation name
RestartJobName= <JobName of CN>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of CN>/ensemble/<JobName of CN>
# Old component name for restart (if empty, use new name)
OldName=

#D-- Surface BioGeochemistry ( STOMATE ) subjobs
[SBG]
WriteFrequency="1M"
Restart= y # NVres Y
#-- Last day of the experience used as restart
RestartDate= 1900-12-31
# Define restart simulation name
RestartJobName= <ID of CN>HIST
RestartPath=<Path to IGCM>/IGCM_OUT/OL2/<SpaceName of CN>/ensemble/<JobName of CN>
# Old component name for restart (if empty, use new name)
OldName=

<JobName?> and <SpaceName?> can be found in the config.card of the CN simulation.

copy the modified Job_ENSEMBLE:

cp /home/surface3/common/CNP_files/revXX/Job_ENSEMBLE_restart Job_ENSEMBLE

1.4 adjust the job as usual to your specific needs

  1. modify config.card following http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation

give information on Job Name, Tag Name, Dates and Restart.

Make sure the path ARCHIVE is set to the same path in config.card ...

ARCHIVE=/home/surface3/$LOGIN

... and in the ../SPINUP/SUBJOB/OOL_SEC_STO/config.card

ARCHIVE=/home/surface3/$LOGIN

REMARK: it seems you are not allowed to put the ARCHIVE on obelix to /home/scratch01/$LOGIN.

  1. modify run.def following http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation
  1. modify fluxnet.card following http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation

1.5 create job

../../../libIGCM/ins_job

1.6 adjust job

on curie You must deactivate BigBrother? by setting the existing entry from true to false:

BigBrother=false

on obelix Change the queue to longp to increase the wallclock time (the default queue mediump is too short for this setting and you will exceed wall clock time). WARNING: In case you have set level_hist=10 (fluxnet.card) you need to take the extra long queue:

#PBS -q longp

If you want to debug on obelix: set the variable RUN_DIR_PATH to disk with enough space (the default path is not accessible for the user)

# for example on scratch01
RUN_DIR_PATH=/home/scratch01/dgoll/RUN_DIR

ACHTUNG: On curie, do not set RUN_DIR_PATH in the Job_*. If you do, the configuration will crash. You find the default path to the RUN_DIR_PATH in the section "3.2.8 check simulations".

1.7 start simulations

ksh
nohup ./Job_ENSEMBLE fluxnet > out_Job 2>&1 &

1.8 check simulations

to see the decomposition of the spatial domain among the processors check in the run dir "Load_balance_orchidee.dat" on curie The RUN_DIR is in

/ccc/scratch/cont003/dsm/<userid>/RUN_DIR

to check the status of the job; type

ccc_mstat -u <userid>

to delete a job; type

ccc_mdel <jobid>

1.8.1 what to do when a simulation crash

You cannot use clean_month and clean_year. You should remove the sites that crashed from the IGCM_OUT path and rerun the same experiment with a fluxnet.card containing only the sites which crashed.

1.8.2. IMPORTANT: check if you reached an equilibrium

It is not given that you will achieve an equilibrium when having all nutrient interactions on. Therefore you need to confirm that the NEE is close to zero averaged over your forcing period. The figure below shows the evolution of the carbon soil pools (CENTURY) during two CNP simulations with:

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=7
# ... for duree_sechiba years
duree_sechiba=50

duree_final=250

The temporal evolution of the CENTURY C pools during a FLUXNET run (with 7*50yr Analytical spinup (green) + 250 OOL_SEC_STO (blue).

# Loop over ORCHIDEE runs (used for spin-up)
# The analytical spinup is restarted n_iter times ...
n_iter=1
# ... for duree_sechiba years
duree_sechiba=50

duree_final=750

as runAway.png but w/o the analytical spinup steps

1.9 post processing

ensure the Jobname and the path to the reference simulation is set in fluxnet.card; in case there is no reference set it to NONE

# History file of former ORCHIDEE runs (Reference) to compare with the current simulations
reference_base_path=/home/scratch01/dgoll/IGCM_OUT/OL2/DEVT/ensemble
reference_config=DSG04EnsCN

start the skript

ksh
nohup ./Job_Post_FLUXNET fluxnet > out_Post 2>&1 &

you will find the graphics in the experiment folder in the experiment folder in the IGCM_OUT path. The plots are in the folder and start with "fluxnet_taylor_diff". You can view the html file using firefox on curie ( I couldn't figure out which program can open pdf).

OPTIONAL FEATURES

specify the name of the exectuable of orchidee

If you want to use different version of the orchidee_ol which you store in modipsl/bin you have to do the following: add the following line to the [Executable] section in ../SPINUP/config.card

OOL=(orchidee_ol, orchidee_ol)

add the following line to Job_ENSEMBLE; make sure it is exactly at this spot (before # Give path for ...):

IGCM_card_WriteOption ${New_SUBMIT_DIR}/config.card Executable OOL ${config_Executable_OOL}

# Give path for all skeleton SubJob scripts to SPINUP directory
IGCM_card_WriteOption ${New_SUBMIT_DIR}/COMP/spinup.card UserChoices SubJobPath ${Skeleton_Path}

At the moment I cannot figure out how to set the variable ${config_Executable_OOL} according to the config.card; thus I hardcode the name in Job_ENSEMBLE for the moment:

IGCM_card_WriteOption ${New_SUBMIT_DIR}/config.card Executable OOL "(my_orchidee_ol, orchidee_ol)"

PROBLEM: although my_orchidee_ol is not present in modipsl/bin a orchidee_ol is present in the RUN_DIR.

output for the spinup simulations

In case you want to check if the spinup works well, increase output level of the spinup simulations & prevent libIGCM from removing these files. The files can be found in the respective site folders in the experiment folder.

modify in fluxnet.card and ensemble.card:

# level of the history files for each iteration of the SPINUP
level_hist=10
# level of the history files for the final iteration of the SPINUP
level_hist_final=10

# DEBUG mode for SPINUP 
#
# This mode keep all SPINUP directory in ARCHIVE
# If disable, all ARCHIVE is automaticly cleaned.
DEBUG_SPIN=y
# If you don't want to keep old spinup steps, but last one
CONSERVE=y

You find the files in the folder "JobName?"STOI in the IGCM_OUT path for the STOI simulation. Exchange "STOI" with "ORC-1", "FIN" etc for the simulations you want to check.

additional site specific parameters

You can add more site specific parameter values by adding them to the fluxnet.card, as long as the parameters the shape of PFT parameters (NbPFTs in fluxnet.card) or a single value. If you have a different shape (see below).

Here an example how to add site specific information for the soil fractions

SOIL_FRACTIONS = 1. , 0. , 0.

First, edit the variable for the number of site specific parameters in fluxnet.card:

#  Number of parameters to modify for each site
NbSitesParam= 2

Second, add the parameter name to the list of parameters to be site specific in fluxnet.card:

NameSitesParam= ( SECHIBA_VEGMAX SOIL_FRACTIONS )

Third, add the values for the new parameters in fluxnet.card:

Tface=      ( US-Thu, US-Thu_1999-2014.nc , 1980 , 36 , 0,1,0,0,0,0,0,0,0,0,0,0,0 , 1.,0.,0.)

Fourth, make sure the parameter is present in PARAM/orchidee.default. Make sure that there is a blank after each comma or the script will think it is a single value variable.:

SOIL_FRACTIONS =  0.6, 0.2, 0.2

You can check if the site parameter are correctly initialized after you submitted the job. You should find in job_Job:

ORCHIDEE_def : SECHIBA_VEGMAX 0 1 0 0 0 0 0 0 0 0 0 0 0
ORCHIDEE_def : SOIL_FRACTIONS 1. 0. 0.  

additional site specific parameters (other than NbPFTs)

You need to modify fluxnet.card, Job_ENSEMBLE, ensemble.ksh.

Here an example to read in SOIL_FRACTIONS which should get 3 entries

in fluxnet.card add:

NbSOILs= 3

tell ensemble.ksh to read the new variable by adding:

    # - List of the sites/information to be treated :
    #   * Number of PFTs
    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/ensemble.card CONFIG NbPFTs
    NbPFTs=${ensemble_CONFIG_NbPFTs}

DSGadded
    # - List of the sites/information to be treated :
    #   * Number of SOILs
    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/ensemble.card CONFIG NbSOILs
    NbSOILs=${ensemble_CONFIG_NbSOILs}
DSGadded

tell ensemble.ksh to use the new variable by modifying:

#DSGmod
  if [ ${ensemble_CONFIG_NameSitesParam[${iphys}]} = "SOIL_FRACTIONS" ] ; then
      echo 'nbSOILs:' $NbSOILs
       DSGcunt=$NbSOILs
    else
      echo 'nbPFTs:' $NbPFTs
       DSGcunt=$NbPFTs
    fi

    case ${SearchParam} in
        undefined)
        echo "ERROR : parameter ${ensemble_CONFIG_NameSitesParam[${iphys}]} doesn't exist in run.def parameter file !"
        echo "You must correct NameSiteParam option in your card file or add new parameter in run.def file"
        echo "We must STOP here."
        exit 1
        ;;
        value)
        (( NumInfosBySite = NumInfosBySite + 1 ))
        ;;
        line)
        (( NumInfosBySite = NumInfosBySite + DSGcunt ))
        ;;
        vector)
        (( NumInfosBySite = NumInfosBySite + DSGcunt ))
        ;;
    esac
#DSGmod

Now, let Job_ENSEMBLE know that, too, by modifying:

        if [ ${ensemble_CONFIG_NameSitesParam[${iphys}]} == "SOIL_FRACTIONS" ] ; then
           unset ValueSiteParams
           ipft=0
           while [ $ipft -lt 3 ] ; do
               ValueSiteParams[${ipft}]=${group_site[$(( indext )) ]}
               (( indext = indext + 1 ))
               (( ipft = ipft + 1 ))
           done
        else

           unset ValueSiteParams
           ipft=0
           while [ $ipft -lt ${NbPFTs} ] ; do
               ValueSiteParams[${ipft}]=${group_site[$(( indext )) ]}
               (( indext = indext + 1 ))
               (( ipft = ipft + 1 ))
           done
        fi

Attachments (7)

Download all attachments as: .zip