= WARNING: this setup is not maintained anymore for the CNP branch. Use the OOL_SEC_STO configuration.= fluxnet sites [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation] [[Image(ORC-CNP- report 201603.2.png)]] Figure shows the sequence of simulations the FLUXNET configuration will produce in a grey box. Prognostic nutrient cycles have to be activated in a three steps. First, a FLUXNET simulation with imposed N and P tissue concentration is done (C-only). Second, a simulation with full-dynamics N cycle but prescribed tissue P concentration (CN) -or - a simulation with full dynamic N & P cycles (CNP). One can skip the second step in the scheme and directly initialize the full flexible CNP stoichiometry. ---- ---- '''WARNING: The FLUXNET configuration is very rigid compared to the other configurations. You should follow exactly what is written below. The strategy should be to minimize any personalization of the paths or settings.''' ---- ---- == 0.0 in case you do not use the ORCHIDEE-CN-P code == The forcing of many FLUXNET sites has a duration of less than 7 years. Per default the variable spinup_period in ORCHIDEE is increased to period larger than 6. This automatic correction doesn't work with the FLUXNET configuration. It will prevent the analytical spinup in the FOR-X simulation. Therefore you need to deactivate the automatic correction in src_stomate/stomate.f90: {{{ ELSE IF (spinup_period <= 6) THEN ! Adjust to bigger period. The period must be a multiple of the original period. - WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be adjusted.' - spinup_period = spinup_period*(INT(9/spinup_period)+1) + !DSG WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be adjusted.' + WRITE(numout,*) 'Initial spinup_period =',spinup_period,' will be not adjusted.' + !DSG: spinup_period = spinup_period*(INT(9/spinup_period)+1) }}} then, re-compile the model. ---- == 0.1 deactivate BigBrother in libIGCM when running on curie == You should deactivate the BigBrother on curie. The model runs also with BigBrother but thee curie staff asked us not to use it. You have to set it to false in modipsl/libIGCM/AA_job {{{ BigBrother=false }}} Then all jobs you are going to create will automatically be without BigBrother. In case you already run the libIGCM command to create a job (*) at least once (FLUXNET or not), you need to also set {{{ BigBrother=false }}} in: modipsl/config/ORCHIDEE_OL/SPINUP/Job_Spinup modipsl/config/ORCHIDEE_OL/SPINUP/SUBJOB/OOL_SEC_STO/Job_AUTO (*) this is what I am talking about: {{{ cd modipsl/config/ORCHIDEE_OL/SPINUP ../../../libIGCM/ins_job }}} == 1.1 create new experiment folder == {{{ cd ../../config/ORCHIDEE_OL/ cp -fr ENSEMBLE/ ENSEMBLE_CNP/ cd ENSEMBLE_CNP/ }}} == 1.2 tell libIGCM about the new input files == For now, I store all files needed on the IMBALANCE-P disk on obelix (you find the path below). Please ensure you are using the newest revision. The configuration which are used are stored in ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/ so, do as in 4.1.2: change the path of soil_param.nc and add the new files ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/sechiba.card: {{{ (/home/surface3/common/CNP_files/revXX/soils_param.nc, .), \ (/home/users/dgoll/ORC_data/N_deposition/LMDzINCA_DEP_2000.nc, .), \ }}} ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/stomate.card: {{{ ListNonDel= (${R_IN}/SRF/reftemp.nc, .), \ (/home/surface3/common/CNP_files/revXX/USDA_SoilSuborder.nc, .), \ (/home/surface3/common/CNP_files/revXX/lithology.nc, .), \ (/XXX/deposition_2000.nc, .) }}} REMARK: make sure there are no blanks behind the "\"! REMARK: Although the file with N deposition is used in sechiba we add the file also to the stomate.card. This is needed to ensure it is copied when we restart from a simulation with impose_cn. In that case, the N deposition variable will not be present in the sechiba_restart_out.nc but the libIGCM assumes all sechiba variables are present. REMARK: see this page for deposition files: [https://forge.ipsl.jussieu.fr/orchidee/wiki/DevelopmentActivities/inputRoutine] == 1.3 adjust the libIGCM == === 1.3.0 exchange the spinup.driver === exchange the spinup.driver file in ../SPINUP/COMP/ with the new modified driver from NV. {{{ cp /home/surface3/common/CNP_files/revXX/spinup.driver ../SPINUP/COMP/ }}} the new driver will use analytical spinup steps instead of forcsoil steps which a) will not work with nutrients and b) are anyway outdated. === 1.3.1 PARAM/run.def === copy a run.def with parameter values for the ORCHIDEE-CAN components. Make sure you are using the newest version here: {{{ cp /home/surface3/common/CNP_files/revXX/run.def PARAM/run.def }}} make sure the variables to turn on/off the nutrients cycles are in PARAM/run.def: {{{ # switch for N cycle STOMATE_OK_NCYCLE=y # switch for P cycle STOMATE_OK_PCYCLE=y # switch to either impose N content of vegetation (to surpass N limitation): STOMATE_IMPOSE_CN=y # switch to either impose N content of vegetation (to surpass P limitation): STOMATE_IMPOSE_NP=y # do not fake the initial height (DOFOCO parameter) LD_FAKE_HEIGHT=n }}} if you want to debug the model, set printlev to 4 for full debug information in PARAM/run.def: {{{ PRINTLEV=4 }}} === 1.3.2 PARAM/orchidee.default (optional) === you can add defaults for the nutrient switches into PARAM/orchidee.default {{{ # STOMATE_OK_NCYLE ([FLAG]) : Activate the nitrogen cycle {OK_STOMATE} STOMATE_OK_NCYCLE = n # STOMATE_OK_PCYLE ([FLAG]) : Activate the phosphorus cycle {OK_STOMATE} STOMATE_OK_PCYCLE = n # STOMATE_MASS_CONSERVATION ([FLAG]) : activate mass conservation checks which force stop if violated {OK_STOMATE} STOMATE_MASS_CONSERVATION = n # STOMATE_DSG_DEBUG ([FLAG]) : activate checks for stoichiometry and negative pools {OK_STOMATE} STOMATE_DSG_DEBUG = n # LD_FAKE_HEIGHT ([FLAG]) : fakes height of vegetation (ORCHIDEE-CAN related) {OK_STOMATE} LD_FAKE_HEIGHT = n }}} === 1.3.3 fluxnet.card === '''WARNING''': Do not enable river routing in site scale simulation, is per default disabled in "fluxnet.card" '''WARNING''': make sure that the variable SPINUP_ANALYTIC is not present in any section of fluxnet.card (per default SPINUP_ANALYTIC can be found twice in the fluxnet.card. Remove both entries. ) {{{ # in case this variable is set to "n" or "y" the configuration will run but the analytic spinup is either constantly on/off. #SPINUP_ANALYTIC=n }}} Modify the configuration of the different phases in the FLUXNET simulations. These variables were originally used for the outdated spinup method using FORCESOIL. Thus the variable names are misleading. Depending on the simulations (C-only, CN, CNP) the variables have to set differently. {{{ [SPINUP] # Initialisation for spin-up : # orchidee with sechiba alone (!!! if ok_stomate == n !!!) duree_nostomate=0 # normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years duree_inistomate=250 # teststomate (only if duree_inistomate > 0); DSG: don't touch this one duree_offlineini=0 # Loop over ORCHIDEE runs (used for spin-up) # The analytical spinup is restarted n_iter times ... n_iter=3 # ... for duree_sechiba years duree_sechiba=15 # teststomate; DSG: don't touch this one duree_stomate=0 # forcesoil; DSG: don't touch this one duree_carbonsol=1 # Final run after the analytical spinup # This last parameter must be non-zero. duree_final=250 }}} === 1.3.1 C-only simulations === All parameter settings above are for a C-only === 1.3.2 CN simulations === Here I specify the modification you have to the do the setting of the C-only simulation to start a CN simulation. You need to have restart files from a C-only simulation. modify in fluxnet.card: {{{ # normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years # this must be zero for use with restart files duree_inistomate=0 # Loop over ORCHIDEE runs (used for spin-up) # The analytical spinup is restarted n_iter times ... n_iter=5 # ... for duree_sechiba years duree_sechiba=50 }}} modify PARAM/run.def: {{{ # switch to either impose N content of vegetation (to surpass N limitation): STOMATE_IMPOSE_CN=n #this switch adds nutrients when immobilisation demands exceed supply # to avoid crashing the model). This problem of overuse happens regularly during spinups. # a mass conserving solution would be better, but I have no solution at the moment. SPINUP_CNP=y }}} Specify the path to the restart files for SECHIBA and STOMATE of the C-only FLUXNET simulations (you must have performed before); modify config.card: {{{ #D-- SRF options passed to SuRFace (SECHIBA) subjobs [SRF] WriteFrequency="1800s" Restart=y #-- Last day of the experience used as restart RestartDate=1997-12-31 # Define restart simulation name RestartJobName=HIST RestartPath=/IGCM_OUT/OL2//ensemble/ # Old component name for restart (if empty, use new name) OldName= #D-- Surface BioGeochemistry ( STOMATE ) subjobs [SBG] WriteFrequency="1M" Restart=y #-- Last day of the experience used as restart RestartDate= 1900-12-31 # Define restart simulation name RestartJobName= HIST RestartPath=/IGCM_OUT/OL2//ensemble/ # Old component name for restart (if empty, use new name) OldName= }}} and can be found in the config.card of the C-only simulation. copy the modified Job_ENSEMBLE: {{{ cp /home/surface3/common/CNP_files/revXX/Job_ENSEMBLE_restart Job_ENSEMBLE }}} === 1.3.2 CNP simulations === Here I specify the modification you have to the do the setting of the C-only simulation to start a CNP simulation. You need to have restart files from a CN or a C simulation (see steps before). modify in fluxnet.card: {{{ # normal SEC_STO simulation to fill biomass pools (equilibrate litter fluxes) of duree_inistomate years duree_inistomate=0 # Loop over ORCHIDEE runs (used for spin-up) # The analytical spinup is restarted n_iter times ... n_iter=5 # ... for duree_sechiba years duree_sechiba=50 }}} modify PARAM/run.def: {{{ # switch to either impose N content of vegetation (to surpass N limitation): STOMATE_IMPOSE_CN=n STOMATE_IMPOSE_NP=n #this switch adds nutrients when immobilisation demands exceed supply # to avoid crashing the model). This problem of overuse happens regularly during spinups # a better solution would be good, but I have no solution at the moment. SPINUP_CNP=y }}} Specify the path to the restart files for SECHIBA and STOMATE of the CN FLUXNET simulations (you must have performed before); modify config.card: {{{ #D-- SRF options passed to SuRFace (SECHIBA) subjobs [SRF] WriteFrequency="1800s" Restart= y # NVres Y #-- Last day of the experience used as restart RestartDate=1997-12-31 # Define restart simulation name RestartJobName= HIST RestartPath=/IGCM_OUT/OL2//ensemble/ # Old component name for restart (if empty, use new name) OldName= #D-- Surface BioGeochemistry ( STOMATE ) subjobs [SBG] WriteFrequency="1M" Restart= y # NVres Y #-- Last day of the experience used as restart RestartDate= 1900-12-31 # Define restart simulation name RestartJobName= HIST RestartPath=/IGCM_OUT/OL2//ensemble/ # Old component name for restart (if empty, use new name) OldName= }}} and can be found in the config.card of the CN simulation. copy the modified Job_ENSEMBLE: {{{ cp /home/surface3/common/CNP_files/revXX/Job_ENSEMBLE_restart Job_ENSEMBLE }}} == 1.4 adjust the job as usual to your specific needs == 1. modify config.card following [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation] give information on Job Name, Tag Name, Dates and Restart. Make sure the path ARCHIVE is set to the same path in config.card ... {{{ ARCHIVE=/home/surface3/$LOGIN }}} ... '''and''' in the ../SPINUP/SUBJOB/OOL_SEC_STO/config.card {{{ ARCHIVE=/home/surface3/$LOGIN }}} REMARK: it seems you are not allowed to put the ARCHIVE on obelix to /home/scratch01/$LOGIN. 2. modify run.def following [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation] 3. modify fluxnet.card following [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation] == 1.5 create job == {{{ ../../../libIGCM/ins_job }}} == 1.6 adjust job == '''on curie''' You must deactivate BigBrother by setting the existing entry from true to false: {{{ BigBrother=false }}} '''on obelix''' Change the queue to longp to increase the wallclock time (the default queue mediump is too short for this setting and you will exceed wall clock time). '''WARNING:''' In case you have set level_hist=10 (fluxnet.card) you need to take the extra long queue: {{{ #PBS -q longp }}} If you want to debug on obelix: set the variable RUN_DIR_PATH to disk with enough space (the default path is not accessible for the user) {{{ # for example on scratch01 RUN_DIR_PATH=/home/scratch01/dgoll/RUN_DIR }}} ACHTUNG: On curie, do not set RUN_DIR_PATH in the Job_*. If you do, the configuration will crash. You find the default path to the RUN_DIR_PATH in the section "3.2.8 check simulations". == 1.7 start simulations == {{{ ksh nohup ./Job_ENSEMBLE fluxnet > out_Job 2>&1 & }}} == 1.8 check simulations == to see the decomposition of the spatial domain among the processors check in the run dir "Load_balance_orchidee.dat" '''on curie''' The RUN_DIR is in {{{ /ccc/scratch/cont003/dsm//RUN_DIR }}} to check the status of the job; type {{{ ccc_mstat -u }}} to delete a job; type {{{ ccc_mdel }}} === 1.8.1 what to do when a simulation crash === You cannot use clean_month and clean_year. You should remove the sites that crashed from the IGCM_OUT path and rerun the same experiment with a fluxnet.card containing only the sites which crashed. === 1.8.2. IMPORTANT: check if you reached an equilibrium === It is not given that you will achieve an equilibrium when having all nutrient interactions on. Therefore you need to confirm that the NEE is close to zero averaged over your forcing period. The figure below shows the evolution of the carbon soil pools (CENTURY) during two CNP simulations with: {{{ # Loop over ORCHIDEE runs (used for spin-up) # The analytical spinup is restarted n_iter times ... n_iter=7 # ... for duree_sechiba years duree_sechiba=50 duree_final=250 }}} [[Image(runAwayP.png)]] {{{ # Loop over ORCHIDEE runs (used for spin-up) # The analytical spinup is restarted n_iter times ... n_iter=1 # ... for duree_sechiba years duree_sechiba=50 duree_final=750 }}} [[Image(runAwayP2.png)]] == 1.9 post processing == ensure the Jobname and the path to the reference simulation is set in fluxnet.card; in case there is no reference set it to NONE {{{ # History file of former ORCHIDEE runs (Reference) to compare with the current simulations reference_base_path=/home/scratch01/dgoll/IGCM_OUT/OL2/DEVT/ensemble reference_config=DSG04EnsCN }}} start the skript {{{ ksh nohup ./Job_Post_FLUXNET fluxnet > out_Post 2>&1 & }}} you will find the graphics in the experiment folder in the experiment folder in the IGCM_OUT path. The plots are in the folder and start with "fluxnet_taylor_diff". You can view the html file using firefox on curie ( I couldn't figure out which program can open pdf). == OPTIONAL FEATURES == === specify the name of the exectuable of orchidee === If you want to use different version of the orchidee_ol which you store in modipsl/bin you have to do the following: add the following line to the [Executable] section in ../SPINUP/config.card {{{ OOL=(orchidee_ol, orchidee_ol) }}} add the following line to Job_ENSEMBLE; make sure it is exactly at this spot (before # Give path for ...): {{{ IGCM_card_WriteOption ${New_SUBMIT_DIR}/config.card Executable OOL ${config_Executable_OOL} # Give path for all skeleton SubJob scripts to SPINUP directory IGCM_card_WriteOption ${New_SUBMIT_DIR}/COMP/spinup.card UserChoices SubJobPath ${Skeleton_Path} }}} At the moment I cannot figure out how to set the variable ${config_Executable_OOL} according to the config.card; thus I hardcode the name in Job_ENSEMBLE for the moment: {{{ IGCM_card_WriteOption ${New_SUBMIT_DIR}/config.card Executable OOL "(my_orchidee_ol, orchidee_ol)" }}} PROBLEM: although my_orchidee_ol is not present in modipsl/bin a orchidee_ol is present in the RUN_DIR. === output for the spinup simulations === In case you want to check if the spinup works well, increase output level of the spinup simulations & prevent libIGCM from removing these files. The files can be found in the respective site folders in the experiment folder. modify in fluxnet.card and ensemble.card: {{{ # level of the history files for each iteration of the SPINUP level_hist=10 # level of the history files for the final iteration of the SPINUP level_hist_final=10 # DEBUG mode for SPINUP # # This mode keep all SPINUP directory in ARCHIVE # If disable, all ARCHIVE is automaticly cleaned. DEBUG_SPIN=y # If you don't want to keep old spinup steps, but last one CONSERVE=y }}} You find the files in the folder "JobName"STOI in the IGCM_OUT path for the STOI simulation. Exchange "STOI" with "ORC-1", "FIN" etc for the simulations you want to check. === additional site specific parameters === You can add more site specific parameter values by adding them to the fluxnet.card, as long as the parameters the shape of PFT parameters (NbPFTs in fluxnet.card) or a single value. If you have a different shape (see below). Here an example how to add site specific information for the soil fractions {{{ SOIL_FRACTIONS = 1. , 0. , 0. }}} First, edit the variable for the number of site specific parameters in fluxnet.card: {{{ # Number of parameters to modify for each site NbSitesParam= 2 }}} Second, add the parameter name to the list of parameters to be site specific in fluxnet.card: {{{ NameSitesParam= ( SECHIBA_VEGMAX SOIL_FRACTIONS ) }}} Third, add the values for the new parameters in fluxnet.card: {{{ Tface= ( US-Thu, US-Thu_1999-2014.nc , 1980 , 36 , 0,1,0,0,0,0,0,0,0,0,0,0,0 , 1.,0.,0.) }}} Fourth, make sure the parameter is present in PARAM/orchidee.default. Make sure that there is a blank after each comma or the script will think it is a single value variable.: {{{ SOIL_FRACTIONS = 0.6, 0.2, 0.2 }}} You can check if the site parameter are correctly initialized after you submitted the job. You should find in job_Job: {{{ ORCHIDEE_def : SECHIBA_VEGMAX 0 1 0 0 0 0 0 0 0 0 0 0 0 ORCHIDEE_def : SOIL_FRACTIONS 1. 0. 0. }}} === additional site specific parameters (other than NbPFTs) === You need to modify fluxnet.card, Job_ENSEMBLE, ensemble.ksh. Here an example to read in SOIL_FRACTIONS which should get 3 entries in fluxnet.card add: {{{ NbSOILs= 3 }}} tell ensemble.ksh to read the new variable by adding: {{{ # - List of the sites/information to be treated : # * Number of PFTs IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/ensemble.card CONFIG NbPFTs NbPFTs=${ensemble_CONFIG_NbPFTs} DSGadded # - List of the sites/information to be treated : # * Number of SOILs IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/ensemble.card CONFIG NbSOILs NbSOILs=${ensemble_CONFIG_NbSOILs} DSGadded }}} tell ensemble.ksh to use the new variable by modifying: {{{ #DSGmod if [ ${ensemble_CONFIG_NameSitesParam[${iphys}]} = "SOIL_FRACTIONS" ] ; then echo 'nbSOILs:' $NbSOILs DSGcunt=$NbSOILs else echo 'nbPFTs:' $NbPFTs DSGcunt=$NbPFTs fi case ${SearchParam} in undefined) echo "ERROR : parameter ${ensemble_CONFIG_NameSitesParam[${iphys}]} doesn't exist in run.def parameter file !" echo "You must correct NameSiteParam option in your card file or add new parameter in run.def file" echo "We must STOP here." exit 1 ;; value) (( NumInfosBySite = NumInfosBySite + 1 )) ;; line) (( NumInfosBySite = NumInfosBySite + DSGcunt )) ;; vector) (( NumInfosBySite = NumInfosBySite + DSGcunt )) ;; esac #DSGmod }}} Now, let Job_ENSEMBLE know that, too, by modifying: {{{ if [ ${ensemble_CONFIG_NameSitesParam[${iphys}]} == "SOIL_FRACTIONS" ] ; then unset ValueSiteParams ipft=0 while [ $ipft -lt 3 ] ; do ValueSiteParams[${ipft}]=${group_site[$(( indext )) ]} (( indext = indext + 1 )) (( ipft = ipft + 1 )) done else unset ValueSiteParams ipft=0 while [ $ipft -lt ${NbPFTs} ] ; do ValueSiteParams[${ipft}]=${group_site[$(( indext )) ]} (( indext = indext + 1 )) (( ipft = ipft + 1 )) done fi }}}