Context Navigation

FLUXNETValidation

Timestamp:: 2019-12-20T11:44:54+01:00 (4 years ago)
Author:: mmcgrath
Comment:: Refining the instructions to work from a fresh svn install...not yet there part 2

Legend:

: Unmodified
: Added
: Removed
: Modified

Documentation/UserGuide/FLUXNETValidation

-                      v17
+                      v18
 The general procedure is that the Job_ENSEMBLE script will create and launch the following runs for every site:
 STOI (a fast spinup, length controlled by duree_inistomate)
 ORC-1 (a longer spinup, length controlled by duree_sechiba)
 CLEARCUT (added for CAN...aboveground biomass is removed before the run, to ensure forests have a specific age at the end...length controlled by the fifth field in the site description in fluxnet.card)
 FIN (the final fast production run, length controlled by duree_final)
+* STOI (a fast spinup, length controlled by duree_inistomate)
+* ORC-1 (a longer spinup, length controlled by duree_sechiba)
+* CLEARCUT (added for CAN...aboveground biomass is removed before the run, to ensure forests have a specific age at the end...length controlled by the fifth field in the site description in fluxnet.card)
+* FIN (the final fast production run, length controlled by duree_final)
 The length of these phases can be modified, and additional longer spinups can be added (by changing n_iter, creating ORC-2, ORC-3, etc.), but they are typically not necessary.  The final production data (from FIN) is always saved, and output from the other stages can be saved as well, but it's not recommended.  In particular, the data for the ORC run can get pretty large when half-hourly output is used.
 I have found the following files are used.  A lot of the work below goes to ensure that conflicting options are not specified in these files (assuming you are running analytical spinup, sechiba, and stomate):
+I have found the following files are used.  A lot of the work below goes to ensure that conflicting options are not specified in these files.  OOL_SEC_STO is selected when you are run both stomate and sechiba active (otherwise, OOL_SEC may be used):
 {{{
 …
 The following gives a general flow of how the script works, which should give an idea of the priority.
 ) ENSEMBLE_initialize/ensemble.ksh reads in the options from fluxnet.card (launching the script as "./Job_ENSEMBLE fluxnet" copies fluxnet.card to ensemble.card...Job_ENSEMBLE always reads all options from ensemble.card).  These are global variables while the runs are being set-up.  But, when the individual runs (each stage/site) launch, they may get overwritten by PARAM and COMP options in the launch directory.
+) ENSEMBLE_initialize/ensemble.ksh reads in the options from fluxnet.card (launching the script as "./Job_ENSEMBLE fluxnet" copies fluxnet.card to ensemble.card...Job_ENSEMBLE always reads all options from ensemble.card).  These are global variables while the runs are being set-up.  But, when the individual runs (each stage/site) launch, they may get overwritten by PARAM and COMP options in the launch directory.  The following sections are parsed in fluxnet.card:
    a) Section [SPINUP]
    b) Section [UserChoices]
    c) Section [CONFIG] (explicitly looks for ForcingPath, NbPFTs, NbSitesParam, NameSitesParam, Groups)
 The script below creates multiple new submission directories.  I use the notation ${}/PARAM/run.def and ${}/COMP/*card to refer to the run.def and various .card files **after** they have been copied from their original locations.
+The JOB_ENSEMBLE script and the spinup.driver create multiple new submission directories, using the SPINUP and SPINUP/SUBJOBS directories as templates.  I use the notation ${}/PARAM/run.def and ${}/COMP/*card to refer to the run.def and various .card files **after** they have been copied from their original locations.
 The following is then done for every site in Groups in fluxnet.card:
 …
 ) the Job_ file for the spinup is modified
 ) The FLUXNET/PARAM/run.def is copied to the spinup directory ${}/PARAM/run.def
 ) The script checks that all options which appear in fluxnet.card/SubJobParams also appear in FLUXNET/PARAM/run.def
+) The script checks that all options which appear in fluxnet.card[SubJobParams] also appear in FLUXNET/PARAM/run.def
 ) The script writes all of these options into the ${}/COMP/spinup.card and ${}/PARAM/run.def
 ) The script checks to make sure that the NbSitesParam variables in the fluxnet.card appear in the ${}/PARAM/run.def
 …
 ) The spinup job is launched
 From here, the Job_ENSEMBLE script is not used anymore.  Now the independent spinup jobs control the show.  Things are a little more difficult to follow here unless you are really used to libIGCM.  libIGCM does a very good job of generalizing functions, but that can make it more difficult to find what you are looking for.  It helps to remember that each file name is COMP is a "component", and libIGCM treats them all identically: initilizing with IGCM_comp_Initialize, for example.  As defined in SPINUP/config.card, we only have a single component in a spinup job: SPIN.
 A spinup job runs like the following.  Notice that all of this is carried out in ${}, which is a new directory the above script has created in the FLUXNET directory (something like FI_HyyFLUXNET), NOT in the SPINUP directory.
+From here, the Job_ENSEMBLE script is not used anymore.  Now the independent spinup jobs control the show.  Things are a little more difficult to follow here unless you are really used to libIGCM.  libIGCM does a very good job of generalizing functions, but that can make it more difficult to find what you are looking for.  It helps to remember that each file name in the COMP directory is a "component", and libIGCM treats them all identically: initilizing with IGCM_comp_Initialize, for example.  As defined in SPINUP/config.card, we only have a single component in a spinup job: SPIN.
+A spinup job runs like the following.  Notice that all of this is carried out in ${}, which is a new directory the above script has created in the FLUXNET directory (e.g., FLUXNET/FI_HyyFLUXNET), NOT in the SPINUP directory.
 ) IGCM_comp_Initialize/libIGCM_comp.ksh reads in the UserChoices from ${}/COMP/spinup.card, which was created in step (2) above and modified in steps (3) and (7)
 ) IGCM_comp_Update/libIGCM_comp.ksh calls SPIN_Update from ${}/COMP/spinup.driver, which determines we are on the "start" stage and therefore need to follow STOI instructions for the next step (referred to as SECSTOINI inside spinup.driver)
 …
 == Practical steps  ==
 This section gives step-by-step instructions for getting the simulations working with a clean svn install of ORCHIDEE-CAN r6378.  The "General steps" section gives an idea of how to figure things out yourself.
 First, duplicate the ENSEMBLE directory to a name of your own choosing (I choose the name FLUXNET here).
+This section gives step-by-step instructions for getting the simulations working with a clean svn install of ORCHIDEE-CAN r6414.  The "General steps" section gives an idea of how to figure things out yourself.
+First, duplicate the ENSEMBLE directory to a name of your own choosing (I choose the name FLUXNET here, to match the previous section).
 {{{
 cd config/ORCHIDEE_OL
 cp -r ENSEMBLE FLUXNET
+cd FLUXNET
 }}}
 …
 }}}
 I have noticed that the script will complain if a value is specified in fluxnet.card but not the run.def.  It will not complain if a value is specified in run.def and not fluxnet.card.  Check the [UserChoices] and [SubJobParams] sections of fluxnet.card.  Many of the UserChoices are already in SPINUP/COMP/spinup.card, and many of the SubJobParams are in the run.def.  It seems that the scripts make decisions based on what is in fluxnet.card, so this should typically take precedence.
+I have noticed that the script will complain if a value is specified in fluxnet.card but not the run.def.  It will not complain if a value is specified in run.def and not fluxnet.card.  Check the [UserChoices] and [SubJobParams] sections of fluxnet.card.  Many of the UserChoices are already in SPINUP/COMP/spinup.card, and many of the SubJobParams are in the run.def.  It seems that the scripts make decisions based on what is in fluxnet.card, so this should typically take precedence.  I will point out the exact changes I make for r6414 below.
 Before we get to some specifics, let's create the jobs.
 …
 {{{
 vi Job_FLUXNET
+(make the queue "medium" instead of "mediump" on obelix: #PBS -q mediump)
 (change RUN_DIR_PATH=/home/scratch01/mmcgrath/RUN_DIR)
 (change JobType=DEV if you are not sure this will work)
 …
 }}}
 COMMENT OUT ANY OTHER GROUPS LINES.  Else, when you submit the job, you will launch a run over all of the sites in all groups, and you have to cancel them one at a time.
 The length of the spinup also matters.  I use the following for the moment (in fluxnet.card, also in SPINUP/spinup.card)
+COMMENT OUT ANY OTHER LINES THAT BEGIN WITH "Groups".  Else, when you submit the job, you will launch a run over all of the sites in all groups, and you have to cancel them one at a time.  From experience, this is painful.
+The length of the spinup also matters.  I use the following for production runs at the moment (in fluxnet.card...I also change the values in SPINUP/COMP/spinup.card, even though those should be overwritten by Job_FLUXNET)
 {{{
 …
 (I use duree_sechiba=50 for my test run, so that it goes a little faster).  All of the other duree values I set to 0.  This launches a simulation over one loop of the forcing file, then 500 years (regardless of the length of the forcing file), and then one final loop for analysis.
 The section in the fluxnet.card with [SubJobParams] deserves special mention.  As of a recent version of CAN, the run.def has been restructured to include two files: orchidee.def, orchidee_pft.def. This makes the run.def much neater and matches what is done in the coupled simulations.  However, the Job_ENSEMBLE script attempts to change some variables in the run.def that fall under the [SubJobParams] section.  To do this, it looks at the actual run.def file, not any included file.  If it does not find a line in the run.def corresponding to the lines in [SubJobParams], it will crash.  So make sure all the lines you specific under [SubJobParams] in fluxnet.card also explicitly appear in the PARAM/run.def file.
+The section in the fluxnet.card with [SubJobParams] deserves special mention.  As of a recent version of CAN, the run.def has been restructured to include two files: orchidee.def, orchidee_pft.def. This makes the run.def much neater and matches what is done in the coupled simulations.  However, the Job_ENSEMBLE script attempts to change some variables in the run.def that fall under the [SubJobParams] section.  To do this, it looks at the actual run.def file, not any included file.  If it does not find a line in the run.def corresponding to the lines in [SubJobParams], it will crash.  So make sure all the lines you specify under [SubJobParams] in fluxnet.card also explicitly appear in the PARAM/run.def file.
 {{{
 …
 NINPUT_SUFFIX_YEAR = n
 (remove the following lines, otherwise ORCHIDEE will not find a land point for any site outside of this window)
+(make sure the following lines are commented out, otherwise ORCHIDEE will not find a land point for any site outside of this window)
 LIMIT_WEST=8
 LIMIT_NORTH=48
 …
 }}}
+Note that many of the Nitrogen variables above were also in orchidee.def!  Remove them there.  Also remove ATM_CO2 that was already existing in the run.def.
+Note that many of the Nitrogen variables above were also in PARAM/orchidee.def!  Remove the following from PARAM/orchidee.def:
+{{{
+Nammonium_FILE = ndep_nhx.nc
+Nammonium_VAR = nhx
+Nnitrate_FILE = ndep_noy.nc
+Nnitrate_VAR = noy
+Nfert_FILE = NONE
+Nfert_VAR = nfer
+Nmanure_FILE = NONE
+Nmanure_VAR = Nmanure
+Nfert_cropland_FILE = nfert_cropland.nc
+Nfert_cropland_VAR = nfer
+Nmanure_cropland_FILE = nmanure_cropland.nc
+Nmanure_cropland_VAR = Nmanure
+Nfert_pasture_FILE = nfert_pasture.nc
+Nfert_pasture_VAR = Nfer
+Nmanure_pasture_FILE = nmanure_pasture.nc
+Nmanure_pasture_VAR = Nmanure
+Nbnf_FILE= bnf.nc
+Nbnf_VAR= BNF_MGN_PERM2_PERYR
+}}}
+Also remove the ATM_CO2 that was already existing in the PARAM/run.def:
+{{{
+ATM_CO2 = _AUTO_: DEFAULT = 350.
+}}}
 The addition of the orchidee.def and orchidee_pft.def required adding them to the [ParametersFiles] in SPINUP/SUBJOBS/OOL_SEC_STO/COMP/orchidee_ol.card, so that libIGCM copies the new files to the PARAM directory of the running code.  It also required changes to the driver, to select from the correct orchidee_pft.def file.  To fix this, I simply copied OOL_SEC_STO_FG2/COMP/orchidee_ol.* to SPINUP/SUBJOB/OOL_SEC_STO/COMP/.
 …
 cp ../OOL_SEC_STO_FG2/COMP/orchidee_ol.* ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/
 }}}
 This also required adding the following to the [UserChoices] section in SPINUP/SUBJOB/OOL_SEC_STO/COMP/orchidee_ol.card
+This also required adding the following to the [UserChoices] section in SPINUP/SUBJOB/OOL_SEC_STO/COMP/orchidee_ol.card, since the SPINUP/COMP/spinup.driver looks for them:
 {{{
 …
 TIMELENGTH=y
 }}}
+Notice that the SPINUP/SUBJOB/OOL_SEC_STO/COMP/orchidee_ol.card defines the age classes and PFTs that you will be using.  For the moment, we have selected our fluxnet.card to have a certain number of PFTs and age classes, but we have **not** conveyed this choice to libIGCM in any way.  We do that by changing the SPINUP/SUBJOB/OOL_SEC_STO/COMP/orchidee_ol.card value of DefSuffix:
+{{{
+vi ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/orchidee_ol.card
+DefSuffix = 28pft.1ac
+}}}
+Make sure this matches with the fluxnet.card that you copied at the beginning!
 The script expects any variables in the NameSitesParam keyword of fluxnet.card to be present in PARAM/run.def.  SECHIBA_VEGMAX is currently in PARAM/orchidee_pft.def_*.  So, depending on what you have present in SPINUP/SUBJOB/OOL_SEC_STO/COMP/orchidee_ol.card for DefSuffix, you need to move the following lines to PARAM/run.def from PARAM/orchidee_pft.def_DefSuffix, removing the lines in PARAM/orchidee_pft.def_DefSuffix.  The the specific case of 28 PFTs that we are using here:
 …
 SECHIBA_VEGMAX__28=0.0357142857143
+cp PARAM/*def ../SPINUP/SUBJOB/OOL_SEC_STO/PARAM/
 }}}