= HOWTO install/compile/run ORCHIDEE-CN-P for offline use with libIGCM. == This howto is based on the information which can be found in the ORCHIDEE wiki regarding the trunk version of ORCHIDEE. Please check the wiki for information regarding the respective commands listed here. ---- ---- == 1. install libIGCM & ORCHIDEE-CN-P == === 1.1 where to install === You should install the model NOT in the home directory; it is even too small to handle diagnostics like "out_execution" or "out_orchidee". You can use , for example, your work directory: curie: {{{ /ccc/work/cont003/dsm/ }}} or project storage; here example for the IMBALANCE-P project on obelix: {{{ /home/surface3/dgoll }}} === 1.2 install libIGCM & trunk ORCHIDEE === {{{ svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl cd modipsl/util ./model ORCHIDEE_trunk }}} === 1.3 Exchange the trunk ORCHIDEE with ORCHIDEE-CN-P === {{{ cd ../modeles rm -fr ORCHIDEE svn co svn://forge.ipsl.jussieu.fr/orchidee/branches/ORCHIDEE-CN-P ORCHIDEE cd ../util ; ./ins_make }}} ---- ---- == 2. Compile ORCHIDEE == === 2.1 modify IOIPSL === adjust IOIPSL to be able to write more variables into the output by increase the value of max_var by 50 in ../modeles/IOIPSL/src/restcom.f90 {{{ & max_var=550, max_file=50, max_dim=NF90_MAX_VAR_DIMS }}} === 2.2 compile orchidee === ==== A: on obelix ==== {{{ cd ../modeles/ORCHIDEE }}} First do a "normal" make to compile XIOS & IPOSL components which are needed; don't worry if the compilation of ORCHIDEE crashes {{{ make }}} then compile (do compile always with "-parallel mpi" as this executable can be run in sequential mode, too. But not vice versa) {{{ ./makeorchidee_fcm -driver -arch ifort_LSCE -noxios -parallel mpi }}} ==== B: on curie ==== WARNING: before you do anything on curie; make sure you have copied the p86ipsl login environment to your home according to [https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocBenvBtgcc]: {{{ cp ~p86ipsl/.bashrc_curie ~/. cp ~p86ipsl/.bash_profile ~/. }}} Then compile the standard way: {{{ cd ../config/ORCHIDEE_OL gmake without_xios }}} Optional: I usually re-compile ORCHIDEE again with makeorchidee_fcm to set options (don't forget to disable xios): {{{ cd ../../modeles/ORCHIDEE ./makeorchidee_fcm -driver -parallel mpi -arch X64_CURIE -driver -noxios }}} ---- ---- == 3. Run ORCHIDEE == There are different setups possible with libIGCM. Please check the documentation of libIGCM for more information, here I just tell you what you need to do. === 3.1 SPINUP_ANALYTIC === ==== 3.1.1 create new experiment folder ==== {{{ cd ../../config/ORCHIDEE_OL/ cp -fr SPINUP_ANALYTIC/ MY_SPINUP_ANALYTIC/ cd MY_SPINUP_ANALYTIC/ }}} ==== 3.1.2 tell libIGCM about the new input files (write me an email if you need them) ==== modify the variable "ListNonDel" in COMP/stomate.card: {{{ ListNonDel= (${R_IN}/SRF/reftemp.nc, .), \ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/USDA_SoilSuborder.nc, .), \ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/lithology.nc, .), \ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/deposition.nc, .) }}} WARNING: the deposition file is a static field at the moment; it does not include N deposition either. At the moment N deposition is set to P deposition in the ORCHIDEE code. Anyway N deposition is not accounted for in the calculations for the N & P cycle for now. tell that we use the new “soil_param.nc”; in COMP/sechiba.card: {{{ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/soils_param.nc, .), \ }}} ==== 3.1.3 adjust the libIGCM ==== disable XIOS in COMP/orchidee_ol.card {{{ # Use XIOS as output library instead of IOIPSL XIOS=n }}} disable XIOS in PARAM/run.def {{{ # Use XIOS for writing diagnostics file # defulat = n XIOS_ORCHIDEE_OK = n }}} set the IO frequency to avoid crashes in PARAM/run.def {{{ # Writefrequency in seconds in sechiba_history.nc # default = 86400.0 WRITE_STEP = 1800.0 # Writefrequency in days in stomate_history.nc # default = 10. STOMATE_HIST_DT = 1 }}} add the new flags for the nutrient cycles in PARAM/run.def: {{{ # to activate P cycle set =y STOMATE_OK_PCYCLE=y # to activate N cycle set =y STOMATE_OK_NCYCLE=y # make sure we dont fake LD_FAKE_HEIGHT=n # you can enable mass conservation and stoichiometry checks by =y STOMATE_MASS_CONSERVATION=y STOMATE_DSG_DEBUG=y }}} in addition you should add all the PFT and soil parameter values as set in /home/users/dgoll/ORCHIDEE/DEFS_n_CARDS/run.def disable river routing if you run on non-global scale in run.def {{{ # Activate river routing RIVER_ROUTING = n }}} ==== 3.1.4 adjust the job as usual ==== modify config.card as usual (see wiki) modify run.def as usual (see wiki) plus if you want to archive, for example in the IMBALANCE-P storage; add to config.card: {{{ ARCHIVE=/home/surface3/dgoll }}} ==== 3.1.5 create job ==== {{{ ../../../libIGCM/ins_job }}} ==== 3.1.6 adjust job_file (optional) ==== Header to run a site scale simulation on obelix {{{ ###################### ## OBELIX LSCE ## ###################### #PBS -N test #PBS -m a #PBS -j oe #PBS -q short #PBS -o Script_Output_test.000001 #PBS -S /bin/ksh #PBS -v BATCH_NUM_PROC_TOT=1 #PBS -l nodes=1:ppn=1 }}} to automatically have existing files deleted, set JobType=DEV {{{ #D- Experience type : DEB(ug), DEV(elopment), RUN (default) JobType=DEV }}} to avoid queuing take full advantage of your job time; the value depends on #PBS -q short and computation time of orchidee {{{ #D- Number of execution in one job PeriodNb=60 }}} to have a temp directory you can access on obelix; exchange dgoll with your ID {{{ #D- Define running directory #D- Default=${TMPDIR} ie temporary batch directory #D- #RUN_DIR_PATH=/workdir/or/scratchdir/of/this/machine RUN_DIR_PATH=/home/scratch01/dgoll/RUN_DIR }}} ==== 3.1.7 submit the job ==== on obelix: {{{ qsub Job_ID }}} on curie {{{ ccc_msub Job_ID }}} === 3.2 ENSEMBLE: fluxnet sites [http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation] === ==== 3.2.1 create new experiment folder ==== {{{ cd ../../config/ORCHIDEE_OL/ cp -fr ENSEMBLE/ MY_ENSEMBLE/ cd MY_ENSEMBLE/ }}} ==== 3.2.2 tell libIGCM about the new input files (write me an email if you need them) ==== The configuration which are used are stored in ../SPINUP/SUBJOB/OOL_SEC_STO/COMP/ so, do as in 4.1.2: change the path of soil_param.nc and add the new files sechiba.card: {{{ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/soils_param.nc, .), \ }}} stomate.card: {{{ ListNonDel= (${R_IN}/SRF/reftemp.nc, .) \ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/USDA_SoilSuborder.nc, .), \ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/lithology.nc, .), \ (/home/scratch01/dgoll/ORCHIDEE/ORC-CNP/simulations/simple_forcing_DSG/deposition.nc, .) }}} WARNING: the deposition file is a static field at the moment; it does not include N deposition either. At the moment N deposition is set to P deposition in the ORCHIDEE code. Anyway N deposition is not accounted for in the calculations for the N & P cycle for now. copy a run.def with all parameters needed to PARAM, for example this one: {{{ cp ~/ORCHIDEE/DEFS_n_CARDS/run.def_NV PARAM/run.def }}} ==== 3.2.3 adjust the libIGCM ==== add the new flags for the nutrient cycles in fluxnet.card: {{{ # to activate P cycle set =y STOMATE_OK_PCYCLE=y # to activate N cycle set =y STOMATE_OK_NCYCLE=y # make sure we dont fake LD_FAKE_HEIGHT=n # you can enable mass conservation and stoichiometry checks by =y STOMATE_MASS_CONSERVATION=y STOMATE_DSG_DEBUG=y }}} you need to add defaults for these switches into PARAM/orchidee.default {{{ # STOMATE_OK_NCYLE ([FLAG]) : Activate the nitrogen cycle {OK_STOMATE} STOMATE_OK_NCYCLE = n # STOMATE_OK_PCYLE ([FLAG]) : Activate the phosphorus cycle {OK_STOMATE} STOMATE_OK_PCYCLE = n # STOMATE_MASS_CONSERVATION ([FLAG]) : activate mass conservation checks which force stop if violated {OK_STOMATE} STOMATE_MASS_CONSERVATION = n # STOMATE_DSG_DEBUG ([FLAG]) : activate checks for stoichiometry and negative pools {OK_STOMATE} STOMATE_DSG_DEBUG = n # LD_FAKE_HEIGHT ([FLAG]) : fakes height of vegetation (ORCHIDEE-CAN related) {OK_STOMATE} LD_FAKE_HEIGHT = n }}} specify the spinup simulations in the config.card: {{{ [SPINUP] # Initialisation for spin-up : # orchidee with sechiba alone (!!! if ok_stomate == n !!!) duree_nostomate=0 # orchidee with stomate; number of years before n_iter cycles of duree_sechiba years of analytical spinup cycles duree_inistomate=250 # teststomate (only if duree_inistomate > 0) duree_offlineini=0 # Loop over ORCHIDEE runs (used for spin-up) # The whole job is restarted n_iter times # n_iter: the number of cycles with duree_sechiba years n_iter=3 # orchidee with sechiba (and stomate if ok_stomate=y below) duree_sechiba=50 # teststomate duree_stomate=0 # forcesoil duree_carbonsol=1 # Final run (full ORCHIDEE) # This last parameter must be non-zero. duree_final=100 }}} xios is per default disabled in "PARAM/orchidee.default" routing is per default disabled in "fluxnet.card" ==== 3.2.4 adjust the job as usual ==== modify config.card as usual ([http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation]) modify run.def as usual ([http://forge.ipsl.jussieu.fr/orchidee/wiki/Scripts/FluxnetValidation]) plus if you want to archive, for example in the IMBALANCE-P storage; add to config.card: {{{ ARCHIVE=/home/surface3/dgoll }}} modify fluxnet.card as usual (no information): - if you want to keep the output files of the spinup you need to set the following variables {{{ # DEBUG mode for SPINUP # # This mode keep all SPINUP directory in ARCHIVE # If disable, all ARCHIVE is automaticly cleaned. DEBUG_SPIN=y # If you don't want to keep old spinup steps, but last one CONSERVE=y }}} ==== 3.2.5 create job ==== {{{ ../../../libIGCM/ins_job }}} ==== 3.2.6 adjust job (optional) ==== ==== 3.2.7 start simulations ==== {{{ my_prompt>> ksh my_prompt>> nohup ./Job_ENSEMBLE fluxnet > out_Job 2>&1 & }}} ==== 3.2.8 check simulations ==== to see the decomposition of the spatial domain among the processors check in the run dir "Load_balance_orchidee.dat" on curie {{{ ccc_mstat -u userid }}} {{{ ccc_mdel jobid }}} ==== 3.2.8 post processing ==== ensure the Jobname and the path to the reference simulation is set in fluxnet.card; in case there is no reference set it to NONE {{{ # History file of former ORCHIDEE runs (Reference) to compare with the current simulations reference_base_path=/home/scratch01/dgoll/IGCM_OUT/OL2/DEVT/ensemble reference_config=DSG04EnsCN }}}