Last edited [[Timestamp]] [[BR]] [[PageOutline]] = '''Using libIGCM library with ORCA2_LIM''' = [[BR]] == Purposes: == [[BR]] * Document the using of libIGCM for NEMO, in configuration ORCA2_LIM [[BR]] [[BR]] == Quick start guide: == [[BR]] * Define the following alias to use SVN : [[BR]] {{{ alias svn_ano='svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl' }}} * Create and go into working directory : [[BR]] {{{ mkdir TRY ; cd TRY }}} * Extract modipsl : [[BR]] {{{ svn_ano }}} * Extract NEMO : [[BR]] {{{ cd modipsl/util ./model NEMO_IGCM }}} * Choosing ORCA2_LIM configuration : [[BR]] {{{ cd ../modeles/UTIL ./fait_config ORCA2_LIM }}} * Install the makefiles : [[BR]] {{{ cd ../../util ./ins_make }}} * Compile : [[BR]] {{{ cd ../config/ORCA2_LIM gmake }}} * Submit Job : [[BR]] {{{ ( Editing at this point modipsl/config/ORCA2_LIM/IGCM00/config.card ) ../../util/ins_job cd IGCM00 qsub Job_NL1 }}} [[BR]] [[BR]] == Set up a simulation: == [[BR]] * '''First of all see official documentation''' http://forge.ipsl.jussieu.fr/libigcm/wiki/ * '''All parameters in config.card file''' In modipsl/config/ORCA2_LIM/IGCM00/'''config.card''' file you can set all parameters to manage your experience: '''How to manage restart files ( in the same simulation )''' in config.card you have to set duration of your experience and setting !PeriodLenght parameter, so you create restart files. ( for example : if you have and experience of 10 years and a !PeriodLenght of 1 year, you will have 10 restart files ) #-- Début et fin de Job [[BR]] #-- "YYYY-MM-DD" [[BR]] DateBegin=2001-01-01 [[BR]] DateEnd=2010-12-31 [[BR]] #============================ [[BR]] #-- 1Y, 1M, 5D, 1D [[BR]] PeriodLength=1Y [[BR]] '''Note : ''' If you want to use restart data from another simulation ( already run not in this moment ) you have to manage "Restarts" part of config.card #================ [[BR]] #D-- Restarts -[[BR]] [Restarts] '''How to manage write output frequency''' in config.card you have to set !WriteFrequency of your experience ( for example : if you want output files every 5 days you have to put for Ocean #=================== [[BR]] #D-- OCE - [[BR]] [OCE] [[BR]] WriteFrequency="5D" and if you want output files every 5 days you have to put for ICE #=================== [[BR]] #D-- ICE - [[BR]] [ICE] [[BR]] WriteFrequency="5D" ) '''How to manage calendar type''' in config.card you have to set !CalendarType like 360d or leap or noleap ( for example : if you want 360 days you have to be careful of DateEnd=2010-12-3'''0''' [[BR]] because every month has 30 days If you put for example DateEnd=2005-12-31, you will have another run, of duration like your !PeriodLenght ) '''How to manage rdt ( i.e. timestep )''' in COMP/nemo.driver you have to change parameter : ORCA_NPDT_JOUR=15 ( 15 corresponds to 86400s/5760s; it does mean that in one day you have 15 iterations; and it does mean that timestep is 5760) '''How control all parameters in modipsl/config/ORCA2_LIM/IGCM00/PARAM/namelist''' libIGCM change namelist's values but not in namelist's file, so it is possible to control all changes of namelist's parameters looking at : {OUTPUT_DIR}/IGCM_OUT/ORCA2_LIM/N360d_5d/OCE/Debug/!JobName_date_date_ocean.output '''How to activate Monitoring''' in config.card file #D- If you want to produce time series, this flag determines #D- frequency of post-processing submission TimeSeriesFrequency=NONE you have to put you experience lenght. If you do an experience during 10 years you have to put:[[BR]] TimeSeriesFrequency=10Y '''How to activate Atlas''' in config.card file #D- If you want to produce seasonal average, this flag determines #D- the period of this average SeasonalFrequency=NONE you have to put you experience lenght. If you do an experience during 10 years you have to put:[[BR]] SeasonalFrequency=10Y '''How to manage frequency of post processing''' ( Note: in this moment is not possible to have atlas and monitoring of frequency different from '''1M''', if we want this we've to see it with Patrick Brockman!! ) in COMP/nemo.card you have to change in [OutputFiles] chapter: Post_'''1M'''_grid_T (U,V,W) and you have to change [Post_'''1M'''_grid_T(U,V,W)] chapter. and in in COMP/lim2.card you have to change in [OutputFiles] chapiter: List= (${PREFIX_NWRITE}_${DATE_OPA}_icemod.nc, ${R_OUT_ICE_NWRITE}/${PREFIX}_${WF1}_icemod.nc, Post_'''1M'''_icemod) and you have to change [Post_'''1M'''_icemod] chapter. * '''How to make a parallel run ''' first of all add a cpp key: http://www.nemo-ocean.eu/index.php//Using-NEMO/User-Guides/Advanced/How-to-add-modify-new-modules-or-new-cpp-keys and then see: http://www.nemo-ocean.eu/index.php//Using-NEMO/User-Guides/Advanced/How-to-set-up-one-simulation#eztoc1210_5 * '''How to have mail of end of Job, sur ccrt's machines''' on mercure: in Job_Name: #PBS -M name@mail on platine: In Job_Name: #BSUB -u name@mail [[BR]] [[BR]] == Run only post processing: == [[BR]] * '''If you have already run a job and you need only to run post processing''' you have to move Script_Output.xxx in Job_Name you have to change: #PBS -l cputim_job=4:00:00 # Limite temps [[BR]] DRYRUN=3 [[BR]] [[BR]] == List of possible errors in modipsl/config/ORCA2_LIM/IGCM00/Script_Output_''!JobName'' : == [[BR]] * '''Error : TERM = (vt100) ${HOME:-.}/.profile: .bashrc: not found''' * '''Error : ksh: D--------------------------------------------------------------------==: not found''' it can depends on '''.profile''' file. To solve it:: .profile file it can be like: «# .profile (idris) [[BR]] # Fichier d environnement a charger sur la frontale TX7 unix LINUX [[BR]] # Vous pouvez mettre a jour le fichier .profile_tx [[BR]] if [ !`hostname!` = "brodie" -a -f $HOME/.profile_tx ]; then [[BR]] . $HOME/.profile_tx [[BR]] # Fichier d environnement a charger sur les noeuds SX8 unix SUPER-UX [[BR]] # Vous pouvez mettre a jour le fichier .profile_sx [[BR]] elif [ !`hostname!` != "brodie" -a -f $HOME/.profile_sx ]; then [[BR]] . $HOME/.profile_sx [[BR]] else [[BR]] echo [[BR]] fi » * '''Error : cat: cannot open ........./modipsl/config/ORCA2_LIM/IGCM00/../.resol ''' : it is due to modipsl/libIGCM/AA_job file. To solve it:: change in this way: #-- Resolution FLAG for oasis input [[BR]] #RESOL=$(cat ${SUBMIT_DIR}/../.resol) [[BR]] RESOL=ORCA2 '''NOTE: Be careful because this is ok for ORCA2, but if we use ORCA0.5 ore ORCA0.25 is NOT TRUE!!!!!''' go in modipsl/config/ORCA2_LIM/IGCM00 directory cancel '''run.card.init''' and '''Job_''Name''' files and then do an '''ins_job''' again. (does not still fixed in libIGCM_v1_'''2''') * '''Error : Input files and BC for NEMO DO NOT EXIST (i.e. sst_data.nc, sss_data.nc, geothermal_heating.nc, bathy_level.nc, etc...)''' IGCM_sys_Get, ERROR : regular file ...../OCE/ORCA2_LIM/bathy_level.nc DOES NOT EXIST . To solve it:: In this moment we've still to download on gaya.idris.fr:~/!reee512/NEMO/ORCA2_LIM_nemo_v3.tar file, and to untar it into the directory: ${R_BC}/OCE/ORCA2_LIM and where ${R_BC} is a variable that can be changed in modipsl/config/ORCA2_LIM/IGCM00/config.card file. * '''Error : IGCM_debug_Exit : IGCM_config_Initialize Error !PeriodState : Fatal ''' To solve it:: change line in modipsl/config/ORCA2_LIM/IGCM00/run.card !PeriodState=Fatal [[BR]] in [[BR]] !PeriodState= or rm modipsl/config/ORCA2_LIM/IGCM00/run.card * '''Error : compiling ''' ''**** 96 Loop count is greater than that assumed by the compiler : loop-count=27118 eln=94 PROG=fldread.fld_read ELN=94(4001a8cf0) Called from sbcblk_clio.sbc_blk_clio ELN=190(4001c7fb8) Called from sbcmod.sbc ELN=264(4001154d0) Called from step.stp ELN=236(40005c9c4) Called from opa.opa_model ELN=133(400000e28) Called from model ELN=19(4000008b0)''' To solve it:: Compiling with new files: new '''mod.def''', '''fldread.F90''', '''fait_AA_make''', '''BB_make''' (in KEEPED_FILES) * '''ls: ERROR: Cannot access stack: No such file or directory''' in Script_Outputxxx: ( '''is it a problem with filemane'''? ) ( '''which is the difference between Out and OCE/Output directories???''' ) IGCM_sys_Put_Out : out_nemo /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N365d_5d/Out/N365d_5d_20030101_20031231_out_nemo [[BR]] IGCM_sys_MkdirArchive : /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N365d_5d/Out [[BR]] IGCM_sys_Chmod : 444 out_nemo [[BR]] --Debug1--> OCE [[BR]] --Debug1--> ICE [[BR]] Files that will be deleted before next period-run : [[BR]] ls: ERROR: Cannot access stack: No such file or directory [[BR]] -r--r--r-- N365d_5d_00016425_restart.nc [[BR]] -r--r--r-- N365d_5d_00016425_restart_ice.nc [[BR]] (see if it is ok in libIGCM_v1_'''2''') [[BR]] [[BR]] == List of possible remarks and suggestions : == [[BR]] * '''be careful at group (for example for a gensi one)''' if we use a common account on CCRT we have to be sure that everygroup can have access to it, this is important for chmod 444, do another test!!!!!! * '''can we leave these files if the don't exist anymore?????''' WARNING : IGCM_sys_Put_Out xxx_trends.nc DOES NOT EXIST [[BR]] WARNING : IGCM_sys_Put_Out xxx_diagap.nc DOES NOT EXIST [[BR]] WARNING : IGCM_sys_Put_Out output.abort.nc DOES NOT EXIST * '''Every time that we change modipsl/config/ORCA2_LIM/IGCM00/config.card''' we have to cancel modipsl/config/ORCA2_LIM/IGCM00/run.card.init cancel modipsl/config/ORCA2_LIM/IGCM00/Job_''Name'' and run ./modipsl/util/ns_job script * '''To perform a second simulation after a run with !PeriodState=Fatal''' cancel modipsl/config/ORCA2_LIM/IGCM00/stack_error, modipsl/config/ORCA2_LIM/IGCM00/Script_Output_''!JobName'' but do not cancel modipsl/config/ORCA2_LIM/IGCM00/run.card.init file, if not run.card file is not re-created. ( File run.card.init is created with ins_job script ) * '''Does it exists a caption file for run.card.init options?''' In to run.card.init file who can we undesrtand syntax of? : !TimeSeriesRunning=n [[BR]] !TimeSeriesCompleted= [[BR]] !SeasonalRunning=n [[BR]] Is there a reference file? * '''test with trunk/libIGCM, ( and tags/libIGCM_v1_2 too? )''' To use libIGCM trunk version it is necessary to add in config.card in POST Session: !RebuildFromArchive=NONE NOTE: We wait for !http://forge.ipsl.jussieu.fr/libigcm/browser/tags version 2 to do changes in config.card: !http://forge.ipsl.jussieu.fr/nemo/svn/branches/libIGCM/ORCA2_LIM/IGCM00/ * '''Change in nemo.card or in nemo.driver variable: ORCA_NPDT_JOUR and OPA_NPDT_JOURS''' * '''Have informations about mailing system for good jobs, on ccrt and idris''' [[BR]] [[BR]] == Done : == [[BR]] * '''run on CCRT machines ( mercure )''' Tests for Calendar Type, Output frequency and Restarts: [[BR]] '''Simulation environment''' #- Repository informations [[BR]] #-[[BR]] #-S- 1 cvs !anonymous@cvs.ipsl.jussieu.fr:/home/ioipsl/CVSROOT[[BR]] #-S- 2 cvs !sechiba@cvs.ipsl.jussieu.fr:/home/ssipsl/CVSREP[[BR]] #-S- 3 cvs !lmdzbrowse@cvs.lmd.jussieu.fr:/home/cvsroot[[BR]] #-S- 4 cvs !opa@cvs.ipsl.jussieu.fr:/home/opalod/CVSROOT[[BR]] #-S- 5 cvs !nemo@cvs.ipsl.jussieu.fr:/home/opalod/NEMOCVSROOT[[BR]] #-S- 6 cvs !inca@cvs.ipsl.jussieu.fr:/home/incaipsl/CVSROOT[[BR]] #-S- 7 svn --username nemo_user !http://forge.ipsl.jussieu.fr/nemo/svn [[BR]] #-S- 8 svn !http://forge.ipsl.jussieu.fr/igcmg/svn [[BR]] #-S- 9 svn !http://forge.ipsl.jussieu.fr/inca/svn [[BR]] #-S- 10 svn !http://forge.ipsl.jussieu.fr/libigcm/svn [[BR]] #-H- NEMO_IGCM NEMO[[BR]] #-H- NEMO_IGCM OPA[[BR]] #-H- NEMO_IGCM LIM[[BR]] #-H- NEMO_IGCM TOP[[BR]] #-H- NEMO_IGCM IOIPSL/src - svn - tag v2_1_4[[BR]] #-H- NEMO_IGCM libIGCM - svn -[[BR]] #-H- NEMO_IGCM NEMO sources and configurations - svn - trunk[[BR]] #-M- NEMO_IGCM !opatlod@locean-ipsl.upmc.fr[[BR]] #-C- NEMO_IGCM IOIPSL/tags/v2_1_4/src HEAD 8 IOIPSL/src modeles [[BR]] #-C- NEMO_IGCM trunk/libIGCM 47 10 libIGCM . [[BR]] #-C- NEMO_IGCM tags/nemo_v3_1_beta/NEMO HEAD 7 . modeles [[BR]] #-C- NEMO_IGCM tags/nemo_v3_1_beta/UTIL HEAD 7 . modeles [[BR]] #-C- NEMO_IGCM tags/nemo_v3_1_beta/CONFIG/GYRE_LOBSTER HEAD 7 GYRE_LOBSTER config [[BR]] #-C- NEMO_IGCM tags/nemo_v3_1_beta/CONFIG/ORCA2_LIM HEAD 7 ORCA2_LIM config [[BR]] #-C- NEMO_IGCM branches/libIGCM/ORCA2_LIM/IGCM00 HEAD 7 IGCM00 config/ORCA2_LIM [[BR]] Simulation duration = '''5 year ''' !PeriodLength = '''1Y''' timestep: rdt = '''5760''' equivalent to ORCA_NPDT_JOUR = '''15''' || || || '''Output Frequency''' || '''Output Frequency''' || '''Output Frequency''' || || || || 5d || 1M || 1Y || || '''Calendar Type '''|| 360d || '''ok''' || '''ok''' || '''ok''' || || '''Calendar Type '''|| 365d || '''ok''' || '''ko''' || '''ok''' || || '''Calendar Type '''|| leap || '''ok''' || '''ko''' || to do || ko = there are only 11 output ( point how NEMO do averadge for every month ) '''NOTE: control in:''' - why in /dmnfs13/cont003/flavoni/IGCM_OUT/ORCA2_LIM/N360d_1m/OCE/Debug/*_ocean.output Namelist namrun job number no = '''0''' is EVER = 0???? '''N365d_1m_20011231_restart.nc''' Control if file name is correct!!!! (my job is N365d_1m) [[BR]] [[BR]] == To do list : == [[BR]] * simulation=100Y ( ongoing with tags/libIGCM_v1_2 ) [[BR]] periodlenght=10y[[BR]] and sortiee=1Y[[BR]] * '''monitoring''' : ongoing on mercure for experience=10Y, !PeriodLenght=1Y, !WriteFrequency=1M * '''atlas''' : ongoing on mercure for experience=10Y, !PeriodLenght=1Y, !WriteFrequency=1M NOTE: change COMP/nemo.card and set OutputFiles and Post_''xX''_grid_T[U,V,W], where ''xX''=!WriteFrequency * test mpi [[BR]] [[BR]] == NOTE files to change for NEMO : == [[BR]] * for 360d CalendarType CHANGE config/ORCA2_LIM/IGCM00/COMP/'''nemo.driver''' [[BR]] and modipsl/modeles/NEMO/WORK/'''domain.F90''' ( see in !http://forge.ipsl.jussieu.fr/igcmg/browser/CONFIG/IPSLCM/IPSLCM5/trunk/EXP00/COMP and there is not "leap" parameter in all *.driver or *.card???? how does it work for Sébastien and Arnaud????????? ) * modipsl/modeles/UTIL/'''fait_AA_make''' difference with the old one: in #- Compilation options --- Committed: shade 10000 (for mercure) * modipsl/libIGCM/'''AA_job''' difference with the old one: [[BR]] #RESOL=$(cat ${SUBMIT_DIR}/../.resol) [[BR]] '''RESOL=ORCA2''' * modipsl/modeles/NEMO/WORK/'''domain.F90''' difference with the old one: in line 171 WRITE(numout,*) ' leap year calendar (0/1/'''30''') nleapy = ', nleapy * modipsl/config/ORCA2_LIM/IGCM00/COMP/'''nemo.driver''' difference with the old one: added nleapy variable that corresponds to nbisex on opa.driver # nleapy leap year calendar (0/1) (30 for 360d) PAT_NLEAPY=$( supergrep nleapy ${SUBMIT_DIR}/PARAM/namelist ) in lines 141-151 case ${config_UserChoices_CalendarType} in [[BR]] leap !||Gregorian)[[BR]] ORCA_NLEAPY=1;;[[BR]] noleap)[[BR]] ORCA_NLEAPY=0;;[[BR]] 360d)[[BR]] ORCA_NLEAPY=30;;[[BR]] esac[[BR]] in line 158 -e "s%${PAT_NLEAPY}% nleapy=${ORCA_NLEAPY}%" \ [[BR]] [[BR]] Simona Flavoni last update: [[Timestamp]]