source: tags/libIGCM_v1_4/AA_job

Last change on this file was 185, checked in by sdipsl, 15 years ago

Some tweak with PeriodNB
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Date Author Revision
File size: 18.3 KB
Line 
1#-Q- platine #!/usr/bin/ksh
2#-Q- platine ##################
3#-Q- platine ## PLATINE   CEA ##
4#-Q- platine ##################
5#-Q- platine #BSUB -J ::Jobname::       # nom de la requete
6#-Q- platine #BSUB -N                   # message a la fin du job
7#-Q- platine #BSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
8#-Q- platine #BSUB -n ::JobNumProcTot::             # reservation des processeurs pour le job
9#-Q- platine #BSUB -W 1:00           # Limite temps
10#-Q- platine BATCH_NUM_PROC_TOT=$SLURM_NPROCS
11#-Q- sx8brodie #######################
12#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
13#-Q- sx8brodie #######################
14#-Q- sx8brodie #PBS -N ::Jobname::         # nom de la requete
15#-Q- sx8brodie #PBS -m a                   # message a la fin du job
16#-Q- sx8brodie #PBS -j o                   # regroupement des sorties du job : standard error et output
17#-Q- sx8brodie #PBS -o Script_Output_::Jobname::.000001  # nom du fichier de sortie
18#-Q- sx8brodie #PBS -S /usr/bin/ksh        # shell de soumission
19#-Q- sx8brodie #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
20#-Q- sx8brodie #PBS -q ::JobClass::                # classe multi pour BATCH_NUM_PROC_TOT superieur a 2, mono sinon
21#-Q- sx8brodie #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}  # reservation des processeurs pour le job
22#-Q- sx8brodie #PBS -l memsz_job=8.0gb     # Limite memoire
23#-Q- sx8brodie #PBS -l cputim_job=1:00:00  # Limite temps
24#-Q- aix6 #!/usr/bin/ksh
25#-Q- aix6 # ######################
26#-Q- aix6 # ##   VARGAS IDRIS   ##
27#-Q- aix6 # ######################
28#-Q- aix6 # Nom de la requete
29#-Q- aix6 # @ job_name = ::Jobname::
30#-Q- aix6 # Type de travail
31#-Q- aix6 # @ job_type = parallel
32#-Q- aix6 # Fichier de sortie standard
33#-Q- aix6 # @ output = $(job_name).$(jobid)
34#-Q- aix6 # Fichier de sortie erreur (le meme)
35#-Q- aix6 # @ error =  $(job_name).$(jobid)
36#-Q- aix6 # Nombre de processus demandes
37#-Q- aix6 # @ total_tasks = ::JobNumProcTot::
38#-Q- aix6 # @ environment = "BATCH_NUM_PROC_TOT=::JobNumProcTot::"
39#-Q- aix6 # Temps CPU max. par processus MPI hh:mm:ss
40#-Q- aix6 # @ wall_clock_limit = 1:00:00
41#-Q- aix6 # Memoire max. utilisee par processus
42#-Q- aix6 # @ data_limit = 3.2gb
43#-Q- aix6 # Memoire stack demandee
44#-Q- aix6 # @ stack_limit = 0.3gb,0.3gb
45#-Q- aix6 # Pas d OpenMP
46#-Q- aix6 # @ resources = ConsumableCpus(1)
47#-Q- aix6 # Fin de l entete
48#-Q- aix6 # @ queue
49#-Q- sx8mercure #!/bin/ksh
50#-Q- sx8mercure ######################
51#-Q- sx8mercure ## SX8MERCURE   CEA ##
52#-Q- sx8mercure ######################
53#-Q- sx8mercure #PBS -N ::Jobname::           # nom de la requete
54#-Q- sx8mercure #PBS -m a                     # message a la fin du job
55#-Q- sx8mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
56#-Q- sx8mercure ####PBS -A gen6178
57#-Q- sx8mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
58#-Q- sx8mercure #PBS -S /usr/bin/ksh          # shell de soumission
59#-Q- sx8mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
60#-Q- sx8mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
61#-Q- sx8mercure #PBS -l memsz_job=8.0gb       # limite memoire
62#-Q- sx8mercure #PBS -l elapstim_req=01:00:00 # limite en temps elapsed
63#-Q- sx9mercure #!/bin/ksh
64#-Q- sx9mercure ######################
65#-Q- sx9mercure ## SX9MERCURE   CEA ##
66#-Q- sx9mercure ######################
67#-Q- sx9mercure #PBS -N ::Jobname::           # nom de la requete
68#-Q- sx9mercure #PBS -m a                     # message a la fin du job
69#-Q- sx9mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
70#-Q- sx9mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
71#-Q- sx9mercure #PBS -S /usr/bin/ksh          # shell de soumission
72#-Q- sx9mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
73#-Q- sx9mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
74#-Q- sx9mercure #PBS -l memsz_job=16.0gb       # limite memoire
75#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # limite en temps elapsed
76#-Q- sx9mercure #PBS -A gen6178               # groupe obligatoire sur SX9
77#-Q- default #!/bin/ksh
78#-Q- default ##################
79#-Q- default ## DEFAULT HOST ##
80#-Q- default ##################
81#-Q- default #For MPI use, uncomment next line :
82#-Q- default #BATCH_NUM_PROC_TOT=::JobNumProcTot::
83
84# $Date$
85# $Author$
86# $Revision$
87# IPSL (2006)
88#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
89
90#set -eu
91#set -vx
92
93date
94echo
95echo "#######################################"
96echo "#       ANOTHER GREAT SIMULATION      #"
97echo "#######################################"
98echo
99
100#D--------------------------------------------------------------------==
101#D--------------------------------------------------------------------==
102#D-
103#D-                      Job to launch IGCM models
104#D-
105#D--------------------------------------------------------------------==
106#D--------------------------------------------------------------------==
107#D-
108
109MODIPSL=::modipsl::
110libIGCM=${MODIPSL}/libIGCM
111
112#D--------------------------------------------------------------------==
113#D-                   -1. User modification place
114#D-                      - Job Verbosity
115#D-                      - PeriodNb
116#D-                      - Experience type : DEB(ug), DEV(elopment), RUN
117#D--------------------------------------------------------------------==
118
119#D- Increased verbosity (1, 2, 3)
120Verbosity=3
121#D- Number of execution in one job
122PeriodNb=1
123#-Q- sx9mercure PeriodNb=60
124
125#D-
126# --------------------------------------------------------------------
127#D- ! Optionnal features /!\ Use with care !
128# --------------------------------------------------------------------
129
130#D- Low level debug : to bypass lib test checks and stack construction
131#D- Default=true
132DEBUG_debug=false
133
134#D- Set DEBUG_sys to false to disable output calls of function
135#D- Default=true
136#DEBUG_sys=false
137
138#D- Experience type : DEB(ug), DEV(elopment), RUN (default)
139JobType=RUN
140
141#D- Define running directory
142#D- Default=${TMPDIR} ie temporary batch directory
143#RUN_DIR_PATH=/workdir/or/scratchdir/of/this/machine
144
145#D- Define submit directory
146#D- Default= where you launch qsub (variable from scheduler)
147#SUBMIT_DIR=$( pwd )
148
149#D- Define BIG_DIR directory
150#D- Default=(strongly machine dependant/see libIGCM_sys for details)
151#BIG_DIR=/somewhere/with/lot/of/space/scratchdir/may/be
152
153#D- Turn in dry run mode ? (0,1,2,3)
154#D- Default=0
155#DRYRUN=3
156# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
157# -------------------------------------------------------------------------------------
158# | DRYRUN=  |  Date computations, | sys_Get    |  Exe    | sys_Put_Out; sys_Put_Rest |
159# |          |  Cp/Exe param files |            |  Chmod  |                           |
160# |          |      Qsub           |            |         |                           |
161# -------------------------------------------------------------------------------------
162# |    0     |       yes           |    yes     |  yes    |      yes                  |
163# -------------------------------------------------------------------------------------
164# |    1     |       yes           |    yes     |  yes    |      no                   |
165# -------------------------------------------------------------------------------------
166# |    2     |       yes           |    yes     |  no     |      no                   |
167# -------------------------------------------------------------------------------------
168# |    3     |       yes           |    no      |  no     |      no                   |
169# -------------------------------------------------------------------------------------
170
171#D- Define input file root directory
172#D- Default=/IGCMG/common/account/of/this/machine
173#R_IN=/u/rech/por/rpor111/DATA
174
175#D-
176#D--------------------------------------------------------------------==
177#D-                   0. System Environment
178#D-                      - Define variables for Number of processors
179#D-                      - Define MPI variables
180#D-                      - Define batch scheduler variables
181#D-                      - Source IGCM Library
182#D-                      - Get RESOLution in .resol file (temporary)
183#D--------------------------------------------------------------------==
184
185#D--------------------------------------------------------------------==
186
187. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
188       ( ${DEBUG_debug} ) && IGCM_debug_Check
189. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
190       ( ${DEBUG_debug} ) && IGCM_card_Check
191. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
192       ( ${DEBUG_debug} ) && IGCM_date_Check
193#-------
194. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
195. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
196. ${libIGCM}/libIGCM_comp/libIGCM_comp.ksh
197. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
198
199#D--------------------------------------------------------------------==
200
201#-- Resolution FLAG for oasis input
202[ -f ${SUBMIT_DIR}/../.resol ] && RESOL=$(head -1 ${SUBMIT_DIR}/../.resol)
203#RESOL="ORCA2xLMD9671"
204
205#D-
206#D--------------------------------------------------------------------==
207#D-           1. INITIALIZE CONFIGURATION AND THEN COMPONENTS
208#D--------------------------------------------------------------------==
209
210# --------------------------------------------------------------------==
211#D- Initialize configuration :
212#D-              - Simulation configuration
213#D-              - Simulation parameters
214#D-              - Execution parameter
215#D-              - Define input files directory
216#D-              - Read or initialize CumulPeriod
217#D-              - run.card
218#D-
219# --------------------------------------------------------------------==
220IGCM_config_Initialize
221
222# --------------------------------------------------------------------==
223#D- Define, create and cd RUN_DIR
224#D- 
225# --------------------------------------------------------------------==
226RUN_DIR=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}.${$}
227IGCM_sys_MkdirWork ${RUN_DIR}
228IGCM_sys_Cd ${RUN_DIR}
229
230# --------------------------------------------------------------------==
231#D- Initialize each components taking part
232#D- into simulation configuration :
233#D-         - Define component executable
234#D-         - Define Write Frequency by component "1M" "1D" ...
235#D-         - Source ${comp}.driver
236#D-         - Define Storage Places by component
237#D-         - ${comp}_Initialize :
238#D-         - Patterns to sed
239#D-         - Variables from namelist
240#D-
241# --------------------------------------------------------------------==
242IGCM_comp_Initialize
243
244# --------------------------------------------------------------------==
245#D- Verify compatibility of period length, write frequencies, ...
246#D- Verify compatibility of rebuild choices and post-process choices
247#D- Will stop here if something is wrong
248# --------------------------------------------------------------------==
249#IGCM_config_Check
250
251#D--------------------------------------------------------------------==
252#D-                  2. ENTER THE DEEP EXECUTION LOOP
253#D-                  !!!! FASTEN YOUR SEAT BELTS !!!!
254#D-                  !!!!!      PLEASE.          !!!!
255#D-                  !! WE WILL CROSS TURBULENCES. !!
256#D--------------------------------------------------------------------==
257
258Period=1
259
260PeriodContinue=false
261if [ ${Period} -le ${PeriodNb} ]; then
262    PeriodContinue=true
263fi
264while ( ${PeriodContinue} ) ; do
265
266    echo
267    echo "Starting iteration ${Period} / ${PeriodNb}"
268
269    #D-
270    # ------------------------------------------------------------------
271    #D- COMPUTE AND DEFINE DATE STUFF.
272    # ------------------------------------------------------------------
273    IGCM_config_PeriodStart
274    #D- => ${PeriodDateBegin}
275    #D- => ${PeriodDateEnd}
276    #D- => ${CumulPeriod}
277    #D- => ${DatesPeriod}=${PeriodDateBegin}_${PeriodDateEnd}
278    #D- => ${PeriodLength} => JOURS=31 , 28, 29, 30 ; MOIS=0 ; ANS=0
279    #D- => update run.card value
280   
281    #D-
282    # ------------------------------------------------------------------
283    #D- Get parameters text files updated by job (.def, namelist ...)
284    #D- READ AND USE BY GCM AT EACH EXECUTION.
285    # ------------------------------------------------------------------
286    IGCM_comp_GetInputParametersFiles
287
288    #D-
289    # ------------------------------------------------------------------
290    #D- Get initial state (Etat0, carteveg,relief...)
291    #D- NECESSARY ONLY IF CumulPeriod= (=> NUMERO) == 1
292    #D- AND ???_Restart=NO
293    #D- READ AND USE BY GCM FOR ONLY ONE EXECUTION.
294    # ------------------------------------------------------------------
295    IGCM_comp_GetInputInitialStateFiles
296
297    #D-
298    # ------------------------------------------------------------------
299    #D- Get Boundaries Conditions (SST, WIND[X,Y,Z], LAI ...)
300    #D- READ AND USE BY GCM AT EACH EXECUTION.
301    # ------------------------------------------------------------------
302    IGCM_comp_GetInputBoundaryFiles
303
304    #D-
305    # ------------------------------------------------------------------
306    #D- Get restart files (restartphy.nc, orca_restart.nc ...)
307    #D- READ AND USE BY GCM AT EACH EXECUTION.
308    #D-
309    #D- IF A COMPONENT DO NOT RESTART FROM PREVIOULSY COMPUTED RESTART
310    #D- ONLY IF CumulPeriod= (=> NUMERO) == 1
311    #D- MUST EXECUTE CREATE ETAT0_LIMIT (TYPICALLY LMDZ AND ./create_etat0_limit.e)
312    #D-
313    #D- IF CumulPeriod == 1
314    #D- AND A COMPONENT RESTART FROM PREVIOULSY COMPUTED RESTART
315    #D- MUST BRING THIS RESTART IN ${RUN_DIR_PATH}
316    #D-
317    #D- ELSE BRING RESTART FROM ${JobName} IN ${RUN_DIR_PATH}
318    # ------------------------------------------------------------------
319    IGCM_comp_GetInputRestartFiles
320   
321    #D-
322    # ------------------------------------------------------------------
323    #D- Test if all was right before Update
324    # ------------------------------------------------------------------
325    IGCM_debug_Verif_Exit
326
327    # ------------------------------------------------------------------
328    #D- Activate running environnement variables
329    # ------------------------------------------------------------------
330    IGCM_sys_activ_variables
331
332    #D-
333    # ------------------------------------------------------------------
334    #D- UPDATE ParametersFiles (.def, namelist ...) with
335    #D- current value of (ORCA_NIT00, ORCA_LRSTAR, RAZ_DATE, ...)
336    # ------------------------------------------------------------------
337    IGCM_comp_Update
338
339    #D-
340    # ------------------------------------------------------------------
341    #D- Test if all was right before execution
342    # ------------------------------------------------------------------
343    IGCM_debug_Verif_Exit
344
345    #D-
346    #D----------------------------------------------------------------==
347    #D-                          3. Execution
348    #D----------------------------------------------------------------==
349
350    echo
351    echo "#######################################"
352    echo "#      DIR BEFORE RUN EXECUTION       #"
353    echo "#######################################"
354    echo
355    ls -lrt
356
357    echo "========================================================================"
358    if ( [ X${BATCH_NUM_PROC_TOT} != X ] && [ "${BATCH_NUM_PROC_TOT}" -gt 1 ] ) ; then
359        MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND}
360        eval MPIRUN_OPTIONS=${config_UserChoices_JobRunOptions}
361    else
362        MPIRUN_COMMAND=${MPIRUN_COMMAND:="time "}
363        MPIRUN_OPTIONS=${MPIRUN_OPTIONS:=""}
364    fi
365
366    if [ ${DRYRUN} -le 1 ] ; then
367        REAL_DATE_INIT=$( date )
368        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1"
369        typeset RET
370        RUN_DATE_BEGIN=$( date '+%Y-%m-%dT%H:%M:%S' )
371        ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1
372        RET=$?
373        RUN_DATE_END=$( date '+%Y-%m-%dT%H:%M:%S' )
374        if [ ${RET} -gt 0 ] ; then
375            echo "Return code of executable :" ${RET}
376            IGCM_debug_Exit "EXECUTABLE"
377            IGCM_sys_Cp ${Exe_Output} ${SUBMIT_DIR}/${PREFIX}_${Exe_Output}_error
378        fi
379        echo "libIGCM RunDateBegin=${RUN_DATE_BEGIN}"  >> ${Exe_Output}
380        echo "libIGCM RunDateEnd=${RUN_DATE_END}" >> ${Exe_Output}
381    else
382        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} simulated for DRYRUN = " $DRYRUN
383        if ( $DEBUG_debug ) ; then
384            echo "FOR EXECUTION DRYRUN mode = " $DRYRUN >> stack
385        fi
386    fi
387    echo "========================================================================"
388
389    echo
390    echo "#######################################"
391    echo "#       DIR AFTER RUN EXECUTION       #"
392    echo "#######################################"
393    echo
394    ls -lrt
395
396    #D-
397    # ------------------------------------------------------------------
398    #D- Desactivate running environnement variables
399    # ------------------------------------------------------------------
400    IGCM_sys_desactiv_variables
401
402    #D-
403    #D----------------------------------------------------------------==
404    #D-                    4. Post execution operations
405    #D----------------------------------------------------------------==
406
407    #D-
408    # ------------------------------------------------------------------
409    #D- Save on archive machine netcdf outputs, restarts
410    #D- and text files of models
411    # ------------------------------------------------------------------
412    IGCM_comp_Finalize
413
414    #D-
415    # ------------------------------------------------------------------
416    #D- Test if all was right after Finalize
417    # ------------------------------------------------------------------
418    IGCM_debug_Verif_Exit
419
420    #D-
421    # ------------------------------------------------------------------
422    #D- Launch remote rebuild or/and post-treatment process if necessary
423    # ------------------------------------------------------------------
424    IGCM_post_Configure
425
426    #D-
427    # ------------------------------------------------------------------
428    #D- Save job output
429    #D- Manage executable size to save only different binary
430    #D- Write in run.card user, system and elapse time
431    #D- Check that everything went well ! No ? then we stop.
432    #D- Determine next computed period
433    # ------------------------------------------------------------------
434    IGCM_config_PeriodEnd
435
436    echo "Ending iteration ${Period}"
437    (( Period = Period + 1 ))
438
439    # End loop if date end is reached
440    if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
441        break ;
442    fi
443
444    PeriodContinue=false
445    if [ ${Period} -le ${PeriodNb} ]; then
446        PeriodContinue=true
447    fi
448done
449
450#D-
451#D--------------------------------------------------------------------==
452#D-     5. Submit next job or send mail if simulation over.
453#D--------------------------------------------------------------------==
454IGCM_config_Finalize
455
456date
Note: See TracBrowser for help on using the repository browser.