source: trunk/libIGCM/AA_job @ 182

Last change on this file since 182 was 182, checked in by sdipsl, 15 years ago
  • clean up
  • change default class on SX9. 1h is the test class and SX9 is mainly a production machine
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Date Author Revision
File size: 18.3 KB
Line 
1#-Q- platine #!/usr/bin/ksh
2#-Q- platine ##################
3#-Q- platine ## PLATINE   CEA ##
4#-Q- platine ##################
5#-Q- platine #BSUB -J ::Jobname::       # nom de la requete
6#-Q- platine #BSUB -N                   # message a la fin du job
7#-Q- platine #BSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
8#-Q- platine #BSUB -n ::JobNumProcTot::             # reservation des processeurs pour le job
9#-Q- platine #BSUB -W 1:00           # Limite temps
10#-Q- platine BATCH_NUM_PROC_TOT=$SLURM_NPROCS
11#-Q- sx8brodie #######################
12#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
13#-Q- sx8brodie #######################
14#-Q- sx8brodie #PBS -N ::Jobname::         # nom de la requete
15#-Q- sx8brodie #PBS -m a                   # message a la fin du job
16#-Q- sx8brodie #PBS -j o                   # regroupement des sorties du job : standard error et output
17#-Q- sx8brodie #PBS -o Script_Output_::Jobname::.000001  # nom du fichier de sortie
18#-Q- sx8brodie #PBS -S /usr/bin/ksh        # shell de soumission
19#-Q- sx8brodie #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
20#-Q- sx8brodie #PBS -q ::JobClass::                # classe multi pour BATCH_NUM_PROC_TOT superieur a 2, mono sinon
21#-Q- sx8brodie #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}  # reservation des processeurs pour le job
22#-Q- sx8brodie #PBS -l memsz_job=8.0gb     # Limite memoire
23#-Q- sx8brodie #PBS -l cputim_job=1:00:00  # Limite temps
24#-Q- aix6 #!/usr/bin/ksh
25#-Q- aix6 # ######################
26#-Q- aix6 # ##   VARGAS IDRIS   ##
27#-Q- aix6 # ######################
28#-Q- aix6 # Nom de la requete
29#-Q- aix6 # @ job_name = ::Jobname::
30#-Q- aix6 # Type de travail
31#-Q- aix6 # @ job_type = parallel
32#-Q- aix6 # Fichier de sortie standard
33#-Q- aix6 # @ output = $(job_name).$(jobid)
34#-Q- aix6 # Fichier de sortie erreur (le meme)
35#-Q- aix6 # @ error =  $(job_name).$(jobid)
36#-Q- aix6 # Nombre de processus demandes
37#-Q- aix6 # @ total_tasks = ::JobNumProcTot::
38#-Q- aix6 # @ environment = "BATCH_NUM_PROC_TOT=::JobNumProcTot::"
39#-Q- aix6 # Temps CPU max. par processus MPI hh:mm:ss
40#-Q- aix6 # @ wall_clock_limit = 1:00:00
41#-Q- aix6 # Memoire max. utilisee par processus
42#-Q- aix6 # @ data_limit = 3.2gb
43#-Q- aix6 # Memoire stack demandee
44#-Q- aix6 # @ stack_limit = 0.3gb,0.3gb
45#-Q- aix6 # Pas d OpenMP
46#-Q- aix6 # @ resources = ConsumableCpus(1)
47#-Q- aix6 # Fin de l entete
48#-Q- aix6 # @ queue
49#-Q- sx8mercure #!/bin/ksh
50#-Q- sx8mercure ######################
51#-Q- sx8mercure ## SX8MERCURE   CEA ##
52#-Q- sx8mercure ######################
53#-Q- sx8mercure #PBS -N ::Jobname::           # nom de la requete
54#-Q- sx8mercure #PBS -m a                     # message a la fin du job
55#-Q- sx8mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
56#-Q- sx8mercure ####PBS -A gen6178
57#-Q- sx8mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
58#-Q- sx8mercure #PBS -S /usr/bin/ksh          # shell de soumission
59#-Q- sx8mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
60#-Q- sx8mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
61#-Q- sx8mercure #PBS -l memsz_job=8.0gb       # limite memoire
62#-Q- sx8mercure #PBS -l elapstim_req=01:00:00 # limite en temps elapsed
63#-Q- sx9mercure #!/bin/ksh
64#-Q- sx9mercure ######################
65#-Q- sx9mercure ## SX9MERCURE   CEA ##
66#-Q- sx9mercure ######################
67#-Q- sx9mercure #PBS -N ::Jobname::           # nom de la requete
68#-Q- sx9mercure #PBS -m a                     # message a la fin du job
69#-Q- sx9mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
70#-Q- sx9mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
71#-Q- sx9mercure #PBS -S /usr/bin/ksh          # shell de soumission
72#-Q- sx9mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
73#-Q- sx9mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
74#-Q- sx9mercure #PBS -l memsz_job=16.0gb       # limite memoire
75#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # limite en temps elapsed
76#-Q- sx9mercure #PBS -A gen6178               # groupe obligatoire sur SX9
77#-Q- default #!/bin/ksh
78#-Q- default ##################
79#-Q- default ## DEFAULT HOST ##
80#-Q- default ##################
81#-Q- default #For MPI use, uncomment next line :
82#-Q- default #BATCH_NUM_PROC_TOT=::JobNumProcTot::
83
84# $Date$
85# $Author$
86# $Revision$
87# IPSL (2006)
88#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
89
90#set -eu
91#set -vx
92
93date
94echo
95echo "#######################################"
96echo "#       ANOTHER GREAT SIMULATION      #"
97echo "#######################################"
98echo
99
100#D--------------------------------------------------------------------==
101#D--------------------------------------------------------------------==
102#D-
103#D-                      Job to launch IGCM models
104#D-
105#D--------------------------------------------------------------------==
106#D--------------------------------------------------------------------==
107#D-
108
109MODIPSL=::modipsl::
110libIGCM=${MODIPSL}/libIGCM
111
112#D--------------------------------------------------------------------==
113#D-                   -1. User modification place
114#D-                      - Job Verbosity
115#D-                      - PeriodNb
116#D-                      - Experience type : DEB(ug), DEV(elopment), RUN
117#D--------------------------------------------------------------------==
118
119#D- Increased verbosity (1, 2, 3)
120Verbosity=3
121#D- Number of execution in one job
122PeriodNb=60
123
124#D-
125# --------------------------------------------------------------------
126#D- ! Optionnal features /!\ Use with care !
127# --------------------------------------------------------------------
128
129#D- Low level debug : to bypass lib test checks and stack construction
130#D- Default=true
131DEBUG_debug=false
132
133#D- Set DEBUG_sys to false to disable output calls of function
134#D- Default=true
135#DEBUG_sys=false
136
137#D- Experience type : DEB(ug), DEV(elopment), RUN (default)
138JobType=RUN
139
140#D- Define running directory
141#D- Default=${TMPDIR} ie temporary batch directory
142#RUN_DIR_PATH=/workdir/or/scratchdir/of/this/machine
143
144#D- Define submit directory
145#D- Default= where you launch qsub (variable from scheduler)
146#SUBMIT_DIR=$( pwd )
147
148#D- Define BIG_DIR directory
149#D- Default=(strongly machine dependant/see libIGCM_sys for details)
150#BIG_DIR=/somewhere/with/lot/of/space/scratchdir/may/be
151
152#D- Turn in dry run mode ? (0,1,2,3)
153#D- Default=0
154#DRYRUN=3
155# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
156# -------------------------------------------------------------------------------------
157# | DRYRUN=  |  Date computations, | sys_Get    |  Exe    | sys_Put_Out; sys_Put_Rest |
158# |          |  Cp/Exe param files |            |  Chmod  |                           |
159# |          |      Qsub           |            |         |                           |
160# -------------------------------------------------------------------------------------
161# |    0     |       yes           |    yes     |  yes    |      yes                  |
162# -------------------------------------------------------------------------------------
163# |    1     |       yes           |    yes     |  yes    |      no                   |
164# -------------------------------------------------------------------------------------
165# |    2     |       yes           |    yes     |  no     |      no                   |
166# -------------------------------------------------------------------------------------
167# |    3     |       yes           |    no      |  no     |      no                   |
168# -------------------------------------------------------------------------------------
169
170#D- Define input file root directory
171#D- Default=/IGCMG/common/account/of/this/machine
172#R_IN=/u/rech/por/rpor111/DATA
173
174#D-
175#D--------------------------------------------------------------------==
176#D-                   0. System Environment
177#D-                      - Define variables for Number of processors
178#D-                      - Define MPI variables
179#D-                      - Define batch scheduler variables
180#D-                      - Source IGCM Library
181#D-                      - Get RESOLution in .resol file (temporary)
182#D--------------------------------------------------------------------==
183
184#D--------------------------------------------------------------------==
185
186. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
187       ( ${DEBUG_debug} ) && IGCM_debug_Check
188. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
189       ( ${DEBUG_debug} ) && IGCM_card_Check
190. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
191       ( ${DEBUG_debug} ) && IGCM_date_Check
192#-------
193. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
194. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
195. ${libIGCM}/libIGCM_comp/libIGCM_comp.ksh
196. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
197
198#D--------------------------------------------------------------------==
199
200#-- Resolution FLAG for oasis input
201[ -f ${SUBMIT_DIR}/../.resol ] && RESOL=$(head -1 ${SUBMIT_DIR}/../.resol)
202#RESOL="ORCA2xLMD9671"
203
204#D-
205#D--------------------------------------------------------------------==
206#D-           1. INITIALIZE CONFIGURATION AND THEN COMPONENTS
207#D--------------------------------------------------------------------==
208
209# --------------------------------------------------------------------==
210#D- Initialize configuration :
211#D-              - Simulation configuration
212#D-              - Simulation parameters
213#D-              - Execution parameter
214#D-              - Define input files directory
215#D-              - Read or initialize CumulPeriod
216#D-              - run.card
217#D-
218# --------------------------------------------------------------------==
219IGCM_config_Initialize
220
221# --------------------------------------------------------------------==
222#D- Define, create and cd RUN_DIR
223#D- 
224# --------------------------------------------------------------------==
225RUN_DIR=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}.${$}
226IGCM_sys_MkdirWork ${RUN_DIR}
227IGCM_sys_Cd ${RUN_DIR}
228
229# --------------------------------------------------------------------==
230#D- Initialize each components taking part
231#D- into simulation configuration :
232#D-         - Define component executable
233#D-         - Define Write Frequency by component "1M" "1D" ...
234#D-         - Source ${comp}.driver
235#D-         - Define Storage Places by component
236#D-         - ${comp}_Initialize :
237#D-         - Patterns to sed
238#D-         - Variables from namelist
239#D-
240# --------------------------------------------------------------------==
241IGCM_comp_Initialize
242
243# --------------------------------------------------------------------==
244#D- Verify compatibility of period length, write frequencies, ...
245#D- Verify compatibility of rebuild choices and post-process choices
246#D- Will stop here if something is wrong
247# --------------------------------------------------------------------==
248#IGCM_config_Check
249
250#D--------------------------------------------------------------------==
251#D-                  2. ENTER THE DEEP EXECUTION LOOP
252#D-                  !!!! FASTEN YOUR SEAT BELTS !!!!
253#D-                  !!!!!      PLEASE.          !!!!
254#D-                  !! WE WILL CROSS TURBULENCES. !!
255#D--------------------------------------------------------------------==
256
257Period=1
258
259PeriodContinue=false
260if [ ${Period} -le ${PeriodNb} ]; then
261    PeriodContinue=true
262fi
263while ( ${PeriodContinue} ) ; do
264
265    echo
266    echo "Starting iteration ${Period} / ${PeriodNb}"
267
268    #D-
269    # ------------------------------------------------------------------
270    #D- COMPUTE AND DEFINE DATE STUFF.
271    # ------------------------------------------------------------------
272    IGCM_config_PeriodStart
273    #D- => ${PeriodDateBegin}
274    #D- => ${PeriodDateEnd}
275    #D- => ${CumulPeriod}
276    #D- => ${DatesPeriod}=${PeriodDateBegin}_${PeriodDateEnd}
277    #D- => ${PeriodLength} => JOURS=31 , 28, 29, 30 ; MOIS=0 ; ANS=0
278    #D- => update run.card value
279   
280    #D-
281    # ------------------------------------------------------------------
282    #D- Get parameters text files updated by job (.def, namelist ...)
283    #D- READ AND USE BY GCM AT EACH EXECUTION.
284    # ------------------------------------------------------------------
285    IGCM_comp_GetInputParametersFiles
286
287    #D-
288    # ------------------------------------------------------------------
289    #D- Get initial state (Etat0, carteveg,relief...)
290    #D- NECESSARY ONLY IF CumulPeriod= (=> NUMERO) == 1
291    #D- AND ???_Restart=NO
292    #D- READ AND USE BY GCM FOR ONLY ONE EXECUTION.
293    # ------------------------------------------------------------------
294    IGCM_comp_GetInputInitialStateFiles
295
296    #D-
297    # ------------------------------------------------------------------
298    #D- Get Boundaries Conditions (SST, WIND[X,Y,Z], LAI ...)
299    #D- READ AND USE BY GCM AT EACH EXECUTION.
300    # ------------------------------------------------------------------
301    IGCM_comp_GetInputBoundaryFiles
302
303    #D-
304    # ------------------------------------------------------------------
305    #D- Get restart files (restartphy.nc, orca_restart.nc ...)
306    #D- READ AND USE BY GCM AT EACH EXECUTION.
307    #D-
308    #D- IF A COMPONENT DO NOT RESTART FROM PREVIOULSY COMPUTED RESTART
309    #D- ONLY IF CumulPeriod= (=> NUMERO) == 1
310    #D- MUST EXECUTE CREATE ETAT0_LIMIT (TYPICALLY LMDZ AND ./create_etat0_limit.e)
311    #D-
312    #D- IF CumulPeriod == 1
313    #D- AND A COMPONENT RESTART FROM PREVIOULSY COMPUTED RESTART
314    #D- MUST BRING THIS RESTART IN ${RUN_DIR_PATH}
315    #D-
316    #D- ELSE BRING RESTART FROM ${JobName} IN ${RUN_DIR_PATH}
317    # ------------------------------------------------------------------
318    IGCM_comp_GetInputRestartFiles
319   
320    #D-
321    # ------------------------------------------------------------------
322    #D- Test if all was right before Update
323    # ------------------------------------------------------------------
324    IGCM_debug_Verif_Exit
325
326    # ------------------------------------------------------------------
327    #D- Activate running environnement variables
328    # ------------------------------------------------------------------
329    IGCM_sys_activ_variables
330
331    #D-
332    # ------------------------------------------------------------------
333    #D- UPDATE ParametersFiles (.def, namelist ...) with
334    #D- current value of (ORCA_NIT00, ORCA_LRSTAR, RAZ_DATE, ...)
335    # ------------------------------------------------------------------
336    IGCM_comp_Update
337
338    #D-
339    # ------------------------------------------------------------------
340    #D- Test if all was right before execution
341    # ------------------------------------------------------------------
342    IGCM_debug_Verif_Exit
343
344    #D-
345    #D----------------------------------------------------------------==
346    #D-                          3. Execution
347    #D----------------------------------------------------------------==
348
349    echo
350    echo "#######################################"
351    echo "#      DIR BEFORE RUN EXECUTION       #"
352    echo "#######################################"
353    echo
354    ls -lrt
355
356    echo "========================================================================"
357    if ( [ X${BATCH_NUM_PROC_TOT} != X ] && [ "${BATCH_NUM_PROC_TOT}" -gt 1 ] ) ; then
358        MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND}
359        eval MPIRUN_OPTIONS=${config_UserChoices_JobRunOptions}
360    else
361        MPIRUN_COMMAND=${MPIRUN_COMMAND:="time "}
362        MPIRUN_OPTIONS=${MPIRUN_OPTIONS:=""}
363    fi
364
365    if [ ${DRYRUN} -le 1 ] ; then
366        REAL_DATE_INIT=$( date )
367        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1"
368        typeset RET
369        RUN_DATE_BEGIN=$( date '+%Y-%m-%dT%H:%M:%S' )
370        ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1
371        RET=$?
372        RUN_DATE_END=$( date '+%Y-%m-%dT%H:%M:%S' )
373        if [ ${RET} -gt 0 ] ; then
374            echo "Return code of executable :" ${RET}
375            IGCM_debug_Exit "EXECUTABLE"
376            IGCM_sys_Cp ${Exe_Output} ${SUBMIT_DIR}/${PREFIX}_${Exe_Output}_error
377        fi
378        echo "libIGCM RunDateBegin=${RUN_DATE_BEGIN}"  >> ${Exe_Output}
379        echo "libIGCM RunDateEnd=${RUN_DATE_END}" >> ${Exe_Output}
380    else
381        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} simulated for DRYRUN = " $DRYRUN
382        if ( $DEBUG_debug ) ; then
383            echo "FOR EXECUTION DRYRUN mode = " $DRYRUN >> stack
384        fi
385    fi
386    echo "========================================================================"
387
388    echo
389    echo "#######################################"
390    echo "#       DIR AFTER RUN EXECUTION       #"
391    echo "#######################################"
392    echo
393    ls -lrt
394
395    #D-
396    # ------------------------------------------------------------------
397    #D- Desactivate running environnement variables
398    # ------------------------------------------------------------------
399    IGCM_sys_desactiv_variables
400
401    #D-
402    #D----------------------------------------------------------------==
403    #D-                    4. Post execution operations
404    #D----------------------------------------------------------------==
405
406    #D-
407    # ------------------------------------------------------------------
408    #D- Save on archive machine netcdf outputs, restarts
409    #D- and text files of models
410    # ------------------------------------------------------------------
411    IGCM_comp_Finalize
412
413    #D-
414    # ------------------------------------------------------------------
415    #D- Test if all was right after Finalize
416    # ------------------------------------------------------------------
417    IGCM_debug_Verif_Exit
418
419    #D-
420    # ------------------------------------------------------------------
421    #D- Launch remote rebuild or/and post-treatment process if necessary
422    # ------------------------------------------------------------------
423    IGCM_post_Configure
424
425    #D-
426    # ------------------------------------------------------------------
427    #D- Save job output
428    #D- Manage executable size to save only different binary
429    #D- Write in run.card user, system and elapse time
430    #D- Check that everything went well ! No ? then we stop.
431    #D- Determine next computed period
432    # ------------------------------------------------------------------
433    IGCM_config_PeriodEnd
434
435    echo "Ending iteration ${Period}"
436    (( Period = Period + 1 ))
437
438    # End loop if date end is reached
439    if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
440        break ;
441    fi
442
443    PeriodContinue=false
444    if [ ${Period} -le ${PeriodNb} ]; then
445        PeriodContinue=true
446    fi
447done
448
449#D-
450#D--------------------------------------------------------------------==
451#D-     5. Submit next job or send mail if simulation over.
452#D--------------------------------------------------------------------==
453IGCM_config_Finalize
454
455date
Note: See TracBrowser for help on using the repository browser.