source: trunk/libIGCM/AA_job @ 222

Last change on this file since 222 was 222, checked in by aclsce, 14 years ago
  • Added titane (CCRT Xeon machine) as target machine :

for the moment, post-treatment will be done on cesium.

  • Removed the use of dmget command on platine (CCRT Itanium machine)
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Date Author Revision
File size: 18.8 KB
Line 
1#-Q- platine #!/usr/bin/ksh
2#-Q- platine ##################
3#-Q- platine ## PLATINE   CEA ##
4#-Q- platine ##################
5#-Q- platine #BSUB -J ::Jobname::       # nom de la requete
6#-Q- platine #BSUB -N                   # message a la fin du job
7#-Q- platine #BSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
8#-Q- platine #BSUB -n ::JobNumProcTot::             # reservation des processeurs pour le job
9#-Q- platine #BSUB -W 1:00           # Limite temps
10#-Q- platine BATCH_NUM_PROC_TOT=$SLURM_NPROCS
11#-Q- titane #!/usr/bin/ksh
12#-Q- titane ##################
13#-Q- titane ## TITANE   CEA ##
14#-Q- titane ##################
15#-Q- titane #MSUB -r ::Jobname::       # nom de la requete
16#-Q- titane #MSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
17#-Q- titane #MSUB -n ::JobNumProcTot::             # reservation des processeurs pour le job
18#-Q- titane #MSUB -T 86400           # Limite temps (en secondes)
19#-Q- titane #MSUB -p gen2211
20#-Q- titane BATCH_NUM_PROC_TOT=$BRIDGE_MSUB_NPROC
21#-Q- sx8brodie #######################
22#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
23#-Q- sx8brodie #######################
24#-Q- sx8brodie #PBS -N ::Jobname::         # nom de la requete
25#-Q- sx8brodie #PBS -m a                   # message a la fin du job
26#-Q- sx8brodie #PBS -j o                   # regroupement des sorties du job : standard error et output
27#-Q- sx8brodie #PBS -o Script_Output_::Jobname::.000001  # nom du fichier de sortie
28#-Q- sx8brodie #PBS -S /usr/bin/ksh        # shell de soumission
29#-Q- sx8brodie #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
30#-Q- sx8brodie #PBS -q ::JobClass::                # classe multi pour BATCH_NUM_PROC_TOT superieur a 2, mono sinon
31#-Q- sx8brodie #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}  # reservation des processeurs pour le job
32#-Q- sx8brodie #PBS -l memsz_job=15.0gb     # Limite memoire
33#-Q- sx8brodie #PBS -l cputim_job=1:00:00  # Limite temps
34#-Q- aix6 #!/usr/bin/ksh
35#-Q- aix6 # ######################
36#-Q- aix6 # ##   VARGAS IDRIS   ##
37#-Q- aix6 # ######################
38#-Q- aix6 # Nom de la requete
39#-Q- aix6 # @ job_name = ::Jobname::
40#-Q- aix6 # Type de travail
41#-Q- aix6 # @ job_type = parallel
42#-Q- aix6 # Fichier de sortie standard
43#-Q- aix6 # @ output = $(job_name).$(jobid)
44#-Q- aix6 # Fichier de sortie erreur (le meme)
45#-Q- aix6 # @ error =  $(job_name).$(jobid)
46#-Q- aix6 # Nombre de processus demandes
47#-Q- aix6 # @ total_tasks = ::JobNumProcTot::
48#-Q- aix6 # @ environment = "BATCH_NUM_PROC_TOT=::JobNumProcTot::"
49#-Q- aix6 # Temps CPU max. par processus MPI hh:mm:ss
50#-Q- aix6 # @ wall_clock_limit = 1:00:00
51#-Q- aix6 # Memoire max. utilisee par processus
52#-Q- aix6 # @ data_limit = 3.2gb
53#-Q- aix6 # Memoire stack demandee
54#-Q- aix6 # @ stack_limit = 0.3gb,0.3gb
55#-Q- aix6 # Pas d OpenMP
56#-Q- aix6 # @ resources = ConsumableCpus(1)
57#-Q- aix6 # Fin de l entete
58#-Q- aix6 # @ queue
59#-Q- sx8mercure #!/bin/ksh
60#-Q- sx8mercure ######################
61#-Q- sx8mercure ## SX8MERCURE   CEA ##
62#-Q- sx8mercure ######################
63#-Q- sx8mercure #PBS -N ::Jobname::           # nom de la requete
64#-Q- sx8mercure #PBS -m a                     # message a la fin du job
65#-Q- sx8mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
66#-Q- sx8mercure ####PBS -A gen6178
67#-Q- sx8mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
68#-Q- sx8mercure #PBS -S /usr/bin/ksh          # shell de soumission
69#-Q- sx8mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
70#-Q- sx8mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
71#-Q- sx8mercure #PBS -l memsz_job=15.0gb       # limite memoire
72#-Q- sx8mercure #PBS -l elapstim_req=01:00:00 # limite en temps elapsed
73#-Q- sx9mercure #!/bin/ksh
74#-Q- sx9mercure ######################
75#-Q- sx9mercure ## SX9MERCURE   CEA ##
76#-Q- sx9mercure ######################
77#-Q- sx9mercure #PBS -N ::Jobname::           # nom de la requete
78#-Q- sx9mercure #PBS -m a                     # message a la fin du job
79#-Q- sx9mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
80#-Q- sx9mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
81#-Q- sx9mercure #PBS -S /usr/bin/ksh          # shell de soumission
82#-Q- sx9mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
83#-Q- sx9mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
84#-Q- sx9mercure #PBS -l memsz_job=16.0gb       # limite memoire
85#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # limite en temps elapsed
86#-Q- sx9mercure #PBS -A gen6178               # groupe obligatoire sur SX9
87#-Q- default #!/bin/ksh
88#-Q- default ##################
89#-Q- default ## DEFAULT HOST ##
90#-Q- default ##################
91#-Q- default #For MPI use, uncomment next line :
92#-Q- default #BATCH_NUM_PROC_TOT=::JobNumProcTot::
93
94# $Date$
95# $Author$
96# $Revision$
97# IPSL (2006)
98#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
99
100#set -eu
101#set -vx
102
103date
104echo
105echo "#######################################"
106echo "#       ANOTHER GREAT SIMULATION      #"
107echo "#######################################"
108echo
109
110#D--------------------------------------------------------------------==
111#D--------------------------------------------------------------------==
112#D-
113#D-                      Job to launch IGCM models
114#D-
115#D--------------------------------------------------------------------==
116#D--------------------------------------------------------------------==
117#D-
118
119MODIPSL=::modipsl::
120libIGCM=${MODIPSL}/libIGCM
121
122#D--------------------------------------------------------------------==
123#D-                   -1. User modification place
124#D-                      - Job Verbosity
125#D-                      - PeriodNb
126#D-                      - Experience type : DEB(ug), DEV(elopment), RUN
127#D--------------------------------------------------------------------==
128
129#D- Increased verbosity (1, 2, 3)
130Verbosity=3
131#D- Number of execution in one job
132PeriodNb=1
133#-Q- sx9mercure PeriodNb=60
134
135#D-
136# --------------------------------------------------------------------
137#D- ! Optionnal features /!\ Use with care !
138# --------------------------------------------------------------------
139
140#D- Low level debug : to bypass lib test checks and stack construction
141#D- Default=true
142DEBUG_debug=false
143
144#D- Set DEBUG_sys to false to disable output calls of function
145#D- Default=true
146#DEBUG_sys=false
147
148#D- Experience type : DEB(ug), DEV(elopment), RUN (default)
149JobType=RUN
150
151#D- Define running directory
152#D- Default=${TMPDIR} ie temporary batch directory
153#RUN_DIR_PATH=/workdir/or/scratchdir/of/this/machine
154
155#D- Define submit directory
156#D- Default= where you launch qsub (variable from scheduler)
157#SUBMIT_DIR=$( pwd )
158
159#D- Define BIG_DIR directory
160#D- Default=(strongly machine dependant/see libIGCM_sys for details)
161#BIG_DIR=/somewhere/with/lot/of/space/scratchdir/may/be
162
163#D- Turn in dry run mode ? (0,1,2,3)
164#D- Default=0
165#DRYRUN=3
166# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
167# -------------------------------------------------------------------------------------
168# | DRYRUN=  |  Date computations, | sys_Get    |  Exe    | sys_Put_Out; sys_Put_Rest |
169# |          |  Cp/Exe param files |            |  Chmod  |                           |
170# |          |      Qsub           |            |         |                           |
171# -------------------------------------------------------------------------------------
172# |    0     |       yes           |    yes     |  yes    |      yes                  |
173# -------------------------------------------------------------------------------------
174# |    1     |       yes           |    yes     |  yes    |      no                   |
175# -------------------------------------------------------------------------------------
176# |    2     |       yes           |    yes     |  no     |      no                   |
177# -------------------------------------------------------------------------------------
178# |    3     |       yes           |    no      |  no     |      no                   |
179# -------------------------------------------------------------------------------------
180
181#D- Define input file root directory
182#D- Default=/IGCMG/common/account/of/this/machine
183#R_IN=/u/rech/por/rpor111/DATA
184
185#D-
186#D--------------------------------------------------------------------==
187#D-                   0. System Environment
188#D-                      - Define variables for Number of processors
189#D-                      - Define MPI variables
190#D-                      - Define batch scheduler variables
191#D-                      - Source IGCM Library
192#D-                      - Get RESOLution in .resol file (temporary)
193#D--------------------------------------------------------------------==
194
195#D--------------------------------------------------------------------==
196
197. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
198       ( ${DEBUG_debug} ) && IGCM_debug_Check
199. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
200       ( ${DEBUG_debug} ) && IGCM_card_Check
201. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
202       ( ${DEBUG_debug} ) && IGCM_date_Check
203#-------
204. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
205. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
206. ${libIGCM}/libIGCM_comp/libIGCM_comp.ksh
207. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
208
209#D--------------------------------------------------------------------==
210
211#-- Resolution FLAG for oasis input
212[ -f ${SUBMIT_DIR}/../.resol ] && RESOL=$(head -1 ${SUBMIT_DIR}/../.resol)
213#RESOL="ORCA2xLMD9671"
214
215#D-
216#D--------------------------------------------------------------------==
217#D-           1. INITIALIZE CONFIGURATION AND THEN COMPONENTS
218#D--------------------------------------------------------------------==
219
220# --------------------------------------------------------------------==
221#D- Initialize configuration :
222#D-              - Simulation configuration
223#D-              - Simulation parameters
224#D-              - Execution parameter
225#D-              - Define input files directory
226#D-              - Read or initialize CumulPeriod
227#D-              - run.card
228#D-
229# --------------------------------------------------------------------==
230IGCM_config_Initialize
231
232# --------------------------------------------------------------------==
233#D- Define, create and cd RUN_DIR
234#D- 
235# --------------------------------------------------------------------==
236RUN_DIR=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}.${$}
237IGCM_sys_MkdirWork ${RUN_DIR}
238IGCM_sys_Cd ${RUN_DIR}
239
240# --------------------------------------------------------------------==
241#D- Initialize each components taking part
242#D- into simulation configuration :
243#D-         - Define component executable
244#D-         - Define Write Frequency by component "1M" "1D" ...
245#D-         - Source ${comp}.driver
246#D-         - Define Storage Places by component
247#D-         - ${comp}_Initialize :
248#D-         - Patterns to sed
249#D-         - Variables from namelist
250#D-
251# --------------------------------------------------------------------==
252IGCM_comp_Initialize
253
254# --------------------------------------------------------------------==
255#D- Verify compatibility of period length, write frequencies, ...
256#D- Verify compatibility of rebuild choices and post-process choices
257#D- Will stop here if something is wrong
258# --------------------------------------------------------------------==
259#IGCM_config_Check
260
261#D--------------------------------------------------------------------==
262#D-                  2. ENTER THE DEEP EXECUTION LOOP
263#D-                  !!!! FASTEN YOUR SEAT BELTS !!!!
264#D-                  !!!!!      PLEASE.          !!!!
265#D-                  !! WE WILL CROSS TURBULENCES. !!
266#D--------------------------------------------------------------------==
267
268Period=1
269
270PeriodContinue=false
271if [ ${Period} -le ${PeriodNb} ]; then
272    PeriodContinue=true
273fi
274while ( ${PeriodContinue} ) ; do
275
276    echo
277    echo "Starting iteration ${Period} / ${PeriodNb}"
278
279    #D-
280    # ------------------------------------------------------------------
281    #D- COMPUTE AND DEFINE DATE STUFF.
282    # ------------------------------------------------------------------
283    IGCM_config_PeriodStart
284    #D- => ${PeriodDateBegin}
285    #D- => ${PeriodDateEnd}
286    #D- => ${CumulPeriod}
287    #D- => ${DatesPeriod}=${PeriodDateBegin}_${PeriodDateEnd}
288    #D- => ${PeriodLength} => JOURS=31 , 28, 29, 30 ; MOIS=0 ; ANS=0
289    #D- => update run.card value
290   
291    #D-
292    # ------------------------------------------------------------------
293    #D- Get parameters text files updated by job (.def, namelist ...)
294    #D- READ AND USE BY GCM AT EACH EXECUTION.
295    # ------------------------------------------------------------------
296    IGCM_comp_GetInputParametersFiles
297
298    #D-
299    # ------------------------------------------------------------------
300    #D- Get initial state (Etat0, carteveg,relief...)
301    #D- NECESSARY ONLY IF CumulPeriod= (=> NUMERO) == 1
302    #D- AND ???_Restart=NO
303    #D- READ AND USE BY GCM FOR ONLY ONE EXECUTION.
304    # ------------------------------------------------------------------
305    IGCM_comp_GetInputInitialStateFiles
306
307    #D-
308    # ------------------------------------------------------------------
309    #D- Get Boundaries Conditions (SST, WIND[X,Y,Z], LAI ...)
310    #D- READ AND USE BY GCM AT EACH EXECUTION.
311    # ------------------------------------------------------------------
312    IGCM_comp_GetInputBoundaryFiles
313
314    #D-
315    # ------------------------------------------------------------------
316    #D- Get restart files (restartphy.nc, orca_restart.nc ...)
317    #D- READ AND USE BY GCM AT EACH EXECUTION.
318    #D-
319    #D- IF A COMPONENT DO NOT RESTART FROM PREVIOULSY COMPUTED RESTART
320    #D- ONLY IF CumulPeriod= (=> NUMERO) == 1
321    #D- MUST EXECUTE CREATE ETAT0_LIMIT (TYPICALLY LMDZ AND ./create_etat0_limit.e)
322    #D-
323    #D- IF CumulPeriod == 1
324    #D- AND A COMPONENT RESTART FROM PREVIOULSY COMPUTED RESTART
325    #D- MUST BRING THIS RESTART IN ${RUN_DIR_PATH}
326    #D-
327    #D- ELSE BRING RESTART FROM ${JobName} IN ${RUN_DIR_PATH}
328    # ------------------------------------------------------------------
329    IGCM_comp_GetInputRestartFiles
330   
331    #D-
332    # ------------------------------------------------------------------
333    #D- Test if all was right before Update
334    # ------------------------------------------------------------------
335    IGCM_debug_Verif_Exit
336
337    # ------------------------------------------------------------------
338    #D- Activate running environnement variables
339    # ------------------------------------------------------------------
340    IGCM_sys_activ_variables
341
342    #D-
343    # ------------------------------------------------------------------
344    #D- UPDATE ParametersFiles (.def, namelist ...) with
345    #D- current value of (ORCA_NIT00, ORCA_LRSTAR, RAZ_DATE, ...)
346    # ------------------------------------------------------------------
347    IGCM_comp_Update
348
349    #D-
350    # ------------------------------------------------------------------
351    #D- Test if all was right before execution
352    # ------------------------------------------------------------------
353    IGCM_debug_Verif_Exit
354
355    #D-
356    #D----------------------------------------------------------------==
357    #D-                          3. Execution
358    #D----------------------------------------------------------------==
359
360    echo
361    echo "#######################################"
362    echo "#      DIR BEFORE RUN EXECUTION       #"
363    echo "#######################################"
364    echo
365    ls -lrt
366
367    echo "========================================================================"
368    if ( [ X${BATCH_NUM_PROC_TOT} != X ] && [ "${BATCH_NUM_PROC_TOT}" -gt 1 ] ) ; then
369        MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND}
370        eval MPIRUN_OPTIONS=${config_UserChoices_JobRunOptions}
371    else
372        MPIRUN_COMMAND=${MPIRUN_COMMAND:="time "}
373        MPIRUN_OPTIONS=${MPIRUN_OPTIONS:=""}
374    fi
375
376    if [ ${DRYRUN} -le 1 ] ; then
377        REAL_DATE_INIT=$( date )
378        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1"
379        typeset RET
380        RUN_DATE_BEGIN=$( date '+%Y-%m-%dT%H:%M:%S' )
381        ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1
382        RET=$?
383        RUN_DATE_END=$( date '+%Y-%m-%dT%H:%M:%S' )
384        if [ ${RET} -gt 0 ] ; then
385            echo "Return code of executable :" ${RET}
386            IGCM_debug_Exit "EXECUTABLE"
387            IGCM_sys_Cp ${Exe_Output} ${SUBMIT_DIR}/${PREFIX}_${Exe_Output}_error
388        fi
389        echo "libIGCM RunDateBegin=${RUN_DATE_BEGIN}"  >> ${Exe_Output}
390        echo "libIGCM RunDateEnd=${RUN_DATE_END}" >> ${Exe_Output}
391    else
392        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} simulated for DRYRUN = " $DRYRUN
393        if ( $DEBUG_debug ) ; then
394            echo "FOR EXECUTION DRYRUN mode = " $DRYRUN >> stack
395        fi
396    fi
397    echo "========================================================================"
398
399    echo
400    echo "#######################################"
401    echo "#       DIR AFTER RUN EXECUTION       #"
402    echo "#######################################"
403    echo
404    ls -lrt
405
406    #D-
407    # ------------------------------------------------------------------
408    #D- Desactivate running environnement variables
409    # ------------------------------------------------------------------
410    IGCM_sys_desactiv_variables
411
412    #D-
413    #D----------------------------------------------------------------==
414    #D-                    4. Post execution operations
415    #D----------------------------------------------------------------==
416
417    #D-
418    # ------------------------------------------------------------------
419    #D- Save on archive machine netcdf outputs, restarts
420    #D- and text files of models
421    # ------------------------------------------------------------------
422    IGCM_comp_Finalize
423
424    #D-
425    # ------------------------------------------------------------------
426    #D- Test if all was right after Finalize
427    # ------------------------------------------------------------------
428    IGCM_debug_Verif_Exit
429
430    #D-
431    # ------------------------------------------------------------------
432    #D- Launch remote rebuild or/and post-treatment process if necessary
433    # ------------------------------------------------------------------
434    IGCM_post_Configure
435
436    #D-
437    # ------------------------------------------------------------------
438    #D- Save job output
439    #D- Manage executable size to save only different binary
440    #D- Write in run.card user, system and elapse time
441    #D- Check that everything went well ! No ? then we stop.
442    #D- Determine next computed period
443    # ------------------------------------------------------------------
444    IGCM_config_PeriodEnd
445
446    echo "Ending iteration ${Period}"
447    (( Period = Period + 1 ))
448
449    # End loop if date end is reached
450    if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
451        break ;
452    fi
453
454    PeriodContinue=false
455    if [ ${Period} -le ${PeriodNb} ]; then
456        PeriodContinue=true
457    fi
458done
459
460#D-
461#D--------------------------------------------------------------------==
462#D-     5. Submit next job or send mail if simulation over.
463#D--------------------------------------------------------------------==
464IGCM_config_Finalize
465
466date
Note: See TracBrowser for help on using the repository browser.