source: tags/libIGCM_v1_3/AA_job @ 1456

Last change on this file since 1456 was 138, checked in by sdipsl, 15 years ago

SD+MAF : prepare .resol to contain more information
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Date Author Revision
File size: 17.6 KB
Line 
1#-Q- platine #!/usr/bin/ksh
2#-Q- platine ##################
3#-Q- platine ## PLATINE   CEA ##
4#-Q- platine ##################
5#-Q- platine #BSUB -J ::Jobname::       # nom de la requete
6#-Q- platine #BSUB -N                   # message a la fin du job
7#-Q- platine #BSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
8#-Q- platine #BSUB -n ::JobNumProcTot::             # reservation des processeurs pour le job
9#-Q- platine #BSUB -W 1:00           # Limite temps
10#-Q- platine BATCH_NUM_PROC_TOT=$SLURM_NPROCS
11#-Q- sx8brodie #######################
12#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
13#-Q- sx8brodie #######################
14#-Q- sx8brodie #PBS -N ::Jobname::         # nom de la requete
15#-Q- sx8brodie #PBS -m a                   # message a la fin du job
16#-Q- sx8brodie #PBS -j o                   # regroupement des sorties du job : standard error et output
17#-Q- sx8brodie #PBS -o Script_Output_::Jobname::.000001  # nom du fichier de sortie
18#-Q- sx8brodie #PBS -S /usr/bin/ksh        # shell de soumission
19#-Q- sx8brodie #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
20#-Q- sx8brodie #PBS -q ::JobClass::                # classe multi pour BATCH_NUM_PROC_TOT superieur a 2, mono sinon
21#-Q- sx8brodie #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}  # reservation des processeurs pour le job
22#-Q- sx8brodie #PBS -l memsz_job=8.0gb     # Limite memoire
23#-Q- sx8brodie #PBS -l cputim_job=1:00:00  # Limite temps
24#-Q- aix6 #!/usr/bin/ksh
25#-Q- aix6 # ######################
26#-Q- aix6 # ##   VARGAS IDRIS   ##
27#-Q- aix6 # ######################
28#-Q- aix6 # Nom de la requete
29#-Q- aix6 # @ job_name = ::Jobname::
30#-Q- aix6 # Type de travail
31#-Q- aix6 # @ job_type = parallel
32#-Q- aix6 # Fichier de sortie standard
33#-Q- aix6 # @ output = $(job_name).$(jobid)
34#-Q- aix6 # Fichier de sortie erreur (le meme)
35#-Q- aix6 # @ error =  $(job_name).$(jobid)
36#-Q- aix6 # Nombre de processus demandes
37#-Q- aix6 # @ total_tasks = ::JobNumProcTot::
38#-Q- aix6 # @ environment = "BATCH_NUM_PROC_TOT=::JobNumProcTot::"
39#-Q- aix6 # Temps CPU max. par processus MPI hh:mm:ss
40#-Q- aix6 # @ wall_clock_limit = 1:00:00
41#-Q- aix6 # Memoire max. utilisee par processus
42#-Q- aix6 # @ data_limit = 3.2gb
43#-Q- aix6 # Memoire stack demandee
44#-Q- aix6 # @ stack_limit = 0.3gb,0.3gb
45#-Q- aix6 # Pas d OpenMP
46#-Q- aix6 # @ resources = ConsumableCpus(1)
47#-Q- aix6 # Fin de l entete
48#-Q- aix6 # @ queue
49#-Q- sx8mercure #!/bin/ksh
50#-Q- sx8mercure ######################
51#-Q- sx8mercure ## SX8MERCURE   CEA ##
52#-Q- sx8mercure ######################
53#-Q- sx8mercure #PBS -N ::Jobname::           # nom de la requete
54#-Q- sx8mercure #PBS -m a                     # message a la fin du job
55#-Q- sx8mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
56#-Q- sx8mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
57#-Q- sx8mercure #PBS -S /usr/bin/ksh          # shell de soumission
58#-Q- sx8mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
59#-Q- sx8mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
60#-Q- sx8mercure #PBS -l memsz_job=8.0gb       # limite memoire
61#-Q- sx8mercure #PBS -l elapstim_req=01:00:00 # limite en temps elapsed
62#-Q- default #!/bin/ksh
63#-Q- default ##################
64#-Q- default ## DEFAULT HOST ##
65#-Q- default ##################
66#-Q- default #For MPI use, uncomment next line :
67#-Q- default #BATCH_NUM_PROC_TOT=::JobNumProcTot::
68
69# $Date$
70# $Author$
71# $Revision$
72# IPSL (2006)
73#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
74
75#set -eu
76#set -vx
77
78date
79echo
80echo "#######################################"
81echo "#       ANOTHER GREAT SIMULATION      #"
82echo "#######################################"
83echo
84
85#D--------------------------------------------------------------------==
86#D--------------------------------------------------------------------==
87#D-
88#D-                      Job to launch IGCM models
89#D-
90#D--------------------------------------------------------------------==
91#D--------------------------------------------------------------------==
92#D-
93
94MODIPSL=::modipsl::
95libIGCM=${MODIPSL}/libIGCM
96
97#D--------------------------------------------------------------------==
98#D-                   -1. User modification place
99#D-                      - Job Verbosity
100#D-                      - PeriodNb
101#D-                      - Experience type : DEB(ug), DEV(elopment), RUN
102#D--------------------------------------------------------------------==
103
104#D- Increased verbosity (1, 2, 3)
105Verbosity=3
106#D- Number of execution in one job
107PeriodNb=1
108
109#D-
110# --------------------------------------------------------------------
111#D- ! Optionnal features /!\ Use with care !
112# --------------------------------------------------------------------
113
114#D- Low level debug : to bypass lib test checks and stack construction
115#D- Default=true
116DEBUG_debug=false
117
118#D- Set DEBUG_sys to false to disable output calls of function
119#D- Default=true
120#DEBUG_sys=false
121
122#D- Experience type : DEB(ug), DEV(elopment), RUN (default)
123JobType=RUN
124
125#D- Define running directory
126#D- Default=${TMPDIR} ie temporary batch directory
127#RUN_DIR_PATH=/workdir/of/this/machine
128
129#D- Define submit directory
130#D- Default= where you launch qsub (variable from scheduler)
131#SUBMIT_DIR=$( pwd )
132
133#D- Turn in dry run mode ? (0,1,2,3)
134#D- Default=0
135#DRYRUN=3
136# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
137# -------------------------------------------------------------------------------------
138# | DRYRUN=  |  Date computations, | sys_Get    |  Exe    | sys_Put_Out; sys_Put_Rest |
139# |          |  Cp/Exe param files |            |  Chmod  |                           |
140# |          |      Qsub           |            |         |                           |
141# -------------------------------------------------------------------------------------
142# |    0     |       yes           |    yes     |  yes    |      yes                  |
143# -------------------------------------------------------------------------------------
144# |    1     |       yes           |    yes     |  yes    |      no                   |
145# -------------------------------------------------------------------------------------
146# |    2     |       yes           |    yes     |  no     |      no                   |
147# -------------------------------------------------------------------------------------
148# |    3     |       yes           |    no      |  no     |      no                   |
149# -------------------------------------------------------------------------------------
150
151#D- Define input file root directory
152#D- Default=/IGCMG/common/account/of/this/machine
153#R_IN=/u/rech/por/rpor111/DATA
154
155#D-
156#D--------------------------------------------------------------------==
157#D-                   0. System Environment
158#D-                      - Define variables for Number of processors
159#D-                      - Define MPI variables
160#D-                      - Define batch scheduler variables
161#D-                      - Source IGCM Library
162#D-                      - Get RESOLution in .resol file (temporary)
163#D--------------------------------------------------------------------==
164
165#D--------------------------------------------------------------------==
166
167. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
168       ( ${DEBUG_debug} ) && IGCM_debug_Check
169. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
170       ( ${DEBUG_debug} ) && IGCM_card_Check
171. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
172       ( ${DEBUG_debug} ) && IGCM_date_Check
173#-------
174. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
175. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
176. ${libIGCM}/libIGCM_comp/libIGCM_comp.ksh
177. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
178
179#D--------------------------------------------------------------------==
180
181#-- Resolution FLAG for oasis input
182[ -f ${SUBMIT_DIR}/../.resol ] && RESOL=$(head -1 ${SUBMIT_DIR}/../.resol)
183#RESOL="ORCA2xLMD9671"
184
185#D-
186#D--------------------------------------------------------------------==
187#D-           1. INITIALIZE CONFIGURATION AND THEN COMPONENTS
188#D--------------------------------------------------------------------==
189
190# --------------------------------------------------------------------==
191#D- Initialize configuration :
192#D-              - Simulation configuration
193#D-              - Simulation parameters
194#D-              - Execution parameter
195#D-              - Define input files directory
196#D-              - Read or initialize CumulPeriod
197#D-              - run.card
198#D-
199# --------------------------------------------------------------------==
200IGCM_config_Initialize
201
202# --------------------------------------------------------------------==
203#D- Define, create and cd RUN_DIR
204#D- 
205# --------------------------------------------------------------------==
206RUN_DIR=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
207IGCM_sys_MkdirWork ${RUN_DIR}
208IGCM_sys_Cd ${RUN_DIR}
209
210# --------------------------------------------------------------------==
211#D- Define REBUILD_DIR :
212#D- where we store files needing rebuild process
213#D- 
214# --------------------------------------------------------------------==
215if [ X${config_Post_RebuildFromArchive} = Xtrue ] ; then
216    REBUILD_DIR=${R_SAVE}/TMP
217    IGCM_sys_MkdirArchive ${REBUILD_DIR}
218else
219    REBUILD_DIR=${RUN_DIR}
220fi
221
222# --------------------------------------------------------------------==
223#D- Initialize each components taking part
224#D- into simulation configuration :
225#D-         - Define component executable
226#D-         - Define Write Frequency by component "1M" "1D" ...
227#D-         - Source ${comp}.driver
228#D-         - Define Storage Places by component
229#D-         - ${comp}_Initialize :
230#D-         - Patterns to sed
231#D-         - Variables from namelist
232#D-
233# --------------------------------------------------------------------==
234IGCM_comp_Initialize
235
236# --------------------------------------------------------------------==
237#D- Verify compatibility of period length, write frequencies, ...
238#D- Verify compatibility of rebuild choices and post-process choices
239#D- Will stop here if something is wrong
240# --------------------------------------------------------------------==
241#IGCM_config_Check
242
243#D--------------------------------------------------------------------==
244#D-                  2. ENTER THE DEEP EXECUTION LOOP
245#D-                  !!!! FASTEN YOUR SEAT BELTS !!!!
246#D-                  !!!!!      PLEASE.          !!!!
247#D-                  !! WE WILL CROSS TURBULENCES. !!
248#D--------------------------------------------------------------------==
249
250Period=1
251
252PeriodContinue=false
253if [ ${Period} -le ${PeriodNb} ]; then
254    PeriodContinue=true
255fi
256while ( ${PeriodContinue} ) ; do
257
258    echo
259    echo "Starting iteration ${Period} / ${PeriodNb}"
260
261    #D-
262    # ------------------------------------------------------------------
263    #D- COMPUTE AND DEFINE DATE STUFF.
264    # ------------------------------------------------------------------
265    IGCM_config_PeriodStart
266    #D- => ${PeriodDateBegin}
267    #D- => ${PeriodDateEnd}
268    #D- => ${CumulPeriod}
269    #D- => ${DatesPeriod}=${PeriodDateBegin}_${PeriodDateEnd}
270    #D- => ${PeriodLength} => JOURS=31 , 28, 29, 30 ; MOIS=0 ; ANS=0
271    #D- => update run.card value
272   
273    #D-
274    # ------------------------------------------------------------------
275    #D- Get parameters text files updated by job (.def, namelist ...)
276    #D- READ AND USE BY GCM AT EACH EXECUTION.
277    # ------------------------------------------------------------------
278    IGCM_comp_GetInputParametersFiles
279
280    #D-
281    # ------------------------------------------------------------------
282    #D- Get initial state (Etat0, carteveg,relief...)
283    #D- NECESSARY ONLY IF CumulPeriod= (=> NUMERO) == 1
284    #D- AND ???_Restart=NO
285    #D- READ AND USE BY GCM FOR ONLY ONE EXECUTION.
286    # ------------------------------------------------------------------
287    IGCM_comp_GetInputInitialStateFiles
288
289    #D-
290    # ------------------------------------------------------------------
291    #D- Get Boundaries Conditions (SST, WIND[X,Y,Z], LAI ...)
292    #D- READ AND USE BY GCM AT EACH EXECUTION.
293    # ------------------------------------------------------------------
294    IGCM_comp_GetInputBoundaryFiles
295
296    #D-
297    # ------------------------------------------------------------------
298    #D- Get restart files (restartphy.nc, orca_restart.nc ...)
299    #D- READ AND USE BY GCM AT EACH EXECUTION.
300    #D-
301    #D- IF A COMPONENT DO NOT RESTART FROM PREVIOULSY COMPUTED RESTART
302    #D- ONLY IF CumulPeriod= (=> NUMERO) == 1
303    #D- MUST EXECUTE CREATE ETAT0_LIMIT (TYPICALLY LMDZ AND ./create_etat0_limit.e)
304    #D-
305    #D- IF CumulPeriod == 1
306    #D- AND A COMPONENT RESTART FROM PREVIOULSY COMPUTED RESTART
307    #D- MUST BRING THIS RESTART IN ${RUN_DIR_PATH}
308    #D-
309    #D- ELSE BRING RESTART FROM ${JobName} IN ${RUN_DIR_PATH}
310    # ------------------------------------------------------------------
311    IGCM_comp_GetInputRestartFiles
312   
313    #D-
314    # ------------------------------------------------------------------
315    #D- Test if all was right before Update
316    # ------------------------------------------------------------------
317    IGCM_debug_Verif_Exit
318
319    # ------------------------------------------------------------------
320    #D- Activate running environnement variables
321    # ------------------------------------------------------------------
322    IGCM_sys_activ_variables
323
324    #D-
325    # ------------------------------------------------------------------
326    #D- UPDATE ParametersFiles (.def, namelist ...) with
327    #D- current value of (ORCA_NIT00, ORCA_LRSTAR, RAZ_DATE, ...)
328    # ------------------------------------------------------------------
329    IGCM_comp_Update
330
331    #D-
332    # ------------------------------------------------------------------
333    #D- Test if all was right before execution
334    # ------------------------------------------------------------------
335    IGCM_debug_Verif_Exit
336
337    #D-
338    #D----------------------------------------------------------------==
339    #D-                          3. Execution
340    #D----------------------------------------------------------------==
341
342    echo
343    echo "#######################################"
344    echo "#      DIR BEFORE RUN EXECUTION       #"
345    echo "#######################################"
346    echo
347    ls -lrt
348
349    echo "========================================================================"
350    if ( [ X${BATCH_NUM_PROC_TOT} != X ] && [ "${BATCH_NUM_PROC_TOT}" -gt 1 ] ) ; then
351        MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND}
352        eval MPIRUN_OPTIONS=${config_UserChoices_JobRunOptions}
353    else
354        MPIRUN_COMMAND=${MPIRUN_COMMAND:="time "}
355        MPIRUN_OPTIONS=${MPIRUN_OPTIONS:=""}
356    fi
357
358    if [ ${DRYRUN} -le 1 ] ; then
359        REAL_DATE_INIT=$( date )
360        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1"
361        typeset RET
362        RUN_DATE_BEGIN=$( date '+%Y-%m-%dT%H:%M:%S' )
363        ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1
364        RET=$?
365        RUN_DATE_END=$( date '+%Y-%m-%dT%H:%M:%S' )
366        if [ ${RET} -gt 0 ] ; then
367            echo "Return code of executable :" ${RET}
368            IGCM_debug_Exit "EXECUTABLE"
369            IGCM_sys_Cp ${Exe_Output} ${SUBMIT_DIR}/${PREFIX}_${Exe_Output}_error
370        fi
371        echo "libIGCM RunDateBegin=${RUN_DATE_BEGIN}"  >> ${Exe_Output}
372        echo "libIGCM RunDateEnd=${RUN_DATE_END}" >> ${Exe_Output}
373    else
374        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} simulated for DRYRUN = " $DRYRUN
375        if ( $DEBUG_debug ) ; then
376            echo "FOR EXECUTION DRYRUN mode = " $DRYRUN >> stack
377        fi
378    fi
379    echo "========================================================================"
380
381    echo
382    echo "#######################################"
383    echo "#       DIR AFTER RUN EXECUTION       #"
384    echo "#######################################"
385    echo
386    ls -lrt
387
388    #D-
389    # ------------------------------------------------------------------
390    #D- Desactivate running environnement variables
391    # ------------------------------------------------------------------
392    IGCM_sys_desactiv_variables
393
394    #D-
395    #D----------------------------------------------------------------==
396    #D-                    4. Post execution operations
397    #D----------------------------------------------------------------==
398
399    #D-
400    # ------------------------------------------------------------------
401    #D- Save on archive machine netcdf outputs, restarts
402    #D- and text files of models
403    # ------------------------------------------------------------------
404    IGCM_comp_Finalize
405
406    #D-
407    # ------------------------------------------------------------------
408    #D- Test if all was right after Finalize
409    # ------------------------------------------------------------------
410    IGCM_debug_Verif_Exit
411
412    #D-
413    # ------------------------------------------------------------------
414    #D- Launch remote rebuild or/and post-treatment process if necessary
415    # ------------------------------------------------------------------
416    IGCM_post_Configure
417
418    #D-
419    # ------------------------------------------------------------------
420    #D- Save job output
421    #D- Manage executable size to save only different binary
422    #D- Write in run.card user, system and elapse time
423    #D- Check that everything went well ! No ? then we stop.
424    #D- Determine next computed period
425    # ------------------------------------------------------------------
426    IGCM_config_PeriodEnd
427
428    echo "Ending iteration ${Period}"
429    (( Period = Period + 1 ))
430
431    # End loop if date end is reached
432    if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
433        break ;
434    fi
435
436    PeriodContinue=false
437    if [ ${Period} -le ${PeriodNb} ]; then
438        PeriodContinue=true
439    fi
440done 
441
442#D-
443#D--------------------------------------------------------------------==
444#D-     5. Submit next job or send mail if simulation over.
445#D--------------------------------------------------------------------==
446IGCM_config_Finalize
447
448date
Note: See TracBrowser for help on using the repository browser.