source: trunk/libIGCM/AA_job @ 437

Last change on this file since 437 was 437, checked in by sdipsl, 13 years ago

Send mail notification when computing job stops
It could be that we send a notification if a post-processing job stops in PROD mode ---> need some refactoring and caution...

  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 22.9 KB
Line 
1#-Q- cesium #!/bin/ksh
2#-Q- cesium ###################################################
3#-Q- cesium ## CESIUM   CEA                                  ##
4#-Q- cesium ## Usefull for postprocessing. AA_job not tested ##
5#-Q- cesium ###################################################
6#-Q- cesium #MSUB -r ::Jobname::    # Nom du job               
7#-Q- cesium #MSUB -N 1              # Reservation du noeud
8#-Q- cesium #MSUB -n 1              # Reservation du processus
9#-Q- cesium #MSUB -T 86400          # Limite de temps elapsed du job
10#-Q- cesium #MSUB -E "-j o"
11#-Q- cesium #MSUB -E "-S /bin/ksh"
12#-Q- platine #!/usr/bin/ksh
13#-Q- platine ##################
14#-Q- platine ## PLATINE   CEA ##
15#-Q- platine ##################
16#-Q- platine #BSUB -J ::Jobname::       # nom de la requete
17#-Q- platine #BSUB -N                   # message a la fin du job
18#-Q- platine #BSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
19#-Q- platine #BSUB -n ::JobNumProcTot::             # reservation des processeurs pour le job
20#-Q- platine #BSUB -W 1:00           # Limite temps
21#-Q- platine #BSUB -L /bin/ksh
22#-Q- platine BATCH_NUM_PROC_TOT=$SLURM_NPROCS
23#-Q- titane #!/usr/bin/ksh
24#-Q- titane ##################
25#-Q- titane ## TITANE   CEA ##
26#-Q- titane ##################
27#-Q- titane #MSUB -r ::Jobname::       # nom de la requete
28#-Q- titane #MSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
29#-Q- titane #MSUB -n ::JobNumProcTot::             # reservation des processeurs pour le job
30#-Q- titane #MSUB -T 86400           # Limite temps (en secondes)
31#-Q- titane #MSUB -p gen2211
32#-Q- titane BATCH_NUM_PROC_TOT=$BRIDGE_MSUB_NPROC
33#-Q- sx8brodie #######################
34#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
35#-Q- sx8brodie #######################
36#-Q- sx8brodie #PBS -N ::Jobname::         # nom de la requete
37#-Q- sx8brodie #PBS -m a                   # message a la fin du job
38#-Q- sx8brodie #PBS -j o                   # regroupement des sorties du job : standard error et output
39#-Q- sx8brodie #PBS -o Script_Output_::Jobname::.000001  # nom du fichier de sortie
40#-Q- sx8brodie #PBS -S /usr/bin/ksh        # shell de soumission
41#-Q- sx8brodie #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
42#-Q- sx8brodie #PBS -q ::JobClass::                # classe multi pour BATCH_NUM_PROC_TOT superieur a 2, mono sinon
43#-Q- sx8brodie #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}  # reservation des processeurs pour le job
44#-Q- sx8brodie #PBS -l memsz_job=15.0gb     # Limite memoire
45#-Q- sx8brodie #PBS -l cputim_job=1:00:00  # Limite temps
46#-Q- aix6 #!/usr/bin/ksh
47#-Q- aix6 # ######################
48#-Q- aix6 # ##   VARGAS IDRIS   ##
49#-Q- aix6 # ######################
50#-Q- aix6 # Nom de la requete
51#-Q- aix6 # @ job_name = ::Jobname::
52#-Q- aix6 # Type de travail
53#-Q- aix6 # @ job_type = parallel
54#-Q- aix6 # Fichier de sortie standard
55#-Q- aix6 # @ output = $(job_name).$(jobid)
56#-Q- aix6 # Fichier de sortie erreur (le meme)
57#-Q- aix6 # @ error =  $(job_name).$(jobid)
58#-Q- aix6 # Nombre de processus demandes
59#-Q- aix6 # @ total_tasks = ::JobNumProcTot::
60#-Q- aix6 # @ environment = "BATCH_NUM_PROC_TOT=::JobNumProcTot::"
61#-Q- aix6 # Temps CPU max. par processus MPI hh:mm:ss
62#-Q- aix6 # @ wall_clock_limit = 1:00:00
63#-Q- aix6 # Memoire max. utilisee par processus
64#-Q- aix6 # @ data_limit = 3.2gb
65#-Q- aix6 # Memoire stack demandee
66#-Q- aix6 # @ stack_limit = 0.3gb,0.3gb
67#-Q- aix6 # Pas d OpenMP
68#-Q- aix6 # @ resources = ConsumableCpus(1)
69#-Q- aix6 # Fin de l entete
70#-Q- aix6 # @ queue
71#-Q- ulam #!/usr/bin/ksh
72#-Q- ulam # ####################################################
73#-Q- ulam # ##  ULAM IDRIS                                    ##
74#-Q- ulam # ##  Usefull for postprocessing. AA_job not tested ##
75#-Q- ulam # ####################################################
76#-Q- ulam #
77#-Q- sx8mercure #!/bin/ksh
78#-Q- sx8mercure ######################
79#-Q- sx8mercure ## SX8MERCURE   CEA ##
80#-Q- sx8mercure ######################
81#-Q- sx8mercure #PBS -N ::Jobname::           # nom de la requete
82#-Q- sx8mercure #PBS -m a                     # message a la fin du job
83#-Q- sx8mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
84#-Q- sx8mercure ####PBS -A gen6178
85#-Q- sx8mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
86#-Q- sx8mercure #PBS -S /usr/bin/ksh          # shell de soumission
87#-Q- sx8mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
88#-Q- sx8mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
89#-Q- sx8mercure #PBS -l memsz_job=15.0gb       # limite memoire
90#-Q- sx8mercure #PBS -l elapstim_req=01:00:00 # limite en temps elapsed
91#-Q- sx9mercure #!/bin/ksh
92#-Q- sx9mercure ######################
93#-Q- sx9mercure ## SX9MERCURE   CEA ##
94#-Q- sx9mercure ######################
95#-Q- sx9mercure #PBS -N ::Jobname::           # nom de la requete
96#-Q- sx9mercure #PBS -m a                     # message a la fin du job
97#-Q- sx9mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
98#-Q- sx9mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
99#-Q- sx9mercure #PBS -S /usr/bin/ksh          # shell de soumission
100#-Q- sx9mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
101#-Q- sx9mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
102#-Q- sx9mercure #PBS -l memsz_job=16.0gb       # limite memoire
103#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # limite en temps elapsed
104#-Q- sx9mercure #PBS -A gen6178               # groupe obligatoire sur SX9
105#-Q- sx9mercure #PBS -q para-sx9              # para-sx9 or prio-sx9 (for CMIP5 top priorities), bckg-sx9 (run with free slot) and test-sx9 (if elapsed<1h)
106#-Q- lxiv8 ######################
107#-Q- lxiv8 ## OBELIX      LSCE ##
108#-Q- lxiv8 ######################
109#-Q- lxiv8 #PBS -N ::Jobname::
110#-Q- lxiv8 #PBS -m a
111#-Q- lxiv8 #PBS -j oe
112#-Q- lxiv8 #PBS -q medium
113#-Q- lxiv8 #PBS -o Script_Output_::Jobname::.000001
114#-Q- lxiv8 #PBS -S /bin/ksh
115#-Q- lxiv8 #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
116#-Q- lxiv8 #PBS -l nodes=1:ppn=::JobNumProcTot::
117#-Q- default #!/bin/ksh
118#-Q- default ##################
119#-Q- default ## DEFAULT HOST ##
120#-Q- default ##################
121#-Q- default #For MPI use, uncomment next line :
122#-Q- default #BATCH_NUM_PROC_TOT=::JobNumProcTot::
123
124#**************************************************************
125# Author: Sebastien Denvil
126# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
127# $Revision::                                          $ Revision of last commit
128# $Author::                                            $ Author of last commit
129# $Date::                                              $ Date of last commit
130# IPSL (2006)
131#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
132#
133#**************************************************************
134
135#set -eu
136#set -vx
137
138date
139echo
140echo "#######################################"
141echo "#       ANOTHER GREAT SIMULATION      #"
142echo "#######################################"
143echo
144
145#D--------------------------------------------------------------------==
146#D--------------------------------------------------------------------==
147#D-
148#D-                      Job to launch IGCM models
149#D-
150#D--------------------------------------------------------------------==
151#D--------------------------------------------------------------------==
152#D-
153
154MODIPSL=::modipsl::
155libIGCM=${MODIPSL}/libIGCM
156
157#D--------------------------------------------------------------------==
158#D-                   -1. User modification place
159#D-                      - Job Verbosity
160#D-                      - PeriodNb
161#D-                      - Experience type : DEB(ug), DEV(elopment), RUN
162#D--------------------------------------------------------------------==
163
164#D- Increased verbosity (1, 2, 3)
165Verbosity=3
166#D- Number of execution in one job
167PeriodNb=1
168#-Q- sx9mercure PeriodNb=60
169
170#D-
171# --------------------------------------------------------------------
172#D- ! Optionnal features /!\ Use with care !
173# --------------------------------------------------------------------
174
175#D- Low level debug : to bypass lib test checks and stack construction
176#D- Default=true
177DEBUG_debug=false
178
179#D- Set DEBUG_sys to false to disable output calls of function
180#D- Default=true
181#DEBUG_sys=false
182
183#D- Experience type : DEB(ug), DEV(elopment), RUN (default)
184JobType=RUN
185
186#D- Define running directory
187#D- Default=${TMPDIR} ie temporary batch directory
188#RUN_DIR_PATH=/workdir/or/scratchdir/of/this/machine
189
190#D- Define submit directory
191#D- Default= where you launch qsub (variable from scheduler)
192#SUBMIT_DIR=$( pwd )
193
194#D- Define BIG_DIR directory
195#D- Default=(strongly machine dependant/see libIGCM_sys for details)
196#BIG_DIR=/somewhere/with/lot/of/space/scratchdir/may/be
197
198#D- Turn in dry run mode ? (0,1,2,3)
199#D- Default=0
200#DRYRUN=3
201# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
202# -------------------------------------------------------------------------------------
203# | DRYRUN=  |  Date computations, | sys_Get    |  Exe    | sys_Put_Out; sys_Put_Rest |
204# |          |  Cp/Exe param files |            |  Chmod  |                           |
205# |          |      Qsub           |            |         |                           |
206# -------------------------------------------------------------------------------------
207# |    0     |       yes           |    yes     |  yes    |      yes                  |
208# -------------------------------------------------------------------------------------
209# |    1     |       yes           |    yes     |  yes    |      no                   |
210# -------------------------------------------------------------------------------------
211# |    2     |       yes           |    yes     |  no     |      no                   |
212# -------------------------------------------------------------------------------------
213# |    3     |       yes           |    no      |  no     |      no                   |
214# -------------------------------------------------------------------------------------
215
216#D- Define input file root directory
217#D- Default=/IGCMG/common/account/of/this/machine
218#R_IN=/u/rech/por/rpor111/DATA
219
220#D-
221#D--------------------------------------------------------------------==
222#D-                   0. System Environment
223#D-                      - Define variables for Number of processors
224#D-                      - Define MPI variables
225#D-                      - Define batch scheduler variables
226#D-                      - Source IGCM Library
227#D-                      - Get RESOLution in .resol file (temporary)
228#D--------------------------------------------------------------------==
229
230#D--------------------------------------------------------------------==
231
232. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
233       ( ${DEBUG_debug} ) && IGCM_debug_Check
234. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
235       ( ${DEBUG_debug} ) && IGCM_card_Check
236. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
237       ( ${DEBUG_debug} ) && IGCM_date_Check
238#-------
239. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
240. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
241. ${libIGCM}/libIGCM_comp/libIGCM_comp.ksh
242. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
243
244#D--------------------------------------------------------------------==
245
246#-- Resolution FLAG for oasis input
247[ -f ${SUBMIT_DIR}/../.resol ] && RESOL=$(head -1 ${SUBMIT_DIR}/../.resol)
248#RESOL="ORCA2xLMD9671"
249
250#D-
251#D--------------------------------------------------------------------==
252#D-           1. INITIALIZE CONFIGURATION AND THEN COMPONENTS
253#D--------------------------------------------------------------------==
254
255# --------------------------------------------------------------------==
256#D- Initialize configuration :
257#D-              - Simulation configuration
258#D-              - Simulation parameters
259#D-              - Execution parameter
260#D-              - Define input files directory
261#D-              - Read or initialize CumulPeriod
262#D-              - run.card
263#D-
264# --------------------------------------------------------------------==
265IGCM_config_Initialize
266
267# --------------------------------------------------------------------==
268#D- Define, create and cd RUN_DIR
269#D- 
270# --------------------------------------------------------------------==
271RUN_DIR=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}.${$}
272IGCM_sys_MkdirWork ${RUN_DIR}
273IGCM_sys_Cd ${RUN_DIR}
274
275# ------------------------------------------------------------------
276# Test if all was right before proceeding further
277# ------------------------------------------------------------------
278IGCM_debug_Verif_Exit
279
280# --------------------------------------------------------------------==
281#D- Initialize each components taking part
282#D- into simulation configuration :
283#D-         - Define component executable
284#D-         - Define Write Frequency by component "1M" "1D" ...
285#D-         - Source ${comp}.driver
286#D-         - Define Storage Places by component
287#D-         - ${comp}_Initialize :
288#D-         - Patterns to sed
289#D-         - Variables from namelist
290#D-
291# --------------------------------------------------------------------==
292IGCM_comp_Initialize
293
294# --------------------------------------------------------------------==
295#D- Verify compatibility of period length, write frequencies, ...
296#D- Verify compatibility of rebuild choices and post-process choices
297#D- Will stop here if something is wrong
298# --------------------------------------------------------------------==
299IGCM_config_Check
300
301# ------------------------------------------------------------------
302#D- Test if all was right before entering the period loop
303# ------------------------------------------------------------------
304IGCM_debug_Verif_Exit
305
306#D--------------------------------------------------------------------==
307#D-                  2. ENTER THE DEEP EXECUTION LOOP
308#D-                  !!!! FASTEN YOUR SEAT BELTS !!!!
309#D-                  !!!!!      PLEASE.          !!!!
310#D-                  !! WE WILL CROSS TURBULENCES. !!
311#D--------------------------------------------------------------------==
312
313Period=1
314
315PeriodContinue=false
316if [ ${Period} -le ${PeriodNb} ]; then
317    PeriodContinue=true
318fi
319while ( ${PeriodContinue} ) ; do
320
321    echo
322    echo "Starting iteration ${Period} / ${PeriodNb}"
323
324    #D-
325    # ------------------------------------------------------------------
326    #D- COMPUTE AND DEFINE DATE STUFF.
327    # ------------------------------------------------------------------
328    IGCM_config_PeriodStart
329    #D- => ${PeriodDateBegin}
330    #D- => ${PeriodDateEnd}
331    #D- => ${CumulPeriod}
332    #D- => ${DatesPeriod}=${PeriodDateBegin}_${PeriodDateEnd}
333    #D- => ${PeriodLength} => JOURS=31 , 28, 29, 30 ; MOIS=0 ; ANS=0
334    #D- => update run.card value
335   
336    #D-
337    # ------------------------------------------------------------------
338    #D- BEGIN COMPUTATION ONLY IF NEEDED
339    # ------------------------------------------------------------------
340    if [ ${SimulationLengthInDays} -gt ${ExperienceLengthInDays} ] ; then
341        break ;
342    fi
343   
344    #D-
345    # ------------------------------------------------------------------
346    #D- Optionnal function in driver
347    #D- to set special variables used in forward lists (Param, Init or Bc).
348    # ------------------------------------------------------------------
349    IGCM_comp_PeriodStart
350   
351    #D-
352    # ------------------------------------------------------------------
353    #D- Get parameters text files updated by job (.def, namelist ...)
354    #D- READ AND USE BY GCM AT EACH EXECUTION.
355    # ------------------------------------------------------------------
356    IGCM_comp_GetInputParametersFiles
357
358    #D-
359    # ------------------------------------------------------------------
360    #D- Get initial state (Etat0, carteveg,relief...)
361    #D- NECESSARY ONLY IF CumulPeriod= (=> NUMERO) == 1
362    #D- AND ???_Restart=NO
363    #D- READ AND USE BY GCM FOR ONLY ONE EXECUTION.
364    # ------------------------------------------------------------------
365    IGCM_comp_GetInputInitialStateFiles
366
367    #D-
368    # ------------------------------------------------------------------
369    #D- Get Boundaries Conditions (SST, WIND[X,Y,Z], LAI ...)
370    #D- READ AND USE BY GCM AT EACH EXECUTION.
371    # ------------------------------------------------------------------
372    IGCM_comp_GetInputBoundaryFiles
373
374    #D-
375    # ------------------------------------------------------------------
376    #D- Get SmoothFiles Conditions (SST, WIND[X,Y,Z], LAI ...)
377    #D- READ AND USE BY GCM AT EACH EXECUTION but varying in time
378    # ------------------------------------------------------------------
379    IGCM_comp_GetInputSmoothFiles
380
381    #D-
382    # ------------------------------------------------------------------
383    #D- Get restart files (restartphy.nc, orca_restart.nc ...)
384    #D- READ AND USE BY GCM AT EACH EXECUTION.
385    #D-
386    #D- IF A COMPONENT DO NOT RESTART FROM PREVIOULSY COMPUTED RESTART
387    #D- ONLY IF CumulPeriod= (=> NUMERO) == 1
388    #D- MUST EXECUTE CREATE ETAT0_LIMIT (TYPICALLY LMDZ AND ./create_etat0_limit.e)
389    #D-
390    #D- IF CumulPeriod == 1
391    #D- AND A COMPONENT RESTART FROM PREVIOULSY COMPUTED RESTART
392    #D- MUST BRING THIS RESTART IN ${RUN_DIR_PATH}
393    #D-
394    #D- ELSE BRING RESTART FROM ${JobName} IN ${RUN_DIR_PATH}
395    # ------------------------------------------------------------------
396    IGCM_comp_GetInputRestartFiles
397   
398    #D-
399    # ------------------------------------------------------------------
400    #D- Test if all was right before Update
401    # ------------------------------------------------------------------
402    IGCM_debug_Verif_Exit
403
404    # ------------------------------------------------------------------
405    #D- Activate running environnement variables
406    # ------------------------------------------------------------------
407    IGCM_sys_activ_variables
408
409    #D-
410    # ------------------------------------------------------------------
411    #D- UPDATE ParametersFiles (.def, namelist ...) with
412    #D- current value of (ORCA_NIT00, ORCA_LRSTAR, RAZ_DATE, ...)
413    # ------------------------------------------------------------------
414    IGCM_comp_Update
415
416    #D-
417    # ------------------------------------------------------------------
418    #D- Test if all was right before execution
419    # ------------------------------------------------------------------
420    IGCM_debug_Verif_Exit
421
422    #D-
423    #D----------------------------------------------------------------==
424    #D-                          3. Execution
425    #D----------------------------------------------------------------==
426
427    echo
428    echo "#######################################"
429    echo "#      DIR BEFORE RUN EXECUTION       #"
430    echo "#######################################"
431    echo
432    ls -lrt
433
434    echo "========================================================================"
435    if ( [ X${BATCH_NUM_PROC_TOT} != X ] && [ "${BATCH_NUM_PROC_TOT}" -gt 1 ] ) ; then
436        MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND}
437        eval MPIRUN_OPTIONS=${config_UserChoices_JobRunOptions}
438    else
439        MPIRUN_COMMAND=${MPIRUN_COMMAND:="time "}
440        MPIRUN_OPTIONS=${MPIRUN_OPTIONS:=""}
441    fi
442
443    if [ ${DRYRUN} -le 1 ] ; then
444        REAL_DATE_INIT=$( date )
445        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name}   > ${Exe_Output} 2>&1"
446        echo                                                                                   > ${Exe_Output}
447        echo "#######################################"                                        >> ${Exe_Output}
448        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name}" >> ${Exe_Output}
449        echo                                                                                  >> ${Exe_Output}
450        typeset RET
451        RUN_DATE_BEGIN=$( date '+%Y-%m-%dT%H:%M:%S' )
452        ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} >> ${Exe_Output} 2>&1
453        RET=$?
454        RUN_DATE_END=$( date '+%Y-%m-%dT%H:%M:%S' )
455        if [ ${RET} -gt 0 ] ; then
456            echo "Return code of executable :" ${RET}
457            IGCM_debug_Exit "EXECUTABLE"
458            IGCM_sys_Cp ${Exe_Output} ${SUBMIT_DIR}/${PREFIX}_${Exe_Output}_error
459            ExecutionFail=true
460        fi
461        echo                                            >> ${Exe_Output}
462        echo "#######################################"  >> ${Exe_Output}
463        echo "libIGCM RunDateBegin=${RUN_DATE_BEGIN}"   >> ${Exe_Output}
464        echo "libIGCM RunDateEnd=${RUN_DATE_END}"       >> ${Exe_Output}
465        echo                                            >> ${Exe_Output}
466    else
467        echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} simulated for DRYRUN = " $DRYRUN
468        if ( $DEBUG_debug ) ; then
469            echo "FOR EXECUTION DRYRUN mode = " $DRYRUN >> stack
470        fi
471    fi
472    echo "========================================================================"
473
474    echo
475    echo "#######################################"
476    echo "#       DIR AFTER RUN EXECUTION       #"
477    echo "#######################################"
478    echo
479    ls -lrt
480
481    #D-
482    # ------------------------------------------------------------------
483    #D- Desactivate running environnement variables
484    # ------------------------------------------------------------------
485    IGCM_sys_desactiv_variables
486
487    #D-
488    #D----------------------------------------------------------------==
489    #D-                    4. Post execution operations
490    #D----------------------------------------------------------------==
491
492    #D-
493    # ------------------------------------------------------------------
494    #D- Save on archive machine netcdf outputs, restarts
495    #D- and text files of models
496    # ------------------------------------------------------------------
497    IGCM_comp_Finalize
498
499    #D-
500    # ------------------------------------------------------------------
501    #D- Test if all was right after Finalize
502    # ------------------------------------------------------------------
503    IGCM_debug_Verif_Exit
504
505    #D-
506    # ------------------------------------------------------------------
507    #D- Configure asynchronous post-treatment (Level 0) when necessary (rebuild)
508    #D- Configure classic post-treatment (Level 1) when necessary (TimeSeries/Seasonal)
509    # ------------------------------------------------------------------
510    IGCM_post_Configure
511
512    #D-
513    # ------------------------------------------------------------------
514    #D- Launch remote rebuild or/and post-treatment process if necessary
515    # ------------------------------------------------------------------
516    IGCM_post_Submit
517
518    #D-
519    # ------------------------------------------------------------------
520    #D- Save job output
521    #D- Manage executable size to save only different binary
522    #D- Write in run.card user, system and elapse time
523    #D- Check that everything went well ! No ? then we stop.
524    #D- Determine next computed period
525    # ------------------------------------------------------------------
526    IGCM_config_PeriodEnd
527
528    echo "Ending iteration ${Period}"
529    (( Period = Period + 1 ))
530
531    # End loop if date end is reached
532    if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
533        break ;
534    fi
535
536    PeriodContinue=false
537    if [ ${Period} -le ${PeriodNb} ]; then
538        PeriodContinue=true
539    fi
540done
541
542#D-
543#D--------------------------------------------------------------------==
544#D-     5. Submit next job or send mail if simulation over.
545#D--------------------------------------------------------------------==
546IGCM_config_Finalize
547
548date
Note: See TracBrowser for help on using the repository browser.