source: trunk/libIGCM/AA_rebuild_fromWorkdir @ 1093

Last change on this file since 1093 was 1070, checked in by sdipsl, 10 years ago
  • propagate supervision down to post-processing. See #210
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 6.9 KB
Line 
1#-Q- curie ######################
2#-Q- curie ## CURIE   TGCC/CEA ##
3#-Q- curie ######################
4#-Q- curie #MSUB -r REBUILDWRK     # Nom du job
5#-Q- curie #MSUB -eo
6#-Q- curie #MSUB -n 1              # Reservation du processus
7#-Q- curie #MSUB -T 36000          # Limite de temps elapsed du job
8#-Q- curie #MSUB -q xlarge         # fat node
9#-Q- curie ##MSUB -q standard      # thin nodes
10#-Q- curie ##MSUB -x               # exclusive node (needed to use high speed disk below)
11#-Q- curie #MSUB -Q normal
12#-Q- curie #MSUB -A ::default_project::
13#-Q- curie #RUN_DIR_PATH=/tmp/REBUILD_DIR_MR_$$     # RAM high speed disk partition ; 250GB/node. Along with -q xlarge
14#-Q- curie #RUN_DIR_PATH=/dev/shm/REBUILD_DIR_MR_$$ # SSD high speed disk partition ; 64GB/node.  Along with -q standard
15#-Q- curie set +x
16#-Q- ada #!/bin/ksh
17#-Q- ada #######################
18#-Q- ada ## ADA         IDRIS ##
19#-Q- ada #######################
20#-Q- ada # @ job_type = serial
21#-Q- ada # @ requirements = (Feature == "prepost")
22#-Q- ada # Temps Elapsed max. d'une requete hh:mm:ss
23#-Q- ada # @ wall_clock_limit = 10:00:00
24#-Q- ada # Nom du travail LoadLeveler
25#-Q- ada # @ job_name   = REBUILDWRK
26#-Q- ada # Fichier de sortie standard du travail
27#-Q- ada # @ output     = $(job_name).$(jobid)
28#-Q- ada # Fichier de sortie d'erreur du travail
29#-Q- ada # @ error      =  $(job_name).$(jobid)
30#-Q- ada # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
31#-Q- ada # @ notification = error
32#-Q- ada # @ environment  = $DEBUG_debug ; $BigBrother ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $POST_DIR ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
33#-Q- ada # @ queue
34#-Q- lxiv8 ######################
35#-Q- lxiv8 ## OBELIX      LSCE ##
36#-Q- lxiv8 ######################
37#-Q- lxiv8 #PBS -N REBUILDWRK
38#-Q- lxiv8 #PBS -m a
39#-Q- lxiv8 #PBS -j oe
40#-Q- lxiv8 #PBS -q medium
41#-Q- lxiv8 #PBS -o REBUILDWRK.$$
42#-Q- lxiv8 #PBS -S /bin/ksh
43#-Q- default #!/bin/ksh
44#-Q- default ##################
45#-Q- default ## DEFAULT HOST ##
46#-Q- default ##################
47
48#**************************************************************
49# Author: Sebastien Denvil
50# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
51# $Revision::                                          $ Revision of last commit
52# $Author::                                            $ Author of last commit
53# $Date::                                              $ Date of last commit
54# IPSL (2006)
55#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
56#
57#**************************************************************
58
59#set -eu
60#set -vx
61
62date
63
64#-Q- ada export OMP_NUM_THREADS=1
65
66#D- Task type (computing or post-processing)
67TaskType=post-processing
68
69########################################################################
70
71#D- Flag to determine if this job in a standalone mode
72#D- Default : value from AA_job if any
73StandAlone=${StandAlone:=true}
74
75#D- Path to libIGCM
76#D- Default : value from AA_job if any
77# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
78# WARNING you must check MirrorlibIGCM variable in sys library.
79# WARNING If this variable is true, you must use libIGCM_POST path instead
80# WARNING of your running libIGCM directory.
81libIGCM=${libIGCM:=::modipsl::/libIGCM}
82
83#-D- $hostname of the MASTER job when SUBMIT_DIR is not visible on postprocessing computer.
84MASTER=${MASTER:=ada|curie}
85
86#D- Directory where files we need to rebuild are store
87#D- Default : value from AA_job if any
88#D- REBUILD_DIR=${BIGDIR}/REBUILD/TagName/JobName/
89#D- example : /scratch/cont003/p86denv/REBUILD/IPSLCM5/SCAL-NEW
90REBUILD_DIR=${REBUILD_DIR:=/path/to/your/TMP/REBUILD/FILES}
91
92#D- How many directory to rebuild we have to consider
93#D- Default : value from AA_job if any
94NbRebuildDir=${NbRebuildDir:=1000}
95
96#D- Suffix date we will use to determine which directory to rebuild
97#D- We will rebuild NbRebuildDir before and including PeriodDateBegin
98#D- Default : value from AA_job if any
99LastPeriodForRebuild=${LastPeriodForRebuild:=${PeriodDateBegin:=18901201}}
100
101#D- Flag to determine atlas job's output directory
102#D- Default : value from libIGCM_post.ksh if any
103POST_DIR=${POST_DIR:=${PBS_O_WORKDIR}}
104
105#-D- libIGCM_SX contains libIGCM on Master computer. Required on some center. Always OK.
106libIGCM_SX=${libIGCM_SX:=::modipsl::/libIGCM}
107
108#D- Increased verbosity (1, 2, 3)
109#D- Default : value from AA_job if any
110Verbosity=${Verbosity:=3}
111
112#D- Low level debug : to bypass lib test checks and stack construction
113#D- Default : value from AA_job if any
114DEBUG_debug=${DEBUG_debug:=false}
115
116#D- TEMPORARY Flag to determine atmospheric resolution
117#D- Default : value from atmospheric driver if any
118RESOL_ATM=${RESOL_ATM:=ALL}
119
120#D- TEMPORARY Flag to determine ocean resolution
121#D- Default : value from ocean driver if any
122RESOL_OCE=${RESOL_OCE:=ORCA2}
123
124#D- TEMPORARY Flag to determine ice resolution
125#D- Default : value from ice driver if any
126RESOL_ICE=${RESOL_ICE:=ORCA2}
127
128#D- TEMPORARY Flag to determine biogeochemistry resolution
129#D- Default : value from ice driver if any
130RESOL_MBG=${RESOL_MBG:=ORCA2}
131
132#D- Flag to determine surface resolution
133#D- Default : value from surface driver if any
134RESOL_SRF=ALL
135
136#D- Flag to determine surface resolution
137#D- Default : value from surface driver if any
138RESOL_SBG=ALL
139
140########################################################################
141
142. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
143. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
144. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
145#-------
146. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
147. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
148#-------
149( ${DEBUG_debug} ) && IGCM_debug_Check
150( ${DEBUG_debug} ) && IGCM_card_Check
151( ${DEBUG_debug} ) && IGCM_date_Check
152
153########################################################################
154
155#set -vx
156
157RUN_DIR=${RUN_DIR_PATH}
158IGCM_sys_MkdirWork ${RUN_DIR}
159IGCM_sys_Cd ${RUN_DIR}
160
161# ------------------------------------------------------------------
162# Test if all was right before proceeding further
163# ------------------------------------------------------------------
164IGCM_debug_Verif_Exit_Post
165#
166ListAllRebuildDir=$( IGCM_sys_RshMaster ls -d ${REBUILD_DIR}/REBUILD_* | grep -B $(( ${NbRebuildDir} -1 )) ${LastPeriodForRebuild} )
167#
168
169for directory in ${ListAllRebuildDir} ; do
170  #
171  IGCM_sys_Get_Master ${directory} ${RUN_DIR}
172  #
173  IGCM_sys_Cd ${RUN_DIR}/$( basename ${directory} )
174
175  # --------------------------------------------------------------------
176  # Source function include in the REBUILD ksh and rebuild
177  # --------------------------------------------------------------------
178  . ${RUN_DIR}/$( basename ${directory} )/rebuild.ksh
179  IGCM_FlushRebuild
180  #
181  IGCM_sys_Cd ${RUN_DIR}
182  # Clean Up
183  # local clean up
184  IGCM_sys_Rm -rf $( basename ${directory} )
185  # remote clean up
186  IGCM_sys_RshMaster rm -rf ${directory}
187done
188
189# Clean RUN_DIR_PATH (necessary for cesium and titane only)
190IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
191
192date
Note: See TracBrowser for help on using the repository browser.