source: tags/libIGCM_v2.0_beta3/AA_monitoring @ 1137

Last change on this file since 1137 was 681, checked in by mafoipsl, 12 years ago

On curie : Computing job on -q standard ie thin nodes, post-processing jobs on -q large ie large nodes. Stop default echo of commands.
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 15.2 KB
Line 
1#-Q- cesium #!/bin/ksh
2#-Q- cesium ######################
3#-Q- cesium ## CESIUM   CEA ##
4#-Q- cesium ######################
5#-Q- cesium #MSUB -r MONITORING     # Nom du job               
6#-Q- cesium #MSUB -N 1              # Reservation du noeud
7#-Q- cesium #MSUB -n 1              # Reservation du processus
8#-Q- cesium #MSUB -T 86400          # Limite de temps elapsed du job
9#-Q- cesium #MSUB -E "-j o"
10#-Q- cesium #MSUB -E "-S /bin/ksh"
11#-Q- curie ######################
12#-Q- curie ## CURIE   TGCC/CEA ##
13#-Q- curie ######################
14#-Q- curie #MSUB -r MONITORING     # Nom du job               
15#-Q- curie #MSUB -eo
16#-Q- curie #MSUB -n 1              # Reservation du processus
17#-Q- curie #MSUB -T 86400          # Limite de temps elapsed du job
18#-Q- curie #MSUB -E '--no-requeue'
19#-Q- curie #MSUB -q large
20#-Q- curie set +x
21#-Q- platine #!/usr/bin/ksh
22#-Q- platine ##################
23#-Q- platine ## PLATINE   CEA ##
24#-Q- platine ##################
25#-Q- platine #BSUB -J MONITORING             # Nom du job
26#-Q- platine #BSUB -N                        # message a la fin du job
27#-Q- platine #BSUB -n 1                      # reservation des processeurs pour le job
28#-Q- platine #BSUB -W 1:00                   # Limite temps
29#-Q- platine #BSUB -q post              # Passage en queue post
30#-Q- platine #BSUB -L /bin/ksh
31#-Q- sx8brodie #!/bin/ksh
32#-Q- sx8brodie #######################
33#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
34#-Q- sx8brodie #######################
35#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
36#-Q- sx8brodie # @ wall_clock_limit = 1:00:00
37#-Q- sx8brodie # Nom du travail LoadLeveler
38#-Q- sx8brodie # @ job_name   = MONITORING
39#-Q- sx8brodie # Fichier de sortie standard du travail       
40#-Q- sx8brodie # @ output     = $(job_name).$(jobid)
41#-Q- sx8brodie # Fichier de sortie d'erreur du travail
42#-Q- sx8brodie # @ error      =  $(job_name).$(jobid)
43#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
44#-Q- sx8brodie # @ notification = error
45#-Q- sx8brodie # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ;  $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
46#-Q- sx8brodie # @ queue
47#-Q- aix6 #!/bin/ksh
48#-Q- aix6 #######################
49#-Q- aix6 ##   VARGAS   IDRIS  ##
50#-Q- aix6 #######################
51#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
52#-Q- aix6 # @ wall_clock_limit = 1:00:00
53#-Q- aix6 # Nom du travail LoadLeveler
54#-Q- aix6 # @ job_name   = MONITORING
55#-Q- aix6 # Fichier de sortie standard du travail       
56#-Q- aix6 # @ output     = $(job_name).$(jobid)
57#-Q- aix6 # Fichier de sortie d'erreur du travail
58#-Q- aix6 # @ error      =  $(job_name).$(jobid)
59#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
60#-Q- aix6 # @ notification = error
61#-Q- aix6 # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
62#-Q- aix6 # @ queue
63#-Q- ulam #!/bin/ksh
64#-Q- ulam #######################
65#-Q- ulam ##   ULAM   IDRIS  ##
66#-Q- ulam #######################
67#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
68#-Q- ulam # @ wall_clock_limit = 1:00:00
69#-Q- ulam # Nom du travail LoadLeveler
70#-Q- ulam # @ job_name   = MONITORING
71#-Q- ulam # Fichier de sortie standard du travail       
72#-Q- ulam # @ output     = $(job_name).$(jobid)
73#-Q- ulam # Fichier de sortie d'erreur du travail
74#-Q- ulam # @ error      =  $(job_name).$(jobid)
75#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
76#-Q- ulam # @ notification = error
77#-Q- ulam # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
78#-Q- ulam # @ queue
79#-Q- sx8mercure #!/bin/ksh
80#-Q- sx8mercure ######################
81#-Q- sx8mercure ## SX8MERCURE   CEA ##
82#-Q- sx8mercure ######################
83#-Q- sx8mercure #PBS -N  MONITORING          # Nom du job
84#-Q- sx8mercure #PBS -j o                    # regroupement des stdout et stderr
85#-Q- sx8mercure #PBS -S /usr/bin/ksh         # shell de soumission
86#-Q- sx8mercure #PBS -l memsz_job=1gb        # Limite memoire a 1 Go
87#-Q- sx8mercure #PBS -l cputim_job=1:00:00   # Limite temps a 2 heures
88#-Q- sx8mercure #PBS -q scalaire
89#-Q- sx8mercure #PBS -r n
90#-Q- sx9mercure #!/bin/ksh
91#-Q- sx9mercure ######################
92#-Q- sx9mercure ## SX9MERCURE  CCRT ##
93#-Q- sx9mercure ######################
94#-Q- sx9mercure #PBS -N  MONITORING          # Nom du job
95#-Q- sx9mercure #PBS -j o                    # regroupement des stdout et stderr
96#-Q- sx9mercure #PBS -S /usr/bin/ksh         # shell de soumission
97#-Q- sx9mercure #PBS -l memsz_job=1gb        # Limite memoire a 1 Go
98#-Q- sx9mercure #PBS -l elapstim_req=1:00:00   # Limite temps a 2 heures
99#-Q- sx9mercure #PBS -q scalaire
100#-Q- sx9mercure #PBS -r n
101#-Q- titane #!/bin/ksh
102#-Q- titane ######################
103#-Q- titane ## TITANE   CEA ##
104#-Q- titane ######################
105#-Q- titane #MSUB -r MONITORING     # Nom du job               
106#-Q- titane #MSUB -eo
107#-Q- titane #MSUB -n 1              # Reservation du processus
108#-Q- titane #MSUB -T 86400          # Limite de temps elapsed du job
109#-Q- titane #MSUB -q mono
110#-Q- titane #MSUB -E '-rn'
111#-Q- lxiv8 ######################
112#-Q- lxiv8 ## OBELIX      LSCE ##
113#-Q- lxiv8 ######################
114#-Q- lxiv8 #PBS -N MONITORING
115#-Q- lxiv8 #PBS -m a
116#-Q- lxiv8 #PBS -j oe
117#-Q- lxiv8 #PBS -q medium
118#-Q- lxiv8 #PBS -o MONITORING.$$
119#-Q- lxiv8 #PBS -S /bin/ksh
120#-Q- default #!/bin/ksh
121#-Q- default ##################
122#-Q- default ## DEFAULT HOST ##
123#-Q- default ##################
124
125#**************************************************************
126# Author: Sebastien Denvil
127# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
128# $Revision::                                          $ Revision of last commit
129# $Author::                                            $ Author of last commit
130# $Date::                                              $ Date of last commit
131# IPSL (2006)
132#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
133#
134#**************************************************************
135
136#set -eu
137#set -vx
138
139date
140
141#-Q- sx8brodie export OMP_NUM_THREADS=1
142#-Q- aix6 export OMP_NUM_THREADS=1
143#-Q- ulam export OMP_NUM_THREADS=1
144
145########################################################################
146
147#D- Flag to determine if this job in a standalone mode
148#D- Default : value from AA_job if any
149StandAlone=${StandAlone:=true}
150
151#D- Path to libIGCM
152#D- Default : value from AA_job if any
153# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
154# WARNING you must check MirrorlibIGCM variable in sys library.
155# WARNING If this variable is true, you must use libIGCM_POST path instead
156# WARNING of your running libIGCM directory.
157libIGCM=${libIGCM:=::modipsl::/libIGCM}
158
159#D- Increased verbosity (1, 2, 3)
160#D- Default : value from AA_job if any
161Verbosity=${Verbosity:=3}
162
163#D- Low level debug : to bypass lib test checks and stack construction
164#D- Default : value from AA_job if any
165DEBUG_debug=${DEBUG_debug:=false}
166
167#D- TEMPORARY Flag to determine atmospheric resolution
168#D- Default : value from atmospheric driver if any
169RESOL_ATM=${RESOL_ATM:=ALL}
170
171#D- Flag to determine surface resolution
172#D- Default : value from surface driver if any
173RESOL_SRF=${RESOL_SRF:=ALL}
174
175#D- Flag to determine surface resolution
176#D- Default : value from surface driver if any
177RESOL_SBG=${RESOL_SBG:=ALL}
178
179#D- TEMPORARY Flag to determine ocean resolution
180#D- Default : value from ocean driver if any
181RESOL_OCE=${RESOL_OCE:=ORCA2}
182
183#D- TEMPORARY Flag to determine ice resolution
184#D- Default : value from ice driver if any
185RESOL_ICE=${RESOL_ICE:=ORCA2}
186
187#D- TEMPORARY Flag to determine marine biogeochemistry resolution
188#D- Default : value from ice driver if any
189RESOL_MBG=${RESOL_MBG:=ORCA2}
190
191######################################################################
192
193. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
194      ( ${DEBUG_debug} ) && IGCM_debug_Check
195. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
196     ( ${DEBUG_debug} ) && IGCM_card_Check
197. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
198     ( ${DEBUG_debug} ) && IGCM_date_Check
199#-------
200. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
201
202######################################################################
203
204#set -vx
205
206#===========================================
207RUN_DIR=${RUN_DIR_PATH}
208IGCM_sys_MkdirWork ${RUN_DIR}
209IGCM_sys_Cd ${RUN_DIR}
210
211# ------------------------------------------------------------------
212# Test if all was right before proceeding further
213# ------------------------------------------------------------------
214IGCM_debug_Verif_Exit_Post
215
216if [ ${StandAlone} = true ] ; then
217    CARD_DIR=${SUBMIT_DIR}
218else
219    CARD_DIR=${RUN_DIR}
220    IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
221    IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card    ${RUN_DIR_PATH}
222    IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP        ${RUN_DIR_PATH}
223    IGCM_sys_Get_Master ${SUBMIT_DIR}/POST        ${RUN_DIR_PATH}
224fi
225
226#
227# First of all
228#
229IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card UserChoices
230typeset option
231for option in ${config_UserChoices[*]} ; do
232    IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
233done
234
235IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card Ensemble
236typeset option
237for option in ${config_Ensemble[*]} ; do
238    IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Ensemble ${option}
239done
240IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card ListOfComponents
241
242#==================================
243#R_SAVE : Job output directory
244#R_BUFR : Job output buffered directory
245if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
246  FreeName=$( echo ${config_UserChoices_JobName} | sed 's/.*_//' )
247  if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
248    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
249    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
250    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
251    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
252  else
253    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
254    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
255    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
256    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
257  fi
258else
259  if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
260    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
261    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
262    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
263    R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
264  else
265    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
266    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
267    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
268    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
269  fi
270fi
271
272IGCM_sys_MkdirArchive ${R_FIGR}
273#
274IGCM_sys_TestDirArchive ${R_FIGR}/MONITORING
275if [ $? = 0 ] ; then
276    IGCM_debug_Print 1 "Get MONITORING directory from archive"
277    IGCM_sys_Get_Dir ${R_FIGR}/MONITORING ${RUN_DIR}
278else
279    IGCM_debug_Print 1 "MONITORING first pass. Nothing has been done before"
280fi
281# --------------------------------------------
282# Insert your commands between III...III lines
283# and precise produced directories to save
284# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
285
286for comp in ${config_ListOfComponents[*]} ; do
287    IGCM_debug_Print 1 "################## Component: ${comp} ######################"
288    liste_file_monitoring=""
289    IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
290    eval compname=\${config_ListOfComponents_${comp}[0]}                > /dev/null 2>&1
291
292    PATH_monitoring_file=""
293    eval monitoring_file=monitoring01_${compname}_\${RESOL_${comp}}.cfg > /dev/null 2>&1
294    if [[ -d ${CARD_DIR}/POST && -f ${CARD_DIR}/POST/monitoring01_${compname}.cfg ]] ; then
295        PATH_monitoring_file=${CARD_DIR}/POST/monitoring01_${compname}.cfg
296    elif [ -f ${FER_ATLAS}/${monitoring_file} ] ; then
297        PATH_monitoring_file=${FER_ATLAS}/${monitoring_file}
298    else
299        IGCM_debug_Print 1 "No monitoring file found for this component. Was expecting ${monitoring_file}"
300        IGCM_debug_Print 1 "Step to next component"
301        continue
302    fi
303    #
304    if [ X${PATH_monitoring_file} != X"" ] ; then
305        IGCM_debug_Print 1 "Monitoring file used : ${PATH_monitoring_file}"
306        IGCM_debug_Print 1 "Determine which files we need."
307        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
308        liste_file_monitoring=$( . monitoring01 -q -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO )
309    fi
310    #
311    if [ ! "X${liste_file_monitoring}" = X ] ; then
312        IGCM_sys_Get /l liste_file_monitoring ${RUN_DIR}
313        IGCM_debug_Print 1 "monitoring01 -c ${CARD_DIR} -p ${comp} --time -t \"${config_UserChoices_JobName} monitoring\" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} ."
314        IGCM_debug_Print 1 "monitoring01 starts ................................................."
315        monitoring01 -c ${CARD_DIR} -p ${comp} --time -t "${config_UserChoices_JobName} monitoring" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} .
316    else
317        IGCM_debug_Print 1 "No time series detected by this command :"
318        IGCM_debug_Print 1 "monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO"
319        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
320    fi
321    #
322done
323
324# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
325
326# Save files
327IGCM_sys_Put_Dir MONITORING ${R_FIGR}
328
329# Dods copy
330IGCM_sys_Put_Dods MONITORING
331
332# Clean RUN_DIR_PATH (necessary for cesium and titane only)
333IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
Note: See TracBrowser for help on using the repository browser.