source: tags/libIGCM_v2.0_beta2/AA_monitoring @ 1170

Last change on this file since 1170 was 658, checked in by labetoulle, 12 years ago

Add Ensemble functionality to libIGCM - First step :
modify output directories according to an option in section [Ensemble]
in config.card (modification of config.card yet to be done).
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 15.2 KB
Line 
1#-Q- cesium #!/bin/ksh
2#-Q- cesium ######################
3#-Q- cesium ## CESIUM   CEA ##
4#-Q- cesium ######################
5#-Q- cesium #MSUB -r MONITORING     # Nom du job               
6#-Q- cesium #MSUB -N 1              # Reservation du noeud
7#-Q- cesium #MSUB -n 1              # Reservation du processus
8#-Q- cesium #MSUB -T 86400          # Limite de temps elapsed du job
9#-Q- cesium #MSUB -E "-j o"
10#-Q- cesium #MSUB -E "-S /bin/ksh"
11#-Q- curie ######################
12#-Q- curie ## CURIE   TGCC/CEA ##
13#-Q- curie ######################
14#-Q- curie #MSUB -r MONITORING     # Nom du job               
15#-Q- curie #MSUB -eo
16#-Q- curie #MSUB -n 1              # Reservation du processus
17#-Q- curie #MSUB -T 86400          # Limite de temps elapsed du job
18#-Q- curie #MSUB -E '--no-requeue'
19#-Q- platine #!/usr/bin/ksh
20#-Q- platine ##################
21#-Q- platine ## PLATINE   CEA ##
22#-Q- platine ##################
23#-Q- platine #BSUB -J MONITORING             # Nom du job
24#-Q- platine #BSUB -N                        # message a la fin du job
25#-Q- platine #BSUB -n 1                      # reservation des processeurs pour le job
26#-Q- platine #BSUB -W 1:00                   # Limite temps
27#-Q- platine #BSUB -q post              # Passage en queue post
28#-Q- platine #BSUB -L /bin/ksh
29#-Q- sx8brodie #!/bin/ksh
30#-Q- sx8brodie #######################
31#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
32#-Q- sx8brodie #######################
33#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
34#-Q- sx8brodie # @ wall_clock_limit = 1:00:00
35#-Q- sx8brodie # Nom du travail LoadLeveler
36#-Q- sx8brodie # @ job_name   = MONITORING
37#-Q- sx8brodie # Fichier de sortie standard du travail       
38#-Q- sx8brodie # @ output     = $(job_name).$(jobid)
39#-Q- sx8brodie # Fichier de sortie d'erreur du travail
40#-Q- sx8brodie # @ error      =  $(job_name).$(jobid)
41#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
42#-Q- sx8brodie # @ notification = error
43#-Q- sx8brodie # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ;  $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
44#-Q- sx8brodie # @ queue
45#-Q- aix6 #!/bin/ksh
46#-Q- aix6 #######################
47#-Q- aix6 ##   VARGAS   IDRIS  ##
48#-Q- aix6 #######################
49#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
50#-Q- aix6 # @ wall_clock_limit = 1:00:00
51#-Q- aix6 # Nom du travail LoadLeveler
52#-Q- aix6 # @ job_name   = MONITORING
53#-Q- aix6 # Fichier de sortie standard du travail       
54#-Q- aix6 # @ output     = $(job_name).$(jobid)
55#-Q- aix6 # Fichier de sortie d'erreur du travail
56#-Q- aix6 # @ error      =  $(job_name).$(jobid)
57#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
58#-Q- aix6 # @ notification = error
59#-Q- aix6 # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
60#-Q- aix6 # @ queue
61#-Q- ulam #!/bin/ksh
62#-Q- ulam #######################
63#-Q- ulam ##   ULAM   IDRIS  ##
64#-Q- ulam #######################
65#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
66#-Q- ulam # @ wall_clock_limit = 1:00:00
67#-Q- ulam # Nom du travail LoadLeveler
68#-Q- ulam # @ job_name   = MONITORING
69#-Q- ulam # Fichier de sortie standard du travail       
70#-Q- ulam # @ output     = $(job_name).$(jobid)
71#-Q- ulam # Fichier de sortie d'erreur du travail
72#-Q- ulam # @ error      =  $(job_name).$(jobid)
73#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
74#-Q- ulam # @ notification = error
75#-Q- ulam # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
76#-Q- ulam # @ queue
77#-Q- sx8mercure #!/bin/ksh
78#-Q- sx8mercure ######################
79#-Q- sx8mercure ## SX8MERCURE   CEA ##
80#-Q- sx8mercure ######################
81#-Q- sx8mercure #PBS -N  MONITORING          # Nom du job
82#-Q- sx8mercure #PBS -j o                    # regroupement des stdout et stderr
83#-Q- sx8mercure #PBS -S /usr/bin/ksh         # shell de soumission
84#-Q- sx8mercure #PBS -l memsz_job=1gb        # Limite memoire a 1 Go
85#-Q- sx8mercure #PBS -l cputim_job=1:00:00   # Limite temps a 2 heures
86#-Q- sx8mercure #PBS -q scalaire
87#-Q- sx8mercure #PBS -r n
88#-Q- sx9mercure #!/bin/ksh
89#-Q- sx9mercure ######################
90#-Q- sx9mercure ## SX9MERCURE  CCRT ##
91#-Q- sx9mercure ######################
92#-Q- sx9mercure #PBS -N  MONITORING          # Nom du job
93#-Q- sx9mercure #PBS -j o                    # regroupement des stdout et stderr
94#-Q- sx9mercure #PBS -S /usr/bin/ksh         # shell de soumission
95#-Q- sx9mercure #PBS -l memsz_job=1gb        # Limite memoire a 1 Go
96#-Q- sx9mercure #PBS -l elapstim_req=1:00:00   # Limite temps a 2 heures
97#-Q- sx9mercure #PBS -q scalaire
98#-Q- sx9mercure #PBS -r n
99#-Q- titane #!/bin/ksh
100#-Q- titane ######################
101#-Q- titane ## TITANE   CEA ##
102#-Q- titane ######################
103#-Q- titane #MSUB -r MONITORING     # Nom du job               
104#-Q- titane #MSUB -eo
105#-Q- titane #MSUB -n 1              # Reservation du processus
106#-Q- titane #MSUB -T 86400          # Limite de temps elapsed du job
107#-Q- titane #MSUB -q mono
108#-Q- titane #MSUB -E '-rn'
109#-Q- lxiv8 ######################
110#-Q- lxiv8 ## OBELIX      LSCE ##
111#-Q- lxiv8 ######################
112#-Q- lxiv8 #PBS -N MONITORING
113#-Q- lxiv8 #PBS -m a
114#-Q- lxiv8 #PBS -j oe
115#-Q- lxiv8 #PBS -q medium
116#-Q- lxiv8 #PBS -o MONITORING.$$
117#-Q- lxiv8 #PBS -S /bin/ksh
118#-Q- default #!/bin/ksh
119#-Q- default ##################
120#-Q- default ## DEFAULT HOST ##
121#-Q- default ##################
122
123#**************************************************************
124# Author: Sebastien Denvil
125# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
126# $Revision::                                          $ Revision of last commit
127# $Author::                                            $ Author of last commit
128# $Date::                                              $ Date of last commit
129# IPSL (2006)
130#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
131#
132#**************************************************************
133
134#set -eu
135#set -vx
136
137date
138
139#-Q- sx8brodie export OMP_NUM_THREADS=1
140#-Q- aix6 export OMP_NUM_THREADS=1
141#-Q- ulam export OMP_NUM_THREADS=1
142
143########################################################################
144
145#D- Flag to determine if this job in a standalone mode
146#D- Default : value from AA_job if any
147StandAlone=${StandAlone:=true}
148
149#D- Path to libIGCM
150#D- Default : value from AA_job if any
151# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
152# WARNING you must check MirrorlibIGCM variable in sys library.
153# WARNING If this variable is true, you must use libIGCM_POST path instead
154# WARNING of your running libIGCM directory.
155libIGCM=${libIGCM:=::modipsl::/libIGCM}
156
157#D- Increased verbosity (1, 2, 3)
158#D- Default : value from AA_job if any
159Verbosity=${Verbosity:=3}
160
161#D- Low level debug : to bypass lib test checks and stack construction
162#D- Default : value from AA_job if any
163DEBUG_debug=${DEBUG_debug:=false}
164
165#D- TEMPORARY Flag to determine atmospheric resolution
166#D- Default : value from atmospheric driver if any
167RESOL_ATM=${RESOL_ATM:=ALL}
168
169#D- Flag to determine surface resolution
170#D- Default : value from surface driver if any
171RESOL_SRF=${RESOL_SRF:=ALL}
172
173#D- Flag to determine surface resolution
174#D- Default : value from surface driver if any
175RESOL_SBG=${RESOL_SBG:=ALL}
176
177#D- TEMPORARY Flag to determine ocean resolution
178#D- Default : value from ocean driver if any
179RESOL_OCE=${RESOL_OCE:=ORCA2}
180
181#D- TEMPORARY Flag to determine ice resolution
182#D- Default : value from ice driver if any
183RESOL_ICE=${RESOL_ICE:=ORCA2}
184
185#D- TEMPORARY Flag to determine marine biogeochemistry resolution
186#D- Default : value from ice driver if any
187RESOL_MBG=${RESOL_MBG:=ORCA2}
188
189######################################################################
190
191. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
192      ( ${DEBUG_debug} ) && IGCM_debug_Check
193. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
194     ( ${DEBUG_debug} ) && IGCM_card_Check
195. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
196     ( ${DEBUG_debug} ) && IGCM_date_Check
197#-------
198. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
199
200######################################################################
201
202#set -vx
203
204#===========================================
205RUN_DIR=${RUN_DIR_PATH}
206IGCM_sys_MkdirWork ${RUN_DIR}
207IGCM_sys_Cd ${RUN_DIR}
208
209# ------------------------------------------------------------------
210# Test if all was right before proceeding further
211# ------------------------------------------------------------------
212IGCM_debug_Verif_Exit_Post
213
214if [ ${StandAlone} = true ] ; then
215    CARD_DIR=${SUBMIT_DIR}
216else
217    CARD_DIR=${RUN_DIR}
218    IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
219    IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card    ${RUN_DIR_PATH}
220    IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP        ${RUN_DIR_PATH}
221    IGCM_sys_Get_Master ${SUBMIT_DIR}/POST        ${RUN_DIR_PATH}
222fi
223
224#
225# First of all
226#
227IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card UserChoices
228typeset option
229for option in ${config_UserChoices[*]} ; do
230    IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
231done
232
233IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card Ensemble
234typeset option
235for option in ${config_Ensemble[*]} ; do
236    IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Ensemble ${option}
237done
238IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card ListOfComponents
239
240#==================================
241#R_SAVE : Job output directory
242#R_BUFR : Job output buffered directory
243if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
244  FreeName=$( echo ${config_UserChoices_JobName} | sed 's/.*_//' )
245  if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
246    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
247    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
248    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
249    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
250  else
251    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
252    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
253    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
254    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
255  fi
256else
257  if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
258    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
259    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
260    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
261    R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
262  else
263    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
264    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
265    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
266    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
267  fi
268fi
269
270IGCM_sys_MkdirArchive ${R_FIGR}
271#
272IGCM_sys_TestDirArchive ${R_FIGR}/MONITORING
273if [ $? = 0 ] ; then
274    IGCM_debug_Print 1 "Get MONITORING directory from archive"
275    IGCM_sys_Get_Dir ${R_FIGR}/MONITORING ${RUN_DIR}
276else
277    IGCM_debug_Print 1 "MONITORING first pass. Nothing has been done before"
278fi
279# --------------------------------------------
280# Insert your commands between III...III lines
281# and precise produced directories to save
282# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
283
284for comp in ${config_ListOfComponents[*]} ; do
285    IGCM_debug_Print 1 "################## Component: ${comp} ######################"
286    liste_file_monitoring=""
287    IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
288    eval compname=\${config_ListOfComponents_${comp}[0]}                > /dev/null 2>&1
289
290    PATH_monitoring_file=""
291    eval monitoring_file=monitoring01_${compname}_\${RESOL_${comp}}.cfg > /dev/null 2>&1
292    if [[ -d ${CARD_DIR}/POST && -f ${CARD_DIR}/POST/monitoring01_${compname}.cfg ]] ; then
293        PATH_monitoring_file=${CARD_DIR}/POST/monitoring01_${compname}.cfg
294    elif [ -f ${FER_ATLAS}/${monitoring_file} ] ; then
295        PATH_monitoring_file=${FER_ATLAS}/${monitoring_file}
296    else
297        IGCM_debug_Print 1 "No monitoring file found for this component. Was expecting ${monitoring_file}"
298        IGCM_debug_Print 1 "Step to next component"
299        continue
300    fi
301    #
302    if [ X${PATH_monitoring_file} != X"" ] ; then
303        IGCM_debug_Print 1 "Monitoring file used : ${PATH_monitoring_file}"
304        IGCM_debug_Print 1 "Determine which files we need."
305        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
306        liste_file_monitoring=$( . monitoring01 -q -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO )
307    fi
308    #
309    if [ ! "X${liste_file_monitoring}" = X ] ; then
310        IGCM_sys_Get /l liste_file_monitoring ${RUN_DIR}
311        IGCM_debug_Print 1 "monitoring01 -c ${CARD_DIR} -p ${comp} --time -t \"${config_UserChoices_JobName} monitoring\" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} ."
312        IGCM_debug_Print 1 "monitoring01 starts ................................................."
313        monitoring01 -c ${CARD_DIR} -p ${comp} --time -t "${config_UserChoices_JobName} monitoring" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} .
314    else
315        IGCM_debug_Print 1 "No time series detected by this command :"
316        IGCM_debug_Print 1 "monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO"
317        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
318    fi
319    #
320done
321
322# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
323
324# Save files
325IGCM_sys_Put_Dir MONITORING ${R_FIGR}
326
327# Dods copy
328IGCM_sys_Put_Dods MONITORING
329
330# Clean RUN_DIR_PATH (necessary for cesium and titane only)
331IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
Note: See TracBrowser for help on using the repository browser.