source: trunk/libIGCM/AA_monitoring @ 386

Last change on this file since 386 was 386, checked in by mafoipsl, 13 years ago

Add ulam header to facilitate post-processing on ulam directly.
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 12.2 KB
Line 
1#-Q- cesium #!/bin/ksh
2#-Q- cesium ######################
3#-Q- cesium ## CESIUM   CEA ##
4#-Q- cesium ######################
5#-Q- cesium #MSUB -r MONITORING     # Nom du job               
6#-Q- cesium #MSUB -N 1              # Reservation du noeud
7#-Q- cesium #MSUB -n 1              # Reservation du processus
8#-Q- cesium #MSUB -T 86400          # Limite de temps elapsed du job
9#-Q- cesium #MSUB -E "-j o"
10#-Q- cesium #MSUB -E "-S /bin/ksh"
11#-Q- platine #!/usr/bin/ksh
12#-Q- platine ##################
13#-Q- platine ## PLATINE   CEA ##
14#-Q- platine ##################
15#-Q- platine #BSUB -J MONITORING             # Nom du job
16#-Q- platine #BSUB -N                        # message a la fin du job
17#-Q- platine #BSUB -n 1                      # reservation des processeurs pour le job
18#-Q- platine #BSUB -W 1:00                   # Limite temps
19#-Q- platine #BSUB -q post              # Passage en queue post
20#-Q- sx8brodie #!/bin/ksh
21#-Q- sx8brodie #######################
22#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
23#-Q- sx8brodie #######################
24#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
25#-Q- sx8brodie # @ wall_clock_limit = 20:00:00
26#-Q- sx8brodie # Nom du travail LoadLeveler
27#-Q- sx8brodie # @ job_name   = MONITORING
28#-Q- sx8brodie # Fichier de sortie standard du travail       
29#-Q- sx8brodie # @ output     = $(job_name).$(jobid)
30#-Q- sx8brodie # Fichier de sortie d'erreur du travail
31#-Q- sx8brodie # @ error      =  $(job_name).$(jobid)
32#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
33#-Q- sx8brodie # @ notification = error
34#-Q- sx8brodie # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ;  $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
35#-Q- sx8brodie # @ queue
36#-Q- aix6 #!/bin/ksh
37#-Q- aix6 #######################
38#-Q- aix6 ##   VARGAS   IDRIS  ##
39#-Q- aix6 #######################
40#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
41#-Q- aix6 # @ wall_clock_limit = 20:00:00
42#-Q- aix6 # Nom du travail LoadLeveler
43#-Q- aix6 # @ job_name   = MONITORING
44#-Q- aix6 # Fichier de sortie standard du travail       
45#-Q- aix6 # @ output     = $(job_name).$(jobid)
46#-Q- aix6 # Fichier de sortie d'erreur du travail
47#-Q- aix6 # @ error      =  $(job_name).$(jobid)
48#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
49#-Q- aix6 # @ notification = error
50#-Q- aix6 # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
51#-Q- aix6 # @ queue
52#-Q- ulam #!/bin/ksh
53#-Q- ulam #######################
54#-Q- ulam ##   ULAM   IDRIS  ##
55#-Q- ulam #######################
56#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
57#-Q- ulam # @ wall_clock_limit = 20:00:00
58#-Q- ulam # Nom du travail LoadLeveler
59#-Q- ulam # @ job_name   = MONITORING
60#-Q- ulam # Fichier de sortie standard du travail       
61#-Q- ulam # @ output     = $(job_name).$(jobid)
62#-Q- ulam # Fichier de sortie d'erreur du travail
63#-Q- ulam # @ error      =  $(job_name).$(jobid)
64#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
65#-Q- ulam # @ notification = error
66#-Q- ulam # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
67#-Q- ulam # @ queue
68#-Q- sx8mercure #!/bin/ksh
69#-Q- sx8mercure ######################
70#-Q- sx8mercure ## SX8MERCURE   CEA ##
71#-Q- sx8mercure ######################
72#-Q- sx8mercure #PBS -N  MONITORING          # Nom du job
73#-Q- sx8mercure #PBS -j o                    # regroupement des stdout et stderr
74#-Q- sx8mercure #PBS -S /usr/bin/ksh         # shell de soumission
75#-Q- sx8mercure #PBS -l memsz_job=1gb        # Limite memoire a 1 Go
76#-Q- sx8mercure #PBS -l cputim_job=1:00:00   # Limite temps a 2 heures
77#-Q- sx8mercure #PBS -q scalaire
78#-Q- sx9mercure #!/bin/ksh
79#-Q- sx9mercure #########################
80#-Q- sx9mercure ## CESIUM FOR SX9  CEA ##
81#-Q- sx9mercure #########################
82#-Q- sx9mercure #MSUB -r MONITORING     # Nom du job               
83#-Q- sx9mercure #MSUB -N 1              # Reservation du noeud
84#-Q- sx9mercure #MSUB -n 1              # Reservation du processus
85#-Q- sx9mercure #MSUB -T 86400          # Limite de temps elapsed du job
86#-Q- sx9mercure #MSUB -E "-j o"
87#-Q- sx9mercure #MSUB -E "-S /bin/ksh"
88#-Q- titane #!/bin/ksh
89#-Q- titane ######################
90#-Q- titane ## TITANE   CEA ##
91#-Q- titane ######################
92#-Q- titane #MSUB -r MONITORING     # Nom du job               
93#-Q- titane #MSUB -N 1              # Reservation du noeud
94#-Q- titane #MSUB -n 1              # Reservation du processus
95#-Q- titane #MSUB -T 86400          # Limite de temps elapsed du job
96#-Q- titane #MSUB -E "-j o"
97#-Q- titane #MSUB -E "-S /bin/ksh"
98#-Q- titane ##MSUB -e nco.out        # Sortie standard
99#-Q- titane ##MSUB -o nco.out        # Sortie standard
100#-Q- lxiv8 ######################
101#-Q- lxiv8 ## OBELIX      LSCE ##
102#-Q- lxiv8 ######################
103#-Q- lxiv8 #PBS -N MONITORING
104#-Q- lxiv8 #PBS -m a
105#-Q- lxiv8 #PBS -j oe
106#-Q- lxiv8 #PBS -q medium
107#-Q- lxiv8 #PBS -o MONITORING.$$
108#-Q- lxiv8 #PBS -S /bin/ksh
109#-Q- default #!/bin/ksh
110#-Q- default ##################
111#-Q- default ## DEFAULT HOST ##
112#-Q- default ##################
113
114#**************************************************************
115# Author: Sebastien Denvil
116# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
117# $Revision::                                          $ Revision of last commit
118# $Author::                                            $ Author of last commit
119# $Date::                                              $ Date of last commit
120# IPSL (2006)
121#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
122#
123#**************************************************************
124
125#set -eu
126#set -vx
127
128date
129
130#-Q- sx8brodie export OMP_NUM_THREADS=1
131#-Q- aix6 export OMP_NUM_THREADS=1
132#-Q- ulam export OMP_NUM_THREADS=1
133
134########################################################################
135
136#D- Flag to determine if this job in a standalone mode
137#D- Default : value from AA_job if any
138StandAlone=${StandAlone:=true}
139
140#D- Low level debug : to bypass lib test checks and stack construction
141#D- Default : value from AA_job if any
142libIGCM=${libIGCM:=::modipsl::/libIGCM}
143# WARNING for StandAlone used : To run this script on some machine,
144# you must check MirrorlibIGCM variable in sys library.
145# If this variable is true, you must use libIGCM_POST path instead
146# of your running libIGCM directory.
147
148#D- Increased verbosity (1, 2, 3)
149#D- Default : value from AA_job if any
150Verbosity=${Verbosity:=3}
151
152#D- Low level debug : to bypass lib test checks and stack construction
153#D- Default : value from AA_job if any
154DEBUG_debug=${DEBUG_debug:=false}
155
156#D- TEMPORARY Flag to determine atmospheric resolution
157#D- Default : value from atmospheric driver if any
158RESOL_ATM=${RESOL_ATM:=ALL}
159
160#D- Flag to determine surface resolution
161#D- Default : value from surface driver if any
162RESOL_SRF=${RESOL_SRF:=ALL}
163
164#D- Flag to determine surface resolution
165#D- Default : value from surface driver if any
166RESOL_SBG=${RESOL_SBG:=ALL}
167
168#D- TEMPORARY Flag to determine ocean resolution
169#D- Default : value from ocean driver if any
170RESOL_OCE=${RESOL_OCE:=ORCA2}
171
172#D- TEMPORARY Flag to determine ice resolution
173#D- Default : value from ice driver if any
174RESOL_ICE=${RESOL_ICE:=ORCA2}
175
176#D- TEMPORARY Flag to determine marine biogeochemistry resolution
177#D- Default : value from ice driver if any
178RESOL_MBG=${RESOL_MBG:=ORCA2}
179
180######################################################################
181
182. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
183      ( ${DEBUG_debug} ) && IGCM_debug_Check
184. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
185     ( ${DEBUG_debug} ) && IGCM_card_Check
186. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
187     ( ${DEBUG_debug} ) && IGCM_date_Check
188#-------
189. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
190
191######################################################################
192
193#set -vx
194
195#===========================================
196RUN_DIR=${RUN_DIR_PATH}
197IGCM_sys_MkdirWork ${RUN_DIR}
198IGCM_sys_Cd ${RUN_DIR}
199
200if [ ${StandAlone} = true ] ; then
201    CARD_DIR=${SUBMIT_DIR}
202else
203    CARD_DIR=${RUN_DIR}/$( basename ${SUBMIT_DIR} )
204    IGCM_sys_Get_Master ${SUBMIT_DIR} ${RUN_DIR}
205fi
206
207#
208# First of all
209#
210IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card UserChoices
211typeset option
212for option in ${config_UserChoices[*]} ; do
213    IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
214done
215IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card ListOfComponents
216
217#==================================
218#R_SAVE : Job output directory
219if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
220    FreeName=$( echo ${config_UserChoices_JobName} | sed 's/.*_//' )
221    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
222    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
223else
224    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
225    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
226fi
227#
228IGCM_sys_TestDirArchive ${R_SAVE}/MONITORING
229if [ $? = 0 ] ; then
230    IGCM_debug_Print 1 "Get MONITORING directory from archive"
231    IGCM_sys_Get_Dir ${R_SAVE}/MONITORING ${RUN_DIR}
232else
233    IGCM_debug_Print 1 "MONITORING first pass. Nothing has been done before"
234fi
235# --------------------------------------------
236# Insert your commands between III...III lines
237# and precise produced directories to save
238# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
239
240for comp in ${config_ListOfComponents[*]} ; do
241    IGCM_debug_Print 1 "################## Component: ${comp} ######################"
242    liste_file_monitoring=""
243    IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
244    eval compname=\${config_ListOfComponents_${comp}[0]}                > /dev/null 2>&1
245
246    PATH_monitoring_file=""
247    eval monitoring_file=monitoring01_${compname}_\${RESOL_${comp}}.cfg > /dev/null 2>&1
248    if [[ -d ${CARD_DIR}/POST && -f ${CARD_DIR}/POST/monitoring01_${compname}.cfg ]] ; then
249        PATH_monitoring_file=${CARD_DIR}/POST/monitoring01_${compname}.cfg
250    elif [ -f ${FER_ATLAS}/${monitoring_file} ] ; then
251        PATH_monitoring_file=${FER_ATLAS}/${monitoring_file}
252    else
253        IGCM_debug_Print 1 "No monitoring file found for this component. Was expecting ${monitoring_file}"
254        IGCM_debug_Print 1 "Step to next component"
255        continue
256    fi
257    #
258    if [ X${PATH_monitoring_file} != X"" ] ; then
259        IGCM_debug_Print 1 "Monitoring file used : ${PATH_monitoring_file}"
260        IGCM_debug_Print 1 "Determine which files we need."
261        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
262        liste_file_monitoring=$( . monitoring01 -q -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO )
263    fi
264    #
265    if [ ! "X${liste_file_monitoring}" = X ] ; then
266        IGCM_sys_Get /l liste_file_monitoring ${RUN_DIR}
267        IGCM_debug_Print 1 "monitoring01 -c ${CARD_DIR} -p ${comp} --time -t \"${config_UserChoices_JobName} monitoring\" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} ."
268        IGCM_debug_Print 1 "monitoring01 starts ................................................."
269        monitoring01 -c ${CARD_DIR} -p ${comp} --time -t "${config_UserChoices_JobName} monitoring" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} .
270    else
271        IGCM_debug_Print 1 "No time series detected by this command :"
272        IGCM_debug_Print 1 "monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO"
273        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
274    fi
275    #
276done
277
278# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
279
280# Save files
281IGCM_sys_Put_Dir MONITORING ${R_SAVE}
282
283# Dods copy
284IGCM_sys_Put_Dods MONITORING
285
286# Clean RUN_DIR_PATH (necessary for cesium)
287IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
Note: See TracBrowser for help on using the repository browser.