source: trunk/libIGCM/AA_monitoring @ 614

Last change on this file since 614 was 591, checked in by sdipsl, 12 years ago
  • adapt TSC and create_ts to work with packed files
  • adapt dods related functions to work with work/store
  • bugfixes
  • replace tabulation by 2 space in TSC (svn diff -x -w)
    • sorry for the inconvenience but it will really help on the long run.
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 12.8 KB
RevLine 
[314]1#-Q- cesium #!/bin/ksh
2#-Q- cesium ######################
3#-Q- cesium ## CESIUM   CEA ##
4#-Q- cesium ######################
5#-Q- cesium #MSUB -r MONITORING     # Nom du job               
6#-Q- cesium #MSUB -N 1              # Reservation du noeud
7#-Q- cesium #MSUB -n 1              # Reservation du processus
8#-Q- cesium #MSUB -T 86400          # Limite de temps elapsed du job
9#-Q- cesium #MSUB -E "-j o"
10#-Q- cesium #MSUB -E "-S /bin/ksh"
[2]11#-Q- platine #!/usr/bin/ksh
12#-Q- platine ##################
13#-Q- platine ## PLATINE   CEA ##
14#-Q- platine ##################
15#-Q- platine #BSUB -J MONITORING             # Nom du job
16#-Q- platine #BSUB -N                        # message a la fin du job
17#-Q- platine #BSUB -n 1                      # reservation des processeurs pour le job
18#-Q- platine #BSUB -W 1:00                   # Limite temps
[18]19#-Q- platine #BSUB -q post              # Passage en queue post
[426]20#-Q- platine #BSUB -L /bin/ksh
[59]21#-Q- sx8brodie #!/bin/ksh
[2]22#-Q- sx8brodie #######################
23#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
24#-Q- sx8brodie #######################
[58]25#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
[504]26#-Q- sx8brodie # @ wall_clock_limit = 1:00:00
[58]27#-Q- sx8brodie # Nom du travail LoadLeveler
[60]28#-Q- sx8brodie # @ job_name   = MONITORING
[58]29#-Q- sx8brodie # Fichier de sortie standard du travail       
30#-Q- sx8brodie # @ output     = $(job_name).$(jobid)
31#-Q- sx8brodie # Fichier de sortie d'erreur du travail
32#-Q- sx8brodie # @ error      =  $(job_name).$(jobid)
33#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
34#-Q- sx8brodie # @ notification = error
[178]35#-Q- sx8brodie # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ;  $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
[58]36#-Q- sx8brodie # @ queue
[68]37#-Q- aix6 #!/bin/ksh
38#-Q- aix6 #######################
39#-Q- aix6 ##   VARGAS   IDRIS  ##
40#-Q- aix6 #######################
41#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
[504]42#-Q- aix6 # @ wall_clock_limit = 1:00:00
[68]43#-Q- aix6 # Nom du travail LoadLeveler
44#-Q- aix6 # @ job_name   = MONITORING
45#-Q- aix6 # Fichier de sortie standard du travail       
46#-Q- aix6 # @ output     = $(job_name).$(jobid)
47#-Q- aix6 # Fichier de sortie d'erreur du travail
48#-Q- aix6 # @ error      =  $(job_name).$(jobid)
49#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
50#-Q- aix6 # @ notification = error
[178]51#-Q- aix6 # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
[68]52#-Q- aix6 # @ queue
[386]53#-Q- ulam #!/bin/ksh
54#-Q- ulam #######################
55#-Q- ulam ##   ULAM   IDRIS  ##
56#-Q- ulam #######################
57#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
[500]58#-Q- ulam # @ wall_clock_limit = 1:00:00
[386]59#-Q- ulam # Nom du travail LoadLeveler
60#-Q- ulam # @ job_name   = MONITORING
61#-Q- ulam # Fichier de sortie standard du travail       
62#-Q- ulam # @ output     = $(job_name).$(jobid)
63#-Q- ulam # Fichier de sortie d'erreur du travail
64#-Q- ulam # @ error      =  $(job_name).$(jobid)
65#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
66#-Q- ulam # @ notification = error
67#-Q- ulam # @ environment  = $POST_DIR ; $SUBMIT_DIR ; $libIGCM ; $libIGCM_SX ; $R_INIT ; $R_BC ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $R_SAVE ; $config_UserChoices_JobName ; $config_UserChoices_TagName ; $YEARS ; $MASTER
68#-Q- ulam # @ queue
[2]69#-Q- sx8mercure #!/bin/ksh
70#-Q- sx8mercure ######################
71#-Q- sx8mercure ## SX8MERCURE   CEA ##
72#-Q- sx8mercure ######################
73#-Q- sx8mercure #PBS -N  MONITORING          # Nom du job
74#-Q- sx8mercure #PBS -j o                    # regroupement des stdout et stderr
75#-Q- sx8mercure #PBS -S /usr/bin/ksh         # shell de soumission
76#-Q- sx8mercure #PBS -l memsz_job=1gb        # Limite memoire a 1 Go
77#-Q- sx8mercure #PBS -l cputim_job=1:00:00   # Limite temps a 2 heures
[299]78#-Q- sx8mercure #PBS -q scalaire
[172]79#-Q- sx9mercure #!/bin/ksh
[316]80#-Q- sx9mercure #########################
81#-Q- sx9mercure ## CESIUM FOR SX9  CEA ##
82#-Q- sx9mercure #########################
83#-Q- sx9mercure #MSUB -r MONITORING     # Nom du job               
84#-Q- sx9mercure #MSUB -N 1              # Reservation du noeud
85#-Q- sx9mercure #MSUB -n 1              # Reservation du processus
86#-Q- sx9mercure #MSUB -T 86400          # Limite de temps elapsed du job
87#-Q- sx9mercure #MSUB -E "-j o"
88#-Q- sx9mercure #MSUB -E "-S /bin/ksh"
[222]89#-Q- titane #!/bin/ksh
90#-Q- titane ######################
91#-Q- titane ## TITANE   CEA ##
92#-Q- titane ######################
93#-Q- titane #MSUB -r MONITORING     # Nom du job               
[544]94#-Q- titane #MSUB -eo
[222]95#-Q- titane #MSUB -n 1              # Reservation du processus
96#-Q- titane #MSUB -T 86400          # Limite de temps elapsed du job
[544]97#-Q- titane #MSUB -q mono
[281]98#-Q- lxiv8 ######################
99#-Q- lxiv8 ## OBELIX      LSCE ##
100#-Q- lxiv8 ######################
101#-Q- lxiv8 #PBS -N MONITORING
102#-Q- lxiv8 #PBS -m a
103#-Q- lxiv8 #PBS -j oe
104#-Q- lxiv8 #PBS -q medium
105#-Q- lxiv8 #PBS -o MONITORING.$$
106#-Q- lxiv8 #PBS -S /bin/ksh
[2]107#-Q- default #!/bin/ksh
108#-Q- default ##################
109#-Q- default ## DEFAULT HOST ##
110#-Q- default ##################
111
[373]112#**************************************************************
113# Author: Sebastien Denvil
114# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
115# $Revision::                                          $ Revision of last commit
116# $Author::                                            $ Author of last commit
117# $Date::                                              $ Date of last commit
[2]118# IPSL (2006)
119#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
[373]120#
121#**************************************************************
[2]122
[108]123#set -eu
124#set -vx
125
[2]126date
127
128#-Q- sx8brodie export OMP_NUM_THREADS=1
[108]129#-Q- aix6 export OMP_NUM_THREADS=1
[386]130#-Q- ulam export OMP_NUM_THREADS=1
[2]131
[280]132########################################################################
133
134#D- Flag to determine if this job in a standalone mode
135#D- Default : value from AA_job if any
136StandAlone=${StandAlone:=true}
137
[554]138#D- Path to libIGCM
[280]139#D- Default : value from AA_job if any
[554]140# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
141# WARNING you must check MirrorlibIGCM variable in sys library.
142# WARNING If this variable is true, you must use libIGCM_POST path instead
143# WARNING of your running libIGCM directory.
[280]144libIGCM=${libIGCM:=::modipsl::/libIGCM}
145
[334]146#D- Increased verbosity (1, 2, 3)
147#D- Default : value from AA_job if any
148Verbosity=${Verbosity:=3}
149
150#D- Low level debug : to bypass lib test checks and stack construction
151#D- Default : value from AA_job if any
152DEBUG_debug=${DEBUG_debug:=false}
153
154#D- TEMPORARY Flag to determine atmospheric resolution
155#D- Default : value from atmospheric driver if any
156RESOL_ATM=${RESOL_ATM:=ALL}
157
158#D- Flag to determine surface resolution
159#D- Default : value from surface driver if any
160RESOL_SRF=${RESOL_SRF:=ALL}
161
162#D- Flag to determine surface resolution
163#D- Default : value from surface driver if any
164RESOL_SBG=${RESOL_SBG:=ALL}
165
166#D- TEMPORARY Flag to determine ocean resolution
167#D- Default : value from ocean driver if any
168RESOL_OCE=${RESOL_OCE:=ORCA2}
169
170#D- TEMPORARY Flag to determine ice resolution
171#D- Default : value from ice driver if any
172RESOL_ICE=${RESOL_ICE:=ORCA2}
173
174#D- TEMPORARY Flag to determine marine biogeochemistry resolution
175#D- Default : value from ice driver if any
176RESOL_MBG=${RESOL_MBG:=ORCA2}
177
[2]178######################################################################
179
[108]180. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
181      ( ${DEBUG_debug} ) && IGCM_debug_Check
182. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
183     ( ${DEBUG_debug} ) && IGCM_card_Check
184. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
185     ( ${DEBUG_debug} ) && IGCM_date_Check
186#-------
187. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
[2]188
189######################################################################
190
191#set -vx
192
193#===========================================
[36]194RUN_DIR=${RUN_DIR_PATH}
[34]195IGCM_sys_MkdirWork ${RUN_DIR}
196IGCM_sys_Cd ${RUN_DIR}
197
[436]198# ------------------------------------------------------------------
199# Test if all was right before proceeding further
200# ------------------------------------------------------------------
201IGCM_debug_Verif_Exit_Post
202
[2]203if [ ${StandAlone} = true ] ; then
204    CARD_DIR=${SUBMIT_DIR}
205else
[36]206    CARD_DIR=${RUN_DIR}/$( basename ${SUBMIT_DIR} )
207    IGCM_sys_Get_Master ${SUBMIT_DIR} ${RUN_DIR}
[2]208fi
209
210#
211# First of all
212#
213IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card UserChoices
214typeset option
215for option in ${config_UserChoices[*]} ; do
216    IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
217done
218IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card ListOfComponents
219
[263]220#==================================
221#R_SAVE : Job output directory
[546]222#R_BUFR : Job output buffered directory
[263]223if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
224    FreeName=$( echo ${config_UserChoices_JobName} | sed 's/.*_//' )
225    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
[587]226    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
[546]227    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
[263]228    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
229else
230    R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
[587]231    R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
[546]232    R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
[263]233    R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
234fi
[591]235
236IGCM_sys_MkdirArchive ${R_FIGR}
[102]237#
[591]238IGCM_sys_TestDirArchive ${R_FIGR}/MONITORING
[102]239if [ $? = 0 ] ; then
240    IGCM_debug_Print 1 "Get MONITORING directory from archive"
[591]241    IGCM_sys_Get_Dir ${R_FIGR}/MONITORING ${RUN_DIR}
[102]242else
243    IGCM_debug_Print 1 "MONITORING first pass. Nothing has been done before"
244fi
[2]245# --------------------------------------------
246# Insert your commands between III...III lines
247# and precise produced directories to save
248# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
249
250for comp in ${config_ListOfComponents[*]} ; do
[253]251    IGCM_debug_Print 1 "################## Component: ${comp} ######################"
[2]252    liste_file_monitoring=""
253    IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
254    eval compname=\${config_ListOfComponents_${comp}[0]}                > /dev/null 2>&1
[181]255
256    PATH_monitoring_file=""
[2]257    eval monitoring_file=monitoring01_${compname}_\${RESOL_${comp}}.cfg > /dev/null 2>&1
[251]258    if [[ -d ${CARD_DIR}/POST && -f ${CARD_DIR}/POST/monitoring01_${compname}.cfg ]] ; then
[252]259        PATH_monitoring_file=${CARD_DIR}/POST/monitoring01_${compname}.cfg
[181]260    elif [ -f ${FER_ATLAS}/${monitoring_file} ] ; then
261        PATH_monitoring_file=${FER_ATLAS}/${monitoring_file}
262    else
[251]263        IGCM_debug_Print 1 "No monitoring file found for this component. Was expecting ${monitoring_file}"
264        IGCM_debug_Print 1 "Step to next component"
265        continue
[181]266    fi
[2]267    #
[181]268    if [ X${PATH_monitoring_file} != X"" ] ; then
[250]269        IGCM_debug_Print 1 "Monitoring file used : ${PATH_monitoring_file}"
[134]270        IGCM_debug_Print 1 "Determine which files we need."
[181]271        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
272        liste_file_monitoring=$( . monitoring01 -q -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO )
[102]273    fi
[2]274    #
275    if [ ! "X${liste_file_monitoring}" = X ] ; then
[35]276        IGCM_sys_Get /l liste_file_monitoring ${RUN_DIR}
[250]277        IGCM_debug_Print 1 "monitoring01 -c ${CARD_DIR} -p ${comp} --time -t \"${config_UserChoices_JobName} monitoring\" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} ."
278        IGCM_debug_Print 1 "monitoring01 starts ................................................."
[181]279        monitoring01 -c ${CARD_DIR} -p ${comp} --time -t "${config_UserChoices_JobName} monitoring" -o ${RUN_DIR}/MONITORING ${PATH_monitoring_file} .
[102]280    else
281        IGCM_debug_Print 1 "No time series detected by this command :"
[181]282        IGCM_debug_Print 1 "monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO"
283        . monitoring01 -l2 --listcommand 'IGCM_sys_RshArchive ls' ${PATH_monitoring_file} ${R_SAVE}/${comp}/Analyse/TS_MO
[2]284    fi
285    #
286done
287
288# IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
289
290# Save files
[587]291IGCM_sys_Put_Dir MONITORING ${R_FIGR}
[2]292
293# Dods copy
294IGCM_sys_Put_Dods MONITORING
[203]295
[590]296# Clean RUN_DIR_PATH (necessary for cesium and titane only)
[203]297IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
Note: See TracBrowser for help on using the repository browser.