source: trunk/libIGCM/AA_create_ts

Last change on this file was 1625, checked in by aclsce, 3 months ago

Fixed bug in case of several output files to be treated to generate Time Series. This bug was introduced with changes related to "pack frequency per file" functionality.
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 34.2 KB
Line 
1#-Q- curie ######################
2#-Q- curie ## CURIE   TGCC/CEA ##
3#-Q- curie ######################
4#-Q- curie #MSUB -r TS             # Nom du job
5#-Q- curie #MSUB -o TS.out_%I
6#-Q- curie #MSUB -e TS.out_%I
7#-Q- curie #MSUB -n 1              # Reservation du processus
8#-Q- curie #MSUB -T 80000          # Limite de temps elapsed du job
9#-Q- curie #MSUB -q ::default_node::
10#-Q- curie #MSUB -c ::default_core::
11#-Q- curie #MSUB -Q normal
12#-Q- curie #MSUB -A ::default_project::
13#-Q- curie set +x
14#-Q- irene ######################
15#-Q- irene ## IRENE   TGCC/CEA ##
16#-Q- irene ######################
17#-Q- irene #MSUB -r TS            # Job name
18#-Q- irene #MSUB -o TS.out_%I
19#-Q- irene #MSUB -e TS.out_%I
20#-Q- irene #MSUB -n 1             # Number of cores
21#-Q- irene #MSUB -T 80000         # Maximum elapsed time
22#-Q- irene #MSUB -q ::default_node::
23#-Q- irene #MSUB -c ::default_core::
24#-Q- irene #MSUB -Q normal
25#-Q- irene #MSUB -A ::default_post_project::
26#-Q- irene #MSUB -m store,work,scratch
27#-Q- irene set +x
28#-Q- irene-amd ######################
29#-Q- irene-amd ## IRENE   TGCC/CEA ##
30#-Q- irene-amd ######################
31#-Q- irene-amd #MSUB -r TS            # Job name
32#-Q- irene-amd #MSUB -o TS.out_%I
33#-Q- irene-amd #MSUB -e TS.out_%I
34#-Q- irene-amd #MSUB -n 1             # Number of cores
35#-Q- irene-amd #MSUB -T 80000         # Maximum elapsed time
36#-Q- irene-amd #MSUB -q ::default_node::
37#-Q- irene-amd #MSUB -c ::default_core::
38#-Q- irene-amd #MSUB -Q normal
39#-Q- irene-amd #MSUB -A ::default_post_project::
40#-Q- irene-amd #MSUB -m store,work,scratch
41#-Q- irene-amd set +x
42#-Q- jeanzay #!/bin/ksh
43#-Q- jeanzay ######################
44#-Q- jeanzay ## JEANZAY    IDRIS ##
45#-Q- jeanzay ######################
46#-Q- jeanzay #SBATCH --job-name=TS         # Job Name
47#-Q- jeanzay #SBATCH --output=TS.out_%I    # standard output
48#-Q- jeanzay #SBATCH --error=TS.out_%I     # error output
49#-Q- jeanzay #SBATCH -N  1                        # Number of core
50#-Q- jeanzay #SBATCH --partition=prepost          # Post-processing partition
51#-Q- jeanzay #SBATCH --time=10:00:00       # Wall clock limit (seconds)
52#-Q- jeanzay #SBATCH --account ::default_project::@cpu
53#-Q- jeanzay set +x
54#-Q- ada #!/bin/ksh
55#-Q- ada #######################
56#-Q- ada ## ADA         IDRIS ##
57#-Q- ada #######################
58#-Q- ada # @ job_type = mpich
59#-Q- ada # @ requirements = (Feature == "prepost")
60#-Q- ada # Temps Elapsed max. d'une requete hh:mm:ss
61#-Q- ada # @ wall_clock_limit = 20:00:00
62#-Q- ada # Nom du travail LoadLeveler
63#-Q- ada # @ job_name   = TS
64#-Q- ada # Fichier de sortie standard du travail
65#-Q- ada # @ output     = $(job_name).$(jobid)
66#-Q- ada # Fichier de sortie d'erreur du travail
67#-Q- ada # @ error      =  $(job_name).$(jobid)
68#-Q- ada # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
69#-Q- ada # @ notification = error
70#-Q- ada # @ environment  = $DEBUG_debug ; $BigBrother ; $postProcessingStopLevel ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $POST_DIR ; $Script_Post_Output ; $MASTER ; $RebuildFrequency ; $DateBegin ; $PeriodDateEnd ; $StandAlone ; $CompletedFlag ; $TsTask ; $CompToRead ; $FlagToRead ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; wall_clock_limit=$(wall_clock_limit)
71#-Q- ada # @ queue
72#-Q- obelix ######################
73#-Q- obelix ## OBELIX      LSCE ##
74#-Q- obelix ######################
75#-Q- obelix #PBS -N TS
76#-Q- obelix #PBS -m a
77#-Q- obelix #PBS -j oe
78#-Q- obelix #PBS -q medium
79#-Q- obelix #PBS -o TS.$$
80#-Q- obelix #PBS -S /bin/ksh
81#-Q- mesoipsl #!/bin/ksh
82#-Q- mesoipsl ######################
83#-Q- mesoipsl ## MESO ESPRI IPSL  ##
84#-Q- mesoipsl ######################
85#-Q- mesoipsl #SBATCH --job-name=TS         # Job Name
86#-Q- mesoipsl #SBATCH --output=TS.out_%I    # standard output
87#-Q- mesoipsl #SBATCH --error=TS.out_%I     # error output
88#-Q- mesoipsl #SBATCH -N  1                        # Number of core
89#-Q- mesoipsl #SBATCH --time=10:00:00       # Wall clock limit (seconds)
90#-Q- ifort_CICLAD ######################
91#-Q- ifort_CICLAD ##   CICLAD    IPSL ##
92#-Q- ifort_CICLAD ######################
93#-Q- ifort_CICLAD #PBS -N TS
94#-Q- ifort_CICLAD #PBS -m a
95#-Q- ifort_CICLAD #PBS -j oe
96#-Q- ifort_CICLAD ###PBS -q h12       # Queue for 12 hours at ciclad only
97#-Q- ifort_CICLAD #PBS -S /bin/ksh
98#-Q- default #!/bin/ksh
99#-Q- default ##################
100#-Q- default ## DEFAULT HOST ##
101#-Q- default ##################
102
103#**************************************************************
104# Author: Sebastien Denvil
105# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
106# $Revision::                                          $ Revision of last commit
107# $Author::                                            $ Author of last commit
108# $Date::                                              $ Date of last commit
109# IPSL (2006)
110#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
111#
112#**************************************************************
113
114#set -eu
115#set -vx
116
117date
118
119#-Q- ada export OMP_NUM_THREADS=1
120
121#D- Task type DO NOT CHANGE (computing, post-processing or checking)
122TaskType=post-processing
123
124########################################################################
125
126#D- Flag to determine if this job in a standalone mode
127#D- Default : value from AA_job if any
128StandAlone=${StandAlone:=true}
129
130#D- Path to libIGCM
131#D- Default : value from AA_job if any
132libIGCM=${libIGCM:=::modipsl::/libIGCM}
133
134#D- Flag to determine allready produced time series. Empty if you start from the beginning
135#D- Default : Value from AA_job if any. Usefull in StandAlone case if you want to continue a time series
136CompletedFlag=${CompletedFlag:=}
137
138#D- Flag to determine task type this script will perform.
139#D- Possible Value : 2D, 3D, Chunck2D, Chunck3D
140#D- Default : Value from AA_job if any. Usefull in StandAlone case.
141TsTask=${TsTask:=2D}
142if [ X${TsTask} = Xempty ] ; then
143  TsTask=""
144fi
145
146#D- Flag to determine if rebuild process has been performed asynchronously
147#D- Possible value true or false.
148#D- If true raw files has allready been patched by rebuild job
149#D- If false create_ts will patch the raw files
150#D- Default : Value from AA_job if any. Usefull in StandAlone case.
151RebuildFrequency=${RebuildFrequency:=true}
152
153#D- Flag to determine job's output directory
154#D- Default : value from libIGCM_post.ksh if any
155POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
156
157#D- Increased verbosity (1, 2, 3)
158#D- Default : value from AA_job if any
159Verbosity=${Verbosity:=3}
160
161#D- Low level debug : to bypass lib test checks and stack construction
162#D- Default : value from AA_job if any
163DEBUG_debug=${DEBUG_debug:=false}
164
165#D- TEMPORARY Flag to determine atmospheric resolution
166#D- Default : value from atmospheric driver if any
167RESOL_ATM=${RESOL_ATM:=ALL}
168
169#D- Flag to determine surface resolution
170#D- Default : value from surface driver if any
171RESOL_SRF=${RESOL_SRF:=ALL}
172
173#D- Flag to determine surface resolution
174#D- Default : value from surface driver if any
175RESOL_SBG=${RESOL_SBG:=ALL}
176
177#D- TEMPORARY Flag to determine ocean resolution
178#D- Default : value from ocean driver if any
179RESOL_OCE=${RESOL_OCE:=ORCA2}
180
181#D- TEMPORARY Flag to determine ice resolution
182#D- Default : value from ice driver if any
183RESOL_ICE=${RESOL_OCE:=ORCA2}
184
185#D- TEMPORARY Flag to determine marine biogeochemistry resolution
186#D- Default : value from ice driver if any
187RESOL_MBG=${RESOL_OCE:=ORCA2}
188
189########################################################################
190
191. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
192. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
193. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
194#-------
195. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
196. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
197#-------
198IGCM_sys_MkdirWork ${RUN_DIR_PATH}
199IGCM_sys_Cd ${RUN_DIR_PATH}
200#-------
201( ${DEBUG_debug} ) && IGCM_debug_Check
202( ${DEBUG_debug} ) && IGCM_card_Check
203( ${DEBUG_debug} ) && IGCM_date_Check
204
205########################################################################
206
207#set -vx
208
209# Useful cleaning function
210MENAGE ()
211{
212  typeset ExistFlag
213  ExistFlag=$( IGCM_sys_RshArchive "[ -f $1/$3 ] && echo 0 || echo 1" )
214  [ ${ExistFlag} -eq 0 ] && ( IGCM_sys_RshArchive rm -f $1/$2 ;)
215}
216
217########################################################################
218
219# ------------------------------------------------------------------
220# Test if all was right before proceeding further
221# ------------------------------------------------------------------
222IGCM_debug_Verif_Exit
223
224if [ ${StandAlone} = true ] ; then
225  CARD_DIR=${SUBMIT_DIR}
226else
227  CARD_DIR=${RUN_DIR_PATH}
228
229  IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
230
231  IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card    ${RUN_DIR_PATH}
232
233  IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP        ${RUN_DIR_PATH}
234
235  IGCM_sys_Get_Master ${SUBMIT_DIR}/POST        ${RUN_DIR_PATH}
236fi
237
238#==================================
239# First of all
240#
241# Read libIGCM compatibility version in config.card
242# Read UserChoices section
243# Read Ensemble section
244# Read Post section
245# Define all netcdf output directories
246#==================================
247IGCM_config_CommonConfiguration ${CARD_DIR}/config.card
248
249# ------------------------------------------------------------------
250# Activate BigBrother so as to supervise this job
251# ------------------------------------------------------------------
252IGCM_debug_BigBro_Initialize
253
254#==================================
255# Define default value to keep compatibility with previous card: means before changes due to TGCC
256if [ X${PackDefault} = Xtrue ] ; then
257  [ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
258else
259  config_Post_PackFrequency=NONE
260fi
261
262#==================================
263# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
264if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
265  config_UserChoices_PeriodLength=${config_Post_PackFrequency}
266fi
267
268#
269# Determine component to take care of depending on kind of task create_ts will perform
270#
271[ ${CompToRead} ] && set -A config_ListOfComponents ${CompToRead} || IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
272#
273# Determine period for time series. Default : value from AA_job or from launch_TS if any
274#
275[ X${CompletedFlag} = X ] || CompletedFlag=$( IGCM_date_ConvertFormatToGregorian ${CompletedFlag} )
276
277DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
278#
279PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
280#
281# Determine Dimension of the produced time series : 2D or 3D
282if (   [ X${TsTask} = XChunck2D ] || [ X${TsTask} = X2D ] ) ; then
283  Dimension=2D
284elif ( [ X${TsTask} = XChunck3D ] || [ X${TsTask} = X3D ] ) ; then
285  Dimension=3D
286else
287  Dimension=""
288fi
289
290# ------------------------------------------------------------------
291#D- Test if all was right before entering the loop
292# ------------------------------------------------------------------
293IGCM_debug_Verif_Exit
294
295########################################################################
296#      Depending on requested time series fill following variables :
297#
298#                           - LISTE_FILE_${comp}[*]
299#                           - LISTE_VARS_${file}[*]
300#                           - LISTE_AXIS_${file}[*]
301#                           - LISTE_PATCH_${file}[*]
302########################################################################
303
304#
305# For each selected component determine which files need post-processing
306#
307DoJob=false
308for comp in ${config_ListOfComponents[*]} ; do
309  #
310  ActiveComp=false
311  #
312  IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
313  eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
314  #
315  card=${CARD_DIR}/COMP/${compname}.card
316
317  IGCM_card_DefineArrayFromOption ${card} OutputFiles List
318  #
319  ListFilesName=${compname}_OutputFiles_List
320  eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
321  #
322  if [ X${FileName0} != X${NULL_STR} ] ; then
323    #
324    #IGCM_debug_Print 1 "Component      : ${compname}"
325    #
326    # INITIALISATION
327    #
328    typeset i
329    if [ ${FlagToRead} ] ; then
330      ((NbFiles=${FlagToRead}))
331      ((i=${NbFiles}))
332    else
333      eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
334      i=2
335    fi
336    #
337    until [ $i -gt $NbFiles ]; do
338      #
339      eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
340      #
341      if [ X${flag_post} != XNONE ] ; then
342        #
343        # For each comp and each file who needs post-processing
344        # determine which variables need time series.
345        #
346        FILE=$( echo ${flag_post} | awk "-FPost_" '{print $2}' )
347        IGCM_card_DefineArrayFromOption ${card}    Post_${FILE} TimeSeriesVars${Dimension}
348        IGCM_card_DefineVariableFromOption ${card} Post_${FILE} ChunckJob${Dimension}
349        #
350        # If TimeSeriesVars list is empty we skip
351        #
352        if [ $( eval echo \${${compname}_Post_${FILE}_TimeSeriesVars${Dimension}} ) = ${NULL_STR} ] ; then
353          #
354          IGCM_debug_Print 3 "Nothing to do for ${FILE} according to ${card}"
355          IGCM_debug_Print 3 "TimeSeriesVars${Dimension} for ${FILE} is empty"
356          (( i_ = i+1 ))
357          eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
358          eval Testvar=${Testvar_}
359          if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
360              (( i = i+4 ))
361          else
362          (( i=i+3 ))
363          fi         
364          continue
365          #
366        fi
367        #
368        # If files/variables has been switch off by ChunckJob${Dimension}=0 we skip
369        #
370        if [ X$( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} ) = XOFF ] ; then
371          IGCM_debug_Print 3 "Nothing to do for ${FILE} according to ${card}"
372          IGCM_debug_Print 3 "ChunckJob${Dimension}=OFF for ${FILE} so it has been switched off"
373          (( i_ = i+1 ))
374          eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
375          eval Testvar=${Testvar_}
376          if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
377              (( i = i+4 ))
378          else
379          (( i=i+3 ))
380          fi           
381          continue
382        fi
383        #
384          # If we are not a Chunck type task, we skip files/variables handled by chunck task
385          #
386        if ( [ ! X${TsTask} = XChunck${Dimension} ] && [ ! X${TsTask} = X ] && [ ! XChunck${Dimension} = XChunck ] ) ; then
387          if [ ! $( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} ) = NONE ] ; then
388            IGCM_debug_Print 3 "Nothing to do for ${FILE} TimeSeriesVars${Dimension} according to ${card}"
389            IGCM_debug_Print 3 "Let another chunck task do the job $( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} )"
390          (( i_ = i+1 ))
391          eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
392          eval Testvar=${Testvar_}
393          if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
394              (( i = i+4 ))
395          else
396            (( i=i+3 ))
397          fi
398            continue
399          fi
400        fi
401        #
402        # Now on we know we have things to do
403        #
404
405        #
406        #  We need coordinates axis to properly fill produced time series
407        #
408        IGCM_card_DefineArrayFromOption ${card} Post_${FILE} GatherWithInternal
409        #
410        if [ $( eval echo \${${compname}_Post_${FILE}_GatherWithInternal} ) = ${NULL_STR} ] ; then
411          #
412          IGCM_debug_Print 1 "ERROR !! We miss coordinate axis for ${FILE} according to ${card}"
413          IGCM_debug_Print 1 "So no time series will be produced for ${FILE} !!"
414          (( i_ = i+1 ))
415          eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
416          eval Testvar=${Testvar_}
417          if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
418              (( i = i+4 ))
419          else
420          (( i=i+3 ))
421          fi         
422          continue
423          #
424        fi
425        #
426        # We need LIST of variables not allready produced (useful for standalone mode)
427        #
428        FlagDir=$( echo ${FILE} | awk -F "_" '{print $1}' )
429        case ${FlagDir} in
430          *Y)  TS_Dir=TS_YE  ;;
431          *M)  TS_Dir=TS_MO  ;;
432          *D)  TS_Dir=TS_DA  ;;
433          HF)  TS_Dir=TS_HF  ;;
434          3H)  TS_Dir=TS_HF  ;;
435          INS) TS_Dir=TS_INS ;;
436        esac
437        #
438        count=0
439        for var in $( eval echo \${${compname}_Post_${FILE}_TimeSeriesVars${Dimension}[*]} ) ; do
440          #
441          DIRECTORY=${R_SAVE}/${comp}/Analyse/${TS_Dir}
442          TestedFile=${config_UserChoices_JobName}_${DateBegin}_${PeriodDateEnd}_${FlagDir}_${var}.nc
443          #
444          IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
445          if [ ! $? = 0 ] ; then
446            eval set +A LISTE_VARS_${FILE} \${LISTE_VARS_${FILE}[*]} ${var}
447            ActiveComp=true
448            (( count = count + 1 ))
449          fi
450        done
451        #
452        if [ ${count} -gt 0 ] ; then
453          #
454          eval set +A LISTE_FILE_${comp} \${LISTE_FILE_${comp}[*]} ${FILE}
455          eval set +A LISTE_AXIS_${FILE} \${${compname}_Post_${FILE}_GatherWithInternal[*]}
456          (( i_ = i+1 ))
457          eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
458          eval Testvar=${Testvar_}
459          if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
460              eval PeriodPack_FileType=${Testvar}
461          else
462              PeriodPack_FileType=${config_Post_PackFrequency}
463          fi
464          eval set +A LISTE_PACKFREQUENCY_${comp} \${LISTE_PACKFREQUENCY_${comp}[*]} ${PeriodPack_FileType}
465          #
466          if [ ! X${RebuildFrequency} = Xtrue ] ; then
467            IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
468            eval set +A LISTE_PATCHES_${FILE} \${${compname}_Post_${FILE}_Patches[*]}
469            if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) !=  X${NULL_STR} ]; then
470              for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
471                if [ Xload${Patch} != X ] ; then
472                  . ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
473                  eval load${Patch}=loaded
474                fi
475              done
476            fi
477          fi
478        fi
479      fi
480      (( i_ = i+1 ))
481      eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
482      eval Testvar=${Testvar_}
483      if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
484          (( i = i+4 ))
485      else
486      (( i=i+3 ))
487      fi         
488    done
489  fi
490  #
491  if [ ${ActiveComp} = true ] ; then
492    set +A ActiveListOfComponents ${ActiveListOfComponents[*]} ${comp}
493  fi
494done
495
496set -A config_ListOfComponents ${ActiveListOfComponents[*]}
497[ ! -z "${ActiveListOfComponents}" ] && DoJob=true
498
499########################################################################
500#
501#               IS THERE SOME ALLREADY PRODUCED TIME SERIES ?
502#                 IF SO BRING THEM IN THE WORKING DIRECTORY
503#
504# PS : Keep in mind that IGCM_sys_Get here is a weak link :
505#      - especially for IDRiS
506#      - no special protection against "command too long" !
507#      - libIGCM_sys should handle that or modify following code part
508#
509########################################################################
510
511if [ ${StandAlone} != true ] ; then
512  Running_Flag=$( IGCM_sys_RshMaster "IGCM_card_DefineVariableFromOption \
513  ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning ; \
514  echo \${run_PostProcessing_TimeSeriesRunning}" )
515
516  #if [ X${Running_Flag} = Xy ] ; then
517  #  # Time Series Job allready running
518  #  IGCM_debug_Print 1 "Time Series Job allready running exit"
519  #  exit
520  #fi
521  # Now run.card know we are running
522
523
524
525
526  IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "y"
527
528
529
530
531
532  # Get information from last execution
533  #CompletedFlag=$( IGCM_sys_RshMaster \
534  # "IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted ;\
535  #        echo \${run_PostProcessing_TimeSeriesCompleted}" )
536fi
537
538# Is it the first submission or not ?
539
540if ( [ X${CompletedFlag} = X ] || [ X${CompletedFlag} = X$( IGCM_date_AddDaysToGregorianDate ${DateBegin} -1 ) ] ) ; then
541  # First Time Series Submission
542  FIRST_PASS=TRUE
543else
544  # Last Completed Time Series
545  FIRST_PASS=FALSE
546  DATE_FIN_JOB_B=${CompletedFlag}
547  date=${DateBegin}_${DATE_FIN_JOB_B}
548  #
549  for comp in ${config_ListOfComponents[*]} ; do
550    #
551    IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
552    #
553    IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
554    #
555    unset liste_file
556    #
557    file_path=${R_SAVE}/${comp}/Analyse
558    #
559    i=0
560    file=${NULL_STR}
561    #
562    for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
563      #
564      # Determine in which output can we find file
565      # IE : Analyse/TS or Analyse/DA2TS or ...
566      #
567      FlagDir=$( echo ${file} | awk -F "_" '{print $1}' )
568      case ${FlagDir} in
569        *Y)  TS_Dir=TS_YE  ;;
570        *M)  TS_Dir=TS_MO  ;;
571        *D)  TS_Dir=TS_DA  ;;
572        HF)  TS_Dir=TS_HF  ;;
573        3H)  TS_Dir=TS_HF  ;;
574        INS) TS_Dir=TS_INS ;;
575      esac
576      for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
577        #
578        IGCM_sys_TestFileArchive ${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc
579        if [ $? = 0 ] ; then
580          liste_file[${#liste_file[*]}]=${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc
581        else
582          IGCM_debug_Print 1 "WARNING ${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc dont exist"
583        fi
584        #
585      done
586    done
587    if [ X${file} != X${NULL_STR} ] ; then
588      IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
589      IGCM_debug_Verif_Exit
590    fi
591  done
592  #
593  LEVEL_INIT=0
594  #
595fi
596
597########################################################################
598#
599#                          DEFINE LOOP PARAMETERS
600#
601########################################################################
602
603
604# THEN FOR EACH FILE TYPE WE LOOP BETWEEN DATE_COURANTE AND DATE_FIN.
605for comp in ${config_ListOfComponents[*]} ; do
606    #
607    IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
608      #
609    IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
610    #
611    eval R_OUT_${comp}=${R_SAVE}/${comp}
612    #
613    i=0
614    #
615    for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
616        ( [ -n "${LEVEL_INIT}" ] && [ ${LEVEL_INIT} -eq 0 ] ) && NBRE_FILE_TOT=0 || NBRE_FILE_TOT=1
617        # If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
618        if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
619           config_UserChoices_PeriodLength=$( eval echo \${LISTE_PACKFREQUENCY_${comp}[$i]} )
620        fi
621
622        if [ ${FIRST_PASS} = TRUE ] ; then
623            Length=$( IGCM_date_DaysInPreviousPeriod ${DateBegin} ${config_UserChoices_PeriodLength} begin) 
624            DATE_FIN_JOB_B=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( Length - 1 )) )
625        fi
626DATE_COUNT=${DATE_FIN_JOB_B}
627DATE_FIN_JOB_B_LOOP=${DATE_FIN_JOB_B}
628if [ ${DoJob} = true ] ; then
629  while [ ${DATE_COUNT} -lt ${PeriodDateEnd} ] ; do
630    (( NBRE_FILE_TOT = NBRE_FILE_TOT + 1 ))
631    Length=$( IGCM_date_DaysInNextPeriod ${DATE_COUNT} ${config_UserChoices_PeriodLength} )
632    DATE_COUNT=$( IGCM_date_AddDaysToGregorianDate ${DATE_COUNT} ${Length} )
633  done
634else
635  NBRE_FILE_TOT=0
636fi
637
638if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then
639  NBRE_TOUR=1
640elif [ ${NBRE_FILE_TOT}%${UNIX_MAX_LIMIT} -eq 0 ] ; then
641  let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT}
642else
643  let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT}+1
644fi
645
646CURRENT_LOOP=1
647# THIS LOOP GIVES PARAMETERS FOR THE DATE'S LOOP ie : DATE_COURANTE, DATE_FIN.
648# WE HAVE 3 LEVELS
649#    - LEVEL 1 : JUST ONE LOOP TO COMPLETE
650#    - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
651#    - LEVEL 3 : WE ARE IN THE LAST LOOP
652LEVEL=${LEVEL_INIT}
653while [ ${CURRENT_LOOP} -le  ${NBRE_TOUR} ] ; do
654
655  if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then
656    # - LEVEL 1 : JUST ONE LOOP TO COMPLETE
657    NBRE_FILE_LOOP=${NBRE_FILE_TOT}
658
659    if [ ${FIRST_PASS} = TRUE ] ; then
660      DATE_COURANTE=${DATE_FIN_JOB_B_LOOP}
661    else
662      Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN_JOB_B_LOOP} ${config_UserChoices_PeriodLength} )
663      DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
664    fi
665
666    DATE_FIN=${PeriodDateEnd}
667    DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP}
668
669    [ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
670    [ -n "${LEVEL}" ] && FLAG_B="TRUE" || FLAG_B="FALSE"
671
672    LEVEL=1
673    DEPOT="TRUE"
674  elif [ ${CURRENT_LOOP} -ne  ${NBRE_TOUR} ] ; then
675    # - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
676    NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
677
678    if [ ! ${FIRST_PASS} = TRUE ] && [ ${CURRENT_LOOP} -eq 1 ] ; then
679      Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN_JOB_B_LOOP} ${config_UserChoices_PeriodLength} )
680      DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
681    elif [ -z "${DATE_COURANTE}" ] ; then
682      DATE_COURANTE=${DATE_FIN_JOB_B}
683    else
684      Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN} ${config_UserChoices_PeriodLength} )
685      DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
686    fi
687
688    (( TotLength = 0 ))
689    COMPTEUR=1
690    DATE_LOOP=${DATE_COURANTE}
691
692    while [ ${COMPTEUR} -lt ${NBRE_FILE_LOOP} ] ; do
693      #
694      Length=$( IGCM_date_DaysInNextPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} )
695      DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length} )
696      (( TotLength = TotLength + Length ))
697      (( COMPTEUR = COMPTEUR + 1 ))
698      #
699    done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
700    DATE_FIN=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} ${TotLength} )
701
702    [ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
703    [ -n "${LEVEL}" ] && [ ${LEVEL} -eq 0 ] && FLAG_B="TRUE" || [ ${CURRENT_LOOP} -gt 1 ] && FLAG_B="TRUE" || FLAG_B="FALSE"
704    DEPOT="FALSE"
705    LEVEL=2
706  else
707    # - LEVEL 3 : WE ARE IN THE LAST LOOP
708    NBRE_FILE_LOOP=$(( ${NBRE_FILE_TOT} % ( ${UNIX_MAX_LIMIT} * ( ${CURRENT_LOOP} - 1 ) ) ))
709    [ ${NBRE_FILE_LOOP} -eq 0 ] && NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
710
711    Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN} ${config_UserChoices_PeriodLength} )
712    DATE_COURANTE=$(IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
713    DATE_FIN=${PeriodDateEnd}
714    [ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
715
716    LEVEL=3
717    DEPOT="TRUE"
718    FLAG_B="TRUE"
719  fi
720
721  # THEN FOR EACH FILE TYPE WE LOOP BETWEEN DATE_COURANTE AND DATE_FIN.
722      #
723      # Determine in which output we can find file
724      # IE : Output/MO or Output/DA or ...
725      #
726      FlagDir=$( echo ${file} | awk -F "_" '{print $1}' )
727      case ${FlagDir} in
728        *Y)    TS_Dir=TS_YE  ; FreqDir=YE  ;;
729        *M)    TS_Dir=TS_MO  ; FreqDir=MO  ;;
730        *D)    TS_Dir=TS_DA  ; FreqDir=DA  ;;
731        HF)    TS_Dir=TS_HF  ; FreqDir=HF  ;;
732        3H)    TS_Dir=TS_HF  ; FreqDir=HF  ;;
733        INS)   TS_Dir=TS_INS ; FreqDir=INS ;;
734        *)     IGCM_debug_Print 1 "Error in FlagDir=${FlagDir} for file ${file}."
735               IGCM_debug_Exit "Job create_ts" ;;
736      esac
737
738      # Initialize array
739      unset liste_file
740      unset liste_file_tmp
741      eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
742
743      COMPTEUR=1
744      MissingFile=FALSE
745      DATE_LOOP=${DATE_COURANTE}
746
747      while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] ; do
748        #
749        Length1=$( IGCM_date_DaysInPreviousPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} end)
750        DATE_TAB=$(  IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} $(( 1 - Length1 )) )_${DATE_LOOP}
751
752        Length2=$( IGCM_date_DaysInNextPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} )
753        DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length2} )
754
755        IGCM_sys_TestFileArchive ${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
756        if [ $? = 0 ] ; then
757          liste_file[${#liste_file[*]}]=${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
758          liste_file_tmp[${#liste_file_tmp[*]}]=${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
759        else
760          IGCM_debug_Print 1 "WARNING ${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc dont exist"
761          MissingFile=TRUE
762        fi
763        (( COMPTEUR = COMPTEUR + 1 ))
764        #
765      done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
766
767      # We skip this file type if one of them is missing.
768      if [ "${MissingFile}" = "TRUE" ] ; then
769        IGCM_debug_Print 1 "WARNING We skip ${file} type processing"
770                continue 2
771      fi
772
773      DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} -${Length2} )
774
775      # Get selected files
776      IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
777      IGCM_debug_Verif_Exit
778
779      # Apply IGCM_Patch if needed
780      if [ ! X${RebuildFrequency} = Xtrue ] ; then
781        if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) !=  X${NULL_STR} ]; then
782          for file_t in $( eval echo ${liste_file_tmp[*]} ); do
783            for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
784              IGCM_${Patch} ${file_t}
785              IGCM_debug_Verif_Exit
786            done
787          done
788        fi
789      fi
790
791      # WE CAN CONCATENATE FILES AT THIS POINT
792      [ "${FLAG_B}" = "TRUE" ] && DATE_BUILD_B=${DateBegin}_${DATE_FIN_JOB_B_LOOP_PREC}
793      Length=$( IGCM_date_DaysInPreviousPeriod ${DATE_COURANTE} ${config_UserChoices_PeriodLength} end)
794      DATE_BUILD1=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} -$(( Length - 1 )) )
795      DATE_BUILD=${DATE_BUILD1}_${DATE_FIN}
796      DATE_BUILD_END=${DateBegin}_${DATE_FIN}
797
798      liste_coord=" "
799      for axis in $( eval echo \${LISTE_AXIS_${file}[*]} ); do
800        liste_coord=${liste_coord}${axis}","
801      done
802
803      # Time axis must be the last dimension
804      time_axis=${axis}
805
806      # SWITCH BETWEEN MODEL TO FIT PARTICULAR CASE
807      for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
808        #
809        # We skip variables not in the first file of the list.
810        # Seems overkill to test thel all... but will do if needed
811        ncdump -hv ${var} ${liste_file_tmp[0]} > /dev/null 2>&1
812        if [ ! $? = 0 ] ; then
813          IGCM_debug_Print 1 "WARNING We skip ${var}, not present within ${liste_file_tmp[0]}"
814          continue
815        fi
816        #
817        if [ "${FLAG_B}" = "TRUE" ] ; then
818          # WE CONCATENATE WITH EXISTING FILES
819          file1=${config_UserChoices_JobName}_${DATE_BUILD_B}_${FlagDir}_${var}.nc
820          file_out=${config_UserChoices_JobName}_${DATE_BUILD_END}_${FlagDir}_${var}.nc
821          IGCM_sys_ncrcat --hst -v ${liste_coord}${var} ${file1} ${liste_file_tmp[*]} ${file_out}
822          IGCM_debug_Verif_Exit
823          #
824          IGCM_sys_Rm ${file1}
825          if [ ! "${DEPOT}" = "TRUE" ] ; then
826            eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}}/Analyse/${TS_Dir}/${file_out}
827            # "${file1} = file_before ?
828            [ ! "${file1}" = "${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file1} ${file_out}
829          fi
830        else
831          # OR NOT
832          file_out=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
833          IGCM_sys_ncrcat --hst -v ${liste_coord}${var} ${liste_file_tmp[*]} ${file_out}
834          IGCM_debug_Verif_Exit
835        fi
836
837        #
838        if [ "${DEPOT}" = "TRUE" ] ; then
839          # WE PUT FINAL FILE ON FILE SYSTEM
840          [ "${FLAG_B}" = "FALSE" ] && file_out=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
841          file_before=${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc
842          #
843          eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}}/Analyse/${TS_Dir}/${file_out}
844          [ "${FLAG_B}" = "TRUE" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file_before} ${file_out}
845          if [ ! "X${file1}" = "X" ] ; then
846            [ ! "${file1}" = "${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file1} ${file_out}
847          fi
848          #
849          [ -f ${file_out} ] && IGCM_sys_Rm ${file_out}
850          #
851        fi
852        #
853      done                              # for var in ${liste_var} ; do
854      IGCM_sys_Rm ${liste_file_tmp[*]}
855            (( CURRENT_LOOP = CURRENT_LOOP + 1 ))
856      (( i = i + 1 ))
857    done                                # for file in ${liste_file_brut} do
858  done                                  # for comp in ${config_ListOfComponents[*]} ; do
859  # GO AHEAD IN DEEPEST LOOP
860done                                    # while [ ${CURRENT_LOOP} -le  ${NBRE_TOUR} ] do
861# FINISH
862
863# DODS copy
864for comp in ${config_ListOfComponents[*]} ; do
865  for TS_Dir in TS_YE TS_MO TS_DA TS_HF TS_INS ; do
866    IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/${TS_Dir}
867    [ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/${TS_Dir}
868  done
869done
870
871export DEBUG_debug; export BigBrother ; export libIGCM; export SUBMIT_DIR; export POST_DIR; export Script_Output ;
872export StandAlone ; export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ;
873export RESOL_SBG
874listVarEnv="DEBUG_debug,BigBrother,libIGCM,SUBMIT_DIR,POST_DIR,Script_Post_Output,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG"
875export listVarEnv
876
877#D-
878# --------------------------------------------------------------------
879#D- Test if all was right
880# --------------------------------------------------------------------
881IGCM_debug_Verif_Exit
882# --------------------------------------------------------------------
883#D- Submit MONITORING and C-ESM-EP
884# --------------------------------------------------------------------
885if ( [ X${TsTask} = X2D ] || [ X${TsTask} = X3D ] || [ X${TsTask} = X ] ) ; then
886  # From 2D and 3D it is the last one to finish that will submit the monitoring.
887  if ( [ $( ls ${POST_DIR}/create_ts.${PeriodDateEnd}.[23]D.out | wc -l ) = 1 ] || [ ! "X$( grep "IGCM_sys_RmRunDir" ${POST_DIR}/create_ts.${PeriodDateEnd}.[23]D.out 2> /dev/null )" = "X" ] ) ; then
888    Script_Post_Output=monitoring.${PeriodDateEnd}
889    IGCM_sys_QsubPost monitoring
890    if [ $PostCesmep = TS ] ; then
891        ${SUBMIT_DIR}/cesmep_lite/libIGCM_post.sh $DateBegin $PeriodDateEnd
892    fi
893  fi
894fi
895#D-
896# --------------------------------------------------------------------
897#D- Update the run.card
898# --------------------------------------------------------------------
899if [ ${StandAlone} != true ] ; then
900  # Put in run.card end period of time series
901  [ X"${DATE_FIN}" = X ] || IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted "${DATE_FIN}"
902  # We have finish the job !
903  IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "n"
904fi
905
906# Clean RUN_DIR_PATH (necessary for cesium and titane only)
907IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
908
909# ------------------------------------------------------------------
910# Finalize BigBrother to inform that the jobs end
911# ------------------------------------------------------------------
912IGCM_debug_BigBro_Finalize
913
914date
Note: See TracBrowser for help on using the repository browser.