source: tags/libIGCM_v1_1/AA_create_ts @ 1137

Last change on this file since 1137 was 18, checked in by mmaipsl, 16 years ago

Aclsce, MM: Add queue post for platine
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Date Author Revision
File size: 24.1 KB
Line 
1#-Q- platine #!/usr/bin/ksh
2#-Q- platine ##################
3#-Q- platine ## PLATINE   CEA ##
4#-Q- platine ##################
5#-Q- platine #BSUB -J TS                     # Nom du job
6#-Q- platine #BSUB -N                        # message a la fin du job
7#-Q- platine #BSUB -n 1                      # reservation des processeurs pour le job
8#-Q- platine #BSUB -W 1:00                   # Limite temps
9#-Q- platine #BSUB -q post              # Passage en queue post
10#-Q- sx8brodie #######################
11#-Q- sx8brodie ## SX8BRODIE   IDRIS ##
12#-Q- sx8brodie #######################
13#-Q- sx8brodie #QSUB -r TS                   # Nom du job
14#-Q- sx8brodie #QSUB -lT 1:00:00             # limite en temps total
15#-Q- sx8brodie #QSUB -lt 1:00:00             # limite en temps par process
16#-Q- sx8brodie #QSUB -lM 1Gb 
17#-Q- sx8brodie #QSUB -eo                     # rassemble standard error et output
18#-Q- sx8brodie #QSUB -J m
19#-Q- sx8brodie #QSUB -l mpp_p=4
20#-Q- sx8brodie #QSUB -s /bin/ksh             # shell du job
21#-Q- sx8mercure #!/bin/ksh
22#-Q- sx8mercure ######################
23#-Q- sx8mercure ## SX8MERCURE   CEA ##
24#-Q- sx8mercure ######################
25#-Q- sx8mercure #PBS -N TS                   # Nom du job
26#-Q- sx8mercure #PBS -j o                    # regroupement des stdout et stderr
27#-Q- sx8mercure #PBS -S /usr/bin/ksh         # shell de soumission
28#-Q- sx8mercure #PBS -l memsz_job=1gb        # Limite memoire a 1 Go
29#-Q- sx8mercure #PBS -l cputim_job=2:00:00   # Limite temps a 2 heures
30#-Q- sx8mercure #PBS -q scalaire
31#-Q- default #!/bin/ksh
32#-Q- default ##################
33#-Q- default ## DEFAULT HOST ##
34#-Q- default ##################
35
36#set -vx
37
38date
39
40#-Q- sx8brodie export OMP_NUM_THREADS=1
41
42# $Date$
43# $Author$
44# $Revision$
45# IPSL (2006)
46#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
47
48########################################################################
49
50#D- Flag to determine if this job in a standalone mode
51#D- Default : value from AA_job if any
52StandAlone=${StandAlone:=true}
53
54#D- Flag to determine job's output directory
55#D- Default : value from libIGCM_post.ksh if any
56POST_DIR=${POST_DIR:=${PBS_O_WORKDIR}}
57
58#D- Increased verbosity (1, 2, 3)
59#D- Default : value from AA_job if any
60Verbosity=${Verbosity:=3}
61
62#D- Low level debug : to bypass lib test checks and stack construction
63#D- Default : value from AA_job if any
64DEBUG_debug=${DEBUG_debug:=false}
65
66#D- Low level debug : to bypass lib test checks and stack construction
67#D- Default : value from AA_job if any
68libIGCM=${libIGCM:=/work/cont003/p86denv/PARA_SX8_ORCA2xLMD144142/Prelude/modipsl/libIGCM}
69
70#D- TEMPORARY Flag to determine atmospheric resolution
71#D- Default : value from atmospheric driver if any
72RESOL_ATM=${RESOL_ATM:=LMD144142}
73
74#D- TEMPORARY Flag to determine ocean resolution
75#D- Default : value from ocean driver if any
76RESOL_OCE=${RESOL_OCE:=ORCA2}
77
78#D- TEMPORARY Flag to determine ice resolution
79#D- Default : value from ice driver if any
80RESOL_ICE=${RESOL_ICE:=ORCA2}
81
82#D- Flag to determine surface resolution
83#D- Default : value from surface driver if any
84RESOL_SRF=${RESOL_SRF:=LMD144142}
85
86########################################################################
87
88. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh ;
89. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh     #; IGCM_debug_Check
90. ${libIGCM}/libIGCM_card/libIGCM_card.ksh   #; IGCM_card_Check
91. ${libIGCM}/libIGCM_date/libIGCM_date.ksh   #; IGCM_date_Check
92
93########################################################################
94
95#set -vx
96
97# Max number of files we use with a command (ex : ncrcat, ...)
98UNIX_MAX_LIMIT=120
99
100# Useful cleaning function
101MENAGE () 
102{ 
103    [ -f $3 ] & ( rm -f $1/$2 ;) 
104}
105
106########################################################################
107
108#set -vx
109
110if [ ${DEBUG_debug} = true ] ; then
111    IGCM_sys_MkdirWork ${RUN_DIR_PATH}
112    echo "RUN_DIR_PATH ${RUN_DIR_PATH} ok."
113fi
114IGCM_sys_Cd ${RUN_DIR_PATH}
115
116if [ ${StandAlone} = true ] ; then
117    CARD_DIR=${SUBMIT_DIR}
118else
119    CARD_DIR=${RUN_DIR_PATH}/$( basename ${SUBMIT_DIR} )
120    IGCM_sys_Get_Master ${SUBMIT_DIR} ${RUN_DIR_PATH}
121fi
122
123#
124# First of all
125#
126IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card UserChoices
127typeset option
128for option in ${config_UserChoices[*]} ; do
129    IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
130done
131#
132echo
133IGCM_debug_Print 1 "DefineArrayFromOption  : config_UserChoices"
134IGCM_debug_PrintVariables 3 config_UserChoices_JobName
135IGCM_debug_PrintVariables 3 config_UserChoices_LongName
136IGCM_debug_PrintVariables 3 config_UserChoices_TagName
137IGCM_debug_PrintVariables 3 config_UserChoices_CalendarType
138IGCM_debug_PrintVariables 3 config_UserChoices_DateBegin
139IGCM_debug_PrintVariables 3 config_UserChoices_DateEnd
140IGCM_debug_PrintVariables 3 config_UserChoices_PeriodLength
141echo
142
143#==================================
144
145R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
146#
147IGCM_card_DefineArrayFromSection       ${CARD_DIR}/config.card ListOfComponents
148#
149# Determine period for time series. Default : value from AA_job if any
150#
151DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
152#
153PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
154
155########################################################################
156#
157#      Depending on requested time series fill following variables :
158#
159#                           - LISTE_FILE_${comp}[*]
160#                           - LISTE_VARS_${file}[*]
161#                           - LISTE_AXIS_${file}[*]
162#                           - LISTE_PATCH_${file}[*]
163########################################################################
164
165#
166# For each selected component determine which files need post-processing
167#
168DoJob=false
169for comp in ${config_ListOfComponents[*]} ; do
170    #
171    ActiveComp=false
172    #
173    IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
174    eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
175    #
176    card=${CARD_DIR}/COMP/${compname}.card
177    IGCM_card_DefineArrayFromOption ${card} OutputFiles List
178    #
179    ListFilesName=${compname}_OutputFiles_List
180    eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
181    #
182    if [ X${FileName0} != X${NULL_STR} ] ; then
183        #
184        #IGCM_debug_Print 1 "Component      : ${compname}"
185        #
186        # INITIALISATION
187        #
188        eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
189        typeset i=0
190        #
191        until [ $i -eq $NbFiles ]; do
192            #
193            eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
194            eval file_in=${file_in_}
195            let $(( i_ = i+2 ))
196            eval flag_post_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
197            eval flag_post=${flag_post_}
198            #
199            if [ X${flag_post} != XNONE ] ; then
200                #
201                # Initialize ${flag_post}_${compname}_${post_freq}
202                #
203                eval ${flag_post}_${compname}_TimeSeriesFrequency=true
204            fi
205            let $(( i=i+3 ))
206        done
207    fi
208
209    for element in $( set | grep Post_ | grep ${compname}_TimeSeriesFrequency | grep -i "=true" ) ; do
210        #
211        FILE=$( echo ${element} | awk -F "_${compname}" '{print $1}' | awk "-FPost_" '{print $2}' )
212        #
213        # For each comp and each file who needs post-processing
214        # determine which variables need time series.
215        #
216        IGCM_card_DefineArrayFromOption ${card} Post_${FILE} TimeSeriesVars
217        #
218        if [ $( eval echo \$\{${compname}_Post_${FILE}_TimeSeriesVars} ) = ${NULL_STR} ] ; then
219            #
220            IGCM_debug_Print 3 "Nothing to do for ${FILE} according to ${card}"
221            continue
222            #
223        fi
224        #
225        IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
226        #
227        # So we need coordinates axis to properly fill produced time series
228        #
229        IGCM_card_DefineArrayFromOption ${card} Post_${FILE} GatherWithInternal
230        #
231        if [ $( eval echo \$\{${compname}_Post_${FILE}_GatherWithInternal} ) = ${NULL_STR} ] ; then
232            #
233            IGCM_debug_Print 1 "ERROR !! We miss coordinate axis for ${FILE} according to ${card}"
234            IGCM_debug_Print 1 "So no time series will be produced for ${FILE} !!"
235            continue
236            #
237        fi
238        #
239        # CREATE LIST
240        #
241        FlagDir=$( echo ${FILE} | awk -F "_" '{print $1}' )
242        case ${FlagDir} in
243            1Y)    TS_Dir=TS_YE  ;;
244            1M)    TS_Dir=TS_MO  ;;
245            5D|1D) TS_Dir=TS_DA  ;;
246            HF)    TS_Dir=TS_HF  ;;
247            INS)   TS_Dir=TS_INS ;;
248        esac
249
250        count=0
251        for var in $( eval echo \$\{${compname}_Post_${FILE}_TimeSeriesVars[*]} ) ; do
252            #
253            DIRECTORY=${R_SAVE}/${comp}/Analyse/${TS_Dir}
254            TestedFile=${config_UserChoices_JobName}_${DateBegin}_${PeriodDateEnd}_${FlagDir}_${var}.nc
255            #
256            if [ ! -f ${DIRECTORY}/${TestedFile} ] ; then
257                eval set +A LISTE_VARS_${FILE} \$\{LISTE_VARS_${FILE}[*]} ${var}
258                ActiveComp=true
259                (( count = count + 1 ))
260            fi
261        done
262
263        if [ ${count} -gt 0 ] ; then
264            #
265            eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
266            eval set +A LISTE_AXIS_${FILE} \$\{${compname}_Post_${FILE}_GatherWithInternal[*]}
267            eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
268            if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) !=  X${NULL_STR} ]; then
269                for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
270                    if [ Xload${Patch} != X ] ; then
271                        . ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
272                        eval load${Patch}=loaded
273                    fi
274                done
275            fi
276            #
277        fi
278    done
279
280    if [ ${ActiveComp} = true ] ; then
281        set +A ActiveListOfComponents ${ActiveListOfComponents[*]} ${comp}
282    fi
283
284done
285
286set -A config_ListOfComponents ${ActiveListOfComponents[*]}
287[ ! -z "${ActiveListOfComponents}" ] && DoJob=true
288
289########################################################################
290#
291#                          DEFINE DATE FUNCTIONS
292#
293########################################################################
294
295# Give the end of the next $1 period from $2 date
296function create_ts_next_date {
297    IGCM_debug_PushStack "create_ts_next_date" $@
298    typeset year month Length
299    case ${1} in
300        1Y|1y)
301            IGCM_date_GetYearMonth ${2} year month
302            Length=$( IGCM_date_DaysInYear $(( year + 1 )) ) ;;
303        1M|1m)
304            IGCM_date_GetYearMonth ${2} year month
305            if [ $month -lt 12 ] ; then
306                Length=$( IGCM_date_DaysInMonth $year $(( month + 1 )) )
307            else
308                Length=$( IGCM_date_DaysInMonth $(( year + 1 )) 1 )
309            fi ;;
310        *D|*d)
311            Length=$( echo ${1} | sed -e "s/[dD]//" ) ;;
312        *)
313            IGCM_debug_Exit "create_ts " ${1} " invalid PeriodLength : choose in 1Y, 1M, 5D, 1D."
314            IGCM_debug_Verif_Exit_Post ;;
315    esac
316    echo ${Length}
317
318    IGCM_debug_PopStack "create_ts_next_date"
319}
320
321# Give the end of the actual $1 period from $2 date
322function create_ts_begin_date {
323    IGCM_debug_PushStack "create_ts_begin_date" $@
324    typeset year month Length
325    case ${1} in
326        1Y|1y)
327            IGCM_date_GetYearMonth ${2} year month
328            Length=$( IGCM_date_DaysInYear $year ) ;;
329        1M|1m)
330            IGCM_date_GetYearMonth ${2} year month
331            Length=$( IGCM_date_DaysInMonth $year $month ) ;;
332        *D|*d)
333            Length=$( echo ${1} | sed -e "s/[dD]//" ) ;;
334        *)
335            IGCM_debug_Exit "create_ts " ${1} " invalid PeriodLength : choose in 1Y, 1M, 5D, 1D."
336            IGCM_debug_Verif_Exit_Post ;;
337    esac
338    echo ${Length}
339
340    IGCM_debug_PopStack "create_ts_begin_date"
341}
342
343########################################################################
344#
345#               IS THERE SOME ALLREADY PRODUCED TIME SERIES ?
346#                 IF SO BRING THEM IN THE WORKING DIRECTORY
347#
348# PS : Keep in mind that IGCM_sys_Get here is a weak link :
349#      - especially for IDRiS
350#      - no special protection against "command too long" !
351#      - libIGCM_sys should handle that or modify following code part
352#
353########################################################################
354
355if [ ${StandAlone} != true ] ; then
356    Running_Flag=$( IGCM_sys_RshMaster "IGCM_card_DefineVariableFromOption \
357       ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning ; \
358       echo \${run_PostProcessing_TimeSeriesRunning}" )
359
360    if [ X${Running_Flag} = Xy ] ; then
361        # Time Series Job allready running
362        IGCM_debug_Print 1 "Time Series Job allready running exit"
363        exit
364    fi
365    # Now run.card know we are running
366    IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "y"
367    # Get information from last execution
368    CompletedFlag=$( IGCM_sys_RshMaster \
369        "IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted ;\
370        echo \${run_PostProcessing_TimeSeriesCompleted}" )
371fi
372
373# Is it the first submission or not ?
374if [ X${CompletedFlag} = X ] ; then
375    # First Time Series Submission
376    FIRST_PASS=TRUE
377    Length=$( create_ts_begin_date ${config_UserChoices_PeriodLength} ${DateBegin} )
378    DATE_FIN_JOB_B=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( Length - 1 )) )
379else
380    # Last Completed Time Series
381    FIRST_PASS=FALSE
382    DATE_FIN_JOB_B=${CompletedFlag}
383    date=${DateBegin}_${DATE_FIN_JOB_B}
384    for comp in ${config_ListOfComponents[*]} ; do
385        #
386        IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
387        #
388        IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
389        #
390        set -A liste_file
391        #
392        file_path=${R_SAVE}/${comp}/Analyse
393        #
394        i=0
395        file=${NULL_STR}
396        #
397        for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
398            #
399            # Determine in which output can we find file
400            # IE : Analyse/TS or Analyse/DA2TS or ...
401            #
402            FlagDir=$( echo ${file} | awk -F "_" '{print $1}' )
403            case ${FlagDir} in
404                1Y)    TS_Dir=TS_YE  ;;
405                1M)    TS_Dir=TS_MO  ;;
406                5D|1D) TS_Dir=TS_DA  ;;
407                HF)    TS_Dir=TS_HF  ;;
408                INS)   TS_Dir=TS_INS ;;
409            esac
410            for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
411                #
412                set +A liste_file ${liste_file[*]} ${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc
413                #
414            done
415        done
416        if [ X${file} != X${NULL_STR} ] ; then
417            IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
418            IGCM_debug_Verif_Exit_Post
419        fi
420    done
421    #
422    LEVEL=0
423    #
424fi
425
426########################################################################
427#
428#                          DEFINE LOOP PARAMETERS
429#
430########################################################################
431
432DATE_COUNT=${DATE_FIN_JOB_B}
433DATE_FIN_JOB_B_LOOP=${DATE_FIN_JOB_B}
434
435( [ -n "${LEVEL}" ] && [ ${LEVEL} -eq 0 ] ) && NBRE_FILE_TOT=0 || NBRE_FILE_TOT=1
436
437if [ ${DoJob} = true ] ; then
438    while [ ${DATE_COUNT} -lt ${PeriodDateEnd} ] ; do
439        (( NBRE_FILE_TOT = NBRE_FILE_TOT + 1 ))
440        Length=$( create_ts_next_date ${config_UserChoices_PeriodLength} ${DATE_COUNT} )
441        DATE_COUNT=$( IGCM_date_AddDaysToGregorianDate ${DATE_COUNT} ${Length} )
442    done
443else
444    NBRE_FILE_TOT=0
445fi
446
447if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then
448    NBRE_TOUR=1
449elif [ ${NBRE_FILE_TOT}%${UNIX_MAX_LIMIT} -eq 0 ] ; then
450    let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT} 
451else
452    let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT}+1
453fi
454
455CURRENT_LOOP=1
456# THIS LOOP GIVES PARAMETERS FOR THE DATE'S LOOP ie : DATE_COURANTE, DATE_FIN.
457# WE HAVE 3 LEVELS
458#    - LEVEL 1 : JUST ONE LOOP TO COMPLETE
459#    - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
460#    - LEVEL 3 : WE ARE IN THE LAST LOOP
461
462while [ ${CURRENT_LOOP} -le  ${NBRE_TOUR} ] ; do
463     
464    if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then 
465        # - LEVEL 1 : JUST ONE LOOP TO COMPLETE
466        NBRE_FILE_LOOP=${NBRE_FILE_TOT}
467
468        if [ ${FIRST_PASS} = TRUE ] ; then
469            DATE_COURANTE=${DATE_FIN_JOB_B_LOOP}
470        else
471            Length=$( create_ts_next_date ${config_UserChoices_PeriodLength} ${DATE_FIN_JOB_B_LOOP} )
472            DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
473        fi
474
475        DATE_FIN=${PeriodDateEnd}
476        DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP}
477
478        [ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
479        [ -n "${LEVEL}" ] && FLAG_B="TRUE" || FLAG_B="FALSE"
480
481        LEVEL=1
482        DEPOT="TRUE"     
483    elif [ ${CURRENT_LOOP} -ne  ${NBRE_TOUR} ] ; then
484        # - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
485        NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
486
487        if [ ! ${FIRST_PASS} = TRUE ] && [ ${CURRENT_LOOP} -eq 1 ] ; then
488            Length=$( create_ts_next_date ${config_UserChoices_PeriodLength} ${DATE_FIN_JOB_B_LOOP} )
489            DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
490        elif [ -z "${DATE_COURANTE}" ] ; then
491            DATE_COURANTE=${DATE_FIN_JOB_B}
492        else
493            Length=$( create_ts_next_date ${config_UserChoices_PeriodLength} ${DATE_FIN} )
494            DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
495        fi
496         
497        (( TotLength = 0 ))
498        COMPTEUR=1
499        DATE_LOOP=${DATE_COURANTE}
500
501        while [ ${COMPTEUR} -lt ${NBRE_FILE_LOOP} ] ; do
502            #
503            Length=$( create_ts_next_date ${config_UserChoices_PeriodLength} ${DATE_LOOP} )
504            DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length} )
505            (( TotLength = TotLength + Length ))
506            (( COMPTEUR = COMPTEUR + 1 ))
507            #
508        done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
509        DATE_FIN=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} ${TotLength} )
510
511        [ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
512        [ -n "${LEVEL}" ] && [ ${LEVEL} -eq 0 ] && FLAG_B="TRUE" || [ ${CURRENT_LOOP} -gt 1 ] && FLAG_B="TRUE" || FLAG_B="FALSE"
513        DEPOT="FALSE"
514        LEVEL=2
515    else
516        # - LEVEL 3 : WE ARE IN THE LAST LOOP
517        NBRE_FILE_LOOP=$( expr ${NBRE_FILE_TOT} % $(( UNIX_MAX_LIMIT * $(( CURRENT_LOOP - 1 )) )) )
518        [ ${NBRE_FILE_LOOP} -eq 0 ] && NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
519
520        Length=$( create_ts_next_date ${config_UserChoices_PeriodLength} ${DATE_FIN} )
521        DATE_COURANTE=$(IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
522        DATE_FIN=${PeriodDateEnd}
523        [ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
524
525        LEVEL=3
526        DEPOT="TRUE"
527        FLAG_B="TRUE"
528    fi
529
530    # THEN FOR EACH FILE TYPE WE LOOP BETWEEN DATE_COURANTE AND DATE_FIN.
531    for comp in ${config_ListOfComponents[*]} ; do
532        #
533        IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
534        #
535        IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
536        #
537        eval R_OUT_${comp}=${R_SAVE}/${comp}
538        #
539        i=0
540        #
541        for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
542            #
543            # Determine in which output can we find file
544            # IE : Output/MO or Output/DA or ...
545            #
546            FlagDir=$( echo ${file} | awk -F "_" '{print $1}' )
547            case ${FlagDir} in
548                1Y)    TS_Dir=TS_YE  ; FreqDir=AN  ;;
549                1M)    TS_Dir=TS_MO  ; FreqDir=MO  ;;
550                *D)    TS_Dir=TS_DA  ; FreqDir=DA  ;;
551                HF)    TS_Dir=TS_HF  ; FreqDir=HF  ;;
552                INS)   TS_Dir=TS_INS ; FreqDir=INS ;;
553                *) 
554                       IGCM_debug_Print 1 "Error in FlagDir=${FlagDir} for file ${file}."
555                       IGCM_sys_Exit "Job create_ts" ;;
556            esac
557
558            # Initialize array
559            set -A liste_file
560            set -A liste_file_tmp
561
562            eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
563
564            COMPTEUR=1
565            DATE_LOOP=${DATE_COURANTE}
566
567            while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] ; do
568                #
569                Length1=$( create_ts_begin_date ${config_UserChoices_PeriodLength} ${DATE_LOOP} )
570                DATE_TAB=$(  IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} $(( 1 - Length1 )) )_${DATE_LOOP}
571
572                Length2=$( create_ts_next_date ${config_UserChoices_PeriodLength} ${DATE_LOOP} )
573                DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length2} )
574
575                set +A liste_file ${liste_file[*]} ${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
576                set +A liste_file_tmp ${liste_file_tmp[*]} ${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
577                (( COMPTEUR = COMPTEUR + 1 ))
578                #
579            done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
580            DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} -${Length2} )
581
582            # Get selected files
583            IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
584            eval ExitFlag=false
585            IGCM_debug_Verif_Exit_Post
586
587            # Apply IGCM_Patch if needed
588            if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) !=  X${NULL_STR} ]; then
589                for file_t in $( eval echo ${liste_file_tmp[*]} ); do           
590                    for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
591                        IGCM_${Patch} ${file_t}
592                    done
593                done
594            fi
595
596            # WE CAN CONCATENATE FILES AT THIS POINT
597            [ "${FLAG_B}" = "TRUE" ] && DATE_BUILD_B=${DateBegin}_${DATE_FIN_JOB_B_LOOP_PREC}
598            Length=$( create_ts_begin_date ${config_UserChoices_PeriodLength} ${DATE_COURANTE} )
599            DATE_BUILD1=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} -$(( Length - 1 )) )
600            DATE_BUILD=${DATE_BUILD1}_${DATE_FIN}
601            DATE_BUILD_END=${DateBegin}_${DATE_FIN}
602
603            liste_coord=" "
604            for axis in $( eval echo \${LISTE_AXIS_${file}[*]} ); do
605              liste_coord=${liste_coord}${axis}","
606            done
607
608            # Time axis must be the last dimension
609            time_axis=${axis}
610
611            # SWITCH BETWEEN MODEL TO FIT PARTICULAR CASE
612            for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
613                #
614                IGCM_sys_ncrcat --hst -v ${liste_coord}${var} ${liste_file_tmp[*]} ${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
615                #
616#                 # Change time axis name to be CF compliant
617#                 if [ ${time_axis} != time_counter ]; then
618#                   IGCM_sys_ncrename -O -v ${time_axis},time_counter ${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
619#                 fi
620                #
621                if [ "${FLAG_B}" = "TRUE" ] ; then
622                    # WE CONCATENATE WITH EXISTING FILES
623                    file1=${config_UserChoices_JobName}_${DATE_BUILD_B}_${FlagDir}_${var}.nc
624                    file2=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
625                    file_out=${config_UserChoices_JobName}_${DATE_BUILD_END}_${FlagDir}_${var}.nc
626                    IGCM_sys_ncrcat --hst ${file1} ${file2} ${file_out}
627                    #
628                    IGCM_sys_Rm ${file1} ${file2}
629                fi
630                #
631                if [ "${DEPOT}" = "TRUE" ] ; then
632                    # WE PUT FINAL FILE ON FILE SYSTEM
633                    [ "${FLAG_B}" = "FALSE" ] && file_out=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
634                    #${ncatted} -a modulo,time_counter,c,c," " ${file_out}
635                    file_before=${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc
636                    eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}}/Analyse/${TS_Dir}/${file_out}
637                    [ "${FLAG_B}" = "TRUE" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file_before} ${file_out}
638                    IGCM_sys_Rm ${file_out}
639                    #
640                fi
641                #
642                eval ExitFlag=false
643                IGCM_debug_Verif_Exit_Post
644            done     # for var in ${liste_var} ; do
645            IGCM_sys_Rm ${liste_file_tmp[*]}
646        done         # for file in ${liste_file_brut} do
647    done             # for comp in ${config_ListOfComponents[*]} ; do
648    eval ExitFlag=false
649    IGCM_debug_Verif_Exit_Post
650    # GO AHEAD IN DEEPEST LOOP
651    (( CURRENT_LOOP = CURRENT_LOOP + 1 ))
652done                 # while [ ${CURRENT_LOOP} -le  ${NBRE_TOUR} ] do
653# FINISH
654
655# DODS copy
656for comp in ${config_ListOfComponents[*]} ; do
657    for TS_Dir in $( ls ${R_SAVE}/${comp}/Analyse/ 2>&1 | grep TS_ ) ; do
658        [ -d ${R_SAVE}/${comp}/Analyse/${TS_Dir} ] && IGCM_sys_Put_Dods ${comp}/Analyse/${TS_Dir}
659    done
660done
661
662# clean environment variables and export usefull one :
663if [ X$( hostname -s ) = Xrhodes ] ; then
664    LISTE_ENV=$( env | grep -iv MYLANG | grep -iv HOST | grep -iv LOGIN | grep -iv config_ | grep -iv R_ | grep -iv libIGCM | grep -iv PERIOD \
665        | grep -iv DMFDIR | grep -iv AN_| grep -iv YEARS | grep -iv ex | grep -iv RESOL_ | grep -iv PATH | grep -iv WORKDIR | grep -iv USER   \
666        | grep -iv LOGNAME | grep -iv PROFILE | grep -iv ENV | grep -iv TMPDIR | grep -iv POST_DIR | grep -iv StandAlone | grep -iv DB | awk -F= '{print $1}' )
667 
668    for variables in ${LISTE_ENV} ; do
669        unset ${variables}
670    done
671fi
672
673export POST_DIR; export SUBMIT_DIR; export libIGCM; export R_INIT; export R_BC; export StandAlone
674export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_SRF
675
676listVarEnv="POST_DIR,SUBMIT_DIR,libIGCM,R_INIT,R_BC,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_SRF"
677export listVarEnv
678
679#D-
680# --------------------------------------------------------------------
681#D- Test if all was right
682# --------------------------------------------------------------------
683IGCM_debug_Verif_Exit_Post
684# --------------------------------------------------------------------
685#D- Submit MONITORING
686# --------------------------------------------------------------------
687IGCM_sys_QsubPost monitoring
688
689#D-
690# --------------------------------------------------------------------
691#D- Update the run.card
692# --------------------------------------------------------------------
693if [ ${StandAlone} != true ] ; then
694    # Put in run.card end period of time series
695    IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted "${DATE_FIN}"
696    # We have finish the job !
697    IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "n"
698fi
699date
Note: See TracBrowser for help on using the repository browser.