Context Navigation

AA_create_se @ 395

Last change on this file since 395 was 387, checked in by sdipsl, 14 years ago

function IGCM_sys_Get_Master on cesium will try 10 times to scp files from master machine. Hopefully the last cesium consolidation
function IGCM_config_Check will check chunck2D/3D consistency with TimeSeriesFrequency?.
change the way the output from post-processing script was named. We will now have:

-rebuild_fromWorkdir.20051231.out
-create_ts.20051231.2D.out
-create_ts.20051231.3D.out
-atlas_PISCES.19991231.out
-create_ts.20051231.Chunck2D.ATM.POST_1D_histday.out
-create_ts.20051231.Chunck2D.OCE.POST_1D_grid_T.out

remove modipsl headers from AA_SE_Checker, AA_TimeSeries_Checker, AA_move-and-rename
- they check and submit job. They do not use resources and so there is no point to submit them
- they will be rename shortly SE_Checker.ksh, TimeSeries_Checker.ksh, move-and-rename.ksh
cosmetics

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 28.5 KB

Line
1	#-Q- cesium #!/bin/ksh
2	#-Q- cesium ######################
3	#-Q- cesium ## CESIUM CEA ##
4	#-Q- cesium ######################
5	#-Q- cesium #MSUB -r SE # Nom du job
6	#-Q- cesium #MSUB -N 1 # Reservation du noeud
7	#-Q- cesium #MSUB -n 1 # Reservation du processus
8	#-Q- cesium #MSUB -T 86400 # Limite de temps elapsed du job
9	#-Q- cesium #MSUB -E "-j o"
10	#-Q- cesium #MSUB -E "-S /bin/ksh"
11	#-Q- platine #!/usr/bin/ksh
12	#-Q- platine ###################
13	#-Q- platine ## PLATINE CEA ##
14	#-Q- platine ###################
15	#-Q- platine #BSUB -J SE # Nom du job
16	#-Q- platine #BSUB -N # message a la fin du job
17	#-Q- platine #BSUB -n 1 # reservation des processeurs pour le job
18	#-Q- platine #BSUB -W 1:00 # Limite temps
19	#-Q- platine #BSUB -q post # Passage en queue post
20	#-Q- sx8brodie #!/bin/ksh
21	#-Q- sx8brodie #######################
22	#-Q- sx8brodie ## SX8BRODIE IDRIS ##
23	#-Q- sx8brodie #######################
24	#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
25	#-Q- sx8brodie # @ wall_clock_limit = 20:00:00
26	#-Q- sx8brodie # Nom du travail LoadLeveler
27	#-Q- sx8brodie # @ job_name = SE
28	#-Q- sx8brodie # Fichier de sortie standard du travail
29	#-Q- sx8brodie # @ output = $(job_name).$(jobid)
30	#-Q- sx8brodie # Fichier de sortie d'erreur du travail
31	#-Q- sx8brodie # @ error = $(job_name).$(jobid)
32	#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
33	#-Q- sx8brodie # @ notification = error
34	#-Q- sx8brodie # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
35	#-Q- sx8brodie # @ queue
36	#-Q- aix6 #!/bin/ksh
37	#-Q- aix6 #######################
38	#-Q- aix6 ## VARGAS IDRIS ##
39	#-Q- aix6 #######################
40	#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
41	#-Q- aix6 # @ wall_clock_limit = 20:00:00
42	#-Q- aix6 # Nom du travail LoadLeveler
43	#-Q- aix6 # @ job_name = SE
44	#-Q- aix6 # Fichier de sortie standard du travail
45	#-Q- aix6 # @ output = $(job_name).$(jobid)
46	#-Q- aix6 # Fichier de sortie d'erreur du travail
47	#-Q- aix6 # @ error = $(job_name).$(jobid)
48	#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
49	#-Q- aix6 # @ notification = error
50	#-Q- aix6 # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
51	#-Q- aix6 # @ queue
52	#-Q- ulam #!/bin/ksh
53	#-Q- ulam #######################
54	#-Q- ulam ## ULAM IDRIS ##
55	#-Q- ulam #######################
56	#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
57	#-Q- ulam # @ wall_clock_limit = 20:00:00
58	#-Q- ulam # Nom du travail LoadLeveler
59	#-Q- ulam # @ job_name = SE
60	#-Q- ulam # Fichier de sortie standard du travail
61	#-Q- ulam # @ output = $(job_name).$(jobid)
62	#-Q- ulam # Fichier de sortie d'erreur du travail
63	#-Q- ulam # @ error = $(job_name).$(jobid)
64	#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
65	#-Q- ulam # @ notification = error
66	#-Q- ulam # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
67	#-Q- ulam # @ queue
68	#-Q- sx8mercure #!/bin/ksh
69	#-Q- sx8mercure ######################
70	#-Q- sx8mercure ## SX8MERCURE CEA ##
71	#-Q- sx8mercure ######################
72	#-Q- sx8mercure #PBS -N SE # Nom du job
73	#-Q- sx8mercure #PBS -j o # regroupement des stdout et stderr
74	#-Q- sx8mercure #PBS -S /usr/bin/ksh # shell de soumission
75	#-Q- sx8mercure #PBS -l memsz_job=1gb # Limite memoire a 1 Go
76	#-Q- sx8mercure #PBS -l cputim_job=1:00:00 # Limite temps a 1 heures
77	#-Q- sx8mercure #PBS -q scalaire
78	#-Q- sx9mercure #!/bin/ksh
79	#-Q- sx9mercure #########################
80	#-Q- sx9mercure ## CESIUM FOR SX9 CEA ##
81	#-Q- sx9mercure #########################
82	#-Q- sx9mercure #MSUB -r SE # Nom du job
83	#-Q- sx9mercure #MSUB -N 1 # Reservation du noeud
84	#-Q- sx9mercure #MSUB -n 1 # Reservation du processus
85	#-Q- sx9mercure #MSUB -T 86400 # Limite de temps elapsed du job
86	#-Q- sx9mercure #MSUB -E "-j o"
87	#-Q- sx9mercure #MSUB -E "-S /bin/ksh"
88	#-Q- titane #!/bin/ksh
89	#-Q- titane ######################
90	#-Q- titane ## TITANE CEA ##
91	#-Q- titane ######################
92	#-Q- titane #MSUB -r SE # Nom du job
93	#-Q- titane #MSUB -N 1 # Reservation du noeud
94	#-Q- titane #MSUB -n 1 # Reservation du processus
95	#-Q- titane #MSUB -T 86400 # Limite de temps elapsed du job
96	#-Q- titane #MSUB -E "-j o"
97	#-Q- titane #MSUB -E "-S /bin/ksh"
98	#-Q- titane ##MSUB -e nco.out # Sortie standard
99	#-Q- titane ##MSUB -o nco.out # Sortie standard
100	#-Q- lxiv8 ######################
101	#-Q- lxiv8 ## OBELIX LSCE ##
102	#-Q- lxiv8 ######################
103	#-Q- lxiv8 #PBS -N SE
104	#-Q- lxiv8 #PBS -m a
105	#-Q- lxiv8 #PBS -j oe
106	#-Q- lxiv8 #PBS -q medium
107	#-Q- lxiv8 #PBS -o SE.$$
108	#-Q- lxiv8 #PBS -S /bin/ksh
109	#-Q- default #!/bin/ksh
110	#-Q- default ##################
111	#-Q- default ## DEFAULT HOST ##
112	#-Q- default ##################
113
114	#**************************************************************
115	# Author: Sebastien Denvil
116	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
117	# $Revision:: $ Revision of last commit
118	# $Author:: $ Author of last commit
119	# $Date:: $ Date of last commit
120	# IPSL (2006)
121	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
122	#
123	#**************************************************************
124
125	#set -eu
126	#set -vx
127
128	date
129
130	#-Q- sx8brodie export OMP_NUM_THREADS=1
131	#-Q- aix6 export OMP_NUM_THREADS=1
132	#-Q- ulam export OMP_NUM_THREADS=1
133
134	########################################################################
135
136	#D- Flag to determine if this job in a standalone mode
137	#D- Default : value from AA_job if any
138	StandAlone=${StandAlone:=true}
139
140	#D- Low level debug : to bypass lib test checks and stack construction
141	#D- Default : value from AA_job if any
142	libIGCM=${libIGCM:=::modipsl::/libIGCM}
143	# WARNING for StandAlone used : To run this script on some machine,
144	# you must check MirrorlibIGCM variable in sys library.
145	# If this variable is true, you must use libIGCM_POST path instead
146	# of your running libIGCM directory.
147
148	#D- Flag to determine if rebuild process has been performed asynchronously
149	#D- Possible value true or false.
150	#D- If true raw files has allready been patched by rebuild job
151	#D- If false create_ts will patch the raw files
152	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
153	RebuildFrequency=${RebuildFrequency:=true}
154
155	#D- Flag to determine atlas job's output directory
156	#D- Default : value from libIGCM_post.ksh if any
157	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
158
159	#D- Increased verbosity (1, 2, 3)
160	#D- Default : value from AA_job if any
161	Verbosity=${Verbosity:=3}
162
163	#D- Low level debug : to bypass lib test checks and stack construction
164	#D- Default : value from AA_job if any
165	DEBUG_debug=${DEBUG_debug:=false}
166
167	#D- TEMPORARY Flag to determine atmospheric resolution
168	#D- Default : value from atmospheric driver if any
169	RESOL_ATM=${RESOL_ATM:=ALL}
170
171	#D- Flag to determine surface resolution
172	#D- Default : value from surface driver if any
173	RESOL_SRF=ALL
174
175	#D- Flag to determine surface resolution
176	#D- Default : value from surface driver if any
177	RESOL_SBG=ALL
178
179	#D- TEMPORARY Flag to determine ocean resolution
180	#D- Default : value from ocean driver if any
181	RESOL_OCE=${RESOL_OCE:=ORCA2}
182
183	#D- TEMPORARY Flag to determine ice resolution
184	#D- Default : value from ice driver if any
185	RESOL_ICE=${RESOL_ICE:=ORCA2}
186
187	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
188	#D- Default : value from ice driver if any
189	RESOL_MBG=${RESOL_MBG:=ORCA2}
190
191	########################################################################
192
193	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
194	( ${DEBUG_debug} ) && IGCM_debug_Check
195	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
196	( ${DEBUG_debug} ) && IGCM_card_Check
197	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
198	( ${DEBUG_debug} ) && IGCM_date_Check
199	#-------
200	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
201
202	########################################################################
203
204	#set -vx
205
206	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
207	IGCM_sys_Cd ${RUN_DIR_PATH}
208
209	if [ ${StandAlone} = true ] ; then
210	CARD_DIR=${SUBMIT_DIR}
211	else
212	CARD_DIR=${RUN_DIR_PATH}/$( basename ${SUBMIT_DIR} )
213	IGCM_sys_Get_Master ${SUBMIT_DIR} ${RUN_DIR_PATH}
214	fi
215
216	#
217	# First of all
218	#
219	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card UserChoices
220	typeset option
221	for option in ${config_UserChoices[*]} ; do
222	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
223	done
224	#
225	echo
226	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
227	IGCM_debug_PrintVariables 3 config_UserChoices_JobName
228	IGCM_debug_PrintVariables 3 config_UserChoices_LongName
229	IGCM_debug_PrintVariables 3 config_UserChoices_TagName
230	IGCM_debug_PrintVariables 3 config_UserChoices_SpaceName
231	IGCM_debug_PrintVariables 3 config_UserChoices_ExperimentName
232	IGCM_debug_PrintVariables 3 config_UserChoices_CalendarType
233	IGCM_debug_PrintVariables 3 config_UserChoices_DateBegin
234	IGCM_debug_PrintVariables 3 config_UserChoices_DateEnd
235	IGCM_debug_PrintVariables 3 config_UserChoices_PeriodLength
236	echo
237
238	#==================================
239	#R_SAVE : Job output directory
240	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
241	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
242	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
243	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
244	else
245	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
246	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
247	fi
248	#
249	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
250	#
251	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
252	#
253	# Determine period for seasonnal average. Default : value from AA_job if any
254	#
255	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
256	#
257	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
258	#
259	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
260	#
261	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
262	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
263	an_fin=$( IGCM_date_YearDigit ${an_fin} )
264	#
265	# For each component selected determine which files need post-processing
266	#
267	for comp in ${config_ListOfComponents[*]} ; do
268	#
269	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
270	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
271	#
272	card=${CARD_DIR}/COMP/${compname}.card
273	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
274	#
275	ListFilesName=${compname}_OutputFiles_List
276	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
277	#
278	if [ X${FileName0} != X${NULL_STR} ] ; then
279	#
280	# INITIALISATION
281	#
282	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
283	typeset i=2
284	#
285	until [ $i -gt $NbFiles ]; do
286	#
287	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
288	#
289	if [ X${flag_post} != XNONE ] ; then
290	#
291	# Check if seasonal is required for this file
292	#
293	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
294	#
295	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
296	#
297	# variable option allready typeset above
298	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
299	if [ ${option} = Seasonal ] ; then
300	FoundSeasonal=true
301	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
302	fi
303	done
304	#
305	if [ ! X${FoundSeasonal} = Xtrue ] ; then
306	eval ${compname}_${flag_post}_Seasonal=ON
307	fi
308	#
309	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
310	(( i=i+3 ))
311	continue
312	fi
313	#
314	# CREATE LIST
315	#
316	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
317	#
318	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
319	#
320	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
321	#
322	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
323	if [ ! $? = 0 ] ; then
324	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
325	if [ ! ${RebuildFrequency} = true ] ; then
326	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
327	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
328	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
329	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
330	if [ Xload${Patch} != X ] ; then
331	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
332	eval load${Patch}=loaded
333	fi
334	done
335	fi
336	fi
337	fi
338	fi
339	(( i=i+3 ))
340	done # until [ $i -eq $NbFiles ]
341	fi
342	done # for comp
343
344	for comp in ${config_ListOfComponents[*]} ; do
345	#
346	eval R_OUT_${comp}=${R_SAVE}/${comp}
347	#
348	i=0
349	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
350	#
351	# Determine in which output can we find file
352	# IE : Output/MO or Output/DA or ...
353	#
354	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
355	case ${FlagDir} in
356	1Y) FreqDir=YE ;;
357	1M) FreqDir=MO ;;
358	5D\|1D) FreqDir=DA ;;
359	HF) FreqDir=HF ;;
360	3H) FreqDir=HF ;;
361	INS) FreqDir=INS ;;
362	esac
363	#
364	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
365	# Do not redoo an existing SE
366	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
367	if [ $? = 0 ] ; then
368	continue
369	fi
370	#
371	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
372	#
373	case ${config_UserChoices_PeriodLength} in
374	# For Yearly simulation :
375	1Y\|1y)
376	(( an = an_deb ))
377	# Initialize array
378	#set -A liste_file_se
379	#set -A liste_file_se_tmp
380	unset liste_file_se
381	unset liste_file_se_tmp
382	while [ ${an} -le ${an_fin} ] ; do
383	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
384	#
385	an4=$(printf "%04i\n" ${an} )
386	#
387	diy=$( IGCM_date_DaysInYear $an4 )
388	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
389	date=${an4}0101_${afin}
390	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
391	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
392	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
393	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
394	(( an = an + 1 ))
395	done
396	#
397	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
398	eval ExitFlag=false
399	IGCM_debug_Verif_Exit_Post
400
401	# Apply IGCM_Patch if needed
402	if [ ! ${RebuildFrequency} = true ] ; then
403	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
404	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
405	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
406	IGCM_${Patch} ${file_t}
407	done
408	done
409	fi
410	fi
411
412	# Search of common list of variables to be treated
413	nbfile=0
414	for file_common_var in ${liste_file_se_tmp[*]} ; do
415	ncks -m ${file_common_var} \| grep '# dim. =' \| cut -d ':' -f 1 >> tmpfile_ncra
416	(( nbfile = nbfile + 1 ))
417	done
418	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
419	IGCM_sys_Rm tmpfile_ncra
420
421	# detect time counter : time or time_counter ie unlimited variable
422	var_unlim=$(ncdump -h ${liste_file_se_tmp[1]}\|grep UNLIMITED\| cut -d ' ' -f 1)
423
424	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
425	#
426	(( nb = ${mois} - 1 ))
427	#
428	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
429	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
430	done
431	IGCM_sys_Rm ${liste_file_se_tmp[*]}
432	eval ExitFlag=false
433	IGCM_debug_Verif_Exit_Post
434	;;
435	# For Monthly simulation :
436	1M\|1m)
437	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
438	#
439	(( nb = ${mois} - 1 ))
440	#
441	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
442	#
443	(( an = an_deb ))
444	# Initialize array
445	#set -A liste_file_se
446	#set -A liste_file_se_tmp
447	unset liste_file_se
448	unset liste_file_se_tmp
449	while [ ${an} -le ${an_fin} ] ; do
450	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
451	#
452	an4=$(printf "%04i\n" ${an} )
453	#
454	date=${an4}${mois}01_${an4}${mois}${moisL}
455	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
456	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
457	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
458	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
459
460	(( an = an + 1 ))
461	done
462	#
463	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
464	eval ExitFlag=false
465	IGCM_debug_Verif_Exit_Post
466
467	# Apply IGCM_Patch if needed
468	if [ ! ${RebuildFrequency} = true ] ; then
469	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
470	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
471	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
472	IGCM_${Patch} ${file_t}
473	done
474	done
475	fi
476	fi
477
478	# Search of common list of variables to be treated
479	nbfile=0
480	for file_common_var in ${liste_file_se_tmp[*]} ; do
481	ncks -m ${file_common_var} \| grep '# dim. =' \| cut -d ':' -f 1 >> tmpfile_ncra
482	(( nbfile = nbfile + 1 ))
483	done
484	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
485	IGCM_sys_Rm tmpfile_ncra
486
487	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
488
489	IGCM_sys_Rm ${liste_file_se_tmp[*]}
490	eval ExitFlag=false
491	IGCM_debug_Verif_Exit_Post
492	done
493	;;
494	D\|d)
495	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
496	#
497	(( nb = ${mois} - 1 ))
498	#
499	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
500	#
501	(( an = an_deb ))
502	# Initialize array
503	#set -A liste_file_se
504	#set -A liste_file_se_tmp
505	unset liste_file_se
506	unset liste_file_se_tmp
507	while [ ${an} -le ${an_fin} ] ; do
508	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
509	#
510	an4=$(printf "%04i\n" ${an} )
511	#
512	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
513	# for one day
514	if [ ${length_d} -eq 1 ] ; then
515	(( length_d = 0 ))
516	fi
517	(( day = 1 ))
518	while [ ${day} -le ${moisL} ] ; do
519	day2=$(printf "%02i\n" ${day} )
520	day_beg=${an4}${mois}${day2}
521	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
522	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
523	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
524	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
525	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
526	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
527	(( day = day + 1 ))
528	done
529	(( an = an + 1 ))
530	done
531	#
532	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
533	eval ExitFlag=false
534	IGCM_debug_Verif_Exit_Post
535
536	# Apply IGCM_Patch if needed
537	if [ ! ${RebuildFrequency} = true ] ; then
538	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
539	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
540	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
541	IGCM_${Patch} ${file_t}
542	done
543	done
544	fi
545	fi
546
547	# Search of common list of variables to be treated
548	nbfile=0
549	for file_common_var in ${liste_file_se_tmp[*]} ; do
550	ncks -m ${file_common_var} \| grep '# dim. =' \| cut -d ':' -f 1 >> tmpfile_ncra
551	(( nbfile = nbfile + 1 ))
552	done
553	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
554	IGCM_sys_Rm tmpfile_ncra
555
556	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
557	IGCM_sys_Rm ${liste_file_se_tmp[*]}
558	eval ExitFlag=false
559	IGCM_debug_Verif_Exit_Post
560	done
561	;;
562	esac
563	#
564
565	# Search of common list of variables to be treated
566	nbfile=0
567	for file_common_var in ${OUT_SE[*]} ; do
568	ncks -m ${file_common_var} \| grep '# dim. =' \| cut -d ':' -f 1 >> tmpfile_ncrcat
569	(( nbfile = nbfile + 1 ))
570	done
571	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
572	IGCM_sys_Rm tmpfile_ncrcat
573
574	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
575	#
576	IGCM_sys_Rm ${OUT_SE[*]}
577	#
578	# detect time counter : time or time_counter ie unlimited variable
579	var_unlim=$(ncdump -h ${RESULT_SE}\|grep UNLIMITED\| cut -d ' ' -f 1)
580
581	# Put correct climatological axis following calendar
582	case ${config_UserChoices_CalendarType} in
583	360d)
584	# 360 days
585	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN 0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
586	;;
587	noleap)
588	# 365 days
589	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN 0000 00:00:00\" ; ${var_unlim}@calendar=\"noleap\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
590	;;
591	all_leap)
592	# 366 days
593	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN 0000 00:00:00\" ; ${var_unlim}@calendar=\"all_leap\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
594	;;
595	leap\|gregorian)
596	# 365.2425 days
597	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN 0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
598	;;
599	esac
600	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
601	IGCM_sys_Rm ${RESULT_SE}
602	IGCM_sys_Mv tmp.nc ${RESULT_SE}
603	#
604	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
605	IGCM_sys_Rm ${RESULT_SE}
606	#
607	eval ExitFlag=false
608	IGCM_debug_Verif_Exit_Post
609	done
610	done
611
612	# DODS copy
613	for comp in ${config_ListOfComponents[*]} ; do
614	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
615	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
616	done
617
618	YEARS=${an_deb}_${an_fin}
619	DIR_OUT=SE
620
621	export POST_DIR; export SUBMIT_DIR; export libIGCM; export R_INIT; export R_BC; export StandAlone
622	export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ; export RESOL_SBG
623	export R_SAVE; export config_UserChoices_JobName; export config_UserChoices_TagName; export YEARS; export DIR_OUT
624	listVarEnv="POST_DIR,SUBMIT_DIR,libIGCM,R_INIT,R_BC,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,R_SAVE,config_UserChoices_JobName,config_UserChoices_TagName,YEARS,DIR_OUT"
625	export listVarEnv;
626
627	#D-
628	# --------------------------------------------------------------------
629	#D- Test if all was right before ATLAS
630	# --------------------------------------------------------------------
631	IGCM_debug_Verif_Exit_Post
632
633	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
634	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
635	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
636	#
637	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
638	IGCM_sys_QsubPost atlas_ORCHIDEE ;
639	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
640	IGCM_sys_QsubPost atlas_LMDZ ;
641	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
642	IGCM_sys_QsubPost atlas_ORCA_LIM ;
643	Script_Post_Output=additionnal.${PeriodDateEnd} ;
644	IGCM_sys_QsubPost additionnal ;
645	#
646	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
647	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
648	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] ; then
649	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
650	IGCM_sys_QsubPost atlas_ORCHIDEE ;
651	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
652	IGCM_sys_QsubPost atlas_LMDZ ;
653	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
654	IGCM_sys_QsubPost atlas_ORCA_LIM ;
655	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
656	IGCM_sys_QsubPost atlas_PISCES ;
657	#
658	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
659	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
660	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
661	#
662	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
663	IGCM_sys_QsubPost atlas_LMDZ ;
664	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
665	IGCM_sys_QsubPost atlas_ORCHIDEE ;
666	#
667	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
668	#
669	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
670	IGCM_sys_QsubPost atlas_LMDZ ;
671	#
672	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
673	[ ${config_UserChoices_TagName} = "OL2" ] ; then
674	#
675	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
676	IGCM_sys_QsubPost atlas_ORCHIDEE ;
677	#
678	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\|
679	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] ; then
680	#
681	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
682	IGCM_sys_QsubPost atlas_ORCA_LIM ;
683	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] ; then
684	#
685	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
686	IGCM_sys_QsubPost atlas_ORCA_LIM ;
687	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
688	IGCM_sys_QsubPost atlas_PISCES ;
689	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
690	#
691	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
692	IGCM_sys_QsubPost atlas_PISCES ;
693	#
694	fi
695
696	# Clean RUN_DIR_PATH (necessary for cesium)
697	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/AA_create_se @ 395

Download in other formats: