Context Navigation

source: trunk/libIGCM/AA_create_se @ 763

Last change on this file since 763 was 758, checked in by mafoipsl, 11 years ago
On curie, delete #MSUB -E '--no-requeue' option. The system will requeue jobs automatically.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 28.9 KB

Line
1	#-Q- cesium #!/bin/ksh
2	#-Q- cesium ######################
3	#-Q- cesium ## CESIUM CEA ##
4	#-Q- cesium ######################
5	#-Q- cesium #MSUB -r SE # Nom du job
6	#-Q- cesium #MSUB -N 1 # Reservation du noeud
7	#-Q- cesium #MSUB -n 1 # Reservation du processus
8	#-Q- cesium #MSUB -T 86400 # Limite de temps elapsed du job
9	#-Q- cesium #MSUB -E "-j o"
10	#-Q- cesium #MSUB -E "-S /bin/ksh"
11	#-Q- curie ######################
12	#-Q- curie ## CURIE TGCC/CEA ##
13	#-Q- curie ######################
14	#-Q- curie #MSUB -r SE # Nom du job
15	#-Q- curie #MSUB -eo
16	#-Q- curie #MSUB -n 1 # Reservation du processus
17	#-Q- curie #MSUB -T 86400 # Limite de temps elapsed du job
18	#-Q- curie #MSUB -q xlarge
19	#-Q- curie #MSUB -Q normal
20	#-Q- curie set +x
21	#-Q- platine #!/usr/bin/ksh
22	#-Q- platine ###################
23	#-Q- platine ## PLATINE CEA ##
24	#-Q- platine ###################
25	#-Q- platine #BSUB -J SE # Nom du job
26	#-Q- platine #BSUB -N # message a la fin du job
27	#-Q- platine #BSUB -n 1 # reservation des processeurs pour le job
28	#-Q- platine #BSUB -W 1:00 # Limite temps
29	#-Q- platine #BSUB -q post # Passage en queue post
30	#-Q- platine #BSUB -L /bin/ksh
31	#-Q- sx8brodie #!/bin/ksh
32	#-Q- sx8brodie #######################
33	#-Q- sx8brodie ## SX8BRODIE IDRIS ##
34	#-Q- sx8brodie #######################
35	#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
36	#-Q- sx8brodie # @ wall_clock_limit = 10:00:00
37	#-Q- sx8brodie # Nom du travail LoadLeveler
38	#-Q- sx8brodie # @ job_name = SE
39	#-Q- sx8brodie # Fichier de sortie standard du travail
40	#-Q- sx8brodie # @ output = $(job_name).$(jobid)
41	#-Q- sx8brodie # Fichier de sortie d'erreur du travail
42	#-Q- sx8brodie # @ error = $(job_name).$(jobid)
43	#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
44	#-Q- sx8brodie # @ notification = error
45	#-Q- sx8brodie # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
46	#-Q- sx8brodie # @ queue
47	#-Q- aix6 #!/bin/ksh
48	#-Q- aix6 #######################
49	#-Q- aix6 ## VARGAS IDRIS ##
50	#-Q- aix6 #######################
51	#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
52	#-Q- aix6 # @ wall_clock_limit = 10:00:00
53	#-Q- aix6 # Nom du travail LoadLeveler
54	#-Q- aix6 # @ job_name = SE
55	#-Q- aix6 # Fichier de sortie standard du travail
56	#-Q- aix6 # @ output = $(job_name).$(jobid)
57	#-Q- aix6 # Fichier de sortie d'erreur du travail
58	#-Q- aix6 # @ error = $(job_name).$(jobid)
59	#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
60	#-Q- aix6 # @ notification = error
61	#-Q- aix6 # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
62	#-Q- aix6 # @ queue
63	#-Q- ulam #!/bin/ksh
64	#-Q- ulam #######################
65	#-Q- ulam ## ULAM IDRIS ##
66	#-Q- ulam #######################
67	#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
68	#-Q- ulam # @ wall_clock_limit = 10:00:00
69	#-Q- ulam # Nom du travail LoadLeveler
70	#-Q- ulam # @ job_name = SE
71	#-Q- ulam # Fichier de sortie standard du travail
72	#-Q- ulam # @ output = $(job_name).$(jobid)
73	#-Q- ulam # Fichier de sortie d'erreur du travail
74	#-Q- ulam # @ error = $(job_name).$(jobid)
75	#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
76	#-Q- ulam # @ notification = error
77	#-Q- ulam # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
78	#-Q- ulam # @ queue
79	#-Q- sx8mercure #!/bin/ksh
80	#-Q- sx8mercure ######################
81	#-Q- sx8mercure ## SX8MERCURE CEA ##
82	#-Q- sx8mercure ######################
83	#-Q- sx8mercure #PBS -N SE # Nom du job
84	#-Q- sx8mercure #PBS -j o # regroupement des stdout et stderr
85	#-Q- sx8mercure #PBS -S /usr/bin/ksh # shell de soumission
86	#-Q- sx8mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
87	#-Q- sx8mercure #PBS -l cputim_job=24:00:00 # Limite temps a 1 heures
88	#-Q- sx8mercure #PBS -q scalaire
89	#-Q- sx8mercure #PBS -r n
90	#-Q- sx9mercure #!/bin/ksh
91	#-Q- sx9mercure ######################
92	#-Q- sx9mercure ## SX9MERCURE CCRT ##
93	#-Q- sx9mercure ######################
94	#-Q- sx9mercure #PBS -N SE # Nom du job
95	#-Q- sx9mercure #PBS -j o # regroupement des stdout et stderr
96	#-Q- sx9mercure #PBS -S /usr/bin/ksh # shell de soumission
97	#-Q- sx9mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
98	#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # Limite temps a 1 heures
99	#-Q- sx9mercure #PBS -q scalaire
100	#-Q- sx9mercure #PBS -r n
101	#-Q- titane #!/bin/ksh
102	#-Q- titane ######################
103	#-Q- titane ## TITANE CEA ##
104	#-Q- titane ######################
105	#-Q- titane #MSUB -r SE # Nom du job
106	#-Q- titane #MSUB -eo
107	#-Q- titane #MSUB -n 1 # Reservation du processus
108	#-Q- titane #MSUB -T 86400 # Limite de temps elapsed du job
109	#-Q- titane #MSUB -q mono
110	#-Q- titane #MSUB -E '-rn'
111	#-Q- lxiv8 ######################
112	#-Q- lxiv8 ## OBELIX LSCE ##
113	#-Q- lxiv8 ######################
114	#-Q- lxiv8 #PBS -N SE
115	#-Q- lxiv8 #PBS -m a
116	#-Q- lxiv8 #PBS -j oe
117	#-Q- lxiv8 #PBS -q medium
118	#-Q- lxiv8 #PBS -o SE.$$
119	#-Q- lxiv8 #PBS -S /bin/ksh
120	#-Q- default #!/bin/ksh
121	#-Q- default ##################
122	#-Q- default ## DEFAULT HOST ##
123	#-Q- default ##################
124
125	#**************************************************************
126	# Author: Sebastien Denvil
127	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
128	# $Revision:: $ Revision of last commit
129	# $Author:: $ Author of last commit
130	# $Date:: $ Date of last commit
131	# IPSL (2006)
132	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
133	#
134	#**************************************************************
135
136	#set -eu
137	#set -vx
138
139	date
140
141	#-Q- sx8brodie export OMP_NUM_THREADS=1
142	#-Q- aix6 export OMP_NUM_THREADS=1
143	#-Q- ulam export OMP_NUM_THREADS=1
144
145	#D- Task type (computing or post-processing)
146	TaskType=post-processing
147
148	########################################################################
149
150	#D- Flag to determine if this job in a standalone mode
151	#D- Default : value from AA_job if any
152	StandAlone=${StandAlone:=true}
153
154	#D- Path to libIGCM
155	#D- Default : value from AA_job if any
156	# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
157	# WARNING you must check MirrorlibIGCM variable in sys library.
158	# WARNING If this variable is true, you must use libIGCM_POST path instead
159	# WARNING of your running libIGCM directory.
160	libIGCM=${libIGCM:=::modipsl::/libIGCM}
161
162	#D- Flag to determine if rebuild process has been performed asynchronously
163	#D- Possible value true or false.
164	#D- If true raw files has allready been patched by rebuild job
165	#D- If false create_ts will patch the raw files
166	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
167	RebuildFrequency=${RebuildFrequency:=true}
168
169	#D- Flag to determine atlas job's output directory
170	#D- Default : value from libIGCM_post.ksh if any
171	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
172
173	#D- Increased verbosity (1, 2, 3)
174	#D- Default : value from AA_job if any
175	Verbosity=${Verbosity:=3}
176
177	#D- Low level debug : to bypass lib test checks and stack construction
178	#D- Default : value from AA_job if any
179	DEBUG_debug=${DEBUG_debug:=false}
180
181	#D- TEMPORARY Flag to determine atmospheric resolution
182	#D- Default : value from atmospheric driver if any
183	RESOL_ATM=${RESOL_ATM:=ALL}
184
185	#D- Flag to determine surface resolution
186	#D- Default : value from surface driver if any
187	RESOL_SRF=ALL
188
189	#D- Flag to determine surface resolution
190	#D- Default : value from surface driver if any
191	RESOL_SBG=ALL
192
193	#D- TEMPORARY Flag to determine ocean resolution
194	#D- Default : value from ocean driver if any
195	RESOL_OCE=${RESOL_OCE:=ORCA2}
196
197	#D- TEMPORARY Flag to determine ice resolution
198	#D- Default : value from ice driver if any
199	RESOL_ICE=${RESOL_ICE:=ORCA2}
200
201	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
202	#D- Default : value from ice driver if any
203	RESOL_MBG=${RESOL_MBG:=ORCA2}
204
205	########################################################################
206
207	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
208	( ${DEBUG_debug} ) && IGCM_debug_Check
209	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
210	( ${DEBUG_debug} ) && IGCM_card_Check
211	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
212	( ${DEBUG_debug} ) && IGCM_date_Check
213	#-------
214	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
215	. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
216
217	########################################################################
218
219	#set -vx
220
221	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
222	IGCM_sys_Cd ${RUN_DIR_PATH}
223
224	# ------------------------------------------------------------------
225	# Test if all was right before proceeding further
226	# ------------------------------------------------------------------
227	IGCM_debug_Verif_Exit_Post
228
229	if [ ${StandAlone} = true ] ; then
230	CARD_DIR=${SUBMIT_DIR}
231	else
232	CARD_DIR=${RUN_DIR_PATH}
233	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
234	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
235	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
236	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
237	fi
238
239	#==================================
240	# First of all
241	#
242	# Read libIGCM compatibility version in config.card
243	# Read UserChoices section
244	# Read Ensemble section
245	# Read Post section
246	# Define all netcdf output directories
247	#==================================
248	IGCM_config_CommonConfiguration ${CARD_DIR}/config.card
249
250	#==================================
251	# Define default value to keep compatibility with previous card: means before changes due to TGCC
252	if [ X${PackDefault} = Xtrue ] ; then
253	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
254	else
255	config_Post_PackFrequency=NONE
256	fi
257
258	#==================================
259	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
260	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
261	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
262	fi
263
264	#==================================
265	# Read ListOfComponents section:
266	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
267
268	#==================================
269	# Read SeasonalFrequency:
270	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
271
272	#
273	# Determine period for seasonnal average. Default : value from AA_job if any
274	#
275	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
276	#
277	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
278	#
279	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
280	#
281	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
282	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
283	an_fin=$( IGCM_date_YearDigit ${an_fin} )
284	#
285	# For each component selected determine which files need post-processing
286	#
287	for comp in ${config_ListOfComponents[*]} ; do
288	#
289	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
290	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
291	#
292	card=${CARD_DIR}/COMP/${compname}.card
293	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
294	#
295	ListFilesName=${compname}_OutputFiles_List
296	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
297	#
298	if [ X${FileName0} != X${NULL_STR} ] ; then
299	#
300	# INITIALISATION
301	#
302	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
303	typeset i=2
304	#
305	until [ $i -gt $NbFiles ]; do
306	#
307	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
308	#
309	if [ X${flag_post} != XNONE ] ; then
310	#
311	# Check if seasonal is required for this file
312	#
313	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
314	#
315	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
316	#
317	# variable option allready typeset above
318	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
319	if [ ${option} = Seasonal ] ; then
320	FoundSeasonal=true
321	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
322	fi
323	done
324	#
325	if [ ! X${FoundSeasonal} = Xtrue ] ; then
326	eval ${compname}_${flag_post}_Seasonal=ON
327	fi
328	#
329	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
330	(( i=i+3 ))
331	continue
332	fi
333	#
334	# CREATE LIST
335	#
336	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
337	#
338	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
339	#
340	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
341	#
342	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
343	if [ ! $? = 0 ] ; then
344	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
345	if [ ! ${RebuildFrequency} = true ] ; then
346	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
347	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
348	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
349	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
350	if [ Xload${Patch} != X ] ; then
351	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
352	eval load${Patch}=loaded
353	fi
354	done
355	fi
356	fi
357	fi
358	fi
359	(( i=i+3 ))
360	done # until [ $i -eq $NbFiles ]
361	fi
362	done # for comp
363
364	for comp in ${config_ListOfComponents[*]} ; do
365	#
366	eval R_OUT_${comp}=${R_SAVE}/${comp}
367	#
368	i=0
369	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
370	#
371	# Determine in which output can we find file
372	# IE : Output/MO or Output/DA or ...
373	#
374	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
375	case ${FlagDir} in
376	1Y) FreqDir=YE ;;
377	1M) FreqDir=MO ;;
378	5D\|1D) FreqDir=DA ;;
379	HF) FreqDir=HF ;;
380	3H) FreqDir=HF ;;
381	INS) FreqDir=INS ;;
382	esac
383	#
384	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
385	# Do not redoo an existing SE
386	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
387	if [ $? = 0 ] ; then
388	continue
389	fi
390	#
391	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
392	#
393	case ${config_UserChoices_PeriodLength} in
394	# For Yearly simulation :
395	1Y\|1y)
396	(( an = an_deb ))
397	# Initialize array
398	#set -A liste_file_se
399	#set -A liste_file_se_tmp
400	unset liste_file_se
401	unset liste_file_se_tmp
402	while [ ${an} -le ${an_fin} ] ; do
403	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
404	#
405	an4=$(printf "%04i\n" ${an} )
406	#
407	diy=$( IGCM_date_DaysInYear $an4 )
408	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
409	date=${an4}0101_${afin}
410	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
411	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
412	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
413	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
414	(( an = an + 1 ))
415	done
416	#
417	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
418	IGCM_debug_Verif_Exit_Post
419
420	# Apply IGCM_Patch if needed
421	if [ ! ${RebuildFrequency} = true ] ; then
422	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
423	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
424	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
425	IGCM_${Patch} ${file_t}
426	done
427	done
428	fi
429	fi
430
431	# Search of common list of variables to be treated
432	nbfile=0
433	for file_common_var in ${liste_file_se_tmp[*]} ; do
434	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
435	(( nbfile = nbfile + 1 ))
436	done
437	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
438	IGCM_sys_Rm tmpfile_ncra
439
440	# detect time counter : time or time_counter ie unlimited variable
441	var_unlim=$(ncdump -h ${liste_file_se_tmp[1]}\|grep UNLIMITED\|awk '{print $1}')
442
443	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
444	#
445	(( nb = ${mois} - 1 ))
446	#
447	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
448	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
449	done
450	IGCM_sys_Rm ${liste_file_se_tmp[*]}
451	;;
452	# For Monthly simulation :
453	1M\|1m)
454	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
455	#
456	(( nb = ${mois} - 1 ))
457	#
458	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
459	#
460	(( an = an_deb ))
461	# Initialize array
462	#set -A liste_file_se
463	#set -A liste_file_se_tmp
464	unset liste_file_se
465	unset liste_file_se_tmp
466	while [ ${an} -le ${an_fin} ] ; do
467	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
468	#
469	an4=$(printf "%04i\n" ${an} )
470	#
471	date=${an4}${mois}01_${an4}${mois}${moisL}
472	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
473	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
474	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
475	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
476
477	(( an = an + 1 ))
478	done
479	#
480	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
481	IGCM_debug_Verif_Exit_Post
482
483	# Apply IGCM_Patch if needed
484	if [ ! ${RebuildFrequency} = true ] ; then
485	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
486	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
487	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
488	IGCM_${Patch} ${file_t}
489	done
490	done
491	fi
492	fi
493
494	# Search of common list of variables to be treated
495	nbfile=0
496	for file_common_var in ${liste_file_se_tmp[*]} ; do
497	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
498	(( nbfile = nbfile + 1 ))
499	done
500	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
501	IGCM_sys_Rm tmpfile_ncra
502
503	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
504
505	IGCM_sys_Rm ${liste_file_se_tmp[*]}
506	done
507	;;
508	D\|d)
509	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
510	#
511	(( nb = ${mois} - 1 ))
512	#
513	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
514	#
515	(( an = an_deb ))
516	# Initialize array
517	#set -A liste_file_se
518	#set -A liste_file_se_tmp
519	unset liste_file_se
520	unset liste_file_se_tmp
521	while [ ${an} -le ${an_fin} ] ; do
522	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
523	#
524	an4=$(printf "%04i\n" ${an} )
525	#
526	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
527	# for one day
528	if [ ${length_d} -eq 1 ] ; then
529	(( length_d = 0 ))
530	fi
531	(( day = 1 ))
532	while [ ${day} -le ${moisL} ] ; do
533	day2=$(printf "%02i\n" ${day} )
534	day_beg=${an4}${mois}${day2}
535	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
536	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
537	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
538	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
539	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
540	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
541	(( day = day + 1 ))
542	done
543	(( an = an + 1 ))
544	done
545	#
546	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
547	IGCM_debug_Verif_Exit_Post
548
549	# Apply IGCM_Patch if needed
550	if [ ! ${RebuildFrequency} = true ] ; then
551	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
552	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
553	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
554	IGCM_${Patch} ${file_t}
555	done
556	done
557	fi
558	fi
559
560	# Search of common list of variables to be treated
561	nbfile=0
562	for file_common_var in ${liste_file_se_tmp[*]} ; do
563	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
564	(( nbfile = nbfile + 1 ))
565	done
566	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
567	IGCM_sys_Rm tmpfile_ncra
568
569	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
570	IGCM_sys_Rm ${liste_file_se_tmp[*]}
571	done
572	;;
573	esac
574	#
575
576	# Search of common list of variables to be treated
577	nbfile=0
578	for file_common_var in ${OUT_SE[*]} ; do
579	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
580	(( nbfile = nbfile + 1 ))
581	done
582	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
583	IGCM_sys_Rm tmpfile_ncrcat
584
585	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
586	#
587	IGCM_sys_Rm ${OUT_SE[*]}
588	#
589	# detect time counter : time or time_counter ie unlimited variable
590	var_unlim=$(ncdump -h ${RESULT_SE}\|grep UNLIMITED\| cut -d ' ' -f 1)
591
592	# Put correct climatological axis following calendar
593	case ${config_UserChoices_CalendarType} in
594	360d\|360_day)
595	# 360 days
596	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
597	;;
598	noleap\|365_day)
599	# 365 days
600	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
601	;;
602	all_leap\|366_day)
603	# 366 days
604	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
605	;;
606	leap\|gregorian\|standard)
607	# 365.2425 days
608	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
609	;;
610	esac
611	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
612	IGCM_sys_Rm ${RESULT_SE}
613	IGCM_sys_Mv tmp.nc ${RESULT_SE}
614	#
615	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
616	IGCM_sys_Rm ${RESULT_SE}
617	#
618	done
619	done
620
621	# DODS copy
622	for comp in ${config_ListOfComponents[*]} ; do
623	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
624	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
625	done
626
627	YEARS=${an_deb}_${an_fin}
628	DIR_OUT=SE
629
630	export POST_DIR; export SUBMIT_DIR; export libIGCM; export R_INIT; export R_BC; export StandAlone
631	export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ; export RESOL_SBG
632	export R_SAVE; export config_UserChoices_JobName; export config_UserChoices_TagName; export YEARS; export DIR_OUT
633	listVarEnv="POST_DIR,SUBMIT_DIR,libIGCM,R_INIT,R_BC,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,R_SAVE,config_UserChoices_JobName,config_UserChoices_TagName,YEARS,DIR_OUT"
634	export listVarEnv;
635
636	#D-
637	# --------------------------------------------------------------------
638	#D- Test if all was right before ATLAS
639	# --------------------------------------------------------------------
640	IGCM_debug_Verif_Exit_Post
641
642	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
643	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
644	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
645	#
646	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
647	IGCM_sys_QsubPost atlas_ORCHIDEE ;
648	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
649	IGCM_sys_QsubPost atlas_LMDZ ;
650	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
651	IGCM_sys_QsubPost atlas_ORCA_LIM ;
652	Script_Post_Output=additionnal.${PeriodDateEnd} ;
653	IGCM_sys_QsubPost additionnal ;
654	#
655	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
656	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
657	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
658	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] ; then
659	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
660	IGCM_sys_QsubPost atlas_ORCHIDEE ;
661	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
662	IGCM_sys_QsubPost atlas_LMDZ ;
663	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
664	IGCM_sys_QsubPost atlas_ORCA_LIM ;
665	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
666	IGCM_sys_QsubPost atlas_PISCES ;
667	#
668	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
669	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
670	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
671	#
672	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
673	IGCM_sys_QsubPost atlas_LMDZ ;
674	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
675	IGCM_sys_QsubPost atlas_ORCHIDEE ;
676	#
677	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
678	#
679	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
680	IGCM_sys_QsubPost atlas_LMDZ ;
681	#
682	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
683	[ ${config_UserChoices_TagName} = "OL2" ] ; then
684	#
685	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
686	IGCM_sys_QsubPost atlas_ORCHIDEE ;
687	#
688	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\|
689	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] ; then
690	#
691	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
692	IGCM_sys_QsubPost atlas_ORCA_LIM ;
693	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] ; then
694	#
695	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
696	IGCM_sys_QsubPost atlas_ORCA_LIM ;
697	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
698	IGCM_sys_QsubPost atlas_PISCES ;
699	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
700	#
701	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
702	IGCM_sys_QsubPost atlas_PISCES ;
703	#
704	fi
705
706	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
707	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: