Context Navigation

source: trunk/libIGCM/AA_create_se @ 640

Last change on this file since 640 was 640, checked in by mafoipsl, 12 years ago
Add norerun option on SX front-end. Transfer specific files ie config.card COMP ... with IGCM_sys_Get_Master and modify IGCM_sys_Get_Master to allow file transfer. tested on ulam and SX front-end.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 30.0 KB

Line
1	#-Q- cesium #!/bin/ksh
2	#-Q- cesium ######################
3	#-Q- cesium ## CESIUM CEA ##
4	#-Q- cesium ######################
5	#-Q- cesium #MSUB -r SE # Nom du job
6	#-Q- cesium #MSUB -N 1 # Reservation du noeud
7	#-Q- cesium #MSUB -n 1 # Reservation du processus
8	#-Q- cesium #MSUB -T 86400 # Limite de temps elapsed du job
9	#-Q- cesium #MSUB -E "-j o"
10	#-Q- cesium #MSUB -E "-S /bin/ksh"
11	#-Q- curie ######################
12	#-Q- curie ## CURIE TGCC/CEA ##
13	#-Q- curie ######################
14	#-Q- curie #MSUB -r SE # Nom du job
15	#-Q- curie #MSUB -eo
16	#-Q- curie #MSUB -n 1 # Reservation du processus
17	#-Q- curie #MSUB -T 86400 # Limite de temps elapsed du job
18	#-Q- platine #!/usr/bin/ksh
19	#-Q- platine ###################
20	#-Q- platine ## PLATINE CEA ##
21	#-Q- platine ###################
22	#-Q- platine #BSUB -J SE # Nom du job
23	#-Q- platine #BSUB -N # message a la fin du job
24	#-Q- platine #BSUB -n 1 # reservation des processeurs pour le job
25	#-Q- platine #BSUB -W 1:00 # Limite temps
26	#-Q- platine #BSUB -q post # Passage en queue post
27	#-Q- platine #BSUB -L /bin/ksh
28	#-Q- sx8brodie #!/bin/ksh
29	#-Q- sx8brodie #######################
30	#-Q- sx8brodie ## SX8BRODIE IDRIS ##
31	#-Q- sx8brodie #######################
32	#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
33	#-Q- sx8brodie # @ wall_clock_limit = 10:00:00
34	#-Q- sx8brodie # Nom du travail LoadLeveler
35	#-Q- sx8brodie # @ job_name = SE
36	#-Q- sx8brodie # Fichier de sortie standard du travail
37	#-Q- sx8brodie # @ output = $(job_name).$(jobid)
38	#-Q- sx8brodie # Fichier de sortie d'erreur du travail
39	#-Q- sx8brodie # @ error = $(job_name).$(jobid)
40	#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
41	#-Q- sx8brodie # @ notification = error
42	#-Q- sx8brodie # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
43	#-Q- sx8brodie # @ queue
44	#-Q- aix6 #!/bin/ksh
45	#-Q- aix6 #######################
46	#-Q- aix6 ## VARGAS IDRIS ##
47	#-Q- aix6 #######################
48	#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
49	#-Q- aix6 # @ wall_clock_limit = 10:00:00
50	#-Q- aix6 # Nom du travail LoadLeveler
51	#-Q- aix6 # @ job_name = SE
52	#-Q- aix6 # Fichier de sortie standard du travail
53	#-Q- aix6 # @ output = $(job_name).$(jobid)
54	#-Q- aix6 # Fichier de sortie d'erreur du travail
55	#-Q- aix6 # @ error = $(job_name).$(jobid)
56	#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
57	#-Q- aix6 # @ notification = error
58	#-Q- aix6 # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
59	#-Q- aix6 # @ queue
60	#-Q- ulam #!/bin/ksh
61	#-Q- ulam #######################
62	#-Q- ulam ## ULAM IDRIS ##
63	#-Q- ulam #######################
64	#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
65	#-Q- ulam # @ wall_clock_limit = 10:00:00
66	#-Q- ulam # Nom du travail LoadLeveler
67	#-Q- ulam # @ job_name = SE
68	#-Q- ulam # Fichier de sortie standard du travail
69	#-Q- ulam # @ output = $(job_name).$(jobid)
70	#-Q- ulam # Fichier de sortie d'erreur du travail
71	#-Q- ulam # @ error = $(job_name).$(jobid)
72	#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
73	#-Q- ulam # @ notification = error
74	#-Q- ulam # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
75	#-Q- ulam # @ queue
76	#-Q- sx8mercure #!/bin/ksh
77	#-Q- sx8mercure ######################
78	#-Q- sx8mercure ## SX8MERCURE CEA ##
79	#-Q- sx8mercure ######################
80	#-Q- sx8mercure #PBS -N SE # Nom du job
81	#-Q- sx8mercure #PBS -j o # regroupement des stdout et stderr
82	#-Q- sx8mercure #PBS -S /usr/bin/ksh # shell de soumission
83	#-Q- sx8mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
84	#-Q- sx8mercure #PBS -l cputim_job=24:00:00 # Limite temps a 1 heures
85	#-Q- sx8mercure #PBS -q scalaire
86	#-Q- sx8mercure #PBS -r n
87	#-Q- sx9mercure #!/bin/ksh
88	#-Q- sx9mercure ######################
89	#-Q- sx9mercure ## SX9MERCURE CCRT ##
90	#-Q- sx9mercure ######################
91	#-Q- sx9mercure #PBS -N SE # Nom du job
92	#-Q- sx9mercure #PBS -j o # regroupement des stdout et stderr
93	#-Q- sx9mercure #PBS -S /usr/bin/ksh # shell de soumission
94	#-Q- sx9mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
95	#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # Limite temps a 1 heures
96	#-Q- sx9mercure #PBS -q scalaire
97	#-Q- sx9mercure #PBS -r n
98	#-Q- titane #!/bin/ksh
99	#-Q- titane ######################
100	#-Q- titane ## TITANE CEA ##
101	#-Q- titane ######################
102	#-Q- titane #MSUB -r SE # Nom du job
103	#-Q- titane #MSUB -eo
104	#-Q- titane #MSUB -n 1 # Reservation du processus
105	#-Q- titane #MSUB -T 86400 # Limite de temps elapsed du job
106	#-Q- titane #MSUB -q mono
107	#-Q- lxiv8 ######################
108	#-Q- lxiv8 ## OBELIX LSCE ##
109	#-Q- lxiv8 ######################
110	#-Q- lxiv8 #PBS -N SE
111	#-Q- lxiv8 #PBS -m a
112	#-Q- lxiv8 #PBS -j oe
113	#-Q- lxiv8 #PBS -q medium
114	#-Q- lxiv8 #PBS -o SE.$$
115	#-Q- lxiv8 #PBS -S /bin/ksh
116	#-Q- default #!/bin/ksh
117	#-Q- default ##################
118	#-Q- default ## DEFAULT HOST ##
119	#-Q- default ##################
120
121	#**************************************************************
122	# Author: Sebastien Denvil
123	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
124	# $Revision:: $ Revision of last commit
125	# $Author:: $ Author of last commit
126	# $Date:: $ Date of last commit
127	# IPSL (2006)
128	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
129	#
130	#**************************************************************
131
132	#set -eu
133	#set -vx
134
135	date
136
137	#-Q- sx8brodie export OMP_NUM_THREADS=1
138	#-Q- aix6 export OMP_NUM_THREADS=1
139	#-Q- ulam export OMP_NUM_THREADS=1
140
141	########################################################################
142
143	#D- Flag to determine if this job in a standalone mode
144	#D- Default : value from AA_job if any
145	StandAlone=${StandAlone:=true}
146
147	#D- Path to libIGCM
148	#D- Default : value from AA_job if any
149	# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
150	# WARNING you must check MirrorlibIGCM variable in sys library.
151	# WARNING If this variable is true, you must use libIGCM_POST path instead
152	# WARNING of your running libIGCM directory.
153	libIGCM=${libIGCM:=::modipsl::/libIGCM}
154
155	#D- Flag to determine if rebuild process has been performed asynchronously
156	#D- Possible value true or false.
157	#D- If true raw files has allready been patched by rebuild job
158	#D- If false create_ts will patch the raw files
159	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
160	RebuildFrequency=${RebuildFrequency:=true}
161
162	#D- Flag to determine atlas job's output directory
163	#D- Default : value from libIGCM_post.ksh if any
164	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
165
166	#D- Increased verbosity (1, 2, 3)
167	#D- Default : value from AA_job if any
168	Verbosity=${Verbosity:=3}
169
170	#D- Low level debug : to bypass lib test checks and stack construction
171	#D- Default : value from AA_job if any
172	DEBUG_debug=${DEBUG_debug:=false}
173
174	#D- TEMPORARY Flag to determine atmospheric resolution
175	#D- Default : value from atmospheric driver if any
176	RESOL_ATM=${RESOL_ATM:=ALL}
177
178	#D- Flag to determine surface resolution
179	#D- Default : value from surface driver if any
180	RESOL_SRF=ALL
181
182	#D- Flag to determine surface resolution
183	#D- Default : value from surface driver if any
184	RESOL_SBG=ALL
185
186	#D- TEMPORARY Flag to determine ocean resolution
187	#D- Default : value from ocean driver if any
188	RESOL_OCE=${RESOL_OCE:=ORCA2}
189
190	#D- TEMPORARY Flag to determine ice resolution
191	#D- Default : value from ice driver if any
192	RESOL_ICE=${RESOL_ICE:=ORCA2}
193
194	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
195	#D- Default : value from ice driver if any
196	RESOL_MBG=${RESOL_MBG:=ORCA2}
197
198	########################################################################
199
200	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
201	( ${DEBUG_debug} ) && IGCM_debug_Check
202	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
203	( ${DEBUG_debug} ) && IGCM_card_Check
204	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
205	( ${DEBUG_debug} ) && IGCM_date_Check
206	#-------
207	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
208
209	########################################################################
210
211	#set -vx
212
213	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
214	IGCM_sys_Cd ${RUN_DIR_PATH}
215
216	# ------------------------------------------------------------------
217	# Test if all was right before proceeding further
218	# ------------------------------------------------------------------
219	IGCM_debug_Verif_Exit_Post
220
221	if [ ${StandAlone} = true ] ; then
222	CARD_DIR=${SUBMIT_DIR}
223	else
224	CARD_DIR=${RUN_DIR_PATH}/$( basename ${SUBMIT_DIR} )
225	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
226	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
227	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
228	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
229	fi
230
231	#==================================
232	# First of all
233	#
234	typeset option
235
236	#==================================
237	# Read UserChoices section:
238	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card UserChoices
239	echo
240	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
241	for option in ${config_UserChoices[*]} ; do
242	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
243	eval auxprint=\${config_UserChoices_${option}}
244	IGCM_debug_Print 3 "${option} : ${auxprint}"
245	done
246	#
247	echo
248
249	#==================================
250	# Read Post section:
251	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card Post
252	echo
253	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
254	for option in ${config_Post[*]} ; do
255	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post ${option}
256	eval auxprint=\${config_Post_${option}}
257	IGCM_debug_Print 3 "${option} : ${auxprint}"
258	done
259	#
260	echo
261
262	#==================================
263	# Define default value to keep compatibility with previous card: means before changes due to TGCC
264	if [ X${PackDefault} = Xtrue ] ; then
265	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
266	else
267	config_Post_PackFrequency=NONE
268	fi
269
270	#==================================
271	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
272	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
273	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
274	fi
275
276	#==================================
277	#R_SAVE : Job output directory
278	#R_BUFR : Job output buffered directory
279	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
280	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
281	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
282	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
283	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
284	else
285	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
286	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
287	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
288	fi
289	#
290	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
291	#
292	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
293	#
294	# Determine period for seasonnal average. Default : value from AA_job if any
295	#
296	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
297	#
298	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
299	#
300	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
301	#
302	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
303	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
304	an_fin=$( IGCM_date_YearDigit ${an_fin} )
305	#
306	# For each component selected determine which files need post-processing
307	#
308	for comp in ${config_ListOfComponents[*]} ; do
309	#
310	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
311	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
312	#
313	card=${CARD_DIR}/COMP/${compname}.card
314	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
315	#
316	ListFilesName=${compname}_OutputFiles_List
317	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
318	#
319	if [ X${FileName0} != X${NULL_STR} ] ; then
320	#
321	# INITIALISATION
322	#
323	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
324	typeset i=2
325	#
326	until [ $i -gt $NbFiles ]; do
327	#
328	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
329	#
330	if [ X${flag_post} != XNONE ] ; then
331	#
332	# Check if seasonal is required for this file
333	#
334	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
335	#
336	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
337	#
338	# variable option allready typeset above
339	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
340	if [ ${option} = Seasonal ] ; then
341	FoundSeasonal=true
342	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
343	fi
344	done
345	#
346	if [ ! X${FoundSeasonal} = Xtrue ] ; then
347	eval ${compname}_${flag_post}_Seasonal=ON
348	fi
349	#
350	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
351	(( i=i+3 ))
352	continue
353	fi
354	#
355	# CREATE LIST
356	#
357	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
358	#
359	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
360	#
361	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
362	#
363	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
364	if [ ! $? = 0 ] ; then
365	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
366	if [ ! ${RebuildFrequency} = true ] ; then
367	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
368	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
369	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
370	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
371	if [ Xload${Patch} != X ] ; then
372	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
373	eval load${Patch}=loaded
374	fi
375	done
376	fi
377	fi
378	fi
379	fi
380	(( i=i+3 ))
381	done # until [ $i -eq $NbFiles ]
382	fi
383	done # for comp
384
385	for comp in ${config_ListOfComponents[*]} ; do
386	#
387	eval R_OUT_${comp}=${R_SAVE}/${comp}
388	#
389	i=0
390	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
391	#
392	# Determine in which output can we find file
393	# IE : Output/MO or Output/DA or ...
394	#
395	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
396	case ${FlagDir} in
397	1Y) FreqDir=YE ;;
398	1M) FreqDir=MO ;;
399	5D\|1D) FreqDir=DA ;;
400	HF) FreqDir=HF ;;
401	3H) FreqDir=HF ;;
402	INS) FreqDir=INS ;;
403	esac
404	#
405	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
406	# Do not redoo an existing SE
407	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
408	if [ $? = 0 ] ; then
409	continue
410	fi
411	#
412	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
413	#
414	case ${config_UserChoices_PeriodLength} in
415	# For Yearly simulation :
416	1Y\|1y)
417	(( an = an_deb ))
418	# Initialize array
419	#set -A liste_file_se
420	#set -A liste_file_se_tmp
421	unset liste_file_se
422	unset liste_file_se_tmp
423	while [ ${an} -le ${an_fin} ] ; do
424	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
425	#
426	an4=$(printf "%04i\n" ${an} )
427	#
428	diy=$( IGCM_date_DaysInYear $an4 )
429	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
430	date=${an4}0101_${afin}
431	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
432	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
433	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
434	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
435	(( an = an + 1 ))
436	done
437	#
438	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
439	IGCM_debug_Verif_Exit_Post
440
441	# Apply IGCM_Patch if needed
442	if [ ! ${RebuildFrequency} = true ] ; then
443	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
444	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
445	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
446	IGCM_${Patch} ${file_t}
447	done
448	done
449	fi
450	fi
451
452	# Search of common list of variables to be treated
453	nbfile=0
454	for file_common_var in ${liste_file_se_tmp[*]} ; do
455	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
456	(( nbfile = nbfile + 1 ))
457	done
458	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
459	IGCM_sys_Rm tmpfile_ncra
460
461	# detect time counter : time or time_counter ie unlimited variable
462	var_unlim=$(ncdump -h ${liste_file_se_tmp[1]}\|grep UNLIMITED\| cut -d ' ' -f 1)
463
464	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
465	#
466	(( nb = ${mois} - 1 ))
467	#
468	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
469	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
470	done
471	IGCM_sys_Rm ${liste_file_se_tmp[*]}
472	;;
473	# For Monthly simulation :
474	1M\|1m)
475	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
476	#
477	(( nb = ${mois} - 1 ))
478	#
479	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
480	#
481	(( an = an_deb ))
482	# Initialize array
483	#set -A liste_file_se
484	#set -A liste_file_se_tmp
485	unset liste_file_se
486	unset liste_file_se_tmp
487	while [ ${an} -le ${an_fin} ] ; do
488	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
489	#
490	an4=$(printf "%04i\n" ${an} )
491	#
492	date=${an4}${mois}01_${an4}${mois}${moisL}
493	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
494	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
495	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
496	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
497
498	(( an = an + 1 ))
499	done
500	#
501	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
502	IGCM_debug_Verif_Exit_Post
503
504	# Apply IGCM_Patch if needed
505	if [ ! ${RebuildFrequency} = true ] ; then
506	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
507	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
508	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
509	IGCM_${Patch} ${file_t}
510	done
511	done
512	fi
513	fi
514
515	# Search of common list of variables to be treated
516	nbfile=0
517	for file_common_var in ${liste_file_se_tmp[*]} ; do
518	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
519	(( nbfile = nbfile + 1 ))
520	done
521	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
522	IGCM_sys_Rm tmpfile_ncra
523
524	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
525
526	IGCM_sys_Rm ${liste_file_se_tmp[*]}
527	done
528	;;
529	D\|d)
530	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
531	#
532	(( nb = ${mois} - 1 ))
533	#
534	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
535	#
536	(( an = an_deb ))
537	# Initialize array
538	#set -A liste_file_se
539	#set -A liste_file_se_tmp
540	unset liste_file_se
541	unset liste_file_se_tmp
542	while [ ${an} -le ${an_fin} ] ; do
543	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
544	#
545	an4=$(printf "%04i\n" ${an} )
546	#
547	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
548	# for one day
549	if [ ${length_d} -eq 1 ] ; then
550	(( length_d = 0 ))
551	fi
552	(( day = 1 ))
553	while [ ${day} -le ${moisL} ] ; do
554	day2=$(printf "%02i\n" ${day} )
555	day_beg=${an4}${mois}${day2}
556	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
557	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
558	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
559	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
560	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
561	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
562	(( day = day + 1 ))
563	done
564	(( an = an + 1 ))
565	done
566	#
567	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
568	IGCM_debug_Verif_Exit_Post
569
570	# Apply IGCM_Patch if needed
571	if [ ! ${RebuildFrequency} = true ] ; then
572	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
573	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
574	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
575	IGCM_${Patch} ${file_t}
576	done
577	done
578	fi
579	fi
580
581	# Search of common list of variables to be treated
582	nbfile=0
583	for file_common_var in ${liste_file_se_tmp[*]} ; do
584	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
585	(( nbfile = nbfile + 1 ))
586	done
587	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
588	IGCM_sys_Rm tmpfile_ncra
589
590	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
591	IGCM_sys_Rm ${liste_file_se_tmp[*]}
592	done
593	;;
594	esac
595	#
596
597	# Search of common list of variables to be treated
598	nbfile=0
599	for file_common_var in ${OUT_SE[*]} ; do
600	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
601	(( nbfile = nbfile + 1 ))
602	done
603	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
604	IGCM_sys_Rm tmpfile_ncrcat
605
606	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
607	#
608	IGCM_sys_Rm ${OUT_SE[*]}
609	#
610	# detect time counter : time or time_counter ie unlimited variable
611	var_unlim=$(ncdump -h ${RESULT_SE}\|grep UNLIMITED\| cut -d ' ' -f 1)
612
613	# Put correct climatological axis following calendar
614	case ${config_UserChoices_CalendarType} in
615	360d\|360_day)
616	# 360 days
617	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
618	;;
619	noleap\|365_day)
620	# 365 days
621	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
622	;;
623	all_leap\|366_day)
624	# 366 days
625	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
626	;;
627	leap\|gregorian\|standard)
628	# 365.2425 days
629	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
630	;;
631	esac
632	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
633	IGCM_sys_Rm ${RESULT_SE}
634	IGCM_sys_Mv tmp.nc ${RESULT_SE}
635	#
636	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
637	IGCM_sys_Rm ${RESULT_SE}
638	#
639	done
640	done
641
642	# DODS copy
643	for comp in ${config_ListOfComponents[*]} ; do
644	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
645	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
646	done
647
648	YEARS=${an_deb}_${an_fin}
649	DIR_OUT=SE
650
651	export POST_DIR; export SUBMIT_DIR; export libIGCM; export R_INIT; export R_BC; export StandAlone
652	export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ; export RESOL_SBG
653	export R_SAVE; export config_UserChoices_JobName; export config_UserChoices_TagName; export YEARS; export DIR_OUT
654	listVarEnv="POST_DIR,SUBMIT_DIR,libIGCM,R_INIT,R_BC,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,R_SAVE,config_UserChoices_JobName,config_UserChoices_TagName,YEARS,DIR_OUT"
655	export listVarEnv;
656
657	#D-
658	# --------------------------------------------------------------------
659	#D- Test if all was right before ATLAS
660	# --------------------------------------------------------------------
661	IGCM_debug_Verif_Exit_Post
662
663	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
664	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
665	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
666	#
667	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
668	IGCM_sys_QsubPost atlas_ORCHIDEE ;
669	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
670	IGCM_sys_QsubPost atlas_LMDZ ;
671	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
672	IGCM_sys_QsubPost atlas_ORCA_LIM ;
673	Script_Post_Output=additionnal.${PeriodDateEnd} ;
674	IGCM_sys_QsubPost additionnal ;
675	#
676	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
677	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
678	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
679	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] ; then
680	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
681	IGCM_sys_QsubPost atlas_ORCHIDEE ;
682	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
683	IGCM_sys_QsubPost atlas_LMDZ ;
684	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
685	IGCM_sys_QsubPost atlas_ORCA_LIM ;
686	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
687	IGCM_sys_QsubPost atlas_PISCES ;
688	#
689	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
690	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
691	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
692	#
693	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
694	IGCM_sys_QsubPost atlas_LMDZ ;
695	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
696	IGCM_sys_QsubPost atlas_ORCHIDEE ;
697	#
698	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
699	#
700	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
701	IGCM_sys_QsubPost atlas_LMDZ ;
702	#
703	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
704	[ ${config_UserChoices_TagName} = "OL2" ] ; then
705	#
706	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
707	IGCM_sys_QsubPost atlas_ORCHIDEE ;
708	#
709	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\|
710	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] ; then
711	#
712	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
713	IGCM_sys_QsubPost atlas_ORCA_LIM ;
714	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] ; then
715	#
716	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
717	IGCM_sys_QsubPost atlas_ORCA_LIM ;
718	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
719	IGCM_sys_QsubPost atlas_PISCES ;
720	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
721	#
722	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
723	IGCM_sys_QsubPost atlas_PISCES ;
724	#
725	fi
726
727	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
728	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: