Context Navigation

source: tags/libIGCM_v2.0_beta2/AA_create_se @ 1312

Last change on this file since 1312 was 658, checked in by labetoulle, 12 years ago
Add Ensemble functionality to libIGCM - First step : modify output directories according to an option in section [Ensemble] in config.card (modification of config.card yet to be done).
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 31.7 KB

Line
1	#-Q- cesium #!/bin/ksh
2	#-Q- cesium ######################
3	#-Q- cesium ## CESIUM CEA ##
4	#-Q- cesium ######################
5	#-Q- cesium #MSUB -r SE # Nom du job
6	#-Q- cesium #MSUB -N 1 # Reservation du noeud
7	#-Q- cesium #MSUB -n 1 # Reservation du processus
8	#-Q- cesium #MSUB -T 86400 # Limite de temps elapsed du job
9	#-Q- cesium #MSUB -E "-j o"
10	#-Q- cesium #MSUB -E "-S /bin/ksh"
11	#-Q- curie ######################
12	#-Q- curie ## CURIE TGCC/CEA ##
13	#-Q- curie ######################
14	#-Q- curie #MSUB -r SE # Nom du job
15	#-Q- curie #MSUB -eo
16	#-Q- curie #MSUB -n 1 # Reservation du processus
17	#-Q- curie #MSUB -T 86400 # Limite de temps elapsed du job
18	#-Q- curie #MSUB -E '--no-requeue'
19	#-Q- platine #!/usr/bin/ksh
20	#-Q- platine ###################
21	#-Q- platine ## PLATINE CEA ##
22	#-Q- platine ###################
23	#-Q- platine #BSUB -J SE # Nom du job
24	#-Q- platine #BSUB -N # message a la fin du job
25	#-Q- platine #BSUB -n 1 # reservation des processeurs pour le job
26	#-Q- platine #BSUB -W 1:00 # Limite temps
27	#-Q- platine #BSUB -q post # Passage en queue post
28	#-Q- platine #BSUB -L /bin/ksh
29	#-Q- sx8brodie #!/bin/ksh
30	#-Q- sx8brodie #######################
31	#-Q- sx8brodie ## SX8BRODIE IDRIS ##
32	#-Q- sx8brodie #######################
33	#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
34	#-Q- sx8brodie # @ wall_clock_limit = 10:00:00
35	#-Q- sx8brodie # Nom du travail LoadLeveler
36	#-Q- sx8brodie # @ job_name = SE
37	#-Q- sx8brodie # Fichier de sortie standard du travail
38	#-Q- sx8brodie # @ output = $(job_name).$(jobid)
39	#-Q- sx8brodie # Fichier de sortie d'erreur du travail
40	#-Q- sx8brodie # @ error = $(job_name).$(jobid)
41	#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
42	#-Q- sx8brodie # @ notification = error
43	#-Q- sx8brodie # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
44	#-Q- sx8brodie # @ queue
45	#-Q- aix6 #!/bin/ksh
46	#-Q- aix6 #######################
47	#-Q- aix6 ## VARGAS IDRIS ##
48	#-Q- aix6 #######################
49	#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
50	#-Q- aix6 # @ wall_clock_limit = 10:00:00
51	#-Q- aix6 # Nom du travail LoadLeveler
52	#-Q- aix6 # @ job_name = SE
53	#-Q- aix6 # Fichier de sortie standard du travail
54	#-Q- aix6 # @ output = $(job_name).$(jobid)
55	#-Q- aix6 # Fichier de sortie d'erreur du travail
56	#-Q- aix6 # @ error = $(job_name).$(jobid)
57	#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
58	#-Q- aix6 # @ notification = error
59	#-Q- aix6 # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
60	#-Q- aix6 # @ queue
61	#-Q- ulam #!/bin/ksh
62	#-Q- ulam #######################
63	#-Q- ulam ## ULAM IDRIS ##
64	#-Q- ulam #######################
65	#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
66	#-Q- ulam # @ wall_clock_limit = 10:00:00
67	#-Q- ulam # Nom du travail LoadLeveler
68	#-Q- ulam # @ job_name = SE
69	#-Q- ulam # Fichier de sortie standard du travail
70	#-Q- ulam # @ output = $(job_name).$(jobid)
71	#-Q- ulam # Fichier de sortie d'erreur du travail
72	#-Q- ulam # @ error = $(job_name).$(jobid)
73	#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
74	#-Q- ulam # @ notification = error
75	#-Q- ulam # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
76	#-Q- ulam # @ queue
77	#-Q- sx8mercure #!/bin/ksh
78	#-Q- sx8mercure ######################
79	#-Q- sx8mercure ## SX8MERCURE CEA ##
80	#-Q- sx8mercure ######################
81	#-Q- sx8mercure #PBS -N SE # Nom du job
82	#-Q- sx8mercure #PBS -j o # regroupement des stdout et stderr
83	#-Q- sx8mercure #PBS -S /usr/bin/ksh # shell de soumission
84	#-Q- sx8mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
85	#-Q- sx8mercure #PBS -l cputim_job=24:00:00 # Limite temps a 1 heures
86	#-Q- sx8mercure #PBS -q scalaire
87	#-Q- sx8mercure #PBS -r n
88	#-Q- sx9mercure #!/bin/ksh
89	#-Q- sx9mercure ######################
90	#-Q- sx9mercure ## SX9MERCURE CCRT ##
91	#-Q- sx9mercure ######################
92	#-Q- sx9mercure #PBS -N SE # Nom du job
93	#-Q- sx9mercure #PBS -j o # regroupement des stdout et stderr
94	#-Q- sx9mercure #PBS -S /usr/bin/ksh # shell de soumission
95	#-Q- sx9mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
96	#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # Limite temps a 1 heures
97	#-Q- sx9mercure #PBS -q scalaire
98	#-Q- sx9mercure #PBS -r n
99	#-Q- titane #!/bin/ksh
100	#-Q- titane ######################
101	#-Q- titane ## TITANE CEA ##
102	#-Q- titane ######################
103	#-Q- titane #MSUB -r SE # Nom du job
104	#-Q- titane #MSUB -eo
105	#-Q- titane #MSUB -n 1 # Reservation du processus
106	#-Q- titane #MSUB -T 86400 # Limite de temps elapsed du job
107	#-Q- titane #MSUB -q mono
108	#-Q- titane #MSUB -E '-rn'
109	#-Q- lxiv8 ######################
110	#-Q- lxiv8 ## OBELIX LSCE ##
111	#-Q- lxiv8 ######################
112	#-Q- lxiv8 #PBS -N SE
113	#-Q- lxiv8 #PBS -m a
114	#-Q- lxiv8 #PBS -j oe
115	#-Q- lxiv8 #PBS -q medium
116	#-Q- lxiv8 #PBS -o SE.$$
117	#-Q- lxiv8 #PBS -S /bin/ksh
118	#-Q- default #!/bin/ksh
119	#-Q- default ##################
120	#-Q- default ## DEFAULT HOST ##
121	#-Q- default ##################
122
123	#**************************************************************
124	# Author: Sebastien Denvil
125	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
126	# $Revision:: $ Revision of last commit
127	# $Author:: $ Author of last commit
128	# $Date:: $ Date of last commit
129	# IPSL (2006)
130	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
131	#
132	#**************************************************************
133
134	#set -eu
135	#set -vx
136
137	date
138
139	#-Q- sx8brodie export OMP_NUM_THREADS=1
140	#-Q- aix6 export OMP_NUM_THREADS=1
141	#-Q- ulam export OMP_NUM_THREADS=1
142
143	########################################################################
144
145	#D- Flag to determine if this job in a standalone mode
146	#D- Default : value from AA_job if any
147	StandAlone=${StandAlone:=true}
148
149	#D- Path to libIGCM
150	#D- Default : value from AA_job if any
151	# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
152	# WARNING you must check MirrorlibIGCM variable in sys library.
153	# WARNING If this variable is true, you must use libIGCM_POST path instead
154	# WARNING of your running libIGCM directory.
155	libIGCM=${libIGCM:=::modipsl::/libIGCM}
156
157	#D- Flag to determine if rebuild process has been performed asynchronously
158	#D- Possible value true or false.
159	#D- If true raw files has allready been patched by rebuild job
160	#D- If false create_ts will patch the raw files
161	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
162	RebuildFrequency=${RebuildFrequency:=true}
163
164	#D- Flag to determine atlas job's output directory
165	#D- Default : value from libIGCM_post.ksh if any
166	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
167
168	#D- Increased verbosity (1, 2, 3)
169	#D- Default : value from AA_job if any
170	Verbosity=${Verbosity:=3}
171
172	#D- Low level debug : to bypass lib test checks and stack construction
173	#D- Default : value from AA_job if any
174	DEBUG_debug=${DEBUG_debug:=false}
175
176	#D- TEMPORARY Flag to determine atmospheric resolution
177	#D- Default : value from atmospheric driver if any
178	RESOL_ATM=${RESOL_ATM:=ALL}
179
180	#D- Flag to determine surface resolution
181	#D- Default : value from surface driver if any
182	RESOL_SRF=ALL
183
184	#D- Flag to determine surface resolution
185	#D- Default : value from surface driver if any
186	RESOL_SBG=ALL
187
188	#D- TEMPORARY Flag to determine ocean resolution
189	#D- Default : value from ocean driver if any
190	RESOL_OCE=${RESOL_OCE:=ORCA2}
191
192	#D- TEMPORARY Flag to determine ice resolution
193	#D- Default : value from ice driver if any
194	RESOL_ICE=${RESOL_ICE:=ORCA2}
195
196	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
197	#D- Default : value from ice driver if any
198	RESOL_MBG=${RESOL_MBG:=ORCA2}
199
200	########################################################################
201
202	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
203	( ${DEBUG_debug} ) && IGCM_debug_Check
204	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
205	( ${DEBUG_debug} ) && IGCM_card_Check
206	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
207	( ${DEBUG_debug} ) && IGCM_date_Check
208	#-------
209	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
210
211	########################################################################
212
213	#set -vx
214
215	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
216	IGCM_sys_Cd ${RUN_DIR_PATH}
217
218	# ------------------------------------------------------------------
219	# Test if all was right before proceeding further
220	# ------------------------------------------------------------------
221	IGCM_debug_Verif_Exit_Post
222
223	if [ ${StandAlone} = true ] ; then
224	CARD_DIR=${SUBMIT_DIR}
225	else
226	CARD_DIR=${RUN_DIR_PATH}
227	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
228	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
229	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
230	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
231	fi
232
233	#==================================
234	# First of all
235	#
236	typeset option
237
238	#==================================
239	# Read UserChoices section:
240	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card UserChoices
241	echo
242	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
243	for option in ${config_UserChoices[*]} ; do
244	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
245	eval auxprint=\${config_UserChoices_${option}}
246	IGCM_debug_Print 3 "${option} : ${auxprint}"
247	done
248	#
249	echo
250
251	#==================================
252	# Read Ensemble section:
253	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card Ensemble
254	echo
255	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
256	for option in ${config_Ensemble[*]} ; do
257	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Ensemble ${option}
258	eval auxprint=\${config_Ensemble_${option}}
259	IGCM_debug_Print 3 "${option} : ${auxprint}"
260	done
261	#
262	echo
263
264	#==================================
265	# Read Post section:
266	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card Post
267	echo
268	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
269	for option in ${config_Post[*]} ; do
270	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post ${option}
271	eval auxprint=\${config_Post_${option}}
272	IGCM_debug_Print 3 "${option} : ${auxprint}"
273	done
274	#
275	echo
276
277	#==================================
278	# Define default value to keep compatibility with previous card: means before changes due to TGCC
279	if [ X${PackDefault} = Xtrue ] ; then
280	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
281	else
282	config_Post_PackFrequency=NONE
283	fi
284
285	#==================================
286	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
287	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
288	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
289	fi
290
291	#==================================
292	#R_SAVE : Job output directory
293	#R_BUFR : Job output buffered directory
294	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
295	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
296	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
297	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
298	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
299	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
300	else
301	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
302	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
303	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
304	fi
305	else
306	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
307	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
308	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
309	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
310	else
311	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
312	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
313	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
314	fi
315	fi
316	#
317	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
318	#
319	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
320	#
321	# Determine period for seasonnal average. Default : value from AA_job if any
322	#
323	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
324	#
325	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
326	#
327	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
328	#
329	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
330	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
331	an_fin=$( IGCM_date_YearDigit ${an_fin} )
332	#
333	# For each component selected determine which files need post-processing
334	#
335	for comp in ${config_ListOfComponents[*]} ; do
336	#
337	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
338	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
339	#
340	card=${CARD_DIR}/COMP/${compname}.card
341	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
342	#
343	ListFilesName=${compname}_OutputFiles_List
344	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
345	#
346	if [ X${FileName0} != X${NULL_STR} ] ; then
347	#
348	# INITIALISATION
349	#
350	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
351	typeset i=2
352	#
353	until [ $i -gt $NbFiles ]; do
354	#
355	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
356	#
357	if [ X${flag_post} != XNONE ] ; then
358	#
359	# Check if seasonal is required for this file
360	#
361	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
362	#
363	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
364	#
365	# variable option allready typeset above
366	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
367	if [ ${option} = Seasonal ] ; then
368	FoundSeasonal=true
369	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
370	fi
371	done
372	#
373	if [ ! X${FoundSeasonal} = Xtrue ] ; then
374	eval ${compname}_${flag_post}_Seasonal=ON
375	fi
376	#
377	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
378	(( i=i+3 ))
379	continue
380	fi
381	#
382	# CREATE LIST
383	#
384	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
385	#
386	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
387	#
388	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
389	#
390	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
391	if [ ! $? = 0 ] ; then
392	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
393	if [ ! ${RebuildFrequency} = true ] ; then
394	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
395	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
396	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
397	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
398	if [ Xload${Patch} != X ] ; then
399	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
400	eval load${Patch}=loaded
401	fi
402	done
403	fi
404	fi
405	fi
406	fi
407	(( i=i+3 ))
408	done # until [ $i -eq $NbFiles ]
409	fi
410	done # for comp
411
412	for comp in ${config_ListOfComponents[*]} ; do
413	#
414	eval R_OUT_${comp}=${R_SAVE}/${comp}
415	#
416	i=0
417	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
418	#
419	# Determine in which output can we find file
420	# IE : Output/MO or Output/DA or ...
421	#
422	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
423	case ${FlagDir} in
424	1Y) FreqDir=YE ;;
425	1M) FreqDir=MO ;;
426	5D\|1D) FreqDir=DA ;;
427	HF) FreqDir=HF ;;
428	3H) FreqDir=HF ;;
429	INS) FreqDir=INS ;;
430	esac
431	#
432	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
433	# Do not redoo an existing SE
434	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
435	if [ $? = 0 ] ; then
436	continue
437	fi
438	#
439	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
440	#
441	case ${config_UserChoices_PeriodLength} in
442	# For Yearly simulation :
443	1Y\|1y)
444	(( an = an_deb ))
445	# Initialize array
446	#set -A liste_file_se
447	#set -A liste_file_se_tmp
448	unset liste_file_se
449	unset liste_file_se_tmp
450	while [ ${an} -le ${an_fin} ] ; do
451	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
452	#
453	an4=$(printf "%04i\n" ${an} )
454	#
455	diy=$( IGCM_date_DaysInYear $an4 )
456	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
457	date=${an4}0101_${afin}
458	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
459	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
460	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
461	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
462	(( an = an + 1 ))
463	done
464	#
465	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
466	IGCM_debug_Verif_Exit_Post
467
468	# Apply IGCM_Patch if needed
469	if [ ! ${RebuildFrequency} = true ] ; then
470	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
471	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
472	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
473	IGCM_${Patch} ${file_t}
474	done
475	done
476	fi
477	fi
478
479	# Search of common list of variables to be treated
480	nbfile=0
481	for file_common_var in ${liste_file_se_tmp[*]} ; do
482	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
483	(( nbfile = nbfile + 1 ))
484	done
485	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
486	IGCM_sys_Rm tmpfile_ncra
487
488	# detect time counter : time or time_counter ie unlimited variable
489	var_unlim=$(ncdump -h ${liste_file_se_tmp[1]}\|grep UNLIMITED\| cut -d ' ' -f 1)
490
491	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
492	#
493	(( nb = ${mois} - 1 ))
494	#
495	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
496	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
497	done
498	IGCM_sys_Rm ${liste_file_se_tmp[*]}
499	;;
500	# For Monthly simulation :
501	1M\|1m)
502	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
503	#
504	(( nb = ${mois} - 1 ))
505	#
506	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
507	#
508	(( an = an_deb ))
509	# Initialize array
510	#set -A liste_file_se
511	#set -A liste_file_se_tmp
512	unset liste_file_se
513	unset liste_file_se_tmp
514	while [ ${an} -le ${an_fin} ] ; do
515	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
516	#
517	an4=$(printf "%04i\n" ${an} )
518	#
519	date=${an4}${mois}01_${an4}${mois}${moisL}
520	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
521	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
522	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
523	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
524
525	(( an = an + 1 ))
526	done
527	#
528	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
529	IGCM_debug_Verif_Exit_Post
530
531	# Apply IGCM_Patch if needed
532	if [ ! ${RebuildFrequency} = true ] ; then
533	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
534	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
535	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
536	IGCM_${Patch} ${file_t}
537	done
538	done
539	fi
540	fi
541
542	# Search of common list of variables to be treated
543	nbfile=0
544	for file_common_var in ${liste_file_se_tmp[*]} ; do
545	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
546	(( nbfile = nbfile + 1 ))
547	done
548	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
549	IGCM_sys_Rm tmpfile_ncra
550
551	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
552
553	IGCM_sys_Rm ${liste_file_se_tmp[*]}
554	done
555	;;
556	D\|d)
557	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
558	#
559	(( nb = ${mois} - 1 ))
560	#
561	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
562	#
563	(( an = an_deb ))
564	# Initialize array
565	#set -A liste_file_se
566	#set -A liste_file_se_tmp
567	unset liste_file_se
568	unset liste_file_se_tmp
569	while [ ${an} -le ${an_fin} ] ; do
570	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
571	#
572	an4=$(printf "%04i\n" ${an} )
573	#
574	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
575	# for one day
576	if [ ${length_d} -eq 1 ] ; then
577	(( length_d = 0 ))
578	fi
579	(( day = 1 ))
580	while [ ${day} -le ${moisL} ] ; do
581	day2=$(printf "%02i\n" ${day} )
582	day_beg=${an4}${mois}${day2}
583	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
584	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
585	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
586	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
587	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
588	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
589	(( day = day + 1 ))
590	done
591	(( an = an + 1 ))
592	done
593	#
594	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
595	IGCM_debug_Verif_Exit_Post
596
597	# Apply IGCM_Patch if needed
598	if [ ! ${RebuildFrequency} = true ] ; then
599	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
600	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
601	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
602	IGCM_${Patch} ${file_t}
603	done
604	done
605	fi
606	fi
607
608	# Search of common list of variables to be treated
609	nbfile=0
610	for file_common_var in ${liste_file_se_tmp[*]} ; do
611	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
612	(( nbfile = nbfile + 1 ))
613	done
614	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
615	IGCM_sys_Rm tmpfile_ncra
616
617	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
618	IGCM_sys_Rm ${liste_file_se_tmp[*]}
619	done
620	;;
621	esac
622	#
623
624	# Search of common list of variables to be treated
625	nbfile=0
626	for file_common_var in ${OUT_SE[*]} ; do
627	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
628	(( nbfile = nbfile + 1 ))
629	done
630	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
631	IGCM_sys_Rm tmpfile_ncrcat
632
633	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
634	#
635	IGCM_sys_Rm ${OUT_SE[*]}
636	#
637	# detect time counter : time or time_counter ie unlimited variable
638	var_unlim=$(ncdump -h ${RESULT_SE}\|grep UNLIMITED\| cut -d ' ' -f 1)
639
640	# Put correct climatological axis following calendar
641	case ${config_UserChoices_CalendarType} in
642	360d\|360_day)
643	# 360 days
644	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
645	;;
646	noleap\|365_day)
647	# 365 days
648	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
649	;;
650	all_leap\|366_day)
651	# 366 days
652	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
653	;;
654	leap\|gregorian\|standard)
655	# 365.2425 days
656	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
657	;;
658	esac
659	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
660	IGCM_sys_Rm ${RESULT_SE}
661	IGCM_sys_Mv tmp.nc ${RESULT_SE}
662	#
663	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
664	IGCM_sys_Rm ${RESULT_SE}
665	#
666	done
667	done
668
669	# DODS copy
670	for comp in ${config_ListOfComponents[*]} ; do
671	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
672	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
673	done
674
675	YEARS=${an_deb}_${an_fin}
676	DIR_OUT=SE
677
678	export POST_DIR; export SUBMIT_DIR; export libIGCM; export R_INIT; export R_BC; export StandAlone
679	export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ; export RESOL_SBG
680	export R_SAVE; export config_UserChoices_JobName; export config_UserChoices_TagName; export YEARS; export DIR_OUT
681	listVarEnv="POST_DIR,SUBMIT_DIR,libIGCM,R_INIT,R_BC,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,R_SAVE,config_UserChoices_JobName,config_UserChoices_TagName,YEARS,DIR_OUT"
682	export listVarEnv;
683
684	#D-
685	# --------------------------------------------------------------------
686	#D- Test if all was right before ATLAS
687	# --------------------------------------------------------------------
688	IGCM_debug_Verif_Exit_Post
689
690	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
691	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
692	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
693	#
694	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
695	IGCM_sys_QsubPost atlas_ORCHIDEE ;
696	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
697	IGCM_sys_QsubPost atlas_LMDZ ;
698	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
699	IGCM_sys_QsubPost atlas_ORCA_LIM ;
700	Script_Post_Output=additionnal.${PeriodDateEnd} ;
701	IGCM_sys_QsubPost additionnal ;
702	#
703	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
704	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
705	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
706	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] ; then
707	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
708	IGCM_sys_QsubPost atlas_ORCHIDEE ;
709	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
710	IGCM_sys_QsubPost atlas_LMDZ ;
711	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
712	IGCM_sys_QsubPost atlas_ORCA_LIM ;
713	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
714	IGCM_sys_QsubPost atlas_PISCES ;
715	#
716	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
717	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
718	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
719	#
720	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
721	IGCM_sys_QsubPost atlas_LMDZ ;
722	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
723	IGCM_sys_QsubPost atlas_ORCHIDEE ;
724	#
725	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
726	#
727	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
728	IGCM_sys_QsubPost atlas_LMDZ ;
729	#
730	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
731	[ ${config_UserChoices_TagName} = "OL2" ] ; then
732	#
733	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
734	IGCM_sys_QsubPost atlas_ORCHIDEE ;
735	#
736	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\|
737	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] ; then
738	#
739	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
740	IGCM_sys_QsubPost atlas_ORCA_LIM ;
741	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] ; then
742	#
743	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
744	IGCM_sys_QsubPost atlas_ORCA_LIM ;
745	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
746	IGCM_sys_QsubPost atlas_PISCES ;
747	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
748	#
749	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
750	IGCM_sys_QsubPost atlas_PISCES ;
751	#
752	fi
753
754	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
755	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: