Context Navigation

← Previous Revision
Next Revision →
Blame
Revision Log

AA_create_se

Last change on this file was 1621, checked in by jgipsl, 3 months ago
Correction for the case where no pack is done (SpaceName?=TEST or at spirit and obelix). The error was introduced in [1603].
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 31.3 KB

Line
1	#-Q- curie ######################
2	#-Q- curie ## CURIE TGCC/CEA ##
3	#-Q- curie ######################
4	#-Q- curie #MSUB -r SE # Nom du job
5	#-Q- curie #MSUB -o SE.out_%I
6	#-Q- curie #MSUB -e SE.out_%I
7	#-Q- curie #MSUB -n 1 # Reservation du processus
8	#-Q- curie #MSUB -T 36000 # Limite de temps elapsed du job
9	#-Q- curie #MSUB -q ::default_node::
10	#-Q- curie #MSUB -c ::default_core::
11	#-Q- curie #MSUB -Q normal
12	#-Q- curie #MSUB -A ::default_project::
13	#-Q- curie set +x
14	#-Q- irene ######################
15	#-Q- irene ## IRENE TGCC/CEA ##
16	#-Q- irene ######################
17	#-Q- irene #MSUB -r SE # Job name
18	#-Q- irene #MSUB -o SE.out_%I
19	#-Q- irene #MSUB -e SE.out_%I
20	#-Q- irene #MSUB -n 1
21	#-Q- irene #MSUB -T 36000 # Maximum elapsed time
22	#-Q- irene #MSUB -q ::default_node::
23	#-Q- irene #MSUB -c ::default_core::
24	#-Q- irene #MSUB -Q normal
25	#-Q- irene #MSUB -A ::default_post_project::
26	#-Q- irene #MSUB -m store,work,scratch
27	#-Q- irene set +x
28	#-Q- irene-amd ######################
29	#-Q- irene-amd ## IRENE TGCC/CEA ##
30	#-Q- irene-amd ######################
31	#-Q- irene-amd #MSUB -r SE # Job name
32	#-Q- irene-amd #MSUB -o SE.out_%I
33	#-Q- irene-amd #MSUB -e SE.out_%I
34	#-Q- irene-amd #MSUB -n 1 # Number of cores
35	#-Q- irene-amd #MSUB -T 36000 # Maximum elapsed time
36	#-Q- irene-amd #MSUB -q ::default_node::
37	#-Q- irene-amd #MSUB -c ::default_core::
38	#-Q- irene-amd #MSUB -Q normal
39	#-Q- irene-amd #MSUB -A ::default_post_project::
40	#-Q- irene-amd #MSUB -m store,work,scratch
41	#-Q- irene-amd set +x
42	#-Q- jeanzay #!/bin/ksh
43	#-Q- jeanzay ######################
44	#-Q- jeanzay ## JEANZAY IDRIS ##
45	#-Q- jeanzay ######################
46	#-Q- jeanzay #SBATCH --job-name=SE # Job Name
47	#-Q- jeanzay #SBATCH --output=SE.out_%I # standard output
48	#-Q- jeanzay #SBATCH --error=SE.out_%I # error output
49	#-Q- jeanzay #SBATCH -N 1 # Number of core
50	#-Q- jeanzay #SBATCH --partition=prepost # Post-processing partition
51	#-Q- jeanzay #SBATCH --time=10:00:00 # Wall clock limit (seconds)
52	#-Q- jeanzay #SBATCH --account ::default_project::@cpu
53	#-Q- jeanzay set +x
54	#-Q- ada #!/bin/ksh
55	#-Q- ada #######################
56	#-Q- ada ## ADA IDRIS ##
57	#-Q- ada #######################
58	#-Q- ada # @ job_type = mpich
59	#-Q- ada # @ requirements = (Feature == "prepost")
60	#-Q- ada # Temps Elapsed max. d'une requete hh:mm:ss
61	#-Q- ada # @ wall_clock_limit = 10:00:00
62	#-Q- ada # Memory required for ncrcat
63	#-Q- ada # @ as_limit = 30Gb
64	#-Q- ada # Nom du travail LoadLeveler
65	#-Q- ada # @ job_name = SE
66	#-Q- ada # Fichier de sortie standard du travail
67	#-Q- ada # @ output = $(job_name).$(jobid)
68	#-Q- ada # Fichier de sortie d'erreur du travail
69	#-Q- ada # @ error = $(job_name).$(jobid)
70	#-Q- ada # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
71	#-Q- ada # @ notification = error
72	#-Q- ada # @ environment = $DEBUG_debug ; $BigBrother ; $postProcessingStopLevel ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $POST_DIR ; $Script_Post_Output ; $MASTER ; $DateBegin ; $PeriodDateEnd ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; wall_clock_limit=$(wall_clock_limit)
73	#-Q- ada # @ queue
74	#-Q- obelix ######################
75	#-Q- obelix ## OBELIX LSCE ##
76	#-Q- obelix ######################
77	#-Q- obelix #PBS -N SE
78	#-Q- obelix #PBS -m a
79	#-Q- obelix #PBS -j oe
80	#-Q- obelix #PBS -q medium
81	#-Q- obelix #PBS -o SE.$$
82	#-Q- obelix #PBS -S /bin/ksh
83	#-Q- mesoipsl #!/bin/ksh
84	#-Q- mesoipsl ######################
85	#-Q- mesoipsl ## MESO ESPRI IPSL ##
86	#-Q- mesoipsl ######################
87	#-Q- mesoipsl #SBATCH --job-name=TS # Job Name
88	#-Q- mesoipsl #SBATCH --output=TS.out_%I # standard output
89	#-Q- mesoipsl #SBATCH --error=TS.out_%I # error output
90	#-Q- mesoipsl #SBATCH -N 1 # Number of core
91	#-Q- mesoipsl #SBATCH --time=10:00:00 # Wall clock limit (seconds)
92	#-Q- ifort_CICLAD ######################
93	#-Q- ifort_CICLAD ## CICLAD IPSL ##
94	#-Q- ifort_CICLAD ######################
95	#-Q- ifort_CICLAD #PBS -N SE
96	#-Q- ifort_CICLAD #PBS -m a
97	#-Q- ifort_CICLAD #PBS -j oe
98	#-Q- ifort_CICLAD ###PBS -q h12 # Queue for 12 hours at ciclad only
99	#-Q- ifort_CICLAD #PBS -S /bin/ksh
100	#-Q- default #!/bin/ksh
101	#-Q- default ##################
102	#-Q- default ## DEFAULT HOST ##
103	#-Q- default ##################
104
105	#**************************************************************
106	# Author: Sebastien Denvil
107	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
108	# $Revision:: $ Revision of last commit
109	# $Author:: $ Author of last commit
110	# $Date:: $ Date of last commit
111	# IPSL (2006)
112	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
113	#
114	#**************************************************************
115
116	#set -eu
117	#set -vx
118
119	date
120
121	#-Q- ada export OMP_NUM_THREADS=1
122
123	#D- Task type DO NOT CHANGE (computing, post-processing or checking)
124	TaskType=post-processing
125
126	########################################################################
127
128	#D- Flag to determine if this job in a standalone mode
129	#D- Default : value from AA_job if any
130	StandAlone=${StandAlone:=true}
131
132	#D- Path to libIGCM
133	#D- Default : value from AA_job if any
134	libIGCM=${libIGCM:=::modipsl::/libIGCM}
135
136	#D- Flag to determine if rebuild process has been performed asynchronously
137	#D- Possible value true or false.
138	#D- If true raw files has allready been patched by rebuild job
139	#D- If false create_ts will patch the raw files
140	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
141	RebuildFrequency=${RebuildFrequency:=true}
142
143	#D- Flag to determine atlas job's output directory
144	#D- Default : value from libIGCM_post.ksh if any
145	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
146
147	#D- Increased verbosity (1, 2, 3)
148	#D- Default : value from AA_job if any
149	Verbosity=${Verbosity:=3}
150
151	#D- Low level debug : to bypass lib test checks and stack construction
152	#D- Default : value from AA_job if any
153	DEBUG_debug=${DEBUG_debug:=false}
154
155	#D- TEMPORARY Flag to determine atmospheric resolution
156	#D- Default : value from atmospheric driver if any
157	RESOL_ATM=${RESOL_ATM:=ALL}
158
159	#D- Flag to determine surface resolution
160	#D- Default : value from surface driver if any
161	RESOL_SRF=ALL
162
163	#D- Flag to determine surface resolution
164	#D- Default : value from surface driver if any
165	RESOL_SBG=ALL
166
167	#D- TEMPORARY Flag to determine ocean resolution
168	#D- Default : value from ocean driver if any
169	RESOL_OCE=${RESOL_OCE:=ORCA2}
170
171	#D- TEMPORARY Flag to determine ice resolution
172	#D- Default : value from ice driver if any
173	RESOL_ICE=${RESOL_OCE:=ORCA2}
174
175	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
176	#D- Default : value from ice driver if any
177	RESOL_MBG=${RESOL_OCE:=ORCA2}
178
179	########################################################################
180
181	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
182	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
183	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
184	#-------
185	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
186	. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
187	#-------
188	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
189	IGCM_sys_Cd ${RUN_DIR_PATH}
190	#-------
191	( ${DEBUG_debug} ) && IGCM_debug_Check
192	( ${DEBUG_debug} ) && IGCM_card_Check
193	( ${DEBUG_debug} ) && IGCM_date_Check
194
195	########################################################################
196
197	#set -vx
198
199	# ------------------------------------------------------------------
200	# Test if all was right before proceeding further
201	# ------------------------------------------------------------------
202	IGCM_debug_Verif_Exit
203
204	if [ ${StandAlone} = true ] ; then
205	CARD_DIR=${SUBMIT_DIR}
206	else
207	CARD_DIR=${RUN_DIR_PATH}
208	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
209	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
210	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
211	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
212	fi
213
214	#==================================
215	# First of all
216	#
217	# Read libIGCM compatibility version in config.card
218	# Read UserChoices section
219	# Read Ensemble section
220	# Read Post section
221	# Define all netcdf output directories
222	#==================================
223	IGCM_config_CommonConfiguration ${CARD_DIR}/config.card
224
225	# ------------------------------------------------------------------
226	# Activate BigBrother so as to supervise this job
227	# ------------------------------------------------------------------
228	IGCM_debug_BigBro_Initialize
229
230	#==================================
231	# Define default value to keep compatibility with previous card: means before changes due to TGCC
232	if [ X${PackDefault} = Xtrue ] ; then
233	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
234	else
235	config_Post_PackFrequency=NONE
236	fi
237
238	#==================================
239	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
240	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
241	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
242	fi
243
244	#==================================
245	# Read ListOfComponents section:
246	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
247
248	#==================================
249	# Read SeasonalFrequency:
250	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
251
252	#
253	# Determine period for seasonnal average. Default : value from AA_job if any
254	#
255	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
256	#
257	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
258	#
259	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
260	#
261	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
262	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
263	an_fin=$( IGCM_date_YearDigit ${an_fin} )
264	#
265	# For each component selected determine which files need post-processing
266	#
267	for comp in ${config_ListOfComponents[*]} ; do
268	#
269	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
270	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
271	#
272	card=${CARD_DIR}/COMP/${compname}.card
273	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
274	#
275	ListFilesName=${compname}_OutputFiles_List
276	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
277	#
278	if [ X${FileName0} != X${NULL_STR} ] ; then
279	#
280	# INITIALISATION
281	#
282	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
283	typeset i=2
284	#
285	until [ $i -gt $NbFiles ]; do
286	#
287	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
288	#
289	if [ X${flag_post} != XNONE ] ; then
290	#
291	# Check if seasonal is required for this file
292	#
293	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
294	#
295	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
296	#
297	# variable option allready typeset above
298	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
299	if [ ${option} = Seasonal ] ; then
300	FoundSeasonal=true
301	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
302	fi
303	done
304	#
305	if [ ! X${FoundSeasonal} = Xtrue ] ; then
306	eval ${compname}_${flag_post}_Seasonal=ON
307	fi
308	#
309	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
310	(( i_ = i+1 ))
311	eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
312	eval Testvar=${Testvar_}
313	if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
314	(( i = i+4 ))
315	else
316	(( i=i+3 ))
317	fi
318	continue
319	fi
320	#
321	# CREATE LIST
322	#
323	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
324	#
325	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
326	#
327	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
328	#
329	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
330	if [ ! $? = 0 ] ; then
331	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
332	(( i_ = i+1 ))
333	eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
334	eval Testvar=${Testvar_}
335	if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
336	eval PeriodPack_FileType=${Testvar}
337	else
338	PeriodPack_FileType=${config_Post_PackFrequency}
339	fi
340	eval set +A LISTE_PACKFREQUENCY_${comp} \${LISTE_PACKFREQUENCY_${comp}[*]} ${PeriodPack_FileType}
341	if [ ! ${RebuildFrequency} = true ] ; then
342	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
343	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
344	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
345	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
346	if [ Xload${Patch} != X ] ; then
347	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
348	eval load${Patch}=loaded
349	fi
350	done
351	fi
352	fi
353	fi
354	fi
355	(( i_ = i+1 ))
356	eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
357	eval Testvar=${Testvar_}
358	if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
359	(( i = i+4 ))
360	else
361	(( i=i+3 ))
362	fi
363	done # until [ $i -eq $NbFiles ]
364	fi
365	done # for comp
366
367	for comp in ${config_ListOfComponents[*]} ; do
368	#
369	eval R_OUT_${comp}=${R_SAVE}/${comp}
370	#
371	i=0
372	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
373
374	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
375	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
376	config_UserChoices_PeriodLength=$( eval echo \${LISTE_PACKFREQUENCY_${comp}[$i]} )
377	fi
378
379	#
380	# Determine in which output can we find file
381	# IE : Output/MO or Output/DA or ...
382	#
383	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
384	case ${FlagDir} in
385	1Y) FreqDir=YE ;;
386	1M) FreqDir=MO ;;
387	5D\|1D) FreqDir=DA ;;
388	HF) FreqDir=HF ;;
389	3H) FreqDir=HF ;;
390	INS) FreqDir=INS ;;
391	esac
392	#
393	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
394	# Do not redoo an existing SE
395	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
396	if [ $? = 0 ] ; then
397	continue
398	fi
399	#
400	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
401	#
402	case ${config_UserChoices_PeriodLength} in
403	# For Yearly simulation :
404	Y\|y)
405	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
406	(( an = an_deb ))
407	# Initialize array
408	unset liste_file_se
409	unset liste_file_se_tmp
410	while [ ${an} -le ${an_fin} ] ; do
411	#
412	an4=$(printf "%04i\n" ${an} )
413	#
414	diy=$( IGCM_date_DaysInCurrentPeriod ${an4}0101 ${config_UserChoices_PeriodLength} )
415	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
416	#
417	date=${an4}0101_${afin}
418	#
419	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
420	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
421	(( an = an + ${PeriodLengthInYears} ))
422	done
423	#
424	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
425	IGCM_debug_Verif_Exit
426
427	# Apply IGCM_Patch if needed
428	if [ ! ${RebuildFrequency} = true ] ; then
429	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
430	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
431	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
432	IGCM_${Patch} ${file_t}
433	done
434	done
435	fi
436	fi
437
438	# Search of common list of variables to be treated
439	nbfile=0
440	for file_common_var in ${liste_file_se_tmp[*]} ; do
441	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
442	(( nbfile = nbfile + 1 ))
443	done
444	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
445	IGCM_sys_Rm tmpfile_ncra
446
447	# detect time counter : time or time_counter ie unlimited variable
448	var_unlim=$(ncdump -h ${liste_file_se_tmp[0]}\|grep UNLIMITED\|awk '{print $1}')
449
450	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
451	#
452	(( nb = ${mois} - 1 ))
453	#
454	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
455	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
456	done
457	IGCM_sys_Rm ${liste_file_se_tmp[*]}
458	;;
459	# For Monthly simulation :
460	1M\|1m)
461	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
462	#
463	(( nb = ${mois} - 1 ))
464	#
465	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
466	#
467	(( an = an_deb ))
468	# Initialize array
469	unset liste_file_se
470	unset liste_file_se_tmp
471	while [ ${an} -le ${an_fin} ] ; do
472	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
473	#
474	an4=$(printf "%04i\n" ${an} )
475	#
476	date=${an4}${mois}01_${an4}${mois}${moisL}
477	#
478	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
479	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
480
481	(( an = an + 1 ))
482	done
483	#
484	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
485	IGCM_debug_Verif_Exit
486
487	# Apply IGCM_Patch if needed
488	if [ ! ${RebuildFrequency} = true ] ; then
489	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
490	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
491	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
492	IGCM_${Patch} ${file_t}
493	done
494	done
495	fi
496	fi
497
498	# Search of common list of variables to be treated
499	nbfile=0
500	for file_common_var in ${liste_file_se_tmp[*]} ; do
501	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
502	(( nbfile = nbfile + 1 ))
503	done
504	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
505	IGCM_sys_Rm tmpfile_ncra
506
507	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
508
509	IGCM_sys_Rm ${liste_file_se_tmp[*]}
510	done
511	;;
512	D\|d)
513	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
514	#
515	(( nb = ${mois} - 1 ))
516	#
517	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
518	#
519	(( an = an_deb ))
520	# Initialize array
521	unset liste_file_se
522	unset liste_file_se_tmp
523	while [ ${an} -le ${an_fin} ] ; do
524	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
525	#
526	an4=$(printf "%04i\n" ${an} )
527	#
528	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
529	# for one day
530	if [ ${length_d} -eq 1 ] ; then
531	(( length_d = 0 ))
532	fi
533	(( day = 1 ))
534	while [ ${day} -le ${moisL} ] ; do
535	day2=$(printf "%02i\n" ${day} )
536	day_beg=${an4}${mois}${day2}
537	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
538	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
539	#
540	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
541	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
542	(( day = day + 1 ))
543	done
544	(( an = an + 1 ))
545	done
546	#
547	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
548	IGCM_debug_Verif_Exit
549
550	# Apply IGCM_Patch if needed
551	if [ ! ${RebuildFrequency} = true ] ; then
552	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
553	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
554	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
555	IGCM_${Patch} ${file_t}
556	done
557	done
558	fi
559	fi
560
561	# Search of common list of variables to be treated
562	nbfile=0
563	for file_common_var in ${liste_file_se_tmp[*]} ; do
564	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
565	(( nbfile = nbfile + 1 ))
566	done
567	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
568	IGCM_sys_Rm tmpfile_ncra
569
570	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
571	IGCM_sys_Rm ${liste_file_se_tmp[*]}
572	done
573	;;
574	esac
575	#
576
577	# Search of common list of variables to be treated
578	nbfile=0
579	for file_common_var in ${OUT_SE[*]} ; do
580	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
581	(( nbfile = nbfile + 1 ))
582	done
583	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| grep -v ^time_ \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
584	IGCM_sys_Rm tmpfile_ncrcat
585
586	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
587	#
588	IGCM_sys_Rm ${OUT_SE[*]}
589	#
590	# detect time counter : time or time_counter ie unlimited variable
591	var_unlim=$( ncdump -h ${RESULT_SE} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[[:space:]]*//' )
592
593	# Put correct climatological axis following calendar
594	case ${config_UserChoices_CalendarType} in
595	360d\|360_day)
596	# 360 days
597	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
598	;;
599	noleap\|365_day)
600	# 365 days
601	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
602	;;
603	all_leap\|366_day)
604	# 366 days
605	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
606	;;
607	leap\|gregorian\|standard)
608	# 365.2425 days
609	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
610	;;
611	esac
612	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
613	IGCM_sys_Rm ${RESULT_SE}
614	IGCM_sys_Mv tmp.nc ${RESULT_SE}
615	#
616	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
617	[ -f ${RESULT_SE} ] && IGCM_sys_Rm ${RESULT_SE}
618	#
619	(( i = i + 1 ))
620	done
621	done
622
623	# DODS copy
624	if [ X${config_Post_DodsCopy} = XTRUE ] ; then
625	for comp in ${config_ListOfComponents[*]} ; do
626	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
627	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
628	done
629	fi
630
631	# Temporal coverage id for atlas
632	YEARS=${an_deb}_${an_fin}
633	DIR_OUT=SE
634
635	export DEBUG_debug; export BigBrother ; export libIGCM; export SUBMIT_DIR; export POST_DIR; export Script_Output ;
636	export StandAlone ; export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ;
637	export RESOL_SBG ; export YEARS; export DIR_OUT
638	listVarEnv="DEBUG_debug,BigBrother,libIGCM,SUBMIT_DIR,POST_DIR,Script_Post_Output,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,YEARS,DIR_OUT"
639	export listVarEnv;
640
641	#D-
642	# --------------------------------------------------------------------
643	#D- Test if all was right before ATLAS
644	# --------------------------------------------------------------------
645	IGCM_debug_Verif_Exit
646
647	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
648	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
649	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
650	# ATLAS
651	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
652	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
653	IGCM_sys_QsubPost atlas_ORCHIDEE ;
654	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
655	IGCM_sys_QsubPost atlas_LMDZ ;
656	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
657	IGCM_sys_QsubPost atlas_ORCA_LIM ;
658	export Script_Post_Output=additionnal.${PeriodDateEnd} ;
659	IGCM_sys_QsubPost additionnal ;
660	fi
661	#
662	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
663	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
664	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
665	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] \|\| \
666	[ ${config_UserChoices_TagName} = "IPSLCM5A2" ] \|\| \
667	[ ${config_UserChoices_TagName} = "IPSLCM6" ] ; then
668	# METRICS
669	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
670	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
671	IGCM_sys_QsubPost metrics_LMDZ ;
672	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
673	IGCM_sys_QsubPost metrics_NEMO ;
674	fi
675	# ATLAS
676	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
677	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
678	IGCM_sys_QsubPost atlas_ORCHIDEE ;
679	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
680	IGCM_sys_QsubPost atlas_LMDZ ;
681	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
682	IGCM_sys_QsubPost atlas_ORCA_LIM ;
683	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
684	IGCM_sys_QsubPost atlas_PISCES ;
685	fi
686	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
687	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
688	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
689	# METRICS
690	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
691	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
692	IGCM_sys_QsubPost metrics_LMDZ ;
693	fi
694	# ATLAS
695	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
696	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
697	IGCM_sys_QsubPost atlas_LMDZ ;
698	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
699	IGCM_sys_QsubPost atlas_ORCHIDEE ;
700	fi
701	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
702	# METRICS
703	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
704	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
705	IGCM_sys_QsubPost metrics_LMDZ ;
706	fi
707	# ATLAS
708	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
709	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
710	IGCM_sys_QsubPost atlas_LMDZ ;
711	fi
712	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
713	[ ${config_UserChoices_TagName} = "OL2" ] ; then
714	# METRICS
715	# ATLAS
716	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
717	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
718	IGCM_sys_QsubPost atlas_ORCHIDEE ;
719	fi
720	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\| \
721	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] \|\| \
722	[ ${config_UserChoices_TagName} = "ORCA2_LIM3" ] \|\| \
723	[ ${config_UserChoices_TagName} = "ORCA1_LIM2" ] \|\| \
724	[ ${config_UserChoices_TagName} = "ORCA1_LIM3" ] ; then
725	# METRICS
726	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
727	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
728	IGCM_sys_QsubPost metrics_NEMO ;
729	fi
730	# ATLAS
731	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
732	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
733	IGCM_sys_QsubPost atlas_ORCA_LIM ;
734	fi
735	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] \|\| \
736	[ ${config_UserChoices_TagName} = "ORCA2_LIM3_PISCES" ] \|\| \
737	[ ${config_UserChoices_TagName} = "ORCA1_LIM2_PISCES" ] \|\| \
738	[ ${config_UserChoices_TagName} = "ORCA1_LIM3_PISCES" ] ; then
739	# METRICS
740	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
741	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
742	IGCM_sys_QsubPost metrics_NEMO ;
743	fi
744	# ATLAS
745	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
746	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
747	IGCM_sys_QsubPost atlas_ORCA_LIM ;
748	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
749	IGCM_sys_QsubPost atlas_PISCES ;
750	fi
751	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
752	# METRICS
753	# ATLAS
754	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
755	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
756	IGCM_sys_QsubPost atlas_PISCES ;
757	fi
758	fi
759
760	# Execute C-ESM-EP driver (which launches jobs)
761	if [ $PostCesmep = SE ] ; then
762	${SUBMIT_DIR}/cesmep_lite/libIGCM_post.sh $DateBegin $PeriodDateEnd
763	fi
764
765	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
766	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
767
768	# ------------------------------------------------------------------
769	# Finalize BigBrother to inform that the jobs end
770	# ------------------------------------------------------------------
771	IGCM_debug_BigBro_Finalize

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/AA_create_se

Download in other formats: