Context Navigation

source: branches/libIGCM_CESMEP/AA_create_se @ 1553

Last change on this file since 1553 was 1525, checked in by cetlod, 4 years ago
Update Scripts for Post-processing on Irene-Rome
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 29.6 KB

Line
1	#-Q- curie ######################
2	#-Q- curie ## CURIE TGCC/CEA ##
3	#-Q- curie ######################
4	#-Q- curie #MSUB -r SE # Nom du job
5	#-Q- curie #MSUB -o SE.out_%I
6	#-Q- curie #MSUB -e SE.out_%I
7	#-Q- curie #MSUB -n 1 # Reservation du processus
8	#-Q- curie #MSUB -T 36000 # Limite de temps elapsed du job
9	#-Q- curie #MSUB -q ::default_node::
10	#-Q- curie #MSUB -c ::default_core::
11	#-Q- curie #MSUB -Q normal
12	#-Q- curie #MSUB -A ::default_project::
13	#-Q- curie set +x
14	#-Q- irene ######################
15	#-Q- irene ## IRENE TGCC/CEA ##
16	#-Q- irene ######################
17	#-Q- irene #MSUB -r SE # Job name
18	#-Q- irene #MSUB -o SE.out_%I
19	#-Q- irene #MSUB -e SE.out_%I
20	#-Q- irene #MSUB -n 1
21	#-Q- irene #MSUB -T 36000 # Maximum elapsed time
22	#-Q- irene #MSUB -q ::default_node::
23	#-Q- irene #MSUB -c ::default_core::
24	#-Q- irene #MSUB -Q normal
25	#-Q- irene #MSUB -A ::default_post_project::
26	#-Q- irene #MSUB -m store,work,scratch
27	#-Q- irene set +x
28	#-Q- irene-amd ######################
29	#-Q- irene-amd ## IRENE TGCC/CEA ##
30	#-Q- irene-amd ######################
31	#-Q- irene-amd #MSUB -r SE # Job name
32	#-Q- irene-amd #MSUB -o SE.out_%I
33	#-Q- irene-amd #MSUB -e SE.out_%I
34	#-Q- irene-amd #MSUB -n 1 # Number of cores
35	#-Q- irene-amd #MSUB -T 36000 # Maximum elapsed time
36	#-Q- irene-amd #MSUB -q ::default_node::
37	#-Q- irene-amd #MSUB -c ::default_core::
38	#-Q- irene-amd #MSUB -Q normal
39	#-Q- irene-amd #MSUB -A ::default_post_project::
40	#-Q- irene-amd #MSUB -m store,work,scratch
41	#-Q- irene-amd set +x
42	#-Q- jeanzay #!/bin/ksh
43	#-Q- jeanzay ######################
44	#-Q- jeanzay ## JEANZAY IDRIS ##
45	#-Q- jeanzay ######################
46	#-Q- jeanzay #SBATCH --job-name=SE # Job Name
47	#-Q- jeanzay #SBATCH --output=SE.out_%I # standard output
48	#-Q- jeanzay #SBATCH --error=SE.out_%I # error output
49	#-Q- jeanzay #SBATCH -N 1 # Number of core
50	#-Q- jeanzay #SBATCH --partition=prepost # Post-processing partition
51	#-Q- jeanzay #SBATCH --time=10:00:00 # Wall clock limit (seconds)
52	#-Q- jeanzay #SBATCH --account ::default_project::@cpu
53	#-Q- jeanzay set +x
54	#-Q- ada #!/bin/ksh
55	#-Q- ada #######################
56	#-Q- ada ## ADA IDRIS ##
57	#-Q- ada #######################
58	#-Q- ada # @ job_type = mpich
59	#-Q- ada # @ requirements = (Feature == "prepost")
60	#-Q- ada # Temps Elapsed max. d'une requete hh:mm:ss
61	#-Q- ada # @ wall_clock_limit = 10:00:00
62	#-Q- ada # Memory required for ncrcat
63	#-Q- ada # @ as_limit = 30Gb
64	#-Q- ada # Nom du travail LoadLeveler
65	#-Q- ada # @ job_name = SE
66	#-Q- ada # Fichier de sortie standard du travail
67	#-Q- ada # @ output = $(job_name).$(jobid)
68	#-Q- ada # Fichier de sortie d'erreur du travail
69	#-Q- ada # @ error = $(job_name).$(jobid)
70	#-Q- ada # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
71	#-Q- ada # @ notification = error
72	#-Q- ada # @ environment = $DEBUG_debug ; $BigBrother ; $postProcessingStopLevel ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $POST_DIR ; $Script_Post_Output ; $MASTER ; $DateBegin ; $PeriodDateEnd ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; wall_clock_limit=$(wall_clock_limit)
73	#-Q- ada # @ queue
74	#-Q- lxiv8 ######################
75	#-Q- lxiv8 ## OBELIX LSCE ##
76	#-Q- lxiv8 ######################
77	#-Q- lxiv8 #PBS -N SE
78	#-Q- lxiv8 #PBS -m a
79	#-Q- lxiv8 #PBS -j oe
80	#-Q- lxiv8 #PBS -q medium
81	#-Q- lxiv8 #PBS -o SE.$$
82	#-Q- lxiv8 #PBS -S /bin/ksh
83	#-Q- ifort_CICLAD ######################
84	#-Q- ifort_CICLAD ## CICLAD IPSL ##
85	#-Q- ifort_CICLAD ######################
86	#-Q- ifort_CICLAD #PBS -N SE
87	#-Q- ifort_CICLAD #PBS -m a
88	#-Q- ifort_CICLAD #PBS -j oe
89	#-Q- ifort_CICLAD ###PBS -q h12 # Queue for 12 hours at ciclad only
90	#-Q- ifort_CICLAD #PBS -S /bin/ksh
91	#-Q- default #!/bin/ksh
92	#-Q- default ##################
93	#-Q- default ## DEFAULT HOST ##
94	#-Q- default ##################
95
96	#**************************************************************
97	# Author: Sebastien Denvil
98	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
99	# $Revision:: $ Revision of last commit
100	# $Author:: $ Author of last commit
101	# $Date:: $ Date of last commit
102	# IPSL (2006)
103	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
104	#
105	#**************************************************************
106
107	#set -eu
108	#set -vx
109
110	date
111
112	#-Q- ada export OMP_NUM_THREADS=1
113
114	#D- Task type DO NOT CHANGE (computing, post-processing or checking)
115	TaskType=post-processing
116
117	########################################################################
118
119	#D- Flag to determine if this job in a standalone mode
120	#D- Default : value from AA_job if any
121	StandAlone=${StandAlone:=true}
122
123	#D- Path to libIGCM
124	#D- Default : value from AA_job if any
125	libIGCM=${libIGCM:=::modipsl::/libIGCM}
126
127	#D- Flag to determine if rebuild process has been performed asynchronously
128	#D- Possible value true or false.
129	#D- If true raw files has allready been patched by rebuild job
130	#D- If false create_ts will patch the raw files
131	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
132	RebuildFrequency=${RebuildFrequency:=true}
133
134	#D- Flag to determine atlas job's output directory
135	#D- Default : value from libIGCM_post.ksh if any
136	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
137
138	#D- Increased verbosity (1, 2, 3)
139	#D- Default : value from AA_job if any
140	Verbosity=${Verbosity:=3}
141
142	#D- Low level debug : to bypass lib test checks and stack construction
143	#D- Default : value from AA_job if any
144	DEBUG_debug=${DEBUG_debug:=false}
145
146	#D- TEMPORARY Flag to determine atmospheric resolution
147	#D- Default : value from atmospheric driver if any
148	RESOL_ATM=${RESOL_ATM:=ALL}
149
150	#D- Flag to determine surface resolution
151	#D- Default : value from surface driver if any
152	RESOL_SRF=ALL
153
154	#D- Flag to determine surface resolution
155	#D- Default : value from surface driver if any
156	RESOL_SBG=ALL
157
158	#D- TEMPORARY Flag to determine ocean resolution
159	#D- Default : value from ocean driver if any
160	RESOL_OCE=${RESOL_OCE:=ORCA2}
161
162	#D- TEMPORARY Flag to determine ice resolution
163	#D- Default : value from ice driver if any
164	RESOL_ICE=${RESOL_OCE:=ORCA2}
165
166	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
167	#D- Default : value from ice driver if any
168	RESOL_MBG=${RESOL_OCE:=ORCA2}
169
170	########################################################################
171
172	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
173	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
174	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
175	#-------
176	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
177	. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
178	#-------
179	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
180	IGCM_sys_Cd ${RUN_DIR_PATH}
181	#-------
182	( ${DEBUG_debug} ) && IGCM_debug_Check
183	( ${DEBUG_debug} ) && IGCM_card_Check
184	( ${DEBUG_debug} ) && IGCM_date_Check
185
186	########################################################################
187
188	#set -vx
189
190	# ------------------------------------------------------------------
191	# Test if all was right before proceeding further
192	# ------------------------------------------------------------------
193	IGCM_debug_Verif_Exit
194
195	if [ ${StandAlone} = true ] ; then
196	CARD_DIR=${SUBMIT_DIR}
197	else
198	CARD_DIR=${RUN_DIR_PATH}
199	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
200	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
201	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
202	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
203	fi
204
205	#==================================
206	# First of all
207	#
208	# Read libIGCM compatibility version in config.card
209	# Read UserChoices section
210	# Read Ensemble section
211	# Read Post section
212	# Define all netcdf output directories
213	#==================================
214	IGCM_config_CommonConfiguration ${CARD_DIR}/config.card
215
216	# ------------------------------------------------------------------
217	# Activate BigBrother so as to supervise this job
218	# ------------------------------------------------------------------
219	IGCM_debug_BigBro_Initialize
220
221	#==================================
222	# Define default value to keep compatibility with previous card: means before changes due to TGCC
223	if [ X${PackDefault} = Xtrue ] ; then
224	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
225	else
226	config_Post_PackFrequency=NONE
227	fi
228
229	#==================================
230	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
231	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
232	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
233	fi
234
235	#==================================
236	# Read ListOfComponents section:
237	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
238
239	#==================================
240	# Read SeasonalFrequency:
241	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
242
243	#
244	# Determine period for seasonnal average. Default : value from AA_job if any
245	#
246	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
247	#
248	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
249	#
250	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
251	#
252	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
253	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
254	an_fin=$( IGCM_date_YearDigit ${an_fin} )
255	#
256	# For each component selected determine which files need post-processing
257	#
258	for comp in ${config_ListOfComponents[*]} ; do
259	#
260	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
261	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
262	#
263	card=${CARD_DIR}/COMP/${compname}.card
264	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
265	#
266	ListFilesName=${compname}_OutputFiles_List
267	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
268	#
269	if [ X${FileName0} != X${NULL_STR} ] ; then
270	#
271	# INITIALISATION
272	#
273	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
274	typeset i=2
275	#
276	until [ $i -gt $NbFiles ]; do
277	#
278	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
279	#
280	if [ X${flag_post} != XNONE ] ; then
281	#
282	# Check if seasonal is required for this file
283	#
284	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
285	#
286	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
287	#
288	# variable option allready typeset above
289	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
290	if [ ${option} = Seasonal ] ; then
291	FoundSeasonal=true
292	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
293	fi
294	done
295	#
296	if [ ! X${FoundSeasonal} = Xtrue ] ; then
297	eval ${compname}_${flag_post}_Seasonal=ON
298	fi
299	#
300	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
301	(( i=i+3 ))
302	continue
303	fi
304	#
305	# CREATE LIST
306	#
307	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
308	#
309	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
310	#
311	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
312	#
313	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
314	if [ ! $? = 0 ] ; then
315	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
316	if [ ! ${RebuildFrequency} = true ] ; then
317	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
318	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
319	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
320	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
321	if [ Xload${Patch} != X ] ; then
322	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
323	eval load${Patch}=loaded
324	fi
325	done
326	fi
327	fi
328	fi
329	fi
330	(( i=i+3 ))
331	done # until [ $i -eq $NbFiles ]
332	fi
333	done # for comp
334
335	for comp in ${config_ListOfComponents[*]} ; do
336	#
337	eval R_OUT_${comp}=${R_SAVE}/${comp}
338	#
339	i=0
340	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
341	#
342	# Determine in which output can we find file
343	# IE : Output/MO or Output/DA or ...
344	#
345	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
346	case ${FlagDir} in
347	1Y) FreqDir=YE ;;
348	1M) FreqDir=MO ;;
349	5D\|1D) FreqDir=DA ;;
350	HF) FreqDir=HF ;;
351	3H) FreqDir=HF ;;
352	INS) FreqDir=INS ;;
353	esac
354	#
355	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
356	# Do not redoo an existing SE
357	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
358	if [ $? = 0 ] ; then
359	continue
360	fi
361	#
362	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
363	#
364	case ${config_UserChoices_PeriodLength} in
365	# For Yearly simulation :
366	Y\|y)
367	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
368	(( an = an_deb ))
369	# Initialize array
370	unset liste_file_se
371	unset liste_file_se_tmp
372	while [ ${an} -le ${an_fin} ] ; do
373	#
374	an4=$(printf "%04i\n" ${an} )
375	#
376	diy=$( IGCM_date_DaysInCurrentPeriod ${an4}0101 ${config_UserChoices_PeriodLength} )
377	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
378	#
379	date=${an4}0101_${afin}
380	#
381	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
382	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
383	(( an = an + ${PeriodLengthInYears} ))
384	done
385	#
386	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
387	IGCM_debug_Verif_Exit
388
389	# Apply IGCM_Patch if needed
390	if [ ! ${RebuildFrequency} = true ] ; then
391	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
392	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
393	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
394	IGCM_${Patch} ${file_t}
395	done
396	done
397	fi
398	fi
399
400	# Search of common list of variables to be treated
401	nbfile=0
402	for file_common_var in ${liste_file_se_tmp[*]} ; do
403	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
404	(( nbfile = nbfile + 1 ))
405	done
406	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
407	IGCM_sys_Rm tmpfile_ncra
408
409	# detect time counter : time or time_counter ie unlimited variable
410	var_unlim=$(ncdump -h ${liste_file_se_tmp[0]}\|grep UNLIMITED\|awk '{print $1}')
411
412	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
413	#
414	(( nb = ${mois} - 1 ))
415	#
416	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
417	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
418	done
419	IGCM_sys_Rm ${liste_file_se_tmp[*]}
420	;;
421	# For Monthly simulation :
422	1M\|1m)
423	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
424	#
425	(( nb = ${mois} - 1 ))
426	#
427	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
428	#
429	(( an = an_deb ))
430	# Initialize array
431	unset liste_file_se
432	unset liste_file_se_tmp
433	while [ ${an} -le ${an_fin} ] ; do
434	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
435	#
436	an4=$(printf "%04i\n" ${an} )
437	#
438	date=${an4}${mois}01_${an4}${mois}${moisL}
439	#
440	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
441	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
442
443	(( an = an + 1 ))
444	done
445	#
446	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
447	IGCM_debug_Verif_Exit
448
449	# Apply IGCM_Patch if needed
450	if [ ! ${RebuildFrequency} = true ] ; then
451	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
452	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
453	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
454	IGCM_${Patch} ${file_t}
455	done
456	done
457	fi
458	fi
459
460	# Search of common list of variables to be treated
461	nbfile=0
462	for file_common_var in ${liste_file_se_tmp[*]} ; do
463	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
464	(( nbfile = nbfile + 1 ))
465	done
466	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
467	IGCM_sys_Rm tmpfile_ncra
468
469	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
470
471	IGCM_sys_Rm ${liste_file_se_tmp[*]}
472	done
473	;;
474	D\|d)
475	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
476	#
477	(( nb = ${mois} - 1 ))
478	#
479	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
480	#
481	(( an = an_deb ))
482	# Initialize array
483	unset liste_file_se
484	unset liste_file_se_tmp
485	while [ ${an} -le ${an_fin} ] ; do
486	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
487	#
488	an4=$(printf "%04i\n" ${an} )
489	#
490	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
491	# for one day
492	if [ ${length_d} -eq 1 ] ; then
493	(( length_d = 0 ))
494	fi
495	(( day = 1 ))
496	while [ ${day} -le ${moisL} ] ; do
497	day2=$(printf "%02i\n" ${day} )
498	day_beg=${an4}${mois}${day2}
499	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
500	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
501	#
502	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
503	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
504	(( day = day + 1 ))
505	done
506	(( an = an + 1 ))
507	done
508	#
509	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
510	IGCM_debug_Verif_Exit
511
512	# Apply IGCM_Patch if needed
513	if [ ! ${RebuildFrequency} = true ] ; then
514	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
515	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
516	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
517	IGCM_${Patch} ${file_t}
518	done
519	done
520	fi
521	fi
522
523	# Search of common list of variables to be treated
524	nbfile=0
525	for file_common_var in ${liste_file_se_tmp[*]} ; do
526	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
527	(( nbfile = nbfile + 1 ))
528	done
529	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
530	IGCM_sys_Rm tmpfile_ncra
531
532	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
533	IGCM_sys_Rm ${liste_file_se_tmp[*]}
534	done
535	;;
536	esac
537	#
538
539	# Search of common list of variables to be treated
540	nbfile=0
541	for file_common_var in ${OUT_SE[*]} ; do
542	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
543	(( nbfile = nbfile + 1 ))
544	done
545	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| grep -v ^time_ \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
546	IGCM_sys_Rm tmpfile_ncrcat
547
548	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
549	#
550	IGCM_sys_Rm ${OUT_SE[*]}
551	#
552	# detect time counter : time or time_counter ie unlimited variable
553	var_unlim=$( ncdump -h ${RESULT_SE} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[[:space:]]*//' )
554
555	# Put correct climatological axis following calendar
556	case ${config_UserChoices_CalendarType} in
557	360d\|360_day)
558	# 360 days
559	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
560	;;
561	noleap\|365_day)
562	# 365 days
563	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
564	;;
565	all_leap\|366_day)
566	# 366 days
567	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
568	;;
569	leap\|gregorian\|standard)
570	# 365.2425 days
571	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
572	;;
573	esac
574	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
575	IGCM_sys_Rm ${RESULT_SE}
576	IGCM_sys_Mv tmp.nc ${RESULT_SE}
577	#
578	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
579	[ -f ${RESULT_SE} ] && IGCM_sys_Rm ${RESULT_SE}
580	#
581	done
582	done
583
584	# DODS copy
585	if [ X${config_Post_DodsCopy} = XTRUE ] ; then
586	for comp in ${config_ListOfComponents[*]} ; do
587	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
588	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
589	done
590	fi
591
592	# Temporal coverage id for atlas
593	YEARS=${an_deb}_${an_fin}
594	DIR_OUT=SE
595
596	export DEBUG_debug; export BigBrother ; export libIGCM; export SUBMIT_DIR; export POST_DIR; export Script_Output ;
597	export StandAlone ; export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ;
598	export RESOL_SBG ; export YEARS; export DIR_OUT
599	listVarEnv="DEBUG_debug,BigBrother,libIGCM,SUBMIT_DIR,POST_DIR,Script_Post_Output,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,YEARS,DIR_OUT"
600	export listVarEnv;
601
602	#D-
603	# --------------------------------------------------------------------
604	#D- Test if all was right before ATLAS
605	# --------------------------------------------------------------------
606	IGCM_debug_Verif_Exit
607
608	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
609	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
610	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
611	# ATLAS
612	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
613	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
614	IGCM_sys_QsubPost atlas_ORCHIDEE ;
615	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
616	IGCM_sys_QsubPost atlas_LMDZ ;
617	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
618	IGCM_sys_QsubPost atlas_ORCA_LIM ;
619	export Script_Post_Output=additionnal.${PeriodDateEnd} ;
620	IGCM_sys_QsubPost additionnal ;
621	fi
622	#
623	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
624	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
625	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
626	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] \|\| \
627	[ ${config_UserChoices_TagName} = "IPSLCM5A2" ] \|\| \
628	[ ${config_UserChoices_TagName} = "IPSLCM6" ] ; then
629	# METRICS
630	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
631	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
632	IGCM_sys_QsubPost metrics_LMDZ ;
633	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
634	IGCM_sys_QsubPost metrics_NEMO ;
635	fi
636	# ATLAS
637	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
638	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
639	IGCM_sys_QsubPost atlas_ORCHIDEE ;
640	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
641	IGCM_sys_QsubPost atlas_LMDZ ;
642	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
643	IGCM_sys_QsubPost atlas_ORCA_LIM ;
644	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
645	IGCM_sys_QsubPost atlas_PISCES ;
646	fi
647	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
648	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
649	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
650	# METRICS
651	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
652	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
653	IGCM_sys_QsubPost metrics_LMDZ ;
654	fi
655	# ATLAS
656	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
657	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
658	IGCM_sys_QsubPost atlas_LMDZ ;
659	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
660	IGCM_sys_QsubPost atlas_ORCHIDEE ;
661	fi
662	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
663	# METRICS
664	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
665	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
666	IGCM_sys_QsubPost metrics_LMDZ ;
667	fi
668	# ATLAS
669	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
670	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
671	IGCM_sys_QsubPost atlas_LMDZ ;
672	fi
673	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
674	[ ${config_UserChoices_TagName} = "OL2" ] ; then
675	# METRICS
676	# ATLAS
677	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
678	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
679	IGCM_sys_QsubPost atlas_ORCHIDEE ;
680	fi
681	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\| \
682	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] \|\| \
683	[ ${config_UserChoices_TagName} = "ORCA2_LIM3" ] \|\| \
684	[ ${config_UserChoices_TagName} = "ORCA1_LIM2" ] \|\| \
685	[ ${config_UserChoices_TagName} = "ORCA1_LIM3" ] ; then
686	# METRICS
687	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
688	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
689	IGCM_sys_QsubPost metrics_NEMO ;
690	fi
691	# ATLAS
692	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
693	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
694	IGCM_sys_QsubPost atlas_ORCA_LIM ;
695	fi
696	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] \|\| \
697	[ ${config_UserChoices_TagName} = "ORCA2_LIM3_PISCES" ] \|\| \
698	[ ${config_UserChoices_TagName} = "ORCA1_LIM2_PISCES" ] \|\| \
699	[ ${config_UserChoices_TagName} = "ORCA1_LIM3_PISCES" ] ; then
700	# METRICS
701	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
702	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
703	IGCM_sys_QsubPost metrics_NEMO ;
704	fi
705	# ATLAS
706	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
707	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
708	IGCM_sys_QsubPost atlas_ORCA_LIM ;
709	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
710	IGCM_sys_QsubPost atlas_PISCES ;
711	fi
712	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
713	# METRICS
714	# ATLAS
715	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
716	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
717	IGCM_sys_QsubPost atlas_PISCES ;
718	fi
719	fi
720
721	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
722	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
723
724	# ------------------------------------------------------------------
725	# Finalize BigBrother to inform that the jobs end
726	# ------------------------------------------------------------------
727	IGCM_debug_BigBro_Finalize

Note: See TracBrowser for help on using the repository browser.

Download in other formats: