Context Navigation

AA_create_se @ 1486

Last change on this file since 1486 was 1468, checked in by mafoipsl, 6 years ago

On curie : set output files for all jobs.

On irene :

set output files for all jobs like curie
add the possibility to use xlarge nodes (for free) for post-processing or skylake nodes
add a question to set project used for post-processing and delete -A option in libIGCM_sys_irene
display only available projects for computing or post-processing on irene

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 28.4 KB

Line
1	#-Q- curie ######################
2	#-Q- curie ## CURIE TGCC/CEA ##
3	#-Q- curie ######################
4	#-Q- curie #MSUB -r SE # Nom du job
5	#-Q- curie #MSUB -o SE.out_%I
6	#-Q- curie #MSUB -e SE.out_%I
7	#-Q- curie #MSUB -n 1 # Reservation du processus
8	#-Q- curie #MSUB -T 36000 # Limite de temps elapsed du job
9	#-Q- curie #MSUB -q ::default_node::
10	#-Q- curie #MSUB -c ::default_core::
11	#-Q- curie #MSUB -Q normal
12	#-Q- curie #MSUB -A ::default_project::
13	#-Q- curie set +x
14	#-Q- irene ######################
15	#-Q- irene ## IRENE TGCC/CEA ##
16	#-Q- irene ######################
17	#-Q- irene #MSUB -r SE # Job name
18	#-Q- irene #MSUB -o SE.out_%I
19	#-Q- irene #MSUB -e SE.out_%I
20	#-Q- irene #MSUB -n 1
21	#-Q- irene #MSUB -T 36000 # Maximum elapsed time
22	#-Q- irene #MSUB -q ::default_node::
23	#-Q- irene #MSUB -c ::default_core::
24	#-Q- irene #MSUB -Q normal
25	#-Q- irene #MSUB -A ::default_post_project::
26	#-Q- irene #MSUB -m store,work,scratch
27	#-Q- irene set +x
28	#-Q- ada #!/bin/ksh
29	#-Q- ada #######################
30	#-Q- ada ## ADA IDRIS ##
31	#-Q- ada #######################
32	#-Q- ada # @ job_type = mpich
33	#-Q- ada # @ requirements = (Feature == "prepost")
34	#-Q- ada # Temps Elapsed max. d'une requete hh:mm:ss
35	#-Q- ada # @ wall_clock_limit = 10:00:00
36	#-Q- ada # Memory required for ncrcat
37	#-Q- ada # @ as_limit = 30Gb
38	#-Q- ada # Nom du travail LoadLeveler
39	#-Q- ada # @ job_name = SE
40	#-Q- ada # Fichier de sortie standard du travail
41	#-Q- ada # @ output = $(job_name).$(jobid)
42	#-Q- ada # Fichier de sortie d'erreur du travail
43	#-Q- ada # @ error = $(job_name).$(jobid)
44	#-Q- ada # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
45	#-Q- ada # @ notification = error
46	#-Q- ada # @ environment = $DEBUG_debug ; $BigBrother ; $postProcessingStopLevel ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $POST_DIR ; $Script_Post_Output ; $MASTER ; $DateBegin ; $PeriodDateEnd ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; wall_clock_limit=$(wall_clock_limit)
47	#-Q- ada # @ queue
48	#-Q- lxiv8 ######################
49	#-Q- lxiv8 ## OBELIX LSCE ##
50	#-Q- lxiv8 ######################
51	#-Q- lxiv8 #PBS -N SE
52	#-Q- lxiv8 #PBS -m a
53	#-Q- lxiv8 #PBS -j oe
54	#-Q- lxiv8 #PBS -q medium
55	#-Q- lxiv8 #PBS -o SE.$$
56	#-Q- lxiv8 #PBS -S /bin/ksh
57	#-Q- ifort_CICLAD ######################
58	#-Q- ifort_CICLAD ## CICLAD IPSL ##
59	#-Q- ifort_CICLAD ######################
60	#-Q- ifort_CICLAD #PBS -N SE
61	#-Q- ifort_CICLAD #PBS -m a
62	#-Q- ifort_CICLAD #PBS -j oe
63	#-Q- ifort_CICLAD ###PBS -q h12 # Queue for 12 hours at ciclad only
64	#-Q- ifort_CICLAD #PBS -S /bin/ksh
65	#-Q- default #!/bin/ksh
66	#-Q- default ##################
67	#-Q- default ## DEFAULT HOST ##
68	#-Q- default ##################
69
70	#**************************************************************
71	# Author: Sebastien Denvil
72	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
73	# $Revision:: $ Revision of last commit
74	# $Author:: $ Author of last commit
75	# $Date:: $ Date of last commit
76	# IPSL (2006)
77	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
78	#
79	#**************************************************************
80
81	#set -eu
82	#set -vx
83
84	date
85
86	#-Q- ada export OMP_NUM_THREADS=1
87
88	#D- Task type DO NOT CHANGE (computing, post-processing or checking)
89	TaskType=post-processing
90
91	########################################################################
92
93	#D- Flag to determine if this job in a standalone mode
94	#D- Default : value from AA_job if any
95	StandAlone=${StandAlone:=true}
96
97	#D- Path to libIGCM
98	#D- Default : value from AA_job if any
99	libIGCM=${libIGCM:=::modipsl::/libIGCM}
100
101	#D- Flag to determine if rebuild process has been performed asynchronously
102	#D- Possible value true or false.
103	#D- If true raw files has allready been patched by rebuild job
104	#D- If false create_ts will patch the raw files
105	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
106	RebuildFrequency=${RebuildFrequency:=true}
107
108	#D- Flag to determine atlas job's output directory
109	#D- Default : value from libIGCM_post.ksh if any
110	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
111
112	#D- Increased verbosity (1, 2, 3)
113	#D- Default : value from AA_job if any
114	Verbosity=${Verbosity:=3}
115
116	#D- Low level debug : to bypass lib test checks and stack construction
117	#D- Default : value from AA_job if any
118	DEBUG_debug=${DEBUG_debug:=false}
119
120	#D- TEMPORARY Flag to determine atmospheric resolution
121	#D- Default : value from atmospheric driver if any
122	RESOL_ATM=${RESOL_ATM:=ALL}
123
124	#D- Flag to determine surface resolution
125	#D- Default : value from surface driver if any
126	RESOL_SRF=ALL
127
128	#D- Flag to determine surface resolution
129	#D- Default : value from surface driver if any
130	RESOL_SBG=ALL
131
132	#D- TEMPORARY Flag to determine ocean resolution
133	#D- Default : value from ocean driver if any
134	RESOL_OCE=${RESOL_OCE:=ORCA2}
135
136	#D- TEMPORARY Flag to determine ice resolution
137	#D- Default : value from ice driver if any
138	RESOL_ICE=${RESOL_OCE:=ORCA2}
139
140	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
141	#D- Default : value from ice driver if any
142	RESOL_MBG=${RESOL_OCE:=ORCA2}
143
144	########################################################################
145
146	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
147	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
148	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
149	#-------
150	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
151	. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
152	#-------
153	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
154	IGCM_sys_Cd ${RUN_DIR_PATH}
155	#-------
156	( ${DEBUG_debug} ) && IGCM_debug_Check
157	( ${DEBUG_debug} ) && IGCM_card_Check
158	( ${DEBUG_debug} ) && IGCM_date_Check
159
160	########################################################################
161
162	#set -vx
163
164	# ------------------------------------------------------------------
165	# Test if all was right before proceeding further
166	# ------------------------------------------------------------------
167	IGCM_debug_Verif_Exit
168
169	if [ ${StandAlone} = true ] ; then
170	CARD_DIR=${SUBMIT_DIR}
171	else
172	CARD_DIR=${RUN_DIR_PATH}
173	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
174	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
175	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
176	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
177	fi
178
179	#==================================
180	# First of all
181	#
182	# Read libIGCM compatibility version in config.card
183	# Read UserChoices section
184	# Read Ensemble section
185	# Read Post section
186	# Define all netcdf output directories
187	#==================================
188	IGCM_config_CommonConfiguration ${CARD_DIR}/config.card
189
190	# ------------------------------------------------------------------
191	# Activate BigBrother so as to supervise this job
192	# ------------------------------------------------------------------
193	IGCM_debug_BigBro_Initialize
194
195	#==================================
196	# Define default value to keep compatibility with previous card: means before changes due to TGCC
197	if [ X${PackDefault} = Xtrue ] ; then
198	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
199	else
200	config_Post_PackFrequency=NONE
201	fi
202
203	#==================================
204	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
205	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
206	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
207	fi
208
209	#==================================
210	# Read ListOfComponents section:
211	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
212
213	#==================================
214	# Read SeasonalFrequency:
215	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
216
217	#
218	# Determine period for seasonnal average. Default : value from AA_job if any
219	#
220	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
221	#
222	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
223	#
224	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
225	#
226	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
227	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
228	an_fin=$( IGCM_date_YearDigit ${an_fin} )
229	#
230	# For each component selected determine which files need post-processing
231	#
232	for comp in ${config_ListOfComponents[*]} ; do
233	#
234	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
235	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
236	#
237	card=${CARD_DIR}/COMP/${compname}.card
238	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
239	#
240	ListFilesName=${compname}_OutputFiles_List
241	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
242	#
243	if [ X${FileName0} != X${NULL_STR} ] ; then
244	#
245	# INITIALISATION
246	#
247	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
248	typeset i=2
249	#
250	until [ $i -gt $NbFiles ]; do
251	#
252	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
253	#
254	if [ X${flag_post} != XNONE ] ; then
255	#
256	# Check if seasonal is required for this file
257	#
258	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
259	#
260	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
261	#
262	# variable option allready typeset above
263	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
264	if [ ${option} = Seasonal ] ; then
265	FoundSeasonal=true
266	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
267	fi
268	done
269	#
270	if [ ! X${FoundSeasonal} = Xtrue ] ; then
271	eval ${compname}_${flag_post}_Seasonal=ON
272	fi
273	#
274	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
275	(( i=i+3 ))
276	continue
277	fi
278	#
279	# CREATE LIST
280	#
281	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
282	#
283	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
284	#
285	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
286	#
287	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
288	if [ ! $? = 0 ] ; then
289	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
290	if [ ! ${RebuildFrequency} = true ] ; then
291	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
292	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
293	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
294	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
295	if [ Xload${Patch} != X ] ; then
296	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
297	eval load${Patch}=loaded
298	fi
299	done
300	fi
301	fi
302	fi
303	fi
304	(( i=i+3 ))
305	done # until [ $i -eq $NbFiles ]
306	fi
307	done # for comp
308
309	for comp in ${config_ListOfComponents[*]} ; do
310	#
311	eval R_OUT_${comp}=${R_SAVE}/${comp}
312	#
313	i=0
314	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
315	#
316	# Determine in which output can we find file
317	# IE : Output/MO or Output/DA or ...
318	#
319	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
320	case ${FlagDir} in
321	1Y) FreqDir=YE ;;
322	1M) FreqDir=MO ;;
323	5D\|1D) FreqDir=DA ;;
324	HF) FreqDir=HF ;;
325	3H) FreqDir=HF ;;
326	INS) FreqDir=INS ;;
327	esac
328	#
329	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
330	# Do not redoo an existing SE
331	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
332	if [ $? = 0 ] ; then
333	continue
334	fi
335	#
336	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
337	#
338	case ${config_UserChoices_PeriodLength} in
339	# For Yearly simulation :
340	Y\|y)
341	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
342	(( an = an_deb ))
343	# Initialize array
344	unset liste_file_se
345	unset liste_file_se_tmp
346	while [ ${an} -le ${an_fin} ] ; do
347	#
348	an4=$(printf "%04i\n" ${an} )
349	#
350	diy=$( IGCM_date_DaysInCurrentPeriod ${an4}0101 ${config_UserChoices_PeriodLength} )
351	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
352	#
353	date=${an4}0101_${afin}
354	#
355	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
356	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
357	(( an = an + ${PeriodLengthInYears} ))
358	done
359	#
360	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
361	IGCM_debug_Verif_Exit
362
363	# Apply IGCM_Patch if needed
364	if [ ! ${RebuildFrequency} = true ] ; then
365	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
366	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
367	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
368	IGCM_${Patch} ${file_t}
369	done
370	done
371	fi
372	fi
373
374	# Search of common list of variables to be treated
375	nbfile=0
376	for file_common_var in ${liste_file_se_tmp[*]} ; do
377	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
378	(( nbfile = nbfile + 1 ))
379	done
380	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
381	IGCM_sys_Rm tmpfile_ncra
382
383	# detect time counter : time or time_counter ie unlimited variable
384	var_unlim=$(ncdump -h ${liste_file_se_tmp[0]}\|grep UNLIMITED\|awk '{print $1}')
385
386	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
387	#
388	(( nb = ${mois} - 1 ))
389	#
390	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
391	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
392	done
393	IGCM_sys_Rm ${liste_file_se_tmp[*]}
394	;;
395	# For Monthly simulation :
396	1M\|1m)
397	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
398	#
399	(( nb = ${mois} - 1 ))
400	#
401	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
402	#
403	(( an = an_deb ))
404	# Initialize array
405	unset liste_file_se
406	unset liste_file_se_tmp
407	while [ ${an} -le ${an_fin} ] ; do
408	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
409	#
410	an4=$(printf "%04i\n" ${an} )
411	#
412	date=${an4}${mois}01_${an4}${mois}${moisL}
413	#
414	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
415	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
416
417	(( an = an + 1 ))
418	done
419	#
420	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
421	IGCM_debug_Verif_Exit
422
423	# Apply IGCM_Patch if needed
424	if [ ! ${RebuildFrequency} = true ] ; then
425	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
426	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
427	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
428	IGCM_${Patch} ${file_t}
429	done
430	done
431	fi
432	fi
433
434	# Search of common list of variables to be treated
435	nbfile=0
436	for file_common_var in ${liste_file_se_tmp[*]} ; do
437	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
438	(( nbfile = nbfile + 1 ))
439	done
440	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
441	IGCM_sys_Rm tmpfile_ncra
442
443	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
444
445	IGCM_sys_Rm ${liste_file_se_tmp[*]}
446	done
447	;;
448	D\|d)
449	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
450	#
451	(( nb = ${mois} - 1 ))
452	#
453	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
454	#
455	(( an = an_deb ))
456	# Initialize array
457	unset liste_file_se
458	unset liste_file_se_tmp
459	while [ ${an} -le ${an_fin} ] ; do
460	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
461	#
462	an4=$(printf "%04i\n" ${an} )
463	#
464	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
465	# for one day
466	if [ ${length_d} -eq 1 ] ; then
467	(( length_d = 0 ))
468	fi
469	(( day = 1 ))
470	while [ ${day} -le ${moisL} ] ; do
471	day2=$(printf "%02i\n" ${day} )
472	day_beg=${an4}${mois}${day2}
473	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
474	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
475	#
476	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
477	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
478	(( day = day + 1 ))
479	done
480	(( an = an + 1 ))
481	done
482	#
483	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
484	IGCM_debug_Verif_Exit
485
486	# Apply IGCM_Patch if needed
487	if [ ! ${RebuildFrequency} = true ] ; then
488	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
489	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
490	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
491	IGCM_${Patch} ${file_t}
492	done
493	done
494	fi
495	fi
496
497	# Search of common list of variables to be treated
498	nbfile=0
499	for file_common_var in ${liste_file_se_tmp[*]} ; do
500	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
501	(( nbfile = nbfile + 1 ))
502	done
503	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
504	IGCM_sys_Rm tmpfile_ncra
505
506	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
507	IGCM_sys_Rm ${liste_file_se_tmp[*]}
508	done
509	;;
510	esac
511	#
512
513	# Search of common list of variables to be treated
514	nbfile=0
515	for file_common_var in ${OUT_SE[*]} ; do
516	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
517	(( nbfile = nbfile + 1 ))
518	done
519	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| grep -v ^time_ \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
520	IGCM_sys_Rm tmpfile_ncrcat
521
522	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
523	#
524	IGCM_sys_Rm ${OUT_SE[*]}
525	#
526	# detect time counter : time or time_counter ie unlimited variable
527	var_unlim=$( ncdump -h ${RESULT_SE} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[[:space:]]*//' )
528
529	# Put correct climatological axis following calendar
530	case ${config_UserChoices_CalendarType} in
531	360d\|360_day)
532	# 360 days
533	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
534	;;
535	noleap\|365_day)
536	# 365 days
537	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
538	;;
539	all_leap\|366_day)
540	# 366 days
541	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
542	;;
543	leap\|gregorian\|standard)
544	# 365.2425 days
545	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
546	;;
547	esac
548	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
549	IGCM_sys_Rm ${RESULT_SE}
550	IGCM_sys_Mv tmp.nc ${RESULT_SE}
551	#
552	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
553	[ -f ${RESULT_SE} ] && IGCM_sys_Rm ${RESULT_SE}
554	#
555	done
556	done
557
558	# DODS copy
559	if [ X${config_Post_DodsCopy} = XTRUE ] ; then
560	for comp in ${config_ListOfComponents[*]} ; do
561	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
562	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
563	done
564	fi
565
566	# Temporal coverage id for atlas
567	YEARS=${an_deb}_${an_fin}
568	DIR_OUT=SE
569
570	export DEBUG_debug; export BigBrother ; export libIGCM; export SUBMIT_DIR; export POST_DIR; export Script_Output ;
571	export StandAlone ; export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ;
572	export RESOL_SBG ; export YEARS; export DIR_OUT
573	listVarEnv="DEBUG_debug,BigBrother,libIGCM,SUBMIT_DIR,POST_DIR,Script_Post_Output,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,YEARS,DIR_OUT"
574	export listVarEnv;
575
576	#D-
577	# --------------------------------------------------------------------
578	#D- Test if all was right before ATLAS
579	# --------------------------------------------------------------------
580	IGCM_debug_Verif_Exit
581
582	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
583	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
584	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
585	# ATLAS
586	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
587	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
588	IGCM_sys_QsubPost atlas_ORCHIDEE ;
589	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
590	IGCM_sys_QsubPost atlas_LMDZ ;
591	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
592	IGCM_sys_QsubPost atlas_ORCA_LIM ;
593	export Script_Post_Output=additionnal.${PeriodDateEnd} ;
594	IGCM_sys_QsubPost additionnal ;
595	fi
596	#
597	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
598	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
599	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
600	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] \|\| \
601	[ ${config_UserChoices_TagName} = "IPSLCM5A2" ] \|\| \
602	[ ${config_UserChoices_TagName} = "IPSLCM6" ] ; then
603	# METRICS
604	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
605	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
606	IGCM_sys_QsubPost metrics_LMDZ ;
607	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
608	IGCM_sys_QsubPost metrics_NEMO ;
609	fi
610	# ATLAS
611	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
612	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
613	IGCM_sys_QsubPost atlas_ORCHIDEE ;
614	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
615	IGCM_sys_QsubPost atlas_LMDZ ;
616	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
617	IGCM_sys_QsubPost atlas_ORCA_LIM ;
618	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
619	IGCM_sys_QsubPost atlas_PISCES ;
620	fi
621	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
622	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
623	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
624	# METRICS
625	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
626	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
627	IGCM_sys_QsubPost metrics_LMDZ ;
628	fi
629	# ATLAS
630	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
631	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
632	IGCM_sys_QsubPost atlas_LMDZ ;
633	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
634	IGCM_sys_QsubPost atlas_ORCHIDEE ;
635	fi
636	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
637	# METRICS
638	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
639	export Script_Post_Output=metrics_LMDZ.${PeriodDateEnd} ;
640	IGCM_sys_QsubPost metrics_LMDZ ;
641	fi
642	# ATLAS
643	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
644	export Script_Post_Output=atlas_LMDZ.${PeriodDateEnd} ;
645	IGCM_sys_QsubPost atlas_LMDZ ;
646	fi
647	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
648	[ ${config_UserChoices_TagName} = "OL2" ] ; then
649	# METRICS
650	# ATLAS
651	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
652	export Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
653	IGCM_sys_QsubPost atlas_ORCHIDEE ;
654	fi
655	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\| \
656	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] \|\| \
657	[ ${config_UserChoices_TagName} = "ORCA2_LIM3" ] \|\| \
658	[ ${config_UserChoices_TagName} = "ORCA1_LIM2" ] \|\| \
659	[ ${config_UserChoices_TagName} = "ORCA1_LIM3" ] ; then
660	# METRICS
661	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
662	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
663	IGCM_sys_QsubPost metrics_NEMO ;
664	fi
665	# ATLAS
666	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
667	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
668	IGCM_sys_QsubPost atlas_ORCA_LIM ;
669	fi
670	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] \|\| \
671	[ ${config_UserChoices_TagName} = "ORCA2_LIM3_PISCES" ] \|\| \
672	[ ${config_UserChoices_TagName} = "ORCA1_LIM2_PISCES" ] \|\| \
673	[ ${config_UserChoices_TagName} = "ORCA1_LIM3_PISCES" ] ; then
674	# METRICS
675	if [ X${config_Post_MetricsPCMDI} = XTRUE ] ; then
676	export Script_Post_Output=metrics_NEMO.${PeriodDateEnd} ;
677	IGCM_sys_QsubPost metrics_NEMO ;
678	fi
679	# ATLAS
680	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
681	export Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
682	IGCM_sys_QsubPost atlas_ORCA_LIM ;
683	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
684	IGCM_sys_QsubPost atlas_PISCES ;
685	fi
686	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
687	# METRICS
688	# ATLAS
689	if [ X${config_Post_AtlasIPSL} = XTRUE ] ; then
690	export Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
691	IGCM_sys_QsubPost atlas_PISCES ;
692	fi
693	fi
694
695	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
696	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
697
698	# ------------------------------------------------------------------
699	# Finalize BigBrother to inform that the jobs end
700	# ------------------------------------------------------------------
701	IGCM_debug_BigBro_Finalize

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

Original Format