Context Navigation

AA_create_se @ 544

Last change on this file since 544 was 544, checked in by sdipsl, 13 years ago

Prepare libIGCM for TGCC migration. For now work for CCRT machines ONLY
SX8, SX9 and Titane are independant from DMNFS
- 30 seconds between execution instead of at least 3 minutes ; sometimes hours
SX8, SX9 and Titane use their SCRATCHDIR as a BUFFER (for restart and text)
Titane simulations are entirely post-processed on titane mono
- Working except create_se and atlas (missing software : ncap2, openpbm, ImageMagick?)
Cards are the same up to now
New indentation to ease readability ("svn diff -x -b" to ignore space when differencing)
Project (gen2211) is inherited from AA_job by post-processing jobs
- will stop during config_check if Project is empty
Prepare easy switching (submitting post-processing here or there)
Homogenize CCRTs libIGCM_sys
Bugfixes
Cosmetics

First steps and still a long way to go.

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 28.5 KB

Line
1	#-Q- cesium #!/bin/ksh
2	#-Q- cesium ######################
3	#-Q- cesium ## CESIUM CEA ##
4	#-Q- cesium ######################
5	#-Q- cesium #MSUB -r SE # Nom du job
6	#-Q- cesium #MSUB -N 1 # Reservation du noeud
7	#-Q- cesium #MSUB -n 1 # Reservation du processus
8	#-Q- cesium #MSUB -T 86400 # Limite de temps elapsed du job
9	#-Q- cesium #MSUB -E "-j o"
10	#-Q- cesium #MSUB -E "-S /bin/ksh"
11	#-Q- platine #!/usr/bin/ksh
12	#-Q- platine ###################
13	#-Q- platine ## PLATINE CEA ##
14	#-Q- platine ###################
15	#-Q- platine #BSUB -J SE # Nom du job
16	#-Q- platine #BSUB -N # message a la fin du job
17	#-Q- platine #BSUB -n 1 # reservation des processeurs pour le job
18	#-Q- platine #BSUB -W 1:00 # Limite temps
19	#-Q- platine #BSUB -q post # Passage en queue post
20	#-Q- platine #BSUB -L /bin/ksh
21	#-Q- sx8brodie #!/bin/ksh
22	#-Q- sx8brodie #######################
23	#-Q- sx8brodie ## SX8BRODIE IDRIS ##
24	#-Q- sx8brodie #######################
25	#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
26	#-Q- sx8brodie # @ wall_clock_limit = 10:00:00
27	#-Q- sx8brodie # Nom du travail LoadLeveler
28	#-Q- sx8brodie # @ job_name = SE
29	#-Q- sx8brodie # Fichier de sortie standard du travail
30	#-Q- sx8brodie # @ output = $(job_name).$(jobid)
31	#-Q- sx8brodie # Fichier de sortie d'erreur du travail
32	#-Q- sx8brodie # @ error = $(job_name).$(jobid)
33	#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
34	#-Q- sx8brodie # @ notification = error
35	#-Q- sx8brodie # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
36	#-Q- sx8brodie # @ queue
37	#-Q- aix6 #!/bin/ksh
38	#-Q- aix6 #######################
39	#-Q- aix6 ## VARGAS IDRIS ##
40	#-Q- aix6 #######################
41	#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
42	#-Q- aix6 # @ wall_clock_limit = 10:00:00
43	#-Q- aix6 # Nom du travail LoadLeveler
44	#-Q- aix6 # @ job_name = SE
45	#-Q- aix6 # Fichier de sortie standard du travail
46	#-Q- aix6 # @ output = $(job_name).$(jobid)
47	#-Q- aix6 # Fichier de sortie d'erreur du travail
48	#-Q- aix6 # @ error = $(job_name).$(jobid)
49	#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
50	#-Q- aix6 # @ notification = error
51	#-Q- aix6 # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
52	#-Q- aix6 # @ queue
53	#-Q- ulam #!/bin/ksh
54	#-Q- ulam #######################
55	#-Q- ulam ## ULAM IDRIS ##
56	#-Q- ulam #######################
57	#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
58	#-Q- ulam # @ wall_clock_limit = 10:00:00
59	#-Q- ulam # Nom du travail LoadLeveler
60	#-Q- ulam # @ job_name = SE
61	#-Q- ulam # Fichier de sortie standard du travail
62	#-Q- ulam # @ output = $(job_name).$(jobid)
63	#-Q- ulam # Fichier de sortie d'erreur du travail
64	#-Q- ulam # @ error = $(job_name).$(jobid)
65	#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
66	#-Q- ulam # @ notification = error
67	#-Q- ulam # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
68	#-Q- ulam # @ queue
69	#-Q- sx8mercure #!/bin/ksh
70	#-Q- sx8mercure ######################
71	#-Q- sx8mercure ## SX8MERCURE CEA ##
72	#-Q- sx8mercure ######################
73	#-Q- sx8mercure #PBS -N SE # Nom du job
74	#-Q- sx8mercure #PBS -j o # regroupement des stdout et stderr
75	#-Q- sx8mercure #PBS -S /usr/bin/ksh # shell de soumission
76	#-Q- sx8mercure #PBS -l memsz_job=1gb # Limite memoire a 1 Go
77	#-Q- sx8mercure #PBS -l cputim_job=1:00:00 # Limite temps a 1 heures
78	#-Q- sx8mercure #PBS -q scalaire
79	#-Q- sx9mercure #!/bin/ksh
80	#-Q- sx9mercure #########################
81	#-Q- sx9mercure ## CESIUM FOR SX9 CEA ##
82	#-Q- sx9mercure #########################
83	#-Q- sx9mercure #MSUB -r SE # Nom du job
84	#-Q- sx9mercure #MSUB -N 1 # Reservation du noeud
85	#-Q- sx9mercure #MSUB -n 1 # Reservation du processus
86	#-Q- sx9mercure #MSUB -T 86400 # Limite de temps elapsed du job
87	#-Q- sx9mercure #MSUB -E "-j o"
88	#-Q- sx9mercure #MSUB -E "-S /bin/ksh"
89	#-Q- titane #!/bin/ksh
90	#-Q- titane ######################
91	#-Q- titane ## TITANE CEA ##
92	#-Q- titane ######################
93	#-Q- titane #MSUB -r SE # Nom du job
94	#-Q- titane #MSUB -eo
95	#-Q- titane #MSUB -n 1 # Reservation du processus
96	#-Q- titane #MSUB -T 86400 # Limite de temps elapsed du job
97	#-Q- titane #MSUB -q mono
98	#-Q- lxiv8 ######################
99	#-Q- lxiv8 ## OBELIX LSCE ##
100	#-Q- lxiv8 ######################
101	#-Q- lxiv8 #PBS -N SE
102	#-Q- lxiv8 #PBS -m a
103	#-Q- lxiv8 #PBS -j oe
104	#-Q- lxiv8 #PBS -q medium
105	#-Q- lxiv8 #PBS -o SE.$$
106	#-Q- lxiv8 #PBS -S /bin/ksh
107	#-Q- default #!/bin/ksh
108	#-Q- default ##################
109	#-Q- default ## DEFAULT HOST ##
110	#-Q- default ##################
111
112	#**************************************************************
113	# Author: Sebastien Denvil
114	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
115	# $Revision:: $ Revision of last commit
116	# $Author:: $ Author of last commit
117	# $Date:: $ Date of last commit
118	# IPSL (2006)
119	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
120	#
121	#**************************************************************
122
123	#set -eu
124	#set -vx
125
126	date
127
128	#-Q- sx8brodie export OMP_NUM_THREADS=1
129	#-Q- aix6 export OMP_NUM_THREADS=1
130	#-Q- ulam export OMP_NUM_THREADS=1
131
132	########################################################################
133
134	#D- Flag to determine if this job in a standalone mode
135	#D- Default : value from AA_job if any
136	StandAlone=${StandAlone:=true}
137
138	#D- Low level debug : to bypass lib test checks and stack construction
139	#D- Default : value from AA_job if any
140	libIGCM=${libIGCM:=::modipsl::/libIGCM}
141	# WARNING for StandAlone used : To run this script on some machine,
142	# you must check MirrorlibIGCM variable in sys library.
143	# If this variable is true, you must use libIGCM_POST path instead
144	# of your running libIGCM directory.
145
146	#D- Flag to determine if rebuild process has been performed asynchronously
147	#D- Possible value true or false.
148	#D- If true raw files has allready been patched by rebuild job
149	#D- If false create_ts will patch the raw files
150	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
151	RebuildFrequency=${RebuildFrequency:=true}
152
153	#D- Flag to determine atlas job's output directory
154	#D- Default : value from libIGCM_post.ksh if any
155	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
156
157	#D- Increased verbosity (1, 2, 3)
158	#D- Default : value from AA_job if any
159	Verbosity=${Verbosity:=3}
160
161	#D- Low level debug : to bypass lib test checks and stack construction
162	#D- Default : value from AA_job if any
163	DEBUG_debug=${DEBUG_debug:=false}
164
165	#D- TEMPORARY Flag to determine atmospheric resolution
166	#D- Default : value from atmospheric driver if any
167	RESOL_ATM=${RESOL_ATM:=ALL}
168
169	#D- Flag to determine surface resolution
170	#D- Default : value from surface driver if any
171	RESOL_SRF=ALL
172
173	#D- Flag to determine surface resolution
174	#D- Default : value from surface driver if any
175	RESOL_SBG=ALL
176
177	#D- TEMPORARY Flag to determine ocean resolution
178	#D- Default : value from ocean driver if any
179	RESOL_OCE=${RESOL_OCE:=ORCA2}
180
181	#D- TEMPORARY Flag to determine ice resolution
182	#D- Default : value from ice driver if any
183	RESOL_ICE=${RESOL_ICE:=ORCA2}
184
185	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
186	#D- Default : value from ice driver if any
187	RESOL_MBG=${RESOL_MBG:=ORCA2}
188
189	########################################################################
190
191	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
192	( ${DEBUG_debug} ) && IGCM_debug_Check
193	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
194	( ${DEBUG_debug} ) && IGCM_card_Check
195	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
196	( ${DEBUG_debug} ) && IGCM_date_Check
197	#-------
198	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
199
200	########################################################################
201
202	#set -vx
203
204	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
205	IGCM_sys_Cd ${RUN_DIR_PATH}
206
207	# ------------------------------------------------------------------
208	# Test if all was right before proceeding further
209	# ------------------------------------------------------------------
210	IGCM_debug_Verif_Exit_Post
211
212	if [ ${StandAlone} = true ] ; then
213	CARD_DIR=${SUBMIT_DIR}
214	else
215	CARD_DIR=${RUN_DIR_PATH}/$( basename ${SUBMIT_DIR} )
216	IGCM_sys_Get_Master ${SUBMIT_DIR} ${RUN_DIR_PATH}
217	fi
218
219	#
220	# First of all
221	#
222	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card UserChoices
223	typeset option
224	for option in ${config_UserChoices[*]} ; do
225	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
226	done
227	#
228	echo
229	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
230	IGCM_debug_PrintVariables 3 config_UserChoices_JobName
231	IGCM_debug_PrintVariables 3 config_UserChoices_LongName
232	IGCM_debug_PrintVariables 3 config_UserChoices_TagName
233	IGCM_debug_PrintVariables 3 config_UserChoices_SpaceName
234	IGCM_debug_PrintVariables 3 config_UserChoices_ExperimentName
235	IGCM_debug_PrintVariables 3 config_UserChoices_CalendarType
236	IGCM_debug_PrintVariables 3 config_UserChoices_DateBegin
237	IGCM_debug_PrintVariables 3 config_UserChoices_DateEnd
238	IGCM_debug_PrintVariables 3 config_UserChoices_PeriodLength
239	echo
240
241	#==================================
242	#R_SAVE : Job output directory
243	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
244	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
245	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
246	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
247	else
248	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
249	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
250	fi
251	#
252	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
253	#
254	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
255	#
256	# Determine period for seasonnal average. Default : value from AA_job if any
257	#
258	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
259	#
260	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
261	#
262	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
263	#
264	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
265	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
266	an_fin=$( IGCM_date_YearDigit ${an_fin} )
267	#
268	# For each component selected determine which files need post-processing
269	#
270	for comp in ${config_ListOfComponents[*]} ; do
271	#
272	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
273	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
274	#
275	card=${CARD_DIR}/COMP/${compname}.card
276	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
277	#
278	ListFilesName=${compname}_OutputFiles_List
279	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
280	#
281	if [ X${FileName0} != X${NULL_STR} ] ; then
282	#
283	# INITIALISATION
284	#
285	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
286	typeset i=2
287	#
288	until [ $i -gt $NbFiles ]; do
289	#
290	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
291	#
292	if [ X${flag_post} != XNONE ] ; then
293	#
294	# Check if seasonal is required for this file
295	#
296	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
297	#
298	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
299	#
300	# variable option allready typeset above
301	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
302	if [ ${option} = Seasonal ] ; then
303	FoundSeasonal=true
304	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
305	fi
306	done
307	#
308	if [ ! X${FoundSeasonal} = Xtrue ] ; then
309	eval ${compname}_${flag_post}_Seasonal=ON
310	fi
311	#
312	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
313	(( i=i+3 ))
314	continue
315	fi
316	#
317	# CREATE LIST
318	#
319	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
320	#
321	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
322	#
323	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
324	#
325	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
326	if [ ! $? = 0 ] ; then
327	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
328	if [ ! ${RebuildFrequency} = true ] ; then
329	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
330	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
331	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
332	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
333	if [ Xload${Patch} != X ] ; then
334	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
335	eval load${Patch}=loaded
336	fi
337	done
338	fi
339	fi
340	fi
341	fi
342	(( i=i+3 ))
343	done # until [ $i -eq $NbFiles ]
344	fi
345	done # for comp
346
347	for comp in ${config_ListOfComponents[*]} ; do
348	#
349	eval R_OUT_${comp}=${R_SAVE}/${comp}
350	#
351	i=0
352	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
353	#
354	# Determine in which output can we find file
355	# IE : Output/MO or Output/DA or ...
356	#
357	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
358	case ${FlagDir} in
359	1Y) FreqDir=YE ;;
360	1M) FreqDir=MO ;;
361	5D\|1D) FreqDir=DA ;;
362	HF) FreqDir=HF ;;
363	3H) FreqDir=HF ;;
364	INS) FreqDir=INS ;;
365	esac
366	#
367	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
368	# Do not redoo an existing SE
369	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
370	if [ $? = 0 ] ; then
371	continue
372	fi
373	#
374	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
375	#
376	case ${config_UserChoices_PeriodLength} in
377	# For Yearly simulation :
378	1Y\|1y)
379	(( an = an_deb ))
380	# Initialize array
381	#set -A liste_file_se
382	#set -A liste_file_se_tmp
383	unset liste_file_se
384	unset liste_file_se_tmp
385	while [ ${an} -le ${an_fin} ] ; do
386	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
387	#
388	an4=$(printf "%04i\n" ${an} )
389	#
390	diy=$( IGCM_date_DaysInYear $an4 )
391	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
392	date=${an4}0101_${afin}
393	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
394	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
395	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
396	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
397	(( an = an + 1 ))
398	done
399	#
400	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
401	IGCM_debug_Verif_Exit_Post
402
403	# Apply IGCM_Patch if needed
404	if [ ! ${RebuildFrequency} = true ] ; then
405	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
406	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
407	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
408	IGCM_${Patch} ${file_t}
409	done
410	done
411	fi
412	fi
413
414	# Search of common list of variables to be treated
415	nbfile=0
416	for file_common_var in ${liste_file_se_tmp[*]} ; do
417	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
418	(( nbfile = nbfile + 1 ))
419	done
420	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
421	IGCM_sys_Rm tmpfile_ncra
422
423	# detect time counter : time or time_counter ie unlimited variable
424	var_unlim=$(ncdump -h ${liste_file_se_tmp[1]}\|grep UNLIMITED\| cut -d ' ' -f 1)
425
426	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
427	#
428	(( nb = ${mois} - 1 ))
429	#
430	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
431	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
432	done
433	IGCM_sys_Rm ${liste_file_se_tmp[*]}
434	;;
435	# For Monthly simulation :
436	1M\|1m)
437	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
438	#
439	(( nb = ${mois} - 1 ))
440	#
441	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
442	#
443	(( an = an_deb ))
444	# Initialize array
445	#set -A liste_file_se
446	#set -A liste_file_se_tmp
447	unset liste_file_se
448	unset liste_file_se_tmp
449	while [ ${an} -le ${an_fin} ] ; do
450	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
451	#
452	an4=$(printf "%04i\n" ${an} )
453	#
454	date=${an4}${mois}01_${an4}${mois}${moisL}
455	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
456	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
457	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
458	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
459
460	(( an = an + 1 ))
461	done
462	#
463	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
464	IGCM_debug_Verif_Exit_Post
465
466	# Apply IGCM_Patch if needed
467	if [ ! ${RebuildFrequency} = true ] ; then
468	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
469	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
470	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
471	IGCM_${Patch} ${file_t}
472	done
473	done
474	fi
475	fi
476
477	# Search of common list of variables to be treated
478	nbfile=0
479	for file_common_var in ${liste_file_se_tmp[*]} ; do
480	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
481	(( nbfile = nbfile + 1 ))
482	done
483	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
484	IGCM_sys_Rm tmpfile_ncra
485
486	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
487
488	IGCM_sys_Rm ${liste_file_se_tmp[*]}
489	done
490	;;
491	D\|d)
492	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
493	#
494	(( nb = ${mois} - 1 ))
495	#
496	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
497	#
498	(( an = an_deb ))
499	# Initialize array
500	#set -A liste_file_se
501	#set -A liste_file_se_tmp
502	unset liste_file_se
503	unset liste_file_se_tmp
504	while [ ${an} -le ${an_fin} ] ; do
505	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
506	#
507	an4=$(printf "%04i\n" ${an} )
508	#
509	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
510	# for one day
511	if [ ${length_d} -eq 1 ] ; then
512	(( length_d = 0 ))
513	fi
514	(( day = 1 ))
515	while [ ${day} -le ${moisL} ] ; do
516	day2=$(printf "%02i\n" ${day} )
517	day_beg=${an4}${mois}${day2}
518	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
519	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
520	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
521	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
522	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
523	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
524	(( day = day + 1 ))
525	done
526	(( an = an + 1 ))
527	done
528	#
529	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
530	IGCM_debug_Verif_Exit_Post
531
532	# Apply IGCM_Patch if needed
533	if [ ! ${RebuildFrequency} = true ] ; then
534	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
535	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
536	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
537	IGCM_${Patch} ${file_t}
538	done
539	done
540	fi
541	fi
542
543	# Search of common list of variables to be treated
544	nbfile=0
545	for file_common_var in ${liste_file_se_tmp[*]} ; do
546	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
547	(( nbfile = nbfile + 1 ))
548	done
549	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
550	IGCM_sys_Rm tmpfile_ncra
551
552	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
553	IGCM_sys_Rm ${liste_file_se_tmp[*]}
554	done
555	;;
556	esac
557	#
558
559	# Search of common list of variables to be treated
560	nbfile=0
561	for file_common_var in ${OUT_SE[*]} ; do
562	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
563	(( nbfile = nbfile + 1 ))
564	done
565	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
566	IGCM_sys_Rm tmpfile_ncrcat
567
568	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
569	#
570	IGCM_sys_Rm ${OUT_SE[*]}
571	#
572	# detect time counter : time or time_counter ie unlimited variable
573	var_unlim=$(ncdump -h ${RESULT_SE}\|grep UNLIMITED\| cut -d ' ' -f 1)
574
575	# Put correct climatological axis following calendar
576	case ${config_UserChoices_CalendarType} in
577	360d\|360_day)
578	# 360 days
579	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
580	;;
581	noleap\|365_day)
582	# 365 days
583	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
584	;;
585	all_leap\|366_day)
586	# 366 days
587	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
588	;;
589	leap\|gregorian\|standard)
590	# 365.2425 days
591	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
592	;;
593	esac
594	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
595	IGCM_sys_Rm ${RESULT_SE}
596	IGCM_sys_Mv tmp.nc ${RESULT_SE}
597	#
598	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
599	IGCM_sys_Rm ${RESULT_SE}
600	#
601	done
602	done
603
604	# DODS copy
605	for comp in ${config_ListOfComponents[*]} ; do
606	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
607	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
608	done
609
610	YEARS=${an_deb}_${an_fin}
611	DIR_OUT=SE
612
613	export POST_DIR; export SUBMIT_DIR; export libIGCM; export R_INIT; export R_BC; export StandAlone
614	export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ; export RESOL_SBG
615	export R_SAVE; export config_UserChoices_JobName; export config_UserChoices_TagName; export YEARS; export DIR_OUT
616	listVarEnv="POST_DIR,SUBMIT_DIR,libIGCM,R_INIT,R_BC,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,R_SAVE,config_UserChoices_JobName,config_UserChoices_TagName,YEARS,DIR_OUT"
617	export listVarEnv;
618
619	#D-
620	# --------------------------------------------------------------------
621	#D- Test if all was right before ATLAS
622	# --------------------------------------------------------------------
623	IGCM_debug_Verif_Exit_Post
624
625	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
626	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
627	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
628	#
629	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
630	IGCM_sys_QsubPost atlas_ORCHIDEE ;
631	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
632	IGCM_sys_QsubPost atlas_LMDZ ;
633	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
634	IGCM_sys_QsubPost atlas_ORCA_LIM ;
635	Script_Post_Output=additionnal.${PeriodDateEnd} ;
636	IGCM_sys_QsubPost additionnal ;
637	#
638	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
639	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
640	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
641	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] ; then
642	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
643	IGCM_sys_QsubPost atlas_ORCHIDEE ;
644	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
645	IGCM_sys_QsubPost atlas_LMDZ ;
646	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
647	IGCM_sys_QsubPost atlas_ORCA_LIM ;
648	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
649	IGCM_sys_QsubPost atlas_PISCES ;
650	#
651	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
652	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
653	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
654	#
655	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
656	IGCM_sys_QsubPost atlas_LMDZ ;
657	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
658	IGCM_sys_QsubPost atlas_ORCHIDEE ;
659	#
660	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
661	#
662	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
663	IGCM_sys_QsubPost atlas_LMDZ ;
664	#
665	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
666	[ ${config_UserChoices_TagName} = "OL2" ] ; then
667	#
668	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
669	IGCM_sys_QsubPost atlas_ORCHIDEE ;
670	#
671	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\|
672	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] ; then
673	#
674	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
675	IGCM_sys_QsubPost atlas_ORCA_LIM ;
676	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] ; then
677	#
678	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
679	IGCM_sys_QsubPost atlas_ORCA_LIM ;
680	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
681	IGCM_sys_QsubPost atlas_PISCES ;
682	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
683	#
684	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
685	IGCM_sys_QsubPost atlas_PISCES ;
686	#
687	fi
688
689	# Clean RUN_DIR_PATH (necessary for cesium)
690	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/AA_create_se @ 544

Download in other formats: