Context Navigation

libIGCM_comp.ksh @ 236

Last change on this file since 236 was 236, checked in by sdipsl, 14 years ago

Slight modification in the way we read and interpret card for post-processing. Actual card will not work with this revision

List=(histmth.nc, ${R_OUT_ATM_O_M}/${PREFIX}_1M_histmth.nc, Post_1M_histmth), \

(histday.nc, ${R_OUT_ATM_O_D}/${PREFIX}_1D_histday.nc, Post_1D_histday), \
(histhf.nc, ${R_OUT_ATM_O_H}/${PREFIX}_HF_histhf.nc, Post_HF_histhf)

ChunckJob2D, ChunckJob3d have control over TS, with the folliwing keywords :

OFF (TimeSeries? not active), NONE (TimeSeries? active but without chunck), 10Y (TimeSeries? active, but produce as many as needed 10 years lenght time series)

Seasonnal (ON/OFF), have control over SE files production.
Patch can be applied systematicaly even if no Time Series or Seasonal post-proceesing are required
A first step towards the merge between rebuildFromArchive and rebuildFrowWorkdir jobs
We may change rebuildFromArchive job name as it is not that meaningful in this context
We may change ChunckJob2D/ChunckJob3D as it is not that meaningful in this context
Some bugfix and optimisation

For testing purpose only (still try to have backward compatibility)

Exemple :

[Post_1D_histday]
Patches= (Patch_20091030_histcom_time_axis)
GatherWithInternal? = (lon, lat, presnivs, time_counter)
TimeSeriesVars2D = (bils, cldh, ...)
ChunckJob2D = 50Y
TimeSeriesVars3D = (temp, theta, ovap, ovapinit, geop, ...)
ChunckJob3D = OFF
Seasonal=OFF

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Date Author Revision

File size: 30.4 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil@ipsl.jussieu.fr Martial.Mancip@ipsl.jussieu.fr
6	# $Date$
7	# $Author$
8	# $Revision$
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	# History:
12	# Modification:
13	#
14	#**************************************************************
15
16	#========================================================================
17	function IGCM_comp_Initialize
18	{
19	IGCM_debug_PushStack "IGCM_comp_Initialize"
20
21	# Debug Print :
22	echo
23	IGCM_debug_Print 1 "IGCM_comp_Initialize :"
24	echo
25
26	typeset comp compname comptagname CompatibilityTag auxprint card_UserChoices first_option i j
27	for comp in ${config_ListOfComponents[*]} ; do
28	# Debug Print
29	IGCM_debug_Print 1 ${comp}
30	# Define component
31	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
32	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
33	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
34
35	# Read libIGCM compatibility version in ${compname}.card
36	card=${SUBMIT_DIR}/COMP/${compname}.card
37	IGCM_card_DefineVariableFromOption ${card} Compatibility libIGCM
38	eval CompatibilityTag=\${${compname}_Compatibility_libIGCM} > /dev/null 2>&1
39
40	if [ ! "${CompatibilityTag}" = "${libIGCM_CurrentTag}" ] ; then
41	IGCM_debug_Exit "${compname}.card is not compatible with libIGCM version ${libIGCM_CurrentTag} see libIGCM FAQ http://wiki.ipsl.jussieu.fr/wiki_ipsl/IGCMG/libIGCM/DocUtilisateur/FAQ ."
42	fi
43
44	# Manage component executable
45	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card Executable ${comp}
46
47	# Read component Write Frequency in config.card
48	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} WriteFrequency
49	# Debug Print :
50	eval auxprint=\${config_${comp}_WriteFrequency}
51	IGCM_debug_Print 1 "Write frequency for ${compname} : ${auxprint} "
52	#2> /dev/null
53
54	# Debug Print :
55	IGCM_debug_Print 1 "Initialize following component library :"
56	IGCM_debug_Print 1 ${SUBMIT_DIR}/COMP/${compname}.driver
57	IGCM_debug_Print 3 "With tag : ${comptagname}"
58	# Source component library
59	. ${SUBMIT_DIR}/COMP/${compname}.driver
60
61	# Debug Print
62	IGCM_debug_Print 3 "Initialize ${comp} output directory."
63	# Define Dirs
64	eval R_OUT_${comp}=${R_SAVE}/${comp}
65	eval IGCM_sys_MkdirArchive \${R_OUT_${comp}}
66
67	eval R_OUT_${comp}_O=\${R_OUT_${comp}}/Output
68	eval IGCM_sys_MkdirArchive \${R_OUT_${comp}_O}
69
70	eval R_OUT_${comp}_R=\${R_OUT_${comp}}/Restart
71	eval IGCM_sys_MkdirArchive \${R_OUT_${comp}_R}
72
73	eval R_OUT_${comp}_D=\${R_OUT_${comp}}/Debug
74	eval IGCM_sys_MkdirArchive \${R_OUT_${comp}_D}
75
76	eval R_OUT_${comp}_O_I=\${R_OUT_${comp}_O}/INS
77	eval R_OUT_${comp}_O_H=\${R_OUT_${comp}_O}/HF
78	eval R_OUT_${comp}_O_D=\${R_OUT_${comp}_O}/DA
79	eval R_OUT_${comp}_O_M=\${R_OUT_${comp}_O}/MO
80	eval R_OUT_${comp}_O_Y=\${R_OUT_${comp}_O}/YE
81
82	# Read UserChoices section of component card
83	IGCM_debug_Print 1 "DefineArrayFromSection : ${compname}_UserChoices ${card}"
84	IGCM_card_DefineArrayFromSection ${card} UserChoices
85	eval first_option=\${${compname}_UserChoices[0]} > /dev/null 2>&1
86	if [ X${first_option} != X"Error:" ] ; then
87	eval IGCM_debug_Print 2 "${compname}_UserChoices_Options:" \${${compname}_UserChoices[*]}
88	if [ X${card_UserChoices[0]} != X ] ; then
89	unset card_UserChoices
90	fi
91	eval set +A card_UserChoices -- \${${compname}_UserChoices[*]} > /dev/null 2>&1
92	typeset option
93	for option in ${card_UserChoices[*]} ; do
94	IGCM_card_DefineVariableFromOption ${card} UserChoices ${option}
95	eval IGCM_debug_Print 3 "${compname}_UserChoices_values: ${option} \${card_UserChoices_${option}}"
96	done
97	fi
98
99	# Read and Build Output File stuff
100	IGCM_debug_Print 1 "DefineArrayFromOption : ${compname}_OutputFiles ${card}"
101	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
102	ListFilesName=${compname}_OutputFiles_List
103	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
104	#
105	if [ X${FileName0} != X${NULL_STR} ] ; then
106	#
107	#IGCM_debug_Print 1 "Component : ${compname}"
108	#
109	# INITIALISATION
110	#
111	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
112	i=2
113	#
114	until [ $i -ge $NbFiles ]; do
115	#
116	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
117	#
118	if [ X${flag_post} != XNONE ] ; then
119	# Dimension = vide si vieille card.
120	IGCM_card_DefineArrayFromOption ${card} ${flag_post} TimeSeriesVars
121	IGCM_card_DefineArrayFromOption ${card} ${flag_post} Patches
122	if [ "$( eval echo \${${compname}_${flag_post}_TimeSeriesVars[*]} )" = "Option not found ${flag_post}" ] ; then
123	# New TimeSeriesVar description, with 2D, 3D and associate ChunckJob.
124	ListDimension[0]=2D
125	ListDimension[1]=3D
126	TimeSeries=false
127	iLoop=${#ListDimension[*]}
128	j=0
129	until [ $j -ge ${iLoop} ]; do
130	Dimension=${ListDimension[${j}]}
131	IGCM_card_DefineArrayFromOption ${card} ${flag_post} TimeSeriesVars${Dimension}
132	IGCM_card_DefineVariableFromOption ${card} ${flag_post} ChunckJob${Dimension}
133	#
134	# Time series WITHOUT chunk
135	#
136	if [ ! $( eval echo \${${compname}_${flag_post}_TimeSeriesVars${Dimension}} ) = ${NULL_STR} ] ; then
137	if [ $( eval echo \${${compname}_${flag_post}_ChunckJob${Dimension}} ) = NONE ] ; then
138	IGCM_debug_Print 3 "${Dimension} time series activated for ${flag_post} according to ${card}"
139	eval TimeSeries${Dimension}=true
140	fi
141	fi
142	#
143	# Time series WITH chunk
144	#
145	if [ ! $( eval echo \${${compname}_${flag_post}_TimeSeriesVars${Dimension}} ) = ${NULL_STR} ] ; then
146	chunck_size=$( eval echo \${${compname}_${flag_post}_ChunckJob${Dimension}} )
147	if [ ! ${chunck_size} = NONE ] ; then
148	IGCM_debug_Print 3 "${Dimension} time series activated with chunck for ${flag_post} according to ${card}"
149	eval TimeSeriesChunck${Dimension}=true
150	eval set +A CHUNCK${Dimension}_COMP \${CHUNCK${Dimension}_COMP[*]} ${comp}
151	eval set +A CHUNCK${Dimension}_FLAG \${CHUNCK${Dimension}_FLAG[*]} ${i}
152	eval set +A CHUNCK${Dimension}_SIZE \${CHUNCK${Dimension}_SIZE[*]} ${chunck_size}
153	fi
154	fi
155	(( j=j+1 ))
156	done
157	else
158	ListDimension[0]=""
159	TimeSeries=true
160	TimeSeries2D=false
161	TimeSeries3D=false
162	TimeSeriesChunck2D=false
163	TimeSeriesChunck3D=false
164	fi
165	# Seasonal case
166	IGCM_card_DefineArrayFromOption ${card} ${flag_post} Seasonal
167	if [ $( eval echo \${${compname}_${flag_post}_Seasonal} ) = ON ] ; then
168	Seasonal=true
169	fi
170	fi
171	(( i=i+3 ))
172	done
173	fi
174	# Debug Print
175	IGCM_debug_Print 3 "Initialize ${compname} with driver."
176	# INIT component
177	${comp}_Initialize
178	echo
179	done
180	echo "-----"
181	echo
182	IGCM_debug_Print 1 "DefineArrayFromOption : ListOfComponents"
183	IGCM_debug_PrintVariables 3 config_ListOfComponents
184
185	IGCM_debug_PopStack "IGCM_comp_Initialize"
186	}
187
188	#=======================================================================
189	function IGCM_comp_PrepareDeletedFiles
190	{
191	IGCM_debug_PushStack "IGCM_comp_PrepareDeletedFiles" $@
192
193	if [ X${2} != X. ] ; then
194	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=$( basename ${2} ) > /dev/null 2>&1
195	else
196	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=$( basename ${1} ) > /dev/null 2>&1
197	fi
198
199	IGCM_debug_PopStack "IGCM_comp_PrepareDeletedFiles"
200	}
201
202	#=======================================================================
203	function IGCM_comp_GetInputInitialStateFiles
204	{
205	IGCM_debug_PushStack "IGCM_comp_GetInputInitialStateFiles"
206
207	# Debug Print :
208	echo
209	IGCM_debug_Print 1 "IGCM_comp_GetInputInitialStateFiles :"
210	echo
211
212	# Only the first time step need InitialStateFiles
213	# otherwise it's BoundaryConditions
214	if ( ${FirstInitialize} ) ; then
215	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_
216	typeset file_in_ file_in file_out_ file_out do_init
217	for comp in ${config_ListOfComponents[*]} ; do
218	# Initialize
219	do_init="y"
220	# Do we need to bring initial state file for this component
221	if [ "${config_Restarts_OverRule}" = "y" ] ; then
222	eval do_init="n"
223	else
224	# Read component Restarts parameters
225	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} Restart
226	eval do_start=\${config_${comp}_Restart} > /dev/null 2>&1
227	if [ "${do_start}" = "y" ] ; then
228	do_init="n"
229	else
230	do_init="y"
231	fi
232	fi
233
234	if [ "${do_init}" = "y" ] ; then
235	# Define component
236	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
237	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
238
239	# Debug Print :
240	IGCM_debug_Print 3 "Initialisation files ${compname}"
241
242	card=${SUBMIT_DIR}/COMP/${compname}.card
243
244	IGCM_card_DefineArrayFromOption ${card} InitialStateFiles List
245	ListFilesName=${compname}_InitialStateFiles_List
246
247	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
248	if [ X${FileName0} != X${NULL_STR} ] ; then
249	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
250
251	(( i=0 ))
252	until [ $i -ge $NbFiles ]; do
253	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
254	eval file_in=${file_in_}
255	(( i_ = i+1 ))
256	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
257	eval file_out=${file_out_}
258
259	IGCM_sys_Get ${file_in} ${file_out}
260	#IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
261
262	(( i=i+2 ))
263	done
264	fi
265	fi
266	done
267	fi
268	IGCM_debug_PopStack "IGCM_comp_GetInputInitialStateFiles"
269	}
270
271	#=======================================================================
272	function IGCM_comp_GetInputBoundaryFiles
273	{
274	IGCM_debug_PushStack "IGCM_comp_GetInputBoundaryFiles"
275
276	# Debug Print :
277	echo
278	IGCM_debug_Print 1 "IGCM_comp_GetInputBoundaryFiles :"
279	echo
280
281	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_
282	typeset file_in_ file_in file_out_ file_out
283
284	if [ ${Period} = 1 ]; then
285	ListFixBoundary=" "
286	fi
287
288	for comp in ${config_ListOfComponents[*]} ; do
289
290	echo "-----"
291	# Define component
292	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
293	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
294
295	# Debug Print :
296	IGCM_debug_Print 3 "Boundary files ${compname}"
297
298	card=${SUBMIT_DIR}/COMP/${compname}.card
299
300	IGCM_card_DefineArrayFromOption ${card} BoundaryFiles List
301	ListFilesName=${compname}_BoundaryFiles_List
302	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
303
304	if [ X${FileName0} != X${NULL_STR} ] ; then
305	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
306
307	(( i=0 ))
308	until [ $i -ge $NbFiles ]; do
309	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
310	eval file_in=${file_in_}
311	(( i_ = i+1 ))
312	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
313	eval file_out=${file_out_}
314
315	IGCM_sys_Get ${file_in} ${file_out}
316	IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
317
318	(( i=i+2 ))
319	done
320	fi
321
322	# Get non deleted files
323	if [ ${Period} = 1 ]; then
324
325	IGCM_card_DefineArrayFromOption ${card} BoundaryFiles ListNonDel
326	ListFilesName=${compname}_BoundaryFiles_ListNonDel
327	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
328
329	if [ X${FileName0} != X${NULL_STR} ] ; then
330	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
331
332	(( i=0 ))
333	until [ $i -ge $NbFiles ]; do
334	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
335	eval file_in=${file_in_}
336	(( i_ = i+1 ))
337	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
338	eval file_out=${file_out_}
339
340	IGCM_sys_Get ${file_in} ${file_out}
341
342	if [ X${file_out} != X. ] ; then
343	ListFixBoundary=${ListFixBoundary}" "${file_out}
344	else
345	ListFixBoundary=${ListFixBoundary}" "$( basename ${file_in} )
346	fi
347
348	(( i=i+2 ))
349	done
350	fi
351	fi
352	done
353
354	IGCM_debug_PopStack "IGCM_comp_GetInputBoundaryFiles"
355	}
356
357	#=======================================================================
358	function IGCM_comp_DelFixeBoundaryFiles
359	{
360	IGCM_debug_PushStack "IGCM_comp_DelFixeBoundaryFiles"
361
362	# Debug Print :
363	echo
364	IGCM_debug_Print 1 "IGCM_comp_DelFixeBoundaryFiles :"
365	echo
366
367	ls -l ${ListFixBoundary}
368	rm -f ${ListFixBoundary}
369
370	IGCM_debug_PopStack "IGCM_comp_DelFixeBoundaryFiles"
371	}
372
373	#=======================================================================
374	function IGCM_comp_GetInputParametersFiles
375	{
376	IGCM_debug_PushStack "IGCM_comp_GetInputParametersFiles"
377
378	# Debug Print :
379	echo
380	IGCM_debug_Print 1 "IGCM_comp_GetInputParametersFiles :"
381	echo
382
383	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_ file_in file_out
384	for comp in ${config_ListOfComponents[*]} ; do
385	# Define component
386	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
387	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
388
389	# Debug Print :
390	IGCM_debug_Print 3 "Parameters ${compname}"
391
392	card=${SUBMIT_DIR}/COMP/${compname}.card
393
394	IGCM_card_DefineArrayFromOption ${card} ParametersFiles List
395	ListFilesName=${compname}_ParametersFiles_List
396	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
397
398	if [ X${FileName0} != X${NULL_STR} ] ; then
399	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
400
401	(( i=0 ))
402	until [ $i -ge $NbFiles ]; do
403	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
404	eval file_in=${file_in_}
405	(( i_ = i+1 ))
406	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
407	eval file_out=${file_out_}
408
409	IGCM_sys_Cp ${file_in} ${file_out}
410	IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
411
412	(( i=i+2 ))
413	done
414	fi
415	done
416
417	IGCM_debug_PopStack "IGCM_comp_GetInputParametersFiles"
418	}
419
420	#=======================================================================
421	function IGCM_comp_GetInputRestartFiles
422	{
423	IGCM_debug_PushStack "IGCM_comp_GetInputRestartFiles"
424
425	# Debug Print :
426	echo
427	IGCM_debug_Print 1 "IGCM_comp_GetInputRestartFiles :"
428	echo
429
430	typeset Date_tmp Date_r Path_r do_start CompOldName Path_temp
431	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_
432	typeset file_in file_out file_in_ file_out_ file_in_Name
433	typeset -Z4 j4
434
435	for comp in ${config_ListOfComponents[*]} ; do
436	# Define component
437	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
438	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
439	#
440	card=${SUBMIT_DIR}/COMP/${compname}.card
441	#
442	IGCM_card_DefineArrayFromOption ${card} RestartFiles List
443	ListFilesName=${compname}_RestartFiles_List
444	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
445
446	# Debug Print :
447	IGCM_debug_Print 3 "restart ${compname}"
448
449	if ( ${FirstInitialize} ) ; then
450
451	if [ "${config_Restarts_OverRule}" = "y" ] ; then
452	eval config_${comp}_Restart="y"
453	eval config_${comp}_RestartDate=${config_Restarts_RestartDate}
454	eval config_${comp}_RestartJobName=${config_Restarts_RestartJobName}
455	eval config_${comp}_RestartPath=${config_Restarts_RestartPath}
456	eval do_start=\${config_${comp}_Restart} > /dev/null 2>&1
457	eval CompOldName=${comp}
458	else
459	# Read component Restarts parameters
460	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} Restart
461	eval do_start=\${config_${comp}_Restart} > /dev/null 2>&1
462
463	if [ "${do_start}" = "y" ] ; then
464	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} RestartDate
465	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} RestartJobName
466	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} RestartPath
467	else
468	eval config_${comp}_RestartDate=-1
469	eval config_${comp}_RestartJobName=${NULL_STR}
470	eval config_${comp}_RestartPath=${NULL_STR}
471	fi
472	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} OldName
473	eval CompOldName=\${config_${comp}_OldName}
474	if [ X${CompOldName} = X ] ; then
475	eval CompOldName=${comp}
476	fi
477	fi
478
479	if [ "${do_start}" = "y" ] ; then
480
481	if ( [ X${FileName0} != X${NULL_STR} ] && [ X${FileName0} != XNONE ] ) ; then
482	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
483
484	(( i=1 ))
485	until [ $i -gt $NbFiles ]; do
486	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
487	eval file_in=${file_in_}
488
489	(( i_ = i+1 ))
490	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
491	eval file_out=${file_out_}
492
493	eval Date_tmp=\${config_${comp}_RestartDate}
494	Date_r=$( IGCM_date_ConvertFormatToGregorian ${Date_tmp} )
495	eval Path_r=\${config_${comp}_RestartPath}/\${config_${comp}_RestartJobName}/${CompOldName}/Restart
496	eval file_in_Name=\${config_${comp}_RestartJobName}_${Date_r}_${file_in}
497
498	generic_restart_file_name_in=$( basename ${file_in_Name} .nc )
499	generic_restart_file_name_out=$( basename ${file_out} .nc )
500
501	eval Path_temp=\${Path_r}/${generic_restart_file_name_in}
502	nb_restart_file=$(IGCM_sys_CountFileArchive ${Path_temp}_????.nc)
503
504	if [ ${nb_restart_file} -gt 1 ] ; then
505	j=0
506	until [ $j -ge $nb_restart_file ]; do
507	j4=${j}
508	eval IGCM_sys_Get ${Path_r}/${generic_restart_file_name_in}_${j4}.nc ${generic_restart_file_name_out}_${j4}.nc
509	#IGCM_comp_PrepareDeletedFiles ${generic_restart_file_name_in}_${j4}.nc ${generic_restart_file_name_out}_${j4}.nc
510	#eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${generic_restart_file_name_out}_${j4}.nc" > /dev/null 2>&1
511	(( j=j+1 ))
512	done
513	else
514	eval IGCM_sys_Get ${Path_r}/${file_in_Name} ${file_out}
515	#IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
516	fi
517
518	(( i=i+3 ))
519	done
520	else
521	if [ X${FileName0} != XNONE ] ; then
522	IGCM_debug_Exit "IGCM_comp_GetInputRestartFiles : No file in list for ${compname}."
523	else
524	IGCM_debug_Print 1 "IGCM_comp_GetInputRestartFiles : NONE specified in Restart List ${compname}."
525	fi
526	fi
527	fi
528	elif [ ${Period} -eq 1 ] ; then
529	# if not FirstInitialize and first loop of this job
530
531	# Restore Restarts files
532	#-----------------------
533	if ( [ X${FileName0} != X${NULL_STR} ] && [ X${FileName0} != XNONE ] ) ; then
534	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
535
536	(( i=1 ))
537	until [ $i -gt $NbFiles ]; do
538	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
539	eval file_in=${file_in_}
540	(( i_ = i+1 ))
541	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
542	eval file_out=${file_out_}
543
544	file_in_Name=${run_Configuration_OldPrefix}_${file_in}
545
546	generic_restart_file_name_in=$( basename ${file_in_Name} .nc )
547	generic_restart_file_name_out=$( basename ${file_out} .nc )
548
549	eval Path_temp=\${R_OUT_${comp}_R}/${generic_restart_file_name_in}
550	nb_restart_file=$(IGCM_sys_CountFileArchive ${Path_temp}_????.nc)
551
552	if [ ${nb_restart_file} -gt 1 ] ; then
553	j=0
554	until [ $j -ge $nb_restart_file ]; do
555	j4=${j}
556	eval IGCM_sys_Get \${R_OUT_${comp}_R}/${generic_restart_file_name_in}_${j4}.nc ${generic_restart_file_name_out}_${j4}.nc
557	#IGCM_comp_PrepareDeletedFiles ${generic_restart_file_name_in}_${j4}.nc ${generic_restart_file_name_out}_${j4}.nc
558	#eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${generic_restart_file_name_out}_${j4}.nc" > /dev/null 2>&1
559
560	(( j=j+1 ))
561	done
562	else
563	eval IGCM_sys_Get \${R_OUT_${comp}_R}/${file_in_Name} ${file_out}
564	#IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
565	fi
566
567	(( i=i+3 ))
568	done
569	else
570	if [ X${FileName0} != XNONE ] ; then
571	IGCM_debug_Exit "IGCM_comp_GetInputRestartFiles : No file in list for ${compname}."
572	else
573	IGCM_debug_Print 1 "IGCM_comp_GetInputRestartFiles : NONE specified in Restart List ${compname}."
574	fi
575	fi
576	fi
577	done
578	IGCM_sys_Chmod u+rw *
579
580	IGCM_debug_PopStack "IGCM_comp_GetInputRestartFiles"
581	}
582
583	#=======================================================================
584	function IGCM_comp_Update
585	{
586	IGCM_debug_PushStack "IGCM_comp_Update"
587
588	# Debug Print :
589	echo
590	IGCM_debug_Print 1 "IGCM_comp_Update :"
591	echo
592
593	typeset ExeNameIn ExeNameOut
594	typeset comp compname comptagname
595	for comp in ${config_ListOfComponents[*]} ; do
596	# Define component
597	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
598	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
599
600	# Copy executable for this component
601	eval ExeNameIn=\${config_Executable_${comp}[0]}
602	eval ExeNameOut=\${config_Executable_${comp}[1]}
603
604	# If missing executable then stop!
605	if [ ! X${ExeNameIn} = X ] && [ ! -f ${R_EXE}/${ExeNameIn} ] ; then
606	IGCM_debug_Exit "IGCM_comp_Update missing executable ${ExeNameIn}"
607	fi
608
609	if [ ${Period} -eq 1 ] && [ -f ${R_EXE}/${ExeNameIn} ] ; then
610	eval IGCM_sys_Cp ${R_EXE}/${ExeNameIn} ${ExeNameOut}
611	if [ -f ${RUN_DIR}/${ExeNameOut} ] ; then
612	eval IGCM_sys_Chmod +rx ${RUN_DIR}/${ExeNameOut}
613	fi
614	elif [ -f ${R_EXE}/${ExeNameIn} ] && [ ! -f ${RUN_DIR}/${ExeNameOut} ] ; then
615	eval IGCM_sys_Cp ${R_EXE}/${ExeNameIn} ${ExeNameOut}
616	if [ -f ${RUN_DIR}/${ExeNameOut} ] ; then
617	eval IGCM_sys_Chmod +rx ${RUN_DIR}/${ExeNameOut}
618	fi
619	fi
620
621	# Debug Print
622	IGCM_debug_Print 3 "Update ${compname} Parameter Files."
623	# UPDATE component
624	${comp}_Update
625
626	done
627
628	IGCM_debug_PopStack "IGCM_comp_Update"
629	}
630
631	#=======================================================================
632	function IGCM_comp_Finalize
633	{
634	IGCM_debug_PushStack "IGCM_comp_Finalize"
635
636	# Debug Print :
637	echo
638	IGCM_debug_Print 1 "IGCM_comp_Finalize :"
639	echo
640
641	typeset ListTextName TextName0
642	typeset comp compname comptagname card ListFilesName FileName0 NbFiles SaveOnArchive
643	typeset i i_ file_in file_in_ file_out file_out_ file_outin file_outin_ generic_file_name nb_rebuild_file
644	typeset -Z4 j4
645	for comp in ${config_ListOfComponents[*]} ; do
646	# Define component
647	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
648	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
649
650	# Debug Print
651	IGCM_debug_Print 1 "Finalize ${compname} component."
652	# FINALIZE component
653	${comp}_Finalize
654
655	card=${SUBMIT_DIR}/COMP/${compname}.card
656
657	# Save Restarts files
658	#--------------------
659	IGCM_card_DefineArrayFromOption ${card} RestartFiles List
660	ListFilesName=${compname}_RestartFiles_List
661	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
662
663	if ( [ X${FileName0} != X${NULL_STR} ] && [ X${FileName0} != XNONE ] ) ; then
664	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
665
666	(( i=0 ))
667	until [ $i -ge $NbFiles ]; do
668	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
669	eval file_in=${file_in_}
670
671	(( i_ = i+1 ))
672	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
673	eval file_out=${file_out_}
674
675	(( i_ = i+2 ))
676	eval file_outin_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
677	eval file_outin=${file_outin_}
678
679	generic_restart_file_name_in=$( basename ${file_in} .nc )
680	generic_restart_file_name_out=$( basename ${config_UserChoices_JobName}_${PeriodDateEnd}_${file_out} .nc )
681	generic_restart_file_name_outin=$( basename ${file_outin} .nc )
682
683	nb_restart_file=$( ls ${generic_restart_file_name_in}_????.nc 2>/dev/null \| wc -l )
684	if [ ${nb_restart_file} -gt 1 ] ; then
685	j=0
686	until [ $j -ge $nb_restart_file ]; do
687	j4=${j}
688	eval IGCM_sys_Put_Rest ${generic_restart_file_name_in}_${j4}.nc \${R_OUT_${comp}_R}/${generic_restart_file_name_out}_${j4}.nc
689	[ ! ${file_in} = ${file_outin} ] && IGCM_sys_Mv ${generic_restart_file_name_in}_${j4}.nc ${generic_restart_file_name_outin}_${j4}.nc
690	#eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${generic_restart_file_name_in}_${j4}.nc" > /dev/null 2>&1
691	(( j=j+1 ))
692	done
693	else
694	eval IGCM_sys_Put_Rest ${file_in} \${R_OUT_${comp}_R}/${config_UserChoices_JobName}_${PeriodDateEnd}_${file_out}
695	[ ! ${file_in} = ${file_outin} ] && IGCM_sys_Mv ${file_in} ${file_outin}
696	#eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file_in}" > /dev/null 2>&1
697	fi
698
699	(( i=i+3 ))
700	done
701	else
702	if [ X${FileName0} != XNONE ] ; then
703	IGCM_debug_Exit "IGCM_comp_Finalize : No file in restart list for ${compname}."
704	else
705	IGCM_debug_Print 1 "IGCM_comp_Finalize : NONE specified in Restart List ${compname}."
706	fi
707	fi
708
709	# Save Output files
710	#------------------
711	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
712	ListFilesName=${compname}_OutputFiles_List
713	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
714
715	if [ X${FileName0} != X${NULL_STR} ] ; then
716	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
717
718	(( i=0 ))
719	until [ $i -ge $NbFiles ]; do
720	SaveOnArchive=true
721	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
722	eval file_in=${file_in_}
723	(( i_ = i+1 ))
724	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
725	eval file_out=${file_out_}
726	(( i_ = i+2 ))
727	eval flag_post=\${${ListFilesName}[$i_]} > /dev/null 2>&1
728	#
729	generic_file_name=$( basename ${file_in} .nc )
730	nb_rebuild_file=$( ls \| grep "^${generic_file_name}[_0-9]*.nc" \| wc -l )
731	#
732	if ( [ ${nb_rebuild_file} -eq 1 ] && [ -f ${generic_file_name}_0000.nc ] ) ; then
733	IGCM_debug_Print 2 "Parallelism with 1 process. Rebuilding ${file_in} not needed"
734	IGCM_sys_Mv ${generic_file_name}_0000.nc ${file_in}
735	elif [ ${nb_rebuild_file} -gt 1 ] ; then
736	IGCM_debug_Print 2 "Parallelism detected rebuilding ${file_in} is needed"
737	if ( [ X${config_Post_RebuildFrequency} = X${NULL_STR} ] \|\| [ X${config_Post_RebuildFrequency} = XNONE ] ) ; then
738	IGCM_debug_Print 2 "Rebuilding ${file_in} online"
739	IGCM_sys_rebuild ${file_in} ${generic_file_name}_*.nc
740	else
741	IGCM_debug_Print 2 "Preparing offline rebuild for ${file_in}"
742	[ ! -d ${RUN_DIR}/REBUILD_${PeriodDateBegin} ] && IGCM_sys_Mkdir ${RUN_DIR}/REBUILD_${PeriodDateBegin}
743	IGCM_sys_Mv ${generic_file_name}_????.nc ${RUN_DIR}/REBUILD_${PeriodDateBegin}
744
745	# Prepare headers for the shell dedicated to offline rebuild
746	if [ ! -f ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh ] ; then
747	echo \#!/bin/ksh > ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
748	echo function IGCM_FlushRebuild >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
749	echo { >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
750	echo IGCM_debug_PushStack "IGCM_FlushRebuild" >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
751	echo echo >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
752	echo IGCM_debug_Print 1 "IGCM_FlushRebuild" >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
753	echo echo >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
754	fi
755	# Prepare the shell dedicated to offline rebuild
756	echo IGCM_sys_rebuild ${file_in} ${generic_file_name}_*.nc >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
757	echo IGCM_debug_Verif_Exit >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
758	#
759	# Load Patch we need to apply and apply
760	if [ X$( eval echo \${${compname}_${flag_post}_Patches[0]} ) != X${NULL_STR} ]; then
761	for Patch in $( eval echo \${${compname}_${flag_post}_Patches[*]} ); do
762	echo . ${libIGCM_POST}/libIGCM_post/IGCM_${Patch}.ksh >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
763	echo IGCM_${Patch} ${file_in} >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
764	done
765	fi
766	#
767	echo IGCM_sys_Put_Out ${file_in} ${file_out} >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
768	echo IGCM_debug_Verif_Exit >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
769	echo IGCM_sys_Rm ${generic_file_name}_*.nc >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
770	SaveOnArchive=false
771	fi
772	fi
773	#
774	if [ ${SaveOnArchive} = true ] ; then
775	#
776	# If we need to apply a patch we use TMP DIRECTORY before ARCHIVING if asynchronous rebuild is on
777	#
778	if ( [ ! X$( eval echo \${${compname}_${flag_post}_Patches[0]} ) = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = X ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
779	if [ -f ${file_in} ] ; then
780	IGCM_sys_Mv ${file_in} ${RUN_DIR}/REBUILD_${PeriodDateBegin}
781	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=REBUILD_${PeriodDateBegin}/${file_in} > /dev/null 2>&1
782	#
783	for Patch in $( eval echo \${${compname}_${flag_post}_Patches[*]} ); do
784	echo . ${libIGCM_POST}/libIGCM_post/IGCM_${Patch}.ksh >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
785	echo IGCM_${Patch} ${file_in} >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
786	done
787	#
788	echo IGCM_sys_Put_Out ${file_in} ${file_out} >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
789	echo IGCM_debug_Verif_Exit >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
790	#
791	fi
792	else
793	IGCM_sys_Put_Out ${file_in} ${file_out}
794	[ $? -eq 0 ] && eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file_in}" > /dev/null 2>&1
795	if [ ${nb_rebuild_file} -gt 1 ] ; then
796	for DelFile in $( ls \| grep "${generic_file_name}[_0-9]*.nc" ) ; do
797	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=${DelFile} > /dev/null 2>&1
798	done
799	fi
800	fi
801	fi
802	(( i=i+3 ))
803	done
804	fi
805
806	# Save Output Text files of models
807	#---------------------------------
808
809	IGCM_card_DefineArrayFromOption ${card} OutputText List
810	ListTextName=${compname}_OutputText_List
811
812	eval TextName0=\${${ListTextName}[0]} > /dev/null 2>&1
813	if [ X${TextName0} != X${NULL_STR} ] ; then
814	eval NbFiles=\${#${ListTextName}[@]} > /dev/null 2>&1
815
816	(( i=0 ))
817	until [ $i -eq $NbFiles ]; do
818	eval file_in=\${${ListTextName}[$i]} > /dev/null 2>&1
819	(( i=i+1 ))
820
821	nb_text_file=$( ls ${file_in}* 2>/dev/null \| wc -l )
822	if [ ${nb_text_file} -gt 1 ] ; then
823	list_file=$( ls ${file_in}* )
824	for file in ${list_file}
825	do
826	eval IGCM_sys_Put_Out ${file} \${R_OUT_${comp}_D}/${PREFIX}_${file}
827	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file}" > /dev/null 2>&1
828	done
829	else
830	if ( [ -f ${file_in}_0000 ] \|\| [ -f ${file_in}0 ] ) ; then
831	eval IGCM_sys_Mv ${file_in}* ${file_in}
832	fi
833	eval IGCM_sys_Put_Out ${file_in} \${R_OUT_${comp}_D}/${PREFIX}_${file_in}
834	[ $? -eq 0 ] && eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file_in}" > /dev/null 2>&1
835	fi
836	done
837	fi
838	done
839
840	IGCM_debug_PopStack "IGCM_comp_Finalize"
841	}

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_comp/libIGCM_comp.ksh @ 236

Download in other formats: