Context Navigation

libIGCM_comp.ksh @ 980

Last change on this file since 980 was 980, checked in by jgipsl, 10 years ago

Add possibility to have two output files differing in the name only with an added suffix on one of the files. For exemple file1.nc and file1_bis.nc is now ok.

This modification also gives the possibility to have more or less digits than 4 in suffix for domain decomposition.

see ticket #173

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 54.7 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#========================================================================
15	function IGCM_comp_Initialize
16	{
17	IGCM_debug_PushStack "IGCM_comp_Initialize"
18
19	# Debug Print :
20	echo
21	IGCM_debug_Print 1 "IGCM_comp_Initialize"
22	echo
23
24	typeset comp compname comptagname CompatibilityTag auxprint card_UserChoices first_option option i j
25	for comp in ${config_ListOfComponents[*]} ; do
26
27	# Define component
28	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
29	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
30	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
31
32	# Debug Print
33	IGCM_debug_Print 1 "Initialize ${comp} : ${compname} component."
34
35	# Read libIGCM compatibility version in ${compname}.card
36	card=${SUBMIT_DIR}/COMP/${compname}.card
37	IGCM_card_DefineVariableFromOption ${card} Compatibility libIGCM
38
39	eval CompatibilityTag=\${${compname}_Compatibility_libIGCM} > /dev/null 2>&1
40	if [ ! "${CompatibilityTag}" = "${libIGCM_CurrentTag}" ] ; then
41	IGCM_debug_Exit "${compname}.card is not compatible with libIGCM version ${libIGCM_CurrentTag} see libIGCM FAQ http://wiki.ipsl.jussieu.fr/wiki_ipsl/IGCMG/libIGCM/DocUtilisateur/FAQ ."
42	fi
43
44	# Manage component executable
45	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card Executable ${comp}
46
47	# Define all options in section [comp]
48	IGCM_debug_Print 3 " DefineArrayFromSection : ${comp}"
49	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ${comp}
50	eval config_comp=\${config_${comp}[*]} > /dev/null 2>&1
51	for option in ${config_comp[*]} ; do
52	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} ${option}
53	done
54	IGCM_debug_Print 3 " Found in section config_${comp} : ${config_comp[*]} "
55
56	# Debug Print :
57	eval auxprint=\${config_${comp}_WriteFrequency}
58	IGCM_debug_Print 1 "Write frequency for ${compname} : ${auxprint} "
59	#2> /dev/null
60
61	# Debug Print :
62	IGCM_debug_Print 2 "Initialize following component library"
63
64	# Source drivers in directory DRIVER if it exist
65	# else source them from directory COMP
66	if [ -d ${SUBMIT_DIR}/DRIVER ] ; then
67	IGCM_debug_Print 2 ${SUBMIT_DIR}/DRIVER/${compname}.driver
68	# Source component library
69	. ${SUBMIT_DIR}/DRIVER/${compname}.driver
70	else
71	IGCM_debug_Print 2 ${SUBMIT_DIR}/COMP/${compname}.driver
72	# Source component library
73	. ${SUBMIT_DIR}/COMP/${compname}.driver
74	fi
75	IGCM_debug_Print 3 "With tag : ${comptagname}"
76
77	# Debug Print
78	IGCM_debug_Print 3 "Initialize ${comp} output directory."
79
80	# Define ARCHIVED Dirs
81	eval R_OUT_${comp}=${R_SAVE}/${comp}
82	eval IGCM_sys_MkdirArchive \${R_OUT_${comp}}
83
84	eval R_OUT_${comp}_O=\${R_OUT_${comp}}/Output
85	eval IGCM_sys_MkdirArchive \${R_OUT_${comp}_O}
86
87	eval R_OUT_${comp}_R=\${R_OUT_${comp}}/Restart
88	[ ${config_Post_PackFrequency} = NONE ] && eval IGCM_sys_MkdirArchive \${R_OUT_${comp}_R}
89
90	eval R_OUT_${comp}_D=\${R_OUT_${comp}}/Debug
91	[ ${config_Post_PackFrequency} = NONE ] && eval IGCM_sys_MkdirArchive \${R_OUT_${comp}_D}
92
93	eval R_OUT_${comp}_O_I=\${R_OUT_${comp}_O}/INS
94	eval R_OUT_${comp}_O_H=\${R_OUT_${comp}_O}/HF
95	eval R_OUT_${comp}_O_D=\${R_OUT_${comp}_O}/DA
96	eval R_OUT_${comp}_O_M=\${R_OUT_${comp}_O}/MO
97	eval R_OUT_${comp}_O_Y=\${R_OUT_${comp}_O}/YE
98
99	# Define BUFFERED Dirs
100	if [ ! ${config_Post_PackFrequency} = NONE ] ; then
101	eval R_BUF_${comp}=${R_BUFR}/${comp}
102	eval IGCM_sys_Mkdir \${R_BUF_${comp}}
103
104	eval R_BUF_${comp}_O=\${R_BUF_${comp}}/Output
105	eval IGCM_sys_Mkdir \${R_BUF_${comp}_O}
106
107	eval R_BUF_${comp}_R=\${R_BUF_${comp}}/Restart
108	eval IGCM_sys_Mkdir \${R_BUF_${comp}_R}
109
110	eval R_BUF_${comp}_D=\${R_BUF_${comp}}/Debug
111	eval IGCM_sys_Mkdir \${R_BUF_${comp}_D}
112
113	eval R_BUF_${comp}_O_I=\${R_BUF_${comp}_O}/INS
114	eval R_BUF_${comp}_O_H=\${R_BUF_${comp}_O}/HF
115	eval R_BUF_${comp}_O_D=\${R_BUF_${comp}_O}/DA
116	eval R_BUF_${comp}_O_M=\${R_BUF_${comp}_O}/MO
117	eval R_BUF_${comp}_O_Y=\${R_BUF_${comp}_O}/YE
118	fi
119
120	# Read UserChoices section of component card
121	IGCM_debug_Print 2 "DefineArrayFromSection : ${compname}_UserChoices ${card}"
122	IGCM_card_DefineArrayFromSection ${card} UserChoices
123	eval first_option=\${${compname}_UserChoices[0]} > /dev/null 2>&1
124
125	# If section is not empty we define corresponding variables
126	if [ X${first_option} != X"Error:" ] ; then
127	if [ X${card_UserChoices[0]} != X ] ; then
128	unset card_UserChoices
129	fi
130	eval set +A card_UserChoices -- \${${compname}_UserChoices[*]} > /dev/null 2>&1
131	IGCM_debug_Print 3 "${compname}_UserChoices_values:"
132	for option in ${card_UserChoices[*]} ; do
133	IGCM_card_DefineVariableFromOption ${card} UserChoices ${option}
134	eval IGCM_debug_Print 3 "${option}=\${${compname}_UserChoices_${option}}"
135	done
136	fi
137
138	# Read and Build Output File stuff
139	IGCM_debug_Print 2 "DefineArrayFromOption : ${compname}_OutputFiles ${card}"
140	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
141	ListFilesName=${compname}_OutputFiles_List
142	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
143	#
144	if [ X${FileName0} != X${NULL_STR} ] ; then
145	#
146	#IGCM_debug_Print 1 "Component : ${compname}"
147	#
148	# INITIALISATION
149	#
150	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
151	#
152	i=2
153	#
154	until [ $i -ge $NbFiles ]; do
155	#
156	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
157	#
158	if [ X${flag_post} != XNONE ] ; then
159	#
160	# First of all
161	#
162	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
163	#
164	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
165	#
166	# variable option allready typeset above
167	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
168	if [ ${option} = Seasonal ] ; then
169	FoundSeasonal=true
170	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
171	fi
172	done
173	#
174	if [ ! X${FoundSeasonal} = Xtrue ] ; then
175	eval ${compname}_${flag_post}_Seasonal=ON
176	fi
177	#
178	if [ $( eval echo \${${compname}_${flag_post}_Seasonal} ) = ON ] ; then
179	Seasonal=true
180	fi
181
182	# Dimension = vide si vieille card.
183	IGCM_card_DefineArrayFromOption ${card} ${flag_post} TimeSeriesVars
184	IGCM_card_DefineArrayFromOption ${card} ${flag_post} Patches
185	if [ X"$( eval echo \${${compname}_${flag_post}_TimeSeriesVars[*]} )" = X"Option not" ] ; then
186	# New TimeSeriesVar description, with 2D, 3D and associate ChunckJob.
187	ListDimension[0]=2D
188	ListDimension[1]=3D
189	TimeSeries=false
190	iLoop=${#ListDimension[*]}
191	j=0
192	until [ $j -ge ${iLoop} ]; do
193	Dimension=${ListDimension[${j}]}
194	IGCM_card_DefineArrayFromOption ${card} ${flag_post} TimeSeriesVars${Dimension}
195	IGCM_card_DefineVariableFromOption ${card} ${flag_post} ChunckJob${Dimension}
196	#
197	# Time series WITHOUT chunk
198	#
199	if [ ! $( eval echo \${${compname}_${flag_post}_TimeSeriesVars${Dimension}} ) = ${NULL_STR} ] ; then
200	if [ $( eval echo \${${compname}_${flag_post}_ChunckJob${Dimension}} ) = NONE ] ; then
201	IGCM_debug_Print 3 "${Dimension} time series activated for ${flag_post}"
202	eval TimeSeries${Dimension}=true
203	fi
204	fi
205	#
206	# Time series WITH chunk
207	#
208	if [ ! $( eval echo \${${compname}_${flag_post}_TimeSeriesVars${Dimension}} ) = ${NULL_STR} ] ; then
209	chunck_size=$( eval echo \${${compname}_${flag_post}_ChunckJob${Dimension}} )
210	if [ ! ${chunck_size} = NONE ] && [ ! ${chunck_size} = OFF ] ; then
211	IGCM_debug_Print 3 "${Dimension} time series activated with chunck for ${flag_post}"
212	eval TimeSeriesChunck${Dimension}=true
213	eval CHUNCK${Dimension}_COMP[\${#CHUNCK${Dimension}_COMP[*]}]=${comp}
214	eval CHUNCK${Dimension}_FLAG[\${#CHUNCK${Dimension}_FLAG[*]}]=${i}
215	eval CHUNCK${Dimension}_NAME[\${#CHUNCK${Dimension}_NAME[*]}]=${flag_post}
216	eval CHUNCK${Dimension}_SIZE[\${#CHUNCK${Dimension}_SIZE[*]}]=${chunck_size}
217	fi
218	fi
219	(( j=j+1 ))
220	done
221	else
222	ListDimension[0]=""
223	TimeSeries=true
224	TimeSeries2D=false
225	TimeSeries3D=false
226	TimeSeriesChunck2D=false
227	TimeSeriesChunck3D=false
228	fi
229	fi
230	(( i=i+3 ))
231	done
232	fi
233	# Debug Print
234	IGCM_debug_Print 3 "Initialize ${compname} with driver."
235	# INIT component
236	${comp}_Initialize
237	echo
238	done
239
240	IGCM_debug_PopStack "IGCM_comp_Initialize"
241	}
242
243	#=======================================================================
244	function IGCM_comp_PrepareDeletedFiles
245	{
246	IGCM_debug_PushStack "IGCM_comp_PrepareDeletedFiles" $@
247
248	if [ X${2} != X. ] ; then
249	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=$( basename ${2} ) > /dev/null 2>&1
250	else
251	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=$( basename ${1} ) > /dev/null 2>&1
252	fi
253
254	IGCM_debug_PopStack "IGCM_comp_PrepareDeletedFiles"
255	}
256
257	#=======================================================================
258	function IGCM_comp_GetInputInitialStateFiles
259	{
260	IGCM_debug_PushStack "IGCM_comp_GetInputInitialStateFiles"
261
262	# Debug Print :
263	echo
264	IGCM_debug_Print 1 "IGCM_comp_GetInputInitialStateFiles"
265	echo
266
267	# Only the first time step need InitialStateFiles
268	# otherwise it's BoundaryConditions
269	if ( ${FirstInitialize} ) ; then
270	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_
271	typeset file_in_ file_in file_out_ file_out do_init
272	for comp in ${config_ListOfComponents[*]} ; do
273	# Initialize
274	do_init="y"
275	# Do we need to bring initial state file for this component
276	if [ "${config_Restarts_OverRule}" = "y" ] ; then
277	eval do_init="n"
278	else
279	# Read component Restarts parameters
280	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} Restart
281	eval do_start=\${config_${comp}_Restart} > /dev/null 2>&1
282	if [ "${do_start}" = "y" ] ; then
283	do_init="n"
284	else
285	do_init="y"
286	fi
287	fi
288
289	if [ "${do_init}" = "y" ] ; then
290	# Define component
291	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
292	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
293
294	# Debug Print :
295	IGCM_debug_Print 3 "Initialisation files ${compname}"
296
297	card=${SUBMIT_DIR}/COMP/${compname}.card
298
299	IGCM_card_DefineArrayFromOption ${card} InitialStateFiles List
300	ListFilesName=${compname}_InitialStateFiles_List
301
302	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
303	if [ X${FileName0} != X${NULL_STR} ] ; then
304	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
305	(( i=0 ))
306	until [ $i -ge $NbFiles ]; do
307	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
308	eval file_in=${file_in_}
309	(( i_ = i+1 ))
310	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
311	eval file_out=${file_out_}
312
313	IGCM_sys_IsFileArchived ${file_in}
314	if [ $? = 0 ] ; then
315	IGCM_sys_Get ${file_in} ${file_out}
316	#IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
317	else
318	IGCM_sys_Cp ${file_in} ${file_out}
319	fi
320	(( i=i+2 ))
321	done
322	fi
323	fi
324	done
325	fi
326	IGCM_debug_PopStack "IGCM_comp_GetInputInitialStateFiles"
327	}
328
329	#=======================================================================
330	# Definition of Smooth modulo function
331	# usage :
332	# IGCM_SmoothModulo StringModulo value
333	#
334	# StringModulo : A string of min max and modulo like definition of Scilab vectors.
335	# [min]:[modulo:][max]
336	# where :
337	# [] value are optionnals;
338	# empty min equal 1
339	# empty max equal infinity
340	# modulo not given or empty equal 1
341	# empty string or just ':' equal always.
342	#
343	# value : the value to test with the definition
344	#
345	# return : true(1)/false(0)
346	function IGCM_SmoothModulo
347	{
348	IGCM_debug_PushStack "IGCM_SmoothModulo"
349	typeset defVector ModValue
350
351	eval set +A defVector -- $( echo "${1}" \| \
352	gawk -F ':' '{print ($1 == "" ? 1 : $1) " " (NF==3 ? ($2 == "" ? 1 : $2) : 1) " " (NF==3 ? ($3 == "" ? -1 : $3) : ($2 == "" ? -1 : $2))}' )
353
354	# Save Smooth Min and Max. Needed to call IGCM_sys_Get when appropriate
355	arr[1]=${defVector[0]}
356	arr[2]=${defVector[2]}
357
358	# Test limits :
359	# ${defVector[0]} <= ${2} <= ${defVector[2]}
360	# or ${defVector[2]} == -1
361	if ( [ ${2} -ge ${defVector[0]} ] && ( [ ${2} -le ${defVector[2]} ] \|\| [ ${defVector[2]} -lt 0 ] ) ) ; then
362	# Test modulo
363	ModValue=$( expr $ ${2} - ${defVector[0]} $ % ${defVector[1]} )
364	if [ ${ModValue} -eq 0 ] ; then
365	arr[3]=true
366	echo ${arr[@]}
367	IGCM_debug_PopStack "IGCM_SmoothModulo"
368	return 1
369	else
370	arr[3]=false
371	echo ${arr[@]}
372	IGCM_debug_PopStack "IGCM_SmoothModulo"
373	return 0
374	fi
375	else
376	arr[3]=false
377	echo ${arr[@]}
378	IGCM_debug_PopStack "IGCM_SmoothModulo"
379	return 0
380	fi
381	}
382
383	#=======================================================================
384	function IGCM_comp_GetInputSmoothFiles
385	{
386	IGCM_debug_PushStack "IGCM_comp_GetInputSmoothFiles"
387
388	# Debug Print :
389	echo
390	IGCM_debug_Print 1 "IGCM_comp_GetInputSmoothFiles"
391	echo
392
393	typeset comp compname comptagname card ListFilesName FileName0 NbFiles j i i_ i__
394	typeset file_in_ file_in file_out_ file_out ret SmoothDef aux val
395
396	for comp in ${config_ListOfComponents[*]} ; do
397	# Define component
398	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
399	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
400
401	# Debug Print :
402	IGCM_debug_Print 3 "Smooth files ${compname}"
403
404	card=${SUBMIT_DIR}/COMP/${compname}.card
405
406	IGCM_card_DefineArrayFromOption ${card} SmoothFiles List
407	ListFilesName=${compname}_SmoothFiles_List
408	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
409
410	if ( [ X${FileName0} != X${NULL_STR} ] && [ X${FileName0} != X"Section" ] ) ; then
411	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
412
413	(( i=0 ))
414	until [ $i -ge $NbFiles ]; do
415	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
416	eval file_in=${file_in_}
417	(( i_ = i+1 ))
418	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
419	eval file_out=${file_out_}
420
421	# define CumulPeriod definition for this file
422	(( i__ = i+2 ))
423	eval SmoothDef=\${${ListFilesName}[$i__]}
424	IGCM_debug_Print 3 " ${file_in} ${SmoothDef}"
425	aux=$( IGCM_SmoothModulo ${SmoothDef} ${CumulPeriod} )
426	j=1
427	for val in ${aux} ; do
428	[ ${j} -eq 1 ] && SmoothMin=${val}
429	[ ${j} -eq 2 ] && SmoothMax=${val}
430	[ ${j} -eq 3 ] && ret=${val}
431	(( j=j+1 ))
432	done
433	[ ${SmoothMax} -eq -1 ] && SmoothMax=${CumulPeriod}
434	if ( [ X${ret} = Xtrue ] \|\| ( [ ${Period} -eq 1 ] && [ ${CumulPeriod} -ge ${SmoothMin} ] && [ ${CumulPeriod} -le ${SmoothMax} ] ) ) ; then
435
436	IGCM_sys_IsFileArchived ${file_in}
437	if [ $? = 0 ] ; then
438	IGCM_sys_Get ${file_in} ${file_out}
439	#IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
440	else
441	IGCM_sys_Cp ${file_in} ${file_out}
442	fi
443	fi
444	(( i=i+3 ))
445	done
446	fi
447	done
448
449	IGCM_debug_PopStack "IGCM_comp_GetInputSmoothFiles"
450	}
451
452	#=======================================================================
453	function IGCM_comp_GetInputBoundaryFiles
454	{
455	IGCM_debug_PushStack "IGCM_comp_GetInputBoundaryFiles"
456
457	# Debug Print :
458	echo
459	IGCM_debug_Print 1 "IGCM_comp_GetInputBoundaryFiles"
460	echo
461
462	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_
463	typeset file_in_ file_in file_out_ file_out
464
465	if [ ${Period} = 1 ]; then
466	ListFixBoundary=" "
467	fi
468
469	for comp in ${config_ListOfComponents[*]} ; do
470
471	# Define component
472	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
473	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
474
475	# Debug Print :
476	IGCM_debug_Print 3 "Boundary files ${compname}"
477
478	card=${SUBMIT_DIR}/COMP/${compname}.card
479
480	IGCM_card_DefineArrayFromOption ${card} BoundaryFiles List
481	ListFilesName=${compname}_BoundaryFiles_List
482	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
483
484	if [ X${FileName0} != X${NULL_STR} ] ; then
485	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
486
487	(( i=0 ))
488	until [ $i -ge $NbFiles ]; do
489	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
490	eval file_in=${file_in_}
491	(( i_ = i+1 ))
492	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
493	eval file_out=${file_out_}
494
495	IGCM_sys_IsFileArchived ${file_in}
496	if [ $? = 0 ] ; then
497	IGCM_sys_Get ${file_in} ${file_out}
498	IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
499	else
500	IGCM_sys_Cp ${file_in} ${file_out}
501	fi
502
503	(( i=i+2 ))
504	done
505	fi
506
507	# Get non deleted files
508	if [ ${Period} = 1 ]; then
509
510	IGCM_card_DefineArrayFromOption ${card} BoundaryFiles ListNonDel
511	ListFilesName=${compname}_BoundaryFiles_ListNonDel
512	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
513
514	if [ X${FileName0} != X${NULL_STR} ] ; then
515	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
516
517	(( i=0 ))
518	until [ $i -ge $NbFiles ]; do
519	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
520	eval file_in=${file_in_}
521	(( i_ = i+1 ))
522	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
523	eval file_out=${file_out_}
524
525	IGCM_sys_IsFileArchived ${file_in}
526	if [ $? = 0 ] ; then
527	IGCM_sys_Get ${file_in} ${file_out}
528	#IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
529	else
530	IGCM_sys_Cp ${file_in} ${file_out}
531	fi
532
533	if [ X${file_out} != X. ] ; then
534	ListFixBoundary=${ListFixBoundary}" "${file_out}
535	else
536	ListFixBoundary=${ListFixBoundary}" "$( basename ${file_in} )
537	fi
538
539	(( i=i+2 ))
540	done
541	fi
542	fi
543	done
544
545	IGCM_debug_PopStack "IGCM_comp_GetInputBoundaryFiles"
546	}
547
548	#=======================================================================
549	function IGCM_comp_DelFixeBoundaryFiles
550	{
551	IGCM_debug_PushStack "IGCM_comp_DelFixeBoundaryFiles"
552
553	# Debug Print :
554	echo
555	IGCM_debug_Print 1 "IGCM_comp_DelFixeBoundaryFiles"
556	echo
557
558	ls -l ${ListFixBoundary}
559	rm -f ${ListFixBoundary}
560
561	IGCM_debug_PopStack "IGCM_comp_DelFixeBoundaryFiles"
562	}
563
564	#=======================================================================
565	function IGCM_comp_GetInputParametersFiles
566	{
567	IGCM_debug_PushStack "IGCM_comp_GetInputParametersFiles"
568
569	# Debug Print :
570	echo
571	IGCM_debug_Print 1 "IGCM_comp_GetInputParametersFiles"
572	echo
573
574	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_ file_in file_out
575	for comp in ${config_ListOfComponents[*]} ; do
576	# Define component
577	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
578	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
579
580	# Debug Print :
581	IGCM_debug_Print 3 "Parameters ${compname}"
582
583	card=${SUBMIT_DIR}/COMP/${compname}.card
584
585	IGCM_card_DefineArrayFromOption ${card} ParametersFiles List
586	ListFilesName=${compname}_ParametersFiles_List
587	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
588
589	if [ X${FileName0} != X${NULL_STR} ] ; then
590	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
591
592	(( i=0 ))
593	until [ $i -ge $NbFiles ]; do
594	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
595	eval file_in=${file_in_}
596	(( i_ = i+1 ))
597	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
598	eval file_out=${file_out_}
599
600	IGCM_sys_Cp ${file_in} ${file_out}
601	IGCM_comp_PrepareDeletedFiles ${file_in} ${file_out}
602
603	(( i=i+2 ))
604	done
605	fi
606	done
607
608	IGCM_debug_PopStack "IGCM_comp_GetInputParametersFiles"
609	}
610
611	#=======================================================================
612	function IGCM_comp_GetInputRestartFiles
613	{
614	IGCM_debug_PushStack "IGCM_comp_GetInputRestartFiles"
615
616	# Debug Print :
617	echo
618	IGCM_debug_Print 1 "IGCM_comp_GetInputRestartFiles"
619	echo
620
621	typeset Date_tmp Date_r Path_r do_start CompOldName Path_OUT Path_BUF
622	typeset Buffered Archived Tared PotentialTarFile IsMatching TarFileFound
623	typeset comp compname comptagname card ListFilesName FileName0 NbFiles i i_
624	typeset file_in file_out file_in_ file_out_ file_in_Name
625	typeset -Z4 j4
626	#BASH declare j4
627
628	IsMatching=""
629
630	for comp in ${config_ListOfComponents[*]} ; do
631	# Define component
632	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
633	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
634	#
635	card=${SUBMIT_DIR}/COMP/${compname}.card
636	#
637	IGCM_card_DefineArrayFromOption ${card} RestartFiles List
638	ListFilesName=${compname}_RestartFiles_List
639	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
640
641	# Debug Print :
642	IGCM_debug_Print 3 "restart ${compname}"
643
644	if ( ${FirstInitialize} ) ; then
645
646	if [ "${config_Restarts_OverRule}" = "y" ] ; then
647	eval config_${comp}_Restart="y"
648	eval config_${comp}_RestartDate=${config_Restarts_RestartDate}
649	eval config_${comp}_RestartJobName=${config_Restarts_RestartJobName}
650	eval config_${comp}_RestartPath=${config_Restarts_RestartPath}
651	eval do_start=\${config_${comp}_Restart} > /dev/null 2>&1
652	eval CompOldName=${comp}
653	else
654	# Read component Restarts parameters
655	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} Restart
656	eval do_start=\${config_${comp}_Restart} > /dev/null 2>&1
657
658	if [ "${do_start}" = "y" ] ; then
659	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} RestartDate
660	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} RestartJobName
661	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card ${comp} RestartPath
662	else
663	eval config_${comp}_RestartDate=-1
664	eval config_${comp}_RestartJobName=${NULL_STR}
665	eval config_${comp}_RestartPath=${NULL_STR}
666	fi
667
668	eval CompOldName=\${config_${comp}_OldName}
669	if [ X${CompOldName} = X ] ; then
670	eval CompOldName=${comp}
671	fi
672
673	# Reinitialize IsMatching to allow searching for a different tar file for each component.
674	IsMatching=""
675	TarFileFound=""
676	fi
677
678	if [ "${do_start}" = "y" ] ; then
679
680	# Restore Restarts files
681	#-----------------------
682	if ( [ X${FileName0} != X${NULL_STR} ] && [ X${FileName0} != XNONE ] ) ; then
683	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
684
685	(( i=1 ))
686	until [ $i -gt $NbFiles ]; do
687	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
688	eval file_in=${file_in_}
689	(( i_ = i+1 ))
690	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
691	eval file_out=${file_out_}
692
693	eval Date_tmp=\${config_${comp}_RestartDate}
694	Date_r=$( IGCM_date_ConvertFormatToGregorian ${Date_tmp} )
695	# will be re-use
696	eval RestartPath=\${config_${comp}_RestartPath}
697	eval RestartJobName=\${config_${comp}_RestartJobName}
698	#
699	Path_r=${RestartPath}/${RestartJobName}/${CompOldName}/Restart
700	file_in_Name=${RestartJobName}_${Date_r}_${file_in}
701
702	extension_in=$( echo ${file_in_Name##*.} )
703	extension_out=$( echo ${file_out##*.} )
704
705	generic_restart_file_name_in=$( basename ${file_in_Name} .${extension_in} )
706	generic_restart_file_name_out=$( basename ${file_out} .${extension_out} )
707
708	Path_OUT=${Path_r}/${generic_restart_file_name_in}
709
710	if [ $( IGCM_sys_TestFileBuffer ${Path_OUT}*.${extension_in} ; echo $? ) = 0 ] ; then
711	IGCM_debug_Print 3 "Buffered restart ${Path_OUT}*.${extension_in} "
712	Buffered=true
713	Archived=false
714	Tared=false
715	nb_restart_file=$(IGCM_sys_CountFileBuffer ${Path_OUT}_????.${extension_in})
716	elif [ $( IGCM_sys_TestFileArchive ${Path_OUT}*.${extension_in} ; echo $? ) = 0 ] ; then
717	IGCM_debug_Print 3 "Archived restart ${Path_OUT}*.${extension_in}"
718	Buffered=false
719	Archived=true
720	Tared=false
721	nb_restart_file=$(IGCM_sys_CountFileArchive ${Path_OUT}_????.${extension_in})
722	else
723	IGCM_debug_Print 3 "No restart file in the buffer nor in the archive directory"
724	IGCM_debug_Print 3 "${Path_OUT}*.${extension_in} do not exist"
725	IGCM_debug_Print 3 "Restart files will now be searched for in : ${RestartPath}/${RestartJobName}/RESTART"
726	Buffered=false
727	Archived=false
728	Tared=true
729	# Look after the tar file we want if we did not found it already
730	if [ X${IsMatching} = X ] ; then
731	IGCM_sys_TestDirArchive ${RestartPath}/${RestartJobName}/RESTART
732	if [ $? ] ; then
733	for PotentialTarFile in $( IGCM_sys_RshArchive "find ${RestartPath}/${RestartJobName}/RESTART -name "${RestartJobName}_restart.tar" -print" ) ; do
734	IsMatching=$( echo ${PotentialTarFile##*/} \| \
735	sed "s:_restart::" \| \
736	sed "s:^${RestartJobName}_::" \| \
737	sed "s:\.tar$::" \| \
738	gawk -F_ -v restartdate=${Date_r} \
739	'{if (($1 <= restartdate) && ($2 >= restartdate)) {print $1"_"$2}}' )
740	if [ ! X${IsMatching} = X ] ; then
741	TarFileFound=${PotentialTarFile}
742	break
743	fi
744	done
745	else
746	IGCM_debug_Print 3 "Restart files were not found!"
747	IGCM_debug_Print 3 "Restart files have been searched for in buffer and archive directory."
748	IGCM_debug_Print 3 "They have been searched for in packed and unpacked format."
749	IGCM_debug_Exit "Please double check restart settings in config.card"
750	IGCM_debug_Verif_Exit
751	fi
752	fi
753	IGCM_sys_PrepareTaredRestart ${TarFileFound}
754	TarFileLocation=$( basename ${TarFileFound} )
755	IGCM_debug_Print 1 "tar xvf ${TarFileLocation} ${comp}_${generic_restart_file_name_in}*.${extension_in}"
756	tar xvf ${TarFileLocation} ${comp}_${generic_restart_file_name_in}*.${extension_in}
757	nb_restart_file=$( IGCM_sys_CountFileBuffer ${comp}_${generic_restart_file_name_in}_????.${extension_in} )
758	fi
759
760	if [ ${nb_restart_file} -gt 1 ] ; then
761	j=0 # BASH LINE NOT NEEDED
762	# BASH for j4 in in $( eval echo {0000..$(( nb_restart_file - 1 ))} ) ; do
763	until [ $j -ge ${nb_restart_file} ]; do # BASH LINE NOT NEEDED
764	j4=${j} # BASH LINE NOT NEEDED
765	if [ X${Buffered} = Xtrue ] ; then
766	IGCM_sys_GetBuffer ${Path_OUT}_${j4}.${extension_in} ${generic_restart_file_name_out}_${j4}.${extension_out}
767	elif [ X${Archived} = Xtrue ] ; then
768	IGCM_sys_Get ${Path_OUT}_${j4}.${extension_in} ${generic_restart_file_name_out}_${j4}.${extension_out}
769	elif [ X${Tared} = Xtrue ] ; then
770	IGCM_sys_Mv ${comp}_${generic_restart_file_name_in}_${j4}.${extension_in} ${generic_restart_file_name_out}_${j4}.${extension_out}
771	fi
772	(( j=j+1 )) #BASH LINE NOT NEEDED
773	done
774
775	if [ X${NUM_PROC_OCE} != X ] ; then
776	if [ ${NUM_PROC_OCE} -ne ${nb_restart_file} ] ; then
777	IGCM_sys_rebuild ${generic_restart_file_name_out}.${extension_out} ${generic_restart_file_name_out}_????.${extension_out}
778	IGCM_sys_Rm ${generic_restart_file_name_out}_????.${extension_out}
779	fi
780	fi
781	else
782	if [ X${Buffered} = Xtrue ] ; then
783	IGCM_sys_GetBuffer ${Path_r}/${file_in_Name} ${file_out}
784	elif [ X${Archived} = Xtrue ] ; then
785	IGCM_sys_Get ${Path_r}/${file_in_Name} ${file_out}
786	elif [ X${Tared} = Xtrue ] ; then
787	IGCM_sys_Mv ${comp}_${file_in_Name} ${file_out}
788	fi
789	fi
790	(( i=i+3 ))
791	done
792
793	else
794	if [ X${FileName0} != XNONE ] ; then
795	IGCM_debug_Exit "IGCM_comp_GetInputRestartFiles : No file in list for ${compname}."
796	else
797	IGCM_debug_Print 1 "IGCM_comp_GetInputRestartFiles : NONE specified in Restart List ${compname}."
798	fi
799	fi
800	fi
801	elif ( [ ${Period} -eq 1 ] && [ ${DRYRUN} -eq 0 ] ) ; then
802	# if not FirstInitialize and first loop of this job
803
804	# Restore Restarts files
805	#-----------------------
806	if ( [ X${FileName0} != X${NULL_STR} ] && [ X${FileName0} != XNONE ] ) ; then
807	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
808
809	(( i=1 ))
810	until [ $i -gt $NbFiles ]; do
811	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
812	eval file_in=${file_in_}
813	(( i_ = i+1 ))
814	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
815	eval file_out=${file_out_}
816
817	file_in_Name=${run_Configuration_OldPrefix}_${file_in}
818
819	extension_in=$( echo ${file_in_Name##*.} )
820	extension_out=$( echo ${file_out##*.} )
821
822	generic_restart_file_name_in=$( basename ${file_in_Name} .${extension_in} )
823	generic_restart_file_name_out=$( basename ${file_out} .${extension_out} )
824
825	eval Path_BUF=\${R_BUF_${comp}_R}/${generic_restart_file_name_in}
826	eval Path_OUT=\${R_OUT_${comp}_R}/${generic_restart_file_name_in}
827
828	if [ $( IGCM_sys_TestFileBuffer ${Path_BUF}*.${extension_in} ; echo $? ) = 0 ] ; then
829	IGCM_debug_Print 3 "Buffered restart ${Path_BUF}*.${extension_in}"
830	Buffered=true
831	Archived=false
832	Tared=false
833	nb_restart_file=$(IGCM_sys_CountFileBuffer ${Path_BUF}_????.${extension_in})
834	elif [ $( IGCM_sys_TestFileArchive ${Path_OUT}*.${extension_in} ; echo $? ) = 0 ] ; then
835	IGCM_debug_Print 3 "Archived restart ${Path_OUT}*.${extension_in}"
836	Buffered=false
837	Archived=true
838	Tared=false
839	nb_restart_file=$(IGCM_sys_CountFileArchive ${Path_OUT}_????.${extension_in})
840	else
841	IGCM_debug_Print 3 "No restart file in the buffer nor in the archive directory"
842	IGCM_debug_Print 3 "Restart files will now be searched for in : ${R_SAVE}/RESTART"
843	Buffered=false
844	Archived=false
845	Tared=true
846	# Look after the tar file we want if we did not found it already
847	if [ X${IsMatching} = X ] ; then
848	for PotentialTarFile in $( IGCM_sys_RshArchive "find ${R_SAVE}/RESTART -name "${config_UserChoices_JobName}_*_restart.tar" -print" ) ; do
849	IsMatching=$( echo ${PotentialTarFile##*/} \| sed "s:^${config_UserChoices_JobName}_::" \| sed "s:\.restart\.tar$::" \| gawk -F_ -v restartdate=${LastPeriodDateEnd} '{if (($1 < restartdate) && ($2 >= restartdate)) {print $1"_"$2}}' )
850	if [ ! X${IsMatching} = X ] ; then
851	TarFileFound=${PotentialTarFile}
852	break
853	fi
854	done
855	fi
856	IGCM_sys_PrepareTaredRestart ${TarFileFound}
857	TarFileLocation=$( basename ${TarFileFound} )
858	IGCM_debug_Print 1 "tar xvf ${TarFileLocation} ${comp}_${generic_restart_file_name_in}*.${extension_in}"
859	tar xvf ${TarFileLocation} ${comp}_${generic_restart_file_name_in}*.${extension_in}
860	nb_restart_file=$( IGCM_sys_CountFileBuffer ${comp}_${generic_restart_file_name_in}_????.${extension_in} )
861	fi
862
863	if [ ${nb_restart_file} -gt 1 ] ; then
864	j=0 # BASH LINE NOT NEEDED
865	#BASH for j4 in in $( eval echo {0000..$(( nb_restart_file - 1 ))} ) ; do
866	until [ $j -ge ${nb_restart_file} ]; do # BASH LINE NOT NEEDED
867	j4=${j} # BASH LINE NOT NEEDED
868	if [ X${Buffered} = Xtrue ] ; then
869	IGCM_sys_GetBuffer ${Path_BUF}_${j4}.${extension_in} ${generic_restart_file_name_out}_${j4}.${extension_out}
870	elif [ X${Archived} = Xtrue ] ; then
871	IGCM_sys_Get ${Path_OUT}_${j4}.${extension_in} ${generic_restart_file_name_out}_${j4}.${extension_out}
872	elif [ X${Tared} = Xtrue ] ; then
873	IGCM_sys_Mv ${comp}_${generic_restart_file_name_in}_${j4}.${extension_in} ${generic_restart_file_name_out}_${j4}.${extension_out}
874	fi
875	(( j=j+1 )) # BASH LINE NOT NEEDED
876	done
877	else
878	if [ X${Buffered} = Xtrue ] ; then
879	eval IGCM_sys_GetBuffer \${R_BUF_${comp}_R}/${file_in_Name} ${file_out}
880	elif [ X${Archived} = Xtrue ] ; then
881	eval IGCM_sys_Get \${R_OUT_${comp}_R}/${file_in_Name} ${file_out}
882	elif [ X${Tared} = Xtrue ] ; then
883	IGCM_sys_Mv ${comp}_${file_in_Name} ${file_out}
884	fi
885	fi
886	(( i=i+3 ))
887	done
888	else
889	if [ X${FileName0} != XNONE ] ; then
890	IGCM_debug_Exit "IGCM_comp_GetInputRestartFiles : No file in list for ${compname}."
891	else
892	IGCM_debug_Print 1 "IGCM_comp_GetInputRestartFiles : NONE specified in Restart List ${compname}."
893	fi
894	fi
895	fi
896	done
897
898	NbFiles=$( ls * 2> /dev/null \| wc -l )
899	if [ ${NbFiles} -gt 0 ] ; then
900	IGCM_sys_Chmod u+rw *
901	fi
902
903	IGCM_debug_PopStack "IGCM_comp_GetInputRestartFiles"
904	}
905
906	#=======================================================================
907	function IGCM_comp_PeriodStart
908	{
909	IGCM_debug_PushStack "IGCM_comp_PeriodStart"
910
911	# Debug Print :
912	echo
913	IGCM_debug_Print 1 "IGCM_comp_PeriodStart"
914	echo
915
916	typeset ExeNameIn ExeNameOut
917	typeset comp compname comptagname
918	for comp in ${config_ListOfComponents[*]} ; do
919	# Define component
920	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
921	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
922
923	# Copy executable for this component
924	eval ExeNameIn=\${config_Executable_${comp}[0]}
925	eval ExeNameOut=\${config_Executable_${comp}[1]}
926
927	# Debug Print
928	IGCM_debug_Print 3 "PeriodStart ${compname} Driver Function (if any)."
929	# UPDATE component
930	${comp}_PeriodStart 2> /dev/null
931
932	done
933
934	IGCM_debug_PopStack "IGCM_comp_PeriodStart"
935	}
936
937	#=======================================================================
938	function IGCM_comp_Update
939	{
940	IGCM_debug_PushStack "IGCM_comp_Update"
941
942	# Debug Print :
943	echo
944	IGCM_debug_Print 1 "IGCM_comp_Update"
945	echo
946
947	typeset ExeNameIn ExeNameOut
948	typeset comp compname comptagname
949	for comp in ${config_ListOfComponents[*]} ; do
950	# Define component
951	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
952	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
953
954	# Copy executable for this component
955	eval ExeNameIn=\${config_Executable_${comp}[0]}
956	eval ExeNameOut=\${config_Executable_${comp}[1]}
957
958	# If missing executable and DRYRUN is set to 0 or 1 then stop!
959	if [ ${DRYRUN} -le 1 ] && [ X${ExeNameIn} != X\"\" ] && [ ! -f ${R_EXE}/${ExeNameIn} ] ; then
960	IGCM_debug_Exit "IGCM_comp_Update missing executable ${ExeNameIn}"
961	fi
962
963	if [ ${Period} -eq 1 ] && [ -f ${R_EXE}/${ExeNameIn} ] ; then
964	eval IGCM_sys_Cp ${R_EXE}/${ExeNameIn} ${ExeNameOut}
965	if [ -f ${RUN_DIR}/${ExeNameOut} ] ; then
966	eval IGCM_sys_Chmod +rx ${RUN_DIR}/${ExeNameOut}
967	fi
968	elif [ -f ${R_EXE}/${ExeNameIn} ] && [ ! -f ${RUN_DIR}/${ExeNameOut} ] ; then
969	eval IGCM_sys_Cp ${R_EXE}/${ExeNameIn} ${ExeNameOut}
970	if [ -f ${RUN_DIR}/${ExeNameOut} ] ; then
971	eval IGCM_sys_Chmod +rx ${RUN_DIR}/${ExeNameOut}
972	fi
973	fi
974
975	# Debug Print
976	IGCM_debug_Print 1 "Update ${compname} Parameter Files."
977	# UPDATE component
978	${comp}_Update
979
980	done
981
982	IGCM_debug_PopStack "IGCM_comp_Update"
983	}
984
985	#=======================================================================
986	function IGCM_comp_Finalize
987	{
988	IGCM_debug_PushStack "IGCM_comp_Finalize"
989
990	# Debug Print :
991	echo
992	IGCM_debug_Print 1 "IGCM_comp_Finalize"
993	echo
994
995	typeset ListTextName TextName0
996	typeset comp compname comptagname card ListFilesName FileName0 NbFiles SaveOnArchive
997	typeset i i_ file_in file_in_ file_out file_out_ file_outin file_outin_ generic_file_name nb_rebuild_file
998	typeset -Z4 j4
999	#BASH declare j4
1000	typeset list_file nlist_file
1001	typeset compactoutputs
1002
1003	compactoutputs=false
1004	if [ X${JobType} != XRUN ] ; then
1005	compactoutputs=true
1006	elif [ X${config_UserChoices_CompactText} != Xn ] ; then
1007	compactoutputs=true
1008	fi
1009
1010	for comp in ${config_ListOfComponents[*]} ; do
1011	# Define component
1012	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
1013	eval comptagname=\${config_ListOfComponents_${comp}[1]} > /dev/null 2>&1
1014
1015	# Debug Print
1016	IGCM_debug_Print 1 "Finalize ${comp} : ${compname} component."
1017	# FINALIZE component
1018	${comp}_Finalize
1019
1020	card=${SUBMIT_DIR}/COMP/${compname}.card
1021
1022	# Save Output Text files of models
1023	#---------------------------------
1024	IGCM_debug_Print 2 "Save Output Text files for ${comp} : ${compname} component."
1025	IGCM_card_DefineArrayFromOption ${card} OutputText List
1026	ListTextName=${compname}_OutputText_List
1027
1028	eval TextName0=\${${ListTextName}[0]} > /dev/null 2>&1
1029	if [ X${TextName0} != X${NULL_STR} ] ; then
1030	eval NbFiles=\${#${ListTextName}[@]} > /dev/null 2>&1
1031
1032	(( i=0 ))
1033	until [ $i -eq $NbFiles ]; do
1034	eval file_in=\${${ListTextName}[$i]} > /dev/null 2>&1
1035	eval file_out=${PREFIX}_${file_in}
1036
1037	(( i=i+1 ))
1038
1039	unset list_file
1040	#set +A list_file -- $( ls ${file_in}* \| sort 2>/dev/null )
1041	# result for a a1 a10 a2 with file_in=a a a1 a2 a10
1042	set +A list_file -- $( [ -f ${file_in} ] && ls ${file_in} ; for i in $(ls ${file_in}* 2>/dev/null \| sed "s/${file_in}//" \| sort -n) ; do ls ${file_in}$i ; done )
1043	nlist_file=${#list_file[@]}
1044	if [ ${nlist_file} -gt 1 ] ; then
1045	if ( ${compactoutputs} ) ; then
1046	IGCM_debug_Print 2 "Parallelism of Text Output with ${nlist_file} files."
1047	IGCM_debug_Print 2 "Compact files in ${file_out} : " ${list_file[*]}
1048	echo ${list_file[*]} > ${file_out}
1049	echo "" >> ${file_out}
1050
1051	(( i_ = 0 ))
1052	for file in ${list_file[@]}
1053	do
1054	echo "_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ " >> ${file_out}
1055	echo "\| " ${i_} " " ${file} >> ${file_out}
1056	echo "- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - " >> ${file_out}
1057	cat ${file} \| sed "s/$.*$/${i_}\1/" ${file} >> ${file_out}
1058	echo "" >> ${file_out}
1059	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file}" > /dev/null 2>&1
1060	(( i_ = i_ + 1 ))
1061	done
1062	if [ X${Pack} = Xtrue ] ; then
1063	eval IGCM_sys_PutBuffer_Out ${file_out} \${R_BUF_${comp}_D}/${file_out}
1064	else
1065	eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}_D}/${file_out}
1066	fi
1067	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file_out}" > /dev/null 2>&1
1068
1069	if ( ${ExecutionFail} ) ; then
1070	IGCM_sys_Cp ${file_out} ${SUBMIT_DIR}/Debug
1071	fi
1072	else
1073	for file in ${list_file[@]}
1074	do
1075	if [ X${Pack} = Xtrue ] ; then
1076	eval IGCM_sys_PutBuffer_Out ${file} \${R_BUF_${comp}_D}/${PREFIX}_${file}
1077	else
1078	eval IGCM_sys_Put_Out ${file} \${R_OUT_${comp}_D}/${PREFIX}_${file}
1079	fi
1080	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file}" > /dev/null 2>&1
1081
1082	if ( ${ExecutionFail} ) ; then
1083	IGCM_sys_Cp ${file} ${SUBMIT_DIR}/Debug/${PREFIX}_${file}
1084	fi
1085	done
1086	fi
1087	else
1088	if ( [ -f ${file_in}_0000 ] \|\| [ -f ${file_in}0 ] ) ; then
1089	eval IGCM_sys_Mv ${file_in}* ${file_in}
1090	fi
1091	if [ X${Pack} = Xtrue ] ; then
1092	eval IGCM_sys_PutBuffer_Out ${file_in} \${R_BUF_${comp}_D}/${file_out}
1093	else
1094	eval IGCM_sys_Put_Out ${file_in} \${R_OUT_${comp}_D}/${file_out}
1095	fi
1096	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file_in}" > /dev/null 2>&1
1097
1098	if ( ${ExecutionFail} ) ; then
1099	IGCM_sys_Cp ${file_in} ${SUBMIT_DIR}/Debug/${file_out}
1100	fi
1101	fi
1102	done
1103	fi
1104
1105	# Save Restarts files
1106	#--------------------
1107	IGCM_debug_Print 2 "Save Restart files for ${comp} : ${compname} component."
1108	IGCM_card_DefineArrayFromOption ${card} RestartFiles List
1109	ListFilesName=${compname}_RestartFiles_List
1110	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
1111
1112	if ( [ X${FileName0} != X${NULL_STR} ] && [ X${FileName0} != XNONE ] ) ; then
1113	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
1114
1115	(( i=0 ))
1116	until [ $i -ge $NbFiles ]; do
1117	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
1118	eval file_in=${file_in_}
1119
1120	(( i_ = i+1 ))
1121	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
1122	eval file_out=${file_out_}
1123
1124	(( i_ = i+2 ))
1125	eval file_outin_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
1126	eval file_outin=${file_outin_}
1127
1128	generic_restart_file_name_in=$( basename ${file_in} .nc )
1129	generic_restart_file_name_out=$( basename ${config_UserChoices_JobName}_${PeriodDateEnd}_${file_out} .nc )
1130	generic_restart_file_name_outin=$( basename ${file_outin} .nc )
1131
1132	nb_restart_file=$( ls ${generic_restart_file_name_in}_????.nc 2>/dev/null \| wc -l )
1133	if [ ${nb_restart_file} -gt 1 ] ; then
1134	j=0 # BASH LINE NOT NEEDED
1135	# BASH for j4 in in $( eval echo {0000..$(( nb_restart_file - 1 ))} ) ; do
1136	until [ $j -ge ${nb_restart_file} ]; do # BASH LINE NOT NEEDED
1137	j4=${j} # BASH LINE NOT NEEDED
1138	if [ X${Pack} = Xtrue ] ; then
1139	eval IGCM_sys_PutBuffer_Rest ${generic_restart_file_name_in}_${j4}.nc \${R_BUF_${comp}_R}/${generic_restart_file_name_out}_${j4}.nc
1140	else
1141	eval IGCM_sys_Put_Rest ${generic_restart_file_name_in}_${j4}.nc \${R_OUT_${comp}_R}/${generic_restart_file_name_out}_${j4}.nc
1142	fi
1143	if [ ! ${file_in} = ${file_outin} ] ; then
1144	if ( ${ExitFlag} ) ; then
1145	echo "IGCM_sys_Mv ${generic_restart_file_name_in}_${j4}.nc ${generic_restart_file_name_outin}_${j4}.nc not executed."
1146	else
1147	IGCM_sys_Mv ${generic_restart_file_name_in}_${j4}.nc ${generic_restart_file_name_outin}_${j4}.nc
1148	fi
1149	fi
1150	(( j=j+1 )) # BASH LINE NOT NEEDED
1151	done
1152	else
1153	if [ X${Pack} = Xtrue ] ; then
1154	eval IGCM_sys_PutBuffer_Rest ${file_in} \${R_BUF_${comp}_R}/${config_UserChoices_JobName}_${PeriodDateEnd}_${file_out}
1155	else
1156	eval IGCM_sys_Put_Rest ${file_in} \${R_OUT_${comp}_R}/${config_UserChoices_JobName}_${PeriodDateEnd}_${file_out}
1157	fi
1158	if [ ! ${file_in} = ${file_outin} ] ; then
1159	if ( ${ExitFlag} ) ; then
1160	echo "IGCM_sys_Mv ${file_in} ${file_outin} not executed."
1161	else
1162	IGCM_sys_Mv ${file_in} ${file_outin}
1163	fi
1164	fi
1165	fi
1166
1167	(( i=i+3 ))
1168	done
1169	else
1170	if [ X${FileName0} != XNONE ] ; then
1171	IGCM_debug_Exit "IGCM_comp_Finalize : No file in restart list for ${compname}."
1172	else
1173	IGCM_debug_Print 1 "IGCM_comp_Finalize : NONE specified in Restart List ${compname}."
1174	fi
1175	fi
1176
1177	# Save Output files
1178	#------------------
1179	IGCM_debug_Print 2 "Save Output files for ${comp} : ${compname} component."
1180	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
1181	ListFilesName=${compname}_OutputFiles_List
1182	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
1183
1184	if [ X${FileName0} != X${NULL_STR} ] ; then
1185	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
1186
1187	(( i=0 ))
1188	until [ $i -ge $NbFiles ]; do
1189	SaveOnArchive=true
1190	eval file_in_=\${${ListFilesName}[$i]} > /dev/null 2>&1
1191	eval file_in=${file_in_}
1192	(( i_ = i+1 ))
1193	eval file_out_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
1194	eval file_out=${file_out_}
1195	#
1196	# Override file_out path remplacing R_SAVE by R_BUFR
1197	#
1198	if [ X${Pack} = Xtrue ] ; then
1199	file_out=$( echo $file_out \| sed "s:^$R_SAVE:$R_BUFR:" )
1200	fi
1201	#
1202	# Not necessarily the best option. /!\ Potential side effects /!\
1203	#
1204	(( i_ = i+2 ))
1205	eval flag_post=\${${ListFilesName}[$i_]} > /dev/null 2>&1
1206	#
1207	generic_file_name=$( basename ${file_in} .nc )
1208	nb_rebuild_file=$( ls \| grep "^${generic_file_name}_[0-9]*.nc" \| wc -l )
1209	#
1210	if ( [ ${nb_rebuild_file} -eq 1 ] && [ -f ${generic_file_name}_0000.nc ] ) ; then
1211	IGCM_debug_Print 2 "Parallelism with 1 file. Rebuilding ${file_in} not needed"
1212	IGCM_sys_Mv ${generic_file_name}_0000.nc ${file_in}
1213	elif [ ${nb_rebuild_file} -gt 1 ] ; then
1214	IGCM_debug_Print 2 "Parallelism detected and rebuilding ${file_in} is needed"
1215	if [ X${AsynchronousRebuild} = Xfalse ] ; then
1216	IGCM_debug_Print 2 "Rebuilding ${file_in} online"
1217	IGCM_sys_rebuild ${file_in} ${generic_file_name}_[0-9]*.nc
1218	else
1219	IGCM_debug_Print 2 "Preparing offline rebuild for ${file_in}"
1220	[ ! -d ${RUN_DIR}/REBUILD_${PeriodDateBegin} ] && IGCM_sys_Mkdir ${RUN_DIR}/REBUILD_${PeriodDateBegin}
1221	IGCM_sys_Mv ${generic_file_name}_[0-9]*.nc ${RUN_DIR}/REBUILD_${PeriodDateBegin}
1222
1223	# Prepare headers for the shell dedicated to offline rebuild
1224	if [ ! -f ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh ] ; then
1225	if [ $DRYRUN -le 1 ]; then
1226	echo "#!/bin/ksh " > ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1227	echo "function IGCM_FlushRebuild " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1228	echo "{ " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1229	echo "IGCM_debug_PushStack \"IGCM_FlushRebuild\" " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1230	echo "echo " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1231	echo "IGCM_debug_Print 1 \"IGCM_FlushRebuild\" " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1232	echo "echo " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1233	echo "export R_SAVE=${R_SAVE} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1234	echo "export R_BUFR=${R_BUFR} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1235	echo "export R_OUT_KSH=${R_OUT_KSH} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1236	echo "export R_BUF_KSH=${R_BUF_KSH} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1237	echo "export config_UserChoices_JobName=${config_UserChoices_JobName} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1238	echo "export config_UserChoices_SpaceName=${config_UserChoices_SpaceName} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1239	fi
1240	fi
1241	# Prepare the shell dedicated to offline rebuild
1242	if [ $DRYRUN -le 1 ]; then
1243	if [ ${file_in} = histstn.nc ] ; then
1244	echo "IGCM_sys_rebuild_station ${file_in} ${generic_file_name}_*.nc" >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1245	else
1246	echo "IGCM_sys_rebuild ${file_in} ${generic_file_name}_[0-9]*.nc" >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1247	fi
1248	echo "IGCM_debug_Verif_Exit_Post" >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1249	fi
1250	#
1251	# Load Patch we need to apply and apply
1252	if [ $DRYRUN -le 1 ]; then
1253	if [ X$( eval echo \${${compname}_${flag_post}_Patches[0]} ) != X${NULL_STR} ]; then
1254	for Patch in $( eval echo \${${compname}_${flag_post}_Patches[*]} ); do
1255	echo ". ${libIGCM_POST}/libIGCM_post/IGCM_${Patch}.ksh" >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1256	echo "IGCM_${Patch} ${file_in} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1257	echo "IGCM_debug_Verif_Exit_Post " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1258	done
1259	fi
1260	fi
1261	#
1262	if [ $DRYRUN -le 1 ]; then
1263	if [ X${Pack} = Xtrue ] ; then
1264	echo "IGCM_sys_PutBuffer_Out ${file_in} ${file_out} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1265	else
1266	echo "IGCM_sys_Put_Out ${file_in} ${file_out} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1267	fi
1268	echo "IGCM_debug_Verif_Exit_Post " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1269	echo "IGCM_sys_Rm ${generic_file_name}_[0-9]*.nc " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1270	fi
1271	SaveOnArchive=false
1272	fi
1273	fi
1274	#
1275	if [ ${SaveOnArchive} = true ] ; then
1276	#
1277	# Rebuild has been done online or it was not needed
1278	#
1279	# If we need to apply a patch we use TMP DIRECTORY before ARCHIVING if asynchronous rebuild is on
1280	#
1281	thereisapatch=$( eval echo \${${compname}_${flag_post}_Patches[0]} )
1282	if ( [ ! X${thereisapatch} = X${NULL_STR} ] && [ X${AsynchronousRebuild} = Xtrue ] && [ -f ${file_in} ] ) ; then
1283	[ ! -d ${RUN_DIR}/REBUILD_${PeriodDateBegin} ] && IGCM_sys_Mkdir ${RUN_DIR}/REBUILD_${PeriodDateBegin}
1284	IGCM_sys_Mv ${file_in} ${RUN_DIR}/REBUILD_${PeriodDateBegin}
1285	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=REBUILD_${PeriodDateBegin}/${file_in} > /dev/null 2>&1
1286	#
1287	if [ $DRYRUN -le 1 ]; then
1288	if [ ! -f ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh ] ; then
1289	echo "#!/bin/ksh " > ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1290	echo "function IGCM_FlushRebuild " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1291	echo "{ " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1292	echo "IGCM_debug_PushStack \"IGCM_FlushRebuild\" " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1293	echo "echo " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1294	echo "IGCM_debug_Print 1 \"IGCM_FlushRebuild\" " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1295	echo "echo " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1296	echo "export R_SAVE=${R_SAVE} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1297	echo "export config_UserChoices_JobName=${config_UserChoices_JobName} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1298	fi
1299	#
1300	for Patch in $( eval echo \${${compname}_${flag_post}_Patches[*]} ); do
1301	echo ". ${libIGCM_POST}/libIGCM_post/IGCM_${Patch}.ksh" >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1302	echo "IGCM_${Patch} ${file_in} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1303	echo "IGCM_debug_Verif_Exit_Post " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1304	done
1305	#
1306	if [ X${Pack} = Xtrue ] ; then
1307	echo "IGCM_sys_PutBuffer_Out ${file_in} ${file_out} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1308	else
1309	echo "IGCM_sys_Put_Out ${file_in} ${file_out} " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1310	fi
1311	echo "IGCM_debug_Verif_Exit_Post " >> ${RUN_DIR}/REBUILD_${PeriodDateBegin}/rebuild.ksh
1312	#
1313	fi
1314	else
1315	#
1316	# No Patch, No Asynchronous rebuild, online rebuild has been done or was not needed
1317	#
1318	if [ X${Pack} = Xtrue ] ; then
1319	IGCM_sys_PutBuffer_Out ${file_in} ${file_out}
1320	else
1321	IGCM_sys_Put_Out ${file_in} ${file_out}
1322	fi
1323	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${file_in}" > /dev/null 2>&1
1324	if [ ${nb_rebuild_file} -gt 1 ] ; then
1325	for DelFile in $( ls \| grep "${generic_file_name}[_0-9]*.nc" ) ; do
1326	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=${DelFile} > /dev/null 2>&1
1327	done
1328	fi
1329	fi
1330	fi
1331	(( i=i+3 ))
1332	done
1333	fi
1334	echo
1335	done
1336	IGCM_debug_PopStack "IGCM_comp_Finalize"
1337	}

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_comp/libIGCM_comp.ksh @ 980

Download in other formats: