Context Navigation

libIGCM_config.ksh @ 1603

Last change on this file since 1603 was 1603, checked in by aclsce, 7 months ago

Changed libIGCM behaviour in case of pb in pack_output.job : now, we exit from the job if there is any problem, whatever we are in PROD, DEVT, TEST mode.
Specification of pack frequency per file :

In order to reduce the number of inodes, it is possible to specify by file the frequency of packing. The syntax to do that is in the 4th column of OutputFiles? section of the component.card, for example as follows in lmdz.card :
[OutputFiles?]
List= (histmth.nc, ${R_OUT_ATM_O_M}/${PREFIX}_1M_histmth.nc, Post_1M_histmth, 100Y), \

(histday.nc, ${R_OUT_ATM_O_D}/${PREFIX}_1D_histday.nc, Post_1D_histday, 10Y), \

...

In this example, histmth files will be packed every 100 years and histday files will be packed every 10 years.
The pack frequency you defined in config.card is the frequency of pack by default, that means if a specific frequency of pack is specified for a file in a component.card, this file will be packed at the specific frequency whereas all other files will be packed at global pack frequency (specified in config.card) and in this case, the frequency pack (from the config.card) is the frequency the pack_output job will be launched at.
There is a constraint to use this fonctionality : the Packfrequency you defined in config.card must be greater or equal to the pack frequencies you specified for each type of file in component.card, otherwise the computing job will be stopped (with an explicit error message).
Surpack mode :
A surpack mode functionality is available through the use of pack_output.job. To enable this functionality, you have to put "surpack_mode=y" (default value is n). The way to use is similar to restart post-processing pack_output jobs, as indicated here : http://forge.ipsl.jussieu.fr/igcmg_doc/wiki/Doc/CheckDebug#RestartPack_output. You can either use a global pack frequency in config.card or specific pack frequency per file, as explained above.

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 63.2 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_CommonConfiguration
16	{
17	IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19	# Debug Print :
20	[ ${Verbosity} -gt 0 ] && echo
21	IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23	# config.card path
24	configCardPath=$1
25
26	#==================================
27	typeset option auxprint
28
29	#==================================
30	# Read UserChoices section:
31	[ ${Verbosity} -gt 0 ] && echo
32	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
33
34	IGCM_card_DefineArrayFromSection ${configCardPath} UserChoices
35	for option in ${config_UserChoices[*]} ; do
36	IGCM_card_DefineVariableFromOption ${configCardPath} UserChoices ${option}
37	eval auxprint=\${config_UserChoices_${option}}
38	IGCM_debug_Print 3 "${option} : ${auxprint}"
39	done
40
41	# Set variables with shorter names on some variables which
42	# are used in the executable name in config.card
43	ResolAtm=${config_UserChoices_ResolAtm}
44	ResolOce=${config_UserChoices_ResolOce}
45	ConfChem=${config_UserChoices_ConfChem}
46	OptMode=${config_UserChoices_OptMode}
47	ConfigType=${config_UserChoices_ConfigType}
48
49	#==================================
50	# Read Ensemble section:
51	[ ${Verbosity} -gt 0 ] && echo
52	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
53
54	IGCM_card_DefineArrayFromSection ${configCardPath} Ensemble
55	for option in ${config_Ensemble[*]} ; do
56	IGCM_card_DefineVariableFromOption ${configCardPath} Ensemble ${option}
57	eval auxprint=\${config_Ensemble_${option}}
58	IGCM_debug_Print 3 "${option} : ${auxprint}"
59	done
60
61	#==================================
62	# Read Post section:
63	[ ${Verbosity} -gt 0 ] && echo
64	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
65
66	IGCM_card_DefineArrayFromSection ${configCardPath} Post
67	for option in ${config_Post[*]} ; do
68	IGCM_card_DefineVariableFromOption ${configCardPath} Post ${option}
69	eval auxprint=\${config_Post_${option}}
70	IGCM_debug_Print 3 "${option} : ${auxprint}"
71	done
72	[ ${Verbosity} -gt 0 ] && echo
73
74	#==================================
75	# Define default value to keep compatibility with previous card: means before changes due to TGCC
76	# Apply some overrules to ensure proper usage of computing centres resources
77	#
78
79	# if RebuildFrequency is not define in config.card - we initilize it to NONE
80	if [ X${config_Post_RebuildFrequency} = X ] ; then
81	config_Post_RebuildFrequency=NONE
82	fi
83
84	if [ X${PackDefault} = Xtrue ] ; then
85	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
86	# TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
87	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
88	config_Post_PackFrequency=NONE
89	else
90	# Default to RebuildFrequency if nothing has been set up related to PackFrequency
91	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
92	fi
93	else
94	# If we DO NOT apply pack in this computing center
95	config_Post_PackFrequency=NONE
96	fi
97
98	#====================================================
99	# Define ARCHIVE : Dedicated to large files
100	# Define STORAGE : Dedicated to small/medium files
101	# Define R_OUT : Output tree located on ARCHIVE
102	# Define R_BUF : Output tree located on STORAGE (files waiting treatment, or file lcoation when SpaceName=!PROD)
103	# Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
104	# Define R_TMP : A temporary space used by IGCM_debug_send_AMQP_msg__MAILTUNNEL. Must be persistent in between jobs
105	IGCM_sys_defineArchives
106
107	#====================================================
108	# R_SAVE : Job output directory
109	# R_BUFR : Job output buffered directory
110	# R_CMIP : Job output directory for CMIP6 standard files
111
112	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
113	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
114
115	if ( [ X${config_Post_StorageCMIP} = XSCRATCH ] \|\| [ X${config_Post_StorageCMIP} = Xscratch ] ) ; then
116	IGCM_debug_Print 1 "LIBIGCM CONFIG:: SCRATCH R_BUF=${R_BUF}"
117	StoreCMIP=${R_BUF}
118	elif ( [ X${config_Post_StorageCMIP} = XSTORE ] \|\| [ X${config_Post_StorageCMIP} = Xstore ] ) ; then
119	IGCM_debug_Print 1 "LIBIGCM CONFIG:: SCRATCH R_OUT=${R_OUT}"
120	StoreCMIP=${R_OUT}
121	else
122	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ELSE CMIP_BUF=${CMIP6_BUF}"
123	StoreCMIP=${CMIP6_BUF}
124	fi
125	# Ensemble cases (default all members outputs in same directory)
126	if ( [ ! X${config_Ensemble_EnsembleName} = X ] ) ; then
127	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
128	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
129	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
130	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
131	R_CMIP=${StoreCMIP}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
132	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE R_CMIP=${R_CMIP}"
133
134	# PERTURB Ensemble Outputs
135	if ( [ X${config_Ensemble_EnsembleType} = XEns_PERTURB ] ) ; then
136	if ( [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
137	R_SAVE=${R_SAVE}/${config_Ensemble_EnsembleDate}/${FreeName}
138	R_FIGR=${R_FIGR}/${config_Ensemble_EnsembleDate}/${FreeName}
139	R_BUFR=${R_BUFR}/${config_Ensemble_EnsembleDate}/${FreeName}
140	R_DODS=${R_DODS}/${config_Ensemble_EnsembleDate}/${FreeName}
141	R_CMIP=${R_CMIP}/${config_Ensemble_EnsembleDate}/${FreeName}
142	else
143	R_SAVE=${R_SAVE}/${FreeName}
144	R_FIGR=${R_FIGR}/${FreeName}
145	R_BUFR=${R_BUFR}/${FreeName}
146	R_DODS=${R_DODS}/${FreeName}
147	R_CMIP=${R_CMIP}/${FreeName}
148	fi
149	# DATE Ensemble Outputs
150	elif ( [ X${config_Ensemble_EnsembleType} = XEns_DATE ] ) ; then
151	R_FIGR=${R_FIGR}/${FreeName}
152	R_BUFR=${R_BUFR}/${FreeName}
153	R_DODS=${R_DODS}/${FreeName}
154	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE DATE R_CMIP=${R_CMIP}"
155	if ( [ X${config_Ensemble_EnsembleMergeSave} = X ] \|\| [ X${config_Ensemble_EnsembleMergeSave} = Xn ] \|\|
156	[ X${config_Ensemble_EnsembleMergeSave} = Xno ] ) ; then
157	# One SAVE directory per member, else common output dir for all members
158	R_SAVE=${R_SAVE}/${FreeName}
159	R_CMIP=${R_CMIP}/${FreeName}
160	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE DATE MERGE R_CMIP=${R_CMIP}"
161	fi
162	fi
163	else
164	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
165	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
166	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
167	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
168	R_CMIP=${StoreCMIP}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
169	fi
170	else
171	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
172	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
173	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
174	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
175	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
176	else
177	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
178	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
179	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
180	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
181	fi
182	fi
183
184	IGCM_debug_Print 1 "LIBIGCM CONFIG:: R_CMIP = ${R_CMIP}"
185
186	#====================================================
187	# Define R_OUT_KSH : Storage place for job output
188	# Define R_OUT_EXE : Storage place for binary used during simulation
189	R_OUT_KSH=${R_SAVE}/Out
190	R_OUT_EXE=${R_SAVE}/Exe
191
192	#====================================================
193	# Define R_BUF_KSH : Buffer place for job output
194	# Define R_BUF_EXE : Buffer place for binary used during simulation
195	R_BUF_KSH=${R_BUFR}/Out
196	R_BUF_EXE=${R_BUFR}/Exe
197
198	#====================================================
199	# Define REBUILD_DIR : where we store files needing rebuild process
200	REBUILD_DIR=${R_BUFR}/REBUILD
201	if [ ! X${TaskType} = Xchecking ] ; then
202	IGCM_sys_MkdirWork ${REBUILD_DIR}
203	fi
204
205	#====================================================
206	# DodsCopy : apply default value if not defined
207	if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] \|\| [ X${config_Post_DodsCopy} = X ] ) ; then
208	config_Post_DodsCopy=TRUE
209	fi
210
211	#====================================================
212	# AtlasIPSL : apply default value if not defined
213	if ( [ X${config_Post_AtlasIPSL} = X${NULL_STR} ] \|\| [ X${config_Post_AtlasIPSL} = X ] ) ; then
214	config_Post_AtlasIPSL=FALSE
215	fi
216
217	#====================================================
218	# ParserXIOS : apply default value if not defined
219	if ( [ X${config_Post_ParserXIOS} = X${NULL_STR} ] \|\| [ X${config_Post_ParserXIOS} = X ] ) ; then
220	config_Post_ParserXIOS=FALSE
221	fi
222
223	#====================================================
224	# MetricsPCMDI : apply default value if not defined
225	if ( [ X${config_Post_MetricsPCMDI} = X${NULL_STR} ] \|\| [ X${config_Post_MetricsPCMDI} = X ] ) ; then
226	config_Post_MetricsPCMDI=FALSE
227	fi
228	#====================================================
229	# Cesmep : apply default value FALSE if not defined, and if TRUE
230	# or *Y set it based on an adhoc algorithm
231	if ( [ X${config_Post_Cesmep} = X${NULL_STR} ] \|\| [ X${config_Post_Cesmep} = XNONE ] \|\| \
232	[ X${config_Post_Cesmep} = X ] \|\| [ X${config_Post_Cesmep} = XFALSE ] ) ; then
233	config_Post_Cesmep=FALSE
234	elif [ X${config_Post_Cesmep} = XTRUE ] \|\| [[ X${config_Post_Cesmep} =~ X[1-9][0-9]?Y ]] ; then
235	if [[ X${config_Post_Cesmep} =~ X[1-9][0-9]?Y ]] ; then
236	CesmepPeriod=${config_Post_Cesmep%Y}
237	fi
238	if ([ ! X${config_Post_SeasonalFrequency} = XNONE ] && [ ! X${config_Post_SeasonalFrequency} = X ] ) ; then
239	config_Post_Cesmep="SE"
240	elif ([ ! X${config_Post_TimeSeriesFrequency} = XNONE ] && [ ! X${config_Post_TimeSeriesFrequency} = X ] ) ; then
241	config_Post_Cesmep="TS"
242	elif ([ ! X${config_Post_PackFrequency} = XNONE ] && [ ! X${config_Post_PackFrequency} = X ] ) ; then
243	config_Post_Cesmep="Pack"
244	else
245	# This covers case SpaceName=TEST with no SE nor TS (nor Pack)
246	config_Post_Cesmep=AtEnd
247	fi
248	fi
249	#
250	if [ X${config_Post_Cesmep} != XFALSE ] ; then
251	if [ X${config_Post_Cesmep} = XSE ] ; then
252	if ([ X${config_Post_SeasonalFrequency} = XNONE ] \|\| [ X${config_Post_SeasonalFrequency} = X ] ) ; then
253	IGCM_debug_Exit "Cannot use SE for Cesmep : SE post-processing is not active"
254	else
255	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_SeasonalFrequency%Y}
256	fi
257	elif [ X${config_Post_Cesmep} = XTS ] ; then
258	if ([ X${config_Post_TimeSeriesFrequency} = XNONE ] \|\| [ X${config_Post_TimeSeriesFrequency} = X ] ) ; then
259	IGCM_debug_Exit "Cannot use TS for Cesmep : TS post-processing is not active"
260	else
261	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_TimeSeriesFrequency%Y}
262	fi
263	elif [ X${config_Post_Cesmep} = XPack ] ; then
264	if ([ X${config_Post_PackFrequency} = XNONE ] \|\| [ X${config_Post_PackFrequency} = X ] ) ; then
265	IGCM_debug_Exit "Cannot use Packed output for Cesmep : Pack post-processing is not active"
266	else
267	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_PackFrequency%Y}
268	fi
269	elif [ X${config_Post_Cesmep} != XAtEnd ] ; then
270	IGCM_debug_Exit "config.card's Post section's value for Cesmep (${config_Post_Cesmep}) is invalid; use SE, TS, Pack, AtEnd, TRUE or <n>Y"
271	else
272	[ -z $CesmepPeriod ] && CesmepPeriod=0
273	fi
274
275	if ( [ X${config_Post_CesmepComparison} = XNONE ] \|\| [ X${config_Post_CesmepComparison} = X ] ) ; then
276	config_Post_CesmepComparison=run_comparison
277	fi
278
279	if ( [ X${config_Post_CesmepCode} = XNONE ] \|\| [ X${config_Post_CesmepCode} = X ] ) ; then
280	if ( [ X"${SYSTEM}" == "Xirene" ] \|\| [ X"${SYSTEM}" == "Xirene-amd" ] ); then
281	config_Post_CesmepCode=/ccc/cont003/home/igcmg/igcmg/Tools/cesmep
282	elif ( [ X"${SYSTEM}" == "Xjeanzay" ] ); then
283	config_Post_CesmepCode=/gpfswork/rech/psl/commun/Tools/cesmep
284	elif ( [ X"${SYSTEM}" == "Xmesoipsl" ] ); then
285	config_Post_CesmepCode=/net/nfs/tools/Users/SU/jservon/cesmep_installs/cesmep_for_libIGCM
286	elif [ ${config_Post_Cesmep} != FALSE ] ; then
287	IGCM_debug_Exit "Cannot locate a default C-ESM-EP code on $SYSTEM, please set CesmepCode in section Post"
288	fi
289	fi
290	if ( [ X${config_Post_CesmepSlices} = XNONE ] \|\| [ X${config_Post_CesmepSlices} = X ] ) ; then
291	config_Post_CesmepSlices=8
292	fi
293	fi
294	IGCM_debug_Print 3 "config_Post_Cesmep=${config_Post_Cesmep}, CesmepPeriod=$CesmepPeriod, config_Post_CesmepSlices=$config_Post_CesmepSlices "
295	#====================================================
296	# IgnoreNonMonotonic : apply default value if not defined
297	if ( [ X${config_Post_IgnoreNonMonotonic} = X${NULL_STR} ] \|\| [ X${config_Post_IgnoreNonMonotonic} = X ] ) ; then
298	config_Post_IgnoreNonMonotonic=FALSE
299	fi
300
301	#====================================================
302	# LightRestartPack : apply default value if not defined
303	if ( [ X${config_Post_LightRestartPack} = X${NULL_STR} ] \|\| [ X${config_Post_LightRestartPack} = X ] ) ; then
304	config_Post_LightRestartPack=FALSE
305	fi
306
307	#====================================================
308	# dr2xmlIPSL : apply default value if not defined
309	if ( [ X${config_Post_dr2xmlIPSL} = X${NULL_STR} ] \|\| [ X${config_Post_dr2xmlIPSL} = X ] ) ; then
310	config_Post_dr2xmlIPSL=FALSE
311	fi
312
313
314	#====================================================
315	# Define StackFileLocation : directory where we store stack files
316	# Define StackFileName : stack file containing call tree and instrumentation
317	# Stack file containing call tree will be stored there.
318	if ( $DEBUG_debug ) ; then
319	StackFileLocation=${StackFileLocation:=${R_BUF_KSH}}
320	[ ! -d ${StackFileLocation} ] && mkdir -p ${StackFileLocation}
321	if [ X${TaskType} = Xcomputing ]; then
322	StackFileName=computing.stack.$$
323	elif [ X${TaskType} = Xpost-processing ]; then
324	StackFileName=${Script_Post_Output}.stack.$$
325	elif [ X${TaskType} = Xchecking ]; then
326	StackFileName=checking.stack.$$
327	else
328	IGCM_debug_Exit "IGCM_config_CommonConfiguration unknown TaskType : ${TaskType}"
329	IGCM_debug_Verif_Exit
330	fi
331
332	# This boolean will trigger the filling of the stack
333	# Only now we know where things should be ...
334	# We don't fill the stack when we perform checking task
335	if [ ! X${TaskType} = Xchecking ] ; then
336	ActivateStackFilling=true
337	fi
338	fi
339
340	IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
341	}
342
343	#===================================
344	function IGCM_config_Initialize
345	{
346	IGCM_debug_PushStack "IGCM_config_Initialize"
347
348	# Debug Print :
349	echo
350	IGCM_debug_Print 1 "IGCM_config_Initialize"
351
352	# Test modipsl tree existence.
353	IGCM_sys_TestDir ${MODIPSL}
354	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
355	IGCM_sys_TestDir ${libIGCM}
356	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
357	IGCM_sys_TestDir ${R_EXE}
358	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
359	IGCM_sys_TestDir ${SUBMIT_DIR}
360	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
361
362	if ( $DEBUG_debug ) ; then
363	echo "Keep trace of inital SUBMIT_DIR : "
364	ls -lta ${SUBMIT_DIR}
365	fi
366
367	#==================================
368	# Read ListOfComponents section:
369	echo
370	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
371
372	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
373	for comp in ${config_ListOfComponents[*]} ; do
374	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
375	done
376	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
377
378	#==================================
379	# Read Executable section:
380	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
381
382	#==================================
383	# Read Restarts section:
384	# Restarts : Gerneral rule or local for each component.
385	echo
386	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
387
388	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
389	for option in ${config_Restarts[*]} ; do
390	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
391	eval auxprint=\${config_Restarts_${option}}
392	IGCM_debug_Print 3 "${option} : ${auxprint}"
393	done
394
395	#==================================
396	# Define Job Outputs Name
397	echo
398	IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
399	Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
400	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
401	Exe_Output=out_execution
402	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
403
404	#===================================================================#
405	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
406	# But available to any son functions #
407	#===================================================================#
408
409	# Convert yyyy-mm-dd date to gregorian yyyymmdd
410	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
411	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
412	EndYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateEnd ) / 1000 ))
413	EndYear_p1=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateEnd ) / 1000 + 1 ))
414
415
416	# Period Length In Days between DateBegin and DateEnd
417	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
418	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
419	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
420	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
421	IGCM_debug_Verif_Exit
422	fi
423
424	# Day and Year of Initial State (Given in julian format)
425	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
426	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
427	InitYear_m1=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 - 1 ))
428
429	#================================================================#
430	# Test and Prepare directories #
431	#================================================================#
432
433	# ==> 4 kinds of input files :
434	# 1) R_INIT : Initial State Files (Etat0, carteveg)
435	# 2) R_BC : Boundary Conditions (Forcages, lai)
436	# 3) Parameters files (allready define through ${SUBMIT_DIR})
437	# 4) Restarts files (allready define in IGCM_config_Initialize)
438
439	# Here we offer the possibility to redefine R_INIT, R_BC
440	# and PeriodNb through config.card
441	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
442	echo
443	IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
444	IGCM_debug_Print 3 "R_IN=${R_IN}"
445	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
446	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
447	IGCM_debug_Print 3 "R_BC=${R_BC}"
448	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
449	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
450
451	# SD ADA SPECIFIC #
452	# TO FIX #
453	#IGCM_sys_TestDirArchive ${R_IN}
454	#[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
455
456	if ( ${FirstInitialize} ) ; then
457	IGCM_sys_MkdirArchive ${R_SAVE}
458	[ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
459	else
460	IGCM_sys_TestDirArchive ${R_SAVE}
461	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
462
463	if [ ! ${config_Post_PackFrequency} = NONE ] ; then
464	IGCM_sys_TestDir ${R_BUFR}
465	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
466	fi
467
468	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
469	IGCM_config_StateCheck
470
471	# And EXIT if not OK
472	IGCM_debug_Verif_Exit
473	fi
474
475	#====================================================
476	# Experience type : DEB(ug), DEV(elopment), RUN
477	if [ X${JobType} != XRUN ] ; then
478	echo
479	echo "===================================================="
480	echo "libIGCM JOB is NOT in RUN type mode."
481	echo "!! OUTPUT files will NOT be PROTECTED !!"
482	echo "Be carefull : you can ERASE the result of this job !"
483
484	case ${JobType} in
485	DEB)
486	echo "DEBUG mode : activation of 'set -vx' mode."
487	echo "DEBUG mode : no protection for output files."
488	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
489	;;
490	DEV)
491	echo "DEVelopment mode : no protection for output files."
492	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
493	;;
494	esac
495
496	if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
497	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
498	echo "------------"
499	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
500	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
501	echo "------------"
502	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
503	fi
504	fi
505	echo "===================================================="
506	echo
507	fi
508
509	IGCM_debug_PopStack "IGCM_config_Initialize"
510	}
511
512	#===================================
513	function IGCM_config_DaysInPeriodLength
514	{
515	IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
516
517	typeset i
518
519	# Determine number of day(s) in PeriodLength :
520	case ${config_UserChoices_PeriodLength} in
521	Y\|y)
522	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
523	echo
524	IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
525	PeriodLengthInDays=0
526	i=0
527	until [ $i -ge $PeriodLengthInYears ] ; do
528	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
529	(( i=i+1 ))
530	done
531	;;
532	M\|m)
533	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
534	echo
535	IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
536	PeriodLengthInDays=0
537	i=0
538	until [ $i -ge $PeriodLengthInMonths ] ; do
539	if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
540	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
541	else
542	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
543	fi
544	(( i=i+1 ))
545	done
546	;;
547	D\|d)
548	PeriodLengthInMonths=0
549	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' )
550	echo
551	IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
552	*)
553	IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
554	IGCM_debug_Verif_Exit ;;
555	esac
556
557	IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
558	}
559
560	#===================================
561	function IGCM_config_DateCoherency
562	{
563	IGCM_debug_PushStack "IGCM_config_DateCoherency"
564
565	echo
566	IGCM_debug_Print 1 "IGCM_config_DateCoherency"
567	echo
568
569	typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
570
571	# check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
572	# DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
573	echo
574
575	case ${config_UserChoices_PeriodLength} in
576	Y\|y)
577	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
578	;;
579	M\|m)
580	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
581	;;
582	D\|d)
583	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
584	;;
585	esac
586	VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
587
588	if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
589	IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
590	IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
591	IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
592	IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
593	IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
594	IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
595
596	IGCM_debug_Exit "STOP here to avoid further issues."
597	fi
598
599	# PeriodDateBegin + PeriodLength = PeriodDateEnd
600	VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
601
602	IGCM_debug_PopStack "IGCM_config_DateCoherency"
603	}
604
605	#===================================
606	function IGCM_config_StateCheck
607	{
608	IGCM_debug_PushStack "IGCM_config_StateCheck"
609
610	#Test state of run in run.card
611	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
612
613	if [ ${run_Configuration_PeriodState} = "Fatal" ] ; then
614	echo
615	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
616	IGCM_debug_Print 1 "Check the overall status of your simulation by visiting this page:"
617	IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
618	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
619	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean lastest pack period."
620	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
621	IGCM_debug_Exit
622	elif [ $( echo ${run_Configuration_PeriodState} \| grep Fatal \| wc -l ) -eq 1 ] ; then
623	echo
624	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
625	IGCM_debug_Print 1 "Compute jobs has been stop because at least the above mentionned post-processing jobs fails."
626	IGCM_debug_Print 1 "Check post-processing jobs carefully by visiting this page:"
627	IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
628	IGCM_debug_Print 1 "Please visit that page to see how to fix issues:"
629	IGCM_debug_Print 1 "https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocGmonitor"
630	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
631	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean latest pack period."
632	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
633	IGCM_debug_Exit
634	fi
635
636	IGCM_debug_PopStack "IGCM_config_StateCheck"
637	}
638
639	#===================================
640	function IGCM_config_Check
641	{
642	IGCM_debug_PushStack "IGCM_config_Check"
643
644	# If one of the following modulo is not zero :
645	# we will issue an error then explain and exit in
646	# AA_job IGCM_debug_Verif_Exit call before binary submission
647
648	echo
649	IGCM_debug_Print 1 "IGCM_config_Check"
650	echo
651
652	typeset i
653
654	# Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
655	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
656	AsynchronousRebuild=true
657	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
658	echo
659	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
660	IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
661	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
662	IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
663	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
664	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
665	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
666	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
667	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
668	else
669	AsynchronousRebuild=false
670	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
671	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
672	echo
673	#modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
674	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
675	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
676	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
677	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
678	fi
679
680	# Check PackFrequency against other key frequencies
681	# Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
682	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
683	Pack=true
684	#
685	IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
686	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
687	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
688	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
689	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
690	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
691	else
692	Pack=false
693	fi
694	# Loop over components
695	for comp in ${config_ListOfComponents[*]} ; do
696
697	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
698	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
699	card=${SUBMIT_DIR}/COMP/${compname}.card
700	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
701	ListFilesName=${compname}_OutputFiles_List
702	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
703	if [ X${FileName0} != X${NULL_STR} ] ; then
704	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
705
706	(( i=0 ))
707	until [ $i -ge $NbFiles ]; do
708	(( i_ = i+3 ))
709	eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
710	eval Testvar=${Testvar_}
711	if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
712	IGCM_debug_Print 1 "Check coherence between PackFrequency and specific pack frequencies"
713	DaysTemp1=$( IGCM_date_DaysInPreviousPeriod ${DateBegin} ${Testvar} )
714	DaysTemp2=$( IGCM_date_DaysInPreviousPeriod ${DateBegin} ${config_Post_PackFrequency} )
715	if [ ${DaysTemp2} -lt ${DaysTemp1} ] ; then
716	IGCM_debug_Print 1 "PackFrequency in config.card is smaller than the specific pack frequency"
717	IGCM_debug_Print 1 "we found in ${compname}.card"
718	IGCM_debug_Exit "ERROR You have to modify the PackFrequency in config.card to be always greater or equal to specific pack frequencies defined in comp.card"
719	IGCM_debug_Verif_Exit
720	fi
721	((i = i+4))
722	else
723	((i = i+3))
724	fi
725	done
726	fi
727	done
728
729	# modulo (TimeSeriesFrequency and all Chunck2D) must be zero
730	NbJob=${#CHUNCK2D_SIZE[@]}
731	i=0
732	until [ $i -ge $NbJob ]; do
733	value=${CHUNCK2D_SIZE[${i}]}
734	IGCM_debug_Print 1 "Check coherence between ${CHUNCK2D_NAME[${i}]} Chunck2D frequency and TimeSeriesFrequency"
735	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
736	case ${value} in
737	Y\|y) ;;
738	*)
739	IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year Y\|y in comp.card"
740	IGCM_debug_Exit "This will stop the job" ;;
741	esac
742	(( i=i+1 ))
743	done
744
745	# modulo (TimeSeriesFrequency and all Chunck3D) must be zero
746	NbJob=${#CHUNCK3D_SIZE[@]}
747	i=0
748	until [ $i -ge $NbJob ]; do
749	value=${CHUNCK3D_SIZE[${i}]}
750	IGCM_debug_Print 1 "Check coherence between ${CHUNCK3D_NAME[${i}]} Chunck3D frequency and TimeSeriesFrequency"
751	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
752	case ${value} in
753	Y\|y) ;;
754	*)
755	IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year Y\|y in comp.card"
756	IGCM_debug_Exit "This will stop the job" ;;
757	esac
758	(( i=i+1 ))
759	done
760
761	# check to be sure there is enough space on temporary filesystems to run
762	echo
763	IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
764	IGCM_sys_check_quota
765
766	# check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
767	echo
768	IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
769	IGCM_sys_check_path
770
771
772	IGCM_debug_PopStack "IGCM_config_Check"
773	}
774
775	#===================================
776	function IGCM_config_ConfigureExecution
777	{
778	IGCM_debug_PushStack " IGCM_config_ConfigureExecution"
779
780	#echo
781	IGCM_debug_Print 1 " IGCM_config_ConfigureExecution"
782	#echo
783
784	typeset ExeNameIn ExeNameFirst CompNameFirst configCardPath comp i
785	typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
786
787	# config.card path
788	configCardPath=$1
789
790	coreNumber=0
791	mpiTasks=0
792	openMPthreads=0
793	NbExec=0
794
795	OK_PARA_MPI=false
796	OK_PARA_OMP=false
797	OK_PARA_NOD=false
798	OK_PARA_MPMD=false
799
800	for comp in ${config_ListOfComponents[*]} ; do
801
802	# Manage component executable
803	IGCM_card_DefineArrayFromOption ${configCardPath} Executable ${comp}
804
805	eval ExeNameIn=\${config_Executable_${comp}[0]}
806
807	# NO order in config.card for parallelized values !
808	# just use suffix : MPI , OMP and NOD (for number of NODes.)
809
810	# NOD is the number of NODes allocated
811	eval ${comp}_PROC_NOD=0
812
813	# MPI is the number of MPI processus per nodes
814	eval ${comp}_PROC_MPI=0
815
816	# OMP is the number of OpenMP threads per MPI processus
817	eval ${comp}_PROC_OMP=0
818
819	# DEP is the factor to increase the number of cores per process
820	eval ${comp}_PROC_DEP=0
821
822	# DEDICATED is the flag to activate the use of dedicated node for XIOS server only
823	eval ${comp}_OK_DEDICATED=0
824
825	# Only if we really have an executable for the component :
826	if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
827
828	IGCM_debug_Print 1 ${comp}
829
830	# Keep the first executable found and the first CompName
831	eval ExeNameFirst=${ExeNameIn}
832	CompNameFirst=${comp}
833
834	# Are we a second executable?
835	(( NbExec = NbExec + 1 ))
836
837	# set 1 MPI task, 1 OpenMP thread and 1 node as default
838	eval ${comp}_PROC_MPI=1
839	eval ${comp}_PROC_OMP=1
840	eval ${comp}_PROC_NOD=1
841	eval ${comp}_PROC_DEP=1
842
843	eval NbElts=\${#config_Executable_${comp}[@]}
844	if [ ${NbElts} -gt 2 ] ; then
845	#
846	# CURRENT METHOD TO SPECIFY MPI AND OMP RESSOURCES
847	#
848	i=2
849	while [ ${i} -lt ${NbElts} ] ; do
850	eval tempvar=\${config_Executable_${comp}[${i}]}
851	IGCM_debug_Print 2 ${tempvar}
852
853	if [ X${tempvar} = X ] ; then
854	IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
855	IGCM_debug_Exit "Check your config.card. Exit now"
856	IGCM_debug_Verif_Exit
857	fi
858
859	case ${tempvar} in
860	[mM][pP][iI])
861	# Read MPI parameter for composante
862	eval ${comp}_PROC_MPI=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/MPI//" )
863	OK_PARA_MPI=true;;
864	[oO][mM][pP])
865	# Read OMP parameter for composante
866	eval ${comp}_PROC_OMP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/OMP//" )
867	;;
868	[nN][oO][dD])
869	# Read NOD (NumBer of Nodes) parameter for composante
870	eval ${comp}_PROC_NOD=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/NOD//" )
871	OK_PARA_NOD=true
872	OK_PARA_MPI=true
873	;;
874	[dD][eE][pP])
875	# Read DEP parameter for composante
876	eval ${comp}_PROC_DEP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/DEP//" )
877	;;
878	[dD][eE][dD][iI][cC][aA][tT][eE][dD])
879	# Read DEDICATED parameter for composante
880	eval ${comp}_OK_DEDICATED=1
881	esac
882	(( i = i + 1 ))
883	done
884	else
885	#
886	# BACKWARD COMPATIBILITY NOT SUPPORTED ANYMORE
887	#
888	IGCM_debug_Exit "You are using a deprecated ressources specification mechanism in your config.card"
889	IGCM_debug_Exit "Please check : https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocEsetup#ThesectionExecutable"
890	IGCM_debug_Exit "Please modify ${configCardPath}"
891	exit
892	fi
893	if [ ${comp}_PROC_NOD -gt 1 ] ; then
894	if ( [ ${comp}_PROC_MPI -gt 1 ] && [ ${comp}_PROC_OMP -gt 1 ] ) ; then
895	IGCM_debug_Print 2 "Error using MPI/OMP/NOD parameters !!!"
896	IGCM_debug_Exit "It is not allowed to use MPI and OMP and NOD parameters for the same component"
897	IGCM_debug_Exit "Check your config.card. Exit now"
898	IGCM_debug_Verif_Exit
899	fi
900	if [ "${comp}" != "IOS" ] ; then
901	IGCM_debug_Print 2 "Error using NOD parameter !!!"
902	IGCM_debug_Exit "NOD parameter is only available for IOS component "
903	IGCM_debug_Exit "Check your config.card. Exit now"
904	IGCM_debug_Verif_Exit
905	fi
906	fi
907	if ( [ ${comp}_OK_DEDICATED -eq 1 ] && [ "${comp}" != "IOS" ] ) ; then
908	IGCM_debug_Print 2 "Error using DEDICATED parameter !!!"
909	IGCM_debug_Exit "DEDICATED parameter is only available for IOS component "
910	IGCM_debug_Exit "Check your config.card. Exit now"
911	IGCM_debug_Verif_Exit
912	fi
913
914	eval tempvarMPI=\${${comp}_PROC_MPI}
915	eval tempvarNOD=\${${comp}_PROC_NOD}
916	eval tempvarOMP=\${${comp}_PROC_OMP}
917	eval tempvarDEP=\${${comp}_PROC_DEP}
918
919	# set OMP mode if more than 1 OMP thread.
920	[ ${tempvarOMP} -ge 2 ] && OK_PARA_OMP=true
921
922	# Number of OMP threads
923	[ ${openMPthreads} -lt ${tempvarOMP} ] && openMPthreads=${tempvarOMP}
924
925	# SUM UP NUMBER OF CORES
926	if [ ${comp}_PROC_NOD -gt 1 ] ; then
927	(( coreNumber = coreNumber + tempvarNOD * NB_CORE_PER_NODE ))
928	else
929	(( coreNumber = coreNumber + tempvarMPI * tempvarNOD * tempvarOMP * tempvarDEP ))
930	fi
931	# SUM UP NUMBER OF MPI TASKS
932	(( mpiTasks = mpiTasks + tempvarMPI * tempvarNOD ))
933	fi
934	done
935
936	# MANDATORY FOR THE OPA9.DRIVER. USED TO EDIT OPA NAMELIST
937	# WE SHOULD PLANIFY NUM_PROC_??? DEPRECATION
938	NUM_PROC_CPL=${CPL_PROC_MPI}
939	NUM_PROC_OCE=${OCE_PROC_MPI}
940	NUM_PROC_ATM=${ATM_PROC_MPI}
941
942	# set MPMD mode if more than 2 executable names.
943	[ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
944
945	# Define the execution type we are running in
946	if ( ${OK_PARA_MPMD} ) ; then
947	if ( ${OK_PARA_MPI} ) ; then
948	# MPMD always implies MPI
949	executionType=1
950	fi
951	if ( ${OK_PARA_OMP} ) ; then
952	# MPMD + MPI/OMP
953	executionType=2
954	fi
955	else
956	if ( ( ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
957	# SPMD + MPI/OMP
958	executionType=3
959	elif ( ( ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
960	# SPMD + MPI only
961	executionType=4
962	elif ( ( ! ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
963	# SPMD + OMP only
964	executionType=5
965	elif ( ( ! ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
966	# SEQUENTIAL THEN
967	executionType=6
968	coreNumber=1
969	fi
970	fi
971
972	IGCM_debug_Print 1 "MPI/OMP treatment coreNumber = ${coreNumber}"
973	IGCM_debug_Print 1 "MPI/OMP treatment mpiTasks = ${mpiTasks}"
974	IGCM_debug_Print 1 "MPI/OMP treatment openMPthreads = ${openMPthreads}"
975	IGCM_debug_Print 1 "MPI/OMP treatment executionType = ${executionType}"
976
977	IGCM_debug_PopStack "IGCM_config_ConfigureExecution"
978	}
979
980	#===================================
981	function IGCM_config_PeriodStart
982	{
983	IGCM_debug_PushStack "IGCM_config_PeriodStart"
984
985	echo
986	IGCM_debug_Print 1 "IGCM_config_PeriodStart"
987	echo
988
989	if ( ${FirstInitialize} ) ; then
990	#================================================#
991	# Initialize date/period information #
992	#================================================#
993
994	IGCM_date_GetYearMonthDay ${DateBegin} year month day
995	IGCM_config_DaysInPeriodLength
996
997	PeriodDateBegin=${DateBegin}
998	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
999	CumulPeriod=1
1000
1001	#=================================================#
1002	# Write updated run.card #
1003	#=================================================#
1004
1005	#Correct run.card Configuration for this period
1006	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1007	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1008	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1009	if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
1010	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
1011	fi
1012
1013	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
1014
1015	else
1016	#================================================#
1017	# The file run.card allready exist #
1018	#================================================#
1019
1020	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1021	IGCM_config_StateCheck
1022	# And EXIT if not OK
1023	IGCM_debug_Verif_Exit
1024
1025	#===================================#
1026	# Read updated run.card #
1027	#===================================#
1028
1029	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
1030	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
1031	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
1032
1033	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
1034	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
1035	CumulPeriod=${run_Configuration_CumulPeriod}
1036
1037	LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
1038
1039	if [ ${Period} = 1 ]; then
1040	# save last Job output and current run.card
1041	typeset Potential
1042	IGCM_sys_Cd ${SUBMIT_DIR}
1043	#
1044	IGCM_debug_Print 2 "Save previous ksh job output"
1045	for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
1046	if [ X${Pack} = Xtrue ] ; then
1047	( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${Potential} ) \|\| IGCM_sys_Cp ${Potential} ${R_BUF_KSH}/${Potential}.$$
1048	else
1049	( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) \|\| IGCM_sys_Put_Out ${Potential} ${R_OUT_KSH}/${Potential}.$$ NOMOVE
1050	IGCM_debug_Print 1 "Save of previous ksh job output is not done for Pack=false"
1051	fi
1052	done
1053
1054	#
1055	IGCM_debug_Print 2 "Save current run.card"
1056	IGCM_card_CheckConflict run.card
1057
1058	if [ X${Pack} = Xtrue ] ; then
1059	IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card
1060	else
1061	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card NOMOVE
1062	IGCM_debug_Print 1 "Save of run.card is not done for Pack=false"
1063	fi
1064	#
1065	IGCM_sys_Cd ${RUN_DIR}
1066	else
1067	unset FileToBeDeleted
1068	fi
1069
1070	# Determine number of day(s) in PeriodLength
1071	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1072	IGCM_config_DaysInPeriodLength
1073
1074	# Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
1075	IGCM_config_DateCoherency
1076	# And EXIT if not OK
1077	IGCM_debug_Verif_Exit
1078
1079	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1080	IGCM_config_StateCheck
1081	# And EXIT if not OK
1082	IGCM_debug_Verif_Exit
1083
1084	# We can say we are "Running" now.
1085	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
1086	fi
1087
1088	# BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
1089	# Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
1090	year_m1=$(( year - 1 ))
1091	year_p1=$(( year + 1 ))
1092	# Compute month_m1 (month minus 1M)
1093	# Compute yyyymm_m1 (yyyymm minus 1M)
1094	month_m1=$(( 10#${month} - 1 ))
1095	if [ ${month_m1} = 0 ]; then
1096	month_m1=12
1097	yyyymm_m1=${year_m1}12
1098	elif [ ${month_m1} -le 9 ]; then
1099	month_m1=0${month_m1}
1100	yyyymm_m1=${year}${month_m1}
1101	else
1102	yyyymm_m1=${year}${month_m1}
1103	fi
1104	# Compute month_p1 (month plus 1M)
1105	# Compute yyyymm_p1 (yyyymm plus 1M)
1106	month_p1=$(( 10#${month} + 1 ))
1107	if [ ${month_p1} = 13 ]; then
1108	month_p1=01
1109	yyyymm_p1=${year_p1}01
1110	elif [ ${month_p1} -le 9 ]; then
1111	month_p1=0${month_p1}
1112	yyyymm_p1=${year}${month_p1}
1113	else
1114	yyyymm_p1=${year}${month_p1}
1115	fi
1116
1117
1118	#===================================================================#
1119	# Calculate CyclicYear to be used for looping over a given forcing #
1120	# period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
1121	#===================================================================#
1122
1123	# To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
1124	# CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
1125	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
1126	CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
1127	#CyclicYear_p1= #NOTDEFINED
1128
1129	# For current year
1130	yeartmp=$year
1131	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
1132	while [ $diffy -lt 0 ] ; do
1133	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
1134	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
1135	done
1136	CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
1137
1138	# For next coming year
1139	CyclicYear_p1=$(( CyclicYear + 1 ))
1140
1141	# For previous year
1142	CyclicYear_m1=$(( CyclicYear - 1 ))
1143
1144	IGCM_debug_Print 1 "CyclicYear = ${CyclicYear}, current year=$year"
1145	IGCM_debug_Print 1 "CyclicYear_p1 = ${CyclicYear_p1}, CyclicYear_m1 = ${CyclicYear_m1} "
1146	else
1147	CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
1148	CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
1149	IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
1150	fi
1151
1152	# END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
1153
1154	#===================================================================#
1155	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
1156	# But available to any son functions #
1157	#===================================================================#
1158
1159	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
1160	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
1161
1162	# Debug Print :
1163	echo
1164	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
1165	IGCM_debug_Print 1 "Year of simulation : ${year}"
1166	IGCM_debug_Print 1 "Month of simulation : ${month}"
1167	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1168	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1169	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1170	IGCM_debug_Print 1 "SimulationLengthInDays : ${SimulationLengthInDays}"
1171	IGCM_debug_Print 1 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
1172
1173	#================================================================#
1174	# Prepare variables available for comp_finalyze #
1175	#================================================================#
1176
1177	# Period for save files
1178	DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
1179
1180	# Prefix for save files of this period
1181	PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
1182
1183	# List of files that will be deleted in RUN_DIR after run
1184	[ -f stack ] && FileToBeDeleted[0]="stack"
1185
1186	# Test if the same run as already been saved :
1187	if [ X${JobType} = XRUN ] ; then
1188	if [ ${DRYRUN} -le 0 ] ; then
1189	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1190	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently re-running an old job."
1191	IGCM_debug_Print 1 "Because of readonly permissions, you can't re-run a job when saved files"
1192	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
1193	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See different clean_*.job in ${libIGCM} directory."
1194	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
1195	IGCM_debug_Verif_Exit
1196	fi
1197	fi
1198	else
1199	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1200	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
1201	fi
1202	fi
1203
1204	#================================================================#
1205	# Prepare variables available for binary execution #
1206	# Call function for creation of run script #
1207	# Only done once per job #
1208	#================================================================#
1209
1210	if [ ${Period} -eq 1 ]; then
1211	# Define the execution context (MPMD, SPMD, MPI/OMP ...)
1212	IGCM_config_ConfigureExecution ${SUBMIT_DIR}/config.card
1213	# Create the execution script for the current context
1214	IGCM_sys_build_execution_scripts
1215	fi
1216
1217	ExecutionFail=false
1218
1219	# Update the rabbitMQ queue
1220	IGCM_debug_BigBro_Update
1221
1222	IGCM_debug_PopStack "IGCM_config_PeriodStart"
1223	}
1224
1225	#===================================
1226	function IGCM_config_SaveSourceModifications
1227	{
1228	IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
1229
1230	typeset ExeOutDateMax listVarEnv
1231	ExeOutDateMax=$1
1232
1233	listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
1234	IGCM_sys_RshMaster "\
1235	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
1236	export ExeOutDateMax=${ExeOutDateMax};\
1237	export R_OUT_EXE=${R_OUT_EXE};\
1238	export PREFIX=${PREFIX};\
1239	export SUBMIT_DIR=${SUBMIT_DIR};\
1240	export listVarEnv=${listVarEnv};\
1241	Script_Output=out_SaveSourceModifications;\
1242	IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
1243
1244	IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
1245	}
1246
1247	#===================================
1248	function IGCM_config_PeriodEnd
1249	{
1250	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
1251
1252	echo
1253	IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
1254	echo
1255
1256	if [ ${DRYRUN} -le 1 ] ; then
1257
1258	IGCM_debug_Print 1 "Check components binary : size and creation date"
1259
1260	typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
1261	typeset comp i
1262	typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
1263
1264	#==================================#
1265	# Get last Exe Size #
1266	#==================================#
1267
1268	(( i=0 ))
1269	if ( ${FirstInitialize} ) ; then
1270	run_Log_LastExeSize=""
1271	for comp in ${config_ListOfComponents[*]} ; do
1272	run_Log_LastExeSize[$i]=0
1273	(( i=i+1 ))
1274	done
1275	else
1276	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
1277	fi
1278	#==================================#
1279	# And Build ExeDate #
1280	#==================================#
1281
1282	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
1283	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
1284	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
1285	ExeDate=""
1286	NextExeSize="( "
1287	(( i=0 ))
1288	UpdateExe=false
1289	(( ExeSecDateMax = 0 ))
1290	for comp in ${config_ListOfComponents[*]} ; do
1291
1292	IGCM_debug_Print 3 ${comp}
1293
1294	eval ExeNameIn=\${config_Executable_${comp}[0]}
1295	eval ExeNameOut=\${config_Executable_${comp}[1]}
1296	# Only if we really have an executable for the component :
1297	if [ X${ExeNameIn} = X\"\" ] ; then
1298	# If there is no exe file for this component
1299	(( ExeSize=0 ))
1300	else
1301	eval ExeNameIn=${ExeNameIn}
1302	LS_bin=${R_EXE}/${ExeNameIn}
1303	IGCM_sys_FileSize ${LS_bin} ExeSize
1304
1305	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
1306	if [ X${ExeDate} = X ] ; then
1307	# First component exe date
1308	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
1309	else
1310	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1311	fi
1312	ExeDate=${ExeDate}_${LS_comp[7]}
1313	fi
1314
1315	if [ ${i} -eq 0 ] ; then
1316	# First component
1317	NextExeSize="( "${ExeSize}
1318	else
1319	NextExeSize=${NextExeSize}", "${ExeSize}
1320	fi
1321	LastCompExeSize=${run_Log_LastExeSize[$i]}
1322	(( i=i+1 ))
1323
1324	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1325	if ( ${FirstInitialize} ) ; then
1326	IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1327	else
1328	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1329	IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
1330	FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1331	fi
1332	IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1333	UpdateExe=true
1334
1335	# SD : switch off for now
1336	#IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1337	#if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1338	# ExeSecDateMax=$ExeSecDate
1339	#fi
1340	fi
1341	done
1342
1343	# SD : switch off for now
1344	#if ( ${UpdateExe} ) ; then
1345	# echo "Launch SaveSourceModifications."
1346	# IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1347	#fi
1348
1349	NextExeSize=${NextExeSize}" )"
1350	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1351
1352	if [ ${DRYRUN} -le 1 ] ; then
1353	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1354	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1355	RET=$?
1356	if [ $RET -eq 0 ] ; then
1357	# ExeCpuLog variable contents 5 fields
1358	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
1359	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
1360	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1361	fi
1362	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
1363	fi
1364
1365	fi
1366
1367	#==================================#
1368	# Save Job output #
1369	#==================================#
1370	if [ X${Pack} = Xtrue ] ; then
1371	IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1372	else
1373	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1374	fi
1375	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1376
1377	# All was right ? no ? then we stop.
1378	IGCM_debug_Verif_Exit
1379
1380	# If all was OK, we can delete all files not necessary for next Job
1381	echo
1382	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1383
1384	if [ ${DRYRUN} -le 2 ] ; then
1385	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1386	else
1387	echo ${FileToBeDeleted[@]}
1388	fi
1389
1390	# Send some accounting element to the user if CumulPeriod=3
1391	if [ ${CumulPeriod} -eq 3 ] ; then
1392	echo
1393	IGCM_debug_Print 1 "Send email containing some accounting information : "
1394
1395	RealCpuTime=$( echo ${ExeCpuLog} \| gawk '{print $3}' )
1396
1397	consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${coreNumber}/3600" \| bc )
1398
1399	consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" \| bc )
1400
1401	recommendedPeriodNb=$( echo "scale=6;${jobWarningDelay}/3600/${consumeHoursPerPeriod}*${coreNumber}" \| bc )
1402
1403	IGCM_sys_SendMail Accounting
1404	fi
1405
1406	#=================================================#
1407	# Modification of libIGCM behaviour #
1408	#=================================================#
1409
1410	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
1411	# and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1412	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1413	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1414	echo
1415	echo "########################################################################"
1416	echo "!!! Modification of libIGCM behaviour !!!"
1417	echo
1418
1419	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
1420	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1421	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1422
1423	# Special treatments for libIGCM internals
1424	for option in ${libIGCM_UserChanges[*]} ; do
1425	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1426
1427	echo "We will change : ${option}."
1428	eval echo "Previous value : " \${${option}}
1429	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1430
1431	eval ${option}=\${libIGCM_UserChanges_${option}}
1432
1433	case ${option} in
1434	config_UserChoices_DateEnd)
1435	IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1436	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1437
1438	# Period Length In Days between DateBegin and DateEnd
1439	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
1440	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1441	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1442	IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
1443	IGCM_debug_Verif_Exit
1444	fi
1445	;;
1446	config_UserChoices_PeriodLength)
1447	IGCM_debug_Print 1 "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1448	;;
1449	PeriodNb)
1450	IGCM_debug_Print 1 "Loop in main Job with ${PeriodNb} period(s)"
1451	;;
1452	config_Post_RebuildFrequency)
1453	IGCM_debug_Print 1 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1454	IGCM_post_Configure
1455	;;
1456	config_Post_TimeSeriesFrequency)
1457	IGCM_debug_Print 1 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1458	IGCM_post_Configure
1459	;;
1460	config_Post_SeasonalFrequency)
1461	IGCM_debug_Print 1 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1462	IGCM_post_Configure
1463	;;
1464	esac
1465	done
1466
1467	echo
1468	echo "########################################################################"
1469	echo
1470	fi
1471
1472	#=================================================#
1473	# Determine next computed period #
1474	#=================================================#
1475
1476	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1477	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1478	year_m1=$(( year - 1 ))
1479	year_p1=$(( year + 1 ))
1480	IGCM_config_DaysInPeriodLength
1481	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1482
1483	# Debug Print :
1484	echo
1485	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1486	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1487	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1488	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1489
1490	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1491	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1492
1493	(( CumulPeriod = CumulPeriod + 1 ))
1494
1495	# Debug Print :
1496	echo
1497	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1498	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
1499	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
1500
1501	#=================================================#
1502	# Write updated run.card #
1503	#=================================================#
1504
1505	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1506	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1507	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1508
1509	if ( ${FirstInitialize} ) ; then
1510	# It's no more the first time
1511	FirstInitialize=false
1512	fi
1513
1514	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1515	}
1516
1517	#===================================
1518	function IGCM_config_Finalize
1519	{
1520	IGCM_debug_PushStack "IGCM_config_Finalize"
1521
1522	echo
1523	IGCM_debug_Print 1 "IGCM_config_Finalize"
1524	echo
1525
1526	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1527	IGCM_config_StateCheck
1528
1529	# And EXIT if not OK
1530	IGCM_debug_Verif_Exit
1531
1532	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1533	#==========================#
1534	# End of entire simulation #
1535	#==========================#
1536	simulationIsOver=true
1537
1538	# Mail notification
1539	IGCM_sys_SendMail
1540	#
1541	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1542	#
1543	IGCM_debug_Print 1 "Normal End of computation."
1544
1545	else
1546	#=================#
1547	# Submit next job #
1548	#=================#
1549	simulationIsOver=false
1550
1551	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1552
1553	# Name of next Ksh Script output :
1554	Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
1555
1556	IGCM_debug_Print 1 "Submit next job"
1557	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1558	IGCM_sys_Cd ${SUBMIT_DIR}
1559	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1560	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null \| wc -l )
1561	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
1562	# Submit next job and come back
1563	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1564	IGCM_sys_Cd -
1565	fi
1566
1567	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1568	# Only for production run (No clean up in DEV or DEB mode)
1569	# and command sent from .. directory.
1570	IGCM_sys_Cd ..
1571	[ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
1572
1573	# Inform the rabbitMQ queue
1574	IGCM_debug_BigBro_Finalize
1575
1576	IGCM_debug_PopStack "IGCM_config_Finalize"
1577	}
1578
1579	#===================================

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 1603

Download in other formats: