Context Navigation

← Previous Revision
Next Revision →
Blame
Revision Log

libIGCM_config.ksh

Last change on this file was 1638, checked in by aclsce, 5 weeks ago
Added functionality to run concurrent execution with 2 or 3 model instances.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 64.8 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_CommonConfiguration
16	{
17	IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19	# Debug Print :
20	[ ${Verbosity} -gt 0 ] && echo
21	IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23	# config.card path
24	configCardPath=$1
25
26	#==================================
27	typeset option auxprint
28
29	#==================================
30	# Read UserChoices section:
31	[ ${Verbosity} -gt 0 ] && echo
32	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
33
34	IGCM_card_DefineArrayFromSection ${configCardPath} UserChoices
35	for option in ${config_UserChoices[*]} ; do
36	IGCM_card_DefineVariableFromOption ${configCardPath} UserChoices ${option}
37	eval auxprint=\${config_UserChoices_${option}}
38	IGCM_debug_Print 3 "${option} : ${auxprint}"
39	done
40
41	# Set variables with shorter names on some variables which
42	# are used in the executable name in config.card
43	ResolAtm=${config_UserChoices_ResolAtm}
44	ResolOce=${config_UserChoices_ResolOce}
45	ConfChem=${config_UserChoices_ConfChem}
46	OptMode=${config_UserChoices_OptMode}
47	ConfigType=${config_UserChoices_ConfigType}
48
49	#==================================
50	# Read Ensemble section:
51	[ ${Verbosity} -gt 0 ] && echo
52	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
53
54	IGCM_card_DefineArrayFromSection ${configCardPath} Ensemble
55	for option in ${config_Ensemble[*]} ; do
56	IGCM_card_DefineVariableFromOption ${configCardPath} Ensemble ${option}
57	eval auxprint=\${config_Ensemble_${option}}
58	IGCM_debug_Print 3 "${option} : ${auxprint}"
59	done
60
61	#==================================
62	# Read Post section:
63	[ ${Verbosity} -gt 0 ] && echo
64	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
65
66	IGCM_card_DefineArrayFromSection ${configCardPath} Post
67	for option in ${config_Post[*]} ; do
68	IGCM_card_DefineVariableFromOption ${configCardPath} Post ${option}
69	eval auxprint=\${config_Post_${option}}
70	IGCM_debug_Print 3 "${option} : ${auxprint}"
71	done
72	[ ${Verbosity} -gt 0 ] && echo
73
74	#==================================
75	# Define default value to keep compatibility with previous card: means before changes due to TGCC
76	# Apply some overrules to ensure proper usage of computing centres resources
77	#
78
79	# if RebuildFrequency is not define in config.card - we initilize it to NONE
80	if [ X${config_Post_RebuildFrequency} = X ] ; then
81	config_Post_RebuildFrequency=NONE
82	fi
83
84	if [ X${PackDefault} = Xtrue ] ; then
85	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
86	# TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
87	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
88	config_Post_PackFrequency=NONE
89	else
90	# Default to RebuildFrequency if nothing has been set up related to PackFrequency
91	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
92	fi
93	else
94	# If we DO NOT apply pack in this computing center
95	config_Post_PackFrequency=NONE
96	fi
97
98	#====================================================
99	# Define ARCHIVE : Dedicated to large files
100	# Define STORAGE : Dedicated to small/medium files
101	# Define R_OUT : Output tree located on ARCHIVE
102	# Define R_BUF : Output tree located on STORAGE (files waiting treatment, or file lcoation when SpaceName=!PROD)
103	# Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
104	# Define R_TMP : A temporary space used by IGCM_debug_send_AMQP_msg__MAILTUNNEL. Must be persistent in between jobs
105	IGCM_sys_defineArchives
106
107	#====================================================
108	# R_SAVE : Job output directory
109	# R_BUFR : Job output buffered directory
110	# R_CMIP : Job output directory for CMIP6 standard files
111
112	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
113	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
114
115	if ( [ X${config_Post_StorageCMIP} = XSCRATCH ] \|\| [ X${config_Post_StorageCMIP} = Xscratch ] ) ; then
116	IGCM_debug_Print 1 "LIBIGCM CONFIG:: SCRATCH R_BUF=${R_BUF}"
117	StoreCMIP=${R_BUF}
118	elif ( [ X${config_Post_StorageCMIP} = XSTORE ] \|\| [ X${config_Post_StorageCMIP} = Xstore ] ) ; then
119	IGCM_debug_Print 1 "LIBIGCM CONFIG:: SCRATCH R_OUT=${R_OUT}"
120	StoreCMIP=${R_OUT}
121	else
122	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ELSE CMIP_BUF=${CMIP6_BUF}"
123	StoreCMIP=${CMIP6_BUF}
124	fi
125	# Ensemble cases (default all members outputs in same directory)
126	if ( [ ! X${config_Ensemble_EnsembleName} = X ] ) ; then
127	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
128	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
129	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
130	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
131	R_CMIP=${StoreCMIP}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
132	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE R_CMIP=${R_CMIP}"
133
134	# PERTURB Ensemble Outputs
135	if ( [ X${config_Ensemble_EnsembleType} = XEns_PERTURB ] ) ; then
136	if ( [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
137	R_SAVE=${R_SAVE}/${config_Ensemble_EnsembleDate}/${FreeName}
138	R_FIGR=${R_FIGR}/${config_Ensemble_EnsembleDate}/${FreeName}
139	R_BUFR=${R_BUFR}/${config_Ensemble_EnsembleDate}/${FreeName}
140	R_DODS=${R_DODS}/${config_Ensemble_EnsembleDate}/${FreeName}
141	R_CMIP=${R_CMIP}/${config_Ensemble_EnsembleDate}/${FreeName}
142	else
143	R_SAVE=${R_SAVE}/${FreeName}
144	R_FIGR=${R_FIGR}/${FreeName}
145	R_BUFR=${R_BUFR}/${FreeName}
146	R_DODS=${R_DODS}/${FreeName}
147	R_CMIP=${R_CMIP}/${FreeName}
148	fi
149	# DATE Ensemble Outputs
150	elif ( [ X${config_Ensemble_EnsembleType} = XEns_DATE ] ) ; then
151	R_FIGR=${R_FIGR}/${FreeName}
152	R_BUFR=${R_BUFR}/${FreeName}
153	R_DODS=${R_DODS}/${FreeName}
154	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE DATE R_CMIP=${R_CMIP}"
155	if ( [ X${config_Ensemble_EnsembleMergeSave} = X ] \|\| [ X${config_Ensemble_EnsembleMergeSave} = Xn ] \|\|
156	[ X${config_Ensemble_EnsembleMergeSave} = Xno ] ) ; then
157	# One SAVE directory per member, else common output dir for all members
158	R_SAVE=${R_SAVE}/${FreeName}
159	R_CMIP=${R_CMIP}/${FreeName}
160	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE DATE MERGE R_CMIP=${R_CMIP}"
161	fi
162	fi
163	else
164	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
165	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
166	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
167	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
168	R_CMIP=${StoreCMIP}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
169	fi
170	else
171	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
172	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
173	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
174	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
175	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
176	else
177	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
178	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
179	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
180	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
181	fi
182	fi
183
184	IGCM_debug_Print 1 "LIBIGCM CONFIG:: R_CMIP = ${R_CMIP}"
185
186	#====================================================
187	# Define R_OUT_KSH : Storage place for job output
188	# Define R_OUT_EXE : Storage place for binary used during simulation
189	R_OUT_KSH=${R_SAVE}/Out
190	R_OUT_EXE=${R_SAVE}/Exe
191
192	#====================================================
193	# Define R_BUF_KSH : Buffer place for job output
194	# Define R_BUF_EXE : Buffer place for binary used during simulation
195	R_BUF_KSH=${R_BUFR}/Out
196	R_BUF_EXE=${R_BUFR}/Exe
197
198	#====================================================
199	# Define REBUILD_DIR : where we store files needing rebuild process
200	REBUILD_DIR=${R_BUFR}/REBUILD
201	if [ ! X${TaskType} = Xchecking ] ; then
202	IGCM_sys_MkdirWork ${REBUILD_DIR}
203	fi
204
205	#====================================================
206	# DodsCopy : apply default value if not defined
207	if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] \|\| [ X${config_Post_DodsCopy} = X ] ) ; then
208	config_Post_DodsCopy=TRUE
209	fi
210
211	#====================================================
212	# AtlasIPSL : apply default value if not defined
213	if ( [ X${config_Post_AtlasIPSL} = X${NULL_STR} ] \|\| [ X${config_Post_AtlasIPSL} = X ] ) ; then
214	config_Post_AtlasIPSL=FALSE
215	fi
216
217	#====================================================
218	# ParserXIOS : apply default value if not defined
219	if ( [ X${config_Post_ParserXIOS} = X${NULL_STR} ] \|\| [ X${config_Post_ParserXIOS} = X ] ) ; then
220	config_Post_ParserXIOS=FALSE
221	fi
222
223	#====================================================
224	# MetricsPCMDI : apply default value if not defined
225	if ( [ X${config_Post_MetricsPCMDI} = X${NULL_STR} ] \|\| [ X${config_Post_MetricsPCMDI} = X ] ) ; then
226	config_Post_MetricsPCMDI=FALSE
227	fi
228	#====================================================
229	# Cesmep : apply default value FALSE if not defined, and if TRUE
230	# or *Y set it based on an adhoc algorithm
231	if ( [ X${config_Post_Cesmep} = X${NULL_STR} ] \|\| [ X${config_Post_Cesmep} = XNONE ] \|\| \
232	[ X${config_Post_Cesmep} = X ] \|\| [ X${config_Post_Cesmep} = XFALSE ] ) ; then
233	config_Post_Cesmep=FALSE
234	elif [ X${config_Post_Cesmep} = XTRUE ] ; then
235	if ([ ! X${config_Post_SeasonalFrequency} = XNONE ] && [ ! X${config_Post_SeasonalFrequency} = X ] ) ; then
236	config_Post_Cesmep="SE"
237	elif ([ ! X${config_Post_TimeSeriesFrequency} = XNONE ] && [ ! X${config_Post_TimeSeriesFrequency} = X ] ) ; then
238	config_Post_Cesmep="TS"
239	elif ([ ! X${config_Post_PackFrequency} = XNONE ] && [ ! X${config_Post_PackFrequency} = X ] ) ; then
240	config_Post_Cesmep="Pack"
241	else
242	# This covers case SpaceName=TEST with no SE nor TS (nor Pack)
243	config_Post_Cesmep=AtEnd
244	fi
245	fi
246	#
247	if [ X${config_Post_Cesmep} != XFALSE ] ; then
248	if ( [ X${config_Post_CesmepPeriod} != X${NULL_STR} ] && [ X${config_Post_CesmepPeriod} != X ] ) ; then
249	CesmepPeriod=${config_Post_CesmepPeriod%Y}
250	fi
251	if [ X${config_Post_Cesmep} = XSE ] ; then
252	if ([ X${config_Post_SeasonalFrequency} = XNONE ] \|\| [ X${config_Post_SeasonalFrequency} = X ] ) ; then
253	IGCM_debug_Exit "Cannot use SE for Cesmep : SE post-processing is not active"
254	else
255	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_SeasonalFrequency%Y}
256	fi
257	elif [ X${config_Post_Cesmep} = XTS ] ; then
258	if ([ X${config_Post_TimeSeriesFrequency} = XNONE ] \|\| [ X${config_Post_TimeSeriesFrequency} = X ] ) ; then
259	IGCM_debug_Exit "Cannot use TS for Cesmep : TS post-processing is not active"
260	else
261	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_TimeSeriesFrequency%Y}
262	fi
263	elif [ X${config_Post_Cesmep} = XPack ] ; then
264	if ([ X${config_Post_PackFrequency} = XNONE ] \|\| [ X${config_Post_PackFrequency} = X ] ) ; then
265	IGCM_debug_Exit "Cannot use Packed output for Cesmep : Pack post-processing is not active"
266	else
267	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_PackFrequency%Y}
268	fi
269	elif [ X${config_Post_Cesmep} != XAtEnd ] ; then
270	IGCM_debug_Exit "config.card's Post section's value for Cesmep (${config_Post_Cesmep}) is invalid; use SE, TS, Pack, AtEnd or TRUE"
271	else
272	[ -z $CesmepPeriod ] && CesmepPeriod=0
273	fi
274
275	if ( [ X${config_Post_CesmepComparison} = XNONE ] \|\| [ X${config_Post_CesmepComparison} = X ] ) ; then
276	config_Post_CesmepComparison=run_comparison
277	fi
278
279	if ( [ X${config_Post_CesmepCode} = XNONE ] \|\| [ X${config_Post_CesmepCode} = X ] ) ; then
280	if ( [ X"${SYSTEM}" == "Xirene" ] \|\| [ X"${SYSTEM}" == "Xirene-amd" ] ); then
281	config_Post_CesmepCode=/ccc/cont003/home/igcmg/igcmg/Tools/cesmep
282	elif ( [ X"${SYSTEM}" == "Xjeanzay" ] ); then
283	config_Post_CesmepCode=/gpfswork/rech/psl/commun/Tools/cesmep
284	elif ( [ X"${SYSTEM}" == "Xmesoipsl" ] ); then
285	config_Post_CesmepCode=/net/nfs/tools/Users/SU/jservon/cesmep_installs/cesmep_for_libIGCM
286	elif [ ${config_Post_Cesmep} != FALSE ] ; then
287	IGCM_debug_Exit "Cannot locate a default C-ESM-EP code on $SYSTEM, please set CesmepCode in section Post"
288	fi
289	fi
290	if ( [ X${config_Post_CesmepSlices} = XNONE ] \|\| [ X${config_Post_CesmepSlices} = X ] ) ; then
291	config_Post_CesmepSlices=8
292	fi
293	if ( [ X${config_Post_CesmepSlicesDuration} != X${NULL_STR} ] && [ X${config_Post_CesmepSlicesDuration} != X ] &&\
294	[ X${config_Post_CesmepSlicesDuration} != XNONE ] ) ; then
295	CesmepSlicesDuration=${config_Post_CesmepSlicesDuration%Y}
296	else
297	CesmepSlicesDuration=$CesmepPeriod
298	fi
299	if [ $CesmepSlicesDuration -gt $CesmepPeriod ]; then
300	IGCM_debug_Exit "C-ESM-EP : you cannot set a CesmepSlicesDuration ($CesmepSlicesDuration) longer than CesmepPeriod ($CesmepPeriod)"
301	fi
302	if [ X${config_Post_CesmepReferences} = X${NULL_STR} ] \|\| [ X${config_Post_CesmepReferences} = XNONE ] \|\| \
303	[ X${config_Post_CesmepReferences} = X ] ; then
304	config_Post_CesmepReferences=NONE
305	fi
306	fi
307	IGCM_debug_Print 3 "config_Post_Cesmep=${config_Post_Cesmep}, CesmepPeriod=$CesmepPeriod, config_Post_CesmepSlices=$config_Post_CesmepSlices "
308	#====================================================
309	# IgnoreNonMonotonic : apply default value if not defined
310	if ( [ X${config_Post_IgnoreNonMonotonic} = X${NULL_STR} ] \|\| [ X${config_Post_IgnoreNonMonotonic} = X ] ) ; then
311	config_Post_IgnoreNonMonotonic=FALSE
312	fi
313
314	#====================================================
315	# LightRestartPack : apply default value if not defined
316	if ( [ X${config_Post_LightRestartPack} = X${NULL_STR} ] \|\| [ X${config_Post_LightRestartPack} = X ] ) ; then
317	config_Post_LightRestartPack=FALSE
318	fi
319
320	#====================================================
321	# dr2xmlIPSL : apply default value if not defined
322	if ( [ X${config_Post_dr2xmlIPSL} = X${NULL_STR} ] \|\| [ X${config_Post_dr2xmlIPSL} = X ] ) ; then
323	config_Post_dr2xmlIPSL=FALSE
324	fi
325
326
327	#====================================================
328	# Define StackFileLocation : directory where we store stack files
329	# Define StackFileName : stack file containing call tree and instrumentation
330	# Stack file containing call tree will be stored there.
331	if ( $DEBUG_debug ) ; then
332	StackFileLocation=${StackFileLocation:=${R_BUF_KSH}}
333	[ ! -d ${StackFileLocation} ] && mkdir -p ${StackFileLocation}
334	if [ X${TaskType} = Xcomputing ]; then
335	StackFileName=computing.stack.$$
336	elif [ X${TaskType} = Xpost-processing ]; then
337	StackFileName=${Script_Post_Output}.stack.$$
338	elif [ X${TaskType} = Xchecking ]; then
339	StackFileName=checking.stack.$$
340	else
341	IGCM_debug_Exit "IGCM_config_CommonConfiguration unknown TaskType : ${TaskType}"
342	IGCM_debug_Verif_Exit
343	fi
344
345	# This boolean will trigger the filling of the stack
346	# Only now we know where things should be ...
347	# We don't fill the stack when we perform checking task
348	if [ ! X${TaskType} = Xchecking ] ; then
349	ActivateStackFilling=true
350	fi
351	fi
352
353	IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
354	}
355
356	#===================================
357	function IGCM_config_Initialize
358	{
359	IGCM_debug_PushStack "IGCM_config_Initialize"
360
361	# Debug Print :
362	echo
363	IGCM_debug_Print 1 "IGCM_config_Initialize"
364
365	# Test modipsl tree existence.
366	IGCM_sys_TestDir ${MODIPSL}
367	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
368	IGCM_sys_TestDir ${libIGCM}
369	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
370	IGCM_sys_TestDir ${R_EXE}
371	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
372	IGCM_sys_TestDir ${SUBMIT_DIR}
373	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
374
375	if ( $DEBUG_debug ) ; then
376	echo "Keep trace of inital SUBMIT_DIR : "
377	ls -lta ${SUBMIT_DIR}
378	fi
379
380	#==================================
381	# Read ListOfComponents section:
382	echo
383	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
384
385	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
386	for comp in ${config_ListOfComponents[*]} ; do
387	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
388	done
389	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
390
391	#==================================
392	# Read Executable section:
393	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
394
395	#==================================
396	# Read Restarts section:
397	# Restarts : Gerneral rule or local for each component.
398	echo
399	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
400
401	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
402	for option in ${config_Restarts[*]} ; do
403	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
404	eval auxprint=\${config_Restarts_${option}}
405	IGCM_debug_Print 3 "${option} : ${auxprint}"
406	done
407
408	#==================================
409	# Define Job Outputs Name
410	echo
411	IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
412	Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
413	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
414	Exe_Output=out_execution
415	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
416
417	#===================================================================#
418	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
419	# But available to any son functions #
420	#===================================================================#
421
422	# Convert yyyy-mm-dd date to gregorian yyyymmdd
423	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
424	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
425	EndYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateEnd ) / 1000 ))
426	EndYear_p1=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateEnd ) / 1000 + 1 ))
427
428
429	# Period Length In Days between DateBegin and DateEnd
430	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
431	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
432	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
433	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
434	IGCM_debug_Verif_Exit
435	fi
436
437	# Day and Year of Initial State (Given in julian format)
438	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
439	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
440	InitYear_m1=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 - 1 ))
441
442	#================================================================#
443	# Test and Prepare directories #
444	#================================================================#
445
446	# ==> 4 kinds of input files :
447	# 1) R_INIT : Initial State Files (Etat0, carteveg)
448	# 2) R_BC : Boundary Conditions (Forcages, lai)
449	# 3) Parameters files (allready define through ${SUBMIT_DIR})
450	# 4) Restarts files (allready define in IGCM_config_Initialize)
451
452	# Here we offer the possibility to redefine R_INIT, R_BC
453	# and NbPeriodsPerJob through config.card
454	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
455	echo
456	IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and NbPeriodsPerJob"
457	IGCM_debug_Print 3 "R_IN=${R_IN}"
458	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
459	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
460	IGCM_debug_Print 3 "R_BC=${R_BC}"
461	NbPeriodsPerJob=${config_UserChoices_NbPeriodsPerJob:=${NbPeriodsPerJob}}
462	IGCM_debug_Print 3 "Loop in main Job with ${NbPeriodsPerJob} period(s)"
463
464	# SD ADA SPECIFIC #
465	# TO FIX #
466	#IGCM_sys_TestDirArchive ${R_IN}
467	#[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
468
469	if ( ${FirstInitialize} ) ; then
470	IGCM_sys_MkdirArchive ${R_SAVE}
471	[ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
472	else
473	IGCM_sys_TestDirArchive ${R_SAVE}
474	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
475
476	if [ ! ${config_Post_PackFrequency} = NONE ] ; then
477	IGCM_sys_TestDir ${R_BUFR}
478	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
479	fi
480
481	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
482	IGCM_config_StateCheck
483
484	# And EXIT if not OK
485	IGCM_debug_Verif_Exit
486	fi
487
488	#====================================================
489	# Experience type : DEB(ug), DEV(elopment), RUN
490	if [ X${JobType} != XRUN ] ; then
491	echo
492	echo "===================================================="
493	echo "libIGCM JOB is NOT in RUN type mode."
494	echo "!! OUTPUT files will NOT be PROTECTED !!"
495	echo "Be carefull : you can ERASE the result of this job !"
496
497	case ${JobType} in
498	DEB)
499	echo "DEBUG mode : activation of 'set -vx' mode."
500	echo "DEBUG mode : no protection for output files."
501	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
502	;;
503	DEV)
504	echo "DEVelopment mode : no protection for output files."
505	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
506	;;
507	esac
508
509	if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
510	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
511	echo "------------"
512	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
513	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
514	echo "------------"
515	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
516	fi
517	fi
518	echo "===================================================="
519	echo
520	fi
521
522	IGCM_debug_PopStack "IGCM_config_Initialize"
523	}
524
525	#===================================
526	function IGCM_config_DaysInPeriodLength
527	{
528	IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
529
530	typeset i
531
532	# Determine number of day(s) in PeriodLength :
533	case ${config_UserChoices_PeriodLength} in
534	Y\|y)
535	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
536	echo
537	IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
538	PeriodLengthInDays=0
539	i=0
540	until [ $i -ge $PeriodLengthInYears ] ; do
541	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
542	(( i=i+1 ))
543	done
544	;;
545	M\|m)
546	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
547	echo
548	IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
549	PeriodLengthInDays=0
550	i=0
551	until [ $i -ge $PeriodLengthInMonths ] ; do
552	if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
553	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
554	else
555	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
556	fi
557	(( i=i+1 ))
558	done
559	;;
560	D\|d)
561	PeriodLengthInMonths=0
562	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' )
563	echo
564	IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
565	*)
566	IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
567	IGCM_debug_Verif_Exit ;;
568	esac
569
570	IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
571	}
572
573	#===================================
574	function IGCM_config_DateCoherency
575	{
576	IGCM_debug_PushStack "IGCM_config_DateCoherency"
577
578	echo
579	IGCM_debug_Print 1 "IGCM_config_DateCoherency"
580	echo
581
582	typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
583
584	# check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
585	# DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
586	echo
587
588	case ${config_UserChoices_PeriodLength} in
589	Y\|y)
590	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
591	;;
592	M\|m)
593	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
594	;;
595	D\|d)
596	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
597	;;
598	esac
599	VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
600
601	if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
602	IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
603	IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
604	IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
605	IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
606	IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
607	IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
608
609	IGCM_debug_Exit "STOP here to avoid further issues."
610	fi
611
612	# PeriodDateBegin + PeriodLength = PeriodDateEnd
613	VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
614
615	IGCM_debug_PopStack "IGCM_config_DateCoherency"
616	}
617
618	#===================================
619	function IGCM_config_StateCheck
620	{
621	IGCM_debug_PushStack "IGCM_config_StateCheck"
622
623	#Test state of run in run.card
624	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
625
626	if [ ${run_Configuration_PeriodState} = "Fatal" ] ; then
627	echo
628	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
629	IGCM_debug_Print 1 "Check the overall status of your simulation by visiting this page:"
630	IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
631	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
632	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean lastest pack period."
633	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
634	IGCM_debug_Exit
635	elif [ $( echo ${run_Configuration_PeriodState} \| grep Fatal \| wc -l ) -eq 1 ] ; then
636	echo
637	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
638	IGCM_debug_Print 1 "Compute jobs has been stop because at least the above mentionned post-processing jobs fails."
639	IGCM_debug_Print 1 "Check post-processing jobs carefully by visiting this page:"
640	IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
641	IGCM_debug_Print 1 "Please visit that page to see how to fix issues:"
642	IGCM_debug_Print 1 "https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocGmonitor"
643	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
644	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean latest pack period."
645	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
646	IGCM_debug_Exit
647	fi
648
649	IGCM_debug_PopStack "IGCM_config_StateCheck"
650	}
651
652	#===================================
653	function IGCM_config_Check
654	{
655	IGCM_debug_PushStack "IGCM_config_Check"
656
657	# If one of the following modulo is not zero :
658	# we will issue an error then explain and exit in
659	# AA_job IGCM_debug_Verif_Exit call before binary submission
660
661	echo
662	IGCM_debug_Print 1 "IGCM_config_Check"
663	echo
664
665	typeset i
666
667	# Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
668	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
669	AsynchronousRebuild=true
670	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
671	echo
672	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
673	IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
674	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
675	IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
676	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
677	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
678	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
679	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
680	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
681	else
682	AsynchronousRebuild=false
683	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
684	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
685	echo
686	#modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
687	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
688	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
689	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
690	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
691	fi
692
693	# Check PackFrequency against other key frequencies
694	# Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
695	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
696	Pack=true
697	#
698	IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
699	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
700	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
701	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
702	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
703	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
704	else
705	Pack=false
706	fi
707	# Loop over components
708	for comp in ${config_ListOfComponents[*]} ; do
709
710	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
711	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
712	card=${SUBMIT_DIR}/COMP/${compname}.card
713	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
714	ListFilesName=${compname}_OutputFiles_List
715	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
716	if [ X${FileName0} != X${NULL_STR} ] ; then
717	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
718
719	(( i=0 ))
720	until [ $i -ge $NbFiles ]; do
721	(( i_ = i+3 ))
722	eval Testvar_=\${${ListFilesName}[$i_]} > /dev/null 2>&1
723	eval Testvar=${Testvar_}
724	if [[ ${Testvar} =~ [0-9][DMY]$ ]] ; then
725	IGCM_debug_Print 1 "Check coherence between PackFrequency and specific pack frequencies"
726	DaysTemp1=$( IGCM_date_DaysInPreviousPeriod ${DateBegin} ${Testvar} )
727	DaysTemp2=$( IGCM_date_DaysInPreviousPeriod ${DateBegin} ${config_Post_PackFrequency} )
728	if [ ${DaysTemp2} -lt ${DaysTemp1} ] ; then
729	IGCM_debug_Print 1 "PackFrequency in config.card is smaller than the specific pack frequency"
730	IGCM_debug_Print 1 "we found in ${compname}.card"
731	IGCM_debug_Exit "ERROR You have to modify the PackFrequency in config.card to be always greater or equal to specific pack frequencies defined in comp.card"
732	IGCM_debug_Verif_Exit
733	fi
734	((i = i+4))
735	else
736	((i = i+3))
737	fi
738	done
739	fi
740	done
741
742	# modulo (TimeSeriesFrequency and all Chunck2D) must be zero
743	NbJob=${#CHUNCK2D_SIZE[@]}
744	i=0
745	until [ $i -ge $NbJob ]; do
746	value=${CHUNCK2D_SIZE[${i}]}
747	IGCM_debug_Print 1 "Check coherence between ${CHUNCK2D_NAME[${i}]} Chunck2D frequency and TimeSeriesFrequency"
748	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
749	case ${value} in
750	Y\|y) ;;
751	*)
752	IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year Y\|y in comp.card"
753	IGCM_debug_Exit "This will stop the job" ;;
754	esac
755	(( i=i+1 ))
756	done
757
758	# modulo (TimeSeriesFrequency and all Chunck3D) must be zero
759	NbJob=${#CHUNCK3D_SIZE[@]}
760	i=0
761	until [ $i -ge $NbJob ]; do
762	value=${CHUNCK3D_SIZE[${i}]}
763	IGCM_debug_Print 1 "Check coherence between ${CHUNCK3D_NAME[${i}]} Chunck3D frequency and TimeSeriesFrequency"
764	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
765	case ${value} in
766	Y\|y) ;;
767	*)
768	IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year Y\|y in comp.card"
769	IGCM_debug_Exit "This will stop the job" ;;
770	esac
771	(( i=i+1 ))
772	done
773
774	# check to be sure there is enough space on temporary filesystems to run
775	echo
776	IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
777	IGCM_sys_check_quota
778
779	# check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
780	echo
781	IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
782	IGCM_sys_check_path
783
784
785	IGCM_debug_PopStack "IGCM_config_Check"
786	}
787
788	#===================================
789	function IGCM_config_ConfigureExecution
790	{
791	IGCM_debug_PushStack " IGCM_config_ConfigureExecution"
792
793	#echo
794	IGCM_debug_Print 1 " IGCM_config_ConfigureExecution"
795	#echo
796
797	typeset ExeNameIn ExeNameFirst CompNameFirst configCardPath comp i
798	typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
799
800	# config.card path
801	configCardPath=$1
802
803	coreNumber=0
804	mpiTasks=0
805	openMPthreads=0
806	NbExec=0
807	first_slurm_comp=0
808	second_slurm_comp=0
809
810	OK_PARA_MPI=false
811	OK_PARA_OMP=false
812	OK_PARA_NOD=false
813	OK_PARA_MPMD=false
814
815	for comp in ${config_ListOfComponents[*]} ; do
816
817	# Add a node in case of many execution in concurrent mode
818	number_rundir=$(echo ${comp} \| sed 's/[^0-9]*//g')
819	if [ X${number_rundir} != X ] ; then
820	if [ X${number_rundir} = X2 ] ; then
821	if [ ${first_slurm_comp} = "0" ] ; then
822	if [ $(( $coreNumber % $NB_CORE_PER_NODE )) -ne 0 ] ; then
823	(( coreNumber = coreNumber + NB_CORE_PER_NODE - coreNumber % NB_CORE_PER_NODE ))
824	fi
825	echo $coreNumber
826	first_slurm_comp=1 ;
827	fi
828	elif [ X${number_rundir} = X3 ] ; then
829	if [ ${second_slurm_comp} = "0" ] ; then
830	if [ $(( $coreNumber % $NB_CORE_PER_NODE )) -ne 0 ] ; then
831	(( coreNumber = coreNumber + NB_CORE_PER_NODE - coreNumber % NB_CORE_PER_NODE ))
832	fi
833	echo $coreNumber
834	second_slurm_comp=1 ;
835	fi
836	fi
837	fi
838
839	# Manage component executable
840	IGCM_card_DefineArrayFromOption ${configCardPath} Executable ${comp}
841
842	eval ExeNameIn=\${config_Executable_${comp}[0]}
843
844	# NO order in config.card for parallelized values !
845	# just use suffix : MPI , OMP and NOD (for number of NODes.)
846
847	# NOD is the number of NODes allocated
848	eval ${comp}_PROC_NOD=0
849
850	# MPI is the number of MPI processus per nodes
851	eval ${comp}_PROC_MPI=0
852
853	# OMP is the number of OpenMP threads per MPI processus
854	eval ${comp}_PROC_OMP=0
855
856	# DEP is the factor to increase the number of cores per process
857	eval ${comp}_PROC_DEP=0
858
859	# DEDICATED is the flag to activate the use of dedicated node for XIOS server only
860	eval ${comp}_OK_DEDICATED=0
861
862	# Only if we really have an executable for the component :
863	if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
864
865	IGCM_debug_Print 1 ${comp}
866
867	# Keep the first executable found and the first CompName
868	eval ExeNameFirst=${ExeNameIn}
869	CompNameFirst=${comp}
870
871	# Are we a second executable?
872	(( NbExec = NbExec + 1 ))
873
874	# set 1 MPI task, 1 OpenMP thread and 1 node as default
875	eval ${comp}_PROC_MPI=1
876	eval ${comp}_PROC_OMP=1
877	eval ${comp}_PROC_NOD=1
878	eval ${comp}_PROC_DEP=1
879
880	eval NbElts=\${#config_Executable_${comp}[@]}
881	if [ ${NbElts} -gt 2 ] ; then
882	#
883	# CURRENT METHOD TO SPECIFY MPI AND OMP RESSOURCES
884	#
885	i=2
886	while [ ${i} -lt ${NbElts} ] ; do
887	eval tempvar=\${config_Executable_${comp}[${i}]}
888	IGCM_debug_Print 2 ${tempvar}
889
890	if [ X${tempvar} = X ] ; then
891	IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
892	IGCM_debug_Exit "Check your config.card. Exit now"
893	IGCM_debug_Verif_Exit
894	fi
895
896	case ${tempvar} in
897	[mM][pP][iI])
898	# Read MPI parameter for composante
899	eval ${comp}_PROC_MPI=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/MPI//" )
900	OK_PARA_MPI=true;;
901	[oO][mM][pP])
902	# Read OMP parameter for composante
903	eval ${comp}_PROC_OMP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/OMP//" )
904	;;
905	[nN][oO][dD])
906	# Read NOD (NumBer of Nodes) parameter for composante
907	eval ${comp}_PROC_NOD=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/NOD//" )
908	OK_PARA_NOD=true
909	OK_PARA_MPI=true
910	;;
911	[dD][eE][pP])
912	# Read DEP parameter for composante
913	eval ${comp}_PROC_DEP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/DEP//" )
914	;;
915	[dD][eE][dD][iI][cC][aA][tT][eE][dD])
916	# Read DEDICATED parameter for composante
917	eval ${comp}_OK_DEDICATED=1
918	esac
919	(( i = i + 1 ))
920	done
921	else
922	#
923	# BACKWARD COMPATIBILITY NOT SUPPORTED ANYMORE
924	#
925	IGCM_debug_Exit "You are using a deprecated ressources specification mechanism in your config.card"
926	IGCM_debug_Exit "Please check : https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocEsetup#ThesectionExecutable"
927	IGCM_debug_Exit "Please modify ${configCardPath}"
928	exit
929	fi
930	if [ ${comp}_PROC_NOD -gt 1 ] ; then
931	if ( [ ${comp}_PROC_MPI -gt 1 ] && [ ${comp}_PROC_OMP -gt 1 ] ) ; then
932	IGCM_debug_Print 2 "Error using MPI/OMP/NOD parameters !!!"
933	IGCM_debug_Exit "It is not allowed to use MPI and OMP and NOD parameters for the same component"
934	IGCM_debug_Exit "Check your config.card. Exit now"
935	IGCM_debug_Verif_Exit
936	fi
937	if [ "${comp}" != "IOS" ] ; then
938	IGCM_debug_Print 2 "Error using NOD parameter !!!"
939	IGCM_debug_Exit "NOD parameter is only available for IOS component "
940	IGCM_debug_Exit "Check your config.card. Exit now"
941	IGCM_debug_Verif_Exit
942	fi
943	fi
944	if ( [ ${comp}_OK_DEDICATED -eq 1 ] && [ "${comp}" != "IOS" ] ) ; then
945	IGCM_debug_Print 2 "Error using DEDICATED parameter !!!"
946	IGCM_debug_Exit "DEDICATED parameter is only available for IOS component "
947	IGCM_debug_Exit "Check your config.card. Exit now"
948	IGCM_debug_Verif_Exit
949	fi
950
951	eval tempvarMPI=\${${comp}_PROC_MPI}
952	eval tempvarNOD=\${${comp}_PROC_NOD}
953	eval tempvarOMP=\${${comp}_PROC_OMP}
954	eval tempvarDEP=\${${comp}_PROC_DEP}
955
956	# set OMP mode if more than 1 OMP thread.
957	[ ${tempvarOMP} -ge 2 ] && OK_PARA_OMP=true
958
959	# Number of OMP threads
960	[ ${openMPthreads} -lt ${tempvarOMP} ] && openMPthreads=${tempvarOMP}
961
962	# SUM UP NUMBER OF CORES
963	if [ ${comp}_PROC_NOD -gt 1 ] ; then
964	(( coreNumber = coreNumber + tempvarNOD * NB_CORE_PER_NODE ))
965	else
966	(( coreNumber = coreNumber + tempvarMPI * tempvarNOD * tempvarOMP * tempvarDEP ))
967	fi
968	# SUM UP NUMBER OF MPI TASKS
969	(( mpiTasks = mpiTasks + tempvarMPI * tempvarNOD ))
970	fi
971	done
972
973	# MANDATORY FOR THE OPA9.DRIVER. USED TO EDIT OPA NAMELIST
974	# WE SHOULD PLANIFY NUM_PROC_??? DEPRECATION
975	NUM_PROC_CPL=${CPL_PROC_MPI}
976	NUM_PROC_OCE=${OCE_PROC_MPI}
977	NUM_PROC_ATM=${ATM_PROC_MPI}
978
979	# set MPMD mode if more than 2 executable names.
980	[ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
981
982	# Define the execution type we are running in
983	if ( ${OK_PARA_MPMD} ) ; then
984	if ( ${OK_PARA_MPI} ) ; then
985	# MPMD always implies MPI
986	executionType=1
987	fi
988	if ( ${OK_PARA_OMP} ) ; then
989	# MPMD + MPI/OMP
990	executionType=2
991	fi
992	else
993	if ( ( ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
994	# SPMD + MPI/OMP
995	executionType=3
996	elif ( ( ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
997	# SPMD + MPI only
998	executionType=4
999	elif ( ( ! ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
1000	# SPMD + OMP only
1001	executionType=5
1002	elif ( ( ! ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
1003	# SEQUENTIAL THEN
1004	executionType=6
1005	coreNumber=1
1006	fi
1007	fi
1008
1009	IGCM_debug_Print 1 "MPI/OMP treatment coreNumber = ${coreNumber}"
1010	IGCM_debug_Print 1 "MPI/OMP treatment mpiTasks = ${mpiTasks}"
1011	IGCM_debug_Print 1 "MPI/OMP treatment openMPthreads = ${openMPthreads}"
1012	IGCM_debug_Print 1 "MPI/OMP treatment executionType = ${executionType}"
1013
1014	IGCM_debug_PopStack "IGCM_config_ConfigureExecution"
1015	}
1016
1017	#===================================
1018	function IGCM_config_PeriodStart
1019	{
1020	IGCM_debug_PushStack "IGCM_config_PeriodStart"
1021
1022	echo
1023	IGCM_debug_Print 1 "IGCM_config_PeriodStart"
1024	echo
1025
1026	if ( ${FirstInitialize} ) ; then
1027	#================================================#
1028	# Initialize date/period information #
1029	#================================================#
1030
1031	IGCM_date_GetYearMonthDay ${DateBegin} year month day
1032	IGCM_config_DaysInPeriodLength
1033
1034	PeriodDateBegin=${DateBegin}
1035	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1036	CumulPeriod=1
1037
1038	#=================================================#
1039	# Write updated run.card #
1040	#=================================================#
1041
1042	#Correct run.card Configuration for this period
1043	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1044	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1045	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1046	if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
1047	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
1048	fi
1049
1050	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
1051
1052	else
1053	#================================================#
1054	# The file run.card allready exist #
1055	#================================================#
1056
1057	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1058	IGCM_config_StateCheck
1059	# And EXIT if not OK
1060	IGCM_debug_Verif_Exit
1061
1062	#===================================#
1063	# Read updated run.card #
1064	#===================================#
1065
1066	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
1067	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
1068	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
1069
1070	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
1071	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
1072	CumulPeriod=${run_Configuration_CumulPeriod}
1073
1074	LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
1075
1076	if [ ${Period} = 1 ]; then
1077	# save last Job output and current run.card
1078	typeset Potential
1079	IGCM_sys_Cd ${SUBMIT_DIR}
1080	#
1081	IGCM_debug_Print 2 "Save previous ksh job output"
1082	for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
1083	if [ X${Pack} = Xtrue ] ; then
1084	( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${Potential} ) \|\| IGCM_sys_Cp ${Potential} ${R_BUF_KSH}/${Potential}.$$
1085	else
1086	( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) \|\| IGCM_sys_Put_Out ${Potential} ${R_OUT_KSH}/${Potential}.$$ NOMOVE
1087	IGCM_debug_Print 1 "Save of previous ksh job output is not done for Pack=false"
1088	fi
1089	done
1090
1091	#
1092	IGCM_debug_Print 2 "Save current run.card"
1093	IGCM_card_CheckConflict run.card
1094
1095	if [ X${Pack} = Xtrue ] ; then
1096	IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card
1097	else
1098	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card NOMOVE
1099	IGCM_debug_Print 1 "Save of run.card is not done for Pack=false"
1100	fi
1101	#
1102	IGCM_sys_Cd ${RUN_DIR}
1103	else
1104	unset FileToBeDeleted
1105	fi
1106
1107	# Determine number of day(s) in PeriodLength
1108	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1109	IGCM_config_DaysInPeriodLength
1110
1111	# Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
1112	IGCM_config_DateCoherency
1113	# And EXIT if not OK
1114	IGCM_debug_Verif_Exit
1115
1116	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1117	IGCM_config_StateCheck
1118	# And EXIT if not OK
1119	IGCM_debug_Verif_Exit
1120
1121	# We can say we are "Running" now.
1122	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
1123	fi
1124
1125	# BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
1126	# Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
1127	year_m1=$(( year - 1 ))
1128	year_p1=$(( year + 1 ))
1129	# Compute month_m1 (month minus 1M)
1130	# Compute yyyymm_m1 (yyyymm minus 1M)
1131	month_m1=$(( 10#${month} - 1 ))
1132	if [ ${month_m1} = 0 ]; then
1133	month_m1=12
1134	yyyymm_m1=${year_m1}12
1135	elif [ ${month_m1} -le 9 ]; then
1136	month_m1=0${month_m1}
1137	yyyymm_m1=${year}${month_m1}
1138	else
1139	yyyymm_m1=${year}${month_m1}
1140	fi
1141	# Compute month_p1 (month plus 1M)
1142	# Compute yyyymm_p1 (yyyymm plus 1M)
1143	month_p1=$(( 10#${month} + 1 ))
1144	if [ ${month_p1} = 13 ]; then
1145	month_p1=01
1146	yyyymm_p1=${year_p1}01
1147	elif [ ${month_p1} -le 9 ]; then
1148	month_p1=0${month_p1}
1149	yyyymm_p1=${year}${month_p1}
1150	else
1151	yyyymm_p1=${year}${month_p1}
1152	fi
1153
1154
1155	#===================================================================#
1156	# Calculate CyclicYear to be used for looping over a given forcing #
1157	# period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
1158	#===================================================================#
1159
1160	# To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
1161	# CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
1162	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
1163	CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
1164	#CyclicYear_p1= #NOTDEFINED
1165
1166	# For current year
1167	yeartmp=$year
1168	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
1169	while [ $diffy -lt 0 ] ; do
1170	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
1171	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
1172	done
1173	CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
1174
1175	# For next coming year
1176	CyclicYear_p1=$(( CyclicYear + 1 ))
1177
1178	# For previous year
1179	CyclicYear_m1=$(( CyclicYear - 1 ))
1180
1181	IGCM_debug_Print 1 "CyclicYear = ${CyclicYear}, current year=$year"
1182	IGCM_debug_Print 1 "CyclicYear_p1 = ${CyclicYear_p1}, CyclicYear_m1 = ${CyclicYear_m1} "
1183	else
1184	CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
1185	CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
1186	IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
1187	fi
1188
1189	# END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
1190
1191	#===================================================================#
1192	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
1193	# But available to any son functions #
1194	#===================================================================#
1195
1196	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
1197	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
1198
1199	# Debug Print :
1200	echo
1201	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
1202	IGCM_debug_Print 1 "Year of simulation : ${year}"
1203	IGCM_debug_Print 1 "Month of simulation : ${month}"
1204	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1205	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1206	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1207	IGCM_debug_Print 1 "SimulationLengthInDays : ${SimulationLengthInDays}"
1208	IGCM_debug_Print 1 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
1209
1210	#================================================================#
1211	# Prepare variables available for comp_finalyze #
1212	#================================================================#
1213
1214	# Period for save files
1215	DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
1216
1217	# Prefix for save files of this period
1218	PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
1219
1220	# List of files that will be deleted in RUN_DIR after run
1221	[ -f stack ] && FileToBeDeleted[0]="stack"
1222
1223	# Test if the same run as already been saved :
1224	if [ X${JobType} = XRUN ] ; then
1225	if [ ${DRYRUN} -le 0 ] ; then
1226	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1227	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently re-running an old job."
1228	IGCM_debug_Print 1 "Because of readonly permissions, you can't re-run a job when saved files"
1229	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
1230	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See different clean_*.job in ${libIGCM} directory."
1231	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
1232	IGCM_debug_Verif_Exit
1233	fi
1234	fi
1235	else
1236	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1237	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
1238	fi
1239	fi
1240
1241	#================================================================#
1242	# Prepare variables available for binary execution #
1243	# Call function for creation of run script #
1244	# Only done once per job #
1245	#================================================================#
1246
1247	if [ ${Period} -eq 1 ]; then
1248	# Define the execution context (MPMD, SPMD, MPI/OMP ...)
1249	IGCM_config_ConfigureExecution ${SUBMIT_DIR}/config.card
1250	# Create the execution script for the current context
1251	IGCM_sys_build_execution_scripts
1252	fi
1253
1254	ExecutionFail=false
1255
1256	# Update the rabbitMQ queue
1257	IGCM_debug_BigBro_Update
1258
1259	IGCM_debug_PopStack "IGCM_config_PeriodStart"
1260	}
1261
1262	#===================================
1263	function IGCM_config_SaveSourceModifications
1264	{
1265	IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
1266
1267	typeset ExeOutDateMax listVarEnv
1268	ExeOutDateMax=$1
1269
1270	listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
1271	IGCM_sys_RshMaster "\
1272	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
1273	export ExeOutDateMax=${ExeOutDateMax};\
1274	export R_OUT_EXE=${R_OUT_EXE};\
1275	export PREFIX=${PREFIX};\
1276	export SUBMIT_DIR=${SUBMIT_DIR};\
1277	export listVarEnv=${listVarEnv};\
1278	Script_Output=out_SaveSourceModifications;\
1279	IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
1280
1281	IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
1282	}
1283
1284	#===================================
1285	function IGCM_config_PeriodEnd
1286	{
1287	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
1288
1289	echo
1290	IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
1291	echo
1292
1293	if [ ${DRYRUN} -le 1 ] ; then
1294
1295	IGCM_debug_Print 1 "Check components binary : size and creation date"
1296
1297	typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
1298	typeset comp i
1299	typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
1300
1301	#==================================#
1302	# Get last Exe Size #
1303	#==================================#
1304
1305	(( i=0 ))
1306	if ( ${FirstInitialize} ) ; then
1307	run_Log_LastExeSize=""
1308	for comp in ${config_ListOfComponents[*]} ; do
1309	run_Log_LastExeSize[$i]=0
1310	(( i=i+1 ))
1311	done
1312	else
1313	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
1314	fi
1315	#==================================#
1316	# And Build ExeDate #
1317	#==================================#
1318
1319	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
1320	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
1321	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
1322	ExeDate=""
1323	NextExeSize="( "
1324	(( i=0 ))
1325	UpdateExe=false
1326	(( ExeSecDateMax = 0 ))
1327	for comp in ${config_ListOfComponents[*]} ; do
1328
1329	IGCM_debug_Print 3 ${comp}
1330
1331	eval ExeNameIn=\${config_Executable_${comp}[0]}
1332	eval ExeNameOut=\${config_Executable_${comp}[1]}
1333	# Only if we really have an executable for the component :
1334	if [ X${ExeNameIn} = X\"\" ] ; then
1335	# If there is no exe file for this component
1336	(( ExeSize=0 ))
1337	else
1338	eval ExeNameIn=${ExeNameIn}
1339	LS_bin=${R_EXE}/${ExeNameIn}
1340	IGCM_sys_FileSize ${LS_bin} ExeSize
1341
1342	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
1343	if [ X${ExeDate} = X ] ; then
1344	# First component exe date
1345	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
1346	else
1347	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1348	fi
1349	ExeDate=${ExeDate}_${LS_comp[7]}
1350	fi
1351
1352	if [ ${i} -eq 0 ] ; then
1353	# First component
1354	NextExeSize="( "${ExeSize}
1355	else
1356	NextExeSize=${NextExeSize}", "${ExeSize}
1357	fi
1358	LastCompExeSize=${run_Log_LastExeSize[$i]}
1359	(( i=i+1 ))
1360
1361	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1362	if ( ${FirstInitialize} ) ; then
1363	IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1364	else
1365	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1366	IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
1367	FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1368	fi
1369	IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1370	UpdateExe=true
1371
1372	# SD : switch off for now
1373	#IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1374	#if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1375	# ExeSecDateMax=$ExeSecDate
1376	#fi
1377	fi
1378	done
1379
1380	# SD : switch off for now
1381	#if ( ${UpdateExe} ) ; then
1382	# echo "Launch SaveSourceModifications."
1383	# IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1384	#fi
1385
1386	NextExeSize=${NextExeSize}" )"
1387	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1388
1389	if [ ${DRYRUN} -le 1 ] ; then
1390	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1391	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1392	RET=$?
1393	if [ $RET -eq 0 ] ; then
1394	# ExeCpuLog variable contents 5 fields
1395	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
1396	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
1397	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1398	fi
1399	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
1400	fi
1401
1402	fi
1403
1404	#==================================#
1405	# Save Job output #
1406	#==================================#
1407	if [ X${Pack} = Xtrue ] ; then
1408	IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1409	else
1410	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1411	fi
1412	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1413
1414	# All was right ? no ? then we stop.
1415	IGCM_debug_Verif_Exit
1416
1417	# If all was OK, we can delete all files not necessary for next Job
1418	echo
1419	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1420
1421	if [ ${DRYRUN} -le 2 ] ; then
1422	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1423	else
1424	echo ${FileToBeDeleted[@]}
1425	fi
1426
1427	# Send some accounting element to the user if CumulPeriod=3
1428	if [ ${CumulPeriod} -eq 3 ] ; then
1429	echo
1430	IGCM_debug_Print 1 "Send email containing some accounting information : "
1431
1432	RealCpuTime=$( echo ${ExeCpuLog} \| gawk '{print $3}' )
1433
1434	consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${coreNumber}/3600" \| bc )
1435
1436	consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" \| bc )
1437
1438	recommendedNbPeriodsPerJob=$( echo "scale=6;${jobWarningDelay}/3600/${consumeHoursPerPeriod}*${coreNumber}" \| bc )
1439
1440	IGCM_sys_SendMail Accounting
1441	fi
1442
1443	#=================================================#
1444	# Modification of libIGCM behaviour #
1445	#=================================================#
1446
1447	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
1448	# and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1449	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1450	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1451	echo
1452	echo "########################################################################"
1453	echo "!!! Modification of libIGCM behaviour !!!"
1454	echo
1455
1456	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
1457	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1458	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1459
1460	# Special treatments for libIGCM internals
1461	for option in ${libIGCM_UserChanges[*]} ; do
1462	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1463
1464	echo "We will change : ${option}."
1465	eval echo "Previous value : " \${${option}}
1466	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1467
1468	eval ${option}=\${libIGCM_UserChanges_${option}}
1469
1470	case ${option} in
1471	config_UserChoices_DateEnd)
1472	IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1473	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1474
1475	# Period Length In Days between DateBegin and DateEnd
1476	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
1477	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1478	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1479	IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
1480	IGCM_debug_Verif_Exit
1481	fi
1482	;;
1483	config_UserChoices_PeriodLength)
1484	IGCM_debug_Print 1 "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1485	;;
1486	NbPeriodsPerJob)
1487	IGCM_debug_Print 1 "Loop in main Job with ${NbPeriodsPerJob} period(s)"
1488	;;
1489	config_Post_RebuildFrequency)
1490	IGCM_debug_Print 1 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1491	IGCM_post_Configure
1492	;;
1493	config_Post_TimeSeriesFrequency)
1494	IGCM_debug_Print 1 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1495	IGCM_post_Configure
1496	;;
1497	config_Post_SeasonalFrequency)
1498	IGCM_debug_Print 1 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1499	IGCM_post_Configure
1500	;;
1501	esac
1502	done
1503
1504	echo
1505	echo "########################################################################"
1506	echo
1507	fi
1508
1509	#=================================================#
1510	# Determine next computed period #
1511	#=================================================#
1512
1513	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1514	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1515	year_m1=$(( year - 1 ))
1516	year_p1=$(( year + 1 ))
1517	IGCM_config_DaysInPeriodLength
1518	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1519
1520	# Debug Print :
1521	echo
1522	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1523	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1524	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1525	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1526
1527	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1528	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1529
1530	(( CumulPeriod = CumulPeriod + 1 ))
1531
1532	# Debug Print :
1533	echo
1534	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1535	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
1536	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
1537
1538	#=================================================#
1539	# Write updated run.card #
1540	#=================================================#
1541
1542	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1543	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1544	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1545
1546	if ( ${FirstInitialize} ) ; then
1547	# It's no more the first time
1548	FirstInitialize=false
1549	fi
1550
1551	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1552	}
1553
1554	#===================================
1555	function IGCM_config_Finalize
1556	{
1557	IGCM_debug_PushStack "IGCM_config_Finalize"
1558
1559	echo
1560	IGCM_debug_Print 1 "IGCM_config_Finalize"
1561	echo
1562
1563	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1564	IGCM_config_StateCheck
1565
1566	# And EXIT if not OK
1567	IGCM_debug_Verif_Exit
1568
1569	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1570	#==========================#
1571	# End of entire simulation #
1572	#==========================#
1573	simulationIsOver=true
1574
1575	# Mail notification
1576	IGCM_sys_SendMail
1577	#
1578	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1579	#
1580	IGCM_debug_Print 1 "Normal End of computation."
1581
1582	else
1583	#=================#
1584	# Submit next job #
1585	#=================#
1586	simulationIsOver=false
1587
1588	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1589
1590	# Name of next Ksh Script output :
1591	Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
1592
1593	IGCM_debug_Print 1 "Submit next job"
1594	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1595	IGCM_sys_Cd ${SUBMIT_DIR}
1596	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1597	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null \| wc -l )
1598	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
1599	# Submit next job and come back
1600	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1601	IGCM_sys_Cd -
1602	fi
1603
1604	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1605	# Only for production run (No clean up in DEV or DEB mode)
1606	# and command sent from .. directory.
1607	IGCM_sys_Cd ..
1608	[ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
1609
1610	# Inform the rabbitMQ queue
1611	IGCM_debug_BigBro_Finalize
1612
1613	IGCM_debug_PopStack "IGCM_config_Finalize"
1614	}
1615
1616	#===================================

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: branches/libIGCM_concurrent/libIGCM_config/libIGCM_config.ksh

Download in other formats: