Context Navigation

source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 1598

Last change on this file since 1598 was 1598, checked in by falletti, 9 months ago
Integration of C-ESM-EP (post-processing) in libIGCM. From the work done by Stephane Senesi. To be activated in config.card [Post] with the keyword Cesmep (default: FALSE).
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 61.7 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_CommonConfiguration
16	{
17	IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19	# Debug Print :
20	[ ${Verbosity} -gt 0 ] && echo
21	IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23	# config.card path
24	configCardPath=$1
25
26	#==================================
27	typeset option auxprint
28
29	#==================================
30	# Read UserChoices section:
31	[ ${Verbosity} -gt 0 ] && echo
32	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
33
34	IGCM_card_DefineArrayFromSection ${configCardPath} UserChoices
35	for option in ${config_UserChoices[*]} ; do
36	IGCM_card_DefineVariableFromOption ${configCardPath} UserChoices ${option}
37	eval auxprint=\${config_UserChoices_${option}}
38	IGCM_debug_Print 3 "${option} : ${auxprint}"
39	done
40
41	# Set variables with shorter names on some variables which
42	# are used in the executable name in config.card
43	ResolAtm=${config_UserChoices_ResolAtm}
44	ResolOce=${config_UserChoices_ResolOce}
45	ConfChem=${config_UserChoices_ConfChem}
46	OptMode=${config_UserChoices_OptMode}
47
48	#==================================
49	# Read Ensemble section:
50	[ ${Verbosity} -gt 0 ] && echo
51	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
52
53	IGCM_card_DefineArrayFromSection ${configCardPath} Ensemble
54	for option in ${config_Ensemble[*]} ; do
55	IGCM_card_DefineVariableFromOption ${configCardPath} Ensemble ${option}
56	eval auxprint=\${config_Ensemble_${option}}
57	IGCM_debug_Print 3 "${option} : ${auxprint}"
58	done
59
60	#==================================
61	# Read Post section:
62	[ ${Verbosity} -gt 0 ] && echo
63	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
64
65	IGCM_card_DefineArrayFromSection ${configCardPath} Post
66	for option in ${config_Post[*]} ; do
67	IGCM_card_DefineVariableFromOption ${configCardPath} Post ${option}
68	eval auxprint=\${config_Post_${option}}
69	IGCM_debug_Print 3 "${option} : ${auxprint}"
70	done
71	[ ${Verbosity} -gt 0 ] && echo
72
73	#==================================
74	# Define default value to keep compatibility with previous card: means before changes due to TGCC
75	# Apply some overrules to ensure proper usage of computing centres resources
76	#
77
78	# if RebuildFrequency is not define in config.card - we initilize it to NONE
79	if [ X${config_Post_RebuildFrequency} = X ] ; then
80	config_Post_RebuildFrequency=NONE
81	fi
82
83	if [ X${PackDefault} = Xtrue ] ; then
84	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
85	# TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
86	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
87	config_Post_PackFrequency=NONE
88	else
89	# Default to RebuildFrequency if nothing has been set up related to PackFrequency
90	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
91	fi
92	else
93	# If we DO NOT apply pack in this computing center
94	config_Post_PackFrequency=NONE
95	fi
96
97	#====================================================
98	# Define ARCHIVE : Dedicated to large files
99	# Define STORAGE : Dedicated to small/medium files
100	# Define R_OUT : Output tree located on ARCHIVE
101	# Define R_BUF : Output tree located on STORAGE (files waiting treatment, or file lcoation when SpaceName=!PROD)
102	# Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
103	# Define R_TMP : A temporary space used by IGCM_debug_send_AMQP_msg__MAILTUNNEL. Must be persistent in between jobs
104	IGCM_sys_defineArchives
105
106	#====================================================
107	# R_SAVE : Job output directory
108	# R_BUFR : Job output buffered directory
109	# R_CMIP : Job output directory for CMIP6 standard files
110
111	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
112	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
113
114	if ( [ X${config_Post_StorageCMIP} = XSCRATCH ] \|\| [ X${config_Post_StorageCMIP} = Xscratch ] ) ; then
115	IGCM_debug_Print 1 "LIBIGCM CONFIG:: SCRATCH R_BUF=${R_BUF}"
116	StoreCMIP=${R_BUF}
117	elif ( [ X${config_Post_StorageCMIP} = XSTORE ] \|\| [ X${config_Post_StorageCMIP} = Xstore ] ) ; then
118	IGCM_debug_Print 1 "LIBIGCM CONFIG:: SCRATCH R_OUT=${R_OUT}"
119	StoreCMIP=${R_OUT}
120	else
121	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ELSE CMIP_BUF=${CMIP6_BUF}"
122	StoreCMIP=${CMIP6_BUF}
123	fi
124	# Ensemble cases (default all members outputs in same directory)
125	if ( [ ! X${config_Ensemble_EnsembleName} = X ] ) ; then
126	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
127	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
128	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
129	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
130	R_CMIP=${StoreCMIP}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}
131	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE R_CMIP=${R_CMIP}"
132
133	# PERTURB Ensemble Outputs
134	if ( [ X${config_Ensemble_EnsembleType} = XEns_PERTURB ] ) ; then
135	if ( [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
136	R_SAVE=${R_SAVE}/${config_Ensemble_EnsembleDate}/${FreeName}
137	R_FIGR=${R_FIGR}/${config_Ensemble_EnsembleDate}/${FreeName}
138	R_BUFR=${R_BUFR}/${config_Ensemble_EnsembleDate}/${FreeName}
139	R_DODS=${R_DODS}/${config_Ensemble_EnsembleDate}/${FreeName}
140	R_CMIP=${R_CMIP}/${config_Ensemble_EnsembleDate}/${FreeName}
141	else
142	R_SAVE=${R_SAVE}/${FreeName}
143	R_FIGR=${R_FIGR}/${FreeName}
144	R_BUFR=${R_BUFR}/${FreeName}
145	R_DODS=${R_DODS}/${FreeName}
146	R_CMIP=${R_CMIP}/${FreeName}
147	fi
148	# DATE Ensemble Outputs
149	elif ( [ X${config_Ensemble_EnsembleType} = XEns_DATE ] ) ; then
150	R_FIGR=${R_FIGR}/${FreeName}
151	R_BUFR=${R_BUFR}/${FreeName}
152	R_DODS=${R_DODS}/${FreeName}
153	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE DATE R_CMIP=${R_CMIP}"
154	if ( [ X${config_Ensemble_EnsembleMergeSave} = X ] \|\| [ X${config_Ensemble_EnsembleMergeSave} = Xn ] \|\|
155	[ X${config_Ensemble_EnsembleMergeSave} = Xno ] ) ; then
156	# One SAVE directory per member, else common output dir for all members
157	R_SAVE=${R_SAVE}/${FreeName}
158	R_CMIP=${R_CMIP}/${FreeName}
159	IGCM_debug_Print 1 "LIBIGCM CONFIG:: ENSEMBLE DATE MERGE R_CMIP=${R_CMIP}"
160	fi
161	fi
162	else
163	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
164	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
165	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
166	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
167	R_CMIP=${StoreCMIP}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
168	fi
169	else
170	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
171	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
172	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
173	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
174	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
175	else
176	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
177	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
178	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
179	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
180	fi
181	fi
182
183	IGCM_debug_Print 1 "LIBIGCM CONFIG:: R_CMIP = ${R_CMIP}"
184
185	#====================================================
186	# Define R_OUT_KSH : Storage place for job output
187	# Define R_OUT_EXE : Storage place for binary used during simulation
188	R_OUT_KSH=${R_SAVE}/Out
189	R_OUT_EXE=${R_SAVE}/Exe
190
191	#====================================================
192	# Define R_BUF_KSH : Buffer place for job output
193	# Define R_BUF_EXE : Buffer place for binary used during simulation
194	R_BUF_KSH=${R_BUFR}/Out
195	R_BUF_EXE=${R_BUFR}/Exe
196
197	#====================================================
198	# Define REBUILD_DIR : where we store files needing rebuild process
199	REBUILD_DIR=${R_BUFR}/REBUILD
200	if [ ! X${TaskType} = Xchecking ] ; then
201	IGCM_sys_MkdirWork ${REBUILD_DIR}
202	fi
203
204	#====================================================
205	# DodsCopy : apply default value if not defined
206	if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] \|\| [ X${config_Post_DodsCopy} = X ] ) ; then
207	config_Post_DodsCopy=TRUE
208	fi
209
210	#====================================================
211	# AtlasIPSL : apply default value if not defined
212	if ( [ X${config_Post_AtlasIPSL} = X${NULL_STR} ] \|\| [ X${config_Post_AtlasIPSL} = X ] ) ; then
213	config_Post_AtlasIPSL=FALSE
214	fi
215
216	#====================================================
217	# ParserXIOS : apply default value if not defined
218	if ( [ X${config_Post_ParserXIOS} = X${NULL_STR} ] \|\| [ X${config_Post_ParserXIOS} = X ] ) ; then
219	config_Post_ParserXIOS=FALSE
220	fi
221
222	#====================================================
223	# MetricsPCMDI : apply default value if not defined
224	if ( [ X${config_Post_MetricsPCMDI} = X${NULL_STR} ] \|\| [ X${config_Post_MetricsPCMDI} = X ] ) ; then
225	config_Post_MetricsPCMDI=FALSE
226	fi
227	#====================================================
228	# Cesmep : apply default value FALSE if not defined, and if TRUE
229	# or *Y set it based on an adhoc algorithm
230	if ( [ X${config_Post_Cesmep} = X${NULL_STR} ] \|\| [ X${config_Post_Cesmep} = XNONE ] \|\| \
231	[ X${config_Post_Cesmep} = X ] \|\| [ X${config_Post_Cesmep} = XFALSE ] ) ; then
232	config_Post_Cesmep=FALSE
233	elif [ X${config_Post_Cesmep} = XTRUE ] \|\| [[ X${config_Post_Cesmep} =~ X[1-9][0-9]?Y ]] ; then
234	if [[ X${config_Post_Cesmep} =~ X[1-9][0-9]?Y ]] ; then
235	CesmepPeriod=${config_Post_Cesmep%Y}
236	fi
237	if ([ ! X${config_Post_SeasonalFrequency} = XNONE ] && [ ! X${config_Post_SeasonalFrequency} = X ] ) ; then
238	config_Post_Cesmep="SE"
239	elif ([ ! X${config_Post_TimeSeriesFrequency} = XNONE ] && [ ! X${config_Post_TimeSeriesFrequency} = X ] ) ; then
240	config_Post_Cesmep="TS"
241	elif ([ ! X${config_Post_PackFrequency} = XNONE ] && [ ! X${config_Post_PackFrequency} = X ] ) ; then
242	config_Post_Cesmep="Pack"
243	else
244	# This covers case SpaceName=TEST with no SE nor TS (nor Pack)
245	config_Post_Cesmep=AtEnd
246	fi
247	fi
248	#
249	if [ X${config_Post_Cesmep} != XFALSE ] ; then
250	if [ X${config_Post_Cesmep} = XSE ] ; then
251	if ([ X${config_Post_SeasonalFrequency} = XNONE ] \|\| [ X${config_Post_SeasonalFrequency} = X ] ) ; then
252	IGCM_debug_Exit "Cannot use SE for Cesmep : SE post-processing is not active"
253	else
254	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_SeasonalFrequency%Y}
255	fi
256	elif [ X${config_Post_Cesmep} = XTS ] ; then
257	if ([ X${config_Post_TimeSeriesFrequency} = XNONE ] \|\| [ X${config_Post_TimeSeriesFrequency} = X ] ) ; then
258	IGCM_debug_Exit "Cannot use TS for Cesmep : TS post-processing is not active"
259	else
260	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_TimeSeriesFrequency%Y}
261	fi
262	elif [ X${config_Post_Cesmep} = XPack ] ; then
263	if ([ X${config_Post_PackFrequency} = XNONE ] \|\| [ X${config_Post_PackFrequency} = X ] ) ; then
264	IGCM_debug_Exit "Cannot use Packed output for Cesmep : Pack post-processing is not active"
265	else
266	[ -z $CesmepPeriod ] && CesmepPeriod=${config_Post_PackFrequency%Y}
267	fi
268	elif [ X${config_Post_Cesmep} != XAtEnd ] ; then
269	IGCM_debug_Exit "config.card's Post section's value for Cesmep (${config_Post_Cesmep}) is invalid; use SE, TS, Pack, AtEnd, TRUE or <n>Y"
270	else
271	[ -z $CesmepPeriod ] && CesmepPeriod=0
272	fi
273
274	if ( [ X${config_Post_CesmepComparison} = XNONE ] \|\| [ X${config_Post_CesmepComparison} = X ] ) ; then
275	config_Post_CesmepComparison=run_comparison
276	fi
277
278	if ( [ X${config_Post_CesmepCode} = XNONE ] \|\| [ X${config_Post_CesmepCode} = X ] ) ; then
279	if ( [ X"${SYSTEM}" == "Xirene" ] \|\| [ X"${SYSTEM}" == "Xirene-amd" ] ); then
280	config_Post_CesmepCode=/ccc/cont003/home/igcmg/igcmg/Tools/cesmep
281	elif ( [ X"${SYSTEM}" == "Xjeanzay" ] ); then
282	config_Post_CesmepCode=/gpfswork/rech/psl/commun/Tools/cesmep
283	elif ( [ X"${SYSTEM}" == "Xmesoipsl" ] ); then
284	config_Post_CesmepCode=/net/nfs/tools/Users/SU/jservon/cesmep_installs/cesmep_for_libIGCM
285	elif [ ${config_Post_Cesmep} != FALSE ] ; then
286	IGCM_debug_Exit "Cannot locate a default C-ESM-EP code on $SYSTEM, please set CesmepCode in section Post"
287	fi
288	fi
289	if ( [ X${config_Post_CesmepSlices} = XNONE ] \|\| [ X${config_Post_CesmepSlices} = X ] ) ; then
290	config_Post_CesmepSlices=8
291	fi
292	fi
293	IGCM_debug_Print 3 "config_Post_Cesmep=${config_Post_Cesmep}, CesmepPeriod=$CesmepPeriod, config_Post_CesmepSlices=$config_Post_CesmepSlices "
294	#====================================================
295	# IgnoreNonMonotonic : apply default value if not defined
296	if ( [ X${config_Post_IgnoreNonMonotonic} = X${NULL_STR} ] \|\| [ X${config_Post_IgnoreNonMonotonic} = X ] ) ; then
297	config_Post_IgnoreNonMonotonic=FALSE
298	fi
299
300	#====================================================
301	# LightRestartPack : apply default value if not defined
302	if ( [ X${config_Post_LightRestartPack} = X${NULL_STR} ] \|\| [ X${config_Post_LightRestartPack} = X ] ) ; then
303	config_Post_LightRestartPack=FALSE
304	fi
305
306	#====================================================
307	# dr2xmlIPSL : apply default value if not defined
308	if ( [ X${config_Post_dr2xmlIPSL} = X${NULL_STR} ] \|\| [ X${config_Post_dr2xmlIPSL} = X ] ) ; then
309	config_Post_dr2xmlIPSL=FALSE
310	fi
311
312
313	#====================================================
314	# Define StackFileLocation : directory where we store stack files
315	# Define StackFileName : stack file containing call tree and instrumentation
316	# Stack file containing call tree will be stored there.
317	if ( $DEBUG_debug ) ; then
318	StackFileLocation=${StackFileLocation:=${R_BUF_KSH}}
319	[ ! -d ${StackFileLocation} ] && mkdir -p ${StackFileLocation}
320	if [ X${TaskType} = Xcomputing ]; then
321	StackFileName=computing.stack.$$
322	elif [ X${TaskType} = Xpost-processing ]; then
323	StackFileName=${Script_Post_Output}.stack.$$
324	elif [ X${TaskType} = Xchecking ]; then
325	StackFileName=checking.stack.$$
326	else
327	IGCM_debug_Exit "IGCM_config_CommonConfiguration unknown TaskType : ${TaskType}"
328	IGCM_debug_Verif_Exit
329	fi
330
331	# This boolean will trigger the filling of the stack
332	# Only now we know where things should be ...
333	# We don't fill the stack when we perform checking task
334	if [ ! X${TaskType} = Xchecking ] ; then
335	ActivateStackFilling=true
336	fi
337	fi
338
339	IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
340	}
341
342	#===================================
343	function IGCM_config_Initialize
344	{
345	IGCM_debug_PushStack "IGCM_config_Initialize"
346
347	# Debug Print :
348	echo
349	IGCM_debug_Print 1 "IGCM_config_Initialize"
350
351	# Test modipsl tree existence.
352	IGCM_sys_TestDir ${MODIPSL}
353	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
354	IGCM_sys_TestDir ${libIGCM}
355	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
356	IGCM_sys_TestDir ${R_EXE}
357	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
358	IGCM_sys_TestDir ${SUBMIT_DIR}
359	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
360
361	if ( $DEBUG_debug ) ; then
362	echo "Keep trace of inital SUBMIT_DIR : "
363	ls -lta ${SUBMIT_DIR}
364	fi
365
366	#==================================
367	# Read ListOfComponents section:
368	echo
369	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
370
371	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
372	for comp in ${config_ListOfComponents[*]} ; do
373	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
374	done
375	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
376
377	#==================================
378	# Read Executable section:
379	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
380
381	#==================================
382	# Read Restarts section:
383	# Restarts : Gerneral rule or local for each component.
384	echo
385	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
386
387	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
388	for option in ${config_Restarts[*]} ; do
389	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
390	eval auxprint=\${config_Restarts_${option}}
391	IGCM_debug_Print 3 "${option} : ${auxprint}"
392	done
393
394	#==================================
395	# Define Job Outputs Name
396	echo
397	IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
398	Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
399	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
400	Exe_Output=out_execution
401	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
402
403	#===================================================================#
404	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
405	# But available to any son functions #
406	#===================================================================#
407
408	# Convert yyyy-mm-dd date to gregorian yyyymmdd
409	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
410	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
411	EndYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateEnd ) / 1000 ))
412	EndYear_p1=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateEnd ) / 1000 + 1 ))
413
414
415	# Period Length In Days between DateBegin and DateEnd
416	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
417	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
418	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
419	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
420	IGCM_debug_Verif_Exit
421	fi
422
423	# Day and Year of Initial State (Given in julian format)
424	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
425	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
426	InitYear_m1=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 - 1 ))
427
428	#================================================================#
429	# Test and Prepare directories #
430	#================================================================#
431
432	# ==> 4 kinds of input files :
433	# 1) R_INIT : Initial State Files (Etat0, carteveg)
434	# 2) R_BC : Boundary Conditions (Forcages, lai)
435	# 3) Parameters files (allready define through ${SUBMIT_DIR})
436	# 4) Restarts files (allready define in IGCM_config_Initialize)
437
438	# Here we offer the possibility to redefine R_INIT, R_BC
439	# and PeriodNb through config.card
440	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
441	echo
442	IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
443	IGCM_debug_Print 3 "R_IN=${R_IN}"
444	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
445	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
446	IGCM_debug_Print 3 "R_BC=${R_BC}"
447	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
448	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
449
450	# SD ADA SPECIFIC #
451	# TO FIX #
452	#IGCM_sys_TestDirArchive ${R_IN}
453	#[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
454
455	if ( ${FirstInitialize} ) ; then
456	IGCM_sys_MkdirArchive ${R_SAVE}
457	[ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
458	else
459	IGCM_sys_TestDirArchive ${R_SAVE}
460	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
461
462	if [ ! ${config_Post_PackFrequency} = NONE ] ; then
463	IGCM_sys_TestDir ${R_BUFR}
464	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
465	fi
466
467	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
468	IGCM_config_StateCheck
469
470	# And EXIT if not OK
471	IGCM_debug_Verif_Exit
472	fi
473
474	#====================================================
475	# Experience type : DEB(ug), DEV(elopment), RUN
476	if [ X${JobType} != XRUN ] ; then
477	echo
478	echo "===================================================="
479	echo "libIGCM JOB is NOT in RUN type mode."
480	echo "!! OUTPUT files will NOT be PROTECTED !!"
481	echo "Be carefull : you can ERASE the result of this job !"
482
483	case ${JobType} in
484	DEB)
485	echo "DEBUG mode : activation of 'set -vx' mode."
486	echo "DEBUG mode : no protection for output files."
487	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
488	;;
489	DEV)
490	echo "DEVelopment mode : no protection for output files."
491	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
492	;;
493	esac
494
495	if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
496	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
497	echo "------------"
498	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
499	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
500	echo "------------"
501	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
502	fi
503	fi
504	echo "===================================================="
505	echo
506	fi
507
508	IGCM_debug_PopStack "IGCM_config_Initialize"
509	}
510
511	#===================================
512	function IGCM_config_DaysInPeriodLength
513	{
514	IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
515
516	typeset i
517
518	# Determine number of day(s) in PeriodLength :
519	case ${config_UserChoices_PeriodLength} in
520	Y\|y)
521	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
522	echo
523	IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
524	PeriodLengthInDays=0
525	i=0
526	until [ $i -ge $PeriodLengthInYears ] ; do
527	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
528	(( i=i+1 ))
529	done
530	;;
531	M\|m)
532	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
533	echo
534	IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
535	PeriodLengthInDays=0
536	i=0
537	until [ $i -ge $PeriodLengthInMonths ] ; do
538	if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
539	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
540	else
541	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
542	fi
543	(( i=i+1 ))
544	done
545	;;
546	D\|d)
547	PeriodLengthInMonths=0
548	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' )
549	echo
550	IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
551	*)
552	IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
553	IGCM_debug_Verif_Exit ;;
554	esac
555
556	IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
557	}
558
559	#===================================
560	function IGCM_config_DateCoherency
561	{
562	IGCM_debug_PushStack "IGCM_config_DateCoherency"
563
564	echo
565	IGCM_debug_Print 1 "IGCM_config_DateCoherency"
566	echo
567
568	typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
569
570	# check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
571	# DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
572	echo
573
574	case ${config_UserChoices_PeriodLength} in
575	Y\|y)
576	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
577	;;
578	M\|m)
579	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
580	;;
581	D\|d)
582	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
583	;;
584	esac
585	VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
586
587	if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
588	IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
589	IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
590	IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
591	IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
592	IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
593	IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
594
595	IGCM_debug_Exit "STOP here to avoid further issues."
596	fi
597
598	# PeriodDateBegin + PeriodLength = PeriodDateEnd
599	VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
600
601	IGCM_debug_PopStack "IGCM_config_DateCoherency"
602	}
603
604	#===================================
605	function IGCM_config_StateCheck
606	{
607	IGCM_debug_PushStack "IGCM_config_StateCheck"
608
609	#Test state of run in run.card
610	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
611
612	if [ ${run_Configuration_PeriodState} = "Fatal" ] ; then
613	echo
614	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
615	IGCM_debug_Print 1 "Check the overall status of your simulation by visiting this page:"
616	IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
617	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
618	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean lastest pack period."
619	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
620	IGCM_debug_Exit
621	elif [ $( echo ${run_Configuration_PeriodState} \| grep Fatal \| wc -l ) -eq 1 ] ; then
622	echo
623	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
624	IGCM_debug_Print 1 "Compute jobs has been stop because at least the above mentionned post-processing jobs fails."
625	IGCM_debug_Print 1 "Check post-processing jobs carefully by visiting this page:"
626	IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
627	IGCM_debug_Print 1 "Please visit that page to see how to fix issues:"
628	IGCM_debug_Print 1 "https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocGmonitor"
629	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
630	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean latest pack period."
631	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
632	IGCM_debug_Exit
633	fi
634
635	IGCM_debug_PopStack "IGCM_config_StateCheck"
636	}
637
638	#===================================
639	function IGCM_config_Check
640	{
641	IGCM_debug_PushStack "IGCM_config_Check"
642
643	# If one of the following modulo is not zero :
644	# we will issue an error then explain and exit in
645	# AA_job IGCM_debug_Verif_Exit call before binary submission
646
647	echo
648	IGCM_debug_Print 1 "IGCM_config_Check"
649	echo
650
651	typeset i
652
653	# Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
654	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
655	AsynchronousRebuild=true
656	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
657	echo
658	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
659	IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
660	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
661	IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
662	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
663	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
664	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
665	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
666	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
667	else
668	AsynchronousRebuild=false
669	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
670	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
671	echo
672	#modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
673	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
674	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
675	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
676	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
677	fi
678
679	# Check PackFrequency against other key frequencies
680	# Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
681	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
682	Pack=true
683	#
684	IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
685	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
686	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
687	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
688	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
689	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
690	else
691	Pack=false
692	fi
693
694	# modulo (TimeSeriesFrequency and all Chunck2D) must be zero
695	NbJob=${#CHUNCK2D_SIZE[@]}
696	i=0
697	until [ $i -ge $NbJob ]; do
698	value=${CHUNCK2D_SIZE[${i}]}
699	IGCM_debug_Print 1 "Check coherence between ${CHUNCK2D_NAME[${i}]} Chunck2D frequency and TimeSeriesFrequency"
700	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
701	case ${value} in
702	Y\|y) ;;
703	*)
704	IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year Y\|y in comp.card"
705	IGCM_debug_Exit "This will stop the job" ;;
706	esac
707	(( i=i+1 ))
708	done
709
710	# modulo (TimeSeriesFrequency and all Chunck3D) must be zero
711	NbJob=${#CHUNCK3D_SIZE[@]}
712	i=0
713	until [ $i -ge $NbJob ]; do
714	value=${CHUNCK3D_SIZE[${i}]}
715	IGCM_debug_Print 1 "Check coherence between ${CHUNCK3D_NAME[${i}]} Chunck3D frequency and TimeSeriesFrequency"
716	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
717	case ${value} in
718	Y\|y) ;;
719	*)
720	IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year Y\|y in comp.card"
721	IGCM_debug_Exit "This will stop the job" ;;
722	esac
723	(( i=i+1 ))
724	done
725
726	# check to be sure there is enough space on temporary filesystems to run
727	echo
728	IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
729	IGCM_sys_check_quota
730
731	# check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
732	echo
733	IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
734	IGCM_sys_check_path
735
736
737	IGCM_debug_PopStack "IGCM_config_Check"
738	}
739
740	#===================================
741	function IGCM_config_ConfigureExecution
742	{
743	IGCM_debug_PushStack " IGCM_config_ConfigureExecution"
744
745	#echo
746	IGCM_debug_Print 1 " IGCM_config_ConfigureExecution"
747	#echo
748
749	typeset ExeNameIn ExeNameFirst CompNameFirst configCardPath comp i
750	typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
751
752	# config.card path
753	configCardPath=$1
754
755	coreNumber=0
756	mpiTasks=0
757	openMPthreads=0
758	NbExec=0
759
760	OK_PARA_MPI=false
761	OK_PARA_OMP=false
762	OK_PARA_NOD=false
763	OK_PARA_MPMD=false
764
765	for comp in ${config_ListOfComponents[*]} ; do
766
767	# Manage component executable
768	IGCM_card_DefineArrayFromOption ${configCardPath} Executable ${comp}
769
770	eval ExeNameIn=\${config_Executable_${comp}[0]}
771
772	# NO order in config.card for parallelized values !
773	# just use suffix : MPI , OMP and NOD (for number of NODes.)
774
775	# NOD is the number of NODes allocated
776	eval ${comp}_PROC_NOD=0
777
778	# MPI is the number of MPI processus per nodes
779	eval ${comp}_PROC_MPI=0
780
781	# OMP is the number of OpenMP threads per MPI processus
782	eval ${comp}_PROC_OMP=0
783
784	# DEP is the factor to increase the number of cores per process
785	eval ${comp}_PROC_DEP=0
786
787	# DEDICATED is the flag to activate the use of dedicated node for XIOS server only
788	eval ${comp}_OK_DEDICATED=0
789
790	# Only if we really have an executable for the component :
791	if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
792
793	IGCM_debug_Print 1 ${comp}
794
795	# Keep the first executable found and the first CompName
796	eval ExeNameFirst=${ExeNameIn}
797	CompNameFirst=${comp}
798
799	# Are we a second executable?
800	(( NbExec = NbExec + 1 ))
801
802	# set 1 MPI task, 1 OpenMP thread and 1 node as default
803	eval ${comp}_PROC_MPI=1
804	eval ${comp}_PROC_OMP=1
805	eval ${comp}_PROC_NOD=1
806	eval ${comp}_PROC_DEP=1
807
808	eval NbElts=\${#config_Executable_${comp}[@]}
809	if [ ${NbElts} -gt 2 ] ; then
810	#
811	# CURRENT METHOD TO SPECIFY MPI AND OMP RESSOURCES
812	#
813	i=2
814	while [ ${i} -lt ${NbElts} ] ; do
815	eval tempvar=\${config_Executable_${comp}[${i}]}
816	IGCM_debug_Print 2 ${tempvar}
817
818	if [ X${tempvar} = X ] ; then
819	IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
820	IGCM_debug_Exit "Check your config.card. Exit now"
821	IGCM_debug_Verif_Exit
822	fi
823
824	case ${tempvar} in
825	[mM][pP][iI])
826	# Read MPI parameter for composante
827	eval ${comp}_PROC_MPI=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/MPI//" )
828	OK_PARA_MPI=true;;
829	[oO][mM][pP])
830	# Read OMP parameter for composante
831	eval ${comp}_PROC_OMP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/OMP//" )
832	;;
833	[nN][oO][dD])
834	# Read NOD (NumBer of Nodes) parameter for composante
835	eval ${comp}_PROC_NOD=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/NOD//" )
836	OK_PARA_NOD=true
837	OK_PARA_MPI=true
838	;;
839	[dD][eE][pP])
840	# Read DEP parameter for composante
841	eval ${comp}_PROC_DEP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/DEP//" )
842	;;
843	[dD][eE][dD][iI][cC][aA][tT][eE][dD])
844	# Read DEDICATED parameter for composante
845	eval ${comp}_OK_DEDICATED=1
846	esac
847	(( i = i + 1 ))
848	done
849	else
850	#
851	# BACKWARD COMPATIBILITY NOT SUPPORTED ANYMORE
852	#
853	IGCM_debug_Exit "You are using a deprecated ressources specification mechanism in your config.card"
854	IGCM_debug_Exit "Please check : https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocEsetup#ThesectionExecutable"
855	IGCM_debug_Exit "Please modify ${configCardPath}"
856	exit
857	fi
858	if [ ${comp}_PROC_NOD -gt 1 ] ; then
859	if ( [ ${comp}_PROC_MPI -gt 1 ] && [ ${comp}_PROC_OMP -gt 1 ] ) ; then
860	IGCM_debug_Print 2 "Error using MPI/OMP/NOD parameters !!!"
861	IGCM_debug_Exit "It is not allowed to use MPI and OMP and NOD parameters for the same component"
862	IGCM_debug_Exit "Check your config.card. Exit now"
863	IGCM_debug_Verif_Exit
864	fi
865	if [ "${comp}" != "IOS" ] ; then
866	IGCM_debug_Print 2 "Error using NOD parameter !!!"
867	IGCM_debug_Exit "NOD parameter is only available for IOS component "
868	IGCM_debug_Exit "Check your config.card. Exit now"
869	IGCM_debug_Verif_Exit
870	fi
871	fi
872	if ( [ ${comp}_OK_DEDICATED -eq 1 ] && [ "${comp}" != "IOS" ] ) ; then
873	IGCM_debug_Print 2 "Error using DEDICATED parameter !!!"
874	IGCM_debug_Exit "DEDICATED parameter is only available for IOS component "
875	IGCM_debug_Exit "Check your config.card. Exit now"
876	IGCM_debug_Verif_Exit
877	fi
878
879	eval tempvarMPI=\${${comp}_PROC_MPI}
880	eval tempvarNOD=\${${comp}_PROC_NOD}
881	eval tempvarOMP=\${${comp}_PROC_OMP}
882	eval tempvarDEP=\${${comp}_PROC_DEP}
883
884	# set OMP mode if more than 1 OMP thread.
885	[ ${tempvarOMP} -ge 2 ] && OK_PARA_OMP=true
886
887	# Number of OMP threads
888	[ ${openMPthreads} -lt ${tempvarOMP} ] && openMPthreads=${tempvarOMP}
889
890	# SUM UP NUMBER OF CORES
891	if [ ${comp}_PROC_NOD -gt 1 ] ; then
892	(( coreNumber = coreNumber + tempvarNOD * NB_CORE_PER_NODE ))
893	else
894	(( coreNumber = coreNumber + tempvarMPI * tempvarNOD * tempvarOMP * tempvarDEP ))
895	fi
896	# SUM UP NUMBER OF MPI TASKS
897	(( mpiTasks = mpiTasks + tempvarMPI * tempvarNOD ))
898	fi
899	done
900
901	# MANDATORY FOR THE OPA9.DRIVER. USED TO EDIT OPA NAMELIST
902	# WE SHOULD PLANIFY NUM_PROC_??? DEPRECATION
903	NUM_PROC_CPL=${CPL_PROC_MPI}
904	NUM_PROC_OCE=${OCE_PROC_MPI}
905	NUM_PROC_ATM=${ATM_PROC_MPI}
906
907	# set MPMD mode if more than 2 executable names.
908	[ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
909
910	# Define the execution type we are running in
911	if ( ${OK_PARA_MPMD} ) ; then
912	if ( ${OK_PARA_MPI} ) ; then
913	# MPMD always implies MPI
914	executionType=1
915	fi
916	if ( ${OK_PARA_OMP} ) ; then
917	# MPMD + MPI/OMP
918	executionType=2
919	fi
920	else
921	if ( ( ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
922	# SPMD + MPI/OMP
923	executionType=3
924	elif ( ( ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
925	# SPMD + MPI only
926	executionType=4
927	elif ( ( ! ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
928	# SPMD + OMP only
929	executionType=5
930	elif ( ( ! ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
931	# SEQUENTIAL THEN
932	executionType=6
933	coreNumber=1
934	fi
935	fi
936
937	IGCM_debug_Print 1 "MPI/OMP treatment coreNumber = ${coreNumber}"
938	IGCM_debug_Print 1 "MPI/OMP treatment mpiTasks = ${mpiTasks}"
939	IGCM_debug_Print 1 "MPI/OMP treatment openMPthreads = ${openMPthreads}"
940	IGCM_debug_Print 1 "MPI/OMP treatment executionType = ${executionType}"
941
942	IGCM_debug_PopStack "IGCM_config_ConfigureExecution"
943	}
944
945	#===================================
946	function IGCM_config_PeriodStart
947	{
948	IGCM_debug_PushStack "IGCM_config_PeriodStart"
949
950	echo
951	IGCM_debug_Print 1 "IGCM_config_PeriodStart"
952	echo
953
954	if ( ${FirstInitialize} ) ; then
955	#================================================#
956	# Initialize date/period information #
957	#================================================#
958
959	IGCM_date_GetYearMonthDay ${DateBegin} year month day
960	IGCM_config_DaysInPeriodLength
961
962	PeriodDateBegin=${DateBegin}
963	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
964	CumulPeriod=1
965
966	#=================================================#
967	# Write updated run.card #
968	#=================================================#
969
970	#Correct run.card Configuration for this period
971	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
972	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
973	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
974	if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
975	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
976	fi
977
978	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
979
980	else
981	#================================================#
982	# The file run.card allready exist #
983	#================================================#
984
985	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
986	IGCM_config_StateCheck
987	# And EXIT if not OK
988	IGCM_debug_Verif_Exit
989
990	#===================================#
991	# Read updated run.card #
992	#===================================#
993
994	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
995	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
996	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
997
998	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
999	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
1000	CumulPeriod=${run_Configuration_CumulPeriod}
1001
1002	LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
1003
1004	if [ ${Period} = 1 ]; then
1005	# save last Job output and current run.card
1006	typeset Potential
1007	IGCM_sys_Cd ${SUBMIT_DIR}
1008	#
1009	IGCM_debug_Print 2 "Save previous ksh job output"
1010	for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
1011	if [ X${Pack} = Xtrue ] ; then
1012	( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${Potential} ) \|\| IGCM_sys_Cp ${Potential} ${R_BUF_KSH}/${Potential}.$$
1013	else
1014	( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) \|\| IGCM_sys_Put_Out ${Potential} ${R_OUT_KSH}/${Potential}.$$ NOMOVE
1015	IGCM_debug_Print 1 "Save of previous ksh job output is not done for Pack=false"
1016	fi
1017	done
1018
1019	#
1020	IGCM_debug_Print 2 "Save current run.card"
1021	IGCM_card_CheckConflict run.card
1022
1023	if [ X${Pack} = Xtrue ] ; then
1024	IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card
1025	else
1026	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card NOMOVE
1027	IGCM_debug_Print 1 "Save of run.card is not done for Pack=false"
1028	fi
1029	#
1030	IGCM_sys_Cd ${RUN_DIR}
1031	else
1032	unset FileToBeDeleted
1033	fi
1034
1035	# Determine number of day(s) in PeriodLength
1036	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1037	IGCM_config_DaysInPeriodLength
1038
1039	# Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
1040	IGCM_config_DateCoherency
1041	# And EXIT if not OK
1042	IGCM_debug_Verif_Exit
1043
1044	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1045	IGCM_config_StateCheck
1046	# And EXIT if not OK
1047	IGCM_debug_Verif_Exit
1048
1049	# We can say we are "Running" now.
1050	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
1051	fi
1052
1053	# BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
1054	# Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
1055	year_m1=$(( year - 1 ))
1056	year_p1=$(( year + 1 ))
1057	# Compute month_m1 (month minus 1M)
1058	# Compute yyyymm_m1 (yyyymm minus 1M)
1059	month_m1=$(( 10#${month} - 1 ))
1060	if [ ${month_m1} = 0 ]; then
1061	month_m1=12
1062	yyyymm_m1=${year_m1}12
1063	elif [ ${month_m1} -le 9 ]; then
1064	month_m1=0${month_m1}
1065	yyyymm_m1=${year}${month_m1}
1066	else
1067	yyyymm_m1=${year}${month_m1}
1068	fi
1069	# Compute month_p1 (month plus 1M)
1070	# Compute yyyymm_p1 (yyyymm plus 1M)
1071	month_p1=$(( 10#${month} + 1 ))
1072	if [ ${month_p1} = 13 ]; then
1073	month_p1=01
1074	yyyymm_p1=${year_p1}01
1075	elif [ ${month_p1} -le 9 ]; then
1076	month_p1=0${month_p1}
1077	yyyymm_p1=${year}${month_p1}
1078	else
1079	yyyymm_p1=${year}${month_p1}
1080	fi
1081
1082
1083	#===================================================================#
1084	# Calculate CyclicYear to be used for looping over a given forcing #
1085	# period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
1086	#===================================================================#
1087
1088	# To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
1089	# CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
1090	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
1091	CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
1092	#CyclicYear_p1= #NOTDEFINED
1093
1094	# For current year
1095	yeartmp=$year
1096	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
1097	while [ $diffy -lt 0 ] ; do
1098	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
1099	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
1100	done
1101	CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
1102
1103	# For next coming year
1104	CyclicYear_p1=$(( CyclicYear + 1 ))
1105
1106	# For previous year
1107	CyclicYear_m1=$(( CyclicYear - 1 ))
1108
1109	IGCM_debug_Print 1 "CyclicYear = ${CyclicYear}, current year=$year"
1110	IGCM_debug_Print 1 "CyclicYear_p1 = ${CyclicYear_p1}, CyclicYear_m1 = ${CyclicYear_m1} "
1111	else
1112	CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
1113	CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
1114	IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
1115	fi
1116
1117	# END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
1118
1119	#===================================================================#
1120	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
1121	# But available to any son functions #
1122	#===================================================================#
1123
1124	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
1125	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
1126
1127	# Debug Print :
1128	echo
1129	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
1130	IGCM_debug_Print 1 "Year of simulation : ${year}"
1131	IGCM_debug_Print 1 "Month of simulation : ${month}"
1132	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1133	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1134	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1135	IGCM_debug_Print 1 "SimulationLengthInDays : ${SimulationLengthInDays}"
1136	IGCM_debug_Print 1 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
1137
1138	#================================================================#
1139	# Prepare variables available for comp_finalyze #
1140	#================================================================#
1141
1142	# Period for save files
1143	DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
1144
1145	# Prefix for save files of this period
1146	PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
1147
1148	# List of files that will be deleted in RUN_DIR after run
1149	[ -f stack ] && FileToBeDeleted[0]="stack"
1150
1151	# Test if the same run as already been saved :
1152	if [ X${JobType} = XRUN ] ; then
1153	if [ ${DRYRUN} -le 0 ] ; then
1154	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1155	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently re-running an old job."
1156	IGCM_debug_Print 1 "Because of readonly permissions, you can't re-run a job when saved files"
1157	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
1158	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See different clean_*.job in ${libIGCM} directory."
1159	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
1160	IGCM_debug_Verif_Exit
1161	fi
1162	fi
1163	else
1164	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1165	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
1166	fi
1167	fi
1168
1169	#================================================================#
1170	# Prepare variables available for binary execution #
1171	# Call function for creation of run script #
1172	# Only done once per job #
1173	#================================================================#
1174
1175	if [ ${Period} -eq 1 ]; then
1176	# Define the execution context (MPMD, SPMD, MPI/OMP ...)
1177	IGCM_config_ConfigureExecution ${SUBMIT_DIR}/config.card
1178	# Create the execution script for the current context
1179	IGCM_sys_build_execution_scripts
1180	fi
1181
1182	ExecutionFail=false
1183
1184	# Update the rabbitMQ queue
1185	IGCM_debug_BigBro_Update
1186
1187	IGCM_debug_PopStack "IGCM_config_PeriodStart"
1188	}
1189
1190	#===================================
1191	function IGCM_config_SaveSourceModifications
1192	{
1193	IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
1194
1195	typeset ExeOutDateMax listVarEnv
1196	ExeOutDateMax=$1
1197
1198	listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
1199	IGCM_sys_RshMaster "\
1200	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
1201	export ExeOutDateMax=${ExeOutDateMax};\
1202	export R_OUT_EXE=${R_OUT_EXE};\
1203	export PREFIX=${PREFIX};\
1204	export SUBMIT_DIR=${SUBMIT_DIR};\
1205	export listVarEnv=${listVarEnv};\
1206	Script_Output=out_SaveSourceModifications;\
1207	IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
1208
1209	IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
1210	}
1211
1212	#===================================
1213	function IGCM_config_PeriodEnd
1214	{
1215	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
1216
1217	echo
1218	IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
1219	echo
1220
1221	if [ ${DRYRUN} -le 1 ] ; then
1222
1223	IGCM_debug_Print 1 "Check components binary : size and creation date"
1224
1225	typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
1226	typeset comp i
1227	typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
1228
1229	#==================================#
1230	# Get last Exe Size #
1231	#==================================#
1232
1233	(( i=0 ))
1234	if ( ${FirstInitialize} ) ; then
1235	run_Log_LastExeSize=""
1236	for comp in ${config_ListOfComponents[*]} ; do
1237	run_Log_LastExeSize[$i]=0
1238	(( i=i+1 ))
1239	done
1240	else
1241	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
1242	fi
1243	#==================================#
1244	# And Build ExeDate #
1245	#==================================#
1246
1247	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
1248	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
1249	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
1250	ExeDate=""
1251	NextExeSize="( "
1252	(( i=0 ))
1253	UpdateExe=false
1254	(( ExeSecDateMax = 0 ))
1255	for comp in ${config_ListOfComponents[*]} ; do
1256
1257	IGCM_debug_Print 3 ${comp}
1258
1259	eval ExeNameIn=\${config_Executable_${comp}[0]}
1260	eval ExeNameOut=\${config_Executable_${comp}[1]}
1261	# Only if we really have an executable for the component :
1262	if [ X${ExeNameIn} = X\"\" ] ; then
1263	# If there is no exe file for this component
1264	(( ExeSize=0 ))
1265	else
1266	eval ExeNameIn=${ExeNameIn}
1267	LS_bin=${R_EXE}/${ExeNameIn}
1268	IGCM_sys_FileSize ${LS_bin} ExeSize
1269
1270	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
1271	if [ X${ExeDate} = X ] ; then
1272	# First component exe date
1273	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
1274	else
1275	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1276	fi
1277	ExeDate=${ExeDate}_${LS_comp[7]}
1278	fi
1279
1280	if [ ${i} -eq 0 ] ; then
1281	# First component
1282	NextExeSize="( "${ExeSize}
1283	else
1284	NextExeSize=${NextExeSize}", "${ExeSize}
1285	fi
1286	LastCompExeSize=${run_Log_LastExeSize[$i]}
1287	(( i=i+1 ))
1288
1289	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1290	if ( ${FirstInitialize} ) ; then
1291	IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1292	else
1293	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1294	IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
1295	FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1296	fi
1297	IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1298	UpdateExe=true
1299
1300	# SD : switch off for now
1301	#IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1302	#if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1303	# ExeSecDateMax=$ExeSecDate
1304	#fi
1305	fi
1306	done
1307
1308	# SD : switch off for now
1309	#if ( ${UpdateExe} ) ; then
1310	# echo "Launch SaveSourceModifications."
1311	# IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1312	#fi
1313
1314	NextExeSize=${NextExeSize}" )"
1315	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1316
1317	if [ ${DRYRUN} -le 1 ] ; then
1318	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1319	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1320	RET=$?
1321	if [ $RET -eq 0 ] ; then
1322	# ExeCpuLog variable contents 5 fields
1323	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
1324	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
1325	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1326	fi
1327	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
1328	fi
1329
1330	fi
1331
1332	#==================================#
1333	# Save Job output #
1334	#==================================#
1335	if [ X${Pack} = Xtrue ] ; then
1336	IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1337	else
1338	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1339	fi
1340	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1341
1342	# All was right ? no ? then we stop.
1343	IGCM_debug_Verif_Exit
1344
1345	# If all was OK, we can delete all files not necessary for next Job
1346	echo
1347	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1348
1349	if [ ${DRYRUN} -le 2 ] ; then
1350	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1351	else
1352	echo ${FileToBeDeleted[@]}
1353	fi
1354
1355	# Send some accounting element to the user if CumulPeriod=3
1356	if [ ${CumulPeriod} -eq 3 ] ; then
1357	echo
1358	IGCM_debug_Print 1 "Send email containing some accounting information : "
1359
1360	RealCpuTime=$( echo ${ExeCpuLog} \| gawk '{print $3}' )
1361
1362	consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${coreNumber}/3600" \| bc )
1363
1364	consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" \| bc )
1365
1366	recommendedPeriodNb=$( echo "scale=6;${jobWarningDelay}/3600/${consumeHoursPerPeriod}*${coreNumber}" \| bc )
1367
1368	IGCM_sys_SendMail Accounting
1369	fi
1370
1371	#=================================================#
1372	# Modification of libIGCM behaviour #
1373	#=================================================#
1374
1375	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
1376	# and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1377	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1378	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1379	echo
1380	echo "########################################################################"
1381	echo "!!! Modification of libIGCM behaviour !!!"
1382	echo
1383
1384	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
1385	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1386	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1387
1388	# Special treatments for libIGCM internals
1389	for option in ${libIGCM_UserChanges[*]} ; do
1390	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1391
1392	echo "We will change : ${option}."
1393	eval echo "Previous value : " \${${option}}
1394	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1395
1396	eval ${option}=\${libIGCM_UserChanges_${option}}
1397
1398	case ${option} in
1399	config_UserChoices_DateEnd)
1400	IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1401	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1402
1403	# Period Length In Days between DateBegin and DateEnd
1404	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
1405	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1406	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1407	IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
1408	IGCM_debug_Verif_Exit
1409	fi
1410	;;
1411	config_UserChoices_PeriodLength)
1412	IGCM_debug_Print 1 "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1413	;;
1414	PeriodNb)
1415	IGCM_debug_Print 1 "Loop in main Job with ${PeriodNb} period(s)"
1416	;;
1417	config_Post_RebuildFrequency)
1418	IGCM_debug_Print 1 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1419	IGCM_post_Configure
1420	;;
1421	config_Post_TimeSeriesFrequency)
1422	IGCM_debug_Print 1 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1423	IGCM_post_Configure
1424	;;
1425	config_Post_SeasonalFrequency)
1426	IGCM_debug_Print 1 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1427	IGCM_post_Configure
1428	;;
1429	esac
1430	done
1431
1432	echo
1433	echo "########################################################################"
1434	echo
1435	fi
1436
1437	#=================================================#
1438	# Determine next computed period #
1439	#=================================================#
1440
1441	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1442	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1443	year_m1=$(( year - 1 ))
1444	year_p1=$(( year + 1 ))
1445	IGCM_config_DaysInPeriodLength
1446	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1447
1448	# Debug Print :
1449	echo
1450	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1451	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1452	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1453	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1454
1455	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1456	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1457
1458	(( CumulPeriod = CumulPeriod + 1 ))
1459
1460	# Debug Print :
1461	echo
1462	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1463	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
1464	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
1465
1466	#=================================================#
1467	# Write updated run.card #
1468	#=================================================#
1469
1470	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1471	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1472	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1473
1474	if ( ${FirstInitialize} ) ; then
1475	# It's no more the first time
1476	FirstInitialize=false
1477	fi
1478
1479	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1480	}
1481
1482	#===================================
1483	function IGCM_config_Finalize
1484	{
1485	IGCM_debug_PushStack "IGCM_config_Finalize"
1486
1487	echo
1488	IGCM_debug_Print 1 "IGCM_config_Finalize"
1489	echo
1490
1491	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1492	IGCM_config_StateCheck
1493
1494	# And EXIT if not OK
1495	IGCM_debug_Verif_Exit
1496
1497	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1498	#==========================#
1499	# End of entire simulation #
1500	#==========================#
1501	simulationIsOver=true
1502
1503	# Mail notification
1504	IGCM_sys_SendMail
1505	#
1506	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1507	#
1508	IGCM_debug_Print 1 "Normal End of computation."
1509
1510	else
1511	#=================#
1512	# Submit next job #
1513	#=================#
1514	simulationIsOver=false
1515
1516	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1517
1518	# Name of next Ksh Script output :
1519	Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
1520
1521	IGCM_debug_Print 1 "Submit next job"
1522	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1523	IGCM_sys_Cd ${SUBMIT_DIR}
1524	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1525	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null \| wc -l )
1526	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
1527	# Submit next job and come back
1528	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1529	IGCM_sys_Cd -
1530	fi
1531
1532	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1533	# Only for production run (No clean up in DEV or DEB mode)
1534	# and command sent from .. directory.
1535	IGCM_sys_Cd ..
1536	[ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
1537
1538	# Inform the rabbitMQ queue
1539	IGCM_debug_BigBro_Finalize
1540
1541	IGCM_debug_PopStack "IGCM_config_Finalize"
1542	}
1543
1544	#===================================

Note: See TracBrowser for help on using the repository browser.

Download in other formats: