Context Navigation

libIGCM_config.ksh @ 1396

Last change on this file since 1396 was 1395, checked in by sdipsl, 7 years ago

Add a LightRestartPack? option. Section [Post] in config.card. If TRUE will save only the latest restart files set. See #302

It is false by default for ascendant compatibility

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 53.4 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_CommonConfiguration
16	{
17	IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19	# Debug Print :
20	[ ${Verbosity} -gt 0 ] && echo
21	IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23	# config.card path
24	configCardPath=$1
25
26	#==================================
27	typeset option auxprint
28
29	#==================================
30	# Read UserChoices section:
31	[ ${Verbosity} -gt 0 ] && echo
32	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
33
34	IGCM_card_DefineArrayFromSection ${configCardPath} UserChoices
35	for option in ${config_UserChoices[*]} ; do
36	IGCM_card_DefineVariableFromOption ${configCardPath} UserChoices ${option}
37	eval auxprint=\${config_UserChoices_${option}}
38	IGCM_debug_Print 3 "${option} : ${auxprint}"
39	done
40
41	#==================================
42	# Read Ensemble section:
43	[ ${Verbosity} -gt 0 ] && echo
44	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
45
46	IGCM_card_DefineArrayFromSection ${configCardPath} Ensemble
47	for option in ${config_Ensemble[*]} ; do
48	IGCM_card_DefineVariableFromOption ${configCardPath} Ensemble ${option}
49	eval auxprint=\${config_Ensemble_${option}}
50	IGCM_debug_Print 3 "${option} : ${auxprint}"
51	done
52
53	#==================================
54	# Read Post section:
55	[ ${Verbosity} -gt 0 ] && echo
56	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
57
58	IGCM_card_DefineArrayFromSection ${configCardPath} Post
59	for option in ${config_Post[*]} ; do
60	IGCM_card_DefineVariableFromOption ${configCardPath} Post ${option}
61	eval auxprint=\${config_Post_${option}}
62	IGCM_debug_Print 3 "${option} : ${auxprint}"
63	done
64	[ ${Verbosity} -gt 0 ] && echo
65
66	#==================================
67	# Define default value to keep compatibility with previous card: means before changes due to TGCC
68	# Apply some overrules to ensure proper usage of computing centres resources
69	#
70	if [ X${PackDefault} = Xtrue ] ; then
71	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
72	# TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
73	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
74	config_Post_PackFrequency=NONE
75	else
76	# Default to RebuildFrequency if nothing has been set up related to PackFrequency
77	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
78	fi
79	else
80	# If we DO NOT apply pack in this computing center
81	config_Post_PackFrequency=NONE
82	fi
83
84	#====================================================
85	# Define ARCHIVE : Dedicated to large files
86	# Define STORAGE : Dedicated to small/medium files
87	# Define R_OUT : Output tree located on ARCHIVE
88	# Define R_BUF : Output tree located on STORAGE (files waiting treatment, or file lcoation when SpaceName=!PROD)
89	# Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
90	# Define R_TMP : A temporary space used by IGCM_debug_send_AMQP_msg__MAILTUNNEL. Must be persistent in between jobs
91	IGCM_sys_defineArchives
92
93	#====================================================
94	# R_SAVE : Job output directory
95	# R_BUFR : Job output buffered directory
96
97	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
98	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
99	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
100	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
101	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
102	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
103	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
104	else
105	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
106	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
107	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
108	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
109	fi
110	else
111	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
112	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
113	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
114	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
115	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
116	else
117	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
118	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
119	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
120	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
121	fi
122	fi
123
124	#====================================================
125	# Define R_OUT_KSH : Storage place for job output
126	# Define R_OUT_EXE : Storage place for binary used during simulation
127	R_OUT_KSH=${R_SAVE}/Out
128	R_OUT_EXE=${R_SAVE}/Exe
129
130	#====================================================
131	# Define R_BUF_KSH : Buffer place for job output
132	# Define R_BUF_EXE : Buffer place for binary used during simulation
133	R_BUF_KSH=${R_BUFR}/Out
134	R_BUF_EXE=${R_BUFR}/Exe
135
136	#====================================================
137	# Define REBUILD_DIR : where we store files needing rebuild process
138	REBUILD_DIR=${R_BUFR}/REBUILD
139	if [ ! X${TaskType} = Xchecking ] ; then
140	IGCM_sys_MkdirWork ${REBUILD_DIR}
141	fi
142
143	#====================================================
144	# DodsCopy : apply default value if not defined
145	if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] \|\| [ X${config_Post_DodsCopy} = X ] ) ; then
146	config_Post_DodsCopy=TRUE
147	fi
148
149	#====================================================
150	# AtlasIPSL : apply default value if not defined
151	if ( [ X${config_Post_AtlasIPSL} = X${NULL_STR} ] \|\| [ X${config_Post_AtlasIPSL} = X ] ) ; then
152	config_Post_AtlasIPSL=TRUE
153	fi
154
155	#====================================================
156	# ParserXIOS : apply default value if not defined
157	if ( [ X${config_Post_ParserXIOS} = X${NULL_STR} ] \|\| [ X${config_Post_ParserXIOS} = X ] ) ; then
158	config_Post_ParserXIOS=TRUE
159	fi
160
161	#====================================================
162	# MetricsPCMDI : apply default value if not defined
163	if ( [ X${config_Post_MetricsPCMDI} = X${NULL_STR} ] \|\| [ X${config_Post_MetricsPCMDI} = X ] ) ; then
164	config_Post_MetricsPCMDI=FALSE
165	fi
166
167	#====================================================
168	# IgnoreNonMonotonic : apply default value if not defined
169	if ( [ X${config_Post_IgnoreNonMonotonic} = X${NULL_STR} ] \|\| [ X${config_Post_IgnoreNonMonotonic} = X ] ) ; then
170	config_Post_IgnoreNonMonotonic=FALSE
171	fi
172
173	#====================================================
174	# LightRestartPack : apply default value if not defined
175	if ( [ X${config_Post_LightRestartPack} = X${NULL_STR} ] \|\| [ X${config_Post_LightRestartPack} = X ] ) ; then
176	config_Post_LightRestartPack=FALSE
177	fi
178
179	#====================================================
180	# Define StackFileLocation : directory where we store stack files
181	# Define StackFileName : stack file containing call tree and instrumentation
182	# Stack file containing call tree will be stored there.
183	if ( $DEBUG_debug ) ; then
184	StackFileLocation=${StackFileLocation:=${R_BUF_KSH}}
185	[ ! -d ${StackFileLocation} ] && mkdir -p ${StackFileLocation}
186	if [ X${TaskType} = Xcomputing ]; then
187	StackFileName=computing.stack.$$
188	elif [ X${TaskType} = Xpost-processing ]; then
189	StackFileName=${Script_Post_Output}.stack.$$
190	elif [ X${TaskType} = Xchecking ]; then
191	StackFileName=checking.stack.$$
192	else
193	IGCM_debug_Exit "IGCM_config_CommonConfiguration unknown TaskType : ${TaskType}"
194	IGCM_debug_Verif_Exit
195	fi
196
197	# This boolean will trigger the filling of the stack
198	# Only now we know where things should be ...
199	# We don't fill the stack when we perform checking task
200	if [ ! X${TaskType} = Xchecking ] ; then
201	ActivateStackFilling=true
202	fi
203	fi
204
205	IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
206	}
207
208	#===================================
209	function IGCM_config_Initialize
210	{
211	IGCM_debug_PushStack "IGCM_config_Initialize"
212
213	# Debug Print :
214	echo
215	IGCM_debug_Print 1 "IGCM_config_Initialize"
216
217	# Test modipsl tree existence.
218	IGCM_sys_TestDir ${MODIPSL}
219	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
220	IGCM_sys_TestDir ${libIGCM}
221	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
222	IGCM_sys_TestDir ${R_EXE}
223	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
224	IGCM_sys_TestDir ${SUBMIT_DIR}
225	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
226
227	if ( $DEBUG_debug ) ; then
228	echo "Keep trace of inital SUBMIT_DIR : "
229	ls -lta ${SUBMIT_DIR}
230	fi
231
232	#==================================
233	# Read ListOfComponents section:
234	echo
235	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
236
237	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
238	for comp in ${config_ListOfComponents[*]} ; do
239	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
240	done
241	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
242
243	#==================================
244	# Read Executable section:
245	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
246
247	#==================================
248	# Read Restarts section:
249	# Restarts : Gerneral rule or local for each component.
250	echo
251	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
252
253	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
254	for option in ${config_Restarts[*]} ; do
255	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
256	eval auxprint=\${config_Restarts_${option}}
257	IGCM_debug_Print 3 "${option} : ${auxprint}"
258	done
259
260	#==================================
261	# Define Job Outputs Name
262	echo
263	IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
264	Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
265	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
266	Exe_Output=out_execution
267	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
268
269	#===================================================================#
270	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
271	# But available to any son functions #
272	#===================================================================#
273
274	# Convert yyyy-mm-dd date to gregorian yyyymmdd
275	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
276	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
277
278	# Period Length In Days between DateBegin and DateEnd
279	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
280	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
281	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
282	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
283	IGCM_debug_Verif_Exit
284	fi
285
286	# Day and Year of Initial State (Given in julian format)
287	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
288	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
289
290	#================================================================#
291	# Test and Prepare directories #
292	#================================================================#
293
294	# ==> 4 kinds of input files :
295	# 1) R_INIT : Initial State Files (Etat0, carteveg)
296	# 2) R_BC : Boundary Conditions (Forcages, lai)
297	# 3) Parameters files (allready define through ${SUBMIT_DIR})
298	# 4) Restarts files (allready define in IGCM_config_Initialize)
299
300	# Here we offer the possibility to redefine R_INIT, R_BC
301	# and PeriodNb through config.card
302	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
303	echo
304	IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
305	IGCM_debug_Print 3 "R_IN=${R_IN}"
306	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
307	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
308	IGCM_debug_Print 3 "R_BC=${R_BC}"
309	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
310	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
311
312	# SD ADA SPECIFIC #
313	# TO FIX #
314	#IGCM_sys_TestDirArchive ${R_IN}
315	#[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
316
317	if ( ${FirstInitialize} ) ; then
318	IGCM_sys_MkdirArchive ${R_SAVE}
319	[ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
320	else
321	IGCM_sys_TestDirArchive ${R_SAVE}
322	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
323
324	if [ ! ${config_Post_PackFrequency} = NONE ] ; then
325	IGCM_sys_TestDir ${R_BUFR}
326	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
327	fi
328
329	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
330	IGCM_config_StateCheck
331
332	# And EXIT if not OK
333	IGCM_debug_Verif_Exit
334	fi
335
336	#====================================================
337	# Experience type : DEB(ug), DEV(elopment), RUN
338	if [ X${JobType} != XRUN ] ; then
339	echo
340	echo "===================================================="
341	echo "libIGCM JOB is NOT in RUN type mode."
342	echo "!! OUTPUT files will NOT be PROTECTED !!"
343	echo "Be carefull : you can ERASE the result of this job !"
344
345	case ${JobType} in
346	DEB)
347	echo "DEBUG mode : activation of 'set -vx' mode."
348	echo "DEBUG mode : no protection for output files."
349	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
350	;;
351	DEV)
352	echo "DEVelopment mode : no protection for output files."
353	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
354	;;
355	esac
356
357	if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
358	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
359	echo "------------"
360	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
361	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
362	echo "------------"
363	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
364	fi
365	fi
366	echo "===================================================="
367	echo
368	fi
369
370	IGCM_debug_PopStack "IGCM_config_Initialize"
371	}
372
373	#===================================
374	function IGCM_config_DaysInPeriodLength
375	{
376	IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
377
378	typeset i
379
380	# Determine number of day(s) in PeriodLength :
381	case ${config_UserChoices_PeriodLength} in
382	Y\|y)
383	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
384	echo
385	IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
386	PeriodLengthInDays=0
387	i=0
388	until [ $i -ge $PeriodLengthInYears ] ; do
389	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
390	(( i=i+1 ))
391	done
392	;;
393	M\|m)
394	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
395	echo
396	IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
397	PeriodLengthInDays=0
398	i=0
399	until [ $i -ge $PeriodLengthInMonths ] ; do
400	if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
401	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
402	else
403	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
404	fi
405	(( i=i+1 ))
406	done
407	;;
408	D\|d)
409	PeriodLengthInMonths=0
410	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' )
411	echo
412	IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
413	*)
414	IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
415	IGCM_debug_Verif_Exit ;;
416	esac
417
418	IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
419	}
420
421	#===================================
422	function IGCM_config_DateCoherency
423	{
424	IGCM_debug_PushStack "IGCM_config_DateCoherency"
425
426	echo
427	IGCM_debug_Print 1 "IGCM_config_DateCoherency"
428	echo
429
430	typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
431
432	# check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
433	# DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
434	echo
435
436	case ${config_UserChoices_PeriodLength} in
437	Y\|y)
438	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
439	;;
440	M\|m)
441	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
442	;;
443	D\|d)
444	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
445	;;
446	esac
447	VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
448
449	if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
450	IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
451	IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
452	IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
453	IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
454	IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
455	IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
456
457	IGCM_debug_Exit "STOP here to avoid further issues."
458	fi
459
460	# PeriodDateBegin + PeriodLength = PeriodDateEnd
461	VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
462
463	IGCM_debug_PopStack "IGCM_config_DateCoherency"
464	}
465
466	#===================================
467	function IGCM_config_StateCheck
468	{
469	IGCM_debug_PushStack "IGCM_config_StateCheck"
470
471	#Test state of run in run.card
472	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
473
474	if [ ${run_Configuration_PeriodState} = "Fatal" ] ; then
475	echo
476	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
477	IGCM_debug_Print 1 "Check the overall status of your simulation by visiting this page:"
478	IGCM_debug_Print 1 "http://prodiguer-test-web.ipsl.fr/static/simulation.detail.html?uid=${simuid}"
479	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
480	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean lastest pack period."
481	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
482	IGCM_debug_Exit
483	elif [ $( echo ${run_Configuration_PeriodState} \| grep Fatal \| wc -l ) -eq 1 ] ; then
484	echo
485	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
486	IGCM_debug_Print 1 "Compute jobs has been stop because at least the above mentionned post-processing jobs fails."
487	IGCM_debug_Print 1 "Check post-processing jobs carefully by visiting this page:"
488	IGCM_debug_Print 1 "http://prodiguer-test-web.ipsl.fr/static/simulation.detail.html?uid=${simuid}"
489	IGCM_debug_Print 1 "Please visit that page to see how to fix issues:"
490	IGCM_debug_Print 1 "https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocGmonitor"
491	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
492	IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean latest pack period."
493	IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
494	IGCM_debug_Exit
495	fi
496
497	IGCM_debug_PopStack "IGCM_config_StateCheck"
498	}
499
500	#===================================
501	function IGCM_config_Check
502	{
503	IGCM_debug_PushStack "IGCM_config_Check"
504
505	# If one of the following modulo is not zero :
506	# we will issue an error then explain and exit in
507	# AA_job IGCM_debug_Verif_Exit call before binary submission
508
509	echo
510	IGCM_debug_Print 1 "IGCM_config_Check"
511	echo
512
513	typeset i
514
515	# Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
516	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
517	AsynchronousRebuild=true
518	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
519	echo
520	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
521	IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
522	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
523	IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
524	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
525	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
526	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
527	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
528	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
529	else
530	AsynchronousRebuild=false
531	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
532	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
533	echo
534	#modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
535	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
536	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
537	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
538	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
539	fi
540
541	# Check PackFrequency against other key frequencies
542	# Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
543	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
544	Pack=true
545	#
546	IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
547	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
548	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
549	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
550	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
551	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
552	else
553	Pack=false
554	fi
555
556	# modulo (TimeSeriesFrequency and all Chunck2D) must be zero
557	NbJob=${#CHUNCK2D_SIZE[@]}
558	i=0
559	until [ $i -ge $NbJob ]; do
560	value=${CHUNCK2D_SIZE[${i}]}
561	IGCM_debug_Print 1 "Check coherence between ${CHUNCK2D_NAME[${i}]} Chunck2D frequency and TimeSeriesFrequency"
562	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
563	case ${value} in
564	Y\|y) ;;
565	*)
566	IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year Y\|y in comp.card"
567	IGCM_debug_Exit "This will stop the job" ;;
568	esac
569	(( i=i+1 ))
570	done
571
572	# modulo (TimeSeriesFrequency and all Chunck3D) must be zero
573	NbJob=${#CHUNCK3D_SIZE[@]}
574	i=0
575	until [ $i -ge $NbJob ]; do
576	value=${CHUNCK3D_SIZE[${i}]}
577	IGCM_debug_Print 1 "Check coherence between ${CHUNCK3D_NAME[${i}]} Chunck3D frequency and TimeSeriesFrequency"
578	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
579	case ${value} in
580	Y\|y) ;;
581	*)
582	IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year Y\|y in comp.card"
583	IGCM_debug_Exit "This will stop the job" ;;
584	esac
585	(( i=i+1 ))
586	done
587
588	# check to be sure there is enough space on temporary filesystems to run
589	echo
590	IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
591	IGCM_sys_check_quota
592
593	# check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
594	echo
595	IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
596	IGCM_sys_check_path
597
598
599	IGCM_debug_PopStack "IGCM_config_Check"
600	}
601
602	#===================================
603	function IGCM_config_ConfigureExecution
604	{
605	IGCM_debug_PushStack " IGCM_config_ConfigureExecution"
606
607	#echo
608	IGCM_debug_Print 1 " IGCM_config_ConfigureExecution"
609	#echo
610
611	typeset ExeNameIn ExeNameFirst CompNameFirst configCardPath comp i
612	typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
613
614	# config.card path
615	configCardPath=$1
616
617	coreNumber=0
618	mpiTasks=0
619	openMPthreads=0
620	NbExec=0
621
622	OK_PARA_MPI=false
623	OK_PARA_OMP=false
624	OK_PARA_NOD=false
625	OK_PARA_MPMD=false
626
627	for comp in ${config_ListOfComponents[*]} ; do
628
629	# Manage component executable
630	IGCM_card_DefineArrayFromOption ${configCardPath} Executable ${comp}
631
632	eval ExeNameIn=\${config_Executable_${comp}[0]}
633
634	# NO order in config.card for parallelized values !
635	# just use suffix : MPI , OMP and NOD (for number of NODes.)
636
637	# NOD is the number of NODes allocated
638	eval ${comp}_PROC_NOD=0
639
640	# MPI is the number of MPI processus per nodes
641	eval ${comp}_PROC_MPI=0
642
643	# OMP is the number of OpenMP threads per MPI processus
644	eval ${comp}_PROC_OMP=0
645
646	# Only if we really have an executable for the component :
647	if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
648
649	IGCM_debug_Print 1 ${comp}
650
651	# Keep the first executable found and the first CompName
652	ExeNameFirst=${ExeNameIn}
653	CompNameFirst=${comp}
654
655	# Are we a second executable?
656	(( NbExec = NbExec + 1 ))
657
658	# set 1 MPI task, 1 OpenMP thread and 1 node as default
659	eval ${comp}_PROC_MPI=1
660	eval ${comp}_PROC_OMP=1
661	eval ${comp}_PROC_NOD=1
662
663	eval NbElts=\${#config_Executable_${comp}[@]}
664
665	if [ ${NbElts} -gt 2 ] ; then
666	#
667	# CURRENT METHOD TO SPECIFY MPI AND OMP RESSOURCES
668	#
669	i=2
670	while [ ${i} -lt ${NbElts} ] ; do
671	eval tempvar=\${config_Executable_${comp}[${i}]}
672	IGCM_debug_Print 2 ${tempvar}
673
674	if [ X${tempvar} = X ] ; then
675	IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
676	IGCM_debug_Exit "Check your config.card. Exit now"
677	IGCM_debug_Verif_Exit
678	fi
679
680	case ${tempvar} in
681	[mM][pP][iI])
682	# Read MPI parameter for composante
683	eval ${comp}_PROC_MPI=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/MPI//" )
684	OK_PARA_MPI=true;;
685	[oO][mM][pP])
686	# Read OMP parameter for composante
687	eval ${comp}_PROC_OMP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/OMP//" )
688	;;
689	[nN][oO][dD])
690	# Read NOD (NumBer of Nodes) parameter for composante
691	eval ${comp}_PROC_NOD=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/NOD//" )
692	OK_PARA_NOD=true
693	OK_PARA_MPI=true
694	;;
695	esac
696	(( i = i + 1 ))
697	done
698	else
699	#
700	# BACKWARD COMPATIBILITY NOT SUPPORTED ANYMORE
701	#
702	IGCM_debug_Exit "You are using a deprecated ressources specification mechanism in your config.card"
703	IGCM_debug_Exit "Please check : https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocEsetup#ThesectionExecutable"
704	IGCM_debug_Exit "Please modify ${configCardPath}"
705	exit
706	fi
707	eval tempvarMPI=\${${comp}_PROC_MPI}
708	eval tempvarNOD=\${${comp}_PROC_NOD}
709	eval tempvarOMP=\${${comp}_PROC_OMP}
710
711	# set OMP mode if more than 1 OMP thread.
712	[ ${tempvarOMP} -ge 2 ] && OK_PARA_OMP=true
713
714	# Number of OMP threads
715	[ ${openMPthreads} -lt ${tempvarOMP} ] && openMPthreads=${tempvarOMP}
716
717	# SUM UP NUMBER OF CORES
718	(( coreNumber = coreNumber + tempvarMPI * tempvarNOD * tempvarOMP ))
719
720	# SUM UP NUMBER OF MPI TASKS
721	(( mpiTasks = mpiTasks + tempvarMPI * tempvarNOD ))
722	fi
723	done
724
725	# MANDATORY FOR THE OPA9.DRIVER. USED TO EDIT OPA NAMELIST
726	# WE SHOULD PLANIFY NUM_PROC_??? DEPRECATION
727	NUM_PROC_CPL=${CPL_PROC_MPI}
728	NUM_PROC_OCE=${OCE_PROC_MPI}
729	NUM_PROC_ATM=${ATM_PROC_MPI}
730
731	# set MPMD mode if more than 2 executable names.
732	[ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
733
734	# Define the execution type we are running in
735	if ( ${OK_PARA_MPMD} ) ; then
736	if ( ${OK_PARA_MPI} ) ; then
737	# MPMD always implies MPI
738	executionType=1
739	fi
740	if ( ${OK_PARA_OMP} ) ; then
741	# MPMD + MPI/OMP
742	executionType=2
743	fi
744	else
745	if ( ( ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
746	# SPMD + MPI/OMP
747	executionType=3
748	elif ( ( ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
749	# SPMD + MPI only
750	executionType=4
751	elif ( ( ! ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
752	# SPMD + OMP only
753	executionType=5
754	elif ( ( ! ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
755	# SEQUENTIAL THEN
756	executionType=6
757	coreNumber=1
758	fi
759	fi
760
761	IGCM_debug_Print 1 "MPI/OMP treatment coreNumber = ${coreNumber}"
762	IGCM_debug_Print 1 "MPI/OMP treatment mpiTasks = ${mpiTasks}"
763	IGCM_debug_Print 1 "MPI/OMP treatment openMPthreads = ${openMPthreads}"
764	IGCM_debug_Print 1 "MPI/OMP treatment executionType = ${executionType}"
765
766	IGCM_debug_PopStack "IGCM_config_ConfigureExecution"
767	}
768
769	#===================================
770	function IGCM_config_PeriodStart
771	{
772	IGCM_debug_PushStack "IGCM_config_PeriodStart"
773
774	echo
775	IGCM_debug_Print 1 "IGCM_config_PeriodStart"
776	echo
777
778	if ( ${FirstInitialize} ) ; then
779	#================================================#
780	# Initialize date/period information #
781	#================================================#
782
783	IGCM_date_GetYearMonthDay ${DateBegin} year month day
784	IGCM_config_DaysInPeriodLength
785
786	PeriodDateBegin=${DateBegin}
787	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
788	CumulPeriod=1
789
790	#=================================================#
791	# Write updated run.card #
792	#=================================================#
793
794	#Correct run.card Configuration for this period
795	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
796	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
797	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
798	if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
799	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
800	fi
801
802	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
803
804	else
805	#================================================#
806	# The file run.card allready exist #
807	#================================================#
808
809	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
810	IGCM_config_StateCheck
811	# And EXIT if not OK
812	IGCM_debug_Verif_Exit
813
814	#===================================#
815	# Read updated run.card #
816	#===================================#
817
818	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
819	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
820	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
821
822	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
823	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
824	CumulPeriod=${run_Configuration_CumulPeriod}
825
826	LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
827
828	if [ ${Period} = 1 ]; then
829	# save last Job output and current run.card
830	typeset Potential
831	IGCM_sys_Cd ${SUBMIT_DIR}
832	#
833	IGCM_debug_Print 2 "Save previous ksh job output"
834	for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
835	if [ X${Pack} = Xtrue ] ; then
836	( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${Potential} ) \|\| IGCM_sys_PutBuffer_Out ${Potential} ${R_BUF_KSH}/${Potential}.$$ NOCHMOD
837	else
838	( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) \|\| IGCM_sys_Put_Out ${Potential} ${R_OUT_KSH}/${Potential}.$$ NOCHMOD
839	fi
840	done
841	#
842	IGCM_debug_Print 2 "Save current run.card"
843	IGCM_card_CheckConflict run.card
844
845	if [ X${Pack} = Xtrue ] ; then
846	IGCM_sys_PutBuffer_Out ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/${PREFIX}_${Exe_Output} NOCHMOD
847	else
848	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/${PREFIX}_${Exe_Output} NOCHMOD
849	fi
850	#
851	IGCM_sys_Cd ${RUN_DIR}
852	else
853	unset FileToBeDeleted
854	fi
855
856	# Determine number of day(s) in PeriodLength
857	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
858	IGCM_config_DaysInPeriodLength
859
860	# Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
861	IGCM_config_DateCoherency
862	# And EXIT if not OK
863	IGCM_debug_Verif_Exit
864
865	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
866	IGCM_config_StateCheck
867	# And EXIT if not OK
868	IGCM_debug_Verif_Exit
869
870	# We can say we are "Running" now.
871	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
872	fi
873
874	# BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
875	# Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
876	year_m1=$(( year - 1 ))
877	year_p1=$(( year + 1 ))
878	# Compute month_m1 (month minus 1M)
879	# Compute yyyymm_m1 (yyyymm minus 1M)
880	month_m1=$(( 10#${month} - 1 ))
881	if [ ${month_m1} = 0 ]; then
882	month_m1=12
883	yyyymm_m1=${year_m1}12
884	elif [ ${month_m1} -le 9 ]; then
885	month_m1=0${month_m1}
886	yyyymm_m1=${year}${month_m1}
887	else
888	yyyymm_m1=${year}${month_m1}
889	fi
890	# Compute month_p1 (month plus 1M)
891	# Compute yyyymm_p1 (yyyymm plus 1M)
892	month_p1=$(( 10#${month} + 1 ))
893	if [ ${month_p1} = 13 ]; then
894	month_p1=01
895	yyyymm_p1=${year_p1}01
896	elif [ ${month_p1} -le 9 ]; then
897	month_p1=0${month_p1}
898	yyyymm_p1=${year}${month_p1}
899	else
900	yyyymm_p1=${year}${month_p1}
901	fi
902
903	#===================================================================#
904	# Calculate CyclicYear to be used for looping over a given forcing #
905	# period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
906	#===================================================================#
907
908	# To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
909	# CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
910	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
911	CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
912	CyclicYear_p1=NOTDEFINED
913
914	# For current year
915	yeartmp=$year
916	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
917	while [ $diffy -lt 0 ] ; do
918	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
919	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
920	done
921	CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
922
923	# For next coming year
924	yeartmp=$(( $year + 1 ))
925	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
926	while [ $diffy -lt 0 ] ; do
927	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
928	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
929	done
930	CyclicYear_p1=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
931
932	IGCM_debug_Print 1 "CyclicYear = ${CyclicYear}, CyclicYear_p1 = ${CyclicYear_p1}, current year=$year"
933	else
934	CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
935	CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
936	IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
937	fi
938
939	# END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
940
941	#===================================================================#
942	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
943	# But available to any son functions #
944	#===================================================================#
945
946	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
947	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
948
949	# Debug Print :
950	echo
951	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
952	IGCM_debug_Print 1 "Year of simulation : ${year}"
953	IGCM_debug_Print 1 "Month of simulation : ${month}"
954	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
955	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
956	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
957	IGCM_debug_Print 1 "SimulationLengthInDays : ${SimulationLengthInDays}"
958	IGCM_debug_Print 1 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
959
960	#================================================================#
961	# Prepare variables available for comp_finalyze #
962	#================================================================#
963
964	# Period for save files
965	DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
966
967	# Prefix for save files of this period
968	PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
969
970	# List of files that will be deleted in RUN_DIR after run
971	[ -f stack ] && FileToBeDeleted[0]="stack"
972
973	# Test if the same run as already been saved :
974	if [ X${JobType} = XRUN ] ; then
975	if [ ${DRYRUN} -le 0 ] ; then
976	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
977	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently re-running an old job."
978	IGCM_debug_Print 1 "Because of readonly permissions, you can't re-run a job when saved files"
979	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
980	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See different clean_*.job in ${libIGCM} directory."
981	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
982	IGCM_debug_Verif_Exit
983	fi
984	fi
985	else
986	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
987	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
988	fi
989	fi
990
991	#================================================================#
992	# Prepare variables available for binary execution #
993	# Call function for creation of run script #
994	# Only done once per job #
995	#================================================================#
996
997	if [ ${Period} -eq 1 ]; then
998	# Define the execution context (MPMD, SPMD, MPI/OMP ...)
999	IGCM_config_ConfigureExecution ${SUBMIT_DIR}/config.card
1000	# Create the execution script for the current context
1001	IGCM_sys_build_execution_scripts
1002	fi
1003
1004	ExecutionFail=false
1005
1006	# Update the rabbitMQ queue
1007	IGCM_debug_BigBro_Update
1008
1009	IGCM_debug_PopStack "IGCM_config_PeriodStart"
1010	}
1011
1012	#===================================
1013	function IGCM_config_SaveSourceModifications
1014	{
1015	IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
1016
1017	typeset ExeOutDateMax listVarEnv
1018	ExeOutDateMax=$1
1019
1020	listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
1021	IGCM_sys_RshMaster "\
1022	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
1023	export ExeOutDateMax=${ExeOutDateMax};\
1024	export R_OUT_EXE=${R_OUT_EXE};\
1025	export PREFIX=${PREFIX};\
1026	export SUBMIT_DIR=${SUBMIT_DIR};\
1027	export listVarEnv=${listVarEnv};\
1028	Script_Output=out_SaveSourceModifications;\
1029	IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
1030
1031	IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
1032	}
1033
1034	#===================================
1035	function IGCM_config_PeriodEnd
1036	{
1037	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
1038
1039	echo
1040	IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
1041	echo
1042
1043	if [ ${DRYRUN} -le 1 ] ; then
1044
1045	IGCM_debug_Print 1 "Check components binary : size and creation date"
1046
1047	typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
1048	typeset comp i
1049	typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
1050
1051	#==================================#
1052	# Get last Exe Size #
1053	#==================================#
1054
1055	(( i=0 ))
1056	if ( ${FirstInitialize} ) ; then
1057	run_Log_LastExeSize=""
1058	for comp in ${config_ListOfComponents[*]} ; do
1059	run_Log_LastExeSize[$i]=0
1060	(( i=i+1 ))
1061	done
1062	else
1063	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
1064	fi
1065	#==================================#
1066	# And Build ExeDate #
1067	#==================================#
1068
1069	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
1070	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
1071	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
1072	ExeDate=""
1073	NextExeSize="( "
1074	(( i=0 ))
1075	UpdateExe=false
1076	(( ExeSecDateMax = 0 ))
1077	for comp in ${config_ListOfComponents[*]} ; do
1078
1079	IGCM_debug_Print 3 ${comp}
1080
1081	eval ExeNameIn=\${config_Executable_${comp}[0]}
1082	eval ExeNameOut=\${config_Executable_${comp}[1]}
1083	# Only if we really have an executable for the component :
1084	if [ X${ExeNameIn} = X\"\" ] ; then
1085	# If there is no exe file for this component
1086	(( ExeSize=0 ))
1087	else
1088	LS_bin=${R_EXE}/${ExeNameIn}
1089	IGCM_sys_FileSize ${LS_bin} ExeSize
1090
1091	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
1092	if [ X${ExeDate} = X ] ; then
1093	# First component exe date
1094	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
1095	else
1096	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1097	fi
1098	ExeDate=${ExeDate}_${LS_comp[7]}
1099	fi
1100
1101	if [ ${i} -eq 0 ] ; then
1102	# First component
1103	NextExeSize="( "${ExeSize}
1104	else
1105	NextExeSize=${NextExeSize}", "${ExeSize}
1106	fi
1107	LastCompExeSize=${run_Log_LastExeSize[$i]}
1108	(( i=i+1 ))
1109
1110	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1111	if ( ${FirstInitialize} ) ; then
1112	IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1113	else
1114	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1115	IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
1116	FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1117	fi
1118	IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1119	UpdateExe=true
1120
1121	# SD : switch off for now
1122	#IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1123	#if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1124	# ExeSecDateMax=$ExeSecDate
1125	#fi
1126	fi
1127	done
1128
1129	# SD : switch off for now
1130	#if ( ${UpdateExe} ) ; then
1131	# echo "Launch SaveSourceModifications."
1132	# IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1133	#fi
1134
1135	NextExeSize=${NextExeSize}" )"
1136	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1137
1138	if [ ${DRYRUN} -le 1 ] ; then
1139	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1140	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1141	RET=$?
1142	if [ $RET -eq 0 ] ; then
1143	# ExeCpuLog variable contents 5 fields
1144	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
1145	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
1146	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1147	fi
1148	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
1149	fi
1150
1151	fi
1152
1153	#==================================#
1154	# Save Job output #
1155	#==================================#
1156	if [ X${Pack} = Xtrue ] ; then
1157	IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1158	else
1159	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1160	fi
1161	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1162
1163	# All was right ? no ? then we stop.
1164	IGCM_debug_Verif_Exit
1165
1166	# If all was OK, we can delete all files not necessary for next Job
1167	echo
1168	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1169
1170	if [ ${DRYRUN} -le 2 ] ; then
1171	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1172	else
1173	echo ${FileToBeDeleted[@]}
1174	fi
1175
1176	# Send some accounting element to the user if CumulPeriod=3
1177	if [ ${CumulPeriod} -eq 3 ] ; then
1178	echo
1179	IGCM_debug_Print 1 "Send email containing some accounting information : "
1180
1181	RealCpuTime=$( echo ${ExeCpuLog} \| gawk '{print $3}' )
1182
1183	consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${coreNumber}/3600" \| bc )
1184
1185	consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" \| bc )
1186
1187	recommendedPeriodNb=$( echo "scale=6;${jobWarningDelay}/3600/${consumeHoursPerPeriod}*${coreNumber}" \| bc )
1188
1189	IGCM_sys_SendMail Accounting
1190	fi
1191
1192	#=================================================#
1193	# Modification of libIGCM behaviour #
1194	#=================================================#
1195
1196	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
1197	# and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1198	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1199	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1200	echo
1201	echo "########################################################################"
1202	echo "!!! Modification of libIGCM behaviour !!!"
1203	echo
1204
1205	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
1206	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1207	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1208
1209	# Special treatments for libIGCM internals
1210	for option in ${libIGCM_UserChanges[*]} ; do
1211	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1212
1213	echo "We will change : ${option}."
1214	eval echo "Previous value : " \${${option}}
1215	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1216
1217	eval ${option}=\${libIGCM_UserChanges_${option}}
1218
1219	case ${option} in
1220	config_UserChoices_DateEnd)
1221	IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1222	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1223
1224	# Period Length In Days between DateBegin and DateEnd
1225	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
1226	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1227	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1228	IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
1229	IGCM_debug_Verif_Exit
1230	fi
1231	;;
1232	config_UserChoices_PeriodLength)
1233	IGCM_debug_Print 1 "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1234	;;
1235	PeriodNb)
1236	IGCM_debug_Print 1 "Loop in main Job with ${PeriodNb} period(s)"
1237	;;
1238	config_Post_RebuildFrequency)
1239	IGCM_debug_Print 1 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1240	IGCM_post_Configure
1241	;;
1242	config_Post_TimeSeriesFrequency)
1243	IGCM_debug_Print 1 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1244	IGCM_post_Configure
1245	;;
1246	config_Post_SeasonalFrequency)
1247	IGCM_debug_Print 1 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1248	IGCM_post_Configure
1249	;;
1250	esac
1251	done
1252
1253	echo
1254	echo "########################################################################"
1255	echo
1256	fi
1257
1258	#=================================================#
1259	# Determine next computed period #
1260	#=================================================#
1261
1262	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1263	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1264	year_m1=$(( year - 1 ))
1265	year_p1=$(( year + 1 ))
1266	IGCM_config_DaysInPeriodLength
1267	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1268
1269	# Debug Print :
1270	echo
1271	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1272	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1273	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1274	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1275
1276	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1277	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1278
1279	(( CumulPeriod = CumulPeriod + 1 ))
1280
1281	# Debug Print :
1282	echo
1283	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1284	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
1285	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
1286
1287	#=================================================#
1288	# Write updated run.card #
1289	#=================================================#
1290
1291	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1292	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1293	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1294
1295	if ( ${FirstInitialize} ) ; then
1296	# It's no more the first time
1297	FirstInitialize=false
1298	fi
1299
1300	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1301	}
1302
1303	#===================================
1304	function IGCM_config_Finalize
1305	{
1306	IGCM_debug_PushStack "IGCM_config_Finalize"
1307
1308	echo
1309	IGCM_debug_Print 1 "IGCM_config_Finalize"
1310	echo
1311
1312	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1313	IGCM_config_StateCheck
1314
1315	# And EXIT if not OK
1316	IGCM_debug_Verif_Exit
1317
1318	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1319	#==========================#
1320	# End of entire simulation #
1321	#==========================#
1322	simulationIsOver=true
1323
1324	# Mail notification
1325	IGCM_sys_SendMail
1326	#
1327	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1328	#
1329	IGCM_debug_Print 1 "Normal End of computation."
1330
1331	else
1332	#=================#
1333	# Submit next job #
1334	#=================#
1335	simulationIsOver=false
1336
1337	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1338
1339	# Name of next Ksh Script output :
1340	Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
1341
1342	IGCM_debug_Print 1 "Submit next job"
1343	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1344	IGCM_sys_Cd ${SUBMIT_DIR}
1345	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1346	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null \| wc -l )
1347	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
1348	# Submit next job and come back
1349	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1350	IGCM_sys_Cd -
1351	fi
1352
1353	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1354	# Only for production run (No clean up in DEV or DEB mode)
1355	# and command sent from .. directory.
1356	IGCM_sys_Cd ..
1357	[ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
1358
1359	# Inform the rabbitMQ queue
1360	IGCM_debug_BigBro_Finalize
1361
1362	IGCM_debug_PopStack "IGCM_config_Finalize"
1363	}
1364
1365	#===================================

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 1396

Download in other formats: