Context Navigation

source: tags/libIGCM_v2.8/libIGCM_config/libIGCM_config.ksh @ 1456

Last change on this file since 1456 was 1280, checked in by sdipsl, 8 years ago
enhance the log message when computing job is stopped by a post processing job. The prost processing job name thjat has trigger the fatal will be written next the Fatal keyword in the run.card
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 52.1 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_CommonConfiguration
16	{
17	IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19	# Debug Print :
20	[ ${Verbosity} -gt 0 ] && echo
21	IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23	# config.card path
24	configCardPath=$1
25
26	#==================================
27	typeset option auxprint
28
29	#==================================
30	# Read UserChoices section:
31	[ ${Verbosity} -gt 0 ] && echo
32	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
33
34	IGCM_card_DefineArrayFromSection ${configCardPath} UserChoices
35	for option in ${config_UserChoices[*]} ; do
36	IGCM_card_DefineVariableFromOption ${configCardPath} UserChoices ${option}
37	eval auxprint=\${config_UserChoices_${option}}
38	IGCM_debug_Print 3 "${option} : ${auxprint}"
39	done
40
41	#==================================
42	# Read Ensemble section:
43	[ ${Verbosity} -gt 0 ] && echo
44	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
45
46	IGCM_card_DefineArrayFromSection ${configCardPath} Ensemble
47	for option in ${config_Ensemble[*]} ; do
48	IGCM_card_DefineVariableFromOption ${configCardPath} Ensemble ${option}
49	eval auxprint=\${config_Ensemble_${option}}
50	IGCM_debug_Print 3 "${option} : ${auxprint}"
51	done
52
53	#==================================
54	# Read Post section:
55	[ ${Verbosity} -gt 0 ] && echo
56	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
57
58	IGCM_card_DefineArrayFromSection ${configCardPath} Post
59	for option in ${config_Post[*]} ; do
60	IGCM_card_DefineVariableFromOption ${configCardPath} Post ${option}
61	eval auxprint=\${config_Post_${option}}
62	IGCM_debug_Print 3 "${option} : ${auxprint}"
63	done
64	[ ${Verbosity} -gt 0 ] && echo
65
66	#==================================
67	# Define default value to keep compatibility with previous card: means before changes due to TGCC
68	# Apply some overrules to ensure proper usage of computing centres resources
69	#
70	if [ X${PackDefault} = Xtrue ] ; then
71	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
72	# TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
73	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
74	config_Post_PackFrequency=NONE
75	else
76	# Default to RebuildFrequency if nothing has been set up related to PackFrequency
77	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
78	fi
79	else
80	# If we DO NOT apply pack in this computing center
81	config_Post_PackFrequency=NONE
82	fi
83
84	#====================================================
85	# Define ARCHIVE : Dedicated to large files
86	# Define STORAGE : Dedicated to small/medium files
87	# Define R_OUT : Output tree located on ARCHIVE
88	# Define R_BUF : Output tree located on STORAGE (files waiting treatment, or file lcoation when SpaceName=!PROD)
89	# Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
90	# Define R_TMP : A temporary space used by IGCM_debug_send_AMQP_msg__MAILTUNNEL. Must be persistent in between jobs
91	IGCM_sys_defineArchives
92
93	#====================================================
94	# R_SAVE : Job output directory
95	# R_BUFR : Job output buffered directory
96
97	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
98	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
99	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
100	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
101	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
102	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
103	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
104	else
105	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
106	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
107	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
108	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
109	fi
110	else
111	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
112	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
113	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
114	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
115	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
116	else
117	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
118	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
119	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
120	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
121	fi
122	fi
123
124	#====================================================
125	# Define R_OUT_KSH : Storage place for job output
126	# Define R_OUT_EXE : Storage place for binary used during simulation
127	R_OUT_KSH=${R_SAVE}/Out
128	R_OUT_EXE=${R_SAVE}/Exe
129
130	#====================================================
131	# Define R_BUF_KSH : Buffer place for job output
132	# Define R_BUF_EXE : Buffer place for binary used during simulation
133	R_BUF_KSH=${R_BUFR}/Out
134	R_BUF_EXE=${R_BUFR}/Exe
135
136	#====================================================
137	# Define REBUILD_DIR : where we store files needing rebuild process
138	REBUILD_DIR=${R_BUFR}/REBUILD
139	if [ ! X${TaskType} = Xchecking ] ; then
140	IGCM_sys_MkdirWork ${REBUILD_DIR}
141	fi
142
143	#====================================================
144	# DodsCopy : apply default value if not defined
145	if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] \|\| [ X${config_Post_DodsCopy} = X ] ) ; then
146	config_Post_DodsCopy=TRUE
147	fi
148
149	#====================================================
150	# IgnoreNonMonotonic : apply default value if not defined
151	if ( [ X${config_Post_IgnoreNonMonotonic} = X${NULL_STR} ] \|\| [ X${config_Post_IgnoreNonMonotonic} = X ] ) ; then
152	config_Post_IgnoreNonMonotonic=FALSE
153	fi
154
155	#====================================================
156	# Define StackFileLocation : directory where we store stack files
157	# Define StackFileName : stack file containing call tree and instrumentation
158	# Stack file containing call tree will be stored there.
159	if ( $DEBUG_debug ) ; then
160	StackFileLocation=${StackFileLocation:=${R_BUF_KSH}}
161	[ ! -d ${StackFileLocation} ] && mkdir -p ${StackFileLocation}
162	if [ X${TaskType} = Xcomputing ]; then
163	StackFileName=computing.stack.$$
164	elif [ X${TaskType} = Xpost-processing ]; then
165	StackFileName=${Script_Post_Output}.stack.$$
166	elif [ X${TaskType} = Xchecking ]; then
167	StackFileName=checking.stack.$$
168	else
169	IGCM_debug_Exit "IGCM_config_CommonConfiguration unknown TaskType : ${TaskType}"
170	IGCM_debug_Verif_Exit
171	fi
172
173	# This boolean will trigger the filling of the stack
174	# Only now we know where things should be ...
175	# We don't fill the stack when we perform checking task
176	if [ ! X${TaskType} = Xchecking ] ; then
177	ActivateStackFilling=true
178	fi
179	fi
180
181	IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
182	}
183
184	#===================================
185	function IGCM_config_Initialize
186	{
187	IGCM_debug_PushStack "IGCM_config_Initialize"
188
189	# Debug Print :
190	echo
191	IGCM_debug_Print 1 "IGCM_config_Initialize"
192
193	# Test modipsl tree existence.
194	IGCM_sys_TestDir ${MODIPSL}
195	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
196	IGCM_sys_TestDir ${libIGCM}
197	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
198	IGCM_sys_TestDir ${R_EXE}
199	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
200	IGCM_sys_TestDir ${SUBMIT_DIR}
201	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
202
203	if ( $DEBUG_debug ) ; then
204	echo "Keep trace of inital SUBMIT_DIR : "
205	ls -lta ${SUBMIT_DIR}
206	fi
207
208	#==================================
209	# Read ListOfComponents section:
210	echo
211	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
212
213	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
214	for comp in ${config_ListOfComponents[*]} ; do
215	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
216	done
217	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
218
219	#==================================
220	# Read Executable section:
221	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
222
223	#==================================
224	# Read Restarts section:
225	# Restarts : Gerneral rule or local for each component.
226	echo
227	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
228
229	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
230	for option in ${config_Restarts[*]} ; do
231	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
232	eval auxprint=\${config_Restarts_${option}}
233	IGCM_debug_Print 3 "${option} : ${auxprint}"
234	done
235
236	#==================================
237	# Define Job Outputs Name
238	echo
239	IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
240	Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
241	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
242	Exe_Output=out_execution
243	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
244
245	#===================================================================#
246	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
247	# But available to any son functions #
248	#===================================================================#
249
250	# Convert yyyy-mm-dd date to gregorian yyyymmdd
251	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
252	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
253
254	# Period Length In Days between DateBegin and DateEnd
255	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
256	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
257	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
258	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
259	IGCM_debug_Verif_Exit
260	fi
261
262	# Day and Year of Initial State (Given in julian format)
263	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
264	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
265
266	#================================================================#
267	# Test and Prepare directories #
268	#================================================================#
269
270	# ==> 4 kinds of input files :
271	# 1) R_INIT : Initial State Files (Etat0, carteveg)
272	# 2) R_BC : Boundary Conditions (Forcages, lai)
273	# 3) Parameters files (allready define through ${SUBMIT_DIR})
274	# 4) Restarts files (allready define in IGCM_config_Initialize)
275
276	# Here we offer the possibility to redefine R_INIT, R_BC
277	# and PeriodNb through config.card
278	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
279	echo
280	IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
281	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
282	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
283	IGCM_debug_Print 3 "R_BC=${R_BC}"
284	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
285	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
286
287	# SD ADA SPECIFIC #
288	# TO FIX #
289	#IGCM_sys_TestDirArchive ${R_IN}
290	#[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
291
292	if ( ${FirstInitialize} ) ; then
293	IGCM_sys_MkdirArchive ${R_SAVE}
294	[ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
295	else
296	IGCM_sys_TestDirArchive ${R_SAVE}
297	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
298
299	if [ ! ${config_Post_PackFrequency} = NONE ] ; then
300	IGCM_sys_TestDir ${R_BUFR}
301	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
302	fi
303
304	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
305	IGCM_config_StateCheck
306
307	# And EXIT if not OK
308	IGCM_debug_Verif_Exit
309	fi
310
311	#====================================================
312	# Experience type : DEB(ug), DEV(elopment), RUN
313	if [ X${JobType} != XRUN ] ; then
314	echo
315	echo "===================================================="
316	echo "libIGCM JOB is NOT in RUN type mode."
317	echo "!! OUTPUT files will NOT be PROTECTED !!"
318	echo "Be carefull : you can ERASE the result of this job !"
319
320	case ${JobType} in
321	DEB)
322	echo "DEBUG mode : activation of 'set -vx' mode."
323	echo "DEBUG mode : no protection for output files."
324	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
325	;;
326	DEV)
327	echo "DEVelopment mode : no protection for output files."
328	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
329	;;
330	esac
331
332	if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
333	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
334	echo "------------"
335	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
336	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
337	echo "------------"
338	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
339	fi
340	fi
341	echo "===================================================="
342	echo
343	fi
344
345	IGCM_debug_PopStack "IGCM_config_Initialize"
346	}
347
348	#===================================
349	function IGCM_config_DaysInPeriodLength
350	{
351	IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
352
353	typeset i
354
355	# Determine number of day(s) in PeriodLength :
356	case ${config_UserChoices_PeriodLength} in
357	Y\|y)
358	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
359	echo
360	IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
361	PeriodLengthInDays=0
362	i=0
363	until [ $i -ge $PeriodLengthInYears ] ; do
364	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
365	(( i=i+1 ))
366	done
367	;;
368	M\|m)
369	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
370	echo
371	IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
372	PeriodLengthInDays=0
373	i=0
374	until [ $i -ge $PeriodLengthInMonths ] ; do
375	if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
376	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
377	else
378	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
379	fi
380	(( i=i+1 ))
381	done
382	;;
383	D\|d)
384	PeriodLengthInMonths=0
385	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' )
386	echo
387	IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
388	*)
389	IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
390	IGCM_debug_Verif_Exit ;;
391	esac
392
393	IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
394	}
395
396	#===================================
397	function IGCM_config_DateCoherency
398	{
399	IGCM_debug_PushStack "IGCM_config_DateCoherency"
400
401	echo
402	IGCM_debug_Print 1 "IGCM_config_DateCoherency"
403	echo
404
405	typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
406
407	# check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
408	# DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
409	echo
410
411	case ${config_UserChoices_PeriodLength} in
412	Y\|y)
413	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
414	;;
415	M\|m)
416	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
417	;;
418	D\|d)
419	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
420	;;
421	esac
422	VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
423
424	if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
425	IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
426	IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
427	IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
428	IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
429	IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
430	IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
431
432	IGCM_debug_Exit "STOP here to avoid further issues."
433	fi
434
435	# PeriodDateBegin + PeriodLength = PeriodDateEnd
436	VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
437
438	IGCM_debug_PopStack "IGCM_config_DateCoherency"
439	}
440
441	#===================================
442	function IGCM_config_StateCheck
443	{
444	IGCM_debug_PushStack "IGCM_config_StateCheck"
445
446	#Test state of run in run.card
447	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
448
449	if [ ${run_Configuration_PeriodState} = "Fatal" ] ; then
450	echo
451	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
452	IGCM_debug_Print 1 "Check the overall status of your simulation by visiting this page:"
453	IGCM_debug_Print 1 "http://prodiguer-test-web.ipsl.fr/static/simulation.detail.html?uid=${simuid}"
454	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_month.job to rerun one period"
455	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_year.job to rerun more."
456	IGCM_debug_Exit
457	elif [ $( echo ${run_Configuration_PeriodState} \| grep Fatal \| wc -l ) -eq 1 ] ; then
458	echo
459	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
460	IGCM_debug_Print 1 "Compute jobs has been stop because at least the above mentionned post-processing jobs fails."
461	IGCM_debug_Print 1 "Check post-processing jobs carefully by visiting this page:"
462	IGCM_debug_Print 1 "http://prodiguer-test-web.ipsl.fr/static/simulation.detail.html?uid=${simuid}"
463	IGCM_debug_Print 1 "Please visit that page to see how to fix issues:"
464	IGCM_debug_Print 1 "https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocGmonitor"
465	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_month.job to rerun one period"
466	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_year.job to rerun more."
467	IGCM_debug_Exit
468	fi
469
470	IGCM_debug_PopStack "IGCM_config_StateCheck"
471	}
472
473	#===================================
474	function IGCM_config_Check
475	{
476	IGCM_debug_PushStack "IGCM_config_Check"
477
478	# If one of the following modulo is not zero :
479	# we will issue an error then explain and exit in
480	# AA_job IGCM_debug_Verif_Exit call before binary submission
481
482	echo
483	IGCM_debug_Print 1 "IGCM_config_Check"
484	echo
485
486	typeset i
487
488	# Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
489	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
490	AsynchronousRebuild=true
491	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
492	echo
493	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
494	IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
495	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
496	IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
497	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
498	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
499	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
500	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
501	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
502	else
503	AsynchronousRebuild=false
504	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
505	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
506	echo
507	#modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
508	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
509	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
510	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
511	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
512	fi
513
514	# Check PackFrequency against other key frequencies
515	# Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
516	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
517	Pack=true
518	#
519	IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
520	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
521	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
522	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
523	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
524	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
525	else
526	Pack=false
527	fi
528
529	# modulo (TimeSeriesFrequency and all Chunck2D) must be zero
530	NbJob=${#CHUNCK2D_SIZE[@]}
531	i=0
532	until [ $i -ge $NbJob ]; do
533	value=${CHUNCK2D_SIZE[${i}]}
534	IGCM_debug_Print 1 "Check coherence between ${CHUNCK2D_NAME[${i}]} Chunck2D frequency and TimeSeriesFrequency"
535	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
536	case ${value} in
537	Y\|y) ;;
538	*)
539	IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year Y\|y in comp.card"
540	IGCM_debug_Exit "This will stop the job" ;;
541	esac
542	(( i=i+1 ))
543	done
544
545	# modulo (TimeSeriesFrequency and all Chunck3D) must be zero
546	NbJob=${#CHUNCK3D_SIZE[@]}
547	i=0
548	until [ $i -ge $NbJob ]; do
549	value=${CHUNCK3D_SIZE[${i}]}
550	IGCM_debug_Print 1 "Check coherence between ${CHUNCK3D_NAME[${i}]} Chunck3D frequency and TimeSeriesFrequency"
551	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
552	case ${value} in
553	Y\|y) ;;
554	*)
555	IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year Y\|y in comp.card"
556	IGCM_debug_Exit "This will stop the job" ;;
557	esac
558	(( i=i+1 ))
559	done
560
561	# check to be sure there is enough space on temporary filesystems to run
562	echo
563	IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
564	IGCM_sys_check_quota
565
566	# check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
567	echo
568	IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
569	IGCM_sys_check_path
570
571
572	IGCM_debug_PopStack "IGCM_config_Check"
573	}
574
575	#===================================
576	function IGCM_config_ConfigureExecution
577	{
578	IGCM_debug_PushStack " IGCM_config_ConfigureExecution"
579
580	#echo
581	IGCM_debug_Print 1 " IGCM_config_ConfigureExecution"
582	#echo
583
584	typeset ExeNameIn ExeNameFirst CompNameFirst configCardPath comp i
585	typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
586
587	# config.card path
588	configCardPath=$1
589
590	coreNumber=0
591	mpiTasks=0
592	openMPthreads=0
593	NbExec=0
594
595	OK_PARA_MPI=false
596	OK_PARA_OMP=false
597	OK_PARA_NOD=false
598	OK_PARA_MPMD=false
599
600	for comp in ${config_ListOfComponents[*]} ; do
601
602	# Manage component executable
603	IGCM_card_DefineArrayFromOption ${configCardPath} Executable ${comp}
604
605	eval ExeNameIn=\${config_Executable_${comp}[0]}
606
607	# NO order in config.card for parallelized values !
608	# just use suffix : MPI , OMP and NOD (for number of NODes.)
609
610	# NOD is the number of NODes allocated
611	eval ${comp}_PROC_NOD=0
612
613	# MPI is the number of MPI processus per nodes
614	eval ${comp}_PROC_MPI=0
615
616	# OMP is the number of OpenMP threads per MPI processus
617	eval ${comp}_PROC_OMP=0
618
619	# Only if we really have an executable for the component :
620	if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
621
622	IGCM_debug_Print 1 ${comp}
623
624	# Keep the first executable found and the first CompName
625	ExeNameFirst=${ExeNameIn}
626	CompNameFirst=${comp}
627
628	# Are we a second executable?
629	(( NbExec = NbExec + 1 ))
630
631	# set 1 MPI task, 1 OpenMP thread and 1 node as default
632	eval ${comp}_PROC_MPI=1
633	eval ${comp}_PROC_OMP=1
634	eval ${comp}_PROC_NOD=1
635
636	eval NbElts=\${#config_Executable_${comp}[@]}
637
638	if [ ${NbElts} -gt 2 ] ; then
639	#
640	# CURRENT METHOD TO SPECIFY MPI AND OMP RESSOURCES
641	#
642	i=2
643	while [ ${i} -lt ${NbElts} ] ; do
644	eval tempvar=\${config_Executable_${comp}[${i}]}
645	IGCM_debug_Print 2 ${tempvar}
646
647	if [ X${tempvar} = X ] ; then
648	IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
649	IGCM_debug_Exit "Check your config.card. Exit now"
650	IGCM_debug_Verif_Exit
651	fi
652
653	case ${tempvar} in
654	[mM][pP][iI])
655	# Read MPI parameter for composante
656	eval ${comp}_PROC_MPI=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/MPI//" )
657	OK_PARA_MPI=true;;
658	[oO][mM][pP])
659	# Read OMP parameter for composante
660	eval ${comp}_PROC_OMP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/OMP//" )
661	;;
662	[nN][oO][dD])
663	# Read NOD (NumBer of Nodes) parameter for composante
664	eval ${comp}_PROC_NOD=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/NOD//" )
665	OK_PARA_NOD=true
666	OK_PARA_MPI=true
667	;;
668	esac
669	(( i = i + 1 ))
670	done
671	else
672	#
673	# BACKWARD COMPATIBILITY NOT SUPPORTED ANYMORE
674	#
675	IGCM_debug_Exit "You are using a deprecated ressources specification mechanism in your config.card"
676	IGCM_debug_Exit "Please check : https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocEsetup#ThesectionExecutable"
677	IGCM_debug_Exit "Please modify ${configCardPath}"
678	exit
679	fi
680	eval tempvarMPI=\${${comp}_PROC_MPI}
681	eval tempvarNOD=\${${comp}_PROC_NOD}
682	eval tempvarOMP=\${${comp}_PROC_OMP}
683
684	# set OMP mode if more than 1 OMP thread.
685	[ ${tempvarOMP} -ge 2 ] && OK_PARA_OMP=true
686
687	# Number of OMP threads
688	[ ${openMPthreads} -lt ${tempvarOMP} ] && openMPthreads=${tempvarOMP}
689
690	# SUM UP NUMBER OF CORES
691	(( coreNumber = coreNumber + tempvarMPI * tempvarNOD * tempvarOMP ))
692
693	# SUM UP NUMBER OF MPI TASKS
694	(( mpiTasks = mpiTasks + tempvarMPI * tempvarNOD ))
695	fi
696	done
697
698	# MANDATORY FOR THE OPA9.DRIVER. USED TO EDIT OPA NAMELIST
699	# WE SHOULD PLANIFY NUM_PROC_??? DEPRECATION
700	NUM_PROC_CPL=${CPL_PROC_MPI}
701	NUM_PROC_OCE=${OCE_PROC_MPI}
702	NUM_PROC_ATM=${ATM_PROC_MPI}
703
704	# set MPMD mode if more than 2 executable names.
705	[ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
706
707	# Define the execution type we are running in
708	if ( ${OK_PARA_MPMD} ) ; then
709	if ( ${OK_PARA_MPI} ) ; then
710	# MPMD always implies MPI
711	executionType=1
712	fi
713	if ( ${OK_PARA_OMP} ) ; then
714	# MPMD + MPI/OMP
715	executionType=2
716	fi
717	else
718	if ( ( ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
719	# SPMD + MPI/OMP
720	executionType=3
721	elif ( ( ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
722	# SPMD + MPI only
723	executionType=4
724	elif ( ( ! ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
725	# SPMD + OMP only
726	executionType=5
727	elif ( ( ! ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
728	# SEQUENTIAL THEN
729	executionType=6
730	coreNumber=1
731	fi
732	fi
733
734	IGCM_debug_Print 1 "MPI/OMP treatment coreNumber = ${coreNumber}"
735	IGCM_debug_Print 1 "MPI/OMP treatment mpiTasks = ${mpiTasks}"
736	IGCM_debug_Print 1 "MPI/OMP treatment openMPthreads = ${openMPthreads}"
737	IGCM_debug_Print 1 "MPI/OMP treatment executionType = ${executionType}"
738
739	IGCM_debug_PopStack "IGCM_config_ConfigureExecution"
740	}
741
742	#===================================
743	function IGCM_config_PeriodStart
744	{
745	IGCM_debug_PushStack "IGCM_config_PeriodStart"
746
747	echo
748	IGCM_debug_Print 1 "IGCM_config_PeriodStart"
749	echo
750
751	if ( ${FirstInitialize} ) ; then
752	#================================================#
753	# Initialize date/period information #
754	#================================================#
755
756	IGCM_date_GetYearMonthDay ${DateBegin} year month day
757	IGCM_config_DaysInPeriodLength
758
759	PeriodDateBegin=${DateBegin}
760	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
761	CumulPeriod=1
762
763	#=================================================#
764	# Write updated run.card #
765	#=================================================#
766
767	#Correct run.card Configuration for this period
768	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
769	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
770	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
771	if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
772	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
773	fi
774
775	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
776
777	else
778	#================================================#
779	# The file run.card allready exist #
780	#================================================#
781
782	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
783	IGCM_config_StateCheck
784	# And EXIT if not OK
785	IGCM_debug_Verif_Exit
786
787	#===================================#
788	# Read updated run.card #
789	#===================================#
790
791	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
792	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
793	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
794
795	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
796	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
797	CumulPeriod=${run_Configuration_CumulPeriod}
798
799	LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
800
801	if [ ${Period} = 1 ]; then
802	# save last Job output and current run.card
803	typeset Potential
804	IGCM_sys_Cd ${SUBMIT_DIR}
805	#
806	IGCM_debug_Print 2 "Save previous ksh job output"
807	for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
808	if [ X${Pack} = Xtrue ] ; then
809	( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${Potential} ) \|\| IGCM_sys_Cp ${Potential} ${R_BUF_KSH}/${Potential}.$$
810	else
811	( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) \|\| IGCM_sys_Cp ${Potential} ${R_OUT_KSH}/${Potential}.$$
812	fi
813	done
814	#
815	IGCM_debug_Print 2 "Save current run.card"
816	IGCM_card_CheckConflict run.card
817	if [ X${Pack} = Xtrue ] ; then
818	IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card
819	else
820	IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card
821	fi
822	#
823	IGCM_sys_Cd ${RUN_DIR}
824	else
825	unset FileToBeDeleted
826	fi
827
828	# Determine number of day(s) in PeriodLength
829	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
830	IGCM_config_DaysInPeriodLength
831
832	# Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
833	IGCM_config_DateCoherency
834	# And EXIT if not OK
835	IGCM_debug_Verif_Exit
836
837	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
838	IGCM_config_StateCheck
839	# And EXIT if not OK
840	IGCM_debug_Verif_Exit
841
842	# We can say we are "Running" now.
843	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
844	fi
845
846	# BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
847	# Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
848	year_m1=$(( year - 1 ))
849	year_p1=$(( year + 1 ))
850	# Compute month_m1 (month minus 1M)
851	# Compute yyyymm_m1 (yyyymm minus 1M)
852	month_m1=$(( 10#${month} - 1 ))
853	if [ ${month_m1} = 0 ]; then
854	month_m1=12
855	yyyymm_m1=${year_m1}12
856	elif [ ${month_m1} -le 9 ]; then
857	month_m1=0${month_m1}
858	yyyymm_m1=${year}${month_m1}
859	else
860	yyyymm_m1=${year}${month_m1}
861	fi
862	# Compute month_p1 (month plus 1M)
863	# Compute yyyymm_p1 (yyyymm plus 1M)
864	month_p1=$(( 10#${month} + 1 ))
865	if [ ${month_p1} = 13 ]; then
866	month_p1=01
867	yyyymm_p1=${year_p1}01
868	elif [ ${month_p1} -le 9 ]; then
869	month_p1=0${month_p1}
870	yyyymm_p1=${year}${month_p1}
871	else
872	yyyymm_p1=${year}${month_p1}
873	fi
874	#IGCM_debug_Print 1 "jg 1 month_m1 = ${month_m1} month_p1 = ${month_p1} "
875	#IGCM_debug_Print 1 "jg 1 calculate yyyymm_m1 = ${yyyymm_m1} "
876	#IGCM_debug_Print 1 "jg 1 calculate yyyymm_p1 = ${yyyymm_p1} "
877
878	#===================================================================#
879	# Calculate CyclicYear to be used for looping over a given forcing #
880	# period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
881	#===================================================================#
882
883	# To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
884	# CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
885	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
886	CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
887	CyclicYear_p1=NOTDEFINED
888
889	# For current year
890	yeartmp=$year
891	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
892	while [ $diffy -lt 0 ] ; do
893	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
894	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
895	done
896	CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
897
898	# For next coming year
899	yeartmp=$(( $year + 1 ))
900	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
901	while [ $diffy -lt 0 ] ; do
902	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
903	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
904	done
905	CyclicYear_p1=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
906
907	IGCM_debug_Print 1 "CyclicYear = ${CyclicYear}, CyclicYear_p1 = ${CyclicYear_p1}, current year=$year"
908	else
909	CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
910	CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
911	IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
912	fi
913
914	# END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
915
916	#===================================================================#
917	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
918	# But available to any son functions #
919	#===================================================================#
920
921	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
922	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
923
924	# Debug Print :
925	echo
926	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
927	IGCM_debug_Print 1 "Year of simulation : ${year}"
928	IGCM_debug_Print 1 "Month of simulation : ${month}"
929	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
930	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
931	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
932	IGCM_debug_Print 1 "SimulationLengthInDays : ${SimulationLengthInDays}"
933	IGCM_debug_Print 1 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
934
935	#================================================================#
936	# Prepare variables available for comp_finalyze #
937	#================================================================#
938
939	# Period for save files
940	DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
941
942	# Prefix for save files of this period
943	PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
944
945	# List of files that will be deleted in RUN_DIR after run
946	[ -f stack ] && FileToBeDeleted[0]="stack"
947
948	# Test if the same run as already been saved :
949	if [ X${JobType} = XRUN ] ; then
950	if [ ${DRYRUN} -le 0 ] ; then
951	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
952	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently RErunning an old job."
953	IGCM_debug_Print 1 "Because of readonly permissions, you can't RErun a job when saved files"
954	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
955	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See clean_month.job in ${libIGCM} directory."
956	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
957	IGCM_debug_Verif_Exit
958	fi
959	fi
960	else
961	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
962	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
963	fi
964	fi
965
966	#================================================================#
967	# Prepare variables available for binary execution #
968	# Call function for creation of run script #
969	# Only done once per job #
970	#================================================================#
971
972	if [ ${Period} -eq 1 ]; then
973	# Define the execution context (MPMD, SPMD, MPI/OMP ...)
974	IGCM_config_ConfigureExecution ${SUBMIT_DIR}/config.card
975	# Create the execution script for the current context
976	IGCM_sys_build_execution_scripts
977	fi
978
979	ExecutionFail=false
980
981	IGCM_debug_PopStack "IGCM_config_PeriodStart"
982	}
983
984	#===================================
985	function IGCM_config_SaveSourceModifications
986	{
987	IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
988
989	typeset ExeOutDateMax listVarEnv
990	ExeOutDateMax=$1
991
992	listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
993	IGCM_sys_RshMaster "\
994	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
995	export ExeOutDateMax=${ExeOutDateMax};\
996	export R_OUT_EXE=${R_OUT_EXE};\
997	export PREFIX=${PREFIX};\
998	export SUBMIT_DIR=${SUBMIT_DIR};\
999	export listVarEnv=${listVarEnv};\
1000	Script_Output=out_SaveSourceModifications;\
1001	IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
1002
1003	IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
1004	}
1005
1006	#===================================
1007	function IGCM_config_PeriodEnd
1008	{
1009	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
1010
1011	echo
1012	IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
1013	echo
1014
1015	if [ ${DRYRUN} -le 1 ] ; then
1016
1017	IGCM_debug_Print 1 "Check components binary : size and creation date"
1018
1019	typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
1020	typeset comp i
1021	typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
1022
1023	#==================================#
1024	# Get last Exe Size #
1025	#==================================#
1026
1027	(( i=0 ))
1028	if ( ${FirstInitialize} ) ; then
1029	run_Log_LastExeSize=""
1030	for comp in ${config_ListOfComponents[*]} ; do
1031	run_Log_LastExeSize[$i]=0
1032	(( i=i+1 ))
1033	done
1034	else
1035	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
1036	fi
1037	#==================================#
1038	# And Build ExeDate #
1039	#==================================#
1040
1041	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
1042	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
1043	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
1044	ExeDate=""
1045	NextExeSize="( "
1046	(( i=0 ))
1047	UpdateExe=false
1048	(( ExeSecDateMax = 0 ))
1049	for comp in ${config_ListOfComponents[*]} ; do
1050
1051	IGCM_debug_Print 3 ${comp}
1052
1053	eval ExeNameIn=\${config_Executable_${comp}[0]}
1054	eval ExeNameOut=\${config_Executable_${comp}[1]}
1055	# Only if we really have an executable for the component :
1056	if [ X${ExeNameIn} = X\"\" ] ; then
1057	# If there is no exe file for this component
1058	(( ExeSize=0 ))
1059	else
1060	LS_bin=${R_EXE}/${ExeNameIn}
1061	IGCM_sys_FileSize ${LS_bin} ExeSize
1062
1063	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
1064	if [ X${ExeDate} = X ] ; then
1065	# First component exe date
1066	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
1067	else
1068	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1069	fi
1070	ExeDate=${ExeDate}_${LS_comp[7]}
1071	fi
1072
1073	if [ ${i} -eq 0 ] ; then
1074	# First component
1075	NextExeSize="( "${ExeSize}
1076	else
1077	NextExeSize=${NextExeSize}", "${ExeSize}
1078	fi
1079	LastCompExeSize=${run_Log_LastExeSize[$i]}
1080	(( i=i+1 ))
1081
1082	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1083	if ( ${FirstInitialize} ) ; then
1084	IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1085	else
1086	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1087	IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
1088	FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1089	fi
1090	IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1091	UpdateExe=true
1092
1093	# SD : switch off for now
1094	#IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1095	#if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1096	# ExeSecDateMax=$ExeSecDate
1097	#fi
1098	fi
1099	done
1100
1101	# SD : switch off for now
1102	#if ( ${UpdateExe} ) ; then
1103	# echo "Launch SaveSourceModifications."
1104	# IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1105	#fi
1106
1107	NextExeSize=${NextExeSize}" )"
1108	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1109
1110	if [ ${DRYRUN} -le 1 ] ; then
1111	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1112	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1113	RET=$?
1114	if [ $RET -eq 0 ] ; then
1115	# ExeCpuLog variable contents 5 fields
1116	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
1117	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
1118	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1119	fi
1120	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
1121	fi
1122
1123	fi
1124
1125	#==================================#
1126	# Save Job output #
1127	#==================================#
1128	if [ X${Pack} = Xtrue ] ; then
1129	IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1130	else
1131	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1132	fi
1133	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1134
1135	# All was right ? no ? then we stop.
1136	IGCM_debug_Verif_Exit
1137
1138	# If all was OK, we can delete all files not necessary for next Job
1139	echo
1140	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1141
1142	if [ ${DRYRUN} -le 2 ] ; then
1143	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1144	else
1145	echo ${FileToBeDeleted[@]}
1146	fi
1147
1148	# Send some accounting element to the user if CumulPeriod=3
1149	if [ ${CumulPeriod} -eq 3 ] ; then
1150	echo
1151	IGCM_debug_Print 1 "Send email containing some accounting information : "
1152
1153	RealCpuTime=$( echo ${ExeCpuLog} \| gawk '{print $3}' )
1154
1155	consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${coreNumber}/3600" \| bc )
1156
1157	consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" \| bc )
1158
1159	recommendedPeriodNb=$( echo "scale=6;${jobWarningDelay}/3600/${consumeHoursPerPeriod}*${coreNumber}" \| bc )
1160
1161	IGCM_sys_SendMail Accounting
1162	fi
1163
1164	#=================================================#
1165	# Modification of libIGCM behaviour #
1166	#=================================================#
1167
1168	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
1169	# and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1170	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1171	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1172	echo
1173	echo "########################################################################"
1174	echo "!!! Modification of libIGCM behaviour !!!"
1175	echo
1176
1177	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
1178	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1179	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1180
1181	# Special treatments for libIGCM internals
1182	for option in ${libIGCM_UserChanges[*]} ; do
1183	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1184
1185	echo "We will change : ${option}."
1186	eval echo "Previous value : " \${${option}}
1187	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1188
1189	eval ${option}=\${libIGCM_UserChanges_${option}}
1190
1191	case ${option} in
1192	config_UserChoices_DateEnd)
1193	IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1194	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1195
1196	# Period Length In Days between DateBegin and DateEnd
1197	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
1198	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1199	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1200	IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
1201	IGCM_debug_Verif_Exit
1202	fi
1203	;;
1204	config_UserChoices_PeriodLength)
1205	IGCM_debug_Print 1 "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1206	;;
1207	PeriodNb)
1208	IGCM_debug_Print 1 "Loop in main Job with ${PeriodNb} period(s)"
1209	;;
1210	config_Post_RebuildFrequency)
1211	IGCM_debug_Print 1 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1212	IGCM_post_Configure
1213	;;
1214	config_Post_TimeSeriesFrequency)
1215	IGCM_debug_Print 1 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1216	IGCM_post_Configure
1217	;;
1218	config_Post_SeasonalFrequency)
1219	IGCM_debug_Print 1 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1220	IGCM_post_Configure
1221	;;
1222	esac
1223	done
1224
1225	echo
1226	echo "########################################################################"
1227	echo
1228	fi
1229
1230	#=================================================#
1231	# Determine next computed period #
1232	#=================================================#
1233
1234	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1235	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1236	year_m1=$(( year - 1 ))
1237	year_p1=$(( year + 1 ))
1238	IGCM_config_DaysInPeriodLength
1239	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1240
1241	# Debug Print :
1242	echo
1243	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1244	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1245	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1246	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1247
1248	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1249	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1250
1251	(( CumulPeriod = CumulPeriod + 1 ))
1252
1253	# Debug Print :
1254	echo
1255	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1256	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
1257	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
1258
1259	#=================================================#
1260	# Write updated run.card #
1261	#=================================================#
1262
1263	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1264	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1265	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1266
1267	if ( ${FirstInitialize} ) ; then
1268	# It's no more the first time
1269	FirstInitialize=false
1270	fi
1271
1272	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1273	}
1274
1275	#===================================
1276	function IGCM_config_Finalize
1277	{
1278	IGCM_debug_PushStack "IGCM_config_Finalize"
1279
1280	echo
1281	IGCM_debug_Print 1 "IGCM_config_Finalize"
1282	echo
1283
1284	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1285	IGCM_config_StateCheck
1286
1287	# And EXIT if not OK
1288	IGCM_debug_Verif_Exit
1289
1290	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1291	#==========================#
1292	# End of entire simulation #
1293	#==========================#
1294	simulationIsOver=true
1295
1296	# Mail notification
1297	IGCM_sys_SendMail
1298	#
1299	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1300	#
1301	IGCM_debug_Print 1 "Normal End of computation."
1302
1303	else
1304	#=================#
1305	# Submit next job #
1306	#=================#
1307	simulationIsOver=false
1308
1309	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1310
1311	# Name of next Ksh Script output :
1312	Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
1313
1314	IGCM_debug_Print 1 "Submit next job"
1315	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1316	IGCM_sys_Cd ${SUBMIT_DIR}
1317	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1318	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null \| wc -l )
1319	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
1320	# Submit next job and come back
1321	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1322	IGCM_sys_Cd -
1323	fi
1324
1325	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1326	# Only for production run (No clean up in DEV or DEB mode)
1327	# and command sent from .. directory.
1328	IGCM_sys_Cd ..
1329	[ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
1330
1331	# Inform the rabbitMQ queue
1332	IGCM_debug_BigBro_Finalize
1333
1334	IGCM_debug_PopStack "IGCM_config_Finalize"
1335	}
1336
1337	#===================================

Note: See TracBrowser for help on using the repository browser.

Download in other formats: