Context Navigation

libIGCM_config.ksh @ 380

Last change on this file since 380 was 380, checked in by sdipsl, 14 years ago

Add a IGCM_config_Check function to test coherency between post-processing frequency. See #29. Will exit before execution if incoherency are found (example rebuild every 6 months, having a 1Y PeriodLenght? and a 2 months TimeSeriesFrequency?...)

libIGCM_post.ksh refactoring, making general function to handle frequency, easier to read hopefully. Still some simplification to add

DRYRUN was not maintain since a long time. Fix that. Minor adaptation still need to be done for sx8 and sx9 (ie when you want to DRYRUN from a machine initially devoted to the post-processing). DRYRUN is aimed for developpement purpose, prefer StandAlone? method to post-process after the fact.

we clean SUBMIT_DIR keeping only the 5 youngest Script_Output_* files. Good for workdir quota.

simplify rebuildFrom* options : remove R_SAVE and config_UserChoices_JobName used by IGCM_Patch_20091118_mask.ksh. rebuild.ksh will define that ; AA_create* already know that.

when exiting because of a RErun an old job : be more explicit and point the files who initiated this action, together with a link to clean_month.job

remove some redundant typeset

clean libIGCM_sys.ksh : remove machines

and cosmetics.

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 30.6 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_Initialize
16	{
17	IGCM_debug_PushStack "IGCM_config_Initialize"
18
19	# Debug Print :
20	echo
21	IGCM_debug_Print 1 "IGCM_config_Initialize :"
22	echo
23
24	# Test modipsl tree existence.
25	IGCM_sys_TestDir ${MODIPSL}
26	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
27	IGCM_sys_TestDir ${libIGCM}
28	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
29	IGCM_sys_TestDir ${R_EXE}
30	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
31	IGCM_sys_TestDir ${SUBMIT_DIR}
32	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
33
34	if ( $DEBUG_debug ) ; then
35	echo "Keep trace of inital SUBMIT_DIR : "
36	ls -lta ${SUBMIT_DIR}
37	fi
38
39	#==================================
40
41	typeset option auxprint CompatibilityTag
42	# Read libIGCM compatibility version in config.card
43	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Compatibility libIGCM
44	eval CompatibilityTag=${config_Compatibility_libIGCM} > /dev/null 2>&1
45
46	if [ ! "${CompatibilityTag}" = "${libIGCM_CurrentTag}" ] ; then
47	IGCM_debug_Exit "config.card is not compatible with libIGCM version ${libIGCM_CurrentTag} see libIGCM FAQ http://wiki.ipsl.jussieu.fr/wiki_ipsl/IGCMG/libIGCM/DocUtilisateur/FAQ ."
48	fi
49
50	#==================================
51	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card UserChoices
52
53	for option in ${config_UserChoices[*]} ; do
54	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card UserChoices ${option}
55	done
56
57	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
58	# Name Space of this experience
59	if [ X${config_UserChoices_SpaceName} != X ] ; then
60	IGCM_debug_PrintVariables 3 config_UserChoices_SpaceName
61	fi
62	# Expericence class of the run
63	if [ X${config_UserChoices_ExperimentName} != X ] ; then
64	IGCM_debug_PrintVariables 3 config_UserChoices_ExperimentName
65	fi
66	IGCM_debug_PrintVariables 3 config_UserChoices_JobName
67	if [ 3 -le ${Verbosity} ] ; then
68	echo "--------------Debug3-->" "config_UserChoices_LongName="
69	echo "--------------Debug3-->" \"${config_UserChoices_LongName}\"
70	fi
71	IGCM_debug_PrintVariables 3 config_UserChoices_TagName
72	IGCM_debug_PrintVariables 3 config_UserChoices_CalendarType
73	IGCM_debug_PrintVariables 3 config_UserChoices_DateBegin
74	IGCM_debug_PrintVariables 3 config_UserChoices_DateEnd
75	IGCM_debug_PrintVariables 3 config_UserChoices_PeriodLength
76
77	#==================================
78
79	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
80
81	echo
82	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
83	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
84	echo
85
86	NbComponents=${#config_ListOfComponents[*]}
87
88	#==================================
89	# Define principal executable
90
91	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
92	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Executable Name
93
94	#==================================
95	# Define Outputs Name
96	IGCM_debug_Print 1 "Define Script_Output_Prefix and Exe_Output"
97	eval Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
98	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
99	eval Exe_Output=out_${config_Executable_Name}
100	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
101
102	#==================================
103
104	#===================================================================#
105	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
106	# But available to any son functions #
107	#===================================================================#
108
109	# Convert yyyy-mm-dd date to gregorian yyyymmdd
110	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
111	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
112
113	# Period Length In Days between DateBegin and DateEnd
114	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
115	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
116	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
117	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
118	IGCM_debug_Verif_Exit
119	fi
120
121	# Day and Year of Initial State (Given in julian format)
122	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
123	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
124
125	#==================================
126	# Restarts : Gerneral rule or local for each component ?
127	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
128
129	echo
130	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
131
132	for option in ${config_Restarts[*]} ; do
133	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
134	eval auxprint=\${config_Restarts_${option}}
135	IGCM_debug_Print 3 "${option} : ${auxprint}"
136	done
137
138	#================================================================#
139	# Test and Prepare directories #
140	#================================================================#
141
142	# ==> 4 kinds of input files :
143	# 1) R_INIT : Initial State Files (Etat0, carteveg)
144	# 2) R_BC : Boundary Conditions (Forcages, lai)
145	# 3) Parameters files (allready define through ${SUBMIT_DIR})
146	# 4) Restarts files (allready define in IGCM_config_Initialize)
147
148	# Here we offer the possibility to redefine R_INIT, R_BC
149	# and PeriodNb through config.card
150	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
151	echo
152	IGCM_debug_Print 1 "(Re)Define R_INIT, R_BC and PeriodNb"
153	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
154	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
155	IGCM_debug_Print 3 "R_BC=${R_BC}"
156	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
157	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
158
159	# Test Archive input/output.
160	IGCM_sys_TestDirArchive ${ARCHIVE}
161	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
162	IGCM_sys_TestDirArchive ${R_IN}
163	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
164
165	#====================================================
166	#R_SAVE : Job output directory
167	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
168	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
169	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
170	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
171	else
172	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
173	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
174	fi
175
176	if [ ! -f ${SUBMIT_DIR}/run.card ]; then
177	IGCM_sys_MkdirArchive ${R_SAVE}
178	else
179	#Test state of run in run.card
180	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
181	if ( [ ${run_Configuration_PeriodState} != "Start" ] && [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
182	echo
183	IGCM_debug_Print 1 "!!!! IGCM_config_Initialize Error PeriodState : " ${run_Configuration_PeriodState} "!!!!!!!!!!"
184	echo
185	IGCM_debug_Exit "IGCM_config_Initialize Error PeriodState : " ${run_Configuration_PeriodState}
186	IGCM_debug_Verif_Exit
187	fi
188	fi
189	IGCM_sys_TestDirArchive ${R_SAVE}
190	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
191
192	#==================================
193	# Post :
194	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Post
195
196	echo
197	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
198
199	for option in ${config_Post[*]} ; do
200	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Post ${option}
201	eval auxprint=\${config_Post_${option}}
202	IGCM_debug_Print 3 "${option} : ${auxprint}"
203	done
204
205	#====================================================
206	# Define REBUILD_DIR : where we store files needing rebuild process
207	if [ X${config_Post_RebuildFromArchive} = Xtrue ] ; then
208	REBUILD_DIR=${R_SAVE}/TMP
209	IGCM_sys_MkdirArchive ${REBUILD_DIR}
210	else
211	REBUILD_DIR=${BIG_DIR}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
212	IGCM_sys_MkdirWork ${REBUILD_DIR}
213	fi
214
215	#====================================================
216	# Experience type : DEB(ug), DEV(elopment), RUN
217	if [ X${JobType} != XRUN ] ; then
218	echo
219	echo "===================================================="
220	echo "libIGCM JOB is NOT in RUN type mode."
221	echo "!! OUTPUT files will NOT be PROTECTED !!"
222	echo "Be carefull : you can ERASE the result of this job !"
223
224	case ${JobType} in
225	DEB)
226	echo "DEBUG mode : activation of 'set -vx' mode."
227	echo "DEBUG mode : no protection for output files."
228	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
229	;;
230	DEV)
231	echo "DEVelopment mode : no protection for output files."
232	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
233	;;
234	esac
235
236	if [ X${config_Post_RebuildFrequency} != XNONE ] ; then
237	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
238	echo "------------"
239	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
240	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
241	echo "------------"
242	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
243	fi
244	fi
245	echo "===================================================="
246	echo
247	fi
248
249	#====================================================
250	#R_OUT_KSH : Storage place for job output
251	#R_OUT_EXE : Storage place for binary used during simulation
252	R_OUT_KSH=${R_SAVE}/Out
253	R_OUT_EXE=${R_SAVE}/Exe
254
255	IGCM_debug_PopStack "IGCM_config_Initialize"
256	}
257
258	#===================================
259	function IGCM_config_Analyse_PeriodLength
260	{
261	IGCM_debug_PushStack "IGCM_config_Analyse_PeriodLength"
262
263	typeset i
264
265	# Determine number of day(s) in PeriodLength :
266	case ${config_UserChoices_PeriodLength} in
267	Y\|y)
268	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
269	IGCM_debug_Print 1 "Number of years for PeriodLength : ${PeriodLengthInYears}"
270	PeriodLengthInDays=0
271	i=0
272	until [ $i -ge $PeriodLengthInYears ] ; do
273	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
274	(( i=i+1 ))
275	done
276	;;
277	M\|m)
278	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
279	IGCM_debug_Print 1 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
280	PeriodLengthInDays=0
281	i=0
282	until [ $i -ge $PeriodLengthInMonths ] ; do
283	if [ $(( month + i )) -lt 13 ] ; then
284	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( month + i )) ) ))
285	else
286	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( month + i - 12 )) ) ))
287	fi
288	(( i=i+1 ))
289	done
290	;;
291	D\|d)
292	PeriodLengthInMonths=0
293	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' );;
294	*)
295	IGCM_debug_Exit "IGCM_config_Analyse_PeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
296	IGCM_debug_Verif_Exit ;;
297	esac
298
299	IGCM_debug_PopStack "IGCM_config_Analyse_PeriodLength"
300	}
301
302	#===================================
303	function IGCM_config_Check
304	{
305	IGCM_debug_PushStack "IGCM_config_Check"
306
307	# If one of the following modulo is not zero :
308	# we will issue an error then explain and exit in
309	# AA_job IGCM_debug_Verif_Exit call before binary submission
310
311	echo
312	IGCM_debug_Print 1 "IGCM_config_Check :"
313	echo
314
315	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] \|\| [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
316	AsynchronousRebuild=true
317	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
318	echo
319	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
320	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
321	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
322	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
323	else
324	AsynchronousRebuild=false
325	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
326	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
327	echo
328	#modulo (TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
329	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
330	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
331	fi
332
333	IGCM_debug_PopStack "IGCM_config_Check"
334	}
335
336	#===================================
337	function IGCM_config_PeriodStart
338	{
339	IGCM_debug_PushStack "IGCM_config_PeriodStart"
340
341	echo
342	IGCM_debug_Print 1 "IGCM_config_PeriodStart :"
343	echo
344
345	if [ ! -r ${SUBMIT_DIR}/run.card ]; then
346
347	#================================================#
348	# The file run.card doesn't exist #
349	#================================================#
350
351	eval FirstInitialize=true
352
353	#copy initial run.card
354	IGCM_sys_Cp ${SUBMIT_DIR}/run.card.init ${SUBMIT_DIR}/run.card
355
356	IGCM_date_GetYearMonth $DateBegin year month
357	year_m1=$(( year - 1 ))
358	year_p1=$(( year + 1 ))
359	IGCM_config_Analyse_PeriodLength
360
361	eval PeriodDateBegin=${DateBegin} > /dev/null 2>&1
362	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
363	eval CumulPeriod=1 > /dev/null 2>&1
364
365	#=================================================#
366	# Creation and write updated run.card #
367	#=================================================#
368
369	#Correct run.card Configuration for this period
370	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
371	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
372	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
373	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
374
375	else
376	#================================================#
377	# The file run.card allready exist #
378	#================================================#
379
380	eval FirstInitialize=false
381
382	#Test state of run in run.card
383	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
384	if ( [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
385	echo
386	IGCM_debug_Print 1 "!!!!!! IGCM_config_PeriodStart Error PeriodState : " ${run_Configuration_PeriodState} "!!!!!!!!!!"
387	echo
388	IGCM_debug_Exit "IGCM_config_PeriodStart Error PeriodState : " ${run_Configuration_PeriodState}
389	IGCM_debug_Verif_Exit
390	fi
391
392	#===================================#
393	# Read updated run.card #
394	#===================================#
395
396	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration OldPrefix
397	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
398	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
399	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
400
401	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
402	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
403	eval CumulPeriod="${run_Configuration_CumulPeriod}" > /dev/null 2>&1
404
405	if [ ${Period} = 1 ]; then
406	# save last Job output
407	typeset LastPeriod iLastPeriod Last_Script_Output_Name
408	(( LastPeriod=CumulPeriod-PeriodNb ))
409	iLastPeriod=$( printf "%06d" ${LastPeriod} )
410	Last_Script_Output_Name="${Script_Output_Prefix}_${config_UserChoices_JobName}.${iLastPeriod}"
411	Last_run_card="run.card.${iLastPeriod}"
412	#eval ls -l ${SUBMIT_DIR}/${Last_Script_Output_Name}
413	#eval IGCM_sys_Put_Out ${SUBMIT_DIR}/${Last_Script_Output_Name} ${R_OUT_KSH}/${Last_Script_Output_Name} > /dev/null 2>&1
414	IGCM_debug_Print 1 "Try to save previous ksh job output"
415	IGCM_sys_Cd ${SUBMIT_DIR}/
416	IGCM_sys_Put_Out ${Last_Script_Output_Name} ${R_OUT_KSH}/${Last_Script_Output_Name}.$$ rw
417	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/${Last_run_card}.$$ rw
418	IGCM_sys_Cd ${RUN_DIR}
419	else
420	unset FileToBeDeleted
421	fi
422
423	IGCM_date_GetYearMonth $PeriodDateBegin year month
424	year_m1=$(( year - 1 ))
425	year_p1=$(( year + 1 ))
426	IGCM_config_Analyse_PeriodLength
427
428	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
429
430	fi
431	OldPrefix=${config_UserChoices_JobName}_${PeriodDateEnd}
432
433	#===================================================================#
434	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
435	# But available to any son functions #
436	#===================================================================#
437
438	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
439	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
440
441	# Debug Print :
442	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
443	IGCM_debug_Print 3 "Year of simulation : ${year}"
444	IGCM_debug_Print 3 "Month of simulation : ${month}"
445	IGCM_debug_Print 3 "PeriodLengthInDays : ${PeriodLengthInDays}"
446	IGCM_debug_Print 3 "PeriodDateBegin : ${PeriodDateBegin}"
447	IGCM_debug_Print 3 "PeriodDateEnd : ${PeriodDateEnd}"
448	IGCM_debug_Print 3 "SimulationLengthInDays : ${SimulationLengthInDays}"
449	IGCM_debug_Print 3 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
450
451	#================================================================#
452	# Prepare variables available for comp_finalyze #
453	#================================================================#
454
455	# Period for save files
456	eval DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd} > /dev/null 2>&1
457
458	# Prefix for save files of this period
459	eval PREFIX=${config_UserChoices_JobName}_${DatesPeriod} > /dev/null 2>&1
460
461	# List of files that will be deleted in RUN_DIR after run
462	[ -f stack ] && FileToBeDeleted[0]="stack"
463
464	# Test if the same run as already been saved :
465	if [ X${JobType} = XRUN ] ; then
466	if [ ${DRYRUN} -le 0 ] ; then
467	if ( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${PREFIX}_${Exe_Output} ) ; then
468	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently RErunning an old job."
469	IGCM_debug_Print 1 "Because of readonly permissions, you can't RErun a job when saved files"
470	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
471	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See clean_month.job in ${libIGCM} directory."
472	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_OUT_KSH}/${PREFIX}_${Exe_Output} exists."
473	IGCM_debug_Verif_Exit
474	fi
475	fi
476	else
477	if ( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${PREFIX}_${Exe_Output} ) ; then
478	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. DEBUG of DEV mode."
479	fi
480	fi
481
482	IGCM_debug_PopStack "IGCM_config_PeriodStart"
483	}
484
485	#===================================
486	function IGCM_config_PeriodEnd
487	{
488	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
489
490	echo
491	IGCM_debug_Print 1 "IGCM_config_PeriodEnd :"
492	echo
493
494	#==================================#
495	# Save Job output #
496	#==================================#
497	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
498	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${Exe_Output}"
499
500	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card rw
501
502	typeset LS_comp LS_bin ExeDate ExeCpuLog ExeSize NextExeSize LastCompExeSize
503	typeset comp i
504	typeset ExeNameIn ExeNameOut
505
506	#==================================#
507	# Get last Exe Size #
508	#==================================#
509
510	if [ ${DRYRUN} -le 1 ] ; then
511	(( i=0 ))
512	if ( ${FirstInitialize} ) ; then
513	run_Log_LastExeSize=""
514	for comp in ${config_ListOfComponents[*]} ; do
515	run_Log_LastExeSize[$i]=0
516	(( i=i+1 ))
517	done
518	else
519	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
520	fi
521	fi
522	#==================================#
523	# And Build ExeDate #
524	#==================================#
525
526	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
527	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
528	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
529	ExeDate=""
530	NextExeSize="( "
531	(( i=0 ))
532
533	for comp in ${config_ListOfComponents[*]} ; do
534
535	IGCM_debug_Print 1 ${comp}
536
537	eval ExeNameIn=\${config_Executable_${comp}[0]}
538	eval ExeNameOut=\${config_Executable_${comp}[1]}
539	# Only if we really have an executable for the component :
540	if [ X${ExeNameIn} = X\"\" ] ; then
541	# If there is no exe file for this component
542	(( ExeSize=0 ))
543	else
544	eval LS_bin=${R_EXE}/${ExeNameIn}
545	IGCM_sys_FileSize ${LS_bin} ExeSize
546
547	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
548	if [ X${ExeDate} = X ] ; then
549	# First component exe date
550	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
551	else
552	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
553	fi
554	ExeDate=${ExeDate}_${LS_comp[7]}
555	fi
556
557	if [ ${i} -eq 0 ] ; then
558	# First component
559	NextExeSize="( "${ExeSize}
560	else
561	NextExeSize=${NextExeSize}", "${ExeSize}
562	fi
563	eval LastCompExeSize=${run_Log_LastExeSize[$i]}
564	(( i=i+1 ))
565
566	if [ ${DRYRUN} -le 1 ] ; then
567	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
568	if ( ${FirstInitialize} ) ; then
569	IGCM_debug_Print 1 "Put first ${ExeNameIn} in ${R_OUT_EXE} !"
570	else
571	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
572	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
573	fi
574	eval IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
575	fi
576	fi
577	done
578
579	if [ ${DRYRUN} -le 1 ] ; then
580	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
581	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
582	RET=$?
583	if [ $RET -eq 0 ] ; then
584	# ExeCpuLog variable contents 5 fields
585	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
586	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
587	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
588	fi
589	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${Exe_Output}_tail.txt"
590	fi
591
592	NextExeSize=${NextExeSize}" )"
593	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
594
595	# All was right ? no ? then we stop.
596	IGCM_debug_Verif_Exit
597
598	# If all was OK, we can delete all files not necessary for next Job
599	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
600
601	if [ ${DRYRUN} -le 2 ] ; then
602	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
603	else
604	echo ${FileToBeDeleted[@]}
605	fi
606
607	#=================================================#
608	# Modification of libIGCM behaviour #
609	#=================================================#
610
611	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
612	# and put it in ${SUBMIT_DIR} directory. Ater modifications of ${SUBMIT_DIR}/libIGCM.card,
613	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
614	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
615	echo
616	echo "########################################################################"
617	echo "!!! Modification of libIGCM behaviour !!!"
618	echo
619
620	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
621	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
622	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
623
624	# Special treatments for libIGCM internals
625	for option in ${libIGCM_UserChanges[*]} ; do
626	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
627
628	echo "We will change : ${option}."
629	eval echo "Previous value : " \${${option}}
630	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
631
632	eval ${option}=\${libIGCM_UserChanges_${option}}
633
634	case ${option} in
635	config_UserChoices_DateEnd)
636	IGCM_debug_PrintVariables 3 config_UserChoices_DateEnd
637	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
638
639	# Period Length In Days between DateBegin and DateEnd
640	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
641	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
642	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
643	IGCM_debug_Exit "IGCM_PeriodEnd" " Wrong Dates."
644	IGCM_debug_Verif_Exit
645	fi
646	;;
647	config_UserChoices_PeriodLength)
648	;;
649	PeriodNb)
650	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
651	;;
652	config_Post_RebuildFrequency)
653	IGCM_debug_Print 3 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
654	IGCM_post_Configure
655	;;
656	config_Post_TimeSeriesFrequency)
657	IGCM_debug_Print 3 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Initialize"
658	IGCM_post_Configure
659	;;
660	config_Post_SeasonalFrequency)
661	IGCM_debug_Print 3 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Initialize"
662	IGCM_post_Configure
663	;;
664	esac
665	done
666
667	echo
668	echo "########################################################################"
669	echo
670	fi
671
672	#=================================================#
673	# Determine next computed period #
674	#=================================================#
675
676	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
677	IGCM_date_GetYearMonth $PeriodDateBegin year month
678	year_m1=$(( year - 1 ))
679	year_p1=$(( year + 1 ))
680	IGCM_config_Analyse_PeriodLength
681	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
682
683	# Debug Print :
684	echo
685	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
686	IGCM_debug_Print 3 "PeriodDateBegin : ${PeriodDateBegin}"
687	IGCM_debug_Print 3 "PeriodDateEnd : ${PeriodDateEnd}"
688	IGCM_debug_Print 3 "PeriodLengthInDays : ${PeriodLengthInDays}"
689
690	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
691	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
692
693	(( CumulPeriod = CumulPeriod + 1 ))
694
695	# Debug Print :
696	echo
697	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
698	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
699	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
700
701	#=================================================#
702	# Write updated run.card #
703	#=================================================#
704
705	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration OldPrefix ${OldPrefix}
706	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
707	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
708	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
709
710	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
711	}
712
713	#===================================
714	function IGCM_config_Finalize
715	{
716	IGCM_debug_PushStack "IGCM_config_Finalize"
717
718	echo
719	IGCM_debug_Print 1 "IGCM_config_Finalize :"
720	echo
721
722	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
723	#==========================#
724	# End of entire simulation #
725	#==========================#
726
727	# Mail notification
728	IGCM_sys_SendMail
729
730	#
731	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
732	IGCM_debug_Print 1 "Normal End of computation."
733	if ( $DEBUG_debug ) ; then
734	echo
735	IGCM_debug_Print 1 "Your files on ${R_OUT} :"
736	IGCM_sys_Tree ${R_SAVE}
737	fi
738	else
739	#=================#
740	# Submit next job #
741	#=================#
742
743	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
744
745	# Name of next Ksh Script output :
746	eval Script_Output="${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )"
747
748	IGCM_debug_Print 1 "Submit next job"
749	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
750	IGCM_sys_Cd ${SUBMIT_DIR}
751	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
752	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? > /dev/null 2>&1 \| wc -l )
753	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
754	# Submit next job and come back
755	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
756	IGCM_sys_Cd -
757	fi
758
759	# Supress Non Deleted Boundary files
760	if [ ${DRYRUN} -le 2 ] ; then
761	IGCM_comp_DelFixeBoundaryFiles
762	ls -la
763	fi
764
765	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
766	# Only for production run (No clean up in DEV or DEB mode)
767	# and command sent from .. directory.
768	IGCM_sys_Cd ..
769	[ X${JobType} = XRUN ] && IGCM_sys_Rm -rf ${RUN_DIR}
770
771	IGCM_debug_PopStack "IGCM_config_Finalize"
772	}
773
774	#===================================

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 380

Download in other formats: