Context Navigation

libIGCM_config.ksh @ 387

Last change on this file since 387 was 387, checked in by sdipsl, 14 years ago

function IGCM_sys_Get_Master on cesium will try 10 times to scp files from master machine. Hopefully the last cesium consolidation
function IGCM_config_Check will check chunck2D/3D consistency with TimeSeriesFrequency?.
change the way the output from post-processing script was named. We will now have:

-rebuild_fromWorkdir.20051231.out
-create_ts.20051231.2D.out
-create_ts.20051231.3D.out
-atlas_PISCES.19991231.out
-create_ts.20051231.Chunck2D.ATM.POST_1D_histday.out
-create_ts.20051231.Chunck2D.OCE.POST_1D_grid_T.out

remove modipsl headers from AA_SE_Checker, AA_TimeSeries_Checker, AA_move-and-rename
- they check and submit job. They do not use resources and so there is no point to submit them
- they will be rename shortly SE_Checker.ksh, TimeSeries_Checker.ksh, move-and-rename.ksh
cosmetics

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 31.1 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_Initialize
16	{
17	IGCM_debug_PushStack "IGCM_config_Initialize"
18
19	# Debug Print :
20	echo
21	IGCM_debug_Print 1 "IGCM_config_Initialize :"
22	echo
23
24	# Test modipsl tree existence.
25	IGCM_sys_TestDir ${MODIPSL}
26	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
27	IGCM_sys_TestDir ${libIGCM}
28	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
29	IGCM_sys_TestDir ${R_EXE}
30	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
31	IGCM_sys_TestDir ${SUBMIT_DIR}
32	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
33
34	if ( $DEBUG_debug ) ; then
35	echo "Keep trace of inital SUBMIT_DIR : "
36	ls -lta ${SUBMIT_DIR}
37	fi
38
39	#==================================
40
41	typeset option auxprint CompatibilityTag
42	# Read libIGCM compatibility version in config.card
43	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Compatibility libIGCM
44	eval CompatibilityTag=${config_Compatibility_libIGCM} > /dev/null 2>&1
45
46	if [ ! "${CompatibilityTag}" = "${libIGCM_CurrentTag}" ] ; then
47	IGCM_debug_Exit "config.card is not compatible with libIGCM version ${libIGCM_CurrentTag} see libIGCM FAQ http://wiki.ipsl.jussieu.fr/wiki_ipsl/IGCMG/libIGCM/DocUtilisateur/FAQ ."
48	fi
49
50	#==================================
51	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card UserChoices
52
53	for option in ${config_UserChoices[*]} ; do
54	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card UserChoices ${option}
55	done
56
57	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
58	# Name Space of this experience
59	if [ X${config_UserChoices_SpaceName} != X ] ; then
60	IGCM_debug_PrintVariables 3 config_UserChoices_SpaceName
61	fi
62	# Expericence class of the run
63	if [ X${config_UserChoices_ExperimentName} != X ] ; then
64	IGCM_debug_PrintVariables 3 config_UserChoices_ExperimentName
65	fi
66	IGCM_debug_PrintVariables 3 config_UserChoices_JobName
67	if [ 3 -le ${Verbosity} ] ; then
68	echo "--------------Debug3-->" "config_UserChoices_LongName="
69	echo "--------------Debug3-->" \"${config_UserChoices_LongName}\"
70	fi
71	IGCM_debug_PrintVariables 3 config_UserChoices_TagName
72	IGCM_debug_PrintVariables 3 config_UserChoices_CalendarType
73	IGCM_debug_PrintVariables 3 config_UserChoices_DateBegin
74	IGCM_debug_PrintVariables 3 config_UserChoices_DateEnd
75	IGCM_debug_PrintVariables 3 config_UserChoices_PeriodLength
76
77	#==================================
78
79	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
80
81	echo
82	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
83	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
84	echo
85
86	NbComponents=${#config_ListOfComponents[*]}
87
88	#==================================
89	# Define principal executable
90
91	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
92	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Executable Name
93
94	#==================================
95	# Define Outputs Name
96	IGCM_debug_Print 1 "Define Script_Output_Prefix and Exe_Output"
97	eval Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
98	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
99	eval Exe_Output=out_${config_Executable_Name}
100	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
101
102	#==================================
103
104	#===================================================================#
105	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
106	# But available to any son functions #
107	#===================================================================#
108
109	# Convert yyyy-mm-dd date to gregorian yyyymmdd
110	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
111	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
112
113	# Period Length In Days between DateBegin and DateEnd
114	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
115	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
116	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
117	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
118	IGCM_debug_Verif_Exit
119	fi
120
121	# Day and Year of Initial State (Given in julian format)
122	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
123	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
124
125	#==================================
126	# Restarts : Gerneral rule or local for each component ?
127	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
128
129	echo
130	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
131
132	for option in ${config_Restarts[*]} ; do
133	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
134	eval auxprint=\${config_Restarts_${option}}
135	IGCM_debug_Print 3 "${option} : ${auxprint}"
136	done
137
138	#================================================================#
139	# Test and Prepare directories #
140	#================================================================#
141
142	# ==> 4 kinds of input files :
143	# 1) R_INIT : Initial State Files (Etat0, carteveg)
144	# 2) R_BC : Boundary Conditions (Forcages, lai)
145	# 3) Parameters files (allready define through ${SUBMIT_DIR})
146	# 4) Restarts files (allready define in IGCM_config_Initialize)
147
148	# Here we offer the possibility to redefine R_INIT, R_BC
149	# and PeriodNb through config.card
150	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
151	echo
152	IGCM_debug_Print 1 "(Re)Define R_INIT, R_BC and PeriodNb"
153	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
154	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
155	IGCM_debug_Print 3 "R_BC=${R_BC}"
156	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
157	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
158
159	# Test Archive input/output.
160	IGCM_sys_TestDirArchive ${ARCHIVE}
161	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
162	IGCM_sys_TestDirArchive ${R_IN}
163	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
164
165	#====================================================
166	#R_SAVE : Job output directory
167	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
168	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
169	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
170	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
171	else
172	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
173	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
174	fi
175
176	if [ ! -f ${SUBMIT_DIR}/run.card ]; then
177	IGCM_sys_MkdirArchive ${R_SAVE}
178	else
179	#Test state of run in run.card
180	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
181	if ( [ ${run_Configuration_PeriodState} != "Start" ] && [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
182	echo
183	IGCM_debug_Print 1 "!!!! IGCM_config_Initialize Error PeriodState : " ${run_Configuration_PeriodState} "!!!!!!!!!!"
184	echo
185	IGCM_debug_Exit "IGCM_config_Initialize Error PeriodState : " ${run_Configuration_PeriodState}
186	IGCM_debug_Verif_Exit
187	fi
188	fi
189	IGCM_sys_TestDirArchive ${R_SAVE}
190	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
191
192	#==================================
193	# Post :
194	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Post
195
196	echo
197	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
198
199	for option in ${config_Post[*]} ; do
200	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Post ${option}
201	eval auxprint=\${config_Post_${option}}
202	IGCM_debug_Print 3 "${option} : ${auxprint}"
203	done
204
205	#====================================================
206	# Define REBUILD_DIR : where we store files needing rebuild process
207	if [ X${config_Post_RebuildFromArchive} = Xtrue ] ; then
208	REBUILD_DIR=${R_SAVE}/TMP
209	IGCM_sys_MkdirArchive ${REBUILD_DIR}
210	else
211	REBUILD_DIR=${BIG_DIR}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
212	IGCM_sys_MkdirWork ${REBUILD_DIR}
213	fi
214
215	#====================================================
216	# Experience type : DEB(ug), DEV(elopment), RUN
217	if [ X${JobType} != XRUN ] ; then
218	echo
219	echo "===================================================="
220	echo "libIGCM JOB is NOT in RUN type mode."
221	echo "!! OUTPUT files will NOT be PROTECTED !!"
222	echo "Be carefull : you can ERASE the result of this job !"
223
224	case ${JobType} in
225	DEB)
226	echo "DEBUG mode : activation of 'set -vx' mode."
227	echo "DEBUG mode : no protection for output files."
228	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
229	;;
230	DEV)
231	echo "DEVelopment mode : no protection for output files."
232	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
233	;;
234	esac
235
236	if [ X${config_Post_RebuildFrequency} != XNONE ] ; then
237	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
238	echo "------------"
239	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
240	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
241	echo "------------"
242	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
243	fi
244	fi
245	echo "===================================================="
246	echo
247	fi
248
249	#====================================================
250	#R_OUT_KSH : Storage place for job output
251	#R_OUT_EXE : Storage place for binary used during simulation
252	R_OUT_KSH=${R_SAVE}/Out
253	R_OUT_EXE=${R_SAVE}/Exe
254
255	IGCM_debug_PopStack "IGCM_config_Initialize"
256	}
257
258	#===================================
259	function IGCM_config_Analyse_PeriodLength
260	{
261	IGCM_debug_PushStack "IGCM_config_Analyse_PeriodLength"
262
263	typeset i
264
265	# Determine number of day(s) in PeriodLength :
266	case ${config_UserChoices_PeriodLength} in
267	Y\|y)
268	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
269	IGCM_debug_Print 1 "Number of years for PeriodLength : ${PeriodLengthInYears}"
270	PeriodLengthInDays=0
271	i=0
272	until [ $i -ge $PeriodLengthInYears ] ; do
273	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
274	(( i=i+1 ))
275	done
276	;;
277	M\|m)
278	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
279	IGCM_debug_Print 1 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
280	PeriodLengthInDays=0
281	i=0
282	until [ $i -ge $PeriodLengthInMonths ] ; do
283	if [ $(( month + i )) -lt 13 ] ; then
284	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( month + i )) ) ))
285	else
286	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( month + i - 12 )) ) ))
287	fi
288	(( i=i+1 ))
289	done
290	;;
291	D\|d)
292	PeriodLengthInMonths=0
293	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' );;
294	*)
295	IGCM_debug_Exit "IGCM_config_Analyse_PeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
296	IGCM_debug_Verif_Exit ;;
297	esac
298
299	IGCM_debug_PopStack "IGCM_config_Analyse_PeriodLength"
300	}
301
302	#===================================
303	function IGCM_config_Check
304	{
305	IGCM_debug_PushStack "IGCM_config_Check"
306
307	# If one of the following modulo is not zero :
308	# we will issue an error then explain and exit in
309	# AA_job IGCM_debug_Verif_Exit call before binary submission
310
311	echo
312	IGCM_debug_Print 1 "IGCM_config_Check :"
313	echo
314
315	typeset i
316
317	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] \|\| [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
318	AsynchronousRebuild=true
319	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
320	echo
321	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
322	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
323	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
324	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
325	else
326	AsynchronousRebuild=false
327	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
328	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
329	echo
330	#modulo (TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
331	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
332	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
333	fi
334
335	#modulo (TimeSeriesFrequency and all Chunck2D) must be zero
336	NbJob=${#CHUNCK2D_SIZE[@]}
337	i=0
338	until [ $i -ge $NbJob ]; do
339	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency ${CHUNCK2D_SIZE[${i}]}
340	(( i=i+1 ))
341	done
342
343	#modulo (TimeSeriesFrequency and all Chunck3D) must be zero
344	NbJob=${#CHUNCK3D_SIZE[@]}
345	i=0
346	until [ $i -ge $NbJob ]; do
347	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency ${CHUNCK3D_SIZE[${i}]}
348	(( i=i+1 ))
349	done
350
351	IGCM_debug_PopStack "IGCM_config_Check"
352	}
353
354	#===================================
355	function IGCM_config_PeriodStart
356	{
357	IGCM_debug_PushStack "IGCM_config_PeriodStart"
358
359	echo
360	IGCM_debug_Print 1 "IGCM_config_PeriodStart :"
361	echo
362
363	if [ ! -r ${SUBMIT_DIR}/run.card ]; then
364
365	#================================================#
366	# The file run.card doesn't exist #
367	#================================================#
368
369	eval FirstInitialize=true
370
371	#copy initial run.card
372	IGCM_sys_Cp ${SUBMIT_DIR}/run.card.init ${SUBMIT_DIR}/run.card
373
374	IGCM_date_GetYearMonth $DateBegin year month
375	year_m1=$(( year - 1 ))
376	year_p1=$(( year + 1 ))
377	IGCM_config_Analyse_PeriodLength
378
379	eval PeriodDateBegin=${DateBegin} > /dev/null 2>&1
380	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
381	eval CumulPeriod=1 > /dev/null 2>&1
382
383	#=================================================#
384	# Creation and write updated run.card #
385	#=================================================#
386
387	#Correct run.card Configuration for this period
388	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
389	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
390	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
391	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
392
393	else
394	#================================================#
395	# The file run.card allready exist #
396	#================================================#
397
398	eval FirstInitialize=false
399
400	#Test state of run in run.card
401	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
402	if ( [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
403	echo
404	IGCM_debug_Print 1 "!!!!!! IGCM_config_PeriodStart Error PeriodState : " ${run_Configuration_PeriodState} "!!!!!!!!!!"
405	echo
406	IGCM_debug_Exit "IGCM_config_PeriodStart Error PeriodState : " ${run_Configuration_PeriodState}
407	IGCM_debug_Verif_Exit
408	fi
409
410	#===================================#
411	# Read updated run.card #
412	#===================================#
413
414	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration OldPrefix
415	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
416	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
417	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
418
419	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
420	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
421	eval CumulPeriod="${run_Configuration_CumulPeriod}" > /dev/null 2>&1
422
423	if [ ${Period} = 1 ]; then
424	# save last Job output
425	typeset LastPeriod iLastPeriod Last_Script_Output_Name
426	(( LastPeriod=CumulPeriod-PeriodNb ))
427	iLastPeriod=$( printf "%06d" ${LastPeriod} )
428	Last_Script_Output_Name="${Script_Output_Prefix}_${config_UserChoices_JobName}.${iLastPeriod}"
429	Last_run_card="run.card.${iLastPeriod}"
430	#eval ls -l ${SUBMIT_DIR}/${Last_Script_Output_Name}
431	#eval IGCM_sys_Put_Out ${SUBMIT_DIR}/${Last_Script_Output_Name} ${R_OUT_KSH}/${Last_Script_Output_Name} > /dev/null 2>&1
432	IGCM_debug_Print 1 "Try to save previous ksh job output"
433	IGCM_sys_Cd ${SUBMIT_DIR}/
434	IGCM_sys_Put_Out ${Last_Script_Output_Name} ${R_OUT_KSH}/${Last_Script_Output_Name}.$$ rw
435	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/${Last_run_card}.$$ rw
436	IGCM_sys_Cd ${RUN_DIR}
437	else
438	unset FileToBeDeleted
439	fi
440
441	IGCM_date_GetYearMonth $PeriodDateBegin year month
442	year_m1=$(( year - 1 ))
443	year_p1=$(( year + 1 ))
444	IGCM_config_Analyse_PeriodLength
445
446	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
447
448	fi
449	OldPrefix=${config_UserChoices_JobName}_${PeriodDateEnd}
450
451	#===================================================================#
452	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
453	# But available to any son functions #
454	#===================================================================#
455
456	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
457	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
458
459	# Debug Print :
460	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
461	IGCM_debug_Print 3 "Year of simulation : ${year}"
462	IGCM_debug_Print 3 "Month of simulation : ${month}"
463	IGCM_debug_Print 3 "PeriodLengthInDays : ${PeriodLengthInDays}"
464	IGCM_debug_Print 3 "PeriodDateBegin : ${PeriodDateBegin}"
465	IGCM_debug_Print 3 "PeriodDateEnd : ${PeriodDateEnd}"
466	IGCM_debug_Print 3 "SimulationLengthInDays : ${SimulationLengthInDays}"
467	IGCM_debug_Print 3 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
468
469	#================================================================#
470	# Prepare variables available for comp_finalyze #
471	#================================================================#
472
473	# Period for save files
474	eval DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd} > /dev/null 2>&1
475
476	# Prefix for save files of this period
477	eval PREFIX=${config_UserChoices_JobName}_${DatesPeriod} > /dev/null 2>&1
478
479	# List of files that will be deleted in RUN_DIR after run
480	[ -f stack ] && FileToBeDeleted[0]="stack"
481
482	# Test if the same run as already been saved :
483	if [ X${JobType} = XRUN ] ; then
484	if [ ${DRYRUN} -le 0 ] ; then
485	if ( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${PREFIX}_${Exe_Output} ) ; then
486	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently RErunning an old job."
487	IGCM_debug_Print 1 "Because of readonly permissions, you can't RErun a job when saved files"
488	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
489	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See clean_month.job in ${libIGCM} directory."
490	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_OUT_KSH}/${PREFIX}_${Exe_Output} exists."
491	IGCM_debug_Verif_Exit
492	fi
493	fi
494	else
495	if ( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${PREFIX}_${Exe_Output} ) ; then
496	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. DEBUG of DEV mode."
497	fi
498	fi
499
500	IGCM_debug_PopStack "IGCM_config_PeriodStart"
501	}
502
503	#===================================
504	function IGCM_config_PeriodEnd
505	{
506	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
507
508	echo
509	IGCM_debug_Print 1 "IGCM_config_PeriodEnd :"
510	echo
511
512	#==================================#
513	# Save Job output #
514	#==================================#
515	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
516	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${Exe_Output}"
517
518	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card rw
519
520	typeset LS_comp LS_bin ExeDate ExeCpuLog ExeSize NextExeSize LastCompExeSize
521	typeset comp i
522	typeset ExeNameIn ExeNameOut
523
524	#==================================#
525	# Get last Exe Size #
526	#==================================#
527
528	if [ ${DRYRUN} -le 1 ] ; then
529	(( i=0 ))
530	if ( ${FirstInitialize} ) ; then
531	run_Log_LastExeSize=""
532	for comp in ${config_ListOfComponents[*]} ; do
533	run_Log_LastExeSize[$i]=0
534	(( i=i+1 ))
535	done
536	else
537	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
538	fi
539	fi
540	#==================================#
541	# And Build ExeDate #
542	#==================================#
543
544	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
545	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
546	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
547	ExeDate=""
548	NextExeSize="( "
549	(( i=0 ))
550
551	for comp in ${config_ListOfComponents[*]} ; do
552
553	IGCM_debug_Print 1 ${comp}
554
555	eval ExeNameIn=\${config_Executable_${comp}[0]}
556	eval ExeNameOut=\${config_Executable_${comp}[1]}
557	# Only if we really have an executable for the component :
558	if [ X${ExeNameIn} = X\"\" ] ; then
559	# If there is no exe file for this component
560	(( ExeSize=0 ))
561	else
562	eval LS_bin=${R_EXE}/${ExeNameIn}
563	IGCM_sys_FileSize ${LS_bin} ExeSize
564
565	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
566	if [ X${ExeDate} = X ] ; then
567	# First component exe date
568	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
569	else
570	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
571	fi
572	ExeDate=${ExeDate}_${LS_comp[7]}
573	fi
574
575	if [ ${i} -eq 0 ] ; then
576	# First component
577	NextExeSize="( "${ExeSize}
578	else
579	NextExeSize=${NextExeSize}", "${ExeSize}
580	fi
581	eval LastCompExeSize=${run_Log_LastExeSize[$i]}
582	(( i=i+1 ))
583
584	if [ ${DRYRUN} -le 1 ] ; then
585	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
586	if ( ${FirstInitialize} ) ; then
587	IGCM_debug_Print 1 "Put first ${ExeNameIn} in ${R_OUT_EXE} !"
588	else
589	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
590	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
591	fi
592	eval IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
593	fi
594	fi
595	done
596
597	if [ ${DRYRUN} -le 1 ] ; then
598	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
599	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
600	RET=$?
601	if [ $RET -eq 0 ] ; then
602	# ExeCpuLog variable contents 5 fields
603	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
604	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
605	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
606	fi
607	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${Exe_Output}_tail.txt"
608	fi
609
610	NextExeSize=${NextExeSize}" )"
611	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
612
613	# All was right ? no ? then we stop.
614	IGCM_debug_Verif_Exit
615
616	# If all was OK, we can delete all files not necessary for next Job
617	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
618
619	if [ ${DRYRUN} -le 2 ] ; then
620	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
621	else
622	echo ${FileToBeDeleted[@]}
623	fi
624
625	#=================================================#
626	# Modification of libIGCM behaviour #
627	#=================================================#
628
629	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
630	# and put it in ${SUBMIT_DIR} directory. Ater modifications of ${SUBMIT_DIR}/libIGCM.card,
631	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
632	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
633	echo
634	echo "########################################################################"
635	echo "!!! Modification of libIGCM behaviour !!!"
636	echo
637
638	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
639	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
640	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
641
642	# Special treatments for libIGCM internals
643	for option in ${libIGCM_UserChanges[*]} ; do
644	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
645
646	echo "We will change : ${option}."
647	eval echo "Previous value : " \${${option}}
648	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
649
650	eval ${option}=\${libIGCM_UserChanges_${option}}
651
652	case ${option} in
653	config_UserChoices_DateEnd)
654	IGCM_debug_PrintVariables 3 config_UserChoices_DateEnd
655	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
656
657	# Period Length In Days between DateBegin and DateEnd
658	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
659	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
660	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
661	IGCM_debug_Exit "IGCM_PeriodEnd" " Wrong Dates."
662	IGCM_debug_Verif_Exit
663	fi
664	;;
665	config_UserChoices_PeriodLength)
666	;;
667	PeriodNb)
668	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
669	;;
670	config_Post_RebuildFrequency)
671	IGCM_debug_Print 3 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
672	IGCM_post_Configure
673	;;
674	config_Post_TimeSeriesFrequency)
675	IGCM_debug_Print 3 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Initialize"
676	IGCM_post_Configure
677	;;
678	config_Post_SeasonalFrequency)
679	IGCM_debug_Print 3 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Initialize"
680	IGCM_post_Configure
681	;;
682	esac
683	done
684
685	echo
686	echo "########################################################################"
687	echo
688	fi
689
690	#=================================================#
691	# Determine next computed period #
692	#=================================================#
693
694	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
695	IGCM_date_GetYearMonth $PeriodDateBegin year month
696	year_m1=$(( year - 1 ))
697	year_p1=$(( year + 1 ))
698	IGCM_config_Analyse_PeriodLength
699	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
700
701	# Debug Print :
702	echo
703	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
704	IGCM_debug_Print 3 "PeriodDateBegin : ${PeriodDateBegin}"
705	IGCM_debug_Print 3 "PeriodDateEnd : ${PeriodDateEnd}"
706	IGCM_debug_Print 3 "PeriodLengthInDays : ${PeriodLengthInDays}"
707
708	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
709	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
710
711	(( CumulPeriod = CumulPeriod + 1 ))
712
713	# Debug Print :
714	echo
715	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
716	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
717	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
718
719	#=================================================#
720	# Write updated run.card #
721	#=================================================#
722
723	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration OldPrefix ${OldPrefix}
724	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
725	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
726	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
727
728	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
729	}
730
731	#===================================
732	function IGCM_config_Finalize
733	{
734	IGCM_debug_PushStack "IGCM_config_Finalize"
735
736	echo
737	IGCM_debug_Print 1 "IGCM_config_Finalize :"
738	echo
739
740	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
741	#==========================#
742	# End of entire simulation #
743	#==========================#
744
745	# Mail notification
746	IGCM_sys_SendMail
747
748	#
749	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
750	IGCM_debug_Print 1 "Normal End of computation."
751	if ( $DEBUG_debug ) ; then
752	echo
753	IGCM_debug_Print 1 "Your files on ${R_OUT} :"
754	IGCM_sys_Tree ${R_SAVE}
755	fi
756	else
757	#=================#
758	# Submit next job #
759	#=================#
760
761	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
762
763	# Name of next Ksh Script output :
764	eval Script_Output="${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )"
765
766	IGCM_debug_Print 1 "Submit next job"
767	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
768	IGCM_sys_Cd ${SUBMIT_DIR}
769	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
770	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? > /dev/null 2>&1 \| wc -l )
771	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
772	# Submit next job and come back
773	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
774	IGCM_sys_Cd -
775	fi
776
777	# Supress Non Deleted Boundary files
778	if [ ${DRYRUN} -le 2 ] ; then
779	IGCM_comp_DelFixeBoundaryFiles
780	ls -la
781	fi
782
783	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
784	# Only for production run (No clean up in DEV or DEB mode)
785	# and command sent from .. directory.
786	IGCM_sys_Cd ..
787	[ X${JobType} = XRUN ] && IGCM_sys_Rm -rf ${RUN_DIR}
788
789	IGCM_debug_PopStack "IGCM_config_Finalize"
790	}
791
792	#===================================

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 387

Download in other formats: