Context Navigation

AA_create_ts @ 1449

Last change on this file since 1449 was 1448, checked in by jgipsl, 6 years ago
Change in headers at irene : temporary remove option -m and use a workaround given by the TGCC. This change is done to avoid problems with resubmitting from job when submitting the main job from workdir.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 30.5 KB

Rev	Line
[622]	1	#-Q- curie ######################
	2	#-Q- curie ## CURIE TGCC/CEA ##
	3	#-Q- curie ######################
[906]	4	#-Q- curie #MSUB -r TS # Nom du job
[622]	5	#-Q- curie #MSUB -eo
	6	#-Q- curie #MSUB -n 1 # Reservation du processus
[880]	7	#-Q- curie #MSUB -T 80000 # Limite de temps elapsed du job
[1154]	8	#-Q- curie #MSUB -q ::default_node::
[1274]	9	#-Q- curie #MSUB -c ::default_core::
[704]	10	#-Q- curie #MSUB -Q normal
[837]	11	#-Q- curie #MSUB -A ::default_project::
[681]	12	#-Q- curie set +x
[1433]	13	#-Q- irene ######################
	14	#-Q- irene ## IRENE TGCC/CEA ##
	15	#-Q- irene ######################
	16	#-Q- irene #MSUB -r TS # Job name
	17	#-Q- irene #MSUB -eo
	18	#-Q- irene #MSUB -n 1 # Number of cores
	19	#-Q- irene #MSUB -T 80000 # Maximum elapsed time
	20	#-Q- irene #MSUB -q skylake
	21	#-Q- irene #MSUB -c 4
	22	#-Q- irene #MSUB -Q normal
[1437]	23	#-Q- irene #MSUB -A ::default_project::
[1448]	24	#-Q- irene ###MSUB -m store,work,scratch
	25	#-Q- irene #MSUB -E '--licenses=fs_unshare,fs_work,fs_store,fs_scratch'
[1433]	26	#-Q- irene set +x
[770]	27	#-Q- ada #!/bin/ksh
	28	#-Q- ada #######################
[929]	29	#-Q- ada ## ADA IDRIS ##
[770]	30	#-Q- ada #######################
[1409]	31	#-Q- ada # @ job_type = mpich
[848]	32	#-Q- ada # @ requirements = (Feature == "prepost")
[770]	33	#-Q- ada # Temps Elapsed max. d'une requete hh:mm:ss
	34	#-Q- ada # @ wall_clock_limit = 20:00:00
	35	#-Q- ada # Nom du travail LoadLeveler
	36	#-Q- ada # @ job_name = TS
	37	#-Q- ada # Fichier de sortie standard du travail
	38	#-Q- ada # @ output = $(job_name).$(jobid)
	39	#-Q- ada # Fichier de sortie d'erreur du travail
	40	#-Q- ada # @ error = $(job_name).$(jobid)
	41	#-Q- ada # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
	42	#-Q- ada # @ notification = error
[1290]	43	#-Q- ada # @ environment = $DEBUG_debug ; $BigBrother ; $postProcessingStopLevel ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $POST_DIR ; $Script_Post_Output ; $MASTER ; $RebuildFrequency ; $DateBegin ; $PeriodDateEnd ; $StandAlone ; $CompletedFlag ; $TsTask ; $CompToRead ; $FlagToRead ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; wall_clock_limit=$(wall_clock_limit)
[770]	44	#-Q- ada # @ queue
[287]	45	#-Q- lxiv8 ######################
	46	#-Q- lxiv8 ## OBELIX LSCE ##
	47	#-Q- lxiv8 ######################
	48	#-Q- lxiv8 #PBS -N TS
	49	#-Q- lxiv8 #PBS -m a
	50	#-Q- lxiv8 #PBS -j oe
	51	#-Q- lxiv8 #PBS -q medium
	52	#-Q- lxiv8 #PBS -o TS.$$
	53	#-Q- lxiv8 #PBS -S /bin/ksh
[1184]	54	#-Q- ifort_CICLAD ######################
	55	#-Q- ifort_CICLAD ## CICLAD IPSL ##
	56	#-Q- ifort_CICLAD ######################
	57	#-Q- ifort_CICLAD #PBS -N TS
	58	#-Q- ifort_CICLAD #PBS -m a
	59	#-Q- ifort_CICLAD #PBS -j oe
[1314]	60	#-Q- ifort_CICLAD ###PBS -q h12 # Queue for 12 hours at ciclad only
[1184]	61	#-Q- ifort_CICLAD #PBS -S /bin/ksh
[2]	62	#-Q- default #!/bin/ksh
	63	#-Q- default ##################
	64	#-Q- default ## DEFAULT HOST ##
	65	#-Q- default ##################
	66
[373]	67	#**************************************************************
	68	# Author: Sebastien Denvil
	69	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
	70	# $Revision:: $ Revision of last commit
	71	# $Author:: $ Author of last commit
	72	# $Date:: $ Date of last commit
[108]	73	# IPSL (2006)
	74	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
[373]	75	#
	76	#**************************************************************
[108]	77
	78	#set -eu
[2]	79	#set -vx
	80
	81	date
	82
[770]	83	#-Q- ada export OMP_NUM_THREADS=1
[2]	84
[1356]	85	#D- Task type DO NOT CHANGE (computing, post-processing or checking)
[712]	86	TaskType=post-processing
	87
[2]	88	########################################################################
	89
	90	#D- Flag to determine if this job in a standalone mode
	91	#D- Default : value from AA_job if any
	92	StandAlone=${StandAlone:=true}
	93
[554]	94	#D- Path to libIGCM
[206]	95	#D- Default : value from AA_job if any
[229]	96	libIGCM=${libIGCM:=::modipsl::/libIGCM}
[206]	97
	98	#D- Flag to determine allready produced time series. Empty if you start from the beginning
[213]	99	#D- Default : Value from AA_job if any. Usefull in StandAlone case if you want to continue a time series
[209]	100	CompletedFlag=${CompletedFlag:=}
[135]	101
[213]	102	#D- Flag to determine task type this script will perform.
	103	#D- Possible Value : 2D, 3D, Chunck2D, Chunck3D
	104	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
[257]	105	TsTask=${TsTask:=2D}
[267]	106	if [ X${TsTask} = Xempty ] ; then
[785]	107	TsTask=""
[267]	108	fi
[213]	109
	110	#D- Flag to determine if rebuild process has been performed asynchronously
[906]	111	#D- Possible value true or false.
[213]	112	#D- If true raw files has allready been patched by rebuild job
	113	#D- If false create_ts will patch the raw files
	114	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
	115	RebuildFrequency=${RebuildFrequency:=true}
	116
[2]	117	#D- Flag to determine job's output directory
	118	#D- Default : value from libIGCM_post.ksh if any
[370]	119	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
[2]	120
	121	#D- Increased verbosity (1, 2, 3)
	122	#D- Default : value from AA_job if any
	123	Verbosity=${Verbosity:=3}
	124
	125	#D- Low level debug : to bypass lib test checks and stack construction
	126	#D- Default : value from AA_job if any
	127	DEBUG_debug=${DEBUG_debug:=false}
	128
	129	#D- TEMPORARY Flag to determine atmospheric resolution
	130	#D- Default : value from atmospheric driver if any
[254]	131	RESOL_ATM=ALL
[2]	132
[257]	133	#D- Flag to determine surface resolution
	134	#D- Default : value from surface driver if any
	135	RESOL_SRF=ALL
	136
	137	#D- Flag to determine surface resolution
	138	#D- Default : value from surface driver if any
	139	RESOL_SBG=ALL
	140
[2]	141	#D- TEMPORARY Flag to determine ocean resolution
	142	#D- Default : value from ocean driver if any
	143	RESOL_OCE=${RESOL_OCE:=ORCA2}
	144
	145	#D- TEMPORARY Flag to determine ice resolution
	146	#D- Default : value from ice driver if any
[1326]	147	RESOL_ICE=${RESOL_OCE:=ORCA2}
[2]	148
[178]	149	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
	150	#D- Default : value from ice driver if any
[1326]	151	RESOL_MBG=${RESOL_OCE:=ORCA2}
[178]	152
[2]	153	########################################################################
	154
[108]	155	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
	156	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
	157	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
	158	#-------
	159	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
[731]	160	. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
[832]	161	#-------
[1192]	162	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
	163	IGCM_sys_Cd ${RUN_DIR_PATH}
	164	#-------
[832]	165	( ${DEBUG_debug} ) && IGCM_debug_Check
	166	( ${DEBUG_debug} ) && IGCM_card_Check
	167	( ${DEBUG_debug} ) && IGCM_date_Check
[2]	168
	169	########################################################################
	170
	171	#set -vx
	172
	173	# Useful cleaning function
[906]	174	MENAGE ()
[81]	175	{
[1306]	176	typeset ExistFlag
	177	ExistFlag=$( IGCM_sys_RshArchive "[ -f $1/$3 ] && echo 0 \|\| echo 1" )
	178	[ ${ExistFlag} -eq 0 ] && ( IGCM_sys_RshArchive rm -f $1/$2 ;)
[2]	179	}
	180
	181	########################################################################
	182
[436]	183	# ------------------------------------------------------------------
	184	# Test if all was right before proceeding further
	185	# ------------------------------------------------------------------
[1206]	186	IGCM_debug_Verif_Exit
[436]	187
[2]	188	if [ ${StandAlone} = true ] ; then
[785]	189	CARD_DIR=${SUBMIT_DIR}
[2]	190	else
[785]	191	CARD_DIR=${RUN_DIR_PATH}
[903]	192
[785]	193	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
[903]	194
[785]	195	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
[903]	196
[785]	197	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
[903]	198
[785]	199	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
[2]	200	fi
	201
[591]	202	#==================================
[2]	203	# First of all
	204	#
[727]	205	# Read libIGCM compatibility version in config.card
	206	# Read UserChoices section
	207	# Read Ensemble section
	208	# Read Post section
	209	# Define all netcdf output directories
[591]	210	#==================================
[727]	211	IGCM_config_CommonConfiguration ${CARD_DIR}/config.card
[591]	212
[1198]	213	# ------------------------------------------------------------------
	214	# Activate BigBrother so as to supervise this job
	215	# ------------------------------------------------------------------
	216	IGCM_debug_BigBro_Initialize
	217
[591]	218	#==================================
[633]	219	# Define default value to keep compatibility with previous card: means before changes due to TGCC
	220	if [ X${PackDefault} = Xtrue ] ; then
	221	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
	222	else
	223	config_Post_PackFrequency=NONE
	224	fi
[591]	225
	226	#==================================
	227	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
	228	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
	229	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
	230	fi
	231
[2]	232	#
[149]	233	# Determine component to take care of depending on kind of task create_ts will perform
[2]	234	#
[392]	235	[ ${CompToRead} ] && set -A config_ListOfComponents ${CompToRead} \|\| IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
[2]	236	#
[149]	237	# Determine period for time series. Default : value from AA_job or from launch_TS if any
	238	#
[309]	239	[ X${CompletedFlag} = X ] \|\| CompletedFlag=$( IGCM_date_ConvertFormatToGregorian ${CompletedFlag} )
[301]	240
[2]	241	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
	242	#
	243	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
[149]	244	#
	245	# Determine Dimension of the produced time series : 2D or 3D
	246	if ( [ X${TsTask} = XChunck2D ] \|\| [ X${TsTask} = X2D ] ) ; then
[785]	247	Dimension=2D
[149]	248	elif ( [ X${TsTask} = XChunck3D ] \|\| [ X${TsTask} = X3D ] ) ; then
[785]	249	Dimension=3D
[149]	250	else
[785]	251	Dimension=""
[149]	252	fi
[2]	253
[441]	254	# ------------------------------------------------------------------
	255	#D- Test if all was right before entering the loop
	256	# ------------------------------------------------------------------
[1206]	257	IGCM_debug_Verif_Exit
[149]	258
[2]	259	########################################################################
	260	# Depending on requested time series fill following variables :
	261	#
	262	# - LISTE_FILE_${comp}[*]
	263	# - LISTE_VARS_${file}[*]
	264	# - LISTE_AXIS_${file}[*]
	265	# - LISTE_PATCH_${file}[*]
	266	########################################################################
	267
	268	#
	269	# For each selected component determine which files need post-processing
	270	#
	271	DoJob=false
	272	for comp in ${config_ListOfComponents[*]} ; do
[785]	273	#
	274	ActiveComp=false
	275	#
	276	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
	277	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
	278	#
	279	card=${CARD_DIR}/COMP/${compname}.card
	280
	281	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
	282	#
	283	ListFilesName=${compname}_OutputFiles_List
	284	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
	285	#
	286	if [ X${FileName0} != X${NULL_STR} ] ; then
[2]	287	#
[785]	288	#IGCM_debug_Print 1 "Component : ${compname}"
[2]	289	#
[785]	290	# INITIALISATION
[2]	291	#
[785]	292	typeset i
	293	if [ ${FlagToRead} ] ; then
	294	((NbFiles=${FlagToRead}))
	295	((i=${NbFiles}))
	296	else
	297	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
	298	i=2
	299	fi
[2]	300	#
[785]	301	until [ $i -gt $NbFiles ]; do
	302	#
	303	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
	304	#
	305	if [ X${flag_post} != XNONE ] ; then
	306	#
	307	# For each comp and each file who needs post-processing
	308	# determine which variables need time series.
	309	#
	310	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
	311	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} TimeSeriesVars${Dimension}
	312	IGCM_card_DefineVariableFromOption ${card} Post_${FILE} ChunckJob${Dimension}
	313	#
	314	# If TimeSeriesVars list is empty we skip
	315	#
	316	if [ $( eval echo \${${compname}_Post_${FILE}_TimeSeriesVars${Dimension}} ) = ${NULL_STR} ] ; then
	317	#
	318	IGCM_debug_Print 3 "Nothing to do for ${FILE} according to ${card}"
	319	IGCM_debug_Print 3 "TimeSeriesVars${Dimension} for ${FILE} is empty"
	320	(( i=i+3 ))
	321	continue
	322	#
	323	fi
	324	#
	325	# If files/variables has been switch off by ChunckJob${Dimension}=0 we skip
	326	#
	327	if [ X$( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} ) = XOFF ] ; then
	328	IGCM_debug_Print 3 "Nothing to do for ${FILE} according to ${card}"
	329	IGCM_debug_Print 3 "ChunckJob${Dimension}=OFF for ${FILE} so it has been switched off"
	330	(( i=i+3 ))
	331	continue
	332	fi
	333	#
	334	# If we are not a Chunck type task, we skip files/variables handled by chunck task
	335	#
	336	if ( [ ! X${TsTask} = XChunck${Dimension} ] && [ ! X${TsTask} = X ] && [ ! XChunck${Dimension} = XChunck ] ) ; then
	337	if [ ! $( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} ) = NONE ] ; then
	338	IGCM_debug_Print 3 "Nothing to do for ${FILE} TimeSeriesVars${Dimension} according to ${card}"
	339	IGCM_debug_Print 3 "Let another chunck task do the job $( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} )"
	340	(( i=i+3 ))
	341	continue
	342	fi
	343	fi
	344	#
	345	# Now on we know we have things to do
	346	#
[236]	347
[785]	348	#
	349	# We need coordinates axis to properly fill produced time series
	350	#
	351	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} GatherWithInternal
	352	#
	353	if [ $( eval echo \${${compname}_Post_${FILE}_GatherWithInternal} ) = ${NULL_STR} ] ; then
	354	#
	355	IGCM_debug_Print 1 "ERROR !! We miss coordinate axis for ${FILE} according to ${card}"
	356	IGCM_debug_Print 1 "So no time series will be produced for ${FILE} !!"
	357	(( i=i+3 ))
	358	continue
	359	#
	360	fi
	361	#
	362	# We need LIST of variables not allready produced (useful for standalone mode)
	363	#
	364	FlagDir=$( echo ${FILE} \| awk -F "_" '{print $1}' )
	365	case ${FlagDir} in
	366	*Y) TS_Dir=TS_YE ;;
	367	*M) TS_Dir=TS_MO ;;
	368	*D) TS_Dir=TS_DA ;;
	369	HF) TS_Dir=TS_HF ;;
	370	3H) TS_Dir=TS_HF ;;
	371	INS) TS_Dir=TS_INS ;;
	372	esac
	373	#
	374	count=0
	375	for var in $( eval echo \${${compname}_Post_${FILE}_TimeSeriesVars${Dimension}[*]} ) ; do
	376	#
	377	DIRECTORY=${R_SAVE}/${comp}/Analyse/${TS_Dir}
	378	TestedFile=${config_UserChoices_JobName}_${DateBegin}_${PeriodDateEnd}_${FlagDir}_${var}.nc
	379	#
	380	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
	381	if [ ! $? = 0 ] ; then
	382	eval set +A LISTE_VARS_${FILE} \${LISTE_VARS_${FILE}[*]} ${var}
	383	ActiveComp=true
	384	(( count = count + 1 ))
	385	fi
	386	done
	387	#
	388	if [ ${count} -gt 0 ] ; then
	389	#
	390	eval set +A LISTE_FILE_${comp} \${LISTE_FILE_${comp}[*]} ${FILE}
	391	eval set +A LISTE_AXIS_${FILE} \${${compname}_Post_${FILE}_GatherWithInternal[*]}
	392	#
	393	if [ ! X${RebuildFrequency} = Xtrue ] ; then
	394	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
	395	eval set +A LISTE_PATCHES_${FILE} \${${compname}_Post_${FILE}_Patches[*]}
	396	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
	397	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
	398	if [ Xload${Patch} != X ] ; then
	399	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
	400	eval load${Patch}=loaded
	401	fi
	402	done
	403	fi
	404	fi
	405	fi
	406	fi
	407	(( i=i+3 ))
	408	done
	409	fi
	410	#
	411	if [ ${ActiveComp} = true ] ; then
	412	set +A ActiveListOfComponents ${ActiveListOfComponents[*]} ${comp}
	413	fi
[2]	414	done
	415
	416	set -A config_ListOfComponents ${ActiveListOfComponents[*]}
	417	[ ! -z "${ActiveListOfComponents}" ] && DoJob=true
	418
	419	########################################################################
	420	#
	421	# IS THERE SOME ALLREADY PRODUCED TIME SERIES ?
	422	# IF SO BRING THEM IN THE WORKING DIRECTORY
	423	#
	424	# PS : Keep in mind that IGCM_sys_Get here is a weak link :
	425	# - especially for IDRiS
	426	# - no special protection against "command too long" !
	427	# - libIGCM_sys should handle that or modify following code part
	428	#
	429	########################################################################
	430
	431	if [ ${StandAlone} != true ] ; then
[785]	432	Running_Flag=$( IGCM_sys_RshMaster "IGCM_card_DefineVariableFromOption \
	433	${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning ; \
	434	echo \${run_PostProcessing_TimeSeriesRunning}" )
[2]	435
[785]	436	#if [ X${Running_Flag} = Xy ] ; then
	437	# # Time Series Job allready running
	438	# IGCM_debug_Print 1 "Time Series Job allready running exit"
	439	# exit
	440	#fi
	441	# Now run.card know we are running
[149]	442
	443
	444
	445
[785]	446	IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "y"
[149]	447
	448
	449
	450
	451
[785]	452	# Get information from last execution
	453	#CompletedFlag=$( IGCM_sys_RshMaster \
	454	# "IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted ;\
	455	# echo \${run_PostProcessing_TimeSeriesCompleted}" )
[2]	456	fi
	457
	458	# Is it the first submission or not ?
[149]	459
	460	if ( [ X${CompletedFlag} = X ] \|\| [ X${CompletedFlag} = X$( IGCM_date_AddDaysToGregorianDate ${DateBegin} -1 ) ] ) ; then
[785]	461	# First Time Series Submission
	462	FIRST_PASS=TRUE
	463	Length=$( IGCM_date_DaysInPreviousPeriod ${DateBegin} ${config_UserChoices_PeriodLength} begin)
	464	DATE_FIN_JOB_B=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( Length - 1 )) )
[2]	465	else
[785]	466	# Last Completed Time Series
	467	FIRST_PASS=FALSE
	468	DATE_FIN_JOB_B=${CompletedFlag}
	469	date=${DateBegin}_${DATE_FIN_JOB_B}
[1038]	470	#
[785]	471	for comp in ${config_ListOfComponents[*]} ; do
	472	#
	473	IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
	474	#
	475	IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
	476	#
	477	unset liste_file
	478	#
	479	file_path=${R_SAVE}/${comp}/Analyse
	480	#
	481	i=0
	482	file=${NULL_STR}
	483	#
	484	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
	485	#
	486	# Determine in which output can we find file
	487	# IE : Analyse/TS or Analyse/DA2TS or ...
	488	#
	489	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
	490	case ${FlagDir} in
	491	*Y) TS_Dir=TS_YE ;;
	492	*M) TS_Dir=TS_MO ;;
	493	*D) TS_Dir=TS_DA ;;
	494	HF) TS_Dir=TS_HF ;;
	495	3H) TS_Dir=TS_HF ;;
	496	INS) TS_Dir=TS_INS ;;
	497	esac
	498	for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
[2]	499	#
[1033]	500	IGCM_sys_TestFileArchive ${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc
	501	if [ $? = 0 ] ; then
	502	liste_file[${#liste_file[*]}]=${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc
	503	else
	504	IGCM_debug_Print 1 "WARNING ${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc dont exist"
	505	fi
[785]	506	#
	507	done
[2]	508	done
[785]	509	if [ X${file} != X${NULL_STR} ] ; then
	510	IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
[1206]	511	IGCM_debug_Verif_Exit
[785]	512	fi
	513	done
	514	#
	515	LEVEL=0
	516	#
[2]	517	fi
	518
	519	########################################################################
	520	#
	521	# DEFINE LOOP PARAMETERS
	522	#
	523	########################################################################
	524
	525	DATE_COUNT=${DATE_FIN_JOB_B}
	526	DATE_FIN_JOB_B_LOOP=${DATE_FIN_JOB_B}
	527
	528	( [ -n "${LEVEL}" ] && [ ${LEVEL} -eq 0 ] ) && NBRE_FILE_TOT=0 \|\| NBRE_FILE_TOT=1
	529
	530	if [ ${DoJob} = true ] ; then
[785]	531	while [ ${DATE_COUNT} -lt ${PeriodDateEnd} ] ; do
	532	(( NBRE_FILE_TOT = NBRE_FILE_TOT + 1 ))
	533	Length=$( IGCM_date_DaysInNextPeriod ${DATE_COUNT} ${config_UserChoices_PeriodLength} )
	534	DATE_COUNT=$( IGCM_date_AddDaysToGregorianDate ${DATE_COUNT} ${Length} )
	535	done
[2]	536	else
[785]	537	NBRE_FILE_TOT=0
[2]	538	fi
	539
	540	if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then
[906]	541	NBRE_TOUR=1
[2]	542	elif [ ${NBRE_FILE_TOT}%${UNIX_MAX_LIMIT} -eq 0 ] ; then
[906]	543	let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT}
[2]	544	else
[785]	545	let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT}+1
[2]	546	fi
	547
	548	CURRENT_LOOP=1
	549	# THIS LOOP GIVES PARAMETERS FOR THE DATE'S LOOP ie : DATE_COURANTE, DATE_FIN.
[906]	550	# WE HAVE 3 LEVELS
[2]	551	# - LEVEL 1 : JUST ONE LOOP TO COMPLETE
	552	# - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
	553	# - LEVEL 3 : WE ARE IN THE LAST LOOP
	554
	555	while [ ${CURRENT_LOOP} -le ${NBRE_TOUR} ] ; do
	556
[906]	557	if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then
[785]	558	# - LEVEL 1 : JUST ONE LOOP TO COMPLETE
	559	NBRE_FILE_LOOP=${NBRE_FILE_TOT}
[2]	560
[785]	561	if [ ${FIRST_PASS} = TRUE ] ; then
	562	DATE_COURANTE=${DATE_FIN_JOB_B_LOOP}
	563	else
	564	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN_JOB_B_LOOP} ${config_UserChoices_PeriodLength} )
	565	DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
	566	fi
[2]	567
[785]	568	DATE_FIN=${PeriodDateEnd}
	569	DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP}
[2]	570
[785]	571	[ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
	572	[ -n "${LEVEL}" ] && FLAG_B="TRUE" \|\| FLAG_B="FALSE"
[2]	573
[785]	574	LEVEL=1
[906]	575	DEPOT="TRUE"
[785]	576	elif [ ${CURRENT_LOOP} -ne ${NBRE_TOUR} ] ; then
	577	# - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
	578	NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
[2]	579
[785]	580	if [ ! ${FIRST_PASS} = TRUE ] && [ ${CURRENT_LOOP} -eq 1 ] ; then
	581	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN_JOB_B_LOOP} ${config_UserChoices_PeriodLength} )
	582	DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
	583	elif [ -z "${DATE_COURANTE}" ] ; then
	584	DATE_COURANTE=${DATE_FIN_JOB_B}
[2]	585	else
[785]	586	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN} ${config_UserChoices_PeriodLength} )
	587	DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
	588	fi
[2]	589
[785]	590	(( TotLength = 0 ))
	591	COMPTEUR=1
	592	DATE_LOOP=${DATE_COURANTE}
[2]	593
[785]	594	while [ ${COMPTEUR} -lt ${NBRE_FILE_LOOP} ] ; do
	595	#
	596	Length=$( IGCM_date_DaysInNextPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} )
	597	DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length} )
	598	(( TotLength = TotLength + Length ))
	599	(( COMPTEUR = COMPTEUR + 1 ))
	600	#
	601	done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
	602	DATE_FIN=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} ${TotLength} )
[2]	603
[785]	604	[ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
	605	[ -n "${LEVEL}" ] && [ ${LEVEL} -eq 0 ] && FLAG_B="TRUE" \|\| [ ${CURRENT_LOOP} -gt 1 ] && FLAG_B="TRUE" \|\| FLAG_B="FALSE"
	606	DEPOT="FALSE"
	607	LEVEL=2
	608	else
	609	# - LEVEL 3 : WE ARE IN THE LAST LOOP
	610	NBRE_FILE_LOOP=$(( ${NBRE_FILE_TOT} % ( ${UNIX_MAX_LIMIT} * ( ${CURRENT_LOOP} - 1 ) ) ))
	611	[ ${NBRE_FILE_LOOP} -eq 0 ] && NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
[2]	612
[785]	613	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN} ${config_UserChoices_PeriodLength} )
	614	DATE_COURANTE=$(IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
	615	DATE_FIN=${PeriodDateEnd}
	616	[ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
[2]	617
[785]	618	LEVEL=3
	619	DEPOT="TRUE"
	620	FLAG_B="TRUE"
	621	fi
[2]	622
[785]	623	# THEN FOR EACH FILE TYPE WE LOOP BETWEEN DATE_COURANTE AND DATE_FIN.
	624	for comp in ${config_ListOfComponents[*]} ; do
	625	#
	626	IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
	627	#
	628	IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
	629	#
	630	eval R_OUT_${comp}=${R_SAVE}/${comp}
	631	#
	632	i=0
	633	#
	634	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
	635	#
	636	# Determine in which output we can find file
	637	# IE : Output/MO or Output/DA or ...
	638	#
	639	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
	640	case ${FlagDir} in
	641	*Y) TS_Dir=TS_YE ; FreqDir=YE ;;
	642	*M) TS_Dir=TS_MO ; FreqDir=MO ;;
	643	*D) TS_Dir=TS_DA ; FreqDir=DA ;;
	644	HF) TS_Dir=TS_HF ; FreqDir=HF ;;
	645	3H) TS_Dir=TS_HF ; FreqDir=HF ;;
	646	INS) TS_Dir=TS_INS ; FreqDir=INS ;;
	647	*) IGCM_debug_Print 1 "Error in FlagDir=${FlagDir} for file ${file}."
[1205]	648	IGCM_debug_Exit "Job create_ts" ;;
[785]	649	esac
[2]	650
[785]	651	# Initialize array
	652	unset liste_file
	653	unset liste_file_tmp
	654	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
[2]	655
[785]	656	COMPTEUR=1
[817]	657	MissingFile=FALSE
[785]	658	DATE_LOOP=${DATE_COURANTE}
[2]	659
[785]	660	while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] ; do
	661	#
	662	Length1=$( IGCM_date_DaysInPreviousPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} end)
	663	DATE_TAB=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} $(( 1 - Length1 )) )_${DATE_LOOP}
[2]	664
[785]	665	Length2=$( IGCM_date_DaysInNextPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} )
	666	DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length2} )
[2]	667
[817]	668	IGCM_sys_TestFileArchive ${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
	669	if [ $? = 0 ] ; then
	670	liste_file[${#liste_file[*]}]=${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
	671	liste_file_tmp[${#liste_file_tmp[*]}]=${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
	672	else
	673	IGCM_debug_Print 1 "WARNING ${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc dont exist"
	674	MissingFile=TRUE
	675	fi
[785]	676	(( COMPTEUR = COMPTEUR + 1 ))
	677	#
	678	done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
[817]	679
	680	# We skip this file type if one of them is missing.
	681	if [ "${MissingFile}" = "TRUE" ] ; then
	682	IGCM_debug_Print 1 "WARNING We skip ${file} type processing"
	683	continue
	684	fi
	685
[785]	686	DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} -${Length2} )
[2]	687
[785]	688	# Get selected files
	689	IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
[1206]	690	IGCM_debug_Verif_Exit
[2]	691
[785]	692	# Apply IGCM_Patch if needed
	693	if [ ! X${RebuildFrequency} = Xtrue ] ; then
	694	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
[906]	695	for file_t in $( eval echo ${liste_file_tmp[*]} ); do
[785]	696	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
	697	IGCM_${Patch} ${file_t}
[1206]	698	IGCM_debug_Verif_Exit
[785]	699	done
	700	done
	701	fi
	702	fi
[2]	703
[785]	704	# WE CAN CONCATENATE FILES AT THIS POINT
	705	[ "${FLAG_B}" = "TRUE" ] && DATE_BUILD_B=${DateBegin}_${DATE_FIN_JOB_B_LOOP_PREC}
	706	Length=$( IGCM_date_DaysInPreviousPeriod ${DATE_COURANTE} ${config_UserChoices_PeriodLength} end)
	707	DATE_BUILD1=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} -$(( Length - 1 )) )
	708	DATE_BUILD=${DATE_BUILD1}_${DATE_FIN}
	709	DATE_BUILD_END=${DateBegin}_${DATE_FIN}
[514]	710
[785]	711	liste_coord=" "
	712	for axis in $( eval echo \${LISTE_AXIS_${file}[*]} ); do
	713	liste_coord=${liste_coord}${axis}","
	714	done
[287]	715
[785]	716	# Time axis must be the last dimension
	717	time_axis=${axis}
[388]	718
[785]	719	# SWITCH BETWEEN MODEL TO FIT PARTICULAR CASE
	720	for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
	721	#
[1033]	722	# We skip variables not in the first file of the list.
	723	# Seems overkill to test thel all... but will do if needed
	724	ncdump -hv ${var} ${liste_file_tmp[0]} > /dev/null 2>&1
	725	if [ ! $? = 0 ] ; then
	726	IGCM_debug_Print 1 "WARNING We skip ${var}, not present within ${liste_file_tmp[0]}"
	727	continue
	728	fi
	729	#
[785]	730	if [ "${FLAG_B}" = "TRUE" ] ; then
	731	# WE CONCATENATE WITH EXISTING FILES
	732	file1=${config_UserChoices_JobName}_${DATE_BUILD_B}_${FlagDir}_${var}.nc
	733	file_out=${config_UserChoices_JobName}_${DATE_BUILD_END}_${FlagDir}_${var}.nc
	734	IGCM_sys_ncrcat --hst -v ${liste_coord}${var} ${file1} ${liste_file_tmp[*]} ${file_out}
[1206]	735	IGCM_debug_Verif_Exit
[785]	736	#
	737	IGCM_sys_Rm ${file1}
	738	if [ ! "${DEPOT}" = "TRUE" ] ; then
	739	eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}}/Analyse/${TS_Dir}/${file_out}
	740	# "${file1} = file_before ?
	741	[ ! "${file1}" = "${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file1} ${file_out}
	742	fi
	743	else
	744	# OR NOT
[1047]	745	file_out=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
	746	IGCM_sys_ncrcat --hst -v ${liste_coord}${var} ${liste_file_tmp[*]} ${file_out}
[1206]	747	IGCM_debug_Verif_Exit
[785]	748	fi
	749
	750	#
	751	if [ "${DEPOT}" = "TRUE" ] ; then
[1042]	752	# WE PUT FINAL FILE ON FILE SYSTEM
	753	[ "${FLAG_B}" = "FALSE" ] && file_out=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
	754	file_before=${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc
[785]	755	#
[1042]	756	eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}}/Analyse/${TS_Dir}/${file_out}
	757	[ "${FLAG_B}" = "TRUE" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file_before} ${file_out}
	758	if [ ! "X${file1}" = "X" ] ; then
	759	[ ! "${file1}" = "${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file1} ${file_out}
	760	fi
	761	#
[1039]	762	[ -f ${file_out} ] && IGCM_sys_Rm ${file_out}
[785]	763	#
	764	fi
	765	#
[1038]	766	done # for var in ${liste_var} ; do
[785]	767	IGCM_sys_Rm ${liste_file_tmp[*]}
[1038]	768	done # for file in ${liste_file_brut} do
	769	done # for comp in ${config_ListOfComponents[*]} ; do
[785]	770	# GO AHEAD IN DEEPEST LOOP
	771	(( CURRENT_LOOP = CURRENT_LOOP + 1 ))
[1038]	772	done # while [ ${CURRENT_LOOP} -le ${NBRE_TOUR} ] do
[2]	773	# FINISH
	774
	775	# DODS copy
	776	for comp in ${config_ListOfComponents[*]} ; do
[1393]	777	for TS_Dir in TS_YE TS_MO TS_DA TS_HF TS_INS ; do
[785]	778	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/${TS_Dir}
	779	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/${TS_Dir}
	780	done
[2]	781	done
	782
[1112]	783	export DEBUG_debug; export BigBrother ; export libIGCM; export SUBMIT_DIR; export POST_DIR; export Script_Output ;
	784	export StandAlone ; export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ;
	785	export RESOL_SBG
	786	listVarEnv="DEBUG_debug,BigBrother,libIGCM,SUBMIT_DIR,POST_DIR,Script_Post_Output,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG"
[2]	787	export listVarEnv
	788
	789	#D-
	790	# --------------------------------------------------------------------
	791	#D- Test if all was right
	792	# --------------------------------------------------------------------
[1206]	793	IGCM_debug_Verif_Exit
[2]	794	# --------------------------------------------------------------------
	795	#D- Submit MONITORING
	796	# --------------------------------------------------------------------
[414]	797	if ( [ X${TsTask} = X2D ] \|\| [ X${TsTask} = X3D ] \|\| [ X${TsTask} = X ] ) ; then
[1439]	798	# From 2D and 3D it is the last one to finish that will submit the monitoring.
	799	if [ ! "X$( grep "IGCM_sys_RmRunDir" ${POST_DIR}/create_ts.${PeriodDateEnd}.[23]D.out 2> /dev/null )" = "X" ] ; then
	800	Script_Post_Output=monitoring.${PeriodDateEnd}
	801	IGCM_sys_QsubPost monitoring
	802	fi
[387]	803	fi
[2]	804
	805	#D-
	806	# --------------------------------------------------------------------
	807	#D- Update the run.card
	808	# --------------------------------------------------------------------
	809	if [ ${StandAlone} != true ] ; then
[785]	810	# Put in run.card end period of time series
	811	[ X"${DATE_FIN}" = X ] \|\| IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted "${DATE_FIN}"
	812	# We have finish the job !
	813	IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "n"
[2]	814	fi
[203]	815
[590]	816	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
[203]	817	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
	818
[1198]	819	# ------------------------------------------------------------------
	820	# Finalize BigBrother to inform that the jobs end
	821	# ------------------------------------------------------------------
	822	IGCM_debug_BigBro_Finalize
	823
[2]	824	date

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

Original Format