Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_ada.ksh @ 1296

Last change on this file since 1296 was 1296, checked in by sdipsl, 8 years ago
Bugfix on ciclad. MPMD was broken. SPMD hybrid MPI/OpenMP was partly broken. cleaning
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 42.1 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sébastien Denvil
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Ada
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=ada
71	# project name
72	typeset PROJECT=$(echo ${LOADL_STEP_GROUP:=NONE})
73	# jobWarningDelay in seconds
74	typeset jobWarningDelay=$( TZ=utc date -d '1970-01-01 '${wall_clock_limit} +%s )
75
76	#D-
77	#D-#==================================================
78	#D-Program used in libIGCM
79	#D-#==================================================
80
81	# Submit command
82	typeset SUBMIT=${SUBMIT:=llsubmit}
83	# rsync with path
84	typeset -r RSYNC=/usr/bin/rsync
85	# RSYNC_opt args to rsync
86	typeset -r RSYNC_opt="-va"
87	# ie storage filesystem
88	typeset -r STOREHOST=ergon
89	typeset -r REMOTE_RSYNC=/usr/bin/rsync
90
91	#====================================================
92	# Set environment tools (ferret, nco, cdo, rebuild, ...)
93	#====================================================
94	source /smplocal/pub/Modules/default/init/ksh
95	. /smphome/rech/psl/rpsl035/.atlas_env_ada_intel_2013_0_bash > /dev/null 2>&1
96	[ ! X${TaskType} = Xchecking ] && IGCM_debug_Print 1 "List of loaded modules:"
97	[ ! X${TaskType} = Xchecking ] && module list
98
99	export PATH=${PATH}:/smphome/rech/psl/rpsl035/AddNoise/src_X64_ADA/bin
100	export PATH=${PATH}:/smphome/rech/psl/rpsl035/AddPerturbation/src_X64_ADA/bin
101	export PATH=${PATH}:/smphome/rech/psl/rpsl035/bin/
102
103	#====================================================
104	# Host specific DIRECTORIES
105	#====================================================
106
107	#====================================================
108	#- MirrorlibIGCM for frontend
109	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
110
111	#====================================================
112	#- libIGCM_POST for frontend
113	typeset -r libIGCM_POST=${libIGCM}
114
115	#====================================================
116	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
117	typeset -r R_EXE="${MODIPSL}/bin"
118
119	#====================================================
120	#- SUBMIT_DIR : submission dir
121	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${LOADL_STEP_INITDIR}}
122
123	#====================================================
124	#- IN
125	typeset -r R_IN=${R_IN:=/workgpfs/rech/psl/rpsl035/IGCM}
126	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/u/rech/psl/rpsl376}
127
128	#====================================================
129	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
130	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${TMPDIR}}
131
132	#====================================================
133	#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
134	typeset -r OUTCOMMAND_PATH=${RUN_DIR_PATH:-/tmp}
135
136	#====================================================
137	#- HOST_MPIRUN_COMMAND
138	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time poe"}
139
140	#====================================================
141	#- Max number of arguments passed to nco operator or demigration command
142	UNIX_MAX_LIMIT=360
143
144	#====================================================
145	#- set PackDefault to true on ada
146	PackDefault=true
147
148	#====================================================
149	#- Default number of MPI task for IPSL coupled model
150	#- required for backward compatibility
151	#-
152	DEFAULT_NUM_PROC_OCE=5
153	DEFAULT_NUM_PROC_CPL=1
154	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
155
156	#D-#==================================================
157	#D-function IGCM_sys_defineArchives
158	#D-* Purpose:
159	#D-* Define ARCHIVE : Dedicated to large files
160	#D-* Define STORAGE : Dedicated to small/medium files
161	#D-* Define R_OUT : Output tree located on ARCHIVE
162	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
163	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
164	#D-* if SpaceName=TEST everything is stored on WORKDIR
165	#D-* Examples:
166	#D-
167	function IGCM_sys_defineArchives {
168	IGCM_debug_PushStack "IGCM_sys_defineArchives"
169
170	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
171	#====================================================
172	#- ARCHIVE (dedicated to large files)
173	ARCHIVE=${config_UserChoices_ARCHIVE}
174	else
175	#====================================================
176	#- ARCHIVE (dedicated to large files)
177	ARCHIVE=$(echo ${HOME} \| sed -e "s,/.*home/,/arch/home/,")
178	fi
179
180	if [ ! X${config_UserChoices_STORAGE} = X ]; then
181	#====================================================
182	#- STORAGE (dedicated to small/medium files)
183	STORAGE=${config_UserChoices_STORAGE}
184	else
185	#====================================================
186	#- STORAGE (dedicated to small/medium files)
187	STORAGE=${WORKDIR}
188	fi
189
190	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
191	#====================================================
192	#- R_OUT
193	R_OUT=${WORKDIR}/IGCM_OUT
194
195	#====================================================
196	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
197	R_FIG=${WORKDIR}/IGCM_OUT
198
199	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
200
201	else
202	#====================================================
203	#- R_OUT
204	R_OUT=${ARCHIVE}/IGCM_OUT
205
206	#====================================================
207	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
208	R_FIG=${ARCHIVE}/IGCM_OUT
209	fi
210
211	#====================================================
212	#- R_BUF (ONLY FOR double copy an scratch)
213	R_BUF=${WORKDIR}/IGCM_OUT
214
215	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
216	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
217	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
218
219	IGCM_debug_PopStack "IGCM_sys_defineArchives"
220	}
221
222	#D-#==================================================
223	#D-function IGCM_sys_RshArchive
224	#D-* Purpose: Archive rsh command
225	#D-* Examples:
226	#D-
227	function IGCM_sys_RshArchive {
228	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
229	DEBUG_sys=false IGCM_sys_IsFileArchived "$@"
230	if [ $? = 0 ] ; then
231	rsh ${STOREHOST} exec /bin/ksh <<-EOF
232	${@}
233	EOF
234	status=$?
235	else
236	/bin/ksh <<-EOF
237	${@}
238	EOF
239	status=$?
240	fi
241	if [ ${status} -gt 0 ] ; then
242	IGCM_debug_Print 2 "IGCM_sys_RshArchive : rsh or command failed error code ${status}"
243	IGCM_debug_Exit "IGCM_sys_RshArchive"
244	fi
245	IGCM_debug_PopStack "IGCM_sys_RshArchive"
246	}
247
248	#D-#==================================================
249	#D-function IGCM_sys_RshArchive_NoError
250	#D-* Purpose: Archive rsh command, without error
251	#D-* used only in monitoring.job
252	#D-* Examples:
253	#D-
254	function IGCM_sys_RshArchive_NoError {
255	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
256	DEBUG_sys=false IGCM_sys_IsFileArchived "$@"
257	if [ $? = 0 ] ; then
258	rsh ${STOREHOST} exec /bin/ksh <<-EOF
259	${@} 2> /dev/null
260	EOF
261	else
262	/bin/ksh <<-EOF
263	${@} 2> /dev/null
264	EOF
265	fi
266	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
267	}
268
269	#D-#==================================================
270	#D-function IGCM_sys_MkdirArchive
271	#D-* Purpose: Mkdir on Archive
272	#D-* Examples:
273	#D-
274	function IGCM_sys_MkdirArchive {
275	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
276	if ( $DEBUG_sys ) ; then
277	echo "IGCM_sys_MkdirArchive :" $@
278	fi
279	#- creation de repertoire sur le serveur fichier
280	DEBUG_sys=false IGCM_sys_IsFileArchived $1
281	if [ $? = 0 ] ; then
282	rsh ${STOREHOST} -n mkdir -p $1
283	status=$?
284	else
285	mkdir -p $1
286	status=$?
287	fi
288
289	if [ ${status} -gt 0 ] ; then
290	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : rsh or mkdir failed error code ${status}"
291	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
292	fi
293	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
294	}
295
296	#D-#==================================================
297	#D-function IGCM_sys_TestDirArchive
298	#D-* Purpose: Test Directory that must exists on Archive
299	#D-* Examples:
300	#D-
301	function IGCM_sys_TestDirArchive {
302	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
303	if ( $DEBUG_sys ) ; then
304	echo "IGCM_sys_TestDirArchive :" $@
305	fi
306	typeset ExistFlag
307	#Command depends on targeted file system
308	DEBUG_sys=false IGCM_sys_IsFileArchived $1
309	if [ $? = 0 ] ; then
310	ExistFlag=$( IGCM_sys_RshArchive "[ -d $1 ] && echo 0 \|\| echo 1" )
311	else
312	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
313	fi
314	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
315	return ${ExistFlag}
316	}
317
318	#D-#==================================================
319	#D-function IGCM_sys_IsFileArchived
320	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
321	#D-* Examples:
322	#D-
323	function IGCM_sys_IsFileArchived {
324	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
325	if ( $DEBUG_sys ) ; then
326	echo "IGCM_sys_IsFileArchived :" $@
327	fi
328	typeset IsArchivedFlag
329	IsArchivedFlag=$( [ "X$( echo $@ \| grep \/arch\/home )" != "X" ] && echo 0 \|\| echo 1 )
330	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
331
332	return ${IsArchivedFlag}
333	}
334
335	#D-#==================================================
336	#D-function IGCM_sys_TestFileArchive
337	#D-* Purpose: Test file that must NOT EXISTS on Archive
338	#D-* Examples:
339	#D-
340	function IGCM_sys_TestFileArchive {
341	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
342	typeset ExistFlag
343	ExistFlag=$( IGCM_sys_RshArchive "[ -f $1 ] && echo 0 \|\| echo 1" )
344	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
345
346	return ${ExistFlag}
347	}
348
349	#D-#==================================================
350	#D-function IGCM_sys_CountFileArchive
351	#D-* Purpose: Count files on Archive filesystem
352	#D-* Examples:
353	#D-
354	function IGCM_sys_CountFileArchive {
355	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
356	#Command depends on targeted file system
357	DEBUG_sys=false IGCM_sys_IsFileArchived $1
358	if [ $? = 0 ] ; then
359	IGCM_sys_RshArchive "ls ${@} 2>/dev/null \| wc -l"
360	status=$?
361	else
362	ls ${@} 2>/dev/null \| wc -l
363	status=$?
364	fi
365	if [ ${status} -gt 0 ] ; then
366	echo "IGCM_sys_CountFileArchive : erreur."
367	fi
368	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
369	}
370
371	#D-#==================================================
372	#D-function IGCM_sys_Tree
373	#D-* Purpose: Tree directories with files on ${ARCHIVE}
374	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
375	#D-
376	function IGCM_sys_Tree {
377	IGCM_debug_PushStack "IGCM_sys_Tree" $@
378	if ( $DEBUG_sys ) ; then
379	echo "IGCM_sys_Tree :" $@
380	fi
381
382	\mfls -R $@
383
384	IGCM_debug_PopStack "IGCM_sys_Tree"
385	}
386
387	#D-#==================================================
388	#D-function IGCM_sys_Qsub
389	#D-* Purpose: Qsub new job
390	#D-* Examples:
391	#D-
392	function IGCM_sys_Qsub {
393	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
394	if ( $DEBUG_sys ) ; then
395	echo "IGCM_sys_Qsub :" $@
396	fi
397	typeset status
398
399	# We have to change output/error file
400	[ ${#@} = 1 ] && REP_FOR_JOB=${SUBMIT_DIR}
401	[ ${#@} = 2 ] && REP_FOR_JOB=${2}
402	sed -e "s:\# \@ output = .:\# \@ output = ${Script_Output}:" \
403	-e "s:\# \@ error = .:\# \@ error = ${Script_Output}:" \
404	$1 > ${REP_FOR_JOB}/JOB_FOR_IGCM
405	cd $REP_FOR_JOB ; /usr/bin/llsubmit JOB_FOR_IGCM > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1 ; status=$? ; cd - ;
406
407	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
408	if [ ${status} -gt 0 ] ; then
409	IGCM_debug_Print 2 "IGCM_sys_Qsub $1 : error code ${status}"
410	IGCM_debug_Exit "IGCM_sys_Qsub"
411	else
412	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ \| tail -1 \| sed -e s/\"//g )
413	IGCM_sys_Rm ${REP_FOR_JOB}/JOB_FOR_IGCM
414	fi
415	IGCM_debug_PopStack "IGCM_sys_Qsub"
416	}
417
418	#D-#==================================================
419	#D-function IGCM_sys_QsubPost
420	#D-* Purpose: Qsub new job on scalaire
421	#D-* Examples:
422	#D-
423	function IGCM_sys_QsubPost {
424	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
425	if ( $DEBUG_sys ) ; then
426	echo "IGCM_sys_QsubPost :" $@
427	fi
428	typeset status
429
430	# We have to change output/error file
431	[ ${#@} = 1 ] && REP_FOR_JOB=${POST_DIR}
432	[ ${#@} = 2 ] && REP_FOR_JOB=${2}
433
434	sed -e "s:\# \@ output = .:\# \@ output = ${Script_Post_Output}.out:" \
435	-e "s:\# \@ error = .:\# \@ error = ${Script_Post_Output}.out:" \
436	${libIGCM_POST}/$1.job > ${REP_FOR_JOB}/JOB_FOR_IGCM_$$
437
438	cd $REP_FOR_JOB ; /usr/bin/llsubmit JOB_FOR_IGCM_$$ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1 ; status=$? ; cd - ;
439
440	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
441	if [ ${status} -gt 0 ] ; then
442	IGCM_debug_Print 2 "IGCM_sys_QsubPost $1 : error code ${status}"
443	IGCM_debug_Exit "IGCM_sys_QsubPost"
444	else
445	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ \| tail -1 \| sed -e s/\"//g )
446	IGCM_sys_Rm ${REP_FOR_JOB}/JOB_FOR_IGCM_$$
447	fi
448	IGCM_debug_PopStack "IGCM_sys_QsubPost"
449	}
450
451	#D-*************************
452	#D- File transfer functions
453	#D-*************************
454	#D-
455
456	#D-#==================================================
457	#D-function IGCM_sys_RmRunDir
458	#D-* Purpose: rm tmpdir (dummy function most of the time batch
459	#D- scheduler will do the job)
460	#D-* Examples:
461	#D-
462	function IGCM_sys_RmRunDir {
463	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
464	if ( $DEBUG_sys ) ; then
465	echo "IGCM_sys_RmRunDir :" $@
466	IGCM_debug_Print 1 "Dummy call, let the scheduler do that."
467	fi
468	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
469	}
470
471	#D-#==================================================
472	#D-function IGCM_sys_Put_Dir
473	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
474	#D-* Examples:
475	#D-
476	function IGCM_sys_Put_Dir {
477	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
478	if ( $DEBUG_sys ) ; then
479	echo "IGCM_sys_Put_Dir :" $@
480	fi
481	if [ $DRYRUN = 0 ]; then
482	if [ ! -d ${1} ] ; then
483	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
484	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
485	return
486	fi
487
488	typeset status
489
490	# Only if we use rsync
491	#IGCM_sys_TestDirArchive $( dirname $2 )
492	#
493	#Command depends on targeted file system
494	DEBUG_sys=false IGCM_sys_IsFileArchived $2
495	if [ $? = 0 ] ; then
496	\rcp -r $1 ${STOREHOST}:$2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
497	status=$?
498	else
499	\cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
500	status=$?
501	fi
502
503	if [ ${status} -gt 0 ] ; then
504	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : rcp or cp failed error code ${status}"
505	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
506	IGCM_debug_Exit "IGCM_sys_Put_Dir"
507	else
508	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
509	fi
510	fi
511	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
512	}
513
514	#D-#==================================================
515	#D-function IGCM_sys_Get_Dir
516	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
517	#D-* Examples:
518	#D-
519	function IGCM_sys_Get_Dir {
520	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
521	if ( $DEBUG_sys ) ; then
522	echo "IGCM_sys_Get_Dir :" $@
523	fi
524	if [ $DRYRUN = 0 ]; then
525	typeset NB_ESSAI DELAI status i
526	# number of tentative
527	NB_ESSAI=3
528	# time delay between tentative
529	DELAI=2
530
531	# Only if we use rsync
532	#IGCM_sys_TestDirArchive $( dirname $2 )
533	#
534	# Command depends on targeted filesystem
535	DEBUG_sys=false IGCM_sys_IsFileArchived $1
536	if [ $? = 0 ] ; then
537	# add dmget (to demigrate all offline files) to reduce time of this command :
538	#IGCM_sys_RshArchive "dmfind $1 -state MIG -o -state OFL -o -state PAR \| dmget -q -n"
539	i=0
540	while [ $i -lt $NB_ESSAI ] ; do
541	\rcp -rp ${STOREHOST}:$1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
542	status=$?
543	if [ ${status} -gt 0 ]; then
544	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : rcp failed error code ${status} ${i}/${NB_ESSAI}"
545	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
546	sleep $DELAI
547	else
548	break
549	fi
550	(( i = i + 1 ))
551	done
552	else
553	\cp -rp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
554	status=$?
555	if [ ${status} -gt 0 ] ; then
556	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
557	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
558	IGCM_debug_Exit "IGCM_sys_Get_Dir"
559	else
560	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
561	fi
562	fi
563	if [ ${status} -gt 0 ] ; then
564	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : (r)cp failed error code ${status}"
565	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
566	IGCM_debug_Exit "IGCM_sys_Get_Dir"
567	else
568	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
569	fi
570	fi
571	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
572	}
573
574	#D-#==================================================
575	#D-function IGCM_sys_Put_Rest
576	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
577	#D- File and target directory must exist.
578	#D-* Examples:
579	#D-
580	function IGCM_sys_Put_Rest {
581	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
582	if ( $DEBUG_sys ) ; then
583	echo "IGCM_sys_Put_Rest :" $@
584	fi
585	if [ $DRYRUN = 0 ]; then
586	if [ ! -f ${1} ] ; then
587	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
588	IGCM_debug_Exit "IGCM_sys_Put_Rest"
589	fi
590
591	typeset status
592	#
593	if [ X${JobType} = XRUN ] ; then
594	IGCM_sys_Chmod 444 ${1}
595	fi
596	#
597	#
598	#Command depends on targeted file system
599	DEBUG_sys=false IGCM_sys_IsFileArchived $2
600	if [ $? = 0 ] ; then
601	mfput $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
602	status=$?
603	else
604	IGCM_sys_MkdirArchive $( dirname $2 )
605	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
606	status=$?
607	fi
608
609	# #RSYNC WITH NETWORK RSH CALL
610	# IGCM_sys_MkdirArchive $( dirname $2 )
611	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
612	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
613
614	# #RSYNC WITH NFS USE
615	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
616	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
617
618	# status=$?
619	# IGCM_sys_Rsync_out $status
620
621	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
622	# (( status=status+$? ))
623
624	if [ ${status} -gt 0 ] ; then
625	IGCM_debug_Print 2 "IGCM_sys_Put_Rest : mfput or cp failed error code ${status}"
626	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
627	IGCM_debug_Exit "IGCM_sys_Put_Rest"
628	else
629	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
630	fi
631	fi
632	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
633	}
634
635	#D-#==================================================
636	#D-function IGCM_sys_Put_Out
637	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
638	#D-* Examples:
639	#D-
640	function IGCM_sys_Put_Out {
641	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
642	if ( $DEBUG_sys ) ; then
643	echo "IGCM_sys_Put_Out :" $@
644	fi
645
646	typeset status
647
648	if [ $DRYRUN = 0 ]; then
649	if [ ! -f ${1} ] ; then
650	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
651	IGCM_debug_PopStack "IGCM_sys_Put_Out"
652	return 1
653	fi
654	#
655	if [ X${JobType} = XRUN ] ; then
656	if [ X${3} = X ] ; then
657	IGCM_sys_Chmod 444 ${1}
658	fi
659	fi
660	#
661	#
662	#Command depends on targeted file system
663	DEBUG_sys=false IGCM_sys_IsFileArchived $2
664	if [ $? = 0 ] ; then
665	mfput $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
666	status=$?
667	else
668	IGCM_sys_MkdirArchive $( dirname $2 )
669	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
670	status=$?
671	fi
672
673	# #RSYNC WITH NETWORK RSH CALL
674	# IGCM_sys_MkdirArchive $( dirname $2 )
675	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
676	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
677
678	# #RSYNC WITH NFS USE
679	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
680	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
681
682	# status=$?
683	# IGCM_sys_Rsync_out $status
684
685	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
686	# (( status=status+$? ))
687
688	if [ ${status} -gt 0 ] ; then
689	IGCM_debug_Print 2 "IGCM_sys_Put_Out : mfput or cp failed error code ${status}"
690	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
691	IGCM_debug_Exit "IGCM_sys_Put_Out"
692	else
693	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
694	fi
695	fi
696	IGCM_debug_PopStack "IGCM_sys_Put_Out"
697	return 0
698	}
699
700	#D-#==================================================
701	#D-function IGCM_sys_Get
702	#D-* Purpose: Get a file from ${ARCHIVE}
703	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
704	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
705	function IGCM_sys_Get {
706	IGCM_debug_PushStack "IGCM_sys_Get" $@
707
708	typeset DEST status dm_liste
709
710	if ( $DEBUG_sys ) ; then
711	echo "IGCM_sys_Get :" $@
712	fi
713	if [ $DRYRUN -le 2 ]; then
714	if [ X${1} = X'/l' ] ; then
715	eval set +A dm_liste \${${2}}
716	else
717	dm_liste=${1}
718	fi
719	eval DEST=\${${#}}
720
721	# test if the (first) file is present in the old computation :
722	DEBUG_sys=false IGCM_sys_IsFileArchived ${dm_liste[0]}
723	if [ $? = 0 ] ; then
724	IGCM_sys_TestFileArchive ${dm_liste[0]}
725	status=$?
726	else
727	IGCM_sys_TestFileBuffer ${dm_liste[0]}
728	status=$?
729	fi
730
731	if [ ${status} -gt 0 ] ; then
732	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
733	IGCM_debug_Exit "IGCM_sys_Get"
734	return
735	fi
736
737	#Command depends on targeted file system
738	DEBUG_sys=false IGCM_sys_IsFileArchived ${dm_liste[0]}
739	if [ $? = 0 ] ; then
740	mfget ${dm_liste[*]} ${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
741	status=$?
742	else
743	\cp ${dm_liste[*]} ${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
744	status=$?
745	fi
746
747	# #RSYNC WITH NETWORK RSH CALL
748	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
749	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
750
751	# #RSYNC WITH NFS USE
752	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
753	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
754
755	# status=$?
756	# IGCM_sys_Rsync_out $status
757
758	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
759	# (( status=status+$? ))
760
761	if [ ${status} -gt 0 ] ; then
762	IGCM_debug_Print 2 "IGCM_sys_Get : mfget or cp failed error code ${status}"
763	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
764	IGCM_debug_Exit "IGCM_sys_Get"
765	else
766	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
767	fi
768	fi
769	IGCM_debug_PopStack "IGCM_sys_Get"
770	}
771
772	#D-#==================================================
773	#D-function IGCM_sys_GetDate_Monitoring
774	#D-* Purpose: get the last year for which the monitoring has been computed
775	#D-* Examples:
776	#D-
777	function IGCM_sys_GetDate_Monitoring {
778	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
779	if ( $DEBUG_sys ) ; then
780	echo "IGCM_sys_GetDate_Monitoring :" $@
781	fi
782
783	IGCM_sys_Cd /tmp
784	# NASTY HACK. We force /tmp here. This function is called only from the front-end and interactively.
785	RUN_DIR_PATH=/tmp IGCM_sys_Get ${1} .
786	eval ${2}=$( cdo showyear $( basename ${1} ) 2> /dev/null \| gawk '{ print $NF }' )
787	# Need to erase this file to avoid collision (permission denied getting file) between users on the front-end
788	IGCM_sys_Rm -f $( basename ${1} )
789	IGCM_sys_Cd -
790
791	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
792	}
793
794	#D-#==================================================
795	#D-function IGCM_sys_Put_Dods
796	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole.
797	#D-* Examples:
798	#D-
799	function IGCM_sys_Put_Dods {
800	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
801	if ( $DEBUG_sys ) ; then
802	echo "IGCM_sys_Put_Dods :" $@
803	fi
804	typeset status
805	if [ $DRYRUN = 0 ]; then
806	# We take our time on that
807	sleep 10
808	IGCM_sys_TestDirArchive ${R_SAVE}/${1}
809	if [ $? != 0 ] ; then
810	echo "WARNING : IGCM_sys_Put_Dods ${R_SAVE}/${1} DOES NOT EXIST ."
811	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
812	return
813	fi
814
815	rsh ${STOREHOST} exec /bin/ksh <<EOF
816	cd ${R_SAVE}
817	/arch/local/bin/dods_rm DODS/pub/${LOGIN}/${R_DODS}/${1} > /dev/null 2>&1
818	/bin/chmod -R u+w ${R_SAVE}/${1}
819	/arch/local/bin/dods_cp ${1} DODS/pub/${LOGIN}/${R_DODS} > /dev/null 2>&1
820	/bin/chmod -R +rX ${R_SAVE}/${1}
821	/bin/chmod -R u+w ${R_SAVE}/${1}
822	EOF
823	status=$?
824
825	if [ ${status} -gt 0 ] ; then
826	echo "IGCM_sys_Put_Dods : error."
827	IGCM_debug_Exit "IGCM_sys_Put_Dods"
828	fi
829	fi
830	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
831	}
832
833	##############################################################
834	# REBUILD OPERATOR
835
836	#D-#==================================================
837	#D-function IGCM_sys_sync
838	#D-* Purpose: flush buffer on disk (dummy function on Ada)
839	#D-* Examples:
840	#D-
841	function IGCM_sys_sync {
842	IGCM_debug_PushStack "IGCM_sys_sync" $@
843	if ( $DEBUG_sys ) ; then
844	echo "IGCM_sys_sync :" $@
845	echo "Dummy call, let the system do that."
846	fi
847	IGCM_debug_PopStack "IGCM_sys_sync"
848	}
849
850	############################################################
851	# Activate Running Environnment Variables
852
853	#D-#==================================================
854	#D-function IGCM_sys_activ_variables
855	#D-* Purpose: set environement variables prior to execution
856	#D-* Examples:
857	#D-
858	function IGCM_sys_activ_variables {
859	IGCM_debug_PushStack "IGCM_sys_activ_variables"
860	if ( $DEBUG_sys ) ; then
861	echo "IGCM_sys_activ_variables"
862	fi
863
864	# --------------------------------------------------------------------
865	#D- MPI specifications
866	# --------------------------------------------------------------------
867
868	# --------------------------------------------------------------------
869	#D- Other specifications
870	# --------------------------------------------------------------------
871
872	ulimit -s unlimited
873	## to be done only one time
874	## export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/smplocal/pub/NetCDF/4.1.3/lib:/smplocal/pub/HDF5/1.8.9/seq/lib
875	## echo ${LD_LIBRARY_PATH} \| grep -i netcdf >/dev/null 2>&1 \|\| export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/smplocal/pub/NetCDF/4.1.3/lib:/smplocal/pub/HDF5/1.8.9/seq/lib
876	IGCM_debug_Print 1 "set LD_LIBRARY_PATH=${LD_LIBRARY_PATH}"
877
878	export MP_STDOUTMODE=combined
879	IGCM_debug_Print 1 "set MP_STDOUTMODE=${MP_STDOUTMODE}"
880
881	## fix to reduce memory usage. Required since 2014/22/04. On ada IDRIS.
882	#export MP_EUILIBPATH=/smplocal/lib/ibmhpc/pe12012/ppe.pami/gnu/lib64/pami64
883	#IGCM_debug_Print 1 "set MP_EUILIBPATH=${MP_EUILIBPATH}"
884
885	IGCM_debug_PopStack "IGCM_sys_activ_variables"
886	}
887
888	############################################################
889	# Desactivate Running Environnment Variables
890
891	#D-#==================================================
892	#D-function IGCM_sys_desactiv_variables
893	#D-* Purpose: unset environement variables after execution
894	#D-* Examples:
895	#D-
896	function IGCM_sys_desactiv_variables {
897	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
898	if ( $DEBUG_sys ) ; then
899	echo "IGCM_sys_desactiv_variables"
900	fi
901	# --------------------------------------------------------------------
902	#D- MPI specifications
903	# --------------------------------------------------------------------
904
905	# --------------------------------------------------------------------
906	#D- Other specifications
907	# --------------------------------------------------------------------
908
909	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
910	}
911
912	############################################################
913	# Update job headers to be used by the scheduler
914
915	#D-#==================================================
916	#D-function IGCM_sys_updateHeaders
917	#D-* Purpose: Update job headers to be used by the scheduler
918	#D-* Examples: IGCM_sys_updateHeaders /path/to/Job_MYEXP
919	#D-
920	function IGCM_sys_updateHeaders {
921	IGCM_debug_PushStack "IGCM_sys_updateHeaders"
922	if ( $DEBUG_sys ) ; then
923	echo "IGCM_sys_updateHeaders"
924	fi
925	typeset file
926	file=$1
927
928	if [ ${executionType} -eq 1 ] ; then
929	# MPMD + MPI
930	sed -e "/::openMPthreads::/d" \
931	-e "s/::JobNumProcTot::/${coreNumber}/" \
932	${file} > ${file}.tmp
933
934	elif [ ${executionType} -eq 2 ] ; then
935	# MPMD + MPI + OMP
936	sed -e "s/::openMPthreads::/${openMPthreads}/" \
937	-e "s/::JobNumProcTot::/${mpiTasks}/" \
938	${file} > ${file}.tmp
939
940	elif [ ${executionType} -eq 3 ] ; then
941	# SPMD + MPI/OMP
942	sed -e "s/::openMPthreads::/${openMPthreads}/" \
943	-e "s/::JobNumProcTot::/${mpiTasks}/" \
944	${file} > ${file}.tmp
945
946	elif [ ${executionType} -eq 4 ] ; then
947	# SPMD + MPI only
948	sed -e "s/::JobNumProcTot::/${mpiTasks}/" \
949	-e "/::openMPthreads::/d" \
950	${file} > ${file}.tmp
951
952	elif [ ${executionType} -eq 5 ] ; then
953	# SPMD + OMP only
954	sed -e "s/::openMPthreads::/${openMPthreads}/" \
955	-e "s/@ job_type = parallel/@ job_type = serial/" \
956	-e "/::JobNumProcTot::/d" \
957	${file} > ${file}.tmp
958
959	elif [ ${executionType} -eq 6 ] ; then
960	# SEQUENTIAL THEN
961	sed -e "s/::JobNumProcTot::/1/" \
962	-e "s/@ job_type = parallel/@ job_type = serial/" \
963	-e "/::openMPthreads::/d" \
964	${file} > ${file}.tmp
965
966	fi
967
968	IGCM_sys_Mv ${file}.tmp ${file}
969
970	IGCM_debug_PopStack "IGCM_sys_updateHeaders"
971	}
972
973	############################################################
974	# Build MPI/OMP scripts run file (dummy function)
975
976	#D-#==================================================
977	#D-function IGCM_sys_build_run_file
978	#D-* Purpose: build run file (deprecated)
979	#D-* Examples:
980	#D-
981	function IGCM_sys_build_run_file {
982
983	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
984
985	}
986
987	############################################################
988	# Build MPI/OMP scripts
989
990	#D-#==================================================
991	#D-function IGCM_sys_build_execution_scripts
992	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
993	#D-* Examples:
994	#D-
995	function IGCM_sys_build_execution_scripts
996	{
997	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
998	if ( $DEBUG_sys ) ; then
999	echo "IGCM_sys_build_execution_scripts " $@
1000	fi
1001
1002	EXECUTION=${HOST_MPIRUN_COMMAND}
1003
1004	# MPMD mode
1005	if ( ${OK_PARA_MPMD} ) ; then
1006
1007	if [ -f run_file ] ; then
1008	IGCM_sys_Rm -f run_file
1009	fi
1010	touch run_file
1011
1012	# Build run_file
1013
1014	# First loop on the components for the coupler ie oasis (only if oasis3)
1015	# the coupler ie oasis3 must be the first one
1016	for comp in ${config_ListOfComponents[*]} ; do
1017
1018	eval ExeNameIn=\${config_Executable_${comp}[0]}
1019	eval ExeNameOut=\${config_Executable_${comp}[1]}
1020
1021	# for CPL component only
1022	if [ "X${comp}" = "XCPL" ] ; then
1023
1024	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1025	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1026
1027	if ( ${OK_PARA_MPI} ) ; then
1028	(( mpi_count = 1 ))
1029	until [ ${mpi_count} -gt ${comp_proc_mpi_loc} ] ; do
1030	echo "./${ExeNameOut}" >> run_file
1031	(( mpi_count = mpi_count + 1 ))
1032	done
1033	else
1034	echo "./${ExeNameOut} " >> run_file
1035	fi
1036	fi
1037	done
1038
1039	# Then second loop on the components
1040	for comp in ${config_ListOfComponents[*]} ; do
1041
1042	eval ExeNameIn=\${config_Executable_${comp}[0]}
1043	eval ExeNameOut=\${config_Executable_${comp}[1]}
1044
1045	# Only if we really have an executable for the component and not the coupler ie oasis:
1046	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
1047
1048	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1049	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1050
1051	if ( ${OK_PARA_MPI} ) ; then
1052	(( mpi_count = 1 ))
1053	until [ ${mpi_count} -gt ${comp_proc_mpi_loc} ] ; do
1054	if ( ${OK_PARA_OMP} ) ; then
1055	# Check if the number of threads is correct
1056	case ${comp_proc_omp_loc} in
1057	1\|2\|4)
1058	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
1059	;;
1060	8\|16)
1061	IGCM_debug_Exit "ERROR with OMP parameters !"
1062	IGCM_debug_Print 2 "Beware : ${comp_proc_omp_loc} is too much for MPMD mode"
1063	IGCM_debug_Print 2 "You will use more CPU than needed"
1064	IGCM_debug_Verif_Exit
1065	;;
1066	*)
1067	IGCM_debug_Exit "ERROR with OMP parameters !"
1068	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
1069	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
1070	IGCM_debug_Verif_Exit
1071	;;
1072	esac
1073	echo "env OMP_NUM_THREADS=$comp_proc_omp_loc ./${ExeNameOut}" >> run_file
1074	else
1075	echo "./${ExeNameOut}" >> run_file
1076	fi
1077	(( mpi_count = mpi_count + 1 ))
1078	done
1079	else
1080	echo "./${ExeNameOut} " >> run_file
1081	fi
1082	if ( ${OK_PARA_OMP} ) ; then
1083	export KMP_STACKSIZE=200m
1084	fi
1085	fi
1086	done
1087
1088	EXECUTION="${HOST_MPIRUN_COMMAND} -pgmmodel mpmd -cmdfile ./run_file"
1089
1090	IGCM_sys_Chmod u+x run_file
1091	if ( $DEBUG_sys ) ; then
1092	echo "run_file contains : "
1093	cat run_file
1094	fi
1095
1096	# Only one executable (SPMD mode).
1097	else
1098
1099	for comp in ${config_ListOfComponents[*]} ; do
1100
1101	# Only if we really have an executable for the component :
1102	eval ExeNameOut=\${config_Executable_${comp}[1]}
1103	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
1104
1105	if ( ${OK_PARA_OMP} ) ; then
1106	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1107	export KMP_STACKSIZE=200m
1108	#export KMP_LIBRARY=turnaround
1109	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1110	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
1111	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
1112	fi
1113	EXECUTION="${HOST_MPIRUN_COMMAND} ./${ExeNameOut}"
1114	fi
1115
1116	done
1117
1118	fi
1119
1120	IGCM_debug_Print 1 "sys ada : execution command is "
1121	IGCM_debug_Print 1 "$EXECUTION"
1122
1123	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
1124	}
1125
1126	#D-#==================================================
1127	#D-function IGCM_sys_check_path
1128	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
1129	#D-* do not point to an important use directory. Stop immediately in that case.
1130	#D-* Examples:
1131	#D-
1132	function IGCM_sys_check_path {
1133	IGCM_debug_PushStack "IGCM_sys_check_path"
1134	if ( $DEBUG_sys ) ; then
1135	echo "IGCM_sys_check_path"
1136	fi
1137
1138	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
1139	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
1140	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
1141	IGCM_debug_Exit "This will stop the job"
1142	fi
1143	IGCM_debug_PopStack "IGCM_sys_check_path"
1144	}
1145
1146	#D-#==================================================
1147	#D-function IGCM_sys_check_quota
1148	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
1149	#D-* Examples:
1150	#D-
1151	function IGCM_sys_check_quota {
1152	IGCM_debug_PushStack "IGCM_sys_check_quota"
1153	if ( $DEBUG_sys ) ; then
1154	echo "IGCM_sys_check_quota"
1155	fi
1156	# Limit of quota (in %)
1157	limit_quota=90
1158
1159	# Check of the volume
1160	volume_quota=$(quota_u -w 2>/dev/null\| grep 'Quota soft' \| gawk '{print $5}')
1161	if [ ! X${volume_quota} = X ] ; then
1162	quota_volume=${volume_quota%%\%}
1163	# echo $quota_volume
1164	if [ $(echo "${quota_volume} > ${limit_quota}" \| bc) -eq 1 ] ; then
1165	IGCM_debug_Print 1 "Please, check your quota of volume on workgpfs"
1166	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
1167	IGCM_debug_Print 1 "Use the quota_u -w command to check"
1168	IGCM_debug_Print 1 "You must have more than 10% available to run"
1169	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1170	IGCM_debug_Verif_Exit
1171	fi
1172
1173	fi
1174	IGCM_debug_PopStack "IGCM_sys_check_quota"
1175	}
1176
1177	#D-#==================================================
1178	#D-function IGCM_sys_projectAccounting
1179	#D-* Purpose: store project accounting information in a file
1180	#D-* Examples:
1181	#D-
1182	function IGCM_sys_projectAccounting {
1183	IGCM_debug_PushStack "IGCM_sys_projectAccounting"
1184	if ( $DEBUG_sys ) ; then
1185	echo "IGCM_sys_check_quota"
1186	fi
1187
1188	cpt > $1
1189
1190	IGCM_debug_PopStack "IGCM_sys_projectAccounting"
1191	}
1192
1193	#D-#==================================================
1194	#D-function IGCM_sys_getJobSchedulerID
1195	#D-* Purpose: Get the job ID during execution
1196	#D-* Examples: IGCM_sys_getJobSchedulerID jobSchedulerID
1197	#D-
1198	function IGCM_sys_getJobSchedulerID {
1199	IGCM_debug_PushStack "IGCM_sys_getJobSchedulerID"
1200	if ( $DEBUG_sys ) ; then
1201	echo "IGCM_sys_getJobSchedulerID"
1202	fi
1203
1204	eval ${1}=$( echo $LOADL_STEP_ID \| awk -F. '{print $4}' )
1205
1206	IGCM_debug_PopStack "IGCM_sys_getJobSchedulerID"
1207	}
1208
1209	#D-#==================================================
1210	#D-function IGCM_sys_GetJobID
1211	#D-* Purpose: Get the job ID from the JobName
1212	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
1213	#D-
1214	function IGCM_sys_GetJobID {
1215	IGCM_debug_PushStack "IGCM_sys_GetJobID"
1216	if ( $DEBUG_sys ) ; then
1217	echo "IGCM_sys_GetJobID"
1218	fi
1219
1220	# Print only the full (-W) JobID (%id) and JobName (%jn)
1221	ID=$( llq -u $2 -W -f %id %jn \| \
1222	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
1223
1224	eval ${3}=${ID}
1225	IGCM_debug_PopStack "IGCM_sys_GetJobID"
1226	}
1227
1228	#D-#==================================================
1229	#D-function IGCM_sys_CountJobInQueue
1230	#D-* Purpose: Count number of users job
1231	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
1232	#D-
1233	function IGCM_sys_CountJobInQueue {
1234	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
1235	if ( $DEBUG_sys ) ; then
1236	echo "IGCM_sys_CountJobInQueue"
1237	fi
1238
1239	# Print only the full (-W) JobName (%jn)
1240	NbRun=$( llq -W -f %jn \| grep -c "$1" )
1241
1242	eval ${2}=${NbRun}
1243
1244	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
1245	}
1246
1247	#D-#==================================================
1248	#D-function IGCM_sys_ListJobInQueue
1249	#D-* Purpose: Produce a list of users jobs
1250	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
1251	#D-
1252	function IGCM_sys_ListJobInQueue {
1253	IGCM_debug cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1254	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1255	IGCM_debug_PopStack "IGCM_sys_atlas"
1256	return 1
1257	else
1258	IGCM_debug_PopStack "IGCM_sys_atlas"
1259	return 0
1260	fi
1261
1262	IGCM_debug_PopStack "IGCM_sys_atlas"
1263	}
1264
1265	_PushStack "IGCM_sys_ListJobInQueue"
1266	if ( $DEBUG_sys ) ; then
1267	echo "IGCM_sys_ListJobInQueue"
1268	fi
1269
1270	# With -W option, column width is as large as necessary
1271	set -A JobList $( llq -u $1 -W -f %jn \| head -n -2 \| tail -n +3 \| \
1272	gawk '( $1 != /TS/ && \
1273	$1 !~ /PACK/ && \
1274	$1 !~ /REBUILD/ && \
1275	$1 !~ /pack/ ) \
1276	{ print $1 }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
1277
1278	eval set -A ${2} ${JobList[*]}
1279
1280	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
1281	}
1282
1283	#D-#==================================================
1284	#D-function IGCM_sys_atlas
1285	#D-* Purpose: encapsulate atlas call so as to manage error code and curie specificity
1286	#D-* Examples:
1287	#D-
1288	function IGCM_sys_atlas {
1289	IGCM_debug_PushStack "IGCM_sys_atlas" $@
1290	if ( $DEBUG_sys ) ; then
1291	echo "IGCM_sys_atlas :" $@
1292	fi
1293
1294	typeset status
1295
1296	\atlas $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1297	status=$?
1298	if [ ${status} -gt 0 ] ; then
1299	echo "IGCM_sys_atlas : error code ${status}"
1300

Note: See TracBrowser for help on using the repository browser.

Download in other formats: