Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_default.ksh @ 1163

Last change on this file since 1163 was 1163, checked in by sdipsl, 9 years ago
Add IGCM_sys_sync function. Do a /bin/sync on Curie and a dummy call otherwise.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 72.1 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys Default host
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset -r MASTER=${HOST}
71
72	#D-
73	#D-#==================================================
74	#D-Program used in libIGCM
75	#D-#==================================================
76
77	# Submit command
78	typeset SUBMIT=${SUBMIT:=qsub}
79	# rsync with path
80	typeset -r RSYNC=/usr/bin/rsync
81	# RSYNC_opt args to rsync
82	typeset -r RSYNC_opt="-va"
83	# ie storage filesystem
84	typeset -r STOREHOST=${MASTER}
85
86	#====================================================
87	# Set environment tools (ferret, nco, cdo)
88	#====================================================
89	#. /home/${LOGIN}/.atlas_env_${HOST}_ksh
90
91	export PATH=${PATH}:/home/jripsl/snapshot/Monitoring/CNClient/
92
93	#====================================================
94	# Host specific DIRECTORIES
95	#====================================================
96
97	#====================================================
98	#- MirrorlibIGCM for frontend
99	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
100
101	#====================================================
102	#- libIGCM_POST for frontend
103	typeset -r libIGCM_POST=${libIGCM}
104
105	#====================================================
106	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
107	typeset -r R_EXE="${MODIPSL}/bin"
108
109	#====================================================
110	#- SUBMIT_DIR : submission dir
111	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PWD}}
112
113	#====================================================
114	#- IN
115	typeset -r R_IN=${R_IN:=/home/${LOGIN}/IGCM}
116	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/home/${LOGIN}/IGCM}
117
118	#====================================================
119	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
120	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/home/${LOGIN}/RUN_DIR/tmp$$}
121
122	#====================================================
123	#- HOST_MPIRUN_COMMAND
124	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="time mpirun"}
125
126	#====================================================
127	#- Max number of arguments passed to nco operator or demigration command
128	UNIX_MAX_LIMIT=360
129
130	#====================================================
131	#- Number of core per node (max number of OpenMP task)
132	NUM_COREPERNODE=1
133
134	#====================================================
135	#- Default number of MPI task for IPSL coupled model
136	#- required for backward compatibility
137	#-
138	DEFAULT_NUM_PROC_OCE=1
139	DEFAULT_NUM_PROC_CPL=1
140	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
141	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
142
143	#D-#==================================================
144	#D-function IGCM_sys_defineArchives
145	#D-* Purpose:
146	#D-* Define ARCHIVE : Dedicated to large files
147	#D-* Define STORAGE : Dedicated to small/medium files
148	#D-* Define R_OUT : Output tree located on ARCHIVE
149	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
150	#D-* Define R_BUF : Output tree located on STORAGE hosting files waiting for rebuild or pack processes
151	#D-* if SpaceName=TEST nothing special will hapen
152	#D-* Examples:
153	#D-
154	function IGCM_sys_defineArchives {
155	IGCM_debug_PushStack "IGCM_sys_defineArchives"
156
157	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
158	#====================================================
159	#- ARCHIVE (dedicated to large files)
160	ARCHIVE=${config_UserChoices_ARCHIVE}
161	else
162	#====================================================
163	#- ARCHIVE (dedicated to large files)
164	ARCHIVE=${ARCHIVE:=/home/${LOGIN}}
165	fi
166
167	if [ ! X${config_UserChoices_STORAGE} = X ]; then
168	#====================================================
169	#- STORAGE (dedicated to small/medium files)
170	STORAGE=${config_UserChoices_STORAGE}
171	else
172	#====================================================
173	#- STORAGE (dedicated to small/medium files)
174	STORAGE=${ARCHIVE}
175	fi
176
177	# ON OBELIX NO SPECIAL CASE WHEN X${config_UserChoices_SpaceName} = XTEST
178
179	#====================================================
180	#- R_OUT
181	R_OUT=${ARCHIVE}/IGCM_OUT
182
183	#====================================================
184	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
185	R_FIG=${STORAGE}/IGCM_OUT
186
187	#====================================================
188	#- R_BUF (ONLY FOR double copy an scratch)
189	R_BUF=${STORAGE}/IGCM_OUT
190
191	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
192	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
193	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
194
195	IGCM_debug_PopStack "IGCM_sys_defineArchives"
196	}
197
198	#D-#==================================================
199	#D-function IGCM_sys_RshMaster
200	#D-* Purpose: Connection to frontend machine.
201	#D-* Examples:
202	#D-
203	function IGCM_sys_RshMaster {
204	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
205	/bin/ksh <<-EOF
206	export libIGCM=${libIGCM}
207	export DEBUG_debug=${DEBUG_debug}
208	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
209	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
210	${@}
211	EOF
212	if [ $? -gt 0 ] ; then
213	echo "IGCM_sys_RshMaster : erreur."
214	IGCM_debug_Exit "IGCM_sys_RshMaster"
215	fi
216	IGCM_debug_PopStack "IGCM_sys_RshMaster"
217	}
218
219	#D-#==================================================
220	#D-function IGCM_sys_RshArchive
221	#D-* Purpose: Archive rsh command
222	#D-* Examples:
223	#D-
224	function IGCM_sys_RshArchive {
225	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
226	/bin/ksh <<-EOF
227	${@}
228	EOF
229	status=$?
230	if [ ${status} -gt 0 ] ; then
231	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
232	IGCM_debug_Exit "IGCM_sys_RshArchive"
233	fi
234	IGCM_debug_PopStack "IGCM_sys_RshArchive"
235	}
236
237	#D-#==================================================
238	#D-function IGCM_sys_RshPost
239	#D-* Purpose: Post-process rsh command
240	#D-* Examples:
241	#D-
242	function IGCM_sys_RshPost {
243	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
244	if ( $DEBUG_sys ) ; then
245	echo "IGCM_sys_RshPost :" $@
246	fi
247
248	# keep standard input (stdin) for the loop onto temporary file
249	cat >tmp_IGCM_sys_RshPost_$$
250
251	/bin/ksh <tmp_IGCM_sys_RshPost_$$
252	if [ $? -gt 0 ] ; then
253	echo "IGCM_sys_RshPost : erreur."
254	IGCM_debug_Exit "IGCM_sys_RshPost"
255	fi
256	# delete temporary file
257	\rm tmp_IGCM_sys_RshPost_$$
258	IGCM_debug_PopStack "IGCM_sys_RshPost"
259	}
260
261	#D-#==================================================
262	#D-function IGCM_sys_SendMail
263	#D-* Purpose: Send mail when simulation is over
264	#D-* Examples:
265	#D-
266	function IGCM_sys_SendMail {
267	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
268	if ( $DEBUG_sys ) ; then
269	echo "IGCM_sys_SendMail :" $@
270	fi
271
272	if [ X${1} = XAccounting ] ; then
273	status=Accounting
274	mailText=jobAccounting.mail
275	elif ( ${ExitFlag} ) ; then
276	status=failed
277	mailText=jobEnd.mail
278	else
279	status=completed
280	mailText=jobEnd.mail
281	fi
282
283	# Update selected mail template
284	while read -r line; do
285	eval echo $line >> mail.txt ;
286	done < ${libIGCM}/libIGCM_sys/${mailText}
287
288	if [ ! -z ${config_UserChoices_MailName} ] ; then
289	mailx -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
290	elif [ -f ~/.forward ] ; then
291	mailx -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
292	fi
293
294	sleep 10
295	rm -f mail.txt
296
297	IGCM_debug_PopStack "IGCM_sys_SendMail"
298	}
299
300	#D-#==================================================
301	#D-function IGCM_sys_Mkdir
302	#D-* Purpose: Master locale mkdir command
303	#D-* Examples:
304	#D-
305	function IGCM_sys_Mkdir {
306	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
307	if ( $DEBUG_sys ) ; then
308	echo "IGCM_sys_Mkdir :" $@
309	fi
310	if [ ! -d ${1} ]; then
311	\mkdir -p $1
312	if [ $? -gt 0 ] ; then
313	echo "IGCM_sys_Mkdir : erreur."
314	IGCM_debug_Exit "IGCM_sys_Mkdir"
315	fi
316	fi
317	# vérification :
318	if [ ! -d ${1} ] ; then
319	echo "IGCM_sys_Mkdir : erreur."
320	IGCM_debug_Exit "IGCM_sys_Mkdir"
321	fi
322	IGCM_debug_PopStack "IGCM_sys_Mkdir"
323	}
324
325	#D-#==================================================
326	#D-function IGCM_sys_MkdirArchive
327	#D-* Purpose: Mkdir on Archive
328	#D-* Examples:
329	#D-
330	function IGCM_sys_MkdirArchive {
331	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
332	if ( $DEBUG_sys ) ; then
333	echo "IGCM_sys_MkdirArchive :" $@
334	fi
335	#- creation de repertoire sur le serveur fichier
336	mkdir -p $1
337	status=$?
338
339	if [ ${status} -gt 0 ] ; then
340	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
341	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
342	fi
343	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
344	}
345
346	#D-#==================================================
347	#D-function IGCM_sys_MkdirWork
348	#D-* Purpose: Mkdir on Work
349	#D-* Examples:
350	#D-
351	function IGCM_sys_MkdirWork {
352	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
353	if ( $DEBUG_sys ) ; then
354	echo "IGCM_sys_MkdirWork :" $@
355	fi
356	#- creation de repertoire sur le serveur fichier
357	if [ ! -d ${1} ]; then
358	\mkdir -p $1
359	if [ $? -gt 0 ] ; then
360	echo "IGCM_sys_MkdirWork : erreur."
361	IGCM_debug_Exit "IGCM_sys_MkdirWork"
362	fi
363	fi
364	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
365	}
366
367	#D-#==================================================
368	#D-function IGCM_sys_Cd
369	#D-* Purpose: master cd command
370	#D-* Examples:
371	#D-
372	function IGCM_sys_Cd {
373	IGCM_debug_PushStack "IGCM_sys_Cd" $@
374	if ( $DEBUG_sys ) ; then
375	echo "IGCM_sys_Cd :" $@
376	fi
377	\cd $1
378	if [ $? -gt 0 ] ; then
379	echo "IGCM_sys_Cd : erreur."
380	IGCM_debug_Exit "IGCM_sys_Cd"
381	fi
382	IGCM_debug_PopStack "IGCM_sys_Cd"
383	}
384
385	#D-#==================================================
386	#D-function IGCM_sys_Chmod
387	#D-* Purpose: Chmod
388	#D-* Examples:
389	#D-
390	function IGCM_sys_Chmod {
391	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
392	if ( $DEBUG_sys ) ; then
393	echo "IGCM_sys_Chmod :" $@
394	fi
395	\chmod $@
396	if [ $? -gt 0 ] ; then
397	echo "IGCM_sys_Chmod : erreur."
398	IGCM_debug_Exit "IGCM_sys_Chmod"
399	fi
400	IGCM_debug_PopStack "IGCM_sys_Chmod"
401	}
402
403	#D-#==================================================
404	#D-function IGCM_sys_FileSize
405	#D-* Purpose: Filesize
406	#D-* Examples:
407	#D-
408	function IGCM_sys_FileSize {
409	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
410
411	typeset sizeF
412	set +A sizeF -- $( ls -la ${1} )
413	if [ $? -gt 0 ] ; then
414	IGCM_debug_Exit "IGCM_sys_FileSize"
415	fi
416	eval ${2}=${sizeF[4]}
417
418	IGCM_debug_PopStack "IGCM_sys_FileSize"
419	}
420
421	#D-#==================================================
422	#D-function IGCM_sys_TestDir
423	#D-* Purpose: Test Directory that must exists
424	#D-* Examples:
425	#D-
426	function IGCM_sys_TestDir {
427	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
428	if ( $DEBUG_sys ) ; then
429	echo "IGCM_sys_TestDir :" $@
430	fi
431	typeset ExistFlag
432	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
433	IGCM_debug_PopStack "IGCM_sys_TestDir"
434
435	return ${ExistFlag}
436	}
437
438	#D-#==================================================
439	#D-function IGCM_sys_TestDirArchive
440	#D-* Purpose: Test Directory that must exists on Archive
441	#D-* Examples:
442	#D-
443	function IGCM_sys_TestDirArchive {
444	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
445	if ( $DEBUG_sys ) ; then
446	echo "IGCM_sys_TestDirArchive :" $@
447	fi
448	typeset ExistFlag
449	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
450	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
451	return ${ExistFlag}
452	}
453
454	#D-#==================================================
455	#D-function IGCM_sys_IsFileArchived
456	#D-* Purpose: Test file that must NOT EXISTS on Archive
457	#D-* Examples:
458	#D-
459	function IGCM_sys_IsFileArchived {
460	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
461	if ( $DEBUG_sys ) ; then
462	echo "IGCM_sys_IsFileArchived :" $@
463	fi
464	typeset IsArchivedFlag
465	# Never archived for this system
466	IsArchivedFlag=1
467	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
468
469	return ${IsArchivedFlag}
470	}
471
472	#D-#==================================================
473	#D-function IGCM_sys_TestFileArchive
474	#D-* Purpose: Test file that must NOT EXISTS on Archive
475	#D-* Examples:
476	#D-
477	function IGCM_sys_TestFileArchive {
478	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
479	typeset ExistFlag
480	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
481	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
482
483	return ${ExistFlag}
484	}
485
486	#D-#==================================================
487	#D-function IGCM_sys_TestFileBuffer
488	#D-* Purpose: Test file that must NOT EXISTS on Buffer
489	#D-* Examples:
490	#D-
491	function IGCM_sys_TestFileBuffer {
492	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
493	typeset ExistFlag
494	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
495	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
496
497	return ${ExistFlag}
498	}
499
500	#D-#==================================================
501	#D-function IGCM_sys_CountFileArchive
502	#D-* Purpose: Count files on Archive filesystem
503	#D-* Examples:
504	#D-
505	function IGCM_sys_CountFileArchive {
506	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
507	ls ${@} 2>/dev/null \| wc -l
508	if [ $? -gt 0 ] ; then
509	echo "IGCM_sys_CountFileArchive : erreur."
510	fi
511	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
512	}
513
514	#D-#==================================================
515	#D-function IGCM_sys_CountFileBuffer
516	#D-* Purpose: Count files on Scratch filesystem
517	#D-* Examples:
518	#D-
519	function IGCM_sys_CountFileBuffer {
520	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
521	ls ${@} 2>/dev/null \| wc -l
522	if [ $? -gt 0 ] ; then
523	echo "IGCM_sys_CountFileBuffer : erreur."
524	fi
525	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
526	}
527
528	#D-#==================================================
529	#D-function IGCM_sys_Tree
530	#D-* Purpose: Tree directories with files on ${ARCHIVE}
531	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
532	#D-
533	function IGCM_sys_Tree {
534	IGCM_debug_PushStack "IGCM_sys_Tree" $@
535	if ( $DEBUG_sys ) ; then
536	echo "IGCM_sys_Tree :" $@
537	fi
538
539	\tree -f $@
540
541	IGCM_debug_PopStack "IGCM_sys_Tree"
542	}
543
544	#D-#==================================================
545	#D-function IGCM_sys_Tar
546	#D-* Purpose: master tar command
547	#D-* Examples:
548	#D-
549	function IGCM_sys_Tar {
550	IGCM_debug_PushStack "IGCM_sys_Tar" $@
551	if ( $DEBUG_sys ) ; then
552	echo "IGCM_sys_Tar :" $@
553	fi
554	\tar cf $@
555	if [ $? -gt 0 ] ; then
556	echo "IGCM_sys_Tar : erreur."
557	IGCM_debug_Exit "IGCM_sys_Tar"
558	fi
559	IGCM_debug_PopStack "IGCM_sys_Tar"
560	}
561
562	#D-#==================================================
563	#D-function IGCM_sys_UnTar
564	#D-* Purpose: master un-tar command
565	#D-* Examples:
566	#D-
567	function IGCM_sys_UnTar {
568	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
569	if ( $DEBUG_sys ) ; then
570	echo "IGCM_sys_UnTar :" $@
571	fi
572	\tar xvf $1
573	if [ $? -gt 0 ] ; then
574	echo "IGCM_sys_UnTar : erreur."
575	IGCM_debug_Exit "IGCM_sys_UnTar"
576	fi
577	IGCM_debug_PopStack "IGCM_sys_UnTar"
578	}
579
580	#D-#==================================================
581	#D-function IGCM_sys_Qsub
582	#D-* Purpose: Qsub new job
583	#D-* Examples:
584	#D-
585	function IGCM_sys_Qsub {
586	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
587	if ( $DEBUG_sys ) ; then
588	echo "IGCM_sys_Qsub :" $@
589	fi
590	echo "${@} > ${SUBMIT_DIR}/${Script_Output} 2>&1 &"
591	${@} > ${SUBMIT_DIR}/${Script_Output} 2>&1 &
592	if [ $? -gt 0 ] ; then
593	echo "IGCM_sys_Qsub : erreur ${options} $1"
594	IGCM_debug_Exit "IGCM_sys_Qsub"
595	fi
596	IGCM_debug_PopStack "IGCM_sys_Qsub"
597	}
598
599	#D-#==================================================
600	#D-function IGCM_sys_QsubPost
601	#D-* Purpose: Qsub new job on scalaire
602	#D-* Examples:
603	#D-
604	function IGCM_sys_QsubPost {
605	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
606	if ( $DEBUG_sys ) ; then
607	echo "IGCM_sys_QsubPost :" $@
608	fi
609	echo "${libIGCM_POST}/$1.job > ${POST_DIR}/$1.${PeriodDateEnd}.out 2>&1 &"
610	${libIGCM_POST}/$1.job > ${POST_DIR}/${Script_Post_Output}.out 2>&1 &
611	if [ $? -gt 0 ] ; then
612	echo "IGCM_sys_QsubPost : erreur " $@
613	IGCM_debug_Exit "IGCM_sys_QsubPost"
614	fi
615	IGCM_debug_PopStack "IGCM_sys_QsubPost"
616	}
617
618	#D-*************************
619	#D- File transfer functions
620	#D-*************************
621	#D-
622
623	#D-#==================================================
624	#D-function IGCM_sys_Rsync_out
625	#D-* Purpose: treat return val of rsync
626	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
627	#D- Error values and explanations can depend on your system version.
628	function IGCM_sys_Rsync_out {
629	status=$1
630	if [ ! $status ] ; then
631	echo "rsync error !"
632	fi
633
634	if [ $MYLANG = "fr" ]; then
635	case $status in
636	0) return ;;
637	1) echo "Erreur de rsync ; RERR_SYNTAX : "
638	echo "Erreur de syntaxe ou d'utilisation."
639	return;;
640	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
641	echo "Incompatibilité de protocole."
642	return;;
643	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
644	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
645	echo "répertoires"
646	return;;
647	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
648	echo "Action demandée non supportée : une tentative de manipulation de"
649	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
650	echo "été faite ; ou une option qui est supportée par le client mais"
651	echo "pas par le serveur a été spécifiée."
652	return;;
653	10) echo "Erreur de rsync ; RERR_SOCKETIO"
654	echo "Erreur dans le socket d'entrée sortie"
655	return;;
656	11) echo "Erreur de rsync ; RERR_FILEIO"
657	echo "Erreur d'entrée sortie fichier"
658	return;;
659	12) echo "Erreur de rsync ; RERR_STREAMIO"
660	echo "Erreur dans flux de donnée du protocole rsync"
661	return;;
662	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
663	echo "Erreur avec les diagnostics du programme"
664	return;;
665	14) echo "Erreur de rsync ; RERR_IPC"
666	echo "Erreur dans le code IPC"
667	return;;
668	20) echo "Erreur de rsync ; RERR_SIGNAL"
669	echo "SIGUSR1 ou SIGINT reçu"
670	return;;
671	21) echo "Erreur de rsync ; RERR_WAITCHILD"
672	echo "Une erreur retournée par waitpid()"
673	return;;
674	22) echo "Erreur de rsync ; RERR_MALLOC"
675	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
676	return;;
677	23) echo ""
678	echo "Erreur fichier inexistant"
679	return;;
680	30) echo "Erreur de rsync ; RERR_TIMEOUT"
681	echo "Temps d'attente écoulé dans l'envoi/réception de données"
682	return;;
683	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
684	return;;
685	esac
686	elif [ $MYLANG = "en" ] ; then
687	case $status in
688	0) return;;
689	1) echo "rsync error : Syntax or usage error "
690	return;;
691	2) echo "rsync error : Protocol incompatibility "
692	return;;
693	3) echo "rsync error : Errors selecting input/output files, dirs"
694	return;;
695	4) echo "rsync error : Requested action not supported: an attempt"
696	echo "was made to manipulate 64-bit files on a platform that cannot support"
697	echo "them; or an option was specified that is supported by the client and"
698	echo "not by the server."
699	return;;
700	5) echo "rsync error : Error starting client-server protocol"
701	return;;
702	10) echo "rsync error : Error in socket I/O "
703	return;;
704	11) echo "rsync error : Error in file I/O "
705	return;;
706	12) echo "rsync error : Error in rsync protocol data stream "
707	return;;
708	13) echo "rsync error : Errors with program diagnostics "
709	return;;
710	14) echo "rsync error : Error in IPC code "
711	return;;
712	20) echo "rsync error : Received SIGUSR1 or SIGINT "
713	return;;
714	21) echo "rsync error : Some error returned by waitpid() "
715	return;;
716	22) echo "rsync error : Error allocating core memory buffers "
717	return;;
718	23) echo "rsync error : Partial transfer due to error"
719	return;;
720	24) echo "rsync error : Partial transfer due to vanished source files"
721	return;;
722	30) echo "rsync error : Timeout in data send/receive "
723	return;;
724	*) echo "rsync error : return code of rsync unknown :" $status
725	return;;
726	esac
727	else
728	echo "unknown language $MYLANG."
729	return
730	fi
731	}
732
733	#D-#==================================================
734	#D-function IGCM_sys_Miror_libIGCM
735	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
736	#D-* Examples:
737	#D-
738	function IGCM_sys_Mirror_libIGCM {
739	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
740	if ( $DEBUG_sys ) ; then
741	echo "IGCM_sys_Mirror_libIGCM"
742	fi
743
744	typeset status
745
746	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
747
748	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
749	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
750	status=$?
751
752	if [ ${status} -gt 0 ] ; then
753	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
754	cat /tmp/out_command.$$
755	fi
756	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
757	}
758
759	#D-#==================================================
760	#D-function IGCM_sys_Cp
761	#D-* Purpose: generic cp
762	#D-* Examples:
763	#D-
764	function IGCM_sys_Cp {
765	IGCM_debug_PushStack "IGCM_sys_Cp" $@
766	if ( $DEBUG_sys ) ; then
767	echo "IGCM_sys_Cp :" $@
768	fi
769
770	typeset status
771
772	echo cp $@ > /tmp/out_command.$$ 2>&1
773	\cp $@ >> /tmp/out_command.$$ 2>&1
774	status=$?
775
776	if [ ${status} -gt 0 ] ; then
777	echo "IGCM_sys_Cp : error code ${status}"
778	cat /tmp/out_command.$$
779	IGCM_debug_Exit "IGCM_sys_Cp"
780	else
781	\rm /tmp/out_command.$$
782	fi
783	IGCM_debug_PopStack "IGCM_sys_Cp"
784	}
785
786	#D-#==================================================
787	#D-function IGCM_sys_Rm
788	#D-* Purpose: generic rm
789	#D-* Examples:
790	#D-
791	function IGCM_sys_Rm {
792	IGCM_debug_PushStack "IGCM_sys_Rm" $@
793	if ( $DEBUG_sys ) ; then
794	echo "IGCM_sys_Rm :" $@
795	fi
796
797	typeset status
798
799	echo rm $@ > /tmp/out_command.$$ 2>&1
800	\rm $@ >> /tmp/out_command.$$ 2>&1
801	status=$?
802
803	if [ ${status} -gt 0 ] ; then
804	echo "IGCM_sys_Rm : error code ${status}"
805	cat /tmp/out_command.$$
806	IGCM_debug_Exit "IGCM_sys_Rm"
807	else
808	\rm /tmp/out_command.$$
809	fi
810	IGCM_debug_PopStack "IGCM_sys_Rm"
811	}
812
813	#D-#==================================================
814	#D-function IGCM_sys_RmRunDir
815	#D-* Purpose: rm tmpdir (dummy function most of the time batch
816	#D- scheduler will do the job)
817	#D-* Examples:
818	#D-
819	function IGCM_sys_RmRunDir {
820	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
821	if ( $DEBUG_sys ) ; then
822	echo "IGCM_sys_RmRunDir :" $@
823	echo "Dummy call, let the user do that."
824	# IGCM_sys_Rm -rf ${RUN_DIR}
825	fi
826	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
827	}
828
829	#D-#==================================================
830	#D-function IGCM_sys_Mv
831	#D-* Purpose: generic move
832	#D-* Examples:
833	#D-
834	function IGCM_sys_Mv {
835	IGCM_debug_PushStack "IGCM_sys_Mv" $@
836	if ( $DEBUG_sys ) ; then
837	echo "IGCM_sys_Mv :" $@
838	fi
839
840	if [ $DRYRUN = 0 ]; then
841
842	typeset status
843
844	echo mv $@ > /tmp/out_command.$$ 2>&1
845	\mv $@ >> /tmp/out_command.$$ 2>&1
846	status=$?
847
848	if [ ${status} -gt 0 ] ; then
849	echo "IGCM_sys_Mv : error code ${status}"
850	cat /tmp/out_command.$$
851	IGCM_debug_Exit "IGCM_sys_Mv"
852	else
853	\rm /tmp/out_command.$$
854	fi
855	fi
856
857	IGCM_debug_PopStack "IGCM_sys_Mv"
858	}
859
860	#D-#==================================================
861	#D-function IGCM_sys_Put_Dir
862	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
863	#D-* Examples:
864	#D-
865	function IGCM_sys_Put_Dir {
866	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
867	if ( $DEBUG_sys ) ; then
868	echo "IGCM_sys_Put_Dir :" $@
869	fi
870	if [ $DRYRUN = 0 ]; then
871	if [ ! -d ${1} ] ; then
872	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
873	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
874	return
875	fi
876
877	typeset status
878
879	# Only if we use rsync
880	#IGCM_sys_TestDirArchive $( dirname $2 )
881	#
882	#USUAL WAY
883	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
884	status=$?
885
886	if [ ${status} -gt 0 ] ; then
887	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
888	cat /tmp/out_command.$$
889	IGCM_debug_Exit "IGCM_sys_Put_Dir"
890	else
891	\rm /tmp/out_command.$$
892	fi
893	fi
894	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
895	}
896
897	#D-#==================================================
898	#D-function IGCM_sys_Get_Dir
899	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
900	#D-* Examples:
901	#D-
902	function IGCM_sys_Get_Dir {
903	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
904	if ( $DEBUG_sys ) ; then
905	echo "IGCM_sys_Get_Dir :" $@
906	fi
907	if [ $DRYRUN = 0 ]; then
908	if [ ! -d ${1} ] ; then
909	echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
910	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
911	return
912	fi
913
914	typeset status
915
916	#USUAL WAY
917	\cp -ur $1 $2 > /tmp/out_command.$$ 2>&1
918	status=$?
919
920	if [ ${status} -gt 0 ] ; then
921	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
922	cat /tmp/out_command.$$
923	IGCM_debug_Exit "IGCM_sys_Get_Dir"
924	else
925	\rm /tmp/out_command.$$
926	fi
927	fi
928	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
929	}
930
931	#D-#==================================================
932	#D-function IGCM_sys_Get_Master
933	#D-* Purpose: Copy a complete directory from MASTER filesystem
934	#D-* Examples:
935	#D-
936	function IGCM_sys_Get_Master {
937	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
938	if ( $DEBUG_sys ) ; then
939	echo "IGCM_sys_Get_Master :" $@
940	fi
941	if [ $DRYRUN = 0 ]; then
942	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
943	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
944	IGCM_debug_PopStack "IGCM_sys_Get_Master"
945	return
946	fi
947
948	typeset NB_ESSAI DELAI status i
949	# number of tentative
950	NB_ESSAI=3
951	# time delay between tentative
952	DELAI=2
953
954	i=0
955	while [ $i -lt $NB_ESSAI ] ; do
956	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
957	status=$?
958	if [ ${status} -gt 0 ]; then
959	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
960	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
961	sleep $DELAI
962	else
963	break
964	fi
965	(( i = i + 1 ))
966	done
967
968	if [ ${status} -gt 0 ] ; then
969	echo "IGCM_sys_Get_Master : error."
970	cat /tmp/out_command.$$
971	IGCM_debug_Exit "IGCM_sys_Get_Master"
972	else
973	\rm /tmp/out_command.$$
974	fi
975	fi
976	IGCM_debug_PopStack "IGCM_sys_Get_Master"
977	}
978
979	#====================================================
980	#- Call IGCM_sys_Mirror_libIGCM now !
981	if ( $MirrorlibIGCM ) ; then
982	IGCM_sys_Mirror_libIGCM
983	fi
984
985	#D-#==================================================
986	#D-function IGCM_sys_Put_Rest
987	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
988	#D- File and target directory must exist.
989	#D-* Examples:
990	#D-
991	function IGCM_sys_Put_Rest {
992	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
993	if ( $DEBUG_sys ) ; then
994	echo "IGCM_sys_Put_Rest :" $@
995	fi
996	if [ $DRYRUN = 0 ]; then
997	if [ ! -f ${1} ] ; then
998	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
999	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1000	fi
1001
1002	typeset status
1003	#
1004	if [ X${JobType} = XRUN ] ; then
1005	IGCM_sys_Chmod 444 ${1}
1006	fi
1007	#
1008	# Only if we use rsync
1009	#IGCM_sys_MkdirArchive $( dirname $2 )
1010	#
1011	#USUAL WAY
1012	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1013	status=$?
1014
1015	# #RSYNC WITH NETWORK SSH CALL
1016	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1017	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1018
1019	# #RSYNC WITH NFS USE
1020	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1021	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1022
1023	# status=$?
1024	# IGCM_sys_Rsync_out $status
1025
1026	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1027	# (( status=status+$? ))
1028
1029	if [ ${status} -gt 0 ] ; then
1030	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
1031	cat /tmp/out_command.$$
1032	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1033	else
1034	\rm /tmp/out_command.$$
1035	fi
1036	fi
1037	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1038	}
1039
1040	#D-#==================================================
1041	#D-function IGCM_sys_PutBuffer_Rest
1042	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1043	#D- File and target directory must exist.
1044	#D-* Examples:
1045	#D-
1046	function IGCM_sys_PutBuffer_Rest {
1047	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1048	if ( $DEBUG_sys ) ; then
1049	echo "IGCM_sys_PutBuffer_Rest :" $@
1050	fi
1051	if [ $DRYRUN = 0 ]; then
1052	if [ ! -f ${1} ] ; then
1053	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1054	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1055	fi
1056
1057	typeset status
1058	#
1059	if [ X${JobType} = XRUN ] ; then
1060	IGCM_sys_Chmod 444 ${1}
1061	fi
1062
1063	#
1064	# USUAL WAY
1065	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1066	status=$?
1067
1068	if [ ${status} -gt 0 ] ; then
1069	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1070	[ -f ${2} ] && ls -l ${2}
1071	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1072	cat /tmp/out_command.$$
1073	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1074	else
1075	\rm /tmp/out_command.$$
1076	fi
1077	fi
1078	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1079	}
1080
1081	#D-#==================================================
1082	#D-function IGCM_sys_PrepareTaredRestart
1083	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
1084	#D-* Examples:
1085	#D-
1086	function IGCM_sys_PrepareTaredRestart {
1087	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1088
1089	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1090	}
1091
1092	#D-#==================================================
1093	#D-function IGCM_sys_Put_Out
1094	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1095	#D-* Examples:
1096	#D-
1097	function IGCM_sys_Put_Out {
1098	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1099	if ( $DEBUG_sys ) ; then
1100	echo "IGCM_sys_Put_Out :" $@
1101	fi
1102
1103	typeset status
1104
1105	if [ $DRYRUN = 0 ]; then
1106	if [ ! -f ${1} ] ; then
1107	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1108	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1109	return 1
1110	fi
1111	#
1112	IGCM_sys_MkdirArchive $( dirname $2 )
1113	#
1114	if [ X${JobType} = XRUN ] ; then
1115	if [ X${3} = X ] ; then
1116	IGCM_sys_Chmod 444 ${1}
1117	fi
1118	fi
1119	#
1120
1121	echo ${RSYNC} ${RSYNC_opt} $1 $2 > /tmp/out_command.$$ 2>&1
1122	${RSYNC} ${RSYNC_opt} $1 $2 >> /tmp/out_command.$$ 2>&1
1123	status=$?
1124	IGCM_sys_Rsync_out $status
1125
1126	${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1127	(( status=status+$? ))
1128
1129	if [ ${status} -gt 0 ] ; then
1130	IGCM_debug_Print 2 "IGCM_sys_Put_Out : rsync failed error code ${status}"
1131	cat /tmp/out_command.$$
1132	IGCM_debug_Exit "IGCM_sys_Put_Out"
1133	else
1134	\rm /tmp/out_command.$$
1135	fi
1136	fi
1137	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1138	return 0
1139	}
1140
1141	#D-#==================================================
1142	#D-function IGCM_sys_PutBuffer_Out
1143	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1144	#D-* Examples:
1145	#D-
1146	function IGCM_sys_PutBuffer_Out {
1147	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1148	if ( $DEBUG_sys ) ; then
1149	echo "IGCM_sys_PutBuffer_Out :" $@
1150	fi
1151
1152	typeset NB_ESSAI DELAI status i exist skip
1153
1154	# number of tentative
1155	NB_ESSAI=3
1156	# time delay between tentative
1157	DELAI=2
1158
1159	if [ $DRYRUN = 0 ]; then
1160	if [ ! -f ${1} ] ; then
1161	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1162	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1163	return 1
1164	fi
1165	#
1166	IGCM_sys_Mkdir $( dirname $2 )
1167	#
1168
1169	exist=false
1170	skip=false
1171	if [ -f $2 ] ; then
1172	IGCM_debug_Print 1 "$2 already exist"
1173	exist=true
1174	if [ "X$( diff $1 $2 )" = X ] ; then
1175	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1176	status=0
1177	skip=true
1178	else
1179	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1180	skip=false
1181	fi
1182	fi
1183	#
1184	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1185	IGCM_sys_Chmod u+w $2
1186	fi
1187
1188	if [ X${skip} = Xfalse ] ; then
1189	i=0
1190	while [ $i -lt $NB_ESSAI ] ; do
1191	# USUAL WAY
1192	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1193	status=$?
1194	if [ ${status} -gt 0 ]; then
1195	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1196	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1197	[ -f ${2} ] && ls -l ${2}
1198	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1199	sleep $DELAI
1200	else
1201	break
1202	fi
1203	(( i = i + 1 ))
1204	done
1205	fi
1206
1207	if [ ${status} -gt 0 ] ; then
1208	echo "IGCM_sys_PutBuffer_Out : error."
1209	[ -f ${2} ] && ls -l ${2}
1210	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1211	cat /tmp/out_command.$$
1212	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1213	else
1214
1215	if [ X${JobType} = XRUN ] ; then
1216	if [ X${3} = X ] ; then
1217	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1218	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1219	fi
1220	fi
1221
1222	\rm /tmp/out_command.$$
1223	fi
1224	fi
1225	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1226	return 0
1227	}
1228
1229	#D-#==================================================
1230	#D-function IGCM_sys_Get
1231	#D-* Purpose: Get a file from ${ARCHIVE}
1232	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1233	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1234	function IGCM_sys_Get {
1235	IGCM_debug_PushStack "IGCM_sys_Get" $@
1236
1237	typeset DEST status dm_liste
1238
1239	if ( $DEBUG_sys ) ; then
1240	echo "IGCM_sys_Get :" $@
1241	fi
1242	if [ $DRYRUN -le 2 ]; then
1243	if [ X${1} = X'/l' ] ; then
1244	eval set +A dm_liste \${${2}}
1245	else
1246	dm_liste=${1}
1247	fi
1248	eval DEST=\${${#}}
1249
1250	# test if the (first) file is present in the old computation :
1251	IGCM_sys_TestFileArchive ${dm_liste[0]}
1252	status=$?
1253	if [ ${status} -gt 0 ] ; then
1254	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
1255	IGCM_debug_Exit "IGCM_sys_Get"
1256	return
1257	fi
1258
1259	#USUAL WAY
1260	\cp ${dm_liste[*]} ${DEST} > /tmp/out_command.$$ 2>&1
1261	status=$?
1262
1263	if [ ${status} -gt 0 ] ; then
1264	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status}"
1265	cat /tmp/out_command.$$
1266	IGCM_debug_Exit "IGCM_sys_Get"
1267	else
1268	\rm /tmp/out_command.$$
1269	fi
1270	fi
1271	IGCM_debug_PopStack "IGCM_sys_Get"
1272	}
1273
1274	#D-#==================================================
1275	#D-function IGCM_sys_GetBuffer
1276	#D-* Purpose: Get a file from ${SCRATCHDIR}
1277	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1278	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1279	function IGCM_sys_GetBuffer {
1280	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1281
1282	typeset DEST buf_liste target file_work
1283	typeset NB_ESSAI DELAI status i
1284
1285	if ( $DEBUG_sys ) ; then
1286	echo "IGCM_sys_GetBuffer :" $@
1287	fi
1288
1289	# number of tentative
1290	NB_ESSAI=3
1291	# time delay between tentative
1292	DELAI=2
1293
1294	if [ $DRYRUN -le 2 ]; then
1295	if [ X${1} = X'/l' ] ; then
1296	eval set +A buf_liste \${${2}}
1297	else
1298	eval set +A buf_liste ${1}
1299	fi
1300	eval DEST=\${${#}}
1301
1302	#USUAL WAY
1303	if [ X${1} = X'/l' ] ; then
1304	for target in ${buf_liste[*]} ; do
1305	local_file=$( basename ${target} )
1306	i=0
1307	while [ $i -lt $NB_ESSAI ] ; do
1308	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1309	status=$?
1310	if [ ${status} -gt 0 ]; then
1311	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1312	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1313	sleep $DELAI
1314	else
1315	break
1316	fi
1317	(( i = i + 1 ))
1318	done
1319	if [ ${status} -gt 0 ] ; then
1320	echo "IGCM_sys_Get : error"
1321	cat /tmp/out_command.$$
1322	\rm /tmp/out_command.$$
1323	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1324	else
1325	\rm /tmp/out_command.$$
1326	fi
1327	done
1328	else
1329	i=0
1330	while [ $i -lt $NB_ESSAI ] ; do
1331	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1332	status=$?
1333	if [ ${status} -gt 0 ]; then
1334	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1335	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1336	sleep $DELAI
1337	else
1338	break
1339	fi
1340	(( i = i + 1 ))
1341	done
1342	if [ ${status} -gt 0 ] ; then
1343	echo "IGCM_sys_Get : error"
1344	cat /tmp/out_command.$$
1345	\rm /tmp/out_command.$$
1346	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1347	else
1348	\rm /tmp/out_command.$$
1349	fi
1350	fi
1351	fi
1352	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1353	}
1354
1355	#D-#==================================================
1356	#D-function IGCM_sys_GetDate_FichWork
1357	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1358	#D-* Examples:
1359	#D-
1360	function IGCM_sys_GetDate_FichWork {
1361	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1362	if ( $DEBUG_sys ) ; then
1363	echo "IGCM_sys_GetDate_FichWork :" $@
1364	fi
1365	typeset dateF
1366	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1367	eval ${2}=${dateF[5]}
1368
1369	# donne la date filesys d'un fichier sur la machine work
1370	IGCM_debug_PopStack "IGCM_sys_FichWork"
1371	}
1372
1373	#D-#==================================================
1374	#D-function IGCM_sys_GetDate_FichArchive
1375	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1376	#D-* Examples:
1377	#D-
1378	function IGCM_sys_GetDate_FichArchive {
1379	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1380	if ( $DEBUG_sys ) ; then
1381	echo "IGCM_sys_GetDate_FichArchive :" $@
1382	fi
1383	typeset dateF
1384	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1385	eval ${2}=${dateF[5]}
1386
1387	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1388	}
1389
1390	#D-#==================================================
1391	#D-function IGCM_sys_Dods_Rm
1392	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1393	#D-* Examples:
1394	#D-
1395	function IGCM_sys_Dods_Rm {
1396	if ( $DEBUG_sys ) ; then
1397	echo "IGCM_sys_Dods_Rm :" $@
1398	fi
1399	return 0
1400	}
1401
1402	#D-#==================================================
1403	#D-function IGCM_sys_Dods_Cp
1404	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1405	#D-* Examples:
1406	#D-
1407	function IGCM_sys_Dods_Cp {
1408	if ( $DEBUG_sys ) ; then
1409	echo "IGCM_sys_Dods_Cp :" $@
1410	fi
1411	return 0
1412	}
1413
1414	#D-#==================================================
1415	#D-function IGCM_sys_Put_Dods
1416	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole. Dummy function here
1417	#D-* Examples:
1418	#D-
1419	function IGCM_sys_Put_Dods {
1420	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1421	if ( $DEBUG_sys ) ; then
1422	echo "IGCM_sys_Put_Dods :" $@
1423	fi
1424	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1425	}
1426
1427	##############################################################
1428	# REBUILD OPERATOR
1429
1430	#D-#==================================================
1431	#D-function IGCM_sys_sync
1432	#D-* Purpose: flush buffer on disk (dummy function on Ada)
1433	#D-* Examples:
1434	#D-
1435	function IGCM_sys_sync {
1436	IGCM_debug_PushStack "IGCM_sys_sync" $@
1437	if ( $DEBUG_sys ) ; then
1438	echo "IGCM_sys_sync :" $@
1439	echo "Dummy call, let the system do that."
1440	fi
1441	IGCM_debug_PopStack "IGCM_sys_sync"
1442	}
1443
1444	#D-#==================================================
1445	#D-function IGCM_sys_rebuild
1446	#D-* Purpose: rebuild parallel files
1447	#D-* Examples:
1448	#D-
1449	function IGCM_sys_rebuild {
1450	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1451	if ( $DEBUG_sys ) ; then
1452	echo "IGCM_sys_rebuild :" $@
1453	fi
1454
1455	typeset NB_ESSAI DELAI status i firstArg
1456	# number of tentative
1457	NB_ESSAI=3
1458	# time delay between tentative
1459	DELAI=2
1460
1461	i=0
1462	while [ $i -lt $NB_ESSAI ] ; do
1463	rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1464	status=$?
1465	if [ ${status} -gt 0 ] ; then
1466	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1467	cat /tmp/out_command.$$
1468	\rm /tmp/out_command.$$
1469	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1470	firstArg=${1}
1471	\rm ${firstArg}
1472	sleep $DELAI
1473	else
1474	\rm /tmp/out_command.$$
1475	break
1476	fi
1477	(( i = i + 1 ))
1478	done
1479
1480	if [ ${status} -gt 0 ] ; then
1481	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1482	IGCM_debug_Exit "rebuild"
1483	fi
1484
1485	IGCM_debug_PopStack "IGCM_sys_rebuild"
1486	}
1487
1488	#D-#==================================================
1489	#D-function IGCM_sys_rebuild_station
1490	#D-* Purpose: rebuild parallel files describing station
1491	#D-* Examples:
1492	#D-
1493	function IGCM_sys_rebuild_station {
1494	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1495	typeset i list_opt file_in file_out prefix_invert list_invert
1496	if ( $DEBUG_sys ) ; then
1497	echo "IGCM_sys_rebuild_station :" $@
1498	fi
1499	list_opt=$@
1500
1501	# Invert Axis : t,x -> x,t
1502	# t,pres,x -> x,t,pres
1503	# So that we can concatenate along x
1504	i=0
1505	for file_in in ${list_opt} ; do
1506	(( i = i + 1))
1507	[ ${i} = 1 ] && file_out=${file_in} && continue
1508	# detect time counter and do the job only if present
1509	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
1510	if [ X${var_unlim} = Xtime_counter ] ; then
1511	prefix_invert=$( basename ${file_in} .nc )
1512	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1513	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1514	fi
1515	done
1516
1517	# Concatenate
1518	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1519
1520	# Re-ivert file
1521	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1522
1523	# Station re-ordering is too expansive to be run within libIGCM
1524	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
1525	# This re-ordering must be done "in memory" by the cmorization process
1526	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
1527	# BEGIN reordering
1528
1529	# Only LMDZ text output contains the exact ordering of the station.
1530	# We isolate this in the code below:
1531	# 0 38 -157.5000000000000 70.98591549295774
1532	# 0 54 27.49999999999999 67.18309859154928
1533	# 0 56 -62.50000000000001 82.39436619718309
1534	# 0 79 12.49999999999999 78.59154929577466
1535	# 0 116 -165.0000000000000 76.05633802816901
1536	# 0 117 130.0000000000000 70.98591549295774
1537	# 0 118 110.0000000000000 87.46478873239437
1538	# 1 40 4.999999999999995 51.97183098591550
1539	# typeset iStation iProc list_opt file_in file_out prefix_invert
1540	# typeset -Z4 j4
1541	# typeset -Z3 j3
1542
1543	# unset list_opt
1544	# set +A list_opt $@
1545
1546	# Filename after rebuild
1547	# file_out=${list_opt[0]}
1548	# Prefix of output files
1549	# prefix_invert=$( basename ${file_out} .nc )
1550	# Number of procs
1551	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
1552
1553	# iProc=0
1554	# while [ ${iProc} -lt ${num_proc} ] ; do
1555	# Array containing Station as a number
1556	# unset proc_stn
1557	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
1558	# Number of stations produced by processor proc
1559	# stationLast=${#proc_stn[*]}
1560	# Proc number on 4 digits
1561	# j4=${iProc}
1562	# Init
1563	# iStation=0
1564	# while [ ${iStation} -lt ${stationLast} ] ; do
1565	# Station number on 3 digits
1566	# j3=${proc_stn[${iStation}]}
1567	# Extract station
1568	# Invert Axis : t,x -> x,t
1569	# t,pres,x -> x,t,pres
1570	# So that we can concatenate along x
1571	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
1572	# (( iStation = iStation + 1 ))
1573	# done
1574	# (( iProc = iProc + 1 ))
1575	# done
1576
1577	# Concatenate all station along x
1578	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
1579
1580	# Re-invert file
1581	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
1582
1583	# END reordering
1584
1585	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
1586	}
1587
1588	############################################################
1589	# Activate Running Environnment Variables
1590
1591	#D-#==================================================
1592	#D-function IGCM_sys_desactiv_variables
1593	#D-* Purpose: set environement variables prior to execution
1594	#D-* Examples:
1595	#D-
1596	function IGCM_sys_activ_variables {
1597	IGCM_debug_PushStack "IGCM_sys_activ_variables"
1598	if ( $DEBUG_sys ) ; then
1599	echo "IGCM_sys_activ_variables"
1600	fi
1601
1602	# --------------------------------------------------------------------
1603	#D- MPI specifications
1604	# --------------------------------------------------------------------
1605
1606	# --------------------------------------------------------------------
1607	#D- Other specifications
1608	# --------------------------------------------------------------------
1609
1610	IGCM_debug_PopStack "IGCM_sys_activ_variables"
1611	}
1612
1613	############################################################
1614	# Desactivate Running Environnment Variables
1615
1616	#D-#==================================================
1617	#D-function IGCM_sys_desactiv_variables
1618	#D-* Purpose: unset environement variables after execution
1619	#D-* Examples:
1620	#D-
1621	function IGCM_sys_desactiv_variables {
1622	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
1623	if ( $DEBUG_sys ) ; then
1624	echo "IGCM_sys_desactiv_variables"
1625	fi
1626	# --------------------------------------------------------------------
1627	#D- MPI specifications
1628	# --------------------------------------------------------------------
1629
1630	# --------------------------------------------------------------------
1631	#D- Other specifications
1632	# --------------------------------------------------------------------
1633
1634	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
1635	}
1636
1637	############################################################
1638	# Build MPI/OMP scripts run file (dummy function)
1639
1640	#D-#==================================================
1641	#D-function IGCM_sys_build_run_file
1642	#D-* Purpose: build run file (deprecated)
1643	#D-* Examples:
1644	#D-
1645	function IGCM_sys_build_run_file {
1646
1647	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
1648
1649	}
1650
1651	############################################################
1652	# Build MPI/OMP scripts
1653
1654	#D-#==================================================
1655	#D-function IGCM_sys_build_execution_scripts
1656	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
1657	#D-* Examples:
1658	#D-
1659	function IGCM_sys_build_execution_scripts
1660	{
1661	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
1662	if ( $DEBUG_sys ) ; then
1663	echo "IGCM_sys_build_execution_scripts " $@
1664	fi
1665	typeset nodes listnodes init_node start_num init_exec comp ExeNameIn ExeNameOut
1666	typeset node_num_current node_current comp_proc_mpi_loc comp_proc_omp_loc
1667	typeset num_corempi nombre_restant_node nombre_restant_comp
1668
1669	if [ ! -f ${SUBMIT_DIR}/Job_${config_UserChoices_JobName} ] ; then
1670	IGCM_debug_Exit "IGCM_sys_default build_execution_scripts : Job_${config_UserChoices_JobName} don't exist in SUBMIT_DIR : ${SUBMIT_DIR} "
1671	fi
1672
1673	if ( ${OK_PARA_MPMD} ) ; then
1674
1675	if [ -f run_file ] ; then
1676	IGCM_sys_Rm -f run_file
1677	fi
1678	touch run_file
1679
1680	if ( ${OK_PARA_OMP} ) ; then
1681
1682	# Hosts treatment
1683
1684	${HOST_MPIRUN_COMMAND} hostname \| sort \| uniq > hosts.tmp
1685
1686	i=0
1687	rm -f hosts
1688	IGCM_debug_Print 1 "sys $( hostname ), Hosts avaible :"
1689	for nodes in `cat hosts.tmp` ; do
1690	host[$i]=$nodes
1691	echo "${host[$i]} slots=1 max_slots=1" >> hosts
1692	IGCM_debug_Print 1 ${host[$i]}
1693	i=$((i+1))
1694	done
1695	rm -f hosts.tmp
1696
1697	listnodes=${host[*]}
1698
1699	EXECUTION="${HOST_MPIRUN_COMMAND} -hostfile hosts"
1700
1701	# Initialisation
1702
1703	init_node=y
1704	node_num_current=0
1705	start_num=0
1706	init_exec=n
1707
1708	# Test : if oasis is there, we put it at the first position
1709
1710	for comp in ${config_ListOfComponents[*]} ; do
1711
1712	if [ "X${comp}" = "XCPL" ] ; then
1713
1714	eval ExeNameIn=\${config_Executable_${comp}[0]}
1715	eval ExeNameOut=\${config_Executable_${comp}[1]}
1716
1717	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1718	echo "" >> script_${ExeNameOut}.ksh
1719	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1720	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
1721	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
1722	echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
1723	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1724
1725	init_node=n
1726
1727	(( nombre_restant_node = NUM_COREPERNODE - 1 ))
1728	node_num_current=0
1729	node_current=${host[${node_num_current}]}
1730
1731	EXECUTION="${EXECUTION} -H ${node_current} -np 1 ./script_${ExeNameOut}.ksh"
1732
1733	init_exec=y
1734	start_num=1
1735
1736	fi
1737
1738	done
1739
1740	# Then loop on the components (except for oasis)
1741
1742	for comp in ${config_ListOfComponents[*]} ; do
1743
1744	eval ExeNameIn=\${config_Executable_${comp}[0]}
1745	eval ExeNameOut=\${config_Executable_${comp}[1]}
1746
1747	# Only if we really have an executable for the component :
1748	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
1749
1750	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1751	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1752
1753	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1754	# echo "set -vx" >> script_${ExeNameOut}.ksh
1755	echo "" >> script_${ExeNameOut}.ksh
1756	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1757	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
1758	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
1759	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1760	echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
1761	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
1762	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1763
1764	node_num=0
1765
1766	# We define the number of MPI process to be assigned for the component
1767
1768	nombre_restant_comp=${comp_proc_mpi_loc}
1769
1770	# Loop on the allocated nodes
1771
1772	for node in ${listnodes} ; do
1773
1774	# We go to the current node
1775	if [ ${node_num} = ${node_num_current} ] ; then
1776
1777	node_current=${host[${node_num_current}]}
1778
1779	# If first time on the node : initialisation
1780
1781	if [ ${init_node} = y ] ; then
1782	nombre_restant_node=${NUM_COREPERNODE}
1783	fi
1784
1785	# Test on the number of OMP threads
1786
1787	if [ ${comp_proc_omp_loc} -gt ${nombre_restant_node} ] ; then
1788	(( node_num = node_num + 1 ))
1789	node_num_current=${node_num}
1790	init_node=y
1791	continue
1792	fi
1793
1794	# Number of MPI process to assign
1795
1796	(( num_corempi = nombre_restant_node / comp_proc_omp_loc ))
1797
1798	if [ ${num_corempi} -gt ${nombre_restant_comp} ] ; then
1799	num_corempi=${nombre_restant_comp}
1800	fi
1801
1802	(( nombre_restant_node = nombre_restant_node - num_corempi * comp_proc_omp_loc ))
1803	(( nombre_restant_comp = nombre_restant_comp - num_corempi ))
1804
1805	if [ ${init_exec} = y ] ; then
1806	EXECUTION="${EXECUTION} : -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
1807	else
1808	EXECUTION="${EXECUTION} -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
1809	init_exec=y
1810	fi
1811
1812	(( start_num = num_corempi + start_num ))
1813
1814	else
1815
1816	(( node_num = node_num + 1 ))
1817	continue
1818	fi
1819
1820	# Test on the number of core/process remaining on the node/component
1821
1822	if [ ${nombre_restant_node} = 0 ] ; then
1823	(( node_num = node_num + 1 ))
1824	node_num_current=${node_num}
1825	init_node=y
1826
1827	if [ ${nombre_restant_comp} = 0 ] ; then
1828	break 1
1829	fi
1830	else
1831
1832	node_num_current=${node_num}
1833	init_node=n
1834
1835	if [ ${nombre_restant_comp} = 0 ] ; then
1836	break 1
1837	fi
1838	fi
1839	done
1840	fi
1841	done
1842
1843	else
1844
1845	# Then first loop on the components for the coupler ie oasis
1846
1847	## the coupler ie oasis must be the first one
1848	for comp in ${config_ListOfComponents[*]} ; do
1849
1850	eval ExeNameOut=\${config_Executable_${comp}[1]}
1851
1852	# for CPL component only
1853	if [ "X${comp}" = "XCPL" ] ; then
1854	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1855	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut} " >> run_file
1856	fi
1857	done
1858
1859	# Then second loop on the components
1860
1861	for comp in ${config_ListOfComponents[*]} ; do
1862
1863	eval ExeNameOut=\${config_Executable_${comp}[1]}
1864
1865	# Only if we really have an executable for the component and not the coupler ie oasis:
1866	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
1867	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1868	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
1869	fi
1870	done
1871	IGCM_sys_Chmod u+x run_file
1872
1873	EXECUTION="${HOST_MPIRUN_COMMAND} --app ./run_file"
1874
1875	fi
1876
1877	else # Only one executable. launch it.
1878
1879	for comp in ${config_ListOfComponents[*]} ; do
1880
1881	# Only if we really have an executable for the component :
1882	eval ExeNameOut=\${config_Executable_${comp}[1]}
1883	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
1884
1885	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1886	echo "" >> script_${ExeNameOut}.ksh
1887	if ( ${OK_PARA_OMP} ) ; then
1888	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1889	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1890	fi
1891	if ( ${OK_PARA_MPI} ) ; then
1892	# Default : mpirun used if nb_proc gt 1
1893	# pour sortie out/err par process
1894	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${OMPI_COMM_WORLD_RANK} 2>out_${ExeNameOut}.err.\${OMPI_COMM_WORLD_RANK}" >> script_${ExeNameOut}.ksh
1895	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
1896	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1897	EXECUTION="${HOST_MPIRUN_COMMAND} ./script_${ExeNameOut}.ksh"
1898	else
1899	# Default : mpirun is NOT used if nb_proc eq 1
1900	# pour sortie out/err par process
1901	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
1902	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
1903	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1904	EXECUTION="time ./script_${ExeNameOut}.ksh"
1905	fi
1906	fi
1907	done
1908
1909	fi
1910
1911	IGCM_debug_Print 1 "sys $( hostname ) : execution command is"
1912	IGCM_debug_Print 1 "$EXECUTION"
1913
1914	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
1915	}
1916
1917	#D-#==================================================
1918	#D-function IGCM_sys_check_path
1919	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
1920	#D-* do not point to an important use directory. Stop immediately in that case.
1921	#D-* Examples:
1922	#D-
1923	function IGCM_sys_check_path {
1924	IGCM_debug_PushStack "IGCM_sys_check_path"
1925	if ( $DEBUG_sys ) ; then
1926	echo "IGCM_sys_check_path"
1927	fi
1928
1929	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
1930	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
1931	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
1932	IGCM_debug_Exit "This will stop the job"
1933	fi
1934	IGCM_debug_PopStack "IGCM_sys_check_path"
1935	}
1936
1937	#D-#==================================================
1938	#D-function IGCM_sys_check_quota. Dummy call here
1939	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
1940	#D-* Examples:
1941	#D-
1942	function IGCM_sys_check_quota {
1943	IGCM_debug_PushStack "IGCM_sys_check_quota"
1944	if ( $DEBUG_sys ) ; then
1945	echo "IGCM_sys_check_quota"
1946	fi
1947	IGCM_debug_PopStack "IGCM_sys_check_quota"
1948	}
1949
1950	#D-#==================================================
1951	#D-function IGCM_sys_CountJobInQueue
1952	#D-* Purpose: Check if job_name is currently
1953	#D- running or in queue
1954	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
1955	#D-
1956	function IGCM_sys_CountJobInQueue {
1957	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
1958	if ( $DEBUG_sys ) ; then
1959	echo "IGCM_sys_CountJobInQueue"
1960	fi
1961	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
1962	}
1963
1964	##############################################################
1965	# NCO OPERATOR
1966
1967	#D-#==================================================
1968	#D-function IGCM_sys_ncap2
1969	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
1970	#D-* Examples:
1971	#D-
1972	function IGCM_sys_ncap2 {
1973	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
1974	if ( $DEBUG_sys ) ; then
1975	echo "IGCM_sys_ncap2 :" $@
1976	fi
1977
1978	typeset NB_ESSAI DELAI status i
1979	# number of tentative
1980	NB_ESSAI=3
1981	# time delay between tentative
1982	DELAI=2
1983
1984	i=0
1985	while [ $i -lt $NB_ESSAI ] ; do
1986	ncap2 -C "$@" > /tmp/out_command.$$ 2>&1
1987	status=$?
1988	if [ ${status} -gt 0 ] ; then
1989	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
1990	cat /tmp/out_command.$$
1991	\rm /tmp/out_command.$$
1992	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1993	sleep $DELAI
1994	else
1995	\rm /tmp/out_command.$$
1996	break
1997	fi
1998	(( i = i + 1 ))
1999	done
2000
2001	if [ ${status} -gt 0 ] ; then
2002	echo "IGCM_sys_ncap2 : ncap2 error"
2003	IGCM_debug_Exit "ncap2"
2004	fi
2005
2006	IGCM_debug_PopStack "IGCM_sys_ncap2"
2007	}
2008
2009	#D-#==================================================
2010	#D-function IGCM_sys_ncatted
2011	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2012	#D-* Examples:
2013	#D-
2014	function IGCM_sys_ncatted {
2015	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2016	if ( $DEBUG_sys ) ; then
2017	echo "IGCM_sys_ncatted :" $@
2018	fi
2019
2020	typeset NB_ESSAI DELAI status i
2021	# number of tentative
2022	NB_ESSAI=3
2023	# time delay between tentative
2024	DELAI=2
2025
2026	i=0
2027	while [ $i -lt $NB_ESSAI ] ; do
2028	ncatted "$@" > /tmp/out_command.$$ 2>&1
2029	status=$?
2030	if [ ${status} -gt 0 ] ; then
2031	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2032	cat /tmp/out_command.$$
2033	\rm /tmp/out_command.$$
2034	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2035	sleep $DELAI
2036	else
2037	\rm /tmp/out_command.$$
2038	break
2039	fi
2040	(( i = i + 1 ))
2041	done
2042
2043	if [ ${status} -gt 0 ] ; then
2044	echo "IGCM_sys_ncatted : ncatted error"
2045	IGCM_debug_Exit "ncatted"
2046	fi
2047
2048	IGCM_debug_PopStack "IGCM_sys_ncatted"
2049	}
2050
2051	#D-#==================================================
2052	#D-function IGCM_sys_ncbo
2053	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2054	#D-* Examples:
2055	#D-
2056	function IGCM_sys_ncbo {
2057	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2058	if ( $DEBUG_sys ) ; then
2059	echo "IGCM_sys_ncbo :" $@
2060	fi
2061
2062	typeset NB_ESSAI DELAI status i
2063	# number of tentative
2064	NB_ESSAI=3
2065	# time delay between tentative
2066	DELAI=2
2067
2068	i=0
2069	while [ $i -lt $NB_ESSAI ] ; do
2070	ncbo -C $@ > /tmp/out_command.$$ 2>&1
2071	status=$?
2072	if [ ${status} -gt 0 ] ; then
2073	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2074	cat /tmp/out_command.$$
2075	\rm /tmp/out_command.$$
2076	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2077	sleep $DELAI
2078	else
2079	\rm /tmp/out_command.$$
2080	break
2081	fi
2082	(( i = i + 1 ))
2083	done
2084
2085	if [ ${status} -gt 0 ] ; then
2086	echo "IGCM_sys_ncbo : ncbo error"
2087	IGCM_debug_Exit "ncbo"
2088	fi
2089
2090	IGCM_debug_PopStack "IGCM_sys_ncbo"
2091	}
2092
2093	#D-#==================================================
2094	#D-function IGCM_sys_ncdif
2095	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2096	#D-* Examples:
2097	#D-
2098	function IGCM_sys_ncdiff {
2099	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2100	if ( $DEBUG_sys ) ; then
2101	echo "IGCM_sys_ncdiff :" $@
2102	fi
2103
2104	typeset NB_ESSAI DELAI status i
2105	# number of tentative
2106	NB_ESSAI=3
2107	# time delay between tentative
2108	DELAI=2
2109
2110	i=0
2111	while [ $i -lt $NB_ESSAI ] ; do
2112	ncdiff -C $@ > /tmp/out_command.$$ 2>&1
2113	status=$?
2114	if [ ${status} -gt 0 ] ; then
2115	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2116	cat /tmp/out_command.$$
2117	\rm /tmp/out_command.$$
2118	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2119	sleep $DELAI
2120	else
2121	\rm /tmp/out_command.$$
2122	break
2123	fi
2124	(( i = i + 1 ))
2125	done
2126
2127	if [ ${status} -gt 0 ] ; then
2128	echo "IGCM_sys_ncdiff : ncdiff error"
2129	IGCM_debug_Exit "ncdiff"
2130	fi
2131
2132	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2133	}
2134
2135	#D-#==================================================
2136	#D-function IGCM_sys_ncea
2137	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2138	#D-* Examples:
2139	#D-
2140	function IGCM_sys_ncea {
2141	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2142	if ( $DEBUG_sys ) ; then
2143	echo "IGCM_sys_ncea :" $@
2144	fi
2145
2146	typeset NB_ESSAI DELAI status i
2147	# number of tentative
2148	NB_ESSAI=3
2149	# time delay between tentative
2150	DELAI=2
2151
2152	i=0
2153	while [ $i -lt $NB_ESSAI ] ; do
2154	ncea -C $@ > /tmp/out_command.$$ 2>&1
2155	status=$?
2156	if [ ${status} -gt 0 ] ; then
2157	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2158	cat /tmp/out_command.$$
2159	\rm /tmp/out_command.$$
2160	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2161	sleep $DELAI
2162	else
2163	\rm /tmp/out_command.$$
2164	break
2165	fi
2166	(( i = i + 1 ))
2167	done
2168
2169	if [ ${status} -gt 0 ] ; then
2170	echo "IGCM_sys_ncea : ncea error"
2171	IGCM_debug_Exit "ncea"
2172	fi
2173
2174	IGCM_debug_PopStack "IGCM_sys_ncea"
2175	}
2176
2177	#D-#==================================================
2178	#D-function IGCM_sys_ncecat
2179	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2180	#D-* Examples:
2181	#D-
2182	function IGCM_sys_ncecat {
2183	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2184	if ( $DEBUG_sys ) ; then
2185	echo "IGCM_sys_ncecat :" $@
2186	fi
2187
2188	typeset NB_ESSAI DELAI status i
2189	# number of tentative
2190	NB_ESSAI=3
2191	# time delay between tentative
2192	DELAI=2
2193
2194	i=0
2195	while [ $i -lt $NB_ESSAI ] ; do
2196	ncecat -C $@ > /tmp/out_command.$$ 2>&1
2197	status=$?
2198	if [ ${status} -gt 0 ] ; then
2199	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2200	cat /tmp/out_command.$$
2201	\rm /tmp/out_command.$$
2202	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2203	sleep $DELAI
2204	else
2205	\rm /tmp/out_command.$$
2206	break
2207	fi
2208	(( i = i + 1 ))
2209	done
2210
2211	if [ ${status} -gt 0 ] ; then
2212	echo "IGCM_sys_ncecat : ncecat error"
2213	IGCM_debug_Exit "ncecat"
2214	fi
2215
2216	IGCM_debug_PopStack "IGCM_sys_ncecat"
2217	}
2218
2219	#D-#==================================================
2220	#D-function IGCM_sys_ncflint
2221	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2222	#D-* Examples:
2223	#D-
2224	function IGCM_sys_ncflint {
2225	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2226	if ( $DEBUG_sys ) ; then
2227	echo "IGCM_sys_ncflint :" $@
2228	fi
2229
2230	typeset NB_ESSAI DELAI status i
2231	# number of tentative
2232	NB_ESSAI=3
2233	# time delay between tentative
2234	DELAI=2
2235
2236	i=0
2237	while [ $i -lt $NB_ESSAI ] ; do
2238	ncflint -C $@ > /tmp/out_command.$$ 2>&1
2239	status=$?
2240	if [ ${status} -gt 0 ] ; then
2241	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2242	cat /tmp/out_command.$$
2243	\rm /tmp/out_command.$$
2244	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2245	sleep $DELAI
2246	else
2247	\rm /tmp/out_command.$$
2248	break
2249	fi
2250	(( i = i + 1 ))
2251	done
2252
2253	if [ ${status} -gt 0 ] ; then
2254	echo "IGCM_sys_ncflint : ncflint error"
2255	IGCM_debug_Exit "ncflint"
2256	fi
2257
2258	IGCM_debug_PopStack "IGCM_sys_ncflint"
2259	}
2260
2261	#D-#==================================================
2262	#D-function IGCM_sys_ncks
2263	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2264	#D-* Examples:
2265	#D-
2266	function IGCM_sys_ncks {
2267	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2268	if ( $DEBUG_sys ) ; then
2269	echo "IGCM_sys_ncks :" $@
2270	fi
2271
2272	typeset NB_ESSAI DELAI status i
2273	# number of tentative
2274	NB_ESSAI=3
2275	# time delay between tentative
2276	DELAI=2
2277
2278	i=0
2279	while [ $i -lt $NB_ESSAI ] ; do
2280	ncks -C $@ > /tmp/out_command.$$ 2>&1
2281	status=$?
2282	if [ ${status} -gt 0 ] ; then
2283	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2284	cat /tmp/out_command.$$
2285	\rm /tmp/out_command.$$
2286	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2287	sleep $DELAI
2288	else
2289	\rm /tmp/out_command.$$
2290	break
2291	fi
2292	(( i = i + 1 ))
2293	done
2294
2295	if [ ${status} -gt 0 ] ; then
2296	echo "IGCM_sys_ncks : ncks error"
2297	IGCM_debug_Exit "ncks"
2298	fi
2299
2300	IGCM_debug_PopStack "IGCM_sys_ncks"
2301	}
2302
2303	#D-#==================================================
2304	#D-function IGCM_sys_ncpdq
2305	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2306	#D-* Examples:
2307	#D-
2308	function IGCM_sys_ncpdq {
2309	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2310	if ( $DEBUG_sys ) ; then
2311	echo "IGCM_sys_ncpdq :" $@
2312	fi
2313
2314	typeset NB_ESSAI DELAI status i
2315	# number of tentative
2316	NB_ESSAI=3
2317	# time delay between tentative
2318	DELAI=2
2319
2320	i=0
2321	while [ $i -lt $NB_ESSAI ] ; do
2322	ncpdq -C $@ > /tmp/out_command.$$ 2>&1
2323	status=$?
2324	if [ ${status} -gt 0 ] ; then
2325	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2326	cat /tmp/out_command.$$
2327	\rm /tmp/out_command.$$
2328	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2329	sleep $DELAI
2330	else
2331	\rm /tmp/out_command.$$
2332	break
2333	fi
2334	(( i = i + 1 ))
2335	done
2336
2337	if [ ${status} -gt 0 ] ; then
2338	echo "IGCM_sys_ncpdq : ncpdq error"
2339	IGCM_debug_Exit "ncpdq"
2340	fi
2341
2342	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2343	}
2344
2345	#D-#==================================================
2346	#D-function IGCM_sys_ncra
2347	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2348	#D-* Examples:
2349	#D-
2350	function IGCM_sys_ncra {
2351	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2352	if ( $DEBUG_sys ) ; then
2353	echo "IGCM_sys_ncra :" $@
2354	fi
2355
2356	typeset NB_ESSAI DELAI status i
2357	# number of tentative
2358	NB_ESSAI=3
2359	# time delay between tentative
2360	DELAI=2
2361
2362	i=0
2363	while [ $i -lt $NB_ESSAI ] ; do
2364	ncra -C $@ > /tmp/out_command.$$ 2>&1
2365	status=$?
2366	if [ ${status} -gt 0 ] ; then
2367	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
2368	cat /tmp/out_command.$$
2369	\rm /tmp/out_command.$$
2370	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2371	sleep $DELAI
2372	else
2373	\rm /tmp/out_command.$$
2374	break
2375	fi
2376	(( i = i + 1 ))
2377	done
2378
2379	if [ ${status} -gt 0 ] ; then
2380	echo "IGCM_sys_ncra : ncra error"
2381	IGCM_debug_Exit "ncra"
2382	fi
2383
2384	IGCM_debug_PopStack "IGCM_sys_ncra"
2385	}
2386
2387	#D-#==================================================
2388	#D-function IGCM_sys_ncrcat
2389	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
2390	#D-* Examples:
2391	#D-
2392	function IGCM_sys_ncrcat {
2393	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
2394	if ( $DEBUG_sys ) ; then
2395	echo "IGCM_sys_ncrcat :" $@
2396	fi
2397
2398	typeset NB_ESSAI DELAI status i lastArg
2399	# number of tentative
2400	NB_ESSAI=3
2401	# time delay between tentative
2402	DELAI=2
2403
2404	i=0
2405	while [ $i -lt $NB_ESSAI ] ; do
2406	ncrcat -C $@ > /tmp/out_command.$$ 2>&1
2407	status=$?
2408	if [ ${status} -gt 0 ] ; then
2409	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
2410	cat /tmp/out_command.$$
2411	\rm /tmp/out_command.$$
2412	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2413	sleep $DELAI
2414	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
2415	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
2416	cat /tmp/out_command.$$
2417	# remove files having corrupted time axis
2418	eval lastArg=\${$#}
2419	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
2420	\rm ${lastArg}
2421	\rm /tmp/out_command.$$
2422	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2423	sleep $DELAI
2424	else
2425	\rm /tmp/out_command.$$
2426	break
2427	fi
2428	(( i = i + 1 ))
2429	done
2430
2431	if [ ${status} -gt 0 ] ; then
2432	echo "IGCM_sys_ncrcat : ncrcat error"
2433	#IGCM_debug_Exit "ncrcat"
2434	fi
2435
2436	IGCM_debug_PopStack "IGCM_sys_ncrcat"
2437	}
2438
2439	#D-#==================================================
2440	#D-function IGCM_sys_ncrename
2441	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
2442	#D-* Examples:
2443	#D-
2444	function IGCM_sys_ncrename {
2445	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
2446	if ( $DEBUG_sys ) ; then
2447	echo "IGCM_sys_ncrename :" $@
2448	fi
2449
2450	typeset NB_ESSAI DELAI status i
2451	# number of tentative
2452	NB_ESSAI=3
2453	# time delay between tentative
2454	DELAI=2
2455
2456	i=0
2457	while [ $i -lt $NB_ESSAI ] ; do
2458	ncrename $@ > /tmp/out_command.$$ 2>&1
2459	status=$?
2460	if [ ${status} -gt 0 ] ; then
2461	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
2462	cat /tmp/out_command.$$
2463	\rm /tmp/out_command.$$
2464	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2465	sleep $DELAI
2466	else
2467	\rm /tmp/out_command.$$
2468	break
2469	fi
2470	(( i = i + 1 ))
2471	done
2472
2473	if [ ${status} -gt 0 ] ; then
2474	echo "IGCM_sys_ncrename : ncrename error"
2475	IGCM_debug_Exit "ncrename"
2476	fi
2477
2478	IGCM_debug_PopStack "IGCM_sys_ncrename"
2479	}
2480
2481	#D-#==================================================
2482	#D-function IGCM_sys_ncwa
2483	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
2484	#D-* Examples:
2485	#D-
2486	function IGCM_sys_ncwa {
2487	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
2488	if ( $DEBUG_sys ) ; then
2489	echo "IGCM_sys_ncwa :" $@
2490	fi
2491
2492	typeset NB_ESSAI DELAI status i
2493	# number of tentative
2494	NB_ESSAI=3
2495	# time delay between tentative
2496	DELAI=2
2497
2498	i=0
2499	while [ $i -lt $NB_ESSAI ] ; do
2500	ncwa -C $@ > /tmp/out_command.$$ 2>&1
2501	status=$?
2502	if [ ${status} -gt 0 ] ; then
2503	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
2504	cat /tmp/out_command.$$
2505	\rm /tmp/out_command.$$
2506	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2507	sleep $DELAI
2508	else
2509	\rm /tmp/out_command.$$
2510	break
2511	fi
2512	(( i = i + 1 ))
2513	done
2514
2515	if [ ${status} -gt 0 ] ; then
2516	echo "IGCM_sys_ncwa : ncwa error"
2517	IGCM_debug_Exit "ncwa"
2518	fi
2519
2520	IGCM_debug_PopStack "IGCM_sys_ncwa"
2521	}
2522
2523	##############################################################
2524	# CDO OPERATOR
2525
2526	#D-#==================================================
2527	#D-function IGCM_sys_cdo
2528	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
2529	#D-* Examples:
2530	#D-
2531	function IGCM_sys_cdo {
2532	IGCM_debug_PushStack "IGCM_sys_cdo" $@
2533	if ( $DEBUG_sys ) ; then
2534	echo "IGCM_sys_cdo :" $@
2535	fi
2536
2537	typeset status
2538
2539	\cdo $@ > /tmp/out_command.$$ 2>&1
2540	status=$?
2541	if [ ${status} -gt 0 ] ; then
2542	echo "IGCM_sys_cdo : error code ${status}"
2543	cat /tmp/out_command.$$
2544	\rm /tmp/out_command.$$
2545	IGCM_debug_PopStack "IGCM_sys_cdo"
2546	return 1
2547	else
2548	IGCM_debug_PopStack "IGCM_sys_cdo"
2549	return 0
2550	fi
2551
2552	IGCM_debug_PopStack "IGCM_sys_cdo"
2553	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: