Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_ada.ksh @ 1092

Last change on this file since 1092 was 1092, checked in by aclsce, 10 years ago
Added functionality to run coupled configuration with hybrid parallelisation (MPI + OpenMP) on ada supercomputer
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 80.6 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sébastien Denvil
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Ada
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=ada
71
72	#D-
73	#D-#==================================================
74	#D-Program used in libIGCM
75	#D-#==================================================
76
77	# Submit command
78	typeset SUBMIT=${SUBMIT:=llsubmit}
79	# rsync with path
80	typeset -r RSYNC=/usr/bin/rsync
81	# RSYNC_opt args to rsync
82	typeset -r RSYNC_opt="-va"
83	# ie storage filesystem
84	###typeset -r STOREHOST=gaya
85	typeset -r STOREHOST=ergon
86	typeset -r REMOTE_RSYNC=/u/rech/ces/rces452/RSYNC/bin/rsync
87
88	#====================================================
89	# Source default environment
90	#====================================================
91	##. /etc/profile
92
93	#====================================================
94	# Set environment tools (ferret, nco, cdo)
95	#====================================================
96	source /smplocal/pub/Modules/default/init/ksh
97	. /smphome/rech/psl/rpsl035/.atlas_env_ada_bash > /dev/null 2>&1
98	export PATH=${PATH}:/smphome/rech/psl/rpsl035/AddNoise/src_X64_ADA/bin
99	export PATH=${PATH}:/smphome/rech/psl/rpsl035/AddPerturbation/src_X64_ADA/bin
100
101	#====================================================
102	# Host specific DIRECTORIES
103	#====================================================
104
105	#====================================================
106	#- MirrorlibIGCM for frontend
107	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
108
109	#====================================================
110	#- libIGCM_POST for frontend
111	typeset -r libIGCM_POST=${libIGCM}
112
113	#====================================================
114	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
115	typeset -r R_EXE="${MODIPSL}/bin"
116
117	#====================================================
118	#- SUBMIT_DIR : submission dir
119	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${LOADL_STEP_INITDIR}}
120
121	#====================================================
122	#- IN
123	typeset -r R_IN=${R_IN:=/workgpfs/rech/psl/rpsl035/IGCM}
124	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/u/rech/psl/rpsl376}
125
126	#====================================================
127	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
128	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${TMPDIR}}
129
130	#====================================================
131	#- HOST_MPIRUN_COMMAND
132	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time poe"}
133
134	#====================================================
135	#- Max number of arguments passed to nco operator or demigration command
136	UNIX_MAX_LIMIT=360
137
138	#====================================================
139	#- set PackDefault true on ada
140	PackDefault=true
141
142	#====================================================
143	#- Number of core per node (max number of OpenMP task)
144	NUM_COREPERNODE=8
145
146	#====================================================
147	#- Default number of MPI task for IPSL coupled model
148	#- required for backward compatibility
149	#-
150	DEFAULT_NUM_PROC_OCE=5
151	DEFAULT_NUM_PROC_CPL=1
152	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
153	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
154
155	#D-#==================================================
156	#D-function IGCM_sys_defineArchives
157	#D-* Purpose:
158	#D-* Define ARCHIVE : Dedicated to large files
159	#D-* Define STORAGE : Dedicated to small/medium files
160	#D-* Define R_OUT : Output tree located on ARCHIVE
161	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
162	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
163	#D-* if SpaceName=TEST everything is stored on WORKDIR
164	#D-* Examples:
165	#D-
166	function IGCM_sys_defineArchives {
167	IGCM_debug_PushStack "IGCM_sys_defineArchives"
168
169	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
170	#====================================================
171	#- ARCHIVE (dedicated to large files)
172	ARCHIVE=${config_UserChoices_ARCHIVE}
173	else
174	#====================================================
175	#- ARCHIVE (dedicated to large files)
176	ARCHIVE=$(echo ${HOME} \| sed -e "s,/.*home/,/arch/home/,")
177	fi
178
179	if [ ! X${config_UserChoices_STORAGE} = X ]; then
180	#====================================================
181	#- STORAGE (dedicated to small/medium files)
182	STORAGE=${config_UserChoices_STORAGE}
183	else
184	#====================================================
185	#- STORAGE (dedicated to small/medium files)
186	STORAGE=${WORKDIR}
187	fi
188
189	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
190	#====================================================
191	#- R_OUT
192	R_OUT=${WORKDIR}/IGCM_OUT
193
194	#====================================================
195	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
196	R_FIG=${WORKDIR}/IGCM_OUT
197
198	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
199
200	else
201	#====================================================
202	#- R_OUT
203	R_OUT=${ARCHIVE}/IGCM_OUT
204
205	#====================================================
206	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
207	R_FIG=${ARCHIVE}/IGCM_OUT
208	fi
209
210	#====================================================
211	#- R_BUF (ONLY FOR double copy an scratch)
212	R_BUF=${WORKDIR}/IGCM_OUT
213
214	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
215	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
216	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
217
218	IGCM_debug_PopStack "IGCM_sys_defineArchives"
219	}
220
221	#D-#==================================================
222	#D-function IGCM_sys_RshMaster
223	#D-* Purpose: Connection to frontend machine.
224	#D-* Examples:
225	#D-
226	function IGCM_sys_RshMaster {
227	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
228	/bin/ksh <<-EOF
229	export libIGCM=${libIGCM}
230	export DEBUG_debug=${DEBUG_debug}
231	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
232	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
233	${@}
234	EOF
235	if [ $? -gt 0 ] ; then
236	echo "IGCM_sys_RshMaster : erreur."
237	IGCM_debug_Exit "IGCM_sys_RshMaster"
238	fi
239	IGCM_debug_PopStack "IGCM_sys_RshMaster"
240	}
241
242	#D-#==================================================
243	#D-function IGCM_sys_RshArchive
244	#D-* Purpose: Archive rsh command
245	#D-* Examples:
246	#D-
247	function IGCM_sys_RshArchive {
248	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
249	typeset ergonCount gayaCount
250	DEBUG_sys=false IGCM_sys_IsFileArchived "$@"
251	if [ $? = 0 ] ; then
252	status=0
253	#special case for the find command. Only used together with RshArchive to find tared restart
254	if [ "X$( echo $@ \| grep ^find \| grep RESTART )" != "X" ] ; then
255	# Ergon first
256	rsh ${STOREHOST} exec /bin/ksh <<-EOF > ergonList.txt
257	${@}
258	EOF
259	(( status=status + $? ))
260	# Gaya turn
261	rsh gaya exec /bin/ksh <<-EOF > gayaList.txt
262	${@}
263	EOF
264	(( status=status + $? ))
265
266	# the more the better
267	ergonCount=$( cat ergonList.txt \| wc -l )
268	gayaCount=$( cat gayaList.txt \| wc -l )
269
270	[ ${ergonCount} -ge ${gayaCount} ] && cat ergonList.txt \|\| cat gayaList.txt
271	rm -f ergonList.txt gayaList.txt
272	else
273	rsh ${STOREHOST} exec /bin/ksh <<-EOF
274	${@}
275	EOF
276	status=$?
277	fi
278	else
279	/bin/ksh <<-EOF
280	${@}
281	EOF
282	status=$?
283	fi
284	if [ ${status} -gt 0 ] ; then
285	IGCM_debug_Print 2 "IGCM_sys_RshArchive : rsh or command failed error code ${status}"
286	IGCM_debug_Exit "IGCM_sys_RshArchive"
287	fi
288	IGCM_debug_PopStack "IGCM_sys_RshArchive"
289	}
290
291	#D-#==================================================
292	#D-function IGCM_sys_RshArchive_NoError
293	#D-* Purpose: Archive rsh command, without error
294	#D-* used only in monitoring.job
295	#D-* Examples:
296	#D-
297	function IGCM_sys_RshArchive_NoError {
298	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
299	DEBUG_sys=false IGCM_sys_IsFileArchived "$@"
300	if [ $? = 0 ] ; then
301	rsh ${STOREHOST} exec /bin/ksh <<-EOF
302	${@} 2> /dev/null
303	EOF
304	else
305	/bin/ksh <<-EOF
306	${@} 2> /dev/null
307	EOF
308	fi
309	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
310	}
311
312	#D-#==================================================
313	#D-function IGCM_sys_RshPost
314	#D-* Purpose: Post-process rsh command
315	#D-* Examples:
316	#D-
317	function IGCM_sys_RshPost {
318	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
319	if ( $DEBUG_sys ) ; then
320	echo "IGCM_sys_RshPost :" $@
321	fi
322	# keep standard input (stdin) for the loop onto temporary file
323	cat >/tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
324
325	/bin/ksh </tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
326	if [ $? -gt 0 ] ; then
327	echo "IGCM_sys_RshPost : erreur."
328	IGCM_debug_Exit "IGCM_sys_RshPost"
329	fi
330	# delete temporary file
331	\rm /tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
332
333	IGCM_debug_PopStack "IGCM_sys_RshPost"
334	}
335
336	#D-#==================================================
337	#D-function IGCM_sys_SendMail
338	#D-* Purpose: Send mail when simulation is over
339	#D-* Examples:
340	#D-
341	function IGCM_sys_SendMail {
342	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
343	if ( $DEBUG_sys ) ; then
344	echo "IGCM_sys_SendMail :" $@
345	fi
346
347	if [ X${1} = XAccounting ] ; then
348	status=Accounting
349	mailText=jobAccounting.mail
350	elif ( ${ExitFlag} ) ; then
351	status=failed
352	mailText=jobEnd.mail
353	else
354	status=completed
355	mailText=jobEnd.mail
356	fi
357
358	# Update selected mail template
359	while read -r line; do
360	eval echo $line >> mail.txt ;
361	done < ${libIGCM}/libIGCM_sys/${mailText}
362
363	if [ ! -z ${config_UserChoices_MailName} ] ; then
364	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
365	elif [ -f ~/.forward ] ; then
366	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
367	else
368	mail -s "${config_UserChoices_JobName} ${status}" ${USER} < mail.txt
369	fi
370
371	sleep 10
372	rm -f mail.txt
373
374	IGCM_debug_PopStack "IGCM_sys_SendMail"
375	}
376
377	#D-#==================================================
378	#D-function IGCM_sys_Mkdir
379	#D-* Purpose: Master locale mkdir command
380	#D-* Examples:
381	#D-
382	function IGCM_sys_Mkdir {
383	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
384	if ( $DEBUG_sys ) ; then
385	echo "IGCM_sys_Mkdir :" $@
386	fi
387	if [ ! -d ${1} ]; then
388	\mkdir -p $1
389	if [ $? -gt 0 ] ; then
390	echo "IGCM_sys_Mkdir : erreur."
391	IGCM_debug_Exit "IGCM_sys_Mkdir"
392	fi
393	fi
394	# vérification :
395	if [ ! -d ${1} ] ; then
396	echo "IGCM_sys_Mkdir : erreur."
397	IGCM_debug_Exit "IGCM_sys_Mkdir"
398	fi
399	IGCM_debug_PopStack "IGCM_sys_Mkdir"
400	}
401
402	#D-#==================================================
403	#D-function IGCM_sys_MkdirArchive
404	#D-* Purpose: Mkdir on Archive
405	#D-* Examples:
406	#D-
407	function IGCM_sys_MkdirArchive {
408	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
409	if ( $DEBUG_sys ) ; then
410	echo "IGCM_sys_MkdirArchive :" $@
411	fi
412	#- creation de repertoire sur le serveur fichier
413	DEBUG_sys=false IGCM_sys_IsFileArchived $1
414	if [ $? = 0 ] ; then
415	rsh ${STOREHOST} -n mkdir -p $1
416	status=$?
417	else
418	mkdir -p $1
419	status=$?
420	fi
421
422	if [ ${status} -gt 0 ] ; then
423	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : rsh or mkdir failed error code ${status}"
424	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
425	fi
426	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
427	}
428
429	#D-#==================================================
430	#D-function IGCM_sys_MkdirWork
431	#D-* Purpose: Mkdir on Work
432	#D-* Examples:
433	#D-
434	function IGCM_sys_MkdirWork {
435	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
436	if ( $DEBUG_sys ) ; then
437	echo "IGCM_sys_MkdirWork :" $@
438	fi
439	#- creation de repertoire sur le serveur fichier
440	if [ ! -d ${1} ]; then
441	\mkdir -p $1
442	if [ $? -gt 0 ] ; then
443	echo "IGCM_sys_MkdirWork : erreur."
444	IGCM_debug_Exit "IGCM_sys_MkdirWork"
445	fi
446	fi
447	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
448	}
449
450	#D-#==================================================
451	#D-function IGCM_sys_Cd
452	#D-* Purpose: master cd command
453	#D-* Examples:
454	#D-
455	function IGCM_sys_Cd {
456	IGCM_debug_PushStack "IGCM_sys_Cd" $@
457	if ( $DEBUG_sys ) ; then
458	echo "IGCM_sys_Cd :" $@
459	fi
460	\cd $1
461	if [ $? -gt 0 ] ; then
462	echo "IGCM_sys_Cd : erreur."
463	IGCM_debug_Exit "IGCM_sys_Cd"
464	fi
465	IGCM_debug_PopStack "IGCM_sys_Cd"
466	}
467
468	#D-#==================================================
469	#D-function IGCM_sys_Chmod
470	#D-* Purpose: Chmod
471	#D-* Examples:
472	#D-
473	function IGCM_sys_Chmod {
474	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
475	if ( $DEBUG_sys ) ; then
476	echo "IGCM_sys_Chmod :" $@
477	fi
478	\chmod $@
479	if [ $? -gt 0 ] ; then
480	echo "IGCM_sys_Chmod : erreur."
481	IGCM_debug_Exit "IGCM_sys_Chmod"
482	fi
483	IGCM_debug_PopStack "IGCM_sys_Chmod"
484	}
485
486	#D-#==================================================
487	#D-function IGCM_sys_FileSize
488	#D-* Purpose: Filesize
489	#D-* Examples:
490	#D-
491	function IGCM_sys_FileSize {
492	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
493
494	typeset sizeF
495	set +A sizeF -- $( ls -la ${1} )
496	if [ $? -gt 0 ] ; then
497	IGCM_debug_Exit "IGCM_sys_FileSize"
498	fi
499	eval ${2}=${sizeF[4]}
500
501	IGCM_debug_PopStack "IGCM_sys_FileSize"
502	}
503
504	#D-#==================================================
505	#D-function IGCM_sys_TestDir
506	#D-* Purpose: Test Directory that must exists
507	#D-* Examples:
508	#D-
509	function IGCM_sys_TestDir {
510	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
511	if ( $DEBUG_sys ) ; then
512	echo "IGCM_sys_TestDir :" $@
513	fi
514	typeset ExistFlag
515	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
516	IGCM_debug_PopStack "IGCM_sys_TestDir"
517
518	return ${ExistFlag}
519	}
520
521	#D-#==================================================
522	#D-function IGCM_sys_TestDirArchive
523	#D-* Purpose: Test Directory that must exists on Archive
524	#D-* Examples:
525	#D-
526	function IGCM_sys_TestDirArchive {
527	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
528	if ( $DEBUG_sys ) ; then
529	echo "IGCM_sys_TestDirArchive :" $@
530	fi
531	typeset ExistFlag
532	#Command depends on targeted file system
533	DEBUG_sys=false IGCM_sys_IsFileArchived $1
534	if [ $? = 0 ] ; then
535	ExistFlag=$( IGCM_sys_RshArchive "[ -d $1 ] && echo 0 \|\| echo 1" )
536	else
537	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
538	fi
539	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
540	return ${ExistFlag}
541	}
542
543	#D-#==================================================
544	#D-function IGCM_sys_IsFileArchived
545	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
546	#D-* Examples:
547	#D-
548	function IGCM_sys_IsFileArchived {
549	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
550	if ( $DEBUG_sys ) ; then
551	echo "IGCM_sys_IsFileArchived :" $@
552	fi
553	typeset IsArchivedFlag
554	IsArchivedFlag=$( ( [ "X$( echo $@ \| grep \/u\/ )" != "X" ] \|\| [ "X$( echo $@ \| grep \/\arch\/home )" != "X" ] ) && echo 0 \|\| echo 1 )
555	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
556
557	return ${IsArchivedFlag}
558	}
559
560	#D-#==================================================
561	#D-function IGCM_sys_TestFileArchive
562	#D-* Purpose: Test file that must NOT EXISTS on Archive
563	#D-* Examples:
564	#D-
565	#function IGCM_sys_TestFileArchive {
566	# IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
567	# typeset ExistFlag TempCommandFile
568	# TempCommandFile="[ -f $1 ] && echo 0 \|\| echo 1"
569	# #Command depends on targeted file system
570	# #We want it quiet
571	# DEBUG_sys=false IGCM_sys_IsFileArchived $1
572	# if [ $? = 0 ] ; then
573	# ExistFlag=$( IGCM_sys_RshArchive ${TempCommandFile} )
574	# else
575	# ExistFlag=$( ${TempCommandFile} )
576	# fi
577	# IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
578	#
579	# return ${ExistFlag}
580	#}
581
582	#D-#==================================================
583	#D-function IGCM_sys_TestFileArchive
584	#D-* Purpose: Test file that must NOT EXISTS on Archive
585	#D-* Examples:
586	#D-
587	function IGCM_sys_TestFileArchive {
588	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
589	typeset ExistFlag finalPosition
590
591	DEBUG_sys=false IGCM_sys_IsFileArchived $1
592	if [ $? = 0 ] ; then
593	finalPosition=$( mfls $1 2>/dev/null \| wc -l )
594	[ ${finalPosition} -eq 4 ] && ExistFlag=1 \|\| ExistFlag=0
595	else
596	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
597	fi
598
599	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
600
601	return ${ExistFlag}
602	}
603
604	#D-#==================================================
605	#D-function IGCM_sys_TestFileBuffer
606	#D-* Purpose: Test file that must NOT EXISTS on Buffer
607	#D-* Examples:
608	#D-
609	function IGCM_sys_TestFileBuffer {
610	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
611	typeset ExistFlag
612	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
613	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
614
615	return ${ExistFlag}
616	}
617
618	##D-#==================================================
619	##D-function IGCM_sys_CountFileArchive
620	##D-* Purpose: Count files on Archive filesystem
621	##D-* Examples:
622	##D-
623	#function IGCM_sys_CountFileArchive {
624	# IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
625	# #Command depends on targeted file system
626	# DEBUG_sys=false IGCM_sys_IsFileArchived $1
627	# if [ $? = 0 ] ; then
628	# IGCM_sys_RshArchive "ls ${@} 2>/dev/null \| wc -l"
629	# status=$?
630	# else
631	# ls ${@} 2>/dev/null \| wc -l
632	# status=$?
633	# fi
634	# if [ ${status} -gt 0 ] ; then
635	# echo "IGCM_sys_CountFileArchive : erreur."
636	# fi
637	# IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
638	#}
639
640	#D-#==================================================
641	#D-function IGCM_sys_CountFileArchive
642	#D-* Purpose: Count files on Archive filesystem
643	#D-* Examples:
644	#D-
645	function IGCM_sys_CountFileArchive {
646	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
647	typeset status gayaPosition ergonPosition finalPosition gayaCount ergonCount
648	#Command depends on targeted file system
649	DEBUG_sys=false IGCM_sys_IsFileArchived $1
650	if [ $? = 0 ] ; then
651	status=0
652	gayaPosition=$( mfls ${@} 2>/dev/null \| grep -n "++ Gaya" \| awk -F\: {'print $1'} )
653	(( status=status + $? ))
654	ergonPosition=$( mfls ${@} 2>/dev/null \| grep -n "++ Ergon" \| awk -F\: {'print $1'} )
655	(( status=status + $? ))
656	finalPosition=$( mfls ${@} 2>/dev/null \| wc -l )
657	(( status=status + $? ))
658
659	(( gayaCount = ergonPosition - gayaPosition - 2 ))
660	(( ergonCount = finalPosition - ergonPosition ))
661
662	# return number of files found. greater the better
663	[ ${ergonCount} -ge ${gayaCount} ] && echo ${ergonCount} \|\| echo ${gayaCount}
664
665	# "Ergon only" case
666	#IGCM_sys_RshArchive "ls ${@} 2>/dev/null \| wc -l"
667	#status=$?
668	else
669	ls ${@} 2>/dev/null \| wc -l
670	status=$?
671	fi
672	if [ ${status} -gt 0 ] ; then
673	echo "IGCM_sys_CountFileArchive : erreur."
674	fi
675	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
676	}
677
678	#D-#==================================================
679	#D-function IGCM_sys_CountFileBuffer
680	#D-* Purpose: Count files on Scratch filesystem
681	#D-* Examples:
682	#D-
683	function IGCM_sys_CountFileBuffer {
684	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
685	ls ${@} 2>/dev/null \| wc -l
686	if [ $? -gt 0 ] ; then
687	echo "IGCM_sys_CountFileBuffer : erreur."
688	fi
689	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
690	}
691
692	#D-#==================================================
693	#D-function IGCM_sys_Tree
694	#D-* Purpose: Tree directories with files on ${ARCHIVE}
695	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
696	#D-
697	function IGCM_sys_Tree {
698	IGCM_debug_PushStack "IGCM_sys_Tree" $@
699	if ( $DEBUG_sys ) ; then
700	echo "IGCM_sys_Tree :" $@
701	fi
702
703	\mfls -R $@
704
705	IGCM_debug_PopStack "IGCM_sys_Tree"
706	}
707
708	#D-#==================================================
709	#D-function IGCM_sys_Tar
710	#D-* Purpose: master tar command
711	#D-* Examples:
712	#D-
713	function IGCM_sys_Tar {
714	IGCM_debug_PushStack "IGCM_sys_Tar" $@
715	if ( $DEBUG_sys ) ; then
716	echo "IGCM_sys_Tar :" $@
717	fi
718	\tar cf $@
719	if [ $? -gt 0 ] ; then
720	echo "IGCM_sys_Tar : erreur."
721	IGCM_debug_Exit "IGCM_sys_Tar"
722	fi
723	IGCM_debug_PopStack "IGCM_sys_Tar"
724	}
725
726	#D-#==================================================
727	#D-function IGCM_sys_UnTar
728	#D-* Purpose: master un-tar command
729	#D-* Examples:
730	#D-
731	function IGCM_sys_UnTar {
732	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
733	if ( $DEBUG_sys ) ; then
734	echo "IGCM_sys_UnTar :" $@
735	fi
736	\tar xvf $1
737	if [ $? -gt 0 ] ; then
738	echo "IGCM_sys_UnTar : erreur."
739	IGCM_debug_Exit "IGCM_sys_UnTar"
740	fi
741	IGCM_debug_PopStack "IGCM_sys_UnTar"
742	}
743
744	#D-#==================================================
745	#D-function IGCM_sys_Qsub
746	#D-* Purpose: Qsub new job
747	#D-* Examples:
748	#D-
749	function IGCM_sys_Qsub {
750	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
751
752	typeset status
753
754	if ( $DEBUG_sys ) ; then
755	echo "IGCM_sys_Qsub :" $@
756	fi
757	# We have to change output/error file
758	[ ${#@} = 1 ] && REP_FOR_JOB=${SUBMIT_DIR}
759	[ ${#@} = 2 ] && REP_FOR_JOB=${2}
760	sed -e "s:\# \@ output = .:\# \@ output = ${Script_Output}:" \
761	-e "s:\# \@ error = .:\# \@ error = ${Script_Output}:" \
762	$1 > ${REP_FOR_JOB}/JOB_FOR_IGCM
763	cd $REP_FOR_JOB ; /usr/bin/llsubmit JOB_FOR_IGCM > /tmp/out_command.$$ 2>&1 ; status=$? ; cd - ;
764
765	cat /tmp/out_command.$$
766	if [ ${status} -gt 0 ] ; then
767	IGCM_debug_Print 2 "IGCM_sys_Qsub $1 : error code ${status}"
768	IGCM_debug_Exit "IGCM_sys_Qsub"
769	else
770	JobID=$( gawk {'print $4'} /tmp/out_command.$$ \| tail -1 \| sed -e s/\"//g )
771	IGCM_sys_Rm ${REP_FOR_JOB}/JOB_FOR_IGCM
772	fi
773	IGCM_debug_PopStack "IGCM_sys_Qsub"
774	}
775
776	#D-#==================================================
777	#D-function IGCM_sys_QsubPost
778	#D-* Purpose: Qsub new job on scalaire
779	#D-* Examples:
780	#D-
781	function IGCM_sys_QsubPost {
782	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
783
784	typeset status
785
786	if ( $DEBUG_sys ) ; then
787	echo "IGCM_sys_QsubPost :" $@
788	fi
789	# We have to change output/error file
790	[ ${#@} = 1 ] && REP_FOR_JOB=${POST_DIR}
791	[ ${#@} = 2 ] && REP_FOR_JOB=${2}
792
793	sed -e "s:\# \@ output = .:\# \@ output = ${Script_Post_Output}.out:" \
794	-e "s:\# \@ error = .:\# \@ error = ${Script_Post_Output}.out:" \
795	${libIGCM_POST}/$1.job > ${REP_FOR_JOB}/JOB_FOR_IGCM_$$
796
797	cd $REP_FOR_JOB ; /usr/bin/llsubmit JOB_FOR_IGCM_$$ > /tmp/out_command.$$ 2>&1 ; status=$? ; cd - ;
798
799	cat /tmp/out_command.$$
800	if [ ${status} -gt 0 ] ; then
801	IGCM_debug_Print 2 "IGCM_sys_QsubPost $1 : error code ${status}"
802	IGCM_debug_Exit "IGCM_sys_QsubPost"
803	else
804	JobID=$( gawk {'print $4'} /tmp/out_command.$$ \| tail -1 \| sed -e s/\"//g )
805	IGCM_sys_Rm ${REP_FOR_JOB}/JOB_FOR_IGCM_$$
806	fi
807	IGCM_debug_PopStack "IGCM_sys_QsubPost"
808	}
809
810	#D-*************************
811	#D- File transfer functions
812	#D-*************************
813	#D-
814
815	#D-#==================================================
816	#D-function IGCM_sys_Rsync_out
817	#D-* Purpose: treat return val of rsync
818	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
819	#D- Error values and explanations can depend on your system version.
820	function IGCM_sys_Rsync_out {
821	status=$1
822	if [ ! $status ] ; then
823	echo "rsync error !"
824	fi
825
826	if [ $MYLANG = "fr" ]; then
827	case $status in
828	0) return ;;
829	1) echo "Erreur de rsync ; RERR_SYNTAX : "
830	echo "Erreur de syntaxe ou d'utilisation."
831	return;;
832	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
833	echo "Incompatibilité de protocole."
834	return;;
835	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
836	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
837	echo "répertoires"
838	return;;
839	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
840	echo "Action demandée non supportée : une tentative de manipulation de"
841	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
842	echo "été faite ; ou une option qui est supportée par le client mais"
843	echo "pas par le serveur a été spécifiée."
844	return;;
845	10) echo "Erreur de rsync ; RERR_SOCKETIO"
846	echo "Erreur dans le socket d'entrée sortie"
847	return;;
848	11) echo "Erreur de rsync ; RERR_FILEIO"
849	echo "Erreur d'entrée sortie fichier"
850	return;;
851	12) echo "Erreur de rsync ; RERR_STREAMIO"
852	echo "Erreur dans flux de donnée du protocole rsync"
853	return;;
854	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
855	echo "Erreur avec les diagnostics du programme"
856	return;;
857	14) echo "Erreur de rsync ; RERR_IPC"
858	echo "Erreur dans le code IPC"
859	return;;
860	20) echo "Erreur de rsync ; RERR_SIGNAL"
861	echo "SIGUSR1 ou SIGINT reçu"
862	return;;
863	21) echo "Erreur de rsync ; RERR_WAITCHILD"
864	echo "Une erreur retournée par waitpid()"
865	return;;
866	22) echo "Erreur de rsync ; RERR_MALLOC"
867	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
868	return;;
869	23) echo ""
870	echo "Erreur fichier inexistant"
871	return;;
872	30) echo "Erreur de rsync ; RERR_TIMEOUT"
873	echo "Temps d'attente écoulé dans l'envoi/réception de données"
874	return;;
875	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
876	return;;
877	esac
878	elif [ $MYLANG = "en" ] ; then
879	case $status in
880	0) return;;
881	1) echo "rsync error : Syntax or usage error "
882	return;;
883	2) echo "rsync error : Protocol incompatibility "
884	return;;
885	3) echo "rsync error : Errors selecting input/output files, dirs"
886	return;;
887	4) echo "rsync error : Requested action not supported: an attempt"
888	echo "was made to manipulate 64-bit files on a platform that cannot support"
889	echo "them; or an option was specified that is supported by the client and"
890	echo "not by the server."
891	return;;
892	5) echo "rsync error : Error starting client-server protocol"
893	return;;
894	10) echo "rsync error : Error in socket I/O "
895	return;;
896	11) echo "rsync error : Error in file I/O "
897	return;;
898	12) echo "rsync error : Error in rsync protocol data stream "
899	return;;
900	13) echo "rsync error : Errors with program diagnostics "
901	return;;
902	14) echo "rsync error : Error in IPC code "
903	return;;
904	20) echo "rsync error : Received SIGUSR1 or SIGINT "
905	return;;
906	21) echo "rsync error : Some error returned by waitpid() "
907	return;;
908	22) echo "rsync error : Error allocating core memory buffers "
909	return;;
910	23) echo "rsync error : Partial transfer due to error"
911	return;;
912	24) echo "rsync error : Partial transfer due to vanished source files"
913	return;;
914	30) echo "rsync error : Timeout in data send/receive "
915	return;;
916	*) echo "rsync error : return code of rsync unknown :" $status
917	return;;
918	esac
919	else
920	echo "unknown language $MYLANG."
921	return
922	fi
923	}
924
925	#D-#==================================================
926	#D-function IGCM_sys_Miror_libIGCM
927	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
928	#D-* Examples:
929	#D-
930	function IGCM_sys_Mirror_libIGCM {
931	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
932	if ( $DEBUG_sys ) ; then
933	echo "IGCM_sys_Mirror_libIGCM"
934	fi
935
936	typeset status
937
938	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
939
940	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
941	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
942	status=$?
943
944	if [ ${status} -gt 0 ] ; then
945	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
946	cat /tmp/out_command.$$
947	fi
948	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
949	}
950
951	#D-#==================================================
952	#D-function IGCM_sys_Cp
953	#D-* Purpose: generic cp
954	#D-* Examples:
955	#D-
956	function IGCM_sys_Cp {
957	IGCM_debug_PushStack "IGCM_sys_Cp" $@
958	if ( $DEBUG_sys ) ; then
959	echo "IGCM_sys_Cp :" $@
960	fi
961
962	typeset status
963
964	echo cp $@ > /tmp/out_command.$$ 2>&1
965	\cp $@ >> /tmp/out_command.$$ 2>&1
966	status=$?
967
968	if [ ${status} -gt 0 ] ; then
969	echo "IGCM_sys_Cp : error code ${status}"
970	cat /tmp/out_command.$$
971	IGCM_debug_Exit "IGCM_sys_Cp"
972	else
973	\rm /tmp/out_command.$$
974	fi
975	IGCM_debug_PopStack "IGCM_sys_Cp"
976	}
977
978	#D-#==================================================
979	#D-function IGCM_sys_Rm
980	#D-* Purpose: generic rm
981	#D-* Examples:
982	#D-
983	function IGCM_sys_Rm {
984	IGCM_debug_PushStack "IGCM_sys_Rm" $@
985	if ( $DEBUG_sys ) ; then
986	echo "IGCM_sys_Rm :" $@
987	fi
988
989	typeset status
990
991	echo rm $@ > /tmp/out_command.$$ 2>&1
992	\rm $@ >> /tmp/out_command.$$ 2>&1
993	status=$?
994
995	if [ ${status} -gt 0 ] ; then
996	echo "IGCM_sys_Rm : error code ${status}"
997	cat /tmp/out_command.$$
998	IGCM_debug_Exit "IGCM_sys_Rm"
999	else
1000	\rm /tmp/out_command.$$
1001	fi
1002	IGCM_debug_PopStack "IGCM_sys_Rm"
1003	}
1004
1005	#D-#==================================================
1006	#D-function IGCM_sys_RmRunDir
1007	#D-* Purpose: rm tmpdir (dummy function most of the time batch
1008	#D- scheduler will do the job)
1009	#D-* Examples:
1010	#D-
1011	function IGCM_sys_RmRunDir {
1012	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
1013	if ( $DEBUG_sys ) ; then
1014	echo "IGCM_sys_RmRunDir :" $@
1015	echo "Dummy call, let the scheduler do that."
1016	fi
1017	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
1018	}
1019
1020	#D-#==================================================
1021	#D-function IGCM_sys_Mv
1022	#D-* Purpose: generic move
1023	#D-* Examples:
1024	#D-
1025	function IGCM_sys_Mv {
1026	IGCM_debug_PushStack "IGCM_sys_Mv" $@
1027	if ( $DEBUG_sys ) ; then
1028	echo "IGCM_sys_Mv :" $@
1029	fi
1030
1031	if [ $DRYRUN = 0 ]; then
1032
1033	typeset status
1034
1035	echo mv $@ > /tmp/out_command.$$ 2>&1
1036	\mv $@ >> /tmp/out_command.$$ 2>&1
1037	status=$?
1038
1039	if [ ${status} -gt 0 ] ; then
1040	echo "IGCM_sys_Mv : error code ${status}"
1041	cat /tmp/out_command.$$
1042	IGCM_debug_Exit "IGCM_sys_Mv"
1043	else
1044	\rm /tmp/out_command.$$
1045	fi
1046	fi
1047
1048	IGCM_debug_PopStack "IGCM_sys_Mv"
1049	}
1050
1051	#D-#==================================================
1052	#D-function IGCM_sys_Put_Dir
1053	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1054	#D-* Examples:
1055	#D-
1056	function IGCM_sys_Put_Dir {
1057	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1058	if ( $DEBUG_sys ) ; then
1059	echo "IGCM_sys_Put_Dir :" $@
1060	fi
1061	if [ $DRYRUN = 0 ]; then
1062	if [ ! -d ${1} ] ; then
1063	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1064	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1065	return
1066	fi
1067
1068	typeset status
1069
1070	# Only if we use rsync
1071	#IGCM_sys_TestDirArchive $( dirname $2 )
1072	#
1073	#Command depends on targeted file system
1074	DEBUG_sys=false IGCM_sys_IsFileArchived $2
1075	if [ $? = 0 ] ; then
1076	\rcp -r $1 ${STOREHOST}:$2 > /tmp/out_command.$$ 2>&1
1077	status=$?
1078	else
1079	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
1080	status=$?
1081	fi
1082
1083	if [ ${status} -gt 0 ] ; then
1084	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : rcp or cp failed error code ${status}"
1085	cat /tmp/out_command.$$
1086	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1087	else
1088	\rm /tmp/out_command.$$
1089	fi
1090	fi
1091	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1092	}
1093
1094	#D-#==================================================
1095	#D-function IGCM_sys_Get_Dir
1096	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1097	#D-* Examples:
1098	#D-
1099	function IGCM_sys_Get_Dir {
1100	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1101	if ( $DEBUG_sys ) ; then
1102	echo "IGCM_sys_Get_Dir :" $@
1103	fi
1104	if [ $DRYRUN = 0 ]; then
1105	# if [ ! -d ${1} ] ; then
1106	# echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
1107	# IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1108	# return
1109	# fi
1110
1111	typeset NB_ESSAI DELAI status i
1112	# number of tentative
1113	NB_ESSAI=3
1114	# time delay between tentative
1115	DELAI=2
1116
1117	# Only if we use rsync
1118	#IGCM_sys_TestDirArchive $( dirname $2 )
1119	#
1120	# Command depends on targeted filesystem
1121	DEBUG_sys=false IGCM_sys_IsFileArchived $1
1122	if [ $? = 0 ] ; then
1123	# add dmget (to demigrate all offline files) to reduce time of this command :
1124	#IGCM_sys_RshArchive "dmfind $1 -state MIG -o -state OFL -o -state PAR \| dmget -q -n"
1125	i=0
1126	while [ $i -lt $NB_ESSAI ] ; do
1127	\rcp -rp ${STOREHOST}:$1 $2 > /tmp/out_command.$$ 2>&1
1128	status=$?
1129	if [ ${status} -gt 0 ]; then
1130	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : rcp failed error code ${status} ${i}/${NB_ESSAI}"
1131	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1132	sleep $DELAI
1133	else
1134	break
1135	fi
1136	(( i = i + 1 ))
1137	done
1138	else
1139	\cp -rp $1 $2 > /tmp/out_command.$$ 2>&1
1140	status=$?
1141	if [ ${status} -gt 0 ] ; then
1142	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
1143	cat /tmp/out_command.$$
1144	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1145	else
1146	\rm /tmp/out_command.$$
1147	fi
1148	fi
1149	if [ ${status} -gt 0 ] ; then
1150	echo "IGCM_sys_Get_Dir : error."
1151	cat /tmp/out_command.$$
1152	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1153	else
1154	\rm /tmp/out_command.$$
1155	fi
1156	fi
1157	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1158	}
1159
1160	#D-#==================================================
1161	#D-function IGCM_sys_Get_Master
1162	#D-* Purpose: Copy a complete directory from MASTER filesystem
1163	#D-* Examples:
1164	#D-
1165	function IGCM_sys_Get_Master {
1166	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1167	if ( $DEBUG_sys ) ; then
1168	echo "IGCM_sys_Get_Master :" $@
1169	fi
1170	if [ $DRYRUN = 0 ]; then
1171	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1172	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1173	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1174	return
1175	fi
1176
1177	typeset NB_ESSAI DELAI status i
1178	# number of tentative
1179	NB_ESSAI=3
1180	# time delay between tentative
1181	DELAI=2
1182
1183	i=0
1184	while [ $i -lt $NB_ESSAI ] ; do
1185	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
1186	status=$?
1187	if [ ${status} -gt 0 ]; then
1188	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1189	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1190	sleep $DELAI
1191	else
1192	break
1193	fi
1194	(( i = i + 1 ))
1195	done
1196
1197	if [ ${status} -gt 0 ] ; then
1198	echo "IGCM_sys_Get_Master : error."
1199	cat /tmp/out_command.$$
1200	IGCM_debug_Exit "IGCM_sys_Get_Master"
1201	else
1202	\rm /tmp/out_command.$$
1203	fi
1204	fi
1205	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1206	}
1207
1208	#====================================================
1209	#- Call IGCM_sys_Mirror_libIGCM now !
1210	if ( $MirrorlibIGCM ) ; then
1211	IGCM_sys_Mirror_libIGCM
1212	fi
1213
1214	#D-#==================================================
1215	#D-function IGCM_sys_Put_Rest
1216	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1217	#D- File and target directory must exist.
1218	#D-* Examples:
1219	#D-
1220	function IGCM_sys_Put_Rest {
1221	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1222	if ( $DEBUG_sys ) ; then
1223	echo "IGCM_sys_Put_Rest :" $@
1224	fi
1225	if [ $DRYRUN = 0 ]; then
1226	if [ ! -f ${1} ] ; then
1227	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1228	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1229	fi
1230
1231	typeset status
1232	#
1233	if [ X${JobType} = XRUN ] ; then
1234	IGCM_sys_Chmod 444 ${1}
1235	fi
1236	#
1237	#
1238	#Command depends on targeted file system
1239	DEBUG_sys=false IGCM_sys_IsFileArchived $2
1240	if [ $? = 0 ] ; then
1241	mfput $1 $2 > /tmp/out_command.$$ 2>&1
1242	status=$?
1243	else
1244	IGCM_sys_MkdirArchive $( dirname $2 )
1245	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1246	status=$?
1247	fi
1248
1249	# #RSYNC WITH NETWORK RSH CALL
1250	# IGCM_sys_MkdirArchive $( dirname $2 )
1251	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1252	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1253
1254	# #RSYNC WITH NFS USE
1255	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1256	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1257
1258	# status=$?
1259	# IGCM_sys_Rsync_out $status
1260
1261	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1262	# (( status=status+$? ))
1263
1264	if [ ${status} -gt 0 ] ; then
1265	echo "IGCM_sys_Put_Rest : mfput or cp failed error code ${status}"
1266	cat /tmp/out_command.$$
1267	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1268	else
1269	\rm /tmp/out_command.$$
1270	fi
1271	fi
1272	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1273	}
1274
1275	#D-#==================================================
1276	#D-function IGCM_sys_PutBuffer_Rest
1277	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1278	#D- File and target directory must exist.
1279	#D-* Examples:
1280	#D-
1281	function IGCM_sys_PutBuffer_Rest {
1282	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1283	if ( $DEBUG_sys ) ; then
1284	echo "IGCM_sys_PutBuffer_Rest :" $@
1285	fi
1286	if [ $DRYRUN = 0 ]; then
1287	if [ ! -f ${1} ] ; then
1288	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1289	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1290	fi
1291
1292	typeset status
1293	#
1294	if [ X${JobType} = XRUN ] ; then
1295	IGCM_sys_Chmod 444 ${1}
1296	fi
1297
1298	#
1299	# USUAL WAY
1300	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1301	status=$?
1302
1303	if [ ${status} -gt 0 ] ; then
1304	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1305	[ -f ${2} ] && ls -l ${2}
1306	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1307	cat /tmp/out_command.$$
1308	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1309	else
1310	\rm /tmp/out_command.$$
1311	fi
1312	fi
1313	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1314	}
1315
1316	#D-#==================================================
1317	#D-function IGCM_sys_PrepareTaredRestart
1318	#D-* Purpose: Prepare tared restart to be access by computing job.
1319	#D-* Examples:
1320	#D-
1321	function IGCM_sys_PrepareTaredRestart {
1322	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1323	if [ $DRYRUN = 0 ]; then
1324	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1325	fi
1326	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1327	}
1328
1329	#D-#==================================================
1330	#D-function IGCM_sys_Put_Out
1331	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1332	#D-* Examples:
1333	#D-
1334	function IGCM_sys_Put_Out {
1335	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1336	if ( $DEBUG_sys ) ; then
1337	echo "IGCM_sys_Put_Out :" $@
1338	fi
1339
1340	typeset status
1341
1342	if [ $DRYRUN = 0 ]; then
1343	if [ ! -f ${1} ] ; then
1344	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1345	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1346	return 1
1347	fi
1348
1349	#
1350	if [ X${JobType} = XRUN ] ; then
1351	if [ X${3} = X ] ; then
1352	IGCM_sys_Chmod 444 ${1}
1353	fi
1354	fi
1355	#
1356	#
1357	#Command depends on targeted file system
1358	DEBUG_sys=false IGCM_sys_IsFileArchived $2
1359	if [ $? = 0 ] ; then
1360	mfput $1 $2 > /tmp/out_command.$$ 2>&1
1361	status=$?
1362	else
1363	IGCM_sys_MkdirArchive $( dirname $2 )
1364	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1365	status=$?
1366	fi
1367
1368	# #RSYNC WITH NETWORK RSH CALL
1369	# IGCM_sys_MkdirArchive $( dirname $2 )
1370	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1371	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1372
1373	# #RSYNC WITH NFS USE
1374	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1375	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1376
1377	# status=$?
1378	# IGCM_sys_Rsync_out $status
1379
1380	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1381	# (( status=status+$? ))
1382
1383	if [ ${status} -gt 0 ] ; then
1384	IGCM_debug_Print 2 "IGCM_sys_Put_Out : mfput or cp failed error code ${status}"
1385	cat /tmp/out_command.$$
1386	IGCM_debug_Exit "IGCM_sys_Put_Out"
1387	else
1388	\rm /tmp/out_command.$$
1389	fi
1390	fi
1391	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1392	return 0
1393	}
1394
1395	#D-#==================================================
1396	#D-function IGCM_sys_PutBuffer_Out
1397	#D-* Purpose: Copy a file on ${WORKDIR} after having chmod it in readonly
1398	#D-* Examples:
1399	#D-
1400	function IGCM_sys_PutBuffer_Out {
1401	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1402	if ( $DEBUG_sys ) ; then
1403	echo "IGCM_sys_PutBuffer_Out :" $@
1404	fi
1405
1406	typeset NB_ESSAI DELAI status i exist skip
1407
1408	# number of tentative
1409	NB_ESSAI=3
1410	# time delay between tentative
1411	DELAI=2
1412
1413	if [ $DRYRUN = 0 ]; then
1414	if [ ! -f ${1} ] ; then
1415	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1416	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1417	return 1
1418	fi
1419	#
1420	IGCM_sys_Mkdir $( dirname $2 )
1421	#
1422
1423	exist=false
1424	skip=false
1425	if [ -f $2 ] ; then
1426	IGCM_debug_Print 1 "$2 already exist"
1427	exist=true
1428	if [ "X$( diff $1 $2 )" = X ] ; then
1429	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1430	status=0
1431	skip=true
1432	else
1433	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1434	skip=false
1435	fi
1436	fi
1437	#
1438	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1439	IGCM_sys_Chmod u+w $2
1440	fi
1441
1442	if [ X${skip} = Xfalse ] ; then
1443	i=0
1444	while [ $i -lt $NB_ESSAI ] ; do
1445	# USUAL WAY
1446	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1447	status=$?
1448	if [ ${status} -gt 0 ]; then
1449	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1450	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1451	[ -f ${2} ] && ls -l ${2}
1452	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1453	sleep $DELAI
1454	else
1455	break
1456	fi
1457	(( i = i + 1 ))
1458	done
1459	fi
1460
1461	if [ ${status} -gt 0 ] ; then
1462	echo "IGCM_sys_PutBuffer_Out : error."
1463	[ -f ${2} ] && ls -l ${2}
1464	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1465	cat /tmp/out_command.$$
1466	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1467	else
1468
1469	if [ X${JobType} = XRUN ] ; then
1470	if [ X${3} = X ] ; then
1471	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1472	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1473	fi
1474	fi
1475
1476	\rm /tmp/out_command.$$
1477	fi
1478	fi
1479	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1480	return 0
1481	}
1482
1483	#D-#==================================================
1484	#D-function IGCM_sys_Get
1485	#D-* Purpose: Get a file from ${ARCHIVE}
1486	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1487	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1488	function IGCM_sys_Get {
1489	IGCM_debug_PushStack "IGCM_sys_Get" $@
1490
1491	typeset DEST status dm_liste
1492
1493	if ( $DEBUG_sys ) ; then
1494	echo "IGCM_sys_Get :" $@
1495	fi
1496	if [ $DRYRUN -le 2 ]; then
1497	if [ X${1} = X'/l' ] ; then
1498	eval set +A dm_liste \${${2}}
1499	else
1500	dm_liste=${1}
1501	fi
1502	eval DEST=\${${#}}
1503
1504	# test if the (first) file is present in the old computation :
1505	DEBUG_sys=false IGCM_sys_IsFileArchived ${dm_liste[0]}
1506	if [ $? = 0 ] ; then
1507	IGCM_sys_TestFileArchive ${dm_liste[0]}
1508	status=$?
1509	else
1510	IGCM_sys_TestFileBuffer ${dm_liste[0]}
1511	status=$?
1512	fi
1513
1514	if [ ${status} -gt 0 ] ; then
1515	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
1516	IGCM_debug_Exit "IGCM_sys_Get"
1517	return
1518	fi
1519
1520	#Command depends on targeted file system
1521	DEBUG_sys=false IGCM_sys_IsFileArchived ${dm_liste[0]}
1522	if [ $? = 0 ] ; then
1523	mfget ${dm_liste[*]} ${DEST} > /tmp/out_command.$$ 2>&1
1524	status=$?
1525	else
1526	\cp ${dm_liste[*]} ${DEST} > /tmp/out_command.$$ 2>&1
1527	status=$?
1528	fi
1529
1530	# #RSYNC WITH NETWORK RSH CALL
1531	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1532	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1533
1534	# #RSYNC WITH NFS USE
1535	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1536	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1537
1538	# status=$?
1539	# IGCM_sys_Rsync_out $status
1540
1541	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1542	# (( status=status+$? ))
1543
1544	if [ ${status} -gt 0 ] ; then
1545	IGCM_debug_Print 2 "IGCM_sys_Get : mfget or cp failed error code ${status}"
1546	cat /tmp/out_command.$$
1547	IGCM_debug_Exit "IGCM_sys_Get"
1548	else
1549	\rm /tmp/out_command.$$
1550	fi
1551	fi
1552	IGCM_debug_PopStack "IGCM_sys_Get"
1553	}
1554
1555	#D-#==================================================
1556	#D-function IGCM_sys_GetBuffer
1557	#D-* Purpose: Get a file from ${SCRATCHDIR}
1558	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1559	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1560	function IGCM_sys_GetBuffer {
1561	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1562
1563	typeset DEST buf_liste target file_work
1564	typeset NB_ESSAI DELAI status i
1565
1566	if ( $DEBUG_sys ) ; then
1567	echo "IGCM_sys_GetBuffer :" $@
1568	fi
1569
1570	# number of tentative
1571	NB_ESSAI=3
1572	# time delay between tentative
1573	DELAI=2
1574
1575	if [ $DRYRUN -le 2 ]; then
1576	if [ X${1} = X'/l' ] ; then
1577	eval set +A buf_liste \${${2}}
1578	else
1579	eval set +A buf_liste ${1}
1580	fi
1581	eval DEST=\${${#}}
1582
1583	#USUAL WAY
1584	if [ X${1} = X'/l' ] ; then
1585	for target in ${buf_liste[*]} ; do
1586	local_file=$( basename ${target} )
1587	i=0
1588	while [ $i -lt $NB_ESSAI ] ; do
1589	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1590	status=$?
1591	if [ ${status} -gt 0 ]; then
1592	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1593	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1594	sleep $DELAI
1595	else
1596	break
1597	fi
1598	(( i = i + 1 ))
1599	done
1600	if [ ${status} -gt 0 ] ; then
1601	echo "IGCM_sys_Get : error"
1602	cat /tmp/out_command.$$
1603	\rm /tmp/out_command.$$
1604	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1605	else
1606	\rm /tmp/out_command.$$
1607	fi
1608	done
1609	else
1610	i=0
1611	while [ $i -lt $NB_ESSAI ] ; do
1612	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1613	status=$?
1614	if [ ${status} -gt 0 ]; then
1615	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1616	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1617	sleep $DELAI
1618	else
1619	break
1620	fi
1621	(( i = i + 1 ))
1622	done
1623	if [ ${status} -gt 0 ] ; then
1624	echo "IGCM_sys_Get : error"
1625	cat /tmp/out_command.$$
1626	\rm /tmp/out_command.$$
1627	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1628	else
1629	\rm /tmp/out_command.$$
1630	fi
1631	fi
1632	fi
1633	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1634	}
1635
1636	#D-#==================================================
1637	#D-function IGCM_sys_GetDate_FichWork
1638	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1639	#D-* Examples:
1640	#D-
1641	function IGCM_sys_GetDate_FichWork {
1642	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1643	if ( $DEBUG_sys ) ; then
1644	echo "IGCM_sys_GetDate_FichWork :" $@
1645	fi
1646
1647	if [ $# -ge 3 ] ; then
1648	mode=$3
1649	TimeStyle=$4
1650	else
1651	mode="default"
1652	TimeStyle="%Y%m%d%H%M%S"
1653	fi
1654
1655	typeset dateF
1656	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1657
1658	case $mode in
1659	"default")
1660	eval ${2}=${dateF[5]}
1661	;;
1662	"SplitFields")
1663	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1664	;;
1665	esac
1666
1667
1668	# donne la date filesys d'un fichier sur la machine work
1669	IGCM_debug_PopStack "IGCM_sys_FichWork"
1670	}
1671
1672	#D-#==================================================
1673	#D-function IGCM_sys_GetDate_FichArchive
1674	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1675	#D-* Examples:
1676	#D-
1677	function IGCM_sys_GetDate_FichArchive {
1678	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1679	if ( $DEBUG_sys ) ; then
1680	echo "IGCM_sys_GetDate_FichArchive :" $@
1681	fi
1682	typeset dateF
1683	set +A dateF -- $( rsh ${STOREHOST} -n ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1684	eval ${2}=${dateF[5]}
1685
1686	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1687	}
1688
1689	#D-#==================================================
1690	#D-function IGCM_sys_GetDate_Monitoring
1691	#D-* Purpose: get the last year for which the monitoring has been computed
1692	#D-* Examples:
1693	#D-
1694	function IGCM_sys_GetDate_Monitoring {
1695	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1696	if ( $DEBUG_sys ) ; then
1697	echo "IGCM_sys_GetDate_Monitoring :" $@
1698	fi
1699
1700	IGCM_sys_Cd /tmp
1701	IGCM_sys_Get ${1} .
1702	eval ${2}=$( cdo showyear $( basename ${1} ) 2> /dev/null \| gawk '{ print $NF }' )
1703	IGCM_sys_Cd -
1704
1705	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1706	}
1707
1708	#D-#==================================================
1709	#D-function IGCM_sys_Put_Dods
1710	#D-* Purpose: Put $(ARCHIVE) files on DODS internet protocole.
1711	#D-* Examples:
1712	#D-
1713	function IGCM_sys_Put_Dods {
1714	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1715	if ( $DEBUG_sys ) ; then
1716	echo "IGCM_sys_Put_Dods :" $@
1717	fi
1718	if [ $DRYRUN = 0 ]; then
1719	# We take our time on that
1720	sleep 10
1721	IGCM_sys_TestDirArchive ${R_SAVE}/${1}
1722	if [ $? != 0 ] ; then
1723	echo "WARNING : IGCM_sys_Put_Dods ${R_SAVE}/${1} DOES NOT EXIST ."
1724	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1725	return
1726	fi
1727
1728	typeset status
1729	#
1730	rsh ${STOREHOST} exec /bin/ksh <<EOF
1731	cd ${R_SAVE}
1732	/arch/local/bin/dods_rm DODS/pub/${LOGIN}/${R_DODS}/${1} > /dev/null 2>&1
1733	/bin/chmod -R u+w ${R_SAVE}/${1}
1734	/arch/local/bin/dods_cp ${1} DODS/pub/${LOGIN}/${R_DODS} > /dev/null 2>&1
1735	/bin/chmod -R +rX ${R_SAVE}/${1}
1736	/bin/chmod -R u+w ${R_SAVE}/${1}
1737	EOF
1738	status=$?
1739
1740	if [ ${status} -gt 0 ] ; then
1741	echo "IGCM_sys_Put_Dods : error."
1742	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1743	fi
1744	fi
1745	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1746	}
1747
1748	##############################################################
1749	# REBUILD OPERATOR
1750
1751	#D-#==================================================
1752	#D-function IGCM_sys_rebuild
1753	#D-* Purpose: rebuild parallel files
1754	#D-* Examples:
1755	#D-
1756	function IGCM_sys_rebuild {
1757	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1758	if ( $DEBUG_sys ) ; then
1759	echo "IGCM_sys_rebuild :" $@
1760	fi
1761
1762	typeset NB_ESSAI DELAI status i firstArg
1763	# number of tentative
1764	NB_ESSAI=3
1765	# time delay between tentative
1766	DELAI=2
1767
1768	i=0
1769	while [ $i -lt $NB_ESSAI ] ; do
1770	/smphome/rech/psl/rpsl035/bin/rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1771	status=$?
1772	if [ ${status} -gt 0 ] ; then
1773	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1774	cat /tmp/out_command.$$
1775	\rm /tmp/out_command.$$
1776	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1777	firstArg=${1}
1778	\rm ${firstArg}
1779	sleep $DELAI
1780	else
1781	\rm /tmp/out_command.$$
1782	break
1783	fi
1784	(( i = i + 1 ))
1785	done
1786
1787	if [ ${status} -gt 0 ] ; then
1788	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1789	IGCM_debug_Exit "rebuild"
1790	fi
1791
1792	IGCM_debug_PopStack "IGCM_sys_rebuild"
1793	}
1794
1795	#D-#==================================================
1796	#D-function IGCM_sys_rebuild_station
1797	#D-* Purpose: rebuild parallel files describing station
1798	#D-* Examples:
1799	#D-
1800	function IGCM_sys_rebuild_station {
1801	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1802	typeset i list_opt file_in file_out prefix_invert list_invert
1803	if ( $DEBUG_sys ) ; then
1804	echo "IGCM_sys_rebuild_station :" $@
1805	fi
1806	list_opt=$@
1807
1808	# Invert Axis : t,x -> x,t
1809	# t,pres,x -> x,t,pres
1810	# So that we can concatenate along x
1811	i=0
1812	for file_in in ${list_opt} ; do
1813	(( i = i + 1))
1814	[ ${i} = 1 ] && file_out=${file_in} && continue
1815	# detect time counter and do the job only if present
1816	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1)
1817	if [ X${var_unlim} = Xtime_counter ] ; then
1818	prefix_invert=$( basename ${file_in} .nc )
1819	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1820	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1821	fi
1822	done
1823
1824	# Concatenate
1825	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1826
1827	# Re-ivert file
1828	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1829
1830	# Station re-ordering is too expansive to be run within libIGCM
1831	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
1832	# This re-ordering must be done "in memory" by the cmorization process
1833	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
1834	# BEGIN reordering
1835
1836	# Only LMDZ text output contains the exact ordering of the station.
1837	# We isolate this in the code below:
1838	# 0 38 -157.5000000000000 70.98591549295774
1839	# 0 54 27.49999999999999 67.18309859154928
1840	# 0 56 -62.50000000000001 82.39436619718309
1841	# 0 79 12.49999999999999 78.59154929577466
1842	# 0 116 -165.0000000000000 76.05633802816901
1843	# 0 117 130.0000000000000 70.98591549295774
1844	# 0 118 110.0000000000000 87.46478873239437
1845	# 1 40 4.999999999999995 51.97183098591550
1846	# typeset iStation iProc list_opt file_in file_out prefix_invert
1847	# typeset -Z4 j4
1848	# typeset -Z3 j3
1849
1850	# unset list_opt
1851	# set +A list_opt $@
1852
1853	# Filename after rebuild
1854	# file_out=${list_opt[0]}
1855	# Prefix of output files
1856	# prefix_invert=$( basename ${file_out} .nc )
1857	# Number of procs
1858	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
1859
1860	# iProc=0
1861	# while [ ${iProc} -lt ${num_proc} ] ; do
1862	# Array containing Station as a number
1863	# unset proc_stn
1864	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
1865	# Number of stations produced by processor proc
1866	# stationLast=${#proc_stn[*]}
1867	# Proc number on 4 digits
1868	# j4=${iProc}
1869	# Init
1870	# iStation=0
1871	# while [ ${iStation} -lt ${stationLast} ] ; do
1872	# Station number on 3 digits
1873	# j3=${proc_stn[${iStation}]}
1874	# Extract station
1875	# Invert Axis : t,x -> x,t
1876	# t,pres,x -> x,t,pres
1877	# So that we can concatenate along x
1878	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
1879	# (( iStation = iStation + 1 ))
1880	# done
1881	# (( iProc = iProc + 1 ))
1882	# done
1883
1884	# Concatenate all station along x
1885	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
1886
1887	# Re-invert file
1888	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
1889
1890	# END reordering
1891
1892	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
1893	}
1894
1895	############################################################
1896	# Activate Running Environnment Variables
1897
1898	#D-#==================================================
1899	#D-function IGCM_sys_desactiv_variables
1900	#D-* Purpose: set environement variables prior to execution
1901	#D-* Examples:
1902	#D-
1903	function IGCM_sys_activ_variables {
1904	IGCM_debug_PushStack "IGCM_sys_activ_variables"
1905	if ( $DEBUG_sys ) ; then
1906	echo "IGCM_sys_activ_variables"
1907	fi
1908
1909	# --------------------------------------------------------------------
1910	#D- MPI specifications
1911	# --------------------------------------------------------------------
1912
1913	# --------------------------------------------------------------------
1914	#D- Other specifications
1915	# --------------------------------------------------------------------
1916
1917	ulimit -s unlimited
1918	## to be done only one time
1919	## export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/smplocal/pub/NetCDF/4.1.3/lib:/smplocal/pub/HDF5/1.8.9/seq/lib
1920	## echo ${LD_LIBRARY_PATH} \| grep -i netcdf >/dev/null 2>&1 \|\| export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/smplocal/pub/NetCDF/4.1.3/lib:/smplocal/pub/HDF5/1.8.9/seq/lib
1921	IGCM_debug_Print 1 "set LD_LIBRARY_PATH=${LD_LIBRARY_PATH}"
1922
1923	export MP_STDOUTMODE=combined
1924	IGCM_debug_Print 1 "set MP_STDOUTMODE=${MP_STDOUTMODE}"
1925
1926	## fix to reduce memory usage. Required since 2014/22/04. On ada IDRIS.
1927	export MP_EUILIBPATH=/smplocal/lib/ibmhpc/pe12012/ppe.pami/gnu/lib64/pami64
1928	IGCM_debug_Print 1 "set MP_EUILIBPATH=${MP_EUILIBPATH}"
1929
1930	IGCM_debug_PopStack "IGCM_sys_activ_variables"
1931	}
1932
1933	############################################################
1934	# Desactivate Running Environnment Variables
1935
1936	#D-#==================================================
1937	#D-function IGCM_sys_desactiv_variables
1938	#D-* Purpose: unset environement variables after execution
1939	#D-* Examples:
1940	#D-
1941	function IGCM_sys_desactiv_variables {
1942	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
1943	if ( $DEBUG_sys ) ; then
1944	echo "IGCM_sys_desactiv_variables"
1945	fi
1946	# --------------------------------------------------------------------
1947	#D- MPI specifications
1948	# --------------------------------------------------------------------
1949
1950	# --------------------------------------------------------------------
1951	#D- Other specifications
1952	# --------------------------------------------------------------------
1953
1954	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
1955	}
1956
1957	############################################################
1958	# Build MPI/OMP scripts run file (dummy function)
1959
1960	#D-#==================================================
1961	#D-function IGCM_sys_build_run_file
1962	#D-* Purpose: build run file (deprecated)
1963	#D-* Examples:
1964	#D-
1965	function IGCM_sys_build_run_file {
1966
1967	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
1968
1969	}
1970
1971	############################################################
1972	# Build MPI/OMP scripts
1973
1974	#D-#==================================================
1975	#D-function IGCM_sys_build_execution_scripts
1976	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
1977	#D-* Examples:
1978	#D-
1979	function IGCM_sys_build_execution_scripts
1980	{
1981	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
1982	if ( $DEBUG_sys ) ; then
1983	echo "IGCM_sys_build_execution_scripts " $@
1984	fi
1985
1986	EXECUTION=${HOST_MPIRUN_COMMAND}
1987
1988	if ( ${OK_PARA_MPMD} ) ; then
1989
1990	if [ -f run_file ] ; then
1991	IGCM_sys_Rm -f run_file
1992	fi
1993	touch run_file
1994
1995	# run_file construction
1996
1997	# Then first loop on the components for the coupler ie oasis
1998
1999	### the coupler ie oasis must be the first one
2000	for comp in ${config_ListOfComponents[*]} ; do
2001
2002	eval ExeNameIn=\${config_Executable_${comp}[0]}
2003	eval ExeNameOut=\${config_Executable_${comp}[1]}
2004
2005	# for CPL component only
2006	if [ "X${comp}" = "XCPL" ] ; then
2007
2008	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2009	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2010
2011	if ( ${OK_PARA_MPI} ) ; then
2012	(( mpi_count = 1 ))
2013	until [ ${mpi_count} -gt ${comp_proc_mpi_loc} ] ; do
2014	echo "./${ExeNameOut}" >> run_file
2015	(( mpi_count = mpi_count + 1 ))
2016	done
2017	else
2018	echo "./${ExeNameOut} " >> run_file
2019	fi
2020	fi
2021	done
2022
2023	# Then second loop on the components
2024
2025	for comp in ${config_ListOfComponents[*]} ; do
2026
2027	eval ExeNameIn=\${config_Executable_${comp}[0]}
2028	eval ExeNameOut=\${config_Executable_${comp}[1]}
2029
2030	# Only if we really have an executable for the component and not the coupler ie oasis:
2031	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2032
2033	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2034	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2035
2036	if ( ${OK_PARA_MPI} ) ; then
2037	(( mpi_count = 1 ))
2038	until [ ${mpi_count} -gt ${comp_proc_mpi_loc} ] ; do
2039	if ( ${OK_PARA_OMP} ) ; then
2040	# Check if the number of threads is correct
2041	case ${comp_proc_omp_loc} in
2042	1\|2)
2043	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2044	;;
2045	4\|8\|16)
2046	IGCM_debug_Exit "ERROR with OMP parameters !"
2047	IGCM_debug_Print 2 "Beware : ${comp_proc_omp_loc} is too much for MPMD mode"
2048	IGCM_debug_Print 2 "You will use more CPU than needed"
2049	IGCM_debug_Verif_Exit
2050	;;
2051	*)
2052	IGCM_debug_Exit "ERROR with OMP parameters !"
2053	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2054	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2055	IGCM_debug_Verif_Exit
2056	;;
2057	esac
2058	echo "env OMP_NUM_THREADS=$comp_proc_omp_loc ./${ExeNameOut}" >> run_file
2059	else
2060	echo "./${ExeNameOut}" >> run_file
2061	fi
2062	(( mpi_count = mpi_count + 1 ))
2063	done
2064	else
2065	echo "./${ExeNameOut} " >> run_file
2066	fi
2067	if ( ${OK_PARA_OMP} ) ; then
2068	export KMP_STACKSIZE=200m
2069	fi
2070	fi
2071	done
2072
2073	EXECUTION="${HOST_MPIRUN_COMMAND} -pgmmodel mpmd -cmdfile ./run_file"
2074
2075	IGCM_sys_Chmod u+x run_file
2076	if ( $DEBUG_sys ) ; then
2077	echo "run_file contains : "
2078	cat run_file
2079	fi
2080
2081	else # Only one executable. launch it.
2082
2083	for comp in ${config_ListOfComponents[*]} ; do
2084
2085	# Only if we really have an executable for the component :
2086	eval ExeNameOut=\${config_Executable_${comp}[1]}
2087	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2088
2089	if ( ${OK_PARA_OMP} ) ; then
2090	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2091	export KMP_STACKSIZE=200m
2092	#export KMP_LIBRARY=turnaround
2093	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2094	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2095	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2096	fi
2097
2098	EXECUTION="${HOST_MPIRUN_COMMAND} ./${ExeNameOut}"
2099	fi
2100
2101	done
2102
2103	fi
2104
2105	IGCM_debug_Print 1 "sys ada : execution command is "
2106	IGCM_debug_Print 1 "$EXECUTION"
2107
2108	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2109	}
2110
2111	#D-#==================================================
2112	#D-function IGCM_sys_check_path
2113	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2114	#D-* do not point to an important use directory. Stop immediately in that case.
2115	#D-* Examples:
2116	#D-
2117	function IGCM_sys_check_path {
2118	IGCM_debug_PushStack "IGCM_sys_check_path"
2119	if ( $DEBUG_sys ) ; then
2120	echo "IGCM_sys_check_path"
2121	fi
2122
2123	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
2124	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2125	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2126	IGCM_debug_Exit "This will stop the job"
2127	fi
2128	IGCM_debug_PopStack "IGCM_sys_check_path"
2129	}
2130
2131	#D-#==================================================
2132	#D-function IGCM_sys_check_quota
2133	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2134	#D-* Examples:
2135	#D-
2136	function IGCM_sys_check_quota {
2137	IGCM_debug_PushStack "IGCM_sys_check_quota"
2138	if ( $DEBUG_sys ) ; then
2139	echo "IGCM_sys_check_quota"
2140	fi
2141	# Limit of quota (in %)
2142	limit_quota=90
2143
2144	# Check of the volume
2145	volume_quota=$(quota_u -w 2>/dev/null\| grep 'Quota soft' \| gawk '{print $5}')
2146	if [ ! X${volume_quota} = X ] ; then
2147	quota_volume=${volume_quota%%\%}
2148	# echo $quota_volume
2149	if [ $(echo "${quota_volume} > ${limit_quota}" \| bc) -eq 1 ] ; then
2150	IGCM_debug_Print 1 "Please, check your quota of volume on workgpfs"
2151	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2152	IGCM_debug_Print 1 "Use the quota_u -w command to check"
2153	IGCM_debug_Print 1 "You must have more than 10% available to run"
2154	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2155	IGCM_debug_Verif_Exit
2156	fi
2157
2158	fi
2159	IGCM_debug_PopStack "IGCM_sys_check_quota"
2160	}
2161
2162	#D-#==================================================
2163	#D-function IGCM_sys_GetJobID
2164	#D-* Purpose: Check if job_name is currently
2165	#D- running or in queue
2166	#D-* Examples: IGCM_sys_GetJobID ${JobName} NbRun
2167	#D-
2168	function IGCM_sys_GetJobID {
2169	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2170	if ( $DEBUG_sys ) ; then
2171	echo "IGCM_sys_GetJobID"
2172	fi
2173
2174	# Print only the full (-W) JobID (%id) and JobName (%jn)
2175	ID=$( llq -u $2 -W -f %id %jn \| \
2176	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2177
2178	eval ${3}=${ID}
2179
2180	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2181	}
2182
2183	#D-#==================================================
2184	#D-function IGCM_sys_CountJobInQueue
2185	#D-* Purpose: Check if job_name is currently
2186	#D- running or in queue
2187	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2188	#D-
2189	function IGCM_sys_CountJobInQueue {
2190	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2191	if ( $DEBUG_sys ) ; then
2192	echo "IGCM_sys_CountJobInQueue"
2193	fi
2194
2195	# Print only the full (-W) JobName (%jn)
2196	NbRun=$( llq -W -f %jn \| grep -c "$1" )
2197
2198	eval ${2}=${NbRun}
2199
2200	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2201	}
2202
2203	#D-#==================================================
2204	#D-function IGCM_sys_ListJobInQueue
2205	#D-* Purpose: Check if job_name is currently
2206	#D- running or in queue
2207	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2208	#D-
2209	function IGCM_sys_ListJobInQueue {
2210	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2211	if ( $DEBUG_sys ) ; then
2212	echo "IGCM_sys_ListJobInQueue"
2213	fi
2214
2215	# With -W option, column width is as large as necessary
2216	set -A JobList $( llq -u $1 -W -f %jn \| head -n -2 \| tail -n +3 \| \
2217	gawk '( $1 != /TS/ && \
2218	$1 !~ /PACK/ && \
2219	$1 !~ /REBUILD/ && \
2220	$1 !~ /pack/ ) \
2221	{ print $1 }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
2222
2223	eval set -A ${2} ${JobList[*]}
2224
2225	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2226	}
2227
2228	##############################################################
2229	# NCO OPERATOR
2230
2231	#D-#==================================================
2232	#D-function IGCM_sys_ncap2
2233	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2234	#D-* Examples:
2235	#D-
2236	function IGCM_sys_ncap2 {
2237	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2238	if ( $DEBUG_sys ) ; then
2239	echo "IGCM_sys_ncap2 :" $@
2240	fi
2241
2242	typeset NB_ESSAI DELAI status i
2243	# number of tentative
2244	NB_ESSAI=3
2245	# time delay between tentative
2246	DELAI=2
2247
2248	i=0
2249	while [ $i -lt $NB_ESSAI ] ; do
2250	ncap2 --nocoords "$@" > /tmp/out_command.$$ 2>&1
2251	status=$?
2252	if [ ${status} -gt 0 ] ; then
2253	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2254	cat /tmp/out_command.$$
2255	\rm /tmp/out_command.$$
2256	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2257	sleep $DELAI
2258	else
2259	\rm /tmp/out_command.$$
2260	break
2261	fi
2262	(( i = i + 1 ))
2263	done
2264
2265	if [ ${status} -gt 0 ] ; then
2266	echo "IGCM_sys_ncap2 : ncap2 error"
2267	IGCM_debug_Exit "ncap2"
2268	fi
2269
2270	IGCM_debug_PopStack "IGCM_sys_ncap2"
2271	}
2272
2273	#D-#==================================================
2274	#D-function IGCM_sys_ncatted
2275	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2276	#D-* Examples:
2277	#D-
2278	function IGCM_sys_ncatted {
2279	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2280	if ( $DEBUG_sys ) ; then
2281	echo "IGCM_sys_ncatted :" $@
2282	fi
2283
2284	typeset NB_ESSAI DELAI status i
2285	# number of tentative
2286	NB_ESSAI=3
2287	# time delay between tentative
2288	DELAI=2
2289
2290	i=0
2291	while [ $i -lt $NB_ESSAI ] ; do
2292	ncatted "$@" > /tmp/out_command.$$ 2>&1
2293	status=$?
2294	if [ ${status} -gt 0 ] ; then
2295	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2296	cat /tmp/out_command.$$
2297	\rm /tmp/out_command.$$
2298	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2299	sleep $DELAI
2300	else
2301	\rm /tmp/out_command.$$
2302	break
2303	fi
2304	(( i = i + 1 ))
2305	done
2306
2307	if [ ${status} -gt 0 ] ; then
2308	echo "IGCM_sys_ncatted : ncatted error"
2309	IGCM_debug_Exit "ncatted"
2310	fi
2311
2312	IGCM_debug_PopStack "IGCM_sys_ncatted"
2313	}
2314
2315	#D-#==================================================
2316	#D-function IGCM_sys_ncbo
2317	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2318	#D-* Examples:
2319	#D-
2320	function IGCM_sys_ncbo {
2321	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2322	if ( $DEBUG_sys ) ; then
2323	echo "IGCM_sys_ncbo :" $@
2324	fi
2325
2326	typeset NB_ESSAI DELAI status i
2327	# number of tentative
2328	NB_ESSAI=3
2329	# time delay between tentative
2330	DELAI=2
2331
2332	i=0
2333	while [ $i -lt $NB_ESSAI ] ; do
2334	ncbo --nocoords $@ > /tmp/out_command.$$ 2>&1
2335	status=$?
2336	if [ ${status} -gt 0 ] ; then
2337	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2338	cat /tmp/out_command.$$
2339	\rm /tmp/out_command.$$
2340	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2341	sleep $DELAI
2342	else
2343	\rm /tmp/out_command.$$
2344	break
2345	fi
2346	(( i = i + 1 ))
2347	done
2348
2349	if [ ${status} -gt 0 ] ; then
2350	echo "IGCM_sys_ncbo : ncbo error"
2351	IGCM_debug_Exit "ncbo"
2352	fi
2353
2354	IGCM_debug_PopStack "IGCM_sys_ncbo"
2355	}
2356
2357	#D-#==================================================
2358	#D-function IGCM_sys_ncdif
2359	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2360	#D-* Examples:
2361	#D-
2362	function IGCM_sys_ncdiff {
2363	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2364	if ( $DEBUG_sys ) ; then
2365	echo "IGCM_sys_ncdiff :" $@
2366	fi
2367
2368	typeset NB_ESSAI DELAI status i
2369	# number of tentative
2370	NB_ESSAI=3
2371	# time delay between tentative
2372	DELAI=2
2373
2374	i=0
2375	while [ $i -lt $NB_ESSAI ] ; do
2376	ncdiff --nocoords $@ > /tmp/out_command.$$ 2>&1
2377	status=$?
2378	if [ ${status} -gt 0 ] ; then
2379	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2380	cat /tmp/out_command.$$
2381	\rm /tmp/out_command.$$
2382	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2383	sleep $DELAI
2384	else
2385	\rm /tmp/out_command.$$
2386	break
2387	fi
2388	(( i = i + 1 ))
2389	done
2390
2391	if [ ${status} -gt 0 ] ; then
2392	echo "IGCM_sys_ncdiff : ncdiff error"
2393	IGCM_debug_Exit "ncdiff"
2394	fi
2395
2396	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2397	}
2398
2399	#D-#==================================================
2400	#D-function IGCM_sys_ncea
2401	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2402	#D-* Examples:
2403	#D-
2404	function IGCM_sys_ncea {
2405	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2406	if ( $DEBUG_sys ) ; then
2407	echo "IGCM_sys_ncea :" $@
2408	fi
2409
2410	typeset NB_ESSAI DELAI status i
2411	# number of tentative
2412	NB_ESSAI=3
2413	# time delay between tentative
2414	DELAI=2
2415
2416	i=0
2417	while [ $i -lt $NB_ESSAI ] ; do
2418	ncea --nocoords $@ > /tmp/out_command.$$ 2>&1
2419	status=$?
2420	if [ ${status} -gt 0 ] ; then
2421	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2422	cat /tmp/out_command.$$
2423	\rm /tmp/out_command.$$
2424	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2425	sleep $DELAI
2426	else
2427	\rm /tmp/out_command.$$
2428	break
2429	fi
2430	(( i = i + 1 ))
2431	done
2432
2433	if [ ${status} -gt 0 ] ; then
2434	echo "IGCM_sys_ncea : ncea error"
2435	IGCM_debug_Exit "ncea"
2436	fi
2437
2438	IGCM_debug_PopStack "IGCM_sys_ncea"
2439	}
2440
2441	#D-#==================================================
2442	#D-function IGCM_sys_ncecat
2443	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2444	#D-* Examples:
2445	#D-
2446	function IGCM_sys_ncecat {
2447	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2448	if ( $DEBUG_sys ) ; then
2449	echo "IGCM_sys_ncecat :" $@
2450	fi
2451
2452	typeset NB_ESSAI DELAI status i
2453	# number of tentative
2454	NB_ESSAI=3
2455	# time delay between tentative
2456	DELAI=2
2457
2458	i=0
2459	while [ $i -lt $NB_ESSAI ] ; do
2460	ncecat --nocoords $@ > /tmp/out_command.$$ 2>&1
2461	status=$?
2462	if [ ${status} -gt 0 ] ; then
2463	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2464	cat /tmp/out_command.$$
2465	\rm /tmp/out_command.$$
2466	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2467	sleep $DELAI
2468	else
2469	\rm /tmp/out_command.$$
2470	break
2471	fi
2472	(( i = i + 1 ))
2473	done
2474
2475	if [ ${status} -gt 0 ] ; then
2476	echo "IGCM_sys_ncecat : ncecat error"
2477	IGCM_debug_Exit "ncecat"
2478	fi
2479
2480	IGCM_debug_PopStack "IGCM_sys_ncecat"
2481	}
2482
2483	#D-#==================================================
2484	#D-function IGCM_sys_ncflint
2485	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2486	#D-* Examples:
2487	#D-
2488	function IGCM_sys_ncflint {
2489	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2490	if ( $DEBUG_sys ) ; then
2491	echo "IGCM_sys_ncflint :" $@
2492	fi
2493
2494	typeset NB_ESSAI DELAI status i
2495	# number of tentative
2496	NB_ESSAI=3
2497	# time delay between tentative
2498	DELAI=2
2499
2500	i=0
2501	while [ $i -lt $NB_ESSAI ] ; do
2502	ncflint --nocoords $@ > /tmp/out_command.$$ 2>&1
2503	status=$?
2504	if [ ${status} -gt 0 ] ; then
2505	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2506	cat /tmp/out_command.$$
2507	\rm /tmp/out_command.$$
2508	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2509	sleep $DELAI
2510	else
2511	\rm /tmp/out_command.$$
2512	break
2513	fi
2514	(( i = i + 1 ))
2515	done
2516
2517	if [ ${status} -gt 0 ] ; then
2518	echo "IGCM_sys_ncflint : ncflint error"
2519	IGCM_debug_Exit "ncflint"
2520	fi
2521
2522	IGCM_debug_PopStack "IGCM_sys_ncflint"
2523	}
2524
2525	#D-#==================================================
2526	#D-function IGCM_sys_ncks
2527	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2528	#D-* Examples:
2529	#D-
2530	function IGCM_sys_ncks {
2531	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2532	if ( $DEBUG_sys ) ; then
2533	echo "IGCM_sys_ncks :" $@
2534	fi
2535
2536	typeset NB_ESSAI DELAI status i
2537	# number of tentative
2538	NB_ESSAI=3
2539	# time delay between tentative
2540	DELAI=2
2541
2542	i=0
2543	while [ $i -lt $NB_ESSAI ] ; do
2544	ncks --nocoords $@ > /tmp/out_command.$$ 2>&1
2545	status=$?
2546	if [ ${status} -gt 0 ] ; then
2547	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2548	cat /tmp/out_command.$$
2549	\rm /tmp/out_command.$$
2550	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2551	sleep $DELAI
2552	else
2553	\rm /tmp/out_command.$$
2554	break
2555	fi
2556	(( i = i + 1 ))
2557	done
2558
2559	if [ ${status} -gt 0 ] ; then
2560	echo "IGCM_sys_ncks : ncks error"
2561	IGCM_debug_Exit "ncks"
2562	fi
2563
2564	IGCM_debug_PopStack "IGCM_sys_ncks"
2565	}
2566
2567	#D-#==================================================
2568	#D-function IGCM_sys_ncpdq
2569	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2570	#D-* Examples:
2571	#D-
2572	function IGCM_sys_ncpdq {
2573	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2574	if ( $DEBUG_sys ) ; then
2575	echo "IGCM_sys_ncpdq :" $@
2576	fi
2577
2578	typeset NB_ESSAI DELAI status i
2579	# number of tentative
2580	NB_ESSAI=3
2581	# time delay between tentative
2582	DELAI=2
2583
2584	i=0
2585	while [ $i -lt $NB_ESSAI ] ; do
2586	ncpdq --nocoords $@ > /tmp/out_command.$$ 2>&1
2587	status=$?
2588	if [ ${status} -gt 0 ] ; then
2589	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2590	cat /tmp/out_command.$$
2591	\rm /tmp/out_command.$$
2592	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2593	sleep $DELAI
2594	else
2595	\rm /tmp/out_command.$$
2596	break
2597	fi
2598	(( i = i + 1 ))
2599	done
2600
2601	if [ ${status} -gt 0 ] ; then
2602	echo "IGCM_sys_ncpdq : ncpdq error"
2603	IGCM_debug_Exit "ncpdq"
2604	fi
2605
2606	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2607	}
2608
2609	#D-#==================================================
2610	#D-function IGCM_sys_ncra
2611	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2612	#D-* Examples:
2613	#D-
2614	function IGCM_sys_ncra {
2615	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2616	if ( $DEBUG_sys ) ; then
2617	echo "IGCM_sys_ncra :" $@
2618	fi
2619
2620	typeset NB_ESSAI DELAI status i
2621	# number of tentative
2622	NB_ESSAI=3
2623	# time delay between tentative
2624	DELAI=2
2625
2626	i=0
2627	while [ $i -lt $NB_ESSAI ] ; do
2628	ncra --nocoords $@ > /tmp/out_command.$$ 2>&1
2629	status=$?
2630	if [ ${status} -gt 0 ] ; then
2631	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
2632	cat /tmp/out_command.$$
2633	\rm /tmp/out_command.$$
2634	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2635	sleep $DELAI
2636	else
2637	\rm /tmp/out_command.$$
2638	break
2639	fi
2640	(( i = i + 1 ))
2641	done
2642
2643	if [ ${status} -gt 0 ] ; then
2644	echo "IGCM_sys_ncra : ncra error"
2645	IGCM_debug_Exit "ncra"
2646	fi
2647
2648	IGCM_debug_PopStack "IGCM_sys_ncra"
2649	}
2650
2651	#D-#==================================================
2652	#D-function IGCM_sys_ncrcat
2653	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
2654	#D-* Examples:
2655	#D-
2656	function IGCM_sys_ncrcat {
2657	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
2658	if ( $DEBUG_sys ) ; then
2659	echo "IGCM_sys_ncrcat :" $@
2660	fi
2661
2662	typeset NB_ESSAI DELAI status i lastArg
2663	# number of tentative
2664	NB_ESSAI=3
2665	# time delay between tentative
2666	DELAI=2
2667
2668	i=0
2669	while [ $i -lt $NB_ESSAI ] ; do
2670	ncrcat --nocoords $@ > /tmp/out_command.$$ 2>&1
2671	status=$?
2672	if [ ${status} -gt 0 ] ; then
2673	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
2674	cat /tmp/out_command.$$
2675	\rm /tmp/out_command.$$
2676	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2677	sleep $DELAI
2678	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
2679	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
2680	cat /tmp/out_command.$$
2681	# remove files having corrupted time axis
2682	eval lastArg=\${$#}
2683	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
2684	\rm ${lastArg}
2685	\rm /tmp/out_command.$$
2686	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2687	sleep $DELAI
2688	else
2689	\rm /tmp/out_command.$$
2690	break
2691	fi
2692	(( i = i + 1 ))
2693	done
2694
2695	if [ ${status} -gt 0 ] ; then
2696	echo "IGCM_sys_ncrcat : ncrcat error"
2697	#IGCM_debug_Exit "ncrcat"
2698	fi
2699
2700	IGCM_debug_PopStack "IGCM_sys_ncrcat"
2701	}
2702
2703	#D-#==================================================
2704	#D-function IGCM_sys_ncrename
2705	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
2706	#D-* Examples:
2707	#D-
2708	function IGCM_sys_ncrename {
2709	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
2710	if ( $DEBUG_sys ) ; then
2711	echo "IGCM_sys_ncrename :" $@
2712	fi
2713
2714	typeset NB_ESSAI DELAI status i
2715	# number of tentative
2716	NB_ESSAI=3
2717	# time delay between tentative
2718	DELAI=2
2719
2720	i=0
2721	while [ $i -lt $NB_ESSAI ] ; do
2722	ncrename $@ > /tmp/out_command.$$ 2>&1
2723	status=$?
2724	if [ ${status} -gt 0 ] ; then
2725	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
2726	cat /tmp/out_command.$$
2727	\rm /tmp/out_command.$$
2728	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2729	sleep $DELAI
2730	else
2731	\rm /tmp/out_command.$$
2732	break
2733	fi
2734	(( i = i + 1 ))
2735	done
2736
2737	if [ ${status} -gt 0 ] ; then
2738	echo "IGCM_sys_ncrename : ncrename error"
2739	IGCM_debug_Exit "ncrename"
2740	fi
2741
2742	IGCM_debug_PopStack "IGCM_sys_ncrename"
2743	}
2744
2745	#D-#==================================================
2746	#D-function IGCM_sys_ncwa
2747	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
2748	#D-* Examples:
2749	#D-
2750	function IGCM_sys_ncwa {
2751	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
2752	if ( $DEBUG_sys ) ; then
2753	echo "IGCM_sys_ncwa :" $@
2754	fi
2755
2756	typeset NB_ESSAI DELAI status i
2757	# number of tentative
2758	NB_ESSAI=3
2759	# time delay between tentative
2760	DELAI=2
2761
2762	i=0
2763	while [ $i -lt $NB_ESSAI ] ; do
2764	ncwa -C $@ > /tmp/out_command.$$ 2>&1
2765	status=$?
2766	if [ ${status} -gt 0 ] ; then
2767	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
2768	cat /tmp/out_command.$$
2769	\rm /tmp/out_command.$$
2770	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2771	sleep $DELAI
2772	else
2773	\rm /tmp/out_command.$$
2774	break
2775	fi
2776	(( i = i + 1 ))
2777	done
2778
2779	if [ ${status} -gt 0 ] ; then
2780	echo "IGCM_sys_ncwa : ncwa error"
2781	IGCM_debug_Exit "ncwa"
2782	fi
2783
2784	IGCM_debug_PopStack "IGCM_sys_ncwa"
2785	}
2786
2787	##############################################################
2788	# CDO OPERATOR
2789
2790	#D-#==================================================
2791	#D-function IGCM_sys_cdo
2792	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
2793	#D-* Examples:
2794	#D-
2795	function IGCM_sys_cdo {
2796	IGCM_debug_PushStack "IGCM_sys_cdo" $@
2797	if ( $DEBUG_sys ) ; then
2798	echo "IGCM_sys_cdo :" $@
2799	fi
2800
2801	typeset status
2802
2803	\cdo $@ > /tmp/out_command.$$ 2>&1
2804	status=$?
2805	if [ ${status} -gt 0 ] ; then
2806	echo "IGCM_sys_cdo : error code ${status}"
2807	cat /tmp/out_command.$$
2808	\rm /tmp/out_command.$$
2809	IGCM_debug_PopStack "IGCM_sys_cdo"
2810	return 1
2811	else
2812	IGCM_debug_PopStack "IGCM_sys_cdo"
2813	return 0
2814	fi
2815
2816	IGCM_debug_PopStack "IGCM_sys_cdo"
2817	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: