Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_obelix.ksh @ 1180

Last change on this file since 1180 was 1180, checked in by sdipsl, 9 years ago
house keeping in libIGCM_sys (third pass). Will ease #248 and #251
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 78.5 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Christian Laguerre
5	# Contact: Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for obelix
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=${MASTER:=$( hostname )}
71
72	#D-
73	#D-#==================================================
74	#D-Program used in libIGCM
75	#D-#==================================================
76
77	# Submit command
78	typeset SUBMIT=${SUBMIT:=qsub}
79	# rsync with path
80	typeset -r RSYNC=/usr/bin/rsync
81	# RSYNC_opt args to rsync
82	typeset -r RSYNC_opt="-va"
83	# ie storage filesystem
84	typeset -r STOREHOST=${MASTER}
85
86	#====================================================
87	# Set environment tools (ferret, nco, cdo, rebuild, ...)
88	#====================================================
89	. /home/users/igcmg/.atlas_env_asterix_ksh
90	export PATH=${PATH}:/home/users/igcmg/rebuild/bin/
91
92	#====================================================
93	# Set lf95 environment
94	#====================================================
95	. /usr/local/install/lf6481/bash_laheyfort_setup
96
97	#====================================================
98	# Host specific DIRECTORIES
99	#====================================================
100
101	#====================================================
102	#- MirrorlibIGCM for frontend
103	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
104
105	#====================================================
106	#- libIGCM_POST for frontend
107	typeset -r libIGCM_POST=${libIGCM}
108
109	#====================================================
110	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
111	typeset -r R_EXE="${MODIPSL}/bin"
112
113	#====================================================
114	#- SUBMIT_DIR : submission dir
115	if [ X${PBS_O_WORKDIR} != X ] ; then
116	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PBS_O_WORKDIR}}
117	else
118	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PWD}}
119	fi
120
121	#====================================================
122	#- IN
123	typeset -r R_IN=${R_IN:=/home/orchideeshare/igcmg/IGCM}
124	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/home/orchideeshare/igcmg/IGCM}
125
126	#====================================================
127	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
128	if [ X${PBS_JOBID} != X ] ; then
129	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/scratch/$PBS_O_LOGNAME.$PBS_JOBID}
130	else
131	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/tmp/tmp$$}
132	fi
133
134	#====================================================
135	#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
136	OUTCOMMAND_PATH=/tmp
137
138	#====================================================
139	#- HOST_MPIRUN_COMMAND
140	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="time mpirun"}
141
142	#====================================================
143	#- Max number of arguments passed to nco operator or demigration command
144	UNIX_MAX_LIMIT=360
145
146	#====================================================
147	#- set PackDefault to false on obelix
148	PackDefault=false
149
150	#====================================================
151	#- Number of core per node (max number of OpenMP task)
152	NUM_COREPERNODE=1
153
154	#====================================================
155	#- Default number of MPI task for IPSL coupled model
156	#- required for backward compatibility
157	#-
158	DEFAULT_NUM_PROC_OCE=1
159	DEFAULT_NUM_PROC_CPL=1
160	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
161	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
162
163	#D-#==================================================
164	#D-function IGCM_sys_defineArchives
165	#D-* Purpose:
166	#D-* Define ARCHIVE : Dedicated to large files
167	#D-* Define STORAGE : Dedicated to small/medium files
168	#D-* Define R_OUT : Output tree located on ARCHIVE
169	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
170	#D-* Define R_BUF : Output tree located on STORAGE hosting files waiting for rebuild or pack processes
171	#D-* if SpaceName=TEST nothing special will hapen
172	#D-* Examples:
173	#D-
174	function IGCM_sys_defineArchives {
175	IGCM_debug_PushStack "IGCM_sys_defineArchives"
176
177	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
178	#====================================================
179	#- ARCHIVE (dedicated to large files)
180	ARCHIVE=${config_UserChoices_ARCHIVE}
181	else
182	#====================================================
183	#- ARCHIVE (dedicated to large files)
184	ARCHIVE=${ARCHIVE:=/home/scratch01/${LOGIN}}
185	fi
186
187	if [ ! X${config_UserChoices_STORAGE} = X ]; then
188	#====================================================
189	#- STORAGE (dedicated to small/medium files)
190	STORAGE=${config_UserChoices_STORAGE}
191	else
192	#====================================================
193	#- STORAGE (dedicated to small/medium files)
194	STORAGE=${ARCHIVE}
195	fi
196
197	# ON OBELIX NO SPECIAL CASE WHEN X${config_UserChoices_SpaceName} = XTEST
198
199	#====================================================
200	#- R_OUT
201	R_OUT=${ARCHIVE}/IGCM_OUT
202
203	#====================================================
204	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
205	R_FIG=${STORAGE}/IGCM_OUT
206
207	#====================================================
208	#- R_BUF (ONLY FOR double copy an scratch)
209	R_BUF=${STORAGE}/IGCM_OUT
210
211	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
212	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
213	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
214
215	IGCM_debug_PopStack "IGCM_sys_defineArchives"
216	}
217
218	#D-#==================================================
219	#D-function IGCM_sys_RshMaster
220	#D-* Purpose: Connection to frontend machine.
221	#D-* Examples:
222	#D-
223	function IGCM_sys_RshMaster {
224	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
225	OUTCOMMAND_PATH=${OUTCOMMAND_PATH} /bin/ksh <<-EOF
226	export libIGCM=${libIGCM}
227	export DEBUG_debug=${DEBUG_debug}
228	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
229	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
230	${@}
231	EOF
232	if [ $? -gt 0 ] ; then
233	echo "IGCM_sys_RshMaster : erreur."
234	IGCM_debug_Exit "IGCM_sys_RshMaster"
235	fi
236	IGCM_debug_PopStack "IGCM_sys_RshMaster"
237	}
238
239	#D-#==================================================
240	#D-function IGCM_sys_RshArchive
241	#D-* Purpose: Archive rsh command
242	#D-* Examples:
243	#D-
244	function IGCM_sys_RshArchive {
245	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
246	/bin/ksh <<-EOF
247	${@}
248	EOF
249	status=$?
250	if [ ${status} -gt 0 ] ; then
251	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
252	IGCM_debug_Exit "IGCM_sys_RshArchive"
253	fi
254	IGCM_debug_PopStack "IGCM_sys_RshArchive"
255	}
256
257	#D-#==================================================
258	#D-function IGCM_sys_RshArchive_NoError
259	#D-* Purpose: Archive rsh command, without error
260	#D-* used only in monitoring.job
261	#D-* Examples:
262	#D-
263	function IGCM_sys_RshArchive_NoError {
264	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
265	/bin/ksh <<-EOF
266	${@} 2> \dev\null
267	EOF
268	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
269	}
270
271	#D-#==================================================
272	#D-function IGCM_sys_RshPost
273	#D-* Purpose: Post-process rsh command
274	#D-* Examples:
275	#D-
276	function IGCM_sys_RshPost {
277	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
278	if ( $DEBUG_sys ) ; then
279	echo "IGCM_sys_RshPost :" $@
280	fi
281	# keep standard input (stdin) for the loop onto temporary file
282	cat >${OUTCOMMAND_PATH}/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
283
284	OUTCOMMAND_PATH=${OUTCOMMAND_PATH} /bin/ksh <${OUTCOMMAND_PATH}/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
285	if [ $? -gt 0 ] ; then
286	echo "IGCM_sys_RshPost : erreur."
287	IGCM_debug_Exit "IGCM_sys_RshPost"
288	fi
289	# delete temporary file
290	\rm ${OUTCOMMAND_PATH}/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
291	IGCM_debug_PopStack "IGCM_sys_RshPost"
292	}
293
294	#D-#==================================================
295	#D-function IGCM_sys_SendMail
296	#D-* Purpose: Send mail when simulation is over
297	#D-* Examples:
298	#D-
299	function IGCM_sys_SendMail {
300	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
301	if ( $DEBUG_sys ) ; then
302	echo "IGCM_sys_SendMail :" $@
303	fi
304
305	if [ X${1} = XAccounting ] ; then
306	status=Accounting
307	mailText=jobAccounting.mail
308	elif ( ${ExitFlag} ) ; then
309	status=failed
310	mailText=jobEnd.mail
311	else
312	status=completed
313	mailText=jobEnd.mail
314	fi
315
316	# Update selected mail template
317	while read -r line; do
318	eval echo $line >> mail.txt ;
319	done < ${libIGCM}/libIGCM_sys/${mailText}
320
321	if [ ! -z ${config_UserChoices_MailName} ] ; then
322	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
323	elif [ -f ~/.forward ] ; then
324	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
325	else
326	mail -s "${config_UserChoices_JobName} ${status}" ${USER} < mail.txt
327	fi
328
329	sleep 10
330	rm -f mail.txt
331
332	IGCM_debug_PopStack "IGCM_sys_SendMail"
333	}
334
335	#D-#==================================================
336	#D-function IGCM_sys_Mkdir
337	#D-* Purpose: Master locale mkdir command
338	#D-* Examples:
339	#D-
340	function IGCM_sys_Mkdir {
341	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
342	if ( $DEBUG_sys ) ; then
343	echo "IGCM_sys_Mkdir :" $@
344	fi
345	if [ ! -d ${1} ]; then
346	\mkdir -p $1
347	if [ $? -gt 0 ] ; then
348	echo "IGCM_sys_Mkdir : erreur."
349	IGCM_debug_Exit "IGCM_sys_Mkdir"
350	fi
351	fi
352	# vérification :
353	if [ ! -d ${1} ] ; then
354	echo "IGCM_sys_Mkdir : erreur."
355	IGCM_debug_Exit "IGCM_sys_Mkdir"
356	fi
357	IGCM_debug_PopStack "IGCM_sys_Mkdir"
358	}
359
360	#D-#==================================================
361	#D-function IGCM_sys_MkdirArchive
362	#D-* Purpose: Mkdir on Archive
363	#D-* Examples:
364	#D-
365	function IGCM_sys_MkdirArchive {
366	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
367	if ( $DEBUG_sys ) ; then
368	echo "IGCM_sys_MkdirArchive :" $@
369	fi
370	#- creation de repertoire sur le serveur fichier
371	if [ ! -d ${1} ]; then
372	\mkdir -p $1
373	status=$?
374
375	if [ ${status} -gt 0 ] ; then
376	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
377	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
378	fi
379	fi
380	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
381	}
382
383	#D-#==================================================
384	#D-function IGCM_sys_MkdirWork
385	#D-* Purpose: Mkdir on Work
386	#D-* Examples:
387	#D-
388	function IGCM_sys_MkdirWork {
389	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
390	if ( $DEBUG_sys ) ; then
391	echo "IGCM_sys_MkdirWork :" $@
392	fi
393	#- creation de repertoire sur le serveur fichier
394	if [ ! -d ${1} ]; then
395	\mkdir -p $1
396	if [ $? -gt 0 ] ; then
397	echo "IGCM_sys_MkdirWork : erreur."
398	IGCM_debug_Exit "IGCM_sys_MkdirWork"
399	fi
400	fi
401	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
402	}
403
404	#D-#==================================================
405	#D-function IGCM_sys_Cd
406	#D-* Purpose: master cd command
407	#D-* Examples:
408	#D-
409	function IGCM_sys_Cd {
410	IGCM_debug_PushStack "IGCM_sys_Cd" $@
411	if ( $DEBUG_sys ) ; then
412	echo "IGCM_sys_Cd :" $@
413	fi
414	\cd $1
415	if [ $? -gt 0 ] ; then
416	echo "IGCM_sys_Cd : erreur."
417	IGCM_debug_Exit "IGCM_sys_Cd"
418	fi
419	IGCM_debug_PopStack "IGCM_sys_Cd"
420	}
421
422	#D-#==================================================
423	#D-function IGCM_sys_Chmod
424	#D-* Purpose: Chmod
425	#D-* Examples:
426	#D-
427	function IGCM_sys_Chmod {
428	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
429	if ( $DEBUG_sys ) ; then
430	echo "IGCM_sys_Chmod :" $@
431	fi
432	\chmod $@
433	if [ $? -gt 0 ] ; then
434	echo "IGCM_sys_Chmod : erreur."
435	IGCM_debug_Exit "IGCM_sys_Chmod"
436	fi
437	IGCM_debug_PopStack "IGCM_sys_Chmod"
438	}
439
440	#D-#==================================================
441	#D-function IGCM_sys_FileSize
442	#D-* Purpose: Filesize
443	#D-* Examples:
444	#D-
445	function IGCM_sys_FileSize {
446	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
447
448	typeset sizeF
449	set +A sizeF -- $( ls -la ${1} )
450	if [ $? -gt 0 ] ; then
451	IGCM_debug_Exit "IGCM_sys_FileSize"
452	fi
453	eval ${2}=${sizeF[4]}
454
455	IGCM_debug_PopStack "IGCM_sys_FileSize"
456	}
457
458	#D-#==================================================
459	#D-function IGCM_sys_TestDir
460	#D-* Purpose: Test Directory that must exists
461	#D-* Examples:
462	#D-
463	function IGCM_sys_TestDir {
464	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
465	if ( $DEBUG_sys ) ; then
466	echo "IGCM_sys_TestDir :" $@
467	fi
468	typeset ExistFlag
469	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
470	IGCM_debug_PopStack "IGCM_sys_TestDir"
471
472	return ${ExistFlag}
473	}
474
475	#D-#==================================================
476	#D-function IGCM_sys_TestDirArchive
477	#D-* Purpose: Test Directory that must exists on Archive
478	#D-* Examples:
479	#D-
480	function IGCM_sys_TestDirArchive {
481	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
482	if ( $DEBUG_sys ) ; then
483	echo "IGCM_sys_TestDirArchive :" $@
484	fi
485	typeset ExistFlag
486	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
487	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
488	return ${ExistFlag}
489	}
490
491	#D-#==================================================
492	#D-function IGCM_sys_IsFileArchived
493	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
494	#D-* Examples:
495	#D-
496	function IGCM_sys_IsFileArchived {
497	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
498	if ( $DEBUG_sys ) ; then
499	echo "IGCM_sys_IsFileArchived :" $@
500	fi
501	typeset IsArchivedFlag
502	# Never archived for this system
503	IsArchivedFlag=1
504	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
505
506	return ${IsArchivedFlag}
507	}
508
509	#D-#==================================================
510	#D-function IGCM_sys_TestFileArchive
511	#D-* Purpose: Test file that must NOT EXISTS on Archive
512	#D-* Examples:
513	#D-
514	function IGCM_sys_TestFileArchive {
515	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
516	typeset ExistFlag
517	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
518	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
519
520	return ${ExistFlag}
521	}
522
523	#D-#==================================================
524	#D-function IGCM_sys_TestFileBuffer
525	#D-* Purpose: Test file that must NOT EXISTS on Buffer
526	#D-* Examples:
527	#D-
528	function IGCM_sys_TestFileBuffer {
529	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
530	typeset ExistFlag
531	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
532	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
533
534	return ${ExistFlag}
535	}
536
537	#D-#==================================================
538	#D-function IGCM_sys_CountFileArchive
539	#D-* Purpose: Count files on Archive filesystem
540	#D-* Examples:
541	#D-
542	function IGCM_sys_CountFileArchive {
543	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
544	ls ${@} 2>/dev/null \| wc -l
545	if [ $? -gt 0 ] ; then
546	echo "IGCM_sys_CountFileArchive : erreur."
547	fi
548	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
549	}
550
551	#D-#==================================================
552	#D-function IGCM_sys_CountFileBuffer
553	#D-* Purpose: Count files on Scratch filesystem
554	#D-* Examples:
555	#D-
556	function IGCM_sys_CountFileBuffer {
557	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
558	ls ${@} 2>/dev/null \| wc -l
559	if [ $? -gt 0 ] ; then
560	echo "IGCM_sys_CountFileBuffer : erreur."
561	fi
562	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
563	}
564
565	#D-#==================================================
566	#D-function IGCM_sys_Tree
567	#D-* Purpose: Tree directories with files on ${ARCHIVE}
568	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
569	#D-
570	function IGCM_sys_Tree {
571	IGCM_debug_PushStack "IGCM_sys_Tree" $@
572	if ( $DEBUG_sys ) ; then
573	echo "IGCM_sys_Tree :" $@
574	fi
575
576	\tree -f $@
577
578	IGCM_debug_PopStack "IGCM_sys_Tree"
579	}
580
581	#D-#==================================================
582	#D-function IGCM_sys_Tar
583	#D-* Purpose: master tar command
584	#D-* Examples:
585	#D-
586	function IGCM_sys_Tar {
587	IGCM_debug_PushStack "IGCM_sys_Tar" $@
588	if ( $DEBUG_sys ) ; then
589	echo "IGCM_sys_Tar :" $@
590	fi
591	\tar cf $@
592	if [ $? -gt 0 ] ; then
593	echo "IGCM_sys_Tar : erreur."
594	IGCM_debug_Exit "IGCM_sys_Tar"
595	fi
596	IGCM_debug_PopStack "IGCM_sys_Tar"
597	}
598
599	#D-#==================================================
600	#D-function IGCM_sys_UnTar
601	#D-* Purpose: master un-tar command
602	#D-* Examples:
603	#D-
604	function IGCM_sys_UnTar {
605	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
606	if ( $DEBUG_sys ) ; then
607	echo "IGCM_sys_UnTar :" $@
608	fi
609	\tar xvf $1
610	if [ $? -gt 0 ] ; then
611	echo "IGCM_sys_UnTar : erreur."
612	IGCM_debug_Exit "IGCM_sys_UnTar"
613	fi
614	IGCM_debug_PopStack "IGCM_sys_UnTar"
615	}
616
617	#D-#==================================================
618	#D-function IGCM_sys_Qsub
619	#D-* Purpose: Qsub new job
620	#D-* Examples:
621	#D-
622	function IGCM_sys_Qsub {
623	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
624	if ( $DEBUG_sys ) ; then
625	echo "IGCM_sys_Qsub :" $@
626	fi
627	typeset options status
628	options="-o ${SUBMIT_DIR}/${Script_Output}"
629	/usr/local/bin/qsub ${options} $1 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
630	status=$?
631
632	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
633	if [ ${status} -gt 0 ] ; then
634	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
635	IGCM_debug_Exit "IGCM_sys_Qsub"
636	else
637	JobID=$( gawk {'print $1'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
638	fi
639	IGCM_debug_PopStack "IGCM_sys_Qsub"
640	}
641
642	#D-#==================================================
643	#D-function IGCM_sys_QsubPost
644	#D-* Purpose: Qsub new job on scalaire
645	#D-* Examples:
646	#D-
647	function IGCM_sys_QsubPost {
648	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
649	if ( $DEBUG_sys ) ; then
650	echo "IGCM_sys_QsubPost :" $@
651	fi
652	typeset options status
653	options="-o ${POST_DIR}/${Script_Post_Output}.out -v ${listVarEnv}"
654	/usr/local/bin/qsub ${options} ${libIGCM_POST}/$1.job > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
655	status=$?
656
657	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
658	if [ ${status} -gt 0 ] ; then
659	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
660	IGCM_debug_Exit "IGCM_sys_QsubPost"
661	else
662	JobID=$( gawk {'print $1'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
663	fi
664	IGCM_debug_PopStack "IGCM_sys_QsubPost"
665	}
666
667	#D-*************************
668	#D- File transfer functions
669	#D-*************************
670	#D-
671
672	#D-#==================================================
673	#D-function IGCM_sys_Rsync_out
674	#D-* Purpose: treat return val of rsync
675	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
676	#D- Error values and explanations can depend on your system version.
677	function IGCM_sys_Rsync_out {
678	status=$1
679	if [ ! $status ] ; then
680	echo "rsync error !"
681	fi
682
683	if [ $MYLANG = "fr" ]; then
684	case $status in
685	0) return ;;
686	1) echo "Erreur de rsync ; RERR_SYNTAX : "
687	echo "Erreur de syntaxe ou d'utilisation."
688	return;;
689	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
690	echo "Incompatibilité de protocole."
691	return;;
692	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
693	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
694	echo "répertoires"
695	return;;
696	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
697	echo "Action demandée non supportée : une tentative de manipulation de"
698	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
699	echo "été faite ; ou une option qui est supportée par le client mais"
700	echo "pas par le serveur a été spécifiée."
701	return;;
702	10) echo "Erreur de rsync ; RERR_SOCKETIO"
703	echo "Erreur dans le socket d'entrée sortie"
704	return;;
705	11) echo "Erreur de rsync ; RERR_FILEIO"
706	echo "Erreur d'entrée sortie fichier"
707	return;;
708	12) echo "Erreur de rsync ; RERR_STREAMIO"
709	echo "Erreur dans flux de donnée du protocole rsync"
710	return;;
711	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
712	echo "Erreur avec les diagnostics du programme"
713	return;;
714	14) echo "Erreur de rsync ; RERR_IPC"
715	echo "Erreur dans le code IPC"
716	return;;
717	20) echo "Erreur de rsync ; RERR_SIGNAL"
718	echo "SIGUSR1 ou SIGINT reçu"
719	return;;
720	21) echo "Erreur de rsync ; RERR_WAITCHILD"
721	echo "Une erreur retournée par waitpid()"
722	return;;
723	22) echo "Erreur de rsync ; RERR_MALLOC"
724	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
725	return;;
726	23) echo ""
727	echo "Erreur fichier inexistant"
728	return;;
729	30) echo "Erreur de rsync ; RERR_TIMEOUT"
730	echo "Temps d'attente écoulé dans l'envoi/réception de données"
731	return;;
732	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
733	return;;
734	esac
735	elif [ $MYLANG = "en" ] ; then
736	case $status in
737	0) return;;
738	1) echo "rsync error : Syntax or usage error "
739	return;;
740	2) echo "rsync error : Protocol incompatibility "
741	return;;
742	3) echo "rsync error : Errors selecting input/output files, dirs"
743	return;;
744	4) echo "rsync error : Requested action not supported: an attempt"
745	echo "was made to manipulate 64-bit files on a platform that cannot support"
746	echo "them; or an option was specified that is supported by the client and"
747	echo "not by the server."
748	return;;
749	5) echo "rsync error : Error starting client-server protocol"
750	return;;
751	10) echo "rsync error : Error in socket I/O "
752	return;;
753	11) echo "rsync error : Error in file I/O "
754	return;;
755	12) echo "rsync error : Error in rsync protocol data stream "
756	return;;
757	13) echo "rsync error : Errors with program diagnostics "
758	return;;
759	14) echo "rsync error : Error in IPC code "
760	return;;
761	20) echo "rsync error : Received SIGUSR1 or SIGINT "
762	return;;
763	21) echo "rsync error : Some error returned by waitpid() "
764	return;;
765	22) echo "rsync error : Error allocating core memory buffers "
766	return;;
767	23) echo "rsync error : Partial transfer due to error"
768	return;;
769	24) echo "rsync error : Partial transfer due to vanished source files"
770	return;;
771	30) echo "rsync error : Timeout in data send/receive "
772	return;;
773	*) echo "rsync error : return code of rsync unknown :" $status
774	return;;
775	esac
776	else
777	echo "unknown language $MYLANG."
778	return
779	fi
780	}
781
782	#D-#==================================================
783	#D-function IGCM_sys_Miror_libIGCM
784	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
785	#D-* Examples:
786	#D-
787	function IGCM_sys_Mirror_libIGCM {
788	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
789	if ( $DEBUG_sys ) ; then
790	echo "IGCM_sys_Mirror_libIGCM"
791	fi
792
793	typeset status
794
795	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
796
797	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
798	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
799	status=$?
800
801	if [ ${status} -gt 0 ] ; then
802	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
803	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
804	fi
805	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
806	}
807
808	#D-#==================================================
809	#D-function IGCM_sys_Cp
810	#D-* Purpose: generic cp
811	#D-* Examples:
812	#D-
813	function IGCM_sys_Cp {
814	IGCM_debug_PushStack "IGCM_sys_Cp" $@
815	if ( $DEBUG_sys ) ; then
816	echo "IGCM_sys_Cp :" $@
817	fi
818
819	typeset status
820
821	echo cp $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
822	\cp $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
823	status=$?
824
825	if [ ${status} -gt 0 ] ; then
826	echo "IGCM_sys_Cp : error code ${status}"
827	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
828	IGCM_debug_Exit "IGCM_sys_Cp"
829	else
830	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
831	fi
832	IGCM_debug_PopStack "IGCM_sys_Cp"
833	}
834
835	#D-#==================================================
836	#D-function IGCM_sys_Rm
837	#D-* Purpose: generic rm
838	#D-* Examples:
839	#D-
840	function IGCM_sys_Rm {
841	IGCM_debug_PushStack "IGCM_sys_Rm" $@
842	if ( $DEBUG_sys ) ; then
843	echo "IGCM_sys_Rm :" $@
844	fi
845
846	typeset status
847
848	echo rm $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
849	\rm $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
850	status=$?
851
852	if [ ${status} -gt 0 ] ; then
853	echo "IGCM_sys_Rm : error code ${status}"
854	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
855	IGCM_debug_Exit "IGCM_sys_Rm"
856	else
857	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
858	fi
859	IGCM_debug_PopStack "IGCM_sys_Rm"
860	}
861
862	#D-#==================================================
863	#D-function IGCM_sys_RmRunDir
864	#D-* Purpose: rm tmpdir (dummy function most of the time batch
865	#D- scheduler will do the job)
866	#D-* Examples:
867	#D-
868	function IGCM_sys_RmRunDir {
869	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
870	if ( $DEBUG_sys ) ; then
871	echo "IGCM_sys_RmRunDir :" $@
872	echo "Dummy call, let the scheduler do that."
873	fi
874	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
875	}
876
877	#D-#==================================================
878	#D-function IGCM_sys_Mv
879	#D-* Purpose: generic move
880	#D-* Examples:
881	#D-
882	function IGCM_sys_Mv {
883	IGCM_debug_PushStack "IGCM_sys_Mv" $@
884	if ( $DEBUG_sys ) ; then
885	echo "IGCM_sys_Mv :" $@
886	fi
887
888	if [ $DRYRUN = 0 ]; then
889
890	typeset status
891
892	echo mv $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
893	\mv $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
894	status=$?
895
896	if [ ${status} -gt 0 ] ; then
897	echo "IGCM_sys_Mv : error code ${status}"
898	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
899	IGCM_debug_Exit "IGCM_sys_Mv"
900	else
901	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
902	fi
903	fi
904	IGCM_debug_PopStack "IGCM_sys_Mv"
905	}
906
907	#D-#==================================================
908	#D-function IGCM_sys_Put_Dir
909	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
910	#D-* Examples:
911	#D-
912	function IGCM_sys_Put_Dir {
913	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
914	if ( $DEBUG_sys ) ; then
915	echo "IGCM_sys_Put_Dir :" $@
916	fi
917	if [ $DRYRUN = 0 ]; then
918	if [ ! -d ${1} ] ; then
919	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
920	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
921	return
922	fi
923
924	typeset status
925
926	# Only if we use rsync
927	#IGCM_sys_TestDirArchive $( dirname $2 )
928	#
929	#USUAL WAY
930	\cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
931	status=$?
932
933	if [ ${status} -gt 0 ] ; then
934	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
935	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
936	IGCM_debug_Exit "IGCM_sys_Put_Dir"
937	else
938	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
939	fi
940	fi
941	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
942	}
943
944	#D-#==================================================
945	#D-function IGCM_sys_Get_Dir
946	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
947	#D-* Examples:
948	#D-
949	function IGCM_sys_Get_Dir {
950	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
951	if ( $DEBUG_sys ) ; then
952	echo "IGCM_sys_Get_Dir :" $@
953	fi
954	if [ $DRYRUN = 0 ]; then
955	typeset NB_ESSAI DELAI status i
956	# number of tentative
957	NB_ESSAI=3
958	# time delay between tentative
959	DELAI=2
960
961	#
962	# USUAL WAY
963	i=0
964	while [ $i -lt $NB_ESSAI ] ; do
965	\cp -ur $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
966	status=$?
967	if [ ${status} -gt 0 ] ; then
968	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
969	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
970	sleep $DELAI
971	else
972	break
973	fi
974	(( i = i + 1 ))
975	done
976
977	if [ ${status} -gt 0 ] ; then
978	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
979	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
980	IGCM_debug_Exit "IGCM_sys_Get_Dir"
981	else
982	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
983	fi
984	fi
985	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
986	}
987
988	#D-#==================================================
989	#D-function IGCM_sys_Get_Master
990	#D-* Purpose: Copy a complete directory from MASTER filesystem
991	#D-* Examples:
992	#D-
993	function IGCM_sys_Get_Master {
994	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
995	if ( $DEBUG_sys ) ; then
996	echo "IGCM_sys_Get_Master :" $@
997	fi
998	if [ $DRYRUN = 0 ]; then
999	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1000	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1001	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1002	return
1003	fi
1004
1005	typeset NB_ESSAI DELAI status i
1006	# number of tentative
1007	NB_ESSAI=3
1008	# time delay between tentative
1009	DELAI=2
1010
1011	i=0
1012	while [ $i -lt $NB_ESSAI ] ; do
1013	\cp -urL $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1014	status=$?
1015	if [ ${status} -gt 0 ]; then
1016	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1017	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1018	sleep $DELAI
1019	else
1020	break
1021	fi
1022	(( i = i + 1 ))
1023	done
1024
1025	if [ ${status} -gt 0 ] ; then
1026	echo "IGCM_sys_Get_Master : error."
1027	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1028	IGCM_debug_Exit "IGCM_sys_Get_Master"
1029	else
1030	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1031	fi
1032	fi
1033	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1034	}
1035
1036	#====================================================
1037	#- Call IGCM_sys_Mirror_libIGCM now !
1038	if ( $MirrorlibIGCM ) ; then
1039	IGCM_sys_Mirror_libIGCM
1040	fi
1041
1042	#D-#==================================================
1043	#D-function IGCM_sys_Put_Rest
1044	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1045	#D- File and target directory must exist.
1046	#D-* Examples:
1047	#D-
1048	function IGCM_sys_Put_Rest {
1049	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1050	if ( $DEBUG_sys ) ; then
1051	echo "IGCM_sys_Put_Rest :" $@
1052	fi
1053	if [ $DRYRUN = 0 ]; then
1054	if [ ! -f ${1} ] ; then
1055	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1056	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1057	fi
1058
1059	typeset status
1060	#
1061	# USUAL WAY
1062	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1063	status=$?
1064
1065	# #RSYNC WITH NETWORK SSH CALL
1066	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1067	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1068
1069	# #RSYNC WITH NFS USE
1070	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1071	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1072
1073	# status=$?
1074	# IGCM_sys_Rsync_out $status
1075
1076	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1077	# (( status=status+$? ))
1078
1079	if [ ${status} -gt 0 ] ; then
1080	IGCM_debug_Print 2 "IGCM_sys_Put_Rest : cp failed error code ${status}"
1081	[ -f ${1} ] && ls -l ${1}
1082	[ -f ${2} ] && ls -l ${2}
1083	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1084	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1085	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1086	else
1087
1088	if [ X${JobType} = XRUN ] ; then
1089	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1090	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1091	fi
1092
1093	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1094	fi
1095	fi
1096	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1097	}
1098
1099	#D-#==================================================
1100	#D-function IGCM_sys_PutBuffer_Rest
1101	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1102	#D- File and target directory must exist.
1103	#D-* Examples:
1104	#D-
1105	function IGCM_sys_PutBuffer_Rest {
1106	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1107	if ( $DEBUG_sys ) ; then
1108	echo "IGCM_sys_PutBuffer_Rest :" $@
1109	fi
1110	if [ $DRYRUN = 0 ]; then
1111	if [ ! -f ${1} ] ; then
1112	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1113	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1114	fi
1115
1116	typeset status
1117	#
1118	# USUAL WAY
1119	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1120	status=$?
1121
1122	if [ ${status} -gt 0 ] ; then
1123	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1124	[ -f ${2} ] && ls -l ${2}
1125	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1126	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1127	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1128	else
1129
1130	if [ X${JobType} = XRUN ] ; then
1131	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1132	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1133	fi
1134
1135	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1136	fi
1137	fi
1138	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1139	}
1140
1141	#D-#==================================================
1142	#D-function IGCM_sys_PrepareTaredRestart
1143	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
1144	#D-* Examples:
1145	#D-
1146	function IGCM_sys_PrepareTaredRestart {
1147	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1148
1149	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1150	}
1151
1152	#D-#==================================================
1153	#D-function IGCM_sys_Put_Out
1154	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1155	#D-* Examples:
1156	#D-
1157	function IGCM_sys_Put_Out {
1158	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1159	if ( $DEBUG_sys ) ; then
1160	echo "IGCM_sys_Put_Out :" $@
1161	fi
1162
1163	typeset status
1164
1165	if [ $DRYRUN = 0 ]; then
1166	if [ ! -f ${1} ] ; then
1167	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1168	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1169	return 1
1170	fi
1171	#
1172	IGCM_sys_MkdirArchive $( dirname $2 )
1173	#
1174	if [ X${JobType} = XRUN ] ; then
1175	if [ X${3} = X ] ; then
1176	IGCM_sys_Chmod 444 ${1}
1177	fi
1178	fi
1179	#
1180
1181	echo ${RSYNC} ${RSYNC_opt} $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1182	${RSYNC} ${RSYNC_opt} $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1183	status=$?
1184	IGCM_sys_Rsync_out $status
1185
1186	${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1187	(( status=status+$? ))
1188
1189	if [ ${status} -gt 0 ] ; then
1190	IGCM_debug_Print 2 "IGCM_sys_Put_Out : rsync failed error code ${status}"
1191	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1192	IGCM_debug_Exit "IGCM_sys_Put_Out"
1193	else
1194	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1195	fi
1196	fi
1197	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1198	return 0
1199	}
1200
1201	#D-#==================================================
1202	#D-function IGCM_sys_PutBuffer_Out
1203	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1204	#D-* Examples:
1205	#D-
1206	function IGCM_sys_PutBuffer_Out {
1207	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1208	if ( $DEBUG_sys ) ; then
1209	echo "IGCM_sys_PutBuffer_Out :" $@
1210	fi
1211
1212	typeset NB_ESSAI DELAI status i exist skip
1213
1214	# number of tentative
1215	NB_ESSAI=3
1216	# time delay between tentative
1217	DELAI=2
1218
1219	if [ $DRYRUN = 0 ]; then
1220	if [ ! -f ${1} ] ; then
1221	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1222	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1223	return 1
1224	fi
1225	#
1226	IGCM_sys_Mkdir $( dirname $2 )
1227	#
1228
1229	exist=false
1230	skip=false
1231	if [ -f $2 ] ; then
1232	IGCM_debug_Print 1 "$2 already exist"
1233	exist=true
1234	if [ "X$( diff $1 $2 )" = X ] ; then
1235	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1236	status=0
1237	skip=true
1238	else
1239	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1240	skip=false
1241	fi
1242	fi
1243	#
1244	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1245	IGCM_sys_Chmod u+w $2
1246	fi
1247
1248	if [ X${skip} = Xfalse ] ; then
1249	i=0
1250	while [ $i -lt $NB_ESSAI ] ; do
1251	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1252	# USUAL WAY
1253	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1254	status=$?
1255	else
1256	# NOT SO USUAL WAY
1257	\mv $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1258	status=$?
1259	fi
1260	if [ ${status} -gt 0 ]; then
1261	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1262	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1263	[ -f ${2} ] && ls -l ${2}
1264	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1265	sleep $DELAI
1266	else
1267	break
1268	fi
1269	(( i = i + 1 ))
1270	done
1271	fi
1272
1273	if [ ${status} -gt 0 ] ; then
1274	echo "IGCM_sys_PutBuffer_Out : error."
1275	[ -f ${2} ] && ls -l ${2}
1276	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1277	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1278	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1279	else
1280
1281	if [ X${JobType} = XRUN ] ; then
1282	if [ X${3} = X ] ; then
1283	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1284	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1285	fi
1286	fi
1287
1288	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1289	fi
1290	fi
1291	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1292	return 0
1293	}
1294
1295	#D-#==================================================
1296	#D-function IGCM_sys_Get
1297	#D-* Purpose: Get a file from ${ARCHIVE}
1298	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1299	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1300	function IGCM_sys_Get {
1301	IGCM_debug_PushStack "IGCM_sys_Get" $@
1302
1303	typeset DEST status dm_liste
1304
1305	if ( $DEBUG_sys ) ; then
1306	echo "IGCM_sys_Get :" $@
1307	fi
1308	if [ $DRYRUN -le 2 ]; then
1309	if [ X${1} = X'/l' ] ; then
1310	eval set +A dm_liste \${${2}}
1311	else
1312	dm_liste=${1}
1313	fi
1314	eval DEST=\${${#}}
1315
1316	# test if the (first) file is present in the old computation :
1317	IGCM_sys_TestFileArchive ${dm_liste[0]}
1318	status=$?
1319	if [ ${status} -gt 0 ] ; then
1320	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
1321	IGCM_debug_Exit "IGCM_sys_Get"
1322	return
1323	fi
1324
1325	#USUAL WAY
1326	\cp ${dm_liste[*]} ${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1327	status=$?
1328
1329	if [ ${status} -gt 0 ] ; then
1330	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status}"
1331	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1332	IGCM_debug_Exit "IGCM_sys_Get"
1333	else
1334	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1335	fi
1336	fi
1337	IGCM_debug_PopStack "IGCM_sys_Get"
1338	}
1339
1340	#D-#==================================================
1341	#D-function IGCM_sys_GetBuffer
1342	#D-* Purpose: Get a file from ${SCRATCHDIR}
1343	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1344	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1345	function IGCM_sys_GetBuffer {
1346	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1347
1348	typeset DEST buf_liste target file_work
1349	typeset NB_ESSAI DELAI status i
1350
1351	if ( $DEBUG_sys ) ; then
1352	echo "IGCM_sys_GetBuffer :" $@
1353	fi
1354
1355	# number of tentative
1356	NB_ESSAI=3
1357	# time delay between tentative
1358	DELAI=2
1359
1360	if [ $DRYRUN -le 2 ]; then
1361	if [ X${1} = X'/l' ] ; then
1362	eval set +A buf_liste \${${2}}
1363	else
1364	eval set +A buf_liste ${1}
1365	fi
1366	eval DEST=\${${#}}
1367
1368	#USUAL WAY
1369	if [ X${1} = X'/l' ] ; then
1370	for target in ${buf_liste[*]} ; do
1371	local_file=$( basename ${target} )
1372	i=0
1373	while [ $i -lt $NB_ESSAI ] ; do
1374	\cp ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1375	status=$?
1376	if [ ${status} -gt 0 ]; then
1377	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1378	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1379	sleep $DELAI
1380	else
1381	break
1382	fi
1383	(( i = i + 1 ))
1384	done
1385	if [ ${status} -gt 0 ] ; then
1386	echo "IGCM_sys_Get : error"
1387	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1388	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1389	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1390	else
1391	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1392	fi
1393	done
1394	else
1395	i=0
1396	while [ $i -lt $NB_ESSAI ] ; do
1397	\cp ${buf_liste} ${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1398	status=$?
1399	if [ ${status} -gt 0 ]; then
1400	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1401	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1402	sleep $DELAI
1403	else
1404	break
1405	fi
1406	(( i = i + 1 ))
1407	done
1408	if [ ${status} -gt 0 ] ; then
1409	echo "IGCM_sys_Get : error"
1410	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1411	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1412	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1413	else
1414	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1415	fi
1416	fi
1417	fi
1418	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1419	}
1420
1421	#D-#==================================================
1422	#D-function IGCM_sys_GetDate_FichWork
1423	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1424	#D-* Examples:
1425	#D-
1426	function IGCM_sys_GetDate_FichWork {
1427	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1428	if ( $DEBUG_sys ) ; then
1429	echo "IGCM_sys_GetDate_FichWork :" $@
1430	fi
1431
1432	if [ $# -ge 3 ] ; then
1433	mode=$3
1434	TimeStyle=$4
1435	else
1436	mode="default"
1437	TimeStyle="%Y%m%d%H%M%S"
1438	fi
1439
1440	typeset dateF
1441	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1442
1443	case $mode in
1444	"default")
1445	eval ${2}=${dateF[5]}
1446	;;
1447	"SplitFields")
1448	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1449	;;
1450	esac
1451
1452
1453	# donne la date filesys d'un fichier sur la machine work
1454	IGCM_debug_PopStack "IGCM_sys_FichWork"
1455	}
1456
1457	#D-#==================================================
1458	#D-function IGCM_sys_GetDate_FichArchive
1459	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1460	#D-* Examples:
1461	#D-
1462	function IGCM_sys_GetDate_FichArchive {
1463	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1464	if ( $DEBUG_sys ) ; then
1465	echo "IGCM_sys_GetDate_FichArchive :" $@
1466	fi
1467	typeset dateF
1468	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1469	eval ${2}=${dateF[5]}
1470
1471	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1472	}
1473
1474	#D-#==================================================
1475	#D-function IGCM_sys_GetDate_Monitoring
1476	#D-* Purpose: get the last year for which the monitoring has been computed
1477	#D-* Examples:
1478	#D-
1479	function IGCM_sys_GetDate_Monitoring {
1480	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1481	if ( $DEBUG_sys ) ; then
1482	echo "IGCM_sys_GetDate_Monitoring :" $@
1483	fi
1484
1485	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1486
1487	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1488	}
1489
1490	#D-#==================================================
1491	#D-function IGCM_sys_Dods_Rm
1492	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1493	#D-* Examples:
1494	#D-
1495	function IGCM_sys_Dods_Rm {
1496	if ( $DEBUG_sys ) ; then
1497	echo "IGCM_sys_Dods_Rm :" $@
1498	fi
1499	return 0
1500	}
1501
1502	#D-#==================================================
1503	#D-function IGCM_sys_Dods_Cp
1504	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1505	#D-* Examples:
1506	#D-
1507	function IGCM_sys_Dods_Cp {
1508	if ( $DEBUG_sys ) ; then
1509	echo "IGCM_sys_Dods_Cp :" $@
1510	fi
1511	return 0
1512	}
1513
1514	#D-#==================================================
1515	#D-function IGCM_sys_Put_Dods
1516	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole. Dummy function here
1517	#D-* Examples:
1518	#D-
1519	function IGCM_sys_Put_Dods {
1520	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1521	if ( $DEBUG_sys ) ; then
1522	echo "IGCM_sys_Put_Dods :" $@
1523	fi
1524	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1525	}
1526
1527	##############################################################
1528	# REBUILD OPERATOR
1529
1530	#D-#==================================================
1531	#D-function IGCM_sys_sync
1532	#D-* Purpose: flush buffer on disk (dummy function on Ada)
1533	#D-* Examples:
1534	#D-
1535	function IGCM_sys_sync {
1536	IGCM_debug_PushStack "IGCM_sys_sync" $@
1537	if ( $DEBUG_sys ) ; then
1538	echo "IGCM_sys_sync :" $@
1539	echo "Dummy call, let the system do that."
1540	fi
1541	IGCM_debug_PopStack "IGCM_sys_sync"
1542	}
1543
1544	#D-#==================================================
1545	#D-function IGCM_sys_rebuild
1546	#D-* Purpose: rebuild parallel files
1547	#D-* Examples:
1548	#D-
1549	function IGCM_sys_rebuild {
1550	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1551	if ( $DEBUG_sys ) ; then
1552	echo "IGCM_sys_rebuild :" $@
1553	fi
1554
1555	typeset NB_ESSAI DELAI status i firstArg
1556	# number of tentative
1557	NB_ESSAI=3
1558	# time delay between tentative
1559	DELAI=2
1560
1561	i=0
1562	while [ $i -lt $NB_ESSAI ] ; do
1563	rebuild -f -o $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1564	status=$?
1565	if [ ${status} -gt 0 ] ; then
1566	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1567	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1568	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1569	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1570	firstArg=${1}
1571	\rm ${firstArg}
1572	sleep $DELAI
1573	else
1574	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1575	break
1576	fi
1577	(( i = i + 1 ))
1578	done
1579
1580	if [ ${status} -gt 0 ] ; then
1581	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1582	IGCM_debug_Exit "rebuild"
1583	fi
1584
1585	IGCM_debug_PopStack "IGCM_sys_rebuild"
1586	}
1587
1588	#D-#==================================================
1589	#D-function IGCM_sys_rebuild_station
1590	#D-* Purpose: rebuild parallel files describing station
1591	#D-* Examples:
1592	#D-
1593	function IGCM_sys_rebuild_station {
1594	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1595	typeset i list_opt file_in file_out prefix_invert list_invert
1596	if ( $DEBUG_sys ) ; then
1597	echo "IGCM_sys_rebuild_station :" $@
1598	fi
1599	list_opt=$@
1600
1601	# Invert Axis : t,x -> x,t
1602	# t,pres,x -> x,t,pres
1603	# So that we can concatenate along x
1604	i=0
1605	for file_in in ${list_opt} ; do
1606	(( i = i + 1))
1607	[ ${i} = 1 ] && file_out=${file_in} && continue
1608	# detect time counter and do the job only if present
1609	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
1610	if [ X${var_unlim} = Xtime_counter ] ; then
1611	prefix_invert=$( basename ${file_in} .nc )
1612	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1613	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1614	fi
1615	done
1616
1617	# Concatenate
1618	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1619
1620	# Re-ivert file
1621	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1622
1623	# Station re-ordering is too expansive to be run within libIGCM
1624	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
1625	# This re-ordering must be done "in memory" by the cmorization process
1626	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
1627	# BEGIN reordering
1628
1629	# Only LMDZ text output contains the exact ordering of the station.
1630	# We isolate this in the code below:
1631	# 0 38 -157.5000000000000 70.98591549295774
1632	# 0 54 27.49999999999999 67.18309859154928
1633	# 0 56 -62.50000000000001 82.39436619718309
1634	# 0 79 12.49999999999999 78.59154929577466
1635	# 0 116 -165.0000000000000 76.05633802816901
1636	# 0 117 130.0000000000000 70.98591549295774
1637	# 0 118 110.0000000000000 87.46478873239437
1638	# 1 40 4.999999999999995 51.97183098591550
1639
1640	# typeset iStation iProc list_opt file_in file_out prefix_invert
1641	# typeset -Z4 j4
1642	# typeset -Z3 j3
1643
1644	# unset list_opt
1645	# set +A list_opt $@
1646
1647	# Filename after rebuild
1648	# file_out=${list_opt[0]}
1649	# Prefix of output files
1650	# prefix_invert=$( basename ${file_out} .nc )
1651	# Number of procs
1652	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
1653
1654	# iProc=0
1655	# while [ ${iProc} -lt ${num_proc} ] ; do
1656	# Array containing Station as a number
1657	# unset proc_stn
1658	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
1659	# Number of stations produced by processor proc
1660	# stationLast=${#proc_stn[*]}
1661	# Proc number on 4 digits
1662	# j4=${iProc}
1663	# Init
1664	# iStation=0
1665	# while [ ${iStation} -lt ${stationLast} ] ; do
1666	# Station number on 3 digits
1667	# j3=${proc_stn[${iStation}]}
1668	# Extract station
1669	# Invert Axis : t,x -> x,t
1670	# t,pres,x -> x,t,pres
1671	# So that we can concatenate along x
1672	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
1673	# (( iStation = iStation + 1 ))
1674	# done
1675	# (( iProc = iProc + 1 ))
1676	# done
1677
1678	# Concatenate all station along x
1679	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
1680
1681	# Re-invert file
1682	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
1683
1684	# END reordering
1685
1686	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
1687	}
1688
1689	############################################################
1690	# Activate Running Environnment Variables
1691
1692	#D-#==================================================
1693	#D-function IGCM_sys_desactiv_variables
1694	#D-* Purpose: set environement variables prior to execution
1695	#D-* Examples:
1696	#D-
1697	function IGCM_sys_activ_variables {
1698	IGCM_debug_PushStack "IGCM_sys_activ_variables"
1699	if ( $DEBUG_sys ) ; then
1700	echo "IGCM_sys_activ_variables"
1701	fi
1702
1703	# --------------------------------------------------------------------
1704	#D- MPI specifications
1705	# --------------------------------------------------------------------
1706
1707	# --------------------------------------------------------------------
1708	#D- Other specifications
1709	# --------------------------------------------------------------------
1710
1711	IGCM_debug_PopStack "IGCM_sys_activ_variables"
1712	}
1713
1714	############################################################
1715	# Desactivate Running Environnment Variables
1716
1717	#D-#==================================================
1718	#D-function IGCM_sys_desactiv_variables
1719	#D-* Purpose: unset environement variables after execution
1720	#D-* Examples:
1721	#D-
1722	function IGCM_sys_desactiv_variables {
1723	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
1724	if ( $DEBUG_sys ) ; then
1725	echo "IGCM_sys_desactiv_variables"
1726	fi
1727	# --------------------------------------------------------------------
1728	#D- MPI specifications
1729	# --------------------------------------------------------------------
1730
1731	# --------------------------------------------------------------------
1732	#D- Other specifications
1733	# --------------------------------------------------------------------
1734
1735	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
1736	}
1737
1738	############################################################
1739	# Build MPI/OMP scripts run file (dummy function)
1740
1741	#D-#==================================================
1742	#D-function IGCM_sys_build_run_file
1743	#D-* Purpose: build run file (deprecated)
1744	#D-* Examples:
1745	#D-
1746	function IGCM_sys_build_run_file {
1747
1748	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
1749
1750	}
1751
1752	############################################################
1753	# Build MPI/OMP scripts
1754
1755	#D-#==================================================
1756	#D-function IGCM_sys_build_execution_scripts
1757	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
1758	#D-* Examples:
1759	#D-
1760	function IGCM_sys_build_execution_scripts
1761	{
1762	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
1763	if ( $DEBUG_sys ) ; then
1764	echo "IGCM_sys_build_execution_scripts " $@
1765	fi
1766
1767	typeset nodes listnodes init_node start_num init_exec comp ExeNameIn ExeNameOut
1768	typeset node_num_current node_current comp_proc_mpi_loc comp_proc_omp_loc
1769	typeset num_corempi nombre_restant_node nombre_restant_comp
1770
1771	if ( ${OK_PARA_MPMD} ) ; then
1772
1773	if [ -f run_file ] ; then
1774	IGCM_sys_Rm -f run_file
1775	fi
1776	touch run_file
1777
1778	if ( ${OK_PARA_OMP} ) ; then
1779
1780	# Hosts treatment
1781
1782	${HOST_MPIRUN_COMMAND} hostname \| sort \| uniq > hosts.tmp
1783
1784	i=0
1785	rm -f hosts
1786	IGCM_debug_Print 1 "sys Obelix, Hosts avaible :"
1787	for nodes in `cat hosts.tmp` ; do
1788	host[$i]=$nodes
1789	echo "${host[$i]} slots=1 max_slots=1" >> hosts
1790	IGCM_debug_Print 1 ${host[$i]}
1791	i=$((i+1))
1792	done
1793	rm -f hosts.tmp
1794
1795	listnodes=${host[*]}
1796
1797	EXECUTION="${HOST_MPIRUN_COMMAND} -hostfile hosts"
1798
1799	# Initialisation
1800
1801	init_node=y
1802	node_num_current=0
1803	start_num=0
1804	init_exec=n
1805
1806	# Build run_file
1807
1808	# First loop on the components for the coupler ie oasis (only if oasis3)
1809	# the coupler ie oasis3 must be the first one
1810	for comp in ${config_ListOfComponents[*]} ; do
1811
1812	if [ "X${comp}" = "XCPL" ] ; then
1813
1814	eval ExeNameIn=\${config_Executable_${comp}[0]}
1815	eval ExeNameOut=\${config_Executable_${comp}[1]}
1816
1817	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1818	echo "" >> script_${ExeNameOut}.ksh
1819	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1820	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
1821	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
1822	echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
1823	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1824
1825	init_node=n
1826
1827	(( nombre_restant_node = NUM_COREPERNODE - 1 ))
1828	node_num_current=0
1829	node_current=${host[${node_num_current}]}
1830
1831	EXECUTION="${EXECUTION} -H ${node_current} -np 1 ./script_${ExeNameOut}.ksh"
1832
1833	init_exec=y
1834	start_num=1
1835
1836	fi
1837
1838	done
1839
1840	# Then loop on the components (except for oasis)
1841
1842	for comp in ${config_ListOfComponents[*]} ; do
1843
1844	eval ExeNameIn=\${config_Executable_${comp}[0]}
1845	eval ExeNameOut=\${config_Executable_${comp}[1]}
1846
1847	# Only if we really have an executable for the component :
1848	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
1849
1850	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1851	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1852
1853	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1854	# echo "set -vx" >> script_${ExeNameOut}.ksh
1855	echo "" >> script_${ExeNameOut}.ksh
1856	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1857	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
1858	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
1859	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1860	echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
1861	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
1862	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1863
1864	node_num=0
1865
1866	# We define the number of MPI process to be assigned for the component
1867
1868	nombre_restant_comp=${comp_proc_mpi_loc}
1869
1870	# Loop on the allocated nodes
1871
1872	for node in ${listnodes} ; do
1873
1874	# We go to the current node
1875	if [ ${node_num} = ${node_num_current} ] ; then
1876
1877	node_current=${host[${node_num_current}]}
1878
1879	# If first time on the node : initialisation
1880
1881	if [ ${init_node} = y ] ; then
1882	nombre_restant_node=${NUM_COREPERNODE}
1883	fi
1884
1885	# Test on the number of OMP threads
1886
1887	if [ ${comp_proc_omp_loc} -gt ${nombre_restant_node} ] ; then
1888	(( node_num = node_num + 1 ))
1889	node_num_current=${node_num}
1890	init_node=y
1891	continue
1892	fi
1893
1894	# Number of MPI process to assign
1895
1896	(( num_corempi = nombre_restant_node / comp_proc_omp_loc ))
1897
1898	if [ ${num_corempi} -gt ${nombre_restant_comp} ] ; then
1899	num_corempi=${nombre_restant_comp}
1900	fi
1901
1902	(( nombre_restant_node = nombre_restant_node - num_corempi * comp_proc_omp_loc ))
1903	(( nombre_restant_comp = nombre_restant_comp - num_corempi ))
1904
1905	if [ ${init_exec} = y ] ; then
1906	EXECUTION="${EXECUTION} : -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
1907	else
1908	EXECUTION="${EXECUTION} -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
1909	init_exec=y
1910	fi
1911
1912	(( start_num = num_corempi + start_num ))
1913
1914	else
1915
1916	(( node_num = node_num + 1 ))
1917	continue
1918	fi
1919
1920	# Test on the number of core/process remaining on the node/component
1921
1922	if [ ${nombre_restant_node} = 0 ] ; then
1923	(( node_num = node_num + 1 ))
1924	node_num_current=${node_num}
1925	init_node=y
1926
1927	if [ ${nombre_restant_comp} = 0 ] ; then
1928	break 1
1929	fi
1930	else
1931
1932	node_num_current=${node_num}
1933	init_node=n
1934
1935	if [ ${nombre_restant_comp} = 0 ] ; then
1936	break 1
1937	fi
1938	fi
1939	done
1940	fi
1941	done
1942
1943	else
1944
1945	# Then first loop on the components for the coupler ie oasis
1946
1947	## the coupler ie oasis must be the first one
1948	for comp in ${config_ListOfComponents[*]} ; do
1949
1950	eval ExeNameOut=\${config_Executable_${comp}[1]}
1951
1952	# for CPL component only
1953	if [ "X${comp}" = "XCPL" ] ; then
1954	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1955	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut} " >> run_file
1956	fi
1957	done
1958
1959	# Then second loop on the components
1960
1961	for comp in ${config_ListOfComponents[*]} ; do
1962
1963	eval ExeNameOut=\${config_Executable_${comp}[1]}
1964
1965	# Only if we really have an executable for the component and not the coupler ie oasis:
1966	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
1967	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1968	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
1969	fi
1970	done
1971	IGCM_sys_Chmod u+x run_file
1972
1973	EXECUTION="${HOST_MPIRUN_COMMAND} --app ./run_file"
1974
1975	fi
1976
1977	else # Only one executable. launch it.
1978
1979	for comp in ${config_ListOfComponents[*]} ; do
1980
1981	# Only if we really have an executable for the component :
1982	eval ExeNameOut=\${config_Executable_${comp}[1]}
1983	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
1984
1985	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1986	echo "" >> script_${ExeNameOut}.ksh
1987	if ( ${OK_PARA_OMP} ) ; then
1988	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1989	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1990	fi
1991	if ( ${OK_PARA_MPI} ) ; then
1992	# Default : mpirun used if nb_proc gt 1
1993	# pour sortie out/err par process
1994	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${OMPI_COMM_WORLD_RANK} 2>out_${ExeNameOut}.err.\${OMPI_COMM_WORLD_RANK}" >> script_${ExeNameOut}.ksh
1995	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
1996	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1997	EXECUTION="${HOST_MPIRUN_COMMAND} ./script_${ExeNameOut}.ksh"
1998	else
1999	# Default : mpirun is NOT used if nb_proc eq 1
2000	# pour sortie out/err par process
2001	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2002	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2003	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2004	EXECUTION="time ./script_${ExeNameOut}.ksh"
2005	fi
2006
2007	IGCM_debug_Print 1 "sys Obelix : script_${ExeNameOut}.ksh contains"
2008	cat script_${ExeNameOut}.ksh
2009
2010	fi
2011
2012	done
2013
2014	fi
2015
2016	IGCM_debug_Print 1 "sys Obelix : execution command is"
2017	IGCM_debug_Print 1 "$EXECUTION"
2018
2019	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2020	}
2021
2022	#D-#==================================================
2023	#D-function IGCM_sys_check_path
2024	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2025	#D-* do not point to an important use directory. Stop immediately in that case.
2026	#D-* Examples:
2027	#D-
2028	function IGCM_sys_check_path {
2029	IGCM_debug_PushStack "IGCM_sys_check_path"
2030	if ( $DEBUG_sys ) ; then
2031	echo "IGCM_sys_check_path"
2032	fi
2033
2034	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
2035	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2036	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2037	IGCM_debug_Exit "This will stop the job"
2038	fi
2039	IGCM_debug_PopStack "IGCM_sys_check_path"
2040	}
2041
2042	#D-#==================================================
2043	#D-function IGCM_sys_check_quota. Dummy call here
2044	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2045	#D-* Examples:
2046	#D-
2047	function IGCM_sys_check_quota {
2048	IGCM_debug_PushStack "IGCM_sys_check_quota"
2049	if ( $DEBUG_sys ) ; then
2050	echo "IGCM_sys_check_quota"
2051	fi
2052	IGCM_debug_PopStack "IGCM_sys_check_quota"
2053	}
2054
2055	#D-#==================================================
2056	#D-function IGCM_sys_GetJobID
2057	#D-* Purpose: Check if job_name is currently
2058	#D- running or in queue
2059	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2060	#D-
2061	function IGCM_sys_GetJobID {
2062	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2063	if ( $DEBUG_sys ) ; then
2064	echo "IGCM_sys_GetJobID"
2065	fi
2066
2067	# With -f option, the full job name is given in the last column
2068	ID="$( qstat -u $2 \| grep -w $1 \| gawk '-F ' '{print $10}' )"
2069
2070	eval ${3}=${ID}
2071	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2072	}
2073
2074	#D-#==================================================
2075	#D-function IGCM_sys_CountJobInQueue
2076	#D-* Purpose: Check if job_name is currently
2077	#D- running or in queue
2078	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2079	#D-
2080	function IGCM_sys_CountJobInQueue {
2081	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2082	if ( $DEBUG_sys ) ; then
2083	echo "IGCM_sys_CountJobInQueue"
2084	fi
2085	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2086	}
2087
2088	##############################################################
2089	# NCO OPERATOR
2090
2091	#D-#==================================================
2092	#D-function IGCM_sys_ncap2
2093	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2094	#D-* Examples:
2095	#D-
2096	function IGCM_sys_ncap2 {
2097	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2098	if ( $DEBUG_sys ) ; then
2099	echo "IGCM_sys_ncap2 :" $@
2100	fi
2101
2102	typeset NB_ESSAI DELAI status i
2103	# number of tentative
2104	NB_ESSAI=3
2105	# time delay between tentative
2106	DELAI=2
2107
2108	i=0
2109	while [ $i -lt $NB_ESSAI ] ; do
2110	ncap2 -C "$@" > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2111	status=$?
2112	if [ ${status} -gt 0 ] ; then
2113	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2114	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2115	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2116	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2117	sleep $DELAI
2118	else
2119	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2120	break
2121	fi
2122	(( i = i + 1 ))
2123	done
2124
2125	if [ ${status} -gt 0 ] ; then
2126	echo "IGCM_sys_ncap2 : ncap2 error"
2127	IGCM_debug_Exit "ncap2"
2128	fi
2129
2130	IGCM_debug_PopStack "IGCM_sys_ncap2"
2131	}
2132
2133	#D-#==================================================
2134	#D-function IGCM_sys_ncatted
2135	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2136	#D-* Examples:
2137	#D-
2138	function IGCM_sys_ncatted {
2139	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2140	if ( $DEBUG_sys ) ; then
2141	echo "IGCM_sys_ncatted :" $@
2142	fi
2143
2144	typeset NB_ESSAI DELAI status i
2145	# number of tentative
2146	NB_ESSAI=3
2147	# time delay between tentative
2148	DELAI=2
2149
2150	i=0
2151	while [ $i -lt $NB_ESSAI ] ; do
2152	ncatted "$@" > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2153	status=$?
2154	if [ ${status} -gt 0 ] ; then
2155	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2156	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2157	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2158	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2159	sleep $DELAI
2160	else
2161	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2162	break
2163	fi
2164	(( i = i + 1 ))
2165	done
2166
2167	if [ ${status} -gt 0 ] ; then
2168	echo "IGCM_sys_ncatted : ncatted error"
2169	IGCM_debug_Exit "ncatted"
2170	fi
2171
2172	IGCM_debug_PopStack "IGCM_sys_ncatted"
2173	}
2174
2175	#D-#==================================================
2176	#D-function IGCM_sys_ncbo
2177	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2178	#D-* Examples:
2179	#D-
2180	function IGCM_sys_ncbo {
2181	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2182	if ( $DEBUG_sys ) ; then
2183	echo "IGCM_sys_ncbo :" $@
2184	fi
2185
2186	typeset NB_ESSAI DELAI status i
2187	# number of tentative
2188	NB_ESSAI=3
2189	# time delay between tentative
2190	DELAI=2
2191
2192	i=0
2193	while [ $i -lt $NB_ESSAI ] ; do
2194	ncbo -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2195	status=$?
2196	if [ ${status} -gt 0 ] ; then
2197	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2198	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2199	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2200	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2201	sleep $DELAI
2202	else
2203	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2204	break
2205	fi
2206	(( i = i + 1 ))
2207	done
2208
2209	if [ ${status} -gt 0 ] ; then
2210	echo "IGCM_sys_ncbo : ncbo error"
2211	IGCM_debug_Exit "ncbo"
2212	fi
2213
2214	IGCM_debug_PopStack "IGCM_sys_ncbo"
2215	}
2216
2217	#D-#==================================================
2218	#D-function IGCM_sys_ncdif
2219	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2220	#D-* Examples:
2221	#D-
2222	function IGCM_sys_ncdiff {
2223	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2224	if ( $DEBUG_sys ) ; then
2225	echo "IGCM_sys_ncdiff :" $@
2226	fi
2227
2228	typeset NB_ESSAI DELAI status i
2229	# number of tentative
2230	NB_ESSAI=3
2231	# time delay between tentative
2232	DELAI=2
2233
2234	i=0
2235	while [ $i -lt $NB_ESSAI ] ; do
2236	ncdiff -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2237	status=$?
2238	if [ ${status} -gt 0 ] ; then
2239	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2240	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2241	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2242	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2243	sleep $DELAI
2244	else
2245	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2246	break
2247	fi
2248	(( i = i + 1 ))
2249	done
2250
2251	if [ ${status} -gt 0 ] ; then
2252	echo "IGCM_sys_ncdiff : ncdiff error"
2253	IGCM_debug_Exit "ncdiff"
2254	fi
2255
2256	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2257	}
2258
2259	#D-#==================================================
2260	#D-function IGCM_sys_ncea
2261	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2262	#D-* Examples:
2263	#D-
2264	function IGCM_sys_ncea {
2265	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2266	if ( $DEBUG_sys ) ; then
2267	echo "IGCM_sys_ncea :" $@
2268	fi
2269
2270	typeset NB_ESSAI DELAI status i
2271	# number of tentative
2272	NB_ESSAI=3
2273	# time delay between tentative
2274	DELAI=2
2275
2276	i=0
2277	while [ $i -lt $NB_ESSAI ] ; do
2278	ncea -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2279	status=$?
2280	if [ ${status} -gt 0 ] ; then
2281	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2282	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2283	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2284	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2285	sleep $DELAI
2286	else
2287	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2288	break
2289	fi
2290	(( i = i + 1 ))
2291	done
2292
2293	if [ ${status} -gt 0 ] ; then
2294	echo "IGCM_sys_ncea : ncea error"
2295	IGCM_debug_Exit "ncea"
2296	fi
2297
2298	IGCM_debug_PopStack "IGCM_sys_ncea"
2299	}
2300
2301	#D-#==================================================
2302	#D-function IGCM_sys_ncecat
2303	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2304	#D-* Examples:
2305	#D-
2306	function IGCM_sys_ncecat {
2307	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2308	if ( $DEBUG_sys ) ; then
2309	echo "IGCM_sys_ncecat :" $@
2310	fi
2311
2312	typeset NB_ESSAI DELAI status i
2313	# number of tentative
2314	NB_ESSAI=3
2315	# time delay between tentative
2316	DELAI=2
2317
2318	i=0
2319	while [ $i -lt $NB_ESSAI ] ; do
2320	ncecat -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2321	status=$?
2322	if [ ${status} -gt 0 ] ; then
2323	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2324	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2325	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2326	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2327	sleep $DELAI
2328	else
2329	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2330	break
2331	fi
2332	(( i = i + 1 ))
2333	done
2334
2335	if [ ${status} -gt 0 ] ; then
2336	echo "IGCM_sys_ncecat : ncecat error"
2337	IGCM_debug_Exit "ncecat"
2338	fi
2339
2340	IGCM_debug_PopStack "IGCM_sys_ncecat"
2341	}
2342
2343	#D-#==================================================
2344	#D-function IGCM_sys_ncflint
2345	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2346	#D-* Examples:
2347	#D-
2348	function IGCM_sys_ncflint {
2349	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2350	if ( $DEBUG_sys ) ; then
2351	echo "IGCM_sys_ncflint :" $@
2352	fi
2353
2354	typeset NB_ESSAI DELAI status i
2355	# number of tentative
2356	NB_ESSAI=3
2357	# time delay between tentative
2358	DELAI=2
2359
2360	i=0
2361	while [ $i -lt $NB_ESSAI ] ; do
2362	ncflint -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2363	status=$?
2364	if [ ${status} -gt 0 ] ; then
2365	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2366	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2367	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2368	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2369	sleep $DELAI
2370	else
2371	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2372	break
2373	fi
2374	(( i = i + 1 ))
2375	done
2376
2377	if [ ${status} -gt 0 ] ; then
2378	echo "IGCM_sys_ncflint : ncflint error"
2379	IGCM_debug_Exit "ncflint"
2380	fi
2381
2382	IGCM_debug_PopStack "IGCM_sys_ncflint"
2383	}
2384
2385	#D-#==================================================
2386	#D-function IGCM_sys_ncks
2387	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2388	#D-* Examples:
2389	#D-
2390	function IGCM_sys_ncks {
2391	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2392	if ( $DEBUG_sys ) ; then
2393	echo "IGCM_sys_ncks :" $@
2394	fi
2395
2396	typeset NB_ESSAI DELAI status i
2397	# number of tentative
2398	NB_ESSAI=3
2399	# time delay between tentative
2400	DELAI=2
2401
2402	i=0
2403	while [ $i -lt $NB_ESSAI ] ; do
2404	ncks -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2405	status=$?
2406	if [ ${status} -gt 0 ] ; then
2407	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2408	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2409	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2410	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2411	sleep $DELAI
2412	else
2413	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2414	break
2415	fi
2416	(( i = i + 1 ))
2417	done
2418
2419	if [ ${status} -gt 0 ] ; then
2420	echo "IGCM_sys_ncks : ncks error"
2421	IGCM_debug_Exit "ncks"
2422	fi
2423
2424	IGCM_debug_PopStack "IGCM_sys_ncks"
2425	}
2426
2427	#D-#==================================================
2428	#D-function IGCM_sys_ncpdq
2429	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2430	#D-* Examples:
2431	#D-
2432	function IGCM_sys_ncpdq {
2433	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2434	if ( $DEBUG_sys ) ; then
2435	echo "IGCM_sys_ncpdq :" $@
2436	fi
2437
2438	typeset NB_ESSAI DELAI status i
2439	# number of tentative
2440	NB_ESSAI=3
2441	# time delay between tentative
2442	DELAI=2
2443
2444	i=0
2445	while [ $i -lt $NB_ESSAI ] ; do
2446	ncpdq -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2447	status=$?
2448	if [ ${status} -gt 0 ] ; then
2449	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2450	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2451	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2452	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2453	sleep $DELAI
2454	else
2455	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2456	break
2457	fi
2458	(( i = i + 1 ))
2459	done
2460
2461	if [ ${status} -gt 0 ] ; then
2462	echo "IGCM_sys_ncpdq : ncpdq error"
2463	IGCM_debug_Exit "ncpdq"
2464	fi
2465
2466	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2467	}
2468
2469	#D-#==================================================
2470	#D-function IGCM_sys_ncra
2471	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2472	#D-* Examples:
2473	#D-
2474	function IGCM_sys_ncra {
2475	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2476	if ( $DEBUG_sys ) ; then
2477	echo "IGCM_sys_ncra :" $@
2478	fi
2479
2480	typeset NB_ESSAI DELAI status i
2481	# number of tentative
2482	NB_ESSAI=3
2483	# time delay between tentative
2484	DELAI=2
2485
2486	i=0
2487	while [ $i -lt $NB_ESSAI ] ; do
2488	ncra -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2489	status=$?
2490	if [ ${status} -gt 0 ] ; then
2491	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
2492	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2493	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2494	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2495	sleep $DELAI
2496	else
2497	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2498	break
2499	fi
2500	(( i = i + 1 ))
2501	done
2502
2503	if [ ${status} -gt 0 ] ; then
2504	echo "IGCM_sys_ncra : ncra error"
2505	IGCM_debug_Exit "ncra"
2506	fi
2507
2508	IGCM_debug_PopStack "IGCM_sys_ncra"
2509	}
2510
2511	#D-#==================================================
2512	#D-function IGCM_sys_ncrcat
2513	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
2514	#D-* Examples:
2515	#D-
2516	function IGCM_sys_ncrcat {
2517	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
2518	if ( $DEBUG_sys ) ; then
2519	echo "IGCM_sys_ncrcat :" $@
2520	fi
2521
2522	typeset NB_ESSAI DELAI status i lastArg
2523	# number of tentative
2524	NB_ESSAI=3
2525	# time delay between tentative
2526	DELAI=2
2527
2528	i=0
2529	while [ $i -lt $NB_ESSAI ] ; do
2530	ncrcat -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2531	status=$?
2532	if [ ${status} -gt 0 ] ; then
2533	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
2534	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2535	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2536	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2537	sleep $DELAI
2538	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )" = "X" ] ; then
2539	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
2540	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2541	# remove files having corrupted time axis
2542	eval lastArg=\${$#}
2543	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
2544	\rm ${lastArg}
2545	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2546	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2547	sleep $DELAI
2548	else
2549	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2550	break
2551	fi
2552	(( i = i + 1 ))
2553	done
2554
2555	if [ ${status} -gt 0 ] ; then
2556	echo "IGCM_sys_ncrcat : ncrcat error"
2557	#IGCM_debug_Exit "ncrcat"
2558	fi
2559
2560	IGCM_debug_PopStack "IGCM_sys_ncrcat"
2561	}
2562
2563	#D-#==================================================
2564	#D-function IGCM_sys_ncrename
2565	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
2566	#D-* Examples:
2567	#D-
2568	function IGCM_sys_ncrename {
2569	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
2570	if ( $DEBUG_sys ) ; then
2571	echo "IGCM_sys_ncrename :" $@
2572	fi
2573
2574	typeset NB_ESSAI DELAI status i
2575	# number of tentative
2576	NB_ESSAI=3
2577	# time delay between tentative
2578	DELAI=2
2579
2580	i=0
2581	while [ $i -lt $NB_ESSAI ] ; do
2582	ncrename $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2583	status=$?
2584	if [ ${status} -gt 0 ] ; then
2585	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
2586	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2587	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2588	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2589	sleep $DELAI
2590	else
2591	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2592	break
2593	fi
2594	(( i = i + 1 ))
2595	done
2596
2597	if [ ${status} -gt 0 ] ; then
2598	echo "IGCM_sys_ncrename : ncrename error"
2599	IGCM_debug_Exit "ncrename"
2600	fi
2601
2602	IGCM_debug_PopStack "IGCM_sys_ncrename"
2603	}
2604
2605	#D-#==================================================
2606	#D-function IGCM_sys_ncwa
2607	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
2608	#D-* Examples:
2609	#D-
2610	function IGCM_sys_ncwa {
2611	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
2612	if ( $DEBUG_sys ) ; then
2613	echo "IGCM_sys_ncwa :" $@
2614	fi
2615
2616	typeset NB_ESSAI DELAI status i
2617	# number of tentative
2618	NB_ESSAI=3
2619	# time delay between tentative
2620	DELAI=2
2621
2622	i=0
2623	while [ $i -lt $NB_ESSAI ] ; do
2624	ncwa -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2625	status=$?
2626	if [ ${status} -gt 0 ] ; then
2627	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
2628	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2629	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2630	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2631	sleep $DELAI
2632	else
2633	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2634	break
2635	fi
2636	(( i = i + 1 ))
2637	done
2638
2639	if [ ${status} -gt 0 ] ; then
2640	echo "IGCM_sys_ncwa : ncwa error"
2641	IGCM_debug_Exit "ncwa"
2642	fi
2643
2644	IGCM_debug_PopStack "IGCM_sys_ncwa"
2645	}
2646
2647	##############################################################
2648	# CDO OPERATOR
2649
2650	#D-#==================================================
2651	#D-function IGCM_sys_cdo
2652	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
2653	#D-* Examples:
2654	#D-
2655	function IGCM_sys_cdo {
2656	IGCM_debug_PushStack "IGCM_sys_cdo" $@
2657	if ( $DEBUG_sys ) ; then
2658	echo "IGCM_sys_cdo :" $@
2659	fi
2660
2661	typeset status
2662
2663	\cdo $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2664	status=$?
2665	if [ ${status} -gt 0 ] ; then
2666	echo "IGCM_sys_cdo : error code ${status}"
2667	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2668	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2669	IGCM_debug_PopStack "IGCM_sys_cdo"
2670	return 1
2671	else
2672	IGCM_debug_PopStack "IGCM_sys_cdo"
2673	return 0
2674	fi
2675
2676	IGCM_debug_PopStack "IGCM_sys_cdo"
2677	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: