Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_curie.ksh @ 1071

Last change on this file since 1071 was 1063, checked in by sdipsl, 10 years ago
bugfixes
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 92.8 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Curie
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=curie
71	# add default project on curie
72	typeset DEFAULT_PROJECT=gen0826
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# Submit command
80	typeset SUBMIT=${SUBMIT:=ccc_msub}
81	# rsync with path
82	typeset -r RSYNC=/usr/bin/rsync
83	# RSYNC_opt args to rsync
84	typeset -r RSYNC_opt="-va"
85	# ie storage filesystem
86	typeset -r STOREHOST=${MASTER}
87
88	#====================================================
89	# Source default environment
90	#====================================================
91	##. /etc/profile
92
93	#====================================================
94	# Set environment tools (ferret, nco, cdo)
95	#====================================================
96	if [ X${TaskType} = Xcomputing ] ; then
97	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
98	# to run with netcdf 3.6.3 ie compilation done before 17/2/2014
99	# uncomment 2 lines :
100	# module unload netcdf
101	# module load netcdf/3.6.3
102	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddNoise/src_X64_CURIE/bin
103	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddPerturbation/src_X64_CURIE/bin
104	else
105	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
106	fi
107
108	#====================================================
109	# Specific for ocean additionnal diagnostic
110	export FER_GO="$FER_GO /home/cont003/p86denv/IGCM_POST_UTIL/JNL /home/cont003/p86denv/GRAF /home/cont003/p86denv/GRAF/GO"
111	export FER_PALETTE="$FER_PALETTE /home/cont003/p86denv/GRAF/PALET"
112
113	#====================================================
114	# Host specific DIRECTORIES
115	#====================================================
116
117	# ============ CESIUM START ============ #
118
119	#====================================================
120	#- Mirror libIGCM from titane to cesium if needed
121	#ROOTSYS=$( echo ${libIGCM} \| gawk -F"/" '{print $3}' )
122	#if [ ! ${ROOTSYS} = "home" ] ; then
123	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
124	#else
125	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
126	#fi
127
128	#====================================================
129	#- libIGCM_POST
130	#if ( ${MirrorlibIGCM} ) ; then
131	# PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
132	# typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
133	#else
134	# typeset -r libIGCM_POST=${libIGCM}
135	#fi
136
137	# ============ CESIUM END ============ #
138
139	#====================================================
140	#- MirrorlibIGCM for frontend
141	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
142
143	#====================================================
144	#- libIGCM_POST for frontend
145	typeset -r libIGCM_POST=${libIGCM}
146
147	#====================================================
148	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
149	typeset -r R_EXE="${MODIPSL}/bin"
150
151	#====================================================
152	#- SUBMIT_DIR : submission dir
153	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${BRIDGE_MSUB_PWD}}
154
155	#====================================================
156	#- IN
157	typeset -r R_IN=${R_IN:=/ccc/work/cont003/dsm/p86ipsl/IGCM}
158	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/ccc/work/cont003/dsm/p24data}
159
160	#====================================================
161	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
162	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCHDIR}/RUN_DIR/${BRIDGE_MSUB_JOBID}_${$}}
163
164	#====================================================
165	#- HOST_MPIRUN_COMMAND
166	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time ccc_mprun -E-K1"}
167
168	#====================================================
169	#- Max number of arguments passed to nco operator or demigration command
170	UNIX_MAX_LIMIT=360
171
172	#====================================================
173	#- set PackDefault true on curie
174	PackDefault=true
175
176	#====================================================
177	#- Number of core per node (max number of OpenMP task)
178	NUM_COREPERNODE=8
179
180	#====================================================
181	#- Default number of MPI task for IPSL coupled model
182	#- required for backward compatibility
183	#-
184	DEFAULT_NUM_PROC_OCE=5
185	DEFAULT_NUM_PROC_CPL=1
186	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
187	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
188
189	#D-#==================================================
190	#D-function IGCM_sys_defineArchives
191	#D-* Purpose:
192	#D-* Define ARCHIVE : Dedicated to large files
193	#D-* Define STORAGE : Dedicated to small/medium files
194	#D-* Define R_OUT : Output tree located on ARCHIVE
195	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
196	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
197	#D-* if SpaceName=TEST everything is stored on SCRATCHDIR
198	#D-* Examples:
199	#D-
200	function IGCM_sys_defineArchives {
201	IGCM_debug_PushStack "IGCM_sys_defineArchives"
202
203	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
204	#====================================================
205	#- ARCHIVE (dedicated to large files)
206	ARCHIVE=${config_UserChoices_ARCHIVE}
207	else
208	#====================================================
209	#- ARCHIVE (dedicated to large files)
210	ARCHIVE=${CCCSTOREDIR}
211	fi
212
213	if [ ! X${config_UserChoices_STORAGE} = X ]; then
214	#====================================================
215	#- STORAGE (dedicated to small/medium files)
216	STORAGE=${config_UserChoices_STORAGE}
217	else
218	#====================================================
219	#- STORAGE (dedicated to small/medium files)
220	STORAGE=${CCCWORKDIR}
221	fi
222
223	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
224	#====================================================
225	#- R_OUT
226	R_OUT=${SCRATCHDIR}/IGCM_OUT
227
228	#====================================================
229	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
230	R_FIG=${SCRATCHDIR}/IGCM_OUT
231
232	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
233
234	else
235	#====================================================
236	#- R_OUT
237	R_OUT=${ARCHIVE}/IGCM_OUT
238
239	#====================================================
240	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
241	R_FIG=${STORAGE}/IGCM_OUT
242	fi
243
244	#====================================================
245	#- R_BUF (ONLY FOR double copy an scratch)
246	R_BUF=${SCRATCHDIR}/IGCM_OUT
247
248	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
249	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
250	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
251
252	IGCM_debug_PopStack "IGCM_sys_defineArchives"
253	}
254
255	#D-#==================================================
256	#D-function IGCM_sys_RshMaster
257	#D-* Purpose: Connection to frontend machine.
258	#D-* Examples:
259	#D-
260	function IGCM_sys_RshMaster {
261	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
262	/bin/ksh <<-EOF
263	export libIGCM=${libIGCM}
264	export DEBUG_debug=${DEBUG_debug}
265	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
266	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
267	${@}
268	EOF
269	if [ $? -gt 0 ] ; then
270	echo "IGCM_sys_RshMaster : erreur."
271	IGCM_debug_Exit "IGCM_sys_RshMaster"
272	fi
273	IGCM_debug_PopStack "IGCM_sys_RshMaster"
274	}
275
276	#D-#==================================================
277	#D-function IGCM_sys_RshArchive
278	#D-* Purpose: Archive rsh command
279	#D-* Examples:
280	#D-
281	function IGCM_sys_RshArchive {
282	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
283	/bin/ksh <<-EOF
284	${@}
285	EOF
286	status=$?
287	if [ ${status} -gt 0 ] ; then
288	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
289	IGCM_debug_Exit "IGCM_sys_RshArchive"
290	fi
291	IGCM_debug_PopStack "IGCM_sys_RshArchive"
292	}
293
294	#D-#==================================================
295	#D-function IGCM_sys_RshArchive_NoError
296	#D-* Purpose: Archive rsh command, without error
297	#D-* used only in monitoring.job
298	#D-* Examples:
299	#D-
300	function IGCM_sys_RshArchive_NoError {
301	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
302	/bin/ksh <<-EOF
303	${@} 2> \dev\null
304	EOF
305	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
306	}
307
308	#D-#==================================================
309	#D-function IGCM_sys_RshPost
310	#D-* Purpose: Post-process rsh command
311	#D-* Examples:
312	#D-
313	function IGCM_sys_RshPost {
314	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
315	if ( $DEBUG_sys ) ; then
316	echo "IGCM_sys_RshPost :" $@
317	fi
318	# keep standard input (stdin) for the loop onto temporary file
319	cat >/tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
320
321	/bin/ksh </tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
322	if [ $? -gt 0 ] ; then
323	echo "IGCM_sys_RshPost : erreur."
324	IGCM_debug_Exit "IGCM_sys_RshPost"
325	fi
326	# delete temporary file
327	\rm /tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
328
329	# ============ FRONTEND END ============ #
330
331	# ============ CESIUM START ============ #
332	# typeset NB_ESSAI DELAI status i
333	# if [ "X$( grep rebuild_from tmp_IGCM_sys_RshPost_$$ )" != "X" ] ; then
334	# #little hack so that rebuild submission is done on titane not an cesium
335	#
336	# libIGCM_POST_sed=$( echo $libIGCM_POST \| sed 's/\//\\\//g' )
337	# POST_DIR_sed=$( echo ${POST_DIR} \| sed 's/\//\\\//g' )
338	# sed "s/IGCM_sys_QsubPost/IGCM_sys_Qsub/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
339	# sed "s/ rebuild_fromWorkdir/ ${libIGCM_POST_sed}\/rebuild_fromWorkdir.job/g" tmp.txt > tmp_IGCM_sys_RshPost_$$
340	# sed "s/ rebuild_fromArchive/ ${libIGCM_POST_sed}\/rebuild_fromArchive.job/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
341	# sed "s/Script_Post_Output=/Script_Output=${POST_DIR_sed}\//g" tmp.txt > tmp_IGCM_sys_RshPost_$$
342	# \mv tmp.txt tmp_IGCM_sys_RshPost_$$
343	#
344	# echo cat tmp_IGCM_sys_RshPost_$$ AFTER
345	# cat tmp_IGCM_sys_RshPost_$$
346	#
347	# /bin/ksh <tmp_IGCM_sys_RshPost_$$
348	# if [ $? -gt 0 ] ; then
349	# echo "IGCM_sys_RshPost : erreur."
350	# IGCM_debug_Exit "IGCM_sys_RshPost"
351	# fi
352	# # delete temporary file
353	# \rm tmp_IGCM_sys_RshPost_$$
354	#
355	# else
356	# # number of tentative
357	# NB_ESSAI=10
358	# # time delay between tentative
359	# DELAI=10
360	# i=0
361	# while [ $i -ne $NB_ESSAI ] ; do
362	# ssh -t titane996 ssh cesium /bin/ksh <tmp_IGCM_sys_RshPost_$$
363	# status=$?
364	# if [ ${status} -ne 0 ];
365	# then
366	# sleep $DELAI
367	# else
368	# break
369	# fi
370	# let i=$i+1
371	# done
372	# # delete temporary file
373	# \rm tmp_IGCM_sys_RshPost_$$
374	#
375	# if [ ${status} -gt 0 ] ; then
376	# echo "IGCM_sys_RshPost : erreur."
377	# IGCM_debug_Exit "IGCM_sys_RshPost"
378	# fi
379	# fi
380
381	# ============ CESIUM END ============ #
382
383	IGCM_debug_PopStack "IGCM_sys_RshPost"
384	}
385
386	#D-#==================================================
387	#D-function IGCM_sys_SendMail
388	#D-* Purpose: Send mail when simulation is over
389	#D-* Examples:
390	#D-
391	function IGCM_sys_SendMail {
392	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
393
394	if ( $DEBUG_sys ) ; then
395	echo "IGCM_sys_SendMail :" $@
396	fi
397
398	if [ X${1} = XAccounting ] ; then
399	status=Accounting
400	mailText=jobAccounting.mail
401	elif ( ${ExitFlag} ) ; then
402	status=failed
403	mailText=jobEnd.mail
404	else
405	status=completed
406	mailText=jobEnd.mail
407	fi
408
409	# Update selected mail template
410	while read -r line; do
411	eval echo $line >> mail.txt ;
412	done < ${libIGCM}/libIGCM_sys/${mailText}
413
414	if [ ! -z ${config_UserChoices_MailName} ] ; then
415	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
416	elif [ -f ~/.forward ] ; then
417	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
418	fi
419
420	sleep 10
421	rm -f mail.txt
422
423	IGCM_debug_PopStack "IGCM_sys_SendMail"
424	}
425
426	#D-#==================================================
427	#D-function IGCM_sys_Mkdir
428	#D-* Purpose: Master locale mkdir command
429	#D-* Examples:
430	#D-
431	function IGCM_sys_Mkdir {
432	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
433	if ( $DEBUG_sys ) ; then
434	echo "IGCM_sys_Mkdir :" $@
435	fi
436	if [ ! -d ${1} ]; then
437	\mkdir -p $1
438	if [ $? -gt 0 ] ; then
439	echo "IGCM_sys_Mkdir : erreur."
440	IGCM_debug_Exit "IGCM_sys_Mkdir"
441	fi
442	fi
443	# vérification :
444	if [ ! -d ${1} ] ; then
445	echo "IGCM_sys_Mkdir : erreur."
446	IGCM_debug_Exit "IGCM_sys_Mkdir"
447	fi
448	IGCM_debug_PopStack "IGCM_sys_Mkdir"
449	}
450
451	#D-#==================================================
452	#D-function IGCM_sys_MkdirArchive
453	#D-* Purpose: Mkdir on Archive
454	#D-* Examples:
455	#D-
456	function IGCM_sys_MkdirArchive {
457	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
458	if ( $DEBUG_sys ) ; then
459	echo "IGCM_sys_MkdirArchive :" $@
460	fi
461	#- creation de repertoire sur le serveur fichier
462	if [ ! -d ${1} ]; then
463	\mkdir -p $1
464	status=$?
465
466	if [ ${status} -gt 0 ] ; then
467	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
468	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
469	fi
470	fi
471	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
472	}
473
474	#D-#==================================================
475	#D-function IGCM_sys_MkdirWork
476	#D-* Purpose: Mkdir on Work
477	#D-* Examples:
478	#D-
479	function IGCM_sys_MkdirWork {
480	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
481	if ( $DEBUG_sys ) ; then
482	echo "IGCM_sys_MkdirWork :" $@
483	fi
484	#- creation de repertoire sur le serveur fichier
485	if [ ! -d ${1} ]; then
486	\mkdir -p $1
487	if [ $? -gt 0 ] ; then
488	echo "IGCM_sys_MkdirWork : erreur."
489	IGCM_debug_Exit "IGCM_sys_MkdirWork"
490	fi
491	fi
492	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
493	}
494
495	#D-#==================================================
496	#D-function IGCM_sys_Cd
497	#D-* Purpose: master cd command
498	#D-* Examples:
499	#D-
500	function IGCM_sys_Cd {
501	IGCM_debug_PushStack "IGCM_sys_Cd" $@
502	if ( $DEBUG_sys ) ; then
503	echo "IGCM_sys_Cd :" $@
504	fi
505	\cd $1
506	if [ $? -gt 0 ] ; then
507	echo "IGCM_sys_Cd : erreur."
508	IGCM_debug_Exit "IGCM_sys_Cd"
509	fi
510	IGCM_debug_PopStack "IGCM_sys_Cd"
511	}
512
513	#D-#==================================================
514	#D-function IGCM_sys_Chmod
515	#D-* Purpose: Chmod
516	#D-* Examples:
517	#D-
518	function IGCM_sys_Chmod {
519	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
520	if ( $DEBUG_sys ) ; then
521	echo "IGCM_sys_Chmod :" $@
522	fi
523	\chmod $@
524	if [ $? -gt 0 ] ; then
525	echo "IGCM_sys_Chmod : erreur."
526	IGCM_debug_Exit "IGCM_sys_Chmod"
527	fi
528	IGCM_debug_PopStack "IGCM_sys_Chmod"
529	}
530
531	#D-#==================================================
532	#D-function IGCM_sys_FileSize
533	#D-* Purpose: Filesize
534	#D-* Examples:
535	#D-
536	function IGCM_sys_FileSize {
537	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
538
539	typeset sizeF
540	set +A sizeF -- $( ls -la ${1} )
541	if [ $? -gt 0 ] ; then
542	IGCM_debug_Exit "IGCM_sys_FileSize"
543	fi
544	eval ${2}=${sizeF[4]}
545
546	IGCM_debug_PopStack "IGCM_sys_FileSize"
547	}
548
549	#D-#==================================================
550	#D-function IGCM_sys_TestDir
551	#D-* Purpose: Test Directory that must exists
552	#D-* Examples:
553	#D-
554	function IGCM_sys_TestDir {
555	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
556	if ( $DEBUG_sys ) ; then
557	echo "IGCM_sys_TestDir :" $@
558	fi
559	typeset ExistFlag
560	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
561	IGCM_debug_PopStack "IGCM_sys_TestDir"
562
563	return ${ExistFlag}
564	}
565
566	#D-#==================================================
567	#D-function IGCM_sys_TestDirArchive
568	#D-* Purpose: Test Directory that must exists on Archive
569	#D-* Examples:
570	#D-
571	function IGCM_sys_TestDirArchive {
572	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
573	if ( $DEBUG_sys ) ; then
574	echo "IGCM_sys_TestDirArchive :" $@
575	fi
576	typeset ExistFlag
577	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
578	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
579	return ${ExistFlag}
580	}
581
582	#D-#==================================================
583	#D-function IGCM_sys_IsFileArchived
584	#D-* Purpose: Test file that must NOT EXISTS on Archive
585	#D-* Examples:
586	#D-
587	function IGCM_sys_IsFileArchived {
588	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
589	if ( $DEBUG_sys ) ; then
590	echo "IGCM_sys_IsFileArchived :" $@
591	fi
592	typeset IsArchivedFlag
593	IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
594	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
595
596	return ${IsArchivedFlag}
597	}
598
599	#D-#==================================================
600	#D-function IGCM_sys_TestFileArchive
601	#D-* Purpose: Test file that must NOT EXISTS on Archive
602	#D-* Examples:
603	#D-
604	function IGCM_sys_TestFileArchive {
605	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
606	typeset ExistFlag
607	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
608	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
609
610	return ${ExistFlag}
611	}
612
613	#D-#==================================================
614	#D-function IGCM_sys_TestFileBuffer
615	#D-* Purpose: Test file that must NOT EXISTS on Buffer
616	#D-* Examples:
617	#D-
618	function IGCM_sys_TestFileBuffer {
619	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
620	typeset ExistFlag
621	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
622	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
623
624	return ${ExistFlag}
625	}
626
627	#D-#==================================================
628	#D-function IGCM_sys_CountFileArchive
629	#D-* Purpose: Count files on Archive filesystem
630	#D-* Examples:
631	#D-
632	function IGCM_sys_CountFileArchive {
633	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
634	ls ${@} 2>/dev/null \| wc -l
635	if [ $? -gt 0 ] ; then
636	echo "IGCM_sys_CountFileArchive : erreur."
637	fi
638	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
639	}
640
641	#D-#==================================================
642	#D-function IGCM_sys_CountFileBuffer
643	#D-* Purpose: Count files on Scratch filesystem
644	#D-* Examples:
645	#D-
646	function IGCM_sys_CountFileBuffer {
647	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
648	ls ${@} 2>/dev/null \| wc -l
649	if [ $? -gt 0 ] ; then
650	echo "IGCM_sys_CountFileBuffer : erreur."
651	fi
652	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
653	}
654
655	#D-#==================================================
656	#D-function IGCM_sys_Tree
657	#D-* Purpose: Tree directories with files on ${ARCHIVE}
658	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
659	#D-
660	function IGCM_sys_Tree {
661	IGCM_debug_PushStack "IGCM_sys_Tree" $@
662	if ( $DEBUG_sys ) ; then
663	echo "IGCM_sys_Tree :" $@
664	fi
665
666	\ls -lR ${@}
667
668	IGCM_debug_PopStack "IGCM_sys_Tree"
669	}
670
671	#D-#==================================================
672	#D-function IGCM_sys_Tar
673	#D-* Purpose: master tar command
674	#D-* Examples:
675	#D-
676	function IGCM_sys_Tar {
677	IGCM_debug_PushStack "IGCM_sys_Tar" $@
678	if ( $DEBUG_sys ) ; then
679	echo "IGCM_sys_Tar :" $@
680	fi
681	\tar cf $@
682	if [ $? -gt 0 ] ; then
683	echo "IGCM_sys_Tar : erreur."
684	IGCM_debug_Exit "IGCM_sys_Tar"
685	fi
686	IGCM_debug_PopStack "IGCM_sys_Tar"
687	}
688
689	#D-#==================================================
690	#D-function IGCM_sys_UnTar
691	#D-* Purpose: master un-tar command
692	#D-* Examples:
693	#D-
694	function IGCM_sys_UnTar {
695	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
696	if ( $DEBUG_sys ) ; then
697	echo "IGCM_sys_UnTar :" $@
698	fi
699	\tar xvf $1
700	if [ $? -gt 0 ] ; then
701	echo "IGCM_sys_UnTar : erreur."
702	IGCM_debug_Exit "IGCM_sys_UnTar"
703	fi
704	IGCM_debug_PopStack "IGCM_sys_UnTar"
705	}
706
707	#D-#==================================================
708	#D-function IGCM_sys_Qsub
709	#D-* Purpose: Qsub new job
710	#D-* Examples:
711	#D-
712	function IGCM_sys_Qsub {
713	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
714	if ( $DEBUG_sys ) ; then
715	echo "IGCM_sys_Qsub :" $@
716	fi
717	typeset options status
718	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
719
720	/usr/bin/ccc_msub ${options} $1 > /tmp/out_command.$$ 2>&1
721	status=$?
722
723	cat /tmp/out_command.$$
724	if [ ${status} -gt 0 ] ; then
725	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
726	IGCM_debug_Exit "IGCM_sys_Qsub"
727	else
728	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
729	fi
730	IGCM_debug_PopStack "IGCM_sys_Qsub"
731	}
732
733	#D-#==================================================
734	#D-function IGCM_sys_QsubPost
735	#D-* Purpose: Qsub new job on scalaire
736	#D-* Examples:
737	#D-
738	function IGCM_sys_QsubPost {
739	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
740	if ( $DEBUG_sys ) ; then
741	echo "IGCM_sys_QsubPost :" $@
742	fi
743	typeset options status
744	MY_PROJECT=$(echo ${BRIDGE_MSUB_PROJECT:=${DEFAULT_PROJECT}} \| cut -d@ -f1 )
745
746	# EASIER TO DO THIS INSTEAD OF DUPLICATING libIGCM_sys_curie.ksh
747	case $( hostname -s ) in
748	curie*)
749	options="-Q normal -A ${MY_PROJECT} -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out";;
750	airain*)
751	options="-q ivybridge -A dsm -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
752	esac
753
754	/usr/bin/ccc_msub ${options} ${libIGCM_POST}/$1.job > /tmp/out_command.$$ 2>&1
755	status=$?
756
757	cat /tmp/out_command.$$
758	if [ ${status} -gt 0 ] ; then
759	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
760	IGCM_debug_Exit "IGCM_sys_QsubPost"
761	else
762	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
763	fi
764	IGCM_debug_PopStack "IGCM_sys_QsubPost"
765	}
766
767	#D-*************************
768	#D- File transfer functions
769	#D-*************************
770	#D-
771
772	#D-#==================================================
773	#D-function IGCM_sys_Rsync_out
774	#D-* Purpose: treat return val of rsync
775	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
776	#D- Error values and explanations can depend on your system version.
777	function IGCM_sys_Rsync_out {
778	status=$1
779	if [ ! $status ] ; then
780	echo "rsync error !"
781	fi
782
783	if [ $MYLANG = "fr" ]; then
784	case $status in
785	0) return ;;
786	1) echo "Erreur de rsync ; RERR_SYNTAX : "
787	echo "Erreur de syntaxe ou d'utilisation."
788	return;;
789	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
790	echo "Incompatibilité de protocole."
791	return;;
792	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
793	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
794	echo "répertoires"
795	return;;
796	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
797	echo "Action demandée non supportée : une tentative de manipulation de"
798	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
799	echo "été faite ; ou une option qui est supportée par le client mais"
800	echo "pas par le serveur a été spécifiée."
801	return;;
802	10) echo "Erreur de rsync ; RERR_SOCKETIO"
803	echo "Erreur dans le socket d'entrée sortie"
804	return;;
805	11) echo "Erreur de rsync ; RERR_FILEIO"
806	echo "Erreur d'entrée sortie fichier"
807	return;;
808	12) echo "Erreur de rsync ; RERR_STREAMIO"
809	echo "Erreur dans flux de donnée du protocole rsync"
810	return;;
811	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
812	echo "Erreur avec les diagnostics du programme"
813	return;;
814	14) echo "Erreur de rsync ; RERR_IPC"
815	echo "Erreur dans le code IPC"
816	return;;
817	20) echo "Erreur de rsync ; RERR_SIGNAL"
818	echo "SIGUSR1 ou SIGINT reçu"
819	return;;
820	21) echo "Erreur de rsync ; RERR_WAITCHILD"
821	echo "Une erreur retournée par waitpid()"
822	return;;
823	22) echo "Erreur de rsync ; RERR_MALLOC"
824	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
825	return;;
826	23) echo ""
827	echo "Erreur fichier inexistant"
828	return;;
829	30) echo "Erreur de rsync ; RERR_TIMEOUT"
830	echo "Temps d'attente écoulé dans l'envoi/réception de données"
831	return;;
832	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
833	return;;
834	esac
835	elif [ $MYLANG = "en" ] ; then
836	case $status in
837	0) return;;
838	1) echo "rsync error : Syntax or usage error "
839	return;;
840	2) echo "rsync error : Protocol incompatibility "
841	return;;
842	3) echo "rsync error : Errors selecting input/output files, dirs"
843	return;;
844	4) echo "rsync error : Requested action not supported: an attempt"
845	echo "was made to manipulate 64-bit files on a platform that cannot support"
846	echo "them; or an option was specified that is supported by the client and"
847	echo "not by the server."
848	return;;
849	5) echo "rsync error : Error starting client-server protocol"
850	return;;
851	10) echo "rsync error : Error in socket I/O "
852	return;;
853	11) echo "rsync error : Error in file I/O "
854	return;;
855	12) echo "rsync error : Error in rsync protocol data stream "
856	return;;
857	13) echo "rsync error : Errors with program diagnostics "
858	return;;
859	14) echo "rsync error : Error in IPC code "
860	return;;
861	20) echo "rsync error : Received SIGUSR1 or SIGINT "
862	return;;
863	21) echo "rsync error : Some error returned by waitpid() "
864	return;;
865	22) echo "rsync error : Error allocating core memory buffers "
866	return;;
867	23) echo "rsync error : Partial transfer due to error"
868	return;;
869	24) echo "rsync error : Partial transfer due to vanished source files"
870	return;;
871	30) echo "rsync error : Timeout in data send/receive "
872	return;;
873	*) echo "rsync error : return code of rsync unknown :" $status
874	return;;
875	esac
876	else
877	echo "unknown language $MYLANG."
878	return
879	fi
880	}
881
882	#D-#==================================================
883	#D-function IGCM_sys_Miror_libIGCM
884	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
885	#D-* Examples:
886	#D-
887	function IGCM_sys_Mirror_libIGCM {
888	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
889	if ( $DEBUG_sys ) ; then
890	echo "IGCM_sys_Mirror_libIGCM"
891	fi
892
893	typeset status
894
895	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
896
897	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
898	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
899	status=$?
900
901	if [ ${status} -gt 0 ] ; then
902	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
903	cat /tmp/out_command.$$
904	fi
905	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
906	}
907
908	#D-#==================================================
909	#D-function IGCM_sys_Cp
910	#D-* Purpose: generic cp
911	#D-* Examples:
912	#D-
913	function IGCM_sys_Cp {
914	IGCM_debug_PushStack "IGCM_sys_Cp" $@
915	if ( $DEBUG_sys ) ; then
916	echo "IGCM_sys_Cp :" $@
917	fi
918
919	typeset status
920
921	echo cp $@ > /tmp/out_command.$$ 2>&1
922	\cp $@ >> /tmp/out_command.$$ 2>&1
923	status=$?
924
925	if [ ${status} -gt 0 ] ; then
926	echo "IGCM_sys_Cp : error code ${status}"
927	cat /tmp/out_command.$$
928	IGCM_debug_Exit "IGCM_sys_Cp"
929	else
930	\rm /tmp/out_command.$$
931	fi
932	IGCM_debug_PopStack "IGCM_sys_Cp"
933	}
934
935	#D-#==================================================
936	#D-function IGCM_sys_Rm
937	#D-* Purpose: generic rm
938	#D-* Examples:
939	#D-
940	function IGCM_sys_Rm {
941	IGCM_debug_PushStack "IGCM_sys_Rm" $@
942	if ( $DEBUG_sys ) ; then
943	echo "IGCM_sys_Rm :" $@
944	fi
945
946	typeset status
947
948	echo rm $@ > /tmp/out_command.$$ 2>&1
949	\rm $@ >> /tmp/out_command.$$ 2>&1
950	status=$?
951
952	if [ ${status} -gt 0 ] ; then
953	echo "IGCM_sys_Rm : error code ${status}"
954	cat /tmp/out_command.$$
955	IGCM_debug_Exit "IGCM_sys_Rm"
956	else
957	\rm /tmp/out_command.$$
958	fi
959	IGCM_debug_PopStack "IGCM_sys_Rm"
960	}
961
962	#D-#==================================================
963	#D-function IGCM_sys_RmRunDir
964	#D-* Purpose: rm tmpdir (dummy function most of the time batch
965	#D- scheduler will do the job)
966	#D-* Examples:
967	#D-
968	function IGCM_sys_RmRunDir {
969	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
970	if ( $DEBUG_sys ) ; then
971	echo "IGCM_sys_RmRunDir :" $@
972	fi
973
974	typeset status
975
976	echo rm $@ > /tmp/out_command.$$ 2>&1
977	\rm $@ >> /tmp/out_command.$$ 2>&1
978	status=$?
979
980	if [ ${status} -gt 0 ] ; then
981	echo "IGCM_sys_RmRunDir : rm error code is ${status}."
982	cat /tmp/out_command.$$
983	IGCM_debug_Exit "IGCM_sys_RmRunDir"
984	else
985	\rm /tmp/out_command.$$
986	fi
987	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
988	}
989
990	#D-#==================================================
991	#D-function IGCM_sys_Mv
992	#D-* Purpose: generic move
993	#D-* Examples:
994	#D-
995	function IGCM_sys_Mv {
996	IGCM_debug_PushStack "IGCM_sys_Mv" $@
997	if ( $DEBUG_sys ) ; then
998	echo "IGCM_sys_Mv :" $@
999	fi
1000
1001	if [ $DRYRUN = 0 ]; then
1002
1003	typeset status
1004
1005	echo mv $@ > /tmp/out_command.$$ 2>&1
1006	\mv $@ >> /tmp/out_command.$$ 2>&1
1007	status=$?
1008
1009	if [ ${status} -gt 0 ] ; then
1010	echo "IGCM_sys_Mv : error code ${status}"
1011	cat /tmp/out_command.$$
1012	IGCM_debug_Exit "IGCM_sys_Mv"
1013	else
1014	\rm /tmp/out_command.$$
1015	fi
1016	fi
1017
1018	IGCM_debug_PopStack "IGCM_sys_Mv"
1019	}
1020
1021	#D-#==================================================
1022	#D-function IGCM_sys_Put_Dir
1023	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1024	#D-* Examples:
1025	#D-
1026	function IGCM_sys_Put_Dir {
1027	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1028	if ( $DEBUG_sys ) ; then
1029	echo "IGCM_sys_Put_Dir :" $@
1030	fi
1031	if [ $DRYRUN = 0 ]; then
1032	if [ ! -d ${1} ] ; then
1033	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1034	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1035	return
1036	fi
1037
1038	typeset status
1039
1040	# Only if we use rsync
1041	#IGCM_sys_TestDirArchive $( dirname $2 )
1042	#
1043	#USUAL WAY
1044	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
1045	status=$?
1046
1047	if [ ${status} -gt 0 ] ; then
1048	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
1049	cat /tmp/out_command.$$
1050	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1051	else
1052	\rm /tmp/out_command.$$
1053	fi
1054	fi
1055	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1056	}
1057
1058	#D-#==================================================
1059	#D-function IGCM_sys_Get_Dir
1060	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1061	#D-* Examples:
1062	#D-
1063	function IGCM_sys_Get_Dir {
1064	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1065	if ( $DEBUG_sys ) ; then
1066	echo "IGCM_sys_Get_Dir :" $@
1067	fi
1068	if [ $DRYRUN = 0 ]; then
1069	# if [ ! -d ${1} ] ; then
1070	# echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
1071	# IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1072	# return
1073	# fi
1074
1075	typeset NB_ESSAI DELAI status i
1076	# number of tentative
1077	NB_ESSAI=3
1078	# time delay between tentative
1079	DELAI=2
1080
1081	# Only if we use rsync
1082	#IGCM_sys_TestDirArchive $( dirname $2 )
1083	#
1084	# USUAL WAY
1085	# add 'ccc_hsm get' (to demigrate all offline files) to reduce time of this command :
1086	ccc_hsm get -r $1
1087
1088	i=0
1089	while [ $i -lt $NB_ESSAI ] ; do
1090	\cp -ur $1 $2 >> /tmp/out_command.$$ 2>&1
1091	status=$?
1092	if [ ${status} -gt 0 ] ; then
1093	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
1094	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1095	sleep $DELAI
1096	else
1097	break
1098	fi
1099	(( i = i + 1 ))
1100	done
1101
1102	if [ ${status} -gt 0 ] ; then
1103	echo "IGCM_sys_Get_Dir : error."
1104	cat /tmp/out_command.$$
1105	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1106	else
1107	\rm /tmp/out_command.$$
1108	fi
1109	fi
1110	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1111	}
1112
1113	#D-#==================================================
1114	#D-function IGCM_sys_Get_Master
1115	#D-* Purpose: Copy a complete directory from MASTER filesystem
1116	#D-* Examples:
1117	#D-
1118	function IGCM_sys_Get_Master {
1119	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1120	if ( $DEBUG_sys ) ; then
1121	echo "IGCM_sys_Get_Master :" $@
1122	fi
1123	if [ $DRYRUN = 0 ]; then
1124	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1125	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1126	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1127	return
1128	fi
1129
1130	typeset NB_ESSAI DELAI status i
1131	# number of tentative
1132	NB_ESSAI=3
1133	# time delay between tentative
1134	DELAI=2
1135
1136	i=0
1137	while [ $i -lt $NB_ESSAI ] ; do
1138	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
1139	status=$?
1140	if [ ${status} -gt 0 ]; then
1141	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1142	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1143	sleep $DELAI
1144	else
1145	break
1146	fi
1147	(( i = i + 1 ))
1148	done
1149
1150	if [ ${status} -gt 0 ] ; then
1151	echo "IGCM_sys_Get_Master : error."
1152	cat /tmp/out_command.$$
1153	IGCM_debug_Exit "IGCM_sys_Get_Master"
1154	else
1155	\rm /tmp/out_command.$$
1156	fi
1157	fi
1158	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1159	}
1160
1161	#====================================================
1162	#- Call IGCM_sys_Mirror_libIGCM now !
1163	if ( $MirrorlibIGCM ) ; then
1164	IGCM_sys_Mirror_libIGCM
1165	fi
1166
1167	#D-#==================================================
1168	#D-function IGCM_sys_Put_Rest
1169	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1170	#D- File and target directory must exist.
1171	#D-* Examples:
1172	#D-
1173	function IGCM_sys_Put_Rest {
1174	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1175	if ( $DEBUG_sys ) ; then
1176	echo "IGCM_sys_Put_Rest :" $@
1177	fi
1178	if [ $DRYRUN = 0 ]; then
1179	if [ ! -f ${1} ] ; then
1180	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1181	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1182	fi
1183
1184	typeset status
1185	#
1186	# USUAL WAY
1187	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1188	status=$?
1189
1190	# #RSYNC WITH NETWORK SSH CALL
1191	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1192	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1193
1194	# #RSYNC WITH NFS USE
1195	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1196	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1197
1198	# status=$?
1199	# IGCM_sys_Rsync_out $status
1200
1201	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1202	# (( status=status+$? ))
1203
1204	if [ ${status} -gt 0 ] ; then
1205	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
1206	[ -f ${1} ] && ls -l ${1}
1207	[ -f ${2} ] && ls -l ${2}
1208	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1209	cat /tmp/out_command.$$
1210	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1211	else
1212
1213	if [ X${JobType} = XRUN ] ; then
1214	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1215	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1216	fi
1217
1218	\rm /tmp/out_command.$$
1219	fi
1220	fi
1221	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1222	}
1223
1224	#D-#==================================================
1225	#D-function IGCM_sys_PutBuffer_Rest
1226	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1227	#D- File and target directory must exist.
1228	#D-* Examples:
1229	#D-
1230	function IGCM_sys_PutBuffer_Rest {
1231	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1232	if ( $DEBUG_sys ) ; then
1233	echo "IGCM_sys_PutBuffer_Rest :" $@
1234	fi
1235	if [ $DRYRUN = 0 ]; then
1236	if [ ! -f ${1} ] ; then
1237	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1238	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1239	fi
1240
1241	typeset status
1242	#
1243	# USUAL WAY
1244	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1245	status=$?
1246
1247	if [ ${status} -gt 0 ] ; then
1248	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1249	[ -f ${2} ] && ls -l ${2}
1250	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1251	cat /tmp/out_command.$$
1252	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1253	else
1254
1255	if [ X${JobType} = XRUN ] ; then
1256	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1257	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1258	fi
1259
1260	\rm /tmp/out_command.$$
1261	fi
1262	fi
1263	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1264	}
1265
1266	#D-#==================================================
1267	#D-function IGCM_sys_PrepareTaredRestart
1268	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
1269	#D-* Examples:
1270	#D-
1271	function IGCM_sys_PrepareTaredRestart {
1272	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1273	if [ $DRYRUN = 0 ]; then
1274	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1275	fi
1276	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1277	}
1278
1279	#D-#==================================================
1280	#D-function IGCM_sys_Put_Out
1281	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1282	#D-* Examples:
1283	#D-
1284	function IGCM_sys_Put_Out {
1285	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1286	if ( $DEBUG_sys ) ; then
1287	echo "IGCM_sys_Put_Out :" $@
1288	fi
1289
1290	typeset NB_ESSAI DELAI status i exist skip
1291
1292	# number of tentative
1293	NB_ESSAI=3
1294	# time delay between tentative
1295	DELAI=2
1296
1297	if [ $DRYRUN = 0 ]; then
1298	if [ ! -f ${1} ] ; then
1299	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1300	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1301	return 1
1302	fi
1303	#
1304	IGCM_sys_MkdirArchive $( dirname $2 )
1305	#
1306
1307	#=====================================================
1308	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1309	#=====================================================
1310
1311	#echo ${2} \| grep "${R_OUT}" > /dev/null 2>&1
1312	#if [ $? -eq 0 ] ; then
1313	# typeset WORKPATH FILEPATH
1314	# WORKPATH=$( dirname $2 \| sed -e "s\|${R_OUT}\|${R_BUF}\|" )
1315	# IGCM_sys_MkdirWork ${WORKPATH}
1316	# FILEPATH=${WORKPATH}/$( basename $2 )
1317	# #
1318	# IGCM_sys_Cp ${1} ${FILEPATH}
1319	#fi
1320
1321	exist=false
1322	skip=false
1323	if [ -f $2 ] ; then
1324	IGCM_debug_Print 1 "$2 already exist"
1325	ccc_hsm get $2
1326	exist=true
1327	if [ "X$( diff $1 $2 )" = X ] ; then
1328	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1329	skip=true
1330	else
1331	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1332	skip=false
1333	fi
1334	fi
1335	#
1336	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1337	IGCM_sys_Chmod u+w $2
1338	fi
1339
1340	if [ X${skip} = Xfalse ] ; then
1341	i=0
1342	while [ $i -lt $NB_ESSAI ] ; do
1343	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1344	# USUAL WAY
1345	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1346	status=$?
1347	else
1348	# NOT SO USUAL WAY
1349	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1350	status=$?
1351	fi
1352	if [ ${status} -gt 0 ]; then
1353	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1354	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
1355	[ -f ${1} ] && ls -l ${1}
1356	[ -f ${2} ] && ls -l ${2}
1357	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1358	sleep $DELAI
1359	else
1360	break
1361	fi
1362	(( i = i + 1 ))
1363	done
1364	fi
1365
1366	# #RSYNC WITH NETWORK SSH CALL
1367	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1368	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1369
1370	# #RSYNC WITH NFS USE
1371	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1372	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1373
1374	# status=$?
1375	# IGCM_sys_Rsync_out $status
1376
1377	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1378	# (( status=status+$? ))
1379
1380	if [ ${status} -gt 0 ] ; then
1381	echo "IGCM_sys_Put_Out : error."
1382	[ -f ${1} ] && ls -l ${1}
1383	[ -f ${2} ] && ls -l ${2}
1384	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1385	cat /tmp/out_command.$$
1386	IGCM_debug_Exit "IGCM_sys_Put_Out"
1387	else
1388
1389	if [ X${JobType} = XRUN ] ; then
1390	if [ X${3} = X ] ; then
1391	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1392	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1393	fi
1394	fi
1395
1396	\rm /tmp/out_command.$$
1397	fi
1398	fi
1399	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1400	return 0
1401	}
1402
1403	#D-#==================================================
1404	#D-function IGCM_sys_PutBuffer_Out
1405	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1406	#D-* Examples:
1407	#D-
1408	function IGCM_sys_PutBuffer_Out {
1409	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1410	if ( $DEBUG_sys ) ; then
1411	echo "IGCM_sys_PutBuffer_Out :" $@
1412	fi
1413
1414	typeset NB_ESSAI DELAI status i exist skip
1415
1416	# number of tentative
1417	NB_ESSAI=3
1418	# time delay between tentative
1419	DELAI=2
1420
1421	if [ $DRYRUN = 0 ]; then
1422	if [ ! -f ${1} ] ; then
1423	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1424	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1425	return 1
1426	fi
1427	#
1428	IGCM_sys_Mkdir $( dirname $2 )
1429	#
1430
1431	exist=false
1432	skip=false
1433	if [ -f $2 ] ; then
1434	IGCM_debug_Print 1 "$2 already exist"
1435	exist=true
1436	if [ "X$( diff $1 $2 )" = X ] ; then
1437	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1438	status=0
1439	skip=true
1440	else
1441	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1442	skip=false
1443	fi
1444	fi
1445	#
1446	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1447	IGCM_sys_Chmod u+w $2
1448	fi
1449
1450	if [ X${skip} = Xfalse ] ; then
1451	i=0
1452	while [ $i -lt $NB_ESSAI ] ; do
1453	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1454	# USUAL WAY
1455	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1456	status=$?
1457	else
1458	# NOT SO USUAL WAY
1459	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1460	status=$?
1461	fi
1462	if [ ${status} -gt 0 ]; then
1463	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1464	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1465	[ -f ${2} ] && ls -l ${2}
1466	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1467	sleep $DELAI
1468	else
1469	break
1470	fi
1471	(( i = i + 1 ))
1472	done
1473	fi
1474
1475	if [ ${status} -gt 0 ] ; then
1476	echo "IGCM_sys_PutBuffer_Out : error."
1477	[ -f ${2} ] && ls -l ${2}
1478	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1479	cat /tmp/out_command.$$
1480	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1481	else
1482
1483	if [ X${JobType} = XRUN ] ; then
1484	if [ X${3} = X ] ; then
1485	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1486	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1487	fi
1488	fi
1489
1490	\rm /tmp/out_command.$$
1491	fi
1492	fi
1493	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1494	return 0
1495	}
1496
1497	#D-#==================================================
1498	#D-function IGCM_sys_Get
1499	#D-* Purpose: Get a file from ${ARCHIVE}
1500	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1501	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1502	function IGCM_sys_Get {
1503	IGCM_debug_PushStack "IGCM_sys_Get" $@
1504
1505	typeset DEST dm_liste target file_work
1506	typeset NB_ESSAI DELAI status i
1507
1508	if ( $DEBUG_sys ) ; then
1509	echo "IGCM_sys_Get :" $@
1510	fi
1511
1512	# number of tentative
1513	NB_ESSAI=3
1514	# time delay between tentative
1515	DELAI=2
1516
1517	if [ $DRYRUN -le 2 ]; then
1518	if [ X${1} = X'/l' ] ; then
1519	eval set +A dm_liste \${${2}}
1520	else
1521	eval set +A dm_liste ${1}
1522	fi
1523	eval DEST=\${${#}}
1524
1525	#=====================================================
1526	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1527	#=====================================================
1528
1529	# Is it an R_OUT file (not R_IN) ?
1530	#echo ${dm_liste[0]} \| grep "${R_OUT}" > /dev/null 2>&1
1531	#if [ $? -eq 0 ] ; then
1532	# # Yes ? then we try to get it in SCRATCHDIR
1533	# set +A file_work $( echo ${dm_liste[*]} \| sed -e "s\|${R_OUT}\|${R_BUF}\|g" )
1534	# if [ -f ${file_work[0]} ] ; then
1535	# IGCM_sys_Cp ${file_work[*]} ${DEST}
1536	# IGCM_debug_PopStack "IGCM_sys_Get"
1537	# return
1538	# fi
1539	#fi
1540
1541	ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1542	status=$?
1543	if [ ${status} -gt 0 ] ; then
1544	echo "WARNING IGCM_sys_Get : error code ${status}"
1545	cat /tmp/out_command.$$
1546	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
1547	fi
1548
1549	#if [ ${status} -gt 0 ] ; then
1550	# if [ ! "X$( grep "Lost dmusrcmd connection" /tmp/out_command.$$ )" = "X" ] ; then
1551	# cat /tmp/out_command.$$
1552	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
1553	# sleep 30
1554	# echo "We try another time"
1555	## dmget ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1556	# ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1557	# status=$?
1558	# if [ ${status} -gt 0 ] ; then
1559	# echo "ERROR IGCM_sys_Get : again demigration error :"
1560	# cat /tmp/out_command.$$
1561	# IGCM_debug_Exit "IGCM_sys_Get"
1562	# fi
1563	# else
1564	# echo "ERROR IGCM_sys_Get : demigration error :"
1565	# cat /tmp/out_command.$$
1566	# IGCM_debug_Exit "IGCM_sys_Get"
1567	# fi
1568	#fi
1569
1570	# #RSYNC WITH NETWORK SSH CALL
1571	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1572	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1573
1574	# #RSYNC WITH NFS USE
1575	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1576	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1577
1578	# status=$?
1579	# IGCM_sys_Rsync_out $status
1580
1581	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1582	# (( status=status+$? ))
1583
1584	#USUAL WAY
1585	if [ X${1} = X'/l' ] ; then
1586	for target in ${dm_liste[*]} ; do
1587	local_file=$( basename ${target} )
1588	# test if the target file is present before the loop
1589	IGCM_sys_TestFileArchive ${target}
1590	status=$?
1591	if [ ${status} -gt 0 ] ; then
1592	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
1593	IGCM_debug_Exit "IGCM_sys_Get"
1594	else
1595	i=0
1596	while [ $i -lt $NB_ESSAI ] ; do
1597	#if [ X${DoLink} = Xtrue ] ; then
1598	# \ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1599	# status=$?
1600	# else
1601	# \cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1602	# status=$?
1603	#fi
1604	\ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1605	status=$?
1606	if [ ${status} -gt 0 ]; then
1607	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1608	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1609	sleep $DELAI
1610	else
1611	break
1612	fi
1613	(( i = i + 1 ))
1614	done
1615	if [ ${status} -gt 0 ] ; then
1616	echo "IGCM_sys_Get : error"
1617	cat /tmp/out_command.$$
1618	\rm /tmp/out_command.$$
1619	IGCM_debug_Exit "IGCM_sys_Get"
1620	else
1621	\rm /tmp/out_command.$$
1622	fi
1623	fi
1624	done
1625	else
1626	i=0
1627	while [ $i -lt $NB_ESSAI ] ; do
1628	\cp ${dm_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1629	status=$?
1630	if [ ${status} -gt 0 ]; then
1631	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1632	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1633	sleep $DELAI
1634	else
1635	break
1636	fi
1637	(( i = i + 1 ))
1638	done
1639	if [ ${status} -gt 0 ] ; then
1640	echo "IGCM_sys_Get : error"
1641	cat /tmp/out_command.$$
1642	\rm /tmp/out_command.$$
1643	IGCM_debug_Exit "IGCM_sys_Get"
1644	else
1645	\rm /tmp/out_command.$$
1646	fi
1647	fi
1648	fi
1649	IGCM_debug_PopStack "IGCM_sys_Get"
1650	}
1651
1652	#D-#==================================================
1653	#D-function IGCM_sys_GetBuffer
1654	#D-* Purpose: Get a file from ${SCRATCHDIR}
1655	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1656	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1657	function IGCM_sys_GetBuffer {
1658	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1659
1660	typeset DEST buf_liste target file_work
1661	typeset NB_ESSAI DELAI status i
1662
1663	if ( $DEBUG_sys ) ; then
1664	echo "IGCM_sys_GetBuffer :" $@
1665	fi
1666
1667	# number of tentative
1668	NB_ESSAI=3
1669	# time delay between tentative
1670	DELAI=2
1671
1672	if [ $DRYRUN -le 2 ]; then
1673	if [ X${1} = X'/l' ] ; then
1674	eval set +A buf_liste \${${2}}
1675	else
1676	eval set +A buf_liste ${1}
1677	fi
1678	eval DEST=\${${#}}
1679
1680	#USUAL WAY
1681	if [ X${1} = X'/l' ] ; then
1682	for target in ${buf_liste[*]} ; do
1683	local_file=$( basename ${target} )
1684	i=0
1685	while [ $i -lt $NB_ESSAI ] ; do
1686	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1687	status=$?
1688	if [ ${status} -gt 0 ]; then
1689	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1690	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1691	sleep $DELAI
1692	else
1693	break
1694	fi
1695	(( i = i + 1 ))
1696	done
1697	if [ ${status} -gt 0 ] ; then
1698	echo "IGCM_sys_Get : error"
1699	cat /tmp/out_command.$$
1700	\rm /tmp/out_command.$$
1701	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1702	else
1703	\rm /tmp/out_command.$$
1704	fi
1705	done
1706	else
1707	i=0
1708	while [ $i -lt $NB_ESSAI ] ; do
1709	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1710	status=$?
1711	if [ ${status} -gt 0 ]; then
1712	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1713	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1714	sleep $DELAI
1715	else
1716	break
1717	fi
1718	(( i = i + 1 ))
1719	done
1720	if [ ${status} -gt 0 ] ; then
1721	echo "IGCM_sys_Get : error"
1722	cat /tmp/out_command.$$
1723	\rm /tmp/out_command.$$
1724	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1725	else
1726	\rm /tmp/out_command.$$
1727	fi
1728	fi
1729	fi
1730	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1731	}
1732
1733	#D-#==================================================
1734	#D-function IGCM_sys_GetDate_FichWork
1735	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1736	#D-* Examples:
1737	#D-
1738	function IGCM_sys_GetDate_FichWork {
1739	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1740	if ( $DEBUG_sys ) ; then
1741	echo "IGCM_sys_GetDate_FichWork :" $@
1742	fi
1743
1744	if [ $# -ge 3 ] ; then
1745	mode=$3
1746	TimeStyle=$4
1747	else
1748	mode="default"
1749	TimeStyle="%Y%m%d%H%M%S"
1750	fi
1751
1752	typeset dateF
1753	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1754
1755	case $mode in
1756	"default")
1757	eval ${2}=${dateF[5]}
1758	;;
1759	"SplitFields")
1760	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1761	;;
1762	esac
1763
1764
1765	# donne la date filesys d'un fichier sur la machine work
1766	IGCM_debug_PopStack "IGCM_sys_FichWork"
1767	}
1768
1769	#D-#==================================================
1770	#D-function IGCM_sys_GetDate_FichArchive
1771	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1772	#D-* Examples:
1773	#D-
1774	function IGCM_sys_GetDate_FichArchive {
1775	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1776	if ( $DEBUG_sys ) ; then
1777	echo "IGCM_sys_GetDate_FichArchive :" $@
1778	fi
1779	typeset dateF
1780	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1781	eval ${2}=${dateF[5]}
1782
1783	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1784	}
1785
1786	#D-#==================================================
1787	#D-function IGCM_sys_GetDate_Monitoring
1788	#D-* Purpose: get the last year for which the monitoring has been computed
1789	#D-* Examples:
1790	#D-
1791	function IGCM_sys_GetDate_Monitoring {
1792	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1793	if ( $DEBUG_sys ) ; then
1794	echo "IGCM_sys_GetDate_Monitoring :" $@
1795	fi
1796
1797	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1798
1799	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1800	}
1801
1802	#D-#==================================================
1803	#D-function IGCM_sys_Dods_Rm
1804	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1805	#D-* Examples:
1806	#D-
1807	function IGCM_sys_Dods_Rm {
1808	if ( $DEBUG_sys ) ; then
1809	echo "IGCM_sys_Dods_Rm :" $@
1810	fi
1811	typeset status
1812	if [ $DRYRUN = 0 ]; then
1813
1814	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
1815	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
1816	# echo "Nothing has been done."
1817	# return
1818	# fi
1819
1820	/ccc/cont003/home/dsm/p86ipsl/bin/dods_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
1821	status=$?
1822
1823	# if [ ${status} -gt 0 ] ; then
1824	# echo "IGCM_sys_Dods_Rm : error."
1825	# cat out_dods_rm
1826	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
1827	# else
1828	# rm out_dods_rm
1829	# fi
1830
1831	fi
1832	return $status
1833	}
1834
1835	#D-#==================================================
1836	#D-function IGCM_sys_Dods_Cp
1837	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1838	#D-* Examples:
1839	#D-
1840	function IGCM_sys_Dods_Cp {
1841	if ( $DEBUG_sys ) ; then
1842	echo "IGCM_sys_Dods_Cp :" $@
1843	fi
1844	typeset status
1845	if [ $DRYRUN = 0 ]; then
1846
1847	# if [ ! -d ${R_SAVE}/${1} ] ; then
1848	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
1849	# echo "Nothing has been done."
1850	# return
1851	# fi
1852
1853	/ccc/cont003/home/dsm/p86ipsl/bin/dods_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
1854	status=$?
1855
1856	# if [ ${status} -gt 0 ] ; then
1857	# echo "IGCM_sys_Dods_Cp : error."
1858	# cat out_dods_cp
1859	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
1860	# else
1861	# rm out_dods_cp
1862	# fi
1863
1864	fi
1865	return $status
1866	}
1867
1868	#D-#==================================================
1869	#D-function IGCM_sys_Put_Dods
1870	#D-* Purpose: Put $(ARCHIVE) files on DODS internet protocole.
1871	#D-* Examples:
1872	#D-
1873	function IGCM_sys_Put_Dods {
1874	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1875	if ( $DEBUG_sys ) ; then
1876	echo "IGCM_sys_Put_Dods :" $@
1877	fi
1878	#set -vx
1879	typeset status
1880	if [ $DRYRUN = 0 ]; then
1881	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
1882	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
1883	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
1884	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
1885	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1886	return
1887	fi
1888
1889	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1890	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1891	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1892	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1893	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1894	return
1895	fi
1896	#
1897	if [ -d ${R_SAVE}/${1} ] ; then
1898	cd ${R_SAVE}
1899	elif [ -d ${R_FIGR}/${1} ] ; then
1900	cd ${R_FIGR}
1901	fi
1902
1903	IGCM_sys_Dods_Rm ${1}
1904	IGCM_sys_Dods_Cp ${1}
1905	status=0
1906
1907	if [ ${status} -gt 0 ] ; then
1908	echo "IGCM_sys_Put_Dods : error."
1909	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1910	fi
1911	fi
1912	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1913	}
1914
1915	##############################################################
1916	# REBUILD OPERATOR
1917
1918	#D-#==================================================
1919	#D-function IGCM_sys_rebuild
1920	#D-* Purpose: rebuild parallel files
1921	#D-* Examples:
1922	#D-
1923	function IGCM_sys_rebuild {
1924	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1925	if ( $DEBUG_sys ) ; then
1926	echo "IGCM_sys_rebuild :" $@
1927	fi
1928
1929	typeset NB_ESSAI DELAI status i firstArg
1930	# number of tentative
1931	NB_ESSAI=3
1932	# time delay between tentative
1933	DELAI=2
1934
1935	i=0
1936	while [ $i -lt $NB_ESSAI ] ; do
1937	~p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1938	status=$?
1939	if [ ${status} -gt 0 ] ; then
1940	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1941	cat /tmp/out_command.$$
1942	\rm /tmp/out_command.$$
1943	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1944	firstArg=${1}
1945	\rm ${firstArg}
1946	sleep $DELAI
1947	else
1948	\rm /tmp/out_command.$$
1949	break
1950	fi
1951	(( i = i + 1 ))
1952	done
1953
1954	if [ ${status} -gt 0 ] ; then
1955	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1956	IGCM_debug_Exit "rebuild"
1957	fi
1958
1959	IGCM_debug_PopStack "IGCM_sys_rebuild"
1960	}
1961
1962	#D-#==================================================
1963	#D-function IGCM_sys_rebuild_station
1964	#D-* Purpose: rebuild parallel files describing station
1965	#D-* Examples:
1966	#D-
1967	function IGCM_sys_rebuild_station {
1968	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1969	typeset i list_opt file_in file_out prefix_invert list_invert
1970	if ( $DEBUG_sys ) ; then
1971	echo "IGCM_sys_rebuild_station :" $@
1972	fi
1973	list_opt=$@
1974
1975	# Invert Axis : t,x -> x,t
1976	# t,pres,x -> x,t,pres
1977	# So that we can concatenate along x
1978	i=0
1979	for file_in in ${list_opt} ; do
1980	(( i = i + 1))
1981	[ ${i} = 1 ] && file_out=${file_in} && continue
1982	# detect time counter and do the job only if present
1983	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1)
1984	if [ X${var_unlim} = Xtime_counter ] ; then
1985	prefix_invert=$( basename ${file_in} .nc )
1986	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1987	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1988	fi
1989	done
1990
1991	# Concatenate
1992	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1993
1994	# Re-ivert file
1995	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1996
1997	# Station re-ordering is too expansive to be run within libIGCM
1998	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
1999	# This re-ordering must be done "in memory" by the cmorization process
2000	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
2001	# BEGIN reordering
2002
2003	# Only LMDZ text output contains the exact ordering of the station.
2004	# We isolate this in the code below:
2005	# 0 38 -157.5000000000000 70.98591549295774
2006	# 0 54 27.49999999999999 67.18309859154928
2007	# 0 56 -62.50000000000001 82.39436619718309
2008	# 0 79 12.49999999999999 78.59154929577466
2009	# 0 116 -165.0000000000000 76.05633802816901
2010	# 0 117 130.0000000000000 70.98591549295774
2011	# 0 118 110.0000000000000 87.46478873239437
2012	# 1 40 4.999999999999995 51.97183098591550
2013
2014	# typeset iStation iProc list_opt file_in file_out prefix_invert
2015	# typeset -Z4 j4
2016	# typeset -Z3 j3
2017
2018	# unset list_opt
2019	# set +A list_opt $@
2020
2021	# Filename after rebuild
2022	# file_out=${list_opt[0]}
2023	# Prefix of output files
2024	# prefix_invert=$( basename ${file_out} .nc )
2025	# Number of procs
2026	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
2027
2028	# iProc=0
2029	# while [ ${iProc} -lt ${num_proc} ] ; do
2030	# Array containing Station as a number
2031	# unset proc_stn
2032	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
2033	# Number of stations produced by processor proc
2034	# stationLast=${#proc_stn[*]}
2035	# Proc number on 4 digits
2036	# j4=${iProc}
2037	# Init
2038	# iStation=0
2039	# while [ ${iStation} -lt ${stationLast} ] ; do
2040	# Station number on 3 digits
2041	# j3=${proc_stn[${iStation}]}
2042	# Extract station
2043	# Invert Axis : t,x -> x,t
2044	# t,pres,x -> x,t,pres
2045	# So that we can concatenate along x
2046	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
2047	# (( iStation = iStation + 1 ))
2048	# done
2049	# (( iProc = iProc + 1 ))
2050	# done
2051
2052	# Concatenate all station along x
2053	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
2054
2055	# Re-invert file
2056	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
2057
2058	# END reordering
2059
2060	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
2061	}
2062
2063	############################################################
2064	# Activate Running Environnment Variables
2065
2066	#D-#==================================================
2067	#D-function IGCM_sys_desactiv_variables
2068	#D-* Purpose: set environement variables prior to execution
2069	#D-* Examples:
2070	#D-
2071	function IGCM_sys_activ_variables {
2072	IGCM_debug_PushStack "IGCM_sys_activ_variables"
2073	if ( $DEBUG_sys ) ; then
2074	echo "IGCM_sys_activ_variables"
2075	fi
2076
2077	# --------------------------------------------------------------------
2078	#D- MPI specifications
2079	# --------------------------------------------------------------------
2080
2081	# --------------------------------------------------------------------
2082	#D- Other specifications
2083	# --------------------------------------------------------------------
2084
2085	ulimit -s unlimited
2086
2087	IGCM_debug_PopStack "IGCM_sys_activ_variables"
2088	}
2089
2090	############################################################
2091	# Desactivate Running Environnment Variables
2092
2093	#D-#==================================================
2094	#D-function IGCM_sys_desactiv_variables
2095	#D-* Purpose: unset environement variables after execution
2096	#D-* Examples:
2097	#D-
2098	function IGCM_sys_desactiv_variables {
2099	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
2100	if ( $DEBUG_sys ) ; then
2101	echo "IGCM_sys_desactiv_variables"
2102	fi
2103	# --------------------------------------------------------------------
2104	#D- MPI specifications
2105	# --------------------------------------------------------------------
2106
2107	# --------------------------------------------------------------------
2108	#D- Other specifications
2109	# --------------------------------------------------------------------
2110
2111	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
2112	}
2113
2114	############################################################
2115	# Build MPI/OMP scripts run file (dummy function)
2116
2117	#D-#==================================================
2118	#D-function IGCM_sys_build_run_file
2119	#D-* Purpose: build run file (deprecated)
2120	#D-* Examples:
2121	#D-
2122	function IGCM_sys_build_run_file {
2123
2124	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
2125
2126	}
2127
2128	############################################################
2129	# Build MPI/OMP scripts
2130
2131	#D-#==================================================
2132	#D-function IGCM_sys_build_execution_scripts
2133	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
2134	#D-* Examples:
2135	#D-
2136	function IGCM_sys_build_execution_scripts
2137	{
2138	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
2139	if ( $DEBUG_sys ) ; then
2140	echo "IGCM_sys_build_execution_scripts " $@
2141	fi
2142
2143	EXECUTION=${HOST_MPIRUN_COMMAND}
2144
2145	# MPMD mode
2146
2147	if ( ${OK_PARA_MPMD} ) ; then
2148
2149	# Only MPI (MPMD)
2150	if ( ! ${OK_PARA_OMP} ) ; then
2151
2152	if [ -f run_file ] ; then
2153	IGCM_sys_Rm -f run_file
2154	fi
2155	touch run_file
2156
2157	# Build run_file
2158
2159	# First loop on the components for the coupler ie oasis (only if oasis3)
2160
2161	## the coupler ie oasis3 must be the first one
2162	for comp in ${config_ListOfComponents[*]} ; do
2163
2164	eval ExeNameIn=\${config_Executable_${comp}[0]}
2165	eval ExeNameOut=\${config_Executable_${comp}[1]}
2166
2167	# for CPL component only
2168	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
2169
2170	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2171	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2172	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2173	fi
2174	done
2175
2176	# Then second loop on the components
2177
2178	for comp in ${config_ListOfComponents[*]} ; do
2179
2180	eval ExeNameIn=\${config_Executable_${comp}[0]}
2181	eval ExeNameOut=\${config_Executable_${comp}[1]}
2182
2183	# Only if we really have an executable for the component and not the coupler ie oasis:
2184	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2185
2186	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2187	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2188	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2189	fi
2190	done
2191
2192	EXECUTION="${HOST_MPIRUN_COMMAND} -f ./run_file"
2193
2194	IGCM_sys_Chmod u+x run_file
2195	if ( $DEBUG_sys ) ; then
2196	echo "run_file contains : "
2197	cat run_file
2198	fi
2199
2200	# MPI-OpenMP (MPMD)
2201	else
2202
2203	# Use of mpirun instead of ccc_mprun
2204	EXECUTION="time mpirun"
2205
2206	# Hosts treatment
2207	${EXECUTION} hostname \| sort \| uniq > hosts.tmp
2208
2209	i=0
2210	rm -f hosts rankfile
2211	IGCM_debug_Print 1 "sys Curie, Hosts available :"
2212	for nodes in `cat hosts.tmp`
2213	do
2214	host[$i]=$nodes
2215	echo "${host[$i]}" >> hosts
2216	IGCM_debug_Print 1 ${host[$i]}
2217	i=$((i+1))
2218	done
2219	rm -f hosts.tmp
2220
2221	listnodes=${host[*]}
2222
2223	EXECUTION="${EXECUTION} -hostfile hosts -rankfile rankfile"
2224
2225	# Initialisation
2226	rank=0
2227	current_core=0
2228	core_per_node=16
2229	init_exec=n
2230
2231	# Loop on the components
2232	for comp in ${config_ListOfComponents[*]} ; do
2233
2234	eval ExeNameIn=\${config_Executable_${comp}[0]}
2235	eval ExeNameOut=\${config_Executable_${comp}[1]}
2236
2237	# Not possible if oasis has an executable (i.e old version of oasis3)
2238	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
2239	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
2240	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
2241	IGCM_debug_Verif_Exit
2242	fi
2243
2244	# Only if we really have an executable for the component :
2245	if [ "X${ExeNameOut}" != X\"\" ] ; then
2246
2247	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2248	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2249
2250	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2251	echo "" >> script_${ExeNameOut}.ksh
2252	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
2253
2254	# Check if the number of threads is correct
2255	case ${comp_proc_omp_loc} in
2256	2\|4\|8\|16)
2257	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2258	;;
2259	*)
2260	IGCM_debug_Exit "ERROR with OMP parameters !"
2261	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2262	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2263	IGCM_debug_Verif_Exit
2264	;;
2265	esac
2266	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2267	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2268	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2269	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2270	fi
2271	# echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
2272	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
2273	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2274	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2275
2276	if [ ${init_exec} = y ] ; then
2277	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2278	else
2279	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2280	init_exec=y
2281	fi
2282
2283	# Build rankfile : method used to assign cores and nodes for the MPI process
2284	# Ex :
2285	#rank 0=curie5296 slot=0,1,2,3
2286	#rank 1=curie5296 slot=4,5,6,7
2287	# Example of final command :
2288	# mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
2289	# with script_lmdz.x.ksh :
2290	# #!/bin/ksh
2291	#export KMP_STACKSIZE=3g
2292	#export KMP_LIBRARY=turnaround
2293	#export MKL_SERIAL=YES
2294	#OMP_NUM_THREADS=4
2295	#./lmdz.x
2296	#
2297
2298	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
2299	(( index_host = current_core / core_per_node ))
2300	host_value=${host[${index_host}]}
2301	(( slot = current_core % core_per_node ))
2302	virg=","
2303	string_final=""
2304	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
2305	string=$index$virg
2306	string_final=$string_final$string
2307	done
2308	string_final=$( echo $string_final \| sed "s/.$//" )
2309	echo "rank $rank=$host_value slot=$string_final" >> rankfile
2310	(( rank = rank + 1 ))
2311	(( current_core = current_core + comp_proc_omp_loc ))
2312	done
2313	fi
2314
2315	done
2316	fi
2317
2318	# Only one executable (SPMD mode).
2319	else
2320
2321	for comp in ${config_ListOfComponents[*]} ; do
2322
2323	# Only if we really have an executable for the component :
2324	eval ExeNameOut=\${config_Executable_${comp}[1]}
2325	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2326
2327	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2328	echo "" >> script_${ExeNameOut}.ksh
2329	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2330
2331	if ( ${OK_PARA_OMP} ) ; then
2332	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2333	echo "" >> script_${ExeNameOut}.ksh
2334	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2335	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2336	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2337	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2338	fi
2339
2340	if ( ${OK_PARA_MPI} ) ; then
2341	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2342	# Default : ccc_mprun used if nb_proc gt 1
2343	# to have out/err per process on different files
2344	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
2345	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2346	EXECUTION="${HOST_MPIRUN_COMMAND} -n ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2347	else
2348	# Default : ccc_mprun is NOT used if nb_proc eq 1
2349	# to have out/err per process on different files
2350	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2351	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2352	EXECUTION="/usr/bin/time ./script_${ExeNameOut}.ksh"
2353	fi
2354
2355	IGCM_debug_Print 1 "sys Curie : script_${ExeNameOut}.ksh contains"
2356	cat script_${ExeNameOut}.ksh
2357
2358	fi
2359
2360	done
2361
2362	fi
2363
2364	IGCM_debug_Print 1 "sys Curie : execution command is "
2365	IGCM_debug_Print 1 "$EXECUTION"
2366
2367	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2368	}
2369
2370	#D-#==================================================
2371	#D-function IGCM_sys_check_path
2372	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2373	#D-* do not point to an important use directory. Stop immediately in that case.
2374	#D-* Examples:
2375	#D-
2376	function IGCM_sys_check_path {
2377	IGCM_debug_PushStack "IGCM_sys_check_path"
2378	if ( $DEBUG_sys ) ; then
2379	echo "IGCM_sys_check_path"
2380	fi
2381
2382	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCHDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCWORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCSTOREDIR} ] ) ; then
2383	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2384	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2385	IGCM_debug_Exit "This will stop the job"
2386	fi
2387	IGCM_debug_PopStack "IGCM_sys_check_path"
2388	}
2389
2390	#D-#==================================================
2391	#D-function IGCM_sys_check_quota
2392	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2393	#D-* Examples:
2394	#D-
2395	function IGCM_sys_check_quota {
2396	IGCM_debug_PushStack "IGCM_sys_check_quota"
2397	if ( $DEBUG_sys ) ; then
2398	echo "IGCM_sys_check_quota"
2399	fi
2400	# Limit of quota (in %)
2401	limit_quota=90
2402
2403	# Check of the volume
2404	volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
2405	volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
2406
2407	if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
2408
2409	unit_avail=${volume_avail: -1}
2410	unit_quota=${volume_quota: -1}
2411
2412	if [ "${unit_quota}" = "*" ] ; then
2413	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2414	IGCM_debug_Print 1 "More than 100% of your quota is used"
2415	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2416	IGCM_debug_Print 1 "You must have more than 10% available to run"
2417	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2418	IGCM_debug_Verif_Exit
2419	fi
2420
2421	temp_avail=${volume_avail%%${volume_avail: -1}*}
2422	temp_quota=${volume_quota%%${volume_quota: -1}*}
2423
2424	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2425
2426	# Convertion
2427	if [ ${volume_avail: -1} = "T" ] ; then
2428	(( temp_avail = temp_avail * 1000000000000 ))
2429	elif [ ${volume_avail: -1} = "G" ] ; then
2430	(( temp_avail = temp_avail * 1000000000 ))
2431	elif [ ${volume_avail: -1} = "M" ] ; then
2432	(( temp_avail = temp_avail * 1000000 ))
2433	elif [ ${volume_avail: -1} = "k" ] ; then
2434	(( temp_avail = temp_avail * 1000 ))
2435	else
2436	(( temp_avail = volume_avail ))
2437	fi
2438	if [ ${volume_quota: -1} = "T" ] ; then
2439	(( temp_quota = temp_quota * 1000000000000 ))
2440	elif [ ${volume_quota: -1} = "G" ] ; then
2441	(( temp_quota = temp_quota * 1000000000 ))
2442	elif [ ${volume_quota: -1} = "M" ] ; then
2443	(( temp_quota = temp_quota * 1000000 ))
2444	elif [ ${volume_quota: -1} = "k" ] ; then
2445	(( temp_quota = temp_quota * 1000 ))
2446	else
2447	(( temp_quota = volume_quota ))
2448	fi
2449	fi
2450
2451	quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2452	# echo "volume ratio is " $quota_volume
2453
2454	if [ ${quota_volume} -ge ${limit_quota} ] ; then
2455	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2456	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2457	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2458	IGCM_debug_Print 1 "You must have more than 10% available to run"
2459	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2460	IGCM_debug_Verif_Exit
2461	fi
2462
2463	fi
2464
2465	# Check of the number of inodes
2466
2467	inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
2468	inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
2469
2470	if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
2471
2472	unit_avail=${inode_avail: -1}
2473	unit_quota=${inode_quota: -1}
2474
2475	if [ "${unit_quota}" = "*" ] ; then
2476	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2477	IGCM_debug_Print 1 "More than 100% of your quota is used"
2478	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2479	IGCM_debug_Print 1 "You must have more than 10% available to run"
2480	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2481	IGCM_debug_Verif_Exit
2482	fi
2483
2484	temp_avail=${inode_avail%%${inode_avail: -1}*}
2485	temp_quota=${inode_quota%%${inode_quota: -1}*}
2486
2487	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2488
2489	# Convertion
2490	if [ ${inode_avail: -1} = "T" ] ; then
2491	(( temp_avail = temp_avail * 1000000000000 ))
2492	elif [ ${inode_avail: -1} = "G" ] ; then
2493	(( temp_avail = temp_avail * 1000000000 ))
2494	elif [ ${inode_avail: -1} = "M" ] ; then
2495	(( temp_avail = temp_avail * 1000000 ))
2496	elif [ ${inode_avail: -1} = "k" ] ; then
2497	(( temp_avail = temp_avail * 1000 ))
2498	else
2499	(( temp_avail = inode_avail ))
2500	fi
2501
2502	if [ ${inode_quota: -1} = "T" ] ; then
2503	(( temp_quota = temp_quota * 1000000000000 ))
2504	elif [ ${inode_quota: -1} = "G" ] ; then
2505	(( temp_quota = temp_quota * 1000000000 ))
2506	elif [ ${inode_quota: -1} = "M" ] ; then
2507	(( temp_quota = temp_quota * 1000000 ))
2508	elif [ ${inode_quota: -1} = "k" ] ; then
2509	(( temp_quota = temp_quota * 1000 ))
2510	else
2511	(( temp_quota = inode_quota ))
2512	fi
2513	fi
2514	quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2515	# echo "inode ratio is " $quota_inode
2516
2517	if [ ${quota_inode} -ge ${limit_quota} ] ; then
2518	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2519	IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
2520	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2521	IGCM_debug_Print 1 "You must have more than 10% available to run"
2522	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2523	IGCM_debug_Verif_Exit
2524	fi
2525	fi
2526	IGCM_debug_PopStack "IGCM_sys_check_quota"
2527	}
2528
2529	#D-#==================================================
2530	#D-function IGCM_sys_GetJobID
2531	#D-* Purpose: Check if job_name is currently
2532	#D- running or in queue
2533	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2534	#D-
2535	function IGCM_sys_GetJobID {
2536	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2537	if ( $DEBUG_sys ) ; then
2538	echo "IGCM_sys_GetJobID"
2539	fi
2540
2541	# With -f option, the full job name is given in the last column
2542	ID=$( ccc_mstat -f -u $2 \| \
2543	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2544
2545	eval ${3}=${ID}
2546
2547	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2548	}
2549
2550	#D-#==================================================
2551	#D-function IGCM_sys_CountJobInQueue
2552	#D-* Purpose: Check if job_name is currently
2553	#D- running or in queue
2554	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2555	#D-
2556	function IGCM_sys_CountJobInQueue {
2557	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2558	if ( $DEBUG_sys ) ; then
2559	echo "IGCM_sys_CountJobInQueue"
2560	fi
2561
2562	#NbRun=$( ccc_mstat -f \| grep -c ${JobName} )
2563
2564	# With -f option, the full job name is given in the last column
2565	NbRun=$( ccc_mstat -f \| gawk -v JobName=$1 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
2566
2567	eval ${2}=${NbRun}
2568
2569	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2570	}
2571
2572	#D-#==================================================
2573	#D-function IGCM_sys_ListJobInQueue
2574	#D-* Purpose: Check if job_name is currently
2575	#D- running or in queue
2576	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2577	#D-
2578	function IGCM_sys_ListJobInQueue {
2579	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2580	if ( $DEBUG_sys ) ; then
2581	echo "IGCM_sys_ListJobInQueue"
2582	fi
2583
2584	# With -f option, the full job name is given in the last column
2585	set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
2586	'( $2 == User && \
2587	$NF != /TS/ && \
2588	$NF !~ /PACK/ && \
2589	$NF !~ /REBUILD/ && \
2590	$NF !~ /pack/ ) \
2591	{ print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
2592
2593	eval set -A ${2} ${JobList[*]}
2594
2595	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2596	}
2597
2598	##############################################################
2599	# NCO OPERATOR
2600
2601	#D-#==================================================
2602	#D-function IGCM_sys_ncap2
2603	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2604	#D-* Examples:
2605	#D-
2606	function IGCM_sys_ncap2 {
2607	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2608	if ( $DEBUG_sys ) ; then
2609	echo "IGCM_sys_ncap2 :" $@
2610	fi
2611
2612	typeset NB_ESSAI DELAI status i
2613	# number of tentative
2614	NB_ESSAI=3
2615	# time delay between tentative
2616	DELAI=2
2617
2618	i=0
2619	while [ $i -lt $NB_ESSAI ] ; do
2620	ncap2 "$@" > /tmp/out_command.$$ 2>&1
2621	status=$?
2622	if [ ${status} -gt 0 ] ; then
2623	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2624	cat /tmp/out_command.$$
2625	\rm /tmp/out_command.$$
2626	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2627	sleep $DELAI
2628	else
2629	\rm /tmp/out_command.$$
2630	break
2631	fi
2632	(( i = i + 1 ))
2633	done
2634
2635	if [ ${status} -gt 0 ] ; then
2636	echo "IGCM_sys_ncap2 : ncap2 error"
2637	IGCM_debug_Exit "ncap2"
2638	fi
2639
2640	IGCM_debug_PopStack "IGCM_sys_ncap2"
2641	}
2642
2643	#D-#==================================================
2644	#D-function IGCM_sys_ncatted
2645	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2646	#D-* Examples:
2647	#D-
2648	function IGCM_sys_ncatted {
2649	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2650	if ( $DEBUG_sys ) ; then
2651	echo "IGCM_sys_ncatted :" $@
2652	fi
2653
2654	typeset NB_ESSAI DELAI status i
2655	# number of tentative
2656	NB_ESSAI=3
2657	# time delay between tentative
2658	DELAI=2
2659
2660	i=0
2661	while [ $i -lt $NB_ESSAI ] ; do
2662	ncatted "$@" > /tmp/out_command.$$ 2>&1
2663	status=$?
2664	if [ ${status} -gt 0 ] ; then
2665	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2666	cat /tmp/out_command.$$
2667	\rm /tmp/out_command.$$
2668	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2669	sleep $DELAI
2670	else
2671	\rm /tmp/out_command.$$
2672	break
2673	fi
2674	(( i = i + 1 ))
2675	done
2676
2677	if [ ${status} -gt 0 ] ; then
2678	echo "IGCM_sys_ncatted : ncatted error"
2679	IGCM_debug_Exit "ncatted"
2680	fi
2681
2682	IGCM_debug_PopStack "IGCM_sys_ncatted"
2683	}
2684
2685	#D-#==================================================
2686	#D-function IGCM_sys_ncbo
2687	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2688	#D-* Examples:
2689	#D-
2690	function IGCM_sys_ncbo {
2691	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2692	if ( $DEBUG_sys ) ; then
2693	echo "IGCM_sys_ncbo :" $@
2694	fi
2695
2696	typeset NB_ESSAI DELAI status i
2697	# number of tentative
2698	NB_ESSAI=3
2699	# time delay between tentative
2700	DELAI=2
2701
2702	i=0
2703	while [ $i -lt $NB_ESSAI ] ; do
2704	ncbo $@ > /tmp/out_command.$$ 2>&1
2705	status=$?
2706	if [ ${status} -gt 0 ] ; then
2707	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2708	cat /tmp/out_command.$$
2709	\rm /tmp/out_command.$$
2710	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2711	sleep $DELAI
2712	else
2713	\rm /tmp/out_command.$$
2714	break
2715	fi
2716	(( i = i + 1 ))
2717	done
2718
2719	if [ ${status} -gt 0 ] ; then
2720	echo "IGCM_sys_ncbo : ncbo error"
2721	IGCM_debug_Exit "ncbo"
2722	fi
2723
2724	IGCM_debug_PopStack "IGCM_sys_ncbo"
2725	}
2726
2727	#D-#==================================================
2728	#D-function IGCM_sys_ncdif
2729	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2730	#D-* Examples:
2731	#D-
2732	function IGCM_sys_ncdiff {
2733	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2734	if ( $DEBUG_sys ) ; then
2735	echo "IGCM_sys_ncdiff :" $@
2736	fi
2737
2738	typeset NB_ESSAI DELAI status i
2739	# number of tentative
2740	NB_ESSAI=3
2741	# time delay between tentative
2742	DELAI=2
2743
2744	i=0
2745	while [ $i -lt $NB_ESSAI ] ; do
2746	ncdiff $@ > /tmp/out_command.$$ 2>&1
2747	status=$?
2748	if [ ${status} -gt 0 ] ; then
2749	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2750	cat /tmp/out_command.$$
2751	\rm /tmp/out_command.$$
2752	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2753	sleep $DELAI
2754	else
2755	\rm /tmp/out_command.$$
2756	break
2757	fi
2758	(( i = i + 1 ))
2759	done
2760
2761	if [ ${status} -gt 0 ] ; then
2762	echo "IGCM_sys_ncdiff : ncdiff error"
2763	IGCM_debug_Exit "ncdiff"
2764	fi
2765
2766	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2767	}
2768
2769	#D-#==================================================
2770	#D-function IGCM_sys_ncea
2771	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2772	#D-* Examples:
2773	#D-
2774	function IGCM_sys_ncea {
2775	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2776	if ( $DEBUG_sys ) ; then
2777	echo "IGCM_sys_ncea :" $@
2778	fi
2779
2780	typeset NB_ESSAI DELAI status i
2781	# number of tentative
2782	NB_ESSAI=3
2783	# time delay between tentative
2784	DELAI=2
2785
2786	i=0
2787	while [ $i -lt $NB_ESSAI ] ; do
2788	ncea $@ > /tmp/out_command.$$ 2>&1
2789	status=$?
2790	if [ ${status} -gt 0 ] ; then
2791	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2792	cat /tmp/out_command.$$
2793	\rm /tmp/out_command.$$
2794	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2795	sleep $DELAI
2796	else
2797	\rm /tmp/out_command.$$
2798	break
2799	fi
2800	(( i = i + 1 ))
2801	done
2802
2803	if [ ${status} -gt 0 ] ; then
2804	echo "IGCM_sys_ncea : ncea error"
2805	IGCM_debug_Exit "ncea"
2806	fi
2807
2808	IGCM_debug_PopStack "IGCM_sys_ncea"
2809	}
2810
2811	#D-#==================================================
2812	#D-function IGCM_sys_ncecat
2813	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2814	#D-* Examples:
2815	#D-
2816	function IGCM_sys_ncecat {
2817	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2818	if ( $DEBUG_sys ) ; then
2819	echo "IGCM_sys_ncecat :" $@
2820	fi
2821
2822	typeset NB_ESSAI DELAI status i
2823	# number of tentative
2824	NB_ESSAI=3
2825	# time delay between tentative
2826	DELAI=2
2827
2828	i=0
2829	while [ $i -lt $NB_ESSAI ] ; do
2830	ncecat $@ > /tmp/out_command.$$ 2>&1
2831	status=$?
2832	if [ ${status} -gt 0 ] ; then
2833	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2834	cat /tmp/out_command.$$
2835	\rm /tmp/out_command.$$
2836	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2837	sleep $DELAI
2838	else
2839	\rm /tmp/out_command.$$
2840	break
2841	fi
2842	(( i = i + 1 ))
2843	done
2844
2845	if [ ${status} -gt 0 ] ; then
2846	echo "IGCM_sys_ncecat : ncecat error"
2847	IGCM_debug_Exit "ncecat"
2848	fi
2849
2850	IGCM_debug_PopStack "IGCM_sys_ncecat"
2851	}
2852
2853	#D-#==================================================
2854	#D-function IGCM_sys_ncflint
2855	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2856	#D-* Examples:
2857	#D-
2858	function IGCM_sys_ncflint {
2859	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2860	if ( $DEBUG_sys ) ; then
2861	echo "IGCM_sys_ncflint :" $@
2862	fi
2863
2864	typeset NB_ESSAI DELAI status i
2865	# number of tentative
2866	NB_ESSAI=3
2867	# time delay between tentative
2868	DELAI=2
2869
2870	i=0
2871	while [ $i -lt $NB_ESSAI ] ; do
2872	ncflint $@ > /tmp/out_command.$$ 2>&1
2873	status=$?
2874	if [ ${status} -gt 0 ] ; then
2875	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2876	cat /tmp/out_command.$$
2877	\rm /tmp/out_command.$$
2878	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2879	sleep $DELAI
2880	else
2881	\rm /tmp/out_command.$$
2882	break
2883	fi
2884	(( i = i + 1 ))
2885	done
2886
2887	if [ ${status} -gt 0 ] ; then
2888	echo "IGCM_sys_ncflint : ncflint error"
2889	IGCM_debug_Exit "ncflint"
2890	fi
2891
2892	IGCM_debug_PopStack "IGCM_sys_ncflint"
2893	}
2894
2895	#D-#==================================================
2896	#D-function IGCM_sys_ncks
2897	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2898	#D-* Examples:
2899	#D-
2900	function IGCM_sys_ncks {
2901	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2902	if ( $DEBUG_sys ) ; then
2903	echo "IGCM_sys_ncks :" $@
2904	fi
2905
2906	typeset NB_ESSAI DELAI status i
2907	# number of tentative
2908	NB_ESSAI=3
2909	# time delay between tentative
2910	DELAI=2
2911
2912	i=0
2913	while [ $i -lt $NB_ESSAI ] ; do
2914	ncks $@ > /tmp/out_command.$$ 2>&1
2915	status=$?
2916	if [ ${status} -gt 0 ] ; then
2917	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2918	cat /tmp/out_command.$$
2919	\rm /tmp/out_command.$$
2920	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2921	sleep $DELAI
2922	else
2923	\rm /tmp/out_command.$$
2924	break
2925	fi
2926	(( i = i + 1 ))
2927	done
2928
2929	if [ ${status} -gt 0 ] ; then
2930	echo "IGCM_sys_ncks : ncks error"
2931	IGCM_debug_Exit "ncks"
2932	fi
2933
2934	IGCM_debug_PopStack "IGCM_sys_ncks"
2935	}
2936
2937	#D-#==================================================
2938	#D-function IGCM_sys_ncpdq
2939	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2940	#D-* Examples:
2941	#D-
2942	function IGCM_sys_ncpdq {
2943	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2944	if ( $DEBUG_sys ) ; then
2945	echo "IGCM_sys_ncpdq :" $@
2946	fi
2947
2948	typeset NB_ESSAI DELAI status i
2949	# number of tentative
2950	NB_ESSAI=3
2951	# time delay between tentative
2952	DELAI=2
2953
2954	i=0
2955	while [ $i -lt $NB_ESSAI ] ; do
2956	ncpdq $@ > /tmp/out_command.$$ 2>&1
2957	status=$?
2958	if [ ${status} -gt 0 ] ; then
2959	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2960	cat /tmp/out_command.$$
2961	\rm /tmp/out_command.$$
2962	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2963	sleep $DELAI
2964	else
2965	\rm /tmp/out_command.$$
2966	break
2967	fi
2968	(( i = i + 1 ))
2969	done
2970
2971	if [ ${status} -gt 0 ] ; then
2972	echo "IGCM_sys_ncpdq : ncpdq error"
2973	IGCM_debug_Exit "ncpdq"
2974	fi
2975
2976	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2977	}
2978
2979	#D-#==================================================
2980	#D-function IGCM_sys_ncra
2981	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2982	#D-* Examples:
2983	#D-
2984	function IGCM_sys_ncra {
2985	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2986	if ( $DEBUG_sys ) ; then
2987	echo "IGCM_sys_ncra :" $@
2988	fi
2989
2990	typeset NB_ESSAI DELAI status i
2991	# number of tentative
2992	NB_ESSAI=3
2993	# time delay between tentative
2994	DELAI=2
2995
2996	i=0
2997	while [ $i -lt $NB_ESSAI ] ; do
2998	ncra $@ > /tmp/out_command.$$ 2>&1
2999	status=$?
3000	if [ ${status} -gt 0 ] ; then
3001	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
3002	cat /tmp/out_command.$$
3003	\rm /tmp/out_command.$$
3004	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3005	sleep $DELAI
3006	else
3007	\rm /tmp/out_command.$$
3008	break
3009	fi
3010	(( i = i + 1 ))
3011	done
3012
3013	if [ ${status} -gt 0 ] ; then
3014	echo "IGCM_sys_ncra : ncra error"
3015	IGCM_debug_Exit "ncra"
3016	fi
3017
3018	IGCM_debug_PopStack "IGCM_sys_ncra"
3019	}
3020
3021	#D-#==================================================
3022	#D-function IGCM_sys_ncrcat
3023	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
3024	#D-* Examples:
3025	#D-
3026	function IGCM_sys_ncrcat {
3027	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
3028	if ( $DEBUG_sys ) ; then
3029	echo "IGCM_sys_ncrcat :" $@
3030	fi
3031
3032	typeset NB_ESSAI DELAI status i lastArg
3033	# number of tentative
3034	NB_ESSAI=3
3035	# time delay between tentative
3036	DELAI=2
3037
3038	i=0
3039	while [ $i -lt $NB_ESSAI ] ; do
3040	ncrcat $@ > /tmp/out_command.$$ 2>&1
3041	status=$?
3042	if [ ${status} -gt 0 ] ; then
3043	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
3044	cat /tmp/out_command.$$
3045	\rm /tmp/out_command.$$
3046	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3047	sleep $DELAI
3048	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
3049	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
3050	cat /tmp/out_command.$$
3051	# remove files having corrupted time axis
3052	eval lastArg=\${$#}
3053	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
3054	\rm ${lastArg}
3055	\rm /tmp/out_command.$$
3056	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3057	sleep $DELAI
3058	else
3059	\rm /tmp/out_command.$$
3060	break
3061	fi
3062	(( i = i + 1 ))
3063	done
3064
3065	if [ ${status} -gt 0 ] ; then
3066	echo "IGCM_sys_ncrcat : ncrcat error"
3067	#IGCM_debug_Exit "ncrcat"
3068	fi
3069
3070	IGCM_debug_PopStack "IGCM_sys_ncrcat"
3071	}
3072
3073	#D-#==================================================
3074	#D-function IGCM_sys_ncrename
3075	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
3076	#D-* Examples:
3077	#D-
3078	function IGCM_sys_ncrename {
3079	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
3080	if ( $DEBUG_sys ) ; then
3081	echo "IGCM_sys_ncrename :" $@
3082	fi
3083
3084	typeset NB_ESSAI DELAI status i
3085	# number of tentative
3086	NB_ESSAI=3
3087	# time delay between tentative
3088	DELAI=2
3089
3090	i=0
3091	while [ $i -lt $NB_ESSAI ] ; do
3092	ncrename $@ > /tmp/out_command.$$ 2>&1
3093	status=$?
3094	if [ ${status} -gt 0 ] ; then
3095	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
3096	cat /tmp/out_command.$$
3097	\rm /tmp/out_command.$$
3098	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3099	sleep $DELAI
3100	else
3101	\rm /tmp/out_command.$$
3102	break
3103	fi
3104	(( i = i + 1 ))
3105	done
3106
3107	if [ ${status} -gt 0 ] ; then
3108	echo "IGCM_sys_ncrename : ncrename error"
3109	IGCM_debug_Exit "ncrename"
3110	fi
3111
3112	IGCM_debug_PopStack "IGCM_sys_ncrename"
3113	}
3114
3115	#D-#==================================================
3116	#D-function IGCM_sys_ncwa
3117	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
3118	#D-* Examples:
3119	#D-
3120	function IGCM_sys_ncwa {
3121	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
3122	if ( $DEBUG_sys ) ; then
3123	echo "IGCM_sys_ncwa :" $@
3124	fi
3125
3126	typeset NB_ESSAI DELAI status i
3127	# number of tentative
3128	NB_ESSAI=3
3129	# time delay between tentative
3130	DELAI=2
3131
3132	i=0
3133	while [ $i -lt $NB_ESSAI ] ; do
3134	ncwa $@ > /tmp/out_command.$$ 2>&1
3135	status=$?
3136	if [ ${status} -gt 0 ] ; then
3137	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
3138	cat /tmp/out_command.$$
3139	\rm /tmp/out_command.$$
3140	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3141	sleep $DELAI
3142	else
3143	\rm /tmp/out_command.$$
3144	break
3145	fi
3146	(( i = i + 1 ))
3147	done
3148
3149	if [ ${status} -gt 0 ] ; then
3150	echo "IGCM_sys_ncwa : ncwa error"
3151	IGCM_debug_Exit "ncwa"
3152	fi
3153
3154	IGCM_debug_PopStack "IGCM_sys_ncwa"
3155	}
3156
3157	##############################################################
3158	# CDO OPERATOR
3159
3160	#D-#==================================================
3161	#D-function IGCM_sys_cdo
3162	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
3163	#D-* Examples:
3164	#D-
3165	function IGCM_sys_cdo {
3166	IGCM_debug_PushStack "IGCM_sys_cdo" $@
3167	if ( $DEBUG_sys ) ; then
3168	echo "IGCM_sys_cdo :" $@
3169	fi
3170
3171	typeset status
3172
3173	\cdo $@ > /tmp/out_command.$$ 2>&1
3174	status=$?
3175	if [ ${status} -gt 0 ] ; then
3176	echo "IGCM_sys_cdo : error code ${status}"
3177	cat /tmp/out_command.$$
3178	\rm /tmp/out_command.$$
3179	IGCM_debug_PopStack "IGCM_sys_cdo"
3180	return 1
3181	else
3182	IGCM_debug_PopStack "IGCM_sys_cdo"
3183	return 0
3184	fi
3185
3186	IGCM_debug_PopStack "IGCM_sys_cdo"
3187	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: