Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_curie.ksh @ 1158

Last change on this file since 1158 was 1158, checked in by sdipsl, 9 years ago
metrics path
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 93.3 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Curie
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=curie
71	# add default project on curie
72	typeset DEFAULT_PROJECT=gen0826
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# Submit command
80	typeset SUBMIT=${SUBMIT:=ccc_msub}
81	# rsync with path
82	typeset -r RSYNC=/usr/bin/rsync
83	# RSYNC_opt args to rsync
84	typeset -r RSYNC_opt="-va"
85	# ie storage filesystem
86	typeset -r STOREHOST=${MASTER}
87
88	#====================================================
89	# Source default environment
90	#====================================================
91	##. /etc/profile
92
93	#====================================================
94	# Set environment tools (ferret, nco, cdo)
95	#====================================================
96	if [ X${TaskType} = Xcomputing ] ; then
97	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
98	# to run with netcdf 3.6.3 ie compilation done before 17/2/2014
99	# uncomment 2 lines :
100	# module unload netcdf
101	# module load netcdf/3.6.3
102	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddNoise/src_X64_CURIE/bin
103	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddPerturbation/src_X64_CURIE/bin
104	else
105	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
106	METRICS_MP=/ccc/work/cont003/dsm/p86ipsl/PCMDI-MP/
107	fi
108
109	#====================================================
110	# Specific for ocean additionnal diagnostic
111	export FER_GO="$FER_GO /home/cont003/p86denv/IGCM_POST_UTIL/JNL /home/cont003/p86denv/GRAF /home/cont003/p86denv/GRAF/GO"
112	export FER_PALETTE="$FER_PALETTE /home/cont003/p86denv/GRAF/PALET"
113
114	#====================================================
115	# Host specific DIRECTORIES
116	#====================================================
117
118	# ============ CESIUM START ============ #
119
120	#====================================================
121	#- Mirror libIGCM from titane to cesium if needed
122	#ROOTSYS=$( echo ${libIGCM} \| gawk -F"/" '{print $3}' )
123	#if [ ! ${ROOTSYS} = "home" ] ; then
124	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
125	#else
126	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
127	#fi
128
129	#====================================================
130	#- libIGCM_POST
131	#if ( ${MirrorlibIGCM} ) ; then
132	# PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
133	# typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
134	#else
135	# typeset -r libIGCM_POST=${libIGCM}
136	#fi
137
138	# ============ CESIUM END ============ #
139
140	#====================================================
141	#- MirrorlibIGCM for frontend
142	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
143
144	#====================================================
145	#- libIGCM_POST for frontend
146	typeset -r libIGCM_POST=${libIGCM}
147
148	#====================================================
149	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
150	typeset -r R_EXE="${MODIPSL}/bin"
151
152	#====================================================
153	#- SUBMIT_DIR : submission dir
154	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${BRIDGE_MSUB_PWD}}
155
156	#====================================================
157	#- IN
158	typeset -r R_IN=${R_IN:=/ccc/work/cont003/dsm/p86ipsl/IGCM}
159	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/ccc/work/cont003/dsm/p24data}
160
161	#====================================================
162	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
163	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCHDIR}/RUN_DIR/${BRIDGE_MSUB_JOBID}_${$}}
164
165	#====================================================
166	#- HOST_MPIRUN_COMMAND
167	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time ccc_mprun -E-K1"}
168
169	#====================================================
170	#- Max number of arguments passed to nco operator or demigration command
171	UNIX_MAX_LIMIT=360
172
173	#====================================================
174	#- set PackDefault true on curie
175	PackDefault=true
176
177	#====================================================
178	#- Number of core per node (max number of OpenMP task)
179	NUM_COREPERNODE=8
180
181	#====================================================
182	#- Default number of MPI task for IPSL coupled model
183	#- required for backward compatibility
184	#-
185	DEFAULT_NUM_PROC_OCE=5
186	DEFAULT_NUM_PROC_CPL=1
187	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
188	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
189
190	#D-#==================================================
191	#D-function IGCM_sys_defineArchives
192	#D-* Purpose:
193	#D-* Define ARCHIVE : Dedicated to large files
194	#D-* Define STORAGE : Dedicated to small/medium files
195	#D-* Define R_OUT : Output tree located on ARCHIVE
196	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
197	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
198	#D-* if SpaceName=TEST everything is stored on SCRATCHDIR
199	#D-* Examples:
200	#D-
201	function IGCM_sys_defineArchives {
202	IGCM_debug_PushStack "IGCM_sys_defineArchives"
203
204	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
205	#====================================================
206	#- ARCHIVE (dedicated to large files)
207	ARCHIVE=${config_UserChoices_ARCHIVE}
208	else
209	#====================================================
210	#- ARCHIVE (dedicated to large files)
211	ARCHIVE=${CCCSTOREDIR}
212	fi
213
214	if [ ! X${config_UserChoices_STORAGE} = X ]; then
215	#====================================================
216	#- STORAGE (dedicated to small/medium files)
217	STORAGE=${config_UserChoices_STORAGE}
218	else
219	#====================================================
220	#- STORAGE (dedicated to small/medium files)
221	STORAGE=${CCCWORKDIR}
222	fi
223
224	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
225	#====================================================
226	#- R_OUT
227	R_OUT=${SCRATCHDIR}/IGCM_OUT
228
229	#====================================================
230	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
231	R_FIG=${SCRATCHDIR}/IGCM_OUT
232
233	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
234
235	else
236	#====================================================
237	#- R_OUT
238	R_OUT=${ARCHIVE}/IGCM_OUT
239
240	#====================================================
241	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
242	R_FIG=${STORAGE}/IGCM_OUT
243	fi
244
245	#====================================================
246	#- R_BUF (ONLY FOR double copy an scratch)
247	R_BUF=${SCRATCHDIR}/IGCM_OUT
248
249	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
250	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
251	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
252
253	IGCM_debug_PopStack "IGCM_sys_defineArchives"
254	}
255
256	#D-#==================================================
257	#D-function IGCM_sys_RshMaster
258	#D-* Purpose: Connection to frontend machine.
259	#D-* Examples:
260	#D-
261	function IGCM_sys_RshMaster {
262	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
263	/bin/ksh <<-EOF
264	export libIGCM=${libIGCM}
265	export DEBUG_debug=${DEBUG_debug}
266	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
267	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
268	${@}
269	EOF
270	if [ $? -gt 0 ] ; then
271	echo "IGCM_sys_RshMaster : erreur."
272	IGCM_debug_Exit "IGCM_sys_RshMaster"
273	fi
274	IGCM_debug_PopStack "IGCM_sys_RshMaster"
275	}
276
277	#D-#==================================================
278	#D-function IGCM_sys_RshArchive
279	#D-* Purpose: Archive rsh command
280	#D-* Examples:
281	#D-
282	function IGCM_sys_RshArchive {
283	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
284	/bin/ksh <<-EOF
285	${@}
286	EOF
287	status=$?
288	if [ ${status} -gt 0 ] ; then
289	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
290	IGCM_debug_Exit "IGCM_sys_RshArchive"
291	fi
292	IGCM_debug_PopStack "IGCM_sys_RshArchive"
293	}
294
295	#D-#==================================================
296	#D-function IGCM_sys_RshArchive_NoError
297	#D-* Purpose: Archive rsh command, without error
298	#D-* used only in monitoring.job
299	#D-* Examples:
300	#D-
301	function IGCM_sys_RshArchive_NoError {
302	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
303	/bin/ksh <<-EOF
304	${@} 2> \dev\null
305	EOF
306	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
307	}
308
309	#D-#==================================================
310	#D-function IGCM_sys_RshPost
311	#D-* Purpose: Post-process rsh command
312	#D-* Examples:
313	#D-
314	function IGCM_sys_RshPost {
315	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
316	if ( $DEBUG_sys ) ; then
317	echo "IGCM_sys_RshPost :" $@
318	fi
319	# keep standard input (stdin) for the loop onto temporary file
320	cat >/tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
321
322	/bin/ksh </tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
323	if [ $? -gt 0 ] ; then
324	echo "IGCM_sys_RshPost : erreur."
325	IGCM_debug_Exit "IGCM_sys_RshPost"
326	fi
327	# delete temporary file
328	\rm /tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
329
330	# ============ FRONTEND END ============ #
331
332	# ============ CESIUM START ============ #
333	# typeset NB_ESSAI DELAI status i
334	# if [ "X$( grep rebuild_from tmp_IGCM_sys_RshPost_$$ )" != "X" ] ; then
335	# #little hack so that rebuild submission is done on titane not an cesium
336	#
337	# libIGCM_POST_sed=$( echo $libIGCM_POST \| sed 's/\//\\\//g' )
338	# POST_DIR_sed=$( echo ${POST_DIR} \| sed 's/\//\\\//g' )
339	# sed "s/IGCM_sys_QsubPost/IGCM_sys_Qsub/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
340	# sed "s/ rebuild_fromWorkdir/ ${libIGCM_POST_sed}\/rebuild_fromWorkdir.job/g" tmp.txt > tmp_IGCM_sys_RshPost_$$
341	# sed "s/ rebuild_fromArchive/ ${libIGCM_POST_sed}\/rebuild_fromArchive.job/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
342	# sed "s/Script_Post_Output=/Script_Output=${POST_DIR_sed}\//g" tmp.txt > tmp_IGCM_sys_RshPost_$$
343	# \mv tmp.txt tmp_IGCM_sys_RshPost_$$
344	#
345	# echo cat tmp_IGCM_sys_RshPost_$$ AFTER
346	# cat tmp_IGCM_sys_RshPost_$$
347	#
348	# /bin/ksh <tmp_IGCM_sys_RshPost_$$
349	# if [ $? -gt 0 ] ; then
350	# echo "IGCM_sys_RshPost : erreur."
351	# IGCM_debug_Exit "IGCM_sys_RshPost"
352	# fi
353	# # delete temporary file
354	# \rm tmp_IGCM_sys_RshPost_$$
355	#
356	# else
357	# # number of tentative
358	# NB_ESSAI=10
359	# # time delay between tentative
360	# DELAI=10
361	# i=0
362	# while [ $i -ne $NB_ESSAI ] ; do
363	# ssh -t titane996 ssh cesium /bin/ksh <tmp_IGCM_sys_RshPost_$$
364	# status=$?
365	# if [ ${status} -ne 0 ];
366	# then
367	# sleep $DELAI
368	# else
369	# break
370	# fi
371	# let i=$i+1
372	# done
373	# # delete temporary file
374	# \rm tmp_IGCM_sys_RshPost_$$
375	#
376	# if [ ${status} -gt 0 ] ; then
377	# echo "IGCM_sys_RshPost : erreur."
378	# IGCM_debug_Exit "IGCM_sys_RshPost"
379	# fi
380	# fi
381
382	# ============ CESIUM END ============ #
383
384	IGCM_debug_PopStack "IGCM_sys_RshPost"
385	}
386
387	#D-#==================================================
388	#D-function IGCM_sys_SendMail
389	#D-* Purpose: Send mail when simulation is over
390	#D-* Examples:
391	#D-
392	function IGCM_sys_SendMail {
393	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
394
395	if ( $DEBUG_sys ) ; then
396	echo "IGCM_sys_SendMail :" $@
397	fi
398
399	if [ X${1} = XAccounting ] ; then
400	status=Accounting
401	mailText=jobAccounting.mail
402	elif ( ${ExitFlag} ) ; then
403	status=failed
404	mailText=jobEnd.mail
405	else
406	status=completed
407	mailText=jobEnd.mail
408	fi
409
410	# Update selected mail template
411	while read -r line; do
412	eval echo $line >> mail.txt ;
413	done < ${libIGCM}/libIGCM_sys/${mailText}
414
415	if [ ! -z ${config_UserChoices_MailName} ] ; then
416	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
417	elif [ -f ~/.forward ] ; then
418	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
419	fi
420
421	sleep 10
422	rm -f mail.txt
423
424	IGCM_debug_PopStack "IGCM_sys_SendMail"
425	}
426
427	#D-#==================================================
428	#D-function IGCM_sys_Mkdir
429	#D-* Purpose: Master locale mkdir command
430	#D-* Examples:
431	#D-
432	function IGCM_sys_Mkdir {
433	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
434	if ( $DEBUG_sys ) ; then
435	echo "IGCM_sys_Mkdir :" $@
436	fi
437	if [ ! -d ${1} ]; then
438	\mkdir -p $1
439	if [ $? -gt 0 ] ; then
440	echo "IGCM_sys_Mkdir : erreur."
441	IGCM_debug_Exit "IGCM_sys_Mkdir"
442	fi
443	fi
444	# vérification :
445	if [ ! -d ${1} ] ; then
446	echo "IGCM_sys_Mkdir : erreur."
447	IGCM_debug_Exit "IGCM_sys_Mkdir"
448	fi
449	IGCM_debug_PopStack "IGCM_sys_Mkdir"
450	}
451
452	#D-#==================================================
453	#D-function IGCM_sys_MkdirArchive
454	#D-* Purpose: Mkdir on Archive
455	#D-* Examples:
456	#D-
457	function IGCM_sys_MkdirArchive {
458	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
459	if ( $DEBUG_sys ) ; then
460	echo "IGCM_sys_MkdirArchive :" $@
461	fi
462	#- creation de repertoire sur le serveur fichier
463	if [ ! -d ${1} ]; then
464	\mkdir -p $1
465	status=$?
466
467	if [ ${status} -gt 0 ] ; then
468	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
469	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
470	fi
471	fi
472	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
473	}
474
475	#D-#==================================================
476	#D-function IGCM_sys_MkdirWork
477	#D-* Purpose: Mkdir on Work
478	#D-* Examples:
479	#D-
480	function IGCM_sys_MkdirWork {
481	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
482	if ( $DEBUG_sys ) ; then
483	echo "IGCM_sys_MkdirWork :" $@
484	fi
485	#- creation de repertoire sur le serveur fichier
486	if [ ! -d ${1} ]; then
487	\mkdir -p $1
488	if [ $? -gt 0 ] ; then
489	echo "IGCM_sys_MkdirWork : erreur."
490	IGCM_debug_Exit "IGCM_sys_MkdirWork"
491	fi
492	fi
493	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
494	}
495
496	#D-#==================================================
497	#D-function IGCM_sys_Cd
498	#D-* Purpose: master cd command
499	#D-* Examples:
500	#D-
501	function IGCM_sys_Cd {
502	IGCM_debug_PushStack "IGCM_sys_Cd" $@
503	if ( $DEBUG_sys ) ; then
504	echo "IGCM_sys_Cd :" $@
505	fi
506	\cd $1
507	if [ $? -gt 0 ] ; then
508	echo "IGCM_sys_Cd : erreur."
509	IGCM_debug_Exit "IGCM_sys_Cd"
510	fi
511	IGCM_debug_PopStack "IGCM_sys_Cd"
512	}
513
514	#D-#==================================================
515	#D-function IGCM_sys_Chmod
516	#D-* Purpose: Chmod
517	#D-* Examples:
518	#D-
519	function IGCM_sys_Chmod {
520	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
521	if ( $DEBUG_sys ) ; then
522	echo "IGCM_sys_Chmod :" $@
523	fi
524	\chmod $@
525	if [ $? -gt 0 ] ; then
526	echo "IGCM_sys_Chmod : erreur."
527	IGCM_debug_Exit "IGCM_sys_Chmod"
528	fi
529	IGCM_debug_PopStack "IGCM_sys_Chmod"
530	}
531
532	#D-#==================================================
533	#D-function IGCM_sys_FileSize
534	#D-* Purpose: Filesize
535	#D-* Examples:
536	#D-
537	function IGCM_sys_FileSize {
538	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
539
540	typeset sizeF
541	set +A sizeF -- $( ls -la ${1} )
542	if [ $? -gt 0 ] ; then
543	IGCM_debug_Exit "IGCM_sys_FileSize"
544	fi
545	eval ${2}=${sizeF[4]}
546
547	IGCM_debug_PopStack "IGCM_sys_FileSize"
548	}
549
550	#D-#==================================================
551	#D-function IGCM_sys_TestDir
552	#D-* Purpose: Test Directory that must exists
553	#D-* Examples:
554	#D-
555	function IGCM_sys_TestDir {
556	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
557	if ( $DEBUG_sys ) ; then
558	echo "IGCM_sys_TestDir :" $@
559	fi
560	typeset ExistFlag
561	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
562	IGCM_debug_PopStack "IGCM_sys_TestDir"
563
564	return ${ExistFlag}
565	}
566
567	#D-#==================================================
568	#D-function IGCM_sys_TestDirArchive
569	#D-* Purpose: Test Directory that must exists on Archive
570	#D-* Examples:
571	#D-
572	function IGCM_sys_TestDirArchive {
573	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
574	if ( $DEBUG_sys ) ; then
575	echo "IGCM_sys_TestDirArchive :" $@
576	fi
577	typeset ExistFlag
578	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
579	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
580	return ${ExistFlag}
581	}
582
583	#D-#==================================================
584	#D-function IGCM_sys_IsFileArchived
585	#D-* Purpose: Test file that must NOT EXISTS on Archive
586	#D-* Examples:
587	#D-
588	function IGCM_sys_IsFileArchived {
589	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
590	if ( $DEBUG_sys ) ; then
591	echo "IGCM_sys_IsFileArchived :" $@
592	fi
593	typeset IsArchivedFlag
594	IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
595	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
596
597	return ${IsArchivedFlag}
598	}
599
600	#D-#==================================================
601	#D-function IGCM_sys_TestFileArchive
602	#D-* Purpose: Test file that must NOT EXISTS on Archive
603	#D-* Examples:
604	#D-
605	function IGCM_sys_TestFileArchive {
606	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
607	typeset ExistFlag
608	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
609	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
610
611	return ${ExistFlag}
612	}
613
614	#D-#==================================================
615	#D-function IGCM_sys_TestFileBuffer
616	#D-* Purpose: Test file that must NOT EXISTS on Buffer
617	#D-* Examples:
618	#D-
619	function IGCM_sys_TestFileBuffer {
620	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
621	typeset ExistFlag
622	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
623	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
624
625	return ${ExistFlag}
626	}
627
628	#D-#==================================================
629	#D-function IGCM_sys_CountFileArchive
630	#D-* Purpose: Count files on Archive filesystem
631	#D-* Examples:
632	#D-
633	function IGCM_sys_CountFileArchive {
634	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
635	ls ${@} 2>/dev/null \| wc -l
636	if [ $? -gt 0 ] ; then
637	echo "IGCM_sys_CountFileArchive : erreur."
638	fi
639	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
640	}
641
642	#D-#==================================================
643	#D-function IGCM_sys_CountFileBuffer
644	#D-* Purpose: Count files on Scratch filesystem
645	#D-* Examples:
646	#D-
647	function IGCM_sys_CountFileBuffer {
648	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
649	ls ${@} 2>/dev/null \| wc -l
650	if [ $? -gt 0 ] ; then
651	echo "IGCM_sys_CountFileBuffer : erreur."
652	fi
653	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
654	}
655
656	#D-#==================================================
657	#D-function IGCM_sys_Tree
658	#D-* Purpose: Tree directories with files on ${ARCHIVE}
659	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
660	#D-
661	function IGCM_sys_Tree {
662	IGCM_debug_PushStack "IGCM_sys_Tree" $@
663	if ( $DEBUG_sys ) ; then
664	echo "IGCM_sys_Tree :" $@
665	fi
666
667	\ls -lR ${@}
668
669	IGCM_debug_PopStack "IGCM_sys_Tree"
670	}
671
672	#D-#==================================================
673	#D-function IGCM_sys_Tar
674	#D-* Purpose: master tar command
675	#D-* Examples:
676	#D-
677	function IGCM_sys_Tar {
678	IGCM_debug_PushStack "IGCM_sys_Tar" $@
679	if ( $DEBUG_sys ) ; then
680	echo "IGCM_sys_Tar :" $@
681	fi
682	\tar cf $@
683	if [ $? -gt 0 ] ; then
684	echo "IGCM_sys_Tar : erreur."
685	IGCM_debug_Exit "IGCM_sys_Tar"
686	fi
687	IGCM_debug_PopStack "IGCM_sys_Tar"
688	}
689
690	#D-#==================================================
691	#D-function IGCM_sys_UnTar
692	#D-* Purpose: master un-tar command
693	#D-* Examples:
694	#D-
695	function IGCM_sys_UnTar {
696	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
697	if ( $DEBUG_sys ) ; then
698	echo "IGCM_sys_UnTar :" $@
699	fi
700	\tar xvf $1
701	if [ $? -gt 0 ] ; then
702	echo "IGCM_sys_UnTar : erreur."
703	IGCM_debug_Exit "IGCM_sys_UnTar"
704	fi
705	IGCM_debug_PopStack "IGCM_sys_UnTar"
706	}
707
708	#D-#==================================================
709	#D-function IGCM_sys_Qsub
710	#D-* Purpose: Qsub new job
711	#D-* Examples:
712	#D-
713	function IGCM_sys_Qsub {
714	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
715	if ( $DEBUG_sys ) ; then
716	echo "IGCM_sys_Qsub :" $@
717	fi
718	typeset options status
719	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
720
721	/usr/bin/ccc_msub ${options} $1 > /tmp/out_command.$$ 2>&1
722	status=$?
723
724	cat /tmp/out_command.$$
725	if [ ${status} -gt 0 ] ; then
726	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
727	IGCM_debug_Exit "IGCM_sys_Qsub"
728	else
729	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
730	fi
731	IGCM_debug_PopStack "IGCM_sys_Qsub"
732	}
733
734	#D-#==================================================
735	#D-function IGCM_sys_QsubPost
736	#D-* Purpose: Qsub new job on scalaire
737	#D-* Examples:
738	#D-
739	function IGCM_sys_QsubPost {
740	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
741	if ( $DEBUG_sys ) ; then
742	echo "IGCM_sys_QsubPost :" $@
743	fi
744	typeset options status
745	MY_PROJECT=$(echo ${BRIDGE_MSUB_PROJECT:=${DEFAULT_PROJECT}} \| cut -d@ -f1 )
746
747	# EASIER TO DO THIS INSTEAD OF DUPLICATING libIGCM_sys_curie.ksh
748	case $( hostname -s ) in
749	curie*)
750	options="-Q normal -A ${MY_PROJECT} -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out";;
751	airain*)
752	options="-q ivybridge -A dsm -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
753	esac
754
755	/usr/bin/ccc_msub ${options} ${libIGCM_POST}/$1.job > /tmp/out_command.$$ 2>&1
756	status=$?
757
758	cat /tmp/out_command.$$
759	if [ ${status} -gt 0 ] ; then
760	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
761	IGCM_debug_Exit "IGCM_sys_QsubPost"
762	else
763	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
764	fi
765	IGCM_debug_PopStack "IGCM_sys_QsubPost"
766	}
767
768	#D-*************************
769	#D- File transfer functions
770	#D-*************************
771	#D-
772
773	#D-#==================================================
774	#D-function IGCM_sys_Rsync_out
775	#D-* Purpose: treat return val of rsync
776	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
777	#D- Error values and explanations can depend on your system version.
778	function IGCM_sys_Rsync_out {
779	status=$1
780	if [ ! $status ] ; then
781	echo "rsync error !"
782	fi
783
784	if [ $MYLANG = "fr" ]; then
785	case $status in
786	0) return ;;
787	1) echo "Erreur de rsync ; RERR_SYNTAX : "
788	echo "Erreur de syntaxe ou d'utilisation."
789	return;;
790	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
791	echo "Incompatibilité de protocole."
792	return;;
793	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
794	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
795	echo "répertoires"
796	return;;
797	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
798	echo "Action demandée non supportée : une tentative de manipulation de"
799	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
800	echo "été faite ; ou une option qui est supportée par le client mais"
801	echo "pas par le serveur a été spécifiée."
802	return;;
803	10) echo "Erreur de rsync ; RERR_SOCKETIO"
804	echo "Erreur dans le socket d'entrée sortie"
805	return;;
806	11) echo "Erreur de rsync ; RERR_FILEIO"
807	echo "Erreur d'entrée sortie fichier"
808	return;;
809	12) echo "Erreur de rsync ; RERR_STREAMIO"
810	echo "Erreur dans flux de donnée du protocole rsync"
811	return;;
812	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
813	echo "Erreur avec les diagnostics du programme"
814	return;;
815	14) echo "Erreur de rsync ; RERR_IPC"
816	echo "Erreur dans le code IPC"
817	return;;
818	20) echo "Erreur de rsync ; RERR_SIGNAL"
819	echo "SIGUSR1 ou SIGINT reçu"
820	return;;
821	21) echo "Erreur de rsync ; RERR_WAITCHILD"
822	echo "Une erreur retournée par waitpid()"
823	return;;
824	22) echo "Erreur de rsync ; RERR_MALLOC"
825	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
826	return;;
827	23) echo ""
828	echo "Erreur fichier inexistant"
829	return;;
830	30) echo "Erreur de rsync ; RERR_TIMEOUT"
831	echo "Temps d'attente écoulé dans l'envoi/réception de données"
832	return;;
833	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
834	return;;
835	esac
836	elif [ $MYLANG = "en" ] ; then
837	case $status in
838	0) return;;
839	1) echo "rsync error : Syntax or usage error "
840	return;;
841	2) echo "rsync error : Protocol incompatibility "
842	return;;
843	3) echo "rsync error : Errors selecting input/output files, dirs"
844	return;;
845	4) echo "rsync error : Requested action not supported: an attempt"
846	echo "was made to manipulate 64-bit files on a platform that cannot support"
847	echo "them; or an option was specified that is supported by the client and"
848	echo "not by the server."
849	return;;
850	5) echo "rsync error : Error starting client-server protocol"
851	return;;
852	10) echo "rsync error : Error in socket I/O "
853	return;;
854	11) echo "rsync error : Error in file I/O "
855	return;;
856	12) echo "rsync error : Error in rsync protocol data stream "
857	return;;
858	13) echo "rsync error : Errors with program diagnostics "
859	return;;
860	14) echo "rsync error : Error in IPC code "
861	return;;
862	20) echo "rsync error : Received SIGUSR1 or SIGINT "
863	return;;
864	21) echo "rsync error : Some error returned by waitpid() "
865	return;;
866	22) echo "rsync error : Error allocating core memory buffers "
867	return;;
868	23) echo "rsync error : Partial transfer due to error"
869	return;;
870	24) echo "rsync error : Partial transfer due to vanished source files"
871	return;;
872	30) echo "rsync error : Timeout in data send/receive "
873	return;;
874	*) echo "rsync error : return code of rsync unknown :" $status
875	return;;
876	esac
877	else
878	echo "unknown language $MYLANG."
879	return
880	fi
881	}
882
883	#D-#==================================================
884	#D-function IGCM_sys_Miror_libIGCM
885	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
886	#D-* Examples:
887	#D-
888	function IGCM_sys_Mirror_libIGCM {
889	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
890	if ( $DEBUG_sys ) ; then
891	echo "IGCM_sys_Mirror_libIGCM"
892	fi
893
894	typeset status
895
896	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
897
898	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
899	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
900	status=$?
901
902	if [ ${status} -gt 0 ] ; then
903	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
904	cat /tmp/out_command.$$
905	fi
906	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
907	}
908
909	#D-#==================================================
910	#D-function IGCM_sys_Cp
911	#D-* Purpose: generic cp
912	#D-* Examples:
913	#D-
914	function IGCM_sys_Cp {
915	IGCM_debug_PushStack "IGCM_sys_Cp" $@
916	if ( $DEBUG_sys ) ; then
917	echo "IGCM_sys_Cp :" $@
918	fi
919
920	typeset status
921
922	echo cp $@ > /tmp/out_command.$$ 2>&1
923	\cp $@ >> /tmp/out_command.$$ 2>&1
924	status=$?
925
926	if [ ${status} -gt 0 ] ; then
927	echo "IGCM_sys_Cp : error code ${status}"
928	cat /tmp/out_command.$$
929	IGCM_debug_Exit "IGCM_sys_Cp"
930	else
931	\rm /tmp/out_command.$$
932	fi
933	IGCM_debug_PopStack "IGCM_sys_Cp"
934	}
935
936	#D-#==================================================
937	#D-function IGCM_sys_Rm
938	#D-* Purpose: generic rm
939	#D-* Examples:
940	#D-
941	function IGCM_sys_Rm {
942	IGCM_debug_PushStack "IGCM_sys_Rm" $@
943	if ( $DEBUG_sys ) ; then
944	echo "IGCM_sys_Rm :" $@
945	fi
946
947	typeset status
948
949	echo rm $@ > /tmp/out_command.$$ 2>&1
950	\rm $@ >> /tmp/out_command.$$ 2>&1
951	status=$?
952
953	if [ ${status} -gt 0 ] ; then
954	echo "IGCM_sys_Rm : error code ${status}"
955	cat /tmp/out_command.$$
956	IGCM_debug_Exit "IGCM_sys_Rm"
957	else
958	\rm /tmp/out_command.$$
959	fi
960	IGCM_debug_PopStack "IGCM_sys_Rm"
961	}
962
963	#D-#==================================================
964	#D-function IGCM_sys_RmRunDir
965	#D-* Purpose: rm tmpdir (dummy function most of the time batch
966	#D- scheduler will do the job)
967	#D-* Examples:
968	#D-
969	function IGCM_sys_RmRunDir {
970	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
971	if ( $DEBUG_sys ) ; then
972	echo "IGCM_sys_RmRunDir :" $@
973	fi
974
975	typeset status
976
977	echo rm $@ > /tmp/out_command.$$ 2>&1
978	\rm $@ >> /tmp/out_command.$$ 2>&1
979	status=$?
980
981	if [ ${status} -gt 0 ] ; then
982	echo "IGCM_sys_RmRunDir : rm error code is ${status}."
983	cat /tmp/out_command.$$
984	IGCM_debug_Exit "IGCM_sys_RmRunDir"
985	else
986	\rm /tmp/out_command.$$
987	fi
988	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
989	}
990
991	#D-#==================================================
992	#D-function IGCM_sys_Mv
993	#D-* Purpose: generic move
994	#D-* Examples:
995	#D-
996	function IGCM_sys_Mv {
997	IGCM_debug_PushStack "IGCM_sys_Mv" $@
998	if ( $DEBUG_sys ) ; then
999	echo "IGCM_sys_Mv :" $@
1000	fi
1001
1002	if [ $DRYRUN = 0 ]; then
1003
1004	typeset status
1005
1006	echo mv $@ > /tmp/out_command.$$ 2>&1
1007	\mv $@ >> /tmp/out_command.$$ 2>&1
1008	status=$?
1009
1010	if [ ${status} -gt 0 ] ; then
1011	echo "IGCM_sys_Mv : error code ${status}"
1012	cat /tmp/out_command.$$
1013	IGCM_debug_Exit "IGCM_sys_Mv"
1014	else
1015	\rm /tmp/out_command.$$
1016	fi
1017	fi
1018
1019	IGCM_debug_PopStack "IGCM_sys_Mv"
1020	}
1021
1022	#D-#==================================================
1023	#D-function IGCM_sys_Put_Dir
1024	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1025	#D-* Examples:
1026	#D-
1027	function IGCM_sys_Put_Dir {
1028	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1029	if ( $DEBUG_sys ) ; then
1030	echo "IGCM_sys_Put_Dir :" $@
1031	fi
1032	if [ $DRYRUN = 0 ]; then
1033	if [ ! -d ${1} ] ; then
1034	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1035	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1036	return
1037	fi
1038
1039	typeset status
1040
1041	# Only if we use rsync
1042	#IGCM_sys_TestDirArchive $( dirname $2 )
1043	#
1044	#USUAL WAY
1045	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
1046	status=$?
1047
1048	if [ ${status} -gt 0 ] ; then
1049	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
1050	cat /tmp/out_command.$$
1051	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1052	else
1053	\rm /tmp/out_command.$$
1054	fi
1055	fi
1056	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1057	}
1058
1059	#D-#==================================================
1060	#D-function IGCM_sys_Get_Dir
1061	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1062	#D-* Examples:
1063	#D-
1064	function IGCM_sys_Get_Dir {
1065	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1066	if ( $DEBUG_sys ) ; then
1067	echo "IGCM_sys_Get_Dir :" $@
1068	fi
1069	if [ $DRYRUN = 0 ]; then
1070	# if [ ! -d ${1} ] ; then
1071	# echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
1072	# IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1073	# return
1074	# fi
1075
1076	typeset NB_ESSAI DELAI status i
1077	# number of tentative
1078	NB_ESSAI=3
1079	# time delay between tentative
1080	DELAI=2
1081
1082	# Only if we use rsync
1083	#IGCM_sys_TestDirArchive $( dirname $2 )
1084	#
1085	# USUAL WAY
1086	# add 'ccc_hsm get' (to demigrate all offline files) to reduce time of this command :
1087	ccc_hsm get -r $1
1088
1089	i=0
1090	while [ $i -lt $NB_ESSAI ] ; do
1091	\cp -ur $1 $2 >> /tmp/out_command.$$ 2>&1
1092	status=$?
1093	if [ ${status} -gt 0 ] ; then
1094	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
1095	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1096	sleep $DELAI
1097	else
1098	break
1099	fi
1100	(( i = i + 1 ))
1101	done
1102
1103	if [ ${status} -gt 0 ] ; then
1104	echo "IGCM_sys_Get_Dir : error."
1105	cat /tmp/out_command.$$
1106	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1107	else
1108	\rm /tmp/out_command.$$
1109	fi
1110	fi
1111	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1112	}
1113
1114	#D-#==================================================
1115	#D-function IGCM_sys_Get_Master
1116	#D-* Purpose: Copy a complete directory from MASTER filesystem
1117	#D-* Examples:
1118	#D-
1119	function IGCM_sys_Get_Master {
1120	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1121	if ( $DEBUG_sys ) ; then
1122	echo "IGCM_sys_Get_Master :" $@
1123	fi
1124	if [ $DRYRUN = 0 ]; then
1125	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1126	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1127	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1128	return
1129	fi
1130
1131	typeset NB_ESSAI DELAI status i
1132	# number of tentative
1133	NB_ESSAI=3
1134	# time delay between tentative
1135	DELAI=2
1136
1137	i=0
1138	while [ $i -lt $NB_ESSAI ] ; do
1139	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
1140	status=$?
1141	if [ ${status} -gt 0 ]; then
1142	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1143	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1144	sleep $DELAI
1145	else
1146	break
1147	fi
1148	(( i = i + 1 ))
1149	done
1150
1151	if [ ${status} -gt 0 ] ; then
1152	echo "IGCM_sys_Get_Master : error."
1153	cat /tmp/out_command.$$
1154	IGCM_debug_Exit "IGCM_sys_Get_Master"
1155	else
1156	\rm /tmp/out_command.$$
1157	fi
1158	fi
1159	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1160	}
1161
1162	#====================================================
1163	#- Call IGCM_sys_Mirror_libIGCM now !
1164	if ( $MirrorlibIGCM ) ; then
1165	IGCM_sys_Mirror_libIGCM
1166	fi
1167
1168	#D-#==================================================
1169	#D-function IGCM_sys_Put_Rest
1170	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1171	#D- File and target directory must exist.
1172	#D-* Examples:
1173	#D-
1174	function IGCM_sys_Put_Rest {
1175	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1176	if ( $DEBUG_sys ) ; then
1177	echo "IGCM_sys_Put_Rest :" $@
1178	fi
1179	if [ $DRYRUN = 0 ]; then
1180	if [ ! -f ${1} ] ; then
1181	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1182	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1183	fi
1184
1185	typeset status
1186	#
1187	# USUAL WAY
1188	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1189	status=$?
1190
1191	# #RSYNC WITH NETWORK SSH CALL
1192	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1193	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1194
1195	# #RSYNC WITH NFS USE
1196	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1197	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1198
1199	# status=$?
1200	# IGCM_sys_Rsync_out $status
1201
1202	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1203	# (( status=status+$? ))
1204
1205	if [ ${status} -gt 0 ] ; then
1206	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
1207	[ -f ${1} ] && ls -l ${1}
1208	[ -f ${2} ] && ls -l ${2}
1209	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1210	cat /tmp/out_command.$$
1211	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1212	else
1213
1214	if [ X${JobType} = XRUN ] ; then
1215	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1216	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1217	fi
1218
1219	\rm /tmp/out_command.$$
1220	fi
1221	fi
1222	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1223	}
1224
1225	#D-#==================================================
1226	#D-function IGCM_sys_PutBuffer_Rest
1227	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1228	#D- File and target directory must exist.
1229	#D-* Examples:
1230	#D-
1231	function IGCM_sys_PutBuffer_Rest {
1232	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1233	if ( $DEBUG_sys ) ; then
1234	echo "IGCM_sys_PutBuffer_Rest :" $@
1235	fi
1236	if [ $DRYRUN = 0 ]; then
1237	if [ ! -f ${1} ] ; then
1238	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1239	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1240	fi
1241
1242	typeset status
1243	#
1244	# USUAL WAY
1245	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1246	status=$?
1247
1248	if [ ${status} -gt 0 ] ; then
1249	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1250	[ -f ${2} ] && ls -l ${2}
1251	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1252	cat /tmp/out_command.$$
1253	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1254	else
1255
1256	if [ X${JobType} = XRUN ] ; then
1257	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1258	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1259	fi
1260
1261	\rm /tmp/out_command.$$
1262	fi
1263	fi
1264	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1265	}
1266
1267	#D-#==================================================
1268	#D-function IGCM_sys_PrepareTaredRestart
1269	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
1270	#D-* Examples:
1271	#D-
1272	function IGCM_sys_PrepareTaredRestart {
1273	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1274	if [ $DRYRUN = 0 ]; then
1275	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1276	fi
1277	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1278	}
1279
1280	#D-#==================================================
1281	#D-function IGCM_sys_Put_Out
1282	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1283	#D-* Examples:
1284	#D-
1285	function IGCM_sys_Put_Out {
1286	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1287	if ( $DEBUG_sys ) ; then
1288	echo "IGCM_sys_Put_Out :" $@
1289	fi
1290
1291	typeset NB_ESSAI DELAI status i exist skip
1292
1293	# number of tentative
1294	NB_ESSAI=3
1295	# time delay between tentative
1296	DELAI=2
1297
1298	if [ $DRYRUN = 0 ]; then
1299	if [ ! -f ${1} ] ; then
1300	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1301	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1302	return 1
1303	fi
1304	#
1305	IGCM_sys_MkdirArchive $( dirname $2 )
1306	#
1307
1308	#=====================================================
1309	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1310	#=====================================================
1311
1312	#echo ${2} \| grep "${R_OUT}" > /dev/null 2>&1
1313	#if [ $? -eq 0 ] ; then
1314	# typeset WORKPATH FILEPATH
1315	# WORKPATH=$( dirname $2 \| sed -e "s\|${R_OUT}\|${R_BUF}\|" )
1316	# IGCM_sys_MkdirWork ${WORKPATH}
1317	# FILEPATH=${WORKPATH}/$( basename $2 )
1318	# #
1319	# IGCM_sys_Cp ${1} ${FILEPATH}
1320	#fi
1321
1322	exist=false
1323	skip=false
1324	if [ -f $2 ] ; then
1325	IGCM_debug_Print 1 "$2 already exist"
1326	ccc_hsm get $2
1327	exist=true
1328	if [ "X$( diff $1 $2 )" = X ] ; then
1329	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1330	skip=true
1331	else
1332	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1333	skip=false
1334	fi
1335	fi
1336	#
1337	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1338	IGCM_sys_Chmod u+w $2
1339	fi
1340
1341	if [ X${skip} = Xfalse ] ; then
1342	i=0
1343	while [ $i -lt $NB_ESSAI ] ; do
1344	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1345	# USUAL WAY
1346	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1347	status=$?
1348	else
1349	# NOT SO USUAL WAY
1350	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1351	status=$?
1352	fi
1353	if [ ${status} -gt 0 ]; then
1354	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1355	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
1356	[ -f ${1} ] && ls -l ${1}
1357	[ -f ${2} ] && ls -l ${2}
1358	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1359	sleep $DELAI
1360	else
1361	break
1362	fi
1363	(( i = i + 1 ))
1364	done
1365	fi
1366
1367	# #RSYNC WITH NETWORK SSH CALL
1368	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1369	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1370
1371	# #RSYNC WITH NFS USE
1372	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1373	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1374
1375	# status=$?
1376	# IGCM_sys_Rsync_out $status
1377
1378	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1379	# (( status=status+$? ))
1380
1381	if [ ${status} -gt 0 ] ; then
1382	echo "IGCM_sys_Put_Out : error."
1383	[ -f ${1} ] && ls -l ${1}
1384	[ -f ${2} ] && ls -l ${2}
1385	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1386	cat /tmp/out_command.$$
1387	IGCM_debug_Exit "IGCM_sys_Put_Out"
1388	else
1389
1390	if [ X${JobType} = XRUN ] ; then
1391	if [ X${3} = X ] ; then
1392	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1393	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1394	fi
1395	fi
1396
1397	\rm /tmp/out_command.$$
1398	fi
1399	fi
1400	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1401	return 0
1402	}
1403
1404	#D-#==================================================
1405	#D-function IGCM_sys_PutBuffer_Out
1406	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1407	#D-* Examples:
1408	#D-
1409	function IGCM_sys_PutBuffer_Out {
1410	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1411	if ( $DEBUG_sys ) ; then
1412	echo "IGCM_sys_PutBuffer_Out :" $@
1413	fi
1414
1415	typeset NB_ESSAI DELAI status i exist skip
1416
1417	# number of tentative
1418	NB_ESSAI=3
1419	# time delay between tentative
1420	DELAI=2
1421
1422	if [ $DRYRUN = 0 ]; then
1423	if [ ! -f ${1} ] ; then
1424	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1425	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1426	return 1
1427	fi
1428	#
1429	IGCM_sys_Mkdir $( dirname $2 )
1430	#
1431
1432	exist=false
1433	skip=false
1434	if [ -f $2 ] ; then
1435	IGCM_debug_Print 1 "$2 already exist"
1436	exist=true
1437	if [ "X$( diff $1 $2 )" = X ] ; then
1438	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1439	status=0
1440	skip=true
1441	else
1442	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1443	skip=false
1444	fi
1445	fi
1446	#
1447	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1448	IGCM_sys_Chmod u+w $2
1449	fi
1450
1451	if [ X${skip} = Xfalse ] ; then
1452	i=0
1453	while [ $i -lt $NB_ESSAI ] ; do
1454	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1455	# USUAL WAY
1456	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1457	status=$?
1458	else
1459	# NOT SO USUAL WAY
1460	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1461	status=$?
1462	fi
1463	if [ ${status} -gt 0 ]; then
1464	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1465	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1466	[ -f ${2} ] && ls -l ${2}
1467	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1468	sleep $DELAI
1469	else
1470	break
1471	fi
1472	(( i = i + 1 ))
1473	done
1474	fi
1475
1476	if [ ${status} -gt 0 ] ; then
1477	echo "IGCM_sys_PutBuffer_Out : error."
1478	[ -f ${2} ] && ls -l ${2}
1479	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1480	cat /tmp/out_command.$$
1481	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1482	else
1483
1484	if [ X${JobType} = XRUN ] ; then
1485	if [ X${3} = X ] ; then
1486	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1487	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1488	fi
1489	fi
1490
1491	\rm /tmp/out_command.$$
1492	fi
1493	fi
1494	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1495	return 0
1496	}
1497
1498	#D-#==================================================
1499	#D-function IGCM_sys_Get
1500	#D-* Purpose: Get a file from ${ARCHIVE}
1501	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1502	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1503	function IGCM_sys_Get {
1504	IGCM_debug_PushStack "IGCM_sys_Get" $@
1505
1506	typeset DEST dm_liste target file_work
1507	typeset NB_ESSAI DELAI status i
1508
1509	if ( $DEBUG_sys ) ; then
1510	echo "IGCM_sys_Get :" $@
1511	fi
1512
1513	# number of tentative
1514	NB_ESSAI=3
1515	# time delay between tentative
1516	DELAI=2
1517
1518	if [ $DRYRUN -le 2 ]; then
1519	if [ X${1} = X'/l' ] ; then
1520	eval set +A dm_liste \${${2}}
1521	else
1522	eval set +A dm_liste ${1}
1523	fi
1524	eval DEST=\${${#}}
1525
1526	#=====================================================
1527	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1528	#=====================================================
1529
1530	# Is it an R_OUT file (not R_IN) ?
1531	#echo ${dm_liste[0]} \| grep "${R_OUT}" > /dev/null 2>&1
1532	#if [ $? -eq 0 ] ; then
1533	# # Yes ? then we try to get it in SCRATCHDIR
1534	# set +A file_work $( echo ${dm_liste[*]} \| sed -e "s\|${R_OUT}\|${R_BUF}\|g" )
1535	# if [ -f ${file_work[0]} ] ; then
1536	# IGCM_sys_Cp ${file_work[*]} ${DEST}
1537	# IGCM_debug_PopStack "IGCM_sys_Get"
1538	# return
1539	# fi
1540	#fi
1541
1542	ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1543	status=$?
1544	if [ ${status} -gt 0 ] ; then
1545	echo "WARNING IGCM_sys_Get : error code ${status}"
1546	cat /tmp/out_command.$$
1547	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
1548	fi
1549
1550	#if [ ${status} -gt 0 ] ; then
1551	# if [ ! "X$( grep "Lost dmusrcmd connection" /tmp/out_command.$$ )" = "X" ] ; then
1552	# cat /tmp/out_command.$$
1553	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
1554	# sleep 30
1555	# echo "We try another time"
1556	## dmget ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1557	# ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1558	# status=$?
1559	# if [ ${status} -gt 0 ] ; then
1560	# echo "ERROR IGCM_sys_Get : again demigration error :"
1561	# cat /tmp/out_command.$$
1562	# IGCM_debug_Exit "IGCM_sys_Get"
1563	# fi
1564	# else
1565	# echo "ERROR IGCM_sys_Get : demigration error :"
1566	# cat /tmp/out_command.$$
1567	# IGCM_debug_Exit "IGCM_sys_Get"
1568	# fi
1569	#fi
1570
1571	# #RSYNC WITH NETWORK SSH CALL
1572	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1573	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1574
1575	# #RSYNC WITH NFS USE
1576	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1577	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1578
1579	# status=$?
1580	# IGCM_sys_Rsync_out $status
1581
1582	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1583	# (( status=status+$? ))
1584
1585	#USUAL WAY
1586	if [ X${1} = X'/l' ] ; then
1587	for target in ${dm_liste[*]} ; do
1588	local_file=$( basename ${target} )
1589	# test if the target file is present before the loop
1590	IGCM_sys_TestFileArchive ${target}
1591	status=$?
1592	if [ ${status} -gt 0 ] ; then
1593	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
1594	IGCM_debug_Exit "IGCM_sys_Get"
1595	else
1596	i=0
1597	while [ $i -lt $NB_ESSAI ] ; do
1598	#if [ X${DoLink} = Xtrue ] ; then
1599	# \ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1600	# status=$?
1601	# else
1602	# \cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1603	# status=$?
1604	#fi
1605	\ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1606	status=$?
1607	if [ ${status} -gt 0 ]; then
1608	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1609	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1610	sleep $DELAI
1611	else
1612	break
1613	fi
1614	(( i = i + 1 ))
1615	done
1616	if [ ${status} -gt 0 ] ; then
1617	echo "IGCM_sys_Get : error"
1618	cat /tmp/out_command.$$
1619	\rm /tmp/out_command.$$
1620	IGCM_debug_Exit "IGCM_sys_Get"
1621	else
1622	\rm /tmp/out_command.$$
1623	fi
1624	fi
1625	done
1626	else
1627	i=0
1628	while [ $i -lt $NB_ESSAI ] ; do
1629	\cp ${dm_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1630	status=$?
1631	if [ ${status} -gt 0 ]; then
1632	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1633	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1634	sleep $DELAI
1635	else
1636	break
1637	fi
1638	(( i = i + 1 ))
1639	done
1640	if [ ${status} -gt 0 ] ; then
1641	echo "IGCM_sys_Get : error"
1642	cat /tmp/out_command.$$
1643	\rm /tmp/out_command.$$
1644	IGCM_debug_Exit "IGCM_sys_Get"
1645	else
1646	\rm /tmp/out_command.$$
1647	fi
1648	fi
1649	fi
1650	IGCM_debug_PopStack "IGCM_sys_Get"
1651	}
1652
1653	#D-#==================================================
1654	#D-function IGCM_sys_GetBuffer
1655	#D-* Purpose: Get a file from ${SCRATCHDIR}
1656	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1657	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1658	function IGCM_sys_GetBuffer {
1659	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1660
1661	typeset DEST buf_liste target file_work
1662	typeset NB_ESSAI DELAI status i
1663
1664	if ( $DEBUG_sys ) ; then
1665	echo "IGCM_sys_GetBuffer :" $@
1666	fi
1667
1668	# number of tentative
1669	NB_ESSAI=3
1670	# time delay between tentative
1671	DELAI=2
1672
1673	if [ $DRYRUN -le 2 ]; then
1674	if [ X${1} = X'/l' ] ; then
1675	eval set +A buf_liste \${${2}}
1676	else
1677	eval set +A buf_liste ${1}
1678	fi
1679	eval DEST=\${${#}}
1680
1681	#USUAL WAY
1682	if [ X${1} = X'/l' ] ; then
1683	for target in ${buf_liste[*]} ; do
1684	local_file=$( basename ${target} )
1685	i=0
1686	while [ $i -lt $NB_ESSAI ] ; do
1687	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1688	status=$?
1689	if [ ${status} -gt 0 ]; then
1690	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1691	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1692	sleep $DELAI
1693	else
1694	break
1695	fi
1696	(( i = i + 1 ))
1697	done
1698	if [ ${status} -gt 0 ] ; then
1699	echo "IGCM_sys_Get : error"
1700	cat /tmp/out_command.$$
1701	\rm /tmp/out_command.$$
1702	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1703	else
1704	\rm /tmp/out_command.$$
1705	fi
1706	done
1707	else
1708	i=0
1709	while [ $i -lt $NB_ESSAI ] ; do
1710	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1711	status=$?
1712	if [ ${status} -gt 0 ]; then
1713	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1714	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1715	sleep $DELAI
1716	else
1717	break
1718	fi
1719	(( i = i + 1 ))
1720	done
1721	if [ ${status} -gt 0 ] ; then
1722	echo "IGCM_sys_Get : error"
1723	cat /tmp/out_command.$$
1724	\rm /tmp/out_command.$$
1725	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1726	else
1727	\rm /tmp/out_command.$$
1728	fi
1729	fi
1730	fi
1731	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1732	}
1733
1734	#D-#==================================================
1735	#D-function IGCM_sys_GetDate_FichWork
1736	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1737	#D-* Examples:
1738	#D-
1739	function IGCM_sys_GetDate_FichWork {
1740	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1741	if ( $DEBUG_sys ) ; then
1742	echo "IGCM_sys_GetDate_FichWork :" $@
1743	fi
1744
1745	if [ $# -ge 3 ] ; then
1746	mode=$3
1747	TimeStyle=$4
1748	else
1749	mode="default"
1750	TimeStyle="%Y%m%d%H%M%S"
1751	fi
1752
1753	typeset dateF
1754	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1755
1756	case $mode in
1757	"default")
1758	eval ${2}=${dateF[5]}
1759	;;
1760	"SplitFields")
1761	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1762	;;
1763	esac
1764
1765
1766	# donne la date filesys d'un fichier sur la machine work
1767	IGCM_debug_PopStack "IGCM_sys_FichWork"
1768	}
1769
1770	#D-#==================================================
1771	#D-function IGCM_sys_GetDate_FichArchive
1772	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1773	#D-* Examples:
1774	#D-
1775	function IGCM_sys_GetDate_FichArchive {
1776	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1777	if ( $DEBUG_sys ) ; then
1778	echo "IGCM_sys_GetDate_FichArchive :" $@
1779	fi
1780	typeset dateF
1781	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1782	eval ${2}=${dateF[5]}
1783
1784	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1785	}
1786
1787	#D-#==================================================
1788	#D-function IGCM_sys_GetDate_Monitoring
1789	#D-* Purpose: get the last year for which the monitoring has been computed
1790	#D-* Examples:
1791	#D-
1792	function IGCM_sys_GetDate_Monitoring {
1793	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1794	if ( $DEBUG_sys ) ; then
1795	echo "IGCM_sys_GetDate_Monitoring :" $@
1796	fi
1797
1798	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1799
1800	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1801	}
1802
1803	#D-#==================================================
1804	#D-function IGCM_sys_Dods_Rm
1805	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1806	#D-* Examples:
1807	#D-
1808	function IGCM_sys_Dods_Rm {
1809	if ( $DEBUG_sys ) ; then
1810	echo "IGCM_sys_Dods_Rm :" $@
1811	fi
1812	typeset status
1813	if [ $DRYRUN = 0 ]; then
1814
1815	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
1816	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
1817	# echo "Nothing has been done."
1818	# return
1819	# fi
1820
1821	/ccc/cont003/home/dsm/p86ipsl/bin/dods_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
1822	status=$?
1823
1824	# if [ ${status} -gt 0 ] ; then
1825	# echo "IGCM_sys_Dods_Rm : error."
1826	# cat out_dods_rm
1827	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
1828	# else
1829	# rm out_dods_rm
1830	# fi
1831
1832	fi
1833	return $status
1834	}
1835
1836	#D-#==================================================
1837	#D-function IGCM_sys_Dods_Cp
1838	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1839	#D-* Examples:
1840	#D-
1841	function IGCM_sys_Dods_Cp {
1842	if ( $DEBUG_sys ) ; then
1843	echo "IGCM_sys_Dods_Cp :" $@
1844	fi
1845	typeset status
1846	if [ $DRYRUN = 0 ]; then
1847
1848	# if [ ! -d ${R_SAVE}/${1} ] ; then
1849	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
1850	# echo "Nothing has been done."
1851	# return
1852	# fi
1853
1854	/ccc/cont003/home/dsm/p86ipsl/bin/dods_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
1855	status=$?
1856
1857	# if [ ${status} -gt 0 ] ; then
1858	# echo "IGCM_sys_Dods_Cp : error."
1859	# cat out_dods_cp
1860	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
1861	# else
1862	# rm out_dods_cp
1863	# fi
1864
1865	fi
1866	return $status
1867	}
1868
1869	#D-#==================================================
1870	#D-function IGCM_sys_Put_Dods
1871	#D-* Purpose: Put $(ARCHIVE) files on DODS internet protocole.
1872	#D-* Examples:
1873	#D-
1874	function IGCM_sys_Put_Dods {
1875	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1876	if ( $DEBUG_sys ) ; then
1877	echo "IGCM_sys_Put_Dods :" $@
1878	fi
1879	#set -vx
1880	typeset status
1881	if [ $DRYRUN = 0 ]; then
1882	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
1883	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
1884	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
1885	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
1886	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1887	return
1888	fi
1889
1890	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1891	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1892	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1893	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1894	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1895	return
1896	fi
1897	#
1898	if [ -d ${R_SAVE}/${1} ] ; then
1899	cd ${R_SAVE}
1900	elif [ -d ${R_FIGR}/${1} ] ; then
1901	cd ${R_FIGR}
1902	fi
1903
1904	IGCM_sys_Dods_Rm ${1}
1905	IGCM_sys_Dods_Cp ${1}
1906	status=0
1907
1908	if [ ${status} -gt 0 ] ; then
1909	echo "IGCM_sys_Put_Dods : error."
1910	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1911	fi
1912	fi
1913	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1914	}
1915
1916	##############################################################
1917	# REBUILD OPERATOR
1918
1919	#D-#==================================================
1920	#D-function IGCM_sys_rebuild
1921	#D-* Purpose: rebuild parallel files
1922	#D-* Examples:
1923	#D-
1924	function IGCM_sys_rebuild {
1925	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1926	if ( $DEBUG_sys ) ; then
1927	echo "IGCM_sys_rebuild :" $@
1928	fi
1929
1930	typeset NB_ESSAI DELAI status i firstArg
1931	# number of tentative
1932	NB_ESSAI=3
1933	# time delay between tentative
1934	DELAI=2
1935
1936	i=0
1937	while [ $i -lt $NB_ESSAI ] ; do
1938	~p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1939	status=$?
1940	if [ ${status} -gt 0 ] ; then
1941	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1942	cat /tmp/out_command.$$
1943	\rm /tmp/out_command.$$
1944	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1945	firstArg=${1}
1946	\rm ${firstArg}
1947	sleep $DELAI
1948	else
1949	\rm /tmp/out_command.$$
1950	break
1951	fi
1952	(( i = i + 1 ))
1953	done
1954
1955	if [ ${status} -gt 0 ] ; then
1956	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1957	IGCM_debug_Exit "rebuild"
1958	fi
1959
1960	IGCM_debug_PopStack "IGCM_sys_rebuild"
1961	}
1962
1963	#D-#==================================================
1964	#D-function IGCM_sys_rebuild_station
1965	#D-* Purpose: rebuild parallel files describing station
1966	#D-* Examples:
1967	#D-
1968	function IGCM_sys_rebuild_station {
1969	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1970	typeset i list_opt file_in file_out prefix_invert list_invert
1971	if ( $DEBUG_sys ) ; then
1972	echo "IGCM_sys_rebuild_station :" $@
1973	fi
1974	list_opt=$@
1975
1976	# Invert Axis : t,x -> x,t
1977	# t,pres,x -> x,t,pres
1978	# So that we can concatenate along x
1979	i=0
1980	for file_in in ${list_opt} ; do
1981	(( i = i + 1))
1982	[ ${i} = 1 ] && file_out=${file_in} && continue
1983	# detect time counter and do the job only if present
1984	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
1985	if [ X${var_unlim} = Xtime_counter ] ; then
1986	prefix_invert=$( basename ${file_in} .nc )
1987	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1988	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1989	fi
1990	done
1991
1992	# Concatenate
1993	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1994
1995	# Re-ivert file
1996	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1997
1998	# Station re-ordering is too expansive to be run within libIGCM
1999	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
2000	# This re-ordering must be done "in memory" by the cmorization process
2001	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
2002	# BEGIN reordering
2003
2004	# Only LMDZ text output contains the exact ordering of the station.
2005	# We isolate this in the code below:
2006	# 0 38 -157.5000000000000 70.98591549295774
2007	# 0 54 27.49999999999999 67.18309859154928
2008	# 0 56 -62.50000000000001 82.39436619718309
2009	# 0 79 12.49999999999999 78.59154929577466
2010	# 0 116 -165.0000000000000 76.05633802816901
2011	# 0 117 130.0000000000000 70.98591549295774
2012	# 0 118 110.0000000000000 87.46478873239437
2013	# 1 40 4.999999999999995 51.97183098591550
2014
2015	# typeset iStation iProc list_opt file_in file_out prefix_invert
2016	# typeset -Z4 j4
2017	# typeset -Z3 j3
2018
2019	# unset list_opt
2020	# set +A list_opt $@
2021
2022	# Filename after rebuild
2023	# file_out=${list_opt[0]}
2024	# Prefix of output files
2025	# prefix_invert=$( basename ${file_out} .nc )
2026	# Number of procs
2027	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
2028
2029	# iProc=0
2030	# while [ ${iProc} -lt ${num_proc} ] ; do
2031	# Array containing Station as a number
2032	# unset proc_stn
2033	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
2034	# Number of stations produced by processor proc
2035	# stationLast=${#proc_stn[*]}
2036	# Proc number on 4 digits
2037	# j4=${iProc}
2038	# Init
2039	# iStation=0
2040	# while [ ${iStation} -lt ${stationLast} ] ; do
2041	# Station number on 3 digits
2042	# j3=${proc_stn[${iStation}]}
2043	# Extract station
2044	# Invert Axis : t,x -> x,t
2045	# t,pres,x -> x,t,pres
2046	# So that we can concatenate along x
2047	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
2048	# (( iStation = iStation + 1 ))
2049	# done
2050	# (( iProc = iProc + 1 ))
2051	# done
2052
2053	# Concatenate all station along x
2054	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
2055
2056	# Re-invert file
2057	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
2058
2059	# END reordering
2060
2061	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
2062	}
2063
2064	############################################################
2065	# Activate Running Environnment Variables
2066
2067	#D-#==================================================
2068	#D-function IGCM_sys_desactiv_variables
2069	#D-* Purpose: set environement variables prior to execution
2070	#D-* Examples:
2071	#D-
2072	function IGCM_sys_activ_variables {
2073	IGCM_debug_PushStack "IGCM_sys_activ_variables"
2074	if ( $DEBUG_sys ) ; then
2075	echo "IGCM_sys_activ_variables"
2076	fi
2077
2078	# --------------------------------------------------------------------
2079	#D- MPI specifications
2080	# --------------------------------------------------------------------
2081
2082	# --------------------------------------------------------------------
2083	#D- Other specifications
2084	# --------------------------------------------------------------------
2085
2086	ulimit -s unlimited
2087
2088	IGCM_debug_PopStack "IGCM_sys_activ_variables"
2089	}
2090
2091	############################################################
2092	# Desactivate Running Environnment Variables
2093
2094	#D-#==================================================
2095	#D-function IGCM_sys_desactiv_variables
2096	#D-* Purpose: unset environement variables after execution
2097	#D-* Examples:
2098	#D-
2099	function IGCM_sys_desactiv_variables {
2100	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
2101	if ( $DEBUG_sys ) ; then
2102	echo "IGCM_sys_desactiv_variables"
2103	fi
2104	# --------------------------------------------------------------------
2105	#D- MPI specifications
2106	# --------------------------------------------------------------------
2107
2108	# --------------------------------------------------------------------
2109	#D- Other specifications
2110	# --------------------------------------------------------------------
2111
2112	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
2113	}
2114
2115	############################################################
2116	# Build MPI/OMP scripts run file (dummy function)
2117
2118	#D-#==================================================
2119	#D-function IGCM_sys_build_run_file
2120	#D-* Purpose: build run file (deprecated)
2121	#D-* Examples:
2122	#D-
2123	function IGCM_sys_build_run_file {
2124
2125	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
2126
2127	}
2128
2129	############################################################
2130	# Build MPI/OMP scripts
2131
2132	#D-#==================================================
2133	#D-function IGCM_sys_build_execution_scripts
2134	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
2135	#D-* Examples:
2136	#D-
2137	function IGCM_sys_build_execution_scripts
2138	{
2139	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
2140	if ( $DEBUG_sys ) ; then
2141	echo "IGCM_sys_build_execution_scripts " $@
2142	fi
2143
2144	EXECUTION=${HOST_MPIRUN_COMMAND}
2145
2146	# MPMD mode
2147	if ( ${OK_PARA_MPMD} ) ; then
2148
2149	# Only MPI (MPMD)
2150	if ( ! ${OK_PARA_OMP} ) ; then
2151
2152	if [ -f run_file ] ; then
2153	IGCM_sys_Rm -f run_file
2154	fi
2155	touch run_file
2156
2157	# Build run_file
2158
2159	# First loop on the components for the coupler ie oasis (only if oasis3)
2160	# the coupler ie oasis3 must be the first one
2161	for comp in ${config_ListOfComponents[*]} ; do
2162
2163	eval ExeNameIn=\${config_Executable_${comp}[0]}
2164	eval ExeNameOut=\${config_Executable_${comp}[1]}
2165
2166	# for CPL component only
2167	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
2168
2169	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2170	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2171	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2172	fi
2173	done
2174
2175	# Then second loop on the components
2176	for comp in ${config_ListOfComponents[*]} ; do
2177
2178	eval ExeNameIn=\${config_Executable_${comp}[0]}
2179	eval ExeNameOut=\${config_Executable_${comp}[1]}
2180
2181	# Only if we really have an executable for the component and not the coupler ie oasis:
2182	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2183
2184	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2185	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2186	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2187	fi
2188	done
2189
2190	EXECUTION="${HOST_MPIRUN_COMMAND} -f ./run_file"
2191
2192	IGCM_sys_Chmod u+x run_file
2193	if ( $DEBUG_sys ) ; then
2194	echo "run_file contains : "
2195	cat run_file
2196	fi
2197
2198	# MPI-OpenMP (MPMD)
2199	else
2200
2201	# Use of mpirun instead of ccc_mprun
2202	EXECUTION="time mpirun"
2203
2204	# Hosts treatment
2205	${EXECUTION} hostname \| sort \| uniq > hosts.tmp
2206
2207	i=0
2208	rm -f hosts rankfile
2209	IGCM_debug_Print 1 "sys Curie, Hosts available :"
2210	for nodes in `cat hosts.tmp`
2211	do
2212	host[$i]=$nodes
2213	echo "${host[$i]}" >> hosts
2214	IGCM_debug_Print 1 ${host[$i]}
2215	i=$((i+1))
2216	done
2217	rm -f hosts.tmp
2218
2219	listnodes=${host[*]}
2220
2221	EXECUTION="${EXECUTION} -hostfile hosts -rankfile rankfile"
2222
2223	# Initialisation
2224	rank=0
2225	current_core=0
2226	core_per_node=16
2227	init_exec=n
2228
2229	# Loop on the components
2230	for comp in ${config_ListOfComponents[*]} ; do
2231
2232	eval ExeNameIn=\${config_Executable_${comp}[0]}
2233	eval ExeNameOut=\${config_Executable_${comp}[1]}
2234
2235	# Not possible if oasis has an executable (i.e old version of oasis3)
2236	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
2237	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
2238	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
2239	IGCM_debug_Verif_Exit
2240	fi
2241
2242	# Only if we really have an executable for the component :
2243	if [ "X${ExeNameOut}" != X\"\" ] ; then
2244
2245	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2246	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2247
2248	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2249	echo "" >> script_${ExeNameOut}.ksh
2250	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
2251
2252	# Check if the number of threads is correct
2253	case ${comp_proc_omp_loc} in
2254	2\|4\|8\|16)
2255	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2256	;;
2257	*)
2258	IGCM_debug_Exit "ERROR with OMP parameters !"
2259	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2260	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2261	IGCM_debug_Verif_Exit
2262	;;
2263	esac
2264	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2265	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2266	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2267	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2268	fi
2269	#echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
2270	#echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
2271	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2272	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2273
2274	if [ ${init_exec} = y ] ; then
2275	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2276	else
2277	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2278	init_exec=y
2279	fi
2280
2281	# Build rankfile : method used to assign cores and nodes for the MPI process
2282	# Ex :
2283	#rank 0=curie5296 slot=0,1,2,3
2284	#rank 1=curie5296 slot=4,5,6,7
2285	# Example of final command :
2286	# mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
2287	# with script_lmdz.x.ksh :
2288	# #!/bin/ksh
2289	#export KMP_STACKSIZE=3g
2290	#export KMP_LIBRARY=turnaround
2291	#export MKL_SERIAL=YES
2292	#OMP_NUM_THREADS=4
2293	#./lmdz.x
2294	#
2295
2296	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
2297	(( index_host = current_core / core_per_node ))
2298	host_value=${host[${index_host}]}
2299	(( slot = current_core % core_per_node ))
2300	virg=","
2301	string_final=""
2302	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
2303	string=$index$virg
2304	string_final=$string_final$string
2305	done
2306	string_final=$( echo $string_final \| sed "s/.$//" )
2307	echo "rank $rank=$host_value slot=$string_final" >> rankfile
2308	(( rank = rank + 1 ))
2309	(( current_core = current_core + comp_proc_omp_loc ))
2310	done
2311	fi
2312
2313	done
2314	fi
2315
2316	# Only one executable (SPMD mode).
2317	else
2318
2319	for comp in ${config_ListOfComponents[*]} ; do
2320
2321	# Only if we really have an executable for the component :
2322	eval ExeNameOut=\${config_Executable_${comp}[1]}
2323	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2324
2325	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2326	echo "" >> script_${ExeNameOut}.ksh
2327	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2328
2329	if ( ${OK_PARA_OMP} ) ; then
2330	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2331	echo "" >> script_${ExeNameOut}.ksh
2332	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2333	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2334	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2335	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2336	fi
2337
2338	if ( ${OK_PARA_MPI} ) ; then
2339	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2340	# Default : ccc_mprun used if nb_proc gt 1
2341	# to have out/err per process on different files
2342	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
2343	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2344	EXECUTION="${HOST_MPIRUN_COMMAND} -n ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2345	else
2346	# Default : ccc_mprun is NOT used if nb_proc eq 1
2347	# to have out/err per process on different files
2348	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2349	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2350	EXECUTION="/usr/bin/time ./script_${ExeNameOut}.ksh"
2351	fi
2352
2353	IGCM_debug_Print 1 "sys Curie : script_${ExeNameOut}.ksh contains"
2354	cat script_${ExeNameOut}.ksh
2355
2356	fi
2357
2358	done
2359
2360	fi
2361
2362	IGCM_debug_Print 1 "sys Curie : execution command is "
2363	IGCM_debug_Print 1 "$EXECUTION"
2364
2365	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2366	}
2367
2368	#D-#==================================================
2369	#D-function IGCM_sys_check_path
2370	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2371	#D-* do not point to an important use directory. Stop immediately in that case.
2372	#D-* Examples:
2373	#D-
2374	function IGCM_sys_check_path {
2375	IGCM_debug_PushStack "IGCM_sys_check_path"
2376	if ( $DEBUG_sys ) ; then
2377	echo "IGCM_sys_check_path"
2378	fi
2379
2380	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCHDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCWORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCSTOREDIR} ] ) ; then
2381	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2382	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2383	IGCM_debug_Exit "This will stop the job"
2384	fi
2385	IGCM_debug_PopStack "IGCM_sys_check_path"
2386	}
2387
2388	#D-#==================================================
2389	#D-function IGCM_sys_check_quota
2390	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2391	#D-* Examples:
2392	#D-
2393	function IGCM_sys_check_quota {
2394	IGCM_debug_PushStack "IGCM_sys_check_quota"
2395	if ( $DEBUG_sys ) ; then
2396	echo "IGCM_sys_check_quota"
2397	fi
2398	# Limit of quota (in %)
2399	limit_quota=90
2400
2401	# Check of the volume
2402	volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
2403	volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
2404
2405	if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
2406
2407	unit_avail=${volume_avail: -1}
2408	unit_quota=${volume_quota: -1}
2409
2410	if [ "${unit_quota}" = "*" ] ; then
2411	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2412	IGCM_debug_Print 1 "More than 100% of your quota is used"
2413	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2414	IGCM_debug_Print 1 "You must have more than 10% available to run"
2415	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2416	IGCM_debug_Verif_Exit
2417	fi
2418
2419	temp_avail=${volume_avail%%${volume_avail: -1}*}
2420	temp_quota=${volume_quota%%${volume_quota: -1}*}
2421
2422	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2423
2424	# Convertion
2425	if [ ${volume_avail: -1} = "T" ] ; then
2426	(( temp_avail = temp_avail * 1000000000000 ))
2427	elif [ ${volume_avail: -1} = "G" ] ; then
2428	(( temp_avail = temp_avail * 1000000000 ))
2429	elif [ ${volume_avail: -1} = "M" ] ; then
2430	(( temp_avail = temp_avail * 1000000 ))
2431	elif [ ${volume_avail: -1} = "k" ] ; then
2432	(( temp_avail = temp_avail * 1000 ))
2433	else
2434	(( temp_avail = volume_avail ))
2435	fi
2436	if [ ${volume_quota: -1} = "T" ] ; then
2437	(( temp_quota = temp_quota * 1000000000000 ))
2438	elif [ ${volume_quota: -1} = "G" ] ; then
2439	(( temp_quota = temp_quota * 1000000000 ))
2440	elif [ ${volume_quota: -1} = "M" ] ; then
2441	(( temp_quota = temp_quota * 1000000 ))
2442	elif [ ${volume_quota: -1} = "k" ] ; then
2443	(( temp_quota = temp_quota * 1000 ))
2444	else
2445	(( temp_quota = volume_quota ))
2446	fi
2447	fi
2448
2449	quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2450	# echo "volume ratio is " $quota_volume
2451
2452	if [ ${quota_volume} -ge ${limit_quota} ] ; then
2453	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2454	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2455	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2456	IGCM_debug_Print 1 "You must have more than 10% available to run"
2457	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2458	IGCM_debug_Verif_Exit
2459	fi
2460
2461	fi
2462
2463	# Check of the number of inodes
2464
2465	inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
2466	inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
2467
2468	if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
2469
2470	unit_avail=${inode_avail: -1}
2471	unit_quota=${inode_quota: -1}
2472
2473	if [ "${unit_quota}" = "*" ] ; then
2474	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2475	IGCM_debug_Print 1 "More than 100% of your quota is used"
2476	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2477	IGCM_debug_Print 1 "You must have more than 10% available to run"
2478	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2479	IGCM_debug_Verif_Exit
2480	fi
2481
2482	temp_avail=${inode_avail%%${inode_avail: -1}*}
2483	temp_quota=${inode_quota%%${inode_quota: -1}*}
2484
2485	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2486
2487	# Convertion
2488	if [ ${inode_avail: -1} = "T" ] ; then
2489	(( temp_avail = temp_avail * 1000000000000 ))
2490	elif [ ${inode_avail: -1} = "G" ] ; then
2491	(( temp_avail = temp_avail * 1000000000 ))
2492	elif [ ${inode_avail: -1} = "M" ] ; then
2493	(( temp_avail = temp_avail * 1000000 ))
2494	elif [ ${inode_avail: -1} = "k" ] ; then
2495	(( temp_avail = temp_avail * 1000 ))
2496	else
2497	(( temp_avail = inode_avail ))
2498	fi
2499
2500	if [ ${inode_quota: -1} = "T" ] ; then
2501	(( temp_quota = temp_quota * 1000000000000 ))
2502	elif [ ${inode_quota: -1} = "G" ] ; then
2503	(( temp_quota = temp_quota * 1000000000 ))
2504	elif [ ${inode_quota: -1} = "M" ] ; then
2505	(( temp_quota = temp_quota * 1000000 ))
2506	elif [ ${inode_quota: -1} = "k" ] ; then
2507	(( temp_quota = temp_quota * 1000 ))
2508	else
2509	(( temp_quota = inode_quota ))
2510	fi
2511	fi
2512	quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2513	# echo "inode ratio is " $quota_inode
2514
2515	if [ ${quota_inode} -ge ${limit_quota} ] ; then
2516	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2517	IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
2518	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2519	IGCM_debug_Print 1 "You must have more than 10% available to run"
2520	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2521	IGCM_debug_Verif_Exit
2522	fi
2523	fi
2524	IGCM_debug_PopStack "IGCM_sys_check_quota"
2525	}
2526
2527	#D-#==================================================
2528	#D-function IGCM_sys_GetJobID
2529	#D-* Purpose: Check if job_name is currently
2530	#D- running or in queue
2531	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2532	#D-
2533	function IGCM_sys_GetJobID {
2534	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2535	if ( $DEBUG_sys ) ; then
2536	echo "IGCM_sys_GetJobID"
2537	fi
2538
2539	# With -f option, the full job name is given in the last column
2540	ID=$( ccc_mstat -f -u $2 \| \
2541	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2542
2543	eval ${3}=${ID}
2544
2545	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2546	}
2547
2548	#D-#==================================================
2549	#D-function IGCM_sys_CountJobInQueue
2550	#D-* Purpose: Check if job_name is currently
2551	#D- running or in queue
2552	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2553	#D-
2554	function IGCM_sys_CountJobInQueue {
2555	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2556	if ( $DEBUG_sys ) ; then
2557	echo "IGCM_sys_CountJobInQueue"
2558	fi
2559
2560	#NbRun=$( ccc_mstat -f \| grep -c ${JobName} )
2561
2562	# With -f option, the full job name is given in the last column
2563	NbRun=$( ccc_mstat -f \| gawk -v JobName=$1 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
2564
2565	eval ${2}=${NbRun}
2566
2567	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2568	}
2569
2570	#D-#==================================================
2571	#D-function IGCM_sys_ListJobInQueue
2572	#D-* Purpose: Check if job_name is currently
2573	#D- running or in queue
2574	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2575	#D-
2576	function IGCM_sys_ListJobInQueue {
2577	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2578	if ( $DEBUG_sys ) ; then
2579	echo "IGCM_sys_ListJobInQueue"
2580	fi
2581
2582	# With -f option, the full job name is given in the last column
2583	set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
2584	'( $2 == User && \
2585	$NF != /TS/ && \
2586	$NF !~ /PACK/ && \
2587	$NF !~ /REBUILD/ && \
2588	$NF !~ /pack/ ) \
2589	{ print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
2590
2591	eval set -A ${2} ${JobList[*]}
2592
2593	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2594	}
2595
2596	##############################################################
2597	# NCO OPERATOR
2598
2599	#D-#==================================================
2600	#D-function IGCM_sys_ncap2
2601	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2602	#D-* Examples:
2603	#D-
2604	function IGCM_sys_ncap2 {
2605	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2606	if ( $DEBUG_sys ) ; then
2607	echo "IGCM_sys_ncap2 :" $@
2608	fi
2609
2610	typeset NB_ESSAI DELAI status i
2611	# number of tentative
2612	NB_ESSAI=3
2613	# time delay between tentative
2614	DELAI=2
2615
2616	i=0
2617	while [ $i -lt $NB_ESSAI ] ; do
2618	ncap2 -C "$@" > /tmp/out_command.$$ 2>&1
2619	status=$?
2620	if [ ${status} -gt 0 ] ; then
2621	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2622	cat /tmp/out_command.$$
2623	\rm /tmp/out_command.$$
2624	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2625	sleep $DELAI
2626	else
2627	\rm /tmp/out_command.$$
2628	break
2629	fi
2630	(( i = i + 1 ))
2631	done
2632
2633	if [ ${status} -gt 0 ] ; then
2634	echo "IGCM_sys_ncap2 : ncap2 error"
2635	IGCM_debug_Exit "ncap2"
2636	fi
2637
2638	IGCM_debug_PopStack "IGCM_sys_ncap2"
2639	}
2640
2641	#D-#==================================================
2642	#D-function IGCM_sys_ncatted
2643	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2644	#D-* Examples:
2645	#D-
2646	function IGCM_sys_ncatted {
2647	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2648	if ( $DEBUG_sys ) ; then
2649	echo "IGCM_sys_ncatted :" $@
2650	fi
2651
2652	typeset NB_ESSAI DELAI status i
2653	# number of tentative
2654	NB_ESSAI=3
2655	# time delay between tentative
2656	DELAI=2
2657
2658	i=0
2659	while [ $i -lt $NB_ESSAI ] ; do
2660	ncatted "$@" > /tmp/out_command.$$ 2>&1
2661	status=$?
2662	if [ ${status} -gt 0 ] ; then
2663	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2664	cat /tmp/out_command.$$
2665	\rm /tmp/out_command.$$
2666	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2667	sleep $DELAI
2668	else
2669	\rm /tmp/out_command.$$
2670	break
2671	fi
2672	(( i = i + 1 ))
2673	done
2674
2675	if [ ${status} -gt 0 ] ; then
2676	echo "IGCM_sys_ncatted : ncatted error"
2677	IGCM_debug_Exit "ncatted"
2678	fi
2679
2680	IGCM_debug_PopStack "IGCM_sys_ncatted"
2681	}
2682
2683	#D-#==================================================
2684	#D-function IGCM_sys_ncbo
2685	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2686	#D-* Examples:
2687	#D-
2688	function IGCM_sys_ncbo {
2689	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2690	if ( $DEBUG_sys ) ; then
2691	echo "IGCM_sys_ncbo :" $@
2692	fi
2693
2694	typeset NB_ESSAI DELAI status i
2695	# number of tentative
2696	NB_ESSAI=3
2697	# time delay between tentative
2698	DELAI=2
2699
2700	i=0
2701	while [ $i -lt $NB_ESSAI ] ; do
2702	ncbo -C $@ > /tmp/out_command.$$ 2>&1
2703	status=$?
2704	if [ ${status} -gt 0 ] ; then
2705	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2706	cat /tmp/out_command.$$
2707	\rm /tmp/out_command.$$
2708	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2709	sleep $DELAI
2710	else
2711	\rm /tmp/out_command.$$
2712	break
2713	fi
2714	(( i = i + 1 ))
2715	done
2716
2717	if [ ${status} -gt 0 ] ; then
2718	echo "IGCM_sys_ncbo : ncbo error"
2719	IGCM_debug_Exit "ncbo"
2720	fi
2721
2722	IGCM_debug_PopStack "IGCM_sys_ncbo"
2723	}
2724
2725	#D-#==================================================
2726	#D-function IGCM_sys_ncdif
2727	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2728	#D-* Examples:
2729	#D-
2730	function IGCM_sys_ncdiff {
2731	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2732	if ( $DEBUG_sys ) ; then
2733	echo "IGCM_sys_ncdiff :" $@
2734	fi
2735
2736	typeset NB_ESSAI DELAI status i
2737	# number of tentative
2738	NB_ESSAI=3
2739	# time delay between tentative
2740	DELAI=2
2741
2742	i=0
2743	while [ $i -lt $NB_ESSAI ] ; do
2744	ncdiff -C $@ > /tmp/out_command.$$ 2>&1
2745	status=$?
2746	if [ ${status} -gt 0 ] ; then
2747	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2748	cat /tmp/out_command.$$
2749	\rm /tmp/out_command.$$
2750	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2751	sleep $DELAI
2752	else
2753	\rm /tmp/out_command.$$
2754	break
2755	fi
2756	(( i = i + 1 ))
2757	done
2758
2759	if [ ${status} -gt 0 ] ; then
2760	echo "IGCM_sys_ncdiff : ncdiff error"
2761	IGCM_debug_Exit "ncdiff"
2762	fi
2763
2764	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2765	}
2766
2767	#D-#==================================================
2768	#D-function IGCM_sys_ncea
2769	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2770	#D-* Examples:
2771	#D-
2772	function IGCM_sys_ncea {
2773	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2774	if ( $DEBUG_sys ) ; then
2775	echo "IGCM_sys_ncea :" $@
2776	fi
2777
2778	typeset NB_ESSAI DELAI status i
2779	# number of tentative
2780	NB_ESSAI=3
2781	# time delay between tentative
2782	DELAI=2
2783
2784	i=0
2785	while [ $i -lt $NB_ESSAI ] ; do
2786	ncea -C $@ > /tmp/out_command.$$ 2>&1
2787	status=$?
2788	if [ ${status} -gt 0 ] ; then
2789	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2790	cat /tmp/out_command.$$
2791	\rm /tmp/out_command.$$
2792	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2793	sleep $DELAI
2794	else
2795	\rm /tmp/out_command.$$
2796	break
2797	fi
2798	(( i = i + 1 ))
2799	done
2800
2801	if [ ${status} -gt 0 ] ; then
2802	echo "IGCM_sys_ncea : ncea error"
2803	IGCM_debug_Exit "ncea"
2804	fi
2805
2806	IGCM_debug_PopStack "IGCM_sys_ncea"
2807	}
2808
2809	#D-#==================================================
2810	#D-function IGCM_sys_ncecat
2811	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2812	#D-* Examples:
2813	#D-
2814	function IGCM_sys_ncecat {
2815	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2816	if ( $DEBUG_sys ) ; then
2817	echo "IGCM_sys_ncecat :" $@
2818	fi
2819
2820	typeset NB_ESSAI DELAI status i
2821	# number of tentative
2822	NB_ESSAI=3
2823	# time delay between tentative
2824	DELAI=2
2825
2826	i=0
2827	while [ $i -lt $NB_ESSAI ] ; do
2828	ncecat -C $@ > /tmp/out_command.$$ 2>&1
2829	status=$?
2830	if [ ${status} -gt 0 ] ; then
2831	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2832	cat /tmp/out_command.$$
2833	\rm /tmp/out_command.$$
2834	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2835	sleep $DELAI
2836	else
2837	\rm /tmp/out_command.$$
2838	break
2839	fi
2840	(( i = i + 1 ))
2841	done
2842
2843	if [ ${status} -gt 0 ] ; then
2844	echo "IGCM_sys_ncecat : ncecat error"
2845	IGCM_debug_Exit "ncecat"
2846	fi
2847
2848	IGCM_debug_PopStack "IGCM_sys_ncecat"
2849	}
2850
2851	#D-#==================================================
2852	#D-function IGCM_sys_ncflint
2853	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2854	#D-* Examples:
2855	#D-
2856	function IGCM_sys_ncflint {
2857	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2858	if ( $DEBUG_sys ) ; then
2859	echo "IGCM_sys_ncflint :" $@
2860	fi
2861
2862	typeset NB_ESSAI DELAI status i
2863	# number of tentative
2864	NB_ESSAI=3
2865	# time delay between tentative
2866	DELAI=2
2867
2868	i=0
2869	while [ $i -lt $NB_ESSAI ] ; do
2870	ncflint -C $@ > /tmp/out_command.$$ 2>&1
2871	status=$?
2872	if [ ${status} -gt 0 ] ; then
2873	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2874	cat /tmp/out_command.$$
2875	\rm /tmp/out_command.$$
2876	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2877	sleep $DELAI
2878	else
2879	\rm /tmp/out_command.$$
2880	break
2881	fi
2882	(( i = i + 1 ))
2883	done
2884
2885	if [ ${status} -gt 0 ] ; then
2886	echo "IGCM_sys_ncflint : ncflint error"
2887	IGCM_debug_Exit "ncflint"
2888	fi
2889
2890	IGCM_debug_PopStack "IGCM_sys_ncflint"
2891	}
2892
2893	#D-#==================================================
2894	#D-function IGCM_sys_ncks
2895	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2896	#D-* Examples:
2897	#D-
2898	function IGCM_sys_ncks {
2899	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2900	if ( $DEBUG_sys ) ; then
2901	echo "IGCM_sys_ncks :" $@
2902	fi
2903
2904	typeset NB_ESSAI DELAI status i
2905	# number of tentative
2906	NB_ESSAI=3
2907	# time delay between tentative
2908	DELAI=2
2909
2910	i=0
2911	while [ $i -lt $NB_ESSAI ] ; do
2912	ncks -C $@ > /tmp/out_command.$$ 2>&1
2913	status=$?
2914	if [ ${status} -gt 0 ] ; then
2915	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2916	cat /tmp/out_command.$$
2917	\rm /tmp/out_command.$$
2918	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2919	sleep $DELAI
2920	else
2921	\rm /tmp/out_command.$$
2922	break
2923	fi
2924	(( i = i + 1 ))
2925	done
2926
2927	if [ ${status} -gt 0 ] ; then
2928	echo "IGCM_sys_ncks : ncks error"
2929	IGCM_debug_Exit "ncks"
2930	fi
2931
2932	IGCM_debug_PopStack "IGCM_sys_ncks"
2933	}
2934
2935	#D-#==================================================
2936	#D-function IGCM_sys_ncpdq
2937	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2938	#D-* Examples:
2939	#D-
2940	function IGCM_sys_ncpdq {
2941	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2942	if ( $DEBUG_sys ) ; then
2943	echo "IGCM_sys_ncpdq :" $@
2944	fi
2945
2946	typeset NB_ESSAI DELAI status i
2947	# number of tentative
2948	NB_ESSAI=3
2949	# time delay between tentative
2950	DELAI=2
2951
2952	i=0
2953	while [ $i -lt $NB_ESSAI ] ; do
2954	ncpdq -C $@ > /tmp/out_command.$$ 2>&1
2955	status=$?
2956	if [ ${status} -gt 0 ] ; then
2957	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2958	cat /tmp/out_command.$$
2959	\rm /tmp/out_command.$$
2960	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2961	sleep $DELAI
2962	else
2963	\rm /tmp/out_command.$$
2964	break
2965	fi
2966	(( i = i + 1 ))
2967	done
2968
2969	if [ ${status} -gt 0 ] ; then
2970	echo "IGCM_sys_ncpdq : ncpdq error"
2971	IGCM_debug_Exit "ncpdq"
2972	fi
2973
2974	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2975	}
2976
2977	#D-#==================================================
2978	#D-function IGCM_sys_ncra
2979	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2980	#D-* Examples:
2981	#D-
2982	function IGCM_sys_ncra {
2983	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2984	if ( $DEBUG_sys ) ; then
2985	echo "IGCM_sys_ncra :" $@
2986	fi
2987
2988	typeset NB_ESSAI DELAI status i
2989	# number of tentative
2990	NB_ESSAI=3
2991	# time delay between tentative
2992	DELAI=2
2993
2994	i=0
2995	while [ $i -lt $NB_ESSAI ] ; do
2996	ncra -C $@ > /tmp/out_command.$$ 2>&1
2997	status=$?
2998	if [ ${status} -gt 0 ] ; then
2999	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
3000	cat /tmp/out_command.$$
3001	\rm /tmp/out_command.$$
3002	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3003	sleep $DELAI
3004	else
3005	\rm /tmp/out_command.$$
3006	break
3007	fi
3008	(( i = i + 1 ))
3009	done
3010
3011	if [ ${status} -gt 0 ] ; then
3012	echo "IGCM_sys_ncra : ncra error"
3013	IGCM_debug_Exit "ncra"
3014	fi
3015
3016	IGCM_debug_PopStack "IGCM_sys_ncra"
3017	}
3018
3019	#D-#==================================================
3020	#D-function IGCM_sys_ncrcat
3021	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
3022	#D-* Examples:
3023	#D-
3024	function IGCM_sys_ncrcat {
3025	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
3026	if ( $DEBUG_sys ) ; then
3027	echo "IGCM_sys_ncrcat :" $@
3028	fi
3029
3030	typeset NB_ESSAI DELAI status i lastArg
3031	# number of tentative
3032	NB_ESSAI=3
3033	# time delay between tentative
3034	DELAI=2
3035
3036	i=0
3037	while [ $i -lt $NB_ESSAI ] ; do
3038	ncrcat -C $@ > /tmp/out_command.$$ 2>&1
3039	status=$?
3040	if [ ${status} -gt 0 ] ; then
3041	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
3042	cat /tmp/out_command.$$
3043	\rm /tmp/out_command.$$
3044	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3045	sleep $DELAI
3046	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
3047	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
3048	cat /tmp/out_command.$$
3049	# remove files having corrupted time axis
3050	eval lastArg=\${$#}
3051	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
3052	\rm ${lastArg}
3053	\rm /tmp/out_command.$$
3054	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3055	sleep $DELAI
3056	else
3057	\rm /tmp/out_command.$$
3058	break
3059	fi
3060	(( i = i + 1 ))
3061	done
3062
3063	if [ ${status} -gt 0 ] ; then
3064	echo "IGCM_sys_ncrcat : ncrcat error"
3065	#IGCM_debug_Exit "ncrcat"
3066	fi
3067
3068	IGCM_debug_PopStack "IGCM_sys_ncrcat"
3069	}
3070
3071	#D-#==================================================
3072	#D-function IGCM_sys_ncrename
3073	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
3074	#D-* Examples:
3075	#D-
3076	function IGCM_sys_ncrename {
3077	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
3078	if ( $DEBUG_sys ) ; then
3079	echo "IGCM_sys_ncrename :" $@
3080	fi
3081
3082	typeset NB_ESSAI DELAI status i
3083	# number of tentative
3084	NB_ESSAI=3
3085	# time delay between tentative
3086	DELAI=2
3087
3088	i=0
3089	while [ $i -lt $NB_ESSAI ] ; do
3090	ncrename $@ > /tmp/out_command.$$ 2>&1
3091	status=$?
3092	if [ ${status} -gt 0 ] ; then
3093	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
3094	cat /tmp/out_command.$$
3095	\rm /tmp/out_command.$$
3096	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3097	sleep $DELAI
3098	else
3099	\rm /tmp/out_command.$$
3100	break
3101	fi
3102	(( i = i + 1 ))
3103	done
3104
3105	if [ ${status} -gt 0 ] ; then
3106	echo "IGCM_sys_ncrename : ncrename error"
3107	IGCM_debug_Exit "ncrename"
3108	fi
3109
3110	IGCM_debug_PopStack "IGCM_sys_ncrename"
3111	}
3112
3113	#D-#==================================================
3114	#D-function IGCM_sys_ncwa
3115	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
3116	#D-* Examples:
3117	#D-
3118	function IGCM_sys_ncwa {
3119	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
3120	if ( $DEBUG_sys ) ; then
3121	echo "IGCM_sys_ncwa :" $@
3122	fi
3123
3124	typeset NB_ESSAI DELAI status i
3125	# number of tentative
3126	NB_ESSAI=3
3127	# time delay between tentative
3128	DELAI=2
3129
3130	i=0
3131	while [ $i -lt $NB_ESSAI ] ; do
3132	ncwa -C $@ > /tmp/out_command.$$ 2>&1
3133	status=$?
3134	if [ ${status} -gt 0 ] ; then
3135	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
3136	cat /tmp/out_command.$$
3137	\rm /tmp/out_command.$$
3138	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3139	sleep $DELAI
3140	else
3141	\rm /tmp/out_command.$$
3142	break
3143	fi
3144	(( i = i + 1 ))
3145	done
3146
3147	if [ ${status} -gt 0 ] ; then
3148	echo "IGCM_sys_ncwa : ncwa error"
3149	IGCM_debug_Exit "ncwa"
3150	fi
3151
3152	IGCM_debug_PopStack "IGCM_sys_ncwa"
3153	}
3154
3155	##############################################################
3156	# CDO OPERATOR
3157
3158	#D-#==================================================
3159	#D-function IGCM_sys_cdo
3160	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
3161	#D-* Examples:
3162	#D-
3163	function IGCM_sys_cdo {
3164	IGCM_debug_PushStack "IGCM_sys_cdo" $@
3165	if ( $DEBUG_sys ) ; then
3166	echo "IGCM_sys_cdo :" $@
3167	fi
3168
3169	typeset status
3170
3171	\cdo $@ > /tmp/out_command.$$ 2>&1
3172	status=$?
3173	if [ ${status} -gt 0 ] ; then
3174	echo "IGCM_sys_cdo : error code ${status}"
3175	cat /tmp/out_command.$$
3176	\rm /tmp/out_command.$$
3177	IGCM_debug_PopStack "IGCM_sys_cdo"
3178	return 1
3179	else
3180	IGCM_debug_PopStack "IGCM_sys_cdo"
3181	return 0
3182	fi
3183
3184	IGCM_debug_PopStack "IGCM_sys_cdo"
3185	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: