Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_curie.ksh @ 1166

Last change on this file since 1166 was 1166, checked in by sdipsl, 9 years ago
the metrics template need more information create IGCM_comp_modifyFile function to simply replace pattern in files code cleanup
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 93.6 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Curie
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=curie
71	# add default project on curie
72	typeset DEFAULT_PROJECT=gen0826
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# Submit command
80	typeset SUBMIT=${SUBMIT:=ccc_msub}
81	# rsync with path
82	typeset -r RSYNC=/usr/bin/rsync
83	# RSYNC_opt args to rsync
84	typeset -r RSYNC_opt="-va"
85	# ie storage filesystem
86	typeset -r STOREHOST=${MASTER}
87
88	#====================================================
89	# Source default environment
90	#====================================================
91	##. /etc/profile
92
93	#====================================================
94	# Set environment tools (ferret, nco, cdo)
95	#====================================================
96	if [ X${TaskType} = Xcomputing ] ; then
97	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
98	# to run with netcdf 3.6.3 ie compilation done before 17/2/2014
99	# uncomment 2 lines :
100	# module unload netcdf
101	# module load netcdf/3.6.3
102	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddNoise/src_X64_CURIE/bin
103	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddPerturbation/src_X64_CURIE/bin
104	else
105	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
106	PCMDI_MP=/ccc/work/cont003/dsm/p86ipsl/PCMDI-MP
107	fi
108
109	#====================================================
110	# Specific for ocean additionnal diagnostic
111	export FER_GO="$FER_GO /home/cont003/p86denv/IGCM_POST_UTIL/JNL /home/cont003/p86denv/GRAF /home/cont003/p86denv/GRAF/GO"
112	export FER_PALETTE="$FER_PALETTE /home/cont003/p86denv/GRAF/PALET"
113
114	#====================================================
115	# Host specific DIRECTORIES
116	#====================================================
117
118	# ============ CESIUM START ============ #
119
120	#====================================================
121	#- Mirror libIGCM from titane to cesium if needed
122	#ROOTSYS=$( echo ${libIGCM} \| gawk -F"/" '{print $3}' )
123	#if [ ! ${ROOTSYS} = "home" ] ; then
124	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
125	#else
126	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
127	#fi
128
129	#====================================================
130	#- libIGCM_POST
131	#if ( ${MirrorlibIGCM} ) ; then
132	# PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
133	# typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
134	#else
135	# typeset -r libIGCM_POST=${libIGCM}
136	#fi
137
138	# ============ CESIUM END ============ #
139
140	#====================================================
141	#- MirrorlibIGCM for frontend
142	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
143
144	#====================================================
145	#- libIGCM_POST for frontend
146	typeset -r libIGCM_POST=${libIGCM}
147
148	#====================================================
149	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
150	typeset -r R_EXE="${MODIPSL}/bin"
151
152	#====================================================
153	#- SUBMIT_DIR : submission dir
154	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${BRIDGE_MSUB_PWD}}
155
156	#====================================================
157	#- IN
158	typeset -r R_IN=${R_IN:=/ccc/work/cont003/dsm/p86ipsl/IGCM}
159	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/ccc/work/cont003/dsm/p24data}
160
161	#====================================================
162	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
163	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCHDIR}/RUN_DIR/${BRIDGE_MSUB_JOBID}_${$}}
164
165	#====================================================
166	#- HOST_MPIRUN_COMMAND
167	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time ccc_mprun -E-K1"}
168
169	#====================================================
170	#- Max number of arguments passed to nco operator or demigration command
171	UNIX_MAX_LIMIT=360
172
173	#====================================================
174	#- set PackDefault true on curie
175	PackDefault=true
176
177	#====================================================
178	#- Number of core per node (max number of OpenMP task)
179	NUM_COREPERNODE=8
180
181	#====================================================
182	#- Default number of MPI task for IPSL coupled model
183	#- required for backward compatibility
184	#-
185	DEFAULT_NUM_PROC_OCE=5
186	DEFAULT_NUM_PROC_CPL=1
187	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
188	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
189
190	#D-#==================================================
191	#D-function IGCM_sys_defineArchives
192	#D-* Purpose:
193	#D-* Define ARCHIVE : Dedicated to large files
194	#D-* Define STORAGE : Dedicated to small/medium files
195	#D-* Define R_OUT : Output tree located on ARCHIVE
196	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
197	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
198	#D-* if SpaceName=TEST everything is stored on SCRATCHDIR
199	#D-* Examples:
200	#D-
201	function IGCM_sys_defineArchives {
202	IGCM_debug_PushStack "IGCM_sys_defineArchives"
203
204	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
205	#====================================================
206	#- ARCHIVE (dedicated to large files)
207	ARCHIVE=${config_UserChoices_ARCHIVE}
208	else
209	#====================================================
210	#- ARCHIVE (dedicated to large files)
211	ARCHIVE=${CCCSTOREDIR}
212	fi
213
214	if [ ! X${config_UserChoices_STORAGE} = X ]; then
215	#====================================================
216	#- STORAGE (dedicated to small/medium files)
217	STORAGE=${config_UserChoices_STORAGE}
218	else
219	#====================================================
220	#- STORAGE (dedicated to small/medium files)
221	STORAGE=${CCCWORKDIR}
222	fi
223
224	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
225	#====================================================
226	#- R_OUT
227	R_OUT=${SCRATCHDIR}/IGCM_OUT
228
229	#====================================================
230	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
231	R_FIG=${SCRATCHDIR}/IGCM_OUT
232
233	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
234
235	else
236	#====================================================
237	#- R_OUT
238	R_OUT=${ARCHIVE}/IGCM_OUT
239
240	#====================================================
241	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
242	R_FIG=${STORAGE}/IGCM_OUT
243	fi
244
245	#====================================================
246	#- R_BUF (ONLY FOR double copy an scratch)
247	R_BUF=${SCRATCHDIR}/IGCM_OUT
248
249	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
250	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
251	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
252
253	IGCM_debug_PopStack "IGCM_sys_defineArchives"
254	}
255
256	#D-#==================================================
257	#D-function IGCM_sys_RshMaster
258	#D-* Purpose: Connection to frontend machine.
259	#D-* Examples:
260	#D-
261	function IGCM_sys_RshMaster {
262	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
263	/bin/ksh <<-EOF
264	export libIGCM=${libIGCM}
265	export DEBUG_debug=${DEBUG_debug}
266	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
267	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
268	${@}
269	EOF
270	if [ $? -gt 0 ] ; then
271	echo "IGCM_sys_RshMaster : erreur."
272	IGCM_debug_Exit "IGCM_sys_RshMaster"
273	fi
274	IGCM_debug_PopStack "IGCM_sys_RshMaster"
275	}
276
277	#D-#==================================================
278	#D-function IGCM_sys_RshArchive
279	#D-* Purpose: Archive rsh command
280	#D-* Examples:
281	#D-
282	function IGCM_sys_RshArchive {
283	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
284	/bin/ksh <<-EOF
285	${@}
286	EOF
287	status=$?
288	if [ ${status} -gt 0 ] ; then
289	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
290	IGCM_debug_Exit "IGCM_sys_RshArchive"
291	fi
292	IGCM_debug_PopStack "IGCM_sys_RshArchive"
293	}
294
295	#D-#==================================================
296	#D-function IGCM_sys_RshArchive_NoError
297	#D-* Purpose: Archive rsh command, without error
298	#D-* used only in monitoring.job
299	#D-* Examples:
300	#D-
301	function IGCM_sys_RshArchive_NoError {
302	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
303	/bin/ksh <<-EOF
304	${@} 2> \dev\null
305	EOF
306	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
307	}
308
309	#D-#==================================================
310	#D-function IGCM_sys_RshPost
311	#D-* Purpose: Post-process rsh command
312	#D-* Examples:
313	#D-
314	function IGCM_sys_RshPost {
315	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
316	if ( $DEBUG_sys ) ; then
317	echo "IGCM_sys_RshPost :" $@
318	fi
319	# keep standard input (stdin) for the loop onto temporary file
320	cat >/tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
321
322	/bin/ksh </tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
323	if [ $? -gt 0 ] ; then
324	echo "IGCM_sys_RshPost : erreur."
325	IGCM_debug_Exit "IGCM_sys_RshPost"
326	fi
327	# delete temporary file
328	\rm /tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
329
330	# ============ FRONTEND END ============ #
331
332	# ============ CESIUM START ============ #
333	# typeset NB_ESSAI DELAI status i
334	# if [ "X$( grep rebuild_from tmp_IGCM_sys_RshPost_$$ )" != "X" ] ; then
335	# #little hack so that rebuild submission is done on titane not an cesium
336	#
337	# libIGCM_POST_sed=$( echo $libIGCM_POST \| sed 's/\//\\\//g' )
338	# POST_DIR_sed=$( echo ${POST_DIR} \| sed 's/\//\\\//g' )
339	# sed "s/IGCM_sys_QsubPost/IGCM_sys_Qsub/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
340	# sed "s/ rebuild_fromWorkdir/ ${libIGCM_POST_sed}\/rebuild_fromWorkdir.job/g" tmp.txt > tmp_IGCM_sys_RshPost_$$
341	# sed "s/ rebuild_fromArchive/ ${libIGCM_POST_sed}\/rebuild_fromArchive.job/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
342	# sed "s/Script_Post_Output=/Script_Output=${POST_DIR_sed}\//g" tmp.txt > tmp_IGCM_sys_RshPost_$$
343	# \mv tmp.txt tmp_IGCM_sys_RshPost_$$
344	#
345	# echo cat tmp_IGCM_sys_RshPost_$$ AFTER
346	# cat tmp_IGCM_sys_RshPost_$$
347	#
348	# /bin/ksh <tmp_IGCM_sys_RshPost_$$
349	# if [ $? -gt 0 ] ; then
350	# echo "IGCM_sys_RshPost : erreur."
351	# IGCM_debug_Exit "IGCM_sys_RshPost"
352	# fi
353	# # delete temporary file
354	# \rm tmp_IGCM_sys_RshPost_$$
355	#
356	# else
357	# # number of tentative
358	# NB_ESSAI=10
359	# # time delay between tentative
360	# DELAI=10
361	# i=0
362	# while [ $i -ne $NB_ESSAI ] ; do
363	# ssh -t titane996 ssh cesium /bin/ksh <tmp_IGCM_sys_RshPost_$$
364	# status=$?
365	# if [ ${status} -ne 0 ];
366	# then
367	# sleep $DELAI
368	# else
369	# break
370	# fi
371	# let i=$i+1
372	# done
373	# # delete temporary file
374	# \rm tmp_IGCM_sys_RshPost_$$
375	#
376	# if [ ${status} -gt 0 ] ; then
377	# echo "IGCM_sys_RshPost : erreur."
378	# IGCM_debug_Exit "IGCM_sys_RshPost"
379	# fi
380	# fi
381
382	# ============ CESIUM END ============ #
383
384	IGCM_debug_PopStack "IGCM_sys_RshPost"
385	}
386
387	#D-#==================================================
388	#D-function IGCM_sys_SendMail
389	#D-* Purpose: Send mail when simulation is over
390	#D-* Examples:
391	#D-
392	function IGCM_sys_SendMail {
393	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
394
395	if ( $DEBUG_sys ) ; then
396	echo "IGCM_sys_SendMail :" $@
397	fi
398
399	if [ X${1} = XAccounting ] ; then
400	status=Accounting
401	mailText=jobAccounting.mail
402	elif ( ${ExitFlag} ) ; then
403	status=failed
404	mailText=jobEnd.mail
405	else
406	status=completed
407	mailText=jobEnd.mail
408	fi
409
410	# Update selected mail template
411	while read -r line; do
412	eval echo $line >> mail.txt ;
413	done < ${libIGCM}/libIGCM_sys/${mailText}
414
415	if [ ! -z ${config_UserChoices_MailName} ] ; then
416	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
417	elif [ -f ~/.forward ] ; then
418	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
419	fi
420
421	sleep 10
422	rm -f mail.txt
423
424	IGCM_debug_PopStack "IGCM_sys_SendMail"
425	}
426
427	#D-#==================================================
428	#D-function IGCM_sys_Mkdir
429	#D-* Purpose: Master locale mkdir command
430	#D-* Examples:
431	#D-
432	function IGCM_sys_Mkdir {
433	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
434	if ( $DEBUG_sys ) ; then
435	echo "IGCM_sys_Mkdir :" $@
436	fi
437	if [ ! -d ${1} ]; then
438	\mkdir -p $1
439	if [ $? -gt 0 ] ; then
440	echo "IGCM_sys_Mkdir : erreur."
441	IGCM_debug_Exit "IGCM_sys_Mkdir"
442	fi
443	fi
444	# vérification :
445	if [ ! -d ${1} ] ; then
446	echo "IGCM_sys_Mkdir : erreur."
447	IGCM_debug_Exit "IGCM_sys_Mkdir"
448	fi
449	IGCM_debug_PopStack "IGCM_sys_Mkdir"
450	}
451
452	#D-#==================================================
453	#D-function IGCM_sys_MkdirArchive
454	#D-* Purpose: Mkdir on Archive
455	#D-* Examples:
456	#D-
457	function IGCM_sys_MkdirArchive {
458	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
459	if ( $DEBUG_sys ) ; then
460	echo "IGCM_sys_MkdirArchive :" $@
461	fi
462	#- creation de repertoire sur le serveur fichier
463	if [ ! -d ${1} ]; then
464	\mkdir -p $1
465	status=$?
466
467	if [ ${status} -gt 0 ] ; then
468	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
469	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
470	fi
471	fi
472	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
473	}
474
475	#D-#==================================================
476	#D-function IGCM_sys_MkdirWork
477	#D-* Purpose: Mkdir on Work
478	#D-* Examples:
479	#D-
480	function IGCM_sys_MkdirWork {
481	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
482	if ( $DEBUG_sys ) ; then
483	echo "IGCM_sys_MkdirWork :" $@
484	fi
485	#- creation de repertoire sur le serveur fichier
486	if [ ! -d ${1} ]; then
487	\mkdir -p $1
488	if [ $? -gt 0 ] ; then
489	echo "IGCM_sys_MkdirWork : erreur."
490	IGCM_debug_Exit "IGCM_sys_MkdirWork"
491	fi
492	fi
493	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
494	}
495
496	#D-#==================================================
497	#D-function IGCM_sys_Cd
498	#D-* Purpose: master cd command
499	#D-* Examples:
500	#D-
501	function IGCM_sys_Cd {
502	IGCM_debug_PushStack "IGCM_sys_Cd" $@
503	if ( $DEBUG_sys ) ; then
504	echo "IGCM_sys_Cd :" $@
505	fi
506	\cd $1
507	if [ $? -gt 0 ] ; then
508	echo "IGCM_sys_Cd : erreur."
509	IGCM_debug_Exit "IGCM_sys_Cd"
510	fi
511	IGCM_debug_PopStack "IGCM_sys_Cd"
512	}
513
514	#D-#==================================================
515	#D-function IGCM_sys_Chmod
516	#D-* Purpose: Chmod
517	#D-* Examples:
518	#D-
519	function IGCM_sys_Chmod {
520	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
521	if ( $DEBUG_sys ) ; then
522	echo "IGCM_sys_Chmod :" $@
523	fi
524	\chmod $@
525	if [ $? -gt 0 ] ; then
526	echo "IGCM_sys_Chmod : erreur."
527	IGCM_debug_Exit "IGCM_sys_Chmod"
528	fi
529	IGCM_debug_PopStack "IGCM_sys_Chmod"
530	}
531
532	#D-#==================================================
533	#D-function IGCM_sys_FileSize
534	#D-* Purpose: Filesize
535	#D-* Examples:
536	#D-
537	function IGCM_sys_FileSize {
538	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
539
540	typeset sizeF
541	set +A sizeF -- $( ls -la ${1} )
542	if [ $? -gt 0 ] ; then
543	IGCM_debug_Exit "IGCM_sys_FileSize"
544	fi
545	eval ${2}=${sizeF[4]}
546
547	IGCM_debug_PopStack "IGCM_sys_FileSize"
548	}
549
550	#D-#==================================================
551	#D-function IGCM_sys_TestDir
552	#D-* Purpose: Test Directory that must exists
553	#D-* Examples:
554	#D-
555	function IGCM_sys_TestDir {
556	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
557	if ( $DEBUG_sys ) ; then
558	echo "IGCM_sys_TestDir :" $@
559	fi
560	typeset ExistFlag
561	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
562	IGCM_debug_PopStack "IGCM_sys_TestDir"
563
564	return ${ExistFlag}
565	}
566
567	#D-#==================================================
568	#D-function IGCM_sys_TestDirArchive
569	#D-* Purpose: Test Directory that must exists on Archive
570	#D-* Examples:
571	#D-
572	function IGCM_sys_TestDirArchive {
573	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
574	if ( $DEBUG_sys ) ; then
575	echo "IGCM_sys_TestDirArchive :" $@
576	fi
577	typeset ExistFlag
578	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
579	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
580	return ${ExistFlag}
581	}
582
583	#D-#==================================================
584	#D-function IGCM_sys_IsFileArchived
585	#D-* Purpose: Test file that must NOT EXISTS on Archive
586	#D-* Examples:
587	#D-
588	function IGCM_sys_IsFileArchived {
589	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
590	if ( $DEBUG_sys ) ; then
591	echo "IGCM_sys_IsFileArchived :" $@
592	fi
593	typeset IsArchivedFlag
594	IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
595	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
596
597	return ${IsArchivedFlag}
598	}
599
600	#D-#==================================================
601	#D-function IGCM_sys_TestFileArchive
602	#D-* Purpose: Test file that must NOT EXISTS on Archive
603	#D-* Examples:
604	#D-
605	function IGCM_sys_TestFileArchive {
606	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
607	typeset ExistFlag
608	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
609	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
610
611	return ${ExistFlag}
612	}
613
614	#D-#==================================================
615	#D-function IGCM_sys_TestFileBuffer
616	#D-* Purpose: Test file that must NOT EXISTS on Buffer
617	#D-* Examples:
618	#D-
619	function IGCM_sys_TestFileBuffer {
620	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
621	typeset ExistFlag
622	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
623	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
624
625	return ${ExistFlag}
626	}
627
628	#D-#==================================================
629	#D-function IGCM_sys_CountFileArchive
630	#D-* Purpose: Count files on Archive filesystem
631	#D-* Examples:
632	#D-
633	function IGCM_sys_CountFileArchive {
634	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
635	ls ${@} 2>/dev/null \| wc -l
636	if [ $? -gt 0 ] ; then
637	echo "IGCM_sys_CountFileArchive : erreur."
638	fi
639	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
640	}
641
642	#D-#==================================================
643	#D-function IGCM_sys_CountFileBuffer
644	#D-* Purpose: Count files on Scratch filesystem
645	#D-* Examples:
646	#D-
647	function IGCM_sys_CountFileBuffer {
648	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
649	ls ${@} 2>/dev/null \| wc -l
650	if [ $? -gt 0 ] ; then
651	echo "IGCM_sys_CountFileBuffer : erreur."
652	fi
653	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
654	}
655
656	#D-#==================================================
657	#D-function IGCM_sys_Tree
658	#D-* Purpose: Tree directories with files on ${ARCHIVE}
659	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
660	#D-
661	function IGCM_sys_Tree {
662	IGCM_debug_PushStack "IGCM_sys_Tree" $@
663	if ( $DEBUG_sys ) ; then
664	echo "IGCM_sys_Tree :" $@
665	fi
666
667	\ls -lR ${@}
668
669	IGCM_debug_PopStack "IGCM_sys_Tree"
670	}
671
672	#D-#==================================================
673	#D-function IGCM_sys_Tar
674	#D-* Purpose: master tar command
675	#D-* Examples:
676	#D-
677	function IGCM_sys_Tar {
678	IGCM_debug_PushStack "IGCM_sys_Tar" $@
679	if ( $DEBUG_sys ) ; then
680	echo "IGCM_sys_Tar :" $@
681	fi
682	\tar cf $@
683	if [ $? -gt 0 ] ; then
684	echo "IGCM_sys_Tar : erreur."
685	IGCM_debug_Exit "IGCM_sys_Tar"
686	fi
687	IGCM_debug_PopStack "IGCM_sys_Tar"
688	}
689
690	#D-#==================================================
691	#D-function IGCM_sys_UnTar
692	#D-* Purpose: master un-tar command
693	#D-* Examples:
694	#D-
695	function IGCM_sys_UnTar {
696	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
697	if ( $DEBUG_sys ) ; then
698	echo "IGCM_sys_UnTar :" $@
699	fi
700	\tar xvf $1
701	if [ $? -gt 0 ] ; then
702	echo "IGCM_sys_UnTar : erreur."
703	IGCM_debug_Exit "IGCM_sys_UnTar"
704	fi
705	IGCM_debug_PopStack "IGCM_sys_UnTar"
706	}
707
708	#D-#==================================================
709	#D-function IGCM_sys_Qsub
710	#D-* Purpose: Qsub new job
711	#D-* Examples:
712	#D-
713	function IGCM_sys_Qsub {
714	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
715	if ( $DEBUG_sys ) ; then
716	echo "IGCM_sys_Qsub :" $@
717	fi
718	typeset options status
719	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
720
721	/usr/bin/ccc_msub ${options} $1 > /tmp/out_command.$$ 2>&1
722	status=$?
723
724	cat /tmp/out_command.$$
725	if [ ${status} -gt 0 ] ; then
726	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
727	IGCM_debug_Exit "IGCM_sys_Qsub"
728	else
729	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
730	fi
731	IGCM_debug_PopStack "IGCM_sys_Qsub"
732	}
733
734	#D-#==================================================
735	#D-function IGCM_sys_QsubPost
736	#D-* Purpose: Qsub new job on scalaire
737	#D-* Examples:
738	#D-
739	function IGCM_sys_QsubPost {
740	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
741	if ( $DEBUG_sys ) ; then
742	echo "IGCM_sys_QsubPost :" $@
743	fi
744	typeset options status
745	MY_PROJECT=$(echo ${BRIDGE_MSUB_PROJECT:=${DEFAULT_PROJECT}} \| cut -d@ -f1 )
746
747	# EASIER TO DO THIS INSTEAD OF DUPLICATING libIGCM_sys_curie.ksh
748	case $( hostname -s ) in
749	curie*)
750	options="-Q normal -A ${MY_PROJECT} -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out";;
751	airain*)
752	options="-q ivybridge -A dsm -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
753	esac
754
755	/usr/bin/ccc_msub ${options} ${libIGCM_POST}/$1.job > /tmp/out_command.$$ 2>&1
756	status=$?
757
758	cat /tmp/out_command.$$
759	if [ ${status} -gt 0 ] ; then
760	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
761	IGCM_debug_Exit "IGCM_sys_QsubPost"
762	else
763	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
764	fi
765	IGCM_debug_PopStack "IGCM_sys_QsubPost"
766	}
767
768	#D-*************************
769	#D- File transfer functions
770	#D-*************************
771	#D-
772
773	#D-#==================================================
774	#D-function IGCM_sys_Rsync_out
775	#D-* Purpose: treat return val of rsync
776	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
777	#D- Error values and explanations can depend on your system version.
778	function IGCM_sys_Rsync_out {
779	status=$1
780	if [ ! $status ] ; then
781	echo "rsync error !"
782	fi
783
784	if [ $MYLANG = "fr" ]; then
785	case $status in
786	0) return ;;
787	1) echo "Erreur de rsync ; RERR_SYNTAX : "
788	echo "Erreur de syntaxe ou d'utilisation."
789	return;;
790	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
791	echo "Incompatibilité de protocole."
792	return;;
793	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
794	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
795	echo "répertoires"
796	return;;
797	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
798	echo "Action demandée non supportée : une tentative de manipulation de"
799	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
800	echo "été faite ; ou une option qui est supportée par le client mais"
801	echo "pas par le serveur a été spécifiée."
802	return;;
803	10) echo "Erreur de rsync ; RERR_SOCKETIO"
804	echo "Erreur dans le socket d'entrée sortie"
805	return;;
806	11) echo "Erreur de rsync ; RERR_FILEIO"
807	echo "Erreur d'entrée sortie fichier"
808	return;;
809	12) echo "Erreur de rsync ; RERR_STREAMIO"
810	echo "Erreur dans flux de donnée du protocole rsync"
811	return;;
812	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
813	echo "Erreur avec les diagnostics du programme"
814	return;;
815	14) echo "Erreur de rsync ; RERR_IPC"
816	echo "Erreur dans le code IPC"
817	return;;
818	20) echo "Erreur de rsync ; RERR_SIGNAL"
819	echo "SIGUSR1 ou SIGINT reçu"
820	return;;
821	21) echo "Erreur de rsync ; RERR_WAITCHILD"
822	echo "Une erreur retournée par waitpid()"
823	return;;
824	22) echo "Erreur de rsync ; RERR_MALLOC"
825	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
826	return;;
827	23) echo ""
828	echo "Erreur fichier inexistant"
829	return;;
830	30) echo "Erreur de rsync ; RERR_TIMEOUT"
831	echo "Temps d'attente écoulé dans l'envoi/réception de données"
832	return;;
833	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
834	return;;
835	esac
836	elif [ $MYLANG = "en" ] ; then
837	case $status in
838	0) return;;
839	1) echo "rsync error : Syntax or usage error "
840	return;;
841	2) echo "rsync error : Protocol incompatibility "
842	return;;
843	3) echo "rsync error : Errors selecting input/output files, dirs"
844	return;;
845	4) echo "rsync error : Requested action not supported: an attempt"
846	echo "was made to manipulate 64-bit files on a platform that cannot support"
847	echo "them; or an option was specified that is supported by the client and"
848	echo "not by the server."
849	return;;
850	5) echo "rsync error : Error starting client-server protocol"
851	return;;
852	10) echo "rsync error : Error in socket I/O "
853	return;;
854	11) echo "rsync error : Error in file I/O "
855	return;;
856	12) echo "rsync error : Error in rsync protocol data stream "
857	return;;
858	13) echo "rsync error : Errors with program diagnostics "
859	return;;
860	14) echo "rsync error : Error in IPC code "
861	return;;
862	20) echo "rsync error : Received SIGUSR1 or SIGINT "
863	return;;
864	21) echo "rsync error : Some error returned by waitpid() "
865	return;;
866	22) echo "rsync error : Error allocating core memory buffers "
867	return;;
868	23) echo "rsync error : Partial transfer due to error"
869	return;;
870	24) echo "rsync error : Partial transfer due to vanished source files"
871	return;;
872	30) echo "rsync error : Timeout in data send/receive "
873	return;;
874	*) echo "rsync error : return code of rsync unknown :" $status
875	return;;
876	esac
877	else
878	echo "unknown language $MYLANG."
879	return
880	fi
881	}
882
883	#D-#==================================================
884	#D-function IGCM_sys_Miror_libIGCM
885	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
886	#D-* Examples:
887	#D-
888	function IGCM_sys_Mirror_libIGCM {
889	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
890	if ( $DEBUG_sys ) ; then
891	echo "IGCM_sys_Mirror_libIGCM"
892	fi
893
894	typeset status
895
896	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
897
898	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
899	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
900	status=$?
901
902	if [ ${status} -gt 0 ] ; then
903	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
904	cat /tmp/out_command.$$
905	fi
906	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
907	}
908
909	#D-#==================================================
910	#D-function IGCM_sys_Cp
911	#D-* Purpose: generic cp
912	#D-* Examples:
913	#D-
914	function IGCM_sys_Cp {
915	IGCM_debug_PushStack "IGCM_sys_Cp" $@
916	if ( $DEBUG_sys ) ; then
917	echo "IGCM_sys_Cp :" $@
918	fi
919
920	typeset status
921
922	echo cp $@ > /tmp/out_command.$$ 2>&1
923	\cp $@ >> /tmp/out_command.$$ 2>&1
924	status=$?
925
926	if [ ${status} -gt 0 ] ; then
927	echo "IGCM_sys_Cp : error code ${status}"
928	cat /tmp/out_command.$$
929	IGCM_debug_Exit "IGCM_sys_Cp"
930	else
931	\rm /tmp/out_command.$$
932	fi
933	IGCM_debug_PopStack "IGCM_sys_Cp"
934	}
935
936	#D-#==================================================
937	#D-function IGCM_sys_Rm
938	#D-* Purpose: generic rm
939	#D-* Examples:
940	#D-
941	function IGCM_sys_Rm {
942	IGCM_debug_PushStack "IGCM_sys_Rm" $@
943	if ( $DEBUG_sys ) ; then
944	echo "IGCM_sys_Rm :" $@
945	fi
946
947	typeset status
948
949	echo rm $@ > /tmp/out_command.$$ 2>&1
950	\rm $@ >> /tmp/out_command.$$ 2>&1
951	status=$?
952
953	if [ ${status} -gt 0 ] ; then
954	echo "IGCM_sys_Rm : error code ${status}"
955	cat /tmp/out_command.$$
956	IGCM_debug_Exit "IGCM_sys_Rm"
957	else
958	\rm /tmp/out_command.$$
959	fi
960	IGCM_debug_PopStack "IGCM_sys_Rm"
961	}
962
963	#D-#==================================================
964	#D-function IGCM_sys_RmRunDir
965	#D-* Purpose: rm tmpdir (dummy function most of the time batch
966	#D- scheduler will do the job)
967	#D-* Examples:
968	#D-
969	function IGCM_sys_RmRunDir {
970	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
971	if ( $DEBUG_sys ) ; then
972	echo "IGCM_sys_RmRunDir :" $@
973	fi
974
975	typeset status
976
977	echo rm $@ > /tmp/out_command.$$ 2>&1
978	\rm $@ >> /tmp/out_command.$$ 2>&1
979	status=$?
980
981	if [ ${status} -gt 0 ] ; then
982	echo "IGCM_sys_RmRunDir : rm error code is ${status}."
983	cat /tmp/out_command.$$
984	IGCM_debug_Exit "IGCM_sys_RmRunDir"
985	else
986	\rm /tmp/out_command.$$
987	fi
988	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
989	}
990
991	#D-#==================================================
992	#D-function IGCM_sys_Mv
993	#D-* Purpose: generic move
994	#D-* Examples:
995	#D-
996	function IGCM_sys_Mv {
997	IGCM_debug_PushStack "IGCM_sys_Mv" $@
998	if ( $DEBUG_sys ) ; then
999	echo "IGCM_sys_Mv :" $@
1000	fi
1001
1002	if [ $DRYRUN = 0 ]; then
1003
1004	typeset status
1005
1006	echo mv $@ > /tmp/out_command.$$ 2>&1
1007	\mv $@ >> /tmp/out_command.$$ 2>&1
1008	status=$?
1009
1010	if [ ${status} -gt 0 ] ; then
1011	echo "IGCM_sys_Mv : error code ${status}"
1012	cat /tmp/out_command.$$
1013	IGCM_debug_Exit "IGCM_sys_Mv"
1014	else
1015	\rm /tmp/out_command.$$
1016	fi
1017	fi
1018
1019	IGCM_debug_PopStack "IGCM_sys_Mv"
1020	}
1021
1022	#D-#==================================================
1023	#D-function IGCM_sys_Put_Dir
1024	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1025	#D-* Examples:
1026	#D-
1027	function IGCM_sys_Put_Dir {
1028	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1029	if ( $DEBUG_sys ) ; then
1030	echo "IGCM_sys_Put_Dir :" $@
1031	fi
1032	if [ $DRYRUN = 0 ]; then
1033	if [ ! -d ${1} ] ; then
1034	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1035	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1036	return
1037	fi
1038
1039	typeset status
1040
1041	# Only if we use rsync
1042	#IGCM_sys_TestDirArchive $( dirname $2 )
1043	#
1044	#USUAL WAY
1045	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
1046	status=$?
1047
1048	if [ ${status} -gt 0 ] ; then
1049	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
1050	cat /tmp/out_command.$$
1051	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1052	else
1053	\rm /tmp/out_command.$$
1054	fi
1055	fi
1056	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1057	}
1058
1059	#D-#==================================================
1060	#D-function IGCM_sys_Get_Dir
1061	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1062	#D-* Examples:
1063	#D-
1064	function IGCM_sys_Get_Dir {
1065	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1066	if ( $DEBUG_sys ) ; then
1067	echo "IGCM_sys_Get_Dir :" $@
1068	fi
1069	if [ $DRYRUN = 0 ]; then
1070	# if [ ! -d ${1} ] ; then
1071	# echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
1072	# IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1073	# return
1074	# fi
1075
1076	typeset NB_ESSAI DELAI status i
1077	# number of tentative
1078	NB_ESSAI=3
1079	# time delay between tentative
1080	DELAI=2
1081
1082	# Only if we use rsync
1083	#IGCM_sys_TestDirArchive $( dirname $2 )
1084	#
1085	# USUAL WAY
1086	# add 'ccc_hsm get' (to demigrate all offline files) to reduce time of this command :
1087	ccc_hsm get -r $1
1088
1089	i=0
1090	while [ $i -lt $NB_ESSAI ] ; do
1091	\cp -ur $1 $2 >> /tmp/out_command.$$ 2>&1
1092	status=$?
1093	if [ ${status} -gt 0 ] ; then
1094	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
1095	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1096	sleep $DELAI
1097	else
1098	break
1099	fi
1100	(( i = i + 1 ))
1101	done
1102
1103	if [ ${status} -gt 0 ] ; then
1104	echo "IGCM_sys_Get_Dir : error."
1105	cat /tmp/out_command.$$
1106	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1107	else
1108	\rm /tmp/out_command.$$
1109	fi
1110	fi
1111	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1112	}
1113
1114	#D-#==================================================
1115	#D-function IGCM_sys_Get_Master
1116	#D-* Purpose: Copy a complete directory from MASTER filesystem
1117	#D-* Examples:
1118	#D-
1119	function IGCM_sys_Get_Master {
1120	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1121	if ( $DEBUG_sys ) ; then
1122	echo "IGCM_sys_Get_Master :" $@
1123	fi
1124	if [ $DRYRUN = 0 ]; then
1125	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1126	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1127	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1128	return
1129	fi
1130
1131	typeset NB_ESSAI DELAI status i
1132	# number of tentative
1133	NB_ESSAI=3
1134	# time delay between tentative
1135	DELAI=2
1136
1137	i=0
1138	while [ $i -lt $NB_ESSAI ] ; do
1139	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
1140	status=$?
1141	if [ ${status} -gt 0 ]; then
1142	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1143	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1144	sleep $DELAI
1145	else
1146	break
1147	fi
1148	(( i = i + 1 ))
1149	done
1150
1151	if [ ${status} -gt 0 ] ; then
1152	echo "IGCM_sys_Get_Master : error."
1153	cat /tmp/out_command.$$
1154	IGCM_debug_Exit "IGCM_sys_Get_Master"
1155	else
1156	\rm /tmp/out_command.$$
1157	fi
1158	fi
1159	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1160	}
1161
1162	#====================================================
1163	#- Call IGCM_sys_Mirror_libIGCM now !
1164	if ( $MirrorlibIGCM ) ; then
1165	IGCM_sys_Mirror_libIGCM
1166	fi
1167
1168	#D-#==================================================
1169	#D-function IGCM_sys_Put_Rest
1170	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1171	#D- File and target directory must exist.
1172	#D-* Examples:
1173	#D-
1174	function IGCM_sys_Put_Rest {
1175	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1176	if ( $DEBUG_sys ) ; then
1177	echo "IGCM_sys_Put_Rest :" $@
1178	fi
1179	if [ $DRYRUN = 0 ]; then
1180	if [ ! -f ${1} ] ; then
1181	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1182	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1183	fi
1184
1185	typeset status
1186	#
1187	# USUAL WAY
1188	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1189	status=$?
1190
1191	# #RSYNC WITH NETWORK SSH CALL
1192	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1193	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1194
1195	# #RSYNC WITH NFS USE
1196	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1197	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1198
1199	# status=$?
1200	# IGCM_sys_Rsync_out $status
1201
1202	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1203	# (( status=status+$? ))
1204
1205	if [ ${status} -gt 0 ] ; then
1206	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
1207	[ -f ${1} ] && ls -l ${1}
1208	[ -f ${2} ] && ls -l ${2}
1209	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1210	cat /tmp/out_command.$$
1211	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1212	else
1213
1214	if [ X${JobType} = XRUN ] ; then
1215	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1216	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1217	fi
1218
1219	\rm /tmp/out_command.$$
1220	fi
1221	fi
1222	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1223	}
1224
1225	#D-#==================================================
1226	#D-function IGCM_sys_PutBuffer_Rest
1227	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1228	#D- File and target directory must exist.
1229	#D-* Examples:
1230	#D-
1231	function IGCM_sys_PutBuffer_Rest {
1232	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1233	if ( $DEBUG_sys ) ; then
1234	echo "IGCM_sys_PutBuffer_Rest :" $@
1235	fi
1236	if [ $DRYRUN = 0 ]; then
1237	if [ ! -f ${1} ] ; then
1238	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1239	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1240	fi
1241
1242	typeset status
1243	#
1244	# USUAL WAY
1245	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1246	status=$?
1247
1248	if [ ${status} -gt 0 ] ; then
1249	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1250	[ -f ${2} ] && ls -l ${2}
1251	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1252	cat /tmp/out_command.$$
1253	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1254	else
1255
1256	if [ X${JobType} = XRUN ] ; then
1257	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1258	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1259	fi
1260
1261	\rm /tmp/out_command.$$
1262	fi
1263	fi
1264	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1265	}
1266
1267	#D-#==================================================
1268	#D-function IGCM_sys_PrepareTaredRestart
1269	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
1270	#D-* Examples:
1271	#D-
1272	function IGCM_sys_PrepareTaredRestart {
1273	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1274	if [ $DRYRUN = 0 ]; then
1275	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1276	fi
1277	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1278	}
1279
1280	#D-#==================================================
1281	#D-function IGCM_sys_Put_Out
1282	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1283	#D-* Examples:
1284	#D-
1285	function IGCM_sys_Put_Out {
1286	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1287	if ( $DEBUG_sys ) ; then
1288	echo "IGCM_sys_Put_Out :" $@
1289	fi
1290
1291	typeset NB_ESSAI DELAI status i exist skip
1292
1293	# number of tentative
1294	NB_ESSAI=3
1295	# time delay between tentative
1296	DELAI=2
1297
1298	if [ $DRYRUN = 0 ]; then
1299	if [ ! -f ${1} ] ; then
1300	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1301	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1302	return 1
1303	fi
1304	#
1305	IGCM_sys_MkdirArchive $( dirname $2 )
1306	#
1307
1308	#=====================================================
1309	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1310	#=====================================================
1311
1312	#echo ${2} \| grep "${R_OUT}" > /dev/null 2>&1
1313	#if [ $? -eq 0 ] ; then
1314	# typeset WORKPATH FILEPATH
1315	# WORKPATH=$( dirname $2 \| sed -e "s\|${R_OUT}\|${R_BUF}\|" )
1316	# IGCM_sys_MkdirWork ${WORKPATH}
1317	# FILEPATH=${WORKPATH}/$( basename $2 )
1318	# #
1319	# IGCM_sys_Cp ${1} ${FILEPATH}
1320	#fi
1321
1322	exist=false
1323	skip=false
1324	if [ -f $2 ] ; then
1325	IGCM_debug_Print 1 "$2 already exist"
1326	ccc_hsm get $2
1327	exist=true
1328	if [ "X$( diff $1 $2 )" = X ] ; then
1329	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1330	skip=true
1331	else
1332	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1333	skip=false
1334	fi
1335	fi
1336	#
1337	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1338	IGCM_sys_Chmod u+w $2
1339	fi
1340
1341	if [ X${skip} = Xfalse ] ; then
1342	i=0
1343	while [ $i -lt $NB_ESSAI ] ; do
1344	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1345	# USUAL WAY
1346	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1347	status=$?
1348	else
1349	# NOT SO USUAL WAY
1350	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1351	status=$?
1352	fi
1353	if [ ${status} -gt 0 ]; then
1354	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1355	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
1356	[ -f ${1} ] && ls -l ${1}
1357	[ -f ${2} ] && ls -l ${2}
1358	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1359	sleep $DELAI
1360	else
1361	break
1362	fi
1363	(( i = i + 1 ))
1364	done
1365	fi
1366
1367	# #RSYNC WITH NETWORK SSH CALL
1368	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1369	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1370
1371	# #RSYNC WITH NFS USE
1372	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1373	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1374
1375	# status=$?
1376	# IGCM_sys_Rsync_out $status
1377
1378	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1379	# (( status=status+$? ))
1380
1381	if [ ${status} -gt 0 ] ; then
1382	echo "IGCM_sys_Put_Out : error."
1383	[ -f ${1} ] && ls -l ${1}
1384	[ -f ${2} ] && ls -l ${2}
1385	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1386	cat /tmp/out_command.$$
1387	IGCM_debug_Exit "IGCM_sys_Put_Out"
1388	else
1389
1390	if [ X${JobType} = XRUN ] ; then
1391	if [ X${3} = X ] ; then
1392	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1393	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1394	fi
1395	fi
1396
1397	\rm /tmp/out_command.$$
1398	fi
1399	fi
1400	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1401	return 0
1402	}
1403
1404	#D-#==================================================
1405	#D-function IGCM_sys_PutBuffer_Out
1406	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1407	#D-* Examples:
1408	#D-
1409	function IGCM_sys_PutBuffer_Out {
1410	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1411	if ( $DEBUG_sys ) ; then
1412	echo "IGCM_sys_PutBuffer_Out :" $@
1413	fi
1414
1415	typeset NB_ESSAI DELAI status i exist skip
1416
1417	# number of tentative
1418	NB_ESSAI=3
1419	# time delay between tentative
1420	DELAI=2
1421
1422	if [ $DRYRUN = 0 ]; then
1423	if [ ! -f ${1} ] ; then
1424	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1425	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1426	return 1
1427	fi
1428	#
1429	IGCM_sys_Mkdir $( dirname $2 )
1430	#
1431
1432	exist=false
1433	skip=false
1434	if [ -f $2 ] ; then
1435	IGCM_debug_Print 1 "$2 already exist"
1436	exist=true
1437	if [ "X$( diff $1 $2 )" = X ] ; then
1438	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1439	status=0
1440	skip=true
1441	else
1442	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1443	skip=false
1444	fi
1445	fi
1446	#
1447	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1448	IGCM_sys_Chmod u+w $2
1449	fi
1450
1451	if [ X${skip} = Xfalse ] ; then
1452	i=0
1453	while [ $i -lt $NB_ESSAI ] ; do
1454	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1455	# USUAL WAY
1456	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1457	status=$?
1458	else
1459	# NOT SO USUAL WAY
1460	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1461	status=$?
1462	fi
1463	if [ ${status} -gt 0 ]; then
1464	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1465	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1466	[ -f ${2} ] && ls -l ${2}
1467	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1468	sleep $DELAI
1469	else
1470	break
1471	fi
1472	(( i = i + 1 ))
1473	done
1474	fi
1475
1476	if [ ${status} -gt 0 ] ; then
1477	echo "IGCM_sys_PutBuffer_Out : error."
1478	[ -f ${2} ] && ls -l ${2}
1479	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1480	cat /tmp/out_command.$$
1481	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1482	else
1483
1484	if [ X${JobType} = XRUN ] ; then
1485	if [ X${3} = X ] ; then
1486	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1487	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1488	fi
1489	fi
1490
1491	\rm /tmp/out_command.$$
1492	fi
1493	fi
1494	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1495	return 0
1496	}
1497
1498	#D-#==================================================
1499	#D-function IGCM_sys_Get
1500	#D-* Purpose: Get a file from ${ARCHIVE}
1501	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1502	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1503	function IGCM_sys_Get {
1504	IGCM_debug_PushStack "IGCM_sys_Get" $@
1505
1506	typeset DEST dm_liste target file_work
1507	typeset NB_ESSAI DELAI status i
1508
1509	if ( $DEBUG_sys ) ; then
1510	echo "IGCM_sys_Get :" $@
1511	fi
1512
1513	# number of tentative
1514	NB_ESSAI=3
1515	# time delay between tentative
1516	DELAI=2
1517
1518	if [ $DRYRUN -le 2 ]; then
1519	if [ X${1} = X'/l' ] ; then
1520	eval set +A dm_liste \${${2}}
1521	else
1522	eval set +A dm_liste ${1}
1523	fi
1524	eval DEST=\${${#}}
1525
1526	#=====================================================
1527	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1528	#=====================================================
1529
1530	# Is it an R_OUT file (not R_IN) ?
1531	#echo ${dm_liste[0]} \| grep "${R_OUT}" > /dev/null 2>&1
1532	#if [ $? -eq 0 ] ; then
1533	# # Yes ? then we try to get it in SCRATCHDIR
1534	# set +A file_work $( echo ${dm_liste[*]} \| sed -e "s\|${R_OUT}\|${R_BUF}\|g" )
1535	# if [ -f ${file_work[0]} ] ; then
1536	# IGCM_sys_Cp ${file_work[*]} ${DEST}
1537	# IGCM_debug_PopStack "IGCM_sys_Get"
1538	# return
1539	# fi
1540	#fi
1541
1542	ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1543	status=$?
1544	if [ ${status} -gt 0 ] ; then
1545	echo "WARNING IGCM_sys_Get : error code ${status}"
1546	cat /tmp/out_command.$$
1547	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
1548	fi
1549
1550	#if [ ${status} -gt 0 ] ; then
1551	# if [ ! "X$( grep "Lost dmusrcmd connection" /tmp/out_command.$$ )" = "X" ] ; then
1552	# cat /tmp/out_command.$$
1553	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
1554	# sleep 30
1555	# echo "We try another time"
1556	## dmget ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1557	# ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1558	# status=$?
1559	# if [ ${status} -gt 0 ] ; then
1560	# echo "ERROR IGCM_sys_Get : again demigration error :"
1561	# cat /tmp/out_command.$$
1562	# IGCM_debug_Exit "IGCM_sys_Get"
1563	# fi
1564	# else
1565	# echo "ERROR IGCM_sys_Get : demigration error :"
1566	# cat /tmp/out_command.$$
1567	# IGCM_debug_Exit "IGCM_sys_Get"
1568	# fi
1569	#fi
1570
1571	# #RSYNC WITH NETWORK SSH CALL
1572	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1573	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1574
1575	# #RSYNC WITH NFS USE
1576	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1577	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1578
1579	# status=$?
1580	# IGCM_sys_Rsync_out $status
1581
1582	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1583	# (( status=status+$? ))
1584
1585	#USUAL WAY
1586	if [ X${1} = X'/l' ] ; then
1587	for target in ${dm_liste[*]} ; do
1588	local_file=$( basename ${target} )
1589	# test if the target file is present before the loop
1590	IGCM_sys_TestFileArchive ${target}
1591	status=$?
1592	if [ ${status} -gt 0 ] ; then
1593	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
1594	IGCM_debug_Exit "IGCM_sys_Get"
1595	else
1596	i=0
1597	while [ $i -lt $NB_ESSAI ] ; do
1598	#if [ X${DoLink} = Xtrue ] ; then
1599	# \ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1600	# status=$?
1601	# else
1602	# \cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1603	# status=$?
1604	#fi
1605	\ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1606	status=$?
1607	if [ ${status} -gt 0 ]; then
1608	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1609	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1610	sleep $DELAI
1611	else
1612	break
1613	fi
1614	(( i = i + 1 ))
1615	done
1616	if [ ${status} -gt 0 ] ; then
1617	echo "IGCM_sys_Get : error"
1618	cat /tmp/out_command.$$
1619	\rm /tmp/out_command.$$
1620	IGCM_debug_Exit "IGCM_sys_Get"
1621	else
1622	\rm /tmp/out_command.$$
1623	fi
1624	fi
1625	done
1626	else
1627	i=0
1628	while [ $i -lt $NB_ESSAI ] ; do
1629	\cp ${dm_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1630	status=$?
1631	if [ ${status} -gt 0 ]; then
1632	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1633	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1634	sleep $DELAI
1635	else
1636	break
1637	fi
1638	(( i = i + 1 ))
1639	done
1640	if [ ${status} -gt 0 ] ; then
1641	echo "IGCM_sys_Get : error"
1642	cat /tmp/out_command.$$
1643	\rm /tmp/out_command.$$
1644	IGCM_debug_Exit "IGCM_sys_Get"
1645	else
1646	\rm /tmp/out_command.$$
1647	fi
1648	fi
1649	fi
1650	IGCM_debug_PopStack "IGCM_sys_Get"
1651	}
1652
1653	#D-#==================================================
1654	#D-function IGCM_sys_GetBuffer
1655	#D-* Purpose: Get a file from ${SCRATCHDIR}
1656	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1657	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1658	function IGCM_sys_GetBuffer {
1659	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1660
1661	typeset DEST buf_liste target file_work
1662	typeset NB_ESSAI DELAI status i
1663
1664	if ( $DEBUG_sys ) ; then
1665	echo "IGCM_sys_GetBuffer :" $@
1666	fi
1667
1668	# number of tentative
1669	NB_ESSAI=3
1670	# time delay between tentative
1671	DELAI=2
1672
1673	if [ $DRYRUN -le 2 ]; then
1674	if [ X${1} = X'/l' ] ; then
1675	eval set +A buf_liste \${${2}}
1676	else
1677	eval set +A buf_liste ${1}
1678	fi
1679	eval DEST=\${${#}}
1680
1681	#USUAL WAY
1682	if [ X${1} = X'/l' ] ; then
1683	for target in ${buf_liste[*]} ; do
1684	local_file=$( basename ${target} )
1685	i=0
1686	while [ $i -lt $NB_ESSAI ] ; do
1687	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1688	status=$?
1689	if [ ${status} -gt 0 ]; then
1690	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1691	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1692	sleep $DELAI
1693	else
1694	break
1695	fi
1696	(( i = i + 1 ))
1697	done
1698	if [ ${status} -gt 0 ] ; then
1699	echo "IGCM_sys_Get : error"
1700	cat /tmp/out_command.$$
1701	\rm /tmp/out_command.$$
1702	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1703	else
1704	\rm /tmp/out_command.$$
1705	fi
1706	done
1707	else
1708	i=0
1709	while [ $i -lt $NB_ESSAI ] ; do
1710	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1711	status=$?
1712	if [ ${status} -gt 0 ]; then
1713	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1714	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1715	sleep $DELAI
1716	else
1717	break
1718	fi
1719	(( i = i + 1 ))
1720	done
1721	if [ ${status} -gt 0 ] ; then
1722	echo "IGCM_sys_Get : error"
1723	cat /tmp/out_command.$$
1724	\rm /tmp/out_command.$$
1725	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1726	else
1727	\rm /tmp/out_command.$$
1728	fi
1729	fi
1730	fi
1731	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1732	}
1733
1734	#D-#==================================================
1735	#D-function IGCM_sys_GetDate_FichWork
1736	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1737	#D-* Examples:
1738	#D-
1739	function IGCM_sys_GetDate_FichWork {
1740	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1741	if ( $DEBUG_sys ) ; then
1742	echo "IGCM_sys_GetDate_FichWork :" $@
1743	fi
1744
1745	if [ $# -ge 3 ] ; then
1746	mode=$3
1747	TimeStyle=$4
1748	else
1749	mode="default"
1750	TimeStyle="%Y%m%d%H%M%S"
1751	fi
1752
1753	typeset dateF
1754	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1755
1756	case $mode in
1757	"default")
1758	eval ${2}=${dateF[5]}
1759	;;
1760	"SplitFields")
1761	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1762	;;
1763	esac
1764
1765
1766	# donne la date filesys d'un fichier sur la machine work
1767	IGCM_debug_PopStack "IGCM_sys_FichWork"
1768	}
1769
1770	#D-#==================================================
1771	#D-function IGCM_sys_GetDate_FichArchive
1772	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1773	#D-* Examples:
1774	#D-
1775	function IGCM_sys_GetDate_FichArchive {
1776	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1777	if ( $DEBUG_sys ) ; then
1778	echo "IGCM_sys_GetDate_FichArchive :" $@
1779	fi
1780	typeset dateF
1781	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1782	eval ${2}=${dateF[5]}
1783
1784	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1785	}
1786
1787	#D-#==================================================
1788	#D-function IGCM_sys_GetDate_Monitoring
1789	#D-* Purpose: get the last year for which the monitoring has been computed
1790	#D-* Examples:
1791	#D-
1792	function IGCM_sys_GetDate_Monitoring {
1793	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1794	if ( $DEBUG_sys ) ; then
1795	echo "IGCM_sys_GetDate_Monitoring :" $@
1796	fi
1797
1798	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1799
1800	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1801	}
1802
1803	#D-#==================================================
1804	#D-function IGCM_sys_Dods_Rm
1805	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1806	#D-* Examples:
1807	#D-
1808	function IGCM_sys_Dods_Rm {
1809	if ( $DEBUG_sys ) ; then
1810	echo "IGCM_sys_Dods_Rm :" $@
1811	fi
1812	typeset status
1813	if [ $DRYRUN = 0 ]; then
1814
1815	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
1816	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
1817	# echo "Nothing has been done."
1818	# return
1819	# fi
1820
1821	/ccc/cont003/home/dsm/p86ipsl/bin/dods_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
1822	status=$?
1823
1824	# if [ ${status} -gt 0 ] ; then
1825	# echo "IGCM_sys_Dods_Rm : error."
1826	# cat out_dods_rm
1827	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
1828	# else
1829	# rm out_dods_rm
1830	# fi
1831
1832	fi
1833	return $status
1834	}
1835
1836	#D-#==================================================
1837	#D-function IGCM_sys_Dods_Cp
1838	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1839	#D-* Examples:
1840	#D-
1841	function IGCM_sys_Dods_Cp {
1842	if ( $DEBUG_sys ) ; then
1843	echo "IGCM_sys_Dods_Cp :" $@
1844	fi
1845	typeset status
1846	if [ $DRYRUN = 0 ]; then
1847
1848	# if [ ! -d ${R_SAVE}/${1} ] ; then
1849	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
1850	# echo "Nothing has been done."
1851	# return
1852	# fi
1853
1854	/ccc/cont003/home/dsm/p86ipsl/bin/dods_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
1855	status=$?
1856
1857	# if [ ${status} -gt 0 ] ; then
1858	# echo "IGCM_sys_Dods_Cp : error."
1859	# cat out_dods_cp
1860	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
1861	# else
1862	# rm out_dods_cp
1863	# fi
1864
1865	fi
1866	return $status
1867	}
1868
1869	#D-#==================================================
1870	#D-function IGCM_sys_Put_Dods
1871	#D-* Purpose: Put $(ARCHIVE) files on DODS internet protocole.
1872	#D-* Examples:
1873	#D-
1874	function IGCM_sys_Put_Dods {
1875	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1876	if ( $DEBUG_sys ) ; then
1877	echo "IGCM_sys_Put_Dods :" $@
1878	fi
1879	#set -vx
1880	typeset status
1881	if [ $DRYRUN = 0 ]; then
1882	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
1883	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
1884	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
1885	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
1886	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1887	return
1888	fi
1889
1890	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1891	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1892	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1893	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1894	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1895	return
1896	fi
1897	#
1898	if [ -d ${R_SAVE}/${1} ] ; then
1899	cd ${R_SAVE}
1900	elif [ -d ${R_FIGR}/${1} ] ; then
1901	cd ${R_FIGR}
1902	fi
1903
1904	IGCM_sys_Dods_Rm ${1}
1905	IGCM_sys_Dods_Cp ${1}
1906	status=0
1907
1908	if [ ${status} -gt 0 ] ; then
1909	echo "IGCM_sys_Put_Dods : error."
1910	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1911	fi
1912	fi
1913	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1914	}
1915
1916	##############################################################
1917	# REBUILD OPERATOR
1918
1919	#D-#==================================================
1920	#D-function IGCM_sys_sync
1921	#D-* Purpose: flush buffer on disk
1922	#D-* Examples:
1923	#D-
1924	function IGCM_sys_sync {
1925	IGCM_debug_PushStack "IGCM_sys_sync" $@
1926	if ( $DEBUG_sys ) ; then
1927	echo "IGCM_sys_sync :" $@
1928	fi
1929
1930	/bin/sync
1931
1932	IGCM_debug_PopStack "IGCM_sys_sync"
1933	}
1934
1935	#D-#==================================================
1936	#D-function IGCM_sys_rebuild
1937	#D-* Purpose: rebuild parallel files
1938	#D-* Examples:
1939	#D-
1940	function IGCM_sys_rebuild {
1941	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1942	if ( $DEBUG_sys ) ; then
1943	echo "IGCM_sys_rebuild :" $@
1944	fi
1945
1946	typeset NB_ESSAI DELAI status i firstArg
1947	# number of tentative
1948	NB_ESSAI=3
1949	# time delay between tentative
1950	DELAI=2
1951
1952	i=0
1953	while [ $i -lt $NB_ESSAI ] ; do
1954	~p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1955	status=$?
1956	if [ ${status} -gt 0 ] ; then
1957	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1958	cat /tmp/out_command.$$
1959	\rm /tmp/out_command.$$
1960	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1961	firstArg=${1}
1962	\rm ${firstArg}
1963	sleep $DELAI
1964	else
1965	\rm /tmp/out_command.$$
1966	break
1967	fi
1968	(( i = i + 1 ))
1969	done
1970
1971	if [ ${status} -gt 0 ] ; then
1972	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1973	IGCM_debug_Exit "rebuild"
1974	fi
1975
1976	IGCM_debug_PopStack "IGCM_sys_rebuild"
1977	}
1978
1979	#D-#==================================================
1980	#D-function IGCM_sys_rebuild_station
1981	#D-* Purpose: rebuild parallel files describing station
1982	#D-* Examples:
1983	#D-
1984	function IGCM_sys_rebuild_station {
1985	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1986	typeset i list_opt file_in file_out prefix_invert list_invert
1987	if ( $DEBUG_sys ) ; then
1988	echo "IGCM_sys_rebuild_station :" $@
1989	fi
1990	list_opt=$@
1991
1992	# Invert Axis : t,x -> x,t
1993	# t,pres,x -> x,t,pres
1994	# So that we can concatenate along x
1995	i=0
1996	for file_in in ${list_opt} ; do
1997	(( i = i + 1))
1998	[ ${i} = 1 ] && file_out=${file_in} && continue
1999	# detect time counter and do the job only if present
2000	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
2001	if [ X${var_unlim} = Xtime_counter ] ; then
2002	prefix_invert=$( basename ${file_in} .nc )
2003	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
2004	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
2005	fi
2006	done
2007
2008	# Concatenate
2009	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
2010
2011	# Re-ivert file
2012	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
2013
2014	# Station re-ordering is too expansive to be run within libIGCM
2015	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
2016	# This re-ordering must be done "in memory" by the cmorization process
2017	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
2018	# BEGIN reordering
2019
2020	# Only LMDZ text output contains the exact ordering of the station.
2021	# We isolate this in the code below:
2022	# 0 38 -157.5000000000000 70.98591549295774
2023	# 0 54 27.49999999999999 67.18309859154928
2024	# 0 56 -62.50000000000001 82.39436619718309
2025	# 0 79 12.49999999999999 78.59154929577466
2026	# 0 116 -165.0000000000000 76.05633802816901
2027	# 0 117 130.0000000000000 70.98591549295774
2028	# 0 118 110.0000000000000 87.46478873239437
2029	# 1 40 4.999999999999995 51.97183098591550
2030
2031	# typeset iStation iProc list_opt file_in file_out prefix_invert
2032	# typeset -Z4 j4
2033	# typeset -Z3 j3
2034
2035	# unset list_opt
2036	# set +A list_opt $@
2037
2038	# Filename after rebuild
2039	# file_out=${list_opt[0]}
2040	# Prefix of output files
2041	# prefix_invert=$( basename ${file_out} .nc )
2042	# Number of procs
2043	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
2044
2045	# iProc=0
2046	# while [ ${iProc} -lt ${num_proc} ] ; do
2047	# Array containing Station as a number
2048	# unset proc_stn
2049	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
2050	# Number of stations produced by processor proc
2051	# stationLast=${#proc_stn[*]}
2052	# Proc number on 4 digits
2053	# j4=${iProc}
2054	# Init
2055	# iStation=0
2056	# while [ ${iStation} -lt ${stationLast} ] ; do
2057	# Station number on 3 digits
2058	# j3=${proc_stn[${iStation}]}
2059	# Extract station
2060	# Invert Axis : t,x -> x,t
2061	# t,pres,x -> x,t,pres
2062	# So that we can concatenate along x
2063	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
2064	# (( iStation = iStation + 1 ))
2065	# done
2066	# (( iProc = iProc + 1 ))
2067	# done
2068
2069	# Concatenate all station along x
2070	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
2071
2072	# Re-invert file
2073	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
2074
2075	# END reordering
2076
2077	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
2078	}
2079
2080	############################################################
2081	# Activate Running Environnment Variables
2082
2083	#D-#==================================================
2084	#D-function IGCM_sys_desactiv_variables
2085	#D-* Purpose: set environement variables prior to execution
2086	#D-* Examples:
2087	#D-
2088	function IGCM_sys_activ_variables {
2089	IGCM_debug_PushStack "IGCM_sys_activ_variables"
2090	if ( $DEBUG_sys ) ; then
2091	echo "IGCM_sys_activ_variables"
2092	fi
2093
2094	# --------------------------------------------------------------------
2095	#D- MPI specifications
2096	# --------------------------------------------------------------------
2097
2098	# --------------------------------------------------------------------
2099	#D- Other specifications
2100	# --------------------------------------------------------------------
2101
2102	ulimit -s unlimited
2103
2104	IGCM_debug_PopStack "IGCM_sys_activ_variables"
2105	}
2106
2107	############################################################
2108	# Desactivate Running Environnment Variables
2109
2110	#D-#==================================================
2111	#D-function IGCM_sys_desactiv_variables
2112	#D-* Purpose: unset environement variables after execution
2113	#D-* Examples:
2114	#D-
2115	function IGCM_sys_desactiv_variables {
2116	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
2117	if ( $DEBUG_sys ) ; then
2118	echo "IGCM_sys_desactiv_variables"
2119	fi
2120	# --------------------------------------------------------------------
2121	#D- MPI specifications
2122	# --------------------------------------------------------------------
2123
2124	# --------------------------------------------------------------------
2125	#D- Other specifications
2126	# --------------------------------------------------------------------
2127
2128	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
2129	}
2130
2131	############################################################
2132	# Build MPI/OMP scripts run file (dummy function)
2133
2134	#D-#==================================================
2135	#D-function IGCM_sys_build_run_file
2136	#D-* Purpose: build run file (deprecated)
2137	#D-* Examples:
2138	#D-
2139	function IGCM_sys_build_run_file {
2140
2141	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
2142
2143	}
2144
2145	############################################################
2146	# Build MPI/OMP scripts
2147
2148	#D-#==================================================
2149	#D-function IGCM_sys_build_execution_scripts
2150	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
2151	#D-* Examples:
2152	#D-
2153	function IGCM_sys_build_execution_scripts
2154	{
2155	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
2156	if ( $DEBUG_sys ) ; then
2157	echo "IGCM_sys_build_execution_scripts " $@
2158	fi
2159
2160	EXECUTION=${HOST_MPIRUN_COMMAND}
2161
2162	# MPMD mode
2163	if ( ${OK_PARA_MPMD} ) ; then
2164
2165	# Only MPI (MPMD)
2166	if ( ! ${OK_PARA_OMP} ) ; then
2167
2168	if [ -f run_file ] ; then
2169	IGCM_sys_Rm -f run_file
2170	fi
2171	touch run_file
2172
2173	# Build run_file
2174
2175	# First loop on the components for the coupler ie oasis (only if oasis3)
2176	# the coupler ie oasis3 must be the first one
2177	for comp in ${config_ListOfComponents[*]} ; do
2178
2179	eval ExeNameIn=\${config_Executable_${comp}[0]}
2180	eval ExeNameOut=\${config_Executable_${comp}[1]}
2181
2182	# for CPL component only
2183	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
2184
2185	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2186	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2187	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2188	fi
2189	done
2190
2191	# Then second loop on the components
2192	for comp in ${config_ListOfComponents[*]} ; do
2193
2194	eval ExeNameIn=\${config_Executable_${comp}[0]}
2195	eval ExeNameOut=\${config_Executable_${comp}[1]}
2196
2197	# Only if we really have an executable for the component and not the coupler ie oasis:
2198	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2199
2200	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2201	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2202	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2203	fi
2204	done
2205
2206	EXECUTION="${HOST_MPIRUN_COMMAND} -f ./run_file"
2207
2208	IGCM_sys_Chmod u+x run_file
2209	if ( $DEBUG_sys ) ; then
2210	echo "run_file contains : "
2211	cat run_file
2212	fi
2213
2214	# MPI-OpenMP (MPMD)
2215	else
2216
2217	# Use of mpirun instead of ccc_mprun
2218	EXECUTION="time mpirun"
2219
2220	# Hosts treatment
2221	${EXECUTION} hostname \| sort \| uniq > hosts.tmp
2222
2223	i=0
2224	rm -f hosts rankfile
2225	IGCM_debug_Print 1 "sys Curie, Hosts available :"
2226	for nodes in `cat hosts.tmp`
2227	do
2228	host[$i]=$nodes
2229	echo "${host[$i]}" >> hosts
2230	IGCM_debug_Print 1 ${host[$i]}
2231	i=$((i+1))
2232	done
2233	rm -f hosts.tmp
2234
2235	listnodes=${host[*]}
2236
2237	EXECUTION="${EXECUTION} -hostfile hosts -rankfile rankfile"
2238
2239	# Initialisation
2240	rank=0
2241	current_core=0
2242	core_per_node=16
2243	init_exec=n
2244
2245	# Loop on the components
2246	for comp in ${config_ListOfComponents[*]} ; do
2247
2248	eval ExeNameIn=\${config_Executable_${comp}[0]}
2249	eval ExeNameOut=\${config_Executable_${comp}[1]}
2250
2251	# Not possible if oasis has an executable (i.e old version of oasis3)
2252	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
2253	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
2254	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
2255	IGCM_debug_Verif_Exit
2256	fi
2257
2258	# Only if we really have an executable for the component :
2259	if [ "X${ExeNameOut}" != X\"\" ] ; then
2260
2261	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2262	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2263
2264	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2265	echo "" >> script_${ExeNameOut}.ksh
2266	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
2267
2268	# Check if the number of threads is correct
2269	case ${comp_proc_omp_loc} in
2270	2\|4\|8\|16)
2271	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2272	;;
2273	*)
2274	IGCM_debug_Exit "ERROR with OMP parameters !"
2275	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2276	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2277	IGCM_debug_Verif_Exit
2278	;;
2279	esac
2280	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2281	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2282	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2283	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2284	fi
2285	#echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
2286	#echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
2287	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2288	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2289
2290	if [ ${init_exec} = y ] ; then
2291	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2292	else
2293	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2294	init_exec=y
2295	fi
2296
2297	# Build rankfile : method used to assign cores and nodes for the MPI process
2298	# Ex :
2299	#rank 0=curie5296 slot=0,1,2,3
2300	#rank 1=curie5296 slot=4,5,6,7
2301	# Example of final command :
2302	# mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
2303	# with script_lmdz.x.ksh :
2304	# #!/bin/ksh
2305	#export KMP_STACKSIZE=3g
2306	#export KMP_LIBRARY=turnaround
2307	#export MKL_SERIAL=YES
2308	#OMP_NUM_THREADS=4
2309	#./lmdz.x
2310	#
2311
2312	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
2313	(( index_host = current_core / core_per_node ))
2314	host_value=${host[${index_host}]}
2315	(( slot = current_core % core_per_node ))
2316	virg=","
2317	string_final=""
2318	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
2319	string=$index$virg
2320	string_final=$string_final$string
2321	done
2322	string_final=$( echo $string_final \| sed "s/.$//" )
2323	echo "rank $rank=$host_value slot=$string_final" >> rankfile
2324	(( rank = rank + 1 ))
2325	(( current_core = current_core + comp_proc_omp_loc ))
2326	done
2327	fi
2328
2329	done
2330	fi
2331
2332	# Only one executable (SPMD mode).
2333	else
2334
2335	for comp in ${config_ListOfComponents[*]} ; do
2336
2337	# Only if we really have an executable for the component :
2338	eval ExeNameOut=\${config_Executable_${comp}[1]}
2339	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2340
2341	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2342	echo "" >> script_${ExeNameOut}.ksh
2343	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2344
2345	if ( ${OK_PARA_OMP} ) ; then
2346	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2347	echo "" >> script_${ExeNameOut}.ksh
2348	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2349	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2350	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2351	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2352	fi
2353
2354	if ( ${OK_PARA_MPI} ) ; then
2355	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2356	# Default : ccc_mprun used if nb_proc gt 1
2357	# to have out/err per process on different files
2358	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
2359	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2360	EXECUTION="${HOST_MPIRUN_COMMAND} -n ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2361	else
2362	# Default : ccc_mprun is NOT used if nb_proc eq 1
2363	# to have out/err per process on different files
2364	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2365	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2366	EXECUTION="/usr/bin/time ./script_${ExeNameOut}.ksh"
2367	fi
2368
2369	IGCM_debug_Print 1 "sys Curie : script_${ExeNameOut}.ksh contains"
2370	cat script_${ExeNameOut}.ksh
2371
2372	fi
2373
2374	done
2375
2376	fi
2377
2378	IGCM_debug_Print 1 "sys Curie : execution command is "
2379	IGCM_debug_Print 1 "$EXECUTION"
2380
2381	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2382	}
2383
2384	#D-#==================================================
2385	#D-function IGCM_sys_check_path
2386	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2387	#D-* do not point to an important use directory. Stop immediately in that case.
2388	#D-* Examples:
2389	#D-
2390	function IGCM_sys_check_path {
2391	IGCM_debug_PushStack "IGCM_sys_check_path"
2392	if ( $DEBUG_sys ) ; then
2393	echo "IGCM_sys_check_path"
2394	fi
2395
2396	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCHDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCWORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCSTOREDIR} ] ) ; then
2397	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2398	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2399	IGCM_debug_Exit "This will stop the job"
2400	fi
2401	IGCM_debug_PopStack "IGCM_sys_check_path"
2402	}
2403
2404	#D-#==================================================
2405	#D-function IGCM_sys_check_quota
2406	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2407	#D-* Examples:
2408	#D-
2409	function IGCM_sys_check_quota {
2410	IGCM_debug_PushStack "IGCM_sys_check_quota"
2411	if ( $DEBUG_sys ) ; then
2412	echo "IGCM_sys_check_quota"
2413	fi
2414	# Limit of quota (in %)
2415	limit_quota=90
2416
2417	# Check of the volume
2418	volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
2419	volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
2420
2421	if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
2422
2423	unit_avail=${volume_avail: -1}
2424	unit_quota=${volume_quota: -1}
2425
2426	if [ "${unit_quota}" = "*" ] ; then
2427	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2428	IGCM_debug_Print 1 "More than 100% of your quota is used"
2429	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2430	IGCM_debug_Print 1 "You must have more than 10% available to run"
2431	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2432	IGCM_debug_Verif_Exit
2433	fi
2434
2435	temp_avail=${volume_avail%%${volume_avail: -1}*}
2436	temp_quota=${volume_quota%%${volume_quota: -1}*}
2437
2438	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2439
2440	# Convertion
2441	if [ ${volume_avail: -1} = "T" ] ; then
2442	(( temp_avail = temp_avail * 1000000000000 ))
2443	elif [ ${volume_avail: -1} = "G" ] ; then
2444	(( temp_avail = temp_avail * 1000000000 ))
2445	elif [ ${volume_avail: -1} = "M" ] ; then
2446	(( temp_avail = temp_avail * 1000000 ))
2447	elif [ ${volume_avail: -1} = "k" ] ; then
2448	(( temp_avail = temp_avail * 1000 ))
2449	else
2450	(( temp_avail = volume_avail ))
2451	fi
2452	if [ ${volume_quota: -1} = "T" ] ; then
2453	(( temp_quota = temp_quota * 1000000000000 ))
2454	elif [ ${volume_quota: -1} = "G" ] ; then
2455	(( temp_quota = temp_quota * 1000000000 ))
2456	elif [ ${volume_quota: -1} = "M" ] ; then
2457	(( temp_quota = temp_quota * 1000000 ))
2458	elif [ ${volume_quota: -1} = "k" ] ; then
2459	(( temp_quota = temp_quota * 1000 ))
2460	else
2461	(( temp_quota = volume_quota ))
2462	fi
2463	fi
2464
2465	quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2466	# echo "volume ratio is " $quota_volume
2467
2468	if [ ${quota_volume} -ge ${limit_quota} ] ; then
2469	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2470	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2471	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2472	IGCM_debug_Print 1 "You must have more than 10% available to run"
2473	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2474	IGCM_debug_Verif_Exit
2475	fi
2476
2477	fi
2478
2479	# Check of the number of inodes
2480
2481	inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
2482	inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
2483
2484	if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
2485
2486	unit_avail=${inode_avail: -1}
2487	unit_quota=${inode_quota: -1}
2488
2489	if [ "${unit_quota}" = "*" ] ; then
2490	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2491	IGCM_debug_Print 1 "More than 100% of your quota is used"
2492	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2493	IGCM_debug_Print 1 "You must have more than 10% available to run"
2494	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2495	IGCM_debug_Verif_Exit
2496	fi
2497
2498	temp_avail=${inode_avail%%${inode_avail: -1}*}
2499	temp_quota=${inode_quota%%${inode_quota: -1}*}
2500
2501	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2502
2503	# Convertion
2504	if [ ${inode_avail: -1} = "T" ] ; then
2505	(( temp_avail = temp_avail * 1000000000000 ))
2506	elif [ ${inode_avail: -1} = "G" ] ; then
2507	(( temp_avail = temp_avail * 1000000000 ))
2508	elif [ ${inode_avail: -1} = "M" ] ; then
2509	(( temp_avail = temp_avail * 1000000 ))
2510	elif [ ${inode_avail: -1} = "k" ] ; then
2511	(( temp_avail = temp_avail * 1000 ))
2512	else
2513	(( temp_avail = inode_avail ))
2514	fi
2515
2516	if [ ${inode_quota: -1} = "T" ] ; then
2517	(( temp_quota = temp_quota * 1000000000000 ))
2518	elif [ ${inode_quota: -1} = "G" ] ; then
2519	(( temp_quota = temp_quota * 1000000000 ))
2520	elif [ ${inode_quota: -1} = "M" ] ; then
2521	(( temp_quota = temp_quota * 1000000 ))
2522	elif [ ${inode_quota: -1} = "k" ] ; then
2523	(( temp_quota = temp_quota * 1000 ))
2524	else
2525	(( temp_quota = inode_quota ))
2526	fi
2527	fi
2528	quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2529	# echo "inode ratio is " $quota_inode
2530
2531	if [ ${quota_inode} -ge ${limit_quota} ] ; then
2532	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2533	IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
2534	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2535	IGCM_debug_Print 1 "You must have more than 10% available to run"
2536	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2537	IGCM_debug_Verif_Exit
2538	fi
2539	fi
2540	IGCM_debug_PopStack "IGCM_sys_check_quota"
2541	}
2542
2543	#D-#==================================================
2544	#D-function IGCM_sys_GetJobID
2545	#D-* Purpose: Check if job_name is currently
2546	#D- running or in queue
2547	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2548	#D-
2549	function IGCM_sys_GetJobID {
2550	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2551	if ( $DEBUG_sys ) ; then
2552	echo "IGCM_sys_GetJobID"
2553	fi
2554
2555	# With -f option, the full job name is given in the last column
2556	ID=$( ccc_mstat -f -u $2 \| \
2557	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2558
2559	eval ${3}=${ID}
2560
2561	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2562	}
2563
2564	#D-#==================================================
2565	#D-function IGCM_sys_CountJobInQueue
2566	#D-* Purpose: Check if job_name is currently
2567	#D- running or in queue
2568	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2569	#D-
2570	function IGCM_sys_CountJobInQueue {
2571	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2572	if ( $DEBUG_sys ) ; then
2573	echo "IGCM_sys_CountJobInQueue"
2574	fi
2575
2576	#NbRun=$( ccc_mstat -f \| grep -c ${JobName} )
2577
2578	# With -f option, the full job name is given in the last column
2579	NbRun=$( ccc_mstat -f \| gawk -v JobName=$1 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
2580
2581	eval ${2}=${NbRun}
2582
2583	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2584	}
2585
2586	#D-#==================================================
2587	#D-function IGCM_sys_ListJobInQueue
2588	#D-* Purpose: Check if job_name is currently
2589	#D- running or in queue
2590	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2591	#D-
2592	function IGCM_sys_ListJobInQueue {
2593	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2594	if ( $DEBUG_sys ) ; then
2595	echo "IGCM_sys_ListJobInQueue"
2596	fi
2597
2598	# With -f option, the full job name is given in the last column
2599	set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
2600	'( $2 == User && \
2601	$NF != /TS/ && \
2602	$NF !~ /PACK/ && \
2603	$NF !~ /REBUILD/ && \
2604	$NF !~ /pack/ ) \
2605	{ print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
2606
2607	eval set -A ${2} ${JobList[*]}
2608
2609	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2610	}
2611
2612	##############################################################
2613	# NCO OPERATOR
2614
2615	#D-#==================================================
2616	#D-function IGCM_sys_ncap2
2617	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2618	#D-* Examples:
2619	#D-
2620	function IGCM_sys_ncap2 {
2621	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2622	if ( $DEBUG_sys ) ; then
2623	echo "IGCM_sys_ncap2 :" $@
2624	fi
2625
2626	typeset NB_ESSAI DELAI status i
2627	# number of tentative
2628	NB_ESSAI=3
2629	# time delay between tentative
2630	DELAI=2
2631
2632	i=0
2633	while [ $i -lt $NB_ESSAI ] ; do
2634	ncap2 -C "$@" > /tmp/out_command.$$ 2>&1
2635	status=$?
2636	if [ ${status} -gt 0 ] ; then
2637	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2638	cat /tmp/out_command.$$
2639	\rm /tmp/out_command.$$
2640	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2641	sleep $DELAI
2642	else
2643	\rm /tmp/out_command.$$
2644	break
2645	fi
2646	(( i = i + 1 ))
2647	done
2648
2649	if [ ${status} -gt 0 ] ; then
2650	echo "IGCM_sys_ncap2 : ncap2 error"
2651	IGCM_debug_Exit "ncap2"
2652	fi
2653
2654	IGCM_debug_PopStack "IGCM_sys_ncap2"
2655	}
2656
2657	#D-#==================================================
2658	#D-function IGCM_sys_ncatted
2659	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2660	#D-* Examples:
2661	#D-
2662	function IGCM_sys_ncatted {
2663	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2664	if ( $DEBUG_sys ) ; then
2665	echo "IGCM_sys_ncatted :" $@
2666	fi
2667
2668	typeset NB_ESSAI DELAI status i
2669	# number of tentative
2670	NB_ESSAI=3
2671	# time delay between tentative
2672	DELAI=2
2673
2674	i=0
2675	while [ $i -lt $NB_ESSAI ] ; do
2676	ncatted "$@" > /tmp/out_command.$$ 2>&1
2677	status=$?
2678	if [ ${status} -gt 0 ] ; then
2679	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2680	cat /tmp/out_command.$$
2681	\rm /tmp/out_command.$$
2682	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2683	sleep $DELAI
2684	else
2685	\rm /tmp/out_command.$$
2686	break
2687	fi
2688	(( i = i + 1 ))
2689	done
2690
2691	if [ ${status} -gt 0 ] ; then
2692	echo "IGCM_sys_ncatted : ncatted error"
2693	IGCM_debug_Exit "ncatted"
2694	fi
2695
2696	IGCM_debug_PopStack "IGCM_sys_ncatted"
2697	}
2698
2699	#D-#==================================================
2700	#D-function IGCM_sys_ncbo
2701	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2702	#D-* Examples:
2703	#D-
2704	function IGCM_sys_ncbo {
2705	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2706	if ( $DEBUG_sys ) ; then
2707	echo "IGCM_sys_ncbo :" $@
2708	fi
2709
2710	typeset NB_ESSAI DELAI status i
2711	# number of tentative
2712	NB_ESSAI=3
2713	# time delay between tentative
2714	DELAI=2
2715
2716	i=0
2717	while [ $i -lt $NB_ESSAI ] ; do
2718	ncbo -C $@ > /tmp/out_command.$$ 2>&1
2719	status=$?
2720	if [ ${status} -gt 0 ] ; then
2721	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2722	cat /tmp/out_command.$$
2723	\rm /tmp/out_command.$$
2724	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2725	sleep $DELAI
2726	else
2727	\rm /tmp/out_command.$$
2728	break
2729	fi
2730	(( i = i + 1 ))
2731	done
2732
2733	if [ ${status} -gt 0 ] ; then
2734	echo "IGCM_sys_ncbo : ncbo error"
2735	IGCM_debug_Exit "ncbo"
2736	fi
2737
2738	IGCM_debug_PopStack "IGCM_sys_ncbo"
2739	}
2740
2741	#D-#==================================================
2742	#D-function IGCM_sys_ncdif
2743	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2744	#D-* Examples:
2745	#D-
2746	function IGCM_sys_ncdiff {
2747	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2748	if ( $DEBUG_sys ) ; then
2749	echo "IGCM_sys_ncdiff :" $@
2750	fi
2751
2752	typeset NB_ESSAI DELAI status i
2753	# number of tentative
2754	NB_ESSAI=3
2755	# time delay between tentative
2756	DELAI=2
2757
2758	i=0
2759	while [ $i -lt $NB_ESSAI ] ; do
2760	ncdiff -C $@ > /tmp/out_command.$$ 2>&1
2761	status=$?
2762	if [ ${status} -gt 0 ] ; then
2763	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2764	cat /tmp/out_command.$$
2765	\rm /tmp/out_command.$$
2766	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2767	sleep $DELAI
2768	else
2769	\rm /tmp/out_command.$$
2770	break
2771	fi
2772	(( i = i + 1 ))
2773	done
2774
2775	if [ ${status} -gt 0 ] ; then
2776	echo "IGCM_sys_ncdiff : ncdiff error"
2777	IGCM_debug_Exit "ncdiff"
2778	fi
2779
2780	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2781	}
2782
2783	#D-#==================================================
2784	#D-function IGCM_sys_ncea
2785	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2786	#D-* Examples:
2787	#D-
2788	function IGCM_sys_ncea {
2789	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2790	if ( $DEBUG_sys ) ; then
2791	echo "IGCM_sys_ncea :" $@
2792	fi
2793
2794	typeset NB_ESSAI DELAI status i
2795	# number of tentative
2796	NB_ESSAI=3
2797	# time delay between tentative
2798	DELAI=2
2799
2800	i=0
2801	while [ $i -lt $NB_ESSAI ] ; do
2802	ncea -C $@ > /tmp/out_command.$$ 2>&1
2803	status=$?
2804	if [ ${status} -gt 0 ] ; then
2805	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2806	cat /tmp/out_command.$$
2807	\rm /tmp/out_command.$$
2808	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2809	sleep $DELAI
2810	else
2811	\rm /tmp/out_command.$$
2812	break
2813	fi
2814	(( i = i + 1 ))
2815	done
2816
2817	if [ ${status} -gt 0 ] ; then
2818	echo "IGCM_sys_ncea : ncea error"
2819	IGCM_debug_Exit "ncea"
2820	fi
2821
2822	IGCM_debug_PopStack "IGCM_sys_ncea"
2823	}
2824
2825	#D-#==================================================
2826	#D-function IGCM_sys_ncecat
2827	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2828	#D-* Examples:
2829	#D-
2830	function IGCM_sys_ncecat {
2831	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2832	if ( $DEBUG_sys ) ; then
2833	echo "IGCM_sys_ncecat :" $@
2834	fi
2835
2836	typeset NB_ESSAI DELAI status i
2837	# number of tentative
2838	NB_ESSAI=3
2839	# time delay between tentative
2840	DELAI=2
2841
2842	i=0
2843	while [ $i -lt $NB_ESSAI ] ; do
2844	ncecat -C $@ > /tmp/out_command.$$ 2>&1
2845	status=$?
2846	if [ ${status} -gt 0 ] ; then
2847	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2848	cat /tmp/out_command.$$
2849	\rm /tmp/out_command.$$
2850	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2851	sleep $DELAI
2852	else
2853	\rm /tmp/out_command.$$
2854	break
2855	fi
2856	(( i = i + 1 ))
2857	done
2858
2859	if [ ${status} -gt 0 ] ; then
2860	echo "IGCM_sys_ncecat : ncecat error"
2861	IGCM_debug_Exit "ncecat"
2862	fi
2863
2864	IGCM_debug_PopStack "IGCM_sys_ncecat"
2865	}
2866
2867	#D-#==================================================
2868	#D-function IGCM_sys_ncflint
2869	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2870	#D-* Examples:
2871	#D-
2872	function IGCM_sys_ncflint {
2873	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2874	if ( $DEBUG_sys ) ; then
2875	echo "IGCM_sys_ncflint :" $@
2876	fi
2877
2878	typeset NB_ESSAI DELAI status i
2879	# number of tentative
2880	NB_ESSAI=3
2881	# time delay between tentative
2882	DELAI=2
2883
2884	i=0
2885	while [ $i -lt $NB_ESSAI ] ; do
2886	ncflint -C $@ > /tmp/out_command.$$ 2>&1
2887	status=$?
2888	if [ ${status} -gt 0 ] ; then
2889	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2890	cat /tmp/out_command.$$
2891	\rm /tmp/out_command.$$
2892	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2893	sleep $DELAI
2894	else
2895	\rm /tmp/out_command.$$
2896	break
2897	fi
2898	(( i = i + 1 ))
2899	done
2900
2901	if [ ${status} -gt 0 ] ; then
2902	echo "IGCM_sys_ncflint : ncflint error"
2903	IGCM_debug_Exit "ncflint"
2904	fi
2905
2906	IGCM_debug_PopStack "IGCM_sys_ncflint"
2907	}
2908
2909	#D-#==================================================
2910	#D-function IGCM_sys_ncks
2911	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2912	#D-* Examples:
2913	#D-
2914	function IGCM_sys_ncks {
2915	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2916	if ( $DEBUG_sys ) ; then
2917	echo "IGCM_sys_ncks :" $@
2918	fi
2919
2920	typeset NB_ESSAI DELAI status i
2921	# number of tentative
2922	NB_ESSAI=3
2923	# time delay between tentative
2924	DELAI=2
2925
2926	i=0
2927	while [ $i -lt $NB_ESSAI ] ; do
2928	ncks -C $@ > /tmp/out_command.$$ 2>&1
2929	status=$?
2930	if [ ${status} -gt 0 ] ; then
2931	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2932	cat /tmp/out_command.$$
2933	\rm /tmp/out_command.$$
2934	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2935	sleep $DELAI
2936	else
2937	\rm /tmp/out_command.$$
2938	break
2939	fi
2940	(( i = i + 1 ))
2941	done
2942
2943	if [ ${status} -gt 0 ] ; then
2944	echo "IGCM_sys_ncks : ncks error"
2945	IGCM_debug_Exit "ncks"
2946	fi
2947
2948	IGCM_debug_PopStack "IGCM_sys_ncks"
2949	}
2950
2951	#D-#==================================================
2952	#D-function IGCM_sys_ncpdq
2953	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2954	#D-* Examples:
2955	#D-
2956	function IGCM_sys_ncpdq {
2957	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2958	if ( $DEBUG_sys ) ; then
2959	echo "IGCM_sys_ncpdq :" $@
2960	fi
2961
2962	typeset NB_ESSAI DELAI status i
2963	# number of tentative
2964	NB_ESSAI=3
2965	# time delay between tentative
2966	DELAI=2
2967
2968	i=0
2969	while [ $i -lt $NB_ESSAI ] ; do
2970	ncpdq -C $@ > /tmp/out_command.$$ 2>&1
2971	status=$?
2972	if [ ${status} -gt 0 ] ; then
2973	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2974	cat /tmp/out_command.$$
2975	\rm /tmp/out_command.$$
2976	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2977	sleep $DELAI
2978	else
2979	\rm /tmp/out_command.$$
2980	break
2981	fi
2982	(( i = i + 1 ))
2983	done
2984
2985	if [ ${status} -gt 0 ] ; then
2986	echo "IGCM_sys_ncpdq : ncpdq error"
2987	IGCM_debug_Exit "ncpdq"
2988	fi
2989
2990	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2991	}
2992
2993	#D-#==================================================
2994	#D-function IGCM_sys_ncra
2995	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2996	#D-* Examples:
2997	#D-
2998	function IGCM_sys_ncra {
2999	IGCM_debug_PushStack "IGCM_sys_ncra" $@
3000	if ( $DEBUG_sys ) ; then
3001	echo "IGCM_sys_ncra :" $@
3002	fi
3003
3004	typeset NB_ESSAI DELAI status i
3005	# number of tentative
3006	NB_ESSAI=3
3007	# time delay between tentative
3008	DELAI=2
3009
3010	i=0
3011	while [ $i -lt $NB_ESSAI ] ; do
3012	ncra -C $@ > /tmp/out_command.$$ 2>&1
3013	status=$?
3014	if [ ${status} -gt 0 ] ; then
3015	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
3016	cat /tmp/out_command.$$
3017	\rm /tmp/out_command.$$
3018	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3019	sleep $DELAI
3020	else
3021	\rm /tmp/out_command.$$
3022	break
3023	fi
3024	(( i = i + 1 ))
3025	done
3026
3027	if [ ${status} -gt 0 ] ; then
3028	echo "IGCM_sys_ncra : ncra error"
3029	IGCM_debug_Exit "ncra"
3030	fi
3031
3032	IGCM_debug_PopStack "IGCM_sys_ncra"
3033	}
3034
3035	#D-#==================================================
3036	#D-function IGCM_sys_ncrcat
3037	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
3038	#D-* Examples:
3039	#D-
3040	function IGCM_sys_ncrcat {
3041	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
3042	if ( $DEBUG_sys ) ; then
3043	echo "IGCM_sys_ncrcat :" $@
3044	fi
3045
3046	typeset NB_ESSAI DELAI status i lastArg
3047	# number of tentative
3048	NB_ESSAI=3
3049	# time delay between tentative
3050	DELAI=2
3051
3052	i=0
3053	while [ $i -lt $NB_ESSAI ] ; do
3054	ncrcat -C $@ > /tmp/out_command.$$ 2>&1
3055	status=$?
3056	if [ ${status} -gt 0 ] ; then
3057	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
3058	cat /tmp/out_command.$$
3059	\rm /tmp/out_command.$$
3060	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3061	sleep $DELAI
3062	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
3063	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
3064	cat /tmp/out_command.$$
3065	# remove files having corrupted time axis
3066	eval lastArg=\${$#}
3067	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
3068	\rm ${lastArg}
3069	\rm /tmp/out_command.$$
3070	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3071	sleep $DELAI
3072	else
3073	\rm /tmp/out_command.$$
3074	break
3075	fi
3076	(( i = i + 1 ))
3077	done
3078
3079	if [ ${status} -gt 0 ] ; then
3080	echo "IGCM_sys_ncrcat : ncrcat error"
3081	#IGCM_debug_Exit "ncrcat"
3082	fi
3083
3084	IGCM_debug_PopStack "IGCM_sys_ncrcat"
3085	}
3086
3087	#D-#==================================================
3088	#D-function IGCM_sys_ncrename
3089	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
3090	#D-* Examples:
3091	#D-
3092	function IGCM_sys_ncrename {
3093	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
3094	if ( $DEBUG_sys ) ; then
3095	echo "IGCM_sys_ncrename :" $@
3096	fi
3097
3098	typeset NB_ESSAI DELAI status i
3099	# number of tentative
3100	NB_ESSAI=3
3101	# time delay between tentative
3102	DELAI=2
3103
3104	i=0
3105	while [ $i -lt $NB_ESSAI ] ; do
3106	ncrename $@ > /tmp/out_command.$$ 2>&1
3107	status=$?
3108	if [ ${status} -gt 0 ] ; then
3109	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
3110	cat /tmp/out_command.$$
3111	\rm /tmp/out_command.$$
3112	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3113	sleep $DELAI
3114	else
3115	\rm /tmp/out_command.$$
3116	break
3117	fi
3118	(( i = i + 1 ))
3119	done
3120
3121	if [ ${status} -gt 0 ] ; then
3122	echo "IGCM_sys_ncrename : ncrename error"
3123	IGCM_debug_Exit "ncrename"
3124	fi
3125
3126	IGCM_debug_PopStack "IGCM_sys_ncrename"
3127	}
3128
3129	#D-#==================================================
3130	#D-function IGCM_sys_ncwa
3131	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
3132	#D-* Examples:
3133	#D-
3134	function IGCM_sys_ncwa {
3135	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
3136	if ( $DEBUG_sys ) ; then
3137	echo "IGCM_sys_ncwa :" $@
3138	fi
3139
3140	typeset NB_ESSAI DELAI status i
3141	# number of tentative
3142	NB_ESSAI=3
3143	# time delay between tentative
3144	DELAI=2
3145
3146	i=0
3147	while [ $i -lt $NB_ESSAI ] ; do
3148	ncwa -C $@ > /tmp/out_command.$$ 2>&1
3149	status=$?
3150	if [ ${status} -gt 0 ] ; then
3151	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
3152	cat /tmp/out_command.$$
3153	\rm /tmp/out_command.$$
3154	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3155	sleep $DELAI
3156	else
3157	\rm /tmp/out_command.$$
3158	break
3159	fi
3160	(( i = i + 1 ))
3161	done
3162
3163	if [ ${status} -gt 0 ] ; then
3164	echo "IGCM_sys_ncwa : ncwa error"
3165	IGCM_debug_Exit "ncwa"
3166	fi
3167
3168	IGCM_debug_PopStack "IGCM_sys_ncwa"
3169	}
3170
3171	##############################################################
3172	# CDO OPERATOR
3173
3174	#D-#==================================================
3175	#D-function IGCM_sys_cdo
3176	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
3177	#D-* Examples:
3178	#D-
3179	function IGCM_sys_cdo {
3180	IGCM_debug_PushStack "IGCM_sys_cdo" $@
3181	if ( $DEBUG_sys ) ; then
3182	echo "IGCM_sys_cdo :" $@
3183	fi
3184
3185	typeset status
3186
3187	\cdo $@ > /tmp/out_command.$$ 2>&1
3188	status=$?
3189	if [ ${status} -gt 0 ] ; then
3190	echo "IGCM_sys_cdo : error code ${status}"
3191	cat /tmp/out_command.$$
3192	\rm /tmp/out_command.$$
3193	IGCM_debug_PopStack "IGCM_sys_cdo"
3194	return 1
3195	else
3196	IGCM_debug_PopStack "IGCM_sys_cdo"
3197	return 0
3198	fi
3199
3200	IGCM_debug_PopStack "IGCM_sys_cdo"
3201	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: