Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_curie.ksh @ 1178

Last change on this file since 1178 was 1178, checked in by sdipsl, 9 years ago
house keeping in libIGCM_sys (first pass). Will ease #248
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 92.4 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Curie
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=curie
71	# add default project on curie
72	typeset DEFAULT_PROJECT=gen0826
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# Submit command
80	typeset SUBMIT=${SUBMIT:=ccc_msub}
81	# rsync with path
82	typeset -r RSYNC=/usr/bin/rsync
83	# RSYNC_opt args to rsync
84	typeset -r RSYNC_opt="-va"
85	# ie storage filesystem
86	typeset -r STOREHOST=${MASTER}
87
88	#====================================================
89	# Set environment tools (ferret, nco, cdo)
90	#====================================================
91	if [ X${TaskType} = Xcomputing ] ; then
92	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
93	# to run with netcdf 3.6.3 ie compilation done before 17/2/2014
94	# uncomment 2 lines :
95	# module unload netcdf
96	# module load netcdf/3.6.3
97	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddNoise/src_X64_CURIE/bin
98	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddPerturbation/src_X64_CURIE/bin
99	else
100	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
101	PCMDI_MP=/ccc/work/cont003/dsm/p86ipsl/PCMDI-MP
102	fi
103
104	#====================================================
105	# Specific for ocean additionnal diagnostic
106	export FER_GO="$FER_GO /home/cont003/p86denv/IGCM_POST_UTIL/JNL /home/cont003/p86denv/GRAF /home/cont003/p86denv/GRAF/GO"
107	export FER_PALETTE="$FER_PALETTE /home/cont003/p86denv/GRAF/PALET"
108
109	#====================================================
110	# Host specific DIRECTORIES
111	#====================================================
112
113	# ============ CESIUM START ============ #
114
115	#====================================================
116	#- Mirror libIGCM from titane to cesium if needed
117	#ROOTSYS=$( echo ${libIGCM} \| gawk -F"/" '{print $3}' )
118	#if [ ! ${ROOTSYS} = "home" ] ; then
119	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
120	#else
121	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
122	#fi
123
124	#====================================================
125	#- libIGCM_POST
126	#if ( ${MirrorlibIGCM} ) ; then
127	# PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
128	# typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
129	#else
130	# typeset -r libIGCM_POST=${libIGCM}
131	#fi
132
133	# ============ CESIUM END ============ #
134
135	#====================================================
136	#- MirrorlibIGCM for frontend
137	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
138
139	#====================================================
140	#- libIGCM_POST for frontend
141	typeset -r libIGCM_POST=${libIGCM}
142
143	#====================================================
144	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
145	typeset -r R_EXE="${MODIPSL}/bin"
146
147	#====================================================
148	#- SUBMIT_DIR : submission dir
149	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${BRIDGE_MSUB_PWD}}
150
151	#====================================================
152	#- IN
153	typeset -r R_IN=${R_IN:=/ccc/work/cont003/dsm/p86ipsl/IGCM}
154	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/ccc/work/cont003/dsm/p24data}
155
156	#====================================================
157	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
158	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCHDIR}/RUN_DIR/${BRIDGE_MSUB_JOBID}_${$}}
159
160	#====================================================
161	#- HOST_MPIRUN_COMMAND
162	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time ccc_mprun -E-K1"}
163
164	#====================================================
165	#- Max number of arguments passed to nco operator or demigration command
166	UNIX_MAX_LIMIT=360
167
168	#====================================================
169	#- set PackDefault true on curie
170	PackDefault=true
171
172	#====================================================
173	#- Number of core per node (max number of OpenMP task)
174	NUM_COREPERNODE=8
175
176	#====================================================
177	#- Default number of MPI task for IPSL coupled model
178	#- required for backward compatibility
179	#-
180	DEFAULT_NUM_PROC_OCE=5
181	DEFAULT_NUM_PROC_CPL=1
182	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
183	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
184
185	#D-#==================================================
186	#D-function IGCM_sys_defineArchives
187	#D-* Purpose:
188	#D-* Define ARCHIVE : Dedicated to large files
189	#D-* Define STORAGE : Dedicated to small/medium files
190	#D-* Define R_OUT : Output tree located on ARCHIVE
191	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
192	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
193	#D-* if SpaceName=TEST everything is stored on SCRATCHDIR
194	#D-* Examples:
195	#D-
196	function IGCM_sys_defineArchives {
197	IGCM_debug_PushStack "IGCM_sys_defineArchives"
198
199	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
200	#====================================================
201	#- ARCHIVE (dedicated to large files)
202	ARCHIVE=${config_UserChoices_ARCHIVE}
203	else
204	#====================================================
205	#- ARCHIVE (dedicated to large files)
206	ARCHIVE=${CCCSTOREDIR}
207	fi
208
209	if [ ! X${config_UserChoices_STORAGE} = X ]; then
210	#====================================================
211	#- STORAGE (dedicated to small/medium files)
212	STORAGE=${config_UserChoices_STORAGE}
213	else
214	#====================================================
215	#- STORAGE (dedicated to small/medium files)
216	STORAGE=${CCCWORKDIR}
217	fi
218
219	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
220	#====================================================
221	#- R_OUT
222	R_OUT=${SCRATCHDIR}/IGCM_OUT
223
224	#====================================================
225	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
226	R_FIG=${SCRATCHDIR}/IGCM_OUT
227
228	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
229
230	else
231	#====================================================
232	#- R_OUT
233	R_OUT=${ARCHIVE}/IGCM_OUT
234
235	#====================================================
236	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
237	R_FIG=${STORAGE}/IGCM_OUT
238	fi
239
240	#====================================================
241	#- R_BUF (ONLY FOR double copy an scratch)
242	R_BUF=${SCRATCHDIR}/IGCM_OUT
243
244	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
245	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
246	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
247
248	IGCM_debug_PopStack "IGCM_sys_defineArchives"
249	}
250
251	#D-#==================================================
252	#D-function IGCM_sys_RshMaster
253	#D-* Purpose: Connection to frontend machine.
254	#D-* Examples:
255	#D-
256	function IGCM_sys_RshMaster {
257	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
258	/bin/ksh <<-EOF
259	export libIGCM=${libIGCM}
260	export DEBUG_debug=${DEBUG_debug}
261	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
262	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
263	${@}
264	EOF
265	if [ $? -gt 0 ] ; then
266	echo "IGCM_sys_RshMaster : erreur."
267	IGCM_debug_Exit "IGCM_sys_RshMaster"
268	fi
269	IGCM_debug_PopStack "IGCM_sys_RshMaster"
270	}
271
272	#D-#==================================================
273	#D-function IGCM_sys_RshArchive
274	#D-* Purpose: Archive rsh command
275	#D-* Examples:
276	#D-
277	function IGCM_sys_RshArchive {
278	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
279	/bin/ksh <<-EOF
280	${@}
281	EOF
282	status=$?
283	if [ ${status} -gt 0 ] ; then
284	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
285	IGCM_debug_Exit "IGCM_sys_RshArchive"
286	fi
287	IGCM_debug_PopStack "IGCM_sys_RshArchive"
288	}
289
290	#D-#==================================================
291	#D-function IGCM_sys_RshArchive_NoError
292	#D-* Purpose: Archive rsh command, without error
293	#D-* used only in monitoring.job
294	#D-* Examples:
295	#D-
296	function IGCM_sys_RshArchive_NoError {
297	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
298	/bin/ksh <<-EOF
299	${@} 2> \dev\null
300	EOF
301	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
302	}
303
304	#D-#==================================================
305	#D-function IGCM_sys_RshPost
306	#D-* Purpose: Post-process rsh command
307	#D-* Examples:
308	#D-
309	function IGCM_sys_RshPost {
310	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
311	if ( $DEBUG_sys ) ; then
312	echo "IGCM_sys_RshPost :" $@
313	fi
314	# keep standard input (stdin) for the loop onto temporary file
315	cat >/tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
316
317	/bin/ksh </tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
318	if [ $? -gt 0 ] ; then
319	echo "IGCM_sys_RshPost : erreur."
320	IGCM_debug_Exit "IGCM_sys_RshPost"
321	fi
322	# delete temporary file
323	\rm /tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
324
325	# ============ FRONTEND END ============ #
326
327	# ============ CESIUM START ============ #
328	# typeset NB_ESSAI DELAI status i
329	# if [ "X$( grep rebuild_from tmp_IGCM_sys_RshPost_$$ )" != "X" ] ; then
330	# #little hack so that rebuild submission is done on titane not an cesium
331	#
332	# libIGCM_POST_sed=$( echo $libIGCM_POST \| sed 's/\//\\\//g' )
333	# POST_DIR_sed=$( echo ${POST_DIR} \| sed 's/\//\\\//g' )
334	# sed "s/IGCM_sys_QsubPost/IGCM_sys_Qsub/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
335	# sed "s/ rebuild_fromWorkdir/ ${libIGCM_POST_sed}\/rebuild_fromWorkdir.job/g" tmp.txt > tmp_IGCM_sys_RshPost_$$
336	# sed "s/ rebuild_fromArchive/ ${libIGCM_POST_sed}\/rebuild_fromArchive.job/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
337	# sed "s/Script_Post_Output=/Script_Output=${POST_DIR_sed}\//g" tmp.txt > tmp_IGCM_sys_RshPost_$$
338	# \mv tmp.txt tmp_IGCM_sys_RshPost_$$
339	#
340	# echo cat tmp_IGCM_sys_RshPost_$$ AFTER
341	# cat tmp_IGCM_sys_RshPost_$$
342	#
343	# /bin/ksh <tmp_IGCM_sys_RshPost_$$
344	# if [ $? -gt 0 ] ; then
345	# echo "IGCM_sys_RshPost : erreur."
346	# IGCM_debug_Exit "IGCM_sys_RshPost"
347	# fi
348	# # delete temporary file
349	# \rm tmp_IGCM_sys_RshPost_$$
350	#
351	# else
352	# # number of tentative
353	# NB_ESSAI=10
354	# # time delay between tentative
355	# DELAI=10
356	# i=0
357	# while [ $i -ne $NB_ESSAI ] ; do
358	# ssh -t titane996 ssh cesium /bin/ksh <tmp_IGCM_sys_RshPost_$$
359	# status=$?
360	# if [ ${status} -ne 0 ];
361	# then
362	# sleep $DELAI
363	# else
364	# break
365	# fi
366	# let i=$i+1
367	# done
368	# # delete temporary file
369	# \rm tmp_IGCM_sys_RshPost_$$
370	#
371	# if [ ${status} -gt 0 ] ; then
372	# echo "IGCM_sys_RshPost : erreur."
373	# IGCM_debug_Exit "IGCM_sys_RshPost"
374	# fi
375	# fi
376
377	# ============ CESIUM END ============ #
378
379	IGCM_debug_PopStack "IGCM_sys_RshPost"
380	}
381
382	#D-#==================================================
383	#D-function IGCM_sys_SendMail
384	#D-* Purpose: Send mail when simulation is over
385	#D-* Examples:
386	#D-
387	function IGCM_sys_SendMail {
388	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
389
390	if ( $DEBUG_sys ) ; then
391	echo "IGCM_sys_SendMail :" $@
392	fi
393
394	if [ X${1} = XAccounting ] ; then
395	status=Accounting
396	mailText=jobAccounting.mail
397	elif ( ${ExitFlag} ) ; then
398	status=failed
399	mailText=jobEnd.mail
400	else
401	status=completed
402	mailText=jobEnd.mail
403	fi
404
405	# Update selected mail template
406	while read -r line; do
407	eval echo $line >> mail.txt ;
408	done < ${libIGCM}/libIGCM_sys/${mailText}
409
410	if [ ! -z ${config_UserChoices_MailName} ] ; then
411	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
412	elif [ -f ~/.forward ] ; then
413	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
414	fi
415
416	sleep 10
417	rm -f mail.txt
418
419	IGCM_debug_PopStack "IGCM_sys_SendMail"
420	}
421
422	#D-#==================================================
423	#D-function IGCM_sys_Mkdir
424	#D-* Purpose: Master locale mkdir command
425	#D-* Examples:
426	#D-
427	function IGCM_sys_Mkdir {
428	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
429	if ( $DEBUG_sys ) ; then
430	echo "IGCM_sys_Mkdir :" $@
431	fi
432	if [ ! -d ${1} ]; then
433	\mkdir -p $1
434	if [ $? -gt 0 ] ; then
435	echo "IGCM_sys_Mkdir : erreur."
436	IGCM_debug_Exit "IGCM_sys_Mkdir"
437	fi
438	fi
439	# vérification :
440	if [ ! -d ${1} ] ; then
441	echo "IGCM_sys_Mkdir : erreur."
442	IGCM_debug_Exit "IGCM_sys_Mkdir"
443	fi
444	IGCM_debug_PopStack "IGCM_sys_Mkdir"
445	}
446
447	#D-#==================================================
448	#D-function IGCM_sys_MkdirArchive
449	#D-* Purpose: Mkdir on Archive
450	#D-* Examples:
451	#D-
452	function IGCM_sys_MkdirArchive {
453	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
454	if ( $DEBUG_sys ) ; then
455	echo "IGCM_sys_MkdirArchive :" $@
456	fi
457	#- creation de repertoire sur le serveur fichier
458	if [ ! -d ${1} ]; then
459	\mkdir -p $1
460	status=$?
461
462	if [ ${status} -gt 0 ] ; then
463	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
464	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
465	fi
466	fi
467	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
468	}
469
470	#D-#==================================================
471	#D-function IGCM_sys_MkdirWork
472	#D-* Purpose: Mkdir on Work
473	#D-* Examples:
474	#D-
475	function IGCM_sys_MkdirWork {
476	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
477	if ( $DEBUG_sys ) ; then
478	echo "IGCM_sys_MkdirWork :" $@
479	fi
480	#- creation de repertoire sur le serveur fichier
481	if [ ! -d ${1} ]; then
482	\mkdir -p $1
483	if [ $? -gt 0 ] ; then
484	echo "IGCM_sys_MkdirWork : erreur."
485	IGCM_debug_Exit "IGCM_sys_MkdirWork"
486	fi
487	fi
488	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
489	}
490
491	#D-#==================================================
492	#D-function IGCM_sys_Cd
493	#D-* Purpose: master cd command
494	#D-* Examples:
495	#D-
496	function IGCM_sys_Cd {
497	IGCM_debug_PushStack "IGCM_sys_Cd" $@
498	if ( $DEBUG_sys ) ; then
499	echo "IGCM_sys_Cd :" $@
500	fi
501	\cd $1
502	if [ $? -gt 0 ] ; then
503	echo "IGCM_sys_Cd : erreur."
504	IGCM_debug_Exit "IGCM_sys_Cd"
505	fi
506	IGCM_debug_PopStack "IGCM_sys_Cd"
507	}
508
509	#D-#==================================================
510	#D-function IGCM_sys_Chmod
511	#D-* Purpose: Chmod
512	#D-* Examples:
513	#D-
514	function IGCM_sys_Chmod {
515	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
516	if ( $DEBUG_sys ) ; then
517	echo "IGCM_sys_Chmod :" $@
518	fi
519	\chmod $@
520	if [ $? -gt 0 ] ; then
521	echo "IGCM_sys_Chmod : erreur."
522	IGCM_debug_Exit "IGCM_sys_Chmod"
523	fi
524	IGCM_debug_PopStack "IGCM_sys_Chmod"
525	}
526
527	#D-#==================================================
528	#D-function IGCM_sys_FileSize
529	#D-* Purpose: Filesize
530	#D-* Examples:
531	#D-
532	function IGCM_sys_FileSize {
533	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
534
535	typeset sizeF
536	set +A sizeF -- $( ls -la ${1} )
537	if [ $? -gt 0 ] ; then
538	IGCM_debug_Exit "IGCM_sys_FileSize"
539	fi
540	eval ${2}=${sizeF[4]}
541
542	IGCM_debug_PopStack "IGCM_sys_FileSize"
543	}
544
545	#D-#==================================================
546	#D-function IGCM_sys_TestDir
547	#D-* Purpose: Test Directory that must exists
548	#D-* Examples:
549	#D-
550	function IGCM_sys_TestDir {
551	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
552	if ( $DEBUG_sys ) ; then
553	echo "IGCM_sys_TestDir :" $@
554	fi
555	typeset ExistFlag
556	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
557	IGCM_debug_PopStack "IGCM_sys_TestDir"
558
559	return ${ExistFlag}
560	}
561
562	#D-#==================================================
563	#D-function IGCM_sys_TestDirArchive
564	#D-* Purpose: Test Directory that must exists on Archive
565	#D-* Examples:
566	#D-
567	function IGCM_sys_TestDirArchive {
568	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
569	if ( $DEBUG_sys ) ; then
570	echo "IGCM_sys_TestDirArchive :" $@
571	fi
572	typeset ExistFlag
573	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
574	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
575	return ${ExistFlag}
576	}
577
578	#D-#==================================================
579	#D-function IGCM_sys_IsFileArchived
580	#D-* Purpose: Test file that must NOT EXISTS on Archive
581	#D-* Examples:
582	#D-
583	function IGCM_sys_IsFileArchived {
584	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
585	if ( $DEBUG_sys ) ; then
586	echo "IGCM_sys_IsFileArchived :" $@
587	fi
588	typeset IsArchivedFlag
589	IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
590	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
591
592	return ${IsArchivedFlag}
593	}
594
595	#D-#==================================================
596	#D-function IGCM_sys_TestFileArchive
597	#D-* Purpose: Test file that must NOT EXISTS on Archive
598	#D-* Examples:
599	#D-
600	function IGCM_sys_TestFileArchive {
601	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
602	typeset ExistFlag
603	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
604	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
605
606	return ${ExistFlag}
607	}
608
609	#D-#==================================================
610	#D-function IGCM_sys_TestFileBuffer
611	#D-* Purpose: Test file that must NOT EXISTS on Buffer
612	#D-* Examples:
613	#D-
614	function IGCM_sys_TestFileBuffer {
615	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
616	typeset ExistFlag
617	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
618	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
619
620	return ${ExistFlag}
621	}
622
623	#D-#==================================================
624	#D-function IGCM_sys_CountFileArchive
625	#D-* Purpose: Count files on Archive filesystem
626	#D-* Examples:
627	#D-
628	function IGCM_sys_CountFileArchive {
629	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
630	ls ${@} 2>/dev/null \| wc -l
631	if [ $? -gt 0 ] ; then
632	echo "IGCM_sys_CountFileArchive : erreur."
633	fi
634	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
635	}
636
637	#D-#==================================================
638	#D-function IGCM_sys_CountFileBuffer
639	#D-* Purpose: Count files on Scratch filesystem
640	#D-* Examples:
641	#D-
642	function IGCM_sys_CountFileBuffer {
643	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
644	ls ${@} 2>/dev/null \| wc -l
645	if [ $? -gt 0 ] ; then
646	echo "IGCM_sys_CountFileBuffer : erreur."
647	fi
648	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
649	}
650
651	#D-#==================================================
652	#D-function IGCM_sys_Tree
653	#D-* Purpose: Tree directories with files on ${ARCHIVE}
654	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
655	#D-
656	function IGCM_sys_Tree {
657	IGCM_debug_PushStack "IGCM_sys_Tree" $@
658	if ( $DEBUG_sys ) ; then
659	echo "IGCM_sys_Tree :" $@
660	fi
661
662	\ls -lR ${@}
663
664	IGCM_debug_PopStack "IGCM_sys_Tree"
665	}
666
667	#D-#==================================================
668	#D-function IGCM_sys_Tar
669	#D-* Purpose: master tar command
670	#D-* Examples:
671	#D-
672	function IGCM_sys_Tar {
673	IGCM_debug_PushStack "IGCM_sys_Tar" $@
674	if ( $DEBUG_sys ) ; then
675	echo "IGCM_sys_Tar :" $@
676	fi
677	\tar cf $@
678	if [ $? -gt 0 ] ; then
679	echo "IGCM_sys_Tar : erreur."
680	IGCM_debug_Exit "IGCM_sys_Tar"
681	fi
682	IGCM_debug_PopStack "IGCM_sys_Tar"
683	}
684
685	#D-#==================================================
686	#D-function IGCM_sys_UnTar
687	#D-* Purpose: master un-tar command
688	#D-* Examples:
689	#D-
690	function IGCM_sys_UnTar {
691	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
692	if ( $DEBUG_sys ) ; then
693	echo "IGCM_sys_UnTar :" $@
694	fi
695	\tar xvf $1
696	if [ $? -gt 0 ] ; then
697	echo "IGCM_sys_UnTar : erreur."
698	IGCM_debug_Exit "IGCM_sys_UnTar"
699	fi
700	IGCM_debug_PopStack "IGCM_sys_UnTar"
701	}
702
703	#D-#==================================================
704	#D-function IGCM_sys_Qsub
705	#D-* Purpose: Qsub new job
706	#D-* Examples:
707	#D-
708	function IGCM_sys_Qsub {
709	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
710	if ( $DEBUG_sys ) ; then
711	echo "IGCM_sys_Qsub :" $@
712	fi
713	typeset options status
714	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
715
716	/usr/bin/ccc_msub ${options} $1 > /tmp/out_command.$$ 2>&1
717	status=$?
718
719	cat /tmp/out_command.$$
720	if [ ${status} -gt 0 ] ; then
721	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
722	IGCM_debug_Exit "IGCM_sys_Qsub"
723	else
724	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
725	fi
726	IGCM_debug_PopStack "IGCM_sys_Qsub"
727	}
728
729	#D-#==================================================
730	#D-function IGCM_sys_QsubPost
731	#D-* Purpose: Qsub new job on scalaire
732	#D-* Examples:
733	#D-
734	function IGCM_sys_QsubPost {
735	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
736	if ( $DEBUG_sys ) ; then
737	echo "IGCM_sys_QsubPost :" $@
738	fi
739	typeset options status
740	MY_PROJECT=$(echo ${BRIDGE_MSUB_PROJECT:=${DEFAULT_PROJECT}} \| cut -d@ -f1 )
741
742	# EASIER TO DO THIS INSTEAD OF DUPLICATING libIGCM_sys_curie.ksh
743	case $( hostname -s ) in
744	curie*)
745	options="-Q normal -A ${MY_PROJECT} -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out";;
746	airain*)
747	options="-q ivybridge -A dsm -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
748	esac
749
750	/usr/bin/ccc_msub ${options} ${libIGCM_POST}/$1.job > /tmp/out_command.$$ 2>&1
751	status=$?
752
753	cat /tmp/out_command.$$
754	if [ ${status} -gt 0 ] ; then
755	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
756	IGCM_debug_Exit "IGCM_sys_QsubPost"
757	else
758	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
759	fi
760	IGCM_debug_PopStack "IGCM_sys_QsubPost"
761	}
762
763	#D-*************************
764	#D- File transfer functions
765	#D-*************************
766	#D-
767
768	#D-#==================================================
769	#D-function IGCM_sys_Rsync_out
770	#D-* Purpose: treat return val of rsync
771	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
772	#D- Error values and explanations can depend on your system version.
773	function IGCM_sys_Rsync_out {
774	status=$1
775	if [ ! $status ] ; then
776	echo "rsync error !"
777	fi
778
779	if [ $MYLANG = "fr" ]; then
780	case $status in
781	0) return ;;
782	1) echo "Erreur de rsync ; RERR_SYNTAX : "
783	echo "Erreur de syntaxe ou d'utilisation."
784	return;;
785	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
786	echo "Incompatibilité de protocole."
787	return;;
788	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
789	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
790	echo "répertoires"
791	return;;
792	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
793	echo "Action demandée non supportée : une tentative de manipulation de"
794	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
795	echo "été faite ; ou une option qui est supportée par le client mais"
796	echo "pas par le serveur a été spécifiée."
797	return;;
798	10) echo "Erreur de rsync ; RERR_SOCKETIO"
799	echo "Erreur dans le socket d'entrée sortie"
800	return;;
801	11) echo "Erreur de rsync ; RERR_FILEIO"
802	echo "Erreur d'entrée sortie fichier"
803	return;;
804	12) echo "Erreur de rsync ; RERR_STREAMIO"
805	echo "Erreur dans flux de donnée du protocole rsync"
806	return;;
807	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
808	echo "Erreur avec les diagnostics du programme"
809	return;;
810	14) echo "Erreur de rsync ; RERR_IPC"
811	echo "Erreur dans le code IPC"
812	return;;
813	20) echo "Erreur de rsync ; RERR_SIGNAL"
814	echo "SIGUSR1 ou SIGINT reçu"
815	return;;
816	21) echo "Erreur de rsync ; RERR_WAITCHILD"
817	echo "Une erreur retournée par waitpid()"
818	return;;
819	22) echo "Erreur de rsync ; RERR_MALLOC"
820	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
821	return;;
822	23) echo ""
823	echo "Erreur fichier inexistant"
824	return;;
825	30) echo "Erreur de rsync ; RERR_TIMEOUT"
826	echo "Temps d'attente écoulé dans l'envoi/réception de données"
827	return;;
828	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
829	return;;
830	esac
831	elif [ $MYLANG = "en" ] ; then
832	case $status in
833	0) return;;
834	1) echo "rsync error : Syntax or usage error "
835	return;;
836	2) echo "rsync error : Protocol incompatibility "
837	return;;
838	3) echo "rsync error : Errors selecting input/output files, dirs"
839	return;;
840	4) echo "rsync error : Requested action not supported: an attempt"
841	echo "was made to manipulate 64-bit files on a platform that cannot support"
842	echo "them; or an option was specified that is supported by the client and"
843	echo "not by the server."
844	return;;
845	5) echo "rsync error : Error starting client-server protocol"
846	return;;
847	10) echo "rsync error : Error in socket I/O "
848	return;;
849	11) echo "rsync error : Error in file I/O "
850	return;;
851	12) echo "rsync error : Error in rsync protocol data stream "
852	return;;
853	13) echo "rsync error : Errors with program diagnostics "
854	return;;
855	14) echo "rsync error : Error in IPC code "
856	return;;
857	20) echo "rsync error : Received SIGUSR1 or SIGINT "
858	return;;
859	21) echo "rsync error : Some error returned by waitpid() "
860	return;;
861	22) echo "rsync error : Error allocating core memory buffers "
862	return;;
863	23) echo "rsync error : Partial transfer due to error"
864	return;;
865	24) echo "rsync error : Partial transfer due to vanished source files"
866	return;;
867	30) echo "rsync error : Timeout in data send/receive "
868	return;;
869	*) echo "rsync error : return code of rsync unknown :" $status
870	return;;
871	esac
872	else
873	echo "unknown language $MYLANG."
874	return
875	fi
876	}
877
878	#D-#==================================================
879	#D-function IGCM_sys_Miror_libIGCM
880	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
881	#D-* Examples:
882	#D-
883	function IGCM_sys_Mirror_libIGCM {
884	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
885	if ( $DEBUG_sys ) ; then
886	echo "IGCM_sys_Mirror_libIGCM"
887	fi
888
889	typeset status
890
891	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
892
893	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
894	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
895	status=$?
896
897	if [ ${status} -gt 0 ] ; then
898	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
899	cat /tmp/out_command.$$
900	fi
901	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
902	}
903
904	#D-#==================================================
905	#D-function IGCM_sys_Cp
906	#D-* Purpose: generic cp
907	#D-* Examples:
908	#D-
909	function IGCM_sys_Cp {
910	IGCM_debug_PushStack "IGCM_sys_Cp" $@
911	if ( $DEBUG_sys ) ; then
912	echo "IGCM_sys_Cp :" $@
913	fi
914
915	typeset status
916
917	echo cp $@ > /tmp/out_command.$$ 2>&1
918	\cp $@ >> /tmp/out_command.$$ 2>&1
919	status=$?
920
921	if [ ${status} -gt 0 ] ; then
922	echo "IGCM_sys_Cp : error code ${status}"
923	cat /tmp/out_command.$$
924	IGCM_debug_Exit "IGCM_sys_Cp"
925	else
926	\rm /tmp/out_command.$$
927	fi
928	IGCM_debug_PopStack "IGCM_sys_Cp"
929	}
930
931	#D-#==================================================
932	#D-function IGCM_sys_Rm
933	#D-* Purpose: generic rm
934	#D-* Examples:
935	#D-
936	function IGCM_sys_Rm {
937	IGCM_debug_PushStack "IGCM_sys_Rm" $@
938	if ( $DEBUG_sys ) ; then
939	echo "IGCM_sys_Rm :" $@
940	fi
941
942	typeset status
943
944	echo rm $@ > /tmp/out_command.$$ 2>&1
945	\rm $@ >> /tmp/out_command.$$ 2>&1
946	status=$?
947
948	if [ ${status} -gt 0 ] ; then
949	echo "IGCM_sys_Rm : error code ${status}"
950	cat /tmp/out_command.$$
951	IGCM_debug_Exit "IGCM_sys_Rm"
952	else
953	\rm /tmp/out_command.$$
954	fi
955	IGCM_debug_PopStack "IGCM_sys_Rm"
956	}
957
958	#D-#==================================================
959	#D-function IGCM_sys_RmRunDir
960	#D-* Purpose: rm tmpdir (dummy function most of the time batch
961	#D- scheduler will do the job)
962	#D-* Examples:
963	#D-
964	function IGCM_sys_RmRunDir {
965	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
966	if ( $DEBUG_sys ) ; then
967	echo "IGCM_sys_RmRunDir :" $@
968	fi
969
970	typeset status
971
972	echo rm $@ > /tmp/out_command.$$ 2>&1
973	\rm $@ >> /tmp/out_command.$$ 2>&1
974	status=$?
975
976	if [ ${status} -gt 0 ] ; then
977	echo "IGCM_sys_RmRunDir : rm error code is ${status}."
978	cat /tmp/out_command.$$
979	IGCM_debug_Exit "IGCM_sys_RmRunDir"
980	else
981	\rm /tmp/out_command.$$
982	fi
983	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
984	}
985
986	#D-#==================================================
987	#D-function IGCM_sys_Mv
988	#D-* Purpose: generic move
989	#D-* Examples:
990	#D-
991	function IGCM_sys_Mv {
992	IGCM_debug_PushStack "IGCM_sys_Mv" $@
993	if ( $DEBUG_sys ) ; then
994	echo "IGCM_sys_Mv :" $@
995	fi
996
997	if [ $DRYRUN = 0 ]; then
998
999	typeset status
1000
1001	echo mv $@ > /tmp/out_command.$$ 2>&1
1002	\mv $@ >> /tmp/out_command.$$ 2>&1
1003	status=$?
1004
1005	if [ ${status} -gt 0 ] ; then
1006	echo "IGCM_sys_Mv : error code ${status}"
1007	cat /tmp/out_command.$$
1008	IGCM_debug_Exit "IGCM_sys_Mv"
1009	else
1010	\rm /tmp/out_command.$$
1011	fi
1012	fi
1013
1014	IGCM_debug_PopStack "IGCM_sys_Mv"
1015	}
1016
1017	#D-#==================================================
1018	#D-function IGCM_sys_Put_Dir
1019	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1020	#D-* Examples:
1021	#D-
1022	function IGCM_sys_Put_Dir {
1023	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1024	if ( $DEBUG_sys ) ; then
1025	echo "IGCM_sys_Put_Dir :" $@
1026	fi
1027	if [ $DRYRUN = 0 ]; then
1028	if [ ! -d ${1} ] ; then
1029	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1030	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1031	return
1032	fi
1033
1034	typeset status
1035
1036	# Only if we use rsync
1037	#IGCM_sys_TestDirArchive $( dirname $2 )
1038	#
1039	#USUAL WAY
1040	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
1041	status=$?
1042
1043	if [ ${status} -gt 0 ] ; then
1044	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
1045	cat /tmp/out_command.$$
1046	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1047	else
1048	\rm /tmp/out_command.$$
1049	fi
1050	fi
1051	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1052	}
1053
1054	#D-#==================================================
1055	#D-function IGCM_sys_Get_Dir
1056	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1057	#D-* Examples:
1058	#D-
1059	function IGCM_sys_Get_Dir {
1060	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1061	if ( $DEBUG_sys ) ; then
1062	echo "IGCM_sys_Get_Dir :" $@
1063	fi
1064	if [ $DRYRUN = 0 ]; then
1065	typeset NB_ESSAI DELAI status i
1066	# number of tentative
1067	NB_ESSAI=3
1068	# time delay between tentative
1069	DELAI=2
1070
1071	# Only if we use rsync
1072	#IGCM_sys_TestDirArchive $( dirname $2 )
1073	#
1074	# USUAL WAY
1075	# add 'ccc_hsm get' (to demigrate all offline files) to reduce time of this command :
1076	ccc_hsm get -r $1
1077
1078	i=0
1079	while [ $i -lt $NB_ESSAI ] ; do
1080	\cp -ur $1 $2 >> /tmp/out_command.$$ 2>&1
1081	status=$?
1082	if [ ${status} -gt 0 ] ; then
1083	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
1084	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1085	sleep $DELAI
1086	else
1087	break
1088	fi
1089	(( i = i + 1 ))
1090	done
1091
1092	if [ ${status} -gt 0 ] ; then
1093	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
1094	cat /tmp/out_command.$$
1095	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1096	else
1097	\rm /tmp/out_command.$$
1098	fi
1099	fi
1100	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1101	}
1102
1103	#D-#==================================================
1104	#D-function IGCM_sys_Get_Master
1105	#D-* Purpose: Copy a complete directory from MASTER filesystem
1106	#D-* Examples:
1107	#D-
1108	function IGCM_sys_Get_Master {
1109	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1110	if ( $DEBUG_sys ) ; then
1111	echo "IGCM_sys_Get_Master :" $@
1112	fi
1113	if [ $DRYRUN = 0 ]; then
1114	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1115	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1116	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1117	return
1118	fi
1119
1120	typeset NB_ESSAI DELAI status i
1121	# number of tentative
1122	NB_ESSAI=3
1123	# time delay between tentative
1124	DELAI=2
1125
1126	i=0
1127	while [ $i -lt $NB_ESSAI ] ; do
1128	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
1129	status=$?
1130	if [ ${status} -gt 0 ]; then
1131	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1132	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1133	sleep $DELAI
1134	else
1135	break
1136	fi
1137	(( i = i + 1 ))
1138	done
1139
1140	if [ ${status} -gt 0 ] ; then
1141	echo "IGCM_sys_Get_Master : error."
1142	cat /tmp/out_command.$$
1143	IGCM_debug_Exit "IGCM_sys_Get_Master"
1144	else
1145	\rm /tmp/out_command.$$
1146	fi
1147	fi
1148	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1149	}
1150
1151	#====================================================
1152	#- Call IGCM_sys_Mirror_libIGCM now !
1153	if ( $MirrorlibIGCM ) ; then
1154	IGCM_sys_Mirror_libIGCM
1155	fi
1156
1157	#D-#==================================================
1158	#D-function IGCM_sys_Put_Rest
1159	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1160	#D- File and target directory must exist.
1161	#D-* Examples:
1162	#D-
1163	function IGCM_sys_Put_Rest {
1164	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1165	if ( $DEBUG_sys ) ; then
1166	echo "IGCM_sys_Put_Rest :" $@
1167	fi
1168	if [ $DRYRUN = 0 ]; then
1169	if [ ! -f ${1} ] ; then
1170	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1171	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1172	fi
1173
1174	typeset status
1175	#
1176	# USUAL WAY
1177	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1178	status=$?
1179
1180	# #RSYNC WITH NETWORK SSH CALL
1181	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1182	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1183
1184	# #RSYNC WITH NFS USE
1185	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1186	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1187
1188	# status=$?
1189	# IGCM_sys_Rsync_out $status
1190
1191	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1192	# (( status=status+$? ))
1193
1194	if [ ${status} -gt 0 ] ; then
1195	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
1196	[ -f ${1} ] && ls -l ${1}
1197	[ -f ${2} ] && ls -l ${2}
1198	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1199	cat /tmp/out_command.$$
1200	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1201	else
1202
1203	if [ X${JobType} = XRUN ] ; then
1204	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1205	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1206	fi
1207
1208	\rm /tmp/out_command.$$
1209	fi
1210	fi
1211	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1212	}
1213
1214	#D-#==================================================
1215	#D-function IGCM_sys_PutBuffer_Rest
1216	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1217	#D- File and target directory must exist.
1218	#D-* Examples:
1219	#D-
1220	function IGCM_sys_PutBuffer_Rest {
1221	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1222	if ( $DEBUG_sys ) ; then
1223	echo "IGCM_sys_PutBuffer_Rest :" $@
1224	fi
1225	if [ $DRYRUN = 0 ]; then
1226	if [ ! -f ${1} ] ; then
1227	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1228	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1229	fi
1230
1231	typeset status
1232	#
1233	# USUAL WAY
1234	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1235	status=$?
1236
1237	if [ ${status} -gt 0 ] ; then
1238	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1239	[ -f ${2} ] && ls -l ${2}
1240	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1241	cat /tmp/out_command.$$
1242	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1243	else
1244
1245	if [ X${JobType} = XRUN ] ; then
1246	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1247	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1248	fi
1249
1250	\rm /tmp/out_command.$$
1251	fi
1252	fi
1253	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1254	}
1255
1256	#D-#==================================================
1257	#D-function IGCM_sys_PrepareTaredRestart
1258	#D-* Purpose: Prepare tared restart to be access by computing job.
1259	#D-* Examples:
1260	#D-
1261	function IGCM_sys_PrepareTaredRestart {
1262	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1263	if [ $DRYRUN = 0 ]; then
1264	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1265	fi
1266	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1267	}
1268
1269	#D-#==================================================
1270	#D-function IGCM_sys_Put_Out
1271	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1272	#D-* Examples:
1273	#D-
1274	function IGCM_sys_Put_Out {
1275	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1276	if ( $DEBUG_sys ) ; then
1277	echo "IGCM_sys_Put_Out :" $@
1278	fi
1279
1280	typeset NB_ESSAI DELAI status i exist skip
1281
1282	# number of tentative
1283	NB_ESSAI=3
1284	# time delay between tentative
1285	DELAI=2
1286
1287	if [ $DRYRUN = 0 ]; then
1288	if [ ! -f ${1} ] ; then
1289	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1290	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1291	return 1
1292	fi
1293	#
1294	IGCM_sys_MkdirArchive $( dirname $2 )
1295	#
1296	exist=false
1297	skip=false
1298	if [ -f $2 ] ; then
1299	IGCM_debug_Print 1 "$2 already exist"
1300	ccc_hsm get $2
1301	exist=true
1302	if [ "X$( diff $1 $2 )" = X ] ; then
1303	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1304	skip=true
1305	else
1306	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1307	skip=false
1308	fi
1309	fi
1310	#
1311	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1312	IGCM_sys_Chmod u+w $2
1313	fi
1314
1315	if [ X${skip} = Xfalse ] ; then
1316	i=0
1317	while [ $i -lt $NB_ESSAI ] ; do
1318	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1319	# USUAL WAY
1320	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1321	status=$?
1322	else
1323	# NOT SO USUAL WAY
1324	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1325	status=$?
1326	fi
1327	if [ ${status} -gt 0 ]; then
1328	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1329	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
1330	[ -f ${1} ] && ls -l ${1}
1331	[ -f ${2} ] && ls -l ${2}
1332	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1333	sleep $DELAI
1334	else
1335	break
1336	fi
1337	(( i = i + 1 ))
1338	done
1339	fi
1340
1341	# #RSYNC WITH NETWORK SSH CALL
1342	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1343	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1344
1345	# #RSYNC WITH NFS USE
1346	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1347	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1348
1349	# status=$?
1350	# IGCM_sys_Rsync_out $status
1351
1352	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1353	# (( status=status+$? ))
1354
1355	if [ ${status} -gt 0 ] ; then
1356	echo "IGCM_sys_Put_Out : error."
1357	[ -f ${1} ] && ls -l ${1}
1358	[ -f ${2} ] && ls -l ${2}
1359	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1360	cat /tmp/out_command.$$
1361	IGCM_debug_Exit "IGCM_sys_Put_Out"
1362	else
1363
1364	if [ X${JobType} = XRUN ] ; then
1365	if [ X${3} = X ] ; then
1366	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1367	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1368	fi
1369	fi
1370
1371	\rm /tmp/out_command.$$
1372	fi
1373	fi
1374	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1375	return 0
1376	}
1377
1378	#D-#==================================================
1379	#D-function IGCM_sys_PutBuffer_Out
1380	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1381	#D-* Examples:
1382	#D-
1383	function IGCM_sys_PutBuffer_Out {
1384	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1385	if ( $DEBUG_sys ) ; then
1386	echo "IGCM_sys_PutBuffer_Out :" $@
1387	fi
1388
1389	typeset NB_ESSAI DELAI status i exist skip
1390
1391	# number of tentative
1392	NB_ESSAI=3
1393	# time delay between tentative
1394	DELAI=2
1395
1396	if [ $DRYRUN = 0 ]; then
1397	if [ ! -f ${1} ] ; then
1398	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1399	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1400	return 1
1401	fi
1402	#
1403	IGCM_sys_Mkdir $( dirname $2 )
1404	#
1405
1406	exist=false
1407	skip=false
1408	if [ -f $2 ] ; then
1409	IGCM_debug_Print 1 "$2 already exist"
1410	exist=true
1411	if [ "X$( diff $1 $2 )" = X ] ; then
1412	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1413	status=0
1414	skip=true
1415	else
1416	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1417	skip=false
1418	fi
1419	fi
1420	#
1421	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1422	IGCM_sys_Chmod u+w $2
1423	fi
1424
1425	if [ X${skip} = Xfalse ] ; then
1426	i=0
1427	while [ $i -lt $NB_ESSAI ] ; do
1428	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1429	# USUAL WAY
1430	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1431	status=$?
1432	else
1433	# NOT SO USUAL WAY
1434	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1435	status=$?
1436	fi
1437	if [ ${status} -gt 0 ]; then
1438	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1439	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1440	[ -f ${2} ] && ls -l ${2}
1441	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1442	sleep $DELAI
1443	else
1444	break
1445	fi
1446	(( i = i + 1 ))
1447	done
1448	fi
1449
1450	if [ ${status} -gt 0 ] ; then
1451	echo "IGCM_sys_PutBuffer_Out : error."
1452	[ -f ${2} ] && ls -l ${2}
1453	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1454	cat /tmp/out_command.$$
1455	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1456	else
1457
1458	if [ X${JobType} = XRUN ] ; then
1459	if [ X${3} = X ] ; then
1460	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1461	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1462	fi
1463	fi
1464
1465	\rm /tmp/out_command.$$
1466	fi
1467	fi
1468	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1469	return 0
1470	}
1471
1472	#D-#==================================================
1473	#D-function IGCM_sys_Get
1474	#D-* Purpose: Get a file from ${ARCHIVE}
1475	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1476	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1477	function IGCM_sys_Get {
1478	IGCM_debug_PushStack "IGCM_sys_Get" $@
1479
1480	typeset DEST dm_liste target file_work
1481	typeset NB_ESSAI DELAI status i
1482
1483	if ( $DEBUG_sys ) ; then
1484	echo "IGCM_sys_Get :" $@
1485	fi
1486
1487	# number of tentative
1488	NB_ESSAI=3
1489	# time delay between tentative
1490	DELAI=2
1491
1492	if [ $DRYRUN -le 2 ]; then
1493	if [ X${1} = X'/l' ] ; then
1494	eval set +A dm_liste \${${2}}
1495	else
1496	eval set +A dm_liste ${1}
1497	fi
1498	eval DEST=\${${#}}
1499	ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1500	status=$?
1501	if [ ${status} -gt 0 ] ; then
1502	echo "WARNING IGCM_sys_Get : error code ${status}"
1503	cat /tmp/out_command.$$
1504	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
1505	fi
1506
1507	#if [ ${status} -gt 0 ] ; then
1508	# if [ ! "X$( grep "Lost dmusrcmd connection" /tmp/out_command.$$ )" = "X" ] ; then
1509	# cat /tmp/out_command.$$
1510	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
1511	# sleep 30
1512	# echo "We try another time"
1513	## dmget ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1514	# ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1515	# status=$?
1516	# if [ ${status} -gt 0 ] ; then
1517	# echo "ERROR IGCM_sys_Get : again demigration error :"
1518	# cat /tmp/out_command.$$
1519	# IGCM_debug_Exit "IGCM_sys_Get"
1520	# fi
1521	# else
1522	# echo "ERROR IGCM_sys_Get : demigration error :"
1523	# cat /tmp/out_command.$$
1524	# IGCM_debug_Exit "IGCM_sys_Get"
1525	# fi
1526	#fi
1527
1528	# #RSYNC WITH NETWORK SSH CALL
1529	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1530	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1531
1532	# #RSYNC WITH NFS USE
1533	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1534	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1535
1536	# status=$?
1537	# IGCM_sys_Rsync_out $status
1538
1539	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1540	# (( status=status+$? ))
1541
1542	#USUAL WAY
1543	if [ X${1} = X'/l' ] ; then
1544	for target in ${dm_liste[*]} ; do
1545	local_file=$( basename ${target} )
1546	# test if the target file is present before the loop
1547	IGCM_sys_TestFileArchive ${target}
1548	status=$?
1549	if [ ${status} -gt 0 ] ; then
1550	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
1551	IGCM_debug_Exit "IGCM_sys_Get"
1552	else
1553	i=0
1554	while [ $i -lt $NB_ESSAI ] ; do
1555	#if [ X${DoLink} = Xtrue ] ; then
1556	# \ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1557	# status=$?
1558	# else
1559	# \cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1560	# status=$?
1561	#fi
1562	\ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1563	status=$?
1564	if [ ${status} -gt 0 ]; then
1565	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1566	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1567	sleep $DELAI
1568	else
1569	break
1570	fi
1571	(( i = i + 1 ))
1572	done
1573	if [ ${status} -gt 0 ] ; then
1574	echo "IGCM_sys_Get : error"
1575	cat /tmp/out_command.$$
1576	\rm /tmp/out_command.$$
1577	IGCM_debug_Exit "IGCM_sys_Get"
1578	else
1579	\rm /tmp/out_command.$$
1580	fi
1581	fi
1582	done
1583	else
1584	i=0
1585	while [ $i -lt $NB_ESSAI ] ; do
1586	\cp ${dm_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1587	status=$?
1588	if [ ${status} -gt 0 ]; then
1589	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1590	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1591	sleep $DELAI
1592	else
1593	break
1594	fi
1595	(( i = i + 1 ))
1596	done
1597	if [ ${status} -gt 0 ] ; then
1598	echo "IGCM_sys_Get : error"
1599	cat /tmp/out_command.$$
1600	\rm /tmp/out_command.$$
1601	IGCM_debug_Exit "IGCM_sys_Get"
1602	else
1603	\rm /tmp/out_command.$$
1604	fi
1605	fi
1606	fi
1607	IGCM_debug_PopStack "IGCM_sys_Get"
1608	}
1609
1610	#D-#==================================================
1611	#D-function IGCM_sys_GetBuffer
1612	#D-* Purpose: Get a file from ${SCRATCHDIR}
1613	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1614	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1615	function IGCM_sys_GetBuffer {
1616	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1617
1618	typeset DEST buf_liste target file_work
1619	typeset NB_ESSAI DELAI status i
1620
1621	if ( $DEBUG_sys ) ; then
1622	echo "IGCM_sys_GetBuffer :" $@
1623	fi
1624
1625	# number of tentative
1626	NB_ESSAI=3
1627	# time delay between tentative
1628	DELAI=2
1629
1630	if [ $DRYRUN -le 2 ]; then
1631	if [ X${1} = X'/l' ] ; then
1632	eval set +A buf_liste \${${2}}
1633	else
1634	eval set +A buf_liste ${1}
1635	fi
1636	eval DEST=\${${#}}
1637
1638	#USUAL WAY
1639	if [ X${1} = X'/l' ] ; then
1640	for target in ${buf_liste[*]} ; do
1641	local_file=$( basename ${target} )
1642	i=0
1643	while [ $i -lt $NB_ESSAI ] ; do
1644	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1645	status=$?
1646	if [ ${status} -gt 0 ]; then
1647	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1648	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1649	sleep $DELAI
1650	else
1651	break
1652	fi
1653	(( i = i + 1 ))
1654	done
1655	if [ ${status} -gt 0 ] ; then
1656	echo "IGCM_sys_Get : error"
1657	cat /tmp/out_command.$$
1658	\rm /tmp/out_command.$$
1659	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1660	else
1661	\rm /tmp/out_command.$$
1662	fi
1663	done
1664	else
1665	i=0
1666	while [ $i -lt $NB_ESSAI ] ; do
1667	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1668	status=$?
1669	if [ ${status} -gt 0 ]; then
1670	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1671	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1672	sleep $DELAI
1673	else
1674	break
1675	fi
1676	(( i = i + 1 ))
1677	done
1678	if [ ${status} -gt 0 ] ; then
1679	echo "IGCM_sys_Get : error"
1680	cat /tmp/out_command.$$
1681	\rm /tmp/out_command.$$
1682	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1683	else
1684	\rm /tmp/out_command.$$
1685	fi
1686	fi
1687	fi
1688	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1689	}
1690
1691	#D-#==================================================
1692	#D-function IGCM_sys_GetDate_FichWork
1693	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1694	#D-* Examples:
1695	#D-
1696	function IGCM_sys_GetDate_FichWork {
1697	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1698	if ( $DEBUG_sys ) ; then
1699	echo "IGCM_sys_GetDate_FichWork :" $@
1700	fi
1701
1702	if [ $# -ge 3 ] ; then
1703	mode=$3
1704	TimeStyle=$4
1705	else
1706	mode="default"
1707	TimeStyle="%Y%m%d%H%M%S"
1708	fi
1709
1710	typeset dateF
1711	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1712
1713	case $mode in
1714	"default")
1715	eval ${2}=${dateF[5]}
1716	;;
1717	"SplitFields")
1718	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1719	;;
1720	esac
1721
1722
1723	# donne la date filesys d'un fichier sur la machine work
1724	IGCM_debug_PopStack "IGCM_sys_FichWork"
1725	}
1726
1727	#D-#==================================================
1728	#D-function IGCM_sys_GetDate_FichArchive
1729	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1730	#D-* Examples:
1731	#D-
1732	function IGCM_sys_GetDate_FichArchive {
1733	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1734	if ( $DEBUG_sys ) ; then
1735	echo "IGCM_sys_GetDate_FichArchive :" $@
1736	fi
1737	typeset dateF
1738	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1739	eval ${2}=${dateF[5]}
1740
1741	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1742	}
1743
1744	#D-#==================================================
1745	#D-function IGCM_sys_GetDate_Monitoring
1746	#D-* Purpose: get the last year for which the monitoring has been computed
1747	#D-* Examples:
1748	#D-
1749	function IGCM_sys_GetDate_Monitoring {
1750	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1751	if ( $DEBUG_sys ) ; then
1752	echo "IGCM_sys_GetDate_Monitoring :" $@
1753	fi
1754
1755	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1756
1757	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1758	}
1759
1760	#D-#==================================================
1761	#D-function IGCM_sys_Dods_Rm
1762	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1763	#D-* Examples:
1764	#D-
1765	function IGCM_sys_Dods_Rm {
1766	if ( $DEBUG_sys ) ; then
1767	echo "IGCM_sys_Dods_Rm :" $@
1768	fi
1769	typeset status
1770	if [ $DRYRUN = 0 ]; then
1771
1772	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
1773	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
1774	# echo "Nothing has been done."
1775	# return
1776	# fi
1777
1778	/ccc/cont003/home/dsm/p86ipsl/bin/dods_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
1779	status=$?
1780
1781	# if [ ${status} -gt 0 ] ; then
1782	# echo "IGCM_sys_Dods_Rm : error."
1783	# cat out_dods_rm
1784	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
1785	# else
1786	# rm out_dods_rm
1787	# fi
1788
1789	fi
1790	return $status
1791	}
1792
1793	#D-#==================================================
1794	#D-function IGCM_sys_Dods_Cp
1795	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1796	#D-* Examples:
1797	#D-
1798	function IGCM_sys_Dods_Cp {
1799	if ( $DEBUG_sys ) ; then
1800	echo "IGCM_sys_Dods_Cp :" $@
1801	fi
1802	typeset status
1803	if [ $DRYRUN = 0 ]; then
1804
1805	# if [ ! -d ${R_SAVE}/${1} ] ; then
1806	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
1807	# echo "Nothing has been done."
1808	# return
1809	# fi
1810
1811	/ccc/cont003/home/dsm/p86ipsl/bin/dods_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
1812	status=$?
1813
1814	# if [ ${status} -gt 0 ] ; then
1815	# echo "IGCM_sys_Dods_Cp : error."
1816	# cat out_dods_cp
1817	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
1818	# else
1819	# rm out_dods_cp
1820	# fi
1821
1822	fi
1823	return $status
1824	}
1825
1826	#D-#==================================================
1827	#D-function IGCM_sys_Put_Dods
1828	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole.
1829	#D-* Examples:
1830	#D-
1831	function IGCM_sys_Put_Dods {
1832	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1833	if ( $DEBUG_sys ) ; then
1834	echo "IGCM_sys_Put_Dods :" $@
1835	fi
1836	#set -vx
1837	typeset status
1838	if [ $DRYRUN = 0 ]; then
1839	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
1840	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
1841	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
1842	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
1843	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1844	return
1845	fi
1846
1847	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1848	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1849	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1850	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1851	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1852	return
1853	fi
1854	#
1855	if [ -d ${R_SAVE}/${1} ] ; then
1856	cd ${R_SAVE}
1857	elif [ -d ${R_FIGR}/${1} ] ; then
1858	cd ${R_FIGR}
1859	fi
1860
1861	IGCM_sys_Dods_Rm ${1}
1862	IGCM_sys_Dods_Cp ${1}
1863	status=0
1864
1865	if [ ${status} -gt 0 ] ; then
1866	echo "IGCM_sys_Put_Dods : error."
1867	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1868	fi
1869	fi
1870	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1871	}
1872
1873	##############################################################
1874	# REBUILD OPERATOR
1875
1876	#D-#==================================================
1877	#D-function IGCM_sys_sync
1878	#D-* Purpose: flush buffer on disk
1879	#D-* Examples:
1880	#D-
1881	function IGCM_sys_sync {
1882	IGCM_debug_PushStack "IGCM_sys_sync" $@
1883	if ( $DEBUG_sys ) ; then
1884	echo "IGCM_sys_sync :" $@
1885	fi
1886
1887	/bin/sync
1888
1889	IGCM_debug_PopStack "IGCM_sys_sync"
1890	}
1891
1892	#D-#==================================================
1893	#D-function IGCM_sys_rebuild
1894	#D-* Purpose: rebuild parallel files
1895	#D-* Examples:
1896	#D-
1897	function IGCM_sys_rebuild {
1898	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1899	if ( $DEBUG_sys ) ; then
1900	echo "IGCM_sys_rebuild :" $@
1901	fi
1902
1903	typeset NB_ESSAI DELAI status i firstArg
1904	# number of tentative
1905	NB_ESSAI=3
1906	# time delay between tentative
1907	DELAI=2
1908
1909	i=0
1910	while [ $i -lt $NB_ESSAI ] ; do
1911	~p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1912	status=$?
1913	if [ ${status} -gt 0 ] ; then
1914	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1915	cat /tmp/out_command.$$
1916	\rm /tmp/out_command.$$
1917	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1918	firstArg=${1}
1919	\rm ${firstArg}
1920	sleep $DELAI
1921	else
1922	\rm /tmp/out_command.$$
1923	break
1924	fi
1925	(( i = i + 1 ))
1926	done
1927
1928	if [ ${status} -gt 0 ] ; then
1929	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1930	IGCM_debug_Exit "rebuild"
1931	fi
1932
1933	IGCM_debug_PopStack "IGCM_sys_rebuild"
1934	}
1935
1936	#D-#==================================================
1937	#D-function IGCM_sys_rebuild_station
1938	#D-* Purpose: rebuild parallel files describing station
1939	#D-* Examples:
1940	#D-
1941	function IGCM_sys_rebuild_station {
1942	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1943	typeset i list_opt file_in file_out prefix_invert list_invert
1944	if ( $DEBUG_sys ) ; then
1945	echo "IGCM_sys_rebuild_station :" $@
1946	fi
1947	list_opt=$@
1948
1949	# Invert Axis : t,x -> x,t
1950	# t,pres,x -> x,t,pres
1951	# So that we can concatenate along x
1952	i=0
1953	for file_in in ${list_opt} ; do
1954	(( i = i + 1))
1955	[ ${i} = 1 ] && file_out=${file_in} && continue
1956	# detect time counter and do the job only if present
1957	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
1958	if [ X${var_unlim} = Xtime_counter ] ; then
1959	prefix_invert=$( basename ${file_in} .nc )
1960	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1961	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1962	fi
1963	done
1964
1965	# Concatenate
1966	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1967
1968	# Re-ivert file
1969	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1970
1971	# Station re-ordering is too expansive to be run within libIGCM
1972	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
1973	# This re-ordering must be done "in memory" by the cmorization process
1974	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
1975	# BEGIN reordering
1976
1977	# Only LMDZ text output contains the exact ordering of the station.
1978	# We isolate this in the code below:
1979	# 0 38 -157.5000000000000 70.98591549295774
1980	# 0 54 27.49999999999999 67.18309859154928
1981	# 0 56 -62.50000000000001 82.39436619718309
1982	# 0 79 12.49999999999999 78.59154929577466
1983	# 0 116 -165.0000000000000 76.05633802816901
1984	# 0 117 130.0000000000000 70.98591549295774
1985	# 0 118 110.0000000000000 87.46478873239437
1986	# 1 40 4.999999999999995 51.97183098591550
1987
1988	# typeset iStation iProc list_opt file_in file_out prefix_invert
1989	# typeset -Z4 j4
1990	# typeset -Z3 j3
1991
1992	# unset list_opt
1993	# set +A list_opt $@
1994
1995	# Filename after rebuild
1996	# file_out=${list_opt[0]}
1997	# Prefix of output files
1998	# prefix_invert=$( basename ${file_out} .nc )
1999	# Number of procs
2000	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
2001
2002	# iProc=0
2003	# while [ ${iProc} -lt ${num_proc} ] ; do
2004	# Array containing Station as a number
2005	# unset proc_stn
2006	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
2007	# Number of stations produced by processor proc
2008	# stationLast=${#proc_stn[*]}
2009	# Proc number on 4 digits
2010	# j4=${iProc}
2011	# Init
2012	# iStation=0
2013	# while [ ${iStation} -lt ${stationLast} ] ; do
2014	# Station number on 3 digits
2015	# j3=${proc_stn[${iStation}]}
2016	# Extract station
2017	# Invert Axis : t,x -> x,t
2018	# t,pres,x -> x,t,pres
2019	# So that we can concatenate along x
2020	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
2021	# (( iStation = iStation + 1 ))
2022	# done
2023	# (( iProc = iProc + 1 ))
2024	# done
2025
2026	# Concatenate all station along x
2027	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
2028
2029	# Re-invert file
2030	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
2031
2032	# END reordering
2033
2034	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
2035	}
2036
2037	############################################################
2038	# Activate Running Environnment Variables
2039
2040	#D-#==================================================
2041	#D-function IGCM_sys_desactiv_variables
2042	#D-* Purpose: set environement variables prior to execution
2043	#D-* Examples:
2044	#D-
2045	function IGCM_sys_activ_variables {
2046	IGCM_debug_PushStack "IGCM_sys_activ_variables"
2047	if ( $DEBUG_sys ) ; then
2048	echo "IGCM_sys_activ_variables"
2049	fi
2050
2051	# --------------------------------------------------------------------
2052	#D- MPI specifications
2053	# --------------------------------------------------------------------
2054	export OMPI_MCA_paffinity_hwloc_priority=41
2055	# --------------------------------------------------------------------
2056	#D- Other specifications
2057	# --------------------------------------------------------------------
2058
2059	ulimit -s unlimited
2060
2061	IGCM_debug_PopStack "IGCM_sys_activ_variables"
2062	}
2063
2064	############################################################
2065	# Desactivate Running Environnment Variables
2066
2067	#D-#==================================================
2068	#D-function IGCM_sys_desactiv_variables
2069	#D-* Purpose: unset environement variables after execution
2070	#D-* Examples:
2071	#D-
2072	function IGCM_sys_desactiv_variables {
2073	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
2074	if ( $DEBUG_sys ) ; then
2075	echo "IGCM_sys_desactiv_variables"
2076	fi
2077	# --------------------------------------------------------------------
2078	#D- MPI specifications
2079	# --------------------------------------------------------------------
2080
2081	# --------------------------------------------------------------------
2082	#D- Other specifications
2083	# --------------------------------------------------------------------
2084
2085	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
2086	}
2087
2088	############################################################
2089	# Build MPI/OMP scripts run file (dummy function)
2090
2091	#D-#==================================================
2092	#D-function IGCM_sys_build_run_file
2093	#D-* Purpose: build run file (deprecated)
2094	#D-* Examples:
2095	#D-
2096	function IGCM_sys_build_run_file {
2097
2098	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
2099
2100	}
2101
2102	############################################################
2103	# Build MPI/OMP scripts
2104
2105	#D-#==================================================
2106	#D-function IGCM_sys_build_execution_scripts
2107	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
2108	#D-* Examples:
2109	#D-
2110	function IGCM_sys_build_execution_scripts
2111	{
2112	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
2113	if ( $DEBUG_sys ) ; then
2114	echo "IGCM_sys_build_execution_scripts " $@
2115	fi
2116
2117	EXECUTION=${HOST_MPIRUN_COMMAND}
2118
2119	# MPMD mode
2120	if ( ${OK_PARA_MPMD} ) ; then
2121
2122	# Only MPI (MPMD)
2123	if ( ! ${OK_PARA_OMP} ) ; then
2124
2125	if [ -f run_file ] ; then
2126	IGCM_sys_Rm -f run_file
2127	fi
2128	touch run_file
2129
2130	# Build run_file
2131
2132	# First loop on the components for the coupler ie oasis (only if oasis3)
2133	# the coupler ie oasis3 must be the first one
2134	for comp in ${config_ListOfComponents[*]} ; do
2135
2136	eval ExeNameIn=\${config_Executable_${comp}[0]}
2137	eval ExeNameOut=\${config_Executable_${comp}[1]}
2138
2139	# for CPL component only
2140	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
2141
2142	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2143	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2144	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2145	fi
2146	done
2147
2148	# Then second loop on the components
2149	for comp in ${config_ListOfComponents[*]} ; do
2150
2151	eval ExeNameIn=\${config_Executable_${comp}[0]}
2152	eval ExeNameOut=\${config_Executable_${comp}[1]}
2153
2154	# Only if we really have an executable for the component and not the coupler ie oasis:
2155	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2156
2157	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2158	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2159	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2160	fi
2161	done
2162
2163	EXECUTION="${HOST_MPIRUN_COMMAND} -f ./run_file"
2164
2165	IGCM_sys_Chmod u+x run_file
2166	if ( $DEBUG_sys ) ; then
2167	echo "run_file contains : "
2168	cat run_file
2169	fi
2170
2171	# MPI-OpenMP (MPMD)
2172	else
2173
2174	# Use of mpirun instead of ccc_mprun
2175	EXECUTION="time mpirun"
2176
2177	# Hosts treatment
2178	${EXECUTION} hostname \| sort \| uniq > hosts.tmp
2179
2180	i=0
2181	rm -f hosts rankfile
2182	IGCM_debug_Print 1 "sys Curie, Hosts available :"
2183	for nodes in `cat hosts.tmp`
2184	do
2185	host[$i]=$nodes
2186	echo "${host[$i]}" >> hosts
2187	IGCM_debug_Print 1 ${host[$i]}
2188	i=$((i+1))
2189	done
2190	rm -f hosts.tmp
2191
2192	listnodes=${host[*]}
2193
2194	EXECUTION="${EXECUTION} -hostfile hosts -rankfile rankfile"
2195
2196	# Initialisation
2197	rank=0
2198	current_core=0
2199	core_per_node=16
2200	init_exec=n
2201
2202	# Loop on the components
2203	for comp in ${config_ListOfComponents[*]} ; do
2204
2205	eval ExeNameIn=\${config_Executable_${comp}[0]}
2206	eval ExeNameOut=\${config_Executable_${comp}[1]}
2207
2208	# Not possible if oasis has an executable (i.e old version of oasis3)
2209	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
2210	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
2211	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
2212	IGCM_debug_Verif_Exit
2213	fi
2214
2215	# Only if we really have an executable for the component :
2216	if [ "X${ExeNameOut}" != X\"\" ] ; then
2217
2218	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2219	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2220
2221	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2222	echo "" >> script_${ExeNameOut}.ksh
2223	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
2224
2225	# Check if the number of threads is correct
2226	case ${comp_proc_omp_loc} in
2227	2\|4\|8\|16)
2228	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2229	;;
2230	*)
2231	IGCM_debug_Exit "ERROR with OMP parameters !"
2232	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2233	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2234	IGCM_debug_Verif_Exit
2235	;;
2236	esac
2237	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2238	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2239	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2240	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2241	fi
2242	#echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
2243	#echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
2244	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2245	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2246
2247	if [ ${init_exec} = y ] ; then
2248	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2249	else
2250	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2251	init_exec=y
2252	fi
2253
2254	# Build rankfile : method used to assign cores and nodes for the MPI process
2255	# Ex :
2256	#rank 0=curie5296 slot=0,2,4,6
2257	#rank 1=curie5296 slot=8,10,12,14
2258	# Example of final command :
2259	# mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
2260	# with script_lmdz.x.ksh :
2261	# #!/bin/ksh
2262	#export KMP_STACKSIZE=3g
2263	#export KMP_LIBRARY=turnaround
2264	#export MKL_SERIAL=YES
2265	#OMP_NUM_THREADS=4
2266	#./lmdz.x
2267	#
2268
2269	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
2270	(( index_host = current_core / core_per_node ))
2271	host_value=${host[${index_host}]}
2272	(( slot = current_core % core_per_node ))
2273	virg=","
2274	string_final=""
2275	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
2276	(( index_slot = index * 2 ))
2277	string=$index_slot$virg
2278	string_final=$string_final$string
2279	done
2280	string_final=$( echo $string_final \| sed "s/.$//" )
2281	echo "rank $rank=$host_value slot=$string_final" >> rankfile
2282	(( rank = rank + 1 ))
2283	(( current_core = current_core + comp_proc_omp_loc ))
2284	done
2285	fi
2286
2287	done
2288	fi
2289
2290	# Only one executable (SPMD mode).
2291	else
2292
2293	for comp in ${config_ListOfComponents[*]} ; do
2294
2295	# Only if we really have an executable for the component :
2296	eval ExeNameOut=\${config_Executable_${comp}[1]}
2297	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2298
2299	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2300	echo "" >> script_${ExeNameOut}.ksh
2301	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2302
2303	if ( ${OK_PARA_OMP} ) ; then
2304	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2305	echo "" >> script_${ExeNameOut}.ksh
2306	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2307	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2308	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2309	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2310	fi
2311
2312	if ( ${OK_PARA_MPI} ) ; then
2313	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2314	# Default : ccc_mprun used if nb_proc gt 1
2315	# to have out/err per process on different files
2316	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
2317	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2318	EXECUTION="${HOST_MPIRUN_COMMAND} -n ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2319	else
2320	# Default : ccc_mprun is NOT used if nb_proc eq 1
2321	# to have out/err per process on different files
2322	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2323	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2324	EXECUTION="/usr/bin/time ./script_${ExeNameOut}.ksh"
2325	fi
2326
2327	IGCM_debug_Print 1 "sys Curie : script_${ExeNameOut}.ksh contains"
2328	cat script_${ExeNameOut}.ksh
2329
2330	fi
2331
2332	done
2333
2334	fi
2335
2336	IGCM_debug_Print 1 "sys Curie : execution command is "
2337	IGCM_debug_Print 1 "$EXECUTION"
2338
2339	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2340	}
2341
2342	#D-#==================================================
2343	#D-function IGCM_sys_check_path
2344	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2345	#D-* do not point to an important use directory. Stop immediately in that case.
2346	#D-* Examples:
2347	#D-
2348	function IGCM_sys_check_path {
2349	IGCM_debug_PushStack "IGCM_sys_check_path"
2350	if ( $DEBUG_sys ) ; then
2351	echo "IGCM_sys_check_path"
2352	fi
2353
2354	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCHDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCWORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCSTOREDIR} ] ) ; then
2355	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2356	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2357	IGCM_debug_Exit "This will stop the job"
2358	fi
2359	IGCM_debug_PopStack "IGCM_sys_check_path"
2360	}
2361
2362	#D-#==================================================
2363	#D-function IGCM_sys_check_quota
2364	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2365	#D-* Examples:
2366	#D-
2367	function IGCM_sys_check_quota {
2368	IGCM_debug_PushStack "IGCM_sys_check_quota"
2369	if ( $DEBUG_sys ) ; then
2370	echo "IGCM_sys_check_quota"
2371	fi
2372	# Limit of quota (in %)
2373	limit_quota=90
2374
2375	# Check of the volume
2376	volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
2377	volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
2378
2379	if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
2380
2381	unit_avail=${volume_avail: -1}
2382	unit_quota=${volume_quota: -1}
2383
2384	if [ "${unit_quota}" = "*" ] ; then
2385	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2386	IGCM_debug_Print 1 "More than 100% of your quota is used"
2387	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2388	IGCM_debug_Print 1 "You must have more than 10% available to run"
2389	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2390	IGCM_debug_Verif_Exit
2391	fi
2392
2393	temp_avail=${volume_avail%%${volume_avail: -1}*}
2394	temp_quota=${volume_quota%%${volume_quota: -1}*}
2395
2396	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2397
2398	# Convertion
2399	if [ ${volume_avail: -1} = "T" ] ; then
2400	(( temp_avail = temp_avail * 1000000000000 ))
2401	elif [ ${volume_avail: -1} = "G" ] ; then
2402	(( temp_avail = temp_avail * 1000000000 ))
2403	elif [ ${volume_avail: -1} = "M" ] ; then
2404	(( temp_avail = temp_avail * 1000000 ))
2405	elif [ ${volume_avail: -1} = "k" ] ; then
2406	(( temp_avail = temp_avail * 1000 ))
2407	else
2408	(( temp_avail = volume_avail ))
2409	fi
2410	if [ ${volume_quota: -1} = "T" ] ; then
2411	(( temp_quota = temp_quota * 1000000000000 ))
2412	elif [ ${volume_quota: -1} = "G" ] ; then
2413	(( temp_quota = temp_quota * 1000000000 ))
2414	elif [ ${volume_quota: -1} = "M" ] ; then
2415	(( temp_quota = temp_quota * 1000000 ))
2416	elif [ ${volume_quota: -1} = "k" ] ; then
2417	(( temp_quota = temp_quota * 1000 ))
2418	else
2419	(( temp_quota = volume_quota ))
2420	fi
2421	fi
2422
2423	quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2424	# echo "volume ratio is " $quota_volume
2425
2426	if [ ${quota_volume} -ge ${limit_quota} ] ; then
2427	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2428	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2429	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2430	IGCM_debug_Print 1 "You must have more than 10% available to run"
2431	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2432	IGCM_debug_Verif_Exit
2433	fi
2434
2435	fi
2436
2437	# Check of the number of inodes
2438
2439	inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
2440	inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
2441
2442	if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
2443
2444	unit_avail=${inode_avail: -1}
2445	unit_quota=${inode_quota: -1}
2446
2447	if [ "${unit_quota}" = "*" ] ; then
2448	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2449	IGCM_debug_Print 1 "More than 100% of your quota is used"
2450	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2451	IGCM_debug_Print 1 "You must have more than 10% available to run"
2452	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2453	IGCM_debug_Verif_Exit
2454	fi
2455
2456	temp_avail=${inode_avail%%${inode_avail: -1}*}
2457	temp_quota=${inode_quota%%${inode_quota: -1}*}
2458
2459	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2460
2461	# Convertion
2462	if [ ${inode_avail: -1} = "T" ] ; then
2463	(( temp_avail = temp_avail * 1000000000000 ))
2464	elif [ ${inode_avail: -1} = "G" ] ; then
2465	(( temp_avail = temp_avail * 1000000000 ))
2466	elif [ ${inode_avail: -1} = "M" ] ; then
2467	(( temp_avail = temp_avail * 1000000 ))
2468	elif [ ${inode_avail: -1} = "k" ] ; then
2469	(( temp_avail = temp_avail * 1000 ))
2470	else
2471	(( temp_avail = inode_avail ))
2472	fi
2473
2474	if [ ${inode_quota: -1} = "T" ] ; then
2475	(( temp_quota = temp_quota * 1000000000000 ))
2476	elif [ ${inode_quota: -1} = "G" ] ; then
2477	(( temp_quota = temp_quota * 1000000000 ))
2478	elif [ ${inode_quota: -1} = "M" ] ; then
2479	(( temp_quota = temp_quota * 1000000 ))
2480	elif [ ${inode_quota: -1} = "k" ] ; then
2481	(( temp_quota = temp_quota * 1000 ))
2482	else
2483	(( temp_quota = inode_quota ))
2484	fi
2485	fi
2486	quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2487	# echo "inode ratio is " $quota_inode
2488
2489	if [ ${quota_inode} -ge ${limit_quota} ] ; then
2490	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2491	IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
2492	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2493	IGCM_debug_Print 1 "You must have more than 10% available to run"
2494	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2495	IGCM_debug_Verif_Exit
2496	fi
2497	fi
2498	IGCM_debug_PopStack "IGCM_sys_check_quota"
2499	}
2500
2501	#D-#==================================================
2502	#D-function IGCM_sys_GetJobID
2503	#D-* Purpose: Check if job_name is currently
2504	#D- running or in queue
2505	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2506	#D-
2507	function IGCM_sys_GetJobID {
2508	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2509	if ( $DEBUG_sys ) ; then
2510	echo "IGCM_sys_GetJobID"
2511	fi
2512
2513	# With -f option, the full job name is given in the last column
2514	ID=$( ccc_mstat -f -u $2 \| \
2515	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2516
2517	eval ${3}=${ID}
2518
2519	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2520	}
2521
2522	#D-#==================================================
2523	#D-function IGCM_sys_CountJobInQueue
2524	#D-* Purpose: Check if job_name is currently
2525	#D- running or in queue
2526	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2527	#D-
2528	function IGCM_sys_CountJobInQueue {
2529	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2530	if ( $DEBUG_sys ) ; then
2531	echo "IGCM_sys_CountJobInQueue"
2532	fi
2533
2534	#NbRun=$( ccc_mstat -f \| grep -c ${JobName} )
2535
2536	# With -f option, the full job name is given in the last column
2537	NbRun=$( ccc_mstat -f \| gawk -v JobName=$1 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
2538
2539	eval ${2}=${NbRun}
2540
2541	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2542	}
2543
2544	#D-#==================================================
2545	#D-function IGCM_sys_ListJobInQueue
2546	#D-* Purpose: Check if job_name is currently
2547	#D- running or in queue
2548	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2549	#D-
2550	function IGCM_sys_ListJobInQueue {
2551	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2552	if ( $DEBUG_sys ) ; then
2553	echo "IGCM_sys_ListJobInQueue"
2554	fi
2555
2556	# With -f option, the full job name is given in the last column
2557	set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
2558	'( $2 == User && \
2559	$NF != /TS/ && \
2560	$NF !~ /PACK/ && \
2561	$NF !~ /REBUILD/ && \
2562	$NF !~ /pack/ ) \
2563	{ print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
2564
2565	eval set -A ${2} ${JobList[*]}
2566
2567	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2568	}
2569
2570	##############################################################
2571	# NCO OPERATOR
2572
2573	#D-#==================================================
2574	#D-function IGCM_sys_ncap2
2575	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2576	#D-* Examples:
2577	#D-
2578	function IGCM_sys_ncap2 {
2579	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2580	if ( $DEBUG_sys ) ; then
2581	echo "IGCM_sys_ncap2 :" $@
2582	fi
2583
2584	typeset NB_ESSAI DELAI status i
2585	# number of tentative
2586	NB_ESSAI=3
2587	# time delay between tentative
2588	DELAI=2
2589
2590	i=0
2591	while [ $i -lt $NB_ESSAI ] ; do
2592	ncap2 -C "$@" > /tmp/out_command.$$ 2>&1
2593	status=$?
2594	if [ ${status} -gt 0 ] ; then
2595	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2596	cat /tmp/out_command.$$
2597	\rm /tmp/out_command.$$
2598	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2599	sleep $DELAI
2600	else
2601	\rm /tmp/out_command.$$
2602	break
2603	fi
2604	(( i = i + 1 ))
2605	done
2606
2607	if [ ${status} -gt 0 ] ; then
2608	echo "IGCM_sys_ncap2 : ncap2 error"
2609	IGCM_debug_Exit "ncap2"
2610	fi
2611
2612	IGCM_debug_PopStack "IGCM_sys_ncap2"
2613	}
2614
2615	#D-#==================================================
2616	#D-function IGCM_sys_ncatted
2617	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2618	#D-* Examples:
2619	#D-
2620	function IGCM_sys_ncatted {
2621	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2622	if ( $DEBUG_sys ) ; then
2623	echo "IGCM_sys_ncatted :" $@
2624	fi
2625
2626	typeset NB_ESSAI DELAI status i
2627	# number of tentative
2628	NB_ESSAI=3
2629	# time delay between tentative
2630	DELAI=2
2631
2632	i=0
2633	while [ $i -lt $NB_ESSAI ] ; do
2634	ncatted "$@" > /tmp/out_command.$$ 2>&1
2635	status=$?
2636	if [ ${status} -gt 0 ] ; then
2637	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2638	cat /tmp/out_command.$$
2639	\rm /tmp/out_command.$$
2640	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2641	sleep $DELAI
2642	else
2643	\rm /tmp/out_command.$$
2644	break
2645	fi
2646	(( i = i + 1 ))
2647	done
2648
2649	if [ ${status} -gt 0 ] ; then
2650	echo "IGCM_sys_ncatted : ncatted error"
2651	IGCM_debug_Exit "ncatted"
2652	fi
2653
2654	IGCM_debug_PopStack "IGCM_sys_ncatted"
2655	}
2656
2657	#D-#==================================================
2658	#D-function IGCM_sys_ncbo
2659	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2660	#D-* Examples:
2661	#D-
2662	function IGCM_sys_ncbo {
2663	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2664	if ( $DEBUG_sys ) ; then
2665	echo "IGCM_sys_ncbo :" $@
2666	fi
2667
2668	typeset NB_ESSAI DELAI status i
2669	# number of tentative
2670	NB_ESSAI=3
2671	# time delay between tentative
2672	DELAI=2
2673
2674	i=0
2675	while [ $i -lt $NB_ESSAI ] ; do
2676	ncbo -C $@ > /tmp/out_command.$$ 2>&1
2677	status=$?
2678	if [ ${status} -gt 0 ] ; then
2679	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2680	cat /tmp/out_command.$$
2681	\rm /tmp/out_command.$$
2682	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2683	sleep $DELAI
2684	else
2685	\rm /tmp/out_command.$$
2686	break
2687	fi
2688	(( i = i + 1 ))
2689	done
2690
2691	if [ ${status} -gt 0 ] ; then
2692	echo "IGCM_sys_ncbo : ncbo error"
2693	IGCM_debug_Exit "ncbo"
2694	fi
2695
2696	IGCM_debug_PopStack "IGCM_sys_ncbo"
2697	}
2698
2699	#D-#==================================================
2700	#D-function IGCM_sys_ncdif
2701	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2702	#D-* Examples:
2703	#D-
2704	function IGCM_sys_ncdiff {
2705	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2706	if ( $DEBUG_sys ) ; then
2707	echo "IGCM_sys_ncdiff :" $@
2708	fi
2709
2710	typeset NB_ESSAI DELAI status i
2711	# number of tentative
2712	NB_ESSAI=3
2713	# time delay between tentative
2714	DELAI=2
2715
2716	i=0
2717	while [ $i -lt $NB_ESSAI ] ; do
2718	ncdiff -C $@ > /tmp/out_command.$$ 2>&1
2719	status=$?
2720	if [ ${status} -gt 0 ] ; then
2721	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2722	cat /tmp/out_command.$$
2723	\rm /tmp/out_command.$$
2724	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2725	sleep $DELAI
2726	else
2727	\rm /tmp/out_command.$$
2728	break
2729	fi
2730	(( i = i + 1 ))
2731	done
2732
2733	if [ ${status} -gt 0 ] ; then
2734	echo "IGCM_sys_ncdiff : ncdiff error"
2735	IGCM_debug_Exit "ncdiff"
2736	fi
2737
2738	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2739	}
2740
2741	#D-#==================================================
2742	#D-function IGCM_sys_ncea
2743	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2744	#D-* Examples:
2745	#D-
2746	function IGCM_sys_ncea {
2747	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2748	if ( $DEBUG_sys ) ; then
2749	echo "IGCM_sys_ncea :" $@
2750	fi
2751
2752	typeset NB_ESSAI DELAI status i
2753	# number of tentative
2754	NB_ESSAI=3
2755	# time delay between tentative
2756	DELAI=2
2757
2758	i=0
2759	while [ $i -lt $NB_ESSAI ] ; do
2760	ncea -C $@ > /tmp/out_command.$$ 2>&1
2761	status=$?
2762	if [ ${status} -gt 0 ] ; then
2763	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2764	cat /tmp/out_command.$$
2765	\rm /tmp/out_command.$$
2766	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2767	sleep $DELAI
2768	else
2769	\rm /tmp/out_command.$$
2770	break
2771	fi
2772	(( i = i + 1 ))
2773	done
2774
2775	if [ ${status} -gt 0 ] ; then
2776	echo "IGCM_sys_ncea : ncea error"
2777	IGCM_debug_Exit "ncea"
2778	fi
2779
2780	IGCM_debug_PopStack "IGCM_sys_ncea"
2781	}
2782
2783	#D-#==================================================
2784	#D-function IGCM_sys_ncecat
2785	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2786	#D-* Examples:
2787	#D-
2788	function IGCM_sys_ncecat {
2789	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2790	if ( $DEBUG_sys ) ; then
2791	echo "IGCM_sys_ncecat :" $@
2792	fi
2793
2794	typeset NB_ESSAI DELAI status i
2795	# number of tentative
2796	NB_ESSAI=3
2797	# time delay between tentative
2798	DELAI=2
2799
2800	i=0
2801	while [ $i -lt $NB_ESSAI ] ; do
2802	ncecat -C $@ > /tmp/out_command.$$ 2>&1
2803	status=$?
2804	if [ ${status} -gt 0 ] ; then
2805	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2806	cat /tmp/out_command.$$
2807	\rm /tmp/out_command.$$
2808	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2809	sleep $DELAI
2810	else
2811	\rm /tmp/out_command.$$
2812	break
2813	fi
2814	(( i = i + 1 ))
2815	done
2816
2817	if [ ${status} -gt 0 ] ; then
2818	echo "IGCM_sys_ncecat : ncecat error"
2819	IGCM_debug_Exit "ncecat"
2820	fi
2821
2822	IGCM_debug_PopStack "IGCM_sys_ncecat"
2823	}
2824
2825	#D-#==================================================
2826	#D-function IGCM_sys_ncflint
2827	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2828	#D-* Examples:
2829	#D-
2830	function IGCM_sys_ncflint {
2831	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2832	if ( $DEBUG_sys ) ; then
2833	echo "IGCM_sys_ncflint :" $@
2834	fi
2835
2836	typeset NB_ESSAI DELAI status i
2837	# number of tentative
2838	NB_ESSAI=3
2839	# time delay between tentative
2840	DELAI=2
2841
2842	i=0
2843	while [ $i -lt $NB_ESSAI ] ; do
2844	ncflint -C $@ > /tmp/out_command.$$ 2>&1
2845	status=$?
2846	if [ ${status} -gt 0 ] ; then
2847	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2848	cat /tmp/out_command.$$
2849	\rm /tmp/out_command.$$
2850	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2851	sleep $DELAI
2852	else
2853	\rm /tmp/out_command.$$
2854	break
2855	fi
2856	(( i = i + 1 ))
2857	done
2858
2859	if [ ${status} -gt 0 ] ; then
2860	echo "IGCM_sys_ncflint : ncflint error"
2861	IGCM_debug_Exit "ncflint"
2862	fi
2863
2864	IGCM_debug_PopStack "IGCM_sys_ncflint"
2865	}
2866
2867	#D-#==================================================
2868	#D-function IGCM_sys_ncks
2869	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2870	#D-* Examples:
2871	#D-
2872	function IGCM_sys_ncks {
2873	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2874	if ( $DEBUG_sys ) ; then
2875	echo "IGCM_sys_ncks :" $@
2876	fi
2877
2878	typeset NB_ESSAI DELAI status i
2879	# number of tentative
2880	NB_ESSAI=3
2881	# time delay between tentative
2882	DELAI=2
2883
2884	i=0
2885	while [ $i -lt $NB_ESSAI ] ; do
2886	ncks -C $@ > /tmp/out_command.$$ 2>&1
2887	status=$?
2888	if [ ${status} -gt 0 ] ; then
2889	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2890	cat /tmp/out_command.$$
2891	\rm /tmp/out_command.$$
2892	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2893	sleep $DELAI
2894	else
2895	\rm /tmp/out_command.$$
2896	break
2897	fi
2898	(( i = i + 1 ))
2899	done
2900
2901	if [ ${status} -gt 0 ] ; then
2902	echo "IGCM_sys_ncks : ncks error"
2903	IGCM_debug_Exit "ncks"
2904	fi
2905
2906	IGCM_debug_PopStack "IGCM_sys_ncks"
2907	}
2908
2909	#D-#==================================================
2910	#D-function IGCM_sys_ncpdq
2911	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2912	#D-* Examples:
2913	#D-
2914	function IGCM_sys_ncpdq {
2915	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2916	if ( $DEBUG_sys ) ; then
2917	echo "IGCM_sys_ncpdq :" $@
2918	fi
2919
2920	typeset NB_ESSAI DELAI status i
2921	# number of tentative
2922	NB_ESSAI=3
2923	# time delay between tentative
2924	DELAI=2
2925
2926	i=0
2927	while [ $i -lt $NB_ESSAI ] ; do
2928	ncpdq -C $@ > /tmp/out_command.$$ 2>&1
2929	status=$?
2930	if [ ${status} -gt 0 ] ; then
2931	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2932	cat /tmp/out_command.$$
2933	\rm /tmp/out_command.$$
2934	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2935	sleep $DELAI
2936	else
2937	\rm /tmp/out_command.$$
2938	break
2939	fi
2940	(( i = i + 1 ))
2941	done
2942
2943	if [ ${status} -gt 0 ] ; then
2944	echo "IGCM_sys_ncpdq : ncpdq error"
2945	IGCM_debug_Exit "ncpdq"
2946	fi
2947
2948	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2949	}
2950
2951	#D-#==================================================
2952	#D-function IGCM_sys_ncra
2953	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2954	#D-* Examples:
2955	#D-
2956	function IGCM_sys_ncra {
2957	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2958	if ( $DEBUG_sys ) ; then
2959	echo "IGCM_sys_ncra :" $@
2960	fi
2961
2962	typeset NB_ESSAI DELAI status i
2963	# number of tentative
2964	NB_ESSAI=3
2965	# time delay between tentative
2966	DELAI=2
2967
2968	i=0
2969	while [ $i -lt $NB_ESSAI ] ; do
2970	ncra -C $@ > /tmp/out_command.$$ 2>&1
2971	status=$?
2972	if [ ${status} -gt 0 ] ; then
2973	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
2974	cat /tmp/out_command.$$
2975	\rm /tmp/out_command.$$
2976	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2977	sleep $DELAI
2978	else
2979	\rm /tmp/out_command.$$
2980	break
2981	fi
2982	(( i = i + 1 ))
2983	done
2984
2985	if [ ${status} -gt 0 ] ; then
2986	echo "IGCM_sys_ncra : ncra error"
2987	IGCM_debug_Exit "ncra"
2988	fi
2989
2990	IGCM_debug_PopStack "IGCM_sys_ncra"
2991	}
2992
2993	#D-#==================================================
2994	#D-function IGCM_sys_ncrcat
2995	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
2996	#D-* Examples:
2997	#D-
2998	function IGCM_sys_ncrcat {
2999	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
3000	if ( $DEBUG_sys ) ; then
3001	echo "IGCM_sys_ncrcat :" $@
3002	fi
3003
3004	typeset NB_ESSAI DELAI status i lastArg
3005	# number of tentative
3006	NB_ESSAI=3
3007	# time delay between tentative
3008	DELAI=2
3009
3010	i=0
3011	while [ $i -lt $NB_ESSAI ] ; do
3012	ncrcat -C $@ > /tmp/out_command.$$ 2>&1
3013	status=$?
3014	if [ ${status} -gt 0 ] ; then
3015	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
3016	cat /tmp/out_command.$$
3017	\rm /tmp/out_command.$$
3018	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3019	sleep $DELAI
3020	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
3021	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
3022	cat /tmp/out_command.$$
3023	# remove files having corrupted time axis
3024	eval lastArg=\${$#}
3025	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
3026	\rm ${lastArg}
3027	\rm /tmp/out_command.$$
3028	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3029	sleep $DELAI
3030	else
3031	\rm /tmp/out_command.$$
3032	break
3033	fi
3034	(( i = i + 1 ))
3035	done
3036
3037	if [ ${status} -gt 0 ] ; then
3038	echo "IGCM_sys_ncrcat : ncrcat error"
3039	#IGCM_debug_Exit "ncrcat"
3040	fi
3041
3042	IGCM_debug_PopStack "IGCM_sys_ncrcat"
3043	}
3044
3045	#D-#==================================================
3046	#D-function IGCM_sys_ncrename
3047	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
3048	#D-* Examples:
3049	#D-
3050	function IGCM_sys_ncrename {
3051	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
3052	if ( $DEBUG_sys ) ; then
3053	echo "IGCM_sys_ncrename :" $@
3054	fi
3055
3056	typeset NB_ESSAI DELAI status i
3057	# number of tentative
3058	NB_ESSAI=3
3059	# time delay between tentative
3060	DELAI=2
3061
3062	i=0
3063	while [ $i -lt $NB_ESSAI ] ; do
3064	ncrename $@ > /tmp/out_command.$$ 2>&1
3065	status=$?
3066	if [ ${status} -gt 0 ] ; then
3067	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
3068	cat /tmp/out_command.$$
3069	\rm /tmp/out_command.$$
3070	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3071	sleep $DELAI
3072	else
3073	\rm /tmp/out_command.$$
3074	break
3075	fi
3076	(( i = i + 1 ))
3077	done
3078
3079	if [ ${status} -gt 0 ] ; then
3080	echo "IGCM_sys_ncrename : ncrename error"
3081	IGCM_debug_Exit "ncrename"
3082	fi
3083
3084	IGCM_debug_PopStack "IGCM_sys_ncrename"
3085	}
3086
3087	#D-#==================================================
3088	#D-function IGCM_sys_ncwa
3089	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
3090	#D-* Examples:
3091	#D-
3092	function IGCM_sys_ncwa {
3093	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
3094	if ( $DEBUG_sys ) ; then
3095	echo "IGCM_sys_ncwa :" $@
3096	fi
3097
3098	typeset NB_ESSAI DELAI status i
3099	# number of tentative
3100	NB_ESSAI=3
3101	# time delay between tentative
3102	DELAI=2
3103
3104	i=0
3105	while [ $i -lt $NB_ESSAI ] ; do
3106	ncwa -C $@ > /tmp/out_command.$$ 2>&1
3107	status=$?
3108	if [ ${status} -gt 0 ] ; then
3109	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
3110	cat /tmp/out_command.$$
3111	\rm /tmp/out_command.$$
3112	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3113	sleep $DELAI
3114	else
3115	\rm /tmp/out_command.$$
3116	break
3117	fi
3118	(( i = i + 1 ))
3119	done
3120
3121	if [ ${status} -gt 0 ] ; then
3122	echo "IGCM_sys_ncwa : ncwa error"
3123	IGCM_debug_Exit "ncwa"
3124	fi
3125
3126	IGCM_debug_PopStack "IGCM_sys_ncwa"
3127	}
3128
3129	##############################################################
3130	# CDO OPERATOR
3131
3132	#D-#==================================================
3133	#D-function IGCM_sys_cdo
3134	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
3135	#D-* Examples:
3136	#D-
3137	function IGCM_sys_cdo {
3138	IGCM_debug_PushStack "IGCM_sys_cdo" $@
3139	if ( $DEBUG_sys ) ; then
3140	echo "IGCM_sys_cdo :" $@
3141	fi
3142
3143	typeset status
3144
3145	\cdo $@ > /tmp/out_command.$$ 2>&1
3146	status=$?
3147	if [ ${status} -gt 0 ] ; then
3148	echo "IGCM_sys_cdo : error code ${status}"
3149	cat /tmp/out_command.$$
3150	\rm /tmp/out_command.$$
3151	IGCM_debug_PopStack "IGCM_sys_cdo"
3152	return 1
3153	else
3154	IGCM_debug_PopStack "IGCM_sys_cdo"
3155	return 0
3156	fi
3157
3158	IGCM_debug_PopStack "IGCM_sys_cdo"
3159	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: