Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_curie.ksh @ 1180

Last change on this file since 1180 was 1180, checked in by sdipsl, 9 years ago
house keeping in libIGCM_sys (third pass). Will ease #248 and #251
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 96.1 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Curie
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=curie
71	# add default project on curie
72	typeset DEFAULT_PROJECT=gen0826
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# Submit command
80	typeset SUBMIT=${SUBMIT:=ccc_msub}
81	# rsync with path
82	typeset -r RSYNC=/usr/bin/rsync
83	# RSYNC_opt args to rsync
84	typeset -r RSYNC_opt="-va"
85	# ie storage filesystem
86	typeset -r STOREHOST=${MASTER}
87	typeset -r REMOTE_RSYNC=/usr/bin/rsync
88
89	#====================================================
90	# Set environment tools (ferret, nco, cdo, rebuild, ...)
91	#====================================================
92	if [ X${TaskType} = Xcomputing ] ; then
93	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
94	# to run with netcdf 3.6.3 ie compilation done before 17/2/2014
95	# uncomment 2 lines :
96	# module unload netcdf
97	# module load netcdf/3.6.3
98	else
99	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
100	PCMDI_MP=/ccc/work/cont003/dsm/p86ipsl/PCMDI-MP
101	fi
102	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddNoise/src_X64_CURIE/bin
103	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddPerturbation/src_X64_CURIE/bin
104	export PATH=${PATH}:$( ccc_home -u p86ipsl )/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/
105
106	#====================================================
107	# Specific for ocean additionnal diagnostic
108	export FER_GO="$FER_GO /home/cont003/p86denv/IGCM_POST_UTIL/JNL /home/cont003/p86denv/GRAF /home/cont003/p86denv/GRAF/GO"
109	export FER_PALETTE="$FER_PALETTE /home/cont003/p86denv/GRAF/PALET"
110
111	#====================================================
112	# Host specific DIRECTORIES
113	#====================================================
114
115	# ============ CESIUM START ============ #
116
117	#====================================================
118	#- Mirror libIGCM from titane to cesium if needed
119	#ROOTSYS=$( echo ${libIGCM} \| gawk -F"/" '{print $3}' )
120	#if [ ! ${ROOTSYS} = "home" ] ; then
121	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
122	#else
123	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
124	#fi
125
126	#====================================================
127	#- libIGCM_POST
128	#if ( ${MirrorlibIGCM} ) ; then
129	# PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
130	# typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
131	#else
132	# typeset -r libIGCM_POST=${libIGCM}
133	#fi
134
135	# ============ CESIUM END ============ #
136
137	#====================================================
138	#- MirrorlibIGCM for frontend
139	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
140
141	#====================================================
142	#- libIGCM_POST for frontend
143	typeset -r libIGCM_POST=${libIGCM}
144
145	#====================================================
146	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
147	typeset -r R_EXE="${MODIPSL}/bin"
148
149	#====================================================
150	#- SUBMIT_DIR : submission dir
151	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${BRIDGE_MSUB_PWD}}
152
153	#====================================================
154	#- IN
155	typeset -r R_IN=${R_IN:=/ccc/work/cont003/dsm/p86ipsl/IGCM}
156	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/ccc/work/cont003/dsm/p24data}
157
158	#====================================================
159	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
160	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCHDIR}/RUN_DIR/${BRIDGE_MSUB_JOBID}_${$}}
161
162	#====================================================
163	#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
164	OUTCOMMAND_PATH=/tmp
165
166	#====================================================
167	#- HOST_MPIRUN_COMMAND
168	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time ccc_mprun -E-K1"}
169
170	#====================================================
171	#- Max number of arguments passed to nco operator or demigration command
172	UNIX_MAX_LIMIT=360
173
174	#====================================================
175	#- set PackDefault to true on curie
176	PackDefault=true
177
178	#====================================================
179	#- Number of core per node (max number of OpenMP task)
180	NUM_COREPERNODE=8
181
182	#====================================================
183	#- Default number of MPI task for IPSL coupled model
184	#- required for backward compatibility
185	#-
186	DEFAULT_NUM_PROC_OCE=5
187	DEFAULT_NUM_PROC_CPL=1
188	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
189	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
190
191	#D-#==================================================
192	#D-function IGCM_sys_defineArchives
193	#D-* Purpose:
194	#D-* Define ARCHIVE : Dedicated to large files
195	#D-* Define STORAGE : Dedicated to small/medium files
196	#D-* Define R_OUT : Output tree located on ARCHIVE
197	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
198	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
199	#D-* if SpaceName=TEST everything is stored on SCRATCHDIR
200	#D-* Examples:
201	#D-
202	function IGCM_sys_defineArchives {
203	IGCM_debug_PushStack "IGCM_sys_defineArchives"
204
205	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
206	#====================================================
207	#- ARCHIVE (dedicated to large files)
208	ARCHIVE=${config_UserChoices_ARCHIVE}
209	else
210	#====================================================
211	#- ARCHIVE (dedicated to large files)
212	ARCHIVE=${CCCSTOREDIR}
213	fi
214
215	if [ ! X${config_UserChoices_STORAGE} = X ]; then
216	#====================================================
217	#- STORAGE (dedicated to small/medium files)
218	STORAGE=${config_UserChoices_STORAGE}
219	else
220	#====================================================
221	#- STORAGE (dedicated to small/medium files)
222	STORAGE=${CCCWORKDIR}
223	fi
224
225	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
226	#====================================================
227	#- R_OUT
228	R_OUT=${SCRATCHDIR}/IGCM_OUT
229
230	#====================================================
231	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
232	R_FIG=${SCRATCHDIR}/IGCM_OUT
233
234	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
235
236	else
237	#====================================================
238	#- R_OUT
239	R_OUT=${ARCHIVE}/IGCM_OUT
240
241	#====================================================
242	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
243	R_FIG=${STORAGE}/IGCM_OUT
244	fi
245
246	#====================================================
247	#- R_BUF (ONLY FOR double copy an scratch)
248	R_BUF=${SCRATCHDIR}/IGCM_OUT
249
250	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
251	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
252	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
253
254	IGCM_debug_PopStack "IGCM_sys_defineArchives"
255	}
256
257	#D-#==================================================
258	#D-function IGCM_sys_RshMaster
259	#D-* Purpose: Connection to frontend machine.
260	#D-* Examples:
261	#D-
262	function IGCM_sys_RshMaster {
263	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
264	OUTCOMMAND_PATH=${OUTCOMMAND_PATH} /bin/ksh <<-EOF
265	export libIGCM=${libIGCM}
266	export DEBUG_debug=${DEBUG_debug}
267	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
268	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
269	${@}
270	EOF
271	if [ $? -gt 0 ] ; then
272	echo "IGCM_sys_RshMaster : erreur."
273	IGCM_debug_Exit "IGCM_sys_RshMaster"
274	fi
275	IGCM_debug_PopStack "IGCM_sys_RshMaster"
276	}
277
278	#D-#==================================================
279	#D-function IGCM_sys_RshArchive
280	#D-* Purpose: Archive rsh command
281	#D-* Examples:
282	#D-
283	function IGCM_sys_RshArchive {
284	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
285	/bin/ksh <<-EOF
286	${@}
287	EOF
288	status=$?
289	if [ ${status} -gt 0 ] ; then
290	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
291	IGCM_debug_Exit "IGCM_sys_RshArchive"
292	fi
293	IGCM_debug_PopStack "IGCM_sys_RshArchive"
294	}
295
296	#D-#==================================================
297	#D-function IGCM_sys_RshArchive_NoError
298	#D-* Purpose: Archive rsh command, without error
299	#D-* used only in monitoring.job
300	#D-* Examples:
301	#D-
302	function IGCM_sys_RshArchive_NoError {
303	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
304	/bin/ksh <<-EOF
305	${@} 2> \dev\null
306	EOF
307	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
308	}
309
310	#D-#==================================================
311	#D-function IGCM_sys_RshPost
312	#D-* Purpose: Post-process rsh command
313	#D-* Examples:
314	#D-
315	function IGCM_sys_RshPost {
316	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
317	if ( $DEBUG_sys ) ; then
318	echo "IGCM_sys_RshPost :" $@
319	fi
320	# keep standard input (stdin) for the loop onto temporary file
321	cat >${OUTCOMMAND_PATH}/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
322
323	OUTCOMMAND_PATH=${OUTCOMMAND_PATH} /bin/ksh <${OUTCOMMAND_PATH}/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
324	if [ $? -gt 0 ] ; then
325	echo "IGCM_sys_RshPost : erreur."
326	IGCM_debug_Exit "IGCM_sys_RshPost"
327	fi
328	# delete temporary file
329	\rm ${OUTCOMMAND_PATH}/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
330
331	# ============ FRONTEND END ============ #
332
333	# ============ CESIUM START ============ #
334	# typeset NB_ESSAI DELAI status i
335	# if [ "X$( grep rebuild_from tmp_IGCM_sys_RshPost_$$ )" != "X" ] ; then
336	# #little hack so that rebuild submission is done on titane not an cesium
337	#
338	# libIGCM_POST_sed=$( echo $libIGCM_POST \| sed 's/\//\\\//g' )
339	# POST_DIR_sed=$( echo ${POST_DIR} \| sed 's/\//\\\//g' )
340	# sed "s/IGCM_sys_QsubPost/IGCM_sys_Qsub/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
341	# sed "s/ rebuild_fromWorkdir/ ${libIGCM_POST_sed}\/rebuild_fromWorkdir.job/g" tmp.txt > tmp_IGCM_sys_RshPost_$$
342	# sed "s/ rebuild_fromArchive/ ${libIGCM_POST_sed}\/rebuild_fromArchive.job/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
343	# sed "s/Script_Post_Output=/Script_Output=${POST_DIR_sed}\//g" tmp.txt > tmp_IGCM_sys_RshPost_$$
344	# \mv tmp.txt tmp_IGCM_sys_RshPost_$$
345	#
346	# echo cat tmp_IGCM_sys_RshPost_$$ AFTER
347	# cat tmp_IGCM_sys_RshPost_$$
348	#
349	# /bin/ksh <tmp_IGCM_sys_RshPost_$$
350	# if [ $? -gt 0 ] ; then
351	# echo "IGCM_sys_RshPost : erreur."
352	# IGCM_debug_Exit "IGCM_sys_RshPost"
353	# fi
354	# # delete temporary file
355	# \rm tmp_IGCM_sys_RshPost_$$
356	#
357	# else
358	# # number of tentative
359	# NB_ESSAI=10
360	# # time delay between tentative
361	# DELAI=10
362	# i=0
363	# while [ $i -ne $NB_ESSAI ] ; do
364	# ssh -t titane996 ssh cesium /bin/ksh <tmp_IGCM_sys_RshPost_$$
365	# status=$?
366	# if [ ${status} -ne 0 ];
367	# then
368	# sleep $DELAI
369	# else
370	# break
371	# fi
372	# let i=$i+1
373	# done
374	# # delete temporary file
375	# \rm tmp_IGCM_sys_RshPost_$$
376	#
377	# if [ ${status} -gt 0 ] ; then
378	# echo "IGCM_sys_RshPost : erreur."
379	# IGCM_debug_Exit "IGCM_sys_RshPost"
380	# fi
381	# fi
382
383	# ============ CESIUM END ============ #
384
385	IGCM_debug_PopStack "IGCM_sys_RshPost"
386	}
387
388	#D-#==================================================
389	#D-function IGCM_sys_SendMail
390	#D-* Purpose: Send mail when simulation is over
391	#D-* Examples:
392	#D-
393	function IGCM_sys_SendMail {
394	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
395	if ( $DEBUG_sys ) ; then
396	echo "IGCM_sys_SendMail :" $@
397	fi
398
399	if [ X${1} = XAccounting ] ; then
400	status=Accounting
401	mailText=jobAccounting.mail
402	elif ( ${ExitFlag} ) ; then
403	status=failed
404	mailText=jobEnd.mail
405	else
406	status=completed
407	mailText=jobEnd.mail
408	fi
409
410	# Update selected mail template
411	while read -r line; do
412	eval echo $line >> mail.txt ;
413	done < ${libIGCM}/libIGCM_sys/${mailText}
414
415	if [ ! -z ${config_UserChoices_MailName} ] ; then
416	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
417	elif [ -f ~/.forward ] ; then
418	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
419	else
420	mail -s "${config_UserChoices_JobName} ${status}" ${USER} < mail.txt
421	fi
422
423	sleep 10
424	rm -f mail.txt
425
426	IGCM_debug_PopStack "IGCM_sys_SendMail"
427	}
428
429	#D-#==================================================
430	#D-function IGCM_sys_Mkdir
431	#D-* Purpose: Master locale mkdir command
432	#D-* Examples:
433	#D-
434	function IGCM_sys_Mkdir {
435	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
436	if ( $DEBUG_sys ) ; then
437	echo "IGCM_sys_Mkdir :" $@
438	fi
439	if [ ! -d ${1} ]; then
440	\mkdir -p $1
441	if [ $? -gt 0 ] ; then
442	echo "IGCM_sys_Mkdir : erreur."
443	IGCM_debug_Exit "IGCM_sys_Mkdir"
444	fi
445	fi
446	# vérification :
447	if [ ! -d ${1} ] ; then
448	echo "IGCM_sys_Mkdir : erreur."
449	IGCM_debug_Exit "IGCM_sys_Mkdir"
450	fi
451	IGCM_debug_PopStack "IGCM_sys_Mkdir"
452	}
453
454	#D-#==================================================
455	#D-function IGCM_sys_MkdirArchive
456	#D-* Purpose: Mkdir on Archive
457	#D-* Examples:
458	#D-
459	function IGCM_sys_MkdirArchive {
460	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
461	if ( $DEBUG_sys ) ; then
462	echo "IGCM_sys_MkdirArchive :" $@
463	fi
464	#- creation de repertoire sur le serveur fichier
465	if [ ! -d ${1} ]; then
466	\mkdir -p $1
467	status=$?
468
469	if [ ${status} -gt 0 ] ; then
470	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
471	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
472	fi
473	fi
474	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
475	}
476
477	#D-#==================================================
478	#D-function IGCM_sys_MkdirWork
479	#D-* Purpose: Mkdir on Work
480	#D-* Examples:
481	#D-
482	function IGCM_sys_MkdirWork {
483	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
484	if ( $DEBUG_sys ) ; then
485	echo "IGCM_sys_MkdirWork :" $@
486	fi
487	#- creation de repertoire sur le serveur fichier
488	if [ ! -d ${1} ]; then
489	\mkdir -p $1
490	if [ $? -gt 0 ] ; then
491	echo "IGCM_sys_MkdirWork : erreur."
492	IGCM_debug_Exit "IGCM_sys_MkdirWork"
493	fi
494	fi
495	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
496	}
497
498	#D-#==================================================
499	#D-function IGCM_sys_Cd
500	#D-* Purpose: master cd command
501	#D-* Examples:
502	#D-
503	function IGCM_sys_Cd {
504	IGCM_debug_PushStack "IGCM_sys_Cd" $@
505	if ( $DEBUG_sys ) ; then
506	echo "IGCM_sys_Cd :" $@
507	fi
508	\cd $1
509	if [ $? -gt 0 ] ; then
510	echo "IGCM_sys_Cd : erreur."
511	IGCM_debug_Exit "IGCM_sys_Cd"
512	fi
513	IGCM_debug_PopStack "IGCM_sys_Cd"
514	}
515
516	#D-#==================================================
517	#D-function IGCM_sys_Chmod
518	#D-* Purpose: Chmod
519	#D-* Examples:
520	#D-
521	function IGCM_sys_Chmod {
522	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
523	if ( $DEBUG_sys ) ; then
524	echo "IGCM_sys_Chmod :" $@
525	fi
526	\chmod $@
527	if [ $? -gt 0 ] ; then
528	echo "IGCM_sys_Chmod : erreur."
529	IGCM_debug_Exit "IGCM_sys_Chmod"
530	fi
531	IGCM_debug_PopStack "IGCM_sys_Chmod"
532	}
533
534	#D-#==================================================
535	#D-function IGCM_sys_FileSize
536	#D-* Purpose: Filesize
537	#D-* Examples:
538	#D-
539	function IGCM_sys_FileSize {
540	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
541
542	typeset sizeF
543	set +A sizeF -- $( ls -la ${1} )
544	if [ $? -gt 0 ] ; then
545	IGCM_debug_Exit "IGCM_sys_FileSize"
546	fi
547	eval ${2}=${sizeF[4]}
548
549	IGCM_debug_PopStack "IGCM_sys_FileSize"
550	}
551
552	#D-#==================================================
553	#D-function IGCM_sys_TestDir
554	#D-* Purpose: Test Directory that must exists
555	#D-* Examples:
556	#D-
557	function IGCM_sys_TestDir {
558	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
559	if ( $DEBUG_sys ) ; then
560	echo "IGCM_sys_TestDir :" $@
561	fi
562	typeset ExistFlag
563	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
564	IGCM_debug_PopStack "IGCM_sys_TestDir"
565
566	return ${ExistFlag}
567	}
568
569	#D-#==================================================
570	#D-function IGCM_sys_TestDirArchive
571	#D-* Purpose: Test Directory that must exists on Archive
572	#D-* Examples:
573	#D-
574	function IGCM_sys_TestDirArchive {
575	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
576	if ( $DEBUG_sys ) ; then
577	echo "IGCM_sys_TestDirArchive :" $@
578	fi
579	typeset ExistFlag
580	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
581	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
582	return ${ExistFlag}
583	}
584
585	#D-#==================================================
586	#D-function IGCM_sys_IsFileArchived
587	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
588	#D-* Examples:
589	#D-
590	function IGCM_sys_IsFileArchived {
591	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
592	if ( $DEBUG_sys ) ; then
593	echo "IGCM_sys_IsFileArchived :" $@
594	fi
595	typeset IsArchivedFlag
596	IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
597	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
598
599	return ${IsArchivedFlag}
600	}
601
602	#D-#==================================================
603	#D-function IGCM_sys_TestFileArchive
604	#D-* Purpose: Test file that must NOT EXISTS on Archive
605	#D-* Examples:
606	#D-
607	function IGCM_sys_TestFileArchive {
608	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
609	typeset ExistFlag
610	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
611	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
612
613	return ${ExistFlag}
614	}
615
616	#D-#==================================================
617	#D-function IGCM_sys_TestFileBuffer
618	#D-* Purpose: Test file that must NOT EXISTS on Buffer
619	#D-* Examples:
620	#D-
621	function IGCM_sys_TestFileBuffer {
622	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
623	typeset ExistFlag
624	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
625	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
626
627	return ${ExistFlag}
628	}
629
630	#D-#==================================================
631	#D-function IGCM_sys_CountFileArchive
632	#D-* Purpose: Count files on Archive filesystem
633	#D-* Examples:
634	#D-
635	function IGCM_sys_CountFileArchive {
636	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
637	ls ${@} 2>/dev/null \| wc -l
638	if [ $? -gt 0 ] ; then
639	echo "IGCM_sys_CountFileArchive : erreur."
640	fi
641	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
642	}
643
644	#D-#==================================================
645	#D-function IGCM_sys_CountFileBuffer
646	#D-* Purpose: Count files on Scratch filesystem
647	#D-* Examples:
648	#D-
649	function IGCM_sys_CountFileBuffer {
650	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
651	ls ${@} 2>/dev/null \| wc -l
652	if [ $? -gt 0 ] ; then
653	echo "IGCM_sys_CountFileBuffer : erreur."
654	fi
655	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
656	}
657
658	#D-#==================================================
659	#D-function IGCM_sys_Tree
660	#D-* Purpose: Tree directories with files on ${ARCHIVE}
661	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
662	#D-
663	function IGCM_sys_Tree {
664	IGCM_debug_PushStack "IGCM_sys_Tree" $@
665	if ( $DEBUG_sys ) ; then
666	echo "IGCM_sys_Tree :" $@
667	fi
668
669	\ls -lR ${@}
670
671	IGCM_debug_PopStack "IGCM_sys_Tree"
672	}
673
674	#D-#==================================================
675	#D-function IGCM_sys_Tar
676	#D-* Purpose: master tar command
677	#D-* Examples:
678	#D-
679	function IGCM_sys_Tar {
680	IGCM_debug_PushStack "IGCM_sys_Tar" $@
681	if ( $DEBUG_sys ) ; then
682	echo "IGCM_sys_Tar :" $@
683	fi
684	\tar cf $@
685	if [ $? -gt 0 ] ; then
686	echo "IGCM_sys_Tar : erreur."
687	IGCM_debug_Exit "IGCM_sys_Tar"
688	fi
689	IGCM_debug_PopStack "IGCM_sys_Tar"
690	}
691
692	#D-#==================================================
693	#D-function IGCM_sys_UnTar
694	#D-* Purpose: master un-tar command
695	#D-* Examples:
696	#D-
697	function IGCM_sys_UnTar {
698	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
699	if ( $DEBUG_sys ) ; then
700	echo "IGCM_sys_UnTar :" $@
701	fi
702	\tar xvf $1
703	if [ $? -gt 0 ] ; then
704	echo "IGCM_sys_UnTar : erreur."
705	IGCM_debug_Exit "IGCM_sys_UnTar"
706	fi
707	IGCM_debug_PopStack "IGCM_sys_UnTar"
708	}
709
710	#D-#==================================================
711	#D-function IGCM_sys_Qsub
712	#D-* Purpose: Qsub new job
713	#D-* Examples:
714	#D-
715	function IGCM_sys_Qsub {
716	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
717	if ( $DEBUG_sys ) ; then
718	echo "IGCM_sys_Qsub :" $@
719	fi
720	typeset options status
721	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
722
723	/usr/bin/ccc_msub ${options} $1 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
724	status=$?
725
726	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
727	if [ ${status} -gt 0 ] ; then
728	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
729	IGCM_debug_Exit "IGCM_sys_Qsub"
730	else
731	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
732	fi
733	IGCM_debug_PopStack "IGCM_sys_Qsub"
734	}
735
736	#D-#==================================================
737	#D-function IGCM_sys_QsubPost
738	#D-* Purpose: Qsub new job on scalaire
739	#D-* Examples:
740	#D-
741	function IGCM_sys_QsubPost {
742	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
743	if ( $DEBUG_sys ) ; then
744	echo "IGCM_sys_QsubPost :" $@
745	fi
746	typeset options status
747	MY_PROJECT=$(echo ${BRIDGE_MSUB_PROJECT:=${DEFAULT_PROJECT}} \| cut -d@ -f1 )
748
749	# EASIER TO DO THIS INSTEAD OF DUPLICATING libIGCM_sys_curie.ksh
750	case $( hostname -s ) in
751	curie*)
752	options="-Q normal -A ${MY_PROJECT} -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out";;
753	airain*)
754	options="-q ivybridge -A dsm -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
755	esac
756
757	/usr/bin/ccc_msub ${options} ${libIGCM_POST}/$1.job > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
758	status=$?
759
760	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
761	if [ ${status} -gt 0 ] ; then
762	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
763	IGCM_debug_Exit "IGCM_sys_QsubPost"
764	else
765	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
766	fi
767	IGCM_debug_PopStack "IGCM_sys_QsubPost"
768	}
769
770	#D-*************************
771	#D- File transfer functions
772	#D-*************************
773	#D-
774
775	#D-#==================================================
776	#D-function IGCM_sys_Rsync_out
777	#D-* Purpose: treat return val of rsync
778	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
779	#D- Error values and explanations can depend on your system version.
780	function IGCM_sys_Rsync_out {
781	status=$1
782	if [ ! $status ] ; then
783	echo "rsync error !"
784	fi
785
786	if [ $MYLANG = "fr" ]; then
787	case $status in
788	0) return ;;
789	1) echo "Erreur de rsync ; RERR_SYNTAX : "
790	echo "Erreur de syntaxe ou d'utilisation."
791	return;;
792	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
793	echo "Incompatibilité de protocole."
794	return;;
795	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
796	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
797	echo "répertoires"
798	return;;
799	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
800	echo "Action demandée non supportée : une tentative de manipulation de"
801	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
802	echo "été faite ; ou une option qui est supportée par le client mais"
803	echo "pas par le serveur a été spécifiée."
804	return;;
805	10) echo "Erreur de rsync ; RERR_SOCKETIO"
806	echo "Erreur dans le socket d'entrée sortie"
807	return;;
808	11) echo "Erreur de rsync ; RERR_FILEIO"
809	echo "Erreur d'entrée sortie fichier"
810	return;;
811	12) echo "Erreur de rsync ; RERR_STREAMIO"
812	echo "Erreur dans flux de donnée du protocole rsync"
813	return;;
814	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
815	echo "Erreur avec les diagnostics du programme"
816	return;;
817	14) echo "Erreur de rsync ; RERR_IPC"
818	echo "Erreur dans le code IPC"
819	return;;
820	20) echo "Erreur de rsync ; RERR_SIGNAL"
821	echo "SIGUSR1 ou SIGINT reçu"
822	return;;
823	21) echo "Erreur de rsync ; RERR_WAITCHILD"
824	echo "Une erreur retournée par waitpid()"
825	return;;
826	22) echo "Erreur de rsync ; RERR_MALLOC"
827	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
828	return;;
829	23) echo ""
830	echo "Erreur fichier inexistant"
831	return;;
832	30) echo "Erreur de rsync ; RERR_TIMEOUT"
833	echo "Temps d'attente écoulé dans l'envoi/réception de données"
834	return;;
835	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
836	return;;
837	esac
838	elif [ $MYLANG = "en" ] ; then
839	case $status in
840	0) return;;
841	1) echo "rsync error : Syntax or usage error "
842	return;;
843	2) echo "rsync error : Protocol incompatibility "
844	return;;
845	3) echo "rsync error : Errors selecting input/output files, dirs"
846	return;;
847	4) echo "rsync error : Requested action not supported: an attempt"
848	echo "was made to manipulate 64-bit files on a platform that cannot support"
849	echo "them; or an option was specified that is supported by the client and"
850	echo "not by the server."
851	return;;
852	5) echo "rsync error : Error starting client-server protocol"
853	return;;
854	10) echo "rsync error : Error in socket I/O "
855	return;;
856	11) echo "rsync error : Error in file I/O "
857	return;;
858	12) echo "rsync error : Error in rsync protocol data stream "
859	return;;
860	13) echo "rsync error : Errors with program diagnostics "
861	return;;
862	14) echo "rsync error : Error in IPC code "
863	return;;
864	20) echo "rsync error : Received SIGUSR1 or SIGINT "
865	return;;
866	21) echo "rsync error : Some error returned by waitpid() "
867	return;;
868	22) echo "rsync error : Error allocating core memory buffers "
869	return;;
870	23) echo "rsync error : Partial transfer due to error"
871	return;;
872	24) echo "rsync error : Partial transfer due to vanished source files"
873	return;;
874	30) echo "rsync error : Timeout in data send/receive "
875	return;;
876	*) echo "rsync error : return code of rsync unknown :" $status
877	return;;
878	esac
879	else
880	echo "unknown language $MYLANG."
881	return
882	fi
883	}
884
885	#D-#==================================================
886	#D-function IGCM_sys_Miror_libIGCM
887	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
888	#D-* Examples:
889	#D-
890	function IGCM_sys_Mirror_libIGCM {
891	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
892	if ( $DEBUG_sys ) ; then
893	echo "IGCM_sys_Mirror_libIGCM"
894	fi
895
896	typeset status
897
898	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
899
900	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
901	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
902	status=$?
903
904	if [ ${status} -gt 0 ] ; then
905	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
906	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
907	fi
908	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
909	}
910
911	#D-#==================================================
912	#D-function IGCM_sys_Cp
913	#D-* Purpose: generic cp
914	#D-* Examples:
915	#D-
916	function IGCM_sys_Cp {
917	IGCM_debug_PushStack "IGCM_sys_Cp" $@
918	if ( $DEBUG_sys ) ; then
919	echo "IGCM_sys_Cp :" $@
920	fi
921
922	typeset status
923
924	echo cp $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
925	\cp $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
926	status=$?
927
928	if [ ${status} -gt 0 ] ; then
929	echo "IGCM_sys_Cp : error code ${status}"
930	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
931	IGCM_debug_Exit "IGCM_sys_Cp"
932	else
933	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
934	fi
935	IGCM_debug_PopStack "IGCM_sys_Cp"
936	}
937
938	#D-#==================================================
939	#D-function IGCM_sys_Rm
940	#D-* Purpose: generic rm
941	#D-* Examples:
942	#D-
943	function IGCM_sys_Rm {
944	IGCM_debug_PushStack "IGCM_sys_Rm" $@
945	if ( $DEBUG_sys ) ; then
946	echo "IGCM_sys_Rm :" $@
947	fi
948
949	typeset status
950
951	echo rm $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
952	\rm $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
953	status=$?
954
955	if [ ${status} -gt 0 ] ; then
956	echo "IGCM_sys_Rm : error code ${status}"
957	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
958	IGCM_debug_Exit "IGCM_sys_Rm"
959	else
960	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
961	fi
962	IGCM_debug_PopStack "IGCM_sys_Rm"
963	}
964
965	#D-#==================================================
966	#D-function IGCM_sys_RmRunDir
967	#D-* Purpose: rm tmpdir (dummy function most of the time batch
968	#D- scheduler will do the job)
969	#D-* Examples:
970	#D-
971	function IGCM_sys_RmRunDir {
972	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
973	if ( $DEBUG_sys ) ; then
974	echo "IGCM_sys_RmRunDir :" $@
975	fi
976
977	typeset status
978
979	echo rm $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
980	\rm $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
981	status=$?
982
983	if [ ${status} -gt 0 ] ; then
984	echo "IGCM_sys_RmRunDir : rm error code is ${status}."
985	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
986	IGCM_debug_Exit "IGCM_sys_RmRunDir"
987	else
988	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
989	fi
990	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
991	}
992
993	#D-#==================================================
994	#D-function IGCM_sys_Mv
995	#D-* Purpose: generic move
996	#D-* Examples:
997	#D-
998	function IGCM_sys_Mv {
999	IGCM_debug_PushStack "IGCM_sys_Mv" $@
1000	if ( $DEBUG_sys ) ; then
1001	echo "IGCM_sys_Mv :" $@
1002	fi
1003
1004	if [ $DRYRUN = 0 ]; then
1005
1006	typeset status
1007
1008	echo mv $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1009	\mv $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1010	status=$?
1011
1012	if [ ${status} -gt 0 ] ; then
1013	echo "IGCM_sys_Mv : error code ${status}"
1014	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1015	IGCM_debug_Exit "IGCM_sys_Mv"
1016	else
1017	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1018	fi
1019	fi
1020	IGCM_debug_PopStack "IGCM_sys_Mv"
1021	}
1022
1023	#D-#==================================================
1024	#D-function IGCM_sys_Put_Dir
1025	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1026	#D-* Examples:
1027	#D-
1028	function IGCM_sys_Put_Dir {
1029	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1030	if ( $DEBUG_sys ) ; then
1031	echo "IGCM_sys_Put_Dir :" $@
1032	fi
1033	if [ $DRYRUN = 0 ]; then
1034	if [ ! -d ${1} ] ; then
1035	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1036	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1037	return
1038	fi
1039
1040	typeset status
1041
1042	# Only if we use rsync
1043	#IGCM_sys_TestDirArchive $( dirname $2 )
1044	#
1045	#USUAL WAY
1046	\cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1047	status=$?
1048
1049	if [ ${status} -gt 0 ] ; then
1050	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
1051	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1052	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1053	else
1054	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1055	fi
1056	fi
1057	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1058	}
1059
1060	#D-#==================================================
1061	#D-function IGCM_sys_Get_Dir
1062	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1063	#D-* Examples:
1064	#D-
1065	function IGCM_sys_Get_Dir {
1066	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1067	if ( $DEBUG_sys ) ; then
1068	echo "IGCM_sys_Get_Dir :" $@
1069	fi
1070	if [ $DRYRUN = 0 ]; then
1071	typeset NB_ESSAI DELAI status i
1072	# number of tentative
1073	NB_ESSAI=3
1074	# time delay between tentative
1075	DELAI=2
1076
1077	#
1078	# USUAL WAY
1079	# add 'ccc_hsm get' (to demigrate all offline files) to reduce time of this command :
1080	ccc_hsm get -r $1
1081
1082	i=0
1083	while [ $i -lt $NB_ESSAI ] ; do
1084	\cp -ur $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1085	status=$?
1086	if [ ${status} -gt 0 ] ; then
1087	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
1088	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1089	sleep $DELAI
1090	else
1091	break
1092	fi
1093	(( i = i + 1 ))
1094	done
1095
1096	if [ ${status} -gt 0 ] ; then
1097	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
1098	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1099	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1100	else
1101	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1102	fi
1103	fi
1104	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1105	}
1106
1107	#D-#==================================================
1108	#D-function IGCM_sys_Get_Master
1109	#D-* Purpose: Copy a complete directory from MASTER filesystem
1110	#D-* Examples:
1111	#D-
1112	function IGCM_sys_Get_Master {
1113	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1114	if ( $DEBUG_sys ) ; then
1115	echo "IGCM_sys_Get_Master :" $@
1116	fi
1117	if [ $DRYRUN = 0 ]; then
1118	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1119	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1120	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1121	return
1122	fi
1123
1124	typeset NB_ESSAI DELAI status i
1125	# number of tentative
1126	NB_ESSAI=3
1127	# time delay between tentative
1128	DELAI=2
1129
1130	i=0
1131	while [ $i -lt $NB_ESSAI ] ; do
1132	\cp -urL $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1133	status=$?
1134	if [ ${status} -gt 0 ]; then
1135	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1136	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1137	sleep $DELAI
1138	else
1139	break
1140	fi
1141	(( i = i + 1 ))
1142	done
1143
1144	if [ ${status} -gt 0 ] ; then
1145	echo "IGCM_sys_Get_Master : error."
1146	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1147	IGCM_debug_Exit "IGCM_sys_Get_Master"
1148	else
1149	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1150	fi
1151	fi
1152	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1153	}
1154
1155	#====================================================
1156	#- Call IGCM_sys_Mirror_libIGCM now !
1157	if ( $MirrorlibIGCM ) ; then
1158	IGCM_sys_Mirror_libIGCM
1159	fi
1160
1161	#D-#==================================================
1162	#D-function IGCM_sys_Put_Rest
1163	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1164	#D- File and target directory must exist.
1165	#D-* Examples:
1166	#D-
1167	function IGCM_sys_Put_Rest {
1168	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1169	if ( $DEBUG_sys ) ; then
1170	echo "IGCM_sys_Put_Rest :" $@
1171	fi
1172	if [ $DRYRUN = 0 ]; then
1173	if [ ! -f ${1} ] ; then
1174	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1175	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1176	fi
1177
1178	typeset status
1179	#
1180	# USUAL WAY
1181	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1182	status=$?
1183
1184	# #RSYNC WITH NETWORK SSH CALL
1185	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1186	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1187
1188	# #RSYNC WITH NFS USE
1189	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1190	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1191
1192	# status=$?
1193	# IGCM_sys_Rsync_out $status
1194
1195	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1196	# (( status=status+$? ))
1197
1198	if [ ${status} -gt 0 ] ; then
1199	IGCM_debug_Print 2 "IGCM_sys_Put_Rest : cp failed error code ${status}"
1200	[ -f ${1} ] && ls -l ${1}
1201	[ -f ${2} ] && ls -l ${2}
1202	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1203	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1204	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1205	else
1206
1207	if [ X${JobType} = XRUN ] ; then
1208	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1209	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1210	fi
1211
1212	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1213	fi
1214	fi
1215	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1216	}
1217
1218	#D-#==================================================
1219	#D-function IGCM_sys_PutBuffer_Rest
1220	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1221	#D- File and target directory must exist.
1222	#D-* Examples:
1223	#D-
1224	function IGCM_sys_PutBuffer_Rest {
1225	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1226	if ( $DEBUG_sys ) ; then
1227	echo "IGCM_sys_PutBuffer_Rest :" $@
1228	fi
1229	if [ $DRYRUN = 0 ]; then
1230	if [ ! -f ${1} ] ; then
1231	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1232	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1233	fi
1234
1235	typeset status
1236	#
1237	# USUAL WAY
1238	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1239	status=$?
1240
1241	if [ ${status} -gt 0 ] ; then
1242	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1243	[ -f ${2} ] && ls -l ${2}
1244	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1245	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1246	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1247	else
1248
1249	if [ X${JobType} = XRUN ] ; then
1250	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1251	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1252	fi
1253
1254	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1255	fi
1256	fi
1257	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1258	}
1259
1260	#D-#==================================================
1261	#D-function IGCM_sys_PrepareTaredRestart
1262	#D-* Purpose: Prepare tared restart to be access by computing job.
1263	#D-* Examples:
1264	#D-
1265	function IGCM_sys_PrepareTaredRestart {
1266	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1267	if [ $DRYRUN = 0 ]; then
1268	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1269	fi
1270	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1271	}
1272
1273	#D-#==================================================
1274	#D-function IGCM_sys_Put_Out
1275	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1276	#D-* Examples:
1277	#D-
1278	function IGCM_sys_Put_Out {
1279	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1280	if ( $DEBUG_sys ) ; then
1281	echo "IGCM_sys_Put_Out :" $@
1282	fi
1283
1284	typeset NB_ESSAI DELAI status i exist skip
1285
1286	# number of tentative
1287	NB_ESSAI=3
1288	# time delay between tentative
1289	DELAI=2
1290
1291	if [ $DRYRUN = 0 ]; then
1292	if [ ! -f ${1} ] ; then
1293	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1294	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1295	return 1
1296	fi
1297	#
1298	IGCM_sys_MkdirArchive $( dirname $2 )
1299	#
1300	exist=false
1301	skip=false
1302	if [ -f $2 ] ; then
1303	IGCM_debug_Print 1 "$2 already exist"
1304	ccc_hsm get $2
1305	exist=true
1306	if [ "X$( diff $1 $2 )" = X ] ; then
1307	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1308	skip=true
1309	else
1310	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1311	skip=false
1312	fi
1313	fi
1314	#
1315	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1316	IGCM_sys_Chmod u+w $2
1317	fi
1318
1319	if [ X${skip} = Xfalse ] ; then
1320	i=0
1321	while [ $i -lt $NB_ESSAI ] ; do
1322	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1323	# USUAL WAY
1324	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1325	status=$?
1326	else
1327	# NOT SO USUAL WAY
1328	\mv $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1329	status=$?
1330	fi
1331	if [ ${status} -gt 0 ]; then
1332	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1333	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
1334	[ -f ${1} ] && ls -l ${1}
1335	[ -f ${2} ] && ls -l ${2}
1336	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1337	sleep $DELAI
1338	else
1339	break
1340	fi
1341	(( i = i + 1 ))
1342	done
1343	fi
1344
1345	# #RSYNC WITH NETWORK SSH CALL
1346	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1347	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1348
1349	# #RSYNC WITH NFS USE
1350	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1351	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1352
1353	# status=$?
1354	# IGCM_sys_Rsync_out $status
1355
1356	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1357	# (( status=status+$? ))
1358
1359	if [ ${status} -gt 0 ] ; then
1360	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status}"
1361	[ -f ${1} ] && ls -l ${1}
1362	[ -f ${2} ] && ls -l ${2}
1363	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1364	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1365	IGCM_debug_Exit "IGCM_sys_Put_Out"
1366	else
1367
1368	if [ X${JobType} = XRUN ] ; then
1369	if [ X${3} = X ] ; then
1370	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1371	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1372	fi
1373	fi
1374
1375	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1376	fi
1377	fi
1378	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1379	return 0
1380	}
1381
1382	#D-#==================================================
1383	#D-function IGCM_sys_PutBuffer_Out
1384	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1385	#D-* Examples:
1386	#D-
1387	function IGCM_sys_PutBuffer_Out {
1388	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1389	if ( $DEBUG_sys ) ; then
1390	echo "IGCM_sys_PutBuffer_Out :" $@
1391	fi
1392
1393	typeset NB_ESSAI DELAI status i exist skip
1394
1395	# number of tentative
1396	NB_ESSAI=3
1397	# time delay between tentative
1398	DELAI=2
1399
1400	if [ $DRYRUN = 0 ]; then
1401	if [ ! -f ${1} ] ; then
1402	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1403	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1404	return 1
1405	fi
1406	#
1407	IGCM_sys_Mkdir $( dirname $2 )
1408	#
1409
1410	exist=false
1411	skip=false
1412	if [ -f $2 ] ; then
1413	IGCM_debug_Print 1 "$2 already exist"
1414	exist=true
1415	if [ "X$( diff $1 $2 )" = X ] ; then
1416	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1417	status=0
1418	skip=true
1419	else
1420	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1421	skip=false
1422	fi
1423	fi
1424	#
1425	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1426	IGCM_sys_Chmod u+w $2
1427	fi
1428
1429	if [ X${skip} = Xfalse ] ; then
1430	i=0
1431	while [ $i -lt $NB_ESSAI ] ; do
1432	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1433	# USUAL WAY
1434	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1435	status=$?
1436	else
1437	# NOT SO USUAL WAY
1438	\mv $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1439	status=$?
1440	fi
1441	if [ ${status} -gt 0 ]; then
1442	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1443	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1444	[ -f ${2} ] && ls -l ${2}
1445	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1446	sleep $DELAI
1447	else
1448	break
1449	fi
1450	(( i = i + 1 ))
1451	done
1452	fi
1453
1454	if [ ${status} -gt 0 ] ; then
1455	echo "IGCM_sys_PutBuffer_Out : error."
1456	[ -f ${2} ] && ls -l ${2}
1457	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1458	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1459	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1460	else
1461
1462	if [ X${JobType} = XRUN ] ; then
1463	if [ X${3} = X ] ; then
1464	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1465	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1466	fi
1467	fi
1468
1469	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1470	fi
1471	fi
1472	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1473	return 0
1474	}
1475
1476	#D-#==================================================
1477	#D-function IGCM_sys_Get
1478	#D-* Purpose: Get a file from ${ARCHIVE}
1479	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1480	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1481	function IGCM_sys_Get {
1482	IGCM_debug_PushStack "IGCM_sys_Get" $@
1483
1484	typeset DEST dm_liste target file_work
1485	typeset NB_ESSAI DELAI status i
1486
1487	if ( $DEBUG_sys ) ; then
1488	echo "IGCM_sys_Get :" $@
1489	fi
1490
1491	# number of tentative
1492	NB_ESSAI=3
1493	# time delay between tentative
1494	DELAI=2
1495
1496	if [ $DRYRUN -le 2 ]; then
1497	if [ X${1} = X'/l' ] ; then
1498	eval set +A dm_liste \${${2}}
1499	else
1500	eval set +A dm_liste ${1}
1501	fi
1502	eval DEST=\${${#}}
1503	ccc_hsm get ${dm_liste[*]} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1504	status=$?
1505	if [ ${status} -gt 0 ] ; then
1506	echo "WARNING IGCM_sys_Get : error code ${status}"
1507	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1508	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
1509	fi
1510
1511	#if [ ${status} -gt 0 ] ; then
1512	# if [ ! "X$( grep "Lost dmusrcmd connection" ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )" = "X" ] ; then
1513	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1514	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
1515	# sleep 30
1516	# echo "We try another time"
1517	## dmget ${dm_liste[*]} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1518	# ccc_hsm get ${dm_liste[*]} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1519	# status=$?
1520	# if [ ${status} -gt 0 ] ; then
1521	# echo "ERROR IGCM_sys_Get : again demigration error :"
1522	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1523	# IGCM_debug_Exit "IGCM_sys_Get"
1524	# fi
1525	# else
1526	# echo "ERROR IGCM_sys_Get : demigration error :"
1527	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1528	# IGCM_debug_Exit "IGCM_sys_Get"
1529	# fi
1530	#fi
1531
1532	# #RSYNC WITH NETWORK SSH CALL
1533	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1534	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1535
1536	# #RSYNC WITH NFS USE
1537	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1538	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1539
1540	# status=$?
1541	# IGCM_sys_Rsync_out $status
1542
1543	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1544	# (( status=status+$? ))
1545
1546	#USUAL WAY
1547	if [ X${1} = X'/l' ] ; then
1548	for target in ${dm_liste[*]} ; do
1549	local_file=$( basename ${target} )
1550	# test if the target file is present before the loop
1551	IGCM_sys_TestFileArchive ${target}
1552	status=$?
1553	if [ ${status} -gt 0 ] ; then
1554	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
1555	IGCM_debug_Exit "IGCM_sys_Get"
1556	else
1557	i=0
1558	while [ $i -lt $NB_ESSAI ] ; do
1559	#if [ X${DoLink} = Xtrue ] ; then
1560	# \ln -s ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1561	# status=$?
1562	# else
1563	# \cp ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1564	# status=$?
1565	#fi
1566	\ln -s ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1567	status=$?
1568	if [ ${status} -gt 0 ]; then
1569	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1570	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1571	sleep $DELAI
1572	else
1573	break
1574	fi
1575	(( i = i + 1 ))
1576	done
1577	if [ ${status} -gt 0 ] ; then
1578	echo "IGCM_sys_Get : error"
1579	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1580	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1581	IGCM_debug_Exit "IGCM_sys_Get"
1582	else
1583	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1584	fi
1585	fi
1586	done
1587	else
1588	i=0
1589	while [ $i -lt $NB_ESSAI ] ; do
1590	\cp ${dm_liste} ${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1591	status=$?
1592	if [ ${status} -gt 0 ]; then
1593	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1594	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1595	sleep $DELAI
1596	else
1597	break
1598	fi
1599	(( i = i + 1 ))
1600	done
1601	if [ ${status} -gt 0 ] ; then
1602	echo "IGCM_sys_Get : error"
1603	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1604	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1605	IGCM_debug_Exit "IGCM_sys_Get"
1606	else
1607	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1608	fi
1609	fi
1610	fi
1611	IGCM_debug_PopStack "IGCM_sys_Get"
1612	}
1613
1614	#D-#==================================================
1615	#D-function IGCM_sys_GetBuffer
1616	#D-* Purpose: Get a file from ${SCRATCHDIR}
1617	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1618	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1619	function IGCM_sys_GetBuffer {
1620	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1621
1622	typeset DEST buf_liste target file_work
1623	typeset NB_ESSAI DELAI status i
1624
1625	if ( $DEBUG_sys ) ; then
1626	echo "IGCM_sys_GetBuffer :" $@
1627	fi
1628
1629	# number of tentative
1630	NB_ESSAI=3
1631	# time delay between tentative
1632	DELAI=2
1633
1634	if [ $DRYRUN -le 2 ]; then
1635	if [ X${1} = X'/l' ] ; then
1636	eval set +A buf_liste \${${2}}
1637	else
1638	eval set +A buf_liste ${1}
1639	fi
1640	eval DEST=\${${#}}
1641
1642	#USUAL WAY
1643	if [ X${1} = X'/l' ] ; then
1644	for target in ${buf_liste[*]} ; do
1645	local_file=$( basename ${target} )
1646	i=0
1647	while [ $i -lt $NB_ESSAI ] ; do
1648	\cp ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1649	status=$?
1650	if [ ${status} -gt 0 ]; then
1651	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1652	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1653	sleep $DELAI
1654	else
1655	break
1656	fi
1657	(( i = i + 1 ))
1658	done
1659	if [ ${status} -gt 0 ] ; then
1660	echo "IGCM_sys_Get : error"
1661	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1662	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1663	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1664	else
1665	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1666	fi
1667	done
1668	else
1669	i=0
1670	while [ $i -lt $NB_ESSAI ] ; do
1671	\cp ${buf_liste} ${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1672	status=$?
1673	if [ ${status} -gt 0 ]; then
1674	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1675	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1676	sleep $DELAI
1677	else
1678	break
1679	fi
1680	(( i = i + 1 ))
1681	done
1682	if [ ${status} -gt 0 ] ; then
1683	echo "IGCM_sys_Get : error"
1684	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1685	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1686	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1687	else
1688	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1689	fi
1690	fi
1691	fi
1692	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1693	}
1694
1695	#D-#==================================================
1696	#D-function IGCM_sys_GetDate_FichWork
1697	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1698	#D-* Examples:
1699	#D-
1700	function IGCM_sys_GetDate_FichWork {
1701	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1702	if ( $DEBUG_sys ) ; then
1703	echo "IGCM_sys_GetDate_FichWork :" $@
1704	fi
1705
1706	if [ $# -ge 3 ] ; then
1707	mode=$3
1708	TimeStyle=$4
1709	else
1710	mode="default"
1711	TimeStyle="%Y%m%d%H%M%S"
1712	fi
1713
1714	typeset dateF
1715	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1716
1717	case $mode in
1718	"default")
1719	eval ${2}=${dateF[5]}
1720	;;
1721	"SplitFields")
1722	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1723	;;
1724	esac
1725
1726
1727	# donne la date filesys d'un fichier sur la machine work
1728	IGCM_debug_PopStack "IGCM_sys_FichWork"
1729	}
1730
1731	#D-#==================================================
1732	#D-function IGCM_sys_GetDate_FichArchive
1733	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1734	#D-* Examples:
1735	#D-
1736	function IGCM_sys_GetDate_FichArchive {
1737	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1738	if ( $DEBUG_sys ) ; then
1739	echo "IGCM_sys_GetDate_FichArchive :" $@
1740	fi
1741	typeset dateF
1742	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1743	eval ${2}=${dateF[5]}
1744
1745	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1746	}
1747
1748	#D-#==================================================
1749	#D-function IGCM_sys_GetDate_Monitoring
1750	#D-* Purpose: get the last year for which the monitoring has been computed
1751	#D-* Examples:
1752	#D-
1753	function IGCM_sys_GetDate_Monitoring {
1754	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1755	if ( $DEBUG_sys ) ; then
1756	echo "IGCM_sys_GetDate_Monitoring :" $@
1757	fi
1758
1759	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1760
1761	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1762	}
1763
1764	#D-#==================================================
1765	#D-function IGCM_sys_Dods_Rm
1766	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1767	#D-* Examples:
1768	#D-
1769	function IGCM_sys_Dods_Rm {
1770	if ( $DEBUG_sys ) ; then
1771	echo "IGCM_sys_Dods_Rm :" $@
1772	fi
1773	typeset status
1774	if [ $DRYRUN = 0 ]; then
1775
1776	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
1777	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
1778	# echo "Nothing has been done."
1779	# return
1780	# fi
1781
1782	/ccc/cont003/home/dsm/p86ipsl/bin/dods_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
1783	status=$?
1784
1785	# if [ ${status} -gt 0 ] ; then
1786	# echo "IGCM_sys_Dods_Rm : error."
1787	# cat out_dods_rm
1788	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
1789	# else
1790	# rm out_dods_rm
1791	# fi
1792
1793	fi
1794	return $status
1795	}
1796
1797	#D-#==================================================
1798	#D-function IGCM_sys_Dods_Cp
1799	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1800	#D-* Examples:
1801	#D-
1802	function IGCM_sys_Dods_Cp {
1803	if ( $DEBUG_sys ) ; then
1804	echo "IGCM_sys_Dods_Cp :" $@
1805	fi
1806	typeset status
1807	if [ $DRYRUN = 0 ]; then
1808
1809	# if [ ! -d ${R_SAVE}/${1} ] ; then
1810	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
1811	# echo "Nothing has been done."
1812	# return
1813	# fi
1814
1815	/ccc/cont003/home/dsm/p86ipsl/bin/dods_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
1816	status=$?
1817
1818	# if [ ${status} -gt 0 ] ; then
1819	# echo "IGCM_sys_Dods_Cp : error."
1820	# cat out_dods_cp
1821	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
1822	# else
1823	# rm out_dods_cp
1824	# fi
1825
1826	fi
1827	return $status
1828	}
1829
1830	#D-#==================================================
1831	#D-function IGCM_sys_Put_Dods
1832	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole.
1833	#D-* Examples:
1834	#D-
1835	function IGCM_sys_Put_Dods {
1836	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1837	if ( $DEBUG_sys ) ; then
1838	echo "IGCM_sys_Put_Dods :" $@
1839	fi
1840	typeset status
1841	if [ $DRYRUN = 0 ]; then
1842	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
1843	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
1844	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
1845	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
1846	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1847	return
1848	fi
1849
1850	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1851	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1852	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1853	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1854	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1855	return
1856	fi
1857	#
1858	if [ -d ${R_SAVE}/${1} ] ; then
1859	cd ${R_SAVE}
1860	elif [ -d ${R_FIGR}/${1} ] ; then
1861	cd ${R_FIGR}
1862	fi
1863
1864	IGCM_sys_Dods_Rm ${1}
1865	IGCM_sys_Dods_Cp ${1}
1866	status=0
1867
1868	if [ ${status} -gt 0 ] ; then
1869	echo "IGCM_sys_Put_Dods : error."
1870	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1871	fi
1872	fi
1873	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1874	}
1875
1876	##############################################################
1877	# REBUILD OPERATOR
1878
1879	#D-#==================================================
1880	#D-function IGCM_sys_sync
1881	#D-* Purpose: flush buffer on disk
1882	#D-* Examples:
1883	#D-
1884	function IGCM_sys_sync {
1885	IGCM_debug_PushStack "IGCM_sys_sync" $@
1886	if ( $DEBUG_sys ) ; then
1887	echo "IGCM_sys_sync :" $@
1888	fi
1889
1890	/bin/sync
1891
1892	IGCM_debug_PopStack "IGCM_sys_sync"
1893	}
1894
1895	#D-#==================================================
1896	#D-function IGCM_sys_rebuild
1897	#D-* Purpose: rebuild parallel files
1898	#D-* Examples:
1899	#D-
1900	function IGCM_sys_rebuild {
1901	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1902	if ( $DEBUG_sys ) ; then
1903	echo "IGCM_sys_rebuild :" $@
1904	fi
1905
1906	typeset NB_ESSAI DELAI status i firstArg
1907	# number of tentative
1908	NB_ESSAI=3
1909	# time delay between tentative
1910	DELAI=2
1911
1912	i=0
1913	while [ $i -lt $NB_ESSAI ] ; do
1914	rebuild -f -o $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1915	status=$?
1916	if [ ${status} -gt 0 ] ; then
1917	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1918	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1919	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1920	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1921	firstArg=${1}
1922	\rm ${firstArg}
1923	sleep $DELAI
1924	else
1925	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1926	break
1927	fi
1928	(( i = i + 1 ))
1929	done
1930
1931	if [ ${status} -gt 0 ] ; then
1932	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1933	IGCM_debug_Exit "rebuild"
1934	fi
1935
1936	IGCM_debug_PopStack "IGCM_sys_rebuild"
1937	}
1938
1939	#D-#==================================================
1940	#D-function IGCM_sys_rebuild_station
1941	#D-* Purpose: rebuild parallel files describing station
1942	#D-* Examples:
1943	#D-
1944	function IGCM_sys_rebuild_station {
1945	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1946	typeset i list_opt file_in file_out prefix_invert list_invert
1947	if ( $DEBUG_sys ) ; then
1948	echo "IGCM_sys_rebuild_station :" $@
1949	fi
1950	list_opt=$@
1951
1952	# Invert Axis : t,x -> x,t
1953	# t,pres,x -> x,t,pres
1954	# So that we can concatenate along x
1955	i=0
1956	for file_in in ${list_opt} ; do
1957	(( i = i + 1))
1958	[ ${i} = 1 ] && file_out=${file_in} && continue
1959	# detect time counter and do the job only if present
1960	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
1961	if [ X${var_unlim} = Xtime_counter ] ; then
1962	prefix_invert=$( basename ${file_in} .nc )
1963	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1964	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1965	fi
1966	done
1967
1968	# Concatenate
1969	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1970
1971	# Re-ivert file
1972	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1973
1974	# Station re-ordering is too expansive to be run within libIGCM
1975	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
1976	# This re-ordering must be done "in memory" by the cmorization process
1977	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
1978	# BEGIN reordering
1979
1980	# Only LMDZ text output contains the exact ordering of the station.
1981	# We isolate this in the code below:
1982	# 0 38 -157.5000000000000 70.98591549295774
1983	# 0 54 27.49999999999999 67.18309859154928
1984	# 0 56 -62.50000000000001 82.39436619718309
1985	# 0 79 12.49999999999999 78.59154929577466
1986	# 0 116 -165.0000000000000 76.05633802816901
1987	# 0 117 130.0000000000000 70.98591549295774
1988	# 0 118 110.0000000000000 87.46478873239437
1989	# 1 40 4.999999999999995 51.97183098591550
1990
1991	# typeset iStation iProc list_opt file_in file_out prefix_invert
1992	# typeset -Z4 j4
1993	# typeset -Z3 j3
1994
1995	# unset list_opt
1996	# set +A list_opt $@
1997
1998	# Filename after rebuild
1999	# file_out=${list_opt[0]}
2000	# Prefix of output files
2001	# prefix_invert=$( basename ${file_out} .nc )
2002	# Number of procs
2003	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
2004
2005	# iProc=0
2006	# while [ ${iProc} -lt ${num_proc} ] ; do
2007	# Array containing Station as a number
2008	# unset proc_stn
2009	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
2010	# Number of stations produced by processor proc
2011	# stationLast=${#proc_stn[*]}
2012	# Proc number on 4 digits
2013	# j4=${iProc}
2014	# Init
2015	# iStation=0
2016	# while [ ${iStation} -lt ${stationLast} ] ; do
2017	# Station number on 3 digits
2018	# j3=${proc_stn[${iStation}]}
2019	# Extract station
2020	# Invert Axis : t,x -> x,t
2021	# t,pres,x -> x,t,pres
2022	# So that we can concatenate along x
2023	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
2024	# (( iStation = iStation + 1 ))
2025	# done
2026	# (( iProc = iProc + 1 ))
2027	# done
2028
2029	# Concatenate all station along x
2030	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
2031
2032	# Re-invert file
2033	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
2034
2035	# END reordering
2036
2037	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
2038	}
2039
2040	############################################################
2041	# Activate Running Environnment Variables
2042
2043	#D-#==================================================
2044	#D-function IGCM_sys_desactiv_variables
2045	#D-* Purpose: set environement variables prior to execution
2046	#D-* Examples:
2047	#D-
2048	function IGCM_sys_activ_variables {
2049	IGCM_debug_PushStack "IGCM_sys_activ_variables"
2050	if ( $DEBUG_sys ) ; then
2051	echo "IGCM_sys_activ_variables"
2052	fi
2053
2054	# --------------------------------------------------------------------
2055	#D- MPI specifications
2056	# --------------------------------------------------------------------
2057	export OMPI_MCA_paffinity_hwloc_priority=41
2058	# --------------------------------------------------------------------
2059	#D- Other specifications
2060	# --------------------------------------------------------------------
2061
2062	ulimit -s unlimited
2063
2064	IGCM_debug_PopStack "IGCM_sys_activ_variables"
2065	}
2066
2067	############################################################
2068	# Desactivate Running Environnment Variables
2069
2070	#D-#==================================================
2071	#D-function IGCM_sys_desactiv_variables
2072	#D-* Purpose: unset environement variables after execution
2073	#D-* Examples:
2074	#D-
2075	function IGCM_sys_desactiv_variables {
2076	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
2077	if ( $DEBUG_sys ) ; then
2078	echo "IGCM_sys_desactiv_variables"
2079	fi
2080	# --------------------------------------------------------------------
2081	#D- MPI specifications
2082	# --------------------------------------------------------------------
2083
2084	# --------------------------------------------------------------------
2085	#D- Other specifications
2086	# --------------------------------------------------------------------
2087
2088	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
2089	}
2090
2091	############################################################
2092	# Build MPI/OMP scripts run file (dummy function)
2093
2094	#D-#==================================================
2095	#D-function IGCM_sys_build_run_file
2096	#D-* Purpose: build run file (deprecated)
2097	#D-* Examples:
2098	#D-
2099	function IGCM_sys_build_run_file {
2100
2101	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
2102
2103	}
2104
2105	############################################################
2106	# Build MPI/OMP scripts
2107
2108	#D-#==================================================
2109	#D-function IGCM_sys_build_execution_scripts
2110	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
2111	#D-* Examples:
2112	#D-
2113	function IGCM_sys_build_execution_scripts
2114	{
2115	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
2116	if ( $DEBUG_sys ) ; then
2117	echo "IGCM_sys_build_execution_scripts " $@
2118	fi
2119
2120	EXECUTION=${HOST_MPIRUN_COMMAND}
2121
2122	# MPMD mode
2123	if ( ${OK_PARA_MPMD} ) ; then
2124
2125	# Only MPI (MPMD)
2126	if ( ! ${OK_PARA_OMP} ) ; then
2127
2128	if [ -f run_file ] ; then
2129	IGCM_sys_Rm -f run_file
2130	fi
2131	touch run_file
2132
2133	# Build run_file
2134
2135	# First loop on the components for the coupler ie oasis (only if oasis3)
2136	# the coupler ie oasis3 must be the first one
2137	for comp in ${config_ListOfComponents[*]} ; do
2138
2139	eval ExeNameIn=\${config_Executable_${comp}[0]}
2140	eval ExeNameOut=\${config_Executable_${comp}[1]}
2141
2142	# for CPL component only
2143	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
2144
2145	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2146	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2147	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2148	fi
2149	done
2150
2151	# Then second loop on the components
2152	for comp in ${config_ListOfComponents[*]} ; do
2153
2154	eval ExeNameIn=\${config_Executable_${comp}[0]}
2155	eval ExeNameOut=\${config_Executable_${comp}[1]}
2156
2157	# Only if we really have an executable for the component and not the coupler ie oasis:
2158	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2159
2160	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2161	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2162	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2163	fi
2164	done
2165
2166	EXECUTION="${HOST_MPIRUN_COMMAND} -f ./run_file"
2167
2168	IGCM_sys_Chmod u+x run_file
2169	if ( $DEBUG_sys ) ; then
2170	echo "run_file contains : "
2171	cat run_file
2172	fi
2173
2174	# MPI-OpenMP (MPMD)
2175	else
2176
2177	# Use of mpirun instead of ccc_mprun
2178	EXECUTION="time mpirun"
2179
2180	# Hosts treatment
2181	${EXECUTION} hostname \| sort \| uniq > hosts.tmp
2182
2183	i=0
2184	rm -f hosts rankfile
2185	IGCM_debug_Print 1 "sys Curie, Hosts available :"
2186	for nodes in `cat hosts.tmp`
2187	do
2188	host[$i]=$nodes
2189	echo "${host[$i]}" >> hosts
2190	IGCM_debug_Print 1 ${host[$i]}
2191	i=$((i+1))
2192	done
2193	rm -f hosts.tmp
2194
2195	listnodes=${host[*]}
2196
2197	EXECUTION="${EXECUTION} -hostfile hosts -rankfile rankfile"
2198
2199	# Initialisation
2200	rank=0
2201	current_core=0
2202	core_per_node=16
2203	init_exec=n
2204
2205	# Loop on the components
2206	for comp in ${config_ListOfComponents[*]} ; do
2207
2208	eval ExeNameIn=\${config_Executable_${comp}[0]}
2209	eval ExeNameOut=\${config_Executable_${comp}[1]}
2210
2211	# Not possible if oasis has an executable (i.e old version of oasis3)
2212	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
2213	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
2214	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
2215	IGCM_debug_Verif_Exit
2216	fi
2217
2218	# Only if we really have an executable for the component :
2219	if [ "X${ExeNameOut}" != X\"\" ] ; then
2220
2221	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2222	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2223
2224	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2225	echo "" >> script_${ExeNameOut}.ksh
2226	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
2227
2228	# Check if the number of threads is correct
2229	case ${comp_proc_omp_loc} in
2230	2\|4\|8\|16)
2231	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2232	;;
2233	*)
2234	IGCM_debug_Exit "ERROR with OMP parameters !"
2235	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2236	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2237	IGCM_debug_Verif_Exit
2238	;;
2239	esac
2240	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2241	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2242	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2243	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2244	fi
2245	#echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
2246	#echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
2247	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2248	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2249
2250	if [ ${init_exec} = y ] ; then
2251	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2252	else
2253	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2254	init_exec=y
2255	fi
2256
2257	# Build rankfile : method used to assign cores and nodes for the MPI process
2258	# Ex :
2259	#rank 0=curie5296 slot=0,2,4,6
2260	#rank 1=curie5296 slot=8,10,12,14
2261	# Example of final command :
2262	# mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
2263	# with script_lmdz.x.ksh :
2264	# #!/bin/ksh
2265	#export KMP_STACKSIZE=3g
2266	#export KMP_LIBRARY=turnaround
2267	#export MKL_SERIAL=YES
2268	#OMP_NUM_THREADS=4
2269	#./lmdz.x
2270	#
2271
2272	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
2273	(( index_host = current_core / core_per_node ))
2274	host_value=${host[${index_host}]}
2275	(( slot = current_core % core_per_node ))
2276	virg=","
2277	string_final=""
2278	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
2279	(( index_slot = index * 2 ))
2280	string=$index_slot$virg
2281	string_final=$string_final$string
2282	done
2283	string_final=$( echo $string_final \| sed "s/.$//" )
2284	echo "rank $rank=$host_value slot=$string_final" >> rankfile
2285	(( rank = rank + 1 ))
2286	(( current_core = current_core + comp_proc_omp_loc ))
2287	done
2288	fi
2289
2290	done
2291	fi
2292
2293	# Only one executable (SPMD mode).
2294	else
2295
2296	for comp in ${config_ListOfComponents[*]} ; do
2297
2298	# Only if we really have an executable for the component :
2299	eval ExeNameOut=\${config_Executable_${comp}[1]}
2300	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2301
2302	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2303	echo "" >> script_${ExeNameOut}.ksh
2304	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2305
2306	if ( ${OK_PARA_OMP} ) ; then
2307	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2308	echo "" >> script_${ExeNameOut}.ksh
2309	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2310	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2311	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2312	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2313	fi
2314
2315	if ( ${OK_PARA_MPI} ) ; then
2316	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2317	# Default : ccc_mprun used if nb_proc gt 1
2318	# to have out/err per process on different files
2319	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
2320	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2321	EXECUTION="${HOST_MPIRUN_COMMAND} -n ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2322	else
2323	# Default : ccc_mprun is NOT used if nb_proc eq 1
2324	# to have out/err per process on different files
2325	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2326	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2327	EXECUTION="/usr/bin/time ./script_${ExeNameOut}.ksh"
2328	fi
2329
2330	IGCM_debug_Print 1 "sys Curie : script_${ExeNameOut}.ksh contains"
2331	cat script_${ExeNameOut}.ksh
2332
2333	fi
2334
2335	done
2336
2337	fi
2338
2339	IGCM_debug_Print 1 "sys Curie : execution command is "
2340	IGCM_debug_Print 1 "$EXECUTION"
2341
2342	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2343	}
2344
2345	#D-#==================================================
2346	#D-function IGCM_sys_check_path
2347	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2348	#D-* do not point to an important use directory. Stop immediately in that case.
2349	#D-* Examples:
2350	#D-
2351	function IGCM_sys_check_path {
2352	IGCM_debug_PushStack "IGCM_sys_check_path"
2353	if ( $DEBUG_sys ) ; then
2354	echo "IGCM_sys_check_path"
2355	fi
2356
2357	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCHDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCWORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCSTOREDIR} ] ) ; then
2358	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2359	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2360	IGCM_debug_Exit "This will stop the job"
2361	fi
2362	IGCM_debug_PopStack "IGCM_sys_check_path"
2363	}
2364
2365	#D-#==================================================
2366	#D-function IGCM_sys_check_quota
2367	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2368	#D-* Examples:
2369	#D-
2370	function IGCM_sys_check_quota {
2371	IGCM_debug_PushStack "IGCM_sys_check_quota"
2372	if ( $DEBUG_sys ) ; then
2373	echo "IGCM_sys_check_quota"
2374	fi
2375	# Limit of quota (in %)
2376	limit_quota=90
2377
2378	# Check of the volume
2379	volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
2380	volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
2381
2382	if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
2383
2384	unit_avail=${volume_avail: -1}
2385	unit_quota=${volume_quota: -1}
2386
2387	if [ "${unit_quota}" = "*" ] ; then
2388	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2389	IGCM_debug_Print 1 "More than 100% of your quota is used"
2390	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2391	IGCM_debug_Print 1 "You must have more than 10% available to run"
2392	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2393	IGCM_debug_Verif_Exit
2394	fi
2395
2396	temp_avail=${volume_avail%%${volume_avail: -1}*}
2397	temp_quota=${volume_quota%%${volume_quota: -1}*}
2398
2399	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2400
2401	# Convertion
2402	if [ ${volume_avail: -1} = "T" ] ; then
2403	(( temp_avail = temp_avail * 1000000000000 ))
2404	elif [ ${volume_avail: -1} = "G" ] ; then
2405	(( temp_avail = temp_avail * 1000000000 ))
2406	elif [ ${volume_avail: -1} = "M" ] ; then
2407	(( temp_avail = temp_avail * 1000000 ))
2408	elif [ ${volume_avail: -1} = "k" ] ; then
2409	(( temp_avail = temp_avail * 1000 ))
2410	else
2411	(( temp_avail = volume_avail ))
2412	fi
2413	if [ ${volume_quota: -1} = "T" ] ; then
2414	(( temp_quota = temp_quota * 1000000000000 ))
2415	elif [ ${volume_quota: -1} = "G" ] ; then
2416	(( temp_quota = temp_quota * 1000000000 ))
2417	elif [ ${volume_quota: -1} = "M" ] ; then
2418	(( temp_quota = temp_quota * 1000000 ))
2419	elif [ ${volume_quota: -1} = "k" ] ; then
2420	(( temp_quota = temp_quota * 1000 ))
2421	else
2422	(( temp_quota = volume_quota ))
2423	fi
2424	fi
2425
2426	quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2427	# echo "volume ratio is " $quota_volume
2428
2429	if [ ${quota_volume} -ge ${limit_quota} ] ; then
2430	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2431	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2432	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2433	IGCM_debug_Print 1 "You must have more than 10% available to run"
2434	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2435	IGCM_debug_Verif_Exit
2436	fi
2437
2438	fi
2439
2440	# Check of the number of inodes
2441
2442	inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
2443	inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
2444
2445	if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
2446
2447	unit_avail=${inode_avail: -1}
2448	unit_quota=${inode_quota: -1}
2449
2450	if [ "${unit_quota}" = "*" ] ; then
2451	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2452	IGCM_debug_Print 1 "More than 100% of your quota is used"
2453	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2454	IGCM_debug_Print 1 "You must have more than 10% available to run"
2455	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2456	IGCM_debug_Verif_Exit
2457	fi
2458
2459	temp_avail=${inode_avail%%${inode_avail: -1}*}
2460	temp_quota=${inode_quota%%${inode_quota: -1}*}
2461
2462	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2463
2464	# Convertion
2465	if [ ${inode_avail: -1} = "T" ] ; then
2466	(( temp_avail = temp_avail * 1000000000000 ))
2467	elif [ ${inode_avail: -1} = "G" ] ; then
2468	(( temp_avail = temp_avail * 1000000000 ))
2469	elif [ ${inode_avail: -1} = "M" ] ; then
2470	(( temp_avail = temp_avail * 1000000 ))
2471	elif [ ${inode_avail: -1} = "k" ] ; then
2472	(( temp_avail = temp_avail * 1000 ))
2473	else
2474	(( temp_avail = inode_avail ))
2475	fi
2476
2477	if [ ${inode_quota: -1} = "T" ] ; then
2478	(( temp_quota = temp_quota * 1000000000000 ))
2479	elif [ ${inode_quota: -1} = "G" ] ; then
2480	(( temp_quota = temp_quota * 1000000000 ))
2481	elif [ ${inode_quota: -1} = "M" ] ; then
2482	(( temp_quota = temp_quota * 1000000 ))
2483	elif [ ${inode_quota: -1} = "k" ] ; then
2484	(( temp_quota = temp_quota * 1000 ))
2485	else
2486	(( temp_quota = inode_quota ))
2487	fi
2488	fi
2489	quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2490	# echo "inode ratio is " $quota_inode
2491
2492	if [ ${quota_inode} -ge ${limit_quota} ] ; then
2493	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2494	IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
2495	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2496	IGCM_debug_Print 1 "You must have more than 10% available to run"
2497	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2498	IGCM_debug_Verif_Exit
2499	fi
2500	fi
2501	IGCM_debug_PopStack "IGCM_sys_check_quota"
2502	}
2503
2504	#D-#==================================================
2505	#D-function IGCM_sys_GetJobID
2506	#D-* Purpose: Check if job_name is currently
2507	#D- running or in queue
2508	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2509	#D-
2510	function IGCM_sys_GetJobID {
2511	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2512	if ( $DEBUG_sys ) ; then
2513	echo "IGCM_sys_GetJobID"
2514	fi
2515
2516	# With -f option, the full job name is given in the last column
2517	ID=$( ccc_mstat -f -u $2 \| \
2518	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2519
2520	eval ${3}=${ID}
2521	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2522	}
2523
2524	#D-#==================================================
2525	#D-function IGCM_sys_CountJobInQueue
2526	#D-* Purpose: Check if job_name is currently
2527	#D- running or in queue
2528	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2529	#D-
2530	function IGCM_sys_CountJobInQueue {
2531	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2532	if ( $DEBUG_sys ) ; then
2533	echo "IGCM_sys_CountJobInQueue"
2534	fi
2535
2536	# With -f option, the full job name is given in the last column
2537	NbRun=$( ccc_mstat -f \| gawk -v JobName=$1 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
2538
2539	eval ${2}=${NbRun}
2540
2541	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2542	}
2543
2544	#D-#==================================================
2545	#D-function IGCM_sys_ListJobInQueue
2546	#D-* Purpose: Check if job_name is currently
2547	#D- running or in queue
2548	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2549	#D-
2550	function IGCM_sys_ListJobInQueue {
2551	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2552	if ( $DEBUG_sys ) ; then
2553	echo "IGCM_sys_ListJobInQueue"
2554	fi
2555
2556	# With -f option, the full job name is given in the last column
2557	set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
2558	'( $2 == User && \
2559	$NF != /TS/ && \
2560	$NF !~ /PACK/ && \
2561	$NF !~ /REBUILD/ && \
2562	$NF !~ /pack/ ) \
2563	{ print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
2564
2565	eval set -A ${2} ${JobList[*]}
2566
2567	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2568	}
2569
2570	##############################################################
2571	# NCO OPERATOR
2572
2573	#D-#==================================================
2574	#D-function IGCM_sys_ncap2
2575	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2576	#D-* Examples:
2577	#D-
2578	function IGCM_sys_ncap2 {
2579	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2580	if ( $DEBUG_sys ) ; then
2581	echo "IGCM_sys_ncap2 :" $@
2582	fi
2583
2584	typeset NB_ESSAI DELAI status i
2585	# number of tentative
2586	NB_ESSAI=3
2587	# time delay between tentative
2588	DELAI=2
2589
2590	i=0
2591	while [ $i -lt $NB_ESSAI ] ; do
2592	ncap2 -C "$@" > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2593	status=$?
2594	if [ ${status} -gt 0 ] ; then
2595	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2596	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2597	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2598	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2599	sleep $DELAI
2600	else
2601	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2602	break
2603	fi
2604	(( i = i + 1 ))
2605	done
2606
2607	if [ ${status} -gt 0 ] ; then
2608	echo "IGCM_sys_ncap2 : ncap2 error"
2609	IGCM_debug_Exit "ncap2"
2610	fi
2611
2612	IGCM_debug_PopStack "IGCM_sys_ncap2"
2613	}
2614
2615	#D-#==================================================
2616	#D-function IGCM_sys_ncatted
2617	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2618	#D-* Examples:
2619	#D-
2620	function IGCM_sys_ncatted {
2621	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2622	if ( $DEBUG_sys ) ; then
2623	echo "IGCM_sys_ncatted :" $@
2624	fi
2625
2626	typeset NB_ESSAI DELAI status i
2627	# number of tentative
2628	NB_ESSAI=3
2629	# time delay between tentative
2630	DELAI=2
2631
2632	i=0
2633	while [ $i -lt $NB_ESSAI ] ; do
2634	ncatted "$@" > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2635	status=$?
2636	if [ ${status} -gt 0 ] ; then
2637	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2638	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2639	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2640	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2641	sleep $DELAI
2642	else
2643	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2644	break
2645	fi
2646	(( i = i + 1 ))
2647	done
2648
2649	if [ ${status} -gt 0 ] ; then
2650	echo "IGCM_sys_ncatted : ncatted error"
2651	IGCM_debug_Exit "ncatted"
2652	fi
2653
2654	IGCM_debug_PopStack "IGCM_sys_ncatted"
2655	}
2656
2657	#D-#==================================================
2658	#D-function IGCM_sys_ncbo
2659	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2660	#D-* Examples:
2661	#D-
2662	function IGCM_sys_ncbo {
2663	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2664	if ( $DEBUG_sys ) ; then
2665	echo "IGCM_sys_ncbo :" $@
2666	fi
2667
2668	typeset NB_ESSAI DELAI status i
2669	# number of tentative
2670	NB_ESSAI=3
2671	# time delay between tentative
2672	DELAI=2
2673
2674	i=0
2675	while [ $i -lt $NB_ESSAI ] ; do
2676	ncbo -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2677	status=$?
2678	if [ ${status} -gt 0 ] ; then
2679	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2680	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2681	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2682	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2683	sleep $DELAI
2684	else
2685	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2686	break
2687	fi
2688	(( i = i + 1 ))
2689	done
2690
2691	if [ ${status} -gt 0 ] ; then
2692	echo "IGCM_sys_ncbo : ncbo error"
2693	IGCM_debug_Exit "ncbo"
2694	fi
2695
2696	IGCM_debug_PopStack "IGCM_sys_ncbo"
2697	}
2698
2699	#D-#==================================================
2700	#D-function IGCM_sys_ncdif
2701	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2702	#D-* Examples:
2703	#D-
2704	function IGCM_sys_ncdiff {
2705	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2706	if ( $DEBUG_sys ) ; then
2707	echo "IGCM_sys_ncdiff :" $@
2708	fi
2709
2710	typeset NB_ESSAI DELAI status i
2711	# number of tentative
2712	NB_ESSAI=3
2713	# time delay between tentative
2714	DELAI=2
2715
2716	i=0
2717	while [ $i -lt $NB_ESSAI ] ; do
2718	ncdiff -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2719	status=$?
2720	if [ ${status} -gt 0 ] ; then
2721	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2722	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2723	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2724	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2725	sleep $DELAI
2726	else
2727	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2728	break
2729	fi
2730	(( i = i + 1 ))
2731	done
2732
2733	if [ ${status} -gt 0 ] ; then
2734	echo "IGCM_sys_ncdiff : ncdiff error"
2735	IGCM_debug_Exit "ncdiff"
2736	fi
2737
2738	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2739	}
2740
2741	#D-#==================================================
2742	#D-function IGCM_sys_ncea
2743	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2744	#D-* Examples:
2745	#D-
2746	function IGCM_sys_ncea {
2747	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2748	if ( $DEBUG_sys ) ; then
2749	echo "IGCM_sys_ncea :" $@
2750	fi
2751
2752	typeset NB_ESSAI DELAI status i
2753	# number of tentative
2754	NB_ESSAI=3
2755	# time delay between tentative
2756	DELAI=2
2757
2758	i=0
2759	while [ $i -lt $NB_ESSAI ] ; do
2760	ncea -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2761	status=$?
2762	if [ ${status} -gt 0 ] ; then
2763	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2764	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2765	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2766	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2767	sleep $DELAI
2768	else
2769	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2770	break
2771	fi
2772	(( i = i + 1 ))
2773	done
2774
2775	if [ ${status} -gt 0 ] ; then
2776	echo "IGCM_sys_ncea : ncea error"
2777	IGCM_debug_Exit "ncea"
2778	fi
2779
2780	IGCM_debug_PopStack "IGCM_sys_ncea"
2781	}
2782
2783	#D-#==================================================
2784	#D-function IGCM_sys_ncecat
2785	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2786	#D-* Examples:
2787	#D-
2788	function IGCM_sys_ncecat {
2789	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2790	if ( $DEBUG_sys ) ; then
2791	echo "IGCM_sys_ncecat :" $@
2792	fi
2793
2794	typeset NB_ESSAI DELAI status i
2795	# number of tentative
2796	NB_ESSAI=3
2797	# time delay between tentative
2798	DELAI=2
2799
2800	i=0
2801	while [ $i -lt $NB_ESSAI ] ; do
2802	ncecat -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2803	status=$?
2804	if [ ${status} -gt 0 ] ; then
2805	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2806	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2807	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2808	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2809	sleep $DELAI
2810	else
2811	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2812	break
2813	fi
2814	(( i = i + 1 ))
2815	done
2816
2817	if [ ${status} -gt 0 ] ; then
2818	echo "IGCM_sys_ncecat : ncecat error"
2819	IGCM_debug_Exit "ncecat"
2820	fi
2821
2822	IGCM_debug_PopStack "IGCM_sys_ncecat"
2823	}
2824
2825	#D-#==================================================
2826	#D-function IGCM_sys_ncflint
2827	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2828	#D-* Examples:
2829	#D-
2830	function IGCM_sys_ncflint {
2831	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2832	if ( $DEBUG_sys ) ; then
2833	echo "IGCM_sys_ncflint :" $@
2834	fi
2835
2836	typeset NB_ESSAI DELAI status i
2837	# number of tentative
2838	NB_ESSAI=3
2839	# time delay between tentative
2840	DELAI=2
2841
2842	i=0
2843	while [ $i -lt $NB_ESSAI ] ; do
2844	ncflint -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2845	status=$?
2846	if [ ${status} -gt 0 ] ; then
2847	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2848	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2849	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2850	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2851	sleep $DELAI
2852	else
2853	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2854	break
2855	fi
2856	(( i = i + 1 ))
2857	done
2858
2859	if [ ${status} -gt 0 ] ; then
2860	echo "IGCM_sys_ncflint : ncflint error"
2861	IGCM_debug_Exit "ncflint"
2862	fi
2863
2864	IGCM_debug_PopStack "IGCM_sys_ncflint"
2865	}
2866
2867	#D-#==================================================
2868	#D-function IGCM_sys_ncks
2869	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2870	#D-* Examples:
2871	#D-
2872	function IGCM_sys_ncks {
2873	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2874	if ( $DEBUG_sys ) ; then
2875	echo "IGCM_sys_ncks :" $@
2876	fi
2877
2878	typeset NB_ESSAI DELAI status i
2879	# number of tentative
2880	NB_ESSAI=3
2881	# time delay between tentative
2882	DELAI=2
2883
2884	i=0
2885	while [ $i -lt $NB_ESSAI ] ; do
2886	ncks -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2887	status=$?
2888	if [ ${status} -gt 0 ] ; then
2889	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2890	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2891	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2892	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2893	sleep $DELAI
2894	else
2895	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2896	break
2897	fi
2898	(( i = i + 1 ))
2899	done
2900
2901	if [ ${status} -gt 0 ] ; then
2902	echo "IGCM_sys_ncks : ncks error"
2903	IGCM_debug_Exit "ncks"
2904	fi
2905
2906	IGCM_debug_PopStack "IGCM_sys_ncks"
2907	}
2908
2909	#D-#==================================================
2910	#D-function IGCM_sys_ncpdq
2911	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2912	#D-* Examples:
2913	#D-
2914	function IGCM_sys_ncpdq {
2915	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2916	if ( $DEBUG_sys ) ; then
2917	echo "IGCM_sys_ncpdq :" $@
2918	fi
2919
2920	typeset NB_ESSAI DELAI status i
2921	# number of tentative
2922	NB_ESSAI=3
2923	# time delay between tentative
2924	DELAI=2
2925
2926	i=0
2927	while [ $i -lt $NB_ESSAI ] ; do
2928	ncpdq -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2929	status=$?
2930	if [ ${status} -gt 0 ] ; then
2931	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2932	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2933	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2934	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2935	sleep $DELAI
2936	else
2937	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2938	break
2939	fi
2940	(( i = i + 1 ))
2941	done
2942
2943	if [ ${status} -gt 0 ] ; then
2944	echo "IGCM_sys_ncpdq : ncpdq error"
2945	IGCM_debug_Exit "ncpdq"
2946	fi
2947
2948	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2949	}
2950
2951	#D-#==================================================
2952	#D-function IGCM_sys_ncra
2953	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2954	#D-* Examples:
2955	#D-
2956	function IGCM_sys_ncra {
2957	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2958	if ( $DEBUG_sys ) ; then
2959	echo "IGCM_sys_ncra :" $@
2960	fi
2961
2962	typeset NB_ESSAI DELAI status i
2963	# number of tentative
2964	NB_ESSAI=3
2965	# time delay between tentative
2966	DELAI=2
2967
2968	i=0
2969	while [ $i -lt $NB_ESSAI ] ; do
2970	ncra -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
2971	status=$?
2972	if [ ${status} -gt 0 ] ; then
2973	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
2974	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2975	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2976	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2977	sleep $DELAI
2978	else
2979	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
2980	break
2981	fi
2982	(( i = i + 1 ))
2983	done
2984
2985	if [ ${status} -gt 0 ] ; then
2986	echo "IGCM_sys_ncra : ncra error"
2987	IGCM_debug_Exit "ncra"
2988	fi
2989
2990	IGCM_debug_PopStack "IGCM_sys_ncra"
2991	}
2992
2993	#D-#==================================================
2994	#D-function IGCM_sys_ncrcat
2995	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
2996	#D-* Examples:
2997	#D-
2998	function IGCM_sys_ncrcat {
2999	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
3000	if ( $DEBUG_sys ) ; then
3001	echo "IGCM_sys_ncrcat :" $@
3002	fi
3003
3004	typeset NB_ESSAI DELAI status i lastArg
3005	# number of tentative
3006	NB_ESSAI=3
3007	# time delay between tentative
3008	DELAI=2
3009
3010	i=0
3011	while [ $i -lt $NB_ESSAI ] ; do
3012	ncrcat -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
3013	status=$?
3014	if [ ${status} -gt 0 ] ; then
3015	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
3016	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3017	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3018	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3019	sleep $DELAI
3020	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )" = "X" ] ; then
3021	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
3022	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3023	# remove files having corrupted time axis
3024	eval lastArg=\${$#}
3025	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
3026	\rm ${lastArg}
3027	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3028	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3029	sleep $DELAI
3030	else
3031	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3032	break
3033	fi
3034	(( i = i + 1 ))
3035	done
3036
3037	if [ ${status} -gt 0 ] ; then
3038	echo "IGCM_sys_ncrcat : ncrcat error"
3039	#IGCM_debug_Exit "ncrcat"
3040	fi
3041
3042	IGCM_debug_PopStack "IGCM_sys_ncrcat"
3043	}
3044
3045	#D-#==================================================
3046	#D-function IGCM_sys_ncrename
3047	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
3048	#D-* Examples:
3049	#D-
3050	function IGCM_sys_ncrename {
3051	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
3052	if ( $DEBUG_sys ) ; then
3053	echo "IGCM_sys_ncrename :" $@
3054	fi
3055
3056	typeset NB_ESSAI DELAI status i
3057	# number of tentative
3058	NB_ESSAI=3
3059	# time delay between tentative
3060	DELAI=2
3061
3062	i=0
3063	while [ $i -lt $NB_ESSAI ] ; do
3064	ncrename $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
3065	status=$?
3066	if [ ${status} -gt 0 ] ; then
3067	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
3068	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3069	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3070	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3071	sleep $DELAI
3072	else
3073	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3074	break
3075	fi
3076	(( i = i + 1 ))
3077	done
3078
3079	if [ ${status} -gt 0 ] ; then
3080	echo "IGCM_sys_ncrename : ncrename error"
3081	IGCM_debug_Exit "ncrename"
3082	fi
3083
3084	IGCM_debug_PopStack "IGCM_sys_ncrename"
3085	}
3086
3087	#D-#==================================================
3088	#D-function IGCM_sys_ncwa
3089	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
3090	#D-* Examples:
3091	#D-
3092	function IGCM_sys_ncwa {
3093	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
3094	if ( $DEBUG_sys ) ; then
3095	echo "IGCM_sys_ncwa :" $@
3096	fi
3097
3098	typeset NB_ESSAI DELAI status i
3099	# number of tentative
3100	NB_ESSAI=3
3101	# time delay between tentative
3102	DELAI=2
3103
3104	i=0
3105	while [ $i -lt $NB_ESSAI ] ; do
3106	ncwa -C $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
3107	status=$?
3108	if [ ${status} -gt 0 ] ; then
3109	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
3110	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3111	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3112	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3113	sleep $DELAI
3114	else
3115	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3116	break
3117	fi
3118	(( i = i + 1 ))
3119	done
3120
3121	if [ ${status} -gt 0 ] ; then
3122	echo "IGCM_sys_ncwa : ncwa error"
3123	IGCM_debug_Exit "ncwa"
3124	fi
3125
3126	IGCM_debug_PopStack "IGCM_sys_ncwa"
3127	}
3128
3129	##############################################################
3130	# CDO OPERATOR
3131
3132	#D-#==================================================
3133	#D-function IGCM_sys_cdo
3134	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
3135	#D-* Examples:
3136	#D-
3137	function IGCM_sys_cdo {
3138	IGCM_debug_PushStack "IGCM_sys_cdo" $@
3139	if ( $DEBUG_sys ) ; then
3140	echo "IGCM_sys_cdo :" $@
3141	fi
3142
3143	typeset status
3144
3145	\cdo $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
3146	status=$?
3147	if [ ${status} -gt 0 ] ; then
3148	echo "IGCM_sys_cdo : error code ${status}"
3149	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3150	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
3151	IGCM_debug_PopStack "IGCM_sys_cdo"
3152	return 1
3153	else
3154	IGCM_debug_PopStack "IGCM_sys_cdo"
3155	return 0
3156	fi
3157
3158	IGCM_debug_PopStack "IGCM_sys_cdo"
3159	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: