Context Navigation

libIGCM_sys_curie.ksh @ 1173

Last change on this file since 1173 was 1173, checked in by aclsce, 9 years ago

MPMD + MPI-OpenMP mode

Modified to fit with new mpirun assignment method (method changed beacuse of activation of hyperthreading on Curie nodes).
Use of classic method in case of 1 OMP thread.

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 93.7 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Curie
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=curie
71	# add default project on curie
72	typeset DEFAULT_PROJECT=gen0826
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# Submit command
80	typeset SUBMIT=${SUBMIT:=ccc_msub}
81	# rsync with path
82	typeset -r RSYNC=/usr/bin/rsync
83	# RSYNC_opt args to rsync
84	typeset -r RSYNC_opt="-va"
85	# ie storage filesystem
86	typeset -r STOREHOST=${MASTER}
87
88	#====================================================
89	# Source default environment
90	#====================================================
91	##. /etc/profile
92
93	#====================================================
94	# Set environment tools (ferret, nco, cdo)
95	#====================================================
96	if [ X${TaskType} = Xcomputing ] ; then
97	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
98	# to run with netcdf 3.6.3 ie compilation done before 17/2/2014
99	# uncomment 2 lines :
100	# module unload netcdf
101	# module load netcdf/3.6.3
102	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddNoise/src_X64_CURIE/bin
103	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddPerturbation/src_X64_CURIE/bin
104	else
105	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
106	PCMDI_MP=/ccc/work/cont003/dsm/p86ipsl/PCMDI-MP
107	fi
108
109	#====================================================
110	# Specific for ocean additionnal diagnostic
111	export FER_GO="$FER_GO /home/cont003/p86denv/IGCM_POST_UTIL/JNL /home/cont003/p86denv/GRAF /home/cont003/p86denv/GRAF/GO"
112	export FER_PALETTE="$FER_PALETTE /home/cont003/p86denv/GRAF/PALET"
113
114	#====================================================
115	# Host specific DIRECTORIES
116	#====================================================
117
118	# ============ CESIUM START ============ #
119
120	#====================================================
121	#- Mirror libIGCM from titane to cesium if needed
122	#ROOTSYS=$( echo ${libIGCM} \| gawk -F"/" '{print $3}' )
123	#if [ ! ${ROOTSYS} = "home" ] ; then
124	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
125	#else
126	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
127	#fi
128
129	#====================================================
130	#- libIGCM_POST
131	#if ( ${MirrorlibIGCM} ) ; then
132	# PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
133	# typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
134	#else
135	# typeset -r libIGCM_POST=${libIGCM}
136	#fi
137
138	# ============ CESIUM END ============ #
139
140	#====================================================
141	#- MirrorlibIGCM for frontend
142	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
143
144	#====================================================
145	#- libIGCM_POST for frontend
146	typeset -r libIGCM_POST=${libIGCM}
147
148	#====================================================
149	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
150	typeset -r R_EXE="${MODIPSL}/bin"
151
152	#====================================================
153	#- SUBMIT_DIR : submission dir
154	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${BRIDGE_MSUB_PWD}}
155
156	#====================================================
157	#- IN
158	typeset -r R_IN=${R_IN:=/ccc/work/cont003/dsm/p86ipsl/IGCM}
159	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/ccc/work/cont003/dsm/p24data}
160
161	#====================================================
162	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
163	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCHDIR}/RUN_DIR/${BRIDGE_MSUB_JOBID}_${$}}
164
165	#====================================================
166	#- HOST_MPIRUN_COMMAND
167	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time ccc_mprun -E-K1"}
168
169	#====================================================
170	#- Max number of arguments passed to nco operator or demigration command
171	UNIX_MAX_LIMIT=360
172
173	#====================================================
174	#- set PackDefault true on curie
175	PackDefault=true
176
177	#====================================================
178	#- Number of core per node (max number of OpenMP task)
179	NUM_COREPERNODE=8
180
181	#====================================================
182	#- Default number of MPI task for IPSL coupled model
183	#- required for backward compatibility
184	#-
185	DEFAULT_NUM_PROC_OCE=5
186	DEFAULT_NUM_PROC_CPL=1
187	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
188	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
189
190	#D-#==================================================
191	#D-function IGCM_sys_defineArchives
192	#D-* Purpose:
193	#D-* Define ARCHIVE : Dedicated to large files
194	#D-* Define STORAGE : Dedicated to small/medium files
195	#D-* Define R_OUT : Output tree located on ARCHIVE
196	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
197	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
198	#D-* if SpaceName=TEST everything is stored on SCRATCHDIR
199	#D-* Examples:
200	#D-
201	function IGCM_sys_defineArchives {
202	IGCM_debug_PushStack "IGCM_sys_defineArchives"
203
204	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
205	#====================================================
206	#- ARCHIVE (dedicated to large files)
207	ARCHIVE=${config_UserChoices_ARCHIVE}
208	else
209	#====================================================
210	#- ARCHIVE (dedicated to large files)
211	ARCHIVE=${CCCSTOREDIR}
212	fi
213
214	if [ ! X${config_UserChoices_STORAGE} = X ]; then
215	#====================================================
216	#- STORAGE (dedicated to small/medium files)
217	STORAGE=${config_UserChoices_STORAGE}
218	else
219	#====================================================
220	#- STORAGE (dedicated to small/medium files)
221	STORAGE=${CCCWORKDIR}
222	fi
223
224	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
225	#====================================================
226	#- R_OUT
227	R_OUT=${SCRATCHDIR}/IGCM_OUT
228
229	#====================================================
230	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
231	R_FIG=${SCRATCHDIR}/IGCM_OUT
232
233	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
234
235	else
236	#====================================================
237	#- R_OUT
238	R_OUT=${ARCHIVE}/IGCM_OUT
239
240	#====================================================
241	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
242	R_FIG=${STORAGE}/IGCM_OUT
243	fi
244
245	#====================================================
246	#- R_BUF (ONLY FOR double copy an scratch)
247	R_BUF=${SCRATCHDIR}/IGCM_OUT
248
249	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
250	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
251	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
252
253	IGCM_debug_PopStack "IGCM_sys_defineArchives"
254	}
255
256	#D-#==================================================
257	#D-function IGCM_sys_RshMaster
258	#D-* Purpose: Connection to frontend machine.
259	#D-* Examples:
260	#D-
261	function IGCM_sys_RshMaster {
262	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
263	/bin/ksh <<-EOF
264	export libIGCM=${libIGCM}
265	export DEBUG_debug=${DEBUG_debug}
266	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
267	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
268	${@}
269	EOF
270	if [ $? -gt 0 ] ; then
271	echo "IGCM_sys_RshMaster : erreur."
272	IGCM_debug_Exit "IGCM_sys_RshMaster"
273	fi
274	IGCM_debug_PopStack "IGCM_sys_RshMaster"
275	}
276
277	#D-#==================================================
278	#D-function IGCM_sys_RshArchive
279	#D-* Purpose: Archive rsh command
280	#D-* Examples:
281	#D-
282	function IGCM_sys_RshArchive {
283	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
284	/bin/ksh <<-EOF
285	${@}
286	EOF
287	status=$?
288	if [ ${status} -gt 0 ] ; then
289	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
290	IGCM_debug_Exit "IGCM_sys_RshArchive"
291	fi
292	IGCM_debug_PopStack "IGCM_sys_RshArchive"
293	}
294
295	#D-#==================================================
296	#D-function IGCM_sys_RshArchive_NoError
297	#D-* Purpose: Archive rsh command, without error
298	#D-* used only in monitoring.job
299	#D-* Examples:
300	#D-
301	function IGCM_sys_RshArchive_NoError {
302	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
303	/bin/ksh <<-EOF
304	${@} 2> \dev\null
305	EOF
306	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
307	}
308
309	#D-#==================================================
310	#D-function IGCM_sys_RshPost
311	#D-* Purpose: Post-process rsh command
312	#D-* Examples:
313	#D-
314	function IGCM_sys_RshPost {
315	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
316	if ( $DEBUG_sys ) ; then
317	echo "IGCM_sys_RshPost :" $@
318	fi
319	# keep standard input (stdin) for the loop onto temporary file
320	cat >/tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
321
322	/bin/ksh </tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
323	if [ $? -gt 0 ] ; then
324	echo "IGCM_sys_RshPost : erreur."
325	IGCM_debug_Exit "IGCM_sys_RshPost"
326	fi
327	# delete temporary file
328	\rm /tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
329
330	# ============ FRONTEND END ============ #
331
332	# ============ CESIUM START ============ #
333	# typeset NB_ESSAI DELAI status i
334	# if [ "X$( grep rebuild_from tmp_IGCM_sys_RshPost_$$ )" != "X" ] ; then
335	# #little hack so that rebuild submission is done on titane not an cesium
336	#
337	# libIGCM_POST_sed=$( echo $libIGCM_POST \| sed 's/\//\\\//g' )
338	# POST_DIR_sed=$( echo ${POST_DIR} \| sed 's/\//\\\//g' )
339	# sed "s/IGCM_sys_QsubPost/IGCM_sys_Qsub/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
340	# sed "s/ rebuild_fromWorkdir/ ${libIGCM_POST_sed}\/rebuild_fromWorkdir.job/g" tmp.txt > tmp_IGCM_sys_RshPost_$$
341	# sed "s/ rebuild_fromArchive/ ${libIGCM_POST_sed}\/rebuild_fromArchive.job/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
342	# sed "s/Script_Post_Output=/Script_Output=${POST_DIR_sed}\//g" tmp.txt > tmp_IGCM_sys_RshPost_$$
343	# \mv tmp.txt tmp_IGCM_sys_RshPost_$$
344	#
345	# echo cat tmp_IGCM_sys_RshPost_$$ AFTER
346	# cat tmp_IGCM_sys_RshPost_$$
347	#
348	# /bin/ksh <tmp_IGCM_sys_RshPost_$$
349	# if [ $? -gt 0 ] ; then
350	# echo "IGCM_sys_RshPost : erreur."
351	# IGCM_debug_Exit "IGCM_sys_RshPost"
352	# fi
353	# # delete temporary file
354	# \rm tmp_IGCM_sys_RshPost_$$
355	#
356	# else
357	# # number of tentative
358	# NB_ESSAI=10
359	# # time delay between tentative
360	# DELAI=10
361	# i=0
362	# while [ $i -ne $NB_ESSAI ] ; do
363	# ssh -t titane996 ssh cesium /bin/ksh <tmp_IGCM_sys_RshPost_$$
364	# status=$?
365	# if [ ${status} -ne 0 ];
366	# then
367	# sleep $DELAI
368	# else
369	# break
370	# fi
371	# let i=$i+1
372	# done
373	# # delete temporary file
374	# \rm tmp_IGCM_sys_RshPost_$$
375	#
376	# if [ ${status} -gt 0 ] ; then
377	# echo "IGCM_sys_RshPost : erreur."
378	# IGCM_debug_Exit "IGCM_sys_RshPost"
379	# fi
380	# fi
381
382	# ============ CESIUM END ============ #
383
384	IGCM_debug_PopStack "IGCM_sys_RshPost"
385	}
386
387	#D-#==================================================
388	#D-function IGCM_sys_SendMail
389	#D-* Purpose: Send mail when simulation is over
390	#D-* Examples:
391	#D-
392	function IGCM_sys_SendMail {
393	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
394
395	if ( $DEBUG_sys ) ; then
396	echo "IGCM_sys_SendMail :" $@
397	fi
398
399	if [ X${1} = XAccounting ] ; then
400	status=Accounting
401	mailText=jobAccounting.mail
402	elif ( ${ExitFlag} ) ; then
403	status=failed
404	mailText=jobEnd.mail
405	else
406	status=completed
407	mailText=jobEnd.mail
408	fi
409
410	# Update selected mail template
411	while read -r line; do
412	eval echo $line >> mail.txt ;
413	done < ${libIGCM}/libIGCM_sys/${mailText}
414
415	if [ ! -z ${config_UserChoices_MailName} ] ; then
416	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
417	elif [ -f ~/.forward ] ; then
418	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
419	fi
420
421	sleep 10
422	rm -f mail.txt
423
424	IGCM_debug_PopStack "IGCM_sys_SendMail"
425	}
426
427	#D-#==================================================
428	#D-function IGCM_sys_Mkdir
429	#D-* Purpose: Master locale mkdir command
430	#D-* Examples:
431	#D-
432	function IGCM_sys_Mkdir {
433	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
434	if ( $DEBUG_sys ) ; then
435	echo "IGCM_sys_Mkdir :" $@
436	fi
437	if [ ! -d ${1} ]; then
438	\mkdir -p $1
439	if [ $? -gt 0 ] ; then
440	echo "IGCM_sys_Mkdir : erreur."
441	IGCM_debug_Exit "IGCM_sys_Mkdir"
442	fi
443	fi
444	# vérification :
445	if [ ! -d ${1} ] ; then
446	echo "IGCM_sys_Mkdir : erreur."
447	IGCM_debug_Exit "IGCM_sys_Mkdir"
448	fi
449	IGCM_debug_PopStack "IGCM_sys_Mkdir"
450	}
451
452	#D-#==================================================
453	#D-function IGCM_sys_MkdirArchive
454	#D-* Purpose: Mkdir on Archive
455	#D-* Examples:
456	#D-
457	function IGCM_sys_MkdirArchive {
458	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
459	if ( $DEBUG_sys ) ; then
460	echo "IGCM_sys_MkdirArchive :" $@
461	fi
462	#- creation de repertoire sur le serveur fichier
463	if [ ! -d ${1} ]; then
464	\mkdir -p $1
465	status=$?
466
467	if [ ${status} -gt 0 ] ; then
468	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
469	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
470	fi
471	fi
472	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
473	}
474
475	#D-#==================================================
476	#D-function IGCM_sys_MkdirWork
477	#D-* Purpose: Mkdir on Work
478	#D-* Examples:
479	#D-
480	function IGCM_sys_MkdirWork {
481	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
482	if ( $DEBUG_sys ) ; then
483	echo "IGCM_sys_MkdirWork :" $@
484	fi
485	#- creation de repertoire sur le serveur fichier
486	if [ ! -d ${1} ]; then
487	\mkdir -p $1
488	if [ $? -gt 0 ] ; then
489	echo "IGCM_sys_MkdirWork : erreur."
490	IGCM_debug_Exit "IGCM_sys_MkdirWork"
491	fi
492	fi
493	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
494	}
495
496	#D-#==================================================
497	#D-function IGCM_sys_Cd
498	#D-* Purpose: master cd command
499	#D-* Examples:
500	#D-
501	function IGCM_sys_Cd {
502	IGCM_debug_PushStack "IGCM_sys_Cd" $@
503	if ( $DEBUG_sys ) ; then
504	echo "IGCM_sys_Cd :" $@
505	fi
506	\cd $1
507	if [ $? -gt 0 ] ; then
508	echo "IGCM_sys_Cd : erreur."
509	IGCM_debug_Exit "IGCM_sys_Cd"
510	fi
511	IGCM_debug_PopStack "IGCM_sys_Cd"
512	}
513
514	#D-#==================================================
515	#D-function IGCM_sys_Chmod
516	#D-* Purpose: Chmod
517	#D-* Examples:
518	#D-
519	function IGCM_sys_Chmod {
520	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
521	if ( $DEBUG_sys ) ; then
522	echo "IGCM_sys_Chmod :" $@
523	fi
524	\chmod $@
525	if [ $? -gt 0 ] ; then
526	echo "IGCM_sys_Chmod : erreur."
527	IGCM_debug_Exit "IGCM_sys_Chmod"
528	fi
529	IGCM_debug_PopStack "IGCM_sys_Chmod"
530	}
531
532	#D-#==================================================
533	#D-function IGCM_sys_FileSize
534	#D-* Purpose: Filesize
535	#D-* Examples:
536	#D-
537	function IGCM_sys_FileSize {
538	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
539
540	typeset sizeF
541	set +A sizeF -- $( ls -la ${1} )
542	if [ $? -gt 0 ] ; then
543	IGCM_debug_Exit "IGCM_sys_FileSize"
544	fi
545	eval ${2}=${sizeF[4]}
546
547	IGCM_debug_PopStack "IGCM_sys_FileSize"
548	}
549
550	#D-#==================================================
551	#D-function IGCM_sys_TestDir
552	#D-* Purpose: Test Directory that must exists
553	#D-* Examples:
554	#D-
555	function IGCM_sys_TestDir {
556	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
557	if ( $DEBUG_sys ) ; then
558	echo "IGCM_sys_TestDir :" $@
559	fi
560	typeset ExistFlag
561	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
562	IGCM_debug_PopStack "IGCM_sys_TestDir"
563
564	return ${ExistFlag}
565	}
566
567	#D-#==================================================
568	#D-function IGCM_sys_TestDirArchive
569	#D-* Purpose: Test Directory that must exists on Archive
570	#D-* Examples:
571	#D-
572	function IGCM_sys_TestDirArchive {
573	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
574	if ( $DEBUG_sys ) ; then
575	echo "IGCM_sys_TestDirArchive :" $@
576	fi
577	typeset ExistFlag
578	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
579	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
580	return ${ExistFlag}
581	}
582
583	#D-#==================================================
584	#D-function IGCM_sys_IsFileArchived
585	#D-* Purpose: Test file that must NOT EXISTS on Archive
586	#D-* Examples:
587	#D-
588	function IGCM_sys_IsFileArchived {
589	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
590	if ( $DEBUG_sys ) ; then
591	echo "IGCM_sys_IsFileArchived :" $@
592	fi
593	typeset IsArchivedFlag
594	IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
595	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
596
597	return ${IsArchivedFlag}
598	}
599
600	#D-#==================================================
601	#D-function IGCM_sys_TestFileArchive
602	#D-* Purpose: Test file that must NOT EXISTS on Archive
603	#D-* Examples:
604	#D-
605	function IGCM_sys_TestFileArchive {
606	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
607	typeset ExistFlag
608	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
609	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
610
611	return ${ExistFlag}
612	}
613
614	#D-#==================================================
615	#D-function IGCM_sys_TestFileBuffer
616	#D-* Purpose: Test file that must NOT EXISTS on Buffer
617	#D-* Examples:
618	#D-
619	function IGCM_sys_TestFileBuffer {
620	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
621	typeset ExistFlag
622	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
623	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
624
625	return ${ExistFlag}
626	}
627
628	#D-#==================================================
629	#D-function IGCM_sys_CountFileArchive
630	#D-* Purpose: Count files on Archive filesystem
631	#D-* Examples:
632	#D-
633	function IGCM_sys_CountFileArchive {
634	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
635	ls ${@} 2>/dev/null \| wc -l
636	if [ $? -gt 0 ] ; then
637	echo "IGCM_sys_CountFileArchive : erreur."
638	fi
639	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
640	}
641
642	#D-#==================================================
643	#D-function IGCM_sys_CountFileBuffer
644	#D-* Purpose: Count files on Scratch filesystem
645	#D-* Examples:
646	#D-
647	function IGCM_sys_CountFileBuffer {
648	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
649	ls ${@} 2>/dev/null \| wc -l
650	if [ $? -gt 0 ] ; then
651	echo "IGCM_sys_CountFileBuffer : erreur."
652	fi
653	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
654	}
655
656	#D-#==================================================
657	#D-function IGCM_sys_Tree
658	#D-* Purpose: Tree directories with files on ${ARCHIVE}
659	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
660	#D-
661	function IGCM_sys_Tree {
662	IGCM_debug_PushStack "IGCM_sys_Tree" $@
663	if ( $DEBUG_sys ) ; then
664	echo "IGCM_sys_Tree :" $@
665	fi
666
667	\ls -lR ${@}
668
669	IGCM_debug_PopStack "IGCM_sys_Tree"
670	}
671
672	#D-#==================================================
673	#D-function IGCM_sys_Tar
674	#D-* Purpose: master tar command
675	#D-* Examples:
676	#D-
677	function IGCM_sys_Tar {
678	IGCM_debug_PushStack "IGCM_sys_Tar" $@
679	if ( $DEBUG_sys ) ; then
680	echo "IGCM_sys_Tar :" $@
681	fi
682	\tar cf $@
683	if [ $? -gt 0 ] ; then
684	echo "IGCM_sys_Tar : erreur."
685	IGCM_debug_Exit "IGCM_sys_Tar"
686	fi
687	IGCM_debug_PopStack "IGCM_sys_Tar"
688	}
689
690	#D-#==================================================
691	#D-function IGCM_sys_UnTar
692	#D-* Purpose: master un-tar command
693	#D-* Examples:
694	#D-
695	function IGCM_sys_UnTar {
696	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
697	if ( $DEBUG_sys ) ; then
698	echo "IGCM_sys_UnTar :" $@
699	fi
700	\tar xvf $1
701	if [ $? -gt 0 ] ; then
702	echo "IGCM_sys_UnTar : erreur."
703	IGCM_debug_Exit "IGCM_sys_UnTar"
704	fi
705	IGCM_debug_PopStack "IGCM_sys_UnTar"
706	}
707
708	#D-#==================================================
709	#D-function IGCM_sys_Qsub
710	#D-* Purpose: Qsub new job
711	#D-* Examples:
712	#D-
713	function IGCM_sys_Qsub {
714	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
715	if ( $DEBUG_sys ) ; then
716	echo "IGCM_sys_Qsub :" $@
717	fi
718	typeset options status
719	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
720
721	/usr/bin/ccc_msub ${options} $1 > /tmp/out_command.$$ 2>&1
722	status=$?
723
724	cat /tmp/out_command.$$
725	if [ ${status} -gt 0 ] ; then
726	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
727	IGCM_debug_Exit "IGCM_sys_Qsub"
728	else
729	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
730	fi
731	IGCM_debug_PopStack "IGCM_sys_Qsub"
732	}
733
734	#D-#==================================================
735	#D-function IGCM_sys_QsubPost
736	#D-* Purpose: Qsub new job on scalaire
737	#D-* Examples:
738	#D-
739	function IGCM_sys_QsubPost {
740	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
741	if ( $DEBUG_sys ) ; then
742	echo "IGCM_sys_QsubPost :" $@
743	fi
744	typeset options status
745	MY_PROJECT=$(echo ${BRIDGE_MSUB_PROJECT:=${DEFAULT_PROJECT}} \| cut -d@ -f1 )
746
747	# EASIER TO DO THIS INSTEAD OF DUPLICATING libIGCM_sys_curie.ksh
748	case $( hostname -s ) in
749	curie*)
750	options="-Q normal -A ${MY_PROJECT} -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out";;
751	airain*)
752	options="-q ivybridge -A dsm -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
753	esac
754
755	/usr/bin/ccc_msub ${options} ${libIGCM_POST}/$1.job > /tmp/out_command.$$ 2>&1
756	status=$?
757
758	cat /tmp/out_command.$$
759	if [ ${status} -gt 0 ] ; then
760	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
761	IGCM_debug_Exit "IGCM_sys_QsubPost"
762	else
763	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
764	fi
765	IGCM_debug_PopStack "IGCM_sys_QsubPost"
766	}
767
768	#D-*************************
769	#D- File transfer functions
770	#D-*************************
771	#D-
772
773	#D-#==================================================
774	#D-function IGCM_sys_Rsync_out
775	#D-* Purpose: treat return val of rsync
776	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
777	#D- Error values and explanations can depend on your system version.
778	function IGCM_sys_Rsync_out {
779	status=$1
780	if [ ! $status ] ; then
781	echo "rsync error !"
782	fi
783
784	if [ $MYLANG = "fr" ]; then
785	case $status in
786	0) return ;;
787	1) echo "Erreur de rsync ; RERR_SYNTAX : "
788	echo "Erreur de syntaxe ou d'utilisation."
789	return;;
790	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
791	echo "Incompatibilité de protocole."
792	return;;
793	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
794	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
795	echo "répertoires"
796	return;;
797	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
798	echo "Action demandée non supportée : une tentative de manipulation de"
799	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
800	echo "été faite ; ou une option qui est supportée par le client mais"
801	echo "pas par le serveur a été spécifiée."
802	return;;
803	10) echo "Erreur de rsync ; RERR_SOCKETIO"
804	echo "Erreur dans le socket d'entrée sortie"
805	return;;
806	11) echo "Erreur de rsync ; RERR_FILEIO"
807	echo "Erreur d'entrée sortie fichier"
808	return;;
809	12) echo "Erreur de rsync ; RERR_STREAMIO"
810	echo "Erreur dans flux de donnée du protocole rsync"
811	return;;
812	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
813	echo "Erreur avec les diagnostics du programme"
814	return;;
815	14) echo "Erreur de rsync ; RERR_IPC"
816	echo "Erreur dans le code IPC"
817	return;;
818	20) echo "Erreur de rsync ; RERR_SIGNAL"
819	echo "SIGUSR1 ou SIGINT reçu"
820	return;;
821	21) echo "Erreur de rsync ; RERR_WAITCHILD"
822	echo "Une erreur retournée par waitpid()"
823	return;;
824	22) echo "Erreur de rsync ; RERR_MALLOC"
825	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
826	return;;
827	23) echo ""
828	echo "Erreur fichier inexistant"
829	return;;
830	30) echo "Erreur de rsync ; RERR_TIMEOUT"
831	echo "Temps d'attente écoulé dans l'envoi/réception de données"
832	return;;
833	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
834	return;;
835	esac
836	elif [ $MYLANG = "en" ] ; then
837	case $status in
838	0) return;;
839	1) echo "rsync error : Syntax or usage error "
840	return;;
841	2) echo "rsync error : Protocol incompatibility "
842	return;;
843	3) echo "rsync error : Errors selecting input/output files, dirs"
844	return;;
845	4) echo "rsync error : Requested action not supported: an attempt"
846	echo "was made to manipulate 64-bit files on a platform that cannot support"
847	echo "them; or an option was specified that is supported by the client and"
848	echo "not by the server."
849	return;;
850	5) echo "rsync error : Error starting client-server protocol"
851	return;;
852	10) echo "rsync error : Error in socket I/O "
853	return;;
854	11) echo "rsync error : Error in file I/O "
855	return;;
856	12) echo "rsync error : Error in rsync protocol data stream "
857	return;;
858	13) echo "rsync error : Errors with program diagnostics "
859	return;;
860	14) echo "rsync error : Error in IPC code "
861	return;;
862	20) echo "rsync error : Received SIGUSR1 or SIGINT "
863	return;;
864	21) echo "rsync error : Some error returned by waitpid() "
865	return;;
866	22) echo "rsync error : Error allocating core memory buffers "
867	return;;
868	23) echo "rsync error : Partial transfer due to error"
869	return;;
870	24) echo "rsync error : Partial transfer due to vanished source files"
871	return;;
872	30) echo "rsync error : Timeout in data send/receive "
873	return;;
874	*) echo "rsync error : return code of rsync unknown :" $status
875	return;;
876	esac
877	else
878	echo "unknown language $MYLANG."
879	return
880	fi
881	}
882
883	#D-#==================================================
884	#D-function IGCM_sys_Miror_libIGCM
885	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
886	#D-* Examples:
887	#D-
888	function IGCM_sys_Mirror_libIGCM {
889	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
890	if ( $DEBUG_sys ) ; then
891	echo "IGCM_sys_Mirror_libIGCM"
892	fi
893
894	typeset status
895
896	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
897
898	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
899	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
900	status=$?
901
902	if [ ${status} -gt 0 ] ; then
903	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
904	cat /tmp/out_command.$$
905	fi
906	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
907	}
908
909	#D-#==================================================
910	#D-function IGCM_sys_Cp
911	#D-* Purpose: generic cp
912	#D-* Examples:
913	#D-
914	function IGCM_sys_Cp {
915	IGCM_debug_PushStack "IGCM_sys_Cp" $@
916	if ( $DEBUG_sys ) ; then
917	echo "IGCM_sys_Cp :" $@
918	fi
919
920	typeset status
921
922	echo cp $@ > /tmp/out_command.$$ 2>&1
923	\cp $@ >> /tmp/out_command.$$ 2>&1
924	status=$?
925
926	if [ ${status} -gt 0 ] ; then
927	echo "IGCM_sys_Cp : error code ${status}"
928	cat /tmp/out_command.$$
929	IGCM_debug_Exit "IGCM_sys_Cp"
930	else
931	\rm /tmp/out_command.$$
932	fi
933	IGCM_debug_PopStack "IGCM_sys_Cp"
934	}
935
936	#D-#==================================================
937	#D-function IGCM_sys_Rm
938	#D-* Purpose: generic rm
939	#D-* Examples:
940	#D-
941	function IGCM_sys_Rm {
942	IGCM_debug_PushStack "IGCM_sys_Rm" $@
943	if ( $DEBUG_sys ) ; then
944	echo "IGCM_sys_Rm :" $@
945	fi
946
947	typeset status
948
949	echo rm $@ > /tmp/out_command.$$ 2>&1
950	\rm $@ >> /tmp/out_command.$$ 2>&1
951	status=$?
952
953	if [ ${status} -gt 0 ] ; then
954	echo "IGCM_sys_Rm : error code ${status}"
955	cat /tmp/out_command.$$
956	IGCM_debug_Exit "IGCM_sys_Rm"
957	else
958	\rm /tmp/out_command.$$
959	fi
960	IGCM_debug_PopStack "IGCM_sys_Rm"
961	}
962
963	#D-#==================================================
964	#D-function IGCM_sys_RmRunDir
965	#D-* Purpose: rm tmpdir (dummy function most of the time batch
966	#D- scheduler will do the job)
967	#D-* Examples:
968	#D-
969	function IGCM_sys_RmRunDir {
970	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
971	if ( $DEBUG_sys ) ; then
972	echo "IGCM_sys_RmRunDir :" $@
973	fi
974
975	typeset status
976
977	echo rm $@ > /tmp/out_command.$$ 2>&1
978	\rm $@ >> /tmp/out_command.$$ 2>&1
979	status=$?
980
981	if [ ${status} -gt 0 ] ; then
982	echo "IGCM_sys_RmRunDir : rm error code is ${status}."
983	cat /tmp/out_command.$$
984	IGCM_debug_Exit "IGCM_sys_RmRunDir"
985	else
986	\rm /tmp/out_command.$$
987	fi
988	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
989	}
990
991	#D-#==================================================
992	#D-function IGCM_sys_Mv
993	#D-* Purpose: generic move
994	#D-* Examples:
995	#D-
996	function IGCM_sys_Mv {
997	IGCM_debug_PushStack "IGCM_sys_Mv" $@
998	if ( $DEBUG_sys ) ; then
999	echo "IGCM_sys_Mv :" $@
1000	fi
1001
1002	if [ $DRYRUN = 0 ]; then
1003
1004	typeset status
1005
1006	echo mv $@ > /tmp/out_command.$$ 2>&1
1007	\mv $@ >> /tmp/out_command.$$ 2>&1
1008	status=$?
1009
1010	if [ ${status} -gt 0 ] ; then
1011	echo "IGCM_sys_Mv : error code ${status}"
1012	cat /tmp/out_command.$$
1013	IGCM_debug_Exit "IGCM_sys_Mv"
1014	else
1015	\rm /tmp/out_command.$$
1016	fi
1017	fi
1018
1019	IGCM_debug_PopStack "IGCM_sys_Mv"
1020	}
1021
1022	#D-#==================================================
1023	#D-function IGCM_sys_Put_Dir
1024	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1025	#D-* Examples:
1026	#D-
1027	function IGCM_sys_Put_Dir {
1028	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1029	if ( $DEBUG_sys ) ; then
1030	echo "IGCM_sys_Put_Dir :" $@
1031	fi
1032	if [ $DRYRUN = 0 ]; then
1033	if [ ! -d ${1} ] ; then
1034	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1035	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1036	return
1037	fi
1038
1039	typeset status
1040
1041	# Only if we use rsync
1042	#IGCM_sys_TestDirArchive $( dirname $2 )
1043	#
1044	#USUAL WAY
1045	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
1046	status=$?
1047
1048	if [ ${status} -gt 0 ] ; then
1049	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
1050	cat /tmp/out_command.$$
1051	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1052	else
1053	\rm /tmp/out_command.$$
1054	fi
1055	fi
1056	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1057	}
1058
1059	#D-#==================================================
1060	#D-function IGCM_sys_Get_Dir
1061	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1062	#D-* Examples:
1063	#D-
1064	function IGCM_sys_Get_Dir {
1065	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1066	if ( $DEBUG_sys ) ; then
1067	echo "IGCM_sys_Get_Dir :" $@
1068	fi
1069	if [ $DRYRUN = 0 ]; then
1070	# if [ ! -d ${1} ] ; then
1071	# echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
1072	# IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1073	# return
1074	# fi
1075
1076	typeset NB_ESSAI DELAI status i
1077	# number of tentative
1078	NB_ESSAI=3
1079	# time delay between tentative
1080	DELAI=2
1081
1082	# Only if we use rsync
1083	#IGCM_sys_TestDirArchive $( dirname $2 )
1084	#
1085	# USUAL WAY
1086	# add 'ccc_hsm get' (to demigrate all offline files) to reduce time of this command :
1087	ccc_hsm get -r $1
1088
1089	i=0
1090	while [ $i -lt $NB_ESSAI ] ; do
1091	\cp -ur $1 $2 >> /tmp/out_command.$$ 2>&1
1092	status=$?
1093	if [ ${status} -gt 0 ] ; then
1094	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
1095	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1096	sleep $DELAI
1097	else
1098	break
1099	fi
1100	(( i = i + 1 ))
1101	done
1102
1103	if [ ${status} -gt 0 ] ; then
1104	echo "IGCM_sys_Get_Dir : error."
1105	cat /tmp/out_command.$$
1106	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1107	else
1108	\rm /tmp/out_command.$$
1109	fi
1110	fi
1111	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1112	}
1113
1114	#D-#==================================================
1115	#D-function IGCM_sys_Get_Master
1116	#D-* Purpose: Copy a complete directory from MASTER filesystem
1117	#D-* Examples:
1118	#D-
1119	function IGCM_sys_Get_Master {
1120	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1121	if ( $DEBUG_sys ) ; then
1122	echo "IGCM_sys_Get_Master :" $@
1123	fi
1124	if [ $DRYRUN = 0 ]; then
1125	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1126	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1127	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1128	return
1129	fi
1130
1131	typeset NB_ESSAI DELAI status i
1132	# number of tentative
1133	NB_ESSAI=3
1134	# time delay between tentative
1135	DELAI=2
1136
1137	i=0
1138	while [ $i -lt $NB_ESSAI ] ; do
1139	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
1140	status=$?
1141	if [ ${status} -gt 0 ]; then
1142	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1143	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1144	sleep $DELAI
1145	else
1146	break
1147	fi
1148	(( i = i + 1 ))
1149	done
1150
1151	if [ ${status} -gt 0 ] ; then
1152	echo "IGCM_sys_Get_Master : error."
1153	cat /tmp/out_command.$$
1154	IGCM_debug_Exit "IGCM_sys_Get_Master"
1155	else
1156	\rm /tmp/out_command.$$
1157	fi
1158	fi
1159	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1160	}
1161
1162	#====================================================
1163	#- Call IGCM_sys_Mirror_libIGCM now !
1164	if ( $MirrorlibIGCM ) ; then
1165	IGCM_sys_Mirror_libIGCM
1166	fi
1167
1168	#D-#==================================================
1169	#D-function IGCM_sys_Put_Rest
1170	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1171	#D- File and target directory must exist.
1172	#D-* Examples:
1173	#D-
1174	function IGCM_sys_Put_Rest {
1175	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1176	if ( $DEBUG_sys ) ; then
1177	echo "IGCM_sys_Put_Rest :" $@
1178	fi
1179	if [ $DRYRUN = 0 ]; then
1180	if [ ! -f ${1} ] ; then
1181	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1182	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1183	fi
1184
1185	typeset status
1186	#
1187	# USUAL WAY
1188	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1189	status=$?
1190
1191	# #RSYNC WITH NETWORK SSH CALL
1192	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1193	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1194
1195	# #RSYNC WITH NFS USE
1196	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1197	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1198
1199	# status=$?
1200	# IGCM_sys_Rsync_out $status
1201
1202	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1203	# (( status=status+$? ))
1204
1205	if [ ${status} -gt 0 ] ; then
1206	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
1207	[ -f ${1} ] && ls -l ${1}
1208	[ -f ${2} ] && ls -l ${2}
1209	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1210	cat /tmp/out_command.$$
1211	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1212	else
1213
1214	if [ X${JobType} = XRUN ] ; then
1215	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1216	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1217	fi
1218
1219	\rm /tmp/out_command.$$
1220	fi
1221	fi
1222	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1223	}
1224
1225	#D-#==================================================
1226	#D-function IGCM_sys_PutBuffer_Rest
1227	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1228	#D- File and target directory must exist.
1229	#D-* Examples:
1230	#D-
1231	function IGCM_sys_PutBuffer_Rest {
1232	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1233	if ( $DEBUG_sys ) ; then
1234	echo "IGCM_sys_PutBuffer_Rest :" $@
1235	fi
1236	if [ $DRYRUN = 0 ]; then
1237	if [ ! -f ${1} ] ; then
1238	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1239	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1240	fi
1241
1242	typeset status
1243	#
1244	# USUAL WAY
1245	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1246	status=$?
1247
1248	if [ ${status} -gt 0 ] ; then
1249	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1250	[ -f ${2} ] && ls -l ${2}
1251	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1252	cat /tmp/out_command.$$
1253	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1254	else
1255
1256	if [ X${JobType} = XRUN ] ; then
1257	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1258	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1259	fi
1260
1261	\rm /tmp/out_command.$$
1262	fi
1263	fi
1264	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1265	}
1266
1267	#D-#==================================================
1268	#D-function IGCM_sys_PrepareTaredRestart
1269	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
1270	#D-* Examples:
1271	#D-
1272	function IGCM_sys_PrepareTaredRestart {
1273	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1274	if [ $DRYRUN = 0 ]; then
1275	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1276	fi
1277	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1278	}
1279
1280	#D-#==================================================
1281	#D-function IGCM_sys_Put_Out
1282	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1283	#D-* Examples:
1284	#D-
1285	function IGCM_sys_Put_Out {
1286	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1287	if ( $DEBUG_sys ) ; then
1288	echo "IGCM_sys_Put_Out :" $@
1289	fi
1290
1291	typeset NB_ESSAI DELAI status i exist skip
1292
1293	# number of tentative
1294	NB_ESSAI=3
1295	# time delay between tentative
1296	DELAI=2
1297
1298	if [ $DRYRUN = 0 ]; then
1299	if [ ! -f ${1} ] ; then
1300	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1301	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1302	return 1
1303	fi
1304	#
1305	IGCM_sys_MkdirArchive $( dirname $2 )
1306	#
1307
1308	#=====================================================
1309	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1310	#=====================================================
1311
1312	#echo ${2} \| grep "${R_OUT}" > /dev/null 2>&1
1313	#if [ $? -eq 0 ] ; then
1314	# typeset WORKPATH FILEPATH
1315	# WORKPATH=$( dirname $2 \| sed -e "s\|${R_OUT}\|${R_BUF}\|" )
1316	# IGCM_sys_MkdirWork ${WORKPATH}
1317	# FILEPATH=${WORKPATH}/$( basename $2 )
1318	# #
1319	# IGCM_sys_Cp ${1} ${FILEPATH}
1320	#fi
1321
1322	exist=false
1323	skip=false
1324	if [ -f $2 ] ; then
1325	IGCM_debug_Print 1 "$2 already exist"
1326	ccc_hsm get $2
1327	exist=true
1328	if [ "X$( diff $1 $2 )" = X ] ; then
1329	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1330	skip=true
1331	else
1332	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1333	skip=false
1334	fi
1335	fi
1336	#
1337	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1338	IGCM_sys_Chmod u+w $2
1339	fi
1340
1341	if [ X${skip} = Xfalse ] ; then
1342	i=0
1343	while [ $i -lt $NB_ESSAI ] ; do
1344	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1345	# USUAL WAY
1346	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1347	status=$?
1348	else
1349	# NOT SO USUAL WAY
1350	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1351	status=$?
1352	fi
1353	if [ ${status} -gt 0 ]; then
1354	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1355	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
1356	[ -f ${1} ] && ls -l ${1}
1357	[ -f ${2} ] && ls -l ${2}
1358	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1359	sleep $DELAI
1360	else
1361	break
1362	fi
1363	(( i = i + 1 ))
1364	done
1365	fi
1366
1367	# #RSYNC WITH NETWORK SSH CALL
1368	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1369	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1370
1371	# #RSYNC WITH NFS USE
1372	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1373	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1374
1375	# status=$?
1376	# IGCM_sys_Rsync_out $status
1377
1378	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1379	# (( status=status+$? ))
1380
1381	if [ ${status} -gt 0 ] ; then
1382	echo "IGCM_sys_Put_Out : error."
1383	[ -f ${1} ] && ls -l ${1}
1384	[ -f ${2} ] && ls -l ${2}
1385	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1386	cat /tmp/out_command.$$
1387	IGCM_debug_Exit "IGCM_sys_Put_Out"
1388	else
1389
1390	if [ X${JobType} = XRUN ] ; then
1391	if [ X${3} = X ] ; then
1392	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1393	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1394	fi
1395	fi
1396
1397	\rm /tmp/out_command.$$
1398	fi
1399	fi
1400	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1401	return 0
1402	}
1403
1404	#D-#==================================================
1405	#D-function IGCM_sys_PutBuffer_Out
1406	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1407	#D-* Examples:
1408	#D-
1409	function IGCM_sys_PutBuffer_Out {
1410	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1411	if ( $DEBUG_sys ) ; then
1412	echo "IGCM_sys_PutBuffer_Out :" $@
1413	fi
1414
1415	typeset NB_ESSAI DELAI status i exist skip
1416
1417	# number of tentative
1418	NB_ESSAI=3
1419	# time delay between tentative
1420	DELAI=2
1421
1422	if [ $DRYRUN = 0 ]; then
1423	if [ ! -f ${1} ] ; then
1424	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1425	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1426	return 1
1427	fi
1428	#
1429	IGCM_sys_Mkdir $( dirname $2 )
1430	#
1431
1432	exist=false
1433	skip=false
1434	if [ -f $2 ] ; then
1435	IGCM_debug_Print 1 "$2 already exist"
1436	exist=true
1437	if [ "X$( diff $1 $2 )" = X ] ; then
1438	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1439	status=0
1440	skip=true
1441	else
1442	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1443	skip=false
1444	fi
1445	fi
1446	#
1447	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1448	IGCM_sys_Chmod u+w $2
1449	fi
1450
1451	if [ X${skip} = Xfalse ] ; then
1452	i=0
1453	while [ $i -lt $NB_ESSAI ] ; do
1454	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1455	# USUAL WAY
1456	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1457	status=$?
1458	else
1459	# NOT SO USUAL WAY
1460	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1461	status=$?
1462	fi
1463	if [ ${status} -gt 0 ]; then
1464	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1465	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1466	[ -f ${2} ] && ls -l ${2}
1467	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1468	sleep $DELAI
1469	else
1470	break
1471	fi
1472	(( i = i + 1 ))
1473	done
1474	fi
1475
1476	if [ ${status} -gt 0 ] ; then
1477	echo "IGCM_sys_PutBuffer_Out : error."
1478	[ -f ${2} ] && ls -l ${2}
1479	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1480	cat /tmp/out_command.$$
1481	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1482	else
1483
1484	if [ X${JobType} = XRUN ] ; then
1485	if [ X${3} = X ] ; then
1486	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1487	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1488	fi
1489	fi
1490
1491	\rm /tmp/out_command.$$
1492	fi
1493	fi
1494	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1495	return 0
1496	}
1497
1498	#D-#==================================================
1499	#D-function IGCM_sys_Get
1500	#D-* Purpose: Get a file from ${ARCHIVE}
1501	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1502	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1503	function IGCM_sys_Get {
1504	IGCM_debug_PushStack "IGCM_sys_Get" $@
1505
1506	typeset DEST dm_liste target file_work
1507	typeset NB_ESSAI DELAI status i
1508
1509	if ( $DEBUG_sys ) ; then
1510	echo "IGCM_sys_Get :" $@
1511	fi
1512
1513	# number of tentative
1514	NB_ESSAI=3
1515	# time delay between tentative
1516	DELAI=2
1517
1518	if [ $DRYRUN -le 2 ]; then
1519	if [ X${1} = X'/l' ] ; then
1520	eval set +A dm_liste \${${2}}
1521	else
1522	eval set +A dm_liste ${1}
1523	fi
1524	eval DEST=\${${#}}
1525
1526	#=====================================================
1527	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1528	#=====================================================
1529
1530	# Is it an R_OUT file (not R_IN) ?
1531	#echo ${dm_liste[0]} \| grep "${R_OUT}" > /dev/null 2>&1
1532	#if [ $? -eq 0 ] ; then
1533	# # Yes ? then we try to get it in SCRATCHDIR
1534	# set +A file_work $( echo ${dm_liste[*]} \| sed -e "s\|${R_OUT}\|${R_BUF}\|g" )
1535	# if [ -f ${file_work[0]} ] ; then
1536	# IGCM_sys_Cp ${file_work[*]} ${DEST}
1537	# IGCM_debug_PopStack "IGCM_sys_Get"
1538	# return
1539	# fi
1540	#fi
1541
1542	ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1543	status=$?
1544	if [ ${status} -gt 0 ] ; then
1545	echo "WARNING IGCM_sys_Get : error code ${status}"
1546	cat /tmp/out_command.$$
1547	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
1548	fi
1549
1550	#if [ ${status} -gt 0 ] ; then
1551	# if [ ! "X$( grep "Lost dmusrcmd connection" /tmp/out_command.$$ )" = "X" ] ; then
1552	# cat /tmp/out_command.$$
1553	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
1554	# sleep 30
1555	# echo "We try another time"
1556	## dmget ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1557	# ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1558	# status=$?
1559	# if [ ${status} -gt 0 ] ; then
1560	# echo "ERROR IGCM_sys_Get : again demigration error :"
1561	# cat /tmp/out_command.$$
1562	# IGCM_debug_Exit "IGCM_sys_Get"
1563	# fi
1564	# else
1565	# echo "ERROR IGCM_sys_Get : demigration error :"
1566	# cat /tmp/out_command.$$
1567	# IGCM_debug_Exit "IGCM_sys_Get"
1568	# fi
1569	#fi
1570
1571	# #RSYNC WITH NETWORK SSH CALL
1572	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1573	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1574
1575	# #RSYNC WITH NFS USE
1576	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1577	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1578
1579	# status=$?
1580	# IGCM_sys_Rsync_out $status
1581
1582	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1583	# (( status=status+$? ))
1584
1585	#USUAL WAY
1586	if [ X${1} = X'/l' ] ; then
1587	for target in ${dm_liste[*]} ; do
1588	local_file=$( basename ${target} )
1589	# test if the target file is present before the loop
1590	IGCM_sys_TestFileArchive ${target}
1591	status=$?
1592	if [ ${status} -gt 0 ] ; then
1593	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
1594	IGCM_debug_Exit "IGCM_sys_Get"
1595	else
1596	i=0
1597	while [ $i -lt $NB_ESSAI ] ; do
1598	#if [ X${DoLink} = Xtrue ] ; then
1599	# \ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1600	# status=$?
1601	# else
1602	# \cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1603	# status=$?
1604	#fi
1605	\ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1606	status=$?
1607	if [ ${status} -gt 0 ]; then
1608	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1609	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1610	sleep $DELAI
1611	else
1612	break
1613	fi
1614	(( i = i + 1 ))
1615	done
1616	if [ ${status} -gt 0 ] ; then
1617	echo "IGCM_sys_Get : error"
1618	cat /tmp/out_command.$$
1619	\rm /tmp/out_command.$$
1620	IGCM_debug_Exit "IGCM_sys_Get"
1621	else
1622	\rm /tmp/out_command.$$
1623	fi
1624	fi
1625	done
1626	else
1627	i=0
1628	while [ $i -lt $NB_ESSAI ] ; do
1629	\cp ${dm_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1630	status=$?
1631	if [ ${status} -gt 0 ]; then
1632	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1633	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1634	sleep $DELAI
1635	else
1636	break
1637	fi
1638	(( i = i + 1 ))
1639	done
1640	if [ ${status} -gt 0 ] ; then
1641	echo "IGCM_sys_Get : error"
1642	cat /tmp/out_command.$$
1643	\rm /tmp/out_command.$$
1644	IGCM_debug_Exit "IGCM_sys_Get"
1645	else
1646	\rm /tmp/out_command.$$
1647	fi
1648	fi
1649	fi
1650	IGCM_debug_PopStack "IGCM_sys_Get"
1651	}
1652
1653	#D-#==================================================
1654	#D-function IGCM_sys_GetBuffer
1655	#D-* Purpose: Get a file from ${SCRATCHDIR}
1656	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1657	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1658	function IGCM_sys_GetBuffer {
1659	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1660
1661	typeset DEST buf_liste target file_work
1662	typeset NB_ESSAI DELAI status i
1663
1664	if ( $DEBUG_sys ) ; then
1665	echo "IGCM_sys_GetBuffer :" $@
1666	fi
1667
1668	# number of tentative
1669	NB_ESSAI=3
1670	# time delay between tentative
1671	DELAI=2
1672
1673	if [ $DRYRUN -le 2 ]; then
1674	if [ X${1} = X'/l' ] ; then
1675	eval set +A buf_liste \${${2}}
1676	else
1677	eval set +A buf_liste ${1}
1678	fi
1679	eval DEST=\${${#}}
1680
1681	#USUAL WAY
1682	if [ X${1} = X'/l' ] ; then
1683	for target in ${buf_liste[*]} ; do
1684	local_file=$( basename ${target} )
1685	i=0
1686	while [ $i -lt $NB_ESSAI ] ; do
1687	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1688	status=$?
1689	if [ ${status} -gt 0 ]; then
1690	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1691	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1692	sleep $DELAI
1693	else
1694	break
1695	fi
1696	(( i = i + 1 ))
1697	done
1698	if [ ${status} -gt 0 ] ; then
1699	echo "IGCM_sys_Get : error"
1700	cat /tmp/out_command.$$
1701	\rm /tmp/out_command.$$
1702	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1703	else
1704	\rm /tmp/out_command.$$
1705	fi
1706	done
1707	else
1708	i=0
1709	while [ $i -lt $NB_ESSAI ] ; do
1710	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1711	status=$?
1712	if [ ${status} -gt 0 ]; then
1713	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1714	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1715	sleep $DELAI
1716	else
1717	break
1718	fi
1719	(( i = i + 1 ))
1720	done
1721	if [ ${status} -gt 0 ] ; then
1722	echo "IGCM_sys_Get : error"
1723	cat /tmp/out_command.$$
1724	\rm /tmp/out_command.$$
1725	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1726	else
1727	\rm /tmp/out_command.$$
1728	fi
1729	fi
1730	fi
1731	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1732	}
1733
1734	#D-#==================================================
1735	#D-function IGCM_sys_GetDate_FichWork
1736	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1737	#D-* Examples:
1738	#D-
1739	function IGCM_sys_GetDate_FichWork {
1740	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1741	if ( $DEBUG_sys ) ; then
1742	echo "IGCM_sys_GetDate_FichWork :" $@
1743	fi
1744
1745	if [ $# -ge 3 ] ; then
1746	mode=$3
1747	TimeStyle=$4
1748	else
1749	mode="default"
1750	TimeStyle="%Y%m%d%H%M%S"
1751	fi
1752
1753	typeset dateF
1754	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1755
1756	case $mode in
1757	"default")
1758	eval ${2}=${dateF[5]}
1759	;;
1760	"SplitFields")
1761	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1762	;;
1763	esac
1764
1765
1766	# donne la date filesys d'un fichier sur la machine work
1767	IGCM_debug_PopStack "IGCM_sys_FichWork"
1768	}
1769
1770	#D-#==================================================
1771	#D-function IGCM_sys_GetDate_FichArchive
1772	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1773	#D-* Examples:
1774	#D-
1775	function IGCM_sys_GetDate_FichArchive {
1776	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1777	if ( $DEBUG_sys ) ; then
1778	echo "IGCM_sys_GetDate_FichArchive :" $@
1779	fi
1780	typeset dateF
1781	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1782	eval ${2}=${dateF[5]}
1783
1784	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1785	}
1786
1787	#D-#==================================================
1788	#D-function IGCM_sys_GetDate_Monitoring
1789	#D-* Purpose: get the last year for which the monitoring has been computed
1790	#D-* Examples:
1791	#D-
1792	function IGCM_sys_GetDate_Monitoring {
1793	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1794	if ( $DEBUG_sys ) ; then
1795	echo "IGCM_sys_GetDate_Monitoring :" $@
1796	fi
1797
1798	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1799
1800	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1801	}
1802
1803	#D-#==================================================
1804	#D-function IGCM_sys_Dods_Rm
1805	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1806	#D-* Examples:
1807	#D-
1808	function IGCM_sys_Dods_Rm {
1809	if ( $DEBUG_sys ) ; then
1810	echo "IGCM_sys_Dods_Rm :" $@
1811	fi
1812	typeset status
1813	if [ $DRYRUN = 0 ]; then
1814
1815	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
1816	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
1817	# echo "Nothing has been done."
1818	# return
1819	# fi
1820
1821	/ccc/cont003/home/dsm/p86ipsl/bin/dods_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
1822	status=$?
1823
1824	# if [ ${status} -gt 0 ] ; then
1825	# echo "IGCM_sys_Dods_Rm : error."
1826	# cat out_dods_rm
1827	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
1828	# else
1829	# rm out_dods_rm
1830	# fi
1831
1832	fi
1833	return $status
1834	}
1835
1836	#D-#==================================================
1837	#D-function IGCM_sys_Dods_Cp
1838	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1839	#D-* Examples:
1840	#D-
1841	function IGCM_sys_Dods_Cp {
1842	if ( $DEBUG_sys ) ; then
1843	echo "IGCM_sys_Dods_Cp :" $@
1844	fi
1845	typeset status
1846	if [ $DRYRUN = 0 ]; then
1847
1848	# if [ ! -d ${R_SAVE}/${1} ] ; then
1849	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
1850	# echo "Nothing has been done."
1851	# return
1852	# fi
1853
1854	/ccc/cont003/home/dsm/p86ipsl/bin/dods_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
1855	status=$?
1856
1857	# if [ ${status} -gt 0 ] ; then
1858	# echo "IGCM_sys_Dods_Cp : error."
1859	# cat out_dods_cp
1860	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
1861	# else
1862	# rm out_dods_cp
1863	# fi
1864
1865	fi
1866	return $status
1867	}
1868
1869	#D-#==================================================
1870	#D-function IGCM_sys_Put_Dods
1871	#D-* Purpose: Put $(ARCHIVE) files on DODS internet protocole.
1872	#D-* Examples:
1873	#D-
1874	function IGCM_sys_Put_Dods {
1875	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1876	if ( $DEBUG_sys ) ; then
1877	echo "IGCM_sys_Put_Dods :" $@
1878	fi
1879	#set -vx
1880	typeset status
1881	if [ $DRYRUN = 0 ]; then
1882	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
1883	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
1884	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
1885	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
1886	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1887	return
1888	fi
1889
1890	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1891	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1892	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1893	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1894	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1895	return
1896	fi
1897	#
1898	if [ -d ${R_SAVE}/${1} ] ; then
1899	cd ${R_SAVE}
1900	elif [ -d ${R_FIGR}/${1} ] ; then
1901	cd ${R_FIGR}
1902	fi
1903
1904	IGCM_sys_Dods_Rm ${1}
1905	IGCM_sys_Dods_Cp ${1}
1906	status=0
1907
1908	if [ ${status} -gt 0 ] ; then
1909	echo "IGCM_sys_Put_Dods : error."
1910	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1911	fi
1912	fi
1913	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1914	}
1915
1916	##############################################################
1917	# REBUILD OPERATOR
1918
1919	#D-#==================================================
1920	#D-function IGCM_sys_sync
1921	#D-* Purpose: flush buffer on disk
1922	#D-* Examples:
1923	#D-
1924	function IGCM_sys_sync {
1925	IGCM_debug_PushStack "IGCM_sys_sync" $@
1926	if ( $DEBUG_sys ) ; then
1927	echo "IGCM_sys_sync :" $@
1928	fi
1929
1930	/bin/sync
1931
1932	IGCM_debug_PopStack "IGCM_sys_sync"
1933	}
1934
1935	#D-#==================================================
1936	#D-function IGCM_sys_rebuild
1937	#D-* Purpose: rebuild parallel files
1938	#D-* Examples:
1939	#D-
1940	function IGCM_sys_rebuild {
1941	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1942	if ( $DEBUG_sys ) ; then
1943	echo "IGCM_sys_rebuild :" $@
1944	fi
1945
1946	typeset NB_ESSAI DELAI status i firstArg
1947	# number of tentative
1948	NB_ESSAI=3
1949	# time delay between tentative
1950	DELAI=2
1951
1952	i=0
1953	while [ $i -lt $NB_ESSAI ] ; do
1954	~p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1955	status=$?
1956	if [ ${status} -gt 0 ] ; then
1957	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1958	cat /tmp/out_command.$$
1959	\rm /tmp/out_command.$$
1960	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1961	firstArg=${1}
1962	\rm ${firstArg}
1963	sleep $DELAI
1964	else
1965	\rm /tmp/out_command.$$
1966	break
1967	fi
1968	(( i = i + 1 ))
1969	done
1970
1971	if [ ${status} -gt 0 ] ; then
1972	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1973	IGCM_debug_Exit "rebuild"
1974	fi
1975
1976	IGCM_debug_PopStack "IGCM_sys_rebuild"
1977	}
1978
1979	#D-#==================================================
1980	#D-function IGCM_sys_rebuild_station
1981	#D-* Purpose: rebuild parallel files describing station
1982	#D-* Examples:
1983	#D-
1984	function IGCM_sys_rebuild_station {
1985	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1986	typeset i list_opt file_in file_out prefix_invert list_invert
1987	if ( $DEBUG_sys ) ; then
1988	echo "IGCM_sys_rebuild_station :" $@
1989	fi
1990	list_opt=$@
1991
1992	# Invert Axis : t,x -> x,t
1993	# t,pres,x -> x,t,pres
1994	# So that we can concatenate along x
1995	i=0
1996	for file_in in ${list_opt} ; do
1997	(( i = i + 1))
1998	[ ${i} = 1 ] && file_out=${file_in} && continue
1999	# detect time counter and do the job only if present
2000	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
2001	if [ X${var_unlim} = Xtime_counter ] ; then
2002	prefix_invert=$( basename ${file_in} .nc )
2003	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
2004	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
2005	fi
2006	done
2007
2008	# Concatenate
2009	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
2010
2011	# Re-ivert file
2012	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
2013
2014	# Station re-ordering is too expansive to be run within libIGCM
2015	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
2016	# This re-ordering must be done "in memory" by the cmorization process
2017	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
2018	# BEGIN reordering
2019
2020	# Only LMDZ text output contains the exact ordering of the station.
2021	# We isolate this in the code below:
2022	# 0 38 -157.5000000000000 70.98591549295774
2023	# 0 54 27.49999999999999 67.18309859154928
2024	# 0 56 -62.50000000000001 82.39436619718309
2025	# 0 79 12.49999999999999 78.59154929577466
2026	# 0 116 -165.0000000000000 76.05633802816901
2027	# 0 117 130.0000000000000 70.98591549295774
2028	# 0 118 110.0000000000000 87.46478873239437
2029	# 1 40 4.999999999999995 51.97183098591550
2030
2031	# typeset iStation iProc list_opt file_in file_out prefix_invert
2032	# typeset -Z4 j4
2033	# typeset -Z3 j3
2034
2035	# unset list_opt
2036	# set +A list_opt $@
2037
2038	# Filename after rebuild
2039	# file_out=${list_opt[0]}
2040	# Prefix of output files
2041	# prefix_invert=$( basename ${file_out} .nc )
2042	# Number of procs
2043	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
2044
2045	# iProc=0
2046	# while [ ${iProc} -lt ${num_proc} ] ; do
2047	# Array containing Station as a number
2048	# unset proc_stn
2049	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
2050	# Number of stations produced by processor proc
2051	# stationLast=${#proc_stn[*]}
2052	# Proc number on 4 digits
2053	# j4=${iProc}
2054	# Init
2055	# iStation=0
2056	# while [ ${iStation} -lt ${stationLast} ] ; do
2057	# Station number on 3 digits
2058	# j3=${proc_stn[${iStation}]}
2059	# Extract station
2060	# Invert Axis : t,x -> x,t
2061	# t,pres,x -> x,t,pres
2062	# So that we can concatenate along x
2063	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
2064	# (( iStation = iStation + 1 ))
2065	# done
2066	# (( iProc = iProc + 1 ))
2067	# done
2068
2069	# Concatenate all station along x
2070	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
2071
2072	# Re-invert file
2073	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
2074
2075	# END reordering
2076
2077	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
2078	}
2079
2080	############################################################
2081	# Activate Running Environnment Variables
2082
2083	#D-#==================================================
2084	#D-function IGCM_sys_desactiv_variables
2085	#D-* Purpose: set environement variables prior to execution
2086	#D-* Examples:
2087	#D-
2088	function IGCM_sys_activ_variables {
2089	IGCM_debug_PushStack "IGCM_sys_activ_variables"
2090	if ( $DEBUG_sys ) ; then
2091	echo "IGCM_sys_activ_variables"
2092	fi
2093
2094	# --------------------------------------------------------------------
2095	#D- MPI specifications
2096	# --------------------------------------------------------------------
2097
2098	# --------------------------------------------------------------------
2099	#D- Other specifications
2100	# --------------------------------------------------------------------
2101
2102	ulimit -s unlimited
2103
2104	IGCM_debug_PopStack "IGCM_sys_activ_variables"
2105	}
2106
2107	############################################################
2108	# Desactivate Running Environnment Variables
2109
2110	#D-#==================================================
2111	#D-function IGCM_sys_desactiv_variables
2112	#D-* Purpose: unset environement variables after execution
2113	#D-* Examples:
2114	#D-
2115	function IGCM_sys_desactiv_variables {
2116	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
2117	if ( $DEBUG_sys ) ; then
2118	echo "IGCM_sys_desactiv_variables"
2119	fi
2120	# --------------------------------------------------------------------
2121	#D- MPI specifications
2122	# --------------------------------------------------------------------
2123
2124	# --------------------------------------------------------------------
2125	#D- Other specifications
2126	# --------------------------------------------------------------------
2127
2128	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
2129	}
2130
2131	############################################################
2132	# Build MPI/OMP scripts run file (dummy function)
2133
2134	#D-#==================================================
2135	#D-function IGCM_sys_build_run_file
2136	#D-* Purpose: build run file (deprecated)
2137	#D-* Examples:
2138	#D-
2139	function IGCM_sys_build_run_file {
2140
2141	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
2142
2143	}
2144
2145	############################################################
2146	# Build MPI/OMP scripts
2147
2148	#D-#==================================================
2149	#D-function IGCM_sys_build_execution_scripts
2150	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
2151	#D-* Examples:
2152	#D-
2153	function IGCM_sys_build_execution_scripts
2154	{
2155	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
2156	if ( $DEBUG_sys ) ; then
2157	echo "IGCM_sys_build_execution_scripts " $@
2158	fi
2159
2160	EXECUTION=${HOST_MPIRUN_COMMAND}
2161
2162	# MPMD mode
2163	if ( ${OK_PARA_MPMD} ) ; then
2164
2165	# Only MPI (MPMD)
2166	if ( ! ${OK_PARA_OMP} ) ; then
2167
2168	if [ -f run_file ] ; then
2169	IGCM_sys_Rm -f run_file
2170	fi
2171	touch run_file
2172
2173	# Build run_file
2174
2175	# First loop on the components for the coupler ie oasis (only if oasis3)
2176	# the coupler ie oasis3 must be the first one
2177	for comp in ${config_ListOfComponents[*]} ; do
2178
2179	eval ExeNameIn=\${config_Executable_${comp}[0]}
2180	eval ExeNameOut=\${config_Executable_${comp}[1]}
2181
2182	# for CPL component only
2183	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
2184
2185	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2186	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2187	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2188	fi
2189	done
2190
2191	# Then second loop on the components
2192	for comp in ${config_ListOfComponents[*]} ; do
2193
2194	eval ExeNameIn=\${config_Executable_${comp}[0]}
2195	eval ExeNameOut=\${config_Executable_${comp}[1]}
2196
2197	# Only if we really have an executable for the component and not the coupler ie oasis:
2198	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2199
2200	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2201	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2202	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2203	fi
2204	done
2205
2206	EXECUTION="${HOST_MPIRUN_COMMAND} -f ./run_file"
2207
2208	IGCM_sys_Chmod u+x run_file
2209	if ( $DEBUG_sys ) ; then
2210	echo "run_file contains : "
2211	cat run_file
2212	fi
2213
2214	# MPI-OpenMP (MPMD)
2215	else
2216
2217	# Use of mpirun instead of ccc_mprun
2218	EXECUTION="time mpirun"
2219
2220	# Hosts treatment
2221	${EXECUTION} hostname \| sort \| uniq > hosts.tmp
2222
2223	i=0
2224	rm -f hosts rankfile
2225	IGCM_debug_Print 1 "sys Curie, Hosts available :"
2226	for nodes in `cat hosts.tmp`
2227	do
2228	host[$i]=$nodes
2229	echo "${host[$i]}" >> hosts
2230	IGCM_debug_Print 1 ${host[$i]}
2231	i=$((i+1))
2232	done
2233	rm -f hosts.tmp
2234
2235	listnodes=${host[*]}
2236
2237	EXECUTION="${EXECUTION} -hostfile hosts -rankfile rankfile"
2238
2239	# Initialisation
2240	rank=0
2241	current_core=0
2242	core_per_node=16
2243	init_exec=n
2244
2245	# Loop on the components
2246	for comp in ${config_ListOfComponents[*]} ; do
2247
2248	eval ExeNameIn=\${config_Executable_${comp}[0]}
2249	eval ExeNameOut=\${config_Executable_${comp}[1]}
2250
2251	# Not possible if oasis has an executable (i.e old version of oasis3)
2252	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
2253	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
2254	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
2255	IGCM_debug_Verif_Exit
2256	fi
2257
2258	# Only if we really have an executable for the component :
2259	if [ "X${ExeNameOut}" != X\"\" ] ; then
2260
2261	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2262	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2263
2264	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2265	echo "" >> script_${ExeNameOut}.ksh
2266	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
2267
2268	# Check if the number of threads is correct
2269	case ${comp_proc_omp_loc} in
2270	2\|4\|8\|16)
2271	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2272	;;
2273	*)
2274	IGCM_debug_Exit "ERROR with OMP parameters !"
2275	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2276	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2277	IGCM_debug_Verif_Exit
2278	;;
2279	esac
2280	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2281	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2282	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2283	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2284	fi
2285	#echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
2286	#echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
2287	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2288	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2289
2290	if [ ${init_exec} = y ] ; then
2291	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2292	else
2293	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2294	init_exec=y
2295	fi
2296
2297	# Build rankfile : method used to assign cores and nodes for the MPI process
2298	# Ex :
2299	#rank 0=curie5296 slot=0,2,4,6
2300	#rank 1=curie5296 slot=8,10,12,14
2301	# Example of final command :
2302	# mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
2303	# with script_lmdz.x.ksh :
2304	# #!/bin/ksh
2305	#export KMP_STACKSIZE=3g
2306	#export KMP_LIBRARY=turnaround
2307	#export MKL_SERIAL=YES
2308	#OMP_NUM_THREADS=4
2309	#./lmdz.x
2310	#
2311
2312	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
2313	(( index_host = current_core / core_per_node ))
2314	host_value=${host[${index_host}]}
2315	(( slot = current_core % core_per_node ))
2316	virg=","
2317	string_final=""
2318	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
2319	(( index_slot = index * 2 ))
2320	string=$index_slot$virg
2321	string_final=$string_final$string
2322	done
2323	string_final=$( echo $string_final \| sed "s/.$//" )
2324	echo "rank $rank=$host_value slot=$string_final" >> rankfile
2325	(( rank = rank + 1 ))
2326	(( current_core = current_core + comp_proc_omp_loc ))
2327	done
2328	fi
2329
2330	done
2331	fi
2332
2333	# Only one executable (SPMD mode).
2334	else
2335
2336	for comp in ${config_ListOfComponents[*]} ; do
2337
2338	# Only if we really have an executable for the component :
2339	eval ExeNameOut=\${config_Executable_${comp}[1]}
2340	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2341
2342	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2343	echo "" >> script_${ExeNameOut}.ksh
2344	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2345
2346	if ( ${OK_PARA_OMP} ) ; then
2347	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2348	echo "" >> script_${ExeNameOut}.ksh
2349	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2350	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2351	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2352	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2353	fi
2354
2355	if ( ${OK_PARA_MPI} ) ; then
2356	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2357	# Default : ccc_mprun used if nb_proc gt 1
2358	# to have out/err per process on different files
2359	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
2360	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2361	EXECUTION="${HOST_MPIRUN_COMMAND} -n ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2362	else
2363	# Default : ccc_mprun is NOT used if nb_proc eq 1
2364	# to have out/err per process on different files
2365	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2366	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2367	EXECUTION="/usr/bin/time ./script_${ExeNameOut}.ksh"
2368	fi
2369
2370	IGCM_debug_Print 1 "sys Curie : script_${ExeNameOut}.ksh contains"
2371	cat script_${ExeNameOut}.ksh
2372
2373	fi
2374
2375	done
2376
2377	fi
2378
2379	IGCM_debug_Print 1 "sys Curie : execution command is "
2380	IGCM_debug_Print 1 "$EXECUTION"
2381
2382	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2383	}
2384
2385	#D-#==================================================
2386	#D-function IGCM_sys_check_path
2387	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2388	#D-* do not point to an important use directory. Stop immediately in that case.
2389	#D-* Examples:
2390	#D-
2391	function IGCM_sys_check_path {
2392	IGCM_debug_PushStack "IGCM_sys_check_path"
2393	if ( $DEBUG_sys ) ; then
2394	echo "IGCM_sys_check_path"
2395	fi
2396
2397	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCHDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCWORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCSTOREDIR} ] ) ; then
2398	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2399	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2400	IGCM_debug_Exit "This will stop the job"
2401	fi
2402	IGCM_debug_PopStack "IGCM_sys_check_path"
2403	}
2404
2405	#D-#==================================================
2406	#D-function IGCM_sys_check_quota
2407	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2408	#D-* Examples:
2409	#D-
2410	function IGCM_sys_check_quota {
2411	IGCM_debug_PushStack "IGCM_sys_check_quota"
2412	if ( $DEBUG_sys ) ; then
2413	echo "IGCM_sys_check_quota"
2414	fi
2415	# Limit of quota (in %)
2416	limit_quota=90
2417
2418	# Check of the volume
2419	volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
2420	volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
2421
2422	if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
2423
2424	unit_avail=${volume_avail: -1}
2425	unit_quota=${volume_quota: -1}
2426
2427	if [ "${unit_quota}" = "*" ] ; then
2428	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2429	IGCM_debug_Print 1 "More than 100% of your quota is used"
2430	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2431	IGCM_debug_Print 1 "You must have more than 10% available to run"
2432	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2433	IGCM_debug_Verif_Exit
2434	fi
2435
2436	temp_avail=${volume_avail%%${volume_avail: -1}*}
2437	temp_quota=${volume_quota%%${volume_quota: -1}*}
2438
2439	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2440
2441	# Convertion
2442	if [ ${volume_avail: -1} = "T" ] ; then
2443	(( temp_avail = temp_avail * 1000000000000 ))
2444	elif [ ${volume_avail: -1} = "G" ] ; then
2445	(( temp_avail = temp_avail * 1000000000 ))
2446	elif [ ${volume_avail: -1} = "M" ] ; then
2447	(( temp_avail = temp_avail * 1000000 ))
2448	elif [ ${volume_avail: -1} = "k" ] ; then
2449	(( temp_avail = temp_avail * 1000 ))
2450	else
2451	(( temp_avail = volume_avail ))
2452	fi
2453	if [ ${volume_quota: -1} = "T" ] ; then
2454	(( temp_quota = temp_quota * 1000000000000 ))
2455	elif [ ${volume_quota: -1} = "G" ] ; then
2456	(( temp_quota = temp_quota * 1000000000 ))
2457	elif [ ${volume_quota: -1} = "M" ] ; then
2458	(( temp_quota = temp_quota * 1000000 ))
2459	elif [ ${volume_quota: -1} = "k" ] ; then
2460	(( temp_quota = temp_quota * 1000 ))
2461	else
2462	(( temp_quota = volume_quota ))
2463	fi
2464	fi
2465
2466	quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2467	# echo "volume ratio is " $quota_volume
2468
2469	if [ ${quota_volume} -ge ${limit_quota} ] ; then
2470	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2471	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2472	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2473	IGCM_debug_Print 1 "You must have more than 10% available to run"
2474	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2475	IGCM_debug_Verif_Exit
2476	fi
2477
2478	fi
2479
2480	# Check of the number of inodes
2481
2482	inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
2483	inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
2484
2485	if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
2486
2487	unit_avail=${inode_avail: -1}
2488	unit_quota=${inode_quota: -1}
2489
2490	if [ "${unit_quota}" = "*" ] ; then
2491	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2492	IGCM_debug_Print 1 "More than 100% of your quota is used"
2493	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2494	IGCM_debug_Print 1 "You must have more than 10% available to run"
2495	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2496	IGCM_debug_Verif_Exit
2497	fi
2498
2499	temp_avail=${inode_avail%%${inode_avail: -1}*}
2500	temp_quota=${inode_quota%%${inode_quota: -1}*}
2501
2502	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2503
2504	# Convertion
2505	if [ ${inode_avail: -1} = "T" ] ; then
2506	(( temp_avail = temp_avail * 1000000000000 ))
2507	elif [ ${inode_avail: -1} = "G" ] ; then
2508	(( temp_avail = temp_avail * 1000000000 ))
2509	elif [ ${inode_avail: -1} = "M" ] ; then
2510	(( temp_avail = temp_avail * 1000000 ))
2511	elif [ ${inode_avail: -1} = "k" ] ; then
2512	(( temp_avail = temp_avail * 1000 ))
2513	else
2514	(( temp_avail = inode_avail ))
2515	fi
2516
2517	if [ ${inode_quota: -1} = "T" ] ; then
2518	(( temp_quota = temp_quota * 1000000000000 ))
2519	elif [ ${inode_quota: -1} = "G" ] ; then
2520	(( temp_quota = temp_quota * 1000000000 ))
2521	elif [ ${inode_quota: -1} = "M" ] ; then
2522	(( temp_quota = temp_quota * 1000000 ))
2523	elif [ ${inode_quota: -1} = "k" ] ; then
2524	(( temp_quota = temp_quota * 1000 ))
2525	else
2526	(( temp_quota = inode_quota ))
2527	fi
2528	fi
2529	quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2530	# echo "inode ratio is " $quota_inode
2531
2532	if [ ${quota_inode} -ge ${limit_quota} ] ; then
2533	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2534	IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
2535	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2536	IGCM_debug_Print 1 "You must have more than 10% available to run"
2537	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2538	IGCM_debug_Verif_Exit
2539	fi
2540	fi
2541	IGCM_debug_PopStack "IGCM_sys_check_quota"
2542	}
2543
2544	#D-#==================================================
2545	#D-function IGCM_sys_GetJobID
2546	#D-* Purpose: Check if job_name is currently
2547	#D- running or in queue
2548	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2549	#D-
2550	function IGCM_sys_GetJobID {
2551	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2552	if ( $DEBUG_sys ) ; then
2553	echo "IGCM_sys_GetJobID"
2554	fi
2555
2556	# With -f option, the full job name is given in the last column
2557	ID=$( ccc_mstat -f -u $2 \| \
2558	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2559
2560	eval ${3}=${ID}
2561
2562	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2563	}
2564
2565	#D-#==================================================
2566	#D-function IGCM_sys_CountJobInQueue
2567	#D-* Purpose: Check if job_name is currently
2568	#D- running or in queue
2569	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2570	#D-
2571	function IGCM_sys_CountJobInQueue {
2572	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2573	if ( $DEBUG_sys ) ; then
2574	echo "IGCM_sys_CountJobInQueue"
2575	fi
2576
2577	#NbRun=$( ccc_mstat -f \| grep -c ${JobName} )
2578
2579	# With -f option, the full job name is given in the last column
2580	NbRun=$( ccc_mstat -f \| gawk -v JobName=$1 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
2581
2582	eval ${2}=${NbRun}
2583
2584	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2585	}
2586
2587	#D-#==================================================
2588	#D-function IGCM_sys_ListJobInQueue
2589	#D-* Purpose: Check if job_name is currently
2590	#D- running or in queue
2591	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2592	#D-
2593	function IGCM_sys_ListJobInQueue {
2594	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2595	if ( $DEBUG_sys ) ; then
2596	echo "IGCM_sys_ListJobInQueue"
2597	fi
2598
2599	# With -f option, the full job name is given in the last column
2600	set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
2601	'( $2 == User && \
2602	$NF != /TS/ && \
2603	$NF !~ /PACK/ && \
2604	$NF !~ /REBUILD/ && \
2605	$NF !~ /pack/ ) \
2606	{ print $NF }' \| sed -e "s/$.$\.[0-9]/\1/" )
2607
2608	eval set -A ${2} ${JobList[*]}
2609
2610	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2611	}
2612
2613	##############################################################
2614	# NCO OPERATOR
2615
2616	#D-#==================================================
2617	#D-function IGCM_sys_ncap2
2618	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2619	#D-* Examples:
2620	#D-
2621	function IGCM_sys_ncap2 {
2622	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2623	if ( $DEBUG_sys ) ; then
2624	echo "IGCM_sys_ncap2 :" $@
2625	fi
2626
2627	typeset NB_ESSAI DELAI status i
2628	# number of tentative
2629	NB_ESSAI=3
2630	# time delay between tentative
2631	DELAI=2
2632
2633	i=0
2634	while [ $i -lt $NB_ESSAI ] ; do
2635	ncap2 -C "$@" > /tmp/out_command.$$ 2>&1
2636	status=$?
2637	if [ ${status} -gt 0 ] ; then
2638	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2639	cat /tmp/out_command.$$
2640	\rm /tmp/out_command.$$
2641	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2642	sleep $DELAI
2643	else
2644	\rm /tmp/out_command.$$
2645	break
2646	fi
2647	(( i = i + 1 ))
2648	done
2649
2650	if [ ${status} -gt 0 ] ; then
2651	echo "IGCM_sys_ncap2 : ncap2 error"
2652	IGCM_debug_Exit "ncap2"
2653	fi
2654
2655	IGCM_debug_PopStack "IGCM_sys_ncap2"
2656	}
2657
2658	#D-#==================================================
2659	#D-function IGCM_sys_ncatted
2660	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2661	#D-* Examples:
2662	#D-
2663	function IGCM_sys_ncatted {
2664	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2665	if ( $DEBUG_sys ) ; then
2666	echo "IGCM_sys_ncatted :" $@
2667	fi
2668
2669	typeset NB_ESSAI DELAI status i
2670	# number of tentative
2671	NB_ESSAI=3
2672	# time delay between tentative
2673	DELAI=2
2674
2675	i=0
2676	while [ $i -lt $NB_ESSAI ] ; do
2677	ncatted "$@" > /tmp/out_command.$$ 2>&1
2678	status=$?
2679	if [ ${status} -gt 0 ] ; then
2680	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2681	cat /tmp/out_command.$$
2682	\rm /tmp/out_command.$$
2683	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2684	sleep $DELAI
2685	else
2686	\rm /tmp/out_command.$$
2687	break
2688	fi
2689	(( i = i + 1 ))
2690	done
2691
2692	if [ ${status} -gt 0 ] ; then
2693	echo "IGCM_sys_ncatted : ncatted error"
2694	IGCM_debug_Exit "ncatted"
2695	fi
2696
2697	IGCM_debug_PopStack "IGCM_sys_ncatted"
2698	}
2699
2700	#D-#==================================================
2701	#D-function IGCM_sys_ncbo
2702	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2703	#D-* Examples:
2704	#D-
2705	function IGCM_sys_ncbo {
2706	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2707	if ( $DEBUG_sys ) ; then
2708	echo "IGCM_sys_ncbo :" $@
2709	fi
2710
2711	typeset NB_ESSAI DELAI status i
2712	# number of tentative
2713	NB_ESSAI=3
2714	# time delay between tentative
2715	DELAI=2
2716
2717	i=0
2718	while [ $i -lt $NB_ESSAI ] ; do
2719	ncbo -C $@ > /tmp/out_command.$$ 2>&1
2720	status=$?
2721	if [ ${status} -gt 0 ] ; then
2722	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2723	cat /tmp/out_command.$$
2724	\rm /tmp/out_command.$$
2725	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2726	sleep $DELAI
2727	else
2728	\rm /tmp/out_command.$$
2729	break
2730	fi
2731	(( i = i + 1 ))
2732	done
2733
2734	if [ ${status} -gt 0 ] ; then
2735	echo "IGCM_sys_ncbo : ncbo error"
2736	IGCM_debug_Exit "ncbo"
2737	fi
2738
2739	IGCM_debug_PopStack "IGCM_sys_ncbo"
2740	}
2741
2742	#D-#==================================================
2743	#D-function IGCM_sys_ncdif
2744	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2745	#D-* Examples:
2746	#D-
2747	function IGCM_sys_ncdiff {
2748	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2749	if ( $DEBUG_sys ) ; then
2750	echo "IGCM_sys_ncdiff :" $@
2751	fi
2752
2753	typeset NB_ESSAI DELAI status i
2754	# number of tentative
2755	NB_ESSAI=3
2756	# time delay between tentative
2757	DELAI=2
2758
2759	i=0
2760	while [ $i -lt $NB_ESSAI ] ; do
2761	ncdiff -C $@ > /tmp/out_command.$$ 2>&1
2762	status=$?
2763	if [ ${status} -gt 0 ] ; then
2764	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2765	cat /tmp/out_command.$$
2766	\rm /tmp/out_command.$$
2767	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2768	sleep $DELAI
2769	else
2770	\rm /tmp/out_command.$$
2771	break
2772	fi
2773	(( i = i + 1 ))
2774	done
2775
2776	if [ ${status} -gt 0 ] ; then
2777	echo "IGCM_sys_ncdiff : ncdiff error"
2778	IGCM_debug_Exit "ncdiff"
2779	fi
2780
2781	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2782	}
2783
2784	#D-#==================================================
2785	#D-function IGCM_sys_ncea
2786	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2787	#D-* Examples:
2788	#D-
2789	function IGCM_sys_ncea {
2790	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2791	if ( $DEBUG_sys ) ; then
2792	echo "IGCM_sys_ncea :" $@
2793	fi
2794
2795	typeset NB_ESSAI DELAI status i
2796	# number of tentative
2797	NB_ESSAI=3
2798	# time delay between tentative
2799	DELAI=2
2800
2801	i=0
2802	while [ $i -lt $NB_ESSAI ] ; do
2803	ncea -C $@ > /tmp/out_command.$$ 2>&1
2804	status=$?
2805	if [ ${status} -gt 0 ] ; then
2806	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2807	cat /tmp/out_command.$$
2808	\rm /tmp/out_command.$$
2809	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2810	sleep $DELAI
2811	else
2812	\rm /tmp/out_command.$$
2813	break
2814	fi
2815	(( i = i + 1 ))
2816	done
2817
2818	if [ ${status} -gt 0 ] ; then
2819	echo "IGCM_sys_ncea : ncea error"
2820	IGCM_debug_Exit "ncea"
2821	fi
2822
2823	IGCM_debug_PopStack "IGCM_sys_ncea"
2824	}
2825
2826	#D-#==================================================
2827	#D-function IGCM_sys_ncecat
2828	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2829	#D-* Examples:
2830	#D-
2831	function IGCM_sys_ncecat {
2832	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2833	if ( $DEBUG_sys ) ; then
2834	echo "IGCM_sys_ncecat :" $@
2835	fi
2836
2837	typeset NB_ESSAI DELAI status i
2838	# number of tentative
2839	NB_ESSAI=3
2840	# time delay between tentative
2841	DELAI=2
2842
2843	i=0
2844	while [ $i -lt $NB_ESSAI ] ; do
2845	ncecat -C $@ > /tmp/out_command.$$ 2>&1
2846	status=$?
2847	if [ ${status} -gt 0 ] ; then
2848	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2849	cat /tmp/out_command.$$
2850	\rm /tmp/out_command.$$
2851	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2852	sleep $DELAI
2853	else
2854	\rm /tmp/out_command.$$
2855	break
2856	fi
2857	(( i = i + 1 ))
2858	done
2859
2860	if [ ${status} -gt 0 ] ; then
2861	echo "IGCM_sys_ncecat : ncecat error"
2862	IGCM_debug_Exit "ncecat"
2863	fi
2864
2865	IGCM_debug_PopStack "IGCM_sys_ncecat"
2866	}
2867
2868	#D-#==================================================
2869	#D-function IGCM_sys_ncflint
2870	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2871	#D-* Examples:
2872	#D-
2873	function IGCM_sys_ncflint {
2874	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2875	if ( $DEBUG_sys ) ; then
2876	echo "IGCM_sys_ncflint :" $@
2877	fi
2878
2879	typeset NB_ESSAI DELAI status i
2880	# number of tentative
2881	NB_ESSAI=3
2882	# time delay between tentative
2883	DELAI=2
2884
2885	i=0
2886	while [ $i -lt $NB_ESSAI ] ; do
2887	ncflint -C $@ > /tmp/out_command.$$ 2>&1
2888	status=$?
2889	if [ ${status} -gt 0 ] ; then
2890	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2891	cat /tmp/out_command.$$
2892	\rm /tmp/out_command.$$
2893	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2894	sleep $DELAI
2895	else
2896	\rm /tmp/out_command.$$
2897	break
2898	fi
2899	(( i = i + 1 ))
2900	done
2901
2902	if [ ${status} -gt 0 ] ; then
2903	echo "IGCM_sys_ncflint : ncflint error"
2904	IGCM_debug_Exit "ncflint"
2905	fi
2906
2907	IGCM_debug_PopStack "IGCM_sys_ncflint"
2908	}
2909
2910	#D-#==================================================
2911	#D-function IGCM_sys_ncks
2912	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2913	#D-* Examples:
2914	#D-
2915	function IGCM_sys_ncks {
2916	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2917	if ( $DEBUG_sys ) ; then
2918	echo "IGCM_sys_ncks :" $@
2919	fi
2920
2921	typeset NB_ESSAI DELAI status i
2922	# number of tentative
2923	NB_ESSAI=3
2924	# time delay between tentative
2925	DELAI=2
2926
2927	i=0
2928	while [ $i -lt $NB_ESSAI ] ; do
2929	ncks -C $@ > /tmp/out_command.$$ 2>&1
2930	status=$?
2931	if [ ${status} -gt 0 ] ; then
2932	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2933	cat /tmp/out_command.$$
2934	\rm /tmp/out_command.$$
2935	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2936	sleep $DELAI
2937	else
2938	\rm /tmp/out_command.$$
2939	break
2940	fi
2941	(( i = i + 1 ))
2942	done
2943
2944	if [ ${status} -gt 0 ] ; then
2945	echo "IGCM_sys_ncks : ncks error"
2946	IGCM_debug_Exit "ncks"
2947	fi
2948
2949	IGCM_debug_PopStack "IGCM_sys_ncks"
2950	}
2951
2952	#D-#==================================================
2953	#D-function IGCM_sys_ncpdq
2954	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2955	#D-* Examples:
2956	#D-
2957	function IGCM_sys_ncpdq {
2958	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2959	if ( $DEBUG_sys ) ; then
2960	echo "IGCM_sys_ncpdq :" $@
2961	fi
2962
2963	typeset NB_ESSAI DELAI status i
2964	# number of tentative
2965	NB_ESSAI=3
2966	# time delay between tentative
2967	DELAI=2
2968
2969	i=0
2970	while [ $i -lt $NB_ESSAI ] ; do
2971	ncpdq -C $@ > /tmp/out_command.$$ 2>&1
2972	status=$?
2973	if [ ${status} -gt 0 ] ; then
2974	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2975	cat /tmp/out_command.$$
2976	\rm /tmp/out_command.$$
2977	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2978	sleep $DELAI
2979	else
2980	\rm /tmp/out_command.$$
2981	break
2982	fi
2983	(( i = i + 1 ))
2984	done
2985
2986	if [ ${status} -gt 0 ] ; then
2987	echo "IGCM_sys_ncpdq : ncpdq error"
2988	IGCM_debug_Exit "ncpdq"
2989	fi
2990
2991	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2992	}
2993
2994	#D-#==================================================
2995	#D-function IGCM_sys_ncra
2996	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2997	#D-* Examples:
2998	#D-
2999	function IGCM_sys_ncra {
3000	IGCM_debug_PushStack "IGCM_sys_ncra" $@
3001	if ( $DEBUG_sys ) ; then
3002	echo "IGCM_sys_ncra :" $@
3003	fi
3004
3005	typeset NB_ESSAI DELAI status i
3006	# number of tentative
3007	NB_ESSAI=3
3008	# time delay between tentative
3009	DELAI=2
3010
3011	i=0
3012	while [ $i -lt $NB_ESSAI ] ; do
3013	ncra -C $@ > /tmp/out_command.$$ 2>&1
3014	status=$?
3015	if [ ${status} -gt 0 ] ; then
3016	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
3017	cat /tmp/out_command.$$
3018	\rm /tmp/out_command.$$
3019	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3020	sleep $DELAI
3021	else
3022	\rm /tmp/out_command.$$
3023	break
3024	fi
3025	(( i = i + 1 ))
3026	done
3027
3028	if [ ${status} -gt 0 ] ; then
3029	echo "IGCM_sys_ncra : ncra error"
3030	IGCM_debug_Exit "ncra"
3031	fi
3032
3033	IGCM_debug_PopStack "IGCM_sys_ncra"
3034	}
3035
3036	#D-#==================================================
3037	#D-function IGCM_sys_ncrcat
3038	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
3039	#D-* Examples:
3040	#D-
3041	function IGCM_sys_ncrcat {
3042	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
3043	if ( $DEBUG_sys ) ; then
3044	echo "IGCM_sys_ncrcat :" $@
3045	fi
3046
3047	typeset NB_ESSAI DELAI status i lastArg
3048	# number of tentative
3049	NB_ESSAI=3
3050	# time delay between tentative
3051	DELAI=2
3052
3053	i=0
3054	while [ $i -lt $NB_ESSAI ] ; do
3055	ncrcat -C $@ > /tmp/out_command.$$ 2>&1
3056	status=$?
3057	if [ ${status} -gt 0 ] ; then
3058	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
3059	cat /tmp/out_command.$$
3060	\rm /tmp/out_command.$$
3061	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3062	sleep $DELAI
3063	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
3064	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
3065	cat /tmp/out_command.$$
3066	# remove files having corrupted time axis
3067	eval lastArg=\${$#}
3068	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
3069	\rm ${lastArg}
3070	\rm /tmp/out_command.$$
3071	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3072	sleep $DELAI
3073	else
3074	\rm /tmp/out_command.$$
3075	break
3076	fi
3077	(( i = i + 1 ))
3078	done
3079
3080	if [ ${status} -gt 0 ] ; then
3081	echo "IGCM_sys_ncrcat : ncrcat error"
3082	#IGCM_debug_Exit "ncrcat"
3083	fi
3084
3085	IGCM_debug_PopStack "IGCM_sys_ncrcat"
3086	}
3087
3088	#D-#==================================================
3089	#D-function IGCM_sys_ncrename
3090	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
3091	#D-* Examples:
3092	#D-
3093	function IGCM_sys_ncrename {
3094	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
3095	if ( $DEBUG_sys ) ; then
3096	echo "IGCM_sys_ncrename :" $@
3097	fi
3098
3099	typeset NB_ESSAI DELAI status i
3100	# number of tentative
3101	NB_ESSAI=3
3102	# time delay between tentative
3103	DELAI=2
3104
3105	i=0
3106	while [ $i -lt $NB_ESSAI ] ; do
3107	ncrename $@ > /tmp/out_command.$$ 2>&1
3108	status=$?
3109	if [ ${status} -gt 0 ] ; then
3110	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
3111	cat /tmp/out_command.$$
3112	\rm /tmp/out_command.$$
3113	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3114	sleep $DELAI
3115	else
3116	\rm /tmp/out_command.$$
3117	break
3118	fi
3119	(( i = i + 1 ))
3120	done
3121
3122	if [ ${status} -gt 0 ] ; then
3123	echo "IGCM_sys_ncrename : ncrename error"
3124	IGCM_debug_Exit "ncrename"
3125	fi
3126
3127	IGCM_debug_PopStack "IGCM_sys_ncrename"
3128	}
3129
3130	#D-#==================================================
3131	#D-function IGCM_sys_ncwa
3132	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
3133	#D-* Examples:
3134	#D-
3135	function IGCM_sys_ncwa {
3136	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
3137	if ( $DEBUG_sys ) ; then
3138	echo "IGCM_sys_ncwa :" $@
3139	fi
3140
3141	typeset NB_ESSAI DELAI status i
3142	# number of tentative
3143	NB_ESSAI=3
3144	# time delay between tentative
3145	DELAI=2
3146
3147	i=0
3148	while [ $i -lt $NB_ESSAI ] ; do
3149	ncwa -C $@ > /tmp/out_command.$$ 2>&1
3150	status=$?
3151	if [ ${status} -gt 0 ] ; then
3152	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
3153	cat /tmp/out_command.$$
3154	\rm /tmp/out_command.$$
3155	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3156	sleep $DELAI
3157	else
3158	\rm /tmp/out_command.$$
3159	break
3160	fi
3161	(( i = i + 1 ))
3162	done
3163
3164	if [ ${status} -gt 0 ] ; then
3165	echo "IGCM_sys_ncwa : ncwa error"
3166	IGCM_debug_Exit "ncwa"
3167	fi
3168
3169	IGCM_debug_PopStack "IGCM_sys_ncwa"
3170	}
3171
3172	##############################################################
3173	# CDO OPERATOR
3174
3175	#D-#==================================================
3176	#D-function IGCM_sys_cdo
3177	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
3178	#D-* Examples:
3179	#D-
3180	function IGCM_sys_cdo {
3181	IGCM_debug_PushStack "IGCM_sys_cdo" $@
3182	if ( $DEBUG_sys ) ; then
3183	echo "IGCM_sys_cdo :" $@
3184	fi
3185
3186	typeset status
3187
3188	\cdo $@ > /tmp/out_command.$$ 2>&1
3189	status=$?
3190	if [ ${status} -gt 0 ] ; then
3191	echo "IGCM_sys_cdo : error code ${status}"
3192	cat /tmp/out_command.$$
3193	\rm /tmp/out_command.$$
3194	IGCM_debug_PopStack "IGCM_sys_cdo"
3195	return 1
3196	else
3197	IGCM_debug_PopStack "IGCM_sys_cdo"
3198	return 0
3199	fi
3200
3201	IGCM_debug_PopStack "IGCM_sys_cdo"
3202	}

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_curie.ksh @ 1173

Download in other formats: