Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_brodie.ksh @ 437

Last change on this file since 437 was 437, checked in by sdipsl, 13 years ago
Send mail notification when computing job stops It could be that we send a notification if a post-processing job stops in PROD mode ---> need some refactoring and caution...
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 32.3 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sébastien Denvil
5	# Contact: sebastien.denvil__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#=================================================
21	#D-LibIGCM_sys for Brodie SX8
22	#D-#=================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe param files \| \| Chmod \| \|
46	# \| \| Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset -r MASTER=brodie
71
72	#D-
73	#D-#==================================================
74	#D-Program used in libIGCM
75	#D-#==================================================
76
77	# rsync with path
78	typeset -r RSYNC=/home/rech/ces/rces452/RSYNC_SX8/bin/rsync
79	# RSYNC_opt args to rsync
80	typeset -r RSYNC_opt="-va"
81	# ie storage filesystem
82	typeset -r RHOST=gaya
83	typeset -r REMOTE_RSYNC=/u/rech/ces/rces452/RSYNC/bin/rsync
84
85	#====================================================
86	# Host specific DIRECTORIES
87	#====================================================
88
89	#====================================================
90	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
91	typeset -r R_EXE="${MODIPSL}/bin"
92
93	#====================================================
94	#- SUBMIT_DIR : submission dir
95	typeset SUBMIT_DIR=${SUBMIT_DIR:=${PBS_O_WORKDIR}}
96
97	#====================================================
98	#- ARCHIVE
99	typeset -r ARCHIVE=$( rsh gaya -n env \| grep PWD \| gawk "-F=" '{print $2}' )
100
101	#====================================================
102	#- Mirror libIGCM from brodie to ulam
103	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
104
105	#====================================================
106	#- libIGCM_POST
107	PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
108	typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
109
110	#====================================================
111	#- IN
112	typeset -r R_IN=${R_IN:=/u/rech/psl/rpsl035/IGCM}
113	typeset -r R_IN_ECMWF=${R_IN:=/u/rech/psl/rpsl376}
114
115	#====================================================
116	#- OUT
117	typeset -r R_OUT=${ARCHIVE}/IGCM_OUT
118
119	#====================================================
120	#- OUT_POST
121	typeset -r R_OUT_POST=$( rsh ulam -n echo $WORKDIR \| gawk "-F=" "{print $2}" )/IGCM_OUT
122
123	#====================================================
124	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
125	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${TMPDIR}}
126
127	#====================================================
128	#- BIG_DIR : BIG_DIR to store files waiting for rebuild
129	typeset -r BIG_DIR=${BIG_DIR:=${WORKDIR}/REBUILD}
130
131	#====================================================
132	#- HOST_MPIRUN_COMMAND
133	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="mpirun"}
134
135	#====================================================
136	#- Max number of arguments passed to nco operator or demigration command
137	UNIX_MAX_LIMIT=360
138
139	#D-#==================================================
140	#D-function IGCM_sys_RshArchive
141	#D-* Purpose: Archive rsh command
142	#D-* Examples:
143	#D-
144	function IGCM_sys_RshArchive {
145	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
146	rsh gaya exec /bin/ksh <<-EOF
147	${@}
148	EOF
149	if [ $? -gt 0 ] ; then
150	echo "IGCM_sys_RshArchive : erreur."
151	IGCM_debug_Exit "IGCM_sys_RshArchive"
152	fi
153	IGCM_debug_PopStack "IGCM_sys_RshArchive"
154	}
155
156	#D-#==================================================
157	#D-function IGCM_sys_RshPost
158	#D-* Purpose: Post-process rsh command
159	#D-* Examples:
160	#D-
161	function IGCM_sys_RshPost {
162	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
163	if ( $DEBUG_sys ) ; then
164	echo "IGCM_sys_RshPost :" $@
165	fi
166	rsh ulam exec /bin/ksh ${@}
167	if [ $? -gt 0 ] ; then
168	echo "IGCM_sys_RshPost : erreur."
169	IGCM_debug_Exit "IGCM_sys_RshPost"
170	fi
171	IGCM_debug_PopStack "IGCM_sys_RshPost"
172	}
173
174	#D-#==================================================
175	#D-function IGCM_sys_SendMail
176	#D-* Purpose: Send mail when simulation is over
177	#D-* Examples:
178	#D-
179	function IGCM_sys_SendMail {
180	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
181	if ( $DEBUG_sys ) ; then
182	echo "IGCM_sys_SendMail :" $@
183	fi
184
185	if ( ${ExitFlag} ) ; then
186	status=FAILED
187	cat << END_MAIL > job_end.mail
188	Dear ${LOGIN},
189
190	Simulation ${config_UserChoices_JobName} failed on supercomputer `hostname`.
191	Job started : ${PeriodDateBegin}
192	Job ended : ${PeriodDateEnd}
193	Output files are available in ${R_SAVE}
194	Script files, Script Outputs and Debug files are available in ${SUBMIT_DIR}
195	END_MAIL
196	else
197	status=COMPLETED
198	cat << END_MAIL > job_end.mail
199	Dear ${LOGIN},
200
201	Simulation ${config_UserChoices_JobName} is finished on supercomputer `hostname`.
202	Job started : ${DateBegin}
203	Job ended : ${DateEnd}
204	Output files are available in ${R_SAVE}
205	Script files and Script Outputs are available in ${SUBMIT_DIR}
206	END_MAIL
207
208	fi
209
210	if [ X${config_UserChoices_MailName} != X ] ; then
211	mailx -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < job_end.mail
212	elif [ -f ~/.forward ] ; then
213	mailx -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < job_end.mail
214	else
215	mailx -s "${config_UserChoices_JobName} ${status}" ${USER} < job_end.mail
216	fi
217
218	if [ $? -gt 0 ] ; then
219	echo "IGCM_sys_SendMail : erreur."
220	IGCM_debug_Exit "IGCM_sys_SendMail"
221	fi
222	IGCM_debug_PopStack "IGCM_sys_SendMail"
223	}
224
225	#D-#==================================================
226	#D-function IGCM_sys_Mkdir
227	#D-* Purpose: Master locale mkdir command
228	#D-* Examples:
229	#D-
230	function IGCM_sys_Mkdir {
231	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
232	if ( $DEBUG_sys ) ; then
233	echo "IGCM_sys_Mkdir :" $@
234	fi
235	if [ ! -d ${1} ]; then
236	\mkdir -p $1
237	if [ $? -gt 0 ] ; then
238	echo "IGCM_sys_Mkdir : erreur."
239	IGCM_debug_Exit "IGCM_sys_Mkdir"
240	fi
241	fi
242	# vérification :
243	if [ ! -d ${1} ] ; then
244	echo "IGCM_sys_Mkdir : erreur."
245	IGCM_debug_Exit "IGCM_sys_Mkdir"
246	fi
247	IGCM_debug_PopStack "IGCM_sys_Mkdir"
248	}
249
250	#D-#==================================================
251	#D-function IGCM_sys_MkdirArchive
252	#D-* Purpose: Mkdir on Archive
253	#D-* Examples:
254	#D-
255	function IGCM_sys_MkdirArchive {
256	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
257	if ( $DEBUG_sys ) ; then
258	echo "IGCM_sys_MkdirArchive :" $@
259	fi
260	#- creation de repertoire sur le serveur fichier
261	rsh gaya -n mkdir -p $1
262
263	if [ $? -gt 0 ] ; then
264	echo "IGCM_sys_MkdirArchive : erreur."
265	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
266	fi
267	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
268	}
269
270	#D-#==================================================
271	#D-function IGCM_sys_MkdirWork
272	#D-* Purpose: Mkdir on Work
273	#D-* Examples:
274	#D-
275	function IGCM_sys_MkdirWork {
276	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
277	if ( $DEBUG_sys ) ; then
278	echo "IGCM_sys_MkdirWork :" $@
279	fi
280	#- creation de repertoire sur le serveur fichier
281	if [ ! -d ${1} ]; then
282	\mkdir -p $1
283	if [ $? -gt 0 ] ; then
284	echo "IGCM_sys_MkdirWork : erreur."
285	IGCM_debug_Exit "IGCM_sys_MkdirWork"
286	fi
287	fi
288	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
289	}
290
291	#D-#==================================================
292	#D-function IGCM_sys_Cd
293	#D-* Purpose: master cd command
294	#D-* Examples:
295	#D-
296	function IGCM_sys_Cd {
297	IGCM_debug_PushStack "IGCM_sys_Cd" $@
298	if ( $DEBUG_sys ) ; then
299	echo "IGCM_sys_Cd :" $@
300	fi
301	\cd $1
302	if [ $? -gt 0 ] ; then
303	echo "IGCM_sys_Cd : erreur."
304	IGCM_debug_Exit "IGCM_sys_Cd"
305	fi
306	IGCM_debug_PopStack "IGCM_sys_Cd"
307	}
308
309	#D-#==================================================
310	#D-function IGCM_sys_Chmod
311	#D-* Purpose: Chmod
312	#D-* Examples:
313	#D-
314	function IGCM_sys_Chmod {
315	IGCM_debug_PushStack "IGCM_sys_Chmod" -- $@
316	if ( $DEBUG_sys ) ; then
317	echo "IGCM_sys_Chmod :" $@
318	fi
319	if [ $DRYRUN -le 1 ]; then
320	\chmod $@
321	if [ $? -gt 0 ] ; then
322	echo "IGCM_sys_Chmod : erreur."
323	IGCM_debug_Exit "IGCM_sys_Chmod"
324	fi
325	else
326	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
327	fi
328	IGCM_debug_PopStack "IGCM_sys_Chmod"
329	}
330
331	#D-#==================================================
332	#D-function IGCM_sys_FileSize
333	#D-* Purpose: Filesize
334	#D-* Examples:
335	#D-
336	function IGCM_sys_FileSize {
337	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
338
339	typeset sizeF
340	set +A sizeF -- $( ls -la ${1} )
341	if [ $? -gt 0 ] ; then
342	IGCM_debug_Exit "IGCM_sys_FileSize"
343	fi
344	eval ${2}=${sizeF[4]}
345
346	IGCM_debug_PopStack "IGCM_sys_FileSize"
347	}
348
349	#D-#==================================================
350	#D-function IGCM_sys_TestDir
351	#D-* Purpose: Test Directory that must exists
352	#D-* Examples:
353	#D-
354	function IGCM_sys_TestDir {
355	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
356	if ( $DEBUG_sys ) ; then
357	echo "IGCM_sys_TestDir :" $@
358	fi
359	typeset ExistFlag
360	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
361	IGCM_debug_PopStack "IGCM_sys_TestDir"
362
363	return ${ExistFlag}
364	}
365
366	#D-#==================================================
367	#D-function IGCM_sys_TestDirArchive
368	#D-* Purpose: Test Directory that must exists on Archive
369	#D-* Examples:
370	#D-
371	function IGCM_sys_TestDirArchive {
372	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
373	if ( $DEBUG_sys ) ; then
374	echo "IGCM_sys_TestDirArchive :" $@
375	fi
376	typeset ExistFlag
377	ExistFlag=$( IGCM_sys_RshArchive "[ -d $1 ] && echo 0 \|\| echo 1" )
378	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
379
380	return ${ExistFlag}
381	}
382
383	#D-#==================================================
384	#D-function IGCM_sys_TestFileArchive
385	#D-* Purpose: Test file that must NOT EXISTS on Archive
386	#D-* Examples:
387	#D-
388	function IGCM_sys_TestFileArchive {
389	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
390	if ( $DEBUG_sys ) ; then
391	echo "IGCM_sys_TestFileArchive :" $@
392	fi
393	typeset ExistFlag
394	ExistFlag=$( IGCM_sys_RshArchive "[ -f $1 ] && echo 0 \|\| echo 1" )
395	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
396
397	return ${ExistFlag}
398	}
399
400	#D-#==================================================
401	#D-function IGCM_sys_CountFileArchive
402	#D-* Purpose: Count files on Archive filesystem
403	#D-* Examples:
404	#D-
405	function IGCM_sys_CountFileArchive {
406	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
407	IGCM_sys_RshArchive "ls ${@} 2>/dev/null \| wc -l"
408	if [ $? -gt 0 ] ; then
409	echo "IGCM_sys_CountFileArchive : erreur."
410	fi
411	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
412	}
413
414	#D-#==================================================
415	#D-function IGCM_sys_Tree
416	#D-* Purpose: Tree directories with files on ${ARCHIVE}
417	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
418	#D-
419	function IGCM_sys_Tree {
420	IGCM_debug_PushStack "IGCM_sys_Tree" $@
421	if ( $DEBUG_sys ) ; then
422	echo "IGCM_sys_Tree :" $@
423	fi
424
425	\mfls -r $@
426
427	IGCM_debug_PopStack "IGCM_sys_Tree"
428	}
429
430	#D-#==================================================
431	#D-function IGCM_sys_Tar
432	#D-* Purpose: master un-tar command
433	#D-* Examples:
434	#D-
435	function IGCM_sys_Tar {
436	IGCM_debug_PushStack "IGCM_sys_Tar" $@
437	if ( $DEBUG_sys ) ; then
438	echo "IGCM_sys_Tar :" $@
439	fi
440	\tar xvf $1
441	if [ $? -gt 0 ] ; then
442	echo "IGCM_sys_Tar : erreur."
443	IGCM_debug_Exit "IGCM_sys_Tar"
444	fi
445	IGCM_debug_PopStack "IGCM_sys_Tar"
446	}
447
448	#D-#==================================================
449	#D-function IGCM_sys_UnTar
450	#D-* Purpose: master un-tar command
451	#D-* Examples:
452	#D-
453	function IGCM_sys_UnTar {
454	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
455	if ( $DEBUG_sys ) ; then
456	echo "IGCM_sys_UnTar :" $@
457	fi
458	\tar xvf $1
459	if [ $? -gt 0 ] ; then
460	echo "IGCM_sys_UnTar : erreur."
461	IGCM_debug_Exit "IGCM_sys_UnTar"
462	fi
463	IGCM_debug_PopStack "IGCM_sys_UnTar"
464	}
465
466	#D-#==================================================
467	#D-function IGCM_sys_Qsub
468	#D-* Purpose: Qsub new job
469	#D-* Examples:
470	#D-
471	function IGCM_sys_Qsub {
472	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
473	if ( $DEBUG_sys ) ; then
474	echo "IGCM_sys_Qsub :" $@
475	fi
476	/usr/bin/nqsII/qsub -o ${Script_Output} -N ${config_UserChoices_JobName}.${CumulPeriod} < $1
477	if [ $? -gt 0 ] ; then
478	echo "IGCM_sys_Qsub : erreur -o ${Script_Output} -N ${config_UserChoices_JobName}.${CumulPeriod} $@."
479	IGCM_debug_Exit "IGCM_sys_Qsub"
480	fi
481	IGCM_debug_PopStack "IGCM_sys_Qsub"
482	}
483
484	#D-#==================================================
485	#D-function IGCM_sys_QsubPost
486	#D-* Purpose: Qsub new job on scalaire
487	#D-* Examples:
488	#D-
489	function IGCM_sys_QsubPost {
490	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
491	if ( $DEBUG_sys ) ; then
492	echo "IGCM_sys_QsubPost :" $@
493	fi
494	cd ${POST_DIR}
495	/opt/ibmll/LoadL/full/bin/llsubmit ${libIGCM_POST}/$1.job
496	cd -
497	if [ $? -gt 0 ] ; then
498	echo "IGCM_sys_QsubPost : erreur " $@
499	IGCM_debug_Exit "IGCM_sys_QsubPost"
500	fi
501	IGCM_debug_PopStack "IGCM_sys_QsubPost"
502	}
503
504	#D-*************************
505	#D- File transfer functions
506	#D-*************************
507	#D-
508
509	#D-#==================================================
510	#D-function IGCM_sys_Rsync_out
511	#D-* Purpose: treat return val of rsync
512	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
513	#D- Error values and explanations can depend on your system version.
514	function IGCM_sys_Rsync_out {
515	RET=$1
516	if [ ! $RET ] ; then
517	echo "rsync error !"
518	fi
519
520	if [ $MYLANG = "fr" ]; then
521	case $RET in
522	0) return ;;
523	1) echo "Erreur de rsync ; RERR_SYNTAX : "
524	echo "Erreur de syntaxe ou d'utilisation."
525	return;;
526	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
527	echo "Incompatibilité de protocole."
528	return;;
529	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
530	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
531	echo "répertoires"
532	return;;
533	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
534	echo "Action demandée non supportée : une tentative de manipulation de"
535	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
536	echo "été faite ; ou une option qui est supportée par le client mais"
537	echo "pas par le serveur a été spécifiée."
538	return;;
539	10) echo "Erreur de rsync ; RERR_SOCKETIO"
540	echo "Erreur dans le socket d'entrée sortie"
541	return;;
542	11) echo "Erreur de rsync ; RERR_FILEIO"
543	echo "Erreur d'entrée sortie fichier"
544	return;;
545	12) echo "Erreur de rsync ; RERR_STREAMIO"
546	echo "Erreur dans flux de donnée du protocole rsync"
547	return;;
548	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
549	echo "Erreur avec les diagnostics du programme"
550	return;;
551	14) echo "Erreur de rsync ; RERR_IPC"
552	echo "Erreur dans le code IPC"
553	return;;
554	20) echo "Erreur de rsync ; RERR_SIGNAL"
555	echo "SIGUSR1 ou SIGINT reçu"
556	return;;
557	21) echo "Erreur de rsync ; RERR_WAITCHILD"
558	echo "Une erreur retournée par waitpid()"
559	return;;
560	22) echo "Erreur de rsync ; RERR_MALLOC"
561	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
562	return;;
563	23) echo ""
564	echo "Erreur fichier inexistant"
565	return;;
566	30) echo "Erreur de rsync ; RERR_TIMEOUT"
567	echo "Temps d'attente écoulé dans l'envoi/réception de données"
568	return;;
569	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $RET
570	return;;
571	esac
572	elif [ $MYLANG = "en" ] ; then
573	case $RET in
574	0) return;;
575	1) echo "rsync error : Syntax or usage error "
576	return;;
577	2) echo "rsync error : Protocol incompatibility "
578	return;;
579	3) echo "rsync error : Errors selecting input/output files, dirs"
580	return;;
581	4) echo "rsync error : Requested action not supported: an attempt"
582	echo "was made to manipulate 64-bit files on a platform that cannot support"
583	echo "them; or an option was specified that is supported by the client and"
584	echo "not by the server."
585	return;;
586	5) echo "rsync error : Error starting client-server protocol"
587	return;;
588	10) echo "rsync error : Error in socket I/O "
589	return;;
590	11) echo "rsync error : Error in file I/O "
591	return;;
592	12) echo "rsync error : Error in rsync protocol data stream "
593	return;;
594	13) echo "rsync error : Errors with program diagnostics "
595	return;;
596	14) echo "rsync error : Error in IPC code "
597	return;;
598	20) echo "rsync error : Received SIGUSR1 or SIGINT "
599	return;;
600	21) echo "rsync error : Some error returned by waitpid() "
601	return;;
602	22) echo "rsync error : Error allocating core memory buffers "
603	return;;
604	23) echo "rsync error : Partial transfer due to error"
605	return;;
606	24) echo "rsync error : Partial transfer due to vanished source files"
607	return;;
608	30) echo "rsync error : Timeout in data send/receive "
609	return;;
610	*) echo "rsync error : return code of rsync unknown :" $RET
611	return;;
612	esac
613	else
614	echo "unknown language $MYLANG."
615	return
616	fi
617	}
618
619
620	#D-#==================================================
621	#D-function IGCM_sys_Miror_libIGCM
622	#D-* Purpose: Mirror libIGCM PATH and lib to ulam
623	#D-* Examples:
624	#D-
625	function IGCM_sys_Mirror_libIGCM {
626	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
627	if ( $DEBUG_sys ) ; then
628	echo "IGCM_sys_Mirror_libIGCM"
629	fi
630
631	typeset RET DEST
632
633	IGCM_sys_RshPost <<-EOF
634	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
635	EOF
636	rsh ulam -n hostname > /dev/null 2>&1
637	RET=$?
638	if [ $RET -eq 0 ] ; then
639	echo ${RSYNC} ${RSYNC_opt} -e "rsh " ${libIGCM} ulam:${HOME}/MIRROR/${PATHlibIGCM} > out_rsync 2>&1
640	${RSYNC} ${RSYNC_opt} -e "rsh " ${libIGCM} ulam:${HOME}/MIRROR/${PATHlibIGCM} >> out_rsync 2>&1
641	RET=$?
642
643	if [ ${RET} -gt 0 ] ; then
644	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on ulam."
645	cat out_rsync
646	fi
647	else
648	echo "No POST-TREATMENT avaible because ulam is down."
649	fi
650	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
651	}
652
653	#====================================================
654	#- Call IGCM_sys_Mirror_libIGCM now !
655	if ( $MirrorlibIGCM ) ; then
656	IGCM_sys_Mirror_libIGCM
657	fi
658
659	#D-#==================================================
660	#D-function IGCM_sys_Cp
661	#D-* Purpose: generic cp
662	#D-* Examples:
663	#D-
664	function IGCM_sys_Cp {
665	IGCM_debug_PushStack "IGCM_sys_Cp" $@
666	if ( $DEBUG_sys ) ; then
667	echo "IGCM_sys_Cp :" $@
668	fi
669
670	typeset RET
671
672	echo cp $@ > out_rsync 2>&1
673	\cp $@ >> out_rsync 2>&1
674	RET=$?
675
676	if [ ${RET} -gt 0 ] ; then
677	echo "IGCM_sys_Cp : error."
678	cat out_rsync
679	IGCM_debug_Exit "IGCM_sys_Cp"
680	fi
681	IGCM_debug_PopStack "IGCM_sys_Cp"
682	}
683
684	#D-#==================================================
685	#D-function IGCM_sys_Rm
686	#D-* Purpose: generic rm
687	#D-* Examples:
688	#D-
689	function IGCM_sys_Rm {
690	IGCM_debug_PushStack "IGCM_sys_Rm" -- $@
691	if ( $DEBUG_sys ) ; then
692	echo "IGCM_sys_Rm :" $@
693	fi
694
695	typeset RET
696
697	echo rm $@ > out_rsync 2>&1
698	\rm $@ >> out_rsync 2>&1
699	RET=$?
700
701	if [ ${RET} -gt 0 ] ; then
702	echo "IGCM_sys_Rm : error."
703	cat out_rsync
704	IGCM_debug_Exit "IGCM_sys_Rm"
705	fi
706	IGCM_debug_PopStack "IGCM_sys_Rm"
707	}
708
709	#D-#==================================================
710	#D-function IGCM_sys_Mv
711	#D-* Purpose: generic move
712	#D-* Examples:
713	#D-
714	function IGCM_sys_Mv {
715	IGCM_debug_PushStack "IGCM_sys_Mv" $@
716	if ( $DEBUG_sys ) ; then
717	echo "IGCM_sys_Mv :" $@
718	fi
719
720	if [ $DRYRUN = 0 ]; then
721
722	typeset RET
723
724	echo mv $@ > out_rsync 2>&1
725	\mv $@ >> out_rsync 2>&1
726	RET=$?
727
728	if [ ${RET} -gt 0 ] ; then
729	echo "IGCM_sys_Mv : error in mv."
730	cat out_rsync
731	IGCM_debug_Exit "IGCM_sys_Mv"
732	fi
733	else
734	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
735	fi
736
737	IGCM_debug_PopStack "IGCM_sys_Mv"
738	}
739
740	#D-#==================================================
741	#D-function IGCM_sys_Put_Dir
742	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
743	#D-* Examples:
744	#D-
745	function IGCM_sys_Put_Dir {
746	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
747	if ( $DEBUG_sys ) ; then
748	echo "IGCM_sys_Put_Dir :" $@
749	fi
750	if [ $DRYRUN = 0 ]; then
751	if [ ! -d ${1} ] ; then
752	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
753	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
754	return
755	fi
756
757	typeset RET
758
759	# Only if we use rsync
760	#IGCM_sys_TestDirArchive $( dirname $2 )
761	#
762	#USUAL WAY
763	rcp -r $1 gaya:$2 > out_rsync 2>&1
764	RET=$?
765
766	if [ ${RET} -gt 0 ] ; then
767	echo "IGCM_sys_Put_Dir : error."
768	cat out_rsync
769	IGCM_debug_Exit "IGCM_sys_Put_Dir"
770	fi
771	else
772	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
773	fi
774	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
775	}
776
777	#D-#==================================================
778	#D-function IGCM_sys_Get_Dir
779	#D-* Purpose: Copy a complete directory from $(ARCHIVE)
780	#D-* Examples:
781	#D-
782	function IGCM_sys_Get_Dir {
783	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
784	if ( $DEBUG_sys ) ; then
785	echo "IGCM_sys_Get_Dir :" $@
786	fi
787	if [ $DRYRUN = 0 ]; then
788	if [ ! -d ${1} ] ; then
789	echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
790	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
791	return
792	fi
793
794	typeset RET
795
796	#USUAL WAY
797	rcp -rp gaya:$1 $2 > out_rsync 2>&1
798	RET=$?
799
800	if [ ${RET} -gt 0 ] ; then
801	echo "IGCM_sys_Get_Dir : error."
802	cat out_rsync
803	IGCM_debug_Exit "IGCM_sys_Get_Dir"
804	fi
805	else
806	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
807	fi
808	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
809	}
810
811	#D-#==================================================
812	#D-function IGCM_sys_Put_Rest
813	#D-* Purpose: Put computied restarts on $(ARCHIVE).
814	#D- File and target directory must exist.
815	#D-* Examples:
816	#D-
817	function IGCM_sys_Put_Rest {
818	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
819	if ( $DEBUG_sys ) ; then
820	echo "IGCM_sys_Put_Rest :" $@
821	fi
822	if [ $DRYRUN = 0 ]; then
823	if [ ! -f ${1} ] ; then
824	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
825	IGCM_debug_Exit "IGCM_sys_Put_Rest"
826	fi
827
828	typeset RET
829	#
830	if [ X${JobType} = XRUN ] ; then
831	IGCM_sys_Chmod 444 ${1}
832	fi
833	#
834	#USUAL WAY
835	mfput $1 $2 > out_rsync 2>&1
836	RET=$?
837
838	# #RSYNC WITH NETWORK RSH CALL
839	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} > out_rsync 2>&1
840	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} >> out_rsync 2>&1
841
842	# #RSYNC WITH NFS USE
843	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > out_rsync 2>&1
844	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> out_rsync 2>&1
845
846	# RET=$?
847	# IGCM_sys_Rsync_out $RET
848
849	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk out_rsync
850	# (( RET=RET+$? ))
851
852	if [ ${RET} -gt 0 ] ; then
853	echo "IGCM_sys_Put_Rest : error."
854	cat out_rsync
855	IGCM_debug_Exit "IGCM_sys_Put_Rest"
856	fi
857	else
858	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
859	fi
860	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
861	}
862
863	#D-#==================================================
864	#D-function IGCM_sys_Put_Out
865	#D-* Purpose: Copy a file on $(ARCHIVE) after have chmod it in readonly
866	#D-* Examples:
867	#D-
868	function IGCM_sys_Put_Out {
869	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
870	if ( $DEBUG_sys ) ; then
871	echo "IGCM_sys_Put_Out :" $@
872	fi
873	if [ $DRYRUN = 0 ]; then
874	if [ ! -f ${1} ] ; then
875	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
876	IGCM_debug_PopStack "IGCM_sys_Put_Out"
877	return 1
878	fi
879
880	typeset RET
881	#
882	if [ X${JobType} = XRUN ] ; then
883	if [ X${3} = X ] ; then
884	IGCM_sys_Chmod 444 ${1}
885	fi
886	fi
887	#
888	#USUAL WAY
889	mfput $1 $2 > out_rsync 2>&1
890	RET=$?
891
892	# #RSYNC WITH NETWORK RSH CALL
893	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} > out_rsync 2>&1
894	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} >> out_rsync 2>&1
895
896	# #RSYNC WITH NFS USE
897	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > out_rsync 2>&1
898	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> out_rsync 2>&1
899
900	# RET=$?
901	# IGCM_sys_Rsync_out $RET
902
903	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk out_rsync
904	# (( RET=RET+$? ))
905
906	if [ ${RET} -gt 0 ] ; then
907	echo "IGCM_sys_Put_Out : error."
908	cat out_rsync
909	IGCM_debug_Exit "IGCM_sys_Put_Out"
910	fi
911	else
912	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
913	fi
914	IGCM_debug_PopStack "IGCM_sys_Put_Out"
915	return 0
916	}
917
918	#D-#==================================================
919	#D-function IGCM_sys_Get
920	#D-* Purpose: Get a file from ${ARCHIVE}
921	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
922	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
923	function IGCM_sys_Get {
924	IGCM_debug_PushStack "IGCM_sys_Get" $@
925
926	typeset DEST RET dm_liste ifile target
927
928	if ( $DEBUG_sys ) ; then
929	echo "IGCM_sys_Get :" $@
930	fi
931	if [ $DRYRUN -le 2 ]; then
932	if [ X${1} = X'/l' ] ; then
933	# test if the first file is present in the old computation :
934	eval set +A dm_liste \${${2}}
935	else
936	dm_liste=${1}
937	fi
938	eval DEST=\${${#}}
939
940	# test if the (first) file is present in the old computation :
941	IGCM_sys_TestFileArchive ${dm_liste[0]}
942	RET=$?
943	if [ ${RET} -gt 0 ] ; then
944	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
945	IGCM_debug_Exit "IGCM_sys_Get"
946	IGCM_debug_PopStack "IGCM_sys_Get"
947	return
948	fi
949
950	# SD : dm_liste is not suited for computing job
951	# because we change filename during transfert
952	# dm_liste is better suited for post-treatment
953	#dm_liste=" "
954	#(( ifile=1 ))
955	#while [ $ifile -lt $# ] ; do
956	# dm_liste=$( eval echo ${dm_liste} " "\${${ifile}} )
957	# (( ifile = ifile + 1 ))
958	#done
959	#DEST=$( eval echo \${${#}} )
960
961	#USUAL WAY
962	mfget ${dm_liste[*]} ${DEST} > out_rsync 2>&1
963	RET=$?
964
965	# #RSYNC WITH NETWORK RSH CALL
966	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RHOST}:"${dm_liste}" ${RHOST}:${RUN_DIR}/${DEST} > out_rsync 2>&1
967	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RHOST}:"${dm_liste}" ${RHOST}:${RUN_DIR}/${DEST} >> out_rsync 2>&1
968
969	# #RSYNC WITH NFS USE
970	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > out_rsync 2>&1
971	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> out_rsync 2>&1
972
973	# RET=$?
974	# IGCM_sys_Rsync_out $RET
975
976	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk out_rsync
977	# (( RET=RET+$? ))
978
979	if [ ${RET} -gt 0 ] ; then
980	echo "IGCM_sys_Get : copy error."
981	cat out_rsync
982	IGCM_debug_Exit "IGCM_sys_Get"
983	fi
984	else
985	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
986	fi
987	IGCM_debug_PopStack "IGCM_sys_Get"
988	}
989
990	############################################################## A FINIR !!
991
992	#D-#==================================================
993	#D-function IGCM_sys_GetDate_FichWork
994	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
995	#D-* Examples:
996	#D-
997	function IGCM_sys_GetDate_FichWork {
998	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
999	if ( $DEBUG_sys ) ; then
1000	echo "IGCM_sys_GetDate_FichWork :" $@
1001	fi
1002	# donne la date filesys d'un fichier sur la machine work
1003	IGCM_debug_PopStack "IGCM_sys_FichWork"
1004	}
1005
1006	#D-#==================================================
1007	#D-function IGCM_sys_GetDate_FichArchive
1008	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1009	#D-* Examples:
1010	#D-
1011	function IGCM_sys_GetDate_FichArchive {
1012	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1013	if ( $DEBUG_sys ) ; then
1014	echo "IGCM_sys_GetDate_FichArchive :" $@
1015	fi
1016	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1017	}
1018
1019	##############################################################
1020	# REBUILD OPERATOR
1021
1022	function IGCM_sys_rebuild {
1023	IGCM_debug_PushStack "IGCM_sys_rebuild" -- $@
1024	if ( $DEBUG_sys ) ; then
1025	echo "IGCM_sys_rebuild :" $@
1026	fi
1027	/home/rech/psl/rpsl035/SX/bin/rebuild -f -o $@
1028	if [ $? -gt 0 ] ; then
1029	echo "IGCM_sys_rebuild : erreur ${@}."
1030	IGCM_debug_Exit "rebuild"
1031	fi
1032
1033	IGCM_debug_PopStack "IGCM_sys_rebuild"
1034	}
1035
1036	############################################################
1037	# Activate Running Environnment Variables
1038
1039	function IGCM_sys_activ_variables {
1040	IGCM_debug_PushStack "IGCM_sys_activ_variables"
1041	if ( $DEBUG_sys ) ; then
1042	echo "IGCM_sys_activ_variables"
1043	fi
1044
1045	# --------------------------------------------------------------------
1046	#D- MPI specifications
1047	# --------------------------------------------------------------------
1048
1049	#D-- MPISUSPEND
1050	export MPISUSPEND=${MPISUSPEND:=OFF}
1051
1052	#D-- MPIPROGINF #other choices : ALL_DETAIL2
1053	export MPIPROGINF=ALL
1054	#D- activate ftrace (with -ftrace)
1055	export F_FTRACE=YES
1056	#D- communication information (with -ftrace)
1057	export MPICOMMINF=DETAIL
1058
1059	# --------------------------------------------------------------------
1060	#D- Other specifications
1061	# --------------------------------------------------------------------
1062
1063	#D- max number of character/line in output job
1064	export F_SYSLEN=5000
1065	#D- number of error that can be admitted on the NEC
1066	export F_ERRCNT=0
1067	#D- global performance
1068	export F_PROGINF=DETAIL
1069
1070	#D- I/O performance (FORTRAN I/O only not netCDF)
1071	export F_FILEINF=${F_FILEINF:=NO}
1072	#D- netCDF I/O performance
1073	export NC_FILEINF=${NC_FILEINF:=NO}
1074
1075	IGCM_debug_PopStack "IGCM_sys_activ_variables"
1076	}
1077
1078	############################################################
1079	# Desactivate Running Environnment Variables
1080
1081	function IGCM_sys_desactiv_variables {
1082	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
1083	if ( $DEBUG_sys ) ; then
1084	echo "IGCM_sys_desactiv_variables"
1085	fi
1086	# --------------------------------------------------------------------
1087	#D- MPI specifications
1088	# --------------------------------------------------------------------
1089
1090	#D-- MPIPROGINF
1091	export MPIPROGINF=NO
1092
1093	# --------------------------------------------------------------------
1094	#D- Other specifications
1095	# --------------------------------------------------------------------
1096
1097	#D- global performance
1098	export F_PROGINF=NO
1099
1100	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
1101
1102	}
1103
1104	############################################################
1105	# Build run file
1106
1107	function IGCM_sys_build_run_file {
1108	IGCM_debug_PushStack "IGCM_sys_build_run_file" $@
1109	if ( $DEBUG_sys ) ; then
1110	echo "IGCM_sys_build_run_file " $@
1111	fi
1112	(( NUM_PROC_ATM = BATCH_NUM_PROC_TOT - 1 ))
1113	(( NUM_PROC_OASIS = BATCH_NUM_PROC_TOT - NUM_PROC_ATM ))
1114	(( NUM_PROC_OCE = BATCH_NUM_PROC_TOT - NUM_PROC_ATM ))
1115
1116	if [ $1 = MPI2 ]; then
1117	cat <<EOF > run_file
1118	-p 1 -np 1 -e ./oasis
1119	EOF
1120	(( NUM_PROCESS = BATCH_NUM_PROC_TOT + 1 ))
1121	config_UserChoices_JobRunOptions='"-max_np ${NUM_PROCESS} -f"'
1122
1123	elif [ $1 = MPI1 ]; then
1124	cat <<EOF > run_file
1125	-p $NUM_PROC_OASIS -e ./oasis
1126	-p $NUM_PROC_ATM -e ./lmdz.x
1127	-p $NUM_PROC_OCE -e ./opa.xx
1128	EOF
1129
1130	fi
1131
1132	IGCM_debug_PopStack "IGCM_sys_build_run_file"
1133
1134	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: