Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_jeanzay.ksh @ 1505

Last change on this file since 1505 was 1505, checked in by aclsce, 4 years ago
unset SLURM_HOSTFILE variable to allow the automatic resoumission of the Job.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 56.6 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Jean-Zay
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host user names project maxCpuTime
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=jeanzay
71	# add default project set by ins_job
72	typeset PROJECT=$( echo ${SLURM_JOB_ACCOUNT:=::default_project::} \| cut -c "1-3" )
73	### jobWarningDelay in seconds
74	##typeset jobWarningDelay=${BRIDGE_MSUB_MAXTIME}
75
76	#D-
77	#D-#==================================================
78	#D-Program used in libIGCM
79	#D-#==================================================
80
81	# Submit command
82	typeset SUBMIT=${SUBMIT:=sbatch}
83	# rsync with path
84	typeset -r RSYNC=/usr/bin/rsync
85	# RSYNC_opt args to rsync
86	typeset -r RSYNC_opt="-va"
87	# ie storage filesystem
88	typeset -r STOREHOST=${MASTER}
89	typeset -r REMOTE_RSYNC=/usr/bin/rsync
90
91	#====================================================
92	# Access to module command
93	#====================================================
94	. /usr/share/Modules/init/ksh
95
96	#====================================================
97	# Set environment tools (ferret, nco, cdo, rebuild, ...)
98	#====================================================
99	if [ X${TaskType} = Xcomputing ] ; then
100	IGCM_debug_Print 1 "Modules will be loaded later in IGCM_sys_activ_variables."
101	else
102	module purge > /dev/null 2>&1
103	. /gpfswork/rech/psl/commun/MachineEnvironment/jeanzay/env_jeanzay > /dev/null 2>&1
104
105	. /gpfswork/rech/psl/commun/MachineEnvironment/jeanzay/env_atlas_jeanzay > /dev/null 2>&1
106
107	# export PCMDI_MP=/ccc/work/cont003/igcmg/igcmg/PCMDI-MP
108	export UVCDAT_ANONYMOUS_LOG=FALSE
109	fi
110
111	# FYI
112	[ ! X${TaskType} = Xchecking ] && IGCM_debug_Print 1 "List of loaded modules:"
113	[ ! X${TaskType} = Xchecking ] && module list
114
115	# For AddNoise (not yet AddPertu3DOCE)
116	export PATH=${PATH}:/gpfswork/rech/psl/commun/Tools/AddNoise/bin
117
118	# For rebuild
119	export PATH=${PATH}:/gpfswork/rech/psl/commun/Tools/rebuild/modipsl_IOIPSL_PLUS_v2_2_4/bin
120
121	#====================================================
122	# Host specific DIRECTORIES
123	#====================================================
124
125	#====================================================
126	#- MirrorlibIGCM for frontend
127	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
128
129	#====================================================
130	#- libIGCM_POST for frontend
131	typeset -r libIGCM_POST=${libIGCM}
132
133	#====================================================
134	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
135	typeset -r R_EXE="${MODIPSL}/bin"
136
137	#====================================================
138	#- SUBMIT_DIR : submission dir
139	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${SLURM_SUBMIT_DIR}}
140
141	#====================================================
142	#- IN
143	typeset -r R_IN=${R_IN:=/gpfswork/rech/psl/commun/IGCM}
144
145	#====================================================
146	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
147	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCH}/RUN_DIR/${SLURM_JOBID}_${$}}
148
149	#====================================================
150	#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
151	typeset -r OUTCOMMAND_PATH=/tmp
152
153	#====================================================
154	#- HOST_MPIRUN_COMMAND
155	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time srun "}
156
157	#====================================================
158	#- Max number of arguments passed to nco operator or demigration command
159	UNIX_MAX_LIMIT=360
160
161	#====================================================
162	#- set PackDefault to true
163	PackDefault=true
164
165	#====================================================
166	#- Default number of MPI task for IPSL coupled model
167	#- required for backward compatibility
168	#-
169	DEFAULT_NUM_PROC_OCE=5
170	DEFAULT_NUM_PROC_CPL=1
171	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
172
173	#====================================================
174	#- Number of computing cores per node
175	#-
176	NB_CORE_PER_NODE=40
177
178	#D-#==================================================
179	#D-function IGCM_sys_defineArchives
180	#D-* Purpose:
181	#D-* Load dfldatadir module : According to project used for submission (default) or set in config.card (optional)
182	#D-* Define ARCHIVE : Dedicated to large files
183	#D-* Define STORAGE : Dedicated to small/medium files
184	#D-* Define R_OUT : Output tree located on ARCHIVE
185	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monsitoring and atlas, and/or small files)
186	#D-* Define R_BUF : Output tree located on SCRATCH hosting files waiting for rebuild or pack processes
187	#D-* if SpaceName=TEST everything is stored on SCRATCH
188	#D-* Examples:
189	#D-
190	function IGCM_sys_defineArchives {
191	IGCM_debug_PushStack "IGCM_sys_defineArchives"
192
193	# Load dfldatadir depending on the project used for submission (default) or set in config.card (optional)
194	if [ X${config_UserChoices_DataProject} = X ] \|\| [ X${config_UserChoices_DataProject} = DEFAULT ]; then
195	# The variable DataProject is not set in in section UserChoices in config.card or it is set to DEFAULT
196	# Default option: Change IDRPROJ according to the project used for submission
197
198	# jg following line does not work in a submitted job:
199	eval $(idrenv -d ${PROJECT})
200	else
201	# Use the project set by the variable DataProject in section UserChoices in config.card
202	eval $(idrenv -d ${config_UserChoices_DataProject})
203	IGCM_debug_Print 1 "Change DataProject red in config.card : ${config_UserChoices_DataProject}"
204	fi
205	IGCM_debug_Print 1 "Data project for output to be used: ${IDRPROJ}"
206
207	#====================================================
208	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
209	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCH}/RUN_DIR/${SLURM_JOBID}_${$}}
210
211	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
212	#====================================================
213	#- ARCHIVE (dedicated to large files)
214	ARCHIVE=${config_UserChoices_ARCHIVE}
215	else
216	#====================================================
217	#- ARCHIVE (dedicated to large files)
218	ARCHIVE=${STORE}
219	fi
220
221	if [ ! X${config_UserChoices_STORAGE} = X ]; then
222	#====================================================
223	#- STORAGE (dedicated to small/medium files)
224	STORAGE=${config_UserChoices_STORAGE}
225	else
226	#====================================================
227	#- STORAGE (dedicated to small/medium files)
228	STORAGE=${WORK}
229	fi
230
231	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
232	#====================================================
233	#- R_OUT
234	R_OUT=${SCRATCH}/IGCM_OUT
235
236	#====================================================
237	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
238	R_FIG=${SCRATCH}/IGCM_OUT
239
240	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
241
242	else
243	#====================================================
244	#- R_OUT
245	R_OUT=${ARCHIVE}/IGCM_OUT
246
247	#====================================================
248	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
249	R_FIG=${STORAGE}/IGCM_OUT
250	fi
251
252	#====================================================
253	#- CMIP6 (hosting CMIP6 files produced by XIOS2 and configured by dr2xml)
254	CMIP6_BUF=${STORAGE}/IGCM_OUT
255
256	#====================================================
257	#- R_BUF (ONLY FOR double copy an scratch)
258	R_BUF=${SCRATCH}/IGCM_OUT
259
260	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
261	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
262	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
263
264	IGCM_debug_PopStack "IGCM_sys_defineArchives"
265	}
266
267	#D-#==================================================
268	#D-function IGCM_sys_RshArchive
269	#D-* Purpose: Archive rsh command
270	#D-* Examples:
271	#D-
272	function IGCM_sys_RshArchive {
273	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
274	/bin/ksh <<-EOF
275	${@}
276	EOF
277	status=$?
278	if [ ${status} -gt 0 ] ; then
279	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
280	IGCM_debug_Exit "IGCM_sys_RshArchive"
281	fi
282	IGCM_debug_PopStack "IGCM_sys_RshArchive"
283	}
284
285	#D-#==================================================
286	#D-function IGCM_sys_RshArchive_NoError
287	#D-* Purpose: Archive rsh command, without error
288	#D-* used only in monitoring.job
289	#D-* Examples:
290	#D-
291	function IGCM_sys_RshArchive_NoError {
292	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
293	/bin/ksh <<-EOF
294	${@} 2> /dev/null
295	EOF
296	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
297	}
298
299	#D-#==================================================
300	#D-function IGCM_sys_MkdirArchive
301	#D-* Purpose: Mkdir on Archive
302	#D-* Examples:
303	#D-
304	function IGCM_sys_MkdirArchive {
305	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
306	if ( $DEBUG_sys ) ; then
307	echo "IGCM_sys_MkdirArchive :" $@
308	fi
309	#- creation de repertoire sur le serveur fichier
310	if [ ! -d ${1} ]; then
311	\mkdir -p $1
312	status=$?
313
314	if [ ${status} -gt 0 ] ; then
315	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
316	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
317	fi
318	fi
319	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
320	}
321
322	#D-#==================================================
323	#D-function IGCM_sys_TestDirArchive
324	#D-* Purpose: Test Directory that must exists on Archive
325	#D-* Examples:
326	#D-
327	function IGCM_sys_TestDirArchive {
328	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
329	if ( $DEBUG_sys ) ; then
330	echo "IGCM_sys_TestDirArchive :" $@
331	fi
332	typeset ExistFlag
333	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
334	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
335	return ${ExistFlag}
336	}
337
338	#D-#==================================================
339	#D-function IGCM_sys_IsFileArchived
340	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
341	#D-* Examples:
342	#D-
343	function IGCM_sys_IsFileArchived {
344	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
345	if ( $DEBUG_sys ) ; then
346	echo "IGCM_sys_IsFileArchived :" $@
347	fi
348	typeset IsArchivedFlag
349	# To be implemented on Jean Zay
350	# IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
351	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
352
353	return ${IsArchivedFlag}
354	}
355
356	#D-#==================================================
357	#D-function IGCM_sys_TestFileArchive
358	#D-* Purpose: Test file that must NOT EXISTS on Archive
359	#D-* Examples:
360	#D-
361	function IGCM_sys_TestFileArchive {
362	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
363	typeset ExistFlag
364	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
365	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
366
367	return ${ExistFlag}
368	}
369
370	#D-#==================================================
371	#D-function IGCM_sys_CountFileArchive
372	#D-* Purpose: Count files on Archive filesystem
373	#D-* Examples:
374	#D-
375	function IGCM_sys_CountFileArchive {
376	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
377	ls ${@} 2>/dev/null \| wc -l
378	if [ $? -gt 0 ] ; then
379	echo "IGCM_sys_CountFileArchive : erreur."
380	fi
381	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
382	}
383
384	#D-#==================================================
385	#D-function IGCM_sys_Tree
386	#D-* Purpose: Tree directories with files on ${ARCHIVE}
387	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
388	#D-
389	function IGCM_sys_Tree {
390	IGCM_debug_PushStack "IGCM_sys_Tree" $@
391	if ( $DEBUG_sys ) ; then
392	echo "IGCM_sys_Tree :" $@
393	fi
394
395	\ls -lR ${@}
396
397	IGCM_debug_PopStack "IGCM_sys_Tree"
398	}
399
400	#D-#==================================================
401	#D-function IGCM_sys_Qsub
402	#D-* Purpose: Qsub new job
403	#D-* Examples:
404	#D-
405	function IGCM_sys_Qsub {
406	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
407	if ( $DEBUG_sys ) ; then
408	echo "IGCM_sys_Qsub :" $@
409	fi
410	typeset options status
411	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
412
413	/usr/bin/time sbatch ${options} $1 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
414	status=$?
415
416	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
417	if [ ${status} -gt 0 ] ; then
418	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
419	IGCM_debug_Exit "IGCM_sys_Qsub"
420	else
421	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
422	fi
423	IGCM_debug_PopStack "IGCM_sys_Qsub"
424	}
425
426	#D-#==================================================
427	#D-function IGCM_sys_QsubPost
428	#D-* Purpose: Qsub new job on scalaire
429	#D-* Examples:
430	#D-
431	function IGCM_sys_QsubPost {
432	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
433	if ( $DEBUG_sys ) ; then
434	echo "IGCM_sys_QsubPost :" $@
435	fi
436	typeset options status
437	options="-o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
438
439	sbatch ${options} ${libIGCM_POST}/$1.job > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
440	status=$?
441
442	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
443	if [ ${status} -gt 0 ] ; then
444	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
445	IGCM_debug_Exit "IGCM_sys_QsubPost"
446	else
447	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
448	fi
449	IGCM_debug_PopStack "IGCM_sys_QsubPost"
450	}
451
452	#D-*************************
453	#D- File transfer functions
454	#D-*************************
455	#D-
456
457	#D-#==================================================
458	#D-function IGCM_sys_RmRunDir
459	#D-* Purpose: rm tmpdir (dummy function most of the time batch
460	#D- scheduler will do the job)
461	#D-* Examples:
462	#D-
463	function IGCM_sys_RmRunDir {
464	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
465	if ( $DEBUG_sys ) ; then
466	echo "IGCM_sys_RmRunDir :" $@
467	fi
468
469	typeset status
470
471	echo rm $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
472	\rm $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
473	status=$?
474
475	if [ ${status} -gt 0 ] ; then
476	IGCM_debug_Print 1 "IGCM_sys_RmRunDir : rm error code is ${status}."
477	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
478	IGCM_debug_Exit "IGCM_sys_RmRunDir"
479	else
480	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
481	fi
482	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
483	}
484
485	#D-#==================================================
486	#D-function IGCM_sys_Put_Dir
487	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
488	#D-* Examples:
489	#D-
490	function IGCM_sys_Put_Dir {
491	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
492	if ( $DEBUG_sys ) ; then
493	echo "IGCM_sys_Put_Dir :" $@
494	fi
495	if [ $DRYRUN = 0 ]; then
496	if [ ! -d ${1} ] ; then
497	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
498	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
499	return
500	fi
501
502	typeset status
503
504	# Only if we use rsync
505	#IGCM_sys_TestDirArchive $( dirname $2 )
506	#
507	#USUAL WAY
508	\cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
509	status=$?
510
511	if [ ${status} -gt 0 ] ; then
512	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
513	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
514	IGCM_debug_Exit "IGCM_sys_Put_Dir"
515	else
516	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
517	fi
518	fi
519	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
520	}
521
522	#D-#==================================================
523	#D-function IGCM_sys_Get_Dir
524	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
525	#D-* Examples:
526	#D-
527	function IGCM_sys_Get_Dir {
528	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
529	if ( $DEBUG_sys ) ; then
530	echo "IGCM_sys_Get_Dir :" $@
531	fi
532	if [ $DRYRUN = 0 ]; then
533	typeset NB_ESSAI DELAI status i
534	# number of tentative
535	NB_ESSAI=3
536	# time delay between tentative
537	DELAI=2
538
539	#
540	# USUAL WAY
541	# add here command to demigrate all offline files to reduce time of this command
542	# actually on Jean Zay files are not archive on bands
543
544	i=0
545	while [ $i -lt $NB_ESSAI ] ; do
546	\cp -ur $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
547	status=$?
548	if [ ${status} -gt 0 ] ; then
549	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
550	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
551	sleep $DELAI
552	else
553	break
554	fi
555	(( i = i + 1 ))
556	done
557
558	if [ ${status} -gt 0 ] ; then
559	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
560	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
561	IGCM_debug_Exit "IGCM_sys_Get_Dir"
562	else
563	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
564	fi
565	fi
566	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
567	}
568
569	#D-#==================================================
570	#D-function IGCM_sys_Put_Rest
571	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
572	#D- File and target directory must exist.
573	#D-* Examples:
574	#D-
575	function IGCM_sys_Put_Rest {
576	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
577	if ( $DEBUG_sys ) ; then
578	echo "IGCM_sys_Put_Rest :" $@
579	fi
580	if [ $DRYRUN = 0 ]; then
581	if [ ! -f ${1} ] ; then
582	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
583	IGCM_debug_Exit "IGCM_sys_Put_Rest"
584	fi
585
586	typeset status
587	#
588	# USUAL WAY
589	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
590	status=$?
591
592	# #RSYNC WITH NETWORK SSH CALL
593	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
594	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
595
596	# #RSYNC WITH NFS USE
597	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
598	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
599
600	# status=$?
601	# IGCM_sys_Rsync_out $status
602
603	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
604	# (( status=status+$? ))
605
606	if [ ${status} -gt 0 ] ; then
607	IGCM_debug_Print 2 "IGCM_sys_Put_Rest : cp failed error code ${status}"
608	[ -f ${1} ] && ls -l ${1}
609	[ -f ${2} ] && ls -l ${2}
610	[ -f ${2}/${1} ] && ls -l ${2}/${1}
611	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
612	IGCM_debug_Exit "IGCM_sys_Put_Rest"
613	else
614
615	if [ X${JobType} = XRUN ] ; then
616	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
617	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
618	fi
619
620	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
621	fi
622	fi
623	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
624	}
625
626	#D-#==================================================
627	#D-function IGCM_sys_Put_Out
628	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
629	#D-* Examples:
630	#D-
631	function IGCM_sys_Put_Out {
632	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
633	if ( $DEBUG_sys ) ; then
634	echo "IGCM_sys_Put_Out :" $@
635	fi
636
637	typeset NB_ESSAI DELAI status i exist skip
638	typeset fileDeviceNumberInHex directoryDeviceNumberInHex
639
640	# number of tentative
641	NB_ESSAI=3
642	# time delay between tentative
643	DELAI=2
644
645	if [ $DRYRUN = 0 ]; then
646	if [ ! -f ${1} ] ; then
647	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
648	IGCM_debug_PopStack "IGCM_sys_Put_Out"
649	return 1
650	fi
651	#
652	IGCM_sys_MkdirArchive $( dirname $2 )
653	#
654	exist=false
655	skip=false
656	if [ -f $2 ] ; then
657	IGCM_debug_Print 1 "$2 already exist"
658	# add here command to demigrate all offline files to reduce time of this command
659	# actually on Jean Zay files are not archive on bands
660
661	exist=true
662	if [ "X$( diff $1 $2 )" = X ] ; then
663	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
664	skip=true
665	else
666	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
667	skip=false
668	fi
669	fi
670	#
671	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
672	IGCM_sys_Chmod u+w $2
673	fi
674
675	if [ X${skip} = Xfalse ] ; then
676	i=0
677	while [ $i -lt $NB_ESSAI ] ; do
678	# Identify file system
679	fileDeviceNumberInHex=$( stat -c %d $1 )
680	status=$?
681	if [ ${status} -gt 0 ] ; then
682	IGCM_debug_Exit "IGCM_sys_Put_Out"
683	fi
684	# Identify file system
685	directoryDeviceNumberInHex=$( stat -c %d $( dirname $2 ) )
686	status=$?
687	if [ ${status} -gt 0 ] ; then
688	IGCM_debug_Exit "IGCM_sys_Put_Out"
689	fi
690
691	if ( [ ${fileDeviceNumberInHex} -ne ${directoryDeviceNumberInHex} ] \|\| [ X$3 = XNOMOVE ] ) ; then
692	# They are not on the same device. USUAL WAY
693	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
694	status=$?
695	else
696	# They are on the same device. NOT SO USUAL WAY
697	\mv $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
698	status=$?
699	fi
700	if [ ${status} -gt 0 ]; then
701	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
702	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
703	[ -f ${1} ] && ls -l ${1}
704	[ -f ${2} ] && ls -l ${2}
705	[ -f ${2}/${1} ] && ls -l ${2}/${1}
706	sleep $DELAI
707	else
708	break
709	fi
710	(( i = i + 1 ))
711	done
712	fi
713
714	# #RSYNC WITH NETWORK SSH CALL
715	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
716	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
717
718	# #RSYNC WITH NFS USE
719	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
720	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
721
722	# status=$?
723	# IGCM_sys_Rsync_out $status
724
725	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
726	# (( status=status+$? ))
727
728	if [ ${status} -gt 0 ] ; then
729	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status}"
730	[ -f ${1} ] && ls -l ${1}
731	[ -f ${2} ] && ls -l ${2}
732	[ -f ${2}/${1} ] && ls -l ${2}/${1}
733	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
734	IGCM_debug_Exit "IGCM_sys_Put_Out"
735	else
736
737	if [ X${JobType} = XRUN ] ; then
738	if [ X${3} = X ] ; then
739	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
740	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
741	fi
742	fi
743
744	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
745	fi
746	fi
747	IGCM_debug_PopStack "IGCM_sys_Put_Out"
748	return 0
749	}
750
751	#D-#==================================================
752	#D-function IGCM_sys_Get
753	#D-* Purpose: Get a file from ${ARCHIVE}
754	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
755	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
756	function IGCM_sys_Get {
757	IGCM_debug_PushStack "IGCM_sys_Get" $@
758
759	typeset DEST dm_liste target file_work
760	typeset NB_ESSAI DELAI status i
761
762	if ( $DEBUG_sys ) ; then
763	echo "IGCM_sys_Get :" $@
764	fi
765
766	# number of tentative
767	NB_ESSAI=3
768	# time delay between tentative
769	DELAI=2
770
771	if [ $DRYRUN -le 2 ]; then
772	if [ X${1} = X'/l' ] ; then
773	eval set +A dm_liste \${${2}}
774	else
775	eval set +A dm_liste ${1}
776	fi
777	eval DEST=\${${#}}
778	# add here command to demigrate all offline files to reduce time of this command
779	# actually on Jean Zay files are not archive on bands
780
781	status=$?
782	if [ ${status} -gt 0 ] ; then
783	echo "WARNING IGCM_sys_Get : error code ${status}"
784	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
785	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
786	fi
787
788	#if [ ${status} -gt 0 ] ; then
789	# if [ ! "X$( grep "Lost dmusrcmd connection" ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )" = "X" ] ; then
790	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
791	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
792	# sleep 30
793	# echo "We try another time"
794	## dmget ${dm_liste[*]} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
795	# ccc_hsm get ${dm_liste[*]} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
796	# status=$?
797	# if [ ${status} -gt 0 ] ; then
798	# echo "ERROR IGCM_sys_Get : again demigration error :"
799	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
800	# IGCM_debug_Exit "IGCM_sys_Get"
801	# fi
802	# else
803	# echo "ERROR IGCM_sys_Get : demigration error :"
804	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
805	# IGCM_debug_Exit "IGCM_sys_Get"
806	# fi
807	#fi
808
809	# #RSYNC WITH NETWORK SSH CALL
810	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
811	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
812
813	# #RSYNC WITH NFS USE
814	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
815	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
816
817	# status=$?
818	# IGCM_sys_Rsync_out $status
819
820	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
821	# (( status=status+$? ))
822
823	#USUAL WAY
824	if [ X${1} = X'/l' ] ; then
825	for target in ${dm_liste[*]} ; do
826	local_file=$( basename ${target} )
827	# test if the target file is present before the loop
828	IGCM_sys_TestFileArchive ${target}
829	status=$?
830	if [ ${status} -gt 0 ] ; then
831	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
832	IGCM_debug_Exit "IGCM_sys_Get"
833	else
834	i=0
835	while [ $i -lt $NB_ESSAI ] ; do
836	#if [ X${DoLink} = Xtrue ] ; then
837	# \ln -s ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
838	# status=$?
839	# else
840	# \cp ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
841	# status=$?
842	#fi
843	\ln -s ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
844	status=$?
845	if [ ${status} -gt 0 ]; then
846	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
847	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
848	sleep $DELAI
849	else
850	break
851	fi
852	(( i = i + 1 ))
853	done
854	if [ ${status} -gt 0 ] ; then
855	echo "IGCM_sys_Get : error"
856	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
857	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
858	IGCM_debug_Exit "IGCM_sys_Get"
859	else
860	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
861	fi
862	fi
863	done
864	else
865	i=0
866	while [ $i -lt $NB_ESSAI ] ; do
867	\cp ${dm_liste} ${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
868	status=$?
869	if [ ${status} -gt 0 ]; then
870	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
871	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
872	sleep $DELAI
873	else
874	break
875	fi
876	(( i = i + 1 ))
877	done
878	if [ ${status} -gt 0 ] ; then
879	echo "IGCM_sys_Get : error"
880	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
881	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
882	IGCM_debug_Exit "IGCM_sys_Get"
883	else
884	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
885	fi
886	fi
887	fi
888	IGCM_debug_PopStack "IGCM_sys_Get"
889	}
890
891	#D-#==================================================
892	#D-function IGCM_sys_GetDate_Monitoring
893	#D-* Purpose: get the last year for which the monitoring has been computed
894	#D-* Examples:
895	#D-
896	function IGCM_sys_GetDate_Monitoring {
897	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
898	if ( $DEBUG_sys ) ; then
899	echo "IGCM_sys_GetDate_Monitoring :" $@
900	fi
901
902	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
903
904	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
905	}
906
907	#D-#==================================================
908	#D-function IGCM_sys_Dods_Rm
909	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
910	#D-* Examples:
911	#D-
912	function IGCM_sys_Dods_Rm {
913	if ( $DEBUG_sys ) ; then
914	echo "IGCM_sys_Dods_Rm :" $@
915	fi
916	typeset status
917	if [ $DRYRUN = 0 ]; then
918
919	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
920	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
921	# echo "Nothing has been done."
922	# return
923	# fi
924
925	if [ "$#" -eq 1 ]; then
926	# $(ccc_home -u igcmg)/Tools/irene/thredds_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
927	status=$?
928	else
929	# $(ccc_home -u igcmg)/Tools/irene/thredds_rm public/${LOGIN}/${R_DODS} # > out_dods_rm 2>&1
930	status=$?
931	fi
932
933	# if [ ${status} -gt 0 ] ; then
934	# echo "IGCM_sys_Dods_Rm : error."
935	# cat out_dods_rm
936	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
937	# else
938	# rm out_dods_rm
939	# fi
940
941	fi
942	return $status
943	}
944
945	#D-#==================================================
946	#D-function IGCM_sys_Dods_Cp
947	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
948	#D-* Examples:
949	#D-
950	function IGCM_sys_Dods_Cp {
951	if ( $DEBUG_sys ) ; then
952	echo "IGCM_sys_Dods_Cp :" $@
953	fi
954	typeset status
955	if [ $DRYRUN = 0 ]; then
956
957	# if [ ! -d ${R_SAVE}/${1} ] ; then
958	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
959	# echo "Nothing has been done."
960	# return
961	# fi
962
963	# $(ccc_home -u igcmg)/Tools/irene/thredds_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
964
965	status=$?
966
967	# if [ ${status} -gt 0 ] ; then
968	# echo "IGCM_sys_Dods_Cp : error."
969	# cat out_dods_cp
970	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
971	# else
972	# rm out_dods_cp
973	# fi
974
975	fi
976	return $status
977	}
978
979	#D-#==================================================
980	#D-function IGCM_sys_Put_Dods
981	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole.
982	#D-* Examples:
983	#D-
984	function IGCM_sys_Put_Dods {
985	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
986	if ( $DEBUG_sys ) ; then
987	echo "IGCM_sys_Put_Dods :" $@
988	fi
989	typeset status
990	if [ $DRYRUN = 0 ]; then
991	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
992	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
993	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
994	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
995	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
996	return
997	fi
998
999	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1000	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1001	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1002	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1003	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1004	return
1005	fi
1006	#
1007	if [ -d ${R_SAVE}/${1} ] ; then
1008	cd ${R_SAVE}
1009	elif [ -d ${R_FIGR}/${1} ] ; then
1010	cd ${R_FIGR}
1011	fi
1012
1013	IGCM_sys_Dods_Rm ${1}
1014	IGCM_sys_Dods_Cp ${1}
1015	status=0
1016
1017	if [ ${status} -gt 0 ] ; then
1018	echo "IGCM_sys_Put_Dods : error."
1019	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1020	fi
1021	fi
1022	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1023	}
1024
1025	##############################################################
1026	# REBUILD OPERATOR
1027
1028	#D-#==================================================
1029	#D-function IGCM_sys_sync
1030	#D-* Purpose: flush buffer on disk
1031	#D-* Examples:
1032	#D-
1033	function IGCM_sys_sync {
1034	IGCM_debug_PushStack "IGCM_sys_sync" $@
1035	if ( $DEBUG_sys ) ; then
1036	echo "IGCM_sys_sync :" $@
1037	fi
1038
1039	/bin/sync
1040
1041	IGCM_debug_PopStack "IGCM_sys_sync"
1042	}
1043
1044	############################################################
1045	# Activate Running Environnment Variables
1046
1047	#D-#==================================================
1048	#D-function IGCM_sys_activ_variables
1049	#D-* Purpose: set environement variables prior to execution
1050	#D-* Examples:
1051	#D-
1052	function IGCM_sys_activ_variables {
1053	IGCM_debug_PushStack "IGCM_sys_activ_variables"
1054	if ( $DEBUG_sys ) ; then
1055	echo "IGCM_sys_activ_variables"
1056	fi
1057
1058	# --------------------------------------------------------------------
1059	#D- MODULE specifications
1060	# --------------------------------------------------------------------
1061	# Source the file EnvFile containing all module load needed to run the model.
1062	# EnvFile can be specified in config.card. If this is the case and if the file exists,
1063	# it will be used. Otherwise a default file will be used.
1064	if [ ! X${config_UserChoices_EnvFile} = X ] && [ -f ${config_UserChoices_EnvFile} ] ; then
1065	# EnvFile is set in config.card and the file exists
1066	IGCM_debug_Print 1 "EnvFile set in config.card will be used"
1067	EnvFile=${config_UserChoices_EnvFile}
1068	else
1069	IGCM_debug_Print 1 "IGCM_sys_active_variables : Default modules will be used"
1070	module purge > /dev/null 2>&1
1071	EnvFile=/gpfswork/rech/psl/commun/MachineEnvironment/jeanzay/env_jeanzay
1072	fi
1073	IGCM_debug_Print 1 "IGCM_sys_active_variables : Following EnvFile will be used :"
1074	IGCM_debug_Print 1 ${EnvFile}
1075	. ${EnvFile}
1076
1077	IGCM_debug_Print 1 "IGCM_sys_active_variables : Now loaded modules for Jean-Zay. "
1078	module list
1079
1080	# --------------------------------------------------------------------
1081	#D- MPI specifications
1082	# --------------------------------------------------------------------
1083
1084	# --------------------------------------------------------------------
1085	#D- Other specifications
1086	# --------------------------------------------------------------------
1087
1088	ulimit -s unlimited
1089
1090	IGCM_debug_PopStack "IGCM_sys_activ_variables"
1091	}
1092
1093	############################################################
1094	# Desactivate Running Environnment Variables
1095
1096	#D-#==================================================
1097	#D-function IGCM_sys_desactiv_variables
1098	#D-* Purpose: unset environement variables after execution
1099	#D-* Examples:
1100	#D-
1101	function IGCM_sys_desactiv_variables {
1102	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
1103	if ( $DEBUG_sys ) ; then
1104	echo "IGCM_sys_desactiv_variables"
1105	fi
1106	# --------------------------------------------------------------------
1107	#D- MPI specifications
1108	# --------------------------------------------------------------------
1109	unset SLURM_HOSTFILE
1110	# --------------------------------------------------------------------
1111	#D- Other specifications
1112	# --------------------------------------------------------------------
1113
1114	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
1115	}
1116
1117	############################################################
1118	# Update job headers to be used by the scheduler
1119
1120	#D-#==================================================
1121	#D-function IGCM_sys_updateHeaders
1122	#D-* Purpose: Update job headers to be used by the scheduler
1123	#D-* Examples: IGCM_sys_updateHeaders /path/to/Job_MYEXP
1124	#D-
1125	function IGCM_sys_updateHeaders {
1126	IGCM_debug_PushStack "IGCM_sys_updateHeaders"
1127	if ( $DEBUG_sys ) ; then
1128	echo "IGCM_sys_updateHeaders"
1129	fi
1130	typeset file
1131	file=$1
1132
1133	if [ ${executionType} -eq 1 ] ; then
1134	# MPMD + MPI
1135	sed -e "/::openMPthreads::/d" \
1136	-e "s/::JobNumProcTot::/${coreNumber}/" \
1137	${file} > ${file}.tmp
1138
1139	elif [ ${executionType} -eq 2 ] ; then
1140	# MPMD + MPI + OMP
1141	(( nodeNumber = coreNumber / NB_CORE_PER_NODE ))
1142	[ $(( ${coreNumber} % ${NB_CORE_PER_NODE} )) -ne 0 ] && (( nodeNumber = nodeNumber + 1 ))
1143	sed -e "/::openMPthreads::/d" \
1144	-e "s/::JobNumProcTot::/${mpiTasks}/" \
1145	-e "/ntasks/i\#SBATCH --nodes=${nodeNumber}"\
1146	-e "/ntasks/i\#SBATCH --exclusive" \
1147	${file} > ${file}.tmp
1148	elif [ ${executionType} -eq 3 ] ; then
1149	# SPMD + MPI/OMP
1150	sed -e "s/::openMPthreads::/${openMPthreads}/" \
1151	-e "s/::JobNumProcTot::/${mpiTasks}/" \
1152	${file} > ${file}.tmp
1153
1154	elif [ ${executionType} -eq 4 ] ; then
1155	# SPMD + MPI only
1156	sed -e "s/::JobNumProcTot::/${mpiTasks}/" \
1157	-e "/::openMPthreads::/d" \
1158	${file} > ${file}.tmp
1159
1160	elif [ ${executionType} -eq 5 ] ; then
1161	# SPMD + OMP only
1162	sed -e "s/::openMPthreads::/${openMPthreads}/" \
1163	-e "/::JobNumProcTot::/d" \
1164	${file} > ${file}.tmp
1165
1166	elif [ ${executionType} -eq 6 ] ; then
1167	# SEQUENTIAL THEN
1168	sed -e "s/::JobNumProcTot::/1/" \
1169	-e "/::openMPthreads::/d" \
1170	${file} > ${file}.tmp
1171
1172	fi
1173
1174	IGCM_sys_Mv ${file}.tmp ${file}
1175
1176	IGCM_debug_PopStack "IGCM_sys_updateHeaders"
1177	}
1178
1179	############################################################
1180	# Build MPI/OMP scripts run file (dummy function)
1181
1182	#D-#==================================================
1183	#D-function IGCM_sys_build_run_file
1184	#D-* Purpose: build run file (deprecated)
1185	#D-* Examples:
1186	#D-
1187	function IGCM_sys_build_run_file {
1188	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
1189	}
1190
1191	############################################################
1192	# Build MPI/OMP scripts
1193
1194	#D-#==================================================
1195	#D-function IGCM_sys_build_execution_scripts
1196	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
1197	#D-* Examples:
1198	#D-
1199	function IGCM_sys_build_execution_scripts
1200	{
1201	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
1202	if ( $DEBUG_sys ) ; then
1203	echo "IGCM_sys_build_execution_scripts " $@
1204	fi
1205	IGCM_debug_Print 1 "executionType= ${executionType}"
1206	EXECUTION=${HOST_MPIRUN_COMMAND}
1207
1208	if ( ${OK_PARA_MPMD} ) ; then
1209
1210	# MPMD mode
1211	# 1 MPI only : executionType=1
1212	# 2 MPI/OpenMP : executionType=2
1213
1214	if [ -f run_file ] ; then
1215	IGCM_sys_Rm -f run_file
1216	fi
1217	touch run_file
1218
1219	# case 1 : Only MPI (MPMD)
1220	if ( ! ${OK_PARA_OMP} ) ; then
1221	# Build run_file
1222	current_core=0
1223	# First loop on the components for the coupler ie oasis (only if oasis3)
1224	# the coupler ie oasis3 must be the first one
1225	for comp in ${config_ListOfComponents[*]} ; do
1226
1227	eval ExeNameIn=\${config_Executable_${comp}[0]}
1228	eval ExeNameOut=\${config_Executable_${comp}[1]}
1229
1230	# Only if the component has an executable
1231	if ( [ "X${ExeNameOut}" != X\"\" ] ) ; then
1232
1233	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1234	(( end_core = ${current_core} + ${comp_proc_mpi_loc} - 1 ))
1235	echo "${current_core}-${end_core} ./${ExeNameOut}" >> run_file
1236	(( current_core = ${end_core} + 1 ))
1237	fi
1238	done
1239
1240	EXECUTION="${HOST_MPIRUN_COMMAND} --multi-prog ./run_file"
1241
1242	IGCM_sys_Chmod u+x run_file
1243	if ( $DEBUG_sys ) ; then
1244	echo "run_file contains : "
1245	cat run_file
1246	fi
1247
1248	else
1249
1250	# 2 MPI/OpenMP : executionType=2
1251
1252	# MPI-OpenMP (MPMD)
1253	# export SLURM_HOSTFILE=./hostlist
1254	# srun --cpu-bind=none --distribution=arbitrary --multi-prog ./run_file
1255	# example of hostlist file :
1256	# r3i3n33
1257	# r3i3n33
1258	# ...
1259	# example of run_file :
1260	# 0 taskset -c 0-4 ./script_lmdz.x.ksh
1261	# 1 taskset -c 5-9 ./script_lmdz.x.ksh
1262	#...
1263	# 32 taskset -c 0 ./script_opa.xx.ksh
1264	# 33 taskset -c 1 ./script_opa.xx.ksh
1265	#...
1266	# 62 taskset -c 30 ./script_xios.x.ksh
1267
1268	# Hosts treatment
1269	IFS=$'\n'; set -f
1270	listnodes=($(< <( scontrol show hostnames $SLURM_JOB_NODELIST )))
1271	unset IFS; set +f
1272	rm -f hostlist
1273
1274	# Loop on the components to build run_file and script_exec files
1275	rank=0
1276	current_core=0
1277
1278	for comp in ${config_ListOfComponents[*]} ; do
1279
1280	eval ExeNameIn=\${config_Executable_${comp}[0]}
1281	eval ExeNameOut=\${config_Executable_${comp}[1]}
1282
1283	# Not possible if oasis has an executable (i.e old version of oasis3)
1284	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
1285	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
1286	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
1287	IGCM_debug_Verif_Exit
1288	fi
1289
1290	# Only if we really have an executable for the component :
1291	if [ "X${ExeNameOut}" != X\"\" ] ; then
1292
1293	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1294	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1295	eval comp_proc_nod_loc=\${${comp}_PROC_NOD}
1296
1297
1298	# Build script files
1299
1300	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1301	echo "" >> script_${ExeNameOut}.ksh
1302	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
1303	# Check if the number of threads is correct
1304	case ${comp_proc_omp_loc} in
1305	2\|4\|5\|10\|20)
1306	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
1307	;;
1308	*)
1309	IGCM_debug_Exit "ERROR with OMP parameters !"
1310	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
1311	IGCM_debug_Print 2 "Only 2,4,5,10,20 as number of OMP threads are possible "
1312	IGCM_debug_Verif_Exit
1313	;;
1314	esac
1315	echo "export OMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1316	echo "export OMP_PLACES=cores" >> script_${ExeNameOut}.ksh
1317	echo "export OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1318
1319	fi
1320
1321	# to have out/err per process on different files
1322	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
1323
1324	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1325
1326	# Complete run_file
1327	if [ ${comp_proc_nod_loc} -gt 1 ] ; then
1328	(( offset_comp_proc_loc = NB_CORE_PER_NODE / (comp_proc_mpi_loc / comp_proc_nod_loc) ))
1329	else
1330	(( offset_comp_proc_loc = comp_proc_omp_loc ))
1331	fi
1332
1333	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
1334	(( index_host = current_core / NB_CORE_PER_NODE ))
1335	host_value=${listnodes[${index_host}]}
1336	(( slot = current_core % NB_CORE_PER_NODE ))
1337	echo "$host_value" >> hostlist
1338	echo "$rank taskset -c $slot-$((slot+comp_proc_omp_loc-1)) ./script_${ExeNameOut}.ksh" >> run_file
1339	(( rank = rank + 1 ))
1340	(( current_core = current_core + offset_comp_proc_loc ))
1341	done
1342	fi
1343	done
1344
1345	## variable added to stop after 60s instead of 600s by default.
1346	## This is used when no error comes from executables and when something stopped an executable without notice.
1347	export SLURM_WAIT=60
1348	export SLURM_HOSTFILE=./hostlist
1349
1350	EXECUTION="${HOST_MPIRUN_COMMAND} --cpu-bind=none --distribution=arbitrary --multi-prog ./run_file"
1351
1352	IGCM_sys_Chmod u+x run_file
1353	if ( $DEBUG_sys ) ; then
1354	echo "run_file contains : "
1355	cat run_file
1356	fi
1357
1358	fi # if ${OK_PARA_MPMD}
1359
1360	else
1361	# Only one executable (SPMD mode): executionType=3, 4, 5 and 6
1362
1363	for comp in ${config_ListOfComponents[*]} ; do
1364
1365	# Only if we really have an executable for the component :
1366	eval ExeNameOut=\${config_Executable_${comp}[1]}
1367	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
1368
1369	# Build script files
1370
1371	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1372	echo "" >> script_${ExeNameOut}.ksh
1373	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1374
1375	if ( ${OK_PARA_OMP} ) ; then
1376	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1377	echo "" >> script_${ExeNameOut}.ksh
1378	echo "export OMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1379	echo "export OMP_PLACES=cores" >> script_${ExeNameOut}.ksh
1380	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1381	fi
1382
1383	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1384
1385	# To have out/err per process on different files
1386	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
1387	EXECUTION="${HOST_MPIRUN_COMMAND} ./script_${ExeNameOut}.ksh"
1388
1389	IGCM_debug_Print 1 "sys Irene : script_${ExeNameOut}.ksh contains"
1390	cat script_${ExeNameOut}.ksh
1391
1392	fi
1393
1394	done
1395
1396	fi # ${OK_PARA_MPMD}
1397
1398	IGCM_debug_Print 1 "sys Irene : execution command is "
1399	IGCM_debug_Print 1 "$EXECUTION"
1400
1401	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
1402	}
1403
1404	#D-#==================================================
1405	#D-function IGCM_sys_check_path
1406	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
1407	#D-* do not point to an important use directory. Stop immediately in that case.
1408	#D-* Examples:
1409	#D-
1410	function IGCM_sys_check_path {
1411	IGCM_debug_PushStack "IGCM_sys_check_path"
1412	if ( $DEBUG_sys ) ; then
1413	echo "IGCM_sys_check_path"
1414	fi
1415
1416	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORK} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCH} ] \|\| [ X${RUN_DIR_PATH} = X${STORE} ] ) ; then
1417	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
1418	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
1419	IGCM_debug_Exit "This will stop the job"
1420	fi
1421	IGCM_debug_PopStack "IGCM_sys_check_path"
1422	}
1423
1424	#D-#==================================================
1425	#D-function IGCM_sys_check_quota
1426	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
1427	#D-* Examples:
1428	#D-
1429	function IGCM_sys_check_quota {
1430	IGCM_debug_PushStack "IGCM_sys_check_quota"
1431	echo "IGCM_sys_check_quota is desactivated on JeanZay because there is actually no way to check scratch quota on this computing center (2019.11.19)"
1432	# if ( $DEBUG_sys ) ; then
1433	# echo "IGCM_sys_check_quota"
1434	# fi
1435	# # Limit of quota (in %)
1436	# limit_quota=90
1437	#
1438	# # Check of the volume
1439	# volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
1440	# volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
1441	#
1442	# if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
1443	#
1444	# unit_avail=${volume_avail: -1}
1445	# unit_quota=${volume_quota: -1}
1446	#
1447	# if [ "${unit_quota}" = "*" ] ; then
1448	# IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
1449	# IGCM_debug_Print 1 "More than 100% of your quota is used"
1450	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1451	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1452	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1453	# IGCM_debug_Verif_Exit
1454	# fi
1455	#
1456	# temp_avail=${volume_avail%%${volume_avail: -1}*}
1457	# temp_quota=${volume_quota%%${volume_quota: -1}*}
1458	#
1459	# if [ ! ${unit_avail} = ${unit_quota} ] ; then
1460	#
1461	# # Convertion
1462	# if [ ${volume_avail: -1} = "T" ] ; then
1463	# (( temp_avail = temp_avail * 1000000000000 ))
1464	# elif [ ${volume_avail: -1} = "G" ] ; then
1465	# (( temp_avail = temp_avail * 1000000000 ))
1466	# elif [ ${volume_avail: -1} = "M" ] ; then
1467	# (( temp_avail = temp_avail * 1000000 ))
1468	# elif [ ${volume_avail: -1} = "k" ] ; then
1469	# (( temp_avail = temp_avail * 1000 ))
1470	# else
1471	# (( temp_avail = volume_avail ))
1472	# fi
1473	# if [ ${volume_quota: -1} = "T" ] ; then
1474	# (( temp_quota = temp_quota * 1000000000000 ))
1475	# elif [ ${volume_quota: -1} = "G" ] ; then
1476	# (( temp_quota = temp_quota * 1000000000 ))
1477	# elif [ ${volume_quota: -1} = "M" ] ; then
1478	# (( temp_quota = temp_quota * 1000000 ))
1479	# elif [ ${volume_quota: -1} = "k" ] ; then
1480	# (( temp_quota = temp_quota * 1000 ))
1481	# else
1482	# (( temp_quota = volume_quota ))
1483	# fi
1484	# fi
1485	#
1486	# quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
1487	## echo "volume ratio is " $quota_volume
1488	#
1489	# if [ ${quota_volume} -ge ${limit_quota} ] ; then
1490	# IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
1491	# IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
1492	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1493	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1494	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1495	# IGCM_debug_Verif_Exit
1496	# fi
1497	#
1498	# fi
1499	#
1500	## Check of the number of inodes
1501	#
1502	# inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
1503	# inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
1504	#
1505	# if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
1506	#
1507	# unit_avail=${inode_avail: -1}
1508	# unit_quota=${inode_quota: -1}
1509	#
1510	# if [ "${unit_quota}" = "*" ] ; then
1511	# IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
1512	# IGCM_debug_Print 1 "More than 100% of your quota is used"
1513	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1514	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1515	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1516	# IGCM_debug_Verif_Exit
1517	# fi
1518	#
1519	# temp_avail=${inode_avail%%${inode_avail: -1}*}
1520	# temp_quota=${inode_quota%%${inode_quota: -1}*}
1521	#
1522	# if [ ! ${unit_avail} = ${unit_quota} ] ; then
1523	#
1524	# # Convertion
1525	# if [ ${inode_avail: -1} = "T" ] ; then
1526	# (( temp_avail = temp_avail * 1000000000000 ))
1527	# elif [ ${inode_avail: -1} = "G" ] ; then
1528	# (( temp_avail = temp_avail * 1000000000 ))
1529	# elif [ ${inode_avail: -1} = "M" ] ; then
1530	# (( temp_avail = temp_avail * 1000000 ))
1531	# elif [ ${inode_avail: -1} = "k" ] ; then
1532	# (( temp_avail = temp_avail * 1000 ))
1533	# else
1534	# (( temp_avail = inode_avail ))
1535	# fi
1536	#
1537	# if [ ${inode_quota: -1} = "T" ] ; then
1538	# (( temp_quota = temp_quota * 1000000000000 ))
1539	# elif [ ${inode_quota: -1} = "G" ] ; then
1540	# (( temp_quota = temp_quota * 1000000000 ))
1541	# elif [ ${inode_quota: -1} = "M" ] ; then
1542	# (( temp_quota = temp_quota * 1000000 ))
1543	# elif [ ${inode_quota: -1} = "k" ] ; then
1544	# (( temp_quota = temp_quota * 1000 ))
1545	# else
1546	# (( temp_quota = inode_quota ))
1547	# fi
1548	# fi
1549	# quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
1550	## echo "inode ratio is " $quota_inode
1551	#
1552	# if [ ${quota_inode} -ge ${limit_quota} ] ; then
1553	# IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
1554	# IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
1555	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1556	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1557	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1558	# IGCM_debug_Verif_Exit
1559	# fi
1560	# fi
1561	IGCM_debug_PopStack "IGCM_sys_check_quota"
1562	}
1563
1564	#D-#==================================================
1565	#D-function IGCM_sys_projectAccounting
1566	#D-* Purpose: store project accounting information in a file
1567	#D-* Examples:
1568	#D-
1569	function IGCM_sys_projectAccounting {
1570	IGCM_debug_PushStack "IGCM_sys_projectAccounting"
1571	if ( $DEBUG_sys ) ; then
1572	echo "IGCM_sys_check_quota"
1573	fi
1574	#need to implemented on Jean Zay
1575	touch $1
1576	# ssh irene191 /usr/bin/ccc_myproject > $1
1577
1578	IGCM_debug_PopStack "IGCM_sys_projectAccounting"
1579	}
1580
1581	#D-#==================================================
1582	#D-function IGCM_sys_getJobSchedulerID
1583	#D-* Purpose: Get the job ID during execution
1584	#D-* Examples: IGCM_sys_getJobSchedulerID jobSchedulerID
1585	#D-
1586	function IGCM_sys_getJobSchedulerID {
1587	IGCM_debug_PushStack "IGCM_sys_getJobSchedulerID"
1588	if ( $DEBUG_sys ) ; then
1589	echo "IGCM_sys_getJobSchedulerID"
1590	fi
1591
1592	eval ${1}=${SLURM_JOBID}
1593
1594	IGCM_debug_PopStack "IGCM_sys_getJobSchedulerID"
1595	}
1596
1597	#D-#==================================================
1598	#D-function IGCM_sys_GetJobID
1599	#D-* Purpose: Get the job ID from the JobName
1600	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
1601	#D-
1602	function IGCM_sys_GetJobID {
1603	IGCM_debug_PushStack "IGCM_sys_GetJobID"
1604	if ( $DEBUG_sys ) ; then
1605	echo "IGCM_sys_GetJobID"
1606	fi
1607
1608	ID=$( squeue -u $user \| grep Job_${config_UserChoices_JobName} \| gawk ' { print $1}' )
1609
1610
1611	eval ${3}=${ID}
1612	IGCM_debug_PopStack "IGCM_sys_GetJobID"
1613	}
1614
1615	#D-#==================================================
1616	#D-function IGCM_sys_CountJobInQueue
1617	#D-* Purpose: Count number of users job
1618	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
1619	#D-
1620	function IGCM_sys_CountJobInQueue {
1621	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
1622	if ( $DEBUG_sys ) ; then
1623	echo "IGCM_sys_CountJobInQueue"
1624	fi
1625
1626	# With -f option, the full job name is given in the last column
1627	NbRun=$(squeue -u $user \| \
1628	grep -v JOBID \| gawk 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
1629
1630	eval ${2}=${NbRun}
1631
1632	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
1633	}
1634
1635	#D-#==================================================
1636	#D-function IGCM_sys_ListJobInQueue
1637	#D-* Purpose: Produce a list of users computing jobs (excluding post-processing)
1638	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
1639	#D-
1640	function IGCM_sys_ListJobInQueue {
1641	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
1642	if ( $DEBUG_sys ) ; then
1643	echo "IGCM_sys_ListJobInQueue"
1644	fi
1645
1646	# to be implemented on Jean Zay
1647	set -A JobList $( squeue -u $user \| gawk '{print $3}' \| grep -v NAME)
1648	# # With -f option, the full job name is given in the last column
1649	# set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
1650	# '( $2 == User && \
1651	# $NF != /TS/ && \
1652	# $NF !~ /PACK/ && \
1653	# $NF !~ /REBUILD/ && \
1654	# $NF !~ /pack/ ) \
1655	# { print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
1656	#
1657	eval set -A ${2} ${JobList[*]}
1658
1659	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
1660	}
1661
1662	#D-#==================================================
1663	#D-function IGCM_sys_atlas
1664	#D-* Purpose: encapsulate atlas call so as to manage error code and curie specificity
1665	#D-* Examples:
1666	#D-
1667	function IGCM_sys_atlas {
1668	IGCM_debug_PushStack "IGCM_sys_atlas" $@
1669	if ( $DEBUG_sys ) ; then
1670	echo "IGCM_sys_atlas :" $@
1671	fi
1672
1673	typeset status
1674	# To be implemented on Jean Zay
1675	#
1676	# \ccc_mprun atlas $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1677	# status=$?
1678	# if [ ${status} -gt 0 ] ; then
1679	# echo "IGCM_sys_atlas : error code ${status}"
1680	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1681	# \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1682	# IGCM_debug_PopStack "IGCM_sys_atlas"
1683	# return 1
1684	# else
1685	# IGCM_debug_PopStack "IGCM_sys_atlas"
1686	# return 0
1687	# fi
1688	#
1689	IGCM_debug_PopStack "IGCM_sys_atlas"
1690	}
1691
1692	#D-#==================================================
1693	#D-function IGCM_sys_rebuild_nemo
1694	#D-* Purpose: rebuild nemo parallel files with nemo specific rebuild on curie
1695	#D-* Examples:
1696	#D-
1697
1698	function IGCM_sys_rebuild_nemo {
1699	IGCM_debug_PushStack "IGCM_sys_rebuild_nemo" $@
1700	if ( $DEBUG_sys ) ; then
1701	echo "IGCM_sys_rebuild_nemo :" $@
1702	fi
1703
1704	/gpfswork/rech/psl/commun/Tools/rebuild_nemo/bin/rebuild_nemo ${1} ${2}
1705
1706	IGCM_debug_PopStack "IGCM_sys_rebuild_nemo"
1707	}
1708

Note: See TracBrowser for help on using the repository browser.

Download in other formats: