Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_jeanzay.ksh @ 1507

Last change on this file since 1507 was 1507, checked in by aclsce, 5 years ago
JeanZay? : added a check on number of OMP threads
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 57.2 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Jean-Zay
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host user names project maxCpuTime
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=jeanzay
71	# add default project set by ins_job
72	typeset PROJECT=$( echo ${SLURM_JOB_ACCOUNT:=::default_project::} \| cut -c "1-3" )
73	### jobWarningDelay in seconds
74	##typeset jobWarningDelay=${BRIDGE_MSUB_MAXTIME}
75
76	#D-
77	#D-#==================================================
78	#D-Program used in libIGCM
79	#D-#==================================================
80
81	# Submit command
82	typeset SUBMIT=${SUBMIT:=sbatch}
83	# rsync with path
84	typeset -r RSYNC=/usr/bin/rsync
85	# RSYNC_opt args to rsync
86	typeset -r RSYNC_opt="-va"
87	# ie storage filesystem
88	typeset -r STOREHOST=${MASTER}
89	typeset -r REMOTE_RSYNC=/usr/bin/rsync
90
91	#====================================================
92	# Access to module command
93	#====================================================
94	. /usr/share/Modules/init/ksh
95
96	#====================================================
97	# Set environment tools (ferret, nco, cdo, rebuild, ...)
98	#====================================================
99	if [ X${TaskType} = Xcomputing ] ; then
100	IGCM_debug_Print 1 "Modules will be loaded later in IGCM_sys_activ_variables."
101	else
102	module purge > /dev/null 2>&1
103	. /gpfswork/rech/psl/commun/MachineEnvironment/jeanzay/env_jeanzay > /dev/null 2>&1
104
105	. /gpfswork/rech/psl/commun/MachineEnvironment/jeanzay/env_atlas_jeanzay > /dev/null 2>&1
106
107	# export PCMDI_MP=/ccc/work/cont003/igcmg/igcmg/PCMDI-MP
108	export UVCDAT_ANONYMOUS_LOG=FALSE
109	fi
110
111	# FYI
112	[ ! X${TaskType} = Xchecking ] && IGCM_debug_Print 1 "List of loaded modules:"
113	[ ! X${TaskType} = Xchecking ] && module list
114
115	# For AddNoise (not yet AddPertu3DOCE)
116	export PATH=${PATH}:/gpfswork/rech/psl/commun/Tools/AddNoise/bin
117
118	# For rebuild
119	export PATH=${PATH}:/gpfswork/rech/psl/commun/Tools/rebuild/modipsl_IOIPSL_PLUS_v2_2_4/bin
120
121	#====================================================
122	# Host specific DIRECTORIES
123	#====================================================
124
125	#====================================================
126	#- MirrorlibIGCM for frontend
127	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
128
129	#====================================================
130	#- libIGCM_POST for frontend
131	typeset -r libIGCM_POST=${libIGCM}
132
133	#====================================================
134	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
135	typeset -r R_EXE="${MODIPSL}/bin"
136
137	#====================================================
138	#- SUBMIT_DIR : submission dir
139	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${SLURM_SUBMIT_DIR}}
140
141	#====================================================
142	#- IN
143	typeset -r R_IN=${R_IN:=/gpfswork/rech/psl/commun/IGCM}
144
145	#====================================================
146	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
147	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCH}/RUN_DIR/${SLURM_JOBID}_${$}}
148
149	#====================================================
150	#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
151	typeset -r OUTCOMMAND_PATH=/tmp
152
153	#====================================================
154	#- HOST_MPIRUN_COMMAND
155	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time srun "}
156
157	#====================================================
158	#- Max number of arguments passed to nco operator or demigration command
159	UNIX_MAX_LIMIT=360
160
161	#====================================================
162	#- set PackDefault to true
163	PackDefault=true
164
165	#====================================================
166	#- Default number of MPI task for IPSL coupled model
167	#- required for backward compatibility
168	#-
169	DEFAULT_NUM_PROC_OCE=5
170	DEFAULT_NUM_PROC_CPL=1
171	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
172
173	#====================================================
174	#- Number of computing cores per node
175	#-
176	NB_CORE_PER_NODE=40
177
178	#D-#==================================================
179	#D-function IGCM_sys_defineArchives
180	#D-* Purpose:
181	#D-* Load dfldatadir module : According to project used for submission (default) or set in config.card (optional)
182	#D-* Define ARCHIVE : Dedicated to large files
183	#D-* Define STORAGE : Dedicated to small/medium files
184	#D-* Define R_OUT : Output tree located on ARCHIVE
185	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monsitoring and atlas, and/or small files)
186	#D-* Define R_BUF : Output tree located on SCRATCH hosting files waiting for rebuild or pack processes
187	#D-* if SpaceName=TEST everything is stored on SCRATCH
188	#D-* Examples:
189	#D-
190	function IGCM_sys_defineArchives {
191	IGCM_debug_PushStack "IGCM_sys_defineArchives"
192
193	# Load dfldatadir depending on the project used for submission (default) or set in config.card (optional)
194	if [ X${config_UserChoices_DataProject} = X ] \|\| [ X${config_UserChoices_DataProject} = DEFAULT ]; then
195	# The variable DataProject is not set in in section UserChoices in config.card or it is set to DEFAULT
196	# Default option: Change IDRPROJ according to the project used for submission
197
198	# jg following line does not work in a submitted job:
199	eval $(idrenv -d ${PROJECT})
200	else
201	# Use the project set by the variable DataProject in section UserChoices in config.card
202	eval $(idrenv -d ${config_UserChoices_DataProject})
203	IGCM_debug_Print 1 "Change DataProject red in config.card : ${config_UserChoices_DataProject}"
204	fi
205	IGCM_debug_Print 1 "Data project for output to be used: ${IDRPROJ}"
206
207	#====================================================
208	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
209	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCH}/RUN_DIR/${SLURM_JOBID}_${$}}
210
211	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
212	#====================================================
213	#- ARCHIVE (dedicated to large files)
214	ARCHIVE=${config_UserChoices_ARCHIVE}
215	else
216	#====================================================
217	#- ARCHIVE (dedicated to large files)
218	ARCHIVE=${STORE}
219	fi
220
221	if [ ! X${config_UserChoices_STORAGE} = X ]; then
222	#====================================================
223	#- STORAGE (dedicated to small/medium files)
224	STORAGE=${config_UserChoices_STORAGE}
225	else
226	#====================================================
227	#- STORAGE (dedicated to small/medium files)
228	STORAGE=${WORK}
229	fi
230
231	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
232	#====================================================
233	#- R_OUT
234	R_OUT=${SCRATCH}/IGCM_OUT
235
236	#====================================================
237	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
238	R_FIG=${SCRATCH}/IGCM_OUT
239
240	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
241
242	else
243	#====================================================
244	#- R_OUT
245	R_OUT=${ARCHIVE}/IGCM_OUT
246
247	#====================================================
248	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
249	R_FIG=${STORAGE}/IGCM_OUT
250	fi
251
252	#====================================================
253	#- CMIP6 (hosting CMIP6 files produced by XIOS2 and configured by dr2xml)
254	CMIP6_BUF=${STORAGE}/IGCM_OUT
255
256	#====================================================
257	#- R_BUF (ONLY FOR double copy an scratch)
258	R_BUF=${SCRATCH}/IGCM_OUT
259
260	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
261	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
262	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
263
264	IGCM_debug_PopStack "IGCM_sys_defineArchives"
265	}
266
267	#D-#==================================================
268	#D-function IGCM_sys_RshArchive
269	#D-* Purpose: Archive rsh command
270	#D-* Examples:
271	#D-
272	function IGCM_sys_RshArchive {
273	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
274	/bin/ksh <<-EOF
275	${@}
276	EOF
277	status=$?
278	if [ ${status} -gt 0 ] ; then
279	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
280	IGCM_debug_Exit "IGCM_sys_RshArchive"
281	fi
282	IGCM_debug_PopStack "IGCM_sys_RshArchive"
283	}
284
285	#D-#==================================================
286	#D-function IGCM_sys_RshArchive_NoError
287	#D-* Purpose: Archive rsh command, without error
288	#D-* used only in monitoring.job
289	#D-* Examples:
290	#D-
291	function IGCM_sys_RshArchive_NoError {
292	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
293	/bin/ksh <<-EOF
294	${@} 2> /dev/null
295	EOF
296	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
297	}
298
299	#D-#==================================================
300	#D-function IGCM_sys_MkdirArchive
301	#D-* Purpose: Mkdir on Archive
302	#D-* Examples:
303	#D-
304	function IGCM_sys_MkdirArchive {
305	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
306	if ( $DEBUG_sys ) ; then
307	echo "IGCM_sys_MkdirArchive :" $@
308	fi
309	#- creation de repertoire sur le serveur fichier
310	if [ ! -d ${1} ]; then
311	\mkdir -p $1
312	status=$?
313
314	if [ ${status} -gt 0 ] ; then
315	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
316	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
317	fi
318	fi
319	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
320	}
321
322	#D-#==================================================
323	#D-function IGCM_sys_TestDirArchive
324	#D-* Purpose: Test Directory that must exists on Archive
325	#D-* Examples:
326	#D-
327	function IGCM_sys_TestDirArchive {
328	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
329	if ( $DEBUG_sys ) ; then
330	echo "IGCM_sys_TestDirArchive :" $@
331	fi
332	typeset ExistFlag
333	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
334	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
335	return ${ExistFlag}
336	}
337
338	#D-#==================================================
339	#D-function IGCM_sys_IsFileArchived
340	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
341	#D-* Examples:
342	#D-
343	function IGCM_sys_IsFileArchived {
344	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
345	if ( $DEBUG_sys ) ; then
346	echo "IGCM_sys_IsFileArchived :" $@
347	fi
348	typeset IsArchivedFlag
349	# To be implemented on Jean Zay
350	# IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
351	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
352
353	return ${IsArchivedFlag}
354	}
355
356	#D-#==================================================
357	#D-function IGCM_sys_TestFileArchive
358	#D-* Purpose: Test file that must NOT EXISTS on Archive
359	#D-* Examples:
360	#D-
361	function IGCM_sys_TestFileArchive {
362	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
363	typeset ExistFlag
364	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
365	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
366
367	return ${ExistFlag}
368	}
369
370	#D-#==================================================
371	#D-function IGCM_sys_CountFileArchive
372	#D-* Purpose: Count files on Archive filesystem
373	#D-* Examples:
374	#D-
375	function IGCM_sys_CountFileArchive {
376	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
377	ls ${@} 2>/dev/null \| wc -l
378	if [ $? -gt 0 ] ; then
379	echo "IGCM_sys_CountFileArchive : erreur."
380	fi
381	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
382	}
383
384	#D-#==================================================
385	#D-function IGCM_sys_Tree
386	#D-* Purpose: Tree directories with files on ${ARCHIVE}
387	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
388	#D-
389	function IGCM_sys_Tree {
390	IGCM_debug_PushStack "IGCM_sys_Tree" $@
391	if ( $DEBUG_sys ) ; then
392	echo "IGCM_sys_Tree :" $@
393	fi
394
395	\ls -lR ${@}
396
397	IGCM_debug_PopStack "IGCM_sys_Tree"
398	}
399
400	#D-#==================================================
401	#D-function IGCM_sys_Qsub
402	#D-* Purpose: Qsub new job
403	#D-* Examples:
404	#D-
405	function IGCM_sys_Qsub {
406	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
407	if ( $DEBUG_sys ) ; then
408	echo "IGCM_sys_Qsub :" $@
409	fi
410	typeset options status
411	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
412
413	/usr/bin/time sbatch ${options} $1 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
414	status=$?
415
416	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
417	if [ ${status} -gt 0 ] ; then
418	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
419	IGCM_debug_Exit "IGCM_sys_Qsub"
420	else
421	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
422	fi
423	IGCM_debug_PopStack "IGCM_sys_Qsub"
424	}
425
426	#D-#==================================================
427	#D-function IGCM_sys_QsubPost
428	#D-* Purpose: Qsub new job on scalaire
429	#D-* Examples:
430	#D-
431	function IGCM_sys_QsubPost {
432	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
433	if ( $DEBUG_sys ) ; then
434	echo "IGCM_sys_QsubPost :" $@
435	fi
436	typeset options status
437	options="-o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
438
439	sbatch ${options} ${libIGCM_POST}/$1.job > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
440	status=$?
441
442	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
443	if [ ${status} -gt 0 ] ; then
444	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
445	IGCM_debug_Exit "IGCM_sys_QsubPost"
446	else
447	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
448	fi
449	IGCM_debug_PopStack "IGCM_sys_QsubPost"
450	}
451
452	#D-*************************
453	#D- File transfer functions
454	#D-*************************
455	#D-
456
457	#D-#==================================================
458	#D-function IGCM_sys_RmRunDir
459	#D-* Purpose: rm tmpdir (dummy function most of the time batch
460	#D- scheduler will do the job)
461	#D-* Examples:
462	#D-
463	function IGCM_sys_RmRunDir {
464	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
465	if ( $DEBUG_sys ) ; then
466	echo "IGCM_sys_RmRunDir :" $@
467	fi
468
469	typeset status
470
471	echo rm $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
472	\rm $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
473	status=$?
474
475	if [ ${status} -gt 0 ] ; then
476	IGCM_debug_Print 1 "IGCM_sys_RmRunDir : rm error code is ${status}."
477	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
478	IGCM_debug_Exit "IGCM_sys_RmRunDir"
479	else
480	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
481	fi
482	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
483	}
484
485	#D-#==================================================
486	#D-function IGCM_sys_Put_Dir
487	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
488	#D-* Examples:
489	#D-
490	function IGCM_sys_Put_Dir {
491	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
492	if ( $DEBUG_sys ) ; then
493	echo "IGCM_sys_Put_Dir :" $@
494	fi
495	if [ $DRYRUN = 0 ]; then
496	if [ ! -d ${1} ] ; then
497	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
498	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
499	return
500	fi
501
502	typeset status
503
504	# Only if we use rsync
505	#IGCM_sys_TestDirArchive $( dirname $2 )
506	#
507	#USUAL WAY
508	\cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
509	status=$?
510
511	if [ ${status} -gt 0 ] ; then
512	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
513	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
514	IGCM_debug_Exit "IGCM_sys_Put_Dir"
515	else
516	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
517	fi
518	fi
519	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
520	}
521
522	#D-#==================================================
523	#D-function IGCM_sys_Get_Dir
524	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
525	#D-* Examples:
526	#D-
527	function IGCM_sys_Get_Dir {
528	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
529	if ( $DEBUG_sys ) ; then
530	echo "IGCM_sys_Get_Dir :" $@
531	fi
532	if [ $DRYRUN = 0 ]; then
533	typeset NB_ESSAI DELAI status i
534	# number of tentative
535	NB_ESSAI=3
536	# time delay between tentative
537	DELAI=2
538
539	#
540	# USUAL WAY
541	# add here command to demigrate all offline files to reduce time of this command
542	# actually on Jean Zay files are not archive on bands
543
544	i=0
545	while [ $i -lt $NB_ESSAI ] ; do
546	\cp -ur $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
547	status=$?
548	if [ ${status} -gt 0 ] ; then
549	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
550	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
551	sleep $DELAI
552	else
553	break
554	fi
555	(( i = i + 1 ))
556	done
557
558	if [ ${status} -gt 0 ] ; then
559	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
560	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
561	IGCM_debug_Exit "IGCM_sys_Get_Dir"
562	else
563	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
564	fi
565	fi
566	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
567	}
568
569	#D-#==================================================
570	#D-function IGCM_sys_Put_Rest
571	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
572	#D- File and target directory must exist.
573	#D-* Examples:
574	#D-
575	function IGCM_sys_Put_Rest {
576	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
577	if ( $DEBUG_sys ) ; then
578	echo "IGCM_sys_Put_Rest :" $@
579	fi
580	if [ $DRYRUN = 0 ]; then
581	if [ ! -f ${1} ] ; then
582	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
583	IGCM_debug_Exit "IGCM_sys_Put_Rest"
584	fi
585
586	typeset status
587	#
588	# USUAL WAY
589	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
590	status=$?
591
592	# #RSYNC WITH NETWORK SSH CALL
593	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
594	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
595
596	# #RSYNC WITH NFS USE
597	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
598	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
599
600	# status=$?
601	# IGCM_sys_Rsync_out $status
602
603	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
604	# (( status=status+$? ))
605
606	if [ ${status} -gt 0 ] ; then
607	IGCM_debug_Print 2 "IGCM_sys_Put_Rest : cp failed error code ${status}"
608	[ -f ${1} ] && ls -l ${1}
609	[ -f ${2} ] && ls -l ${2}
610	[ -f ${2}/${1} ] && ls -l ${2}/${1}
611	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
612	IGCM_debug_Exit "IGCM_sys_Put_Rest"
613	else
614
615	if [ X${JobType} = XRUN ] ; then
616	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
617	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
618	fi
619
620	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
621	fi
622	fi
623	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
624	}
625
626	#D-#==================================================
627	#D-function IGCM_sys_Put_Out
628	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
629	#D-* Examples:
630	#D-
631	function IGCM_sys_Put_Out {
632	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
633	if ( $DEBUG_sys ) ; then
634	echo "IGCM_sys_Put_Out :" $@
635	fi
636
637	typeset NB_ESSAI DELAI status i exist skip
638	typeset fileDeviceNumberInHex directoryDeviceNumberInHex
639
640	# number of tentative
641	NB_ESSAI=3
642	# time delay between tentative
643	DELAI=2
644
645	if [ $DRYRUN = 0 ]; then
646	if [ ! -f ${1} ] ; then
647	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
648	IGCM_debug_PopStack "IGCM_sys_Put_Out"
649	return 1
650	fi
651	#
652	IGCM_sys_MkdirArchive $( dirname $2 )
653	#
654	exist=false
655	skip=false
656	if [ -f $2 ] ; then
657	IGCM_debug_Print 1 "$2 already exist"
658	# add here command to demigrate all offline files to reduce time of this command
659	# actually on Jean Zay files are not archive on bands
660
661	exist=true
662	if [ "X$( diff $1 $2 )" = X ] ; then
663	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
664	skip=true
665	else
666	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
667	skip=false
668	fi
669	fi
670	#
671	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
672	IGCM_sys_Chmod u+w $2
673	fi
674
675	if [ X${skip} = Xfalse ] ; then
676	i=0
677	while [ $i -lt $NB_ESSAI ] ; do
678	# Identify file system
679	fileDeviceNumberInHex=$( stat -c %d $1 )
680	status=$?
681	if [ ${status} -gt 0 ] ; then
682	IGCM_debug_Exit "IGCM_sys_Put_Out"
683	fi
684	# Identify file system
685	directoryDeviceNumberInHex=$( stat -c %d $( dirname $2 ) )
686	status=$?
687	if [ ${status} -gt 0 ] ; then
688	IGCM_debug_Exit "IGCM_sys_Put_Out"
689	fi
690
691	if ( [ ${fileDeviceNumberInHex} -ne ${directoryDeviceNumberInHex} ] \|\| [ X$3 = XNOMOVE ] ) ; then
692	# They are not on the same device. USUAL WAY
693	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
694	status=$?
695	else
696	# They are on the same device. NOT SO USUAL WAY
697	\mv $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
698	status=$?
699	fi
700	if [ ${status} -gt 0 ]; then
701	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
702	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
703	[ -f ${1} ] && ls -l ${1}
704	[ -f ${2} ] && ls -l ${2}
705	[ -f ${2}/${1} ] && ls -l ${2}/${1}
706	sleep $DELAI
707	else
708	break
709	fi
710	(( i = i + 1 ))
711	done
712	fi
713
714	# #RSYNC WITH NETWORK SSH CALL
715	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
716	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
717
718	# #RSYNC WITH NFS USE
719	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
720	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
721
722	# status=$?
723	# IGCM_sys_Rsync_out $status
724
725	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
726	# (( status=status+$? ))
727
728	if [ ${status} -gt 0 ] ; then
729	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status}"
730	[ -f ${1} ] && ls -l ${1}
731	[ -f ${2} ] && ls -l ${2}
732	[ -f ${2}/${1} ] && ls -l ${2}/${1}
733	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
734	IGCM_debug_Exit "IGCM_sys_Put_Out"
735	else
736
737	if [ X${JobType} = XRUN ] ; then
738	if [ X${3} = X ] ; then
739	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
740	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
741	fi
742	fi
743
744	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
745	fi
746	fi
747	IGCM_debug_PopStack "IGCM_sys_Put_Out"
748	return 0
749	}
750
751	#D-#==================================================
752	#D-function IGCM_sys_Get
753	#D-* Purpose: Get a file from ${ARCHIVE}
754	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
755	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
756	function IGCM_sys_Get {
757	IGCM_debug_PushStack "IGCM_sys_Get" $@
758
759	typeset DEST dm_liste target file_work
760	typeset NB_ESSAI DELAI status i
761
762	if ( $DEBUG_sys ) ; then
763	echo "IGCM_sys_Get :" $@
764	fi
765
766	# number of tentative
767	NB_ESSAI=3
768	# time delay between tentative
769	DELAI=2
770
771	if [ $DRYRUN -le 2 ]; then
772	if [ X${1} = X'/l' ] ; then
773	eval set +A dm_liste \${${2}}
774	else
775	eval set +A dm_liste ${1}
776	fi
777	eval DEST=\${${#}}
778	# add here command to demigrate all offline files to reduce time of this command
779	# actually on Jean Zay files are not archive on bands
780
781	status=$?
782	if [ ${status} -gt 0 ] ; then
783	echo "WARNING IGCM_sys_Get : error code ${status}"
784	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
785	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
786	fi
787
788	#if [ ${status} -gt 0 ] ; then
789	# if [ ! "X$( grep "Lost dmusrcmd connection" ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )" = "X" ] ; then
790	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
791	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
792	# sleep 30
793	# echo "We try another time"
794	## dmget ${dm_liste[*]} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
795	# ccc_hsm get ${dm_liste[*]} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
796	# status=$?
797	# if [ ${status} -gt 0 ] ; then
798	# echo "ERROR IGCM_sys_Get : again demigration error :"
799	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
800	# IGCM_debug_Exit "IGCM_sys_Get"
801	# fi
802	# else
803	# echo "ERROR IGCM_sys_Get : demigration error :"
804	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
805	# IGCM_debug_Exit "IGCM_sys_Get"
806	# fi
807	#fi
808
809	# #RSYNC WITH NETWORK SSH CALL
810	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
811	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
812
813	# #RSYNC WITH NFS USE
814	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
815	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
816
817	# status=$?
818	# IGCM_sys_Rsync_out $status
819
820	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
821	# (( status=status+$? ))
822
823	#USUAL WAY
824	if [ X${1} = X'/l' ] ; then
825	for target in ${dm_liste[*]} ; do
826	local_file=$( basename ${target} )
827	# test if the target file is present before the loop
828	IGCM_sys_TestFileArchive ${target}
829	status=$?
830	if [ ${status} -gt 0 ] ; then
831	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
832	IGCM_debug_Exit "IGCM_sys_Get"
833	else
834	i=0
835	while [ $i -lt $NB_ESSAI ] ; do
836	#if [ X${DoLink} = Xtrue ] ; then
837	# \ln -s ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
838	# status=$?
839	# else
840	# \cp ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
841	# status=$?
842	#fi
843	\ln -s ${target} ${DEST}/${local_file} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
844	status=$?
845	if [ ${status} -gt 0 ]; then
846	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
847	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
848	sleep $DELAI
849	else
850	break
851	fi
852	(( i = i + 1 ))
853	done
854	if [ ${status} -gt 0 ] ; then
855	echo "IGCM_sys_Get : error"
856	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
857	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
858	IGCM_debug_Exit "IGCM_sys_Get"
859	else
860	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
861	fi
862	fi
863	done
864	else
865	i=0
866	while [ $i -lt $NB_ESSAI ] ; do
867	\cp ${dm_liste} ${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
868	status=$?
869	if [ ${status} -gt 0 ]; then
870	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
871	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
872	sleep $DELAI
873	else
874	break
875	fi
876	(( i = i + 1 ))
877	done
878	if [ ${status} -gt 0 ] ; then
879	echo "IGCM_sys_Get : error"
880	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
881	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
882	IGCM_debug_Exit "IGCM_sys_Get"
883	else
884	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
885	fi
886	fi
887	fi
888	IGCM_debug_PopStack "IGCM_sys_Get"
889	}
890
891	#D-#==================================================
892	#D-function IGCM_sys_GetDate_Monitoring
893	#D-* Purpose: get the last year for which the monitoring has been computed
894	#D-* Examples:
895	#D-
896	function IGCM_sys_GetDate_Monitoring {
897	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
898	if ( $DEBUG_sys ) ; then
899	echo "IGCM_sys_GetDate_Monitoring :" $@
900	fi
901
902	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
903
904	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
905	}
906
907	#D-#==================================================
908	#D-function IGCM_sys_Dods_Rm
909	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
910	#D-* Examples:
911	#D-
912	function IGCM_sys_Dods_Rm {
913	if ( $DEBUG_sys ) ; then
914	echo "IGCM_sys_Dods_Rm :" $@
915	fi
916	typeset status
917	if [ $DRYRUN = 0 ]; then
918
919	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
920	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
921	# echo "Nothing has been done."
922	# return
923	# fi
924
925	if [ "$#" -eq 1 ]; then
926	# $(ccc_home -u igcmg)/Tools/irene/thredds_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
927	status=$?
928	else
929	# $(ccc_home -u igcmg)/Tools/irene/thredds_rm public/${LOGIN}/${R_DODS} # > out_dods_rm 2>&1
930	status=$?
931	fi
932
933	# if [ ${status} -gt 0 ] ; then
934	# echo "IGCM_sys_Dods_Rm : error."
935	# cat out_dods_rm
936	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
937	# else
938	# rm out_dods_rm
939	# fi
940
941	fi
942	return $status
943	}
944
945	#D-#==================================================
946	#D-function IGCM_sys_Dods_Cp
947	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
948	#D-* Examples:
949	#D-
950	function IGCM_sys_Dods_Cp {
951	if ( $DEBUG_sys ) ; then
952	echo "IGCM_sys_Dods_Cp :" $@
953	fi
954	typeset status
955	if [ $DRYRUN = 0 ]; then
956
957	# if [ ! -d ${R_SAVE}/${1} ] ; then
958	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
959	# echo "Nothing has been done."
960	# return
961	# fi
962
963	# $(ccc_home -u igcmg)/Tools/irene/thredds_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
964
965	status=$?
966
967	# if [ ${status} -gt 0 ] ; then
968	# echo "IGCM_sys_Dods_Cp : error."
969	# cat out_dods_cp
970	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
971	# else
972	# rm out_dods_cp
973	# fi
974
975	fi
976	return $status
977	}
978
979	#D-#==================================================
980	#D-function IGCM_sys_Put_Dods
981	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole.
982	#D-* Examples:
983	#D-
984	function IGCM_sys_Put_Dods {
985	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
986	if ( $DEBUG_sys ) ; then
987	echo "IGCM_sys_Put_Dods :" $@
988	fi
989	typeset status
990	if [ $DRYRUN = 0 ]; then
991	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
992	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
993	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
994	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
995	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
996	return
997	fi
998
999	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1000	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1001	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1002	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1003	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1004	return
1005	fi
1006	#
1007	if [ -d ${R_SAVE}/${1} ] ; then
1008	cd ${R_SAVE}
1009	elif [ -d ${R_FIGR}/${1} ] ; then
1010	cd ${R_FIGR}
1011	fi
1012
1013	IGCM_sys_Dods_Rm ${1}
1014	IGCM_sys_Dods_Cp ${1}
1015	status=0
1016
1017	if [ ${status} -gt 0 ] ; then
1018	echo "IGCM_sys_Put_Dods : error."
1019	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1020	fi
1021	fi
1022	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1023	}
1024
1025	##############################################################
1026	# REBUILD OPERATOR
1027
1028	#D-#==================================================
1029	#D-function IGCM_sys_sync
1030	#D-* Purpose: flush buffer on disk
1031	#D-* Examples:
1032	#D-
1033	function IGCM_sys_sync {
1034	IGCM_debug_PushStack "IGCM_sys_sync" $@
1035	if ( $DEBUG_sys ) ; then
1036	echo "IGCM_sys_sync :" $@
1037	fi
1038
1039	/bin/sync
1040
1041	IGCM_debug_PopStack "IGCM_sys_sync"
1042	}
1043
1044	############################################################
1045	# Activate Running Environnment Variables
1046
1047	#D-#==================================================
1048	#D-function IGCM_sys_activ_variables
1049	#D-* Purpose: set environement variables prior to execution
1050	#D-* Examples:
1051	#D-
1052	function IGCM_sys_activ_variables {
1053	IGCM_debug_PushStack "IGCM_sys_activ_variables"
1054	if ( $DEBUG_sys ) ; then
1055	echo "IGCM_sys_activ_variables"
1056	fi
1057
1058	# --------------------------------------------------------------------
1059	#D- MODULE specifications
1060	# --------------------------------------------------------------------
1061	# Source the file EnvFile containing all module load needed to run the model.
1062	# EnvFile can be specified in config.card. If this is the case and if the file exists,
1063	# it will be used. Otherwise a default file will be used.
1064	if [ ! X${config_UserChoices_EnvFile} = X ] && [ -f ${config_UserChoices_EnvFile} ] ; then
1065	# EnvFile is set in config.card and the file exists
1066	IGCM_debug_Print 1 "EnvFile set in config.card will be used"
1067	EnvFile=${config_UserChoices_EnvFile}
1068	else
1069	IGCM_debug_Print 1 "IGCM_sys_active_variables : Default modules will be used"
1070	module purge > /dev/null 2>&1
1071	EnvFile=/gpfswork/rech/psl/commun/MachineEnvironment/jeanzay/env_jeanzay
1072	fi
1073	IGCM_debug_Print 1 "IGCM_sys_active_variables : Following EnvFile will be used :"
1074	IGCM_debug_Print 1 ${EnvFile}
1075	. ${EnvFile}
1076
1077	IGCM_debug_Print 1 "IGCM_sys_active_variables : Now loaded modules for Jean-Zay. "
1078	module list
1079
1080	# --------------------------------------------------------------------
1081	#D- MPI specifications
1082	# --------------------------------------------------------------------
1083
1084	# --------------------------------------------------------------------
1085	#D- Other specifications
1086	# --------------------------------------------------------------------
1087
1088	ulimit -s unlimited
1089	if [ ${executionType} -eq 2 ] ; then
1090	export SLURM_HOSTFILE=./hostlist
1091	fi
1092
1093
1094	IGCM_debug_PopStack "IGCM_sys_activ_variables"
1095	}
1096
1097	############################################################
1098	# Desactivate Running Environnment Variables
1099
1100	#D-#==================================================
1101	#D-function IGCM_sys_desactiv_variables
1102	#D-* Purpose: unset environement variables after execution
1103	#D-* Examples:
1104	#D-
1105	function IGCM_sys_desactiv_variables {
1106	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
1107	if ( $DEBUG_sys ) ; then
1108	echo "IGCM_sys_desactiv_variables"
1109	fi
1110	# --------------------------------------------------------------------
1111	#D- MPI specifications
1112	# --------------------------------------------------------------------
1113	if [ ${executionType} -eq 2 ] ; then
1114	unset SLURM_HOSTFILE
1115	fi
1116	# --------------------------------------------------------------------
1117	#D- Other specifications
1118	# --------------------------------------------------------------------
1119
1120	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
1121	}
1122
1123	############################################################
1124	# Update job headers to be used by the scheduler
1125
1126	#D-#==================================================
1127	#D-function IGCM_sys_updateHeaders
1128	#D-* Purpose: Update job headers to be used by the scheduler
1129	#D-* Examples: IGCM_sys_updateHeaders /path/to/Job_MYEXP
1130	#D-
1131	function IGCM_sys_updateHeaders {
1132	IGCM_debug_PushStack "IGCM_sys_updateHeaders"
1133	if ( $DEBUG_sys ) ; then
1134	echo "IGCM_sys_updateHeaders"
1135	fi
1136	typeset file
1137	file=$1
1138
1139	if [ ${executionType} -eq 1 ] ; then
1140	# MPMD + MPI
1141	sed -e "/::openMPthreads::/d" \
1142	-e "s/::JobNumProcTot::/${coreNumber}/" \
1143	${file} > ${file}.tmp
1144
1145	elif [ ${executionType} -eq 2 ] ; then
1146	# MPMD + MPI + OMP
1147	(( nodeNumber = coreNumber / NB_CORE_PER_NODE ))
1148	[ $(( ${coreNumber} % ${NB_CORE_PER_NODE} )) -ne 0 ] && (( nodeNumber = nodeNumber + 1 ))
1149	sed -e "/::openMPthreads::/d" \
1150	-e "s/::JobNumProcTot::/${mpiTasks}/" \
1151	-e "/ntasks/i\#SBATCH --nodes=${nodeNumber}"\
1152	-e "/ntasks/i\#SBATCH --exclusive" \
1153	${file} > ${file}.tmp
1154	elif [ ${executionType} -eq 3 ] ; then
1155	# SPMD + MPI/OMP
1156	sed -e "s/::openMPthreads::/${openMPthreads}/" \
1157	-e "s/::JobNumProcTot::/${mpiTasks}/" \
1158	${file} > ${file}.tmp
1159
1160	elif [ ${executionType} -eq 4 ] ; then
1161	# SPMD + MPI only
1162	sed -e "s/::JobNumProcTot::/${mpiTasks}/" \
1163	-e "/::openMPthreads::/d" \
1164	${file} > ${file}.tmp
1165
1166	elif [ ${executionType} -eq 5 ] ; then
1167	# SPMD + OMP only
1168	sed -e "s/::openMPthreads::/${openMPthreads}/" \
1169	-e "/::JobNumProcTot::/d" \
1170	${file} > ${file}.tmp
1171
1172	elif [ ${executionType} -eq 6 ] ; then
1173	# SEQUENTIAL THEN
1174	sed -e "s/::JobNumProcTot::/1/" \
1175	-e "/::openMPthreads::/d" \
1176	${file} > ${file}.tmp
1177
1178	fi
1179
1180	IGCM_sys_Mv ${file}.tmp ${file}
1181
1182	IGCM_debug_PopStack "IGCM_sys_updateHeaders"
1183	}
1184
1185	############################################################
1186	# Build MPI/OMP scripts run file (dummy function)
1187
1188	#D-#==================================================
1189	#D-function IGCM_sys_build_run_file
1190	#D-* Purpose: build run file (deprecated)
1191	#D-* Examples:
1192	#D-
1193	function IGCM_sys_build_run_file {
1194	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
1195	}
1196
1197	############################################################
1198	# Build MPI/OMP scripts
1199
1200	#D-#==================================================
1201	#D-function IGCM_sys_build_execution_scripts
1202	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
1203	#D-* Examples:
1204	#D-
1205	function IGCM_sys_build_execution_scripts
1206	{
1207	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
1208	if ( $DEBUG_sys ) ; then
1209	echo "IGCM_sys_build_execution_scripts " $@
1210	fi
1211	IGCM_debug_Print 1 "executionType= ${executionType}"
1212	EXECUTION=${HOST_MPIRUN_COMMAND}
1213
1214	if ( ${OK_PARA_MPMD} ) ; then
1215
1216	# MPMD mode
1217	# 1 MPI only : executionType=1
1218	# 2 MPI/OpenMP : executionType=2
1219
1220	if [ -f run_file ] ; then
1221	IGCM_sys_Rm -f run_file
1222	fi
1223	touch run_file
1224
1225	# case 1 : Only MPI (MPMD)
1226	if ( ! ${OK_PARA_OMP} ) ; then
1227	# Build run_file
1228	current_core=0
1229	# First loop on the components for the coupler ie oasis (only if oasis3)
1230	# the coupler ie oasis3 must be the first one
1231	for comp in ${config_ListOfComponents[*]} ; do
1232
1233	eval ExeNameIn=\${config_Executable_${comp}[0]}
1234	eval ExeNameOut=\${config_Executable_${comp}[1]}
1235
1236	# Only if the component has an executable
1237	if ( [ "X${ExeNameOut}" != X\"\" ] ) ; then
1238
1239	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1240	(( end_core = ${current_core} + ${comp_proc_mpi_loc} - 1 ))
1241	echo "${current_core}-${end_core} ./${ExeNameOut}" >> run_file
1242	(( current_core = ${end_core} + 1 ))
1243	fi
1244	done
1245
1246	EXECUTION="${HOST_MPIRUN_COMMAND} --multi-prog ./run_file"
1247
1248	IGCM_sys_Chmod u+x run_file
1249	if ( $DEBUG_sys ) ; then
1250	echo "run_file contains : "
1251	cat run_file
1252	fi
1253
1254	else
1255
1256	# 2 MPI/OpenMP : executionType=2
1257
1258	# MPI-OpenMP (MPMD)
1259	# export SLURM_HOSTFILE=./hostlist
1260	# srun --cpu-bind=none --distribution=arbitrary --multi-prog ./run_file
1261	# example of hostlist file :
1262	# r3i3n33
1263	# r3i3n33
1264	# ...
1265	# example of run_file :
1266	# 0 taskset -c 0-4 ./script_lmdz.x.ksh
1267	# 1 taskset -c 5-9 ./script_lmdz.x.ksh
1268	#...
1269	# 32 taskset -c 0 ./script_opa.xx.ksh
1270	# 33 taskset -c 1 ./script_opa.xx.ksh
1271	#...
1272	# 62 taskset -c 30 ./script_xios.x.ksh
1273
1274	# Hosts treatment
1275	IFS=$'\n'; set -f
1276	listnodes=($(< <( scontrol show hostnames $SLURM_JOB_NODELIST )))
1277	unset IFS; set +f
1278	rm -f hostlist
1279
1280	# Loop on the components to build run_file and script_exec files
1281	rank=0
1282	current_core=0
1283
1284	for comp in ${config_ListOfComponents[*]} ; do
1285
1286	eval ExeNameIn=\${config_Executable_${comp}[0]}
1287	eval ExeNameOut=\${config_Executable_${comp}[1]}
1288
1289	# Not possible if oasis has an executable (i.e old version of oasis3)
1290	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
1291	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
1292	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
1293	IGCM_debug_Verif_Exit
1294	fi
1295
1296	# Only if we really have an executable for the component :
1297	if [ "X${ExeNameOut}" != X\"\" ] ; then
1298
1299	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1300	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1301	eval comp_proc_nod_loc=\${${comp}_PROC_NOD}
1302
1303
1304	# Build script files
1305
1306	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1307	echo "" >> script_${ExeNameOut}.ksh
1308	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
1309	# Check if the number of threads is correct
1310	case ${comp_proc_omp_loc} in
1311	2\|4\|5\|10\|20)
1312	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
1313	;;
1314	*)
1315	IGCM_debug_Exit "ERROR with OMP parameters !"
1316	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
1317	IGCM_debug_Print 2 "Only 2,4,5,10,20 as number of OMP threads are possible "
1318	IGCM_debug_Verif_Exit
1319	;;
1320	esac
1321	echo "export OMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1322	echo "export OMP_PLACES=cores" >> script_${ExeNameOut}.ksh
1323	echo "export OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1324
1325	fi
1326
1327	# to have out/err per process on different files
1328	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
1329
1330	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1331
1332	# Complete run_file
1333	if [ ${comp_proc_nod_loc} -gt 1 ] ; then
1334	(( offset_comp_proc_loc = NB_CORE_PER_NODE / (comp_proc_mpi_loc / comp_proc_nod_loc) ))
1335	else
1336	(( offset_comp_proc_loc = comp_proc_omp_loc ))
1337	fi
1338
1339	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
1340	(( index_host = current_core / NB_CORE_PER_NODE ))
1341	host_value=${listnodes[${index_host}]}
1342	(( slot = current_core % NB_CORE_PER_NODE ))
1343	echo "$host_value" >> hostlist
1344	echo "$rank taskset -c $slot-$((slot+comp_proc_omp_loc-1)) ./script_${ExeNameOut}.ksh" >> run_file
1345	(( rank = rank + 1 ))
1346	(( current_core = current_core + offset_comp_proc_loc ))
1347	done
1348	fi
1349	done
1350
1351	## variable added to stop after 60s instead of 600s by default.
1352	## This is used when no error comes from executables and when something stopped an executable without notice.
1353	export SLURM_WAIT=60
1354
1355	EXECUTION="${HOST_MPIRUN_COMMAND} --cpu-bind=none --distribution=arbitrary --multi-prog ./run_file"
1356
1357	IGCM_sys_Chmod u+x run_file
1358	if ( $DEBUG_sys ) ; then
1359	echo "run_file contains : "
1360	cat run_file
1361	fi
1362
1363	fi # if ${OK_PARA_MPMD}
1364
1365	else
1366	# Only one executable (SPMD mode): executionType=3, 4, 5 and 6
1367
1368	for comp in ${config_ListOfComponents[*]} ; do
1369
1370	# Only if we really have an executable for the component :
1371	eval ExeNameOut=\${config_Executable_${comp}[1]}
1372	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
1373
1374	# Build script files
1375
1376	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1377	echo "" >> script_${ExeNameOut}.ksh
1378	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1379
1380	if ( ${OK_PARA_OMP} ) ; then
1381	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1382	# Check if the number of threads is correct
1383	case ${comp_proc_omp_loc} in
1384	2\|4\|5\|10\|20)
1385	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
1386	;;
1387	*)
1388	IGCM_debug_Exit "ERROR with OMP parameters !"
1389	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
1390	IGCM_debug_Print 2 "Only 2,4,5,10,20 as number of OMP threads are possible "
1391	IGCM_debug_Verif_Exit
1392	;;
1393	esac
1394	echo "" >> script_${ExeNameOut}.ksh
1395	echo "export OMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
1396	echo "export OMP_PLACES=cores" >> script_${ExeNameOut}.ksh
1397	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1398	fi
1399
1400	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1401
1402	# To have out/err per process on different files
1403	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
1404	EXECUTION="${HOST_MPIRUN_COMMAND} ./script_${ExeNameOut}.ksh"
1405
1406	IGCM_debug_Print 1 "sys Irene : script_${ExeNameOut}.ksh contains"
1407	cat script_${ExeNameOut}.ksh
1408
1409	fi
1410
1411	done
1412
1413	fi # ${OK_PARA_MPMD}
1414
1415	IGCM_debug_Print 1 "sys Irene : execution command is "
1416	IGCM_debug_Print 1 "$EXECUTION"
1417
1418	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
1419	}
1420
1421	#D-#==================================================
1422	#D-function IGCM_sys_check_path
1423	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
1424	#D-* do not point to an important use directory. Stop immediately in that case.
1425	#D-* Examples:
1426	#D-
1427	function IGCM_sys_check_path {
1428	IGCM_debug_PushStack "IGCM_sys_check_path"
1429	if ( $DEBUG_sys ) ; then
1430	echo "IGCM_sys_check_path"
1431	fi
1432
1433	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORK} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCH} ] \|\| [ X${RUN_DIR_PATH} = X${STORE} ] ) ; then
1434	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
1435	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
1436	IGCM_debug_Exit "This will stop the job"
1437	fi
1438	IGCM_debug_PopStack "IGCM_sys_check_path"
1439	}
1440
1441	#D-#==================================================
1442	#D-function IGCM_sys_check_quota
1443	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
1444	#D-* Examples:
1445	#D-
1446	function IGCM_sys_check_quota {
1447	IGCM_debug_PushStack "IGCM_sys_check_quota"
1448	echo "IGCM_sys_check_quota is desactivated on JeanZay because there is actually no way to check scratch quota on this computing center (2019.11.19)"
1449	# if ( $DEBUG_sys ) ; then
1450	# echo "IGCM_sys_check_quota"
1451	# fi
1452	# # Limit of quota (in %)
1453	# limit_quota=90
1454	#
1455	# # Check of the volume
1456	# volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
1457	# volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
1458	#
1459	# if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
1460	#
1461	# unit_avail=${volume_avail: -1}
1462	# unit_quota=${volume_quota: -1}
1463	#
1464	# if [ "${unit_quota}" = "*" ] ; then
1465	# IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
1466	# IGCM_debug_Print 1 "More than 100% of your quota is used"
1467	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1468	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1469	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1470	# IGCM_debug_Verif_Exit
1471	# fi
1472	#
1473	# temp_avail=${volume_avail%%${volume_avail: -1}*}
1474	# temp_quota=${volume_quota%%${volume_quota: -1}*}
1475	#
1476	# if [ ! ${unit_avail} = ${unit_quota} ] ; then
1477	#
1478	# # Convertion
1479	# if [ ${volume_avail: -1} = "T" ] ; then
1480	# (( temp_avail = temp_avail * 1000000000000 ))
1481	# elif [ ${volume_avail: -1} = "G" ] ; then
1482	# (( temp_avail = temp_avail * 1000000000 ))
1483	# elif [ ${volume_avail: -1} = "M" ] ; then
1484	# (( temp_avail = temp_avail * 1000000 ))
1485	# elif [ ${volume_avail: -1} = "k" ] ; then
1486	# (( temp_avail = temp_avail * 1000 ))
1487	# else
1488	# (( temp_avail = volume_avail ))
1489	# fi
1490	# if [ ${volume_quota: -1} = "T" ] ; then
1491	# (( temp_quota = temp_quota * 1000000000000 ))
1492	# elif [ ${volume_quota: -1} = "G" ] ; then
1493	# (( temp_quota = temp_quota * 1000000000 ))
1494	# elif [ ${volume_quota: -1} = "M" ] ; then
1495	# (( temp_quota = temp_quota * 1000000 ))
1496	# elif [ ${volume_quota: -1} = "k" ] ; then
1497	# (( temp_quota = temp_quota * 1000 ))
1498	# else
1499	# (( temp_quota = volume_quota ))
1500	# fi
1501	# fi
1502	#
1503	# quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
1504	## echo "volume ratio is " $quota_volume
1505	#
1506	# if [ ${quota_volume} -ge ${limit_quota} ] ; then
1507	# IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
1508	# IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
1509	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1510	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1511	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1512	# IGCM_debug_Verif_Exit
1513	# fi
1514	#
1515	# fi
1516	#
1517	## Check of the number of inodes
1518	#
1519	# inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
1520	# inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
1521	#
1522	# if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
1523	#
1524	# unit_avail=${inode_avail: -1}
1525	# unit_quota=${inode_quota: -1}
1526	#
1527	# if [ "${unit_quota}" = "*" ] ; then
1528	# IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
1529	# IGCM_debug_Print 1 "More than 100% of your quota is used"
1530	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1531	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1532	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1533	# IGCM_debug_Verif_Exit
1534	# fi
1535	#
1536	# temp_avail=${inode_avail%%${inode_avail: -1}*}
1537	# temp_quota=${inode_quota%%${inode_quota: -1}*}
1538	#
1539	# if [ ! ${unit_avail} = ${unit_quota} ] ; then
1540	#
1541	# # Convertion
1542	# if [ ${inode_avail: -1} = "T" ] ; then
1543	# (( temp_avail = temp_avail * 1000000000000 ))
1544	# elif [ ${inode_avail: -1} = "G" ] ; then
1545	# (( temp_avail = temp_avail * 1000000000 ))
1546	# elif [ ${inode_avail: -1} = "M" ] ; then
1547	# (( temp_avail = temp_avail * 1000000 ))
1548	# elif [ ${inode_avail: -1} = "k" ] ; then
1549	# (( temp_avail = temp_avail * 1000 ))
1550	# else
1551	# (( temp_avail = inode_avail ))
1552	# fi
1553	#
1554	# if [ ${inode_quota: -1} = "T" ] ; then
1555	# (( temp_quota = temp_quota * 1000000000000 ))
1556	# elif [ ${inode_quota: -1} = "G" ] ; then
1557	# (( temp_quota = temp_quota * 1000000000 ))
1558	# elif [ ${inode_quota: -1} = "M" ] ; then
1559	# (( temp_quota = temp_quota * 1000000 ))
1560	# elif [ ${inode_quota: -1} = "k" ] ; then
1561	# (( temp_quota = temp_quota * 1000 ))
1562	# else
1563	# (( temp_quota = inode_quota ))
1564	# fi
1565	# fi
1566	# quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
1567	## echo "inode ratio is " $quota_inode
1568	#
1569	# if [ ${quota_inode} -ge ${limit_quota} ] ; then
1570	# IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
1571	# IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
1572	# IGCM_debug_Print 1 "Use the ccc_quota command to check"
1573	# IGCM_debug_Print 1 "You must have more than 10% available to run"
1574	# IGCM_debug_Exit "Not enough space to run ! STOP HERE"
1575	# IGCM_debug_Verif_Exit
1576	# fi
1577	# fi
1578	IGCM_debug_PopStack "IGCM_sys_check_quota"
1579	}
1580
1581	#D-#==================================================
1582	#D-function IGCM_sys_projectAccounting
1583	#D-* Purpose: store project accounting information in a file
1584	#D-* Examples:
1585	#D-
1586	function IGCM_sys_projectAccounting {
1587	IGCM_debug_PushStack "IGCM_sys_projectAccounting"
1588	if ( $DEBUG_sys ) ; then
1589	echo "IGCM_sys_check_quota"
1590	fi
1591	#need to implemented on Jean Zay
1592	touch $1
1593	# ssh irene191 /usr/bin/ccc_myproject > $1
1594
1595	IGCM_debug_PopStack "IGCM_sys_projectAccounting"
1596	}
1597
1598	#D-#==================================================
1599	#D-function IGCM_sys_getJobSchedulerID
1600	#D-* Purpose: Get the job ID during execution
1601	#D-* Examples: IGCM_sys_getJobSchedulerID jobSchedulerID
1602	#D-
1603	function IGCM_sys_getJobSchedulerID {
1604	IGCM_debug_PushStack "IGCM_sys_getJobSchedulerID"
1605	if ( $DEBUG_sys ) ; then
1606	echo "IGCM_sys_getJobSchedulerID"
1607	fi
1608
1609	eval ${1}=${SLURM_JOBID}
1610
1611	IGCM_debug_PopStack "IGCM_sys_getJobSchedulerID"
1612	}
1613
1614	#D-#==================================================
1615	#D-function IGCM_sys_GetJobID
1616	#D-* Purpose: Get the job ID from the JobName
1617	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
1618	#D-
1619	function IGCM_sys_GetJobID {
1620	IGCM_debug_PushStack "IGCM_sys_GetJobID"
1621	if ( $DEBUG_sys ) ; then
1622	echo "IGCM_sys_GetJobID"
1623	fi
1624
1625	ID=$( squeue -u $user \| grep Job_${config_UserChoices_JobName} \| gawk ' { print $1}' )
1626
1627
1628	eval ${3}=${ID}
1629	IGCM_debug_PopStack "IGCM_sys_GetJobID"
1630	}
1631
1632	#D-#==================================================
1633	#D-function IGCM_sys_CountJobInQueue
1634	#D-* Purpose: Count number of users job
1635	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
1636	#D-
1637	function IGCM_sys_CountJobInQueue {
1638	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
1639	if ( $DEBUG_sys ) ; then
1640	echo "IGCM_sys_CountJobInQueue"
1641	fi
1642
1643	# With -f option, the full job name is given in the last column
1644	NbRun=$(squeue -u $user \| \
1645	grep -v JOBID \| gawk 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
1646
1647	eval ${2}=${NbRun}
1648
1649	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
1650	}
1651
1652	#D-#==================================================
1653	#D-function IGCM_sys_ListJobInQueue
1654	#D-* Purpose: Produce a list of users computing jobs (excluding post-processing)
1655	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
1656	#D-
1657	function IGCM_sys_ListJobInQueue {
1658	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
1659	if ( $DEBUG_sys ) ; then
1660	echo "IGCM_sys_ListJobInQueue"
1661	fi
1662
1663	# to be implemented on Jean Zay
1664	set -A JobList $( squeue -u $user \| gawk '{print $3}' \| grep -v NAME)
1665	# # With -f option, the full job name is given in the last column
1666	# set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
1667	# '( $2 == User && \
1668	# $NF != /TS/ && \
1669	# $NF !~ /PACK/ && \
1670	# $NF !~ /REBUILD/ && \
1671	# $NF !~ /pack/ ) \
1672	# { print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
1673	#
1674	eval set -A ${2} ${JobList[*]}
1675
1676	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
1677	}
1678
1679	#D-#==================================================
1680	#D-function IGCM_sys_atlas
1681	#D-* Purpose: encapsulate atlas call so as to manage error code and curie specificity
1682	#D-* Examples:
1683	#D-
1684	function IGCM_sys_atlas {
1685	IGCM_debug_PushStack "IGCM_sys_atlas" $@
1686	if ( $DEBUG_sys ) ; then
1687	echo "IGCM_sys_atlas :" $@
1688	fi
1689
1690	typeset status
1691	# To be implemented on Jean Zay
1692	#
1693	# \ccc_mprun atlas $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1694	# status=$?
1695	# if [ ${status} -gt 0 ] ; then
1696	# echo "IGCM_sys_atlas : error code ${status}"
1697	# cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1698	# \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1699	# IGCM_debug_PopStack "IGCM_sys_atlas"
1700	# return 1
1701	# else
1702	# IGCM_debug_PopStack "IGCM_sys_atlas"
1703	# return 0
1704	# fi
1705	#
1706	IGCM_debug_PopStack "IGCM_sys_atlas"
1707	}
1708
1709	#D-#==================================================
1710	#D-function IGCM_sys_rebuild_nemo
1711	#D-* Purpose: rebuild nemo parallel files with nemo specific rebuild on curie
1712	#D-* Examples:
1713	#D-
1714
1715	function IGCM_sys_rebuild_nemo {
1716	IGCM_debug_PushStack "IGCM_sys_rebuild_nemo" $@
1717	if ( $DEBUG_sys ) ; then
1718	echo "IGCM_sys_rebuild_nemo :" $@
1719	fi
1720
1721	/gpfswork/rech/psl/commun/Tools/rebuild_nemo/bin/rebuild_nemo ${1} ${2}
1722
1723	IGCM_debug_PopStack "IGCM_sys_rebuild_nemo"
1724	}
1725

Note: See TracBrowser for help on using the repository browser.

Download in other formats: