WikiPrint - from Polar Technologies

Working on Ada

1. IDRIS users' manual

2. Commands to manage jobs on ada

3. Example of a job to start an executable in MPI

Here is an example of a simple job to start an executable orchidee_ol (or gcm.e commented). The input files and the executable must be in the directory before starting the executable. 

#!/bin/ksh
# ######################
# ##   ADA IDRIS   ##
# ######################
# Query's name
# @ job_name = test
# Job type
# @ job_type = parallel
# Standard output file
# @ output = Script_Output_test.$(jobid)
# Error output file (the same)
# @ error = Script_Output_test.$(jobid)
# Number of requested processes
# @ total_tasks = 8
# max. CPU time per MPI process hh:mm:ss
# @ wall_clock_limit = 1:00:00
# Number of task OpenMP/pthreads per MPI process
### @ parallel_threads = 4
# End of header
# @ queue

poe ./orchidee_ol
#poe ./gcm.e

4. Information on Ergon files from Adapp

Ergon files are visible from Adapp. Use $ARCHIVE to reach Ergon files on Adapp. $ARCHIVE is /arch/home/rech/lab/plabxxx on Adapp. All Unix command are available on Adapp to provides information on Ergon files.

5. Job Header for MPI - MPI/OMP with libIGCM

5.1. Forced model

5.1.1. MPI

To launch a job on XXX MPI tasks, you need to use libIGCM/ins_job script. Check your header. It should be : 

#!/bin/ksh
# ######################
# ##   ADA IDRIS   ##
# ######################
# Job name 
# @ job_name = MyJob
# Job type
# @ job_type = parallel
# Standard output file name
# @ output = Script_Output_MyJob.000001
# Error output file name
# @ error = Script_Output_MyJob.000001
# Total number of tasks
# @ total_tasks = XXX
# @ environment = "BATCH_NUM_PROC_TOT=XXX"
# Maximum CPU time per task hh:mm:ss
# @ wall_clock_limit = 1:00:00
# End of the header options
# @ queue

5.1.2. hybrid MPI-OMP

Hybrid version are only available with _v6 configurations

To launch a job on XXX MPI tasks and YYY threads OMP on each task

5.2. Coupled model

5.2.1. MPI

To launch a job on XXX (32) MPI tasks. 5 for NEMO, 1 for oasis and 26 MPI tasks for LMDZ by default for IPSLCM5A, you need to use libIGCM/ins_job script. Check your header. It should be : 

#!/bin/ksh
# ######################
# ##  ADA       IDRIS ##
# ######################
# Job name 
# @ job_name = MyCoupledJob
# Job type
# @ job_type = parallel
# Standard output file name
# @ output = Script_Output_MyCoupledJob.000001
# Error output file name
# @ error = Script_Output_MyCoupledJob.000001
# Total number of tasks
# @ total_tasks = 32
# @ environment = "BATCH_NUM_PROC_TOT=32"
# Maximum CPU time per task hh:mm:ss
# @ wall_clock_limit = 1:00:00
# End of the header options
# @ queue

5.2.2. hybrid MPI-OMP

Hybrid version are only available with _v6 configurations

To launch a job on XXX (47) MPI tasks and YYY (8) threads OMP for LMDZ, ZZZ (180) MPI tasks for NEMO and SSS (1) XIOS servers :

Note : an authorization is required. Please ask assist_at_idris.fr.

Note : Intel environment 2016.2 is forced by libIGCM since libIGCM v2.8 during execution

Note : this is working with all compiler.

6. Specificities libIGCM on Ada

At IDRIS and for Ada, output files are 'packed' using libIGCM_v2, i.e. they are grouped by periods (in general 1 year) using the command tar or ncrcat for NetCDF output files.

This option implies that files must be temporarily stored on the $WORKDIR space, which means that a large storage is needed (at least 20 To).

The diagram below details all jobs including pack_debug, pack_restart and pack_output as well as the directories those jobs are using. Note that the files are temporarily stored in the $WORKDIR/IGCM_OUT directories before being grouped and sent on Ergon in the IGCM_OUT directories.

You will obtain annual output files with 12 monthly values in the Output/MO directory if you put PeriodLength=1M and PackFrequency=1Y in config.card. This is the default grouping period of most configurations but you can of course change it.

What you must remember:

To learn more about this Section, you can read the documentation on Simulation and post-processing and on Monitor, debug and relaunching.

Finally, in case of panic, visit us or send your questions to the list platform-users.

7. Specificities for Adapp

7.1. IDRIS users' manual for adapp

7.2. Header for adapp job

A post-treatment jobs includes these header lines : 

# @ job_type = serial
# @ requirements = (Feature == "prepost")