source: trunk/libIGCM/libIGCM_ensemble/ENSEMBLE_DOC.txt @ 1008

Last change on this file since 1008 was 1008, checked in by sdipsl, 10 years ago

Documentation for newly integrated ensemble features. Will be move somewhere else somedays.

NL
File size: 6.2 KB
Line 
104/06/2014 - N.LEBAS
2
3**********************************
4ENSEMBLE JOB CREATION
5**********************************
6
7Updates 04/06/2014
8------------------
91) Updates of Ensemble DATE:
10   Fix minor bug: periodic member names start to 0 as for non-periodic
11
12   Create new files "Qclean.month.ENSNAME.sh" and "Qclean.year.ENSNAME.sh" which allow
13   user to use clean_month and clean_year scripts for all members.
14   
15   New option "INITFROM_NONPERIODIC" is avalaible in ensemble.card to use different
16   simulations for restart (use INITFROM option for all if it is not fill).
17   
182) Updates of Ensemble PERTURB:
19   Create new files "Qclean.month.ENSNAME.sh" and "Qclean.year.ENSNAME.sh" which allow
20   user to use clean_month and clean_year scripts for all members.
21   
223) Updates for ins_job:
23   When a new directory is created from config.card, ensemble.card is copy inside if it
24   exists.
25   
26   If -e option is used, ins_job also works when a new directory is created. But it is
27   still only consider the ensemble.card present in the directory where the command was
28   used (it doesn't look for all ensemble.card present in configuration).
29     
30   Update variable name for "ensemble.card" from $FileName to $F_CFG_ENS to be coherent
31   with $F_CFG for 'config.card'.
32 
33
34Updates 11/04/2014
35------------------
361) Updates of libIGCM_ensemble.ksh:
37  New function "IGCM_ensemble_Init" to read the "active" parameter for each
38  ensemble type. Ins_job could then call only functions link to active ensemble.
39 
40  Function "IGCM_ensemble_SetAlpha" fill Alpha and AlphaMonth vectors used to create
41  member name suffix (A,B...). Actually return Alpha= A to Z vector and AlphaMonth=a to l
42  but it could be adapted later to return n suffixes (n = members number).
43 
44  11/04/2014: QsubXXX.txt file become QsubXXX.sh (seems more coherent to it content).
45
462) Updates of Ensemble PERTURB
47  In "ensemble.card" all parameters are in english, so as comments over each param.
48  Read "JobName" in "config.card" directly, so ensemble.card NAME could be different.
49  Creation of symbolic link to DRIVER directory for each member (mandatory with IPSLCM5_v5).
50  Check if directories for each member exist. If this is the case, there is no modification
51  of it (don't touch config.car and Jobxxx inside). Create it else and add command line to
52  Qsub.xxx.txt file.
53 
543) Ensemble DATE
55  Create automatically a set of experiments with defined length and restart file. The
56  descritpion could be defined in a periodic or non-periodic way (or a mix of both).
57  11/04/2014: Update subdir names as ${ENSNAME}${restartnb}A/ and add "_per" suffix for
58  periodic directories.
59   
604) Ensemble PARAMETRIC
61  Test active parameter in ins_job and print "not implemented yet".
62
63
64Usage
65-----
66First of all you need to add a new section called "Ensemble" in config.card - ignored if
67ins_job script is not using for creating an ensemble - as below:
68[Ensemble]
69#D- Ensemble run ? 'y' or 'n'
70#D- If 'y', fill in ensemble.card !!
71EnsembleRun= y
72EnsembleName=
73EnsembleDate=
74EnsembleType=
75
76Note that all empty fields will be filled automatically with the ensemble. All of them
77are only to help you to remind the conditions of the run, they don't change the run of
78the Job.
79
80All parameters for ensemble description are in "ensemble.card" and global template of
81simulations are in the "config.card".
82
83Check that you get COMP, POST, PARAM and DRIVER (for IPSLCM5_v5) directories in experiment
84folder.
85Once "ensemble.card" and "config.card" are correctly written,  to create an ensemble
86simply type:
87
88       ins_job -e
89
90This will create all the directories of the ensemble and Qsub.xxx.sh, a shell file
91containing all command to submit all jobs (PeriodNb=60 for all simulations).
92
93The Qclean.month/year.xxx.sh are bash files to use clean_month or clean_year script for
94all simulations.
95
96NOTE: If a directory exists, ins_job won't modify it. If only some directories of
97the ensemble are presents, it will create the missing one and complete Qsub.xxx.sh
98shell file.
99
100NOTE: If you put ensemble.card and config.card directly in [...]/modipsl/config/IPSLCM5_v5
101folder, it will create a new directory from config.card "JobName" value. Then copy the 2
102card files and generate the ensemble structure.
103
104Configuration
105-------------
106
1071) Peturb ensembles
108   To be done by S. Nguyen.
109
1102) Date restart ensembles
111   
112   In ensemble.card all configuration of this ensemble are in [Ens_DATE] section.
113   Available options for this section are:
114           active*: 'y' for using date ensemble, 'n' else.
115           NAME*: name of the ensemble (used to create root directory)
116           LENGTH*: default length of the simulation for non periodic and duration
117                   for all periodic
118           INITFROM*: Restart name to find all restart files (also use as prefix for
119                     all simulations)
120           INITPATH*: Restart global directory 
121   
122           BEGIN_INIT(1): start date of the first periodic simulation
123           END_INIT(1): start date of the last periodic simulation
124           PERIODICITY(1): duration between the start of 2 periodic simulations
125           BEGIN_RESTART(1): date of the first restart file (next files will be
126                             determined adding periodicity length conform to
127                             config.card calendar).
128           
129           NONPERIODIC(2): list of start dates for all simulations
130           RESTART_NONPERIODIC(2): list of corresponding restart dates
131           INITFROM_NONPERIODIC(2): simulation name to restart for each simulation.
132                                    IF empty all simulations will use INITFROM one.
133           LENGTH_NONPERIODIC: lenght of each simulation. IF empty all simulations
134                               duration will be the default LENGTH option.
135                   
136   All parameters with * must be filled.
137   All parameters with (1) must be filled for periodic description are empty else.
138   All parameters with (2) must be filled for non periodic description are empty else.
139   
140   WARNING: For list variables, use space between values (no coma).
141   
142   ins_job -e will create a "NAME" root ensemble directory containing all periodic and
143   non periodics simulations subdir. Subdir names, such as Jobs, are build by concatening
144   <NAME><RESTART_NB>A (ex: ensNino0A, ensNino1A...) for non periodic and add suffix
145   "_per" for periodic (ex: ensNino0A_per).
146   
1473) Parametric ensembles
148  Not implemented yet.
149
150
151
Note: See TracBrowser for help on using the repository browser.