Changes between Version 10 and Version 11 of Doc/CheckDebug

Timestamp:

09/22/14 14:59:16 (10 years ago)

Author:

ncarenton

Comment:

--

Doc/CheckDebug

@@ -274 +274 @@
 }}}
 
+## Start or restart post processing jobs ##
+Please look at the next paragraph.
+
+# Start or restart post processing jobs #
+
+You can run post processing jobs once the main job is finished (for example if the post processing job was deactivated in [#Lespost-traitementsdansconfig.card config.card] or if you encountered a [#Debug bug]).
+
+On TGCC, the machine used for post processing is the same as the computing machine. On IDRIS, the machine to be used for post-processing is adapp (since July 2013) with same file system available : $WORKDIR, $HOME, ... . You can : 
+  1. work directly in the experiment directory (which looks like `PATH_MODIPSL/config/IPSLCM5A/ST11/`). Possible option but not recommended. 
+  1. work in a dedicated directory located in the experiment directory (e.g. `PATH_MODIPSL/config/IPSLCM5A/ST11/POST_REDO`). Best choice. 
+  1. work in a dedicated directory which is independent of the experiment directory (e.g. `$WORKDIR/POST_REDO`).
+
+For the last two options you must first:
+  * If your post processing directory is a subdirectory located in the experiment directory, copy (or make a link `ln -s`) the files and directories `config.card` `POST`, `COMP` and run.card if you want to post process only part of the simulation
+  * If your post processing directory is an independant directory :
+    * create a dedicated directory (for all simulations) ;
+    * transfer libIGCM and run ins_job (for all simulations) ;
+    * create a directory for the simulation to analyze (i.e. for each simulation) :
+{{{
+#!sh
+cd $PATH_MODIPSL/config/IPSLCM5A/ST11
+mkdir -p POST_REDO
+cd POST_REDO/
+cp -pr ../COMP  . 
+cp -pr ../POST  .
+cp -pr ../config.card  .
+cp -pr ../run.card .
+}}}
+
+
+[[NoteBox(warn, Before submitting a post processing job at TGCC (`rebuild_fromWorkdir.job`\, `pack_debug.job`\, `pack_output.job`\, `pack_restart.job`\, `monitoring.job`\, `create_ts.job`\, `create_se.job`) you must make sure that the submission group in present in the job header (#MSUB -A genxxxx). If it isn't\, add it., 600px)]]
+## Restart REBUILD ##
+
+ * Copy the `rebuild_fromWorkdir.job` file to the experiment directory or to the dedicated directory;
+
+ * Edit it:
+{{{
+#!sh
+StandAlone=true
+
+libIGCM=                    # Points to the libIGCM directory of the experiment 
+
+PeriodDateBegin=            # beginning date of the last serie to be "rebuilded" 
+
+NbRebuildDir=               # Number of directories in the series to be "rebuilded" 
+                            # until the PeriodDateBegin 
+
+
+REBUILD_DIR=                # Path for the backup of files waiting to be reconstructed 
+                            # (looking like $SCRATCHDIR/IGCM_OUT/.../JobName/REBUILD or $SCRATCHDIR/TagName/JobName/REBUILD  for version older than libIGCM_v2.0
+                            # if RebuildFromArchive=NONE) 
+
+MASTER=${MASTER:=curie|ada} # Select the computing machine : MASTER=curie for example
+}}}
+
+
+ * Submit the job:
+{{{
+#!sh
+ccc_msub rebuild_fromWorkdir.job                # TGCC
+
+llsubmit rebuild_fromWorkdir.job                # IDRIS
+}}}
+
+
+[[NoteBox(note, The rebuild job submits `pack_output.job` automatically., 600px)]]
+## Restart Pack_output ##
+
+ The pack_output (e.g. in case it was not submitted by the rebuild job):
+
+  * Copy the `libIGCM/pack_output.job` file to the experiment directory or to the dedicated directory;
+  * Edit it : 
+{{{
+#!sh
+libIGCM=${libIGCM:=::modipsl::/libIGCM}         # path of the libIGCM library
+
+MASTER=${MASTER:=curie|ada}                     # machine on which you work
+
+DateBegin=${DateBegin:=20000101}                # start date of the period to be packed
+  
+DateEnd=${DateEnd:=20691231}                    # end date of the period to be packed
+  
+PeriodPack=${PeriodPack:=10Y}                   # pack frequency
+}}}
+  * Submit the job:
+{{{
+#!sh
+ccc_msub pack_output.job                        # TGCC
+
+llsubmit pack_output.job                        # IDRIS
+}}}
+
+[[NoteBox(note, `create_ts.job` and `create_se.job` are submitted automatically., 600px)]]
+
+## Restart Pack_restart or Pack_debug ##
+
+   * Copy the libIGCM/pack_debug.job and libIGCM/pack_restart.job files to the experiment directory or to the dedicated directory;
+   * Edit them : 
+{{{
+#!sh
+libIGCM=${libIGCM:=::modipsl::/libIGCM}         # path of the libIGCM library
+
+MASTER=${MASTER:=curie|ada}                     # machine on which you work
+
+DateBegin=${DateBegin:=20000101}                # start date of the period to be packed
+  
+DateEnd=${DateEnd:=20691231}                    # end date of the period to be packed
+  
+PeriodPack=${PeriodPack:=10Y}                   # pack frequency
+}}}
+   * Submit the two jobs: 
+{{{
+#!sh
+ccc_msub pack_debug.job ; ccc_msub pack_restart.job      # TGCC
+
+llsubmit pack_debug.job ; llsubmit pack_restart.job      # IDRIS
+}}}
+
+
+
+## Restart the Time series ##
+
+[[NoteBox(tip, In case you haven't done it yet\, retrieve `config.card` `COMP` `POST` and eventually `run.card` (post process only part of the simulation) in the `POST_REDO/` directory., 600px)]]
+
+There are two ways:
+
+### !TimeSeries_checker.job - Recommended method ###
+
+   * Copy the `libIGCM/TimeSeries_Checker.job` file to the experiment directory or to the dedicated directory;
+   * Edit it:
+{{{
+#!sh
+libIGCM=${libIGCM:=...MYEXP/modipsl/libIGCM}    # Path of the libIGCM library
+
+SpaceName=${SpaceName:=DEVT}                 
+
+ExperimentName=${ExperimentName:=pdControl}
+
+JobName=${JobName:=MYEXP}
+
+CARD_DIR=${CARD_DIR:=${CURRENT_DIR}}            # Path of the experiment directory 
+                                                # (including CURRENT_DIR if you copied 
+                                                # TimeSeries_Checker.job properly) 
+
+export BRIDGE_MSUB_PROJECT=gen2211              # number of your genci project
+
+}}}
+   * Run the !TimeSeries_Checker.job in interactive mode. It will call the missing create_ts jobs :
+{{{
+#!sh
+./TimeSeries_Checker.job
+}}}
+     or alternatively, in ksh :
+{{{
+#!sh
+./TimeSeries_Checker.job 2>&1 | tee TSC_OUT     # Create log file
+
+grep Batch TSC_OUT                              # find all the submitted jobs
+}}}
+
+
+### Restart create_ts.job ###
+
+   * Copy the `libIGCM/create_ts.job` file to the experiment directory or to the dedicated directory;
+   * Edit it:
+{{{
+#!sh
+StandAlone=true
+
+libIGCM=                                        # Path of the libIGCM library
+
+PeriodDateEnd                                   # end date of the time series to be created
+
+CompletedFlag                                   # end date of the existing time series 
+
+TsTask=2D                                       # select 2D or 3D
+
+RebuildFrequency=true
+}}}
+   * Run the job: 
+{{{
+#!sh
+ccc_msub create_ts.job                          # TGCC
+
+llsubmit create_ts.job                          # IDRIS
+}}}
+
+[[NoteBox(note, If your time series (TS) are 2D and 3D you must run the create_ts jobs twice and change the !TsTask variable accordingly., 600px)]]
+
+## Restarting the seasonal mean calculations ##
+
+[[NoteBox(tip, Transfer `config.card`\, `COMP`\, `POST`\, and `run.card` (post process part of the simulation only) in the `POST_REDO/` directory if you have not done so yet., 600px)]]
+
+There are two methods: 
+
+### SE_Checker.job (recommended method) ###
+
+ * Copy the `libIGCM/SE_Checker.job` file to the experiment directory or to the dedicated directory;
+ * Edit it ;
+ * Run `SE_checker.job` in interactive mode. This will call the create_se jobs: 
+{{{
+#!sh
+./SE_Checker.job
+}}} 
+   or alternatively, in ksh : 
+{{{
+#!sh
+# Create logfile:
+./SE_Checker.job 2>&1 | tee SE_OUT
+# Find all started jobs :
+grep Batch SE_OUT
+}}}
+ 
+### Restart create_se.job ###
+
+ * Copy the `libIGCM/create_se.job` file to the experiment directory or to the dedicated directory;
+ * Edit it:
+{{{
+#!sh
+StandAlone=true
+
+libIGCM=                                        # path of the libIGCM library
+
+PeriodDateEnd=                                  # end date of the decade to be processed
+}}}
+ * Submit the job: 
+{{{
+#!sh
+ccc_msub create_se.job                          # TGCC
+
+llsubmit create_se.job                          # IDRIS
+}}}