Pretend that you have launched a coupled LMDZOR run for 15 years, since you read on another page that 15 years was the recommended timelength to equilibrate and valid the climate. You are not quite sure what to set the SeasonalFrequency at, though. If you set SeasonalFrequency=5Y, you will produce averages over years 1-5, 6-10, and 11-15. If you set SeasonalFrequency=10Y, you will get averages over years 1-10 and nothing for years 11-15. What you really want is a seasonal average over years 6-15. So you set SeasonalFrequency=5Y and decide you'll figure out the rest later.

Now is later. There is a way to launch ATLAS as a stand-alone job where you can set any period and any length that you wish. This method does both steps of the validation chain: produces the combined files, and launches ATLAS to compute the averages and make the graphs. So you can also use the tips found on other pages to process the combined files afterwards if, say, you want to zoom over a specific region.

Assumptions: Your submit directory (where you submitted the job from) for the LMDZOR coupled run is modipsl/config/LMDZOR_v5.2/MCOUP.

Step 1) Copy the submit directory to a temporary directory.

cp -r modipsl/config/LMDZOR_v5.2/MCOUP modipsl/config/LMDZOR_v5.2/MCOUP.SE

The reason for this is that the ATLAS job will read a bunch of data from the various files in the directory, including the config.card. We are going to fool the script into using different dates by changing the config.card, but we don't want to change the dates in the real run. So we make a copy of the directory.

Now move into this directory and copy the stand-alone libIGCM job file.

cd modipsl/config/LMDZOR_v5.2/MCOUP.SE
cp modipsl/libIGCM/create_se.job .

This file doesn't need to be changed at all. What we are interested in is changing the config.card. Assume, for example, that you launched a 15 year run using the following information in the config.card.

DateBegin=1980-01-01
DateEnd=1994-12-31
TimeSeriesFrequency=5Y
SeasonalFrequency=5Y

Now you want to just average over the last 10 years, 1985-1994. All you have to do is change the following in modipsl/config/LMDZOR_v5.2/MCOUP.SE/config.card

DateBegin=1985-01-01
DateEnd=1994-12-31
TimeSeriesFrequency=10Y
SeasonalFrequency=10Y

Now launch create_se.job as you would normally (ccc_msub create_se.job on Curie, for example). The script will create the merged history file (MCOUP_SE_1985_1994_1M_sechiba_history.nc, for example) and run ATLAS to create all the graphs as usual. Notice that we did NOT change the JobName in the config.card of MCOUP.SE. This is because it uses this JobName to find the history files from the simulation, and we ran the simulation under the MCOUP job.

This technique is also useful for those people who are impatient. Let's say you have the following in the config.card.

DateBegin=1980-01-01
DateEnd=1994-12-31
TimeSeriesFrequency=5Y
SeasonalFrequency=5Y

After four years, though, you want to see results, even while the simulation is still running. You decide you will let the first year be equilibrium and take the averages over the next three years. So you follow the steps above, but this time you change

DateBegin=1981-01-01
DateEnd=1984-12-31
TimeSeriesFrequency=3Y
SeasonalFrequency=3Y

and you launch create_se.job. This will work even if the whole MCOUP simulation is not yet done. As long as year 1984 has finished, you'll get results.

NOTE: It appears that the archive file will only be created for the most recent period. So if, for example, you run from 1981-1990 with a SeasonalFrequency=5Y, only the archive file for 1986-1990 will be created. To obtain the file for 1981-1985, you have to restrict the period to just those years.

NOTE: I got a crash the first time I tried running the script. It was looking for stomate_ipcc_history files and not finding them, since we shut them off in our runs. If this happens, you should make sure that the following line does not appear in COMP/stomate.card in the MCOUP.SE directory.

    (stomate_ipcc_history.nc, ${R_OUT_SBG_O_M}/${PREFIX}_1M_stomate_ipcc_history.nc, Post_1M_stomate_ipcc_history)

With this line, it looks for stomate_ipcc_history.nc, and when it doesn't find it anywhere (because the simulation didn't produce it), it crashes.

Time series jobs, Monitoring, and interMonitoring

Related to the above idea of launching stand-alone ATLAS jobs is the Monitoring function of libIGCM. This feature is particularly useful for helping colleagues who do not currently have access to Irene to check on jobs, as they are displayed on a web browser from anywhere. Much of this information can be found in the Post-processing section of the libIGCM documentation, in particular: ​http://forge.ipsl.jussieu.fr/igcmg_doc/wiki/Doc/CheckDebug#Startorrestartpostprocessingjobs.

In order to use both Monitoring and interMonitoring, you must change your config.card to run in the PROD directory and also request permission to copy files to the THREDDS server. Both tools access this server to display plots, created by the TimeSeries job.

The Monitoring tool (​https://vesg.ipsl.upmc.fr/thredds/catalog/work/USERNAME/catalog.html) allows you to see how far your simulation has progressed without logging into the supercomputer, since it's a website.

The interMonitoring tool (​http://webservices.ipsl.fr/interMonitoring/) is very nice for comparing completed timeseries across different runs that have been placed on the THREDDS server on Irene/Ciclad.

Both the Monitoring and interMonitoring tools rely on the creation of variable TimeSeries files, the creation frequency of which is set in config.card and the output variables set in the various COMP/*card. The primary difference is that the interMonitoring tool allows you to compare multiple simulations, while the Monitoring tool only shows plots for one simulation.

Sometimes, the time series post-processing job does not finish (noticed for r6616, r6642, and others on Irene, global runs). Deleting all the MONITORING files (work/cont003/thredds/.../MONITORING/, work/cont003/gen6328/.../MONITORING) and relaunching the TimeSeries.job (​https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/Doc/CheckDebug#Startorrestartpostprocessingjobs) can make the completed files show up on the monitoring tool (​https://vesg.ipsl.upmc.fr/thredds/catalog/work/USERNAME/catalog.html), but not on the interMonitoring tool. This is reported in ticket #706.