User guide for cpu-Monitoring Tool and Data Management Tools

This page is still in construction.
Update: November 2020 - Eliott Dupont

1. cpu Consumption Monitoring

The cpu Consumption Monitoring is available for each each computing project either at IDRIS or TGCC on this page.

The data used for the graphs is extracted from logs of ccc_myproject at TGCC and logs of idracct at IDRIS.

The graphs are updated daily, at 8 a.m. for TGCC and at 9 a.m. for IDRIS.

How to make sure the graph is up to date ? To check if the graph is up to date, youo should check the title of the first graph. It displays the date of the last measurment of cpu hours. Therefor if today is the 15th of April the graph is up to date is the title says '2020-04-14'.

If a project has hours allocated to multiple types of processors on the same machine (ex: skylake and knl on IRENE) there will be one line plot and one bar plot per type of processor.

Example for project gen7632 :

1.1. Hover Tool

The hover tool can be disable/enabled by clicking on this icon.

When enable it will allow the display of information about curves when hovering over them on the graphs. Here is what you can learn :

  • How many hours the project is in advance/late regarding the optimal consumption curve;

  • A better estimate of the value/date of a given point of a curve.

Example of hovering over 'Total' curve on top plot.

  • The name of the curve (usefull when multiple subproject displayed on the same graph, ex: gencmip6)

  • For the daily consumption plot, how many hours the project consummed for a given day and how the difference with optimal consumption.

Example of hovering to the daily delay/advance on the bottom plot.

1.2. Zooming in and out

You can use the box zoom option to zomm on a selected area.

The x-axis (time) of the two plots are linked for better visibility.

To zoom out it is easier to use the wheel zoom. It will act differently depending if your mouse pointer is on the graph (the zoom will x-and-y and centered on it) or on an axis (the zoom will be only x XOR Y and centered on your mouse pointer).

Zooming out will allow you to see the blue lines corresponding to the date of the end of the allocation (vertical line on the right hand side) and to the lines corresponding to 100% and 125% of the project allocation. 125% is the theorical maximum of hours one project can consume.

Example of box zoom after zomming out with the wheel zoom. The full timeseries is visible.

1.3. Reset Tool

If you want to go back to the original display of the graph, you can click on the reset button.

1.4. Security area

Bellow the optimal consumption curve is the security area. It represent the area your project consumption should dstay in in order to avoid lateness penalty.

At TGCC, you will get 1 mounth penalty if the project is more than 2 mounths late on the 15th of a given month.

At IDRIS, (about the same, complete)

2. Data Management Tools

Documentation is here

Monitoring :

So far, the STORE/WORK/SCRATCH per-project usage monitoring is available on demand. It provides daily update curves displaying the aggregated volume of data and total number of inodes for a given project as a timeseries. Curves will be available at on the link "Stockage [...] TIMESERIES"

The left column represent the volume of data used by the project for STORE/WORK/SCRATCH. The right column represent the number of inodes of sata used by the project for STORE/WORK/SCRATCH.

The data used to plot these graph is extracted from logs of the output of the command ccc_quota -d (details) on IRENE, for each project monitored. Therefore the data displayed does not take into account inodes and volumes of data stored on on THREDDS.

The quota of inodes and volume is visible by zomming out on the different graphs (horizontal red line).

Cleaning :

The cleaning tool consists in a diagnostic application to help the user assess which of their directories have the most data or takes the most inodes. Thanks to the tool, the user can more easily select what experiments they want to delete and download a path list to delete manually on the computing center. The tool is still under development. A beta version is available on demand. Ask edupont at for it.

Step 1 : Data Acquisition from IRENE

The first step for using the app is to log the output from the ccc_tree command at TGCC. Use the appropriate script to have the log with the proper options.

Scp the log file to CICLAD, on the IGCMG account : /home/igcmg/Chantier_DMP_Eliott/PARSER/parsed_logs

Step 2 : Parse the raw log to get yaml file.

To parse the raw log file, you must use the parser locate on the following path : /home/igcmg/Chantier_DMP_Eliott/PARSER/dmp_tools


You must have the appropriate conda environment set up. To create it run the following commands :

module load python/3.6-anaconda50
conda env create -f $PARSER_DIR/environment.yml --name PARSER_env
source activate PARSER_env
pip install anytree

Once this is done or if you already created the appropriate environment, run :

python $PARSER_env/parser/parser_tgcc/src/ -i <name_of_the_input_file.txt>

If everything went correctly, the output file will be written in $PARSER_DIR/../parsed_logs It must be moved to the appropriate folder to be accessible to the Visualisation App.

If any error were met, first, check you imput file. If the error continues you can send an email get some help / report a bug.

Last modified 9 months ago Last modified on 11/19/20 17:58:38

Attachments (9)

Download all attachments as: .zip