wiki:DocDéveloppeur

Documentation Développeur

Auteurs : Sébastien Denvil, Martial Mancip, Patrick Brockmann

Sommaire général libIGCM

Table des matières

  1. Stratégies de développement
  2. Règles de syntaxe
  3. libIGCM_card
    1. Description du format card
    2. Documentation interne
  4. libIGCM_date
    1. Documentation interne
  5. libIGCM_debug
    1. Documentation interne
  6. libIGCM_sys
    1. Documentation interne
  7. libIGCM_comp
  8. libIGCM_config
  9. libIGCM_post
  10. libIGCM_ensemble

Stratégies de développement

  • Développer le plus clairement possible
  • S'appuyer sur des commandes UNIX avec des options standards
  • Inclure une documentation dans le code 
#==================================================
# The documentation of this file can be automatically generated
# if you use the prefix #D- for comments to be extracted.
# Extract with command: cat lib* | grep "^#D-" | cut -c "4-"
#==================================================
#D-This line will be extracted
#D-This one too
  • Fournir une fonction de test pour chaque librairie. Le test se réalise en faisant la différence entre les résulats courants et un fichier de résultats de référence. 
Check libIGCM_debug ..........................................[ FAILED ]

Indique que la librairie libIGCM_debug n'est pas opérationnelle. 

Check libIGCM_debug ..............................................[ OK ]
Check libIGCM_card ...............................................[ OK ]
Check libIGCM_date ...............................................[ OK ]

Indique que toutes ces librairies sont opérationnelles.

Règles de syntaxe

  • Les noms des variables sont définies sans espace avec des majuscules :
    • ceciestunevariable <-- incorrect
    • CeciEstUneVariable <-- correct

libIGCM_card

Cette librairie se charge de faire la lecture et l'écriture dans les fichiers carte.

Description du format card

Ce paragraphe décrit les éléments du format d'un fichier CARD (inspiré du format INI):

  • Sections: Les sections commencent par [ et se termine par ] comme pour les sections [Messages] et [Recipes] utilisées dans l'exemple ci-dessous.
  • Options: L'option Red = 120 ci-dessous est un exemple d'option (aussi connu sous le nom de clé). Les options commencent par une clé Red, suivie du signe égale '=', et elles se terminent par une valeur 120.
  • Commentaires: Toutes les lignes commençant par un '#' sont considérées comme étant des commentaires, et sont ignorées.
  • Continuation de lignes: Une option peut être écrite sur plusieurs lignes si le dernier caractère de la ligne est un '\'
  • Exemple: 
[Messages]
Option1= "Hello Earth"
Option2= "Hello Mars"

# My comments
[Recipes]
Cake= "file1.doc"
Starter= "file2.doc"

[ColorValues]
Red= 120
Blue= 230
Green= 178

[Couples]
List1=   (up, down),   \
         (humid, dry), \
         (hot, cold),  \
         (far, close)
List2=   (ice, fire, air, water)

Documentation interne

#==================================================================
function IGCM_card_PrintOption
* Purpose: Print an option from a given file.card and section
* Usage: IGCM_card_PrintOption file.card section option
* Only used by IGCM_card_Test.ksh

#==================================================================
function IGCM_card_PrintSection
* Purpose: Print all options from a given file.card and section
* Usage: IGCM_card_PrintSection file.card section
* Only used by IGCM_card_Test.ksh

#==================================================================
function IGCM_card_DefineVariableFromOption
* Purpose: Define a variable from a given file.card, section and option
*          Variable name is automatically defined as file_section_option
* Usage: IGCM_card_DefineVariableFromOption file.card section option

#==================================================================
function IGCM_card_DefineArrayFromOption
* Purpose: Define an array variable from a given file.card, section and option
*          Array variable is automatically defined as file_section_option
* Usage: IGCM_card_DefineArrayFromOption file.card section option

#==================================================================
function IGCM_card_DefineArrayFromSection
* Purpose: Define an array variable from a given file.card and section
*          Array variable is automatically defined as file_section
* Usage: IGCM_card_DefineArrayFromSection file.card section

#==================================================================
function IGCM_card_WriteOption
* Purpose: Write an option in a given file.card and section
* Usage: IGCM_card_WriteOption file.card section newvalue
* Examples: IGCM_card_WriteOption file.card Recipes Red 150
            IGCM_card_WriteOption file.card Messages Option2 '"Hello Mercure"'
            IGCM_card_WriteOption file.card Messages ListVal1 '( 1, 2, 3 )'
            listname="(Sebastien, Martial, Patrick)"
            IGCM_card_WriteOption NewTestFile.card Messages ListVal2 "${listname}"

#==================================================================
function IGCM_card_WriteArrayOption
* Purpose: Write an array option a given file.card and section
* Usage: IGCM_card_WriteArrayOption file.card section option newarray
* Examples: set -A MyArray -- 1 2 3
            IGCM_card_WriteArrayOption file.card Recipes List MyArray

#==================================================================
function IGCM_card_Check
* Purpose: Check the present file by comparison with a reference file
* Usage: IGCM_card_Check

libIGCM_date

Documentation interne

#==================================================================
libIGCM_date
This ksh library handles date calculs and convertions in different calendars.
  types of calendars are possible :

  - leap|gregorian|standard (other name leap) :
      The normal calendar. The time origin for the
      julian day in this case is 24 Nov -4713.
  - noleap|365_day :
      A 365 day year without leap years.
  - all_leap|366_day :
      A 366 day year with only leap years.
  - 360d|360_day :
      Year of 360 days with month of equal length.
#==================================================================
function IGCM_date_DaysInYear
* Purpose: Return the number of days in a year
* Usage: IGCM_date_DaysInYear yyyy
         if there is no argument on the command line,
         then assume that a yyyy is being piped in

#==================================================================
function IGCM_date_DaysInMonth
* Purpose: Calculate the number of days in a month
* Usage: IGCM_date_DaysInMonth yyyy mm
         or IGCM_date_DaysInMonth yyyymmdd
         if there are no command line arguments then
         assume that a yyyymmdd is being piped in and read the value.
         if there is only one argument assume it is a yyyymmdd on the command line
         other wise it is a yyyy and mm on the command line
#==================================================================
function IGCM_date_ConvertGregorianDateToJulian
* Purpose: Convert yyyymmdd to yyyyddd
* Usage: IGCM_date_ConvertGregorianDateToJulian 19980429
         if there is no command line argument, then assume that the date
         is coming in on a pipe and use read to collect it

#==================================================================
function IGCM_date_ConvertJulianDateToGregorian()
* Purpose: Convert yyyyddd to yyyymmdd
* Usage: IGCM_date_ConvertJulianDateToGregorian 1998213
         if there is no command line argument, assume one is being
         piped in and read it

#==================================================================
function IGCM_date_AddDaysToJulianDate
* Purpose: Add days to a yyyyddd formatted date
* Usage: IGCM_date_AddDaysToJulianDate 1998312 { ,-}14
         Read the difference from the command lines
         and the date from the command line, or standard input

#==================================================================
function IGCM_date_AddDaysToGregorianDate
* Purpose: Add days to a yyyymmdd formatted date
* Usage: IGCM_date_AddDaysToGregorianDate 19980312 { ,-}14
         Read the difference from the command lines
         and the date from the command line, or standard input

#==================================================================
function IGCM_date_DaysBetweenJulianDate
* Purpose: Calculate the days difference between two dates and reports
           the number days as jul1 - jul2
* Usage: IGCM_date_DaysBetweenJulianDate jul1 jul2
         where julian date is in the form yyyyddd

#==================================================================
function IGCM_date_DaysBetweenGregorianDate ()
* Purpose: Calculate the days difference between two dates and reports
           the number days as grg1 - grg2
* Usage: IGCM_date_DaysBetweenGregorianDate grg1 grg2
         where gregorian date is in the form yyyymmdd

#==================================================================
function IGCM_date_DaysSinceJC ()
* Purpose: Calculate the days difference between a date and 00010101
* Usage: IGCM_date_DaysSinceJC grg1
         where gregorian date is in the form yyyymmdd

#==================================================================
function IGCM_date_DaysInPreviousPeriod ()
* Purpose: Give the numbers of days during the previous prd1 period from grg1 date # OLD create_ts_begin_date
* Usage: IGCM_date_DaysInPreviousPeriod grg1 prd1 [end]
         where grg1 gregorian date is in the form yyyymmdd
         where prd1 period is in the form N[Yy], N[Mm], N[Dd]. N integer
         where [end] is an optionnal keyword to specify grg1 is the end of prd1

#==================================================================
function IGCM_date_DaysInNextPeriod ()
* Purpose: Give the numbers of days during the next prd1 period from grg1 date # OLD create_ts_next_date
* Usage: IGCM_date_DaysInNextPeriod grg1 prd1
         where grg1 gregorian date is in the form yyyymmdd
         where prd1 period is in the form N[Yy], N[Mm], N[Dd]. N integer

#==================================================================
function IGCM_date_DaysInCurrentPeriod ()
* Purpose: Give the numbers of days during the Current prd1 period from grg1 date
* Usage: IGCM_date_DaysInCurrentPeriod grg1 prd1
         where grg1 gregorian date is in the form yyyymmdd
         where prd1 period is in the form N[Yy], N[Mm], N[Dd]. N integer

#==================================================================
function IGCM_date_Check
 * Purpose: Check the present file by comparison with a reference file

libIGCM_debug

Documentation interne

#==================================================================
function IGCM_debug_CallStack
* Purpose: Echo the Stack

#==================================================================
function IGCM_debug_PushStack
* Purpose: Push a function name in the stack

#==================================================================
function IGCM_debug_PopStack
* Purpose: Pop a function name in the stack

#==================================================================
function IGCM_debug_Exit
* Purpose: Print Call Stack and set ExitFlag to true

#==================================================
function IGCM_debug_Verif_Exit
* Purpose: exit with number 1 if ExitFlag is true

#==================================================
function IGCM_debug_Verif_Exit_Post
* Purpose: exit with number 1 if ExitFlag is true for Post-treatment

#==================================================================
function IGCM_debug_Print
* Purpose: Print arguments according to a level of verbosity.

#==================================================================
function IGCM_debug_PrintVariables
* Purpose: Print arguments when match a pattern
           according to a level of verbosity.
#==================================================================
function IGCM_debug_Check
 * Purpose: Check the present file by comparison with a reference file

libIGCM_sys

Documentation interne

#==================================================
LibIGCM_sys Default host
#==================================================

 This ksh library if a layer under some usefull
environment variables and shell commands.
All those definitions depend on host particularities.
It manages a stack mechanism and test validity of operations.
All function described bellow must be prefixed by IGCM_sys.

#==================================================
Program used in libIGCM
#==================================================
#==================================================
function IGCM_sys_ChangeArchive
* Purpose: Just a dummy call on this machine
* Examples:

#==================================================
function IGCM_sys_RshMaster
* Purpose: Connection to frontend machine.
* Examples:

#==================================================
function IGCM_sys_RshArchive
* Purpose: Archive rsh command
* Examples:

#==================================================
function IGCM_sys_RshPost
* Purpose: Post-process rsh command
* Examples:

#==================================================
function IGCM_sys_SendMail
* Purpose: Send mail when simulation is over
* Examples:

#==================================================
function IGCM_sys_Mkdir
* Purpose: Master locale mkdir command
* Examples:

#==================================================
function IGCM_sys_MkdirArchive
* Purpose: Mkdir on Archive
* Examples:

#==================================================
function IGCM_sys_MkdirWork
* Purpose: Mkdir on Work
* Examples:

#==================================================
function IGCM_sys_Cd
* Purpose: master cd command
* Examples:

#==================================================
function IGCM_sys_Chmod
* Purpose: Chmod
* Examples:

#==================================================
function IGCM_sys_FileSize
* Purpose: Filesize
* Examples:

#==================================================
function IGCM_sys_TestDir
* Purpose: Test Directory that must exists
* Examples:

#==================================================
function IGCM_sys_TestDirArchive
* Purpose: Test Directory that must exists on Archive
* Examples:

#==================================================
function IGCM_sys_IsFileArchived
* Purpose: Test file that must NOT EXISTS on Archive
* Examples:

#==================================================
function IGCM_sys_TestFileArchive
* Purpose: Test file that must NOT EXISTS on Archive
* Examples:

#==================================================
function IGCM_sys_TestFileBuffer
* Purpose: Test file that must NOT EXISTS on Buffer
* Examples:

#==================================================
function IGCM_sys_CountFileArchive
* Purpose: Count files on Archive filesystem
* Examples:

#==================================================
function IGCM_sys_CountFileBuffer
* Purpose: Count files on Scratch filesystem
* Examples:

#==================================================
function IGCM_sys_Tree
* Purpose: Tree directories with files on ${ARCHIVE}
* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}

#==================================================
function IGCM_sys_Tar
* Purpose: master tar command
* Examples:

#==================================================
function IGCM_sys_UnTar
* Purpose: master un-tar command
* Examples:

#==================================================
function IGCM_sys_Qsub
* Purpose: Qsub new job
* Examples:

#==================================================
function IGCM_sys_QsubPost
* Purpose: Qsub new job on scalaire
* Examples:

*************************
 File transfer functions
*************************

#==================================================
function IGCM_sys_Rsync_out
* Purpose: treat return val of rsync
* Examples: IGCM_sys_Rsync_out out_RET_rsync
  Error values and explanations can depend on your system version.
#==================================================
function IGCM_sys_Miror_libIGCM
* Purpose: Mirror libIGCM PATH and lib to frontend
* Examples:

#==================================================
function IGCM_sys_Cp
* Purpose: generic cp
* Examples:

#==================================================
function IGCM_sys_Rm
* Purpose: generic rm
* Examples:

#==================================================
function IGCM_sys_RmRunDir
* Purpose: rm tmpdir (dummy function most of the time batch
                      scheduler will do the job)
* Examples:

#==================================================
function IGCM_sys_Mv
* Purpose: generic move
* Examples:

#==================================================
function IGCM_sys_Put_Dir
* Purpose: Copy a complete directory on $(ARCHIVE)
* Examples:

#==================================================
function IGCM_sys_Get_Dir
* Purpose: Copy a complete directory from ${ARCHIVE}
* Examples:

#==================================================
function IGCM_sys_Get_Master
* Purpose: Copy a complete directory from MASTER filesystem
* Examples:

#==================================================
function IGCM_sys_Put_Rest
* Purpose: Put computied restarts on ${ARCHIVE}.
           File and target directory must exist.
* Examples:

#==================================================
function IGCM_sys_PutBuffer_Rest
* Purpose: Put computied restarts on ${SCRATCHDIR}.
           File and target directory must exist.
* Examples:

#==================================================
function IGCM_sys_PrepareTaredRestart
* Purpose: Prepare tared restart to be access by computing job. Identity here.
* Examples:

#==================================================
function IGCM_sys_Put_Out
* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
* Examples:

#==================================================
function IGCM_sys_PutBuffer_Out
* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
* Examples:

#==================================================
function IGCM_sys_Get
* Purpose: Get a file from ${ARCHIVE}
* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
            IGCM_sys_Get /l Array_contain_myfiles /destpath/
#==================================================
function IGCM_sys_GetBuffer
* Purpose: Get a file from ${SCRATCHDIR}
* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
            IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
#==================================================
function IGCM_sys_GetDate_FichWork
* Purpose: donne la date filesys d un fichier sur le filesystem WORK
* Examples:

#==================================================
function IGCM_sys_GetDate_FichArchive
* Purpose: donne la date filesys d un fichier sur le filesystem ARCHIVE
* Examples:

#==================================================
function IGCM_sys_Dods_Rm
* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
* Examples:

#==================================================
function IGCM_sys_Dods_Cp
* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
* Examples:

#==================================================
function IGCM_sys_Put_Dods
* Purpose: Put ${ARCHIVE} files on DODS internet protocole. Dummy function here
* Examples:

#==================================================
function IGCM_sys_rebuild
* Purpose: rebuild parallel files
* Examples:

#==================================================
function IGCM_sys_rebuild_station
* Purpose: rebuild parallel files describing station 
* Examples:

#==================================================
function IGCM_sys_desactiv_variables
* Purpose: set environement variables prior to execution
* Examples:

 MPI specifications
 Other specifications
#==================================================
function IGCM_sys_desactiv_variables
* Purpose: unset environement variables after execution
* Examples:

 MPI specifications
 Other specifications
#==================================================
function IGCM_sys_build_run_file
* Purpose: build run file (deprecated)
* Examples:

#==================================================
function IGCM_sys_build_execution_scripts
* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
* Examples:

#==================================================
function IGCM_sys_check_quota. Dummy call here
* Purpose: check user quota. Stop the simulation if quota above 90%
* Examples:

#==================================================
function IGCM_sys_CountJobInQueue
* Purpose: Check if job_name is currently
  running or in queue
* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun

#==================================================
function IGCM_sys_ncap2
* Purpose: encapsulate ncap2 call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncatted
* Purpose: encapsulate ncatted call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncbo
* Purpose: encapsulate ncbo call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncdif
* Purpose: encapsulate ncdiff call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncea
* Purpose: encapsulate ncea call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncecat
* Purpose: encapsulate ncecat call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncflint
* Purpose: encapsulate ncflint call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncks
* Purpose: encapsulate ncks call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncpdq
* Purpose: encapsulate ncpdq call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncra
* Purpose: encapsulate ncra call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncrcat
* Purpose: encapsulate ncrcat call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncrename
* Purpose: encapsulate ncrename call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_ncwa
* Purpose: encapsulate ncwa call so as to manage error code and retry
* Examples:

#==================================================
function IGCM_sys_cdo
* Purpose: encapsulate cdo call so as to manage error code and retry
* Examples:

libIGCM_comp

libIGCM_config

libIGCM_post

libIGCM_ensemble

Last modified 11 years ago Last modified on 03/05/13 17:43:39