wiki:News

Very Important News

See also platform-users mailing list archive (for registred users only) here : http://courriel.ipsl.jussieu.fr/mailman/listinfo/platform-users or directly here : http://courriel.ipsl.jussieu.fr/mailman/private/platform-users/


12/07/2016 Switchover dods -> esgf

Dear all,

As the dods software is no longer being maintained by its editor
unidata, the dods servers used by IPSL will be shut down on july the
12th 2016. The data that was reachable this way is now reachable from
the thredds servers of the ESGF nodes.


1/ List of switchovers:

UPMC:
http://dods.ipsl.jussieu.fr ->
https://vesg.ipsl.upmc.fr/thredds/catalog/IPSLFS/catalog.html

IDRIS:
http://dods.idris.fr ->
http://prodn.idris.fr/thredds/catalog/ipsl_public/catalog.html
http://dods.idris.fr:81 ->
http://prodn.idris.fr/thredds/catalog/ipsl_private/catalog.html

TGCC:
http://dods.extra.cea.fr/work ->
http://esgf.extra.cea.fr/thredds/catalog/work/catalog.html
http://dods-prive.extra.cea.fr/work ->
http://esgf.extra.cea.fr/thredds/catalog/work_private/catalog.html
http://dods.extra.cea.fr/store ->
http://esgf.extra.cea.fr/thredds/catalog/store/catalog.html
http://dods-prive.extra.cea.fr/store ->
http://esgf.extra.cea.fr/thredds/catalog/store_private/catalog.html

CINES:
Commissioning of http://esgf.cines.fr/thredds to be done later


2/ List of thredds services

Thredds allows to browse trees via some catalogs as well as to access
files via several protocols whose mapping is as follows:

HTTP -> fileServer
OpenDAP -> dodsC
NetcdfSubset -> ncss
WCS -> wcs

3/ New URLs:

>From now on, access to the files should be done using URLs of the form:
http://{esgf_node_fqdn}/thredds/{service}/{path}

Example, replace these URLs:

HTTP:
http://dods.idris.fr/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc
OpenDAP
http://dods.idris.fr/cgi-bin/nph-dods/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc.html

with these ones:

HTTP:
https://prodn.idris.fr/thredds/fileServer/ipsl_public/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc
OpenDAP:
https://prodn.idris.fr/thredds/dodsC/ipsl_public/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc.html


4/ Access control:

Files put online on 'public' spaces are publicly available.
Files put online on 'private' spaces are available to IPSL labs IP
adresses as well as OpenIDs belonging to the IPSL group.

5/ Term:

New ESGF nodes thredds servers at each site are already available
Old dods servers at each site will be de decommissioned on July the 12th

We remain at your disposal for any further information about the
preparation of this switchover.

Thank you

Nicolas Carenton

19/6/2015 : IGCM - Reorganization of the file tree structure in the repository for shared files

A reorganization is done the 19 of June 2015 in the shared repository for input files IGCM at ada and obelix. It has already been done at ergon and ciclad. curie will be done as soon as possible. The reorganization aims to make it easier to find and store files.

After the reorganization the files are now devised into one directory per model component, the following sub directories exist:

  • IGCM/ATM -> input files for the atmospheric model LMDZ
  • IGCM/SRF -> input files for the land surface model ORCHIDEE. The forcing files for ORCHIDEE offline are found in IGCM/SRF/METEO
  • IGCM/CHM -> input files for the chemistry model INCA
  • IGCM/OCE -> input files for the ocean system NEMO (OPA, LIM, PISCES)
  • IGCM/CPL -> input files for the coupler OASIS
  • IGCM/RESTART -> restart files for some reference simulations. This directory is not changed

In addition the old directories INIT and BC are kept as before but all links are updated to point to the new directories. INIT and BC directories are needed only to keep retro compatibility after the reorganization was done. These directories are used in the comp.card in older configurations. INIT and BC will be removed during 2016 but this will be well announced in advance.

  • IGCM/INIT -> temporary directory, contains links to the above directories
  • IGCM/BC -> temporary directory, contains links to the above directories

Before the reorganization following directories existed in IGCM :

  • IGCM/STORAGE -> contains all the files necessary for the different configurations ;
  • IGCM/INIT -> for each configuration links are done to IGCM/STORAGE/INIT. Some files are also stored directly in here.
  • IGCM/BC -> for each configuration links are done to IGCM/STORAGE/BC. Some files are also stored directly in here.
  • IGCM/RESTART -> restart files for some reference simulations

How it works in the configurations

In the distributed libIGCM configurations, the input files are copied from IGCM/INIT or IGCM/BC direcortries. No files are copied directly from IGCM/STORAGE. As the links in IGCM/INIT and IGCM/BC have been updated, these configurations will continue to work as before. In new versions of the configurations, the paths will be set directly to the new directories. This will be done to more easy understand which file is actually copied. For example in orchidee.card and lmdz.card :

Following paths are working before and after the reorganization :
${R_INIT}/SRF/${config_UserChoices_TagName}/routing.nc
${R_BC}/SRF/${config_UserChoices_TagName}/PFTmap_1850to2005_AR5_LUHa.rc2/PFTmap_IPCC_2000.nc
${R_BC}/ATM/${config_UserChoices_TagName}/${RESOL_ATM}/AR5/HISTORIQUE/aerosols_11YearsClim_1995_v5.nc

They will be changed into:
${R_IN}/SRF/routing.nc
${R_IN}/SRF/PFTmap_1850to2005_AR5_LUHa.rc2/PFTmap_IPCC_2000.nc
${R_IN}/ATM/${RESOL_ATM}/AR5/HISTORIQUE/aerosols_11YearsClim_1995_v5.nc

Note that R_IN=../IGCM depending on the machine, R_BC=${R_IN}/BC and R_INIT=${R_IN}/INIT

Back up
At IDRIS/ergon there is a back up of the full IGCM directory before the reorganization. It is found here :

/linkhome/rech/psl/rpsl035/IGCM_backup_2015-06-15

Read also about IGCM directory here: DocBenvEcommonfiles.

5/6/2015 : Transfert très haut débit TGCC-IDRIS

De: Sébastien Denvil 
Objet: [Platform-users] Transfert très haut débit TGCC-IDRIS
Date: 5 juin 2015 11:47:57 UTC+2

Bonjour,

pour information, il est maintenant possible d'utiliser un réseau à très haut débit (10 Gbps) pour transférer des fichiers entre le TGCC et l'IDRIS. 
On obtient à présent un débit de 150 Mo/s, sans cela le débit maximal était de 35 Mo/s. 
La procédure à suivre pour bénéficier de ce réseau est accessible depuis les liens en bas de ce message.

Le réseau des Centres de Calcul Français (CCFR) est un réseau dédié à très haut débit, qui interconnecte les trois centres de calcul nationaux CINES, IDRIS et TGCC. 
Ce réseau CCFR est mis à la disposition des utilisateurs pour faciliter les transferts de données entre les centres nationaux. 
Sont raccordées sur ce réseau les machines Curie au TGCC, Ada et Adapp à l'IDRIS. Le CINES sera intégré dans les prochains mois.

A l'IDRIS:
http://www.idris.fr/faqs/reseau-ccfr.html

Au TGCC, intranet CEA uniquement:
https://www-tgcc.ccc.cea.fr/fr/tgcc/User_documentation.html

bonne journée.
Sébastien
-- 

4/6/2015 : compiler change on Ada-IDRIS

Dear all,

Because of bad results with the compilation environment intel/2013.1, we still advise everyone to go back to the previous one, intel/2013.0, 
as already stated by Laurent Fairhead on May 5th, 2015.

We will put this environment back in the default IPSL environment on ada.

More precisely, the file ~rpsl035/.bash_login will be modified to point towards the validated environment including intel/2013.0. 
This will take effect on Tuesday, June 9th, 2015, at 2 pm.

If you followed the recommendations (https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocBenvAidris), you won't have anything to do, 
otherwise use ~rpsl035/.bash_login as a model to modify your own .bash_login file.

libIGCM tag v2.6 will also point towards this file. You have to update to this libIGCM tag. 
See https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocLfaq#HowdoIuseadifferentversionoflibIGCM for more information.

Be aware that, even with the environment intel/2013.0, there are still a few errors in some variables with the IPSLCM5A model. 
We don't support it any more on ada. Please use IPSLCM5_v5 instead, since it has been checked.

Have good computations on ada! Don't forget to check your results!

4/6/2015 : Change in compilation and runtime environment on Curie-TGCC

In order to fix problems related to NetCDF writing/reading encountered on scratchdir filesytem on Curie machine, 
we are going to switch compilation and running environment with new NetCDF version netcdf/4.3.3.1_hdf5_parallel, nco/4.4.6 and cdo/1.6.7.

Except for IPSLCM5A model (see below), the change of NetCDF version (switch from 4.2 to 4.3.3.1) has been validated and 
the reproducibility of the results has been checked for all of models and configurations available through "model" command in modipsl.
We strongly advice you to switch with this new compilation and running environment which will be available from June, 9th, 2 pm. 
To do that, here are instructions to follow :

1) Environment 
The ~p86ipsl/.bashrc environment file we recommend you to use will be adapted to use the new environment. 
1.1) If you already use this file, as suggested in the documentation (http://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocBenvBtgcc#HowtoinstallyourenvironmentonTGCC), you don't have to do anything.
1.2) If you use your own configuration file (.bashrc_curie, .bashrc,...), you have to modify it (by adding the load of new modules) : you can see ~p86ipsl/.bashrc as an example.

2) Compilation and running
From June, 9th, 2pm, models and configurations dowloaded thanks to the "model" command of modipsl will be adapted to the new environment.
For models and configurations extracted before this date, you will have to :
- compile (or compile again after a gmake clean) using the new environment (see 1)).
- update your libIGCM version in order to use new tag libIGCM_v2.6. 
Here is information about the update of libIGCM version: http://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocLfaq#HowdoIuseadifferentversionoflibIGCM

Particular case of v6 configurations using XIOS output library : to compile (or compile again) with the new environment, you have to modify following files :
  -  modipsl/modeles/XIOS/arch.env as follows (comment lines ) :
  #module unload netcdf
  #module unload hdf5
  #module load hdf5/1.8.9_parallel
  #module load netcdf/4.2_hdf5_parallel
  - modipsl/modeles/XIOS/arch.path as follows :
  HDF5_INCDIR=""
  HDF5_LIBDIR=""
  HDF5_LIB=""

3) IPSLCM5A : particular case
IPSLCM5A model peforms results which are different from NetCDF 4.2 to 4.3.3.2. If you want to keep exactly the same results, 
don't change anything in your environment and don't update your libIGCM library.

If you have any simulations which are running, you have also the possibility to not change anything and 
to keep the previous environment (beware the automatic change : case 1.1 above) : you have to know there is a risk to obtain corrupted files.

A mail will be sent on June, 9th afternoon once all of these changes will be effective.
 
Arnaud

4/5/2015 : Switchover dods -> esgf

Dear all,

As the dods software is no longer being maintained by its editor 
unidata, the IPSL dods servers will be shut down soon.
The data that was reachable this way is now reachable from the thredds 
servers of the ESGF nodes.


1/ List of switchovers:

IDRIS:
http://dods.idris.fr -> 
http://prodn.idris.fr/thredds/catalog/ipsl_public/catalog.html
http://dods.idris.fr:81 -> 
http://prodn.idris.fr/thredds/catalog/ipsl_private/catalog.html

TGCC:
http://dods.extra.cea.fr/work -> 
http://esgf.extra.cea.fr/thredds/catalog/work/catalog.html
http://dods-prive.extra.cea.fr/work -> 
http://esgf.extra.cea.fr/thredds/catalog/work_private/catalog.html
http://dods.extra.cea.fr/store -> 
http://esgf.extra.cea.fr/thredds/catalog/store/catalog.html
http://dods-prive.extra.cea.fr/store -> 
http://esgf.extra.cea.fr/thredds/catalog/store_private/catalog.html

CINES:
Commissioning of http://esgf.cines.fr/thredds to be done later


2/ List of thredds services

Thredds allows to browse trees via some catalogs as well as to access 
files via several protocols whose mapping is as follows:

HTTP -> fileServer
OpenDAP -> dodsC
NetcdfSubset -> ncss
WCS -> wcs
WMS -> wms
NCML -> ncml
UDDC -> uddc
ISO -> iso


3/ New URLs:

 From now on, access to the files should be done using URLs of the form:
http://{esgf_node_fqdn}/thredds/{service}/{path}

Example, replace these URLs:

HTTP:
http://dods.idris.fr/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc
OpenDAP
http://dods.idris.fr/cgi-bin/nph-dods/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc

with these ones:

HTTP:
https://prodn.idris.fr/thredds/fileServer/ipsl_public/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc
OpenDAP:
https://prodn.idris.fr/thredds/dodsC/ipsl_public/rces452/2L36/Atm/TS/2L36_1m_18600101_20001230_BILS.nc


4/ Access control:

Files put online on 'public' spaces are publicly available.
Files put online on 'private' spaces are available to IPSL IP adresses 
as well as OpenIDs belonging to the IPSL group.

5/ Term:

Old dods servers at each site will be de decommissioned on June 15th


We remain at your disposal for any further information about the 
preparation of this switchover.

Thank you

-- 
Nicolas Carenton

Last modified 10 months ago Last modified on 07/12/16 14:59:10