source: trunk/docs/docs_dev/source/guides/updatedata/updatedata_20130725.rst @ 202

.. +
..
.. DESCRIPTION
.. ===========
..
.. procedure to update data on website 20130725
..
.. SEE ALSO
.. ========
..
.. ref:`updatedata`
..
.. TODO
.. =====
..
.. EVOLUTIONS
.. ==========
..
.. $Id$
..
.. $URL$
..
.. - fplod 20130725T141747Z cratos.locean-ipsl.upmc.fr (Linux)
..
..   * creation
..
.. -

.. |project| replace:: TROPFLUX
.. |project_id| replace:: tropflux

.. _updatedata_20130725:

Update data 20130725
====================

Here are details tasks for the update data 20130725 derived from :ref:`updatedata`.

.. _newversion_20130725:

Only time coverage extension from July 2012 to March 2013

Following files are updated :
:file:`${PROJECT}/adm/website/data/changes.rst`
:file:`${PROJECT}/adm/website/data_access.rst`
:file:`${PROJECT}/adm/website/overview.rst`
:file:`${PROJECT}/adm/website/Readme.rst` 
:file:`${PROJECT}/adm/website/_templates/layout.html`

.. _saveold:

Save old version
----------------

.. parsed-literal::

   $ datemodif="20130725"

   $ export datemodif
   $ cd /usr/lodyc/incas/fplod/tropflux_d/
   $ mkdir obsolete_${datemodif}

.. parsed-literal::

   $ mv Readme.rst obsolete_${datemodif}
   $ mv ./daily obsolete_${datemodif}
   $ mv ./monthly obsolete_${datemodif}

.. _copynew1_20130725:

Copy new version (phase 1)
--------------------------

.. parsed-literal::

   $ mkdir /usr/lodyc/incas/fplod/tropflux_d/to_be_published
   $ newdir='/net/adonis/usr/adonis/varclim/tropflux_v2'
   $ export newdir

Is there enought space ?

::

   $ du -sh ${newdir}/daily
   12G     /net/adonis/usr/adonis/varclim/tropflux_v2/daily
   $ du -sh ${newdir}/monthly
   396M    /net/adonis/usr/adonis/varclim/tropflux_v2/monthly
   $ df -h /usr/lodyc/incas/fplod/tropflux_d/to_be_published
   Filesystem            Size  Used Avail Use% Mounted on
   fslodyc:/usr/.lodyc   1.6T  1.6T   20G  99% /.autofs/lodyc

20GB vs 12.4 GB = yes

then, copy ::

   $ cp ${PROJECT}/adm/website/Readme.rst /usr/lodyc/incas/fplod/tropflux_d/to_be_published/Readme.rst
   $ cp -r ${newdir}/daily /usr/lodyc/incas/fplod/tropflux_d/to_be_published
   $ cp -r ${newdir}/monthly /usr/lodyc/incas/fplod/tropflux_d/to_be_published

.. _compareversion_20130725:

Compare versions
----------------

A set of tools must be run to compare obsolete and new versions


see draft tool :ref:`compare.sh`

::

   log=${PROJECT_LOG}/compare.log
   script ${log}
   compare.sh
   exit

Check for results.

First check expected equality for data:

::

   grep "records differ" ${log}

If only "0 of" begin all lines, its ok. Otherwise look inside the log file to
locate troubles.

Second, check for differences on global attributes

On creation_date

::

   grep "^>" ${log} | grep creation_date

If nothing appears, you must modified this attributes. see :ref:`change_creation_date.sh`.

It is what happened in this 20130725 data update, so:

::

   change_creation_date.sh -d /usr/lodyc/incas/fplod/tropflux_d/to_be_published/ -c 20130725
   tlogd.sh change_creation_date


On time_range
::

   grep "^>" ${log} | grep time_range

If nothing appears, you must modified this attributes. see :ref:`change_time_range.sh`

It is what happened in this 20130725 data update, so::

   change_time_range.sh -d /usr/lodyc/incas/fplod/tropflux_d/to_be_published/ -b 19790101 -e 20130331
   tlogd.sh change_time_range

Clean comparison log file::

   rm ${log}

Correct terminology if still necessary:

::

  correct_terminology.sh -d /usr/lodyc/incas/fplod/tropflux_d/to_be_published/ -good "1979_2013" -bad "1979_march2013"
  tlogd.sh correct_terminology

If modifications has been made, re-run comparison (go back to
:ref:`compareversion_20130725`)
before next step

Once comparison is done and PI happy,
++ cleanning or modify newdir

.. _copynew2_20130725:

Copy new version (phase 2)
--------------------------

.. parsed-literal::

   $ cd /usr/lodyc/incas/fplod/tropflux_d/
   $ mv ./to_be_published/Readme.rst .
   $ mv ./to_be_published/daily ./
   $ mv ./to_be_published/monthly ./
   $ ln -sf $(pwd)/Readme.rst daily/
   $ ln -sf $(pwd)/Readme.rst monthly/


.. _desinstalldata_20130725:

Remove old version from the website
-----------------------------------

To avoid confusion, old version has to be removed from the website:

.. warning::

   This sequence do not work  on cratos because no ftp command on this plateform.
   It has been launched on cerbere.locean-ipsl.upmmc.fr

.. parsed-literal::

   ftp www.locean-ipsl.upmc.fr
   cd |project_id|/data/
   mdelete daily/*
   mdelete monthly/*
   bye

.. _installdata_20130725:

Installation of the new version
-------------------------------

L'installation se fait ainsi:

.. warning::

   This sequence do not work  on cratos because no ftp command on this plateform.
   It has been launched on cerbere.locean-ipsl.upmmc.fr

.. parsed-literal::

   cd /usr/lodyc/incas/fplod/tropflux_d/
   ftp www.locean-ipsl.upmc.fr
   cd |project_id|/data/
   put Readme.rst
   cd daily
   lcd daily
   mput \*.nc
   put ../Readme.rst
   cd ../monthly
   lcd ../monthly
   put ../Readme.rst
   mput \*.nc
   bye


Update news of website
----------------------


Files in :file:`${PROJECT}/adm/website/` have been modified to tell the new time coverage so 
see also :ref:`builddoc` to synchronize dataset and documentation

Update svn repository
---------------------

svn commit pour tous les fichiers modifiés lors de cette mise à jour

Cleanning
---------

rm /usr/lodyc/incas/fplod/tropflux_d/to_be_published/