source: trunk/docs/docs_dev/source/guides/updatedata/updatedata_20130725.rst @ 202

Last change on this file since 202 was 202, checked in by pinsard, 11 years ago

end of 20130725 update data

  • Property svn:keywords set to Id URL
File size: 5.3 KB

Update data 20130725

Here are details tasks for the update data 20130725 derived from :ref:`updatedata`.

?

Only time coverage extension from July 2012 to March 2013

Following files are updated : :file:`${PROJECT}/adm/website/data/changes.rst` :file:`${PROJECT}/adm/website/data_access.rst` :file:`${PROJECT}/adm/website/overview.rst` :file:`${PROJECT}/adm/website/Readme.rst` :file:`${PROJECT}/adm/website/_templates/layout.html`

?????

Save old version

$ datemodif="20130725"
$ export datemodif
$ cd /usr/lodyc/incas/fplod/tropflux_d/
$ mkdir obsolete_${datemodif}
$ mv Readme.rst obsolete_${datemodif}
$ mv ./daily obsolete_${datemodif}
$ mv ./monthly obsolete_${datemodif}

Copy new version (phase 1)

$ mkdir /usr/lodyc/incas/fplod/tropflux_d/to_be_published
$ newdir='/net/adonis/usr/adonis/varclim/tropflux_v2'
$ export newdir

Is there enought space ?

$ du -sh ${newdir}/daily
12G     /net/adonis/usr/adonis/varclim/tropflux_v2/daily
$ du -sh ${newdir}/monthly
396M    /net/adonis/usr/adonis/varclim/tropflux_v2/monthly
$ df -h /usr/lodyc/incas/fplod/tropflux_d/to_be_published
Filesystem            Size  Used Avail Use% Mounted on
fslodyc:/usr/.lodyc   1.6T  1.6T   20G  99% /.autofs/lodyc

20GB vs 12.4 GB = yes

then, copy

$ cp ${PROJECT}/adm/website/Readme.rst /usr/lodyc/incas/fplod/tropflux_d/to_be_published/Readme.rst
$ cp -r ${newdir}/daily /usr/lodyc/incas/fplod/tropflux_d/to_be_published
$ cp -r ${newdir}/monthly /usr/lodyc/incas/fplod/tropflux_d/to_be_published

Compare versions

A set of tools must be run to compare obsolete and new versions

see draft tool :ref:`compare.sh`

?
log=${PROJECT_LOG}/compare.log
script ${log}
compare.sh
exit

Check for results.

First check expected equality for data:

grep "records differ" ${log}

If only "0 of" begin all lines, its ok. Otherwise look inside the log file to locate troubles.

Second, check for differences on global attributes

On creation_date

grep "^>" ${log} | grep creation_date

If nothing appears, you must modified this attributes. see :ref:`change_creation_date.sh`.

?

It is what happened in this 20130725 data update, so:

change_creation_date.sh -d /usr/lodyc/incas/fplod/tropflux_d/to_be_published/ -c 20130725
tlogd.sh change_creation_date

On time_range

grep "^>" ${log} | grep time_range

If nothing appears, you must modified this attributes. see :ref:`change_time_range.sh`

?

It is what happened in this 20130725 data update, so:

change_time_range.sh -d /usr/lodyc/incas/fplod/tropflux_d/to_be_published/ -b 19790101 -e 20130331
tlogd.sh change_time_range

Clean comparison log file:

rm ${log}

Correct terminology if still necessary:

correct_terminology.sh -d /usr/lodyc/incas/fplod/tropflux_d/to_be_published/ -good "1979_2013" -bad "1979_march2013"
tlogd.sh correct_terminology

If modifications has been made, re-run comparison (go back to :ref:`compareversion_20130725`) before next step

?

Once comparison is done and PI happy, ++ cleanning or modify newdir

Copy new version (phase 2)

$ cd /usr/lodyc/incas/fplod/tropflux_d/
$ mv ./to_be_published/Readme.rst .
$ mv ./to_be_published/daily ./
$ mv ./to_be_published/monthly ./
$ ln -sf $(pwd)/Readme.rst daily/
$ ln -sf $(pwd)/Readme.rst monthly/

Remove old version from the website

To avoid confusion, old version has to be removed from the website:

Warning

This sequence do not work on cratos because no ftp command on this plateform. It has been launched on cerbere.locean-ipsl.upmmc.fr

ftp www.locean-ipsl.upmc.fr
cd tropflux/data/
mdelete daily/*
mdelete monthly/*
bye

Installation of the new version

L'installation se fait ainsi:

Warning

This sequence do not work on cratos because no ftp command on this plateform. It has been launched on cerbere.locean-ipsl.upmmc.fr

cd /usr/lodyc/incas/fplod/tropflux_d/
ftp www.locean-ipsl.upmc.fr
cd tropflux/data/
put Readme.rst
cd daily
lcd daily
mput *.nc
put ../Readme.rst
cd ../monthly
lcd ../monthly
put ../Readme.rst
mput *.nc
bye

Update news of website

Files in :file:`${PROJECT}/adm/website/` have been modified to tell the new time coverage so see also :ref:`builddoc` to synchronize dataset and documentation

??

Update svn repository

svn commit pour tous les fichiers modifiés lors de cette mise à jour

Cleanning

rm /usr/lodyc/incas/fplod/tropflux_d/to_be_published/

Note: See TracBrowser for help on using the repository browser.