New URL for NEMO forge!   http://forge.nemo-ocean.eu

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
Changeset 11723 for NEMO/trunk/INSTALL.rst – NEMO

Ignore:
Timestamp:
2019-10-18T14:33:36+02:00 (5 years ago)
Author:
nicolasmartin
Message:

Review of INSTALL.rst

File:
1 edited

Legend:

Unmodified
Added
Removed

  • NEMO/trunk/INSTALL.rst

    r11708 r11723  
    1010 
    1111| The NEMO source code is written in *Fortran 95* and 
    12   some of its prerequisite tools and libraries are already included in the ``./ext`` subdirectory. 
     12  some of its prerequisite tools and libraries are already included in the download. 
    1313| It contains the AGRIF_ preprocessing program ``conv``; the FCM_ build system and 
    1414  the IOIPSL_ library for parts of the output. 
    … …  
    4040   This will limit MPI features to those defined within the MPI-2 standard 
    4141   (but will lose some performance benefits). 
    42  
    43 Specifics for NetCDF and HDF 
    44 ---------------------------- 
    45  
    46 NetCDF and HDF versions from . 
    47 However access to all the options available with the XIOS IO-server will require 
    48 the parallel IO support of these libraries which can be unavailable. 
    49  
    50 | **To satisfy these requirements, it is common to have to compile from source 
    51   in this order HDF (C library) then NetCDF (C and Fortran libraries)** 
    52 | It is also necessary to compile these libraries with the same version of the MPI implementation that 
    53   both NEMO and XIOS (see below) are compiled and linked with. 
    54  
    55 .. hint:: 
    56  
    57    | It is difficult to define the options for the compilation as 
    58      they differ from one architecture to another according to 
    59      the hardware used and the software installed. 
    60    | The following is provided without any warranty 
    61  
    62    .. code-block:: console 
    63  
    64       $ ./configure [--{enable-fortran,disable-shared,enable-parallel}] ... 
    65  
    66    It is recommended to build the tests ``--enable-parallel-tests`` and run them with ``make check`` 
    67  
    68 Particular versions of these libraries may have their own restrictions. 
    69 State the following requirements for netCDF-4 support: 
    70  
    71 .. caution:: 
    72  
    73    | When building NetCDF-C library versions older than 4.4.1, use only HDF5 1.8.x versions. 
    74    | Combining older NetCDF-C versions with newer HDF5 1.10 versions will create superblock 3 files 
    75      that are not readable by lots of older software. 
    76  
    77 Extract and install XIOS 
    78 ======================== 
    79  
    80 With the sole exception of running NEMO in mono-processor mode 
    81 (in which case output options are limited to those supported by the ``IOIPSL`` library), 
    82 diagnostic outputs from NEMO are handled by the third party ``XIOS`` library. 
    83 This can be used in two different modes: 
    84  
    85 - *attached* - Every NEMO process also acts as a XIOS server 
    86 - *detached* - Every NEMO process runs as a XIOS client. 
    87   Output is collected and collated by external, stand-alone XIOS server processors. 
    88  
    89 .. important:: 
    90  
    91    In either case, XIOS needs to be compiled before NEMO, 
    92    since the libraries are needed to successfully create the NEMO executable. 
    93  
    94 Instructions on how to obtain and install the software can be found on the :xios:`XIOS wiki<wiki>`. 
    95  
    96 .. hint:: 
    97  
    98    It is recommended to use XIOS version 2.5. 
    99    This version should be more stable (in terms of future code changes) than the XIOS trunk. 
    100    It is also the version used by the NEMO system team when testing all developments and new releases. 
    101  
    102    This particular version has its own branch and can be checked out and downloaded with: 
    103  
    104    .. code:: console 
    105  
    106       $ svn co https://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/branchs/xios-2.5 
    107  
    108 Download the NEMO source code 
    109 ============================= 
    110  
    111 .. code:: console 
    112  
    113    $ svn co https://forge.ipsl.jussieu.fr/nemo/svn/NEMO/trunk 
    114  
    115 Description of directory tree 
    116 ----------------------------- 
    117  
    118 +-----------+------------------------------------------------------------+ 
    119 | Folder    | Purpose                                                    | 
    120 +===========+============================================================+ 
    121 | ``arch``  | Settings (per architecture-compiler pair)                  | 
    122 +-----------+------------------------------------------------------------+ 
    123 | ``cfgs``  | :doc:`Reference configurations <configurations>`           | 
    124 +-----------+------------------------------------------------------------+ 
    125 | ``doc``   | - ``latex``    : LaTex source code for ref. manuals        | 
    126 |           | - ``namelists``: k start guide                             | 
    127 |           | - ``rst``      : ReST files for quick start guide          | 
    128 +-----------+------------------------------------------------------------+ 
    129 | ``ext``   | Dependencies included (``AGRIF``, ``FCM`` & ``IOIPSL``)    | 
    130 +-----------+------------------------------------------------------------+ 
    131 | ``mk``    | Building  routines                                         | 
    132 +-----------+------------------------------------------------------------+ 
    133 | ``src``   | Modelling routines                                         | 
    134 |           |                                                            | 
    135 |           | - ``ICE``: |NEMO-ICE| for sea ice                          | 
    136 |           | - ``NST``: AGRIF for embedded zooms                        | 
    137 |           | - ``OCE``: |NEMO-OCE| for ocean dynamics                   | 
    138 |           | - ``TOP``: |NEMO-MBG| for tracers                          | 
    139 +-----------+------------------------------------------------------------+ 
    140 | ``tests`` | :doc:`Test cases <test_cases>` (unsupported)               | 
    141 +-----------+------------------------------------------------------------+ 
    142 | ``tools`` | :doc:`Utilities <tools>` to [pre|post]process data         | 
    143 +-----------+------------------------------------------------------------+ 
    144  
    145 Setup your architecture configuration file 
    146 ========================================== 
    147  
    148 All compiler options in NEMO are controlled using files in 
    149 ``./arch/arch-'my_arch'.fcm`` where 'my_arch' is the name of the computing 
    150 architecture.  It is recommended to copy and rename an configuration file from 
    151 an architecture similar to your owns. You will need to set appropriate values 
    152 for all of the variables in the file. In particular the FCM variables: 
    153 ``%NCDF_HOME``; ``%HDF5_HOME`` and ``%XIOS_HOME`` should be set to the 
    154 installation directories used for XIOS installation. 
    155  
    156 .. code-block:: sh 
    157  
    158         %NCDF_HOME           /opt/local 
    159         %HDF5_HOME           /opt/local 
    160         %XIOS_HOME           /Users/$( whoami )/xios-2.5 
    161         %OASIS_HOME          /not/defined 
    162  
    163 Compile and create NEMO executable 
    164 ================================== 
    165  
    166 The main script to compile and create executable is called makenemo and located in the CONFIG directory, it is used to identify the routines you need from the source code, to build the makefile and run it. 
    167 As an example, compile GYRE with 'my_arch' to create a 'MY_GYRE' configuration: 
    168  
    169 .. code-block:: sh 
    170  
    171    ./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE' 
    172  
    173 The image below shows the structure and some content of "MY_CONFIG" directory from the launching of the configuration creation (directories and fundamental files created by makenemo). 
    174  
    175 +------------+----------------------------------------------------+ 
    176 | Folder     | Purpose                                            | 
    177 +============+====================================================+ 
    178 | ``BLD``    |                                                    | 
    179 +------------+----------------------------------------------------+ 
    180 | ``EXP00``  |                                                    | 
    181 +------------+----------------------------------------------------+ 
    182 | ``EXPREF`` |                                                    | 
    183 +------------+----------------------------------------------------+ 
    184 | ``MY_SRC`` |                                                    | 
    185 +------------+----------------------------------------------------+ 
    186 | ``WORK``   |                                                    | 
    187 +------------+----------------------------------------------------+ 
    188  
    189 Folder with the symbolic links to all unpreprocessed routines considered in the configuration 
    190 Compilation folder (executables, headers files, libraries, preprocessed routines, flags, …) 
    191 Computation folder for running the model (namelists, xml, executables and inputs-outputs) 
    192 Folder intended to contain your customised routines (modified from initial ones or new entire routines) 
    193  
    194 After successful execution of makenemo command, the executable called opa is created in the EXP00 directory (in the example above, the executable is created in CONFIG/MY_GYRE/EXP00). 
    195  
    196 More makenemo options 
    197 --------------------- 
    198  
    199 ``makenemo`` has several other options that can control which source files are selected and 
    200 the operation of the build process itself. 
    201 These are:: 
    202  
    203    Optional: 
    204       -d  Set of new sub-components (space separated list from ./src directory) 
    205       -e  Path for alternative patch  location (default: 'MY_SRC' in configuration folder) 
    206       -h  Print this help 
    207       -j  Number of processes to compile (0: no build) 
    208       -n  Name for new configuration 
    209       -s  Path for alternative source location (default: 'src' root directory) 
    210       -t  Path for alternative build  location (default: 'BLD' in configuration folder) 
    211       -v  Level of verbosity ([0-3]) 
    212  
    213 These options can be useful for maintaining several code versions with only minor differences but 
    214 they should be used sparingly. 
    215 Note however the ``-j`` option which should be used more routinely to speed up the build process. 
    216 For example: 
    217  
    218 .. code-block:: sh 
    219  
    220         ./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE' -j 8 
    221  
    222 which will compile up to 8 modules simultaneously. 
    223  
    224  
    225 Default behaviour 
    226 ----------------- 
    227  
    228 At the first use, you need the -m option to specify the architecture 
    229 configuration file (compiler and its options, routines and libraries to 
    230 include), then for next compilation, it is assumed you will be using the 
    231 same compiler.  If the –n option is not specified the last compiled configuration 
    232 will be used. 
    233  
    234 Tools used during the process 
    235 ----------------------------- 
    236  
    237 *   functions.sh : bash functions used by makenemo, for instance to create the WORK directory 
    238 *   cfg.txt : text list of configurations and source directories 
    239 *   bld.cfg : FCM rules to compile 
    240  
    241 Examples 
    242 -------- 
    243  
    244 .. code-block:: sh 
    245  
    246         echo "Example to install a new configuration MY_CONFIG"; 
    247         echo "with OPA_SRC and LIM_SRC_2 "; 
    248         echo "makenemo -n MY_CONFIG -d \"OPA_SRC LIM_SRC_2\""; 
    249         echo ""; 
    250         echo "Available configurations :"; cat ${CONFIG_DIR}/cfg.txt; 
    251         echo ""; 
    252         echo "Available unsupported (external) configurations :"; cat ${CONFIG_DIR}/uspcfg.txt; 
    253         echo ""; 
    254         echo "Example to remove bad configuration "; 
    255         echo "./makenemo -n MY_CONFIG clean_config"; 
    256         echo ""; 
    257         echo "Example to clean "; 
    258         echo "./makenemo clean"; 
    259         echo ""; 
    260         echo "Example to list the available keys of a CONFIG "; 
    261         echo "./makenemo list_key"; 
    262         echo ""; 
    263         echo "Example to add and remove keys"; 
    264         echo "./makenemo add_key \"key_iomput key_mpp_mpi\" del_key \"key_agrif\" "; 
    265         echo ""; 
    266         echo "Example to add and remove keys for a new configuration, and do not compile"; 
    267         echo "./makenemo -n MY_CONFIG -j0 add_key \"key_iomput key_mpp_mpi\" del_key \"key_agrif\" "; 
    268  
    269 Running the model 
    270 ================= 
    271  
    272 Once makenemo has run successfully, the opa executable is available in ``CONFIG/MY_CONFIG/EXP00`` 
    273 For the reference configurations, the EXP00 folder also contains the initial input files (namelists, \*xml files for the IOs…). If the configuration also needs NetCDF input files, this should be downloaded here from the corresponding tar file, see Users/Reference Configurations 
    274  
    275 .. code-block:: sh 
    276  
    277         cd 'MY_CONFIG'/EXP00 
    278         mpirun -n $NPROCS ./opa    # $NPROCS is the number of processes ; mpirun is your MPI wrapper 
    279  
    280  
    281 Viewing and changing list of active CPP keys 
    282 ============================================ 
    283  
    284 For a given configuration (here called MY_CONFIG), the list of active CPP keys can be found in: 
    285  
    286 .. code-block:: sh 
    287  
    288         ./cfgs/'MYCONFIG'/cpp_'MY_CONFIG'.fcm 
    289  
    290  
    291 This text file can be edited to change the list of active CPP keys. Once changed, one needs to recompile opa executable using makenemo command in order for this change to be taken in account. 
    292 Note that most NEMO configurations will need to specify the following CPP keys: 
    293 ``key_iomput`` and ``key_mpp_mpi`` 
    294  
    295 .. Links and substitutions 
    29642 
    29743.. |OpenMPI| replace:: *OpenMPI* 
    … …  
    30046.. _MPICH:   https://www.mpich.org 
    30147.. |NetCDF|  replace:: *Network Common Data Form (NetCDF)* 
    302 .. _NetCDF:  https://www.unidata.ucar.edu/downloads/netcdf 
     48.. _NetCDF:  https://www.unidata.ucar.edu 
    30349.. |HDF|     replace:: *Hierarchical Data Form (HDF)* 
    304 .. _HDF:     https://www.hdfgroup.org/downloads 
     50.. _HDF:     https://www.hdfgroup.org 
     51 
     52Specifics for NetCDF and HDF 
     53---------------------------- 
     54 
     55NetCDF and HDF versions from official repositories may have not been compiled with MPI support. 
     56However access to all the options available with the XIOS IO-server will require 
     57the parallelism of these libraries. 
     58 
     59| **To satisfy these requirements, it is common to have to compile from source 
     60  in this order HDF (C library) then NetCDF (C and Fortran libraries)** 
     61| It is also necessary to compile these libraries with the same version of the MPI implementation that 
     62  both NEMO and XIOS (see below) have been compiled and linked with. 
     63 
     64.. hint:: 
     65 
     66   | It is difficult to define the options for the compilation as 
     67     they differ from one architecture to another according to 
     68     the hardware used and the software installed. 
     69   | The following is provided without any warranty 
     70 
     71   .. code-block:: console 
     72 
     73      $ ./configure [--{enable-fortran,disable-shared,enable-parallel}] ... 
     74 
     75   It is recommended to build the tests ``--enable-parallel-tests`` and run them with ``make check`` 
     76 
     77Particular versions of these libraries may have their own restrictions. 
     78State the following requirements for netCDF-4 support: 
     79 
     80.. caution:: 
     81 
     82   | When building NetCDF-C library versions older than 4.4.1, use only HDF5 1.8.x versions. 
     83   | Combining older NetCDF-C versions with newer HDF5 1.10 versions will create superblock 3 files 
     84     that are not readable by lots of older software. 
     85 
     86Extract and install XIOS 
     87======================== 
     88 
     89With the sole exception of running NEMO in mono-processor mode 
     90(in which case output options are limited to those supported by the ``IOIPSL`` library), 
     91diagnostic outputs from NEMO are handled by the third party ``XIOS`` library. 
     92It can be used in two different modes: 
     93 
     94:*attached*:  Every NEMO process also acts as a XIOS server 
     95:*detached*:  Every NEMO process runs as a XIOS client. 
     96  Output is collected and collated by external, stand-alone XIOS server processors. 
     97 
     98Instructions on how to install XIOS can be found on its :xios:`wiki<>`. 
     99 
     100.. hint:: 
     101 
     102   It is recommended to use XIOS 2.5 release. 
     103   This version should be more stable (in terms of future code changes) than the XIOS trunk. 
     104   It is also the one used by the NEMO system team when testing all developments and new releases. 
     105 
     106   This particular version has its own branch and can be checked out with: 
     107 
     108   .. code:: console 
     109 
     110      $ svn co https://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/branchs/xios-2.5 
     111 
     112Download and install the NEMO code 
     113================================== 
     114 
     115Checkout the NEMO sources 
     116------------------------- 
     117 
     118.. code:: console 
     119 
     120   $ svn co https://forge.ipsl.jussieu.fr/nemo/svn/NEMO/trunk 
     121 
     122Description of 1\ :sup:`st` level tree structure 
     123------------------------------------------------ 
     124 
     125+-----------+----------------------------------------+ 
     126| ``arch``  | Compilation settings                   | 
     127+-----------+----------------------------------------+ 
     128| ``cfgs``  | :doc:`Reference configurations <cfgs>` | 
     129+-----------+----------------------------------------+ 
     130| ``doc``   | :doc:`Documentation <doc>`             | 
     131+-----------+----------------------------------------+ 
     132| ``ext``   | Dependencies included                  | 
     133|           | (``AGRIF``, ``FCM`` & ``IOIPSL``)      | 
     134+-----------+----------------------------------------+ 
     135| ``mk``    | Compilation scripts                    | 
     136+-----------+----------------------------------------+ 
     137| ``src``   | :doc:`Modelling routines <src>`        | 
     138+-----------+----------------------------------------+ 
     139| ``tests`` | :doc:`Test cases <tests>`              | 
     140|           | (unsupported)                          | 
     141+-----------+----------------------------------------+ 
     142| ``tools`` | :doc:`Utilities <tools>`               | 
     143|           | to {pre,post}process data              | 
     144+-----------+----------------------------------------+ 
     145 
     146Setup your architecture configuration file 
     147------------------------------------------ 
     148 
     149All compiler options in NEMO are controlled using files in :file:`./arch/arch-'my_arch'.fcm` where 
     150``my_arch`` is the name of the computing architecture 
     151(generally following the pattern ``HPCC-compiler`` or ``OS-compiler``). 
     152It is recommended to copy and rename an configuration file from an architecture similar to your owns. 
     153You will need to set appropriate values for all of the variables in the file. 
     154In particular the FCM variables: 
     155``%NCDF_HOME``; ``%HDF5_HOME`` and ``%XIOS_HOME`` should be set to 
     156the installation directories used for XIOS installation 
     157 
     158.. code-block:: sh 
     159 
     160   %NCDF_HOME    /usr/local/path/to/netcdf 
     161   %HDF5_HOME    /usr/local/path/to/hdf5 
     162   %XIOS_HOME    /home/$( whoami )/path/to/xios-2.5 
     163   %OASIS_HOME   /home/$( whoami )/path/to/oasis 
     164 
     165Create and compile a new configuration 
     166====================================== 
     167 
     168The main script to {re}compile and create executable is called ``makenemo`` located at 
     169the root of the working copy. 
     170It is used to identify the routines you need from the source code, to build the makefile and run it. 
     171As an example, compile a :file:`MY_GYRE` configuration from GYRE with 'my_arch': 
     172 
     173.. code-block:: sh 
     174 
     175   ./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE' 
     176 
     177Then at the end of the configuration compilation, 
     178:file:`MY_GYRE` directory will have the following structure. 
     179 
     180+------------+----------------------------------------------------------------------------+ 
     181| Directory  | Purpose                                                                    | 
     182+============+============================================================================+ 
     183| ``BLD``    | BuiLD folder: target executable, headers, libs, preprocessed routines, ... | 
     184+------------+----------------------------------------------------------------------------+ 
     185| ``EXP00``  | Run   folder: link to executable, namelists, ``*.xml`` and IOs             | 
     186+------------+----------------------------------------------------------------------------+ 
     187| ``EXPREF`` | Files under version control only for :doc:`official configurations <cfgs>` | 
     188+------------+----------------------------------------------------------------------------+ 
     189| ``MY_SRC`` | New routines or modified copies of NEMO sources                            | 
     190+------------+----------------------------------------------------------------------------+ 
     191| ``WORK``   | Links to all raw routines from :file:`./src` considered                    | 
     192+------------+----------------------------------------------------------------------------+ 
     193 
     194After successful execution of ``makenemo`` command, 
     195the executable called `nemo` is available in the :file:`EXP00` directory 
     196 
     197More ``makenemo`` options 
     198------------------------- 
     199 
     200``makenemo`` has several other options that can control which source files are selected and 
     201the operation of the build process itself. 
     202 
     203.. literalinclude:: ../../../makenemo 
     204   :language: text 
     205   :lines: 119-143 
     206   :caption: Output of ``makenemo -h`` 
     207 
     208These options can be useful for maintaining several code versions with only minor differences but 
     209they should be used sparingly. 
     210Note however the ``-j`` option which should be used more routinely to speed up the build process. 
     211For example: 
     212 
     213.. code-block:: sh 
     214 
     215        ./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE' -j 8 
     216 
     217will compile up to 8 processes simultaneously. 
     218 
     219Default behaviour 
     220----------------- 
     221 
     222At the first use, 
     223you need the ``-m`` option to specify the architecture configuration file 
     224(compiler and its options, routines and libraries to include), 
     225then for next compilation, it is assumed you will be using the same compiler. 
     226If the ``-n`` option is not specified the last compiled configuration will be used. 
     227 
     228Tools used during the process 
     229----------------------------- 
     230 
     231* ``functions.sh`` : bash functions used by ``makenemo``, for instance to create the WORK directory 
     232* ``cfg.txt`` : text list of configurations and source directories 
     233* ``bld.cfg`` : FCM rules to compile 
     234 
     235Examples 
     236-------- 
     237 
     238.. literalinclude:: ../../../makenemo 
     239   :language: text 
     240   :lines: 146-153 
     241 
     242Running the model 
     243================= 
     244 
     245Once ``makenemo`` has run successfully, 
     246the ``nemo`` executable is available in ``./cfgs/MY_CONFIG/EXP00``. 
     247For the reference configurations, the ``EXP00`` folder also contains the initial input files 
     248(namelists, ``*.xml`` files for the IOs, ...). 
     249If the configuration needs other input files, they have to be placed here. 
     250 
     251.. code-block:: sh 
     252 
     253   cd 'MY_CONFIG'/EXP00 
     254   mpirun -n $NPROCS ./nemo   # $NPROCS is the number of processes 
     255                              # mpirun is your MPI wrapper 
     256 
     257Viewing and changing list of active CPP keys 
     258============================================ 
     259 
     260For a given configuration (here called ``MY_CONFIG``), 
     261the list of active CPP keys can be found in :file:`./cfgs/'MYCONFIG'/cpp_MY_CONFIG.fcm` 
     262 
     263This text file can be edited by hand or with ``makenemo`` to change the list of active CPP keys. 
     264Once changed, one needs to recompile ``nemo`` in order for this change to be taken in account. 
     265Note that most NEMO configurations will need to specify the following CPP keys: 
     266``key_iomput`` for IOs and ``key_mpp_mpi`` for parallelism. 
Note: See TracChangeset for help on using the changeset viewer.