= Install the modelling framework (NEMO and XIOS) Last edition: '''[[Wikinfo(changed_ts)]]''' by '''[[Wikinfo(changed_by)]]''' == 1. Extract the NEMO code Using your `my_login` /pw registered on this web site: {{{#!sh svn --username "mylogin" co http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE/NEMOGCM }}} == 2. Extract and install XIOS 1 - Install the NetCDF4 library: XIOS needs NetCDF4 (not NetCDF3). In addition, if you want to use the "one_file" mode (see bellow), you will need the hdf/netcdf flibraries properly compiled to allow parallel IO. Note that NetCDF files created by XIOS are not compatible with NetCDF3. The softwares you use to see or analyze NEMO outputs needs to be linked with NetCDF4 and not NetCDF3. 2 - Download XIOS The up to date version of XIOS is XIOS2 (previoulsy XIOS1). XIOS uses xml input files. Those are slightly different from XIOS1 to XIOS2. For the moment, interface for both XIOS1 and XIO2 are available in NEMO, so as the xml files for both. From nemo_v3_6_STABLE and further, use of XIOS2 is now strongly suggested. Download the XIOS2 software (this will create your xios-2.0 directory): cd $HOME svn co -r 819 http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk xios-2.0 ln -s xios-2.0 XIOS Install the XIOS library and executable: Follow the documentation available here: http://forge.ipsl.jussieu.fr/ioserver/wiki/documentation Compile NEMO (see NEMO quickstart guide) using appropriate directives : - NEMO must be compiled with with following cpp keys (see your NEMOCGM/CONFIG/"MY_CONFIG"/cpp*.fcm file: key_iomput and key_mpp_mpi and also (if using XIOS2) key_xios2 - you need to set appropriate PATHS for XIOS libraries in your ARCH/arch-xxxx.fcm file, see for example in NEMOGCM/ARCH, the arch files: arch-macport_osx.fcm, arch-PW6_VARGAS.fcm, arch-X64_CURIE.fcm or NOC Input files for XIOS: You need the following files: domain_def.xml field_def.xml iodef.xml Templates can be found here: NEMOGCM/CONFIG/SHARED for field_def.xml and domain_def.xml see NEMOGCM/CONFIG/ORCA2_LIM for examples of iodef*.xml (Templates for XIOS2 version are different from XIOS1 and can be found here: NEMOGCM/CONFIG/NEMOGCM/CONFIG/GYRE_XIOS for field_def.xml, file_def.xml, domain_def.xml, and example of iodef.xml) Run XIOS in attached mode: XIOS used as a library in NEMO, NEMO is the unique executable This mode has bad performances but is convenient as you have only one executable. It is suitable if you use a relatively low number of cores. Define the attached mode in iodef.xml: false Choose the number of files created: You can create as many files as NEMO cores or produce directly 1 file for all NEMO cores, by choosing the attribute type of the tag "file_definition" in iodef.xml - As many files as NEMO cores : Note that in this case, the files you will get are not located on the subdomains used in the mpi domain decomposition of NEMO. XIOS (even when it is used as a library) creates its own mpi domain decomposition in order to have subdomains containing complete lines (better for IO performances). - A unique file for all NEMO cores: Note that this mode is not available if you compiled XIOS with the option "--netcdf_lib netcdf4_seq" or if the hdf/netcdf flibraries are not properly compiled to allow parallel IO. Check and eventually change the buffer_size The buffer size is defined in iodef.xml according to your domain size, the number of cores you use and the memory available on your machine. Note that you will get a clear error message if this buffer is too small: 10000000 Buffer_size must be bigger than jpi*jpj*jpk*8*3 (with jpi(not jpiglo) and jpj(not jpjglo) NEMO subdomain size) Run NEMO: mpirun -np x ./opa Run XIOS in detached (or server) mode: XIOS used as a library in NEMO and as an external executable. This mode has the best performances but is less convenient as you have to deal with two executables (NEMO and XIOS). Define the mode in iodef.xml: true Choose the number of files created: If you want to create 1 file per XIOS core or produce directly 1 file for all XIOS cores by choosing the attribute type of the tag "file_definition" in iodef.xml - One file per XIOS core : - A unique file for all XIOS cores: Note that this mode is not available if you compiled XIOS with the option "--netcdf_lib netcdf4_seq" or if the hdf/netcdf flibraries are not properly compiled to allow parallel IO. Check and eventually change the buffer_size The buffer size is defined in iodef.xml according to your domain size, the number of cores you use and the memory available on your machine. Note that you will get a clear error message if this buffer is too small: 10000000 Buffer_size must be bigger than jpi*jpj*jpk*8*3 (with jpi(not jpiglo) and jpj(not jpjglo) the subdomain size) Copy xios executable in the directory to run NEMO: cp XIOS/bin/xios_server.exe . Create once for all app.txt file: cat >app.txt -np x ./opa # nemo will run on x cores -np y ./xios_server.exe # xios will run on y cores Run NEMO: mpirun --app app.txt Advanced options: From version 482 of XIOS onwards, several advanced features have been added to the server's capabilities. These include: the ability to perform computations with combinations of output fields; the ability to change the precision of output variables and the ability to add user-defined attributes. The following PDF document contains details and some worked examples: * Documentation: [attachment:wiki:Users/ModelInstall:newxios_functions_zooms.pdf] == 3. Compile and create NEMO executable The main script to compile and create executable is called `makenemo` and located in the `CONFIG` directory. To identify the source code you need, to build the makefile and run it.[[BR]] As an example, compile `GYRE`, with ifort on linux to create a `MY_GYRE` configuration: {{{#!sh cd NEMOGCM/CONFIG; ./makenemo –m ifort_linux –r GYRE -n MY_GYRE }}} === More options * `–n` [CONFIG]: for configuration name, existant or new config to install * `–m` [COMPILER]: compiler choice (use files names in `NEMOGCM/ARCH`, see examples below) * `–h`: help * `–r` [REFCONFIG]: reference configuration as first guess when installing a new one * `–j`[num]: number of processes to compile (`-j 0`:no compilation) * `–d` [DIR]: NEMO subdirectory (not needed) * `–t` [DIR]: temporary directory to store libraries * `-clean` [CONFIG_NAME] : remove the Makefile and all the files created afterwards At the first use, you need the `-m` option to specify compiler, its options and libraries, then for next compilation, it is assumed you will be using the same compiler.[[BR]] If `–n` option is not specified, `ORCA2_LIM` is the default configuration used. `-d` option has been implemented for use in non-interactive shell. The `WORK` directory is created under each configuration directory. === Tools used during the process * `functions.sh` : bash functions used by `makenemo`, for instance to create the `WORK` directory * `cfg.txt` : text list of configurations and source directories * `bld.cfg` : FCM rules to compile === Description of directories * `ARCH` : compilation option files, with format `arch_compiler.fcm`, the compiler name has to be provided with `–m` option * `GYRE`, `ORCA2_LIM` or others : there is now a `cpp.fcm` file containing the list of CPP keys, the `WORK` is specific to each configuration === Examples Compiling `GYRE`, with ifort on linux to create a `MY_GYRE` configuration {{{#!sh makenemo –m ifort_linux –r GYRE -n MY_GYRE }}} Recompile it[[BR]] {{{#!sh makenemo }}} Now, create and compile `ORCA_LIM3` {{{#!sh makenemo –n ORCA_LIM3 (and answer) }}} Now ORCA_LIM3_v2 based on the previous one {{{#!sh makenemo –n ORCA_LIM3_v2 -r ORCA_LIM3 }}} Now, create and compile ORCA2_LIM_2_2, add (`add_key`) and delete (`del_key`) keys, based on `ORCA2_LIM` (just un example) {{{#!sh makenemo –n ORCA2_LIM_2_2 -r ORCA2_LIM del_key "key_iomput" (and answer) }}} To remove a bad configuration {{{#!sh makenemo –n ORCA2_LIM_2_2 clean_config (and answer) }}}