Version 50 (modified by ajornet, 5 years ago) (diff) |
---|
How to compile MICT
Be aware there other ways to compile Orchidee. All of them are valid. But the use of makeorchidee_fcm should make things easier:
- It enables the compilation with multiple cores at the same time.
- No need to modify any file to change the compilation mode (prod/dev/debug).
- Automatically manages all dependencies between files.
modipsl/libIGCM
Despite of having all files inside the modipsl/libIGCM structure. This guide does not guarantee it will work with this framework. For this purpose, go to the appropiate guide.
Libraries
Since MICT V7.2, XIOS 2 is enabled on MICT. This is a new library to manage the output for history files. It is recommended to use over the old one ( IOIPSL). For several reason:
- It is much faster (https://forge.ipsl.jussieu.fr/orchidee/wiki/DevelopmentActivities/Branches/ORCHIDEE-MICT-IMBALANCE-P/SimulationTimes#TrunkvsMICTComparision18022016)
- No need for output history rebuild
- IOIPSL unbalance on Orchidee computation time. All processors except for the last one (always) need to wait. Because it deals with more data (due to sea gridcells).
- IOISPL max number of processors is limited to the number of latitude pixels
- XIOS has no limitation
Set up
This guide assumes modipsl/ligIGCM is already download with the ORCHIDEE_trunk configuration. The following steps describes what is required to change to compile Orchidee-MICT.
There is an implicit step which depends in your architecture (computer). You need to make sure you have chosen the proper libraries (module load ...) to compile Orchidee.
Modules on Irene
- module purge
- module load intel/17.0.4.196
- module load mpi/openmpi/2.0.2
- module load flavor/hdf5/parallel
- module load netcdf-fortran/4.4.4
- module load mkl
- module load hdf5/1.8.20
- module load feature/bridge/heterogenous_mpmd
- module load nco
- module load cdo
- module load ncview
Modules on obelix
- module load intel-fcomp/2018
- module load openmpi/2.1.5
- module load pnetcdf/1.5.0
- module load hdf5/1.10.5
- module load netcdf/4.7
Do NOT use the modules below. There is a bug in one of those libraries. It prevents the writing of the stomate_Cforcing_permafrost.nc under certain cases (e.g: 2nodes + 4 cores each):
- module load netcdf/4p
- module load hdf5/1.8.9p
Note: make sure the python module is NOT loaded. It crashes orchidee.
Apply this patch:
[ajornet@obelix2 XIOS_GOLL]$ svn diff Index: arch/arch-ifort_LSCE.fcm =================================================================== --- arch/arch-ifort_LSCE.fcm (révision 1680) +++ arch/arch-ifort_LSCE.fcm (copie de travail) @@ -2,7 +2,7 @@ %FCOMPILER mpif90 %LINKER mpif90 -nofor-main -%BASE_CFLAGS +%BASE_CFLAGS -std=c++11 %PROD_CFLAGS -O3 -D BOOST_DISABLE_ASSERTS %DEV_CFLAGS -g %DEBUG_CFLAGS -DBZ_DEBUG -g -fno-inline
Modules on curie
- module load intel/16.0.3.210
- module load netcdf/4.3.3.1_hdf5_parallel
- module load hdf5/1.8.9_parallel
IOIPSL
This library takes care multiple Input/Output? tasks:
- Read/write restart files
- Read/write history files
- Read configuration file (run.def)
- Calendar options
After 8.4.0 (rev 4057)
Starting at this version (also included), MICT is using a modification of IOIPSL. Restart subroutines to Read/Write? allows more than 3 dimensions. It allows an increase of performance for the restart files. For such purpose, follow the instructions below in order to install it properly:
1- Go to the modeles folder. All source code is located here.
cd YOURPATH/modipsl/modeles/
2- Remove the current IOIPSL. This one is the IOIPSL trunk version.
rm -rf IOIPSL
3- Download the modified IOISPL-MICT:
3.A- Starting at 8.6.0
svn co -r4850 svn://forge.ipsl.jussieu.fr/orchidee/branches/ORCHIDEE-MICT/IOIPSL IOIPSL3.B- Before 8.6.0
svn co -r4720 svn://forge.ipsl.jussieu.fr/orchidee/branches/ORCHIDEE-MICT/IOIPSL IOIPSL
4- Go to the IOIPSL folder
cd IOIPSL/src
5- Install all makefiles required to compile this library:
../../../util/ins_make
6- Modify the line inside the Makefile (optional)
For Irene:
F_O = -DCPP_PARA -O3 -xHost -align all -align array64byte $(F_D) $(F_P) -I$(MODDIR) -module $(MODDIR) -fp-model precise
For obelix:
F_O = -DCPP_PARA -O3 -align all $(F_D) $(F_P) -I$(MODDIR) -module $(MODDIR) -fp-model precise
For curie/ADA:
F_O = -DCPP_PARA -O3 -xAVX -align all -align array32byte $(F_D) $(F_P) -I$(MODDIR) -module $(MODDIR) -fp-model precise
Before 8.4.0 (rev 4057)
Go to IOIPSL source code folder
cd YOURPATH/modipsl/modeles/IOIPSL/src
Install all makefiles.
../../util/ins_make
Go to the IOIPSL folder
cd ../IOIPSL/src
Modify the files shown below:
Index: histcom.f90 =================================================================== --- histcom.f90 (revision 2705) +++ histcom.f90 (working copy) @@ -73,7 +73,7 @@ !- ! Fixed parameter !- - INTEGER,PARAMETER :: nb_files_max=20,nb_var_max=600, & + INTEGER,PARAMETER :: nb_files_max=20,nb_var_max=1400, & & nb_hax_max=5,nb_zax_max=10,nbopp_max=10 REAL,PARAMETER :: missing_val=nf90_fill_real INTEGER,PARAMETER,PUBLIC :: & Index: restcom.f90 =================================================================== --- restcom.f90 (revision 2705) +++ restcom.f90 (working copy) @@ -39,7 +39,7 @@ ! the out file. !- INTEGER,PARAMETER :: & - & max_var=500, max_file=50, max_dim=NF90_MAX_VAR_DIMS + & max_var=1000, max_file=50, max_dim=NF90_MAX_VAR_DIMS !- CHARACTER(LEN=9),SAVE :: calend_str='unknown' !-
Orchidee-MICT is using lots of variables. So it is necessary to tell IOIPSL increase the maximum allowed.
ORCHIDEE-MICT
In order to compile Orchidee MICT:
Go to the main model folder:
cd YOURPATH/modipsl/modeles/
Remove Orchidee trunk
rm -rf ORCHIDEE
Download MICT from svn (under development)
svn co svn://forge.ipsl.jussieu.fr/orchidee/branches/ORCHIDEE-MICT/ORCHIDEE ORCHIDEE
Compilation
Once modipsl is properly set up for Orchidee-MICT, you can compile the model and its dependencies following the instructions below. For that purpose, you can choose two methods:
- Automatic: the script will do all the steps for you
- Manual: the same as the script but by hand
Automatic
Go to configuration folder for Orchidee offline:
cd YOURPATH/modipsl/config/ORCHIDEE_OL
Modify Makefile to include "-otherexec" from
... # driver : Only compiling ORCHIDEE without linking to XIOS (with fcm method) driver : (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver ; ) # driver_xios : Only compiling ORCHIDEE and linking to XIOS 1.0 (with fcm method) # Note: For linking with XIOS 2.0 change -xios into -xios2 driver_xios : (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -xios2 -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver ; )
to
... # driver : Only compiling ORCHIDEE without linking to XIOS (with fcm method) driver : (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver -otherexec ; ) # driver_xios : Only compiling ORCHIDEE and linking to XIOS 1.0 (with fcm method) # Note: For linking with XIOS 2.0 change -xios into -xios2 driver_xios : (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -xios2 -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver -otherexec ; )
Compile the whole model (IOIPSL, XIOS and Orchidee):
gmake
Manual
Orchidee can also be manually compiled.
IOIPSL
Go to IOIPSL folder
cd modipsl/modeles/IOIPSL/src
Compile IOIPSL
gmake
XIOS
XIOS is a new Input/Output? library aimed to replace IOIPSL at some point. Currently, it only replaces the write of history files. This means IOIPSL is still required and must be always compiled.
Go to XIOS folder:
cd modipsl/modeles/XIOS
Compile XIOS 2
./make_xios --prod --arch SUPPORTED_MACHINE --full --j 8
Orchidee-MICT
Move to Orchidee-MICT
cd modipsl/modeles/ORCHIDEE
Compile Orchidee
./makeorchidee_fcm -parallel mpi -arch SUPPORTED_MACHINE -driver -prod -full -xios2 -j 8 -otherexec
Replace -arch MACHINE with the machine you are working with. Find more information in makeorchidee_fcm -h command.
After that, when you do a modification you can simply run the command line below. So you avoid the whole recompilation from the code:
./makeorchidee_fcm -parallel mpi -arch SUPPORTED_MACHINE -driver -prod -j 8 -xios2 -otherexec
In some cases it might be necessary to recompile from zero.
Debug
If you need to debug use:
./makeorchidee_fcm -parallel mpi -arch SUPPORTED_MACHINE (-xios2) -driver -debug -full -j 8 -otherexec
Notes
- the use of gmake produces an error due to dependencies. For this reason, it is only required to compile orchidee libraries (ioipsl, ...).
- This guide expects you have selected the proper module libraries in the HPC you are working with.
- There is a new option for makeorchidee_fcm called otherexec. It creates forcesoil.exe
Supported machines
Refer to the following keys depending on the machine:
- obelix: ifort_LSCE
- Irene: X64_IRENE
- Curie: X64_CURIE
- ADA: X64_ADA
More machines are available. To discover them:
[ -arch XXX ] : name of the archicture file containg platform dependent compile options. The files arch/arch-XXX.fcm and arch/arch-XXX.path must exist.
Errors
Please, report any error to albert.jornet-puig@…