Opened 4 years ago
Last modified 4 years ago
#2579 new Defect
PGI compiler does allow read values from namelist using buffers
Reported by: | mcastril | Owned by: | systeam |
---|---|---|---|
Priority: | low | Milestone: | Unscheduled |
Component: | DIA | Version: | trunk |
Severity: | minor | Keywords: | GPU, diagnostics, compilation, PGI, |
Cc: |
Description (last modified by nemo)
Context
Compiler arch:
%NCDF_HOME /apps/NETCDF/4.4.1.1/PGI/OPENMPI %HDF5_HOME /apps/HDF5/1.8.20/PGI/OPENMPI %XIOS_HOME /home/bsc99/bsc99214/trunk/xios-2.5.pw9 %NCDF_INC -I%NCDF_HOME/include %NCDF_LIB -L%NCDF_HOME/lib -lnetcdff -lnetcdf -L%HDF5_HOME/lib -lhdf5_hl -lhdf5 -lhdf5 %XIOS_INC -I%XIOS_HOME/inc %XIOS_LIB -L%XIOS_HOME/lib -lxios -lstdc++ -L/apps/GCC/8.1.0/lib/gcc/ppc64le-redhat-linux/lib64/ %OASIS_INC %OASIS_LIB %CPP cpp -P -traditional -Dkey_nosignedzero %FC mpif90 %FCFLAGS -i4 -r8 -g -fast %FFLAGS %FCFLAGS %LD mpif90 %LDFLAGS -i4 -r8 -g -fast %FPPFLAGS -P -Uvector %AR ar %ARFLAGS rs %MK gmake %USER_INC %XIOS_INC %OASIS_INC %NCDF_INC %USER_LIB %XIOS_LIB %OASIS_LIB %NCDF_LIB %CC cc %CFLAGS -O0 -g
The bug activates a warning:
===>>> : W A R N I N G =============== end of record or file while reading namelist nammpp in reference namelist iostat = -1
Analysis
investigating further there is a error in reading values from namelist files:
FIO-F-228/namelist read/internal file/end of file reached without finding group.
Sample code to identify the problem:
integer :: jpni,jpnj CHARACTER(LEN=104) :: buff namelist/nammpp/ jpni,jpnj buff = ' /&nammpp jpni = 10 jpnj = 4 /&namctl /&namsto / EOF ' read(buff,nml=nammpp) print *, jpni end
Recommendation
Example of workaround we implement to manage trunk
integer :: i,j, a, b, pos CHARACTER(LEN=80) :: buff namelist/nammpp/ i,j namelist/namctl/ a,b buff=' / &nammpp i = 10 j = 4 $ / &namctl a = 5 b = 3 / ' pos=index(buff,'&nammpp') read(buff(pos:),nammpp) print *, i, j pos=index(buff,'&namctl') read(buff(pos:),namctl) print *, a,b end
Commit History (0)
(No commits)
Change History (16)
comment:1 Changed 4 years ago by acc
comment:2 Changed 4 years ago by nemo
- Description modified (diff)
comment:3 Changed 4 years ago by nemo
- Description modified (diff)
comment:4 Changed 4 years ago by mcastril
Thank you Andrew, and sorry because I didn't have the other issue in the radar. I told Maicon (our colleague working in the GPUs development) to check that.
Thanks for editing the description too. I tried to modify the ticket's description after I created it but I could modify everything but the description...
comment:5 Changed 4 years ago by mcastril
Hi Andrew,
We tested the proposed script which indeed detected a problem in the namelist_ref namelist. However, after fixing the namelist_ref our problem still persists, so it seems it is unrelated to the slashes
comment:6 Changed 4 years ago by mcastril
This script also shows a failure even without the trailing slash:
[bsc99214@p9login1 ~]$ cat test.f90 integer :: jpni,jpnj,think CHARACTER(LEN=104) :: buff namelist/nammpp/ jpni,jpnj namelist/some/ thing buff = '&some thing = 1 / &nammpp jpni = 10 jpnj = 4 / &namctl / &namsto / EOF ' read(buff,nml=nammpp) print *, jpni end [bsc99214@p9login1 ~]$ pgfortran test.f90 [bsc99214@p9login1 ~]$ ./a.out FIO-F-228/namelist read/internal file/end of file reached without finding group. In source file test.f90, at line number 7 [bsc99214@p9login1 ~]$ gfortran test.f90 [bsc99214@p9login1 ~]$ ./a.out 10
comment:7 Changed 4 years ago by acc
I don't have access to a PGI compiler so could you try this:
integer :: i,j, a, b CHARACTER(LEN=132) :: buff CHARACTER(LEN=1) :: csp namelist/nammpp/ i,j namelist/namctl/ a,b csp = NEW_LINE('A') buff='&nammpp i = 10 j = 4 /'//csp//' &namctl a = 5 b = 3 / ' print *, buff read(buff,nml=nammpp) print *, i,j read(buff,nml=namctl) print *, a,b end
If this works, we can fix this for the PGI compiler by swapping csp definition in LBC/lib_mpp.F90 (load_nml.F90):
!csp = NEW_LINE('A') ! a new line character is the best seperator but some systems (e.g.Cray) ! seem to terminate namelist reads from internal files early if they ! encounter new-lines. Use a single space for safety. csp = ' '
comment:8 Changed 4 years ago by andmirek
It didn't work for me (output):
&nammpp i = 10 j = 4 / &namctl a = 5 b = 3 / 10 4 FIO-F-228/namelist read/internal file/end of file reached without finding group. In source file test3.F90, at line number 13
comment:9 Changed 4 years ago by acc
Ok, There is an issue with some compilers that will need a work-around. The 'big 3' all work and when it doesn't work it is because the behaviour is different between internal and external files. Regardless of the exact wording of the standard, this should not be acceptable. Here is a better test program and results with the compilers I've been able to test. This option has been in the trunk for at least 6 months (possibly longer) so clearly not affecting most users.
integer :: i,j, a, b, pos CHARACTER(LEN=80) :: buff CHARACTER(LEN=1) :: csp namelist/nammpp/ i,j namelist/namctl/ a,b csp=NEW_LINE('A') buff='&nammpp i = 10 j = 4/'//csp//' &namctl a = 5 b = 3 / ' open(unit=15,file='tryme.nl') write(15,*) buff close(15) ! open(unit=15,file='tryme.nl') read(15,nammpp) ; print *, 'nammpp from external file: ',i, j read(15,namctl) ; print *, 'namctl from external file: ',a,b close(15) ! read(buff,nammpp) ; print *, 'nammpp from internal file: ',i, j read(buff,namctl) ; print *, 'namctl from internal file: ',a,b end
---------------------------- gfortran test ---------------------------- gfortran -o test_gf test.f90 ./test_gf nammpp from external file: 10 4 namctl from external file: 5 3 nammpp from internal file: 10 4 namctl from internal file: 5 3 ---------------------------- crayftn test ---------------------------- ftn -o test_cftn test.f90 ./test_cftn nammpp from external file: 10, 4 namctl from external file: 5, 3 nammpp from internal file: 10, 4 namctl from internal file: 5, 3 ---------------------------- ifort test ---------------------------- ifort -o test test.f90 ./test nammpp from external file: 10 4 namctl from external file: 5 3 nammpp from internal file: 10 4 namctl from internal file: 5 3 ---------------------------- allinea test ---------------------------- armflang -o test_arm test.f90 ./test_arm nammpp from external file: 10 4 namctl from external file: 5 3 nammpp from internal file: 10 4 FIO-F-228/namelist read/internal file/end of file reached without finding group. In source file test.f90, at line number 19
&nammpp i = 10 j = 4/ &namctl a = 5 b = 3 /
comment:10 Changed 4 years ago by acc
This might suffice as a workaround. But use with caution because it will edit files in-place; therefore only use it on freshly checked out code with no local modifications AND check the results before using. This hasn't been tested so its more of a proposed work-around at the moment. Please confirm if it works (or can be made to work with a few modifications).
The proposal is this BASH/PERl script:
internal_nl_workaround.pl
INUNITS=( numnam_ref numnam_cfg numnat_ref numnat_cfg numtrc_ref numtrc_cfg numnam_ice_ref numnam_ice_cfg numnamsed_ref numnamsed_cfg numnatp_cfg numnatp_ref ) # # build a list of files that need to be changed # listfile=tmplistfile$$.txt for iunit in ${INUNITS[@]} do grep -l $iunit `find ./ -name '*.[fFh]90'` >> $listfile done allfiles=`cat $listfile | sort -u` echo $allfiles # if [ -f $listfile ] ; then rm $listfile; fi for f in $allfiles do echo "Working on " $f n=0 for n in `seq 0 1 $(( ${#INUNITS[*]} - 1 ))` do numnam=${INUNITS[$n]} perl -ni -e 'unless ( m@.*\s*READ\s*\(\s*'${INUNITS[$n]}'\s*,\s*[a-z,0-9]*.*@) { print } else { $line= $_ ; $line=~s@(.*\s*READ\s*\()(\s*)('${INUNITS[$n]}')(\s*,\s*)([a-z0-9_]*)(.*)@\1\2\3(INDEX(\3,"\5")-1:)\4\5\6@i ; print $line }' $f done done
which can be run in the top directory (on freshly checked out code).
It should locate all the files containing the listed internal files and edit all the READ statements from the likes of:
$ sdiff -s -w 190 ../trunk/src/OCE/nemogcm.F90 src/OCE/nemogcm.F90
Before After
READ ( numnam_ref, namctl, IOSTAT = ios, ERR = 901 ) | READ ( numnam_ref(INDEX(numnam_ref,"namctl")-1:), namctl, IOSTAT = ios, ERR = 901 )
READ ( numnam_cfg, namctl, IOSTAT = ios, ERR = 902 ) | READ ( numnam_cfg(INDEX(numnam_cfg,"namctl")-1:), namctl, IOSTAT = ios, ERR = 902 )
READ ( numnam_ref, namcfg, IOSTAT = ios, ERR = 903 ) | READ ( numnam_ref(INDEX(numnam_ref,"namcfg")-1:), namcfg, IOSTAT = ios, ERR = 903 )
READ ( numnam_cfg, namcfg, IOSTAT = ios, ERR = 904 ) | READ ( numnam_cfg(INDEX(numnam_cfg,"namcfg")-1:), namcfg, IOSTAT = ios, ERR = 904 )
If it works, I'll put together the reverse transformation which will have to be used before commiting any code back to the repository.
This isn't bulletproof; one obvious assumption is that the ampersand is tight with the namelist name (hence the -1 after the index). I think we've been consistent with that but it needs verifying.
comment:11 Changed 4 years ago by acc
I note Maicon has got a guarded but positive response from the nvidia developers of pgfortran:
https://forums.developer.nvidia.com/t/issue-with-namelist-read/156116
The best argument would seem to be that they are already supporting the 'extension' because it is possible to read multiple namelists from a single external file. All we are asking for is that the behaviour is consistent across internal and external files.
Just to complete the list of compilers with the test program from comment 9:
-------------------------------------- pgi compiler -------------------------------------- pgfortran -v Export NVCOMPILER=/lustre/projects/metoffice/pgi-compiler/Linux_x86_64/20.9 pgfortran -o test_pg test.f90 ./test_pg nammpp from external file: 10 4 namctl from external file: 5 3 nammpp from internal file: 10 4 FIO-F-228/namelist read/internal file/end of file reached without finding group. In source file test.f90, at line number 19
comment:12 Changed 4 years ago by maicon
The script compiles fine with Intel and PGI, and only had to add these two modifications to PGI. I was not able to test because the namefiles and inputs are not matching.
-
sbc_phy.F90
old new 1069 1069 REAL(wp), DIMENSION(jpi,jpj), INTENT(in) :: pz0 !: roughness length (for momentum) [m] 1070 1070 !------------------------------------------------------------------- 1071 1071 ! Scalar Re_r relation from Liu et al. 1072 REAL(wp), DIMENSION(8,2) :: dim8x2 1072 1073 REAL(wp), DIMENSION(8,2), PARAMETER :: & 1073 & XA = (/ 0.177, 1.376, 1.026, 1.625, 4.661, 34.904, 1667.19, 5.88e5, & 1074 & 0.292, 1.808, 1.393, 1.956, 4.994, 30.709, 1448.68, 2.98e5 /) 1074 & XA = reshape((/ 0.177, 1.376, 1.026, 1.625, 4.661, 34.904, 1667.19, 5.88e5, & 1075 & 0.292, 1.808, 1.393, 1.956, 4.994, 30.709, 1448.68, 2.98e5 & 1076 & /), shape(dim8x2)) 1075 1077 !! 1076 1078 REAL(wp), DIMENSION(8,2), PARAMETER :: & 1077 & XB = (/ 0., 0.929, -0.599, -1.018, -1.475, -2.067, -2.907, -3.935, & 1078 & 0., 0.826, -0.528, -0.870, -1.297, -1.845, -2.682, -3.616 /) 1079 & XB = reshape((/ 0., 0.929, -0.599, -1.018, -1.475, -2.067, -2.907, -3.935, & 1080 & 0., 0.826, -0.528, -0.870, -1.297, -1.845, -2.682, -3.616 & 1081 & /), shape(dim8x2)) 1079 1082 !! 1080 1083 REAL(wp), DIMENSION(0:8), PARAMETER :: & 1081 1084 & XRAN = (/ 0., 0.11, 0.825, 3.0, 10.0, 30.0, 100., 300., 1000. /)
-
stpctl.F90
25 25 USE lib_mpp ! distributed memory computing 26 26 USE zdf_oce , ONLY : ln_zad_Aimp ! ocean vertical physics variables 27 27 USE wet_dry, ONLY : ll_wd, ssh_ref ! reference depth for negative bathy 28 28 USE, INTRINSIC :: IEEE_ARITHMETIC 29 29 USE netcdf ! NetCDF library 30 30 IMPLICIT NONE 31 31 PRIVATE … … 152 152 & zmax(3) >= 0._wp .OR. & ! negative or zero sea surface salinity 153 153 & zmax(4) >= 100._wp .OR. & ! too large sea surface salinity ( > 100 ) 154 154 & zmax(4) < 0._wp .OR. & ! too large sea surface salinity (keep this line for sea-ice) 155 & I SNAN( zmax(1) + zmax(2) + zmax(3) ) ) ) THEN ! NaN encounter in the tests155 & IEEE_IS_NAN( zmax(1) + zmax(2) + zmax(3) ) ) ) THEN ! NaN encounter in the tests 156 156 IF( lk_mpp .AND. ln_ctl ) THEN 157 157 CALL mpp_maxloc( 'stpctl', ABS(sshn) , ssmask(:,:) , zzz, ih ) 158 158 CALL mpp_maxloc( 'stpctl', ABS(un) , umask (:,:,:), zzz, iu )
comment:13 Changed 4 years ago by maicon
I tested the adjust script with PGI and still there is something wrong in the read of namelists. It works well with Intel compiler, I'm checking what is missing to add to the script.
Runtime error:
===>>> : E R R O R =========== All TOP tracers disabled: change namtrc setting or check if key_top is active
Again, in the same conditions, Intel compilers runs well.
comment:14 Changed 4 years ago by maicon
The issue was because INDEX looks for the first match string, in the namelist_top_cfg, the INDEX(numnat_cfg,"namtrc") goes to namtcr_run. Adding a space solves the issue.
-
trcnam.F90
old new 146 146 IF(lwp) WRITE(numout,*) 'trc_nam_trc : read the passive tracer namelists' 147 147 IF(lwp) WRITE(numout,*) '~~~~~~~~~~~' 148 148 149 READ ( numnat_ref(INDEX(numnat_ref,"namtrc ")-1:), namtrc, IOSTAT = ios, ERR = 901)149 READ ( numnat_ref(INDEX(numnat_ref,"namtrc ")-1:), namtrc, IOSTAT = ios, ERR = 901) 150 150 901 IF( ios /= 0 ) CALL ctl_nam ( ios , 'namtrc in reference namelist' ) 151 READ ( numnat_cfg(INDEX(numnat_cfg,"namtrc ")-1:), namtrc, IOSTAT = ios, ERR = 902 )151 READ ( numnat_cfg(INDEX(numnat_cfg,"namtrc ")-1:), namtrc, IOSTAT = ios, ERR = 902 ) 152 152 902 IF( ios > 0 ) CALL ctl_nam ( ios , 'namtrc in configuration namelist' ) 153 153 IF(lwm) WRITE( numont, namtrc ) 154 154
comment:15 Changed 4 years ago by maicon
The code proposals were tested with SETTE successfully.
comment:16 Changed 4 years ago by mcastril
So as I see it the space mentioned in comment 14 could be added to the perl tool. And the changes mentioned in 12 (sbc_phy.F90 and stpctl.F90) in the code.
Are you sure the problem here isn't just the leading / in your example namelist? A / without a preceding ampersand tends to halt processing on other compilers too; as reported in #2565