New URL for NEMO forge!   http://forge.nemo-ocean.eu

Since March 2022 along with NEMO 4.2 release, the code development moved to a self-hosted GitLab.
This present forge is now archived and remained online for history.
#2622 (on jean-zay @ IDRIS makenemo repeat full compilation even after a succesfull one) – NEMO

Opened 15 months ago

Closed 8 months ago

Last modified 8 months ago

#2622 closed Defect (fixed)

on jean-zay @ IDRIS makenemo repeat full compilation even after a succesfull one

Reported by: molines Owned by: systeam
Priority: low Milestone:
Component: tools Version: v4.0.*
Severity: minor Keywords:
Cc:

Description

Context

makenemo : compilation of the full code is resumed even after a first succesfull compilation and without changing anything (neither F90, nor CPP keys nor arch_xx.fcm ). This behaviour is observed on jean-zay @IDRIS HPC center, and NOT on occigen @ CINES.

Analysis

Not surprisingly the difference between jean-zay and occigen comes from the fcm build statement in makenemo. I notice that the BLD/Makefile is re-created every time on jean-zay, which is not the case on occigen.

I also check the following: after a first successfull compilation with makenemo, if I just go to the BLD directory and use 'make', it is OK, I am told that make: Nothing to be done for 'all'., which is correct.

So the problem is not a Makefile behaviour but in the fcm build procedure

Recommendation

No ideas !

Commit History (2)

ChangesetAuthorTimeChangeLog
15269clem2021-09-17T11:47:48+02:00

4.0-HEAD: change FCM revision number for compilation with JEAN-ZAY. See ticket #2622

15268clem2021-09-17T11:17:05+02:00

commit solution of the defect #2622 found by Jean-Marc for compilation on Jean-Zay machines. Great improvement.

Attachments (1)

ConfigSystem.pm (22.1 KB) - added by molines 14 months ago.

Download all attachments as: .zip

Change History (7)

comment:1 Changed 15 months ago by smasson

I know many people who would be super happy to have a solution... ;-)
How could we figure out if the problem comes from FCM or from Jean-Zay?
Did you contact Jean-Zay support?
Note that up to ~6 months ago, the compilation on Jean-Zay was super fast. It is now super slow independently of the versions of NEMO...

comment:2 Changed 15 months ago by nicolasmartin

As it seems no one has raised a similar issue on other HPCCs, probably a meaningful test would be to change the version of FCM used which is very old (more than 10 years ago).
IncludeSource(/vendors/FCM/README, rev=9621)?
You can't even retrieve this release from the new FCM repository.

A while ago I made a raw test by importing a fresh release of FCM (2017.10.0) into the repository (r10671 & r10672), under my branch I had modified the external reference for FCM accordingly and tried to compile with success.
Unfortunately I cancelled this import last December (r14111) but it is quite straightforward to relaunch a test with the very last version (2019.09.0).

Changed 14 months ago by molines

comment:3 Changed 14 months ago by molines

Hello,

After some time digging in the Perl script of FCM, I found a fix for this problem. To make it short : The problem was that LDFFLAGS and FFLAGS were considered changed between 2 compilations (even though no modifcations done).

So, I found that in fact there was a difference in ConfigSystem.pm and this difference was just a question of leading and trailing white space for those TOOL:: fields. Il implement a 2 lines fix in ConfigSystem.pm (in ext/FCM/lib/Fcm) using the Perl equivalent of TRIM.

324   if (defined($old_lines_ref)) {
325     my %old_val_of
326       = map {($_->label_from_field(1), $_->value())} # converts into a hash
327         grep {$_->label_starts_with($prefix)}        # gets relevant lines
328         @{$old_lines_ref};
329 
330     while (my ($key, $val) = each(%old_val_of)) {
331       if (exists($changed{$key})) {
332         # JMM Eliminate trailing and leading blank before comparison
333         $val=~ s/^\s+|\s+$//g;
334         $new_val_of{$key}=~ s/^\s+|\s+$//g;
335         if ($val eq $new_val_of{$key}) { # no change from old to new
336           delete($changed{$key});
337         }
338       }
339       else { # exists in old but not in new
340         $changed{$key} = undef;
341       }
342     }
343   }

Lines 333 and 334 added. Note that the perl syntax is not that intuitive ...
This works fine for compiling NEMO on Jean-Zay

I attach the modified perl script, I hope it can help some IDRIS users !

Cheers,
Jean-Marc

comment:4 Changed 8 months ago by clem

In 15268:

Error: Failed to load processor CommitTicketReference
No macro or processor named 'CommitTicketReference' found

comment:5 Changed 8 months ago by clem

  • Resolution set to fixed
  • Status changed from new to closed

Thanks Jean-Marc for the solution. It saves me a lot of time.

comment:6 Changed 8 months ago by clem

In 15269:

Error: Failed to load processor CommitTicketReference
No macro or processor named 'CommitTicketReference' found
Note: See TracTickets for help on using tickets.