View Issue Details

IDProjectCategoryView StatusLast Update
0000685OpenFOAMBugpublic2015-02-05 10:26
Reporteruser313Assigned Tohenry  
PriorityhighSeveritymajorReproducibilityalways
Status closedResolutionnot fixable 
PlatformLinux 64bitsOSUbuntu OS Version11
Summary0000685: link problem with new gnu gold
DescriptionError looks like:
/usr/bin/ld: error: --add-needed is not supported but is required for libincompressibleTurbulenceModel.so in /home/danny/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64GccDPOpt/lib/libincompressibleRASModels.so
Make/linux64GccDPOpt/yStar.o:yStar.C:function calcIncompressibleyStar(Foam::fvMesh const&, Foam::Time const&, Foam::GeometricField<Foam::Vector<double>, Foam::fvPatchField, Foam::volMesh> const&, Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh>&): error: undefined reference to 'Foam::incompressible::turbulenceModel::typeName'
Make/linux64GccDPOpt/yStar.o:yStar.C:function calcCompressibleyStar(Foam::fvMesh const&, Foam::Time const&, Foam::GeometricField<Foam::Vector<double>, Foam::fvPatchField, Foam::volMesh> const&, Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh>&): error: undefined reference to 'Foam::compressible::turbulenceModel::typeName'
collect2: ld returned 1 exit status
make: *** [/home/danny/OpenFOAM/OpenFOAM-2.1.0/platforms/linux64GccDPOpt/bin/yStar] Error 1



GNU ld version:
$ ld -v
GNU gold (GNU Binutils for Ubuntu 2.22) 1.11

Uninstall the GNU-gold solves the problem. But even with the old GNU-ld, I have also found that in binutils manual, it says:

--add-needed
--no-add-needed
These two options have been deprecated because of the similarity of their names to the --as-needed and --no-as-needed options. They have been replaced by --copy-dt-needed-entries and --no-copy-dt-needed-entries.

--copy-dt-needed-entries
--no-copy-dt-needed-entries
This option affects the treatment of dynamic libraries referred to by DT_NEEDED tags inside ELF dynamic libraries mentioned on the command line. Normally the linker won't add a DT_NEEDED tag to the output binary for each library mentioned in a DT_NEEDED tag in an input dynamic library. With --copy-dt-needed-entries specified on the command line however any dynamic libraries that follow it will have their DT_NEEDED entries added. The default behaviour can be restored with --no-copy-dt-needed-entries.
This option also has an effect on the resolution of symbols in dynamic libraries. With --copy-dt-needed-entries dynamic libraries mentioned on the command line will be recursively searched, following their DT_NEEDED tags to other libraries, in order to resolve symbols required by the output binary. With the default setting however the searching of dynamic libraries that follow it will stop with the dynamic library itself. No DT_NEEDED links will be traversed to resolve symbols.

  --add-needed Not supported
  --no-add-needed Do not copy DT_NEEDED tags from shared libraries
  --copy-dt-needed-entries Not supported
  --no-copy-dt-needed-entries Do not copy DT_NEEDED tags from shared libraries
TagsNo tags attached.

Activities

user4

2012-11-21 15:34

  ~0001784

Does it work if you replace the --add-needed with --copy-dt-needed-entries?

Do you happen to know from which Ubuntu onwards it supports copy-dt-needed-entries?

wyldckat

2012-11-21 19:58

updater   ~0001785

OK, this is going to be a bit lengthy, so a quick suggestion would be to rely on "WM_CC", "WM_CXX" and "WM_LD" (not yet existent) for defining which binaries to use in the rule files.
(Note: hopefully configurable via "prefs.sh" and inline when sourcing "bashrc" as well, which I don't think it's the current case.)


* On Ubuntu 11.10, "--add-needed" was already being deprecated in favour of "--as-needed" and "--copy-dt-needed-entries": http://manpages.ubuntu.com/manpages/oneiric/man1/ld.1.html

* Ditto on 12.04: http://manpages.ubuntu.com/manpages/precise/man1/ld.1.html

* No manpages available online for 12.10, but the installation I got in a virtual machine indicates that the same options are still available, ld version "2.22.90.20120924"


The problem is that there are two possible versions of LD in Ubuntu (as well as in other distributions, since it's part of binutils), where the default is the "bfd" one and "gold" is the alternative one: http://packages.ubuntu.com/quantal/binutils
Binaries and aliases/links:
* /usr/bin/ld - link to /usr/bin/ld.bfd
* /usr/bin/gold - link to /usr/bin/ld.gold

The "binutils-gold" will switch the link for "ld" to point to the "ld.gold": http://packages.ubuntu.com/quantal/amd64/binutils-gold/filelist


The "gold" one was introduced in 11.10: http://manpages.ubuntu.com/manpages/oneiric/en/man1/gold.1.html
As you can see, the "gold" variant does not support "DT_NEEDED", which is the reason why OpenFOAM doesn't build the way it usually does...
One of the reasons why "gold" would be installed as default is CUDA: https://help.ubuntu.com/community/Cuda


In Fedora they've been looking into replacing the "ld.bfd" for the "ld.gold" for some time now, one of the reasons due to performance when linking:
* http://fedoraproject.org/wiki/Features/GoldLinkerDefault#Benefit_to_Fedora
* http://en.wikipedia.org/wiki/Gold_%28linker%29

wyldckat

2012-11-21 20:08

updater   ~0001786

Last edited: 2012-11-21 20:09

Addendum to the previous comment - the somewhat proper way of choosing "gold" at build time is described here: http://stackoverflow.com/a/8593266

In summary, it's a matter of having a wrapper/glue script that is in an isolated folder (e.g. $HOME/bin/gold) and calling gcc with the additional folder override:
   gcc -B$HOME/bin/gold


Ditto can be done for "ld.bfd", in case "gold" is the default, which adds weight to the suggestion of using "WM_CC" and "WM_CXX" in the rule files.
Which also seems to deprecate the idea of needing for "WM_LD", at least regarding OpenFOAM's build system. I haven't checked what happens when building Scotch and Open-MPI with "gold".

user4

2012-11-22 10:42

  ~0001787

I kind of forgot of what the whole issue was about but the

> the "gold" variant does not support "DT_NEEDED"

does have me worried. The problem is with a library using run-time selection tables with e.g. models or boundary conditions. We might e.g. in another library link to this library so we 'know' these models.

However with all the 'clever' as-needed when we use this top-level library the linker decides we don't actually use anything from our model library and does not link it!

Know of a workaround?

wyldckat

2012-11-22 11:00

updater   ~0001788

> Know of a workaround?

Actually, I know about two workarounds! (if changing "ld" variant is not an option)
When porting OpenFOAM to Windows, both us at blueCAPE and the people at Symscape have had to deal with this problem, because Windows (ironically) doesn't like linking to libraries on which nothing is explicitly needed.


Symscape's solution: add a C dummy function in the libraries that need to be forcefully linked to and then:
1. (the old way) Call said functions in the applications code that need the explicit bind.
2. (the new way) Use the option for the linker for indicating that the function object-reference is missing for linking, forcing the linker to link to the library you already told it to.


blueCAPE's solution: not wanting to babysit libraries and applications, we build things in two (and a half) passes:
- First pass it builds as usual, allowing for things to link as the linker deems necessary.

- An intermediate script is called for comparing the libraries mentioned in "Make/options" with the libraries it actually links to, using "objdump" to get the list (because there is no "ldd" for Windows binaries...).
  This script takes the list of missing libraries and defines a macro with said list of libraries to be loaded at runtime, in the current architecture options file of the respective application.

- On a second pass will build again all of the changed options files, using the injection code that relies on the macro that has the list for loading the libraries.
  The injection code basically acts as if it were loading from the entry "libs" at "system/controlDict".


Symscape's workaround is nested into their patches: http://www.symscape.com/openfoam-2-1-x-on-windows-64-mpi
blueCAPE's workaround is also available online, nested into the respective branch: http://code.google.com/p/bluecfd-singlecore/source/browse/?repo=openfoam21x&name=blueCFD-2.1-1
- you can do a "git diff" between branches "master" and "blueCFD-2.1-1" for the differences.

user4

2012-12-10 16:06

  ~0001815

Thanks guys,

I've added a comment the gnu-gold bug at

http://sourceware.org/bugzilla/show_bug.cgi?id=10238

Feel free to add a bit of weight to this or report a similar bug to your favourite distro.

I guess for now I'll stick with ld.bfd

user313

2012-12-10 16:15

  ~0001816

Haha great, I hope they can do something

henry

2015-02-05 10:25

manager   ~0003691

The GNU Gold linker cannot be used with OpenFOAM as the linker strategy is not compatible with OpenFOAM requirements. Use ld.bfd instead.

Issue History

Date Modified Username Field Change
2012-11-19 20:09 user313 New Issue
2012-11-21 15:34 user4 Note Added: 0001784
2012-11-21 19:58 wyldckat Note Added: 0001785
2012-11-21 20:08 wyldckat Note Added: 0001786
2012-11-21 20:09 wyldckat Note Edited: 0001786
2012-11-22 10:42 user4 Note Added: 0001787
2012-11-22 11:00 wyldckat Note Added: 0001788
2012-12-10 16:06 user4 Note Added: 0001815
2012-12-10 16:15 user313 Note Added: 0001816
2015-02-05 10:25 henry Note Added: 0003691
2015-02-05 10:25 henry Status new => closed
2015-02-05 10:26 henry Assigned To => henry
2015-02-05 10:26 henry Resolution open => not fixable