rpath ignored when system libraries are matched first

[Antoine Martin]

Related to #1256 and #613.

On some platforms (ie: centos7, suse), the linker command will have -L/usr/lib64 ahead of -L/usr/lib64/xpra and so the cython modules will link against the system libraries and we won't use our private libraries.. leading to warnings and sub-optimal behaviour.

[Antoine Martin]

I already had a fix for this: r14750.
Applied to v1.0.x in r14751, also needed r14743 (+ r14744 fixup) to pass the rpath to the setup.py

But since the potential for regressions is high... I am keeping this ticket open, at least until 1.0.2 is released.

[Antoine Martin]

Found the first problem: the linker doesn't use our version of the libraries because /usr/lib64/xpra contains a libvpx.so.4 but /usr/lib64 has a libvpx.so symlink to libvpx.so.1 and that takes precedence because we link with -lvpx.
If I manually add a symlink so to libvpx.so.4 under /usr/lib64/xpra/libvpx.so then the module does get linked against the version of the library we want, but then it crashes during selftests..

[Antoine Martin]

comment:2 was wrong: the problem on this particular system was that the development package libvpx-xpra-devel was not installed correctly (partially removed?) and it contains the required libvpx.so symlink and the correct library headers. With this properly installed, all the problems go away.
At least all the unit tests now run... except when they are ran from the rpmbuild command, then the codec self test crashes with memory corruption in vpx!

[Antoine Martin]

This seemed too risky for the 1.0 stable branch, so r14907 partially reverts r14751, so the library order will be unchanged for now.
This does mean that we're not necessarily using the libraries we want to be using. But that's better than crashing.

[Antoine Martin]

Likely to cause problems, too late for that. Re-scheduling.

[Antoine Martin]

Also partially reverted in trunk as of r15137

[Antoine Martin]

Despite using --enable-rpath with ffmpeg (r15658 for RPM, r15661 for DEB), and setting xpra's PKG_CONFIG_PATH before calling configure (r15664), we can't re-enable the rpath and undo r14907: the centos builds would then crash during the codecs selftests.. Looks like we still have mismatch somewhere between what we build against and what is used at runtime.

@smo: any ideas?

[Antoine Martin]

Found the cause of the problem: the centos build ends up missing the "library_dirs" from pkgconfig, and so we build with the wrong headers!
Looks like it is ignoring our PKG_CONFIG_PATH environment setting.
Running the build command outside rpmbuild does give the expected result and the resulting installation works as expected.
No idea what is different when running from rpmbuild.

[Antoine Martin]

Fixed (at last) in r15675, backport in r15676.
Was really obvious in the end.