Re: s6-svscan can't find shared libraries

From: Jens Rehsack <>
Date: Wed, 4 Dec 2019 07:56:50 +0100

> Am 04.12.2019 um 06:30 schrieb J. Lewis Muir <>:
> On 12/03, Laurent Bercot wrote:
>>> /opt/local/bin/s6-svscan: error while loading shared libraries: cannot open shared object file: No such file or directory
>>> Is this because the '-rpath' linker option is not used at compile time
>>> to add the library directory to the runtime library search path?
>> You could say that - or, more precisely, it's because your system isn't
>> configured to search for shared libraries in /opt/local/lib. It is on
>> purpose that s6 (or other software) does not automatically
>> add rpaths at build time: rpaths prevent run-time relocations, which can
>> be useful in a number of situations. Adding a rpath is hardcoding policy,
>> and by default, software should not hardcode policy.
> Interesting. Up until now, my understanding of best practice was that
> setting rpath was the right thing to do. But I see from your comments
> that this is not necessary true. Thank you for your explanation!

This is typically decided by policy. A number of attacks was possible,
because of bad managed (way to relaxed) relative path's in RPATH.
It's reasonable that you decide for your system the appropriate
approach and set the RPATH based on that.

>> There is no reason for you not to want to modify /etc/ It's
>> the same thing as refusing to add /opt/local/bin to your PATH and then
>> complaining that the binaries are harder to find. Well, yeah, there are
>> conventional directories to host binaries and shared libraries, and the
>> convention exists for a reason; if you want to customize, you're free to
>> do so, but then you should use the adequate mechanism for customization,
>> which is adjusting your paths.
> Understood. Your rationale makes sense. I stand corrected.
> I tried adding /opt/local/lib to /etc/, and then running
> ldconfig, and it does work.

But this makes any shared library can be used from there, even when
lazy loaded (dlopen(3)). Depending on your system, that might be dangerous.

>> (Note that a number of s6 executables *expect* to find other s6
>> executables in their PATH. If /opt/local/bin is not in your default PATH,
>> and you have not used the --enable-absolute-paths configure option,
>> some s6 executables may not work correctly.)
> Yes, I was aware of that; thank you for noting it.

I wanted to do some analyzation before, but then it's on the table:
for some of the executables it might be reasonable to have them (optionally)
in ${PREFIX}/libexec/s6/bin to avoid polluting global search path.

>> But there's another solution: as a software author, I shouldn't hardcode
>> policy, but as an admin, *you* absolutely can! So, if that is your
>> preference, you can add the rpath yourself at build time:
>> env LDFLAGS=-Wl,-rpath,/opt/local/lib ./configure ....
> Indeed, that works as well! Thank you for the solution!
> Now, with no /opt/local/lib in /etc/, ldd shows:
> ----
> # ldd /opt/local/bin/s6-svscan
> => (0x00007fffddfe2000)
> => /opt/local/lib/ (0x00007fcac61dc000)
> => /opt/local/lib/ (0x00007fcac5f9c000)
> => /lib64/ (0x00007fcac5bce000)
> => /lib64/ (0x00007fcac59c6000)
> /lib64/ (0x00007fcac63e9000)
> => /lib64/ (0x00007fcac57aa000)
> ----
> So, problem solved in either of the two ways. Thanks!
> I still wonder about best practice, though. I'm curious about whether
> you have any more comments on this.

That heavily depends on use-case and threat-scenario ;)
You can also use "${ORIGIN}/../lib/" as RPATH, when you need it
relocatable but hardcoded.

> Your rationale for using /etc/ makes sense to me.
> However, at
> David Barr says:
> Half-hearted attempts to improve things
> Some OS's (For example, Linux) have a configurable loader. You can
> configure what run-time paths to look in by modifying /etc/
> This is almost as bad as LD_LIBRARY_PATH! Install scripts should never
> modify this file! This file should contain only the standard library
> locations as shipped with the OS.
> Unfortunately, he does not explain *why* he thinks modifying
> /etc/ is almost as bad as LD_LIBRARY_PATH.

Because it rules for all executables on your system. That can be dangerous.

> I think I can imagine a problem arising around having multiple versions
> of the same library installed in parallel. If the executables and
> libraries do not use rpath and instead depend on /etc/, then
> they can't declare exactly which library they want to use, and maybe you
> could end up with an executable using the wrong library or a bad mix of
> libraries? Obviously, they can declare to some extent using the major
> and minor shared object version. I'm not sure how this could go wrong.
> It seems that Fedora is not a fan of rpath in
> where it says:
> Since the Linux dynamic linker is usually smarter than a hardcoded
> path, we usually do not permit the use of rpath in Fedora.
> But that's in the context of packaging for a distro, and in the "Rpath
> for Internal Libraries" section, it *does* allow for rpath usage for
> internal libraries.
> And Debian also seems not be a fan of rpath in
> where it says:
> While there's no policy dictating the accepted use of RPATH, it's
> been a general consensus that RPATH use is discouraged, given the
> interactions between the above reasons and Debian's way of dealing
> with libraries and package dependencies.
> But again, it allows for using rpath for internal libraries:
> Currently, the only generally accepted use of this feature in Debian
> is to add non-standard library path (like /usr/lib/<package>) to
> libraries that are only intended to be used by the executables or
> other libraries within the same source package.

It's just because they want to permit relocatable installations without
relinking the files. Personally I think it's kinda risky to implement
such a behavior as default in all packages is careless - but hey :)

> Lastly, pkgsrc seems to use rpath based on
> which suggests using libtool with the '-rpath' option for linking.
> With pkgsrc, since it's a cross-platform packaging system, perhaps they
> don't want to depend on a feature like /etc/ since it is only
> available on a subset of the supported platforms.

It's probably just because Joerg Sonnenberger, J. Perkins etc. understand
the implications of careless installation of relocatable binaries
is like playing Russian roulette. Especially when the system
administration doesn't know nor understand the implications of that.

Or short: PkgSrc knows about the human error, most Linux distros just
ignore that - because any Linux user, admin, developer ... is brilliant
and always careful. Continuing that would result in a flame - so ... ;)
Just a personal opinion.

Jens Rehsack -

Received on Wed Dec 04 2019 - 06:56:50 UTC

This archive was generated by hypermail 2.3.0 : Sun May 09 2021 - 19:38:49 UTC