Including system library paths makes it difficult to overide library versions

[hugoduncan]

At present, Config::find returns system library paths. This can make it difficult to override library versions.

The motiviating example is using rust-openssl with ssh2-rs on OS X. On Mac, openssl requires a non-system version of openssl for TLS support, which can successfuly be provided by homebrew, for example. ssh2-rs depends on libz-rs, which pkg-config finds as -L /usr/lib. This library path gets added before the openssl library path, resulting in the system (older) openssl libraries being used, and the linker failing.

[alexcrichton]

Currently system library paths are required to allow the compiler to find static versions of libraries, but I can see where this would cause problems with multiple libs on the system...

[hugoduncan]

Could the compiler not add the system default paths, independently of pkg-config, at the end of the arguments list (so they have the lowest priority)?

[alexcrichton]

Perhaps, but the compiler has no knowledge of the default paths for a system (like the linker does)

[hugoduncan]

What would the process be, to move towards a solution for this?

[alexcrichton]

I can't really think of a great solution for this, so it would probably involve coming up with one first :)

[hugoduncan]

I'm not familiar with all the tooling rust uses to compile. Is there some place I can read up on the linkers used on different platforms? Could the compiler query the linker for the default paths?

[alexcrichton]

We generally just shell out to ld, although I'm not sure if we can query ld for search paths.

[julienw]

So in our project we ended up on exactly the same issue, and no easy solution. Of course the root cause is the awful setup of libssl on MacOS X but we need to find a way around this...

I still don't properly get why we need the system paths for the static libs to be found. Isn't it something that ld can find out by itself ?

[julienw]

If we really need it, here is a dumb idea: call pkg-config twice:

• once with PKG_CONFIG_ALLOW_SYSTEM_LIBS
• once without PKG_CONFIG_ALLOW_SYSTEM_LIBS

Diff the 2 outputs. Then we'd know which libs are the system libs.

[alexcrichton]

I still don't properly get why we need the system paths for the static libs to be found. Isn't it something that ld can find out by itself ?

The compiler needs to load the actual .a archive file to work with it much of the time, so we can't defer it to the linker.

Diff the 2 outputs. Then we'd know which libs are the system libs.

Perhaps!

[julienw]

I just re-read the manual page for pkg-config. Actually the list of paths that pkg-config considers as "system" seems to be hard-coded:

       PKG_CONFIG_ALLOW_SYSTEM_LIBS
              Don't strip -L/usr/lib or -L/lib out of libs.

So maybe it's good enough to just special-case these paths ?

[alexcrichton]

I'd be game!

[julienw]

So now, how do we special case them ? Do you think rustc could actually automatically always add them (and then we can remove them from pkg-config output)? Or would it be too fragile and we should instead add them as a new special build.rs output that cargo would understand ?

[alexcrichton]

It's pretty rare to pick up a staticlib from there, so if we're linking a dylib we just shouldn't print out those two -L paths probably