Discussion:
[gentoo-dev] crossdev and multilib interference
(too old to reply)
hasufell
2014-03-12 15:50:01 UTC
Permalink
We have a problem where the crossdev pkg-config wrapper scripts
interfere with multilib.

crossdev for example sets in their pkg-config wrappers:

PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"

Now, SYSROOT is chosen from multiple conditions. When emerging a
package, that happens to be "/" and thus results in:
"//usr/lib/pkgconfig://usr/share/pkgconfig"

Build systems like autotools will pick the crossdev provided
"i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
find the pkg-config files in /usr/lib64/...

This is not a problem most of the time if the package just wants to
get the libs to link against.

However, every package that tries to access variables that are
different between /usr/lib32/pkgconfig/foo.pc and
/usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
unexpected results.

That already happens for
x11-libs/libva-vdpau-driver
x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)

and there are probably more.

A simple workaround is:
PKG_CONFIG="pkg-config" emerge foo

But I think that is not appropriate to set in the eclass. How can we
solve this? Don't bikeshed.
Alexandre Rostovtsev
2014-03-12 16:10:03 UTC
Permalink
Post by hasufell
We have a problem where the crossdev pkg-config wrapper scripts
interfere with multilib.
PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"
Now, SYSROOT is chosen from multiple conditions. When emerging a
"//usr/lib/pkgconfig://usr/share/pkgconfig"
Build systems like autotools will pick the crossdev provided
"i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
find the pkg-config files in /usr/lib64/...
This is not a problem most of the time if the package just wants to
get the libs to link against.
However, every package that tries to access variables that are
different between /usr/lib32/pkgconfig/foo.pc and
/usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
unexpected results.
That already happens for
x11-libs/libva-vdpau-driver
x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
and there are probably more.
PKG_CONFIG="pkg-config" emerge foo
But I think that is not appropriate to set in the eclass. How can we
solve this? Don't bikeshed.
Two possibilities:
1. Don't allow crossdev to handle targets which are natively handled by
multilib profiles. For example, is there any legitimate reason for
wanting crossdev's i686 wrappers when on a multilib amd64 profile?
2. Have crossdev install its wrappers in a prefix, for example
in /usr/libexec/crossdev, which gets added to PATH by cross-emerge.
Alexis Ballier
2014-03-12 19:00:02 UTC
Permalink
On Wed, 12 Mar 2014 12:06:32 -0400
Post by Alexandre Rostovtsev
1. Don't allow crossdev to handle targets which are natively handled
by multilib profiles. For example, is there any legitimate reason for
wanting crossdev's i686 wrappers when on a multilib amd64 profile?
yes, serving as a distcc server for x86 hosts or using 'cross emerge'
to build a x86 root from scratch
Post by Alexandre Rostovtsev
2. Have crossdev install its wrappers in a prefix, for example
in /usr/libexec/crossdev, which gets added to PATH by cross-emerge.
lgtm
Greg Turner
2014-03-16 12:00:02 UTC
Permalink
Just a few practical notes on this...

On Wed, Mar 12, 2014 at 9:06 AM, Alexandre Rostovtsev
Post by hasufell
Post by hasufell
Now, SYSROOT is chosen from multiple conditions. When emerging a
"//usr/lib/pkgconfig://usr/share/pkgconfig"
Bleh. This is where I obligatorily remind everyone that // != /. POSIX,
cygwin, blahblahblah. In short, paths matching the regex "^//[^/]" are
trouble, and will eventually force me to fix your code once I get back to
gentoo-cygwin hacking.
Post by hasufell
Post by hasufell
Build systems like autotools will pick the crossdev provided
"i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
find the pkg-config files in /usr/lib64/...
So do Gentoo's own toolchain*.eclasses, breaking many $(tc-getFOO)
invocations in non-best multilib-build ABIS for which a matching crossdev
target is installed (substituting ${FOO:-$(tc-getFOO)} works-around these
problems in every instance I've seen; it probably wouldn't hurt to make
such substitutions systematically, when multilib-utizing ebuilds and
eclasses).
Post by hasufell
Post by hasufell
This is not a problem most of the time if the package just wants to
get the libs to link against
lots of "ignoring blahlib.so 'cause it's for the wrong ABI"-type warnings
are a good sign your ebuild may have fallen into this rabbit-hole.
Post by hasufell
However, every package that tries to access variables that are
Post by hasufell
different between /usr/lib32/pkgconfig/foo.pc and
/usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
unexpected results.
That already happens for
x11-libs/libva-vdpau-driver
x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
and there are probably more.
"Every" might be too strong, but... yeah, tons.

cmake-multilib.eclass, for example, breaks in mind-warpingly subtle and
confusing ways on USE="abi_x86_{32,64}" multilib hosts with
i686-pc-linux-gnu crossdev installed (when combined with some other issues
in that eclass, this results in correct qawarns about ignored ldflags on
devel profiles -- an ugly work-around is in my overlay, but I'm not happy
with it, so it's been languishing in my rainy-day todo queue. I'd be
thrilled to see a solution to the underlying problem, so I don't feel
compelled to salvage the ugly parts).

As for how to fix it, if foo-bar-baz-quux crossdev targets are at
${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that, seems
perfectly reasonable... heck, pure speculation, but it might even
noticeably speed up day-to-day $PATH searching on systems with lots of
crossdev targets installed.
Mike Frysinger
2014-03-26 06:10:01 UTC
Permalink
Post by Greg Turner
cmake-multilib.eclass, for example, breaks in mind-warpingly subtle and
confusing ways on USE="abi_x86_{32,64}" multilib hosts with
i686-pc-linux-gnu crossdev installed (when combined with some other issues
in that eclass, this results in correct qawarns about ignored ldflags on
devel profiles -- an ugly work-around is in my overlay, but I'm not happy
with it, so it's been languishing in my rainy-day todo queue. I'd be
thrilled to see a solution to the underlying problem, so I don't feel
compelled to salvage the ugly parts).
cmake is completely broken when it comes to library searching and multilib and
cross-compiling. it will happily look in hardcoded / paths to test for the
existence of files as well as directly execute `pkg-config`. it's a great
example of people saying "autotools is crap, so let's invent our own kind of
crap and ignore lessons learned". this isn't the fault of cmake eclasses, but
it'd be nice if we could someone standardize the hacks in there so we don't
have to duplicate across ebuilds.

just look at how cmake internally utilizes CMAKE_FIND_ROOT_PATH. or grep
"pkg-config" in /usr/share/cmake/. or look at find_library usage in cmake
macros. it's fundamentally screwed up right now :(.
Post by Greg Turner
As for how to fix it, if foo-bar-baz-quux crossdev targets are at
${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that, seems
perfectly reasonable... heck, pure speculation, but it might even
noticeably speed up day-to-day $PATH searching on systems with lots of
crossdev targets installed.
if they're in $PATH, then the exact location is irrelevant. they need not be
in /usr/bin to cause a problem. if they're not in $PATH, then you're breaking
the cross-compilers and that is unacceptable.

putting CHOST things in /usr/CTARGET is incorrect. /usr/CHOST/CTARGET/ is for
hosting helper programs specific to CTARGET that run on CHOST.
-mike
Steven J. Long
2014-03-26 12:10:04 UTC
Permalink
Post by Mike Frysinger
Post by Greg Turner
As for how to fix it, if foo-bar-baz-quux crossdev targets are at
${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that,
seems perfectly reasonable... heck, pure speculation, but it might
even noticeably speed up day-to-day $PATH searching on systems with
lots of crossdev targets installed.
if they're in $PATH, then the exact location is irrelevant.
they need not be in /usr/bin to cause a problem.
if they're not in $PATH, then you're breaking the cross-compilers
and that is unacceptable.
Cross-compilation should be supported via cross-emerge, and perhaps a small
script the cross-compiler sources to setup the env (ie prefix to PATH in
this case) for using CHOST-* tools, like x86-pc-linux-gnu-gcc targetting
a straight x86 platform, instead of the normal multilib setup. The
latter being used by the former (I'd have thought it was already done.)

The cross tools should NOT pollute the default PATH, simply because the
user happened to run crossdev at some point. It's *borked*, plain and
simple, so fix it please or expect people to come up with other solutions
[1]; fragmenting the effort, and making cross-compilers lives harder, as
we try to blend together a working solution from various efforts. The
exact thing crossdev is supposed to answer.
Post by Mike Frysinger
putting CHOST things in /usr/CTARGET is incorrect. /usr/CHOST/CTARGET/
is for hosting helper programs specific to CTARGET that run on CHOST.
Great, make it part of the env script. Though CHOST is usually the "target"
for most packages, so not sure of the relevance apart from toolchain.
Greg appeared to be saying use /usr/CHOST/usr/bin/gcc for example (istr
it's effectively a root under there) or just prefix the relevant bindirs
to PATH, along with whatever other voodoo you need. sed for instance is
still going to come from native CBUILD /bin.

Regards,
steveL

[1] https://github.com/chewi/cross-boss [still in preview]
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
Mike Frysinger
2014-03-26 16:20:02 UTC
Permalink
Post by Steven J. Long
Post by Mike Frysinger
Post by Greg Turner
As for how to fix it, if foo-bar-baz-quux crossdev targets are at
${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something like that,
seems perfectly reasonable... heck, pure speculation, but it might
even noticeably speed up day-to-day $PATH searching on systems with
lots of crossdev targets installed.
if they're in $PATH, then the exact location is irrelevant.
they need not be in /usr/bin to cause a problem.
if they're not in $PATH, then you're breaking the cross-compilers
and that is unacceptable.
Cross-compilation should be supported via cross-emerge, and perhaps a small
script the cross-compiler sources to setup the env (ie prefix to PATH in
this case) for using CHOST-* tools, like x86-pc-linux-gnu-gcc targetting
a straight x86 platform, instead of the normal multilib setup. The
latter being used by the former (I'd have thought it was already done.)
The cross tools should NOT pollute the default PATH, simply because the
user happened to run crossdev at some point. It's *borked*, plain and
simple, so fix it please or expect people to come up with other solutions
[1]; fragmenting the effort, and making cross-compilers lives harder, as
we try to blend together a working solution from various efforts. The
exact thing crossdev is supposed to answer.
that's bs. people install crossdev to get a cross-compile environment, not to
get something that only works through `emerge`. attempting to restrict it so
it only works through `emerge` is unacceptable and it has never been that way.
-mike
Ian Stakenvicius
2014-03-26 16:30:02 UTC
Permalink
Post by Mike Frysinger
Post by Steven J. Long
Post by Mike Frysinger
Post by Greg Turner
As for how to fix it, if foo-bar-baz-quux crossdev targets
are at ${EROOT}/usr/foo-bar-baz-quux, putting wrappers in
${EROOT}/usr/foo-bar-baz-quux/cross-wrappers, or something
like that, seems perfectly reasonable... heck, pure
speculation, but it might even noticeably speed up day-to-day
$PATH searching on systems with lots of crossdev targets
installed.
if they're in $PATH, then the exact location is irrelevant.
they need not be in /usr/bin to cause a problem. if they're not
in $PATH, then you're breaking the cross-compilers and that is
unacceptable.
Cross-compilation should be supported via cross-emerge, and
perhaps a small script the cross-compiler sources to setup the
env (ie prefix to PATH in this case) for using CHOST-* tools,
like x86-pc-linux-gnu-gcc targetting a straight x86 platform,
instead of the normal multilib setup. The latter being used by
the former (I'd have thought it was already done.)
The cross tools should NOT pollute the default PATH, simply
because the user happened to run crossdev at some point. It's
*borked*, plain and simple, so fix it please or expect people to
come up with other solutions [1]; fragmenting the effort, and
making cross-compilers lives harder, as we try to blend together
a working solution from various efforts. The exact thing crossdev
is supposed to answer.
that's bs. people install crossdev to get a cross-compile
environment, not to get something that only works through `emerge`.
attempting to restrict it so it only works through `emerge` is
unacceptable and it has never been that way. -mike
it -does- make sense though to limit anything that one wants to EMERGE
with the crossdev, to require the use of cross-emerge. Would it not
be possible to somehow ensure the crossdev tools are ignored
in/removed from/cannot pollute the standard emerge environment? Are
there any use cases where one -would- want the crossdev to be used in
a standard emerge environment instead of using cross-emerge ?
Mike Frysinger
2014-03-27 02:50:02 UTC
Permalink
Post by Ian Stakenvicius
Post by Mike Frysinger
that's bs. people install crossdev to get a cross-compile
environment, not to get something that only works through `emerge`.
attempting to restrict it so it only works through `emerge` is
unacceptable and it has never been that way.
it -does- make sense though to limit anything that one wants to EMERGE
with the crossdev, to require the use of cross-emerge. Would it not
be possible to somehow ensure the crossdev tools are ignored
in/removed from/cannot pollute the standard emerge environment? Are
there any use cases where one -would- want the crossdev to be used in
a standard emerge environment instead of using cross-emerge ?
you've lost me. when you `emerge-$CTARGET`, that package doesn't go anywhere
near your ROOT=/ system. it's entirely contained in /usr/$CTARGET/.

when you run `crossdev $CTARGET`, it installs all the standard $CTARGET-xxx
tools in /usr/bin. this isn't "polluting" the environment at all ... in fact,
they're living right alongside existing tools.

as i pointed out elsewhere in this thread, the problem is that multilib relies
on automatic detection of the toolchain *failing* so that it falls back to the
native value. in other words, when you run `./configure --host=i686-pc-linux-
gnu`, it tries to find e.g. i686-pc-linux-gnu-ar. it doesn't exist so the
fallback is used (plain `ar`). multilib is using these tuples so that the
standard checks (autoconf/eclasses/etc...) trigger in the right ways for the
cpu/os/userland combinations.

since crossdev installs a full proper toolchain for the target, the one
multilib was using to lie now exists and its toolchain is used instead.
-mike
Alexandre Rostovtsev
2014-03-27 04:50:01 UTC
Permalink
Post by Mike Frysinger
Post by Ian Stakenvicius
Post by Mike Frysinger
that's bs. people install crossdev to get a cross-compile
environment, not to get something that only works through `emerge`.
attempting to restrict it so it only works through `emerge` is
unacceptable and it has never been that way.
it -does- make sense though to limit anything that one wants to EMERGE
with the crossdev, to require the use of cross-emerge. Would it not
be possible to somehow ensure the crossdev tools are ignored
in/removed from/cannot pollute the standard emerge environment? Are
there any use cases where one -would- want the crossdev to be used in
a standard emerge environment instead of using cross-emerge ?
you've lost me. when you `emerge-$CTARGET`, that package doesn't go anywhere
near your ROOT=/ system. it's entirely contained in /usr/$CTARGET/.
when you run `crossdev $CTARGET`, it installs all the standard $CTARGET-xxx
tools in /usr/bin. this isn't "polluting" the environment at all ... in fact,
they're living right alongside existing tools.
as i pointed out elsewhere in this thread, the problem is that multilib relies
on automatic detection of the toolchain *failing* so that it falls back to the
native value. in other words, when you run `./configure --host=i686-pc-linux-
gnu`, it tries to find e.g. i686-pc-linux-gnu-ar. it doesn't exist so the
fallback is used (plain `ar`). multilib is using these tuples so that the
standard checks (autoconf/eclasses/etc...) trigger in the right ways for the
cpu/os/userland combinations.
since crossdev installs a full proper toolchain for the target, the one
multilib was using to lie now exists and its toolchain is used instead.
-mike
Crossdev is "polluting the environment" in the specific case that we are
talking about - secondary native ABIs on a multilib system.

An amd64 multilib system is not expected to build MIPS binaries that
would be hosted on itself. So of course anyone using amd64 undersands
that mips-pc-linux-gnu-ar is part of a cross-compile toolchain, no
matter whether it's in /usr/bin or /usr/libexec/crossdev or anywhere in
the filesystem.

However, an i686 crossdev on an amd64 multilib system is a fundamentally
different situation. An amd64 multilib system *is* expected to build x86
binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
expected to be not a part of any cross-compile toolchain, but a part of
the native toolchain for the machine's secondary native ABI. Especially
when i686-pc-linux-gnu-ar is in /usr/bin.
Mike Frysinger
2014-03-27 06:10:01 UTC
Permalink
Post by Alexandre Rostovtsev
Post by Mike Frysinger
Post by Ian Stakenvicius
Post by Mike Frysinger
that's bs. people install crossdev to get a cross-compile
environment, not to get something that only works through `emerge`.
attempting to restrict it so it only works through `emerge` is
unacceptable and it has never been that way.
it -does- make sense though to limit anything that one wants to EMERGE
with the crossdev, to require the use of cross-emerge. Would it not
be possible to somehow ensure the crossdev tools are ignored
in/removed from/cannot pollute the standard emerge environment? Are
there any use cases where one -would- want the crossdev to be used in
a standard emerge environment instead of using cross-emerge ?
you've lost me. when you `emerge-$CTARGET`, that package doesn't go
anywhere near your ROOT=/ system. it's entirely contained in
/usr/$CTARGET/.
when you run `crossdev $CTARGET`, it installs all the standard $CTARGET-xxx
tools in /usr/bin. this isn't "polluting" the environment at all ... in
fact, they're living right alongside existing tools.
as i pointed out elsewhere in this thread, the problem is that multilib
relies on automatic detection of the toolchain *failing* so that it falls
back to the native value. in other words, when you run `./configure
--host=i686-pc-linux- gnu`, it tries to find e.g. i686-pc-linux-gnu-ar.
it doesn't exist so the fallback is used (plain `ar`). multilib is using
these tuples so that the standard checks (autoconf/eclasses/etc...)
trigger in the right ways for the cpu/os/userland combinations.
since crossdev installs a full proper toolchain for the target, the one
multilib was using to lie now exists and its toolchain is used instead.
Crossdev is "polluting the environment" in the specific case that we are
talking about - secondary native ABIs on a multilib system.
An amd64 multilib system is not expected to build MIPS binaries that
would be hosted on itself. So of course anyone using amd64 undersands
that mips-pc-linux-gnu-ar is part of a cross-compile toolchain, no
matter whether it's in /usr/bin or /usr/libexec/crossdev or anywhere in
the filesystem.
However, an i686 crossdev on an amd64 multilib system is a fundamentally
different situation.
no, it is not
Post by Alexandre Rostovtsev
An amd64 multilib system *is* expected to build x86
binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
expected to be not a part of any cross-compile toolchain, but a part of
the native toolchain for the machine's secondary native ABI. Especially
when i686-pc-linux-gnu-ar is in /usr/bin.
sure, and it works just fine when you use the correct toolchain. if the user
wants to build an ABI using their default toolchain, they can pass the right
ABI flag for it.

but that's irrelevant to how our multilib implementation picks its fake
CHOST_$ABI to take care of implementing things. if anything, the current
system is provably broken because the default ends up executing unqualified
`ar` and `ranlib` friends.
-mike
Alexandre Rostovtsev
2014-03-27 06:40:01 UTC
Permalink
Post by Mike Frysinger
Post by Alexandre Rostovtsev
An amd64 multilib system *is* expected to build x86
binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
expected to be not a part of any cross-compile toolchain, but a part of
the native toolchain for the machine's secondary native ABI. Especially
when i686-pc-linux-gnu-ar is in /usr/bin.
sure, and it works just fine when you use the correct toolchain. if the user
wants to build an ABI using their default toolchain, they can pass the right
ABI flag for it.
They can't pass the right ABI flag because only the core parts of the
toolchain have the concept of an ABI flag.

Sure, binutils and gcc respect "-m32". But what about pkgconfig (and its
clones pkgconf and pkgconfig-openbsd)? What about the *-config tools for
various libraries? Are you going to patch all of them to respect "-m32"?
Mike Frysinger
2014-03-27 06:50:01 UTC
Permalink
Post by Alexandre Rostovtsev
Post by Mike Frysinger
Post by Alexandre Rostovtsev
An amd64 multilib system *is* expected to build x86
binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
expected to be not a part of any cross-compile toolchain, but a part of
the native toolchain for the machine's secondary native ABI. Especially
when i686-pc-linux-gnu-ar is in /usr/bin.
sure, and it works just fine when you use the correct toolchain. if the
user wants to build an ABI using their default toolchain, they can pass
the right ABI flag for it.
They can't pass the right ABI flag because only the core parts of the
toolchain have the concept of an ABI flag.
Sure, binutils and gcc respect "-m32". But what about pkgconfig (and its
clones pkgconf and pkgconfig-openbsd)? What about the *-config tools for
various libraries? Are you going to patch all of them to respect "-m32"?
pkg-config does need fixing in some way. we already know this. it's why the
multilib eclasses currently set PKG_CONFIG_XXX vars -- preciously so the
correct ABI dir is utilized. and this breaks when using some build systems
(like scons) where the env gets blown away (although we also know scons
sucks).

i don't care about the *-config scripts. that's a dead concept long ago
proven to suck and anything still using it needs fixing.
-mike
Samuli Suominen
2014-03-27 07:10:01 UTC
Permalink
Post by Mike Frysinger
Post by Alexandre Rostovtsev
Post by Mike Frysinger
Post by Alexandre Rostovtsev
An amd64 multilib system *is* expected to build x86
binaries that would be hosted on itself. So i686-pc-linux-gnu-ar is
expected to be not a part of any cross-compile toolchain, but a part of
the native toolchain for the machine's secondary native ABI. Especially
when i686-pc-linux-gnu-ar is in /usr/bin.
sure, and it works just fine when you use the correct toolchain. if the
user wants to build an ABI using their default toolchain, they can pass
the right ABI flag for it.
They can't pass the right ABI flag because only the core parts of the
toolchain have the concept of an ABI flag.
Sure, binutils and gcc respect "-m32". But what about pkgconfig (and its
clones pkgconf and pkgconfig-openbsd)? What about the *-config tools for
various libraries? Are you going to patch all of them to respect "-m32"?
pkg-config does need fixing in some way. we already know this. it's why the
multilib eclasses currently set PKG_CONFIG_XXX vars -- preciously so the
correct ABI dir is utilized. and this breaks when using some build systems
(like scons) where the env gets blown away (although we also know scons
sucks).
I pushed 0.28-r1 of dev-util/pkgconfig with ABI_X86 support so that you
can directly
call eg. i686-pc-linux-gnu-pkg-config to search from /usr/lib32/pkgconfig/

I'll try to figure something out for pkgconfig-openbsd too. Don't care
about pkgconf.
Post by Mike Frysinger
i don't care about the *-config scripts. that's a dead concept long ago
proven to suck and anything still using it needs fixing.
nod
Mike Frysinger
2014-03-27 07:30:02 UTC
Permalink
really have no idea what you're ranting about. doesn't look discussion worthy
though.
-mike
Steven J. Long
2014-03-27 08:30:02 UTC
Permalink
Post by Mike Frysinger
Post by Steven J. Long
Post by Mike Frysinger
if they're in $PATH, then the exact location is irrelevant.
they need not be in /usr/bin to cause a problem.
if they're not in $PATH, then you're breaking the cross-compilers
and that is unacceptable.
Cross-compilation should be supported via cross-emerge, and perhaps a small
script the cross-compiler sources to setup the env (ie prefix to PATH in
this case) for using CHOST-* tools, like x86-pc-linux-gnu-gcc targetting
a straight x86 platform, instead of the normal multilib setup. The
latter being used by the former (I'd have thought it was already done.)
The cross tools should NOT pollute the default PATH, simply because the
user happened to run crossdev at some point.
that's bs. people install crossdev to get a cross-compile environment, not to
get something that only works through `emerge`. attempting to restrict it so
it only works through `emerge` is unacceptable and it has never been that way.
That's why I suggested a small sh script to source, to setup that environment.
Personally, I do an awful lot more than that just to build a native chroot,
let alone cross-compile. And I really don't see the hardship for these brave
"cross-compilers" of yours in sourcing a small setup script, which can be
added to ~/.bashrc or even the system-wide one, for anyone who really wants
it to apply whenever they login. Is this somehow beyond our most advanced
userbase?

People may install crossdev to get a cross-compile environment, but it's a
broken design if it's not contained. And BS about how you think it should
ALWAYS go for everybody, only leads to borked "solutions" elsewhere like
telling the people on an amd64 install that they have to run some god-awful
"new" %multilib thing to compile for their secondary ABI. That's just as
counter-intuitive, when you could just fix your borkage and have a clean
setup for everyone.
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
Mike Frysinger
2014-03-28 06:40:02 UTC
Permalink
Post by Steven J. Long
Post by Mike Frysinger
Post by Steven J. Long
Post by Mike Frysinger
if they're in $PATH, then the exact location is irrelevant.
they need not be in /usr/bin to cause a problem.
if they're not in $PATH, then you're breaking the cross-compilers
and that is unacceptable.
Cross-compilation should be supported via cross-emerge, and perhaps a small
script the cross-compiler sources to setup the env (ie prefix to PATH in
this case) for using CHOST-* tools, like x86-pc-linux-gnu-gcc targetting
a straight x86 platform, instead of the normal multilib setup. The
latter being used by the former (I'd have thought it was already done.)
The cross tools should NOT pollute the default PATH, simply because the
user happened to run crossdev at some point.
that's bs. people install crossdev to get a cross-compile environment,
not to get something that only works through `emerge`. attempting to
restrict it so it only works through `emerge` is unacceptable and it has
never been that way.
That's why I suggested a small sh script to source, to setup that
environment. Personally, I do an awful lot more than that just to build a
native chroot, let alone cross-compile. And I really don't see the hardship
for these brave "cross-compilers" of yours in sourcing a small setup
script, which can be added to ~/.bashrc or even the system-wide one, for
anyone who really wants it to apply whenever they login. Is this somehow
beyond our most advanced userbase?
People may install crossdev to get a cross-compile environment, but it's a
broken design if it's not contained. And BS about how you think it should
ALWAYS go for everybody, only leads to borked "solutions" elsewhere like
telling the people on an amd64 install that they have to run some god-awful
"new" %multilib thing to compile for their secondary ABI. That's just as
counter-intuitive, when you could just fix your borkage and have a clean
setup for everyone.
your conclusions are invalid as you're basing them on the assumption that the
current multilib system is working correctly and cleanly. it is provably not.

sticking your head in the sand and blaming crossdev for errors in the multilib
logic is asinine.
-mike
Steven J. Long
2014-03-30 09:40:01 UTC
Permalink
Post by Mike Frysinger
Post by Steven J. Long
Post by Mike Frysinger
Post by Steven J. Long
The cross tools should NOT pollute the default PATH, simply because the
user happened to run crossdev at some point.
that's bs. people install crossdev to get a cross-compile environment,
not to get something that only works through `emerge`. attempting to
restrict it so it only works through `emerge` is unacceptable and it has
never been that way.
That's why I suggested a small sh script to source, to setup that
environment. Personally, I do an awful lot more than that just to build a
native chroot, let alone cross-compile. And I really don't see the hardship
for these brave "cross-compilers" of yours in sourcing a small setup
script, which can be added to ~/.bashrc or even the system-wide one, for
anyone who really wants it to apply whenever they login. Is this somehow
beyond our most advanced userbase?
People may install crossdev to get a cross-compile environment, but it's a
broken design if it's not contained.
your conclusions are invalid as you're basing them on the assumption that
the current multilib system is working correctly and cleanly.
No, I am not. I am basing them on the premise that a "target" SYSROOT is
potentially only one of N such sysroots for the same config.guess stanza.
I expected you to see that usage immediately, and the flexibility that
separation brings. After all, crossdev separates sysroots out already. It's
a trivial change to keep them separate, and it's easy for cross-compilers,
who already deal with this kind of thing, and more, on a daily basis. Only
it gives them more options should they want them, as I for one do.

No, use-cases are not relevant at this point. If you can't see it, that
doesn't meant it's not out there; it just means you're looking in the
wrong direction.
Post by Mike Frysinger
it is provably not.
sticking your head in the sand and blaming crossdev for errors in the
multilib logic is asinine.
And sticking your head in the sand and ignoring flexibility, is simply your
usual ego-waving (as well as providing amusement at your rationalisation-
masquerading-as-logic.) I'd hoped you'd grown past this in the last few years,
since the embarrassing (to you) eclass-manpages awk bug, but evidently not.
It's why you're an awful recruiter, and imo a useless coder, despite being
a very talented QA person and bug-patcher.

The more you code, the more you have to realise how susceptible you are to
mistake. It's *after* you fully realise and _accept_ that, that you become
a truly useful coder. You are clearly still at the wrestling-with-the-ego
stage, after all these years. So you have my sympathy, because I know how
difficult a fight that is, and you're clearly in distress (or you wouldn't
resort to negative personal attacks so frequently, instead of considering
the application. So much bile..) The way to win is to stop fighting,
because you are only fighting yourself.

Accept that you are only a human, and you make mistakes and have blind-spots
just like everyone else. One's just been pointed out to you; do you do the
grown-up thing and accept that, and fix it, or do you keep on with the
tantrums, the dodgy patches and the baroque chain of "fixes" that flow from
your borked "logic"? Grow up, already. Or fire off more bile > /dev/null.

"I'll see you when you get there, if you ever get there.."
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
hasufell
2014-06-15 20:40:01 UTC
Permalink
Post by Steven J. Long
"I'll see you when you get there, if you ever get there.."
No improvements so far. I am going to hardmask sys-devel/crossdev,
unless someone can explain why we are still in broken stage.

More packages are popping up that randomly break. Recent failures were
related to tc-getBUILD_CC.

This isn't stable in any way. I'm not blaming anyone, but that's what
hardmasking is for. A working solution was declined, so...
Chí-Thanh Christopher Nguyễn
2014-06-15 20:50:03 UTC
Permalink
Post by hasufell
No improvements so far. I am going to hardmask sys-devel/crossdev,
unless someone can explain why we are still in broken stage.
More packages are popping up that randomly break. Recent failures were
related to tc-getBUILD_CC.
This isn't stable in any way. I'm not blaming anyone, but that's what
hardmasking is for. A working solution was declined, so...
If I understand correctly, it is not the crossdev package being present on
the system, but the generated toolchains that cause the trouble.

I think there are less intrusive options than hardmask, such as pkg_pretend()
check or blocking offending packages from cross-${CTARGET} category.


Best regards,
Chí-Thanh Christopher Nguyễn
hasufell
2014-06-16 13:40:03 UTC
Permalink
Post by Chí-Thanh Christopher Nguyễn
Post by hasufell
No improvements so far. I am going to hardmask sys-devel/crossdev,
unless someone can explain why we are still in broken stage.
More packages are popping up that randomly break. Recent failures were
related to tc-getBUILD_CC.
This isn't stable in any way. I'm not blaming anyone, but that's what
hardmasking is for. A working solution was declined, so...
If I understand correctly, it is not the crossdev package being present on
the system, but the generated toolchains that cause the trouble.
I think there are less intrusive options than hardmask, such as pkg_pretend()
check or blocking offending packages from cross-${CTARGET} category.
Best regards,
Chí-Thanh Christopher Nguyễn
There was a proposed solution which works perfectly fine: don't clutter
PATH with crossdev links.

If any embedded developer needs these tools in PATH he can add them
temporarily (I'm pretty sure he knows how; an elog can be added as
well), via wrapper scripts or whatnot. That's a reasonable trade-off.

However, toolchain does not agree and I don't randomly touch other
peoples packages (unless there is no response).

So there are only two things left:
* block crossdev within multilib eclasses (that sounds really wrong to me)
* hardmask it, so we are able to communicate this problem to the user
Steev Klimaszewski
2014-06-16 18:50:02 UTC
Permalink
Post by hasufell
Post by Chí-Thanh Christopher Nguyễn
Post by hasufell
No improvements so far. I am going to hardmask sys-devel/crossdev,
unless someone can explain why we are still in broken stage.
More packages are popping up that randomly break. Recent failures were
related to tc-getBUILD_CC.
This isn't stable in any way. I'm not blaming anyone, but that's what
hardmasking is for. A working solution was declined, so...
If I understand correctly, it is not the crossdev package being present on
the system, but the generated toolchains that cause the trouble.
I think there are less intrusive options than hardmask, such as pkg_pretend()
check or blocking offending packages from cross-${CTARGET} category.
Best regards,
Chí-Thanh Christopher Nguyễn
There was a proposed solution which works perfectly fine: don't clutter
PATH with crossdev links.
If any embedded developer needs these tools in PATH he can add them
temporarily (I'm pretty sure he knows how; an elog can be added as
well), via wrapper scripts or whatnot. That's a reasonable trade-off.
However, toolchain does not agree and I don't randomly touch other
peoples packages (unless there is no response).
* block crossdev within multilib eclasses (that sounds really wrong to me)
* hardmask it, so we are able to communicate this problem to the user
I'm someone who uses crossdev (and the cross compilers) quite heavily -
can you point me to a bug that you're talking about? I'm not in the
toolchain, and while I agree that temporarily adding the cross
compiler(s) to the PATH is easy, for some of us, it's easier to allow
Gentoo to do so.

I'm not a huge fan of multilib, but at the same time, I'd like to not
see crossdev being hardmasked, just to prove your point. I don't have
near as much free time as I'd like, but I may try to squeeze some time
in to help out.
hasufell
2014-06-16 19:40:03 UTC
Permalink
Post by Steev Klimaszewski
I'm someone who uses crossdev (and the cross compilers) quite heavily -
can you point me to a bug that you're talking about? I'm not in the
toolchain, and while I agree that temporarily adding the cross
compiler(s) to the PATH is easy, for some of us, it's easier to allow
Gentoo to do so.
I'm not a huge fan of multilib, but at the same time, I'd like to not
see crossdev being hardmasked, just to prove your point. I don't have
near as much free time as I'd like, but I may try to squeeze some time
in to help out.
see https://bugs.gentoo.org/show_bug.cgi?id=500338

emerge a native x86 toolchain and enable ABI_X86="32" globally. A lot of
packages randomly fail, either because of wrong PKG_CONFIG being picked
up or even wrong CC/CXX, causing failure at linking stage and whatnot.

Also check the history of this thread for a few proposed solutions.
hasufell
2014-06-16 19:50:01 UTC
Permalink
On Mon, 16 Jun 2014 19:31:58 +0000
Post by hasufell
Also check the history of this thread for a few proposed solutions.
The history of this thread and the history of gx86-multilib and
crossdev development suggest that crossdev was doing nothing wrong until
gx86-multilib came around and a problem was found between them. Masking
either for the benefit of the other would be, and let me quote the
history of this thread out of context just to fit in with the tone and
mode this sub-thread has taken, "asinine".
This isn't about right or wrong. This is about actual breakage on stable
systems.

Solutions were proposed, nothing has happened for months.

So I don't see what else we can do here other than taking more radical
steps to INFORM users of these possible breakages... and that's exactly
what a hardmask is for.
Joshua Kinard
2014-06-16 20:10:01 UTC
Permalink
Post by hasufell
On Mon, 16 Jun 2014 19:31:58 +0000
Post by hasufell
Also check the history of this thread for a few proposed solutions.
The history of this thread and the history of gx86-multilib and
crossdev development suggest that crossdev was doing nothing wrong until
gx86-multilib came around and a problem was found between them. Masking
either for the benefit of the other would be, and let me quote the
history of this thread out of context just to fit in with the tone and
mode this sub-thread has taken, "asinine".
This isn't about right or wrong. This is about actual breakage on stable
systems.
Solutions were proposed, nothing has happened for months.
So I don't see what else we can do here other than taking more radical
steps to INFORM users of these possible breakages... and that's exactly
what a hardmask is for.
What about those of us who have been using crossdev to generate
cross-compilers for years w/o issue, because we run non-multilib?
Hardmasking crossdev to solve multilib problems doesn't accomplish anything,
other than just irk us. Why not hardmask the multilib stuff instead and
leave crossdev alone?

Neither hardmask solution is viable, since you'll inconvenience one side for
the sake of the other. That's not how you solve problems.

Is my understanding of the issue correct, in that, per Bug #500338, crossdev
was used to merge an i686-pc-linux-gnu cross-toolchain onto an
x86_64-pc-linux-gnu system, resulting in /usr/bin/cross-pkg-config being
linked to /usr/bin/i686-pc-linux-gnu-pkg-config, which causes the problem
reported in that bug?

If so, is it sensible to allow crossdev to install a cross-toolchain when
the underlying machine architecture is the same, just a different ABI?
I.e., would a solution be to prevent i686-on-x86_64 or mips64-on-mips, but
still allow mips64-on-x86_64, and such?
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
hasufell
2014-06-16 20:30:01 UTC
Permalink
Post by Joshua Kinard
Post by hasufell
On Mon, 16 Jun 2014 19:31:58 +0000
Post by hasufell
Also check the history of this thread for a few proposed solutions.
The history of this thread and the history of gx86-multilib and
crossdev development suggest that crossdev was doing nothing wrong until
gx86-multilib came around and a problem was found between them. Masking
either for the benefit of the other would be, and let me quote the
history of this thread out of context just to fit in with the tone and
mode this sub-thread has taken, "asinine".
This isn't about right or wrong. This is about actual breakage on stable
systems.
Solutions were proposed, nothing has happened for months.
So I don't see what else we can do here other than taking more radical
steps to INFORM users of these possible breakages... and that's exactly
what a hardmask is for.
What about those of us who have been using crossdev to generate
cross-compilers for years w/o issue, because we run non-multilib?
Hardmasking crossdev to solve multilib problems doesn't accomplish anything,
other than just irk us. Why not hardmask the multilib stuff instead and
leave crossdev alone?
Hardmask half of the tree instead of a single package? Does not sound
reasonable. The fallout will be _huge_ for users who already run
multilib. You will basically get an emerge dump of 500+ blockers.
Post by Joshua Kinard
If so, is it sensible to allow crossdev to install a cross-toolchain when
the underlying machine architecture is the same, just a different ABI?
I.e., would a solution be to prevent i686-on-x86_64 or mips64-on-mips, but
still allow mips64-on-x86_64, and such?
yes, serving as a distcc server for x86 hosts or using 'cross emerge'
to build a x86 root from scratch
Joshua Kinard
2014-06-16 21:10:04 UTC
Permalink
Post by hasufell
Post by Joshua Kinard
Post by hasufell
On Mon, 16 Jun 2014 19:31:58 +0000
Post by hasufell
Also check the history of this thread for a few proposed solutions.
The history of this thread and the history of gx86-multilib and
crossdev development suggest that crossdev was doing nothing wrong until
gx86-multilib came around and a problem was found between them. Masking
either for the benefit of the other would be, and let me quote the
history of this thread out of context just to fit in with the tone and
mode this sub-thread has taken, "asinine".
This isn't about right or wrong. This is about actual breakage on stable
systems.
Solutions were proposed, nothing has happened for months.
So I don't see what else we can do here other than taking more radical
steps to INFORM users of these possible breakages... and that's exactly
what a hardmask is for.
What about those of us who have been using crossdev to generate
cross-compilers for years w/o issue, because we run non-multilib?
Hardmasking crossdev to solve multilib problems doesn't accomplish anything,
other than just irk us. Why not hardmask the multilib stuff instead and
leave crossdev alone?
Hardmask half of the tree instead of a single package? Does not sound
reasonable. The fallout will be _huge_ for users who already run
multilib. You will basically get an emerge dump of 500+ blockers.
Which is why I followed with the next paragraph that neither hardmask
solution is really viable. You inconvenience a group of people one way or
the other, even if you're only doing it just to raise a point and/or awareness.
Post by hasufell
Post by Joshua Kinard
If so, is it sensible to allow crossdev to install a cross-toolchain when
the underlying machine architecture is the same, just a different ABI?
I.e., would a solution be to prevent i686-on-x86_64 or mips64-on-mips, but
still allow mips64-on-x86_64, and such?
yes, serving as a distcc server for x86 hosts or using 'cross emerge'
to build a x86 root from scratch
Then, can crossdev be augmented to work around the invalid behavior? Has
anyone looked at crossdev's source to see if the issue can be corrected with
a patch? Can the offending feature be made optional via a USE flag? There
are other options available than simply hardmasking a package that many find
useful.
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
hasufell
2014-06-16 22:20:02 UTC
Permalink
Post by Joshua Kinard
Then, can crossdev be augmented to work around the invalid behavior?
Yes, by installing it into prefixes and requiring people to add it to
PATH on their own if they need it outside of cross-emerge.
Joshua Kinard
2014-06-16 23:40:01 UTC
Permalink
Post by hasufell
Post by Joshua Kinard
Then, can crossdev be augmented to work around the invalid behavior?
Yes, by installing it into prefixes and requiring people to add it to
PATH on their own if they need it outside of cross-emerge.
How big of a patch would this change require to the existing crossdev ebuild?

Can $PATH be configured via our existing eselect tool to enable/disable the
crossdev paths when needed?
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
hasufell
2014-06-17 01:50:01 UTC
Permalink
Post by Joshua Kinard
How big of a patch would this change require to the existing crossdev ebuild?
Probably quite trivial, but since vapier said "bs" to that proposal
(translates to "bullshit" I guess) I'll not put any work into that.

So there we go. If you are cool, you can just say "bs", vanish and leave
stable arch in a broken state.

Not even QA cares. Great. I'll try to get it on the next council agenda
then.
Joshua Kinard
2014-06-17 02:20:01 UTC
Permalink
Post by hasufell
Post by Joshua Kinard
How big of a patch would this change require to the existing crossdev ebuild?
Probably quite trivial, but since vapier said "bs" to that proposal
(translates to "bullshit" I guess) I'll not put any work into that.
So there we go. If you are cool, you can just say "bs", vanish and leave
stable arch in a broken state.
Not even QA cares. Great. I'll try to get it on the next council agenda
then.
So you just take your ball and go home then? That's not how it works.

Create the patch, and file it as a bug. Then, raise awareness on the ML.
That's how development works. If your patch is reasonable and doesn't break
things, odds are likely it'll push the other members of toolchain to
consider incorporating it.

Equally using the Council as a hammer all the time doesn't work in the
long-term, either. If you whip a patch up, however, then not only could you
raise this at the next council meeting, but additionally state you've gone
that extra mile and created a patch that addresses the problem.

That's taking the ball and putting it into the goal.
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
hasufell
2014-06-17 12:40:02 UTC
Permalink
Post by Joshua Kinard
Post by hasufell
Post by Joshua Kinard
How big of a patch would this change require to the existing crossdev ebuild?
Probably quite trivial, but since vapier said "bs" to that proposal
(translates to "bullshit" I guess) I'll not put any work into that.
So there we go. If you are cool, you can just say "bs", vanish and leave
stable arch in a broken state.
Not even QA cares. Great. I'll try to get it on the next council agenda
then.
So you just take your ball and go home then? That's not how it works.
Create the patch, and file it as a bug. Then, raise awareness on the ML.
That's how development works. If your patch is reasonable and doesn't break
things, odds are likely it'll push the other members of toolchain to
consider incorporating it.
Equally using the Council as a hammer all the time doesn't work in the
long-term, either. If you whip a patch up, however, then not only could you
raise this at the next council meeting, but additionally state you've gone
that extra mile and created a patch that addresses the problem.
That's taking the ball and putting it into the goal.
No, that's not how opensource works. You don't work on things after
"upstream" said "not interested".

https://bugs.gentoo.org/show_bug.cgi?id=504824
Rich Freeman
2014-06-17 12:50:02 UTC
Permalink
Post by hasufell
No, that's not how opensource works. You don't work on things after
"upstream" said "not interested".
That is hardly true though - which is why we have 47 different
implementations of everything to debate the merits of. :)

Besides, if this were truly an "upstream" issue the Council could
hardly do anything about it.

The solution to this problem isn't annoying crossdev users in the hope
that they will pester the crossdev maintainers. In theory they're the
main ones impacted by the bug in the first place.

Is there a list/etc for crossdev? I'd think that the users and
maintainers of crossdev collectively have the biggest vested interest
in addressing these issues. They're also the ones who can vouch for
how big of a problem this is.

Is this having some kind of adverse impact on anybody outside of the
crossdev community? If the crossdev maintainers were pushing hundreds
of packages to change to accommodate dubious design on their part I
could see this being a distro-wide issue. On the other hand, if this
is an issue that only impacts crossdev users and maintainers, then I'd
think the simplest solution would be let them sort it out.

If somebody in the crossdev community does want to sort it out and the
problem is package squatting, then that might be a valid reason to
escalate. In that case the cleanest solution is to have a crossdev
project, have the interested devs step up, hold a vote for the lead,
and then respect the lead's role in resolving the issue. Nobody
"owns" a package, but in general we should be careful about stepping
in and overriding maintainers, especially if nobody is interested in
stepping in to take the reins long-term.

Rich
hasufell
2014-06-17 14:00:01 UTC
Permalink
Post by Rich Freeman
Post by hasufell
No, that's not how opensource works. You don't work on things after
"upstream" said "not interested".
That is hardly true though - which is why we have 47 different
implementations of everything to debate the merits of. :)
I was excluding the case of forks, because (IMO) it wouldn't make sense
if I start a sys-devel/crossdev-ng package.


The problem is that working around these bugs reliably currently
involves setting dangerous INSTALL_MASK, creating post_install hackery
hooks and additional things to keep crossdev together.

The proposed approach of removing it from PATH would make all this
obsolete and require crossdev users who don't care about this bug and
want the toolchain globally accessible in PATH to add a single line in
their profile/bashrc.

But this single line seems to be a major inconvenience which is reason
enough to break a valid use case.
Post by Rich Freeman
Is there a list/etc for crossdev? I'd think that the users and
maintainers of crossdev collectively have the biggest vested interest
in addressing these issues. They're also the ones who can vouch for
how big of a problem this is.
Is this having some kind of adverse impact on anybody outside of the
crossdev community? If the crossdev maintainers were pushing hundreds
of packages to change to accommodate dubious design on their part I
could see this being a distro-wide issue. On the other hand, if this
is an issue that only impacts crossdev users and maintainers, then I'd
think the simplest solution would be let them sort it out.
I am a crossdev user (I don't just have it installed for fun).


Currently, we don't have any way to communicate this broken use case to
the user. That's not a good situation.

And my hopes for "let them sort it out" are so low, that I won't wait
for it (look at their responses... they consist of colorful threats,
saying "bs" and telling me that I do "dumb" things).


So from my side I can do a number of things:
1) block crossdev from within multilib-minimal.eclass... that brings me
to the question how to do it:
a) just block sys-devel/crossdev if any non-native ABI is activated:
pretty rough and will make some people angry
b) e.g. block cross-i686-pc-linux-gnu/gcc if ABI_X86="32" is
activated: repoman will probably kill me
c) do some sophisticated checks in a phase function that will call
"die" if the the broken use case is detected: pretty hacky, but
more safe than the current situation
2) print an elog that no one will read

All of the solutions above still leave the use case broken.
Joshua Kinard
2014-06-17 14:20:03 UTC
Permalink
Post by Rich Freeman
Is there a list/etc for crossdev? I'd think that the users and
maintainers of crossdev collectively have the biggest vested interest
in addressing these issues. They're also the ones who can vouch for
how big of a problem this is.
toolchain is the primary maintainer for crossdev. I actually wrote the
original implementation of crossdev years ago (~2004?). It was a crappy,
overly-complex bash script that didn't use anything portage related (that I
can remember) to install a cross-toolchain. It worked for simple toolchains
on common arches, i.e., i686 -> mips64 for me to build kernels for the SGI
systems. Mike rewrote it several years later to become the crossdev we know
and love today, integrating it with emerge and taking better advantage of
existing Gentoo capabilities.
Post by Rich Freeman
Is this having some kind of adverse impact on anybody outside of the
crossdev community? If the crossdev maintainers were pushing hundreds
of packages to change to accommodate dubious design on their part I
could see this being a distro-wide issue. On the other hand, if this
is an issue that only impacts crossdev users and maintainers, then I'd
think the simplest solution would be let them sort it out.
I certainly haven't seen crossdev related issues on my systems. Granted, I
tend to run rather simple setups, not doing things like i686-on-x86_64 or
even running X11. From the earlier-quoted bug, it looked like some X11
package is one of several affected by the issue being highlighted here.

What I'd like to see is a list of all affected packages so we all can get a
sense of just how big the actual problem really is. All I am hearing so far
are unsubstantiated claims of tree-wide breakage. Knowing which packages
are broken allows other devs to look at things and maybe come to agreement
that crossdev is the source of the problem or perhaps another solution that
applies to all of them can be worked up.
Post by Rich Freeman
If somebody in the crossdev community does want to sort it out and the
problem is package squatting, then that might be a valid reason to
escalate. In that case the cleanest solution is to have a crossdev
project, have the interested devs step up, hold a vote for the lead,
and then respect the lead's role in resolving the issue. Nobody
"owns" a package, but in general we should be careful about stepping
in and overriding maintainers, especially if nobody is interested in
stepping in to take the reins long-term.
I'm a member of toolchain, but that's mostly historical because I used to
play with a lot of the cross-compile stuff for MIPS and Sparc. Mike and
Ryan are the two primaries in toolchain right now. If they don't see a
problem with crossdev right now, then I do have to question just how big of
a problem this really is.
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
Alexandre Rostovtsev
2014-06-17 15:00:02 UTC
Permalink
Post by Joshua Kinard
What I'd like to see is a list of all affected packages so we all can get a
sense of just how big the actual problem really is. All I am hearing so far
are unsubstantiated claims of tree-wide breakage. Knowing which packages
are broken allows other devs to look at things and maybe come to agreement
that crossdev is the source of the problem or perhaps another solution that
applies to all of them can be worked up.
All multilib packages that use pkgconfig, for one thing. (Which means almost
all multilib packages.) Because current crossdev versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting the binary
belonging to pkgconfig[abi_x86_32].
Michał Górny
2014-06-17 15:20:01 UTC
Permalink
Dnia 2014-06-17, o godz. 10:56:41
Post by Alexandre Rostovtsev
Post by Joshua Kinard
What I'd like to see is a list of all affected packages so we all can get a
sense of just how big the actual problem really is. All I am hearing so far
are unsubstantiated claims of tree-wide breakage. Knowing which packages
are broken allows other devs to look at things and maybe come to agreement
that crossdev is the source of the problem or perhaps another solution that
applies to all of them can be worked up.
All multilib packages that use pkgconfig, for one thing. (Which means almost
all multilib packages.) Because current crossdev versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting the binary
belonging to pkgconfig[abi_x86_32].
Just to shed some more light on this before someone goes into wrong
conclusions.

The initial multilib code didn't use prefixed pkg-config binaries but
simple PKG_CONFIG_PATH override. However, crossdev installing
i686-pc-linux-gnu-pkg-config caused configure scripts to find and use
it instead of plain 'pkg-config', and since the wrapper dumbly
overrides PKG_CONFIG_PATH it broke our original solution.

We specifically made pkg-config packages install the prefixed binaries
to trigger collisions with crossdev -- so that people who have systems
broken with crossdev's i686-pc-linux-gnu-pkg-config would be more
likely informed there's something wrong with their systems.
--
Best regards,
Micha³ Górny
Peter Stuge
2014-06-18 15:30:02 UTC
Permalink
Post by Alexandre Rostovtsev
current crossdev versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting
the binary belonging to pkgconfig[abi_x86_32].
Thanks for getting to the point.

It seems silly for two toolchains (abi_x86_32 and a crossdev i686 toolchain)
to compete for one and the same name.

Is that really desirable?

Both toolchains integrate with emerge; either one can be used to emerge
packages, right? (Where the packages are emerged to doesn't matter.)

If Gentoo would like to support different toolchains for one CHOST
then there obviously needs to be some abstraction, and creating it
should take into account that things like autoconf, cmake, waf and
friends may not support any such abstraction for a long time to come.

An obvious low-finesse solution is to only allow one toolchain per CHOST.

That would formalize the current situation where only one toolchain or
the other works correctly, and would be fine as a first step.

If long-term Gentoo does indeed want to support both an abi_x86_32
and a crossdev-built i686-pc-linux-gnu toolchain, then obviously
someone interested in that will have to come up with how it will
work.

As has been said plenty of times already, doing anything on one side
that inconveniences the other side is not really acceptable. That
makes this problem an interesting one, in that there aren't really
any useful hacks to be done, only an actual proper solution that
benefits both multilib and crossdev users.

Only someone with an interest in doing something good for both those
groups will be able to find a solution.
Post by Alexandre Rostovtsev
Just to shed some more light on this before someone goes into wrong
conclusions.
Thanks for adding this background info! It is very helpful.
Post by Alexandre Rostovtsev
The initial multilib code didn't use prefixed pkg-config binaries
but simple PKG_CONFIG_PATH override.
Do you mean PKG_CONFIG_LIBDIR?
Post by Alexandre Rostovtsev
However, crossdev installing i686-pc-linux-gnu-pkg-config caused
configure scripts to find and use it instead of plain 'pkg-config',
That sounds to me like autoconf's pkg-config support might also be
involved in the problem?
Post by Alexandre Rostovtsev
and since the wrapper dumbly overrides PKG_CONFIG_PATH it broke our
original solution.
We specifically made pkg-config packages install the prefixed binaries
to trigger collisions with crossdev
That strikes me as really unhelpful. You spent time on creating something
you were sure would cause a problem, instead of on something to *avoid*
the problem. Oh well.
Post by Alexandre Rostovtsev
-- so that people who have systems broken with crossdev's
i686-pc-linux-gnu-pkg-config would be more likely informed there's
something wrong with their systems.
You seem to have made an arbitrary decision that crossdev's
pkg-config is at fault. I think we need to look at this in a larger
perspective as outlined above; take a step back.


I'm glad that we're now actually trying to understand what the real
problem is, instead of only treating the symptoms.


//Peter
Michał Górny
2014-06-19 08:00:02 UTC
Permalink
Dnia 2014-06-18, o godz. 17:24:03
Post by Peter Stuge
Post by Alexandre Rostovtsev
current crossdev versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting
the binary belonging to pkgconfig[abi_x86_32].
Thanks for getting to the point.
It seems silly for two toolchains (abi_x86_32 and a crossdev i686 toolchain)
to compete for one and the same name.
Is that really desirable?
No. But the user specifies prefix for crossdev. As far as I'm
concerned, crossdev could refuse prefixes that are already used
in Gentoo profiles.
Post by Peter Stuge
Both toolchains integrate with emerge; either one can be used to emerge
packages, right? (Where the packages are emerged to doesn't matter.)
No, 'integrating with emerge' is a bad word. Crossdev wraps emerge
and runs on top of it. Multilib runs 'below' emerge as run by eclass.
Post by Peter Stuge
If Gentoo would like to support different toolchains for one CHOST
then there obviously needs to be some abstraction, and creating it
should take into account that things like autoconf, cmake, waf and
friends may not support any such abstraction for a long time to come.
An obvious low-finesse solution is to only allow one toolchain per CHOST.
Obviously.
Post by Peter Stuge
That would formalize the current situation where only one toolchain or
the other works correctly, and would be fine as a first step.
If long-term Gentoo does indeed want to support both an abi_x86_32
and a crossdev-built i686-pc-linux-gnu toolchain, then obviously
someone interested in that will have to come up with how it will
work.
I'd rather see multilib gcc installing 'i686-pc-linux-gnu' wrappers
that call the native gcc with proper '-m' option. As far as I know,
there's no real difference between the code emitted by native compiler
with -m32 and cross-compiler for i686. This would remove some need for
crossdev (and therefore some breakages) and allow our users to avoid
rebuilding the same thing twice.
Post by Peter Stuge
Post by Alexandre Rostovtsev
The initial multilib code didn't use prefixed pkg-config binaries
but simple PKG_CONFIG_PATH override.
Do you mean PKG_CONFIG_LIBDIR?
Both.
Post by Peter Stuge
Post by Alexandre Rostovtsev
However, crossdev installing i686-pc-linux-gnu-pkg-config caused
configure scripts to find and use it instead of plain 'pkg-config',
That sounds to me like autoconf's pkg-config support might also be
involved in the problem?
The autoconf's AC_PROG_TOOL macros are proper and very useful. What's
the real problem is that crossdev installs some custom wrapper that
clobbers PKG_CONFIG_* where real pkg-config is expected.
Post by Peter Stuge
Post by Alexandre Rostovtsev
and since the wrapper dumbly overrides PKG_CONFIG_PATH it broke our
original solution.
We specifically made pkg-config packages install the prefixed binaries
to trigger collisions with crossdev
That strikes me as really unhelpful. You spent time on creating something
you were sure would cause a problem, instead of on something to *avoid*
the problem. Oh well.
This was hitting our users and crossdev team didn't care. I did what I
could from our side to fix this, though this probably wasn't good
enough.

Do you prefer if I add some logic to detect i686 crossdev and bail out
completely, telling users to remove it? That wouldn't be very friendly
but at least they wouldn't hit random build failures anymore.
Post by Peter Stuge
Post by Alexandre Rostovtsev
-- so that people who have systems broken with crossdev's
i686-pc-linux-gnu-pkg-config would be more likely informed there's
something wrong with their systems.
You seem to have made an arbitrary decision that crossdev's
pkg-config is at fault. I think we need to look at this in a larger
perspective as outlined above; take a step back.
Because it is at fault. Build systems expect *-pkg-config to be
a binary compatible with pkg-config. When crossdev installs something
that does not respect PKG_CONFIG_{LIBDIR,PATH} and instead uses some
fancy directories, it is inevitable that it will break something.

And just to be clear, we didn't invent anything new. The breakage was
there for years, we were just first ones to mix all the ingredients.

The CHOSTs used by our stuff come from profiles, we didn't invent them.
The multilib_toolchain_setup that sets the build environment comes from
multilib.eclass and was already used for multilib in the past. The code
that causes tc-getBUILD_CC to return incorrect (crossdev) compiler
comes from toolchain-funcs.eclass.

If you look at it all, you'd notice that all code that results in those
failures is maintained by toolchain team, and so is crossdev. So what
can small gx86-multilib team to do to fix it?
--
Best regards,
Micha³ Górny
Joshua Kinard
2014-06-17 15:30:01 UTC
Permalink
Post by Alexandre Rostovtsev
Post by Joshua Kinard
What I'd like to see is a list of all affected packages so we all can get a
sense of just how big the actual problem really is. All I am hearing so far
are unsubstantiated claims of tree-wide breakage. Knowing which packages
are broken allows other devs to look at things and maybe come to agreement
that crossdev is the source of the problem or perhaps another solution that
applies to all of them can be worked up.
All multilib packages that use pkgconfig, for one thing. (Which means almost
all multilib packages.) Because current crossdev versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting the binary
belonging to pkgconfig[abi_x86_32].
But how many packages is that? Is there a way to filter and count the
packages in the tree that are both multilib-capable and rely on pkgconfig?
This still doesn't convey the scale of the perceived problem, and this is
why people are not really convinced that a problem exists and that crossdev
is the source of the problem.

I am intentionally playing the role of the outsider on this because I don't
use multilib yet. Linux/MIPS kinda started the whole multilib thing anyways
w/ the o32/n32/n64 setup that got carried over from IRIX. Later on, you had
x86_64 join the fray, which made the problem much more noticeable. Convince
me that there is a problem and that crossdev is the source of that problem.
Right now, it seems to me that the problem isn't limited to just one
package created by Gentoo, but it's just that a LOT of packages in the
open-source world still haven't updated their build systems to account for
multiple-ABI installs.

Brainstorm: Would making crossdev's installation of the ${CHOST}-pkg-config
wrapper optional solve the problem somewhat? Perhaps as a USE flag that
multilib/pkgconfig packages can check in DEPEND and throw warnings about?
Do any of the other crossdev-installed ${CHOST}- prefixed scripts or
binaries installed in /usr/bin cause similar problems, or does everything
hinge on this one script?
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
Alexandre Rostovtsev
2014-06-18 05:20:02 UTC
Permalink
Post by Joshua Kinard
Post by Alexandre Rostovtsev
Post by Joshua Kinard
What I'd like to see is a list of all affected packages so we all can get a
sense of just how big the actual problem really is. All I am hearing so far
are unsubstantiated claims of tree-wide breakage. Knowing which packages
are broken allows other devs to look at things and maybe come to agreement
that crossdev is the source of the problem or perhaps another solution that
applies to all of them can be worked up.
All multilib packages that use pkgconfig, for one thing. (Which means almost
all multilib packages.) Because current crossdev versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting the binary
belonging to pkgconfig[abi_x86_32].
But how many packages is that? Is there a way to filter and count the
packages in the tree that are both multilib-capable and rely on pkgconfig?
This still doesn't convey the scale of the perceived problem, and this is
why people are not really convinced that a problem exists and that crossdev
is the source of the problem.
$ eix --depend -z virtual/pkgconfig -a --use -z abi_x86_32 -a --use -z abi_x86_64 --only-names | wc -l
285
Joshua Kinard
2014-06-18 06:30:02 UTC
Permalink
Post by Alexandre Rostovtsev
eix --depend -z virtual/pkgconfig -a --use -z abi_x86_32 -a --use -z abi_x86_64 --only-names | wc -l
Interesting, I got 294 (probably from my local dev tree). Close enough, thanks!



So, taking this count-packages script here:
http://dev.gentoo.org/~dberkholz/scripts/count-packages

And running it, I get the following information about the portage tree
(synced about ~2hrs ago):

Statistics for /usr/portage:
Total packages = 20436
Total categories = 174
Average packages per category = 117
Standard deviation of packages per category = 203
Suggested category size within (standard deviation / 2) of average: 16
to 218
Split categories with more than 218 packages, and do not create
categories with fewer than 16 packages.
Post by Alexandre Rostovtsev
print "%.2f" % ((float(294)/20436) * 100)
1.44
Post by Alexandre Rostovtsev
print "%.2f" % ((float(285)/20436) * 100)
1.39

IOW, it looks like less than 1.5% of the tree contains multilib packages
that rely on pkgconfig that could be affected by crossdev installing the
${CHOST}-pkg-config link into PATH.

We all have different measurements of things, but in my book, that doesn't
even begin to qualify for "non-trivial tree-wide problem". If we were
talking close to 5%-10% of packages affected, that to me would be worth some
serious discussion.

So since there isn't a pressing need to do something about it right now,
let's try to think up a proper way to address the problem for the longterm,
because the number of multilib/pkgconfig packages will likely increase in
the future.

Sound fair?
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
Ian Stakenvicius
2014-06-18 14:20:02 UTC
Permalink
Post by Joshua Kinard
IOW, it looks like less than 1.5% of the tree contains multilib
packages that rely on pkgconfig that could be affected by crossdev
installing the ${CHOST}-pkg-config link into PATH.
We all have different measurements of things, but in my book, that
doesn't even begin to qualify for "non-trivial tree-wide problem".
If we were talking close to 5%-10% of packages affected, that to me
would be worth some serious discussion.
Only one problem with that -- if much of that 1.5% is packages in
@system , then it doesn't matter how much of the tree it is in terms
of end-user impact. It's all about how close these packages are to
the root.
Steven J. Long
2014-06-19 21:00:01 UTC
Permalink
Post by Alexandre Rostovtsev
All multilib packages that use pkgconfig, for one thing. (Which means almost
all multilib packages.) Because current crossdev versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting the binary
belonging to pkgconfig[abi_x86_32].
Well I've spent far too long at crossdev code, only to see this and realise
you can simply hard-mask:
cross-i686-pc-linux-gnu/{binutils,gcc,glibc,pkg-config}
in the amd64 multilib profile, unless I'm missing something. You'd be
hard-pushed to install a clashing crossdev with such a mask, afaict.

If you do want to change crossdev[1], afaict you're looking at interaction
between toolchain.eclass (and toolchain-binutils, and likely -funcs),
crossdev and gcc-config. I could well be wrong, as ever. This is just my
preliminary understanding, and maybe it'll provoke a more thorough
explanation.

Crossdev set_links() links the overlay to the tree. toolchain.eclass installs
the versioned links so the internal gcc-bin path is exposed in /usr/bin, in
toolchain_src_install(). For cross-toolchains, it only installs prefixed ones:
dosym ${BINPATH}/${CTARGET}-${x} /usr/bin/${CTARGET}-${x}-${GCC_CONFIG_VER}

For a native compiler it first, in: "${D}"${BINPATH} does:
ln -sf ${CTARGET}-${x} ${x} # unversioned private link
dosym ${BINPATH}/${CTARGET}-${x} /usr/bin/${x}-${GCC_CONFIG_VER}

..then the above. Note that the prefixed link is effectively a race as to
which was installed last, but only in the case of a clash.

gcc-config update_wrappers() makes links in the user's path, though all of
them are run via wrapper.c[2], aka /usr/lib/misc/gcc-config. This is a
multi-switch binary based on argv[0]/$0. However it specifically notes
that it's intended to be used with a PATH-based setup [3]:

/* Find the first file with suitable name in PATH. The idea here is
* that we do not want to bind ourselfs to something static like the
* default profile, or some odd environment variable, but want to be
* able to build something with a non default gcc by just tweaking
* the PATH ... */

crossdev dirs are added to path after system ones in env.d; that's where
gcc-config gets the paths to use from.

crossdev uninstall() removes CTARGET-based links. Note that your native
machine CBUILD == CHOST, also has CTARGET = CHOST, so this would also
be a reason to block/mask according to arch.

I haven't reviewed wrapper.c as well as I'd like: some of it seems a bit
odd but I'd need more time. I did wonder why it doesn't just use
readlink(3P) til I saw that comment.

Anyhow, that all seems a bit pointless when you can just hardmask the
specific crossdev configuration that causes the problem on multilib
profiles, and the same mechanism can be applied elsewhere, as decided
by the arch-team (eg for: o32/n32/n64)

Although canadian-cross and ROOT-based toolchains are another matter.

Regards,
steveL

[1] ie stop installing symlinks in /usr/bin for CTARGET-gcc, as well as
env.d files, and just provide a sh wrapper in each PORTAGE_CONFIGROOT
(= cross-overlay) that can be sourced from /etc/profile or anywhere
else, to add the same settings instead as and when the user chooses.

The most you'd need is the ability to choose whether it takes precedence
over current PATH or not, and that's probably easiest with a variant
'source' ('.' in shell) so cross-builds do, and the profile one doesn't.

[2] http://git.overlays.gentoo.org/gitweb/?p=proj/gcc-config.git;a=blob;f=wrapper.c;hb=HEAD
[3] ll 125-129 h=b00e8187a6063e329049ab9a57023fe9113c598d;hb=HEAD
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
Ian Stakenvicius
2014-06-20 20:20:02 UTC
Permalink
Post by Steven J. Long
Post by Alexandre Rostovtsev
All multilib packages that use pkgconfig, for one thing. (Which
means almost all multilib packages.) Because current crossdev
versions blindly install their
/usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting
the binary belonging to pkgconfig[abi_x86_32].
Well I've spent far too long at crossdev code, only to see this and
cross-i686-pc-linux-gnu/{binutils,gcc,glibc,pkg-config} in the
amd64 multilib profile, unless I'm missing something. You'd be
hard-pushed to install a clashing crossdev with such a mask,
afaict.
If you do want to change crossdev[1], afaict you're looking at
interaction between toolchain.eclass (and toolchain-binutils, and
likely -funcs), crossdev and gcc-config. I could well be wrong, as
ever. This is just my preliminary understanding, and maybe it'll
provoke a more thorough explanation. [ Snip! ]
Thank you for the explanation and research!

Tangental to this, mgorny wrote a little tool yesterday that might
work well as an alternative to crossdev for multilib systems. It
simply wraps all the native toolchain calls with proper -m and
provides the new CTARGETs.

If anybody wants to take a look, this is the link he posted on -dev :

http://git.overlays.gentoo.org/gitweb/?p=dev/mgorny.git;a=blob;f=sys-devel/multilib-gcc-wrapper/multilib-gcc-wrapper-0.ebuild;h=3e304313c0812ffc7da79603e38979fc81a83081;hb=HEAD

Whether or not this suits everyone's needs for an i686 crossdev on
amd64 systems, i don't know. Thoughts?
Greg Turner
2014-06-21 10:40:01 UTC
Permalink
Thoughts [about wrapping gcc, so that non-native multilib-build ABI's can
finally return to a world where [[ ${GCC} != *' '* ]] ]?

TLDR: good idea, I'm strongly in favor of it.

A wrapper would fix horrors like the following, which, last I checked, was
sort-of required* on ABI_X86="*32*" DEFAULT_ABI="amd64" systems with
crossdev i686-pc-linux-gnu targets in ${PATH}:

--- /usr/portage/eclass/cmake-utils.eclass 2014-05-06
08:31:17.000000000 -0700
+++ /usr/local/portage/gogogmt/eclass/cmake-utils.eclass 2014-06-01
12:36:40.753904765 -0700
@@ -373,14 +373,115 @@ enable_cmake-utils_src_prepare() {
}
+
+# retrieve a variable (i.e.: FOO) using i.e.: tc_getFOO,
+# unless the variable is already defined to a nonempty value,


+# in which case, it is returned unaltered


+_cmake-utils_tc_get_if_needed() {


+ local v="$1" s


+ if [[ ${!v} ]] ; then
+ s="${!v}"
+ else
+ s=$(tc-get${v})
+ fi
+ echo "${s}"
+}

+ +


+# trim the preceeding and trailing whitespace from a string,
+# leaving inner spaces intact
+_cmake-utils_trimmed() {
+ local s="$*"
+ while [[ ${s:0:1} =~ [[:space:]] ]] ; do
+ s="${s:1}"
+ done
+ while [[ ${s:$((${#s}-1))} =~ [[:space:]] ]] ; do
+ s="${s:0:$((${#s}-1))}"
+ done
+ echo "${s}"
+}
+
+# given a string, retrieve the first word (consisting of non-space
characters)
+# after trimming any leading whitespace.
+_cmake-utils_first_word_of() {
+ local s=$(_cmake-utils_trimmed "$*")
+ echo "${s%% *}"
+}
+
+# given a string, retrieve the rest of the string after the first word,
+# with any whitespace trimmed, retaining any whitespace between words
+# (except whitespace that was between the first and second word)
+_cmake-utils_subsequent_words_of() {
+ local s=$(_cmake-utils_trimmed "$*")
+ if [[ ${s} ]] ; then
+ s="${s#$(_cmake-utils_first_word_of "${s}")}"
+ echo "$(_cmake-utils_trimmed "${s}")"
+ else
+ echo ""
+ fi
+}
+
+# given a variable-name, retrieve it with _cmake-utils_tc_get_if_needed
+# and then from that retreive the first word
+_cmake-utils_get_first_word() {
+ local s=$(_cmake-utils_tc_get_if_needed "$1")
+ echo "$(_cmake-utils_first_word_of "${s}")"
+}
+
+# given a command-containing variable as the first argument,
+# uses _cmake-utils_tc_get_if_needed to retrieve the var, and then
+# returns type -P of the first word of it, ignoring rest.
+_cmake-utils_get_first_word_executable() {
+ local w=$(_cmake-utils_get_first_word "$1")
+ if [[ ${w} ]] ; then
+ echo "$(type -P "${w}")"
+ else
+ echo ""
+ fi
+}
+
+# fetches any additional bits of a variable not included in
+# _cmake-utils_get_first_word. takes the variable name as an input.
+# May return an empty string.
+_cmake-utils_get_subsequent_words() {
+ local s=$(_cmake-utils_tc_get_if_needed "$1")
+ echo "$(_cmake-utils_subsequent_words_of "${s}")"
}

# @VARIABLE: mycmakeargs
@@ -433,17 +534,46 @@ enable_cmake-utils_src_configure() {

# Prepare Gentoo override rules (set valid compiler, append
CPPFLAGS etc.)
local build_rules=${BUILD_DIR}/gentoo_rules.cmake
+
+ local _CMAKE_C_COMPILER=$(_cmake-utils_get_first_word_executable CC)
+ local _CCORPHANS=$(_cmake-utils_get_subsequent_words CC)
+ local _CMAKE_C_COMPILER_ARG1_COMMAND=
+ local _CMAKE_CXX_COMPILER=$(_cmake-utils_get_first_word_executable
CXX)
+ local _CXXORPHANS=$(_cmake-utils_get_subsequent_words CXX)
+ local _CMAKE_CXX_COMPILER_ARG1_COMMAND=
+
+ # This seems fairly rediculous to me, but if C{,XX}ORPHANS is
nonempty, then, in order to prevent
+ # duplication of the first argument, we must handle the first word
specially
+ if [[ ${_CCORPHANS} ]] ; then
+ _CMAKE_C_COMPILER_ARG1_COMMAND="SET (CMAKE_C_COMPILER_ARG1
$(_cmake-utils_first_word_of "${_CCORPHANS}") CACHE STRING \"C compiler
first argument\" FORCE)"
+ _CCORPHANS=$(_cmake-utils_subsequent_words_of
"${_CCORPHANS}")
+ fi
+ if [[ ${_CXXORPHANS} ]] ; then
+ _CMAKE_CXX_COMPILER_ARG1_COMMAND="SET
(CMAKE_CXX_COMPILER_ARG1 $(_cmake-utils_first_word_of "${_CXXORPHANS}")
CACHE STRING \"C++ compiler first argument\" FORCE)"
+ _CXXORPHANS=$(_cmake-utils_subsequent_words_of
"${_CXXORPHANS}")
+ fi
+
cat > "${build_rules}" <<- _EOF_
- SET (CMAKE_AR $(type -P $(tc-getAR)) CACHE FILEPATH
"Archive manager" FORCE)
- SET (CMAKE_ASM_COMPILE_OBJECT "<CMAKE_C_COMPILER> <DEFINES>
${CFLAGS} <FLAGS> -o <OBJECT> -c <SOURCE>" CACHE STRING "ASM compile
command" FORCE)
- SET (CMAKE_C_COMPILER $(type -P $(tc-getCC)) CACHE FILEPATH
"C compiler" FORCE)
- SET (CMAKE_C_COMPILE_OBJECT "<CMAKE_C_COMPILER> <DEFINES>
${CPPFLAGS} <FLAGS> -o <OBJECT> -c <SOURCE>" CACHE STRING "C compile
command" FORCE)
- SET (CMAKE_CXX_COMPILER $(type -P $(tc-getCXX)) CACHE
FILEPATH "C++ compiler" FORCE)
- SET (CMAKE_CXX_COMPILE_OBJECT "<CMAKE_CXX_COMPILER>
<DEFINES> ${CPPFLAGS} <FLAGS> -o <OBJECT> -c <SOURCE>" CACHE STRING "C++
compile command" FORCE)
- SET (CMAKE_RANLIB $(type -P $(tc-getRANLIB)) CACHE FILEPATH
"Archive index generator" FORCE)
- SET (PKG_CONFIG_EXECUTABLE $(type -P $(tc-getPKG_CONFIG))
CACHE FILEPATH "pkg-config executable" FORCE)
+ SET (CMAKE_C_COMPILER "${_CMAKE_C_COMPILER}" CACHE FILEPATH
"C compiler" FORCE)
+ ${_CMAKE_C_COMPILER_ARG1_COMMAND}
+ SET (CMAKE_CXX_COMPILER "${_CMAKE_CXX_COMPILER}" CACHE
FILEPATH "C++ compiler" FORCE)
+ ${_CMAKE_CXX_COMPILER_ARG1_COMMAND}
+ SET (CMAKE_AR "$(_cmake-utils_get_first_word_executable
AR)" CACHE FILEPATH "Archive manager" FORCE)
+ SET (CMAKE_RANLIB "$(_cmake-utils_get_first_word_executable
RANLIB)" CACHE FILEPATH "Archive index generator" FORCE)
+ SET (CMAKE_ASM_COMPILE_OBJECT "<CMAKE_C_COMPILER>
${_CCORPHANS}${_CCORPHANS:+ }<DEFINES> ${CFLAGS}${CFLAGS:+ }<FLAGS> -o
<OBJECT> -c <SOURCE>" CACHE STRING "ASM compile command" FORCE)
+ SET (CMAKE_C_COMPILE_OBJECT "<CMAKE_C_COMPILER>
${_CCORPHANS}${_CCORPHANS:+ }<DEFINES> ${CFLAGS}${CFLAGS:+ }<FLAGS> -o
<OBJECT> -c <SOURCE>" CACHE STRING "C compile command" FORCE)
+ SET (CMAKE_CXX_COMPILE_OBJECT "<CMAKE_CXX_COMPILER>
${_CXXORPHANS}${_CXXORPHANS:+ }<DEFINES> ${CXXFLAGS}${CXXFLAGS:+ }<FLAGS>
-o <OBJECT> -c <SOURCE>" CACHE STRING "C++ compile command" FORCE)
+ SET (CMAKE_C_LINK_EXECUTABLE "<CMAKE_C_COMPILER>
${_CCORPHANS}${_CCORPHANS:+ }<FLAGS> ${LDFLAGS}${LDFLAGS:+
}<CMAKE_C_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET> <LINK_LIBRARIES>"
CACHE STRING "C link command for executables" FORCE)
+ SET (CMAKE_C_CREATE_SHARED_LIBRARY "<CMAKE_C_COMPILER>
${_CCORPHANS}${_CCORPHANS:+ }<CMAKE_SHARED_LIBRARY_C_FLAGS>
<LANGUAGE_COMPILE_FLAGS> <LINK_FLAGS> ${LDFLAGS}${LDFLAGS:+
}<CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS>
<CMAKE_SHARED_LIBRARY_SONAME_C_FLAG><TARGET_SONAME> -o <TARGET> <OBJECTS>
<LINK_LIBRARIES>" CACHE STRING "C shared library link command" FORCE)
+ SET (CMAKE_C_CREATE_SHARED_MODULE
"\${CMAKE_C_CREATE_SHARED_LIBRARY}" CACHE STRING "C shared module link
command" FORCE)
+ SET (CMAKE_CXX_LINK_EXECUTABLE "<CMAKE_CXX_COMPILER>
${_CXXORPHANS}${_CXXORPHANS:+ }<FLAGS> ${LDFLAGS}${LDFLAGS:+
}<CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET>
<LINK_LIBRARIES>" CACHE STRING "C++ link command for executables" FORCE)
+ SET (CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_CXX_COMPILER>
${_CXXORPHANS}${_CXXORPHANS:+ }<CMAKE_SHARED_LIBRARY_CXX_FLAGS>
<LANGUAGE_COMPILE_FLAGS> <LINK_FLAGS> ${LDFLAGS}${LDFLAGS:+
}<CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS>
<CMAKE_SHARED_LIBRARY_SONAME_CXX_FLAG><TARGET_SONAME> -o <TARGET> <OBJECTS>
<LINK_LIBRARIES>" CACHE STRING "C++ shared library link command" FORCE)
+ SET (CMAKE_CXX_CREATE_SHARED_MODULE
"\${CMAKE_CXX_CREATE_SHARED_LIBRARY}" CACHE STRING "C++ shared module link
command" FORCE)
+ SET (PKG_CONFIG_EXECUTABLE
"$(_cmake-utils_get_first_word_executable PKG_CONFIG)" CACHE FILEPATH
"pkg-config executable" FORCE)
_EOF_

has "${EAPI:-0}" 0 1 2 && ! use prefix && EPREFIX=

Perhaps a straw-man argument, since, the above code could be made more
beautiful, and then it would not be so ugly :P

But, it's not just that cmake can't properly handle a C{C,XX} variables
with spaces in them**. There are a bunch of similar places where we had to
patch Makefiles and autotools to wedge GCC="foo bar" support in.

Without hyperbole, I can honestly say, it's a lame and inelegant hack to
put an /argument/ into GCC. -m${foo} is a CFLAG, not a compiler.
"Everyone" knows GCC is a variable that names an executable -- but in
multilib-build, for some god-awful reason, we've made it into some other
thing, a hybrid-psuedo-thing that really only makes sense in a shell
script, and even then, only in an improperly coded shell script :).

Even if that rubicon was already crossed by ccache, distcc, etc, there is a
reason none of them /require/ it. It's just asking for trouble. We'll
just keep wasting time fixing subtle bugs like the above, because of it
until we break down and fix it. I'd lay pretty long odds that five years
from now we are still clinging to it, should we decide to stick with the
status quo for now. So why not bite the bullet and save ourselves the
hassle?

-gmt

* perhaps today the problem is mitigated, due to the same
gcc-${pseudo-CHOST} executables now existing for upstream multilib-build
pseudo-targets -- but, I'll bet, not solved. Usually trouble only crops up
for larger framework builds with lots of sub-dirs, like qt or mysql-server,
and manifests as incorrect attempts to link native libraries in -- which
the compiler might or might not manage to save you from at the last
moment... also note that patch is mocked up for easy reading -- it won't
apply. See my overlay for the real code. Finally, sorry for the HTML --
I'm writing from the gmail ajax which obstinately refuses to allow
plain-text inline patches.

** IIRC, it would be more precise to say something like "cmake's in-built
hacks for the CC='exectutable argument' circumstance are subtly
incompatible with our means of injecting the portage compiler values into
cmake (or so things used to be; I suspect the problem still lurks to some
extent), and that this ultimately manifests as confusing, intermittent
build failures and correctly issued QA warnings in larger
multilib-build-based ebuilds (or will in the near future; I actually
wouldn't know as I run with the above patches :P), esp. in cases where a
${CC}-${CHOST} or ${CXX}-${CHOST} executables are to be found in ${PATH}."
But if any unlucky person is still reading, they can see fully why I
didn't bother to say all that :P
Michał Górny
2014-06-21 20:50:02 UTC
Permalink
Dnia 2014-06-21, o godz. 03:31:30
Post by Greg Turner
Thoughts [about wrapping gcc, so that non-native multilib-build ABI's can
finally return to a world where [[ ${GCC} != *' '* ]] ]?
TLDR: good idea, I'm strongly in favor of it.
A wrapper would fix horrors like the following, which, last I checked, was
sort-of required* on ABI_X86="*32*" DEFAULT_ABI="amd64" systems with
[...]
But, it's not just that cmake can't properly handle a C{C,XX} variables
with spaces in them**. There are a bunch of similar places where we had to
patch Makefiles and autotools to wedge GCC="foo bar" support in.
I wasn't aware of that.

I honestly would love if toolchain cooperated with us on bringing this.
However, I'm a bit pessimistic about their willing. For one, we'd first
have to fix all multilib arches to use different CHOSTs for
different ABIs -- and so far, people prefer inventing some custom hacks
in the name of keeping status quo.

As far as multilib builds are concerned, I guess I could create
${T}/bin with proper wrappers. We do similar thing in python-r1 suite
already to make sure that all 'python[23]' calls behave correctly. Of
course, this will still require fixing CHOSTs and work only for
multilib ebuilds. But on the other hand, it would allow us to avoid
masking crossdev.
--
Best regards,
Micha³ Górny
Steven J. Long
2014-08-01 08:40:02 UTC
Permalink
Post by Ian Stakenvicius
Post by Steven J. Long
Well I've spent far too long at crossdev code, only to see this and
cross-i686-pc-linux-gnu/{binutils,gcc,glibc,pkg-config} in the
amd64 multilib profile, unless I'm missing something. You'd be
hard-pushed to install a clashing crossdev with such a mask,
afaict.
If you do want to change crossdev[1], afaict you're looking at
interaction between toolchain.eclass (and toolchain-binutils, and
likely -funcs), crossdev and gcc-config. I could well be wrong, as
ever. This is just my preliminary understanding, and maybe it'll
provoke a more thorough explanation. [ Snip! ]
Thank you for the explanation and research!
YW :-) shove autotools.eclass (and supporting) in there too, including
multiprocessing which had a simply painful attempt at cleverness.
I mentioned it in #gentoo-embedded when i saw it, so hopefully it'll
be corrected soon. (bashpid() function: if you can't see why it's
painfully embarrassing, /join #bash and ask.)
Post by Ian Stakenvicius
Tangental to this, mgorny wrote a little tool yesterday that might
work well as an alternative to crossdev for multilib systems. It
simply wraps all the native toolchain calls with proper -m and
provides the new CTARGETs.
<dead url>
Whether or not this suits everyone's needs for an i686 crossdev on
amd64 systems, i don't know. Thoughts?
It's more layer upon layer, I'm afraid. Though that file's gone from
the repo, so I imagine it's already made its way to join the rest of
the misguided hackery that is multilib. Still, it's good that his
bash has come on, though he's still too tricksy for his own good;
likely trying to emulate Frysinger, another one who needs a nice
lie-down sometimes, instead of banging out more. Idiot house-"styles"
will do that to you, as will C++.

I don't know why we can't just mask cross-*/whatever in the multilib
profile, instead of more talk of "masking crossdev" with a heavy
heart.

Nor do know if that's been done already, as I just found that the
profiles directory Changelog stopped in 2013, for some reason, and
I don't have time to chase the files right now.

Sorry for delay, been away and then busy. I was hoping to read
something more than "mask crossdev" yet again, when I got back.

Regards,
igli.
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
Ian Stakenvicius
2014-08-01 14:40:01 UTC
Permalink
Post by Steven J. Long
Post by Ian Stakenvicius
Post by Steven J. Long
Well I've spent far too long at crossdev code, only to see this
cross-i686-pc-linux-gnu/{binutils,gcc,glibc,pkg-config} in the
amd64 multilib profile, unless I'm missing something. You'd be
hard-pushed to install a clashing crossdev with such a mask, afaict.
If you do want to change crossdev[1], afaict you're looking at
interaction between toolchain.eclass (and toolchain-binutils,
and likely -funcs), crossdev and gcc-config. I could well be
wrong, as ever. This is just my preliminary understanding, and
maybe it'll provoke a more thorough explanation. [ Snip! ]
Thank you for the explanation and research!
YW :-) shove autotools.eclass (and supporting) in there too,
including multiprocessing which had a simply painful attempt at
cleverness. I mentioned it in #gentoo-embedded when i saw it, so
hopefully it'll be corrected soon. (bashpid() function: if you
can't see why it's painfully embarrassing, /join #bash and ask.)
Post by Ian Stakenvicius
Tangental to this, mgorny wrote a little tool yesterday that
might work well as an alternative to crossdev for multilib
systems. It simply wraps all the native toolchain calls with
proper -m and provides the new CTARGETs.
<dead url>
Whether or not this suits everyone's needs for an i686 crossdev
on amd64 systems, i don't know. Thoughts?
It's more layer upon layer, I'm afraid. Though that file's gone
from the repo, so I imagine it's already made its way to join the
rest of the misguided hackery that is multilib. Still, it's good
that his bash has come on, though he's still too tricksy for his
own good; likely trying to emulate Frysinger, another one who needs
a nice lie-down sometimes, instead of banging out more. Idiot
house-"styles" will do that to you, as will C++.
I don't know why we can't just mask cross-*/whatever in the
multilib profile, instead of more talk of "masking crossdev" with a
heavy heart.
Nor do know if that's been done already, as I just found that the
profiles directory Changelog stopped in 2013, for some reason, and
I don't have time to chase the files right now.
Sorry for delay, been away and then busy. I was hoping to read
something more than "mask crossdev" yet again, when I got back.
Regards, igli.
It's a package in the tree now, actually --
sys-devel/multilib-gcc-wrapper ; it wraps the native toolchain
appropriately for the different ABI's that this toolchain supports, so
that ie an i686-*-gcc call is implemented properly via the native gcc
instead of a crossdev one. Note, it *also* works on multilib
crossdev, too! I have a ppc crossdev installed on my amd64 host box
and multilib-gcc-wrapper allows it to build for both ppc32 and ppc64
ABIs just fine (presumably; i don't have a ppc32 or ppc64 native
system to actually do runtime tests).

Back to the comment on masking -- would a cross-emerge (which i think
uses the target's profile, right?) end up p.masking its own toolchain?
If it did, would that actually break anything (i'm thinking no since
it's the crossdev that manages toolchain updates for the target rather
than cross-emerge)?? I agree that masks should be minimized, at most
masking the conflicting cross-* packages in a profile. However if
this causes issues within cross-emerge too, then perhaps adjusting the
crossdev tool to warn or error would suffice when a target that will
conflict with the native toolchain is requested.
Steven J. Long
2014-08-01 18:00:02 UTC
Permalink
Post by Ian Stakenvicius
Post by Steven J. Long
I don't know why we can't just mask cross-*/whatever in the
multilib profile, instead of more talk of "masking crossdev" with a
heavy heart.
Nor do know if that's been done already, as I just found that the
profiles directory Changelog stopped in 2013, for some reason, and
I don't have time to chase the files right now.
Sorry for delay, been away and then busy. I was hoping to read
something more than "mask crossdev" yet again, when I got back.
Back to the comment on masking -- would a cross-emerge (which i think
uses the target's profile, right?) end up p.masking its own toolchain?
No. The cross-* part is an overlay on CBUILD (ie the machine building the
software; for a native build this is the same as CHOST in make.conf.)
Post by Ian Stakenvicius
I agree that masks should be minimized, at most
masking the conflicting cross-* packages in a profile. However if
this causes issues within cross-emerge too, then perhaps adjusting the
crossdev tool to warn or error would suffice when a target that will
conflict with the native toolchain is requested.
Well that should happen too: it's a trivial patch, which again has
already been discussed in #-embedded. Either vapier gets on and does
it now he's back, or someone else will, for an arch they care about. I
can't see him caring if it's correct; and after all the mask on the
"overlay" itself (which is on CBUILD, remember) is only possible due
to the separation inherent in the crossdev design.

Nowadays people like to call that "belt'n'braces" or something. When I
learnt to code it was called "common-sense." Discussing it wouldn't
even arise: it goes without saying.

Regards,
igli.
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
Ryan Hill
2014-06-18 04:40:02 UTC
Permalink
On Tue, 17 Jun 2014 10:17:26 -0400
Post by Joshua Kinard
I'm a member of toolchain, but that's mostly historical because I used to
play with a lot of the cross-compile stuff for MIPS and Sparc. Mike and
Ryan are the two primaries in toolchain right now. If they don't see a
problem with crossdev right now, then I do have to question just how big of
a problem this really is.
Thanks for being a voice of reason in this thread.

I just wanted to say that I'm not involved with crossdev development so any
opinions I might be spewing are mine alone and shouldn't be considered an
"upstream" response.
--
Ryan Hill psn: dirtyepic_sk
gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E 7F92 ED38 BD49 957A 8463
Joshua Kinard
2014-06-17 14:10:01 UTC
Permalink
Post by hasufell
Post by Joshua Kinard
Post by hasufell
Post by Joshua Kinard
How big of a patch would this change require to the existing crossdev ebuild?
Probably quite trivial, but since vapier said "bs" to that proposal
(translates to "bullshit" I guess) I'll not put any work into that.
So there we go. If you are cool, you can just say "bs", vanish and leave
stable arch in a broken state.
Not even QA cares. Great. I'll try to get it on the next council agenda
then.
So you just take your ball and go home then? That's not how it works.
Create the patch, and file it as a bug. Then, raise awareness on the ML.
That's how development works. If your patch is reasonable and doesn't break
things, odds are likely it'll push the other members of toolchain to
consider incorporating it.
Equally using the Council as a hammer all the time doesn't work in the
long-term, either. If you whip a patch up, however, then not only could you
raise this at the next council meeting, but additionally state you've gone
that extra mile and created a patch that addresses the problem.
That's taking the ball and putting it into the goal.
No, that's not how opensource works. You don't work on things after
"upstream" said "not interested".
https://bugs.gentoo.org/show_bug.cgi?id=504824
"upstream" didn't say anywhere in that bug that they weren't interested.
They countered your reasoning with a technical argument. QA even states
that you need to file separate bugs for the various build failures. You
could set up a master TRACKER bug for these crossdev-related issues, and
then link in any existing bugs or create new ones tied to it, and that way,
you have things documented.
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
hasufell
2014-06-17 14:40:03 UTC
Permalink
Post by Joshua Kinard
"upstream" didn't say anywhere in that bug that they weren't interested.
They countered your reasoning with a technical argument. QA even states
that you need to file separate bugs for the various build failures. You
could set up a master TRACKER bug for these crossdev-related issues, and
then link in any existing bugs or create new ones tied to it, and that way,
you have things documented.
I appreciate that you want to help, but I'm not sure how many times I
have to explain to you that the PATH idea was neglected by the embedded
gentoo project lead. Check the history of this thread, it starts here:
https://groups.google.com/d/msg/linux.gentoo.dev/KZykx1DAJyM/YCMVUt4CzjUJ

So again, I am not doing work that goes diametral to what the project
lead wants and I am not going to fork crossdev.


I have proposed numerous ways to communicate this problem to the user
without touching any of the precious toolchain/embedded packages. If no
one responds there, I'll just pick one and apply it.
Joshua Kinard
2014-06-17 15:10:02 UTC
Permalink
Post by hasufell
Post by Joshua Kinard
"upstream" didn't say anywhere in that bug that they weren't interested.
They countered your reasoning with a technical argument. QA even states
that you need to file separate bugs for the various build failures. You
could set up a master TRACKER bug for these crossdev-related issues, and
then link in any existing bugs or create new ones tied to it, and that way,
you have things documented.
I appreciate that you want to help, but I'm not sure how many times I
have to explain to you that the PATH idea was neglected by the embedded
https://groups.google.com/d/msg/linux.gentoo.dev/KZykx1DAJyM/YCMVUt4CzjUJ
Post by Joshua Kinard
when you run `crossdev i686-pc-linux-gnu`, it owns that tuple. that includes
i686-pc-linux-gnu-pkg-config.
if we're going to have the multilib system lie and use a tuple that doesn't
(1) override all the vars so they point back to the real toolchain.
this doesn't scale when you consider helper tools like the legacy sdl-config
or the extended set of tools that binutils/gcc/etc... install. it's mitigated
by the fact the set of vars in play most of the time is low.
(2) use tuples with loaded vendor fields to reduce the chance of collisions.
e.g. having an ABI=amd64 system use i686-gentoo%multilib-linux-gnu instead of
i686-pc-linux-gnu would defeat any automatic path searches.
as i pointed out elsewhere in this thread, the problem is that multilib relies
on automatic detection of the toolchain *failing* so that it falls back to the
native value. in other words, when you run `./configure --host=i686-pc-linux-
gnu`, it tries to find e.g. i686-pc-linux-gnu-ar. it doesn't exist so the
fallback is used (plain `ar`). multilib is using these tuples so that the
standard checks (autoconf/eclasses/etc...) trigger in the right ways for the
cpu/os/userland combinations.
since crossdev installs a full proper toolchain for the target, the one
multilib was using to lie now exists and its toolchain is used instead.
pkg-config does need fixing in some way. we already know this. it's why the
multilib eclasses currently set PKG_CONFIG_XXX vars -- preciously so the
correct ABI dir is utilized. and this breaks when using some build systems
(like scons) where the env gets blown away (although we also know scons
sucks).
So again, I am not doing work that goes diametral to what the project
lead wants and I am not going to fork crossdev.
I have proposed numerous ways to communicate this problem to the user
without touching any of the precious toolchain/embedded packages. If no
one responds there, I'll just pick one and apply it.
And what I am trying to tell you is that making hardmask threats don't solve
the core problem. You're threatening to to start a mask/unmask war that
probably won't end well for you. Mike has, in all of the messages I have in
the thread, provided clear technical explanations for why crossdev operates
the way it does, and that it isn't the source of these problems.

Provide a technical counter-argument to that or propose a solution that
people can agree on and you're going to find people are a LOT more willing
to stand with you on fixing the perceived problem.
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
hasufell
2014-06-17 15:20:01 UTC
Permalink
Post by Joshua Kinard
Post by hasufell
I have proposed numerous ways to communicate this problem to the user
without touching any of the precious toolchain/embedded packages. If no
one responds there, I'll just pick one and apply it.
And what I am trying to tell you is that making hardmask threats don't solve
the core problem.
You missed my response to rich then.
hasufell
2014-06-17 15:40:01 UTC
Permalink
Post by Joshua Kinard
Provide a technical counter-argument to that or propose a solution that
people can agree on and you're going to find people are a LOT more willing
to stand with you on fixing the perceived problem.
I start to think here is some confusion going on. We already proposed
solutions and there was no agreement. There are no more possibilities,
except rewriting crossdev, doing non-trivial hackery on toolchain
eclasses or stashing the whole multilib idea.
There were some ideas about a few make.profile hacks, but they didn't
happen either.

I reopened this thread to make clear that we need to at least
communicate this problem to the user and I proposed more ways than just
hardmasking.

Also, I am not going to work on solutions that have no agreement
whatsoever. It does not make sense.

So, I have no idea what you expect me to do that did not already happen.
hasufell
2014-06-17 12:50:01 UTC
Permalink
Post by Joshua Kinard
Equally using the Council as a hammer all the time doesn't work in the
long-term, either.
This is exactly the case where the council has to step in to solve
global issues and those between projects (here it is embedded gentoo
project and multilib project).
Joshua Kinard
2014-06-17 14:40:03 UTC
Permalink
Post by hasufell
Post by Joshua Kinard
Equally using the Council as a hammer all the time doesn't work in the
long-term, either.
This is exactly the case where the council has to step in to solve
global issues and those between projects (here it is embedded gentoo
project and multilib project).
multilib needs a lot of things fixed. Everyone is pretty much aware that we
don't have solid support for multilib, just enough hacks in place that it
seems to JustWork(TM), for now. But I think this is a wider issue in a lot
of GNU packages anyways, as the design mentality for many of them assumed
binary distro installs w/ a single ABI selected.

I am not convinced that a lot of the packages in the tree are being broken
by crossdev, when they are probably making assumptions about the build
environment that aren't multilib-safe. If that's the case, it's those build
systems that need patching, not hard-masking crossdev.

That all said, you still haven't put forth a really convincing argument that
people can agree on. And if you can't convince normal devs, do you think
you can convince council members? What if you're rebuffed at a council
meeting on this issue? What do you do then?

Not everything is a nail that the council needs to whack with a hammer.
Sometimes, you need an impact driver, a blow torch, or a simple Robertson
square-drive screwdriver.
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
Ian Stakenvicius
2014-06-17 13:30:02 UTC
Permalink
Post by Joshua Kinard
Can $PATH be configured via our existing eselect tool to
enable/disable the crossdev paths when needed?
Technically it could but not really ; PATH is an environment thing,
AFAIK all eselect could do is trigger the addition or removal of
entries in /etc/env.d/ , but after that happens one would still need
to 'env-update && . /etc/profile' to get the changes.

Similarly I don't think using an eselect tool to bring the crossdev
tools into the default path (via symlinks) is a great idea either; yes
it'd allow users to un-eselect the crossdev tools when errors occur,
but the errors would still occur every time a user forgets to do this
first.

It would be easier to update PATH yourself manually in the shell, I
expect; perhaps a quick utility could do that for you (maybe opening
up a crossdev-ready subshell) so you don't have to remember the path.
Michał Górny
2014-06-17 14:30:03 UTC
Permalink
Dnia 2014-06-17, o godz. 09:25:32
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Post by Joshua Kinard
Can $PATH be configured via our existing eselect tool to
enable/disable the crossdev paths when needed?
Technically it could but not really ; PATH is an environment thing,
AFAIK all eselect could do is trigger the addition or removal of
entries in /etc/env.d/ , but after that happens one would still need
to 'env-update && . /etc/profile' to get the changes.
Similarly I don't think using an eselect tool to bring the crossdev
tools into the default path (via symlinks) is a great idea either; yes
it'd allow users to un-eselect the crossdev tools when errors occur,
but the errors would still occur every time a user forgets to do this
first.
It would be easier to update PATH yourself manually in the shell, I
expect; perhaps a quick utility could do that for you (maybe opening
up a crossdev-ready subshell) so you don't have to remember the path.
+1. That's how sane tools work.
--
Best regards,
Micha³ Górny
Joshua Kinard
2014-06-17 14:40:02 UTC
Permalink
Post by Michał Górny
Dnia 2014-06-17, o godz. 09:25:32
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Post by Joshua Kinard
Can $PATH be configured via our existing eselect tool to
enable/disable the crossdev paths when needed?
Technically it could but not really ; PATH is an environment thing,
AFAIK all eselect could do is trigger the addition or removal of
entries in /etc/env.d/ , but after that happens one would still need
to 'env-update && . /etc/profile' to get the changes.
Similarly I don't think using an eselect tool to bring the crossdev
tools into the default path (via symlinks) is a great idea either; yes
it'd allow users to un-eselect the crossdev tools when errors occur,
but the errors would still occur every time a user forgets to do this
first.
It would be easier to update PATH yourself manually in the shell, I
expect; perhaps a quick utility could do that for you (maybe opening
up a crossdev-ready subshell) so you don't have to remember the path.
+1. That's how sane tools work.
+1 for providing a technical reason why my off-the-cuff idea won't work.
This is what we need to address this perceived problem: Ideas. Weed out the
ones that don't work and figure out what can work, then apply it.
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
Patrick Lauer
2014-06-16 23:30:02 UTC
Permalink
Post by hasufell
Post by Joshua Kinard
What about those of us who have been using crossdev to generate
cross-compilers for years w/o issue, because we run non-multilib?
Hardmasking crossdev to solve multilib problems doesn't accomplish anything,
other than just irk us. Why not hardmask the multilib stuff instead and
leave crossdev alone?
Hardmask half of the tree instead of a single package? Does not sound
reasonable. The fallout will be _huge_ for users who already run
multilib. You will basically get an emerge dump of 500+ blockers.
That's not a bug ;)

Since I can't figure out how any of these multibuilds work (that's no
longer an ebuild ...) I'm not opposed to have them removed from my
workflow.
Ian Stakenvicius
2014-06-16 20:30:02 UTC
Permalink
Post by Joshua Kinard
Post by hasufell
So I don't see what else we can do here other than taking more
radical steps to INFORM users of these possible breakages... and
that's exactly what a hardmask is for.
What about those of us who have been using crossdev to generate
cross-compilers for years w/o issue, because we run non-multilib?
Hardmasking crossdev to solve multilib problems doesn't accomplish
anything, other than just irk us. Why not hardmask the multilib
stuff instead and leave crossdev alone?
well, we could hardmask in the multilib profiles... but that's a bit
of a digression
Jeroen Roovers
2014-06-16 21:50:04 UTC
Permalink
On Mon, 16 Jun 2014 16:27:19 -0400
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Post by Joshua Kinard
Post by hasufell
So I don't see what else we can do here other than taking more
radical steps to INFORM users of these possible breakages... and
that's exactly what a hardmask is for.
What about those of us who have been using crossdev to generate
cross-compilers for years w/o issue, because we run non-multilib?
Hardmasking crossdev to solve multilib problems doesn't accomplish
anything, other than just irk us. Why not hardmask the multilib
stuff instead and leave crossdev alone?
well, we could hardmask in the multilib profiles... but that's a bit
of a digression
OK, let's sum it up.

We have multilib users.

We have crossdev users.

Some multilib users are crossdev users.

Some multilib users who are crossdev users have built a cross-toolchain.

Some multilib users who have built a cross-toolchain experience bug
#500338.

Masking crossdev would cause issues for all crossdev users.

Masking multilib would cause issues for all multilib users.

Masking crossdev on multilib profiles would cause issues for all users
of both crossdev and multilib who haven't built a cross-toolchain that
irks multilib

Masking crossdev on multilib profiles would also cause issues for all
users of both crossdev and multilib who haven't built a cross-toolchain
at all but now find they want to and run into bug #500338.

In short, masking crossdev in any of the above ways results in very
little progress, and is detrimental to solving the issue since the mask
would prevent testing on a wide range of platforms because of the
inconvenience that masking causes, deterring people from even trying.

jer
Joshua Kinard
2014-06-17 00:10:01 UTC
Permalink
Post by Jeroen Roovers
On Mon, 16 Jun 2014 16:27:19 -0400
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Post by Joshua Kinard
Post by hasufell
So I don't see what else we can do here other than taking more
radical steps to INFORM users of these possible breakages... and
that's exactly what a hardmask is for.
What about those of us who have been using crossdev to generate
cross-compilers for years w/o issue, because we run non-multilib?
Hardmasking crossdev to solve multilib problems doesn't accomplish
anything, other than just irk us. Why not hardmask the multilib
stuff instead and leave crossdev alone?
well, we could hardmask in the multilib profiles... but that's a bit
of a digression
OK, let's sum it up.
We have multilib users.
We have crossdev users.
Some multilib users are crossdev users.
Some multilib users who are crossdev users have built a cross-toolchain.
Some multilib users who have built a cross-toolchain experience bug
#500338.
Masking crossdev would cause issues for all crossdev users.
Masking multilib would cause issues for all multilib users.
Masking crossdev on multilib profiles would cause issues for all users
of both crossdev and multilib who haven't built a cross-toolchain that
irks multilib
Masking crossdev on multilib profiles would also cause issues for all
users of both crossdev and multilib who haven't built a cross-toolchain
at all but now find they want to and run into bug #500338.
In short, masking crossdev in any of the above ways results in very
little progress, and is detrimental to solving the issue since the mask
would prevent testing on a wide range of platforms because of the
inconvenience that masking causes, deterring people from even trying.
jer
+1
--
Joshua Kinard
Gentoo/MIPS
***@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us. And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic
Jeroen Roovers
2014-06-16 19:50:03 UTC
Permalink
On Mon, 16 Jun 2014 19:31:58 +0000
Post by hasufell
Also check the history of this thread for a few proposed solutions.
The history of this thread and the history of gx86-multilib and
crossdev development suggest that crossdev was doing nothing wrong until
gx86-multilib came around and a problem was found between them. Masking
either for the benefit of the other would be, and let me quote the
history of this thread out of context just to fit in with the tone and
mode this sub-thread has taken, "asinine".


jer
hasufell
2014-06-16 20:30:02 UTC
Permalink
Post by Steev Klimaszewski
while I agree that temporarily adding the cross
compiler(s) to the PATH is easy, for some of us, it's easier to allow
Gentoo to do so.
I'm not sure if that is reason enough to cause the current breakage
crossdev and multilib are in.
Ryan Hill
2014-06-16 02:30:01 UTC
Permalink
On Sun, 15 Jun 2014 20:35:53 +0000
Post by hasufell
Post by Steven J. Long
"I'll see you when you get there, if you ever get there.."
No improvements so far. I am going to hardmask sys-devel/crossdev,
unless someone can explain why we are still in broken stage.
Do that and we'll have to take you out behind the woodshed.
--
Ryan Hill psn: dirtyepic_sk
gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E 7F92 ED38 BD49 957A 8463
hasufell
2014-06-16 13:30:01 UTC
Permalink
Post by Ryan Hill
On Sun, 15 Jun 2014 20:35:53 +0000
Post by hasufell
Post by Steven J. Long
"I'll see you when you get there, if you ever get there.."
No improvements so far. I am going to hardmask sys-devel/crossdev,
unless someone can explain why we are still in broken stage.
Do that and we'll have to take you out behind the woodshed.
If you think having broken packages for months in stable arch is ok,
then you are wrong.

And btw., your funny threats don't impress me anymore.

I'll bring this up to the council agenda if you like. This is a
non-trivial tree-wide problem and if toolchain keeps ignoring it, then I
will hardmask the thing.
Ryan Hill
2014-06-17 05:00:02 UTC
Permalink
On Mon, 16 Jun 2014 13:27:29 +0000
Post by hasufell
If you think having broken packages for months in stable arch is ok,
then you are wrong.
And btw., your funny threats don't impress me anymore.
I'll bring this up to the council agenda if you like. This is a
non-trivial tree-wide problem and if toolchain keeps ignoring it, then I
will hardmask the thing.
Well, see, I'm trying funny threats because I don't know how else to get it
through your head that you have no right to start masking packages that don't
belong to you when we won't fix your stupid pet bug. And if you do so there
will be consequences. By masking crossdev you'd effectively be masking the
whole tree for any number of archs. Not to mention anyone building a
cross-compiler for their own use outside of portage. There are several
high-profile groups that rely on crossdev. Hell, several arch teams rely on
it. You're suggesting we mask it because your particular corner-case doesn't
work. Maybe I should start threatening to remove games when they don't run
on my video card.

If doing something dumb like installing a i686 crossdev toolchain on
x86_64 breaks things, it's because you've done something dumb. Stop doing
that and things should work better.
--
Ryan Hill psn: dirtyepic_sk
gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E 7F92 ED38 BD49 957A 8463
hasufell
2014-06-17 12:30:01 UTC
Permalink
Post by Ryan Hill
If doing something dumb like installing a i686 crossdev toolchain on
x86_64 breaks things, it's because you've done something dumb. Stop doing
that and things should work better.
There have been several reasons mentioned to do what you call dumb. I'm
not going to repeat them. Read the thread.
Maciej Mrozowski
2014-03-29 01:30:02 UTC
Permalink
On Wednesday 26 of March 2014 02:07:56 Mike Frysinger wrote:

| cmake is completely broken when it comes to library searching and multilib and
| cross-compiling. it will happily look in hardcoded / paths to test for the
| existence of files as well as directly execute `pkg-config`. it's a great
| example of people saying "autotools is crap, so let's invent our own kind of
| crap and ignore lessons learned". this isn't the fault of cmake eclasses, but
| it'd be nice if we could someone standardize the hacks in there so we don't
| have to duplicate across ebuilds.

If we provided toolchain.cmake (passed to CMake via -DCMAKE_TOOLCHAIN_FILE=<path>) file for each crossdev, that could not only set compiler paths/etc but also override CMAKE_SYSTEM_PREFIX_PATH, we would get pretty close to autotools.

Yes, CMake has pretty rudimentary (to put it mildly..) cross-compilation support but it still can be done there somewhat "right".

regards
MM
Greg Turner
2014-03-16 12:10:02 UTC
Permalink
On Wed, Mar 12, 2014 at 9:06 AM, Alexandre Rostovtsev
Post by Alexandre Rostovtsev
is there any legitimate reason for
wanting crossdev's i686 wrappers when on a multilib amd64 profile?
One other note: legitimacy is in the eye of the legitimizer, I suppose, but
there are sometimes practical reasons to want a no-multilib compiler.

A trivial example is that some primitive build scripts go ape when CC has a
space in it (but this is certainly debatable as to its legitimacy, given
the obvious work-arounds).
Michał Górny
2014-03-13 09:00:02 UTC
Permalink
Dnia 2014-03-12, o godz. 15:46:01
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
We have a problem where the crossdev pkg-config wrapper scripts
interfere with multilib.
PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"
Now, SYSROOT is chosen from multiple conditions. When emerging a
"//usr/lib/pkgconfig://usr/share/pkgconfig"
Build systems like autotools will pick the crossdev provided
"i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
find the pkg-config files in /usr/lib64/...
This is not a problem most of the time if the package just wants to
get the libs to link against.
However, every package that tries to access variables that are
different between /usr/lib32/pkgconfig/foo.pc and
/usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
unexpected results.
That already happens for
x11-libs/libva-vdpau-driver
x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
and there are probably more.
Another possible workaround is to make pkgconfig true-multilib. Then it
would own i686-pc-linux-gnu-pkgconfig, and that executable would work
correctly. More than that, we could work on killing the PKG_CONFIG_PATH
hack.
--
Best regards,
Micha³ Górny
Alexandre Rostovtsev
2014-03-13 12:30:02 UTC
Permalink
Post by Michał Górny
Dnia 2014-03-12, o godz. 15:46:01
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
We have a problem where the crossdev pkg-config wrapper scripts
interfere with multilib.
PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"
Now, SYSROOT is chosen from multiple conditions. When emerging a
"//usr/lib/pkgconfig://usr/share/pkgconfig"
Build systems like autotools will pick the crossdev provided
"i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
find the pkg-config files in /usr/lib64/...
This is not a problem most of the time if the package just wants to
get the libs to link against.
However, every package that tries to access variables that are
different between /usr/lib32/pkgconfig/foo.pc and
/usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
unexpected results.
That already happens for
x11-libs/libva-vdpau-driver
x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
and there are probably more.
Another possible workaround is to make pkgconfig true-multilib. Then it
would own i686-pc-linux-gnu-pkgconfig, and that executable would work
correctly. More than that, we could work on killing the PKG_CONFIG_PATH
hack.
That won't work with current versions of crossdev because they blindly
create/delete their pkg-config wrappers without checking if they are
overwriting or removing something that belongs to another package.
Mike Frysinger
2014-03-26 05:20:01 UTC
Permalink
Post by Michał Górny
Post by hasufell
We have a problem where the crossdev pkg-config wrapper scripts
interfere with multilib.
PKG_CONFIG_LIBDIR="${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgco
nfig"
Now, SYSROOT is chosen from multiple conditions. When emerging a
"//usr/lib/pkgconfig://usr/share/pkgconfig"
this behavior is more bug than feature. the preference is to utilize SYSROOT
in src_* functions first and ROOT in pkg_* functions. i'll have to see if
EBUILD_PHASE is exported to shell scripts ...
Post by Michał Górny
Post by hasufell
Build systems like autotools will pick the crossdev provided
"i686-pc-linux-gnu-pkg-config" for the 32bit ABI which will in turn
override the eclass-exported PKG_CONFIG_LIBDIR and now effectively
find the pkg-config files in /usr/lib64/...
This is not a problem most of the time if the package just wants to
get the libs to link against.
However, every package that tries to access variables that are
different between /usr/lib32/pkgconfig/foo.pc and
/usr/lib64/pkgconfig/foo.pc like "libdir" will fail or produce
unexpected results.
That already happens for
x11-libs/libva-vdpau-driver
x11-libs/libva (https://bugs.gentoo.org/show_bug.cgi?id=500338)
and there are probably more.
there have been reports dating back even longer. e.g. people used crossdev to
create i686-pc-linux-gnu and then tried to build sandbox. it's just the
number of people hitting this was fairly low (like ~1 a year). and for them,
it was possible to just use a diff tuple for their i686 compiler.
Post by Michał Górny
Another possible workaround is to make pkgconfig true-multilib. Then it
would own i686-pc-linux-gnu-pkgconfig, and that executable would work
correctly. More than that, we could work on killing the PKG_CONFIG_PATH
hack.
that by itself doesn't really help. the wrappers are designed to be used with
the respective cross-compiler (SYSROOT by default), not default to the host
system.

when you run `crossdev i686-pc-linux-gnu`, it owns that tuple. that includes
i686-pc-linux-gnu-pkg-config.

if we're going to have the multilib system lie and use a tuple that doesn't
actually exist, we either:

(1) override all the vars so they point back to the real toolchain.
this doesn't scale when you consider helper tools like the legacy sdl-config
or the extended set of tools that binutils/gcc/etc... install. it's mitigated
by the fact the set of vars in play most of the time is low.

(2) use tuples with loaded vendor fields to reduce the chance of collisions.
e.g. having an ABI=amd64 system use i686-gentoo%multilib-linux-gnu instead of
i686-pc-linux-gnu would defeat any automatic path searches.
-mike
Mike Frysinger
2014-03-27 06:20:02 UTC
Permalink
Post by Mike Frysinger
(2) use tuples with loaded vendor fields to reduce the chance of collisions.
e.g. having an ABI=amd64 system use i686-gentoo%multilib-linux-gnu instead
of i686-pc-linux-gnu would defeat any automatic path searches.
this patch keeps the status quo. although the status quo is broken, but we
can sort that out independently.
-mike

--- profiles/arch/amd64/make.defaults 18 Jan 2014 01:03:24 -0000 1.21
+++ profiles/arch/amd64/make.defaults 27 Mar 2014 06:13:22 -0000
@@ -21,17 +21,17 @@ ABI="amd64"
# 64bit specific settings.
CFLAGS_amd64="-m64"
LDFLAGS_amd64="-m elf_x86_64"
-CHOST_amd64="x86_64-pc-linux-gnu"
+CHOST_amd64="${CHOST}"

# 32bit specific settings.
CFLAGS_x86="-m32"
LDFLAGS_x86="-m elf_i386"
-CHOST_x86="i686-pc-linux-gnu"
+CHOST_x86="i686-gentoo%multilib-linux-gnu"

# 64-32bit specific settings.
CFLAGS_x32="-mx32"
LDFLAGS_x32="-m elf32_x86_64"
-CHOST_x32="x86_64-pc-linux-gnux32"
+CHOST_x32="x86_64-gentoo%multilib-linux-gnux32"

# 2006/10/24 - Simon Stelling <***@gentoo.org>
# They are masked, but we can enable them anyway for those who have
Michał Górny
2014-03-27 07:00:01 UTC
Permalink
Dnia 2014-03-27, o godz. 02:13:52
Post by Mike Frysinger
Post by Mike Frysinger
(2) use tuples with loaded vendor fields to reduce the chance of collisions.
e.g. having an ABI=amd64 system use i686-gentoo%multilib-linux-gnu instead
of i686-pc-linux-gnu would defeat any automatic path searches.
this patch keeps the status quo. although the status quo is broken, but we
can sort that out independently.
Except that it breaks stuff that is installed at the point and comes
with no plan of cleaning up the resulting mess.
Post by Mike Frysinger
--- profiles/arch/amd64/make.defaults 18 Jan 2014 01:03:24 -0000 1.21
+++ profiles/arch/amd64/make.defaults 27 Mar 2014 06:13:22 -0000
@@ -21,17 +21,17 @@ ABI="amd64"
# 64bit specific settings.
CFLAGS_amd64="-m64"
LDFLAGS_amd64="-m elf_x86_64"
-CHOST_amd64="x86_64-pc-linux-gnu"
+CHOST_amd64="${CHOST}"
# 32bit specific settings.
CFLAGS_x86="-m32"
LDFLAGS_x86="-m elf_i386"
-CHOST_x86="i686-pc-linux-gnu"
+CHOST_x86="i686-gentoo%multilib-linux-gnu"
Using percent sign here looks like asking for trouble at some point.
I don't see why you can't use plain 'gentoomultilib' that is more
fool-proof.
Post by Mike Frysinger
# 64-32bit specific settings.
CFLAGS_x32="-mx32"
LDFLAGS_x32="-m elf32_x86_64"
-CHOST_x32="x86_64-pc-linux-gnux32"
+CHOST_x32="x86_64-gentoo%multilib-linux-gnux32"
# They are masked, but we can enable them anyway for those who have
--
Best regards,
Micha³ Górny
Mike Frysinger
2014-03-27 07:20:02 UTC
Permalink
Post by Michał Górny
Post by Mike Frysinger
Post by Mike Frysinger
(2) use tuples with loaded vendor fields to reduce the chance of
collisions. e.g. having an ABI=amd64 system use
i686-gentoo%multilib-linux-gnu instead of i686-pc-linux-gnu would
defeat any automatic path searches.
this patch keeps the status quo. although the status quo is broken, but we
can sort that out independently.
Except that it breaks stuff that is installed at the point and comes
with no plan of cleaning up the resulting mess.
such as ... ? vague statements can't be addressed.
Post by Michał Górny
Post by Mike Frysinger
--- profiles/arch/amd64/make.defaults 18 Jan 2014 01:03:24 -0000 1.21
+++ profiles/arch/amd64/make.defaults 27 Mar 2014 06:13:22 -0000
@@ -21,17 +21,17 @@ ABI="amd64"
# 64bit specific settings.
CFLAGS_amd64="-m64"
LDFLAGS_amd64="-m elf_x86_64"
-CHOST_amd64="x86_64-pc-linux-gnu"
+CHOST_amd64="${CHOST}"
# 32bit specific settings.
CFLAGS_x86="-m32"
LDFLAGS_x86="-m elf_i386"
-CHOST_x86="i686-pc-linux-gnu"
+CHOST_x86="i686-gentoo%multilib-linux-gnu"
Using percent sign here looks like asking for trouble at some point.
I don't see why you can't use plain 'gentoomultilib' that is more
fool-proof.
i merely picked a value that was highly unlikely for people to use. the %
should be safe as it is not interpreted by the shell and strongly indicates
"hey man, don't mess with me". it could just as easily be a _ or nothing at
all. i don't feel strongly about it.
-mike
Michał Górny
2014-03-27 09:20:02 UTC
Permalink
Dnia 2014-03-27, o godz. 03:18:31
Post by Mike Frysinger
Post by Michał Górny
Post by Mike Frysinger
Post by Mike Frysinger
(2) use tuples with loaded vendor fields to reduce the chance of
collisions. e.g. having an ABI=amd64 system use
i686-gentoo%multilib-linux-gnu instead of i686-pc-linux-gnu would
defeat any automatic path searches.
this patch keeps the status quo. although the status quo is broken, but we
can sort that out independently.
Except that it breaks stuff that is installed at the point and comes
with no plan of cleaning up the resulting mess.
such as ... ? vague statements can't be addressed.
Such as all the builds that use ${CHOST}-foo currently. If you change
CHOST, our users will have to find and rebuild all packages that
install ${CHOST}-foos or otherwise random breakage will happen.
--
Best regards,
Micha³ Górny
Mike Frysinger
2014-03-27 14:30:01 UTC
Permalink
Post by Michał Górny
Post by Mike Frysinger
Post by Michał Górny
Post by Mike Frysinger
Post by Mike Frysinger
(2) use tuples with loaded vendor fields to reduce the chance of
collisions. e.g. having an ABI=amd64 system use
i686-gentoo%multilib-linux-gnu instead of i686-pc-linux-gnu would
defeat any automatic path searches.
this patch keeps the status quo. although the status quo is broken,
but
we
can sort that out independently.
Except that it breaks stuff that is installed at the point and comes
with no plan of cleaning up the resulting mess.
such as ... ? vague statements can't be addressed.
Such as all the builds that use ${CHOST}-foo currently. If you change
CHOST, our users will have to find and rebuild all packages that
install ${CHOST}-foos or otherwise random breakage will happen.
again, please give a concrete example
-mike
Michał Górny
2014-03-27 14:40:02 UTC
Permalink
Dnia 2014-03-27, o godz. 10:23:30
Post by Mike Frysinger
Post by Michał Górny
Post by Mike Frysinger
Post by Michał Górny
Post by Mike Frysinger
Post by Mike Frysinger
(2) use tuples with loaded vendor fields to reduce the chance of
collisions. e.g. having an ABI=amd64 system use
i686-gentoo%multilib-linux-gnu instead of i686-pc-linux-gnu would
defeat any automatic path searches.
this patch keeps the status quo. although the status quo is broken,
but
we
can sort that out independently.
Except that it breaks stuff that is installed at the point and comes
with no plan of cleaning up the resulting mess.
such as ... ? vague statements can't be addressed.
Such as all the builds that use ${CHOST}-foo currently. If you change
CHOST, our users will have to find and rebuild all packages that
install ${CHOST}-foos or otherwise random breakage will happen.
again, please give a concrete example
glib -- ${CHOST}-gdk-pixbuf-query-loaders (used in gnome2-utils.eclass)
gpg-error -- ${CHOST}-gpg-error-config
libgcrypt -- ${CHOST}-libgcrypt-config
llvm -- ${CHOST}-llvm-config
pango -- ${CHOST}-pango-querymodules

If you change CHOST, all invocations of those tools will fail randomly
until the respective packages are rebuilt. In some cases it will call
the wrong variant (resulting in borked output), in other it will call
non-existing tool.

And let's just hope it's the only issue we're going to hit.
--
Best regards,
Micha³ Górny
Mike Frysinger
2014-03-28 06:40:02 UTC
Permalink
Post by Michał Górny
Post by Mike Frysinger
Post by Michał Górny
Post by Mike Frysinger
Post by Michał Górny
Post by Mike Frysinger
Post by Mike Frysinger
(2) use tuples with loaded vendor fields to reduce the chance of
collisions. e.g. having an ABI=amd64 system use
i686-gentoo%multilib-linux-gnu instead of i686-pc-linux-gnu would
defeat any automatic path searches.
this patch keeps the status quo. although the status quo is broken,
but
we
can sort that out independently.
Except that it breaks stuff that is installed at the point and comes
with no plan of cleaning up the resulting mess.
such as ... ? vague statements can't be addressed.
Such as all the builds that use ${CHOST}-foo currently. If you change
CHOST, our users will have to find and rebuild all packages that
install ${CHOST}-foos or otherwise random breakage will happen.
again, please give a concrete example
glib -- ${CHOST}-gdk-pixbuf-query-loaders (used in gnome2-utils.eclass)
gpg-error -- ${CHOST}-gpg-error-config
libgcrypt -- ${CHOST}-libgcrypt-config
llvm -- ${CHOST}-llvm-config
pango -- ${CHOST}-pango-querymodules
If you change CHOST, all invocations of those tools will fail randomly
until the respective packages are rebuilt. In some cases it will call
the wrong variant (resulting in borked output), in other it will call
non-existing tool.
so the *-config scripts. we already know these are broken, not only from a
conceptual pov, but from real world too -- it relies on CHOST being unique.
but this needs to be sorted out anyways since the current system isn't working
(independent of crossdev usage), and perhaps that means breaking now so things
are cleaner in the future.

for the gnupg projects, we've already disabled -L paths from being emitted, so
the variants will still work (gpg-error-config will give the same answer as
$CHOST-gpg-error-config wrt --libs and such).

the llvm/gdk/pango logic would probably break. we had been looking at
rewriting the gdk/pango stuff to not require direct execution (since this
logic completely fails when cross-compiling) in Chromium OS, but that's been
on the back burner for a while. instead, we hand generate/mung the pango
modules cache file, and we've punted gtk/gdk from the image, so we don't care
it is broken.

the pango stuff is also broken because it uses /etc/pango/$CHOST in an attempt
to get an ABI unique path. we probably should change that to /etc/pango/$ABI
or /etc/pango/$(get_libdir) or move it the multilib dir like the gdk cache
does. the gnome guys would know best.

the llvm impact doesn't look terribly big as very few things use it.
-mike
Michał Górny
2014-03-29 21:40:02 UTC
Permalink
Dnia 2014-03-28, o godz. 02:33:09
Post by Mike Frysinger
the pango stuff is also broken because it uses /etc/pango/$CHOST in an attempt
to get an ABI unique path. we probably should change that to /etc/pango/$ABI
or /etc/pango/$(get_libdir) or move it the multilib dir like the gdk cache
does. the gnome guys would know best.
Because inventing distro-specific directories is always the best
solution.
Post by Mike Frysinger
the llvm impact doesn't look terribly big as very few things use it.
Indeed, most of Gentoo amd64 users don't care about being unable to
build Mesa at all.
--
Best regards,
Micha³ Górny
Loading...