media-libs/x264 fails to build during prepare phase #602

gorrila20 · 2020-09-12T07:05:25Z

media-libs/x264 fails to build on my system while performing an @world update.

Failed attempts I took to fix the issue:

Emerging ~amd64 package
Emerging 9999 package

The lto patch fails in every package version, except for the 9999 one where it gives me a little endian check error.

Error log of normal package no ~amd64:

x264-snapshot-20190214-2245.tar.bz2 BLAKE2B SHA512 size ;-) ... [ ok ]
FEATURES='assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr'
CFLAGS='-march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -pipe -Wl,-O1 -Wl,--as-needed'
CXXFLAGS='-march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -pipe -Wl,-O1 -Wl,--as-needed'
FFLAGS='-march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -pipe -Wl,-O1 -Wl,--as-needed'
FCFLAGS='-march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -pipe -Wl,-O1 -Wl,--as-needed'
F77FLAGS='-march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -pipe -Wl,-O1 -Wl,--as-needed'
LDFLAGS='-Wl,-O1 -Wl,--as-needed -march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -pipe'
MAKEOPTS='-j12'
gcc (Gentoo 10.2.0-r1 p2) 10.2.0
g++ (Gentoo 10.2.0-r1 p2) 10.2.0
GNU ld (Gentoo 2.34 p6) 2.34.0
Linux mbox 5.8.8-gentoo Create CODE_OF_CONDUCT.md #1 SMP Thu Sep 10 18:12:22 CEST 2020 x86_64 AMD Ryzen 5 3600 6-Core Processor AuthenticAMD GNU/Linux
media-libs/x264: 21 seconds average for 2 merges
2020-08-30T20:50:42 >>> media-libs/x264: 21 seconds
2020-08-31T23:14:47 >>> media-libs/x264: 21 seconds

Unpacking source...
Unpacking x264-snapshot-20190214-2245.tar.bz2 to /var/tmp/portage/media-libs/x264-0.0.20190214/work
Source unpacked in /var/tmp/portage/media-libs/x264-0.0.20190214/work

Applying lto.patch ...
patching file configure
Hunk Create CODE_OF_CONDUCT.md #1 FAILED at 1017.
1 out of 1 hunk FAILED -- saving rejects to file configure.rej [ !! ]
ERROR: media-libs/x264-0.0.20190214::gentoo failed (prepare phase):
patch -p1 failed with /var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch
Call stack:

          ebuild.sh, line  762:  Called __ebuild_main 'prepare'

 phase-functions.sh, line  970:  Called __source_all_bashrcs

          ebuild.sh, line  409:  Called __try_source '--no-qa' '/etc/portage/bashrc'

          ebuild.sh, line  458:  Called source '/etc/portage/bashrc'

             bashrc, line    6:  Called BashrcdMain '/etc/portage/bashrc'

        environment, line  600:  Called LTOPatch

        environment, line 1267:  Called eapply '/var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch'

        environment, line 1814:  Called _eapply_patch '/var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch'

        environment, line 1752:  Called __helpers_die 'patch -p1  failed with /var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch'

isolated-functions.sh, line 112: Called die
The specific snippet of code:
```
 	die "$@"
```
If you need support, post the output of emerge --info '=media-libs/x264-0.0.20190214::gentoo',
the complete build log and the output of emerge -pqv '=media-libs/x264-0.0.20190214::gentoo'.
The complete build log is located at '/var/tmp/portage/media-libs/x264-0.0.20190214/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/media-libs/x264-0.0.20190214/temp/environment'.
Working directory: '/var/tmp/portage/media-libs/x264-0.0.20190214/work/x264-snapshot-20190214-2245'
S: '/var/tmp/portage/media-libs/x264-0.0.20190214/work/x264-snapshot-20190214-2245'

Failed to emerge media-libs/x264-0.0.20190214, Log file:

'/var/tmp/portage/media-libs/x264-0.0.20190214/temp/build.log'

Messages for package media-libs/x264-0.0.20190214:
ERROR: media-libs/x264-0.0.20190214::gentoo failed (prepare phase):
patch -p1 failed with /var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch
Call stack:

          ebuild.sh, line  762:  Called __ebuild_main 'prepare'

 phase-functions.sh, line  970:  Called __source_all_bashrcs

          ebuild.sh, line  409:  Called __try_source '--no-qa' '/etc/portage/bashrc'

          ebuild.sh, line  458:  Called source '/etc/portage/bashrc'

             bashrc, line    6:  Called BashrcdMain '/etc/portage/bashrc'

        environment, line  600:  Called LTOPatch

        environment, line 1267:  Called eapply '/var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch'

        environment, line 1814:  Called _eapply_patch '/var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch'

        environment, line 1752:  Called __helpers_die 'patch -p1  failed with /var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch'

isolated-functions.sh, line 112: Called die
The specific snippet of code:
```
 	die "$@"
```
If you need support, post the output of emerge --info '=media-libs/x264-0.0.20190214::gentoo',
the complete build log and the output of emerge -pqv '=media-libs/x264-0.0.20190214::gentoo'.
The complete build log is located at '/var/tmp/portage/media-libs/x264-0.0.20190214/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/media-libs/x264-0.0.20190214/temp/environment'.
Working directory: '/var/tmp/portage/media-libs/x264-0.0.20190214/work/x264-snapshot-20190214-2245'
S: '/var/tmp/portage/media-libs/x264-0.0.20190214/work/x264-snapshot-20190214-2245'

Error for 9999 package:

nitialized empty Git repository in /var/cache/distfiles/git3-src/videolan_x264.git/

Repository id: videolan_x264.git
To override fetched repository properties, use:
EGIT_OVERRIDE_REPO_VIDEOLAN_X264
EGIT_OVERRIDE_BRANCH_VIDEOLAN_X264
EGIT_OVERRIDE_COMMIT_VIDEOLAN_X264
EGIT_OVERRIDE_COMMIT_DATE_VIDEOLAN_X264
Fetching https://code.videolan.org/videolan/x264.git ...
git fetch https://code.videolan.org/videolan/x264.git +HEAD:refs/git-r3/HEAD
remote: Enumerating objects: 22410, done.
remote: Counting objects: 100% (22410/22410), done.
remote: Compressing objects: 100% (3875/3875), done.
remote: Total 22410 (delta 18590), reused 22295 (delta 18492), pack-reused 0
Receiving objects: 100% (22410/22410), 5.21 MiB | 12.67 MiB/s, done.
Resolving deltas: 100% (18590/18590), done.
From https://code.videolan.org/videolan/x264
[new ref] -> refs/git-r3/HEAD
git symbolic-ref refs/git-r3/media-libs/x264/0/main refs/git-r3/HEAD
Checking out https://code.videolan.org/videolan/x264.git to /var/tmp/portage/media-libs/x264-9999/work/x264-9999 ...
git checkout --quiet refs/git-r3/HEAD
GIT NEW branch -->
repository: https://code.videolan.org/videolan/x264.git
at the commit: db0d417728460c647ed4a847222a535b00d3dbcb

Source unpacked in /var/tmp/portage/media-libs/x264-9999/work

Applying lto.patch ... [ ok ]
lto-overlay: LTO patches applied.

Preparing source in /var/tmp/portage/media-libs/x264-9999/work/x264-9999 ...
Source prepared.
Configuring source in /var/tmp/portage/media-libs/x264-9999/work/x264-9999 ...

abi_x86_64.amd64: running multilib-minimal_abi_src_configure
endian test failed
ERROR: media-libs/x264-9999::gentoo failed (configure phase):
(no error message)
Call stack:

ebuild.sh, line  125:  Called src_configure

environment, line 3830: Called multilib-minimal_src_configure
environment, line 3116: Called multilib_foreach_abi 'multilib-minimal_abi_src_configure'
environment, line 3369: Called multibuild_foreach_variant '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_configure'
environment, line 3046: Called _multibuild_run '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_configure'
environment, line 3044: Called _multilib_multibuild_wrapper 'multilib-minimal_abi_src_configure'
environment, line 1848: Called multilib-minimal_abi_src_configure
environment, line 3110: Called multilib_src_configure
environment, line 3609: Called die
The specific snippet of code:

  "${S}/configure" --prefix="${EPREFIX}"/usr --libdir="${EPREFIX}"/usr/$(get_libdir) --disable-cli --disable-avs --disable-lavf --disable-swscale --disable-ffms --disable-gpac --enable-pic --enable-shared --host="${CHOST}" --cross-prefix="${CHOST}-" $(usex interlaced "" "--disable-interlaced") $(usex opencl "" "--disable-opencl") $(usex static-libs "--enable-static" "") $(usex threads "" "--disable-thread") ${asm_conf} || die

If you need support, post the output of emerge --info '=media-libs/x264-9999::gentoo',
the complete build log and the output of emerge -pqv '=media-libs/x264-9999::gentoo'.
The complete build log is located at '/var/tmp/portage/media-libs/x264-9999/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/media-libs/x264-9999/temp/environment'.
Working directory: '/var/tmp/portage/media-libs/x264-9999/work/x264-9999-abi_x86_64.amd64'
S: '/var/tmp/portage/media-libs/x264-9999/work/x264-9999'

Failed to emerge media-libs/x264-9999, Log file:

'/var/tmp/portage/media-libs/x264-9999/temp/build.log'

Messages for package media-libs/x264-9999:
lto-overlay: LTO patches applied.
ERROR: media-libs/x264-9999::gentoo failed (configure phase):
(no error message)
Call stack:

ebuild.sh, line  125:  Called src_configure

environment, line 3830: Called multilib-minimal_src_configure
environment, line 3116: Called multilib_foreach_abi 'multilib-minimal_abi_src_configure'
environment, line 3369: Called multibuild_foreach_variant '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_configure'
environment, line 3046: Called _multibuild_run '_multilib_multibuild_wrapper' 'multilib-minimal_abi_src_configure'
environment, line 3044: Called _multilib_multibuild_wrapper 'multilib-minimal_abi_src_configure'
environment, line 1848: Called multilib-minimal_abi_src_configure
environment, line 3110: Called multilib_src_configure
environment, line 3609: Called die
The specific snippet of code:

  "${S}/configure" --prefix="${EPREFIX}"/usr --libdir="${EPREFIX}"/usr/$(get_libdir) --disable-cli --disable-avs --disable-lavf --disable-swscale --disable-ffms --disable-gpac --enable-pic --enable-shared --host="${CHOST}" --cross-prefix="${CHOST}-" $(usex interlaced "" "--disable-interlaced") $(usex opencl "" "--disable-opencl") $(usex static-libs "--enable-static" "") $(usex threads "" "--disable-thread") ${asm_conf} || die

If you need support, post the output of emerge --info '=media-libs/x264-9999::gentoo',
the complete build log and the output of emerge -pqv '=media-libs/x264-9999::gentoo'.
The complete build log is located at '/var/tmp/portage/media-libs/x264-9999/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/media-libs/x264-9999/temp/environment'.
Working directory: '/var/tmp/portage/media-libs/x264-9999/work/x264-9999-abi_x86_64.amd64'
S: '/var/tmp/portage/media-libs/x264-9999/work/x264-9999'

The text was updated successfully, but these errors were encountered:

InBetweenNames · 2020-09-12T15:03:58Z

I swear, this is the most problematic package right now. I'm just going to fork the ebuild.

gorrila20 · 2020-09-13T13:11:18Z

You have any idea what is causing this or how to resolve the issue?

jiblime · 2020-09-13T21:48:29Z

You can add media-libs/x264 *FLAGS+=-ffat-lto-objects in ltoworkarounds.conf as the lesser of two evils since maintaining forked ebuilds of x264 would be a PITA.

Edit: This works too instead of compiling the entire package with -ffat-lto-objects. Tested with GCC 10.2 (patched local build so ymmv), binutils 2.35, x264-9999::gentoo, x264-0.0.20200810::poly-c, fails to apply with x264-0.0.20190903-r1::gentoo

/var/lib/layman/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264 # cat lto.patch 
--- a/configure 2020-07-07 20:37:00.103200843 -0400
+++ b/configure 2020-07-07 20:36:38.819867863 -0400
@@ -1017,7 +1017,7 @@
 CPU_ENDIAN="little-endian"
 if [ $compiler = GNU ]; then
     echo "int i[2] = {0x42494745,0}; double f[2] = {0x1.0656e6469616ep+102,0};" > conftest.c
-    $CC $CFLAGS conftest.c -c -o conftest.o 2>/dev/null || die "endian test failed"
+    $CC $CFLAGS -ffat-lto-objects conftest.c -c -o conftest.o 2>/dev/null || die "endian test failed"
     if (${STRINGS} -a conftest.o | grep -q BIGE) && (${STRINGS} -a conftest.o | grep -q FPendian) ; then
         define WORDS_BIGENDIAN
         CPU_ENDIAN="big-endian"

gorrila20 · 2020-09-14T10:16:45Z

Great thank you, I will try this solution tomorrow evening when I have some time. I should leave this issue open if the solution works right?

gorrila20 · 2020-09-16T16:23:32Z

Thank the patch worked like a charm! any chance this will be merged upstream?

jiblime · 2020-09-20T06:15:46Z

I'm not sure who should take "responsibility" for this. -ffat-lto-objects is a workaround for x264; should the onus be on GCC devs instead?

mid-kid · 2020-11-07T19:38:31Z

The issue with this patch is that it's generated after the x264-0.0.20190903-STRINGS.patch, but applied before it during build. However, both patches patch about the same thing, so I think the LTO patch should be applied after.
However, the 41-lto-patch.sh runs before src_prepare(), so ultimately, it puts us in a situation where the patch in the gentoo repo should be updated to apply on top of the lto patch, which isn't a very good situation.

I wonder, can the lto patch be applied after the gentoo patch?

telans · 2020-11-07T20:04:59Z

I wonder, can the lto patch be applied after the gentoo patch?

gentooLTO/sys-config/ltoize/files/bashrc.d/41-lto-patch.sh

Line 75 in b02d379

BashrcdPhase prepare LTOPatch

Does changing "prepare" to "configure" work?

mid-kid · 2020-11-07T20:35:55Z

Yeah, that works to apply the patch, at least, but idk if it works with other packages, and the build still fails at endian test failed, but that's a different issue I think.

telans · 2020-11-07T22:04:06Z

I don't think there are any packages which depend on lto patches before gentoo patches, so it should probably be changed to configure @InBetweenNames

aqxa1 · 2020-11-25T23:54:04Z

I wonder, can the lto patch be applied after the gentoo patch?

gentooLTO/sys-config/ltoize/files/bashrc.d/41-lto-patch.sh

Line 75 in b02d379

BashrcdPhase prepare LTOPatch

Does changing "prepare" to "configure" work?

I haven't tested extensively, but this workaround seems to break qtcore setup.

telans · 2020-11-26T02:45:23Z

qtcore or qtgui? I don't think it should be doing anything for a package without patches in this repo

aqxa1 · 2020-11-26T04:15:40Z

My bad, it was qtcore.

telans · 2020-11-26T05:13:37Z

but this workaround seems to break qtcore setup.

I think I know what's going on with this one. The qtcore ebuild now has:

	# fix missing qt_version_tag symbol w/ LTO, bug 674382
	sed -i -e 's/^gcc:ltcg/gcc/' src/corelib/global/global.pri || die

When using "configure" the patch fails due to gcc:ltcg no longer existing of course, but the ebuild should be failing without modifying 41-lto-patch.sh as gcc:ltcg wouldn't exist for sed. I don't know why it doesn't fail there.

The lto patch for qtcore here should be removed. I think this shows "configure" should still be used in 41-lto-patch.sh

jian-lin · 2020-12-06T10:31:56Z

After this issue is closed, I still cannot emerge the media-libs/x264-0.0.20190214. It says:

patching file configure
Hunk #1 FAILED at 1017.
1 out of 1 hunk FAILED -- saving rejects to file configure.rej
 [ !! ]
 * ERROR: media-libs/x264-0.0.20190214::gentoo failed (configure phase):
 *   patch -p1  failed with /var/db/repos/lto-overlay/sys-config/ltoize/files/patches/media-libs/x264/lto.patch

and the configure.rej is:

--- configure 2020-07-07 20:37:00.103200843 -0400
+++ configure 2020-07-07 20:36:38.819867863 -0400
@@ -1017,7 +1017,7 @@
 CPU_ENDIAN="little-endian"
 if [ $compiler = GNU ]; then
     echo "int i[2] = {0x42494745,0}; double f[2] = {0x1.0656e6469616ep+102,0};" > conftest.c
-    $CC $CFLAGS conftest.c -c -o conftest.o 2>/dev/null || die "endian test failed"
+    $CC $CFLAGS conftest.c -c -o conftest.o -shared 2>/dev/null || die "endian test failed"
     if (${STRINGS} -a conftest.o | grep -q BIGE) && (${STRINGS} -a conftest.o | grep -q FPendian) ; then
         define WORDS_BIGENDIAN
         CPU_ENDIAN="big-endian"

After looking at configure, it seems that the line number of the patch is wrong. It should be 994 instead of 1017.

telans · 2020-12-06T10:36:45Z

Whoops, looks like this is still needed: #635

@InBetweenNames please commit below as updated patch

--- a/configure	2020-11-11 12:42:21.977274237 -0500
+++ b/configure	2020-11-11 12:55:30.545286800 -0500
@@ -994,7 +994,7 @@
 CPU_ENDIAN="little-endian"
 if [ $compiler = GNU ]; then
     echo "int i[2] = {0x42494745,0}; double f[2] = {0x1.0656e6469616ep+102,0};" > conftest.c
-    $CC $CFLAGS conftest.c -c -o conftest.o 2>/dev/null || die "endian test failed"
+    $CC -O0 conftest.c -c -o conftest.o 2>/dev/null || die "endian test failed"
     if (${STRINGS} -a conftest.o | grep -q BIGE) && (${STRINGS} -a conftest.o | grep -q FPendian) ; then
         define WORDS_BIGENDIAN
         CPU_ENDIAN="big-endian"

gorrila20 changed the title ~~media-libs/x264 fails to build~~ media-libs/x264 fails to build during prepare phase (Patch doesnt fail on 9999) Sep 12, 2020

gorrila20 changed the title ~~media-libs/x264 fails to build during prepare phase (Patch doesnt fail on 9999)~~ media-libs/x264 fails to build during prepare phase Sep 12, 2020

powpingdone mentioned this issue Oct 19, 2020

media-libs/x264: Minor fix for issue #602 #623

Closed

telans mentioned this issue Nov 26, 2020

lto-patch: change application phase to configure #653

Merged

InBetweenNames closed this as completed in #653 Dec 4, 2020

jiblime mentioned this issue Dec 28, 2020

media-libs/x264-9999: endian test failed #671

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

media-libs/x264 fails to build during prepare phase #602

media-libs/x264 fails to build during prepare phase #602

gorrila20 commented Sep 12, 2020

InBetweenNames commented Sep 12, 2020

gorrila20 commented Sep 13, 2020

jiblime commented Sep 13, 2020 •

edited

Loading

gorrila20 commented Sep 14, 2020

gorrila20 commented Sep 16, 2020

jiblime commented Sep 20, 2020

mid-kid commented Nov 7, 2020

telans commented Nov 7, 2020

mid-kid commented Nov 7, 2020

telans commented Nov 7, 2020 •

edited

Loading

aqxa1 commented Nov 25, 2020 •

edited

Loading

telans commented Nov 26, 2020

aqxa1 commented Nov 26, 2020

telans commented Nov 26, 2020 •

edited

Loading

jian-lin commented Dec 6, 2020

telans commented Dec 6, 2020 •

edited

Loading

media-libs/x264 fails to build during prepare phase #602

media-libs/x264 fails to build during prepare phase #602

Comments

gorrila20 commented Sep 12, 2020

InBetweenNames commented Sep 12, 2020

gorrila20 commented Sep 13, 2020

jiblime commented Sep 13, 2020 • edited Loading

gorrila20 commented Sep 14, 2020

gorrila20 commented Sep 16, 2020

jiblime commented Sep 20, 2020

mid-kid commented Nov 7, 2020

telans commented Nov 7, 2020

mid-kid commented Nov 7, 2020

telans commented Nov 7, 2020 • edited Loading

aqxa1 commented Nov 25, 2020 • edited Loading

telans commented Nov 26, 2020

aqxa1 commented Nov 26, 2020

telans commented Nov 26, 2020 • edited Loading

jian-lin commented Dec 6, 2020

telans commented Dec 6, 2020 • edited Loading

jiblime commented Sep 13, 2020 •

edited

Loading

telans commented Nov 7, 2020 •

edited

Loading

aqxa1 commented Nov 25, 2020 •

edited

Loading

telans commented Nov 26, 2020 •

edited

Loading

telans commented Dec 6, 2020 •

edited

Loading