Add go download script #61
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Example output: |
|
Hey Travis! Thanks for doing this. Can you move this script into a location like |
|
Just wondering - could we not add https://storage.googleapis.com/golang/ or
whatever to the list of binary url bases, and then not host these at all?
…On Wed, Mar 28, 2018 at 3:31 PM, Stu Hood ***@***.***> wrote:
Hey Travis! Thanks for doing this.
Can you *move* this script into a location like
build-support/bin/go/linux/i386/1.10/build.sh, and then symlink the
script directory into the other locations?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#61 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAfS_DLbMq9sdlFIA7gQAV5mbqrU07caks5tjA83gaJpZM4S_doQ>
.
|
|
@benjyw Yeah, I was considering an update to |
|
Makes sense! I might take a swing at it. |
Here we add `build.sh` scripts for Go 1.9.4 and 1.10, and a helper script for generating the `build.sh` scripts. We also clarify in the docs what platforms are supported (i386 and macOS pre 10.8 are not supported going forward).
traviscrawford
force-pushed
the
travis/build-go-script
branch
from
c5b103c
to
b7dc271
Compare
|
Updated with individual |
stuhood
reviewed
Mar 29, 2018
| @@ -0,0 +1,3 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
It looks like all of the OSX scripts can be symlinks to one copy, since golang doesn't have multiple builds?
There was a problem hiding this comment.
But... maybe we don't care. Still trying to work out a good convention here.
There was a problem hiding this comment.
My thought process for separate files is it's more straightforward, since the place we link to changes over time, so you have to hunt around. Also this was easy to script, at least for Go, though I could see how more involved scripts would be a little different.
If you have a suggestion I'm happy to follow it.
There was a problem hiding this comment.
I merged as is, but remembered afterward that one of the reasons for the symlinking is that it avoids actually having to run an identical script a dozen times. So I'm going to leave the recommendation as is, and we should try to use symlinks in the future.
|
cosmicexplorer
added a commit
to cosmicexplorer/binaries
that referenced
this pull request
Nov 16, 2020
Add buildozer 0.6.0-80c7f0d45d7e40fa1f7362852697d4a03df557b3 (pantsbuild#56) Synchronize docker instructions with pantsbuild/pants. (pantsbuild#58) Add clang (pantsbuild#57) Switch to a workflow without committed binaries (pantsbuild#59) * Update the README to indicate that no new binaries should be committed here. * Formalize the script location to improve maintainability. Add Go 1.9.4 and 1.10 (pantsbuild#61) Here we add `build.sh` scripts for Go 1.9.4 and 1.10, and a helper script for generating the `build.sh` scripts. We also clarify in the docs what platforms are supported (i386 and macOS pre 10.8 are not supported going forward). update some of the commands described in the README (pantsbuild#63) add gcc 7.3.0 and clang 6.0.0 cross platform, binutils 2.30 for linux, edit clang 5.0.1 to conform to pantsbuild#59 (pantsbuild#62) ``` $ ./sync-s3.sh upload: bin/clang/linux/x86_64/6.0.0/build-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-cmake.sh upload: bin/clang/linux/x86_64/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build.sh upload: bin/binutils/linux/x86_64/2.30/build.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build.sh upload: bin/binutils/linux/x86_64/2.30/build-binutils.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build-binutils.sh upload: bin/binutils/linux/x86_64/2.30/binutils.tar.gz to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/binutils.tar.gz upload: bin/clang/mac/10.10/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build-clang.sh upload: bin/clang/mac/10.10/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/clang.tar.gz upload: bin/clang/mac/10.11/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build-clang.sh upload: bin/clang/mac/10.11/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh upload: bin/clang/mac/10.12/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build-clang.sh upload: bin/clang/mac/10.12/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build.sh upload: bin/clang/mac/10.10/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/clang.tar.gz upload: bin/clang/mac/10.12/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/clang.tar.gz upload: bin/clang/mac/10.13/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build.sh upload: bin/clang/mac/10.11/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/clang.tar.gz upload: bin/clang/mac/10.8/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/clang.tar.gz upload: bin/clang/mac/10.9/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build-clang.sh upload: bin/clang/mac/10.9/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/clang.tar.gz upload: bin/gcc/linux/x86_64/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build.sh upload: bin/gcc/linux/x86_64/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build-gcc.sh upload: bin/clang/mac/10.9/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/clang.tar.gz upload: bin/gcc/mac/10.10/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.10/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build.sh upload: bin/gcc/mac/10.11/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build-gcc.sh upload: bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.12/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build.sh upload: bin/gcc/mac/10.10/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.13/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.13/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.8/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.8/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build.sh upload: bin/gcc/mac/10.13/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.9/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build.sh upload: bin/gcc/mac/10.8/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/gcc.tar.gz ``` add xz package (pantsbuild#66) cloc and isort are now fetched from `bin`, but without the platform-specific path component. (pantsbuild#68) add --disable-rpath when configuring xz (pantsbuild#71) This removes the need to set `DYLD_LIBRARY_PATH` or `LD_LIBRARY_PATH` in pantsbuild/pants#5936. The `xz` binary will use the version of `xz` and `liblzma` in the installation directory when invoked. add xz wrapper script to set the library path (pantsbuild#72) See pantsbuild#71, where we added `--disable-rpath` to `xz` to stop it from using a hardcoded library path at runtime. However, that still fails in Travis in pantsbuild/pants#5936 with the output: ``` INFO] Fetched llvm.tar.xz binary from: https://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz . INFO] Extracting /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm.tar.xz to /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm . xz: /lib/x86_64-linux-gnu/liblzma.so.5: version `XZ_5.2' not found (required by xz) ``` While we had removed the query for the hardcoded path, we still need to actually set our own path. We had been doing that on the pants side -- instead, this PR adds a (very short) python script at the path of the real `xz` binary (a suggestion from @illicitonion in that PR) which sets `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` appropriately, not clobbering any existing values. This works with the current version of pantsbuild/pants#5936 for me on osx and linux. statically link xz (pantsbuild#74) See pantsbuild#71, pantsbuild#72, pantsbuild#73. Our `xz` is picking up travis's `xz` installation and using its (incompatible) `liblzma` shared library. On the suggestion of @stuhood in pantsbuild/pants#5936, we statically link `xz` here to avoid ever having this problem ever again. Proof: *linux*: ``` > ldd ./xz-5.2.4-linux/xz-install/bin/xz linux-vdso.so.1 (0x00007ffc865cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f1fa385f000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f1fa34a3000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f1fa3cb4000) ``` *osx*: ``` > otool -L ./xz-5.2.4-linux/xz-install/bin/xz ./xz-5.2.4-linux/xz-install/bin/xz: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) ``` add llvm 6.0.0 package. (pantsbuild#75) See pantsbuild/pants#5991. This adds the binary I should have provided when we added the ability to download `BinaryTool`s from external urls, in the exact same way we do it within Pants. Upgrade isort to 4.3.4. (pantsbuild#76) This adds support for more python3 stdlib symbols. Also introduce a build script. Fixup isort path structure. Node.js 10.7.0 and Yarnpkg 1.7.0 (pantsbuild#80) Four small scripts, each named "build.sh" have been added. These will build a specific version of Node.js or Yarnpkg in the current directory. Symlinks from older macOS to 10.13 are included. Add build scripts and symlinks for Node 8.11.3 and Yarnpkg 1.6.0 (pantsbuild#81) Adding Node v10.13.0 build scripts for Mac 10.8-10.13 and Linux X64 (pantsbuild#86) Add go 1.10.5 (pantsbuild#87) Add build script for Thrift 0.11.0 (pantsbuild#83) Add build scripts and symlinks for Yarnpkg 1.12.3 (pantsbuild#89) Add updated go versions to pants binaries (pantsbuild#90) * Add updated go versions to pants binaries * Expanded the mac_versions into a map to encode the OSX min version requirements of the latest builds of go. Add build scripts and symlinks for Yarnpkg 1.16.0 (pantsbuild#91) * Add build scripts and symlinks for Yarnpkg 1.16.0 Add go 1.12.7 (pantsbuild#92) * Add go 1.12.7 Upgrade to cloc 1.82 (pantsbuild#93) Add cloc 1.80 (pantsbuild#94) The current version of cloc (version 1.82) has a bug where the --ignored flag is not working properly ( AlDanial/cloc#401 ). The pants test for the cloc goal tests this behavior, so as long as this bug is not fixed upstream, pants will fail this test. I've confirmed that version 1.80 of cloc does not have this bug, so we should be able to use it until upstream cloc fixes the bug in a new verson. Add go 1.13.8 for linux and mac 10.11+
cosmicexplorer
added a commit
to cosmicexplorer/binaries
that referenced
this pull request
Nov 16, 2020
Add buildozer 0.6.0-80c7f0d45d7e40fa1f7362852697d4a03df557b3 (pantsbuild#56) Synchronize docker instructions with pantsbuild/pants. (pantsbuild#58) Add clang (pantsbuild#57) Switch to a workflow without committed binaries (pantsbuild#59) * Update the README to indicate that no new binaries should be committed here. * Formalize the script location to improve maintainability. Add Go 1.9.4 and 1.10 (pantsbuild#61) Here we add `build.sh` scripts for Go 1.9.4 and 1.10, and a helper script for generating the `build.sh` scripts. We also clarify in the docs what platforms are supported (i386 and macOS pre 10.8 are not supported going forward). update some of the commands described in the README (pantsbuild#63) add gcc 7.3.0 and clang 6.0.0 cross platform, binutils 2.30 for linux, edit clang 5.0.1 to conform to pantsbuild#59 (pantsbuild#62) ``` $ ./sync-s3.sh upload: bin/clang/linux/x86_64/6.0.0/build-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-cmake.sh upload: bin/clang/linux/x86_64/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build.sh upload: bin/binutils/linux/x86_64/2.30/build.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build.sh upload: bin/binutils/linux/x86_64/2.30/build-binutils.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build-binutils.sh upload: bin/binutils/linux/x86_64/2.30/binutils.tar.gz to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/binutils.tar.gz upload: bin/clang/mac/10.10/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build-clang.sh upload: bin/clang/mac/10.10/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/clang.tar.gz upload: bin/clang/mac/10.11/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build-clang.sh upload: bin/clang/mac/10.11/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh upload: bin/clang/mac/10.12/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build-clang.sh upload: bin/clang/mac/10.12/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build.sh upload: bin/clang/mac/10.10/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/clang.tar.gz upload: bin/clang/mac/10.12/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/clang.tar.gz upload: bin/clang/mac/10.13/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build.sh upload: bin/clang/mac/10.11/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/clang.tar.gz upload: bin/clang/mac/10.8/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/clang.tar.gz upload: bin/clang/mac/10.9/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build-clang.sh upload: bin/clang/mac/10.9/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/clang.tar.gz upload: bin/gcc/linux/x86_64/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build.sh upload: bin/gcc/linux/x86_64/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build-gcc.sh upload: bin/clang/mac/10.9/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/clang.tar.gz upload: bin/gcc/mac/10.10/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.10/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build.sh upload: bin/gcc/mac/10.11/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build-gcc.sh upload: bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.12/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build.sh upload: bin/gcc/mac/10.10/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.13/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.13/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.8/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.8/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build.sh upload: bin/gcc/mac/10.13/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.9/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build.sh upload: bin/gcc/mac/10.8/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/gcc.tar.gz ``` add xz package (pantsbuild#66) cloc and isort are now fetched from `bin`, but without the platform-specific path component. (pantsbuild#68) add --disable-rpath when configuring xz (pantsbuild#71) This removes the need to set `DYLD_LIBRARY_PATH` or `LD_LIBRARY_PATH` in pantsbuild/pants#5936. The `xz` binary will use the version of `xz` and `liblzma` in the installation directory when invoked. add xz wrapper script to set the library path (pantsbuild#72) See pantsbuild#71, where we added `--disable-rpath` to `xz` to stop it from using a hardcoded library path at runtime. However, that still fails in Travis in pantsbuild/pants#5936 with the output: ``` INFO] Fetched llvm.tar.xz binary from: https://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz . INFO] Extracting /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm.tar.xz to /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm . xz: /lib/x86_64-linux-gnu/liblzma.so.5: version `XZ_5.2' not found (required by xz) ``` While we had removed the query for the hardcoded path, we still need to actually set our own path. We had been doing that on the pants side -- instead, this PR adds a (very short) python script at the path of the real `xz` binary (a suggestion from @illicitonion in that PR) which sets `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` appropriately, not clobbering any existing values. This works with the current version of pantsbuild/pants#5936 for me on osx and linux. statically link xz (pantsbuild#74) See pantsbuild#71, pantsbuild#72, pantsbuild#73. Our `xz` is picking up travis's `xz` installation and using its (incompatible) `liblzma` shared library. On the suggestion of @stuhood in pantsbuild/pants#5936, we statically link `xz` here to avoid ever having this problem ever again. Proof: *linux*: ``` > ldd ./xz-5.2.4-linux/xz-install/bin/xz linux-vdso.so.1 (0x00007ffc865cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f1fa385f000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f1fa34a3000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f1fa3cb4000) ``` *osx*: ``` > otool -L ./xz-5.2.4-linux/xz-install/bin/xz ./xz-5.2.4-linux/xz-install/bin/xz: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) ``` add llvm 6.0.0 package. (pantsbuild#75) See pantsbuild/pants#5991. This adds the binary I should have provided when we added the ability to download `BinaryTool`s from external urls, in the exact same way we do it within Pants. Upgrade isort to 4.3.4. (pantsbuild#76) This adds support for more python3 stdlib symbols. Also introduce a build script. Fixup isort path structure. Node.js 10.7.0 and Yarnpkg 1.7.0 (pantsbuild#80) Four small scripts, each named "build.sh" have been added. These will build a specific version of Node.js or Yarnpkg in the current directory. Symlinks from older macOS to 10.13 are included. Add build scripts and symlinks for Node 8.11.3 and Yarnpkg 1.6.0 (pantsbuild#81) Adding Node v10.13.0 build scripts for Mac 10.8-10.13 and Linux X64 (pantsbuild#86) Add go 1.10.5 (pantsbuild#87) Add build script for Thrift 0.11.0 (pantsbuild#83) Add build scripts and symlinks for Yarnpkg 1.12.3 (pantsbuild#89) Add updated go versions to pants binaries (pantsbuild#90) * Add updated go versions to pants binaries * Expanded the mac_versions into a map to encode the OSX min version requirements of the latest builds of go. Add build scripts and symlinks for Yarnpkg 1.16.0 (pantsbuild#91) * Add build scripts and symlinks for Yarnpkg 1.16.0 Add go 1.12.7 (pantsbuild#92) * Add go 1.12.7 Upgrade to cloc 1.82 (pantsbuild#93) Add cloc 1.80 (pantsbuild#94) The current version of cloc (version 1.82) has a bug where the --ignored flag is not working properly ( AlDanial/cloc#401 ). The pants test for the cloc goal tests this behavior, so as long as this bug is not fixed upstream, pants will fail this test. I've confirmed that version 1.80 of cloc does not have this bug, so we should be able to use it until upstream cloc fixes the bug in a new verson. Add go 1.13.8 for linux and mac 10.11+
cosmicexplorer
added a commit
to cosmicexplorer/binaries
that referenced
this pull request
Nov 16, 2020
Add buildozer 0.6.0-80c7f0d45d7e40fa1f7362852697d4a03df557b3 (pantsbuild#56) Synchronize docker instructions with pantsbuild/pants. (pantsbuild#58) Add clang (pantsbuild#57) Switch to a workflow without committed binaries (pantsbuild#59) * Update the README to indicate that no new binaries should be committed here. * Formalize the script location to improve maintainability. Add Go 1.9.4 and 1.10 (pantsbuild#61) Here we add `build.sh` scripts for Go 1.9.4 and 1.10, and a helper script for generating the `build.sh` scripts. We also clarify in the docs what platforms are supported (i386 and macOS pre 10.8 are not supported going forward). update some of the commands described in the README (pantsbuild#63) add gcc 7.3.0 and clang 6.0.0 cross platform, binutils 2.30 for linux, edit clang 5.0.1 to conform to pantsbuild#59 (pantsbuild#62) ``` $ ./sync-s3.sh upload: bin/clang/linux/x86_64/6.0.0/build-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-cmake.sh upload: bin/clang/linux/x86_64/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build.sh upload: bin/binutils/linux/x86_64/2.30/build.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build.sh upload: bin/binutils/linux/x86_64/2.30/build-binutils.sh to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/build-binutils.sh upload: bin/binutils/linux/x86_64/2.30/binutils.tar.gz to s3://binaries.pantsbuild.org/bin/binutils/linux/x86_64/2.30/binutils.tar.gz upload: bin/clang/mac/10.10/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build-clang.sh upload: bin/clang/mac/10.10/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/clang.tar.gz upload: bin/clang/mac/10.11/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build-clang.sh upload: bin/clang/mac/10.11/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/build.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang.sh upload: bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh to s3://binaries.pantsbuild.org/bin/clang/linux/x86_64/6.0.0/build-clang-for-linux-with-cmake.sh upload: bin/clang/mac/10.12/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build-clang.sh upload: bin/clang/mac/10.12/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/build.sh upload: bin/clang/mac/10.10/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.10/6.0.0/clang.tar.gz upload: bin/clang/mac/10.12/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.12/6.0.0/clang.tar.gz upload: bin/clang/mac/10.13/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/build.sh upload: bin/clang/mac/10.11/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.11/6.0.0/clang.tar.gz upload: bin/clang/mac/10.8/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/build-clang.sh upload: bin/clang/mac/10.13/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.13/6.0.0/clang.tar.gz upload: bin/clang/mac/10.9/6.0.0/build-clang.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build-clang.sh upload: bin/clang/mac/10.9/6.0.0/build.sh to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/build.sh upload: bin/clang/mac/10.8/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.8/6.0.0/clang.tar.gz upload: bin/gcc/linux/x86_64/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build.sh upload: bin/gcc/linux/x86_64/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/build-gcc.sh upload: bin/clang/mac/10.9/6.0.0/clang.tar.gz to s3://binaries.pantsbuild.org/bin/clang/mac/10.9/6.0.0/clang.tar.gz upload: bin/gcc/mac/10.10/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.10/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/build.sh upload: bin/gcc/mac/10.11/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build-gcc.sh upload: bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/linux/x86_64/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.12/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/build.sh upload: bin/gcc/mac/10.10/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.10/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.11/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.11/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.13/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.13/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/build.sh upload: bin/gcc/mac/10.12/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.12/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.8/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.8/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/build.sh upload: bin/gcc/mac/10.13/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.13/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/build-gcc.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build-gcc.sh upload: bin/gcc/mac/10.9/7.3.0/build.sh to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/build.sh upload: bin/gcc/mac/10.8/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.8/7.3.0/gcc.tar.gz upload: bin/gcc/mac/10.9/7.3.0/gcc.tar.gz to s3://binaries.pantsbuild.org/bin/gcc/mac/10.9/7.3.0/gcc.tar.gz ``` add xz package (pantsbuild#66) cloc and isort are now fetched from `bin`, but without the platform-specific path component. (pantsbuild#68) add --disable-rpath when configuring xz (pantsbuild#71) This removes the need to set `DYLD_LIBRARY_PATH` or `LD_LIBRARY_PATH` in pantsbuild/pants#5936. The `xz` binary will use the version of `xz` and `liblzma` in the installation directory when invoked. add xz wrapper script to set the library path (pantsbuild#72) See pantsbuild#71, where we added `--disable-rpath` to `xz` to stop it from using a hardcoded library path at runtime. However, that still fails in Travis in pantsbuild/pants#5936 with the output: ``` INFO] Fetched llvm.tar.xz binary from: https://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz . INFO] Extracting /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm.tar.xz to /home/travis/.cache/pants/bin/llvm/linux/x86_64/6.0.0/llvm . xz: /lib/x86_64-linux-gnu/liblzma.so.5: version `XZ_5.2' not found (required by xz) ``` While we had removed the query for the hardcoded path, we still need to actually set our own path. We had been doing that on the pants side -- instead, this PR adds a (very short) python script at the path of the real `xz` binary (a suggestion from @illicitonion in that PR) which sets `LD_LIBRARY_PATH` or `DYLD_LIBRARY_PATH` appropriately, not clobbering any existing values. This works with the current version of pantsbuild/pants#5936 for me on osx and linux. statically link xz (pantsbuild#74) See pantsbuild#71, pantsbuild#72, pantsbuild#73. Our `xz` is picking up travis's `xz` installation and using its (incompatible) `liblzma` shared library. On the suggestion of @stuhood in pantsbuild/pants#5936, we statically link `xz` here to avoid ever having this problem ever again. Proof: *linux*: ``` > ldd ./xz-5.2.4-linux/xz-install/bin/xz linux-vdso.so.1 (0x00007ffc865cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f1fa385f000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f1fa34a3000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f1fa3cb4000) ``` *osx*: ``` > otool -L ./xz-5.2.4-linux/xz-install/bin/xz ./xz-5.2.4-linux/xz-install/bin/xz: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) ``` add llvm 6.0.0 package. (pantsbuild#75) See pantsbuild/pants#5991. This adds the binary I should have provided when we added the ability to download `BinaryTool`s from external urls, in the exact same way we do it within Pants. Upgrade isort to 4.3.4. (pantsbuild#76) This adds support for more python3 stdlib symbols. Also introduce a build script. Fixup isort path structure. Node.js 10.7.0 and Yarnpkg 1.7.0 (pantsbuild#80) Four small scripts, each named "build.sh" have been added. These will build a specific version of Node.js or Yarnpkg in the current directory. Symlinks from older macOS to 10.13 are included. Add build scripts and symlinks for Node 8.11.3 and Yarnpkg 1.6.0 (pantsbuild#81) Adding Node v10.13.0 build scripts for Mac 10.8-10.13 and Linux X64 (pantsbuild#86) Add go 1.10.5 (pantsbuild#87) Add build script for Thrift 0.11.0 (pantsbuild#83) Add build scripts and symlinks for Yarnpkg 1.12.3 (pantsbuild#89) Add updated go versions to pants binaries (pantsbuild#90) * Add updated go versions to pants binaries * Expanded the mac_versions into a map to encode the OSX min version requirements of the latest builds of go. Add build scripts and symlinks for Yarnpkg 1.16.0 (pantsbuild#91) * Add build scripts and symlinks for Yarnpkg 1.16.0 Add go 1.12.7 (pantsbuild#92) * Add go 1.12.7 Upgrade to cloc 1.82 (pantsbuild#93) Add cloc 1.80 (pantsbuild#94) The current version of cloc (version 1.82) has a bug where the --ignored flag is not working properly ( AlDanial/cloc#401 ). The pants test for the cloc goal tests this behavior, so as long as this bug is not fixed upstream, pants will fail this test. I've confirmed that version 1.80 of cloc does not have this bug, so we should be able to use it until upstream cloc fixes the bug in a new verson. Add go 1.13.8 for linux and mac 10.11+
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Go publishes pre-built artifacts for both macOS and Linux, simplifying
the pants mirroring of these artifacts. Here we add a script that has a
list of which architecture, macOS versions, and Go versions to download
and setup symlinks for. Running this script localy will setup the
environment for syncing Go binaries to s3.