-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Illegal instruction (core dumped) on armv6l #2517
Comments
Our 32-bit ARM build is currently for armv6h or armhf or however you want to say "hard float" :) We don't have any armv6l boxes handy and the various ARM flavors have proven exceedingly difficult targets to hit -- especially with CGO enabled. We'd like to produce builds without CGO which would make supporting various ARM flavors far far easier. I'm not sure we'll have any resources to invest in fixing this until we removed the CGO dependency. |
I inadvertently muddled things by running from the I definitely realize that supporting the smaller Raspberry Pis might not be worth the effort, and just wanted to clarify. :)
|
I ran into this same issue attempting to run nomad on a Raspberry Pi zero and I was able to get nomad to compile to a state where it runs on the ARMv6l. The caveat is that I haven't fully tested nomad on the zero yet. But, I did want to document how I was able to get it to run. I have an ARMv7 board (I think it's a Raspberry Pi 2) which has more memory than the Zero too. I cloned this repo and checking out the latest tag on the ARMv7 box. I then modified the GNUmakefile file and added the GOARM=6 flag for the Go compiler to this section:
I then finished with a I'll note if I run into any further issues here on this ticket, if anyone wants. But, I did get it to build and thought I would note how I did it. For anyone else following this path, you need GO 1.9 and at least gcc 5 on your build Pi. Thanks! Update: I was able to get redis running on the Pi Zero using the example job. It spun up without issue. |
@muellerkyle Thanks for the fix! Would you, @ttyS0, and anyone else interested in the 32bit ARM build mind testing the binary I attached to #3823? |
I'll pull a copy of your build and try to get it tested tonight. Should there be a seperate GNUMake profile for the ARMv6 vs v7? |
It still crashes for me with The binary is showing up as a arch v7 for me.
:( |
I had the same result ( Ah, nevermind. I see that he did not get it to run. I can post my binary for testing. I believe that I compiled in on a Raspberry Pi 2. I've uploaded my binary here: https://github.com/muellerkyle/nomadpizero/releases/download/v0.7.1/nomad.tar.gz |
Yeah, so far I don't think anyone has figured out how to cross-compile for ARMv6 using our root Vagrant box which cuts our releases. Might have to leave ARMv6 up to community support for the time being... If nothing else I'd be happy to add a note to our downloads page about minimum ARM requirements if someone can concisely describe them. |
@schmichael for the time being I am happy to build on my Pi 3, the releases for 0.7.0 and 0.7.1 are currently there, I will build a 0.8.0 dev release today. |
Hi, not sure if this related or not, but when trying to get binaries built on RHEL 6 (x86_64) to run on RHEL 5 (x86_64), I would add the linker hash type to "both". GNU ld option: |
@shantanugadgil How does linker hash style relate to arm builds? |
It was just a matter of "psychic debugging" on my part. :) |
I got 0.8.4 to work on the pi zero using the vagrant box to build it by chaning CGO_ENABLED to 0
after copying the binary to the pi zero
but Update: I made some edits to this comment because I tried to reproduce all steps from fresh code and realised that I threw myself (and others) in the wrong direction. |
I recently found the core dump error when trying nomad on a small Pi Zero cluster. Using https://github.com/nicholasjackson/nomad-arm6l instructions, I was able to use a Docker image to cross compile, and run on a PiZero W. I've made a PR there: nicholasjackson/nomad-arm6l#2, but I would love to include this into Nomad build process as well. I'll take a look at this at some point during the week |
I've put a compiled zip file on my fork of the builder project. https://github.com/bltavares/nomad-arm6l/releases/tag/v0.8.7-armv6l I was able to get it running on Pi Zero, but not on Pi 3. I'll give it a try to add a new target for armv6l on the build script if this binary works for others. I didn't test further yet, other than connecting it to the cluster. Update: Testing this version, it does not allocate docker containers due to some |
ARM is a quite popular for small computing clusters, and an interesting pet project to try Nomad on. Unfortunately, the current ARM builds don't run on Raspberry Pi's 1, Zero or Zero W as it is compiled with some operations not available there. When executing the current binary, you receive this message: ``` Illegal instruction (core dumped) ``` The difference is caused by the version of the ARM instructions are available on each version of the Pi. | Version | Compiler arch | |-------------+----------------------------| | Pi v1, Zero | armel, floating point (v6) | | Pi v2, v3 | armhf, floating point (v7) | Fortunately, cross-compiling using Go is not so troublesome. It is possible to produce an executable using the non-hf version of `gcc`. This commit introduces a new `make` target for armel binaries. It renames the previous one so it is clear the difference between artifacts versions. A couple of extra compilation dependencies are needed on the build machine. Tested `armel` o a Pi Zero W 1.3 and Pi 3, which executes, but there are not yet any jobs running. The `armel` binary should work on all Raspberry Pi models, while the `armhf` only works on Pi 2/3 and it is optimized. Fixes hashicorp#2517
I was able to produce a new @myoung34 @schmichael I've opened a new PR. #5525 The artifacts should work after a |
I've pre-compiled a new |
@bltavares Ill test 0.9.0 in the next few days |
There is also @nicholasjackson's project, https://github.com/nicholasjackson/nomad-arm6l. He has released a Nomad 0.9.0 binary as well. We are not going to add |
Any I have a bunch of Rasberry Pi Zeros, Docker is running, Consul is running, this one is the last one missing |
I managed to compile It was a messy process of trial and error carried out in a docker container (I don't know |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
When trying to run nomad (v0.5.6) on an armv6l Raspberry Pi (Gen 1 and PiZero), the executable crashes. It does, however run on armv7l Raspberry Pi (Gen 2 & Gen 3). I'm running Arch Linux on the Pis in question, and using the
Linux Arm
nomad binary downloaded directly from nomadproject.io.The following is the last bit of output from strace:
The text was updated successfully, but these errors were encountered: