-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Crash log #267
Comments
I don't know what causes it, but it happens with a certain level of regularity. Jftr, Prometheus itself is compiled at prometheus/prometheus@2f151d0
|
Could you recompile the Alertmanager version you are using with the |
|
Yea the acceptance tests have some known races (have a fix sitting somewhere). |
Same compile flags:
Died after 776 seconds:
|
Also happens with 04fbfb9:
|
It's segfaulting while trying to call into the libc for name resolution. Is
there possibly a mismatch between the libraries in your build and runtime
environments? Maybe try with netgo?
|
All binaries are built on the same machine as they are running on, so other than the Go dependencies which are pulled in automagically, this sounds unlikely. Shouldn't it be possible to catch something like that instead of just keeling over and dying, though? |
No, this is not a recoverable condition. An error handler could not do anything but keel over and die either. |
@matthiasr What do you mean "Maybe try with netgo"? I built this on the same machine I am running it on. 22b37c8 crashes as well in case you are interested in another dump. |
@RichiH depending on the version of Go, when CGO is enabled (which it needs to be for Alertmanager), it will use the system's libc.so to resolve DNS – and from the stack trace, the SEGFAULT happens while within this code path, probably while in the C library. I don't know what's happening there, but I do know that glibc does funky things like trying to load yet another shared library. It's curious that only Alertmanager tickles this, but unless we find out why it's going to be tricky to work around. However, one possible workaround would be to avoid the libc altogether and use Go's builtin resolver – from what I gather, you need to add Alternatively, maybe try with Go 1.6? Sorry I can't be more concretely helpful here. What distribution is this on, and how did you install Go? |
@matthiasr OK, thanks. I can try building with netgo, but I would like to test Go 1.6 first; if that fixes it bumping the requirements would suffice. This is a Fedora system with Go 1.4 installed via packages and Go 1.5 via that automagic requirement handling. I tried bumping the auto-downloader to 1.6 via
but that didn't pull in 1.6 automagically. |
Update: Go 1.6 didn't fix it. |
This fixes the issue:
|
At 8ad6ebc, return code 2.
The text was updated successfully, but these errors were encountered: