Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api/v2: Check if cluster peer is nil -> release-0.16 #1726

Merged
merged 1 commit into from
Jan 31, 2019

Conversation

mxinden
Copy link
Member

@mxinden mxinden commented Jan 25, 2019

When users start Alertmanager with --cluster.listen-address=, the
cluster will not be initialized, hence api.peer will be nil.

With this patch, api v2 skips populating the peers array in case
api.peer is nil.

Fixes #1725

Copy link

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these codes above will panic too

name := api.peer.Name()
status := api.peer.Status()

@mxinden
Copy link
Member Author

mxinden commented Jan 25, 2019

@yeya24 right, thanks for the hint, sorry for the low quality patch. Added a test and the fix.

api/v2/api.go Outdated
Name: &n.Name,
Address: &address,
})
// If api.peers == nil, Alertmanager cluster feature is disabled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would reverse this comment. "If alertmanager cluster feature is disabled, then api.peers == nil".

@mxinden
Copy link
Member Author

mxinden commented Jan 26, 2019

Looks like we will need to remove cluster in alertmanagerStatus from the required fields making it optional. I will follow up with that.

@mxinden mxinden changed the title api/v2: Check if cluster peer is nil [WIP] api/v2: Check if cluster peer is nil Jan 26, 2019
@mxinden mxinden changed the base branch from master to release-0.16 January 28, 2019 10:18
When users start Alertmanager with `--cluster.listen-address=`, the
cluster will not be initialized, hence api.peer will be `nil`. So far
this would result in a nil pointer dereference by the API v2 accessing
the api.peer field.

With this patch, api v2 skips populating the peers array, sets the name
to an empty string and the status to "disabled" in case `api.peer` is
nil.

Signed-off-by: Max Leonard Inden <IndenML@gmail.com>
@mxinden mxinden changed the title [WIP] api/v2: Check if cluster peer is nil [WIP] api/v2: Check if cluster peer is nil -> release-0.16 Jan 28, 2019
@mxinden
Copy link
Member Author

mxinden commented Jan 29, 2019

@stuartnelson3 @simonpasquier any comments on this (bugix for release-0.16, delaying the breaking change to master in #1728)?

@stuartnelson3
Copy link
Contributor

Looks good to me

@simonpasquier
Copy link
Member

👍

@mxinden mxinden changed the title [WIP] api/v2: Check if cluster peer is nil -> release-0.16 api/v2: Check if cluster peer is nil -> release-0.16 Jan 31, 2019
@mxinden mxinden merged commit aaa9b11 into prometheus:release-0.16 Jan 31, 2019
@mxinden mxinden mentioned this pull request Jan 31, 2019
@knweiss
Copy link
Contributor

knweiss commented Mar 13, 2019

@mxinden I just ran into this issue when I was testing HEAD of the alertmanager master branch. Is there a reason why you didn't merge #1726 into the master branch as well?

@mxinden
Copy link
Member Author

mxinden commented Mar 13, 2019

Instead of merging this patch into the master branch, we decided to merge a breaking fix into master via #1728. I have just checked out current master, build Alertmanager and ran it with --cluster.listen-address= which returns the following json on api/v2/status:

  "cluster": {
    "peers": null,
    "status": "disabled"
  },

It does not trigger a nil pointer exception in the Go http handler.

@knweiss if this issue still persists for you, would you mind creating a new Github issue? Thanks for the help!

@knweiss
Copy link
Contributor

knweiss commented Mar 13, 2019

@mxinden Sorry, I'm a little bit in a hurry and only have time for a quick reply: It no longer triggers the nil pointer exception but when I click "Status" in the web UI I get the following error:

Unexpected response from api: Problem with the value at json.cluster.peers: null Expecting a LIST

It comes from this line:

ui/app/src/Utils/Api.elm:            "Unexpected response from api: " ++ err_ 

(My api/v2/status is the same as you mentioned above.)

@mxinden
Copy link
Member Author

mxinden commented Mar 18, 2019

@knweiss we changed peers to be optional along with the patch above:

type alias ClusterStatus =
    { name : Maybe String
    , status : Status
    , peers : Maybe (List PeerStatus)
}

, peers : Maybe (List PeerStatus)

Can you make sure your browser is not using a cached version of the front-end?

@knweiss
Copy link
Contributor

knweiss commented Mar 19, 2019

@mxinden Branch release-0.16 (tag v0.16.1 ) worked fine but master branch (commit f40ecee as of today) did not. I still got the error line and could reproduce the problem by switching between both executables.

However, reloading without Firefox cache did the trick. Now the master branch executable renders the Status page, too! Thanks!

(I could be wrong but I also think I'm seeing the red "disabled" label for the first time in the v0.16.1 version. I'll keep an eye on caching effects from now on.)

@mxinden
Copy link
Member Author

mxinden commented Mar 29, 2019

@knweiss I think we should disable caching of our web assets to prevent confusions like this. I prepared #1817. Let me know what you think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants