Remove prometheus version check?

[miekg]

Hi there,

We're using thanos (0.4) but it checks the version of prometheus (I understand why this is done) on startup. Unfortunate we compile our own prometheus and add some fluff to the version. This makes the check fail and prevents thanos (store) from starting.

We've patched out the check for now.

I'm not sure what the right course of action here is. Either allow the check to fail and still start (via some cmdline flag) or some such?

A better, but more involved way, would be to probe for functionality you need want from prometheus and only abort when that is not available.

[bwplotka]

Thanks for reporting. Yea I think sidecar erroring on custom build of Prometheus is wrong. I am pretty sure we should make this check non-blocking OR remove version fetch totally and try to grab flags and if fails with 404 just continue without those extra checks. What do you think?

PRs welcome (:

[miekg]

Removing code is always my preferred solution :)

Just logging and continuing might just work here? I.e

• version parsed and is OK -> continue
• version parsed not OK -> abort
• version not parsed -> log it -> continue

[bwplotka]

It's more involved if you see the code. AFAIK we check version ONLY to double check if we can continue with extra validation -> as only from 2.2.X there is new flags endpoint (:

I would say if we can drop version check and just check flags and if 404 then log and continue without validation (treat it as below 2.2.x Prometheus)

[miekg]

Ah yes, I see in sidecar.go:

func validatePrometheus(ctx context.Context, logger log.Logger, m *promMetadata) error {
	if m.version == nil {
		level.Warn(logger).Log("msg", "fetched version is nil or invalid. Unable to know whether Prometheus supports /version endpoint, skip validation")
		return nil
	}

	if m.version.LessThan(promclient.FlagsVersion) {
		level.Warn(logger).Log("msg",
			"Prometheus doesn't support flags endpoint, skip validation", "version", m.version.Original())
		return nil
	}

And then it will try to get the flags from prometheus and perform some validation.

From the looks of it, the version stuff can removed and just try to grab the flags; if they are there do the extra validation and otherwise just log something and return nil.

This means removing the version related functions from promclient/promclient.go and related changes in cmd/thanos/sidecar.go

[miekg]

Potential PR: #1163

[GiedriusS]

Closing as the PR was merged.