Skip to content
This repository has been archived by the owner on Aug 13, 2019. It is now read-only.

no overlapping on compaction when an existing block is not within default boundaries. #461

Merged

Conversation

krasi-georgiev
Copy link
Contributor

closes prometheus/prometheus#4643

Signed-off-by: Krasi Georgiev kgeorgie@redhat.com

@krasi-georgiev
Copy link
Contributor Author

@gouthamve , @brian-brazil current master is broken by d50b9a5

The logic for loading the WAL depends on knowing if there are already blocks so db.reload() needs to be called before head.Init()

The test in this PR ensures this behaviour and will pass after reverting d50b9a5

head.go Outdated
@@ -640,13 +640,20 @@ func (h *Head) Appender() Appender {
func (h *Head) appender() *headAppender {
return &headAppender{
head: h,
minValidTime: h.MaxTime() - h.chunkRange/2,
minValidTime: max(h.MinTime(), h.MaxTime()-h.chunkRange/2),
Copy link
Contributor Author

@krasi-georgiev krasi-georgiev Nov 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not entirely sure about this one so let me know if you think it might brake anything else.

With this change when there is no wal and no block the appender is limited to the timestamp of the first appended sample so it brakes few test that rely on adding samples with timestamps lower than the first appended sample.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding samples with timestamps lower than the first appended sample.

That's a valid use case.

block_test.go Outdated Show resolved Hide resolved
db.go Outdated Show resolved Hide resolved
head.go Outdated
@@ -640,13 +640,20 @@ func (h *Head) Appender() Appender {
func (h *Head) appender() *headAppender {
return &headAppender{
head: h,
minValidTime: h.MaxTime() - h.chunkRange/2,
minValidTime: max(h.MinTime(), h.MaxTime()-h.chunkRange/2),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding samples with timestamps lower than the first appended sample.

That's a valid use case.

Krasi Georgiev added 8 commits December 1, 2018 12:38
…ault boundaries.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
This reverts commit c772180.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
@krasi-georgiev
Copy link
Contributor Author

@gouthamve, @brian-brazil I have refactored the logic to what I think makes sense.

TestHead_ReadWAL fails because after loading the wal {Ref: 11, Labels: labels.FromStrings("a", "2")}, is garbage collected as it has no samples. Should it remain there or the test is wrong?

https://github.com/prometheus/tsdb/blob/01e8296ee1cdcf4c90de6c3ea7b2e2012513781f/head_test.go#L95-L132

@brian-brazil
Copy link
Contributor

That shouldn't be GCed, as the GC code isn't called in that test.

@krasi-georgiev
Copy link
Contributor Author

@brian-brazil as I mentioned I refactored the logic , I am asking if the behaviour is correct and whether series should remain in the head if they don't have any sample after loading the WAL.

@brian-brazil
Copy link
Contributor

Ah, I see that you added a manual gc. That behaviour is correct then, and the test needs updating.

Krasi Georgiev added 2 commits December 1, 2018 16:58
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
@krasi-georgiev
Copy link
Contributor Author

all tests passing and also updated the changelog

@krasi-georgiev
Copy link
Contributor Author

@gouthamve one final look would be good.

Copy link
Collaborator

@gouthamve gouthamve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

head.go Outdated Show resolved Hide resolved
Krasi Georgiev added 2 commits December 4, 2018 12:15
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

prometheus is not stable after restoring a snapshot.
3 participants