Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make compact lifecycle more flexible to be overridden for sharded compaction #5964

Merged
merged 26 commits into from
Jul 14, 2023
Merged

Make compact lifecycle more flexible to be overridden for sharded compaction #5964

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
c61ac66
POC on sharded compaction
alexqyle Dec 14, 2022
45a4cfa
Refactored partition info to be part of meta
alexqyle Jan 3, 2023
05f1048
Merge branch 'main' into sharded-compaction
alexqyle Jan 3, 2023
9dbe33e
Override prometheus populate block func
alexqyle Mar 7, 2023
712e401
Merge branch 'main' into sharded-compaction
alexqyle Apr 11, 2023
c71ed9d
Make block populator plugable through CompactionLifecycleCallback
alexqyle Apr 13, 2023
554b19d
Uncommented code
alexqyle Apr 13, 2023
1fa6b60
Merge branch 'main' into sharded-compaction
alexqyle Apr 13, 2023
66f60d8
refactor
alexqyle Apr 13, 2023
42dbe8a
rename
alexqyle Apr 13, 2023
3408c8f
Merge branch 'main' into sharded-compaction
alexqyle Apr 20, 2023
ed90f79
make Group.partitionInfo nil-able
alexqyle Apr 20, 2023
390496b
Merge branch 'main' into sharded-compaction
alexqyle Apr 26, 2023
53b6992
Merge branch 'main' into sharded-compaction
alexqyle May 1, 2023
ed28417
Merge branch 'main' into sharded-compaction
alexqyle May 9, 2023
1766f20
Merge branch 'main' into sharded-compaction
alexqyle May 10, 2023
bc2be70
add extension field to thanos meta
alexqyle Jun 8, 2023
a8d2b0b
Merge branch 'main' into sharded-compaction
alexqyle Jun 8, 2023
28927a3
Merge branch 'main' into sharded-compaction
alexqyle Jul 11, 2023
ff552b1
fixed merge issue
alexqyle Jul 11, 2023
d20a9a4
Merge branch 'main' into sharded-compaction
alexqyle Jul 11, 2023
8b75147
Clean up
alexqyle Jul 12, 2023
8fe0d04
Merge branch 'main' into sharded-compaction
alexqyle Jul 12, 2023
fba2e9f
Merge branch 'main' into sharded-compaction
alexqyle Jul 13, 2023
c40a67f
Merge branch 'main' into sharded-compaction
alexqyle Jul 14, 2023
169f38e
Added comment to ConvertExtensions func
alexqyle Jul 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 47 additions & 49 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,12 @@ module github.com/thanos-io/thanos
go 1.18

require (
cloud.google.com/go/storage v1.27.0 // indirect
cloud.google.com/go/trace v1.4.0
cloud.google.com/go/storage v1.28.1 // indirect
cloud.google.com/go/trace v1.8.0
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v1.8.3
github.com/NYTimes/gziphandler v1.1.1
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137
github.com/alicebob/miniredis/v2 v2.22.0
github.com/armon/go-metrics v0.4.0 // indirect
github.com/blang/semver/v4 v4.0.0
github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b
github.com/cespare/xxhash v1.1.0
Expand All @@ -31,7 +30,7 @@ require (
github.com/gogo/protobuf v1.3.2
github.com/gogo/status v1.1.1
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
github.com/golang/protobuf v1.5.2
github.com/golang/protobuf v1.5.3
github.com/golang/snappy v0.0.4
github.com/googleapis/gax-go v2.0.2+incompatible
github.com/gorilla/mux v1.8.0 // indirect
Expand All @@ -47,7 +46,7 @@ require (
github.com/leanovate/gopter v0.2.9
github.com/lightstep/lightstep-tracer-go v0.25.0
github.com/lovoo/gcloud-opentracing v0.3.0
github.com/miekg/dns v1.1.50
github.com/miekg/dns v1.1.53
github.com/minio/minio-go/v7 v7.0.45 // indirect
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
github.com/oklog/run v1.1.0
Expand All @@ -62,12 +61,12 @@ require (
github.com/prometheus/alertmanager v0.25.0
github.com/prometheus/client_golang v1.14.0
github.com/prometheus/client_model v0.3.0
github.com/prometheus/common v0.39.1-0.20230202092144-f9c1994be032
github.com/prometheus/exporter-toolkit v0.8.2
github.com/prometheus/common v0.42.0
github.com/prometheus/exporter-toolkit v0.9.1
// Prometheus maps version 2.x.y to tags v0.x.y.
github.com/prometheus/prometheus v0.42.0
github.com/prometheus/prometheus v0.43.1-0.20230404063149-1936868e9d13
github.com/sony/gobreaker v0.5.0
github.com/stretchr/testify v1.8.1
github.com/stretchr/testify v1.8.2
github.com/thanos-community/promql-engine v0.0.0-20230224075812-ae04bbea7613
github.com/thanos-io/objstore v0.0.0-20230201072718-11ffbc490204
github.com/uber/jaeger-client-go v2.30.0+incompatible
Expand All @@ -77,21 +76,21 @@ require (
go.elastic.co/apm v1.11.0
go.elastic.co/apm/module/apmot v1.11.0
go.opentelemetry.io/contrib/propagators/ot v1.13.0 // indirect
go.opentelemetry.io/otel v1.13.0
go.opentelemetry.io/otel v1.14.0
go.opentelemetry.io/otel/bridge/opentracing v1.12.0
go.opentelemetry.io/otel/sdk v1.12.0
go.opentelemetry.io/otel/trace v1.13.0
go.opentelemetry.io/otel/sdk v1.14.0
go.opentelemetry.io/otel/trace v1.14.0
go.uber.org/atomic v1.10.0
go.uber.org/automaxprocs v1.5.1
go.uber.org/goleak v1.2.0
golang.org/x/crypto v0.3.0
golang.org/x/net v0.7.0
go.uber.org/goleak v1.2.1
golang.org/x/crypto v0.7.0
golang.org/x/net v0.8.0
golang.org/x/sync v0.1.0
golang.org/x/text v0.7.0
golang.org/x/text v0.8.0
golang.org/x/time v0.3.0
google.golang.org/api v0.108.0 // indirect
google.golang.org/genproto v0.0.0-20230131230820-1c016267d619 // indirect
google.golang.org/grpc v1.52.1
google.golang.org/api v0.114.0 // indirect
google.golang.org/genproto v0.0.0-20230306155012-7f2fa6fef1f4 // indirect
google.golang.org/grpc v1.53.0
google.golang.org/grpc/examples v0.0.0-20211119005141-f45e61797429
gopkg.in/alecthomas/kingpin.v2 v2.2.6
gopkg.in/fsnotify.v1 v1.4.7
Expand All @@ -105,25 +104,25 @@ require (
)

require (
cloud.google.com/go v0.105.0 // indirect
cloud.google.com/go/compute v1.14.0 // indirect
cloud.google.com/go/iam v0.8.0 // indirect
cloud.google.com/go v0.110.0 // indirect
cloud.google.com/go/compute v1.18.0 // indirect
cloud.google.com/go/iam v0.12.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.1.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v0.5.1 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v0.7.0 // indirect
go.opentelemetry.io/contrib/samplers/jaegerremote v0.7.0
go.opentelemetry.io/otel/exporters/jaeger v1.12.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.12.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.12.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.12.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.14.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.14.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.14.0
)

require (
go.opentelemetry.io/contrib/propagators/autoprop v0.38.0
go4.org/intern v0.0.0-20220617035311-6925f38cc365
golang.org/x/exp v0.0.0-20230124195608-d38c7dcee874
golang.org/x/exp v0.0.0-20230307190834-24139beb5833
)

require go4.org/unsafe/assume-no-moving-gc v0.0.0-20230209150437-ee73d164e760 // indirect
Expand All @@ -136,8 +135,8 @@ require (
github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a // indirect
github.com/aliyun/aliyun-oss-go-sdk v2.2.2+incompatible // indirect
github.com/armon/go-radix v1.0.0 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/aws/aws-sdk-go v1.44.187 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aws/aws-sdk-go v1.44.217 // indirect
github.com/aws/aws-sdk-go-v2 v1.16.0 // indirect
github.com/aws/aws-sdk-go-v2/config v1.15.1 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.11.0 // indirect
Expand All @@ -154,7 +153,7 @@ require (
github.com/cenkalti/backoff/v4 v4.2.0 // indirect
github.com/chromedp/sysutil v1.0.0 // indirect
github.com/clbanning/mxj v1.8.4 // indirect
github.com/coreos/go-systemd/v22 v22.4.0 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/dennwc/varint v1.0.0 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/dnaeon/go-vcr v1.2.0 // indirect
Expand All @@ -169,31 +168,30 @@ require (
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-openapi/analysis v0.21.4 // indirect
github.com/go-openapi/errors v0.20.3 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.20.0 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/loads v0.21.2 // indirect
github.com/go-openapi/spec v0.20.7 // indirect
github.com/go-openapi/spec v0.20.8 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/go-openapi/validate v0.22.0 // indirect
github.com/go-openapi/validate v0.22.1 // indirect
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
github.com/gobwas/ws v1.1.0 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/gogo/googleapis v1.4.0 // indirect
github.com/golang-jwt/jwt/v4 v4.4.3 // indirect
github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
github.com/google/go-cmp v0.5.9
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/pprof v0.0.0-20230111200839-76d1ae5aea2b // indirect
github.com/google/pprof v0.0.0-20230228050547-1710fef4ab10 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.1 // indirect
github.com/googleapis/gax-go/v2 v2.7.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.1 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
github.com/googleapis/gax-go/v2 v2.7.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.15.2 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/joeshaw/multierror v0.0.0-20140124173710-69b34d4ec901 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/julienschmidt/httprouter v1.3.0 // indirect
github.com/klauspost/cpuid/v2 v2.1.0 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20210210170715-a8dfcb80d3a7 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
Expand Down Expand Up @@ -229,25 +227,25 @@ require (
github.com/yusufpapurcu/wmi v1.2.2 // indirect
go.elastic.co/apm/module/apmhttp v1.11.0 // indirect
go.elastic.co/fastjson v1.1.0 // indirect
go.mongodb.org/mongo-driver v1.11.0 // indirect
go.mongodb.org/mongo-driver v1.11.2 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.39.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.40.0 // indirect
go.opentelemetry.io/contrib/propagators/aws v1.13.0 // indirect
go.opentelemetry.io/contrib/propagators/b3 v1.13.0 // indirect
go.opentelemetry.io/contrib/propagators/jaeger v1.13.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.12.0 // indirect
go.opentelemetry.io/otel/metric v0.36.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.14.0 // indirect
go.opentelemetry.io/otel/metric v0.37.0 // indirect
go.opentelemetry.io/proto/otlp v0.19.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
golang.org/x/mod v0.7.0 // indirect
golang.org/x/oauth2 v0.4.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/tools v0.5.0 // indirect
golang.org/x/mod v0.9.0 // indirect
golang.org/x/oauth2 v0.6.0 // indirect
golang.org/x/sys v0.6.0 // indirect
golang.org/x/tools v0.7.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
gonum.org/v1/gonum v0.12.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/ini.v1 v1.66.6 // indirect
google.golang.org/protobuf v1.29.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
howett.net/plist v0.0.0-20181124034731-591f970eefbb // indirect
)

Expand Down
Loading