Skip to content

Commit

Permalink
Add OTel tracing bucket
Browse files Browse the repository at this point in the history
Refactor client.NewBucket

Add client.NewInstrumentedBucket

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
  • Loading branch information
kakkoyun committed Jul 6, 2023
1 parent ff1b35b commit 90f9ddf
Show file tree
Hide file tree
Showing 14 changed files with 426 additions and 94 deletions.
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
- [#43](https://github.com/thanos-io/objstore/pull/43) filesystem: abort filesystem bucket operations if the context has been cancelled
- [#44](https://github.com/thanos-io/objstore/pull/44) Add new metric to count total number of fetched bytes from bucket
- [#50](https://github.com/thanos-io/objstore/pull/50) Add Huawei Cloud OBS Object Storage Support
- [#59](https://github.com/thanos-io/objstore/pull/59) Adding method `IsCustomerManagedKeyError` on the bucket interface.
- [#59](https://github.com/thanos-io/objstore/pull/59) Adding method `IsCustomerManagedKeyError` on the bucket interface.
- [#61](https://github.com/thanos-io/objstore/pull/61) Add OpenTelemetry TracingBucket.

### Changed
- [#38](https://github.com/thanos-io/objstore/pull/38) *: Upgrade minio-go version to `v7.0.45`.
Expand Down
24 changes: 13 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ type Bucket interface {
Upload(ctx context.Context, name string, r io.Reader) error

// Delete removes the object with the given name.
// If object does not exists in the moment of deletion, Delete should throw error.
// If object does not exist in the moment of deletion, Delete should throw error.
Delete(ctx context.Context, name string) error

```
Expand Down Expand Up @@ -88,7 +88,7 @@ type BucketReader interface {
// IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.
IsObjNotFoundErr(err error) bool

// Attributes returns information about the specified object.
// IsCustomerManagedKeyError returns true if the permissions for key used to encrypt the object was revoked.
```
Those interfaces represent the object storage operations your code can use from `objstore` clients.
Expand Down Expand Up @@ -128,7 +128,7 @@ Current object storage client implementations:
| [Baidu BOS](#baidu-bos) | Beta | Production Usage | no | @yahaa |
| [Local Filesystem](#filesystem) | Stable | Testing and Demo only | yes | @bwplotka |
| [Oracle Cloud Infrastructure Object Storage](#oracle-cloud-infrastructure-object-storage) | Beta | Production Usage | yes | @aarontams,@gaurav-05,@ericrrath |
| [HuaweiCloud OBS](#huaweicloud-obs) | Beta | Production Usage | no | @setoru |
| [HuaweiCloud OBS](#huaweicloud-obs) | Beta | Production Usage | no | @setoru |
**Missing support to some object storage?** Check out [how to add your client section](#how-to-add-a-new-client-to-thanos)
Expand All @@ -153,6 +153,7 @@ config:
insecure: false
signature_version2: false
secret_key: ""
session_token: ""
put_user_metadata: {}
http_config:
idle_conn_timeout: 1m30s
Expand Down Expand Up @@ -643,18 +644,19 @@ You can also include any of the optional configuration just like the example in

##### HuaweiCloud OBS

To use HuaweiCloud OBS as an object store, you should apply for a HuaweiCloud Account to create an object storage bucket at first.
More details: [HuaweiCloud OBS](https://support.huaweicloud.com/obs/index.html)
To use HuaweiCloud OBS as an object store, you should apply for a HuaweiCloud Account to create an object storage bucket at first. More details: [HuaweiCloud OBS](https://support.huaweicloud.com/obs/index.html)

To configure HuaweiCloud Account to use OBS as storage store you need to set these parameters in YAML format stored in a file:

```yaml mdox-exec="go run scripts/cfggen/main.go --name=cos.Config"
type: OBS
config:
bucket: ""
endpoint: ""
access_key: ""
type: COS
config:
bucket: ""
region: ""
app_id: ""
endpoint: ""
secret_key: ""
secret_id: ""
http_config:
idle_conn_timeout: 1m30s
response_header_timeout: 2m
Expand All @@ -674,7 +676,7 @@ config:
prefix: ""
```

The `access_key` and `secret_key` field is required. The `http_config` field is optional for optimize HTTP transport settings.
The `access_key` and `secret_key` field is required. The `http_config` field is optional for optimize HTTP transport settings.

#### How to add a new client to Thanos?

Expand Down
10 changes: 8 additions & 2 deletions client/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ type BucketConfig struct {

// NewBucket initializes and returns new object storage clients.
// NOTE: confContentYaml can contain secrets.
func NewBucket(logger log.Logger, confContentYaml []byte, reg prometheus.Registerer, component string) (objstore.InstrumentedBucket, error) {
func NewBucket(logger log.Logger, confContentYaml []byte, component string) (objstore.Bucket, error) {
level.Info(logger).Log("msg", "loading bucket configuration")
bucketConf := &BucketConfig{}
if err := yaml.UnmarshalStrict(confContentYaml, bucketConf); err != nil {
Expand Down Expand Up @@ -91,5 +91,11 @@ func NewBucket(logger log.Logger, confContentYaml []byte, reg prometheus.Registe
return nil, errors.Wrap(err, fmt.Sprintf("create %s client", bucketConf.Type))
}

return objstore.NewTracingBucket(objstore.BucketWithMetrics(bucket.Name(), objstore.NewPrefixedBucket(bucket, bucketConf.Prefix), reg)), nil
return objstore.NewPrefixedBucket(bucket, bucketConf.Prefix), nil
}

// NewInstrumentedBucket initializes and returns new object storage clients.
// NOTE: confContentYaml can contain secrets.
func NewInstrumentedBucket(reg prometheus.Registerer, bkt objstore.Bucket) objstore.InstrumentedBucket {
return objstore.BucketWithMetrics(bkt.Name(), bkt, reg)
}
118 changes: 118 additions & 0 deletions client/factory_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
// Copyright (c) The Thanos Authors.
// Licensed under the Apache License 2.0.

package client

import (
"context"
"fmt"
"io/ioutil"

"github.com/go-kit/log"
"github.com/prometheus/client_golang/prometheus"
"go.opentelemetry.io/otel/trace"

"github.com/thanos-io/objstore/tracing/opentelemetry"
"github.com/thanos-io/objstore/tracing/opentracing"
)

func ExampleBucket() {
// Read the configuration file.
confContentYaml, err := ioutil.ReadFile("testconf/filesystem.conf.yml")
if err != nil {
panic(err)
}

// Create a new bucket.
bucket, err := NewBucket(log.NewNopLogger(), confContentYaml, "example")
if err != nil {
panic(err)
}

// Test it.
exists, err := bucket.Exists(context.Background(), "example")
if err != nil {
panic(err)
}
fmt.Println(exists)
// Output:
// false
}

func ExampleInstrumentedBucket() {
// Read the configuration file.
confContentYaml, err := ioutil.ReadFile("testconf/filesystem.conf.yml")
if err != nil {
panic(err)
}

// Create a new bucket.
bucket, err := NewBucket(log.NewNopLogger(), confContentYaml, "example")
if err != nil {
panic(err)
}

// Wrap it with instrumentation.
bucket = NewInstrumentedBucket(prometheus.NewRegistry(), bucket)

// Test it.
exists, err := bucket.Exists(context.Background(), "example")
if err != nil {
panic(err)
}
fmt.Println(exists)
// Output:
// false
}

func ExampleTracingBucketUsingOpenTracing() {

Check failure on line 68 in client/factory_test.go

View workflow job for this annotation

GitHub Actions / Linters (Static Analysis) for Go

tests: ExampleTracingBucketUsingOpenTracing refers to unknown identifier: TracingBucketUsingOpenTracing (govet)
// Read the configuration file.
confContentYaml, err := ioutil.ReadFile("testconf/filesystem.conf.yml")
if err != nil {
panic(err)
}

// Create a new bucket.
bucket, err := NewBucket(log.NewNopLogger(), confContentYaml, "example")
if err != nil {
panic(err)
}

// Wrap it with tracing.
bucket = opentracing.NewTracingBucket(bucket)

// Test it.
exists, err := bucket.Exists(context.Background(), "example")
if err != nil {
panic(err)
}
fmt.Println(exists)
// Output:
// false
}

func ExampleTracingBucketUsingOpenTelemetry() {

Check failure on line 94 in client/factory_test.go

View workflow job for this annotation

GitHub Actions / Linters (Static Analysis) for Go

tests: ExampleTracingBucketUsingOpenTelemetry refers to unknown identifier: TracingBucketUsingOpenTelemetry (govet)
// Read the configuration file.
confContentYaml, err := ioutil.ReadFile("testconf/filesystem.conf.yml")
if err != nil {
panic(err)
}

// Create a new bucket.
bucket, err := NewBucket(log.NewNopLogger(), confContentYaml, "example")
if err != nil {
panic(err)
}

// Wrap it with tracing.
bucket = opentelemetry.NewTracingBucket(trace.NewNoopTracerProvider().Tracer("bucket"), bucket)

// Test it.
exists, err := bucket.Exists(context.Background(), "example")
if err != nil {
panic(err)
}
fmt.Println(exists)
// Output:
// false
}
1 change: 0 additions & 1 deletion client/testconf/blank-gcs.conf.yml

This file was deleted.

3 changes: 0 additions & 3 deletions client/testconf/fake-gcs.conf.yml

This file was deleted.

3 changes: 3 additions & 0 deletions client/testconf/filesystem.conf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
type: "FILESYSTEM"
config:
directory: "./data"
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ require (
github.com/prometheus/client_golang v1.12.2
github.com/prometheus/common v0.36.0
github.com/tencentyun/cos-go-sdk-v5 v0.7.40
go.opentelemetry.io/otel v1.16.0
go.opentelemetry.io/otel/trace v1.16.0
go.uber.org/atomic v1.9.0
golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4
Expand Down Expand Up @@ -92,7 +94,6 @@ require (
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/ini.v1 v1.66.6 // indirect
gopkg.in/yaml.v3 v3.0.0 // indirect
)

require (
Expand Down
11 changes: 7 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -223,8 +223,8 @@ github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
Expand Down Expand Up @@ -385,7 +385,7 @@ github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81P
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1 h1:5TQK59W5E3v0r2duFAb7P95B6hEeOyEnHRa8MjYSMTY=
github.com/stretchr/testify v1.8.3 h1:RP3t2pwF7cMEbC1dqtB6poj3niw/9gnV4Cjg5oW5gtY=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.194/go.mod h1:7sCQWVkxcsR38nffDW057DRGk8mUjK1Ing/EFOK8s8Y=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/kms v1.0.194/go.mod h1:yrBKWhChnDqNz1xuXdSbWXG56XawEq0G5j1lg4VwBD4=
github.com/tencentyun/cos-go-sdk-v5 v0.7.40 h1:W6vDGKCHe4wBACI1d2UgE6+50sJFhRWU4O8IB2ozzxM=
Expand All @@ -403,6 +403,10 @@ go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.opencensus.io v0.23.0 h1:gqCw0LfLxScz8irSi8exQc7fyQ0fKQU/qnC/X8+V/1M=
go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
go.opentelemetry.io/otel v1.16.0 h1:Z7GVAX/UkAXPKsy94IU+i6thsQS4nb7LviLpnaNeW8s=
go.opentelemetry.io/otel v1.16.0/go.mod h1:vl0h9NUa1D5s1nv3A5vZOYWn8av4K8Ml6JDeHrT/bx4=
go.opentelemetry.io/otel/trace v1.16.0 h1:8JRpaObFoW0pxuVPapkgH8UhHQj+bJW8jJsCZEu5MQs=
go.opentelemetry.io/otel/trace v1.16.0/go.mod h1:Yt9vYq1SdNz3xdjZZK7wcXv1qv2pwLkqr2QVwea0ef0=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
Expand Down Expand Up @@ -858,8 +862,7 @@ gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0 h1:hjy8E9ON/egN1tAYqKb61G10WtihqetD4sz2H+8nIeA=
gopkg.in/yaml.v3 v3.0.0/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
Expand Down
22 changes: 0 additions & 22 deletions objstore_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,27 +72,6 @@ func TestMetricBucket_Close(t *testing.T) {
testutil.Assert(t, promtest.ToFloat64(bkt.lastSuccessfulUploadTime) > lastUpload)
}

func TestTracingReader(t *testing.T) {
r := bytes.NewReader([]byte("hello world"))
tr := newTracingReadCloser(NopCloserWithSize(r), nil)

size, err := TryToGetSize(tr)

testutil.Ok(t, err)
testutil.Equals(t, int64(11), size)

smallBuf := make([]byte, 4)
n, err := io.ReadFull(tr, smallBuf)
testutil.Ok(t, err)
testutil.Equals(t, 4, n)

// Verify that size is still the same, after reading 4 bytes.
size, err = TryToGetSize(tr)

testutil.Ok(t, err)
testutil.Equals(t, int64(11), size)
}

func TestDownloadUploadDirConcurrency(t *testing.T) {
r := prometheus.NewRegistry()
m := BucketWithMetrics("", NewInMemBucket(), r)
Expand Down Expand Up @@ -165,7 +144,6 @@ func TestTimingTracingReader(t *testing.T) {
tr = newTimingReadCloser(tr, "", m.opsDuration, m.opsFailures, func(err error) bool {
return false
}, m.opsFetchedBytes)
tr = newTracingReadCloser(tr, nil)

size, err := TryToGetSize(tr)

Expand Down
Loading

0 comments on commit 90f9ddf

Please sign in to comment.