Ingester: bridge OpenTracing spans to Otel

[bboreham]

What this PR does

Set things up so we can see spans and events created inside Prometheus under QueryStream and QueryExemplars.

The bridge added in #2707 works when Prometheus creates a new Otel span, but when Prometheus calls SpanFromContext we need to have already set up a link from the current OpenTracing span to Otel.

As preparation, move the tracing bridge to its own package so we can call it without creating circular dependencies.

Example difference in output - top span is from an ingester without this PR and bottom span is with:

Checklist

• Tests updated.
• NA Documentation added.
• CHANGELOG.md updated.
• NA about-versioning.md updated with experimental features.

[bboreham]

Maybe I should do this here?

mimir/pkg/util/spanlogger/spanlogger.go

Line 33 in 93136aa

func NewWithLogger(ctx context.Context, logger log.Logger, method string, kvps ...interface{}) (*SpanLogger, context.Context) {

[56quarters]

LGTM

Maybe I should do this here?

Since we're just trying to improve ingester traces, that makes sense to me. The way we create spans across the codebase is kinda all over the place.

[pracucci]

I think we can merge as is, and consider doing it in the spanlogger in the future.