instruments creation may cause panic when setting meter provider

[XSAM]

Description

This bug was found in CI after running TestMeterConcurrentSafe test and was introduced in #5754, where creating instruments while calling setDelegate method.

This is because creating instruments can be locked in this line:

opentelemetry-go/internal/global/meter.go

Line 350 in e47618f

m.mtx.Lock()

While setDelegate is cleaning instruments map:

opentelemetry-go/internal/global/meter.go

Line 144 in e47618f

m.instruments = nil

Then, the remaining code in the instrument creation method would trigger panic after the mutex is unlocked:

opentelemetry-go/internal/global/meter.go

Line 360 in e47618f

m.instruments[id] = i

Steps To Reproduce

1. Run

go test -timeout 30s -run ^TestMeterConcurrentSafe$ go.opentelemetry.io/otel/internal/global -count 1000

Expected behavior

No Panic

Proposal

• Fix the panic by cleaning the map but not releasing the map as a quick fix. Fix panic instruments creation when setting meter provider #5758
• This panic behavior shows the possibility that the SetMeterProvider would miss the delegation setting for instruments and registry. We also need to fix this. The simple solution could be to use mutex to lock the whole instrumentation creation. SetMeterProvider might miss the delegation for instruments and registries #5780
• Concurrent safe tests didn't serve the purpose well due to the lack of runs (we only ran it once in CI). We can run the tests for 100 times. Run ConcurrentSafe tests multiple times in CI #5759

[XSAM]

This blocks the release.