Azure: Support authentication using user-assigned managed identity (t…

…hanos-io#4636)

* Azure: Support authentication using user-assigned managed identity

Adds support for [user-assigned managed identity](https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview#managed-identity-types).

When using the `user_assigned_id` the change does not require setting `msi_resource` it defaults to `https://<storage account name>.<endpoint>` unless the config explicitly sets the `msi_resource`.

Tested against Azure.

Signed-off-by: Amr Hanafi (MAHDI)) <amrh@microsoft.com>

* refactor service token fetch into own method

Signed-off-by: Amr Hanafi (MAHDI)) <amrh@microsoft.com>

* do not use deprecated method

Signed-off-by: Amr Hanafi (MAHDI)) <amrh@microsoft.com>

CHANGELOG.md

@@ -25,6 +25,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
 - [#4552](https://github.com/thanos-io/thanos/pull/4552) Compact: Adds `thanos_compact_downsample_duration_seconds` histogram.
 - [#4594](https://github.com/thanos-io/thanos/pull/4594) reloader: Expose metrics in config reloader to give info on the last operation.
 - [#4623](https://github.com/thanos-io/thanos/pull/4623) query-frontend: made HTTP downstream tripper (client) configurable via parameters `--query-range.downstream-tripper-config` and `--query-range.downstream-tripper-config-file`. If your downstream URL is localhost or 127.0.0.1 then it is strongly recommended to bump `max_idle_conns_per_host` to at least 100 so that `query-frontend` could properly use HTTP keep-alive connections and thus reduce the latency of `query-frontend` by about 20%.
+- [#4636](https://github.com/thanos-io/thanos/pull/4636) Azure: Support authentication using user-assigned managed identity
 
 ### Fixed
 

docs/storage.md

@@ -314,6 +314,7 @@ config:
   endpoint: ""
   max_retries: 0
   msi_resource: ""
+  user_assigned_id: ""
   pipeline_config:
     max_tries: 0
     try_timeout: 0s
@@ -333,7 +334,11 @@ config:
     disable_compression: false
 ```
 
-If `msi_resource` is used, authentication is done via ServicePrincipalToken. The value for Azure should be `https://<storage-account-name>.blob.core.windows.net`. The generic `max_retries` will be used as value for the `pipeline_config`'s `max_tries` and `reader_config`'s `max_retry_requests`. For more control, `max_retries` could be ignored (0) and one could set specific retry values.
+If `msi_resource` is used, authentication is done via system-assigned managed identity. The value for Azure should be `https://<storage-account-name>.blob.core.windows.net`.
+
+If `user_assigned_id` is used, authentication is done via user-assigned managed identity. When using `user_assigned_id` the `msi_resource` defaults to `https://<storage_account>.<endpoint>`
+
+The generic `max_retries` will be used as value for the `pipeline_config`'s `max_tries` and `reader_config`'s `max_retry_requests`. For more control, `max_retries` could be ignored (0) and one could set specific retry values.
 
 #### OpenStack Swift
 

go.mod

@@ -5,6 +5,7 @@ require (
 	cloud.google.com/go/storage v1.10.0
 	github.com/Azure/azure-pipeline-go v0.2.3
 	github.com/Azure/azure-storage-blob-go v0.13.0
+	github.com/Azure/go-autorest/autorest/adal v0.9.15
 	github.com/Azure/go-autorest/autorest/azure/auth v0.5.8
 	github.com/NYTimes/gziphandler v1.1.1
 	github.com/alecthomas/units v0.0.0-20210208195552-ff826a37aa15

go.sum

@@ -86,8 +86,9 @@ github.com/Azure/go-autorest/autorest/adal v0.9.2/go.mod h1:/3SMAM86bP6wC9Ev35pe
 github.com/Azure/go-autorest/autorest/adal v0.9.5/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A=
 github.com/Azure/go-autorest/autorest/adal v0.9.11/go.mod h1:nBKAnTomx8gDtl+3ZCJv2v0KACFHWTB2drffI1B68Pk=
 github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M=
-github.com/Azure/go-autorest/autorest/adal v0.9.14 h1:G8hexQdV5D4khOXrWG2YuLCFKhWYmWD8bHYaXN5ophk=
 github.com/Azure/go-autorest/autorest/adal v0.9.14/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M=
+github.com/Azure/go-autorest/autorest/adal v0.9.15 h1:X+p2GF0GWyOiSmqohIaEeuNFNDY4I4EOlVuUQvFdWMk=
+github.com/Azure/go-autorest/autorest/adal v0.9.15/go.mod h1:tGMin8I49Yij6AQ+rvV+Xa/zwxYQB5hmsd6DkfAx2+A=
 github.com/Azure/go-autorest/autorest/azure/auth v0.4.2/go.mod h1:90gmfKdlmKgfjUpnCEpOJzsUEjrWDSLwHIG73tSXddM=
 github.com/Azure/go-autorest/autorest/azure/auth v0.5.8 h1:TzPg6B6fTZ0G1zBf3T54aI7p3cAT6u//TOXGPmFMOXg=
 github.com/Azure/go-autorest/autorest/azure/auth v0.5.8/go.mod h1:kxyKZTSfKh8OVFWPAgOgQ/frrJgeYQJPyR5fLFmXko4=
@@ -765,6 +766,8 @@ github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69
 github.com/gogo/status v1.0.3/go.mod h1:SavQ51ycCLnc7dGyJxp8YAmudx8xqiVrRf+6IXRsugc=
 github.com/gogo/status v1.1.0 h1:+eIkrewn5q6b30y+g/BJINVVdi2xH7je5MPJ3ZPK3JA=
 github.com/gogo/status v1.1.0/go.mod h1:BFv9nrluPLmrS0EmGVvLaPNmRosr9KapBYd5/hpY1WM=
+github.com/golang-jwt/jwt/v4 v4.0.0 h1:RAqyYixv1p7uEnocuy8P1nru5wprCh/MH2BIlW5z5/o=
+github.com/golang-jwt/jwt/v4 v4.0.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg=
 github.com/golang-migrate/migrate/v4 v4.7.0/go.mod h1:Qvut3N4xKWjoH3sokBccML6WyHSnggXm/DvMMnTsQIc=
 github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0=
 github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k=

pkg/objstore/azure/azure.go

@@ -57,6 +57,7 @@ type Config struct {
 	Endpoint           string         `yaml:"endpoint"`
 	MaxRetries         int            `yaml:"max_retries"`
 	MSIResource        string         `yaml:"msi_resource"`
+	UserAssignedID     string         `yaml:"user_assigned_id"`
 	PipelineConfig     PipelineConfig `yaml:"pipeline_config"`
 	ReaderConfig       ReaderConfig   `yaml:"reader_config"`
 	HTTPConfig         HTTPConfig     `yaml:"http_config"`
@@ -98,15 +99,24 @@ func (conf *Config) validate() error {
 
 	var errMsg []string
 	if conf.MSIResource == "" {
-		if conf.StorageAccountName == "" ||
-			conf.StorageAccountKey == "" {
-			errMsg = append(errMsg, "invalid Azure storage configuration")
-		}
-		if conf.StorageAccountName == "" && conf.StorageAccountKey != "" {
-			errMsg = append(errMsg, "no Azure storage_account specified while storage_account_key is present in config file; both should be present")
-		}
-		if conf.StorageAccountName != "" && conf.StorageAccountKey == "" {
-			errMsg = append(errMsg, "no Azure storage_account_key specified while storage_account is present in config file; both should be present")
+		if conf.UserAssignedID == "" {
+			if conf.StorageAccountName == "" ||
+				conf.StorageAccountKey == "" {
+				errMsg = append(errMsg, "invalid Azure storage configuration")
+			}
+			if conf.StorageAccountName == "" && conf.StorageAccountKey != "" {
+				errMsg = append(errMsg, "no Azure storage_account specified while storage_account_key is present in config file; both should be present")
+			}
+			if conf.StorageAccountName != "" && conf.StorageAccountKey == "" {
+				errMsg = append(errMsg, "no Azure storage_account_key specified while storage_account is present in config file; both should be present")
+			}
+		} else {
+			if conf.StorageAccountName == "" {
+				errMsg = append(errMsg, "UserAssignedID is configured but storage account name is missing")
+			}
+			if conf.StorageAccountKey != "" {
+				errMsg = append(errMsg, "UserAssignedID is configured but storage account key is used")
+			}
 		}
 	} else {
 		if conf.StorageAccountName == "" {

pkg/objstore/azure/azure_test.go

@@ -140,6 +140,25 @@ pipeline_config:
 		wantFailParse:    false,
 		wantFailValidate: false,
 	},
+	{
+		name: "Valid User Assigned Identity Config without Resource",
+		config: []byte(`storage_account: "myAccount"
+storage_account_key: ""
+user_assigned_id: "1234-56578678-655"
+container: "MyContainer"`),
+		wantFailParse:    false,
+		wantFailValidate: false,
+	},
+	{
+		name: "Valid User Assigned Identity Config with Resource",
+		config: []byte(`storage_account: "myAccount"
+storage_account_key: ""
+user_assigned_id: "1234-56578678-655"
+msi_resource: "https://example.blob.core.windows.net"
+container: "MyContainer"`),
+		wantFailParse:    false,
+		wantFailValidate: false,
+	},
 }
 
 func TestConfig_validate(t *testing.T) {

pkg/objstore/azure/helpers.go

@@ -15,6 +15,7 @@ import (
 
 	"github.com/Azure/azure-pipeline-go/pipeline"
 	blob "github.com/Azure/azure-storage-blob-go/azblob"
+	"github.com/Azure/go-autorest/autorest/adal"
 	"github.com/Azure/go-autorest/autorest/azure/auth"
 	"github.com/go-kit/kit/log"
 	"github.com/go-kit/kit/log/level"
@@ -37,11 +38,8 @@ func init() {
 }
 
 func getAzureStorageCredentials(logger log.Logger, conf Config) (blob.Credential, error) {
-	if conf.MSIResource != "" {
-		msiConfig := auth.NewMSIConfig()
-		msiConfig.Resource = conf.MSIResource
-
-		spt, err := msiConfig.ServicePrincipalToken()
+	if conf.MSIResource != "" || conf.UserAssignedID != "" {
+		spt, err := getServicePrincipalToken(logger, conf)
 		if err != nil {
 			return nil, err
 		}
@@ -68,6 +66,26 @@ func getAzureStorageCredentials(logger log.Logger, conf Config) (blob.Credential
 	return credential, nil
 }
 
+func getServicePrincipalToken(logger log.Logger, conf Config) (*adal.ServicePrincipalToken, error) {
+	resource := conf.MSIResource
+	if resource == "" {
+		resource = fmt.Sprintf("https://%s.%s", conf.StorageAccountName, conf.Endpoint)
+	}
+
+	msiConfig := auth.MSIConfig{
+		Resource: resource,
+	}
+
+	if conf.UserAssignedID != "" {
+		level.Debug(logger).Log("msg", "using user assigned identity", "clientId", conf.UserAssignedID)
+		msiConfig.ClientID = conf.UserAssignedID
+	} else {
+		level.Debug(logger).Log("msg", "using system assigned identity")
+	}
+
+	return msiConfig.ServicePrincipalToken()
+}
+
 func getContainerURL(ctx context.Context, logger log.Logger, conf Config) (blob.ContainerURL, error) {
 	credentials, err := getAzureStorageCredentials(logger, conf)