Update docs for rate limit xDS config management service

Signed-off-by: Renuka Fernando <renukapiyumal@gmail.com>

README.md

@@ -8,7 +8,7 @@
 - [Building and Testing](#building-and-testing)
   - [Docker-compose setup](#docker-compose-setup)
   - [Full test environment - Configure rate limits through files](#full-test-environment---configure-rate-limits-through-files)
-  - [Full test environment - Configure rate limits through xDS-Server](#full-test-environment---configure-rate-limits-through-xds-server)
+  - [Full test environment - Configure rate limits through an xDS Management Server](#full-test-environment---configure-rate-limits-through-an-xds-management-server)
   - [Self-contained end-to-end integration test](#self-contained-end-to-end-integration-test)
 - [Configuration](#configuration)
   - [The configuration format](#the-configuration-format)
@@ -26,6 +26,8 @@
       - [Example 6](#example-6)
       - [Example 7](#example-7)
   - [Loading Configuration](#loading-configuration)
+    - [File Based Configuration Loading](#file-based-configuration-loading)
+    - [xDS Management Server Based Configuration Loading](#xds-management-server-based-configuration-loading)
   - [Log Format](#log-format)
   - [GRPC Keepalive](#grpc-keepalive)
 - [Request Fields](#request-fields)
@@ -162,7 +164,7 @@ To see the metrics in the example
 curl http://localhost:9102/metrics | grep -i shadow
 ```
 
-## Full test environment - Configure rate limits through xDS-Server
+## Full test environment - Configure rate limits through an xDS Management Server
 
 To run a fully configured environment to demo Envoy based rate limiting, run:
 
@@ -183,7 +185,7 @@ curl localhost:8888/twoheader -H "foo: foo" -H "baz: shady" # This will never be
 curl localhost:8888/twoheader -H "foo: foo" -H "baz: not-so-shady" # This is subject to rate-limiting because the it's now in shadow_mode
 ```
 
-Edit[ `examples/xds-sotw-config-server/resource.go`](examples/xds-sotw-config-server/resource.go) to test different rate limit configs.
+Edit[`examples/xds-sotw-config-server/resource.go`](examples/xds-sotw-config-server/resource.go) to test different rate limit configs.
 
 To see the metrics in the example
 
@@ -524,6 +526,18 @@ descriptors:
 
 ## Loading Configuration
 
+Rate limit service supports following configuration loading methods. You can define which methods to use by configuring environment variable `CONFIG_TYPE`.
+
+| Config Loading Method                                                             | Value for Environment Variable `CONFIG_TYPE` |
+| --------------------------------------------------------------------------------- | -------------------------------------------- |
+| [File Based Configuration Loading](#file-based-configuration-loading)             | `FILE` (Default)                             |
+| [xDS Server Based Configuration Loading](#xds-server-based-configuration-loading) | `GRPC_XDS_SOTW`                              |
+
+When the environment variable `FORCE_START_WITHOUT_INITIAL_CONFIG` set to `false`, the Rate limit service will wait for initial rate limit configuration before
+starting the server (gRPC, Rest server endpoints). When set to `true` the server will start even without initial configuration.
+
+### File Based Configuration Loading
+
 The Ratelimit service uses a library written by Lyft called [goruntime](https://github.com/lyft/goruntime) to do configuration loading. Goruntime monitors
 a designated path, and watches for symlink swaps to files in the directory tree to reload configuration files.
 
@@ -557,6 +571,31 @@ For more information on how runtime works you can read its [README](https://gith
 By default it is not possible to define multiple configuration files within `RUNTIME_SUBDIRECTORY` referencing the same domain.
 To enable this behavior set `MERGE_DOMAIN_CONFIG` to `true`.
 
+### xDS Management Server Based Configuration Loading
+
+xDS Management Server is a gRPC server which implements the [Aggregated Discovery Service (ADS)](https://github.com/envoyproxy/data-plane-api/blob/97b6dae39046f7da1331a4dc57830d20e842fc26/envoy/service/discovery/v3/ads.proto).
+The xDS Management server serves [Discovery Response](https://github.com/envoyproxy/data-plane-api/blob/97b6dae39046f7da1331a4dc57830d20e842fc26/envoy/service/discovery/v3/discovery.proto#L69) with [Ratelimit Configuration Resources](api/ratelimit/config/ratelimit/v3/rls_conf.proto)
+and with Type URL `"type.googleapis.com/ratelimit.config.ratelimit.v3.RateLimitConfig"`.
+The xDS client in the Rate limit service configure Rate limit service with the provided configuration.
+For more information on xDS protocol please refer to the [envoy proxy documentation](https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol).
+
+You can refer to [the sample xDS configuration management server](examples/xds-sotw-config-server/README.md).
+
+The xDS server for listening for configuration can be set via [settings](https://github.com/envoyproxy/ratelimit/blob/master/src/settings/settings.go)
+package with the following environment variables:
+
+```
+CONFIG_GRPC_XDS_NODE_ID default:"default"
+CONFIG_GRPC_XDS_SERVER_URL default:"localhost:18000"
+CONFIG_GRPC_XDS_SERVER_CONNECT_RETRY_INTERVAL default:"3s"
+```
+
+As well Ratelimit supports TLS connections, these can be configured using the following environment variables:
+
+1. `CONFIG_GRPC_XDS_SERVER_USE_TLS`: set to `"true"` to enable a TLS connection with the xDS configuration management server.
+2. `CONFIG_GRPC_XDS_CLIENT_TLS_CERT`, `CONFIG_GRPC_XDS_CLIENT_TLS_KEY`, and `CONFIG_GRPC_XDS_SERVER_TLS_CACERT` to provides files to specify a TLS connection configuration to the xDS configuration management server.
+3. `GRPC_CLIENT_TLS_SAN`: Override the SAN value to validate from the server certificate.
+
 ## Log Format
 
 A centralized log collection system works better with logs in json format. JSON format avoids the need for custom parsing rules.

examples/xds-sotw-config-server/Dockerfile

@@ -1,10 +1,6 @@
 FROM golang:1.18 AS build
 WORKDIR /xds-server
 
-# ENV GOPROXY=https://proxy.golang.org
-# COPY go.mod go.sum /xds-server/
-# RUN go mod download
-
 COPY . .
 
 RUN CGO_ENABLED=0 GOOS=linux go build -o /go/bin/xds-server -v main/main.go

examples/xds-sotw-config-server/README.md

@@ -1,16 +1,17 @@
 # Example Rate-limit Configuration SotW xDS Server
 
-This is an example of a trivial xDS V3 control plane server similar to the example server in [go-control-plane](https://github.com/envoyproxy/go-control-plane/tree/main/internal/example). It serves Rate limit configuration. You can run the example using the project top-level Makefile, e.g.:
+This is an example of a trivial xDS V3 control plane server similar to the example server in [go-control-plane](https://github.com/envoyproxy/go-control-plane/tree/main/internal/example). It serves sample Rate limit configuration. You can run the example using the project top-level docker-compose-example.yml, e.g.:
 
-```
-$ make example
+```bash
+export CONFIG_TYPE=GRPC_XDS_SOTW
+docker-compose -f docker-compose-example.yml --profile xds-config up --build --remove-orphans
 ```
 
-The Makefile builds the example server and then runs `build/example.sh` which runs both Envoy and the example server. The example server serves a configuration defined in `internal/example/resource.go`. If everything works correctly, you should be able to open a browser to [http://localhost:10000](http://localhost:10000) and see Envoy's website.
+The docker-compose builds and runs the example server along with Rate limit server. The example server serves a configuration defined in [`resource.go`](resource.go). If everything works correctly, you can follow the [examples in project top-level README.md file](../../README.md#examples).
 
 ## Files
 
 - [main/main.go](main/main.go) is the example program entrypoint. It instantiates the cache and xDS server and runs the xDS server process.
 - [resource.go](resource.go) generates a `Snapshot` structure which describes the configuration that the xDS server serves to Envoy.
 - [server.go](server.go) runs the xDS control plane server.
-- [logger.go](logger.go) implements the `pkg/log/Logger` interface which provides logging services to the cache.
+- [logger.go](logger.go) is the logger.

examples/xds-sotw-config-server/go.mod

@@ -3,12 +3,11 @@ module github.com/envoyproxy/ratelimit/examples/xds-sotw-config-server
 go 1.18
 
 // TODO: (renuka) Remove this replace once, https://github.com/envoyproxy/go-control-plane/pull/598 is merged
-replace github.com/envoyproxy/go-control-plane => github.com/renuka-fernando/go-control-plane v0.0.1
+replace github.com/envoyproxy/go-control-plane => github.com/renuka-fernando/go-control-plane v0.0.2
 
 require (
 	github.com/envoyproxy/go-control-plane v0.10.1
 	google.golang.org/grpc v1.45.0
-	google.golang.org/protobuf v1.28.0
 )
 
 require (
@@ -20,4 +19,5 @@ require (
 	golang.org/x/sys v0.0.0-20210816183151-1e6c022a8912 // indirect
 	golang.org/x/text v0.3.7 // indirect
 	google.golang.org/genproto v0.0.0-20220329172620-7be39ac1afc7 // indirect
+	google.golang.org/protobuf v1.28.0 // indirect
 )

examples/xds-sotw-config-server/go.sum

@@ -132,8 +132,8 @@ github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZ
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
-github.com/renuka-fernando/go-control-plane v0.0.1 h1:XURYmEmqrBkiT/JNALD3YC3blB4ubbjZyz5oOpqKLI8=
-github.com/renuka-fernando/go-control-plane v0.0.1/go.mod h1:fJJn/j26vwOu972OllsvAgJJM//w9BV6Fxbg2LuVd34=
+github.com/renuka-fernando/go-control-plane v0.0.2 h1:tvvR+mkPz7ab2JZ5N6TQ/V3TOue8PmtTbuMuZxqVKbU=
+github.com/renuka-fernando/go-control-plane v0.0.2/go.mod h1:fJJn/j26vwOu972OllsvAgJJM//w9BV6Fxbg2LuVd34=
 github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=

examples/xds-sotw-config-server/resource.go

@@ -23,6 +23,7 @@ import (
 func makeRlsConfig() []types.Resource {
 	return []types.Resource{
 		&rls_config.RateLimitConfig{
+			Name:   "mongo_cps",
 			Domain: "mongo_cps",
 			Descriptors: []*rls_config.RateLimitDescriptor{
 				{
@@ -44,6 +45,7 @@ func makeRlsConfig() []types.Resource {
 			},
 		},
 		&rls_config.RateLimitConfig{
+			Name:   "rl",
 			Domain: "rl",
 			Descriptors: []*rls_config.RateLimitDescriptor{
 				{

go.mod

@@ -2,8 +2,8 @@ module github.com/envoyproxy/ratelimit
 
 go 1.18
 
-replace github.com/envoyproxy/go-control-plane => /Users/renuka/git/go-control-plane
-// replace github.com/envoyproxy/go-control-plane => github.com/renuka-fernando/go-control-plane v0.0.1
+// TODO: (renuka) Remove this replace once, https://github.com/envoyproxy/go-control-plane/pull/598 is merged
+replace github.com/envoyproxy/go-control-plane => github.com/renuka-fernando/go-control-plane v0.0.2
 
 require (
 	github.com/alicebob/miniredis/v2 v2.23.0

go.sum

@@ -69,7 +69,6 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
 github.com/envoyproxy/protoc-gen-validate v0.6.7 h1:qcZcULcd/abmQg6dwigimCNEyi4gg31M/xaciQlDml8=
 github.com/envoyproxy/protoc-gen-validate v0.6.7/go.mod h1:dyJXwwfPK2VSqiB9Klm1J6romD608Ba7Hij42vrOBCo=
-github.com/envoyproxy/ratelimit v1.4.1-0.20220928150143-0b2f4d5fb04b/go.mod h1:qd8mgua4nBCzIWo+ClmfUqGxFEu+Tm9/CDCs5XRFUsA=
 github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
 github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
 github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
@@ -185,6 +184,8 @@ github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZ
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/renuka-fernando/go-control-plane v0.0.2 h1:tvvR+mkPz7ab2JZ5N6TQ/V3TOue8PmtTbuMuZxqVKbU=
+github.com/renuka-fernando/go-control-plane v0.0.2/go.mod h1:fJJn/j26vwOu972OllsvAgJJM//w9BV6Fxbg2LuVd34=
 github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
 github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
 github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
@@ -367,12 +368,10 @@ golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210816183151-1e6c022a8912/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10 h1:WIoqL4EROvwiPdUtaip4VcDdpZ4kha7wBWZrbVKCIZg=
 golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=

src/config/config_impl.go

@@ -232,8 +232,8 @@ func validateYamlKeys(fileName string, config_map map[interface{}]interface{}) {
 	}
 }
 
-// Load a single YAML config file into the global config.
-// @param config specifies the file contents to load.
+// Load a single YAML config into the global config.
+// @param config specifies the yamlRoot struct to load.
 func (this *rateLimitConfigImpl) loadConfig(config RateLimitConfigToLoad) {
 	root := config.ConfigYaml
 
@@ -341,6 +341,9 @@ func descriptorKey(domain string, descriptor *pb_struct.RateLimitDescriptor) str
 	return domain + "." + rateLimitKey
 }
 
+// ConfigFileContentToYaml converts a single YAML (string content) into yamlRoot struct with validating yaml keys.
+// @param fileName specifies the name of the file.
+// @param content specifies the string content of the yaml file.
 func ConfigFileContentToYaml(fileName, content string) *yamlRoot {
 	// validate keys in config with generic map
 	any := map[interface{}]interface{}{}

src/provider/provider.go

@@ -4,8 +4,14 @@ import (
 	"github.com/envoyproxy/ratelimit/src/config"
 )
 
+// RateLimitConfigProvider is the interface for configurations providers.
 type RateLimitConfigProvider interface {
+	// ConfigUpdateEvent returns a receive-only channel for retrieve configuration updates
+	// The provider implementer should send config update to this channel when it detect a config update
+	// Config receiver waits for configuration updates
 	ConfigUpdateEvent() <-chan ConfigUpdateEvent
+
+	// Stop stops the configuration provider watch for configurations.
 	Stop()
 }