allow specifying installation namespace in script (grafana#335)

* allow specifying installation namespace in script * specify ARN and add to ServiceAccount annotation * fix NAMESPACE typo * add namespace to serviceaccount * add region * update changelog, docs
findingrish · Jan 20, 2021 · e99a0d2 · e99a0d2
1 parent 2f459bb
commit e99a0d2
Show file tree

Hide file tree

Showing 23 changed files with 151 additions and 63 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,5 @@
 NOTE: FreeBSD builds have not been available since v0.6.0 due to a
-cross-compilation issue. The issue for tracking adding support back 
+cross-compilation issue. The issue for tracking adding support back
 can be found at [#317](https://github.com/grafana/agent/issues/317).
 
 # Master (unreleased)
@@ -11,6 +11,9 @@ can be found at [#317](https://github.com/grafana/agent/issues/317).
 
 - [ENHANCEMENT] A sigv4 install script for Prometheus has been added. (@rfratto)
 
+- [ENHANCEMENT] NAMESPACE may be passed as an environment variable to the
+  Kubernetes install scripts to specify an installation namespace. (@rfratto)
+
 - [BUGFIX] The K8s API server scrape job will use the API server Service name
   when resolving IP addresses for Prometheus service discovery using the
   "Endpoints" role. (@hjet)
@@ -20,10 +23,10 @@ can be found at [#317](https://github.com/grafana/agent/issues/317).
 
 # v0.10.0 (2021-01-13)
 
-- [FEATURE] Prometheus `remote_write` now supports SigV4 authentication using 
+- [FEATURE] Prometheus `remote_write` now supports SigV4 authentication using
   the [AWS default credentials
   chain](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html).
-  This enables the Agent to send metrics to Amazon Managed Prometheus without 
+  This enables the Agent to send metrics to Amazon Managed Prometheus without
   needing the [SigV4 Proxy](https://github.com/awslabs/aws-sigv4-proxy).
   (@rfratto)
 
@@ -44,15 +47,15 @@ can be found at [#317](https://github.com/grafana/agent/issues/317).
   work on journals that use +ZSTD compression. (@rfratto)
 
 - [BUGFIX] Integrations will now function if the HTTP listen address was set to
-  a value other than the default. ([#206](https://github.com/grafana/agent/issues/206)) (@mattdurham) 
+  a value other than the default. ([#206](https://github.com/grafana/agent/issues/206)) (@mattdurham)
 
 - [BUGFIX] The default Loki installation will now be able to write its positions
   file. This was prevented by accidentally writing to a readonly volume mount.
   (@rfratto)
 
 # v0.9.1 (2021-01-04)
 
-- [ENHANCEMENT] agentctl will now be installed by the rpm and deb packages as 
+- [ENHANCEMENT] agentctl will now be installed by the rpm and deb packages as
   `grafana-agentctl`. (@rfratto)
 
 # v0.9.0 (2020-12-10)

diff --git a/README.md b/README.md
@@ -66,9 +66,9 @@ Simply copy and paste the following lines in your terminal (requires `envsubst`
 (GNU gettext)):
 
 ```
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl apply -f -
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" | kubectl apply -f -
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-tempo.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-tempo.sh)" | kubectl apply -f -
 ```
 
 Other installation methods can be found in our
@@ -93,9 +93,9 @@ quickly within the Agent. We aim to always base our vendor off of a recent offic
 Prometheus release and to keep the experimental changes not available in the upstream
 repository to a minimum.
 
-Please refer to the pinned 
-[Prometheus Vendor Update Tracking](https://github.com/grafana/agent/issues/112) issue 
-for our current vendored Prometheus release. 
+Please refer to the pinned
+[Prometheus Vendor Update Tracking](https://github.com/grafana/agent/issues/112) issue
+for our current vendored Prometheus release.
 
 For more context on our vendoring strategy, read our
 [repo maintenance guide](./docs/maintaining.md#grafanaprometheus-maintenance).

diff --git a/docs/getting-started.md b/docs/getting-started.md
@@ -60,8 +60,9 @@ applied.
 > **Warning**: Always verify scripts from the internet before running them.
 
 ```
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl -ndefault apply -f -
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl -ndefault apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-tempo.sh)" | kubectl apply -f -
 ```
 
 ### Kubernetes Manifest

diff --git a/production/README.md b/production/README.md
@@ -12,12 +12,14 @@ easiest to hardest:
 
 ## Install Script for Kubernetes
 
-The Grafana Cloud Agent repository comes with an installation script to
-configure remote write and return a Kubernetes manifest that uses our preferred
-defaults. To run the script, copy and paste this line in your terminal:
+The Grafana Cloud Agent repository comes with installation scripts to
+configure components and return a Kubernetes manifest that uses our preferred
+defaults. To run the script, copy and paste this in your terminal:
 
 ```
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl -ndefault apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-tempo.sh)" | kubectl apply -f -
 ```
 
 See the [Kubernetes README](./kubernetes/README.md) for more information.

diff --git a/production/kubernetes/README.md b/production/kubernetes/README.md
@@ -26,12 +26,13 @@ install script does the following:
    step 1.
 4. Prints out the final manifest to stdout without applying it.
 
-Here's a two-line script to copy and paste to install the Agent on
-Kubernetes for collecting metrics and logs (requires `envsubst` (GNU gettext)):
+Here's a script to copy and paste to install the Agent on Kubernetes for
+collecting metrics, logs, and traces (requires `envsubst` (GNU gettext)):
 
 ```
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl -ndefault apply -f -
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" | kubectl -ndefault apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" | kubectl apply -f -
+NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-tempo.sh)" | kubectl apply -f -
 ```
 
 ## Manually Applying

diff --git a/production/kubernetes/agent-bare.yaml b/production/kubernetes/agent-bare.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: grafana-agent
+  namespace: ${NAMESPACE}
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRole
@@ -36,12 +37,13 @@ roleRef:
 subjects:
 - kind: ServiceAccount
   name: grafana-agent
-  namespace: default
+  namespace: ${NAMESPACE}
 ---
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
   name: grafana-agent
+  namespace: ${NAMESPACE}
 spec:
   minReadySeconds: 10
   selector:

diff --git a/production/kubernetes/agent-loki.yaml b/production/kubernetes/agent-loki.yaml
@@ -262,7 +262,7 @@ data:
 kind: ConfigMap
 metadata:
   name: grafana-agent-logs
-  namespace: default
+  namespace: ${NAMESPACE}
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRole
@@ -297,13 +297,13 @@ roleRef:
 subjects:
 - kind: ServiceAccount
   name: grafana-agent-logs
-  namespace: default
+  namespace: ${NAMESPACE}
 ---
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
   name: grafana-agent-logs
-  namespace: default
+  namespace: ${NAMESPACE}
 spec:
   minReadySeconds: 10
   selector:

diff --git a/production/kubernetes/agent-sigv4.yaml b/production/kubernetes/agent-sigv4.yaml
@@ -1,7 +1,10 @@
 apiVersion: v1
 kind: ServiceAccount
 metadata:
+  annotations:
+    eks.amazonaws.com/role-arn: ${ROLE_ARN}
   name: grafana-agent
+  namespace: ${NAMESPACE}
 ---
 apiVersion: v1
 data:
@@ -13,6 +16,7 @@ data:
             remote_write:
               - sigv4:
                     enabled: true
+                    region: ${REGION}
                 url: ${REMOTE_WRITE_URL}
             scrape_configs:
               - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
@@ -88,11 +92,11 @@ data:
                     ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
                     insecure_skip_verify: false
               - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
-                job_name: default/kube-state-metrics
+                job_name: ${NAMESPACE}/kube-state-metrics
                 kubernetes_sd_configs:
                   - namespaces:
                         names:
-                          - default
+                          - ${NAMESPACE}
                     role: pod
                 relabel_configs:
                   - action: keep
@@ -110,11 +114,11 @@ data:
                     ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
                     insecure_skip_verify: false
               - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
-                job_name: default/node-exporter
+                job_name: ${NAMESPACE}/node-exporter
                 kubernetes_sd_configs:
                   - namespaces:
                         names:
-                          - default
+                          - ${NAMESPACE}
                     role: pod
                 relabel_configs:
                   - action: keep
@@ -183,6 +187,7 @@ data:
 kind: ConfigMap
 metadata:
   name: grafana-agent
+  namespace: ${NAMESPACE}
 ---
 apiVersion: v1
 data:
@@ -194,6 +199,7 @@ data:
             remote_write:
               - sigv4:
                     enabled: true
+                    region: ${REGION}
                 url: ${REMOTE_WRITE_URL}
             scrape_configs:
               - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
@@ -227,6 +233,7 @@ data:
 kind: ConfigMap
 metadata:
   name: grafana-agent-deployment
+  namespace: ${NAMESPACE}
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRole
@@ -261,12 +268,13 @@ roleRef:
 subjects:
 - kind: ServiceAccount
   name: grafana-agent
-  namespace: default
+  namespace: ${NAMESPACE}
 ---
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
   name: grafana-agent
+  namespace: ${NAMESPACE}
 spec:
   minReadySeconds: 10
   selector:
@@ -315,6 +323,7 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: grafana-agent-deployment
+  namespace: ${NAMESPACE}
 spec:
   minReadySeconds: 10
   replicas: 1

diff --git a/production/kubernetes/agent-tempo.yaml b/production/kubernetes/agent-tempo.yaml
@@ -61,7 +61,7 @@ data:
 kind: ConfigMap
 metadata:
   name: grafana-agent-traces
-  namespace: default
+  namespace: ${NAMESPACE}
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRole
@@ -96,14 +96,15 @@ roleRef:
 subjects:
 - kind: ServiceAccount
   name: grafana-agent-traces
-  namespace: default
+  namespace: ${NAMESPACE}
 ---
 apiVersion: v1
 kind: Service
 metadata:
   labels:
     name: grafana-agent-traces
   name: grafana-agent-traces
+  namespace: ${NAMESPACE}
 spec:
   ports:
   - name: agent-http-metrics
@@ -144,7 +145,7 @@ apiVersion: apps/v1
 kind: DaemonSet
 metadata:
   name: grafana-agent-traces
-  namespace: default
+  namespace: ${NAMESPACE}
 spec:
   minReadySeconds: 10
   selector:

diff --git a/production/kubernetes/agent.yaml b/production/kubernetes/agent.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: grafana-agent
+  namespace: ${NAMESPACE}
 ---
 apiVersion: v1
 data:
@@ -89,11 +90,11 @@ data:
                     ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
                     insecure_skip_verify: false
               - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
-                job_name: default/kube-state-metrics
+                job_name: ${NAMESPACE}/kube-state-metrics
                 kubernetes_sd_configs:
                   - namespaces:
                         names:
-                          - default
+                          - ${NAMESPACE}
                     role: pod
                 relabel_configs:
                   - action: keep
@@ -111,11 +112,11 @@ data:
                     ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
                     insecure_skip_verify: false
               - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
-                job_name: default/node-exporter
+                job_name: ${NAMESPACE}/node-exporter
                 kubernetes_sd_configs:
                   - namespaces:
                         names:
-                          - default
+                          - ${NAMESPACE}
                     role: pod
                 relabel_configs:
                   - action: keep
@@ -184,6 +185,7 @@ data:
 kind: ConfigMap
 metadata:
   name: grafana-agent
+  namespace: ${NAMESPACE}
 ---
 apiVersion: v1
 data:
@@ -229,6 +231,7 @@ data:
 kind: ConfigMap
 metadata:
   name: grafana-agent-deployment
+  namespace: ${NAMESPACE}
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRole
@@ -263,12 +266,13 @@ roleRef:
 subjects:
 - kind: ServiceAccount
   name: grafana-agent
-  namespace: default
+  namespace: ${NAMESPACE}
 ---
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
   name: grafana-agent
+  namespace: ${NAMESPACE}
 spec:
   minReadySeconds: 10
   selector:
@@ -317,6 +321,7 @@ apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: grafana-agent-deployment
+  namespace: ${NAMESPACE}
 spec:
   minReadySeconds: 10
   replicas: 1

diff --git a/production/kubernetes/build/templates/bare/main.jsonnet b/production/kubernetes/build/templates/bare/main.jsonnet
@@ -6,7 +6,7 @@ agent {
   },
 
   _config+:: {
-    namespace: 'default',
+    namespace: '${NAMESPACE}',
 
     // Since the config map isn't managed by Tanka, we don't want to
     // add the configmap's hash as an annotation for the Kubernetes