From 142aa6dee2f194048a9d7cec7700fcc3053b8723 Mon Sep 17 00:00:00 2001 From: Frank Reno Date: Thu, 30 Jul 2020 10:08:49 -0600 Subject: [PATCH] Doc Updates (#686) * Initial commit. Docs refactoring. * Document configurable parameters and respective defaults. * Refactor Helm install, adding Helm3. * some tweaks and fixes. * some tweaks and fixes. * some tweaks and fixes. * Fixes from some feedback * Final updates to Helm Install * use sections better * fix typo * fix typo * add note on upgrading from pre 1.x * formatting * update non-helm insatllation * fix typos * fix typos * Update docs for existing prometheues operator install * more tweaks, add troubleshooting section * add pointer to merge where relevant * add pre-req and TOC * Update standalone prometheus * Remove legacy Helm3 doc * SideBySide install as well as consistent validaton step * make other methods more clear * Add missing props * For remoteWrite, refer to values.yaml * Update deploy/docs/Installation_with_Helm.md Co-authored-by: Vijit Singhal <56007827+vsinghal13@users.noreply.github.com> * fix issue from conflict merge * delete old helm3 doc and update root README. * Update Installation_with_Helm.md * Update SideBySidePrometheus.md * Update existingPrometheusDoc.md * Update standAlonePrometheus.md * Helm3 has fixed the namespace issue so update steps. Also port in missed content in the different installs for parity. Update non-helm install with improvements. * Fix formatting * Incorporate some feedback * Update deploy/docs/Non_Helm_Installation.md * Update deploy/docs/Non_Helm_Installation.md * Update deploy/docs/Non_Helm_Installation.md * Update deploy/docs/Non_Helm_Installation.md * Add reference to proxy based installation. * Last tweaks Co-authored-by: Vijit Singhal <56007827+vsinghal13@users.noreply.github.com> Co-authored-by: David Duke Co-authored-by: Marcin Stozek <58700054+perk-sumo@users.noreply.github.com> --- deploy/README.md | 2 - deploy/docs/Additional_Fluentd_Plugins.md | 4 +- deploy/docs/Helm3.md | 85 ----- deploy/docs/HelmChartConfiguration.md | 171 ++++++++++ deploy/docs/Installation_with_Helm.md | 132 ++++++-- deploy/docs/Non_Helm_Installation.md | 374 +++++++--------------- deploy/docs/SideBySidePrometheus.md | 180 ++++++++++- deploy/docs/Troubleshoot_Collection.md | 35 +- deploy/docs/existingPrometheusDoc.md | 186 +++++++++-- deploy/docs/standAlonePrometheus.md | 148 ++++++++- 10 files changed, 900 insertions(+), 417 deletions(-) delete mode 100644 deploy/docs/Helm3.md create mode 100644 deploy/docs/HelmChartConfiguration.md diff --git a/deploy/README.md b/deploy/README.md index 894ee08e17..d13bddebc6 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -52,8 +52,6 @@ K8s with AKS | 1.15
1.16
1.17 Helm | 2.14.13 (Linux) kubectl | 1.15.0 -NOTE: Helm 3 compatibility is in the early stages and is not fully tested or supported. Please refer to this [guide](docs/Helm3.md) for more information on Helm 3. We recommend you thoroughly test the use of Helm 3 in your pre-production environments before use. - The following matrix displays the tested package versions for our Helm chart. Sumo Logic Helm Chart | Prometheus Operator | Fluent Bit | Falco | Metrics Server diff --git a/deploy/docs/Additional_Fluentd_Plugins.md b/deploy/docs/Additional_Fluentd_Plugins.md index 3f0b0d6689..70dedcc40d 100644 --- a/deploy/docs/Additional_Fluentd_Plugins.md +++ b/deploy/docs/Additional_Fluentd_Plugins.md @@ -7,9 +7,7 @@ To add additional Fluentd plugins, you can modify `values.yaml` or create a new __Note__: If your plugin requires additional system libraries, it cannot be installed this way. ```yaml -# ... fluentd: - # ... additionalPlugins: - fluent-plugin-route - fluent-plugin-aws-elasticsearch-service @@ -23,7 +21,7 @@ __Note__: You will want to update `` to the [docker tag](https://hub.do FROM sumologic/kubernetes-fluentd: USER root -# Here goes your changes +# Add the plugins you wish to install below. RUN gem install fluent-plugin-route RUN gem install fluent-plugin-aws-elasticsearch-service diff --git a/deploy/docs/Helm3.md b/deploy/docs/Helm3.md deleted file mode 100644 index 286c1c4133..0000000000 --- a/deploy/docs/Helm3.md +++ /dev/null @@ -1,85 +0,0 @@ -# Using this chart with Helm 3 -NOTE: Helm 3 compatibility is in the early stages and is not fully tested or supported. We recommend you thoroughly test the use of Helm 3 in your pre-production environments before use. - -Helm 3 can be used to install the Sumo Logic Helm chart. - -## Migrating from Helm2 to Helm 3 - -If you have installed our Helm chart with Helm 2 and are looking to migrate to Helm 3, we recommend following this [migration guide](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) to transition your Helm 2 release of the Sumo Logic chart. - -## Installing Chart Using Helm 3 - -If you are currently using Helm 3 use the following steps to install the chart. - - 1. Helm 3 no longer creates namespaces. You will need to create a Sumo Logic namespace first. - -``` -kubectl create namespace sumologic -``` - - 2. Change your kubectl context to the sumologic namespace. You can use a tool like [kubens](https://github.com/ahmetb/kubectx) or kubectl to do this. - -```bash -kubectl config set-context --current --namespace=sumologic -``` - - 3. Install the chart using Helm 3. The helm install command has changed slightly in Helm 3. - -``` -helm install collection sumologic/sumologic - --set sumologic.accessId= --set sumologic.accessKey= --set prometheus-operator.prometheus.prometheusSpec.externalLabels.cluster="" --set sumologic.clusterName="" -``` - -NOTE: You may see the following messages however they can usually be ignored. - -```bash -manifest_sorter.go:175: info: skipping unknown hook: "crd-install" -manifest_sorter.go:175: info: skipping unknown hook: "crd-install" -manifest_sorter.go:175: info: skipping unknown hook: "crd-install" -manifest_sorter.go:175: info: skipping unknown hook: "crd-install" -manifest_sorter.go:175: info: skipping unknown hook: "crd-install" -``` - -NOTE: If you need to install the chart with a different release name or namespace you will need to override some configuration fields for both Prometheus and fluent-bit. We recommend using an override file due to the number of fields that need to be overridden. In the following command, replace the `` and `` variables with your values and then run it to download the override file with your replaced values: - -```bash -curl https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/sumologic/values.yaml | \ -sed 's/\-sumologic.sumologic'"/-sumologic./g" | \ -sed 's/\- sumologic'"/- /g" | \ -sed 's/\collection'"//g" > values.yaml -``` - -For example, if your release name is `my-release` and namespace is `my-namespace`: -```bash -curl https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/sumologic/values.yaml | \ -sed 's/\-sumologic.sumologic'"/-sumologic.my-namespace/g" | \ -sed 's/\collection'"/my-release/g" > values.yaml -``` - -Make any changes to the `values.yaml` file as needed and then change your kubectl context to the desired namespace. - -```bash -kubectl config set-context --current --namespace=my-namespace -``` - -Then you can run the following to install the chart with the override file. - -```bash -helm install my-release sumologic/sumologic -f values.yaml --set sumologic.accessId= --set sumologic.accessKey= -``` - -## Uninstalling Chart Using Helm 3 - -In Helm 3 the delete command has changed slightly. By default, history is not preserved yet can be retained by using `--keep-history` - - * Delete without preserving history - -```bash -helm del collection -``` - - * Delete and preserve history - -```bash -helm del --keep-history collection -``` diff --git a/deploy/docs/HelmChartConfiguration.md b/deploy/docs/HelmChartConfiguration.md new file mode 100644 index 0000000000..2aea2352c2 --- /dev/null +++ b/deploy/docs/HelmChartConfiguration.md @@ -0,0 +1,171 @@ +# Configuration +To see all available configuration for our sub-charts, please refer to their documentation. + + * [Falco](https://github.com/helm/charts/tree/master/stable/falco#configuration) - All Falco properties should be prefixed with `falco.` in our values.yaml to override a property not listed below. + * [Prometheus Operator](https://github.com/helm/charts/tree/master/stable/prometheus-operator#configuration) - All Prometheus Operator properties should be prefixed with `prometheus-operator.` in our values.yaml to override a property not listed below. + * [Fluent-bit](https://github.com/helm/charts/tree/master/stable/fluent-bit#configuration) - All Fluent-bit properties should be prefixed with `fluent-bit.` in our values.yaml to override a property not listed below. + * [Metrics Server](https://github.com/helm/charts/tree/master/stable/metrics-server#configuration) - All Metrics Server properties should be prefixed with `metrics-server.` in our values.yaml to override a property not listed below. + +The following table lists the configurable parameters of the Sumo Logic chart and their default values. + +Parameter | Description | Default +--- | --- | --- +`image.repository` | Image repository for Sumo Logic docker container. | `sumologic/kubernetes-fluentd` +`image.tag` | Image tag for Sumo Logic docker container. | `1.0.0-rc.2` +`image.pullPolicy` | Image pullPolicy for Sumo Logic docker container. | `IfNotPresent` +`nameOverride` | Used to override the Chart name. | `Nil` +`sumologic.setupEnabled` | If enabled, a pre-install hook will create Collector and Sources in Sumo Logic. | `true` +`sumologic.logs.enabled` | Set the enabled flag to false for disabling logs ingestion altogether. | `true` +`sumologic.metrics.enabled` | Set the enabled flag to false for disabling metrics ingestion altogether. | `true` +`sumologic.envFromSecret` | If enabled, accessId and accessKey will be sourced from Secret Name given. Be sure to include at least the following env variables in your secret (1) SUMOLOGIC_ACCESSID, (2) SUMOLOGIC_ACCESSKEY | `sumo-api-secret` +`sumologic.accessId` | Sumo access ID. | `Nil` +`sumologic.accessKey` | Sumo access key. | `Nil` +`sumologic.endpoint` | Sumo API endpoint; Leave blank for automatic endpoint discovery and redirection. | `Nil` +`sumologic.collectorName` | The name of the Sumo Logic collector that will be created in the SetUp job. Defaults to `clusterName` if not specified. | `Nil` +`sumologic.clusterName` | An identifier for the Kubernetes cluster. | `kubernetes` +`sumologic.setup.clusterRole.annotations` | Annotations for the ClusterRole. | `[{"helm.sh/hook":"pre-install,pre-upgrade","helm.sh/hook-delete-policy":"before-hook-creation,hook-succeeded","helm.sh/hook-weight":"1"}]` +`sumologic.setup.clusterRoleBinding.annotations` | Annotations for the ClusterRole. | `[{"helm.sh/hook":"pre-install,pre-upgrade","helm.sh/hook-delete-policy":"before-hook-creation,hook-succeeded","helm.sh/hook-weight":"2"}]` +`sumologic.setup.configMap"` | Annotations for the ConfigMap. | `[{"helm.sh/hook":"pre-install,pre-upgrade","helm.sh/hook-delete-policy":"before-hook-creation,hook-succeeded","helm.sh/hook-weight":"2"}]` +`sumologic.setup.job.annotations` | Annotations for the Job. | `[{"helm.sh/hook":"pre-install,pre-upgrade","helm.sh/hook-delete-policy":"before-hook-creation,hook-succeeded","helm.sh/hook-weight":"3"}]` +`sumologic.setup.serviceAccount.annotations` | Annotations for the ServiceAccount. | `[{"helm.sh/hook":"pre-install,pre-upgrade","helm.sh/hook-delete-policy":"before-hook-creation,hook-succeeded","helm.sh/hook-weight":"0"}]` +`fluentd.additionalPlugins` | Additional Fluentd plugins to install from RubyGems. Please see our [documentation](./Additional_Fluentd_Plugins.md) for more information. | `[]` +`fluentd.logLevel` | Sets the fluentd log level. The default log level, if not specified, is info. Sumo will only ingest the error log level and some specific warnings, the info logs can be seen in kubectl logs. | `info` +`fluentd.verifySsl` | Verify SumoLogic HTTPS certificates. | `true` +`fluentd.proxyUri` | Proxy URI for sumologic output plugin. | `Nil` +`fluentd.securityContext` | the securityContext configuration for Fluentd | `{"fsGroup":999}` +`fluentd.persistence.enabled` | Persist data to a persistent volume; When enabled, fluentd uses the file buffer instead of memory buffer. After setting the value to true, run the helm upgrade command with the --force flag. | `false` +`fluentd.persistence.storageClass` | If defined, storageClassName: . If set to "-", storageClassName: "", which disables dynamic provisioning. If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. (gp2 on AWS, standard on GKE, Azure & OpenStack) | `Nil` +`fluentd.persistence.annotations` | Annotations for the persistence. | `Nil` +`fluentd.persistence.accessMode` | The accessMode for persistence. | `ReadWriteOnce` +`fluentd.persistence.size` | The size needed for persistence. | `10Gi` +`fluentd.buffer.type` | Option to specify the Fluentd buffer as file/memory. If `fluentd.persistence.enabled` is `true`, this will be ignored. | `memory` +`fluentd.buffer.flushInterval` | How frequently to push logs to Sumo Logic. | `5s` +`fluentd.buffer.numThreads` | Increase number of http threads to Sumo. May be required in heavy logging/high DPM clusters. | `8` +`fluentd.buffer.chunkLimitSize` | The max size of each chunks: events will be written into chunks until the size of chunks become this size. | `1m` +`fluentd.buffer.queueChunkLimitSize` | Limit the number of queued chunks. | `128` +`fluentd.buffer.totalLimitSize` | The size limitation of this buffer plugin instance. | `128m` +`fluentd.buffer.filePaths` | File paths to buffer to, if Fluentd buffer type is specified as file above. Each sumologic output plugin buffers to its own unique file. | `{"events":"/fluentd/buffer/events","logs":{"containers":"/fluentd/buffer/logs.containers","default":"/fluentd/buffer/logs.default","kubelet":"/fluentd/buffer/logs.kubelet","systemd":"/fluentd/buffer/logs.systemd"},"metrics":{"apiserver":"/fluentd/buffer/metrics.apiserver","container":"/fluentd/buffer/metrics.container","controller":"/fluentd/buffer/metrics.controller","default":"/fluentd/buffer/metrics.default","kubelet":"/fluentd/buffer/metrics.kubelet","node":"/fluentd/buffer/metrics.node","scheduler":"/fluentd/buffer/metrics.scheduler","state":"/fluentd/buffer/metrics.state"},"traces":"/fluentd/buffer/traces"}` +`fluentd.buffer.extraConf` | Additional config for buffer settings | `Nil` +`fluentd.metadata.cacheSize` | Option to control the enabling of metadata filter plugin cache_size. | `10000` +`fluentd.metadata.cacheTtl` | Option to control the enabling of metadata filter plugin cache_ttl (in seconds). | `3600` +`fluentd.metadata.cacheRefresh` | Option to control the interval at which metadata cache is asynchronously refreshed (in seconds). | `1800` +`fluentd.metadata.pluginLogLevel` | Option to give plugin specific log level. | `error` +`fluentd.logs.enabled` | Flag to control deploying the Fluentd logs statefulsets. | `true` +`fluentd.logs.statefulset.nodeSelector` | Node selector for Fluentd log statefulset. | `{}` +`fluentd.logs.statefulset.tolerations` | Tolerations for Fluentd log statefulset. | `{}` +`fluentd.logs.statefulset.affinity` | Affinity for Fluentd log statefulset. | `{}` +`fluentd.logs.statefulset.podAntiAffinity` | PodAntiAffinity for Fluentd log statefulset. | `soft` +`fluentd.logs.statefulset.replicaCount` | Replica count for Fluentd log statefulset. | `3` +`fluentd.logs.statefulset.resources` | Resources for Fluentd log statefulset. | `{"limits":{"cpu":1,"memory":"1Gi"},"requests":{"cpu":0.5,"memory":"768Mi"}}` +`fluentd.logs.autoscaling.enabled` | Option to turn autoscaling on for fluentd and specify params for HPA. Autoscaling needs metrics-server to access cpu metrics. | `false` +`fluentd.logs.autoscaling.minReplicas` | Default min replicas for autoscaling. | `3` +`fluentd.logs.autoscaling.maxReplicas` | Default max replicas for autoscaling. | `10` +`fluentd.logs.autoscaling.targetCPUUtilizationPercentage` | The desired target CPU utilization for autoscaling. | `50` +`fluentd.logs.rawConfig` | Default log configuration. | `@include common.conf @include logs.conf` +`fluentd.logs.output.logFormat` | Format to post logs into Sumo: fields, json, json_merge, or text. | `fields` +`fluentd.logs.output.addTimestamp` | Option to control adding timestamp to logs. | `true` +`fluentd.logs.output.timestampKey` | Field name when add_timestamp is on. | `timestamp` +`fluentd.logs.output.pluginLogLevel` | Option to give plugin specific log level. | `error` +`fluentd.logs.output.extraConf` | Additional config parameters for sumologic output plugin | `Nil` +`fluentd.logs.extraLogs` | Additional config for custom log pipelines. | `Nil` +`fluentd.logs.containers.overrideRawConfig` | To override the entire contents of logs.source.containers.conf file. Leave empty for the default pipeline. | `Nil` +`fluentd.logs.containers.outputConf` | Default output configuration for container logs. | `@include logs.output.conf` +`fluentd.logs.containers.overrideOutputConf` | Override output section for container logs. Leave empty for the default output section. | `Nil` +`fluentd.logs.containers.sourceName` | Set the _sourceName metadata field in Sumo Logic. | `%{namespace}.%{pod}.%{container}` +`fluentd.logs.containers.sourceCategory` | Set the _sourceCategory metadata field in Sumo Logic. | `%{namespace}/%{pod_name}` +`fluentd.logs.containers.sourceCategoryPrefix` | Set the prefix, for _sourceCategory metadata. | `kubernetes/` +`fluentd.logs.containers.sourceCategoryReplaceDash` | Used to replace - with another character. | `/` +`fluentd.logs.containers.excludeContainerRegex` | A regular expression for containers. Matching containers will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.containers.excludeHostRegex` | A regular expression for hosts. Matching hosts will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.containers.excludeNamespaceRegex` | A regular expression for namespaces. Matching namespaces will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.containers.excludePodRegex` | A regular expression for pods. Matching pods will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.containers.k8sMetadataFilter.watch` | Option to control the enabling of metadata filter plugin watch. | `true` +`fluentd.logs.containers.k8sMetadataFilter.caFile` | path to CA file for Kubernetes server certificate validation. | `Nil` +`fluentd.logs.containers.k8sMetadataFilter.verifySsl` | Validate SSL certificates. | `true` +`fluentd.logs.containers.k8sMetadataFilter.clientCert` | Path to a client cert file to authenticate to the API server. | `Nil` +`fluentd.logs.containers.k8sMetadataFilter.clientKey` | Path to a client key file to authenticate to the API server. | `Nil` +`fluentd.logs.containers.k8sMetadataFilter.bearerTokenFile` | Path to a file containing the bearer token to use for authentication. | `Nil` +`fluentd.logs.containers.extraFilterPluginConf` | To use additional filter plugins. | `Nil` +`fluentd.logs.kubelet.enabled` | Collect kubelet logs. | `true` +`fluentd.logs.kubelet.outputConf` | Output configuration for kubelet. | `@include logs.output.conf` +`fluentd.logs.kubelet.overrideOutputConf` | Override output section for kubelet logs. Leave empty for the default output section. | `Nil` +`fluentd.logs.kubelet.sourceName` | Set the _sourceName metadata field in Sumo Logic. | `k8s_kubelet` +`fluentd.logs.kubelet.sourceCategory` | Set the _sourceCategory metadata field in Sumo Logic. | `kubelet` +`fluentd.logs.kubelet.sourceCategoryPrefix` | Set the prefix, for _sourceCategory metadata. | `kubernetes/` +`fluentd.logs.kubelet.sourceCategoryReplaceDash` | Used to replace - with another character. | `/` +`fluentd.logs.kubelet.excludeFacilityRegex` | A regular expression for facility. Matching facility will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.kubelet.excludeHostRegex` | A regular expression for hosts. Matching hosts will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.kubelet.excludePriorityRegex` | A regular expression for priority. Matching priority will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.kubelet.excludeUnitRegex` | A regular expression for unit. Matching unit will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.systemd.enabled` | Collect systemd logs. | `true` +`fluentd.logs.systemd.outputConf` | Output configuration for systemd. | `@include logs.output.conf` +`fluentd.logs.systemd.overrideOutputConf` | Override output section for systemd logs. Leave empty for the default output section. | `Nil` +`fluentd.logs.systemd.sourceCategory` | Set the _sourceCategory metadata field in Sumo Logic. | `system` +`fluentd.logs.systemd.sourceCategoryPrefix` | Set the prefix, for _sourceCategory metadata. | `kubernetes/` +`fluentd.logs.systemd.sourceCategoryReplaceDash` | Used to replace - with another character. | `/` +`fluentd.logs.systemd.excludeFacilityRegex` | A regular expression for facility. Matching facility will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.systemd.excludeHostRegex` | A regular expression for hosts. Matching hosts will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.systemd.excludePriorityRegex` | A regular expression for priority. Matching priority will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.systemd.excludeUnitRegex` | A regular expression for unit. Matching unit will be excluded from Sumo. The logs will still be sent to FluentD. | `Nil` +`fluentd.logs.default.outputConf` | Default log configuration (catch-all). | `@include logs.output.conf` +`fluentd.logs.default.overrideOutputConf` | Override output section for untagged logs. Leave empty for the default output section. | `Nil` +`fluentd.metrics.enabled` | Flag to control deploying the Fluentd metrics statefulsets. | `true` +`fluentd.metrics.statefulset.nodeSelector` | Node selector for Fluentd metrics statefulset. | `{}` +`fluentd.metrics.statefulset.tolerations` | Tolerations for Fluentd metrics statefulset. | `{}` +`fluentd.metrics.statefulset.affinity` | Affinity for Fluentd metrics statefulset. | `{}` +`fluentd.metrics.statefulset.podAntiAffinity` | PodAntiAffinity for Fluentd metrics statefulset. | `soft` +`fluentd.metrics.statefulset.replicaCount` | Replica count for Fluentd metrics statefulset. | `3` +`fluentd.metrics.statefulset.resources` | Resources for Fluentd metrics statefulset. | `{"limits":{"cpu":1,"memory":"1Gi"},"requests":{"cpu":0.5,"memory":"768Mi"}}` +`fluentd.metrics.autoscaling.enabled` | Option to turn autoscaling on for fluentd and specify params for HPA. Autoscaling needs metrics-server to access cpu metrics. | `false` +`fluentd.metrics.autoscaling.minReplicas` | Default min replicas for autoscaling. | `3` +`fluentd.metrics.autoscaling.maxReplicas` | Default max replicas for autoscaling. | `10` +`fluentd.metrics.autoscaling.targetCPUUtilizationPercentage` | The desired target CPU utilization for autoscaling. | `50` +`fluentd.metrics.rawConfig` | Raw config for fluentd metrics. | `@include common.conf @include metrics.conf` +`fluentd.metrics.outputConf` | Configuration for sumologic output plugin. | `@include metrics.output.conf` +`fluentd.metrics.extraFilterPluginConf` | To use additional filter plugins. | `Nil` +`fluentd.metrics.extraOutputPluginConf` | To use additional output plugins. | `Nil` +`fluentd.events.enabled` | If enabled, collect K8s events. | `true` +`fluentd.events.statefulset.nodeSelector` | Node selector for Fluentd events statefulset. | `{}` +`fluentd.events.statefulset.tolerations` | Tolerations for Fluentd events statefulset. | `{}` +`fluentd.events.statefulset.resources` | Resources for Fluentd log statefulset. | `{"limits":{"cpu":"100m","memory":"256Mi"},"requests":{"cpu":"100m","memory":"256Mi"}}` +`fluentd.events.sourceCategory` | Source category for the Events source. Default: "{clusterName}/events" | `Nil` +`metrics-server.enabled` | Set the enabled flag to true for enabling metrics-server. This is required before enabling fluentd autoscaling unless you have an existing metrics-server in the cluster. | `false` +`metrics-server.args` | Arguments for metric server. | `["--kubelet-insecure-tls","--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname"]` +`fluent-bit.resources` | Resources for Fluent-bit daemonsets. | `{}` +`fluent-bit.enabled` | Flag to control deploying Fluent-bit Helm sub-chart. | `true` +`fluent-bit.service.flush` | Frequency to flush fluent-bit buffer to fluentd. | `5` +`fluent-bit.metrics.enabled` | Enable metrics from fluent-bit. | `true` +`fluent-bit.env` | Environment variables for fluent-bit. | `[{"name":"CHART","valueFrom":{"configMapKeyRef":{"key":"fluentdLogs","name":"sumologic-configmap"}}},{"name":"NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}}]` +`fluent-bit.backend.type` | Set the backend to which Fluent-Bit should flush the information it gathers | `forward` +`fluent-bit.backend.forward.host` | Target host where Fluent-Bit or Fluentd are listening for Forward messages. | `${CHART}.${NAMESPACE}.svc.cluster.local.` +`fluent-bit.backend.forward.port` | TCP Port of the target service. | `24321` +`fluent-bit.backend.forward.tls` | Enable or disable TLS support. | `off` +`fluent-bit.backend.forward.tls_verify` | Force certificate validation. | `on` +`fluent-bit.backend.forward.tls_debug` | Set TLS debug verbosity level. It accept the following values: 0-4. | `1` +`fluent-bit.backend.forward.shared_key` | A key string known by the remote Fluentd used for authorization. | `Nil` +`fluent-bit.trackOffsets` | Specify whether to track the file offsets for tailing docker logs. This allows fluent-bit to pick up where it left after pod restarts but requires access to a hostPath. | `true` +`fluent-bit.tolerations` | Optional daemonset tolerations. | `[{"effect":"NoSchedule","operator":"Exists"}]` +`fluent-bit.input.systemd.enabled` | Enable systemd input. | `true` +`fluent-bit.parsers.enabled` | Enable custom parsers. | `true` +`fluent-bit.parsers.regex` | List of regex parsers. | `[{"name":"multi_line","regex":"(?\u003clog\u003e^{\"log\":\"\\d{4}-\\d{1,2}-\\d{1,2}.\\d{2}:\\d{2}:\\d{2}.*)"}]` +`fluent-bit.rawConfig` | DESCRIPTION | `@INCLUDE fluent-bit-service.conf [INPUT] Name tail Path /var/log/containers/*.log Multiline On Parser_Firstline multi_line Tag containers.* Refresh_Interval 1 Rotate_Wait 60 Mem_Buf_Limit 5MB Skip_Long_Lines On DB /tail-db/tail-containers-state-sumo.db DB.Sync Normal [INPUT] Name systemd Tag host.* Systemd_Filter _SYSTEMD_UNIT=addon-config.service Systemd_Filter _SYSTEMD_UNIT=addon-run.service Systemd_Filter _SYSTEMD_UNIT=cfn-etcd-environment.service Systemd_Filter _SYSTEMD_UNIT=cfn-signal.service Systemd_Filter _SYSTEMD_UNIT=clean-ca-certificates.service Systemd_Filter _SYSTEMD_UNIT=containerd.service Systemd_Filter _SYSTEMD_UNIT=coreos-metadata.service Systemd_Filter _SYSTEMD_UNIT=coreos-setup-environment.service Systemd_Filter _SYSTEMD_UNIT=coreos-tmpfiles.service Systemd_Filter _SYSTEMD_UNIT=dbus.service Systemd_Filter _SYSTEMD_UNIT=docker.service Systemd_Filter _SYSTEMD_UNIT=efs.service Systemd_Filter _SYSTEMD_UNIT=etcd-member.service Systemd_Filter _SYSTEMD_UNIT=etcd.service Systemd_Filter _SYSTEMD_UNIT=etcd2.service Systemd_Filter _SYSTEMD_UNIT=etcd3.service Systemd_Filter _SYSTEMD_UNIT=etcdadm-check.service Systemd_Filter _SYSTEMD_UNIT=etcdadm-reconfigure.service Systemd_Filter _SYSTEMD_UNIT=etcdadm-save.service Systemd_Filter _SYSTEMD_UNIT=etcdadm-update-status.service Systemd_Filter _SYSTEMD_UNIT=flanneld.service Systemd_Filter _SYSTEMD_UNIT=format-etcd2-volume.service Systemd_Filter _SYSTEMD_UNIT=kube-node-taint-and-uncordon.service Systemd_Filter _SYSTEMD_UNIT=kubelet.service Systemd_Filter _SYSTEMD_UNIT=ldconfig.service Systemd_Filter _SYSTEMD_UNIT=locksmithd.service Systemd_Filter _SYSTEMD_UNIT=logrotate.service Systemd_Filter _SYSTEMD_UNIT=lvm2-monitor.service Systemd_Filter _SYSTEMD_UNIT=mdmon.service Systemd_Filter _SYSTEMD_UNIT=nfs-idmapd.service Systemd_Filter _SYSTEMD_UNIT=nfs-mountd.service Systemd_Filter _SYSTEMD_UNIT=nfs-server.service Systemd_Filter _SYSTEMD_UNIT=nfs-utils.service Systemd_Filter _SYSTEMD_UNIT=node-problem-detector.service Systemd_Filter _SYSTEMD_UNIT=ntp.service Systemd_Filter _SYSTEMD_UNIT=oem-cloudinit.service Systemd_Filter _SYSTEMD_UNIT=rkt-gc.service Systemd_Filter _SYSTEMD_UNIT=rkt-metadata.service Systemd_Filter _SYSTEMD_UNIT=rpc-idmapd.service Systemd_Filter _SYSTEMD_UNIT=rpc-mountd.service Systemd_Filter _SYSTEMD_UNIT=rpc-statd.service Systemd_Filter _SYSTEMD_UNIT=rpcbind.service Systemd_Filter _SYSTEMD_UNIT=set-aws-environment.service Systemd_Filter _SYSTEMD_UNIT=system-cloudinit.service Systemd_Filter _SYSTEMD_UNIT=systemd-timesyncd.service Systemd_Filter _SYSTEMD_UNIT=update-ca-certificates.service Systemd_Filter _SYSTEMD_UNIT=user-cloudinit.service Systemd_Filter _SYSTEMD_UNIT=var-lib-etcd2.service Max_Entries 1000 Read_From_Tail true @INCLUDE fluent-bit-output.conf` +`prometheus-operator.kubeTargetVersionOverride` | Provide a target gitVersion of K8S, in case .Capabilites.KubeVersion is not available (e.g. helm template). Changing this may break Sumo Logic apps. | `1.13.0-0` +`prometheus-operator.enabled` | Flag to control deploying Prometheus Operator Helm sub-chart. | `true` +`prometheus-operator.alertmanager.enabled` | Deploy alertmanager. | `false` +`prometheus-operator.grafana.enabled` | If true, deploy the grafana sub-chart. | `false` +`prometheus-operator.grafana.defaultDashboardsEnabled` | Deploy default dashboards. These are loaded using the sidecar. | `false` +`prometheus-operator.prometheusOperator.resources` | Resource limits for prometheus operator. Uses sub-chart defaults. | `{}`} +`prometheus-operator.prometheusOperator.admissionWebhooks.enabled` | Create PrometheusRules admission webhooks. Mutating webhook will patch PrometheusRules objects indicating they were validated. Validating webhook will check the rules syntax. | `false` +`prometheus-operator.prometheusOperator.tlsProxy.enabled` | Enable a TLS proxy container. Only the squareup/ghostunnel command line arguments are currently supported and the secret where the cert is loaded from is expected to be provided by the admission webhook. | `false` +`prometheus-operator.prometheusOperator.kube-state-metrics.resources` | Resource limits for kube state metrics. Uses sub-chart defaults. | `{}}` +`prometheus-operator.prometheusOperator.prometheus-node-exporter.resources` | Resource limits for node exporter. Uses sub-chart defaults. | `{}` +`prometheus-operator.prometheus.additionalServiceMonitors` | List of ServiceMonitor objects to create. | `[{"additionalLabels":{"app":"collection-sumologic-fluentd-logs"},"endpoints":[{"port":"metrics"}],"name":"collection-sumologic-fluentd-logs","namespaceSelector":{"matchNames":["sumologic"]},"selector":{"matchLabels":{"app":"collection-sumologic-fluentd-logs"}}},{"additionalLabels":{"app":"collection-sumologic-fluentd-metrics"},"endpoints":[{"port":"metrics"}],"name":"collection-sumologic-fluentd-metrics","namespaceSelector":{"matchNames":["sumologic"]},"selector":{"matchLabels":{"app":"collection-sumologic-fluentd-metrics"}}},{"additionalLabels":{"app":"collection-sumologic-fluentd-events"},"endpoints":[{"port":"metrics"}],"name":"collection-sumologic-fluentd-events","namespaceSelector":{"matchNames":["sumologic"]},"selector":{"matchLabels":{"app":"collection-sumologic-fluentd-events"}}},{"additionalLabels":{"app":"collection-fluent-bit"},"endpoints":[{"path":"/api/v1/metrics/prometheus","port":"metrics"}],"name":"collection-fluent-bit","namespaceSelector":{"matchNames":["sumologic"]},"selector":{"matchLabels":{"app":"fluent-bit"}}},{"additionalLabels":{"app":"collection-sumologic-otelcol"},"endpoints":[{"port":"metrics"}],"name":"collection-sumologic-otelcol","namespaceSelector":{"matchNames":["sumologic"]},"selector":{"matchLabels":{"app":"collection-sumologic-otelcol"}}}]` +`prometheus-operator.prometheus.prometheusSpec.resources` | Resource limits for prometheus. Uses sub-chart defaults. | `{}}` +`prometheus-operator.prometheus.prometheusSpec.thanos.baseImage` | Base image for Thanos container. | `quay.io/thanos/thanos` +`prometheus-operator.prometheus.prometheusSpec.thanos.version` | Image tag for Thanos container. | `v0.10.0` +`prometheus-operator.prometheus.prometheusSpec.containers` | Containers allows injecting additional containers. This is meant to allow adding an authentication proxy to a Prometheus pod. | `[{"env":[{"name":"CHART","valueFrom":{"configMapKeyRef":{"key":"fluentdMetrics","name":"sumologic-configmap"}}},{"name":"NAMESPACE","valueFrom":{"configMapKeyRef":{"key":"fluentdNamespace","name":"sumologic-configmap"}}}],"name":"prometheus-config-reloader"}]` +`prometheus-operator.prometheus.prometheusSpec.remoteWrite` | If specified, the remote_write spec. | See values.yaml +`prometheus-operator.prometheus.prometheusSpec.walCompression` | Enables walCompression in Prometheus | `true` +`falco.enabled` | Flag to control deploying Falco Helm sub-chart. | `false` +`falco.ebpf.enabled` | Enable eBPF support for Falco instead of falco-probe kernel module. Set to false for GKE. | `true` +`falco.falco.jsonOutput` | Output events in json. | `true` \ No newline at end of file diff --git a/deploy/docs/Installation_with_Helm.md b/deploy/docs/Installation_with_Helm.md index 5063195440..30fb5c63cf 100644 --- a/deploy/docs/Installation_with_Helm.md +++ b/deploy/docs/Installation_with_Helm.md @@ -1,20 +1,32 @@ # Installation with Helm -Note: the following steps assume you are installing using Helm 2. Use of Helm 3 is not yet supported. - Our Helm chart deploys Kubernetes resources for collecting Kubernetes logs, metrics, and events; enriching them with deployment, pod, and service level metadata; and sends them to Sumo Logic. - +- [Requirements](#requirements) - [Prerequisite](#prerequisite) - [Installation Steps](#installation-steps) -- [Uninstalling the Chart](#uninstalling-the-chart) +- [Viewing Data In Sumo Logic](#viewing-data-in-sumo-logic) +- [Troubleshooting Installation](#troubleshooting-installation) +- [Customizing Installation](#customizing-installation) +- [Upgrade Sumo Logic Collection](#upgrading-sumo-logic-collection) +- [Uninstalling Sumo Logic Collection](#uninstalling-sumo-logic-collection) +## Requirements + +If you don’t already have a Sumo account, you can create one by clicking the Free Trial button on https://www.sumologic.com/. + +The following are required to set up Sumo Logic's Kubernetes collection. + + * An [Access ID and Access Key](https://help.sumologic.com/Manage/Security/Access-Keys) with [Manage Collectors](https://help.sumologic.com/Manage/Users-and-Roles/Manage-Roles/05-Role-Capabilities#data-management) capability. + * Please review our [minimum requirements](../README.md#minimum-requirements) and [support matrix](../README.md#support-matrix) + + ## Prerequisite -Sumo Logic Apps for Kubernetes and Explore require you to add the following [fields](https://help.sumologic.com/Manage/Fields) in the Sumo Logic UI to your Fields table schema. This is to ensure your logs are tagged with relevant metadata. +Sumo Logic Apps for Kubernetes and Explore require you to add the following [fields](https://help.sumologic.com/Manage/Fields) in the Sumo Logic UI to your Fields table schema. This is to ensure your logs are tagged with relevant metadata. This is a one time setup per Sumo Logic account. - cluster - container - deployment @@ -24,17 +36,24 @@ Sumo Logic Apps for Kubernetes and Explore require you to add the following [fie - pod - service +## Installation Steps + +These steps require that no Prometheus exists. If you already have Prometheus installed select from the following options: + +- [How to install our Chart side by side with your existing Prometheus Operator](./SideBySidePrometheus.md) +- [How to install if you have an existing Prometheus Operator you want to update](./existingPrometheusDoc.md) +- [How to install if you have standalone Prometheus (not using Prometheus Operator)](./standAlonePrometheus.md) + The Helm chart installation requires two parameter overrides: * __sumologic.accessId__ - Sumo [Access ID](https://help.sumologic.com/Manage/Security/Access-Keys). * __sumologic.accessKey__ - Sumo [Access key](https://help.sumologic.com/Manage/Security/Access-Keys). -## Installation Steps +If you are installing the collection in a cluster that requires proxying outbound requests, please see the following [additional properties](./Installation_Behind_Proxy.md) you will need to set. -These steps require that no Prometheus exists. If you already have Prometheus installed select from the following options: +The following parameter is optional, but we recommend setting it. +* __sumologic.clusterName__ - An identifier for your Kubernetes cluster. This is the name you will see for the cluster in Sumo Logic. Default is `kubernetes`. -- [How to install if you have an existing Prometheus operator](./existingPrometheusDoc.md) -- [How to install if you have standalone Prometheus](./standAlonePrometheus.md) -- [How to install our Prometheus side by side with your existing Prometheus](./SideBySidePrometheus.md) +The following helm commands support Helm2 or Helm3. To install the chart, first add the `sumologic` private repo: @@ -42,62 +61,109 @@ To install the chart, first add the `sumologic` private repo: helm repo add sumologic https://sumologic.github.io/sumologic-kubernetes-collection ``` -Install the chart with release name `collection` and namespace `sumologic` +Next you can run `helm upgrade --install` to install our chart. An example command with the minimum parameters is provided below. The following command will install the Sumo Logic chart with the release name `my-release` in the namespace your `kubectl` context is currently set to. ```bash -helm install sumologic/sumologic --name collection --namespace sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" +helm upgrade --install my-release sumologic/sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" ``` +> **Note**: If the release exists, it will be upgraded, otherwise it will be installed. -If you get `Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" already exists`, run the above command with the `--no-crd-hook` flag: +If you wish to install the chart in a different existing namespace you can do the following: ```bash -helm install sumologic/sumologic --name collection --namespace sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --no-crd-hook +helm upgrade --install my-release sumologic/sumologic --namespace=my-namespace --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" ``` -If you get `Error: collector with name 'sumologic' does not exist -sumologic_http_source.default_metrics_source: Importing from ID`, you can safely ignore it and the installation should complete successfully. The installation process creates new [HTTP endpoints](https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source) in your Sumo Logic account, that are used to send data to Sumo. This error occurs if the endpoints had already been created by an earlier run of the installation process. +For Helm3, if the namespace does not exist, you can add the `--create-namespace` flag. + +```bash +helm upgrade --install my-release sumologic/sumologic --namespace=my-namespace --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName=" --create-namespace" +``` + +## Viewing Data In Sumo Logic + +Once you have completed installation, you can [install the Kubernetes App and view the dashboards](https://help.sumologic.com/07Sumo-Logic-Apps/10Containers_and_Orchestration/Kubernetes/Install_the_Kubernetes_App_and_view_the_Dashboards) or [open a new Explore tab](https://help.sumologic.com/Solutions/Kubernetes_Solution/05Navigate_your_Kubernetes_environment) in Sumo Logic. If you do not see data in Sumo Logic, you can review our [troubleshooting guide](./Troubleshoot_Collection.md). -__NOTE__ `Google Kubernetes Engine (GKE)` uses Container-Optimized OS (COS) as the default operating system for its worker node pools. COS is a security-enhanced operating system that limits access to certain parts of the underlying OS. Because of this security constraint, Falco cannot insert its kernel module to process events for system calls. However, COS provides the ability to use extended Berkeley Packet Filter (eBPF) to supply the stream of system calls to the Falco engine. eBPF is currently only supported on GKE and COS. For more information see [Installing Falco](https://falco.org/docs/installation/). +## Troubleshooting Installation -To install on `GKE`, use the provided override file to customize your configuration and uncomment the following lines in the `values.yaml` file referenced below: +### Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" already exists +If you get `Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" already exists` on a Helm2 installation, run the above command with the `--set prometheus-operator.prometheusOperator.createCustomResource=false` parameter: +```bash +helm upgrade --install my-release sumologic/sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.prometheusOperator.createCustomResource=false ``` -ebpf: - enabled: true +### Error: timed out waiting for the condition +If `helm upgrade --install` hangs, it usually means the pre-install setup job is failing and is in a retry loop. Due to a Helm limitation, errors from the setup job cannot be fed back to the `helm upgrade --install` command. Kubernetes schedules the job in a pod, so you can look at logs from the pod to see why the job is failing. First find the pod name in the namespace where the Helm chart was deployed. The pod name will contain `-setup` in the name. + +```sh +kubectl get pods ``` +> **Tip**: If the pod does not exist, it is possible it has been evicted. Re-run the `helm upgrade --install` to recreate it and while that command is running, use another shell to get the name of the pod. -To customize your configuration: +Get the logs from that pod: +``` +kubectl logs POD_NAME -f +``` +### Error: collector with name 'sumologic' does not exist +If you get `Error: collector with name 'sumologic' does not exist +sumologic_http_source.default_metrics_source: Importing from ID`, you can safely ignore it and the installation should complete successfully. The installation process creates new [HTTP endpoints](https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source) in your Sumo Logic account, that are used to send data to Sumo. This error occurs if the endpoints had already been created by an earlier run of the installation process. -- Download the `values.yaml` file by running: +You can find more information in our [troubleshooting documentation](Troubleshoot_Collection.md). +## Customizing Installation +All default properties for the Helm chart can be found in our [documentation](HelmChartConfiguration.md). We recommend creating a new `values.yaml` for each Kubernetes cluster you wish to install collection on and **setting only the properties you wish to override**. Once you have customized you can use the following commands to install or upgrade. + ```bash -curl https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/sumologic/values.yaml > values.yaml +helm upgrade --install my-release sumologic/sumologic -f values.yaml ``` +> **Tip**: To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) + +## Upgrading Sumo Logic Collection + +**Note, if you are upgrading to version 1.x of our collection from a version before 1.x, please see our [migration guide](v1_migration_doc.md).** -- Modify the `values.yaml` file with your customizations, then apply the configuration using the following command: +To upgrade our helm chart to a newer version, you must first run update your local helm repo. ```bash -helm install sumologic/sumologic --name collection --namespace sumologic -f values.yaml --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName= -f values.yaml +helm repo update ``` -#### To install the chart with a different release name or namespace: +Next, you can run `helm upgrade --install` to upgrade to that version. The following upgrades the current version of `my-release` to the latest. ```bash -helm install sumologic/sumologic --name my-release --namespace my-namespace -f values.yaml --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName= +helm upgrade --install my-release sumologic/sumologic -f values.yaml ``` -__NOTE__ To configure a chart with custom cluster settings, [please refer to this document](Terraform.md#kubernetes-terraform-provider) +If you wish to upgrade to a specific version, you can use the `--version` flag. -__NOTE__ To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml --version=1.0.0 +``` -## Uninstalling the Chart +If you no longer have your `values.yaml` from the first installation or do not remember the options you added via `--set` you can run the following to see the values for the currently installed helm chart. For example, if the release is called `my-release` you can run the following. -To uninstall/delete the `collection` release: +```bash +helm get values my-release +``` + +If something goes wrong, or you want to go back to the previous version, +you can [rollback changes using helm](https://helm.sh/docs/helm/helm_rollback/): + +``` +helm history my-release +helm rollback my-release +``` + +## Uninstalling Sumo Logic Collection + +To uninstall/delete the Helm chart: ```bash -helm delete collection +helm delete my-release ``` -> **Tip**: Use helm delete --purge collection to completely remove the release from Helm internal storage +> **Helm2 Tip**: Use helm delete --purge my-release to completely remove the release from Helm internal storage + +> **Helm3 Tip**: In Helm3 the default behavior is to purge history. Use --keep-history to preserve it while deleting the release. The command removes all the Kubernetes components associated with the chart and deletes the release. diff --git a/deploy/docs/Non_Helm_Installation.md b/deploy/docs/Non_Helm_Installation.md index fc886533fc..dc9616aceb 100644 --- a/deploy/docs/Non_Helm_Installation.md +++ b/deploy/docs/Non_Helm_Installation.md @@ -1,56 +1,35 @@ # Non Helm Installation -This document has instructions for setting up collection with Fluentd, FluentBit, and Prometheus. +**Please note that our non-helm installation process still uses Helm to generate the YAML that you will deploy into your Kubernetes cluster. We do not provide YAML that can be directly be applied and it must be generated.** + +This document has instructions for setting up Sumo Logic collection using Fluentd, Fluent-Bit, Prometheus and Falco. -- [Before you start](#before-you-start) -- [Create Sumo Fields and a collector](#create-sumo-fields-a-collector) - - [Automatic Source Creation and Setup YAML](#automatic-source-creation-and-setup-yaml) - - [Manual Source Creation and Setup](#manual-source-creation-and-setup) - - [Create a Hosted Collector and an HTTP Source](#create-a-hosted-collector-and-an-http-source) - - [Create the namespace and secret](#create-the-namespace-and-secret) -- [Deploy Fluentd](#deploy-fluentd) - - [Use default configuration](#use-default-configuration) - - [Customize configuration](#customize-configuration) - - [Verify the pods are running](#verify-the-pods-are-running) -- [Deploy Prometheus](#deploy-prometheus) - - [Missing metrics for `controller-manager` or `scheduler`](#missing-metrics-for-controller-manager-or-scheduler) -- [Deploy FluentBit](#deploy-fluentbit) -- [Deploy Falco](#deploy-falco) -- [Tear down](#tear-down) +- [Requirements](#requirements) +- [Prerequisite](#prerequisite) +- [Installation Steps](#installation-steps) +- [Viewing Data In Sumo Logic](#viewing-data-in-sumo-logic) +- [Troubleshooting Installation](#troubleshooting-installation) +- [Customizing Installation](#customizing-installation) +- [Upgrade Sumo Logic Collection](#upgrading-sumo-logic-collection) +- [Uninstalling Sumo Logic Collection](#uninstalling-sumo-logic-collection) -## Before you start - -* If you haven’t already done so, create your Kubernetes cluster. Verify that you can access the cluster with `kubectl`. -* Verify that the cluster DNS service is enabled. For more information, see [DNS](https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/#dns) in Kubernetes documentation. -* Verify that you are in the correct context. You can check your current context and set your desired context with: -``` -kubectl config current-context -kubectl config use-context DESIRED_CONTEXT_NAME -``` -* In the Non-Helm installation steps, you will never need to run `helm install`, but we use Helm as a templating tool to generate the yaml files to install various components of our solution. Thus you will still need to install Helm: - -*Note the following steps are one way to install Helm, but in order to ensure property security, please be sure to review the [Helm documentation.](https://v2.helm.sh/docs/securing_installation/#securing-your-helm-installation)* - -Download the latest Helm 2 version to generate the yaml files necessary to deploy by running +## Requirements -```bash -brew install helm@2 -export PATH="/usr/local/opt/helm@2/bin:$PATH" -``` - -Reference: https://v2.helm.sh/docs/using_helm/#installing-helm +If you don’t already have a Sumo account, you can create one by clicking the Free Trial button on https://www.sumologic.com/. -__NOTE__ These instructions assume that Prometheus is not already running on your Kubernetes cluster. +The following are required to setup Sumo Logic's Kubernetes collection. -## Create Sumo Fields and a Collector + * An [Access ID and Access Key](https://help.sumologic.com/Manage/Security/Access-Keys) with [Manage Collectors](https://help.sumologic.com/Manage/Users-and-Roles/Manage-Roles/05-Role-Capabilities#data-management) capability. + * Your Kubernetes cluster must allow [outbound access to Sumo Logic](https://help.sumologic.com/APIs/General-API-Information/Sumo-Logic-Endpoints-and-Firewall-Security) to setup collection. Using a proxy is not currently supported. + * Please review our [minimum requirements](../README.md#minimum-requirements) and [support matrix](../README.md#support-matrix) -In this step you create a Sumo Logic Hosted Collector with a set of HTTP Sources to receive your Kubernetes data; creates Kubernetes secrets for the HTTP sources created; and deploy Fluentd using a Sumo-provided .yaml manifest. +## Prerequisite -First, you'll need to set up the relevant [fields](https://help.sumologic.com/Manage/Fields) in the Sumo Logic UI. This is to ensure your logs will be tagged with the correct metadata. +Sumo Logic Apps for Kubernetes and Explore require you to add the following [fields](https://help.sumologic.com/Manage/Fields) in the Sumo Logic UI to your Fields table schema. This is to ensure your logs are tagged with relevant metadata. This is a one time setup per Sumo Logic account. - cluster - container - deployment @@ -60,273 +39,166 @@ First, you'll need to set up the relevant [fields](https://help.sumologic.com/Ma - pod - service -### Automatic Source Creation and Setup YAML - -This approach requires access to the Sumo Logic Collector API. It will create Kubernetes resources in your environment and run a container that uses the [Sumo Logic Terraform provider](https://github.com/SumoLogic/sumologic-terraform-provider) to create a Hosted Collector and multiple HTTP Sources in Sumo. It also uses the [Kubernetes Terraform provider](https://www.terraform.io/docs/providers/kubernetes/index.html) to create a Kubernetes secret to store the HTTP source endpoints to be used by Fluentd later. - -First, create the namespace. (We recommend using `sumologic` for easier setup.) -```sh -kubectl create namespace sumologic -``` - -Run the following command to download and apply the YAML file containing all the Kubernetes resources. Replace the ``, ``, ``, `` and `` variables with your values. - -```sh -curl -s https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/kubernetes/setup-sumologic.yaml.tmpl | \ -sed 's/\$NAMESPACE'"//g" | \ -sed 's/\$SUMOLOGIC_ACCESSID'"//g" | \ -sed 's/\$SUMOLOGIC_ACCESSKEY'"//g" | \ -sed 's/\$COLLECTOR_NAME'"//g" | \ -sed 's/\$CLUSTER_NAME'"//g" | \ -tee setup-sumologic.yaml | \ -kubectl -n sumologic apply -f - -``` - -Run the following command to make sure the `collection-sumologic-setup` job is completed. You should see the status of the job being `Completed`. -```sh -kubectl get jobs -n sumologic -``` - -(Optional) You can delete the setup job which will automatically delete the associated pod as well. -```sh -kubectl delete job collection-sumologic-setup -n sumologic -``` - -Next, you will set up [Fluentd](#deploy-fluentd). +## Installation Steps -### Manual Source Creation and Setup +These steps require that no Prometheus exists. If you already have Prometheus installed select from the following options: -This is a manual alternative approach to the automatic script if you don't have API access or need customized configuration, such as reusing an existing collector. +In this method of installation, you will use our [templating tool](https://github.com/SumoLogic/sumologic-kubernetes-tools#k8s-template-generator) to generate the YAML needed to deploy Sumo Logic collection for Kubernetes. This tool will use our Helm chart to generate the YAML. You will configure the collection the same way that you would for Helm based install. However, instead of using Helm to install the Chart, the tool will output the rendered YAML you can deploy. -#### Create a Hosted Collector and an HTTP Source +The installation requires two parameters: +* __sumologic.accessId__ - Sumo [Access ID](https://help.sumologic.com/Manage/Security/Access-Keys). +* __sumologic.accessKey__ - Sumo [Access key](https://help.sumologic.com/Manage/Security/Access-Keys). -In this step you create a Sumo Logic Hosted Collector with a set of HTTP Sources to receive your Kubernetes data. +If you are installing the collection in a cluster that requires proxying outbound requests, please see the following [additional properties](./Installation_Behind_Proxy.md) you will need to set. -Create a Hosted Collector, following the instructions on [Configure a Hosted Collector](https://help.sumologic.com/03Send-Data/Hosted-Collectors/Configure-a-Hosted-Collector) in Sumo help. If you already have a Sumo Hosted Collector that you want to use, skip this step. +The following parameter is optional, but we recommend setting it. +* __sumologic.clusterName__ - An identifier for your Kubernetes cluster. This is the name you will see for the cluster in Sumo Logic. Default is `kubernetes`. -Create nine HTTP Sources under the collector you created in the previous step, one for each of the Kubernetes components that report metrics in this solution, one for logs, and one for events: +First you will generate the YAML to apply to your cluster. The following command contains the minimum parameters that can generate the YAML to setup Sumo Logic's Kubernetes collection. This command will generate the YAML and pipe it a file called `sumologic.yaml`. Please note that `--namespace` is required -* api-server-metrics -* kubelet-metrics -* controller-manager-metrics -* scheduler-metrics -* kube-state-metrics -* node-exporter-metrics -* default-metrics -* logs -* events - -Follow the instructions on [HTTP Logs and Metrics Source](https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source) to create the Sources, with the following additions: - -* **Naming the sources.** You can assign any name you like to the Sources, but it’s a good idea to assign a name to each Source that reflects the Kubernetes component from which it receives data. For example, you might name the source that receives API Server metrics “api-server-metrics”. -* **HTTP Source URLs.** When you configure each HTTP Source, Sumo will display the URL of the HTTP endpoint. Make a note of the URL. You will use it when you configure the Kubernetes service secrets to send data to Sumo. - -#### Create the namespace and secret - -Run the following command to create namespace `sumologic` - -```sh -kubectl create namespace sumologic -``` - -Run the following command to create a Kubernetes secret containing the 9 HTTP source URLs previously created. - -```sh -kubectl -n sumologic create secret generic sumologic \ - --from-literal=endpoint-metrics=$ENDPOINT_METRICS \ - --from-literal=endpoint-metrics-apiserver=$ENDPOINT_METRICS_APISERVER \ - --from-literal=endpoint-metrics-kube-controller-manager=$ENDPOINT_METRICS_KUBE_CONTROLLER_MANAGER \ - --from-literal=endpoint-metrics-kube-scheduler=$ENDPOINT_METRICS_KUBE_SCHEDULER \ - --from-literal=endpoint-metrics-kube-state=$ENDPOINT_METRICS_KUBE_STATE \ - --from-literal=endpoint-metrics-kubelet=$ENDPOINT_METRICS_KUBELET \ - --from-literal=endpoint-metrics-node-exporter=$ENDPOINT_METRICS_NODE_EXPORTER \ - --from-literal=endpoint-logs=$ENDPOINT_LOGS \ - --from-literal=endpoint-events=$ENDPOINT_EVENTS -``` - -## Deploy Fluentd - -In this step you will deploy Fluentd using a Sumo-provided .yaml manifest. - -### Use default configuration - -If you don't need to customize the configuration apply the `fluentd-sumologic.yaml` manifest with the following command. Replace the `` and `` variables with your values. - -```sh -curl https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/kubernetes/fluentd-sumologic.yaml.tmpl | \ -sed 's/\$NAMESPACE'"//g" | \ -sed 's/cluster kubernetes/cluster /g' | \ -kubectl -n sumologic apply -f - -``` - -### Customize configuration - -If you need to customize the configuration there are two commands to run. First, get the `fluentd-sumologic.yaml` manifest with following command: - -```sh -curl https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/kubernetes/fluentd-sumologic.yaml.tmpl | \ -sed 's/\$NAMESPACE'"/sumologic/g" >> fluentd-sumologic.yaml +```bash +kubectl run tools \ + -it --quiet --rm \ + --restart=Never \ + --image sumologic/kubernetes-tools:1.0.0-rc.0 -- \ + template \ + --name-template 'collection' \ + --set sumologic.accessId='' \ + --set sumologic.accessKey='' \ + --set sumologic.clusterName='' \ + | tee sumologic.yaml ``` -Next, customize the provided YAML file. Our [plugin](../../fluent-plugin-events/README.md#fluent-plugin-events) allows you to configure fields for events. Once done run the following command to apply the `fluentd-sumologic.yaml` manifest. +Next, you will need to apply the required CRD's for the Prometheus Operator. This is required before applying the generated YAML. -```sh -kubectl -n sumologic apply -f fluentd-sumologic.yaml +```bash +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_prometheusrules.yaml +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_thanosrulers.yaml ``` -The manifest will create the Kubernetes resources required by Fluentd. +If you are on Kubernetes 1.14 or earlier, you may received the following error: +```ValidationError(CustomResourceDefinition.spec): unknown field "preserveUnknownFields" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceDefinitionSpec; if you choose to ignore these errors, turn validation off with --validate=false``` -### Verify the pods are running +Simply add the `--validate=false` flag to the `kubectl apply` commands. -```sh -kubectl -n sumologic get pod +```bash +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml --validate=false +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml --validate=false +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml --validate=false +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_prometheusrules.yaml --validate=false +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml --validate=false +kubectl apply -f https://raw.githubusercontent.com/coreos/prometheus-operator/release-0.38/example/prometheus-operator-crd/monitoring.coreos.com_thanosrulers.yaml --validate=false ``` -Next, you will set up [Prometheus](#deploy-prometheus). - -## Deploy Prometheus - -In this step, you will configure the Prometheus server to write metrics to Fluentd. - -Download the Prometheus Operator `prometheus-overrides.yaml` by running +Finally, you can run `kubectl apply` on the file containing the rendered YAML from the previous step. ```bash -$ curl -LJO https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/prometheus-overrides.yaml +kubectl apply -f sumologic.yaml ``` -__NOTE__ If you plan to install Prometheus in a different namespace than you deployed Fluentd to in Step 1, or you have an existing Prometheus you plan to apply our configuration to running in a different namespace, please update the remote write API configuration to use the full service URL like, `http://collection-sumologic.sumologic.svc.cluster.local:9888`. - -You can also [Filter metrics](additional_prometheus_configuration.md#filter-metrics) and [Trim and relabel metrics](additional_prometheus_configuration.md#trim-and-relabel-metrics) in `prometheus-overrides.yaml`. - -Install `prometheus-operator` by generating the yaml files using Helm: +If you with to install the YAML in a different namespace, you can add the `--namespace` flag. The following will render the YAML and install in the `my-namespace` namespace. ```bash -$ helm fetch stable/prometheus-operator --version 8.2.0 -$ helm template prometheus-operator-8.2.0.tgz --name prometheus-operator --namespace=sumologic -f prometheus-overrides.yaml > prometheus.yaml +kubectl run tools \ + -it --quiet --rm \ + --restart=Never \ + --image sumologic/kubernetes-tools:1.0.0-rc.0 -- \ + template \ + --namespace 'my-namespace' \ + --name-template 'collection' \ + --set sumologic.accessId='' \ + --set sumologic.accessKey='' \ + --set sumologic.clusterName='' \ + | tee sumologic.yaml ``` -__NOTE__ Refer to the [requirements.yaml](../helm/sumologic/requirements.yaml) for the currently supported version. - -Before applying, change your default namespace for `kubectl` from `default` to `sumologic`. This is required as the YAML generated will deploy some resources to `kube-system` namespace as well. +Finally, you can run `kubectl apply` on the file containing the rendered YAML from the previous step. You must change your `kubectl` context to the namespace you wish to install in. ```bash -$ kubectl config set-context --current --namespace=sumologic -$ kubectl apply -f prometheus.yaml +kubectl config set-context --current --namespace=my-namespace +kubectl apply -f sumologic.yaml ``` -Verify `prometheus-operator` is running: +## Viewing Data In Sumo Logic -```sh -kubectl -n sumologic logs prometheus-prometheus-operator-prometheus-0 prometheus -f -``` +Once you have completed installation, you can [install the Kubernetes App and view the dashboards](https://help.sumologic.com/07Sumo-Logic-Apps/10Containers_and_Orchestration/Kubernetes/Install_the_Kubernetes_App_and_view_the_Dashboards) or [open a new Explore tab](https://help.sumologic.com/Solutions/Kubernetes_Solution/05Navigate_your_Kubernetes_environment) in Sumo Logic. If you do not see data in Sumo Logic, you can review our [troubleshooting guide](./Troubleshoot_Collection.md). -At this point setup is complete and metrics data is being sent to Sumo Logic. +## Troubleshooting Installation -__NOTE__ You can also [send custom metrics](additional_prometheus_configuration.md#custom-metrics)to Sumo Logic from Prometheus. +### Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" already exists +If you get `Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" already exists` it means you did not apply the CRD's yet. Please make sure you apply CRD's before rendering as documented above. -### Missing metrics for `controller-manager` or `scheduler` +### Fluentd Pods Stuck in CreateContainerConfigError +If the fluentd pods are in `CreateContainerConfigError` it can mean the setup job has not completed yet. Wait for the setup pod to complete and the issue should resolve itself. The setup job creates a secret and the error simply means the secret is not there yet. This usually resolves itself automatically. -Since there is a backward compatibility issue in the current version of chart, you may need to follow a workaround for sending these metrics under `controller-manager` or `scheduler`: +If the issue does not solve resolve automatically, you will need to look at the logs for the setup pod. Kubernetes schedules the job in a pod, so you can look at logs from the pod to see why the job is failing. First find the pod name in the namespace where you installed the rendered YAML. The pod name will contain `-setup` in the name. ```sh -kubectl -n kube-system patch service prometheus-operator-kube-controller-manager -p '{"spec":{"selector":{"k8s-app": "kube-controller-manager"}}}' -kubectl -n kube-system patch service prometheus-operator-kube-scheduler -p '{"spec":{"selector":{"k8s-app": "kube-scheduler"}}}' -kubectl -n kube-system patch service prometheus-operator-kube-controller-manager --type=json -p='[{"op": "remove", "path": "/spec/selector/component"}]' -kubectl -n kube-system patch service prometheus-operator-kube-scheduler --type=json -p='[{"op": "remove", "path": "/spec/selector/component"}]' +kubectl get pods ``` - -## Deploy FluentBit - -In this step, you will deploy FluentBit to forward logs to Fluentd. - -Run the following commands to download the FluentBit `fluent-bit-overrides.yaml` file and install `fluent-bit` - -```bash -$ curl -LJO https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/fluent-bit-overrides.yaml -$ helm fetch stable/fluent-bit --version 2.8.1 -$ helm template fluent-bit-2.8.1.tgz --name fluent-bit --namespace=sumologic -f fluent-bit-overrides.yaml > fluent-bit.yaml -$ kubectl apply -f fluent-bit.yaml +Get the logs from that pod: ``` -__NOTE__ You may see the following errors while applying the `yaml` : - -```bash -unable to recognize "fluent-bit.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1alpha1" -unable to recognize "fluent-bit.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1alpha1" +kubectl logs POD_NAME -f ``` -The above errors can be ignored. - -__NOTE__ Refer to the [requirements.yaml](../helm/sumologic/requirements.yaml) for the currently supported version. - -## Deploy Falco -In this step, you will deploy [Falco](https://falco.org/) to detect anomalous activity and capture Kubernetes Audit Events. This step is required only if you intend to use the Sumo Logic Kubernetes App. +### Error: collector with name 'sumologic' does not exist +If you get `Error: collector with name 'sumologic' does not exist +sumologic_http_source.default_metrics_source: Importing from ID`, you can safely ignore it and the installation should complete successfully. The installation process creates new [HTTP endpoints](https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source) in your Sumo Logic account, that are used to send data to Sumo. This error occurs if the endpoints had already been created by an earlier run of the installation process. -Download the file `falco-overrides.yaml` from GitHub: +You can find more information in our [troubleshooting documentation](Troubleshoot_Collection.md). +## Customizing Installation +All default properties for the Helm chart can be found in our [documentation](HelmChartConfiguration.md). We recommend creating a new `values.yaml` for each Kubernetes cluster you wish to install collection on and **setting only the properties you wish to override**. Once you have customized the file you can generate the YAML. The content of the `values.yaml` can be fed into the template generator as shown below. + ```bash -$ curl -LJO https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/falco-overrides.yaml -$ helm fetch stable/falco --version 1.0.9 -$ helm template falco-1.0.9.tgz --name falco --namespace=sumologic -f falco-overrides.yaml > falco.yaml -$ kubectl apply -f falco.yaml -``` +cat sumo-values.yaml | \ + kubectl run tools \ + -i --quiet --rm \ + --restart=Never \ + --image sumologic/kubernetes-tools:1.0.0-rc.0 -- \ + template \ + --name-template 'collection' \ + | tee sumologic.yaml +``` -__NOTE__ Refer to the [requirements.yaml](../helm/sumologic/requirements.yaml) for the currently supported version. +> **Tip**: To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) -__NOTE__ `Google Kubernetes Engine (GKE)` uses Container-Optimized OS (COS) as the default operating system for its worker node pools. COS is a security-enhanced operating system that limits access to certain parts of the underlying OS. Because of this security constraint, Falco cannot insert its kernel module to process events for system calls. However, COS provides the ability to leverage eBPF (extended Berkeley Packet Filter) to supply the stream of system calls to the Falco engine. eBPF is currently supported only on GKE and COS. More details [here](https://falco.org/docs/installation/). +## Upgrading Sumo Logic Collection -To install `Falco` on `GKE`, uncomment the following lines in the file `falco-overrides.yaml`: +**Note, if you are upgrading to version 1.x of our collection from a version before 1.x, please see our [migration guide](v1_migration_doc.md).** -``` -ebpf: - enabled: true -``` - -Install `falco` by generating the yaml files using Helm: +To upgrade you can simply re-generate the YAML when a new version of the Kubernetes collection is available. If you wish to upgrade to a specific version, you can pass the `--version` flag when generating the YAML. ```bash -$ helm template ./charts/falco --name falco --set dryRun=true -f falco-overrides.yaml > falco.yaml -$ kubectl apply -f falco.yaml -``` - - -## Tear down - -To delete `falco` from the Kubernetes cluster: - -```sh -kubectl delete -f falco.yaml +cat values.yaml | \ + kubectl run tools \ + -i --quiet --rm \ + --restart=Never \ + --image sumologic/kubernetes-tools:1.0.0-rc.0 -- \ + template \ + --name-template 'collection' \ + --version=1.0.0 + | tee sumologic.yaml ``` -To delete `fluent-bit` from the Kubernetes cluster: - -```sh -kubectl delete -f fluent-bit.yaml -``` +## Uninstalling Sumo Logic Collection -To delete `prometheus-operator` from the Kubernetes cluster: +To uninstall/delete, simply run `kubectl delete` on the generated YAML. -```sh -kubectl delete -f prometheus.yaml -``` - -__NOTE__ This command will not remove the Custom Resource Definitions created. - -To delete the `fluentd-sumologic` app: - -```sh -kubectl delete -f ./fluentd-sumologic.yaml +```bash +kubectl delete -f sumologic.yaml ``` -To delete the `sumologic` secret (for recreating collector/sources): +To remove the Kubernetes secret: -```sh -kubectl -n sumologic delete secret sumologic +```bash +kubectl delete secret sumologic ``` -To delete the `sumologic` namespace and all resources under it: - -```sh -kubectl delete namespace sumologic -``` +Then delete the associated hosted collector in the Sumo Logic UI. diff --git a/deploy/docs/SideBySidePrometheus.md b/deploy/docs/SideBySidePrometheus.md index d31109d4d0..cd7031452c 100644 --- a/deploy/docs/SideBySidePrometheus.md +++ b/deploy/docs/SideBySidePrometheus.md @@ -1,23 +1,179 @@ # How to install our Prometheus side by side with your existing Prometheus -When installing our Helm Chart it is possible to have more than one Prometheus server running in the same cluster. However, do note that you cannot have more than one Prometheus Operator running in the same cluster. To install if you have an existing Prometheus Operator, please follow the steps [here](./existingPrometheusDoc.md). +When installing our Helm Chart it is possible to have more than one Prometheus server running in the same cluster. However, do note that you cannot have more than one Prometheus Operator running in the same cluster. This document will take you through the steps to set up Sumo Logic collection when you have an existing Prometheus Operator you wish to keep intact. -To use a different port number than the default 9100 set the following fields for the Prometheus node exporter when installing our Helm Chart. For example: +# Installation with Helm +Our Helm chart deploys Kubernetes resources for collecting Kubernetes logs, metrics, and events; enriching them with deployment, pod, and service level metadata; and sends them to Sumo Logic. + + +- [Requirements](#requirements) +- [Prerequisite](#prerequisite) +- [Installation Steps](#installation-steps) +- [Viewing Data In Sumo Logic](#viewing-data-in-sumo-logic) +- [Troubleshooting Installation](#troubleshooting-installation) +- [Customizing Installation](#customizing-installation) +- [Upgrade Sumo Logic Collection](#upgrading-sumo-logic-collection) +- [Uninstalling Sumo Logic Collection](#uninstalling-sumo-logic-collection) + + + +## Requirements + +If you don’t already have a Sumo account, you can create one by clicking the Free Trial button on https://www.sumologic.com/. + +The following are required to setup Sumo Logic's Kubernetes collection. + + * An [Access ID and Access Key](https://help.sumologic.com/Manage/Security/Access-Keys) with [Manage Collectors](https://help.sumologic.com/Manage/Users-and-Roles/Manage-Roles/05-Role-Capabilities#data-management) capability. + * Please review our [minimum requirements](../README.md#minimum-requirements) and [support matrix](../README.md#support-matrix) + + +## Prerequisite + +Sumo Logic Apps for Kubernetes and Explore require you to add the following [fields](https://help.sumologic.com/Manage/Fields) in the Sumo Logic UI to your Fields table schema. This is to ensure your logs are tagged with relevant metadata. This is a one time setup per Sumo Logic account. +- cluster +- container +- deployment +- host +- namespace +- node +- pod +- service + +## Installation Steps +The Helm chart installation requires two parameter overrides: +* __sumologic.accessId__ - Sumo [Access ID](https://help.sumologic.com/Manage/Security/Access-Keys). +* __sumologic.accessKey__ - Sumo [Access key](https://help.sumologic.com/Manage/Security/Access-Keys). + +The following parameter is optional, but we recommend setting it. +* __sumologic.clusterName__ - An identifier for your Kubernetes cluster. This is the name you will see for the cluster in Sumo Logic. Default is `kubernetes`. + +If you are installing the collection in a cluster that requires proxying outbound requests, please see the following [additional properties](./Installation_Behind_Proxy.md) you will need to set. + +Since we are installing with an existing Prometheus Operator we must also define the following values. + +* __prometheus-operator.prometheusOperator.enabled=false__ - Two operators cannot run in the same cluster at the same time, so this disables ours but preserves the existing. +* __prometheus-operator.prometheus-node-exporter.service.port=9200__ - Since node exporter uses a `NodePort` we have to change the port. +* __prometheus-operator.prometheus-node-exporter.service.targetPort=9200__ - Since node exporter uses a `NodePort` we have to change the port. + +The following helm commands support Helm2 or Helm3. + +To install the chart, first add the `sumologic` private repo: + +```bash +helm repo add sumologic https://sumologic.github.io/sumologic-kubernetes-collection +``` + +Next you can run `helm upgrade --install` to install our chart. An example command with the minimum parameters is provided below. The following command will install the Sumo Logic chart with the release name `my-release` in the namespace your `kubectl` context is currently set to. Node that because this is installing our chart in a cluster where an existing prometheus operator is running, we need to disable our operator and update the node exporter ports. + +```bash +helm upgrade --install my-release sumologic/sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.prometheusOperator.enabled=false --set prometheus-operator.prometheus-node-exporter.service.port=9200 --set prometheus-operator.prometheus-node-exporter.service.targetPort=9200 +``` +> **Note**: If the release exists, it will be upgraded, otherwise it will be installed. + +If you wish to install the chart in a different existing namespace you can do the following: + +```bash +helm upgrade --install my-release sumologic/sumologic --namespace=my-namespace --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.prometheusOperator.enabled=false --set prometheus-operator.prometheus-node-exporter.service.port=9200 --set prometheus-operator.prometheus-node-exporter.service.targetPort=9200 +``` + +For Helm3, if the namespace does not exist, you can add the `--create-namespace` flag. + +```bash +helm upgrade --install my-release sumologic/sumologic --namespace=my-namespace --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.prometheusOperator.enabled=false --set prometheus-operator.prometheus-node-exporter.service.port=9200 --set prometheus-operator.prometheus-node-exporter.service.targetPort=9200 --create-namespace +``` + +## Viewing Data In Sumo Logic + +Once you have completed installation, you can [install the Kubernetes App and view the dashboards](https://help.sumologic.com/07Sumo-Logic-Apps/10Containers_and_Orchestration/Kubernetes/Install_the_Kubernetes_App_and_view_the_Dashboards) or [open a new Explore tab](https://help.sumologic.com/Solutions/Kubernetes_Solution/05Navigate_your_Kubernetes_environment) in Sumo Logic. If you do not see data in Sumo Logic, you can review our [troubleshooting guide](./Troubleshoot_Collection.md). + +## Troubleshooting Installation + +### Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" already exists +If you get `Error: customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" already exists` on a Helm2 installation, run the above command with the `--set prometheus-operator.prometheusOperator.createCustomResource=false` flag: + +```bash +helm upgrade --install my-release sumologic/sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.enabled=false --set prometheus-operator.prometheusOperator.prometheus-node-exporter.service.port=9200 --set prometheus-operator.prometheus-node-exporter.service.targetPort=9200 --set prometheus-operator.prometheusOperator.createCustomResource=false +``` +### Error: timed out waiting for the condition +If `helm upgrade --install` hangs, it usually means the pre-install setup job is failing and is in a retry loop. Due to a Helm limitation, errors from the setup job cannot be fed back to the `helm upgrade --install` command. Kubernetes schedules the job in a pod, so you can look at logs from the pod to see why the job is failing. First find the pod name in the namespace where the Helm chart was deployed. The pod name will contain `-setup` in the name. + +```sh +kubectl get pods +``` +> **Tip**: If the pod does not exist, it is possible it has been evicted. Re-run the `helm upgrade --install` to recreate it and while that command is running, use another shell to get the name of the pod. + +Get the logs from that pod: +``` +kubectl logs POD_NAME -f +``` +### Error: collector with name 'sumologic' does not exist +If you get `Error: collector with name 'sumologic' does not exist +sumologic_http_source.default_metrics_source: Importing from ID`, you can safely ignore it and the installation should complete successfully. The installation process creates new [HTTP endpoints](https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source) in your Sumo Logic account, that are used to send data to Sumo. This error occurs if the endpoints had already been created by an earlier run of the installation process. + +You can find more information in our [troubleshooting documentation](Troubleshoot_Collection.md). + +## Customizing Installation +All default properties for the Helm chart can be found in our [documentation](HelmChartConfiguration.md). We recommend creating a new `values.yaml` for each Kubernetes cluster you wish to install collection on and **setting only the properties you wish to override**. Once you have customized you can use the following commands to install or upgrade. Remember to define the properties in our [requirements section](#requirements) in the `values.yaml` as well or pass them in via `--set` + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml +``` +> **Tip**: To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) + +## Upgrading Sumo Logic Collection + +**Note, if you are upgrading to version 1.x of our collection from a version before 1.x, please see our [migration guide](v1_migration_doc.md).** + +To upgrade our helm chart to a newer version, you must first run update your local helm repo. + +```bash +helm repo update ``` ---set prometheus-operator.prometheus-node-exporter.service.port=9200 --set prometheus-operator.prometheus-node-exporter.service.targetPort=9200 + +Next, you can run `helm upgrade --install` to upgrade to that version. The following upgrades the current version of `my-release` to the latest. + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml ``` -Or add the following to the prometheus-operator section of your overrides values.yaml: +If you wish to upgrade to a specific version, you can use the `--version` flag. +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml --version=1.0.0 ``` -prometheus-operator: - ... - prometheus-node-exporter: - service: - port: 9200 - targetPort: 9200 - ... + +If you no longer have your `values.yaml` from the first installation or do not remember the options you added via `--set` you can run the following to see the values for the currently installed helm chart. For example, if the release is called `my-release` you can run the following. + +```bash +helm get values my-release +``` + +If something goes wrong, or you want to go back to the previous version, +you can [rollback changes using helm](https://helm.sh/docs/helm/helm_rollback/): + +``` +helm history my-release +helm rollback my-release +``` + +## Uninstalling Sumo Logic Collection + +To uninstall/delete the Helm chart: + +```bash +helm delete my-release +``` +> **Helm2 Tip**: Use helm delete --purge my-release to completely remove the release from Helm internal storage + +> **Helm3 Tip**: In Helm3 the default behavior is to purge history. Use --keep-history to preserve it while deleting the release.ease. + +The command removes all the Kubernetes components associated with the chart and deletes the release. + +To remove the Kubernetes secret: + +```bash +kubectl delete secret sumologic ``` -__NOTE__ To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) +Then delete the associated hosted collector in the Sumo Logic UI. \ No newline at end of file diff --git a/deploy/docs/Troubleshoot_Collection.md b/deploy/docs/Troubleshoot_Collection.md index 7feaf8af91..542099ebf8 100644 --- a/deploy/docs/Troubleshoot_Collection.md +++ b/deploy/docs/Troubleshoot_Collection.md @@ -260,7 +260,29 @@ helm upgrade collection sumologic/sumologic --reuse-values --version= - -- [Install Fluentd, Fluent-bit, and Falco](#install-fluentd-fluent-bit-and-falco) -- [Overwrite Prometheus Remote Write Configuration](#overwrite-prometheus-remote-write-configuration) -- [Merge Prometheus Remote Write Configuration](#merge-prometheus-remote-write-configuration) +- [Prerequisite](#prerequisite) +- [Install Sumo Logic Helm Chart](#install-sumo-logic-helm-chart) +- [Update Existing Prometheus Operator Helm Chart](#update-existing-prometheus-operator-helm-chart) +- [Viewing Data In Sumo Logic](#viewing-data-in-sumo-logic) +- [Merge Prometheus Configuration](#merge-prometheus-configuration) +- [Troubleshooting](#troubleshooting) +- [Customizing Installation](#customizing-installation) +- [Upgrade Sumo Logic Collection](#upgrading-sumo-logic-collection) +- [Uninstalling Sumo Logic Collection](#uninstalling-sumo-logic-collection) -## Install Fluentd, Fluent-bit, and Falco +This document will walk you through how to set up Sumo Logic Kubernetes collection when you already have Prometheus running using the Prometheus Operator. In these steps, you will modify your installed Prometheus operator to add in the minimum configuration that Sumo Logic needs. -Installation of collection in a cluster where a prometheus operator already exists requires that you modify Sumo Logic's [values.yaml](https://github.com/SumoLogic/sumologic-kubernetes-collection/blob/master/deploy/helm/sumologic/values.yaml) file. Run the following to download the `values.yaml` file +If you do not wish to modify your Prometheus Operator and wish to run side-by-side with our collection, please refer to our [How to install our Prometheus side by side with your existing Prometheus](./SideBySidePrometheus.md) documentation. + +## Prerequisite + +Sumo Logic Apps for Kubernetes and Explore require you to add the following [fields](https://help.sumologic.com/Manage/Fields) in the Sumo Logic UI to your Fields table schema. This is to ensure your logs are tagged with relevant metadata. This is a one time setup per Sumo Logic account. +- cluster +- container +- deployment +- host +- namespace +- node +- pod +- service + +## Install Sumo Logic Helm Chart + +The Helm chart installation requires two parameter overrides: +* __sumologic.accessId__ - Sumo [Access ID](https://help.sumologic.com/Manage/Security/Access-Keys). +* __sumologic.accessKey__ - Sumo [Access key](https://help.sumologic.com/Manage/Security/Access-Keys). + +If you are installing the collection in a cluster that requires proxying outbound requests, please see the following [additional properties](./Installation_Behind_Proxy.md) you will need to set. + +The following parameter is optional, but we recommend setting it. +* __sumologic.clusterName__ - An identifier for your Kubernetes cluster. This is the name you will see for the cluster in Sumo Logic. Default is `kubernetes`. + +The following helm commands support Helm2 or Helm3. + +To install the chart, first add the `sumologic` private repo: + +```bash +helm repo add sumologic https://sumologic.github.io/sumologic-kubernetes-collection +``` + +Next you can run `helm upgrade --install` to install our chart. An example command with the minimum parameters is provided below. The following command will install the Sumo Logic chart with the release name `my-release` in the namespace your `kubectl` context is currently set to. The below command also disables the `prometheus-operator` sub-chart since we will be modifying the existing prometheus operator install. ```bash -curl -LJO https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/sumologic/values.yaml +helm upgrade --install my-release sumologic/sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.enabled=false ``` +> **Note**: If the release exists, it will be upgraded, otherwise it will be installed. -Edit the `values.yaml` file, setting `prometheus-operator.enabled = false`. This modification will instruct Helm to install all the needed collection components (FluentD, FluentBit, and Falco), but it will not install the Prometheus Operator. Run the following command to install collection on your cluster. +If you wish to install the chart in a different existing namespace you can do the following: ```bash -helm install sumologic/sumologic \ - --name collection --namespace sumologic -f values.yaml \ - --set sumologic.accessId= \ - --set sumologic.accessKey= \ - --set sumologic.clusterName= +helm upgrade --install my-release sumologic/sumologic --namespace=my-namespace --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.enabled=false ``` -**NOTE**: -In case the prometheus-operator is installed in a different namespace as compared to where the Sumo Logic Solution is deployed, you would need to do the following two steps: +For Helm3, if the namespace does not exist, you can add the `--create-namespace` flag. + +```bash +helm upgrade --install my-release sumologic/sumologic --namespace=my-namespace --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.enabled=false --create-namespace +``` + +## Update Existing Prometheus Operator Helm Chart + +**Note that If you have made extensive customization to the current Prometheus Operator Helm install then you will need to [merge your existing configuration with ours](#merge-prometheus-configuration) avoiding conflicts or you may want to [run our Prometheus side-by-side](./SideBySidePrometheus.md).** + +Next you will modify your Prometheus Operator installation with the required configuration to collect the metrics into Sumo Logic. Please note that this process is best when you have not customized the existing Prometheus Operator installation. If you have, please look at [our section on merging the configuration](#merge-prometheus-configuration). -##### 1. Copy one of the configmaps that exposes the release name, which is used in the remote write urls. +If the Prometheus Operator is installed in a different namespace as compared to where the Sumo Logic Chart is deployed, you would need to do the following step to copy the `ConfigMap` that exposes the release name, which is used in the remote write urls. For example:\ If the Sumo Logic Solution is deployed in `` and the existing prometheus-operator is in ``, run the below command: @@ -40,26 +84,28 @@ kubectl get configmap sumologic-configmap \ --namespace= --export -o yaml | \ kubectl apply --namespace= -f - ``` -##### 2. Update Prometheus remote write URL's + Run the following commands to update the [remote write configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write) of the prometheus operator with the prometheus overrides we provide in our [prometheus-overrides.yaml](https://github.com/SumoLogic/sumologic-kubernetes-collection/blob/master/deploy/helm/prometheus-overrides.yaml). -Run the following command to download our prometheus-overrides.yaml file +Run the following command to download our prometheus-overrides.yaml file. Please review our configuration as it will be applied to your existing operator configuration. ```bash -curl -LJO https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/prometheus-overrides.yaml > prometheus-overrides.yaml +curl -LJO https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/prometheus-overrides.yaml ``` -Next run +Next you can upgrade your Prometheus-Operator. The following command assumes it is installed with the release name `prometheus-operator`. Remember, this command will update your Prometheus Operator to be configured with our default settings. If you wish to preserve your settings and merge with what is required for Sumo logic, then please look at the section on [how to merge the configuration](#merge-prometheus-configuration). ```bash -helm upgrade prometheus-operator stable/prometheus-operator -f prometheus-overrides.yaml \ - --set prometheus-operator.prometheus-node-exporter.service.port=9200 \ - --set prometheus-operator.prometheus-node-exporter.service.targetPort=9200 +helm upgrade prometheus-operator stable/prometheus-operator -f prometheus-overrides.yaml ``` -## Merge Prometheus Remote Write Configuration +## Viewing Data In Sumo Logic + +Once you have completed installation, you can [install the Kubernetes App and view the dashboards](https://help.sumologic.com/07Sumo-Logic-Apps/10Containers_and_Orchestration/Kubernetes/Install_the_Kubernetes_App_and_view_the_Dashboards) or [open a new Explore tab](https://help.sumologic.com/Solutions/Kubernetes_Solution/05Navigate_your_Kubernetes_environment) in Sumo Logic. If you do not see data in Sumo Logic, you can review our [troubleshooting guide](./Troubleshoot_Collection.md). + +## Merge Prometheus Configuration -If you have customized your Prometheus remote write configuration, follow these steps to merge the configurations. +If you have customized your Prometheus configuration, follow these steps to merge the configurations. Helm supports providing multiple configuration files, and priority will be given to the last (right-most) file specified. You can obtain your current prometheus configuration by running @@ -74,3 +120,93 @@ helm upgrade prometheus-operator stable/prometheus-operator -f current-values.ya ``` __NOTE__ To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) + +## Troubleshooting + +### UPGRADE FAILED: failed to create resource: Internal error occurred: failed calling webhook "prometheusrulemutate.monitoring.coreos.com" + +If you receive the above error, you can take the following steps and then repeat the `helm upgrade` command. + +```bash +kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io prometheus-operator-admission +kubectl delete MutatingWebhookConfiguration prometheus-operator-admission +``` +### Error: timed out waiting for the condition +If `helm upgrade --install` hangs, it usually means the pre-install setup job is failing and is in a retry loop. Due to a Helm limitation, errors from the setup job cannot be fed back to the `helm upgrade --install` command. Kubernetes schedules the job in a pod, so you can look at logs from the pod to see why the job is failing. First find the pod name in the namespace where the Helm chart was deployed. The pod name will contain `-setup` in the name. + +```sh +kubectl get pods +``` +> **Tip**: If the pod does not exist, it is possible it has been evicted. Re-run the `helm upgrade --install` to recreate it and while that command is running, use another shell to get the name of the pod. + +Get the logs from that pod: +``` +kubectl logs POD_NAME -f +``` + +### Error: collector with name 'sumologic' does not exist +If you get `Error: collector with name 'sumologic' does not exist +sumologic_http_source.default_metrics_source: Importing from ID`, you can safely ignore it and the installation should complete successfully. The installation process creates new [HTTP endpoints](https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source) in your Sumo Logic account, that are used to send data to Sumo. This error occurs if the endpoints had already been created by an earlier run of the installation process. + +You can find more information in our [troubleshooting documentation](Troubleshoot_Collection.md). + +## Customizing Installation +All default properties for the Helm chart can be found in our [documentation](HelmChartConfiguration.md). We recommend creating a new `values.yaml` for each Kubernetes cluster you wish to install collection on and **setting only the properties you wish to override**. Once you have customized you can use the following commands to install or upgrade. Remember to define the properties in our [requirements section](#requirements) in the `values.yaml` as well or pass them in via `--set` + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml +``` +> **Tip**: To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) + +## Upgrading Sumo Logic Collection + +**Note, if you are upgrading to version 1.x of our collection from a version before 1.x, please see our [migration guide](v1_migration_doc.md).** + +To upgrade our helm chart to a newer version, you must first run update your local helm repo. + +```bash +helm repo update +``` + +Next, you can run `helm upgrade --install` to upgrade to that version. The following upgrades the current version of `my-release` to the latest. + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml +``` + +If you wish to upgrade to a specific version, you can use the `--version` flag. + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml --version=1.0.0 +``` + +If you no longer have your `values.yaml` from the first installation or do not remember the options you added via `--set` you can run the following to see the values for the currently installed helm chart. For example, if the release is called `my-release` you can run the following. + +```bash +helm get values my-release +``` + +After upgrading the Sumo Logic chart, you can repeat the steps described in [Update Existing Prometheus Operator Helm Chart](#update-existing-prometheus-operator-helm-chart) to upgrade the Prometheus configuration. + +## Uninstalling Sumo Logic Collection + +To uninstall/delete the Helm chart: + +```bash +helm delete my-release +``` +> **Helm2 Tip**: Use helm delete --purge my-release to completely remove the release from Helm internal storage + +> **Helm3 Tip**: In Helm3 the default behavior is to purge history. Use --keep-history to preserve it while deleting the release.ease. + +The command removes all the Kubernetes components associated with the chart and deletes the release. + +To remove the Kubernetes secret: + +```bash +kubectl delete secret sumologic +``` + +Then delete the associated hosted collector in the Sumo Logic UI. + +Finally, you can restore your Prometheus to the original configuration before you installed Sumo Logic. \ No newline at end of file diff --git a/deploy/docs/standAlonePrometheus.md b/deploy/docs/standAlonePrometheus.md index c40c7bbbbd..1f11ea58fb 100644 --- a/deploy/docs/standAlonePrometheus.md +++ b/deploy/docs/standAlonePrometheus.md @@ -2,13 +2,78 @@ __NOTE__: The Sumo Logic Kubernetes collection process does not support collecting metrics from scaling Prometheus replicas. If you are running multiple Prometheus replicas, please follow our [Side-by-Side](SideBySidePrometheus.md) instructions. -1. Download the Prometheus Operator `prometheus-overrides.yaml` by running + +- [Prerequisite](#prerequisite) +- [Install Sumo Logic Helm Chart](#install-sumo-logic-helm-chart) +- [Update Existing Prometheus](#update-existing-prometheus) +- [Viewing Data In Sumo Logic](#viewing-data-in-sumo-logic) +- [Customizing Installation](#customizing-installation) +- [Upgrade Sumo Logic Collection](#upgrading-sumo-logic-collection) +- [Uninstalling Sumo Logic Collection](#uninstalling-sumo-logic-collection) + + + +This document will walk you through how to set up Sumo Logic Kubernetes collection when you already have Prometheus running, not using the Prometheus Operator. In these steps, you will modify your installed Prometheus to add in the minimum configuration that Sumo Logic needs. If you are using the Prometheus Operator, please refer to our guide on installing with an existing [Prometheus Operator](./existingPrometheusDoc.md). + +## Prerequisite + +Sumo Logic Apps for Kubernetes and Explore require you to add the following [fields](https://help.sumologic.com/Manage/Fields) in the Sumo Logic UI to your Fields table schema. This is to ensure your logs are tagged with relevant metadata. This is a one time setup per Sumo Logic account. +- cluster +- container +- deployment +- host +- namespace +- node +- pod +- service + +## Install Sumo Logic Helm Chart + +The Helm chart installation requires two parameter overrides: +* __sumologic.accessId__ - Sumo [Access ID](https://help.sumologic.com/Manage/Security/Access-Keys). +* __sumologic.accessKey__ - Sumo [Access key](https://help.sumologic.com/Manage/Security/Access-Keys). + +If you are installing the collection in a cluster that requires proxying outbound requests, please see the following [additional properties](./Installation_Behind_Proxy.md) you will need to set. + +The following parameter is optional, but we recommend setting it. +* __sumologic.clusterName__ - An identifier for your Kubernetes cluster. This is the name you will see for the cluster in Sumo Logic. Default is `kubernetes`. + +The following helm commands support Helm2 or Helm3. + +To install the chart, first add the `sumologic` private repo: + +```bash +helm repo add sumologic https://sumologic.github.io/sumologic-kubernetes-collection +``` + +Next you can run `helm upgrade --install` to install our chart. An example command with the minimum parameters is provided below. The following command will install the Sumo Logic chart with the release name `my-release` in the namespace your `kubectl` context is currently set to. The below command also disables the `prometheus-operator` sub-chart since we will be modifying the existing prometheus operator install. + +```bash +helm upgrade --install my-release sumologic/sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.enabled=false +``` +> **Note**: If the release exists, it will be upgraded, otherwise it will be installed. + +If you wish to install the chart in a different existing namespace you can do the following: + +```bash +helm upgrade --install my-release sumologic/sumologic --namespace=my-namespace --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.enabled=false +``` + +For Helm3, if the namespace does not exist, you can add the `--create-namespace` flag. + +```bash +helm upgrade --install my-release sumologic/sumologic --set sumologic.accessId= --set sumologic.accessKey= --set sumologic.clusterName="" --set prometheus-operator.enabled=false --create-namespace +``` + +## Update Existing Prometheus + +First, Download the Prometheus Operator `prometheus-overrides.yaml` by running ```bash $ curl -LJO https://raw.githubusercontent.com/SumoLogic/sumologic-kubernetes-collection/release-v1.0/deploy/helm/prometheus-overrides.yaml ``` -2. Make the following modifications to the `remoteWrite` section of the `prometheus-overrides.yaml` file: +Next, make the following modifications to the `remoteWrite` section of the `prometheus-overrides.yaml` file: * `writeRelabelConfigs:` change to `write_relabel_configs:` * `sourceLabels:` change to `source_labels:` @@ -24,16 +89,89 @@ If you have installed the Sumo Logic helm chart with release name `collection` i ``` `$(CHART).$(NAMESPACE)` will be replaced by `collection-sumologic.prometheus` ``` -3. Copy the modified `remoteWrite` section of the `prometheus-overrides.yaml` file to your Prometheus configuration file’s `remote_write` section, as per the documentation [here](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write) -4. Run the following command to find the existing Prometheus pod. +Next, copy the modified `remoteWrite` section of the `prometheus-overrides.yaml` file to your Prometheus configuration file’s `remote_write` section, as per the documentation [here](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write) + +Then run the following command to find the existing Prometheus pod. ``` kubectl get pods | grep prometheus ``` -5. Delete the existing Prometheus pod so that Kubernetes will respawn it with the updated configuration. + +Finally, delete the existing Prometheus pod so that Kubernetes will respawn it with the updated configuration. ``` kubectl delete pods ``` __NOTE__ To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) + +## Viewing Data In Sumo Logic + +Once you have completed installation, you can [install the Kubernetes App and view the dashboards](https://help.sumologic.com/07Sumo-Logic-Apps/10Containers_and_Orchestration/Kubernetes/Install_the_Kubernetes_App_and_view_the_Dashboards) or [open a new Explore tab](https://help.sumologic.com/Solutions/Kubernetes_Solution/05Navigate_your_Kubernetes_environment) in Sumo Logic. If you do not see data in Sumo Logic, you can review our [troubleshooting guide](./Troubleshoot_Collection.md). + +## Customizing Installation +All default properties for the Helm chart can be found in our [documentation](HelmChartConfiguration.md). We recommend creating a new `values.yaml` for each Kubernetes cluster you wish to install collection on and **setting only the properties you wish to override**. Once you have customized you can use the following commands to install or upgrade. Remember to define the properties in our [requirements section](#requirements) in the `values.yaml` as well or pass them in via `--set` + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml +``` +> **Tip**: To filter or add custom metrics to Prometheus, [please refer to this document](additional_prometheus_configuration.md) + +## Upgrading Sumo Logic Collection + +**Note, if you are upgrading to version 1.x of our collection from a version before 1.x, please see our [migration guide](v1_migration_doc.md).** + +To upgrade our helm chart to a newer version, you must first run update your local helm repo. + +```bash +helm repo update +``` + +Next, you can run `helm upgrade --install` to upgrade to that version. The following upgrades the current version of `my-release` to the latest. + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml +``` + +If you wish to upgrade to a specific version, you can use the `--version` flag. + +```bash +helm upgrade --install my-release sumologic/sumologic -f values.yaml --version=1.0.0 +``` + +If you no longer have your `values.yaml` from the first installation or do not remember the options you added via `--set` you can run the following to see the values for the currently installed helm chart. For example, if the release is called `my-release` you can run the following. + +```bash +helm get values my-release +``` + +If something goes wrong, or you want to go back to the previous version, +you can [rollback changes using helm](https://helm.sh/docs/helm/helm_rollback/): + +``` +helm history my-release +helm rollback my-release +``` + +Finally, you can repeat the steps to [Update Existing Prometheus](#update-existing-prometheus) with the latest. + +## Uninstalling Sumo Logic Collection + +To uninstall/delete the Helm chart: + +```bash +helm delete my-release +``` +> **Helm2 Tip**: Use helm delete --purge my-release to completely remove the release from Helm internal storage + +> **Helm3 Tip**: In Helm3 the default behavior is to purge history. Use --keep-history to preserve it while deleting the release.ease. + +The command removes all the Kubernetes components associated with the chart and deletes the release. + +To remove the Kubernetes secret: + +```bash +kubectl delete secret sumologic +``` + +Then delete the associated hosted collector in the Sumo Logic UI. \ No newline at end of file