diff --git a/CHANGELOG.md b/CHANGELOG.md index 9158a35fb6..3a8cfdd2bc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - chore: upgrade Fluent Bit to v1.6.10-sumo-3 [#2712] - chore: upgrade otelcol to 0.66.0-sumo-0 [#2686] [#2687] [#2692] [#2693] +- feat(otellogs): read from end [#2710] - fix(openshift): changed allowed fsgroups in SecurityContextConstraints [#2717] - fix(openshift): set securityContexts for otelcol-logs-collector [#2717] - fix: obey proxy settings in otelcol [#2719] @@ -21,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [#2687]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2687 [#2693]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2693 [#2692]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2692 +[#2710]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2710 [#2712]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2712 [#2717]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2717 [#2719]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2719 diff --git a/deploy/helm/sumologic/conf/logs/collector/otelcol/config.yaml b/deploy/helm/sumologic/conf/logs/collector/otelcol/config.yaml index d36f76fd12..537f2de371 100644 --- a/deploy/helm/sumologic/conf/logs/collector/otelcol/config.yaml +++ b/deploy/helm/sumologic/conf/logs/collector/otelcol/config.yaml @@ -42,7 +42,6 @@ receivers: filelog/containers: include: - /var/log/pods/*/*/*.log - start_at: beginning ## sets fingerprint_size to 17kb in order to match the longest possible docker line (which by default is 16kb) ## we want to include timestamp, which is at the end of the line fingerprint_size: 17408 diff --git a/docs/v3-migration-doc.md b/docs/v3-migration-doc.md index ab8e53eb35..7ec4c1cbd3 100644 --- a/docs/v3-migration-doc.md +++ b/docs/v3-migration-doc.md @@ -9,8 +9,9 @@ - [Otelcol StatefulSets](#otelcol-statefulsets) - [Tracing/Instrumentation changes](#tracinginstrumentation-changes) - [Additional Service Monitors](#additional-service-monitors) - - [Known issues](#known-issues) - - [Cannot delete pod if using Tailing Sidecar Operator](#cannot-delete-pod-if-using-tailing-sidecar-operator) +- [Known issues](#known-issues) + - [Cannot delete pod if using Tailing Sidecar Operator](#cannot-delete-pod-if-using-tailing-sidecar-operator) + - [OpenTelemetry Collector doesn't read logs from the beginning of files](#opentelemetry-collector-doesnt-read-logs-from-the-beginning-of-files) Based on the feedback from our users, we will be introducing several changes to the Sumo Logic Kubernetes Collection solution. @@ -89,6 +90,11 @@ In this document we detail the changes as well as the exact steps for migration. - Upgrading Tailing Sidecar Operator helm chart to v0.5.5. There is no breaking change if using annotations only. +- OpenTelemetry Logs Collector will read from end of file now. + + See [OpenTelemetry Collector doesn't read logs from the beginning of files](#opentelemetry-collector-doesnt-read-logs-from-the-beginning-of-files) + if you want to keep old behavior. + - Changed `otelagent` from `DaemonSet` to `StatefulSet` - Moved parameters from `otelagent.*` to `otelcolInstrumentation.*` @@ -269,3 +275,19 @@ Please try to remove pod later. [Falco documentation]: https://github.com/falcosecurity/charts/tree/falco-2.4.2/falco [metrics-server-upgrade]: https://github.com/bitnami/charts/tree/5b09f7a7c0d9232f5752840b6c4e5cdc56d7f796/bitnami/metrics-server#to-600 [kube-prometheus-stack-image-migration]: https://github.com/prometheus-community/helm-charts/tree/kube-prometheus-stack-42.1.0/charts/kube-prometheus-stack#from-41x-to-42x + +#### OpenTelemetry Collector doesn't read logs from the beginning of files + +This is done by design. We are not going to read logs from time before the collection has been installed. + +In order to keep old behavior (can result in logs duplication for some cases), please use the following configuration: + +``` +metadata: + logs: + config: + merge: + receivers: + filelog/containers: + start_at: beginning +``` diff --git a/tests/helm/logs_otc/static/basic.output.yaml b/tests/helm/logs_otc/static/basic.output.yaml index af5cffff59..802ad4dd5a 100644 --- a/tests/helm/logs_otc/static/basic.output.yaml +++ b/tests/helm/logs_otc/static/basic.output.yaml @@ -143,7 +143,6 @@ data: - from: body.log to: body type: move - start_at: beginning storage: file_storage journald: directory: /var/log/journal