Memory leak problem with Opentelemetry Collector Using KafkaReceiver

[Dennis8274]

Component(s)

receiver/kafka

What happened?

Description

a memory-leak found in my process,which fetch msg from kafka using sarama,a kafka golang sdk

Steps to Reproduce

Expected Result

Actual Result

Collector version

v0.98.0

Environment information

Environment

go version go1.20.13 darwin/amd64

OpenTelemetry Collector configuration

extensions:
  zpages:
  pprof:

receivers:
  kafka:
    brokers: xxx
    topic: zzz
    encoding:         json
    client_id:        khaos-collector
    group_id:         khaos-collector
    protocol_version: 1.1.1
    initial_offset:   latest
    metadata:
      retry:
        max:     10
        backoff: 3s
    autocommit:
      enable:   true
      interval: 10

processors:
  memory_limiter:
    check_interval: 10s
    limit_percentage: 80
    spike_limit_percentage: 20

exporters:
  nop:

service:
  telemetry:
    logs:
      level: INFO
      encoding: console
      sampling:
        enabled: false
    metrics:
      address: localhost:8888
  extensions: [zpages,pprof]
  pipeline:
    logs:
      receivers: [ kafka ]
      processors: [ memory_limiter ]
      exporters: [ nop ]

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

It looks like the answer from AWS is that this is expected for v1 of the aws-sdk-go package, and there isn't any workaround other than upgrading the dependency. Source: aws/aws-sdk-go#5310

This is essentially a frequency of #33208, requesting all components upgrade to v2 of the AWS golang SDK.

If Sarama needs to upgrade its dependency as well, please check there for any open issues, or open a new issue to request it.

Sorry about this, and thanks for filing!