New component: count connector

[djaglowski]

The purpose and use-cases of the new component

The count connector emits metrics representing the number of signals observed by the connector. For example, the number of spans, data points, or log records.

Initial proposal

A minimal implementation that counts spans, data points, and log records. Counts should be specific to each resource. There should be a reasonable default name for each count metric, but the names should be customizable via the configuration.

Future enhancements

The connector could be enhanced to emit counts on an interval, such as every 60s. Counts would be aggregated during the interval and then cleared upon being emitted.

It should be possible enhance the connector by providing conditional counting capabilities. Likely, this would be based on ottl, but the general idea is that the configuration would allow the user to specify one or more conditions that must be met in order for the signal to be counted. For example:

• Count logs that are error level or higher.
• Count spans that contain a specific attribute.
• Count metric data points with a value greater than 10.

Example configuration for the component

receivers:
  otlp:
exporters:
  otlp:
connectors:
  count:
    logs:
      metric_name: log.record.count

service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [count]
    metrics:
      receivers: [count]
      exporters: [otlp]

Telemetry data types supported

Exporter Pipeline Type	Receiver Pipeline Type	Description
traces	metrics	Counts the number of spans.
logs	metrics	Counts the number of log records.
metrics	metrics	Counts the number of data points.

Is this a vendor-specific component?

• This is a vendor-specific component
• If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

No response

Additional context

Connectors are a new type of component that consume signals from one or more pipelines and emit signals onto one or more other pipelines. For more details on the overall design see open-telemetry/opentelemetry-collector#2336.

[atoulme]

Are connectors now available upstream? Is there a blog post or announcement that would help understand the context of this new work? This looks great :)

[djaglowski]

It is too soon to announce connectors as ready for use, as the collector does not yet contain the logic that will integrate connectors into pipelines. However, the connectors package is fully implemented upstream and I believe this is enough to begin work on individual implementations. This assumes unit testing is on par with that of other components in the repo and that these would stay "in development" until the collector can run pipelines with them.

Looking ahead towards an eventual announcement though, I think it would helpful to launch with a few straightforward connectors such as this one. These can serve as a proof points for the framework and provide some very accessible value right away.

[jpkrohling]

I'll gladly be a sponsor for this one.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[fatsheep9146]

@djaglowski I think this pr can be closed by #18104