Otelcol-dev

My lab for developing a Fluent Forward exporter in Opentelemetry-Collector.

Official documentation: Building a custom collector.

Getting started

Clone the repo and the submodule

git clone --recurse-submodules git@github.com:r0mdau/otelcol-dev.git

Starting your dev journey

Run a Fluent Bit instance that will receive messages over TCP port 24224 through the fluent-forward protocol and send the messages to stdout interface in JSON format every second, but if you want to work with shared_key, TLS and mTLS, see subcommands to start fluentd

make run-fluentbit
# for fluentd with tls
make run-fluentd
# for fluentd with mutual tls authentication (mtls)
make run-fluentd-mtls

Build & start the custom collector

make run
# with tls
make run-tls
# with mtls
make run-mtls

Generate 100 log lines

bash scripts/generate-logs.sh 100 >> testdata/access.log

Look at Fluent logs

docker logs fluentbit
# for fluentd with tls/mtls
docker logs fluentd

Customize the otel modules

A replace instruction is set to use local code in the go.mod file: replace github.com/r0mdau/fluentforwardexporter => ./fluentforwardexporter.

Add the module to the good factory map in the components.go file.

Misc

How-to generate the otelcol-dev content the first time (doc)

ocb --config scripts/builder-config.yaml

Vscode debug config .vscode/launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch Package",
            "type": "go",
            "request": "launch",
            "mode": "auto",
            "program": "${workspaceRoot}",
            "args": ["--config", "${workspaceRoot}/testdata/config.yaml"]
        }
    ]
}