grafana-agent-logs failing to boot on k3d

[clux]

With latest loki yaml from README (run today):

NAMESPACE="monitoring" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/kubernetes/install-loki.sh)" > loki.yml

Leave the grafana-agent-logs pods unable to start. From k describe:

      Message:      failed to create containerd task: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:59: mounting "/etc/machine-id" to rootfs at "/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/8cfe261f373ea2017618ffcec8958a9a2040044974f550bb2703a7733549e953/rootfs/etc/machine-id" caused: not a directory: unknown

but my /etc/machine-id file is the expected 32 character hex.

The loki install worked on a k3d cluster last week on a cluster with k3d cluster create grafana --servers 1 --agents 1 (when it was still called promtail). k3d 4.2.0, kubernetes 1.20.1.

[rfratto]

Thanks for reporting! I ran into this when working on #406 and worked around it by bind mounting that file to the k3d container: https://github.com/grafana/agent/blob/main/example/k3d/scripts/create.bash

/etc/machine-id is typtically used for promtail's journald suppport. Since that's not active our manifests, it would be reasonable to remove that bind mount from the pod.

[rfratto]

I can confirm that I'm able to reproduce your problem with k3d 4.2.0 and that it works as expected when you create the cluster with:

k3d cluster create grafana --servers 1 --agents 1 -v /etc/machine-id:/etc/machine-id

(As long as /etc/machine-id exists on the host)

That being said, I don't think it's an acceptable permanent workaround and the manifests should still be modified.

[clux]

Thanks for the quick response. I am able to work around it with the extra k3d volume mount at cluster creation time :-)

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[laszlocph]

That being said, I don't think it's an acceptable permanent workaround and the manifests should still be modified.

This is the first issue I'm running into when testing Grafana Cloud. The issue should not be closed probably