Linkerd viz integration with Grafana is broken

[ricsanfre]

From linkerd-viz when trying to open any Grafana dashboard, the following message error appears in the browser:

{"message":"invalid username or password","traceID":""}

Grafana's log:

logger=context t=2023-04-15T15:41:08.277401926Z level=error msg="invalid username or password" error="invalid username or password" traceID=
logger=context userId=0 orgId=0 uname= t=2023-04-15T15:41:08.277471789Z level=info msg="Request Completed" method=GET path=/d/linkerd-namespace status=401 remote_addr=10.42.3.42 time_ms=16 duration=16.023782ms size=56 referer=

[ricsanfre]

Integration was broken when I migrated to ArgoCD. Before migration, linkerd-viz namespace was meshed with linkerd, and two Traefik middlewares were used for linkerd-viz ingress resource: HTTP Basic Auth Middleware and Linkerd custom header Middleware.
Meshing linkerd-viz and adding custom header middleware again solves the issue.

It seems that Grafana does not work properly behind a HTTP proxy when basic auth is used. Traefik is passing basic auth user and passwords that seems to cause the error in Grafana.See this post
Grafana does not try to authenticate the user by other means if basic auth headers are present and returns a 401 unauthorized error.

With the second Traefik middleware, after Basic Auth, the auth headers seems not to be passed to Grafana.

Another way to solve the issue is configuring Traefik Basic auth middleware to remove auth headers before forwarding the request to the service . See Basic Auth Midleware removeHeader option

# Basic-auth middleware
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: basic-auth
  namespace: traefik
spec:
  basicAuth:
    secret: basic-auth-secret
    removeHeader: true

With this second solution, it is not needed to mesh linkerd-viz to make the Linkerd-viz/Grafana integration working.