Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No data for "App cost by pod" panel in grafana example dashboard #97

Closed
thiDucTran opened this issue Aug 2, 2023 · 3 comments
Closed
Labels
help wanted Extra attention is needed

Comments

@thiDucTran
Copy link

thiDucTran commented Aug 2, 2023

I imported the dashboard @ https://github.com/opencost/opencost-helm-chart/blob/main/examples/dashboard/kube-prometheus-stack-opencost-dashboard.json to Grafana (kube-prometheus stack)

Seems metrics related to pods or containers are not showing

image

I tried editing some existing panel to use kube_node_labels{job="opencost"} instead of just kube_node_labels as commented in #9 (comment) but does not seem to fix anything

  • AKS: v1.25
  • quay.io/kubecost1/kubecost-cost-model:prod-1.105.1
  • kube-prometheus v0.12.0
@thiDucTran
Copy link
Author

thiDucTran commented Aug 3, 2023

Am I doing something wrong because the metrics, container_memory_allocation_bytes and container_cpu_allocation, do not have the exported_container label...they do have a container label....so if I edit to something like below...I do then see data in the panel

  (sum(sum(container_memory_allocation_bytes{exported_namespace=~"$namespace", container=~"$app.*"}) by (container,exported_instance) * on(exported_instance) group_left() (
				node_ram_hourly_cost{} / 1024 / 1024 / 1024
				+ on(node,instance_type) group_left()
					label_replace
					(
						kube_node_labels{job="opencost"}, "instance_type", "$1", "label_node_kubernetes_io_instance_type", "(.*)"
					) * 0
			)
  + 
  sum(container_cpu_allocation{exported_namespace=~"$namespace", container=~"$app.*"}) by (container,exported_instance) * on(exported_instance) group_left() (
	  			node_cpu_hourly_cost{} + on(node,instance_type) group_left()
		  			label_replace
		  			(
		  				kube_node_labels{job="opencost"}, "instance_type", "$1", "label_node_kubernetes_io_instance_type", "(.*)"
		  			) * 0
		  	))) * 24 * 30

@mattray mattray added the help wanted Extra attention is needed label Aug 28, 2023
@mattray
Copy link
Collaborator

mattray commented Aug 28, 2023

If someone with more Grafana experience wants to weigh in, I'm sure we'd appreciate a PR to fix this.

@ghost
Copy link

ghost commented Sep 5, 2023

Checking the prometheus metrics it seems to just have a wrong label applied as you suspected @thiDucTran so the Prom query is failing. Other panels in the example dashboard don't have this problem, I can submit a PR on it

ghost pushed a commit to spalvarez/opencost-helm-chart that referenced this issue Sep 5, 2023
Signed-off-by: Sean Alvarez <alvie0818@gmail.com>
toscott added a commit that referenced this issue Sep 24, 2023
* #97 Remove exported_ prefix from metric labels in failing panels

Signed-off-by: Sean Alvarez <alvie0818@gmail.com>

* increased readiness and liveness probe delay to 120 (#112)

* increased readiness and liveness probe delay to 120

* Signed-off-by: Matt Ray <github@mattray.dev>

---------

Co-authored-by: Matt Ray <github@mattray.dev>

* Add scrapeTimeout to values.yaml with 10s default

---------

Signed-off-by: Sean Alvarez <alvie0818@gmail.com>
Signed-off-by: Tony Scott <toscott@users.noreply.github.com>
Co-authored-by: Sean Alvarez <alvie0818@gmail.com>
Co-authored-by: Theis Ferré Hjortkjær <theisferre@gmail.com>
Co-authored-by: liorf <lior.franko@ironesrc.com>
Co-authored-by: Tony Scott <toscott@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants