Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(jemalloc): Fix the stats of jemalloc #236

Merged
merged 4 commits into from
Dec 7, 2020
Merged

Conversation

ahsanbarkati
Copy link
Contributor

@ahsanbarkati ahsanbarkati commented Dec 7, 2020

The jemalloc stats are read by using the mallctl() call, this call doesn't return the latest stats unless we refresh the ctl cache. We can get the latest stats after calling an epoch mallctl. This PR fixes this issue.

Note: Calling an epoch mallctl is expensive. An epoch mallctl takes up all the locks as taken by a malloc call. We should use this judiciously.


This change is Reviewable

Copy link
Contributor

@jarifibrahim jarifibrahim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

fmt.Printf("[%d] Current Memory: %05.2f G. Increase? %v\n",
counter, float64(curMem)/float64(1<<30), increase)
var js z.MemStats
z.ReadMemStats(&js)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need this in memtest?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that this can be something which we can use to verify that the stats are as expected.

Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @ahsanbarkati, @jarifibrahim, @manishrjain, and @martinmr)


contrib/memtest/main.go, line 114 at r1 (raw file):

Previously, ahsanbarkati (Ahsan Barkati) wrote…

I was thinking that this can be something which we can use to verify that the stats are as expected.

The memstats can be added on. But, our version of how much memory should be used should still be printed.


z/calloc_jemalloc.go, line 164 at r1 (raw file):

	sz := unsafe.Sizeof(&epoch)
	C.je_mallctl(
		(C.CString)("epoch"),

Link to documentation about this?

@ahsanbarkati
Copy link
Contributor Author


contrib/memtest/main.go, line 114 at r1 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

The memstats can be added on. But, our version of how much memory should be used should still be printed.

Yep, that is still printed.

@ahsanbarkati ahsanbarkati merged commit c72a155 into master Dec 7, 2020
@ahsanbarkati ahsanbarkati deleted the ahsan/jemalloc-stats branch December 7, 2020 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants