Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add chart showing total merge time by part of index #217

Open
mikemccand opened this issue Apr 11, 2023 · 2 comments
Open

Add chart showing total merge time by part of index #217

mikemccand opened this issue Apr 11, 2023 · 2 comments

Comments

@mikemccand
Copy link
Owner

Lucene's infoStream now logs how long each part of the index (postings, doc values, points, vectors, etc.) take to merge. It should be simple to parse these logs from nightly benchmarks and chart these merge costs over time?

@mikemccand
Copy link
Owner Author

Hmm, except, we do not enable infoStream logging in the nightly benchmarks! We could enable it for the single-threaded index, since we don't otherwise count its metrics like docs/sec.

@rmuir
Copy link
Collaborator

rmuir commented Apr 11, 2023

yes this would be nice when discussing issues such as apache/lucene#12203
otherwise, I think merges are currently too opaque when discussing index performance: but we "know" certain parts are way slower than others.

honestly as an even simpler start we could simply make the infostream.txt available somehow (if it isn't too insane amount of disk space). then at least we could investigate the current breakdown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants