org-ql-select function could be faster by caching the buffers variable

[molok]

OS/platform

Linux

Emacs version and provenance

Emacs 29.4 compiled by myself

Org version and provenance

9.7 shipped with emacs

org-ql package version and provenance

v0.8.5 (shipped with doom-emacs)

Description

I've got a usecase where I call org-ql-select 500 times and I've noticed that the caching could be improved since the buffers variable is kind-of expensive to evaluate (unless buffers-or-files is already a list of buffers).

org-ql/org-ql.el

Lines 377 to 393 in b7d4856

	(-let* ((buffers (->> (cl-typecase buffers-or-files
	(null (list (current-buffer)))
	(function (funcall buffers-or-files))
	(list buffers-or-files)
	(otherwise (list buffers-or-files)))
	(--map (cl-etypecase it
	;; NOTE: This etypecase is essential to opening links safely,
	;; as it rejects, e.g. lambdas in the buffers-files argument.
	(buffer it)
	(string (or (find-buffer-visiting it)
	(when (file-readable-p it)
	;; It feels unintuitive that `find-file-noselect' returns
	;; a buffer if the filename doesn't exist.
	(find-file-noselect it))
	(display-warning 'org-ql-select (format "Can't open file: %s" it) :error)))))
	;; Ignore special/hidden buffers.
	(--remove (string-prefix-p " " (buffer-name it)))))

I suspect many uses of org-ql-select pass a string or (org-agenda-files) as the argument of buffers-or-files so caching in the org-ql-select might make sense, but I'm not sure if the obvious implementation will handle all the edge-cases.

On my machine 500 function calls to org-ql-select passing the output of (org-agenda-files) take 5.8s, when passing a list of buffers it takes 1.9s so the difference is quite significant.

Again, I could adapt my code to simply pass a list of buffers, but it seems to be a low-hanging performance improvement that might be beneficial for all callers.

Etc.

No response

[alphapapa]

I doubt that caching that would be appropriate, because caching means adding a new "can of worms" to the whole procedure: how would we know when to invalidate the cache? What would the cache even bypass? The validations you cited are there for good reason.

This seems like an obvious case where, if you are calling org-ql-select 500 times in rapid succession, you should keep the list of buffers in the calling code.

Are you sure that the bottleneck is in org-ql-select and not in org-agenda-files?

Also, please explain more about your use case; it seems unusual, and I wonder if there might be other ways to improve performance.