Patch performance #16
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Uses getResults() method from redland package internals. This is way way faster for returning large numbers of results. This also sidesteps the need to rectangularize query results and manually coerce types; readr instead can handle that for us (as well as one can duck type from strings).
Merge branch 'master' into patch-performance # Conflicts: # inst/examples/profile_performance.Rmd
based on file extension, closes #4 serializer also sets explicit base option serializer defaults to print to character string if doc is NULL.
methods take vocab, base, and key
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A variety of tweaks to improve performance and memory handling when working with large triplestores. Tested with over 6 million triples using disk-based storage in the data-lake vignette.
Key changes:
rdf_querynow bypasses the stupidly slow iteration overgetNextResultapproach and uses an internal redland function call to access all results at once in csv format.as_rdfmethod now uses a poor-man's nquad serializer to rapidly generate rdf (instead of slowly iterating overadd_rdf.