Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch performance #16

Merged
merged 29 commits into from
Feb 20, 2018
Merged
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
8eabf8a
timing notes
cboettig Feb 16, 2018
21e50ae
Much faster, cleaner parsing of SPARQL returns
cboettig Feb 17, 2018
e0e90b1
testing
cboettig Feb 17, 2018
3b4c5f9
tweaking
cboettig Feb 17, 2018
c11eb84
Successful & fast rdf-join :rocket: :sparkles:
cboettig Feb 17, 2018
86376d8
move ex notebook to notebook/
cboettig Feb 17, 2018
702dac3
datalake
cboettig Feb 17, 2018
47875f4
data lake showing gh api example
cboettig Feb 17, 2018
b2d07ad
clean up tmp
cboettig Feb 17, 2018
9fed41c
run results using full lake
cboettig Feb 17, 2018
8d0a4b8
be better about cleaning up temp files
cboettig Feb 17, 2018
0d5c02b
add libs, run full data ex
cboettig Feb 18, 2018
06170d4
make 'data-lake.Rmd' into vignette
cboettig Feb 19, 2018
3bf5216
data lake example
cboettig Feb 19, 2018
26f1beb
suggest nycflights13 data
cboettig Feb 19, 2018
a49fb6a
rdf_add can handle NA as a blank node
cboettig Feb 20, 2018
e8f2927
c() method use turtle to save disk space
cboettig Feb 20, 2018
887294a
parser and serializer will guess format
cboettig Feb 20, 2018
39074cf
cleaning up as_rdf methods
cboettig Feb 20, 2018
481a329
datatype should not be assigned to blank nodes
cboettig Feb 20, 2018
5574938
use rdflib_base_uri throughout
cboettig Feb 20, 2018
3c3aceb
avoid c() by passing rdf arg
cboettig Feb 20, 2018
38598ea
option to reconnect to an existing database
cboettig Feb 20, 2018
7a02e04
indicate storage type in rdf() constructor instead
cboettig Feb 20, 2018
8d55f78
tests
cboettig Feb 20, 2018
7fe8fb6
good practice
cboettig Feb 20, 2018
4332540
newline
cboettig Feb 20, 2018
9efce89
update pkgdown
cboettig Feb 20, 2018
ec81511
skip has_bdb on appveyor
cboettig Feb 20, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
avoid c() by passing rdf arg
  • Loading branch information
cboettig committed Feb 20, 2018
commit 3c3acebc16e44e39de7ec8f2f526603dcd25d2fe
10 changes: 2 additions & 8 deletions vignettes/data-lake.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -167,20 +167,14 @@ gh_tibble %>% arrange(name) %>% head()

## RDF on non-tabular data

The RDF approach merely treats JSON as JSON-LD within a given vocabulary. In this context, nesting implicitly provides important semantic information about relationships between the data, which are captured in the RDF triples. Here, we import the JSON data as RDF:
The RDF approach merely treats JSON as JSON-LD within a given vocabulary. In this context, nesting implicitly provides important semantic information about relationships between the data, which are captured in the RDF triples. Here, we import the JSON data as RDF (and add it to our existing triplestore just for fun)

```{r gh_add}
gh_rdf <- as_rdf.list(gh_data, vocab = "gh:")
gh_rdf <- as_rdf.list(gh_data, rdf = rdf, vocab = "gh:")
```


We can add it to the existing data lake just for kicks:

```{r}
system.time(
rdf <- c(rdf, gh_rdf)
)
```


And we can query it back out of the lake just by selecting the columns of interest.
Expand Down