better get metadata

ropensci · Feb 26, 2020 · 7b28578 · 7b28578
1 parent 1e822c9
commit 7b28578
Show file tree

Hide file tree

Showing 7 changed files with 133 additions and 92 deletions.
diff --git a/DESCRIPTION b/DESCRIPTION
@@ -38,7 +38,8 @@ Suggests:
     testthat,
     purrr,
     tidyr,
-    magrittr
+    magrittr,
+    ggplot2
 VignetteBuilder: knitr
 Encoding: UTF-8
 Roxygen: list(markdown = TRUE)
diff --git a/NEWS.md b/NEWS.md
@@ -11,6 +11,9 @@
 
 * Now using the `snakecase` package to implement name cleaning, 
   providing a broader range of naming styles.
+
+* `nomis_get_metadata()` now makes existence of time concept explicit in the 
+  tibble returned by `nomis_get_metadata({id})`.
 
 
 # nomisr 0.4.1

diff --git a/R/metadata.R b/R/metadata.R
@@ -45,7 +45,7 @@
 #' \donttest{
 #' a <- nomis_get_metadata("NM_1_1")
 #'
-#' tibble::glimpse(a)
+#' print(a)
 #'
 #' b <- nomis_get_metadata("NM_1_1", "geography")
 #'
@@ -89,9 +89,16 @@ nomis_get_metadata <- function(id, concept = NULL, type = NULL, search = NULL,
   if (is.null(concept)) {
     no_code_q <- nomis_data_info(id)
 
-    df <- tibble::as_tibble(
+    df1 <- tibble::as_tibble(
       as.data.frame(no_code_q$components.dimension)
     )
+
+    names(no_code_q) <- gsub("components.timedimension.", "",
+                             names(no_code_q), fixed = TRUE)
+
+    no_code_q <- no_code_q[c("codelist", "conceptref")]
+
+    df <- bind_rows(df1, no_code_q)
 
     df$isfrequencydimension[is.na(df$isfrequencydimension)] <- "false"
   } else {

diff --git a/README.Rmd b/README.Rmd
@@ -51,7 +51,7 @@ devtools::install_github("ropensci/nomisr")
 
 `nomisr` contains functions to search for datasets, identify the query options for different datasets and retrieve data from queries, all done with [`tibbles`](https://tibble.tidyverse.org/), to take advantage of how `tibble` manages list-columns. The use of metadata queries, rather than simply downloading all available data, is useful to avoid overwhelming the rate limits of the API. For full details on all available functions and demonstrations of their use, please see the package [vignette](https://docs.evanodell.com/nomisr/articles/introduction.html).
 
-The example below gets the latest data on Jobseeker's Allowance with rates and proportions, on a national level, with all male claimants and workforce.
+The example below demostrates a workflow to retrieve the latest data on Jobseeker's Allowance with rates and proportions, on a national level, with all male claimants and workforce.
 
 ```{r example}
  library(nomisr)
@@ -86,11 +86,11 @@ Bug reports, suggestions, and code contributions are all welcome. Please see [CO
 
 Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). By participating in this project you agree to abide by its terms.
 
-Please note that this project is not affiliated with the Office for National Statistics or the University of Durham.
+Please note that this project is not affiliated with the Office for National Statistics or the University of Durham (who run Nomis on behalf o the Office for National Statistics).
 
-Get citation information for `nomisr` in R with `citation(package = 'nomisr')`
+Please use the reference below when citing `nomisr`, which is the same as `citation(package = 'nomisr')`.
 
-Odell, (2018). nomisr: Access 'Nomis' UK Labour Market Data. Journal of Open Source Software, 3(27), 859, https://doi.org/10.21105/joss.00859.
+Odell, (2018). nomisr: Access 'Nomis' UK Labour Market Data. _Journal of Open Source Software_, 3(27), 859, https://doi.org/10.21105/joss.00859.
 
 A BibTeX entry for LaTeX users is
 ```

diff --git a/README.md b/README.md
@@ -65,9 +65,9 @@ available functions and demonstrations of their use, please see the
 package
 [vignette](https://docs.evanodell.com/nomisr/articles/introduction.html).
 
-The example below gets the latest data on Jobseeker’s Allowance with
-rates and proportions, on a national level, with all male claimants and
-workforce.
+The example below demostrates a workflow to retrieve the latest data on
+Jobseeker’s Allowance with rates and proportions, on a national level,
+with all male claimants and workforce.
 
 ``` r
  library(nomisr)
@@ -76,20 +76,20 @@ workforce.
  tibble::glimpse(jobseekers_search)
 #> Observations: 17
 #> Variables: 14
-#> $ agencyid                             <chr> "NOMIS", "NOMIS", "NOMIS", "NOMI…
-#> $ id                                   <chr> "NM_1_1", "NM_4_1", "NM_8_1", "N…
-#> $ uri                                  <chr> "Nm-1d1", "Nm-4d1", "Nm-8d1", "N…
-#> $ version                              <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
-#> $ annotations.annotation               <list> [<data.frame[10 x 2]>, <data.fr…
-#> $ components.attribute                 <list> [<data.frame[7 x 4]>, <data.fra…
-#> $ components.dimension                 <list> [<data.frame[5 x 3]>, <data.fra…
-#> $ components.primarymeasure.conceptref <chr> "OBS_VALUE", "OBS_VALUE", "OBS_V…
-#> $ components.timedimension.codelist    <chr> "CL_1_1_TIME", "CL_4_1_TIME", "C…
-#> $ components.timedimension.conceptref  <chr> "TIME", "TIME", "TIME", "TIME", …
-#> $ description.value                    <chr> "Records the number of people cl…
-#> $ description.lang                     <chr> "en", "en", NA, "en", "en", "en"…
-#> $ name.value                           <chr> "Jobseeker's Allowance with rate…
-#> $ name.lang                            <chr> "en", "en", "en", "en", "en", "e…
+#> $ agencyid                             <chr> "NOMIS", "NOMIS", "NOMIS", "NO...
+#> $ id                                   <chr> "NM_1_1", "NM_4_1", "NM_8_1", ...
+#> $ uri                                  <chr> "Nm-1d1", "Nm-4d1", "Nm-8d1", ...
+#> $ version                              <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
+#> $ annotations.annotation               <list> [<data.frame[10 x 2]>, <data....
+#> $ components.attribute                 <list> [<data.frame[7 x 4]>, <data.f...
+#> $ components.dimension                 <list> [<data.frame[5 x 3]>, <data.f...
+#> $ components.primarymeasure.conceptref <chr> "OBS_VALUE", "OBS_VALUE", "OBS...
+#> $ components.timedimension.codelist    <chr> "CL_1_1_TIME", "CL_4_1_TIME", ...
+#> $ components.timedimension.conceptref  <chr> "TIME", "TIME", "TIME", "TIME"...
+#> $ description.value                    <chr> "Records the number of people ...
+#> $ description.lang                     <chr> "en", "en", NA, "en", "en", "e...
+#> $ name.value                           <chr> "Jobseeker's Allowance with ra...
+#> $ name.lang                            <chr> "en", "en", "en", "en", "en", ...
 
  jobseekers_measures <- nomis_get_metadata("NM_1_1", "measures")
 
@@ -106,11 +106,11 @@ workforce.
 #> # A tibble: 6 x 3
 #>   id      label.en                          description.en                      
 #>   <chr>   <chr>                             <chr>                               
-#> 1 TYPE490 government office regions tec / … government office regions tec / lec…
-#> 2 TYPE491 government office regions (forme… government office regions (former i…
+#> 1 TYPE490 government office regions tec / ~ government office regions tec / lec~
+#> 2 TYPE491 government office regions (forme~ government office regions (former i~
 #> 3 TYPE492 standard statistical regions      standard statistical regions        
-#> 4 TYPE496 pre-1996 local authority distric… pre-1996 local authority districts  
-#> 5 TYPE498 pre-1996 counties / scottish reg… pre-1996 counties / scottish regions
+#> 4 TYPE496 pre-1996 local authority distric~ pre-1996 local authority districts  
+#> 5 TYPE498 pre-1996 counties / scottish reg~ pre-1996 counties / scottish regions
 #> 6 TYPE499 countries                         countries
 
  jobseekers_sex <- nomis_get_metadata("NM_1_1", "sex", "TYPE")
@@ -151,40 +151,40 @@ workforce.
  tibble::glimpse(z)
 #> Observations: 70
 #> Variables: 34
-#> $ DATE                <chr> "2019-12", "2019-12", "2019-12", "2019-12", "2019…
-#> $ DATE_NAME           <chr> "December 2019", "December 2019", "December 2019"…
-#> $ DATE_CODE           <chr> "2019-12", "2019-12", "2019-12", "2019-12", "2019…
-#> $ DATE_TYPE           <chr> "date", "date", "date", "date", "date", "date", "…
-#> $ DATE_TYPECODE       <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
-#> $ DATE_SORTORDER      <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
-#> $ GEOGRAPHY           <dbl> 2092957697, 2092957697, 2092957697, 2092957697, 2…
-#> $ GEOGRAPHY_NAME      <chr> "United Kingdom", "United Kingdom", "United Kingd…
-#> $ GEOGRAPHY_CODE      <chr> "K02000001", "K02000001", "K02000001", "K02000001…
-#> $ GEOGRAPHY_TYPE      <chr> "countries", "countries", "countries", "countries…
-#> $ GEOGRAPHY_TYPECODE  <dbl> 499, 499, 499, 499, 499, 499, 499, 499, 499, 499,…
-#> $ GEOGRAPHY_SORTORDER <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1…
-#> $ SEX                 <dbl> 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5…
-#> $ SEX_NAME            <chr> "Male", "Male", "Male", "Male", "Male", "Male", "…
-#> $ SEX_CODE            <dbl> 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5…
-#> $ SEX_TYPE            <chr> "sex", "sex", "sex", "sex", "sex", "sex", "sex", …
-#> $ SEX_TYPECODE        <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
-#> $ SEX_SORTORDER       <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
-#> $ ITEM                <dbl> 1, 1, 2, 2, 3, 3, 4, 4, 9, 9, 1, 1, 2, 2, 3, 3, 4…
-#> $ ITEM_NAME           <chr> "Total claimants", "Total claimants", "Students o…
-#> $ ITEM_CODE           <dbl> 1, 1, 2, 2, 3, 3, 4, 4, 9, 9, 1, 1, 2, 2, 3, 3, 4…
-#> $ ITEM_TYPE           <chr> "item", "item", "item", "item", "item", "item", "…
-#> $ ITEM_TYPECODE       <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
-#> $ ITEM_SORTORDER      <dbl> 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 0, 0, 1, 1, 2, 2, 3…
-#> $ MEASURES            <dbl> 20100, 20201, 20100, 20201, 20100, 20201, 20100, …
-#> $ MEASURES_NAME       <chr> "Persons claiming JSA", "Workplace-based estimate…
-#> $ OBS_VALUE           <dbl> 106180.0, 0.6, NA, NA, NA, NA, NA, NA, NA, NA, 98…
-#> $ OBS_STATUS          <chr> "A", "A", "Q", "Q", "Q", "Q", "Q", "Q", "Q", "Q",…
-#> $ OBS_STATUS_NAME     <chr> "Normal Value", "Normal Value", "These figures ar…
-#> $ OBS_CONF            <lgl> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, …
-#> $ OBS_CONF_NAME       <chr> "Free (free for publication)", "Free (free for pu…
-#> $ URN                 <chr> "Nm-1d1d32316e0d2092957697d5d1d20100", "Nm-1d1d32…
-#> $ RECORD_OFFSET       <dbl> 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,…
-#> $ RECORD_COUNT        <dbl> 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 7…
+#> $ DATE                <chr> "2020-01", "2020-01", "2020-01", "2020-01", "20...
+#> $ DATE_NAME           <chr> "January 2020", "January 2020", "January 2020",...
+#> $ DATE_CODE           <chr> "2020-01", "2020-01", "2020-01", "2020-01", "20...
+#> $ DATE_TYPE           <chr> "date", "date", "date", "date", "date", "date",...
+#> $ DATE_TYPECODE       <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
+#> $ DATE_SORTORDER      <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
+#> $ GEOGRAPHY           <dbl> 2092957697, 2092957697, 2092957697, 2092957697,...
+#> $ GEOGRAPHY_NAME      <chr> "United Kingdom", "United Kingdom", "United Kin...
+#> $ GEOGRAPHY_CODE      <chr> "K02000001", "K02000001", "K02000001", "K020000...
+#> $ GEOGRAPHY_TYPE      <chr> "countries", "countries", "countries", "countri...
+#> $ GEOGRAPHY_TYPECODE  <dbl> 499, 499, 499, 499, 499, 499, 499, 499, 499, 49...
+#> $ GEOGRAPHY_SORTORDER <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1,...
+#> $ SEX                 <dbl> 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,...
+#> $ SEX_NAME            <chr> "Male", "Male", "Male", "Male", "Male", "Male",...
+#> $ SEX_CODE            <dbl> 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,...
+#> $ SEX_TYPE            <chr> "sex", "sex", "sex", "sex", "sex", "sex", "sex"...
+#> $ SEX_TYPECODE        <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
+#> $ SEX_SORTORDER       <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
+#> $ ITEM                <dbl> 1, 1, 2, 2, 3, 3, 4, 4, 9, 9, 1, 1, 2, 2, 3, 3,...
+#> $ ITEM_NAME           <chr> "Total claimants", "Total claimants", "Students...
+#> $ ITEM_CODE           <dbl> 1, 1, 2, 2, 3, 3, 4, 4, 9, 9, 1, 1, 2, 2, 3, 3,...
+#> $ ITEM_TYPE           <chr> "item", "item", "item", "item", "item", "item",...
+#> $ ITEM_TYPECODE       <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
+#> $ ITEM_SORTORDER      <dbl> 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 0, 0, 1, 1, 2, 2,...
+#> $ MEASURES            <dbl> 20100, 20201, 20100, 20201, 20100, 20201, 20100...
+#> $ MEASURES_NAME       <chr> "Persons claiming JSA", "Workplace-based estima...
+#> $ OBS_VALUE           <dbl> 105592.0, 0.6, NA, NA, NA, NA, NA, NA, NA, NA, ...
+#> $ OBS_STATUS          <chr> "A", "A", "Q", "Q", "Q", "Q", "Q", "Q", "Q", "Q...
+#> $ OBS_STATUS_NAME     <chr> "Normal Value", "Normal Value", "These figures ...
+#> $ OBS_CONF            <lgl> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE...
+#> $ OBS_CONF_NAME       <chr> "Free (free for publication)", "Free (free for ...
+#> $ URN                 <chr> "Nm-1d1d32321e0d2092957697d5d1d20100", "Nm-1d1d...
+#> $ RECORD_OFFSET       <dbl> 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 1...
+#> $ RECORD_COUNT        <dbl> 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, 70,...
 ```
 
 There is a lot of data available through Nomis, and there are some
@@ -204,13 +204,15 @@ Conduct](CONDUCT.md). By participating in this project you agree to
 abide by its terms.
 
 Please note that this project is not affiliated with the Office for
-National Statistics or the University of Durham.
+National Statistics or the University of Durham (who run Nomis on behalf
+o the Office for National Statistics).
 
-Get citation information for `nomisr` in R with `citation(package =
-'nomisr')`
+Please use the reference below when citing `nomisr`, which is the same
+as `citation(package = 'nomisr')`.
 
-Odell, (2018). nomisr: Access ‘Nomis’ UK Labour Market Data. Journal of
-Open Source Software, 3(27), 859, <https://doi.org/10.21105/joss.00859>.
+Odell, (2018). nomisr: Access ‘Nomis’ UK Labour Market Data. *Journal of
+Open Source Software*, 3(27), 859,
+<https://doi.org/10.21105/joss.00859>.
 
 A BibTeX entry for LaTeX users is
 

diff --git a/man/nomis_get_metadata.Rd b/man/nomis_get_metadata.Rd