Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop from/to wrappers for format adapters with ISO 19139 #101

Closed
eblondel opened this issue Apr 24, 2018 · 1 comment
Closed

Develop from/to wrappers for format adapters with ISO 19139 #101

eblondel opened this issue Apr 24, 2018 · 1 comment
Assignees
Labels
new feature RConsortium-2019-M3 Skeleton to plug adapters between geometa and other metadata RnD
Milestone

Comments

@eblondel
Copy link
Owner

eblondel commented Apr 24, 2018

The purpose is to develop 2 wrapper functions that will wrap custom functions dedicated to map ISO 19139 from/to other metadata language formats (with candidate adapters #82 #83).

A generic metadata mapper will be drafted to plug adapters between geometa ISOMetadata classes and
other metadata classes (in particular EML and NetCDF-CF). If enough generic this pivot metadata mapper may be externalised in a separate project and used by geometa.

This mapper will be developed first in a separate branch. The mapping tests done with EML package will be handled with the most recent version of EML (from Github) where handling of EML metadata elements has been switched from S4 to list objects. In term of package lifecycle and release to CRAN, this work will be merged to the master branch only when EML package will be up-to-date in CRAN.

@eblondel eblondel self-assigned this Apr 24, 2018
@eblondel eblondel added this to Need sponsors in R Consortium 2019 Sep 14, 2018
@eblondel eblondel moved this from Need sponsors to Unsupported in R Consortium 2019 Sep 14, 2018
@eblondel eblondel moved this from Unsupported to To do in R Consortium 2019 Sep 14, 2018
@eblondel eblondel added RnD and removed new feature labels Jan 24, 2019
@eblondel eblondel added the RConsortium-2019-M3 Skeleton to plug adapters between geometa and other metadata label Apr 24, 2019
eblondel added a commit that referenced this issue Apr 29, 2019
eblondel added a commit that referenced this issue Apr 29, 2019
eblondel added a commit that referenced this issue Apr 29, 2019
eblondel added a commit that referenced this issue Apr 29, 2019
eblondel added a commit that referenced this issue Apr 29, 2019
#101 #82 #83 geometa metadata generic mapper + 1st impl for ncdf4 (from), eml (from/to)
This was referenced Apr 30, 2019
@eblondel
Copy link
Owner Author

eblondel commented Apr 30, 2019

The generic metadata converter implemented consists in set of classes and functions available here.

The design includes the definition of the metadata formats subject to the conversion. A format is defined with the class pivot_format (R6 class). The properties describing a pivot_format are made of a unique id the name of package(s) used (for information only), followed by string expressions (using sprintf formatting):

  • a reader to read metadata properties from the source object
  • a checker for checking existence of a given property in the source metadata object and,
  • a constructor, for creating the target metadata object for the given property. In case the constructor is NULL, then no conversion to this metadata format will be possible (case of ncdf4 in Adapter for NetCDF-CF metadata #82).

Formats for geometa, ncdf4 and EML/emld are then registered by default in geometa. Please note that are not Import packages, as such strong dependency with packages like EML/emld/ncdf4 is not expected in geometa since it is only used for this mapping/conversion feature). Registration of default formats can be done with setMappingFormats() in R user session, otherwise this function will be called when attempting a first conversion. If registered, the mapping formats can be listed with getMappingFormats(). This function will list the formats and indicate if reading/writing is enabled (in case of ncdf4, the column "to" will be FALSE, because there is no writing capacity). A function is already available to register a new mapping format with function registerMappingFormat.

The rules required to convert from one object to another are listed (property by property) into a tabular data file. By default the mapping file used is the one embedded in geometa at: https://github.com/eblondel/geometa/blob/master/inst/extdata/mappings/geometa_mapping.csv An interactive file is available as read-only (and used for developments) at: https://docs.google.com/spreadsheets/d/1N-Xpp8-YjVzQ4CORfdas_9CaRV4hdrRKPbO4xXNs8-Q/edit#gid=0 Each column of this mapping file corresponds to a format. Each row of the mapping file will be modelled as mapping with class pivot_format_mapping designed to map the property from a source object, to a target object. The content of a cell is a string representation of the way to access the property and is modelled with a class named pivot_format_rule. A mapping is then modelled with 2 formats (source,target) to which we associate 2 rules (source,target).

The methodology behind writing a pivot format rule is out of scope of this ticket, but will be documented in exhaustive way in the future to foster collaborative work around the mappings made available in geometa (starting with EML/emld and ncdf4). To summarize, the string representation allows to perform the following tasks:

  • accessing (read) a property from a metadata object, nested or not, single or list items.
  • filtering a property based on another property (e.g. filter a contact which role is "owner")
  • splitting string properties (with "sep" parameter), array item selection (with "index" parameter")
  • perform formatting of target values
    In a summary, the methodology put in place pretends to provide a generic syntax for maping R objects: a source object (in format 1) to an target object (in format 2), in analogy to syntaxes like Xpath/XSLT for XML objects. In that way the description of mapping rules can be done in a data table, instead of hardcoding rules in a R script.

A set of unit functions are in charge of reading source object (get_pivot_source_object), building the target object (create_pivot_target_object, feed_pivot_target_data) in order to apply a mapping (apply_format_mapping). The function convert_metadata constitutes the generic converter, that is then used in different setters for the default formats considered, for use with the generic as function. See #82 and #83 for use cases.

@eblondel eblondel removed this from To do in R Consortium 2019 Apr 30, 2019
@eblondel eblondel added this to the 0.5 milestone Apr 30, 2019
@eblondel eblondel added this to In progress in R Consortium 2019 May 3, 2019
@eblondel eblondel moved this from In progress to Done in R Consortium 2019 May 3, 2019
@eblondel eblondel closed this as completed May 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature RConsortium-2019-M3 Skeleton to plug adapters between geometa and other metadata RnD
Projects
No open projects
Development

No branches or pull requests

1 participant