Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Override doc_values parameter in Spatial XPack Module #53263

Closed
nknize opened this issue Mar 8, 2020 · 5 comments
Closed

Override doc_values parameter in Spatial XPack Module #53263

nknize opened this issue Mar 8, 2020 · 5 comments
Assignees
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@nknize
Copy link
Contributor

nknize commented Mar 8, 2020

In support of the upcoming geo_shape doc values and aggregation feature (licensed Basic). The doc_values field mapper parameter needs to be available in both GeoShapeFieldMapper and ShapeFieldMapper when the spatial xpack module is loaded, and the basic license is available. If the spatial xpack module is not available, existing OSS behavior should remain the same (e.g., MapperParsingException when doc_values parameter is explicitly provided by the user).

This issue will capture discussion and track progress on the design and implementation for overriding doc_values field mapper parameter through the Spatial x-pack module.

The Spatial Doc Value Parameter Feature Branch is available for review and discussion and provides the following behavior:

  • if the Spatial xpack module is available the doc_values parameter defaults to true for GeoShapeFieldMapper and ShapeFieldMapper
  • if the spatial xpack module is available the doc_values parameter throws an Exception for new geo_shape fields defined using the legacy PrefixTree indexing approach (same behavior as before)
  • if the Spatial xpack module is not available (e.g., OSS) the doc_values parameter defaults to false for GeoShapeFieldMapper and ShapeFieldMapper (same behavior as before)
  • if the spatial xpack module is not available (e.g., OSS) explicitly setting the doc_values parameter throws a MapperParsingException (same behavior as before)
  • backwards compatibility is supported

The behavior is achieved using the following "hack":

  • a new GeoShapeFieldMapper.Extension is loaded through SPI that overrides the GeometryIndexer that adds the indexDocValues logic
  • a new AbstractGeometryFieldMapper.ParserExtension is loaded through SPI that supplements TypeParser.parse logic to parse the doc_values parameter in the spatial xpack module
  • IndicesModule.java is "hacked" to load the ParserExtension and DataLoader objects from xpack that provides the overriding behavior described above
@nknize nknize added :Analytics/Geo Indexing, search aggregations of geo points and shapes team-discuss labels Mar 8, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (:Analytics/Geo)

@jpountz
Copy link
Contributor

jpountz commented Mar 9, 2020

@nknize Can you open a pull request for this branch to make commenting easier?

@nknize
Copy link
Contributor Author

nknize commented Mar 9, 2020

#53286 PR opened

@rjernst rjernst added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label May 4, 2020
@talevy
Copy link
Contributor

talevy commented May 8, 2020

hey @nknize, can this be closed now that we've resolved the issue by way of modules?

@nknize
Copy link
Contributor Author

nknize commented May 11, 2020

resolved by #53562

@nknize nknize closed this as completed May 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

No branches or pull requests

5 participants