Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requirement 19 in combination with itemType as defined in Features and Common is problematic #539

Closed
pvretano opened this issue Mar 30, 2021 · 19 comments · Fixed by #669
Closed
Labels
OGC API: Common Issue related to general resources or requirements (see #190) Part 1: Core Issue related to Part 1 - Core

Comments

@pvretano
Copy link
Contributor

Consider that you are a client and you access the /collections endpoint is a multi-resource-type collection environment.

  • You get a list of collection and you start iterating over the list.
  • The first collection you encounter has itemType=feature and so you know (a) there is a /items endpoint and (b) you will get features from the endpoint.
  • The next collection you encounter has itemType=record and again you know (a) there is a /items endpoint and (b) you will get records from the endpoint.
  • The next collection you encounter does no have itemType defined. Now what?

According to features the default itemType is feature so you can assume that (a) the /items endpoint exists and (b) you will get features from the endpoint.
According to common, however, the default itemType is unknown and so you can't assume that an /items endpoint exists and thus no features are accessible.
This is a point of inconsistency.

Features should not declare that the default itemType is feature. Rather features should say that you SHALL include an itemType and its value SHALL be feature if the /items endpoint exists and you can get features from that endpoint.

Also, I would say that Recommendation 2 in common part 2 that says that if the /items endpoint is supported then you SHOULD include an itemType, should be upgraded to a requirement.

@jerstlouis
Copy link
Member

jerstlouis commented Apr 1, 2021

@pvretano Doesn't the presence of a link with an items relation type already solve that issue though?
i.e. if you have an items relation type, and Features says "feature" is the default type of item, then you can safely expect to find Features at /items.

@jampukka
Copy link
Contributor

jampukka commented Apr 1, 2021

I think most clients ignore itemType and don't handle multi-resource-type collection environment at all. Is there a benefit of grouping different types of collections into a single endpoint? I guess it could be nice to have both a metadata record and the actual features for a specific collection available from the same endpoint but even that doesn't work because the collectionIds would collide with each other.

@jerstlouis
Copy link
Member

@jampukka If I understand correctly, a collection can have either Records or Features, but it can also be accessed during other OGC APIs. See issue opengeospatial/ogcapi-common#140 for the context and resolution on allowing different OGC API access mechanism to the same collection of data.

One example is a point cloud can easily be seen as both Coverages and Features.
Another example is accessing building footprints using a 3D Bounding Volume Hierarchy / 3D Tiles, or as Features.
And the common example of accessing a collection as Maps, Tiles (Rasterized or Vector Tiles) or Features.

Example collection available using multiple API: https://maps.ecere.com/ogcapi/collections/SanDiegoCDB:Buildings

@jampukka
Copy link
Contributor

jampukka commented Apr 1, 2021

@jerstlouis that's a little different since you have generalized Features into Data layers. Say you added a collection with itemType=record to the same dataset. Would that collection also support /collections/{collectionId}/styles, /collections/{collectionId}/maps etc?

The issue seems to be if single endpoint can serve both as OGC API Features and OGC API Records service at the same time. And I'm not sure if that's something we should even aim at.

What should happen if I query /collections/{collectionId}/items?q=foo (the query param from OGC API Records) on a collection with itemType=feature? Respond with an exception? But how does the client know that q wasn't appropriate for that collectionId by looking at the OpenAPI document? The goal should be making APIs more mainstream (e.g. using OpenAPI for now) not making stuff that requires deep knowledge of both Features and Records standards.

@jerstlouis
Copy link
Member

@jampukka a single end-point supporting both OGC API Features and Records definitely make sense, at least for Feature collections with support for Records queries at the /collections level.

About some collections supporting some query parameters / resources and others not supporting them, there are plenty of use cases for that, like some collections of a single dataset being raster-based/coverages, while others vector-based/features.

In our API definition we have a dynamic inclusion that defines a different list of valid identifiers for the different end-points/paths, e.g.

      "feature-collection-list" : {
        "$ref" : "./api/feature-collections"
      },
      "coverage-collection-list" : {
        "$ref" : "./api/coverage-collections"
      },

I'm not sure if something similar could be applied with the same /items path, could there be two versions of it, one for Features and one for Records?

See also OAI/OpenAPI-Specification#2453

@jampukka
Copy link
Contributor

jampukka commented Apr 1, 2021

@jerstlouis consider following snippet from your API definition:

"/collections/{collectionId}/items" : {
  "get" : {
    "summary" : "Retrieve vector features from the specified collection.",
    "operationId" : "getFeatures",
    "tags" : [ "Vector Features" ],
    "parameters" : [ {
      "$ref" : "#/components/parameters/collectionId-features"
    }, {
      "$ref" : "#/components/parameters/bbox"
    }, {
      "$ref" : "#/components/parameters/subset"
    }, {
      "$ref" : "#/components/parameters/datetime"
    }, {
      "$ref" : "#/components/parameters/limit-items"
    }, {
      "$ref" : "#/components/parameters/resultType"
    }, {
      "$ref" : "#/components/parameters/propertyName"
    }, {
       "$ref" : "#/components/parameters/filter"
    }, {
       "$ref" : "#/components/parameters/filter-lang"
    }, {
       "$ref" : "#/components/parameters/filter-crs"
    }, {
      "$ref" : "#/components/parameters/f-vector"
    } ],
    ....

How would you edit this to support records resource /collections/{collectionId}/items (https://docs.ogc.org/DRAFTS/20-004.html#records-access) and the required additional query parameters (q, type, externalid)?
I guess you could edit collectionId-features path param to also allow a "record-collection-list" and add the additional query parameters to parameters list. But now you run into the issue I just described with q=foo.

@pvretano
Copy link
Contributor Author

pvretano commented Apr 1, 2021

@jerstlouis I don't think the rel=items links solves the problem because records also uses the rel=items link because, well, they are items in a collection. Another consideration is that we want to be aligned with STAC in Records and they use rel=items.

@pvretano
Copy link
Contributor Author

pvretano commented Apr 1, 2021

@jampukka OUR client does not ignore itemType! The only way to reliably distinguish between a feature and a record is via the itemType. Also, we bundle different resource types into a single endpoint ... see https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa. One collection serves both features and coverages.

@jampukka
Copy link
Contributor

jampukka commented Apr 1, 2021

@pvretano I do believe yours doesn't ignore them. But I do know multiple that also do ignore them :)

I'm not sure if the issue is with Features having a default value for itemType or with Common trying to add something which is somehow retroactively breaking something already specified in Features. I mean we might need to use itemType for something else in Features later on (collection of collections for example?) and shouldn't have to worry if that's possible or not.

There probably are benefits in implementing servers and clients that can handle multiple different types of OGC APIs but I'm not sure if that should validate a breaking changes or even be a focus point when advancing Features.

I guess the question is once Common is done do we inherit definitions from there or do we continue to use our own which might have some additional required fields and other differences (for example changing the default value for itemType from unknown to feature in Collection Description).

If we use the ones from Common directly we are stuck with them. On the other hand if we expand on them we have to check for collisions with every other OGC API because otherwise it's impossible to implement Features + that OGC API in a single endpoint.

@cportele
Copy link
Member

cportele commented Apr 1, 2021

@pvretano - you wrote:

Features should not declare that the default itemType is feature. Rather features should say that you SHALL include an itemType and its value SHALL be feature if the /items endpoint exists and you can get features from that endpoint.

We cannot change that, because it would break be a breaking change. The reasons for the default is that itemType was a late addition to the collections object and it seemed unjustified to break the existing implementations and deployments at the time. It was added as an extension point for records and other data resources. Therefore, the default of "feature" was necessary, just because feature collections were the first collection type that has been standardized. So, any additional types of collections (records and other future data resource types) where the items in the collections are shared via the API at items will have to require that the itemType is explicitly provided with a value specific to the resource type.

If Common sticks to specifying no fixed default for itemType then the only option for a potential Features revision that adds a dependency to Common is to add an additional constraint that any API implementing Features Core must also set the default to "feature". If then another OGC API standard adds a similar requirement for a different default value as Common allows, then no API could conform to Features and that other OGC API resource type.

@jerstlouis
Copy link
Member

@pvretano Right, but records would always set itemType right, since Features says itemType defaults to Features otherwise?

@pvretano
Copy link
Contributor Author

pvretano commented Apr 1, 2021

@cportele perhaps we can add an informative statement to features core saying that any other specification that uses the items endpoint must specify an item type (like records) so that there is no ambiguity.

@cportele
Copy link
Member

cportele commented Apr 2, 2021

@pvretano - Yes, that was my point. It would just be easier, if Common wouldn't introduce subtle changes.

@cportele cportele added OGC API: Common Issue related to general resources or requirements (see #190) Part 1: Core Issue related to Part 1 - Core labels Apr 12, 2021
@cmheazel
Copy link
Contributor

cmheazel commented May 5, 2021

Common has to support all APIs. What is the sense in forcing an API which does not serve Features to implement 'feature' as a default value? By removing a required default value for itemType, API developers can select the default value which makes the most sense for users of their data.

I agree with Peter's suggestion that Recommendation 2 should be a requirement.

@cportele
Copy link
Member

cportele commented May 6, 2021

Fine with me, but I just want to point out that if Recommendation 2 in Common Part 2 would be changed to a requirement, then Features will not be able to have a dependency to Common Part 2. Explicitly stating the itemType is not required in Features and if no itemType is present the value is assumed to be "feature".

This is a general risk, whenever Common is not reusing a standardized building block, but introduces changes that would break implementations.

@jerstlouis
Copy link
Member

jerstlouis commented May 6, 2021

@cportele That would really defeat the purpose of Common Part 2 if Features cannot depend or be compatible with it. Please suggest a way forward in Common so that it can be compatible with Features?

This is a general risk, whenever Common is not reusing a standardized building block, but introduces changes that would break implementations.

Again this points to the problem with developing something in one SWG with not enough consideration for/involvement from other SWGs. At the point where we later start considering making it Common, it has to change, and then it ends up incompatible with the original specification where it was initiated. This is why I keep trying to encourage flagging things as Common and considering developing them as a Common building block as soon as possible (opengeospatial/architecture-dwg#24).

@cportele
Copy link
Member

cportele commented May 9, 2021

Please suggest a way forward in Common so that it can be compatible with Features?

The options are all in the discussion in this thread. Use "feature" as a default or set no default, but leave it optional. The second option has some drawbacks as discussed, but would work, too.

@jampukka
Copy link
Contributor

jampukka commented May 10, 2021

Are Common schemas/definitions meant to be used directly by APIs or more as a guideline?

@cmheazel wrote:

API developers can select the default value which makes the most sense for users of their data.

To me this reads like itemType shouldn't be required or have a default in Common and Features API could extend the definition by setting the default value for itemType to be feature as they currently are doing. Or perhaps another API with multiple types of items could change itemType to be required if that's what they need?

@cportele
Copy link
Member

Meeting 2022-01-31: @cportele create a PR to add a note as mentioned in #539 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OGC API: Common Issue related to general resources or requirements (see #190) Part 1: Core Issue related to Part 1 - Core
Projects
Development

Successfully merging a pull request may close this issue.

5 participants