inconsistent mental model connecting id, kid, and references inside a DID doc

[dhh1128]

Elsewhere (can't find location at the moment but would be glad if others pasted a link...) I saw a discussion about how id should not be required on keys, if those keys are expressed in JWK format, since JWKs have a kid field. The issue generated lots of discussion, and I'm not sure how it resolved.

Implicit in that discussion is the assumption that a JWK's kid and a key's id in classic DID doc specs are intended to perform the same function, and that they have compatible semantics. However, it is clear to me that we are not sufficiently aligned as a community around these assertions, as evidenced by the following:

1. A kid is a hint used for caching, much like the etag header in HTTP. You are supposed to be able to look at the kid and say, "Oh, I've already dereferenced that key. No need to dereference again." (@selfissued , please correct me if I'm mischaracterizing the tribal knowledge about kid in JW* circles...) But currently, the id on a key in a DID doc has explicitly incompatible semantics with this usage, because the value referenced by did:example:abc123#xyz is allowed to be different every time it is dereferenced!

This disconnect is not causing the heartburn that it should. People are proposing to use DID URLs that contain a kid as a pointer to a specific key value, even though the DID URL doesn't contain a timestamp or version stamp and is therefore ambiguous. Validating signatures and doing other cryptographic operations with ambiguous inputs is the opposite of what DIDs are supposed to deliver, IMO.

2. Setting aside that first concern, I am seeing DID URLs in the form did:example:abc123#xyz as references to a key with kid = xyz -- as if kid could play the same role that id plays in fragments. Yet I know of no verbiage in any version of the DID spec that would support this. AFAIK, the only way to refer to a key by kid would be something using matrix parameters, like maybe did:example:abc123;kid=xyz -- and I'm not even sure that works. @peacekeeper, what does the guru say? So are we playing games with @context that map kid into the role that id plays in more traditional JSON-LD--and if so, where is this documented, and why did we think this was a good idea?

I think something has to give. kid and id can't exist in free variation with semantics that overlap in some cases but not others.

[selfissued]

Indeed the kid for a JWK is intended to be used as @dhh1128 says - that if I've already retrieved and cached the key identified by the kid, I know that I don't have to retrieve it again, because I already have it and won't change. We should ensure that we use it in that way in the DID spec.

[awoie]

I agree with @dhh1128 that this discussion is required. Especially, now that the JWK encoding became one of our champions in the DID spec, we should normalize how to use kid in the JOSE stack (JWK/JWT/...). While the kid is typically used to indicate changes, the interpretation is unspecified. IMO, the interpretation can be normalized in the context of the DID spec. The main purpose is still to allow verifiers to determine which key they use for verification purposes.

kid definition as per RF7515 (JWS):

The "kid" (key ID) Header Parameter is a hint indicating which key
was used to secure the JWS. This parameter allows originators to
explicitly signal a change of key to recipients. The structure of
the "kid" value is unspecified. Its value MUST be a case-sensitive
string. Use of this Header Parameter is OPTIONAL.

When used with a JWK, the "kid" value is used to match a JWK "kid"
parameter value.

In this context, we should also take into consideration that the DID resolution results are sometimes cached anyways. I saw some implementations that don't resolve a DID to a DID Document each time they use a DID.

Furthermore, it would be great if we can get more clarity on how to use a kid in a JWT to discover the respective key from a DID Document, e.g., using DID URL HTTP(S) binding. The most common proposal is indeed to use a DID URL that points to the id of the specific verification method or public key.

@peacekeeper what would be a viable and deterministic way to point a DID URL to a version of the verification method or public key in a DID Document?

[dlongley]

An id is still used when a key is represented using JWK (the id is of the verification method). If a verification method is a public key that is represented by a JWK, then it may/may not also include a kid. These two properties can have different uses but there may also be overlap. Whether or not these are the same value depends on application use. One approach could be to say that if a verification method includes publicKeyJwk that the JWK's kid must match the verification method's id and that the JWK must never change.

However, I expect that there are use cases where people want to be able to change out the key whilst keeping the id the same (at least I've heard this from some community members) -- and perhaps they want to also use JWK in those cases. That would point to wanting to have a different value for id (which is used for referencing verification methods in DID URLs) and for kid which has other/additional uses and/or expectations.

[peacekeeper]

Personally I think it would be nice if the id (as used in DID documents) and the kid in JWK were aligned as much as possible.

Fragments in DID URLs are currently used to reference a specific part of the DID document with a certain id (e.g. did:ex:123#keys-1 would reference the part of the DID document that has its id set to that same value), but note that this may change depending on the outcome of the plain JSON vs. JSON-LD discussion (see #128), since the mechanics of a URI fragment are defined for the JSON-LD media type (application/ld+json) but undefined for the plain JSON media type (application/json).

The original proposal to make id optional for services and public keys is #47. The motivation for that had nothing to with JWK, but with the idea that the assignment of id values to parts of DID documents depend on how DID methods work internally, and that some very simple DID methods may not support ids at all. I would agree that such DID methods would not be useful in many situations, but still I felt this should be allowed. And I thought that this could potentially help with JWK alignment, since kid in JWK is also optional.

I agree with @dhh1128 that there is a semantic mismatch between id and kid since the former could point to different content over time. There are some good reasons why you would not want that (see w3c-ccg/did-spec#238), but in general I still feel it's a feature to have DID URLs that can point to different content over time.

If certain applications (DIDComm, OIDC, etc.) require immutable id references to public keys (and therefore be compatible with kid semantics), I think they could just choose to only use certain kinds of DID URLs that fulfill their immutability requirements, e.g.:

• Point to a specific version e.g. did:ex:123;version-id=a484659272a9#main-key or did:ex:123;version-time=1575375498212#main-key. This would only work with DID methods that support this.
• Use fingerprints as fragment e.g. did:ex:123#key-{fingerprint}. You would want to only use DID methods that enforce this, instead of letting you choose ids freely.
• Use UUIDs as fragment e.g. did:ex:123#536c95cb-6717-477b-88cd-a484659272a9. You would want to only use DID methods that enforce this, instead of letting you choose ids freely.

Would this solve the problem? Applications and DID methods that want to use JWK and kid could simply decide to use DID URLs as above that are guaranteed to be immutable and deterministic, but we would not have to change anything about how id and fragments work in the DID document in general?

[kdenhartog]

In general, I've found that this problem turns out to have a wicked amount of impact in a way that I'm still trying to wrap my head around. It seems every time that I try to use a key or a key reference this inconsistency causes a snag. Additionally, it seems any time I want to use a key I have to perform a dereference to make sure the crypto material actually is in a valid and trusted did document. So while I don't quite have any details to offer as a solution yet, I'm actively thinking about the problem and plan to watch this thread for insights.

Also, what are some of the use cases where having mutable references to keys is advantageous? Everyone I can come up with in my head leads to one of two outcomes. Either it introduces a security vulnerability or the cryptographic operation the key is used for fails. I'm sure I'm just not thinking of the use cases where this could be beneficial so can someone help me out here? It would be good to have these use cases listed in this thread (and probably should be provided in the use case document too).

[awoie]

It probably makes sense to stick with the semantics of kid as per https://tools.ietf.org/html/rfc7517. Consequently, id and kid can have different values. I also don't see real benefit of having mutable references for keys.

To make a proposal, one use case that I can think of which is directly impacted of what we are discussing here is verifying a JWT/JWS by a verification method in the DID Document. Then, in order to do discover the specific verification method, you ideally want to use a DID URL, e.g., did:ex:abcd#veri-key1 and the kid. I understood that some people don't expect that this DID URL resolves to the same verification method every time the DID URL gets dereferenced . So, including that in the kid would be inappropriate. On the other hand, we cannot assume that the DID URL will always resolve to a JWK. So, including the DID URL in the jku would be inappropriate as well. If applications want to implement this type of key discovery, e.g., the DID Authn Profile for SIOP, then it probably makes most sense to register a new JOSE header name which is similar to jku, e.g., did_verification_method_uri. In this case, the JWT/JWS could include an optional kid and a did_verification_method_uri to indicate whether the specific verification method needs to be removed from the cache IF jwt.kid != cache.kid for that specific verification method (note the id of the verification method might stay the same). Having such a discovery mechanism in place is necessary in order to avoid having to run through all verification methods inside a DID Document to find the right one that works eventually.

@selfissued @kdenhartog @dhh1128 @dlongley does that make sense?

[peacekeeper]

Also, what are some of the use cases where having mutable references to keys is advantageous?

ActivityPub for example uses key URLs like https://chaos.social/users/peacekeeper#main-key.

Try this:

curl -H "Accept: application/ld+json" https://chaos.social/users/peacekeeper

If the key rotates, I believe the key URL stays the same (although I admit that I haven't actually tried it). So my connected friends would find the new key without first having to figure out what the new URL is. I understand that this enables certain timing or caching attacks which are unacceptable for some use cases, but isn't the simplicitly of being able to always get the latest key under a well-known URL also an advantage?

I understood that some people don't expect that this DID URL resolves to the same verification method every time the DID URL gets dereferenced .

Applications that want this could easily use certain kinds of DID URLs that are guaranteed to always dereference to the same content (see my previous message).

[OR13]

Related: #134

My preference as follow:

did:example:123#primary or
did:example:123#<publicKeyThumbprint> Per https://tools.ietf.org/html/rfc7638

You get the following:

{
  "@context": "https://w3id.org/did/v1",
  "id": "did:example:123",
  "publicKey": [
    {
      "id": "did:example:123#gb3fR275xH8I_9frTsPu4xPAvIvP_SGn0L24Ooi1TMA",
      "type": "Ed25519Signature2018",
      "controller": "did:example:123",
      "publicKeyJwk": {
        "crv": "Ed25519",
        "x": "2xQyOeya0hhRCD4mU0p4QUoUCzlhhLZZYKBq7edP3Uk",
        "kty": "OKP",
        "kid": "gb3fR275xH8I_9frTsPu4xPAvIvP_SGn0L24Ooi1TMA"
      }
    },
...

With valid JWKs:

{
  "keys": [
    {
      "crv": "Ed25519",
      "x": "2xQyOeya0hhRCD4mU0p4QUoUCzlhhLZZYKBq7edP3Uk",
      "d": "bvc2kpLlbZF-Zovrg-_BLQvF21QttajMqFn0qkuxKus",
      "kty": "OKP",
      "kid": "gb3fR275xH8I_9frTsPu4xPAvIvP_SGn0L24Ooi1TMA"
    },
...

I'd rather never see did:example:123#primary... ever.... but primary can be a kid....

Since we now support JWK encoding for all possible key types, it makes even more sense to provide strict guidance around publicKeyJwk.

I think its bad form to inject prefixing into the JWK format, slightly more acceptable to inject prefixed keys into JWE/JWA headers.

Example VC Proof using JWT Proof Mechanism

I think it would be better to have the header look like:

{
  "alg": "ES256K",
  "typ": "JWT",
  "kid": "HgGnHUNTnIQ7mIfSlG4VhHsDGNvpoOCOrS9gdeHE4Us",
  "did": "did:elem:eURSFEEv6J7s3TJ-jhT_ZS4uGRyCDbwc347EWlqpNgw",
  "did_kid": "did:elem:eURSFEEv6J7s3TJ-jhT_ZS4uGRyCDbwc347EWlqpNgw#HgGnHUNTnIQ7mIfSlG4VhHsDGNvpoOCOrS9gdeHE4Us"
}

where did and did_kid are optional.

[OR13]

Note that in Linked Data Proofs which use detached JWS, this is not an issue, since the verificationMethod is stored in the proof, not the JWS header (much cleaner IMO).

Here is any example detached JWS

Scroll down here for the credential associate with the example above: here

[tplooker]

@OR13 can you explain the rationale behind why we would need did and did_kid instead of simply just having the value of kid = the value of did_kid in your example?

[OR13]

if you leave kid as is, there is no change to it. You could use the kid of the form did:example:123#kid in the JWS, but then it would not match directly the kid in the publicKeyJwk... IMO thats a bit wrong since you are pointing to a publicKey not JWK with did:example:123#kid

however, kid can be whatever you want, so its not illegal to do so.

[OR13]

I'd rather did:example:123#kid be an identifier to a publicKeyObject and kid be an identifier for a JWK.

Then use some new identifier for noting that a JWS/JWE can be verified with a publicKeyJwk from a did document... In other words, either:

A.) did:example:123#kid should be everywhere (in the DID Document Public Key ID, JWS Header, and JWKS with the private key).

B.) did:example:123#kid should only be used to identify a did document public key, and kid should be used to identify a JWK, and some other identifier should be used in a JWKS + JWS Header to relate the 2 concepts.

B seems more correct to me... when verifying a JWS if you thought of did:example:123#kid as the resource needed to verify, you would be passing:

{
      "id": "did:example:123#gb3fR275xH8I_9frTsPu4xPAvIvP_SGn0L24Ooi1TMA",
      "type": "Ed25519Signature2018",
      "controller": "did:example:123",
      "publicKeyJwk": {
        "crv": "Ed25519",
        "x": "2xQyOeya0hhRCD4mU0p4QUoUCzlhhLZZYKBq7edP3Uk",
        "kty": "OKP",
        "kid": "gb3fR275xH8I_9frTsPu4xPAvIvP_SGn0L24Ooi1TMA"
      }
    }

To JWS.verify, which should fail, wheras passing:

{
        "crv": "Ed25519",
        "x": "2xQyOeya0hhRCD4mU0p4QUoUCzlhhLZZYKBq7edP3Uk",
        "kty": "OKP",
        "kid": "gb3fR275xH8I_9frTsPu4xPAvIvP_SGn0L24Ooi1TMA"
      }

would succeed... the value for kid in the JWS header, should be the identifier for the resource that would succeed.

@dlongley I know you have thought about this a lot, can you share your thoughts again?... more so on how to relate a DID to a vanilla JWS, how should the DID be expressed in the header?

[tplooker]

Ok I understand that would ensure we maintain greater consistency with how JOSE is used today. However I'd ask the following questions

1. kid to my knowledge is not defined anywhere as needing to be pointing to a JWK, or be a 1-1 mapping to the kid field that appears in the JWK. Your point is more about most JOSE libraries today only accepting a JWK as the input format for a key to a decryption or verify operation?
2. In your example above when I include a public key in the did document, is the kid not a redundant piece of information that could be safely omit as the location of the JWK is identified?
3. What about when the key in the did document is not expressed as a JWK? Do we drop kid from the JOSE header and just have did_kid?

Also I'd point out that there is a big difference in the purpose of the kid when it comes to a JWS vs a JWE.

With a JWS the kid is an identifier to a key that was used to sign the payload, the implications of this is that it somehow needs to be resolved to a public key. In OpenID, signature verification usually includes interpreting the issuer field in the payload and fetching the OPs metadata to get the JWKs url so the verifier can then find the JWK with the corresponding kid.

With a JWE the kid is an identifier to a key that should be meaningful to the correct recipient and enable them to fetch the correct key from their KMS and decrypt the message.

[OR13]

https://tools.ietf.org/html/rfc7638

The resulting hash value can be used for identifying or selecting the key represented by the JWK that is the subject of the thumbprint, for instance, by using the base64url-encoded JWK Thumbprint value as a "kid" (key ID) value.

kid is used to identify JWKs.

https://tools.ietf.org/html/rfc7517#page-8

The "kid" (key ID) parameter is used to match a specific key. This is used, for instance, to choose among a set of keys within a JWK Set during key rollover. The structure of the "kid" value is unspecified. When "kid" values are used within a JWK Set, different keys within the JWK Set SHOULD use distinct "kid" values. (One example in which different keys might use the same "kid" value is if they have different "kty" (key type) values but are considered to be equivalent alternatives by the application using them.) The "kid" value is a case-sensitive string. Use of this member is OPTIONAL. When used with JWS or JWE, the "kid" value is used to match a JWS or JWE "kid" Header Parameter value.

The structure of the "kid" value is unspecified.

You can make this value whatever you want, but doing anything other than using https://tools.ietf.org/html/rfc7638 is not a good idea IF you can avoid it IMO.

kid does not identify a did document public key, I hope we can all agree on that... (although you will see in a bit why we can't have nice things...)

kid is not redundant, because publicKeyJwk is of type JWK and may come from a JWKS which would use the kid to identify it... whereas publicKey.id identifies a linked public key... of course since the structure of kid is not specified you are free to choose whatever terrible values you want for this. Prefixing a did has a number of drawbacks... what if you want to use the same JWKS for multiple DID Documents?...

For did documents that use JWS, but don't express keys as JWK...

This has been one of the most frustrating things I have worked on in the last year.

Everyone does whatever they want. Its a mess. Its the reason we pushed so hard for being allowed to use JWK for all supported key formats.

There is no good solution here IMO. You can inject whatever values you want into your JWS/JWE kid value, or omit it completely and figure out another way... I think its out of scope for the DID WG, it was in scope for VC JWT Proof format:

https://www.w3.org/TR/vc-data-model/

kid MAY be used if there are multiple keys associated with the issuer of the JWT. The key discovery is out of the scope of this specification. For example, the kid can refer to a key in a DID document, or can be the identifier of a key inside a JWKS.

They pretty much created this problem with this language...

https://www.w3.org/TR/vc-data-model/#example-27-jwt-header-of-a-jwt-based-verifiable-credential-using-jws-as-a-proof-non-normative

{
    "alg": "RS256",
    "typ": "JWT",
    "kid": "did:example:abfe13f712120431c276e12ecab#keys-1"
}

note that because kid can be anything, they are choosing to point it to a publicKey, which might encode an RSA key as publicKeyPem or publicKeyJwk....

I would expect did:example:abfe13f712120431c276e12ecab#keys-1 to be present in the DID Document assertionMethod list, if I were using a Linked Data Proof, but its totally unclear to me how using vanilla JOSE with DID Documents, I'm supposed to convey that a key is used for assertions... I suppose there is no way to restrict keys used to create a VC JWT, using a did document, since assertionMethod is not mentioned anywhere regarding JWTs.

This is not a problem for Linked Data Proofs that use JWS, which don't even use the kid field in the header today... because they identify the specific key to be used with verificationMethod, and proofPurpose, assertionMethod...

This has lead me to generally prefer Linked Data Proofs that use Detached JWS, because they have none of these inconsistencies, they rely on canonicalization algorithms, and they support JWS and JWK... so all key types that a DID Document can use, and all signature suites which are part of standard JOSE.

[awoie]

@OR13 That might be irrelevant but not every implementation uses JWK as their underlying format, hence calculating the JWK thumbprint is not possible in all cases, e.g., in case of base58 encoding.

In DIF, we agreed on the convention that kid should be a DID URL pointing to the verification method, meaning it is equivalent to verificationMethod.

The W3C VC spec does not mandate kid. You could use kid in this way but it is up to the protocol to specify key discovery. Note, that the language is MAY and the example is NON-NORMATIVE and please further note that in contrast verificationMethod is also not defined in the W3C VC spec. It is not even defined in the LD Signatures spec as per https://w3c-dvcg.github.io/ld-signatures/ except in non-normative examples. Am I missing something?

Updated:

We could however update the W3C VC Implementation Guide to recommend how key discovery can be used in case DIDs are used, i.e., define either a convention for kid (see above) or introduce a new IANA JWT claim. Would that make sense @msporny @OR13 ?

One of the reasons why we didn't include that in the VC spec is that it can be used also without DIDs and protocols might define their own discovery or don't require dynamic discovery at all.

Updated:

Just recognized that for the above question this is the wrong GitHub and WG :) . So, let's have the discussion in the VC WG when we return to maintenance. Though, this is great input and if we can establish a convention for kid or key discovery in the implementation guide, that would be great.

[OR13]

I think the cleanest would be to introduce an IANA JWT claim for the header value called verificationMethod, then use the fully qualified public key id with it. Not sure how to relate that to proof purposes though... we could also register proofPurpose and the other property inputs to createVerifyData, at the same time... this would draw JWT VCs and Linked Data Proofs, much closer together.

You have correctly pointed out that documentation is poor, I've got a number of open issue trying to fix this, but here is the definition you are looking for: https://w3c-dvcg.github.io/ld-proofs/#verification-method

[awoie]

I don't have a strong opinion on whether use kid or something like did_verification_method. In case we register a new name, then I would prefer did_xyz as we are going to register did and did_doc with IANA as well. I don't see the need for proofPurpose. A verifier that verifies a VC would have to ensure that per convention other keys apart from assertionMethod should be rejected. Although I don't like the naming here because it is inconsistent with the naming of other proof purposes.

However, I would strongly argue for having the different proof purposes, e.g., assertionMethod, to be defined in the DID Core spec to support other proofs than LD-Proofs for DIDs. It is not sufficient to delegate the semantic of proof purposes to the LD-Proof spec only. Take a look at https://uniresolver.io/ and check how many DID methods have support for assertionMethod. Except did:key I don't see any DID Doc with that property.

[tplooker]

In regards to the original scope of the issue raised by @dhh1128, I'd like to point out, even if we make the key id portion of a DID key reference immutably linked to a specific public key, such that the same id within the same did document cannot be used for another public key in future, this does not make the key id entirely immutable. DID documents are a mutable resource where a keys membership in the did document itself can change over time, if we made key ids immutably linked to a specific public key, you can reliable cache this relationship, but if you are verifying the keys usage in relation to the DID subject (such as whether the key is authorised to make assertions), you must still resolve the DID to test this.

To be clear I'm not advocating that there is no merit to immutable key ids in did documents, I just saw worth in pointing out this nuance.

[David-Chadwick]

Sorry for coming in so late to this discussion, but I just wanted to describe the mechanism that we are currently using to pass the public key of a signed JWT to the verifier of the JWT. Basically we encode the public key using RFC 7517, and transmit it as follows:

did:key:jwk:<base64 encoded public key using RFC 7517 Appendix A.1>

for ES256 the public key contains "a" and "b" and the "alg". For RS256 this contains "n" and "e" and the "alg".

An example below for ES256 is

{
  "kid": "did:key:jwk:eyJ4IjoiWXg4MGp5dWk1bU1uZHN1UlRXSHF6ek4xb0lnS2MySjY3WU5GS3hRWGU2VT0iLCJ5IjoiSWRZd05uNHU4bHJYck1sRHpNNEluNVE4aWdWWVNlNGlCYzlTMDFQUnRwOD0iLCJhbGciOiJFUzI1NiJ9",
  "typ": "JWT",
  "alg": "ES256"
}

[OR13]

We need both kid and verificationMethod.id... for many reasons, including interoperability with existing JOSE implementations.

[msporny]

id is the identifier for the verification method, which may contain cryptographic material (like a JWK). kid is the identifier for the JWK. They should never use the same identifier. If they are the same, we have problems. I'll put in a PR to address this concern.

[msporny]

This was addressed by #221, we have consensus, closing.