Idea for permanent mutable links

[Stebalien]

(Was going to be a comment on #9 but got a bit off topic. Also, it might be a good idea to move this to the ipfs/specs repo).

A problem I haven't yet seen addressed is mutable links that never break. Currently, you can do one of two things:

1. Use an IPFS/IPLD link and it will never break (assuming you recursively pin the object).
2. Use a relative link and the target document can be updated, amended, corrected, etc.

From a web perspective, 2 is preferable in the short run but 1 is preferable in the long run. That is, if you, e.g., link to a news article using a permanent link, someone will yell at you for linking to an outdated/incorrect version of the article. However, if you go with 2, someone will eventually yell at you 2+ years down the road when the link fails to resolve (although by then they may not be able to contact you so you may be safe...). I've already seen IPFS users in the wild write down two links (literally "here is my ()" to avoid this problem.

So, it would be really nice to have a simple way to do both at once. That is, link to a mutable object but include a link to a snapshot.

Solution 1:

Embed IPLD links in mutable links. That is, /ipns/XXX+YYY/path/to/object where XXX is the multihash of the IPNS key and YYY is the multihash of the IPLD object.

• Pro: It's "just text". That is, if a user copies and emails or bookmarks the link, they'll get both the IPLD and IPNS link at the same time.
• Pro: It's very simple and easy to read (and easy to remove/update the IPLD hash).
• Pro: It also works with http/https: /http(s)/origin+YYY/path/to/object
• Con: The IPLD link is not canonical (short form). That means I'd need to pin all objects along the path to make it perminent.
• Con: I'm not sure this plays well with more general multiaddrs but I don't know enough about the spec to know.
• Con: It doesn't actually mention that YYY is an IPLD hash.

Solution 2:

Do it at the application level. That is, punt.

• Pro: Less work now.
• Pro: Flexible.
• Con: Less "standardized".
• Con: Not "just text" (personally, this is why I really want to do this at the link level).

Other

There have to be other solutions and are probably ways to improve on solution 1. Thoughts? If this doesn't really make any sense, say so and I'll review it when I'm actually awake.

[jbenet]

I brought this up elsewhere as a "reference". Basically, i'm thinking that a "reference" (in the human writing sense of the word) should be both a mutable link and the immutable snapshot at the time the reference was made (mutable link accessed).

So far, i was struggling to represent this, because yeah, you end up with two links, an /ipns and /ipfs link (or /ipld). Was thinking of an IPLD struct:

some-ref:
  latest: <ipns-link>
  snapshot: <ipld-link>

I really like the idea of combining both links in one. But i'm not sure this belongs in /ipns. Maybe. or maybe it should be a different thing (/ref/<ipns-hash>@<ipld-hash> or something). I do see the value in defining it as a thing for IPNS itself, particularly because i can't think of a legitimate case where i want a mutable link and DON'T want a snapshot at all.

Glad you're thinking about this. There's something important lurking here.

[nicola]

(this feels like authenticated streams, I think we have discussed this in person (actually with both of you in two different conversations), will write about this if it doesn't make sense)

Question: how do you ensure that the ipns has ever pointed to that snapshot?

[jbenet]

This is a different primitive than authenticated streams. It would build on authenticated streams.

The goal here is a concise link that gives the object containing the reference the ability to freeze a snapshot in time, or look up newer things (however that lookup happens, authenticated streams or something else)

[Stebalien]

Although, in the case of IPNS links, it would be nice for them to be authenticated in some way. The IPLD node could point to an IPNS record... But again, that doesn't really extend to other protocols and I'm not sure it's that important.

[joeyh]

Rather than keep a single snapshot, a mutable link could be required to point to a history of past versions. The object that a mutable link points to would be required to be a struct like:

{ latest: <ipld-link>
 , prev: <ipld-link> -- optional, points to another struct of this form
 }

To avoid IPNS later pointing to an object with a disjoint history, the mutable link could include the hash of the initial version of the object: /ref/<ipns-hash>@<ipld-initial-hash> The mutable link would only be resolved successfully when IPNS resolves to an object that has the initial object from the link in its history.

Since the mutable link always points to an object that includes its history, pinning it pins all the history. And regular IPLD paths can be used to access the latest or previous versions:

/ref/<ipns-hash>@<ipld-initial-hash>/latest/
/ref/<ipns-hash>@<ipld-initial-hash>/prev/
/ref/<ipns-hash>@<ipld-initial-hash>/prev/prev/prev/

Note that this does not prevent IPNS changing to point to an object with part of its history rewritten. The only version that stable access is guaranteed to is the initial version (and any prev version that the initial version may point to).

It would be possible to have something that automatically traverses an IPLD object and updates the ipld-initial-hash part of mutable links to be what the mutable link currently points to. The resulting new object would contain the history up to the point it was created. IPNS changes that rewrite history would break those updated mutable links. If mutable links were regularly updated in this way, it would perhaps incentivise against rewriting history.

[singpolyma]

Hmm, I do like the idea of just having an object that represents the link and linking to that. So link to /ipfs/Qm... that has links to the mutable and the permanent version (equivalent of HTTP "multiple choices") and then pick one

[maparent]

Beginning to look at those issues as an outsider, I am certainly missing tons of context. But fwiw, looking at #9 and this issue, I would allow links to specify both the current version and version stream thus:

{
  "someKey": {
	"/": "/ipfs/.....",
	"~": "/ipns/....."
  }
}

It seems more economical to me than storing an ipfs address to a frozen list of versions which contains the ipns which I'd again dereference to get the latest version (because the frozen list obtained with an ipfs link may be outdated.)
Regards

[Stebalien]

@maparent I agree but I'd still like to be able to represent this as a string (e.g., a link in a webpage).

@joeyh sorry for the late response. Versioning is really a separate issue, this issue is more about making links "permanent". IPFS will likely handle versioning by including a previous-version pointer in the file metadata.

[maparent]

What is the link to be to? If to the static (IPLD) snapshot version, maybe each snapshot should contain the IPNS name as part of the IPLD information.
That way a normal /ipfs/YYYY IPLD link would convey the /ipns/XXX information (albeit indirectly.)

The other way around... what @singpolyma proposes looks a bit like a timemap in the Memento protocol. It has the disadvantage of creating a shadow object series for every mutable object.
I prefer timegates to timemaps for complex reasons, but this adds significantly to the protocol. Still, for the record: it would entail IPNS servers keeping a history of values, and being able to get at specific value by adding a timestamp in the "query". (So yes it would have to be a query, and does not map well to fuse etc.)
so you'd query for /ipns/XXX?time=2018-01-12T07:45:00Z and get back a /ipfs/YYY link.

Not that any of that is in contradiction with my proposal for links internal to IPLD data.
Regards

[Stebalien]

That way a normal /ipfs/YYYY IPLD link would convey the /ipns/XXX information (albeit indirectly.)

That's an interesting idea. Kind of like rel="canonial" links in html pages. My goal here is really to point to the IPNS version and only fallback on the snapshot if necessary. They're both useful, just slightly different semantics.

[ilyaigpetrov]

I would like to collaborate on this problem. What is the current status of it?
I like the idea by @maparent of embedding ipns link into ipfs objects.
Is there any code needed to be implemented or some idea to be analyzed?

[ilyaigpetrov]

In ipfs/in-web-browsers#89 format for representing ipfs/ipns links on gateways is discussed. Following this discussion I propose:

1. https://<ipns-hash>.ipns.cf-ipfs.com/<ipfs-hash>/foo/bar format for references on gateways. If ipns is broken then ipfs is used. If ipns is not broken then client is redirected to https://<ipns-hash>.ipns.cf-ipfs.com/<latest-resolved-ipfs-hash>.
2. Outdated and broken ipns hashes fallback to ipfs but ipfs MUST contain ipns link inside it to counter attacks on broken ipns links redirecting user to malicious web pages.
3. For IPLD I feel like /ref/<ipns-hash><separator><ipfs-hash> is the right form where ipfs MUST contain ipns to counter attacks on broken inps.
4. Using only /ref/<ipfs-hash> with ipns link inside ipfs object can result in worsen speed (find peers serving ipfs-hash, download some meta with inps-link, resolve ipns-link VS. resolve ipns-link, fallback to ipfs, check ipns link inside to match ipns-link).

[maparent]

FYI, I did expand on that idea in the IPFS camp, here are my notes. I do not know yet how much I can contribute to implementation (not a go coder, for one thing) but I would be interested in helping to hammer the spec together.

[Stebalien]

@ilyaigpetrov you stated in ipfs/kubo#5982 (comment) that one can use the "base" tag to work around this the issue of the website root. While correct, it's kind of a pain and causes quite few compatibility issues.

But you're right, we can't put the IPFS hash in the subdomain in this case without changing the origin. However, your specific proposal actually introduces a bit of a security concern: given https://<ipns-hash>.ipns.cf-ipfs.com/<ipfs-hash>/foo/bar, an attacker could load https://<ipns-hash>.ipns.cf-ipfs.com/<evil-ipfs-hash>/foo/bar in a hidden iframe to steal private data.

In that light, we may want something like https://<ipns-hash>.ipns-archive.cf-ipfs.com/<signed-ipns-record>/.... Or maybe the query string? https://<ipns-hash>.ipns.cf-ipfs.com/thing/other-thing?backup=<signed-ipns-record>?

Note: For the simple case of "I've visited this website before", we can use etags and/or cookies to tell the gateway about known IPFS hashes.

[rvagg]

Closing due to staleness as per team agreement to clean up the issue tracker a bit (ipld/team-mgmt#28). This doesn't mean this issue is off the table entirely, it's just not on the current active stack but may be revisited in the near future. If you feel there is something pertinent here, please speak up, reopen, or open a new issue. [/boilerplate]