Document suggested IPNS Entry TTL default

[hacdias]

Initial context: ipfs/interface-go-ipfs-core#94 (comment).

Currently, the default IPNS Entry TTL is hard coded to 1 minute. 1 minute is a very short duration for TTL. For DNS, 12 and 24 hours are very common TTL values. If we have a low value from the start, there is barely any performance improvement by setting a TTL manually, since no one is going to manually set it for a larger value.

With this PR, we would like to add a suggested default of 12 hours. If the user (publisher of the website), requires more frequent updates, they can always use the --ttl flag in ipns name publish to override the default.

[hacdias]

Ping @lidel. It's been 3 weeks since I opened this and there hasn't been any comments, so I think it would be fine to merge.

[hacdias]

ping @lidel

[lidel]

@hacdias do you remember why we picked 12h? Was it chosen because it was the Reprovider.Interval in Kubo?
Is 12h implemented anywhere atm?

Since then, we've increased Reprovider.Interval to 22h and expiration of DHT records to 48h (ipfs/kubo#9389, ipfs/kubo#9326). The more special numbers we invent, the harder it will be to debug.

I wonder if we should park this until we have time to evaluate, maybe when we work on #376?

I suspect default closer to 15m or 1h will be safer.

[hacdias]

@lidel the context was DNS, since IPNS Entry and DNS have similar purposes:

For context, when you add DNS record at many registrars, the default TTL for DNS records is either 12 hours or 1 hour, and is lowered manually by the user to 1 minute only for websites which need fast updates.

• @lidel at feat: add namesys publish options interface-go-ipfs-core#94 (comment)

[lidel]

Right. I think I know why 12h made me unease. The concern is around end user UX:

• User is publishing record with TTL 12h
• Loads /ipns/{id} on gateway, record gets cached with TTL 12h
• User complains IPNS update propagation is slow, realizes they can set custom TTL, they do it, but need to wait 12h for update to be in effect, and are stuck with old response.

I think the same UX papercut applies to DNS. I did some spelunking around default TTL in DNS, and higher TTL is used only for things like NS records and roots of TLDs like com or org. Regular websites have TTL of 1h or less.

Some interesting links:

• 1h → https://support.dnsimple.com/articles/what-minimum-time-to-live/
• 300s (5m) → https://developers.cloudflare.com/dns/manage-dns-records/reference/ttl/ (proxied records)
• https://www.ctrl.blog/entry/dns-client-ttl.html
  • Browsers ignore TTL lower than 1-2m, and cache anyway
  • Google has documented their 6 hour maximum TTL value for DNS caching.

In IPNS all websites are final names, so maybe something closer to 5m-15m-60m may be safer default? Allows for caching, but does not cause stale websites for entire day.

(will ping some folks from ecosystem for feedback)

[hacdias]

Then perhaps I would suggest 1 hour. If we set it too low, a user will never need to change it, so there is no big benefit of even having a TTL. And if we set it too high, we have the problem you just mentioned.

[lidel]

My revised understanding is that TTL in IPNS and DNS serve the same purpose: it is a hint about how long a resolution result can be cached on the client, or any intermediate software, before checking for updates.

The difference is that for end user "publishing update" has different impact.
IPNS record update serves the same purpose as (DNS || HTTP) update.

In other words, the TTL for DNS can be higher, but it does not impact end user UX most of the time, because for the most of the time, publishing content update happens on HTTP server. TTL on DNS A record can be 24h, but HTTP server may return Cache-Control which acts as secondary TTL, and HTTP cache could expire faster (e.g. minutes), and Last-Modified header which asks browser to do heuristic caching.

In IPNS, publishing record is the same as publishing content update, which means slower propagation (stale result cached on gateway) will be more noticeable.

To mitigate negative effect on end user, we could:

• pick a lower default TTL (1h, 15m, 5m, or even keep 1m)
• make namesys cache behavior be "stale-while-revalidate" to avoid latency spikes on gateways when TTL expires
  • i think we can do this, because IPNS record has separate expiration field, and as long validity check passes, we can return stale record

Probably good topic to discuss during IPFS Thing in hallway track

[hacdias]

Let's then discuss it during IPFS Thing. I agree we should choose a lower default, but not too low (5m or 15m maybe).

[dokterbob]

I think we should strongly push for/towards making TTL mandatory as publishers typically know about the nature of their content and it does not make sense for the network stack to guess it.

Picking a (somewhat) higher default is in the network's interest and will incentivise publishers to think about TTL's (e.g. for more frequently updated content). Picking too high a value and we might get stale content (e.g. no providers online) and/or bloated caches.

Also note that records might get updated regardless of TTL as updates propagate throughout the network, especially for nodes with a higher centrality. It might even make sense for nodes to be pro-active in propagating updates (e.g. I was caching this so it's probably also relevant to peers).

Concretely, I'd suggest at least 1h and ideally 1d defaults, adding to the docs that frequently updated content should set a lower value. I think more than 1d doesn't make sense cache-wise and less than 1h makes the cache much less useful.

Perhaps we should also consider whether IPNS as it currently stands is indeed the correct mutability framework particularly for frequently updated content (e.g. write-heavy databases). That is: is IPNS future-proof? If not, what is necessary to get there and what would be a realistic roadmap there? Based on that, perhaps it makes sense to map out suitable and less-suitable use cases, which might inform (more) sensible decisions.

[hacdias]

I rebased the PR to make sure it's in sync and changed the status to "blocked" so we discuss it during IPFS þing.

[BigLep]

I rebased the PR to make sure it's in sync and changed the status to "blocked" so we discuss it during IPFS þing.

@hacdias : is there an established forum at IPFS Thing where this is going to be discussed, or do we need to set something up. I'm just making sure that the conversation we want to have happen actually hapens.

[lidel]

If time allows, I think we will create an unconf session on Tue or Wed to discuss what the default TTL should be in boxo/ipns + optionally other IPNS-related topics.

FYI there are three "big" IPNS topics I am aware of:

• Future-proofing: allowing lean V2-only records (IPNS: allow lean V2-only records #376)
• Improve caching: TTL discussed here
• Delegated publishing: this is a wider problem space, some of it identified in
  • IPNI spec IPIP: IPIP-379: Delegated IPNS HTTP API #379
  • Issue about singing IPNS records with crypto wallets (IPNS: support Ethereum wallet signing #323)

From these, TTL is the only one that could be realistically answered during IPFS Thing.

[hacdias]

I rebased this and I'm re-opening this. We did not end up discussing this during IPFS Thing, but many changes have happened in the meanwhile, so maybe we can now come up with a suggestion. It is a suggestion, after all, not a must. I have also changed the suggested default to 1 hour based on the discussions above.