[Content Request] Document IPNI / Indexers

[honghaoq]

We are working on Network indexer that index IPFS and Filecoin contents. We should add indexing doc / pointer on the ipfs page here under peer-to-peer-sharing https://docs.ipfs.tech/concepts/#peer-to-peer-sharing.

We can leverage the indexer gitbook to build the content page: docs.cid.contact

[honghaoq]

@TMoMoreau fyi

[jacobheun]

Perhaps this is a separate issue but documenting reframe would be super useful. There's various resources you can find but querying reframe on the site doesn't yield any results.

https://github.com/ipfs/kubo/releases/tag/v0.16.0 has configuration guidance.
https://blog.ipfs.tech/2022-09-02-introducing-reframe/ has useful information for understanding reframe as well as some curl commands to verify reframe servers.
https://github.com/ipfs-shipyard/someguy is also a useful tool for it, from @aschmahmann -

Generally I'd just use someguy ask --endpoint=https://cid.contact/reframe findprovs <cid> (or put a --pretty before the findprovs keyword) (https://github.com/ipfs-shipyard/someguy).

[lidel]

"IPNI" (InterPlanetary Network Indexer) should be added to glossary: https://docs.ipfs.tech/concepts/glossary/
Ref. https://github.com/ipni

[TMoMoreau]

I'll get a draft PR made for this. Going to make a list here as well, to verify and keep track of what needs to be done.

• Add pointer for the Filecoin network indexer to the list under the peer to peer file sharing subsection, found here: https://docs.ipfs.tech/concepts/#peer-to-peer-file-sharing
• Potentially create new doc for the Filecoin network indexer. I say potentially because I am wondering if the pointer leading to the already existing docs would be the better option.
• Add IPNI to the glossary using https://github.com/ipni as a reference. (Should we also add a link to the IPNI github repo?)

@jacobheun I think it would be best to make a separate issue for the reframe documentation topic.

@lidel @honghaoq

[honghaoq]

Thanks. FYI Torfinn Olsen is gonna be the new indexer PM on this going forward.

[lidel]

I think this is blocked until specs solidify and the implementation of the new API ships with Kubo 0.18.

[lidel]

I guess this is unblocked because we've shipped Kubo 0.18 with default IPNI enabled (https://cid.contact/) by default.

cc @willscott – mind dropping good specs/design docs for docs team to catch up with latest state of things?

[willscott]

ipfs/specs#337 is probably the most useful definition for the API that's been shipped

https://github.com/ipni/storetheindex-docs can be useful from the IPNI side

[ElPaisano]

@lidel can you help the docs team to determine if this is still relevant? https://docs.ipfs.tech/concepts/#peer-to-peer-sharing no longer exists, as it was removed when 2color and I redid the concepts section

[willscott]

@TorfinnOlsen is probably a good point of contact here.

The IPNI team would like to have docs here, and this remains a valid and effective way to do content routing. Since the use of delegated routing is on by default in current releases of kubo we probably should have documentation about it.

[ElPaisano]

Thanks @willscott

[ElPaisano]

I'm thinking we can add a new page in Concepts -> Subsystems and Components called IPNI that describes this subsystem

[TorfinnOlsen]

@ElPaisano one thing to be aware of is a lot of the language around this specific topic is in a bit of flux right now.

The document @willscott mentioned is the most important starting place however upon reviewing it I submitted a PR to include an important update that might be confusing to readers regarding Reframe which has since v0.19 Kubo release been deprecated.

You should also be aware of the Delegated IPNS HTTP API which intends to delegate naming system functionality over HTTP APIs.

I think from a broader descriptive perspective of the IPNI beyond what you'll find in storetheindex-docs you can use our teams overview blogposts as a valuable reference point on some of the more important points.

How does the network indexer work
How to be an index provider

There are a few more of these that dive deeper into topics involving more how to operate the network indexer and less how it routes content efficiently. If that type of documentation is beneficial for context sake let me know and I'll happily load your plate up with additional links.

[ElPaisano]

Cheers thanks @TorfinnOlsen

Initial draft is here #1552. This would be good to quickly discuss in a sync call

[ElPaisano]

@TorfinnOlsen can you let me now if you will have a chance to review this week? I figure it might not happen with IPFS thing

[ElPaisano]

Redraft from @TorfinnOlsen is here https://github.com/TorfinnOlsen/IPNIdocumentation/blob/main/IPFSdocsIPNI.md. Will edit this and add to the PR