Authenticated PubSub

[ianopolous]

@whyrusleeping asked me to post some use cases for an authenticated pubsub api. In this api the topic to subscribe to would be a public key (or one could consider a path starting with a public key for more fine grained control). Anyone can subscribe to a key. However all updates have to be signed by the corresponding private key, and the network would verify these signatures before propagating them.

We would use this in Peergos to allow only users with write access to a part of the filesystem to publish updates.

I can imagine an equivalent to an RSS feed, or even a plain blog, being done using this.

The only sticky point I can think of is ensuring replay attacks aren't possible. Where nodes without the private key republish an old update, but there are techniques to solve this.

[lachenmayer]

I'm very interested in this - is there any documentation around how pubsub works currently / could someone point me towards the right places in the code to play around with this?

[ianopolous]

@lachenmayer There's probably another explanation somewhere, but there's https://github.com/ipfs/blog/blob/4b81aaa0d0819858cf89f5f3f8ab134adfa48128/src/21-pubsub/index.md

[daviddias]

@ianopolous thank you for kicking this again :) See notes from the IPFS workshop here -- #154 -- We've started talking and describing what this would like there :)

[ianopolous]

There is also more discussion here

[RangerMauve]

I would say that instead of authentication, you should use encryption.

If you want to pubsub data that you don't want to be read by random people, share a secret key of some sort and encrypt all the data before publishing it. Then share the secret key with all peers that need it and have them decrypt messages as they get them. If any messages come that can't be decrypted with the key, ignore them.

This makes it easy to have secure pubsub without having to trust the actual pubsub network or worry about people snooping on your messages.

The main limitation here is that people will still be able to pick up on frequency of messages so you should be wary about whether you have any sensitive timing information

[ghost]

@RangerMauve this discussion is not about privacy or encryption -- it's about authenticity of pubsub messages.

[RangerMauve]

@lgierth Encryption acts as authentication in this case. Messages that aren't encrypted with the correct key are ignored.

[ghost]

Ah yes, understood -- now you have to distribute keys though, and the network can't proactively drop invalid messages anymore.

[RangerMauve]

I like the idea of having topics be public keys, but I really like the option to use arbitrary strings for topics. I could see applications that use one particular key for authentication/encryption using multiple topics for different streams of events. For instance different chat rooms. I guess it could have all data in a single stream and have clients filter out messages they don't want, but that would be a lot less efficient.

[RangerMauve]

An application could generate keys from passphrases to make it easier on users. See, for example, how peergos supports "logging in".

One could also use a password that's used with aes encryption.

[RangerMauve]

Sharing keys between peers is also in the works as you can see here

[ianopolous]

Using authentication instead of encryption means the network (without knowledge of any keys) can mitigate DOS attacks from people who try to flood the topic. In Peergos we would be publishing things which are independently encrypted anyway. But pubsub doesn't need to know that.

This means we also don't need to trust the network. Though bear in mind the network can always drop/delay messages, but can't inject messages.

[Stebalien]

@RangerMauve

Encryption acts as authentication in this case. Messages that aren't encrypted with the correct key are ignored.

FYI, that is authentication; it's called authenticated encryption. It just uses symmetric keys for signatures instead of asymmetric keys. Encryption by itself provides no authentication.

---

I've written up a small issue for a general purpose capabilities service here if anyone is interested: #274

[ptescher]

The solution here would probably be that all topics are prefixed with the public key. This would allow for a chat application with a single key to still have separate topics for each channel without having to generate a new key for every new channel.

[ianopolous]

@pat2man Yep, that's what I describe in the original post.

[mitra42]

What we do is use an IPFS hash for an object containing metadata - including title of channel for example, and the public key. Then use that hash as the name of the pubsub channel (currently in Y).
See Dweb example_versions.html for an example. (Note this is authentication of posting, not encryption of the contents which is a separate conversation.)

Note - in our apps we don't have a use-case for channels that lack persistance, so we use "YJS" just to give us the ability to read back in the channel, and from recent conversations this would work as well on OrbitDb.

[ptescher]

One thought on using hashes as topics.

Building on @RangerMauve's comment on efficiency of hashes vs strings, routing can get quite difficult if you can't summarize topics. MQTT does this quite well with their wildcard subscriptions.

Lets say I am running a chat server and I want to create a directory of all channels with a "last message sent at" timestamp. I could write an application that listens to each topic, stores the last timestamp, and then makes that list available as a JSON file via IPFS.

If each topic is just a hash I need to subscribe to each topic individually. In MQTT I can subscribe to something like /topic/# and it will subscribe to all topics with that prefix. Not only am I now subscribe to any existing topics with that prefix, I am also subscribe to any topics created with that prefix.

This can vastly reduce the size of route tables since those wildcard subscriptions can encompass any number of topics.

Instead of pure hash based routing I would instead propose that topics contain both a public key (so that two topics don't collide) and a string. Subscriptions would then contain both the public key and a string that can optionally contain wildcards.

The public key could simply be prefixed to the string, so for our chat channel you would have

• <hash>/rooms/random
• <hash>/rooms/general

You could subscribe to all rooms via <hash>/rooms/#

See https://mosquitto.org/man/mqtt-7.html for some examples.

[RangerMauve]

I like your idea of having a public key hash + topic string, but the MQTT wildcard use case would be very difficult to do in a distributed way.

MQTT is geared towards centralized brokers which are receiving all publishes in the system, they can then easily filter out which client gets which events based on the clients' subscribed patterns.

For pubsub as it is, you listen on a specific topic, and peers publishing to that topic discover that you're listening on them. If wildcards get introduced, publishing nodes don't have an easy way to actually discover peers that are subscribed to patterns for the topic they're publishing to.

If we treat the string as an actual path, then publishing nodes could permutate possible topic patterns that have a wildcard at any point in the path.

e.g.

When publishing foo/bar/baz, look for peers subscribed to the following:

foo/bar/#
foo/#
#
foo/bar/+
foo/+/+
+/+/+
foo/+/bar
+/bar/baz
+/bar/+

Etc.. with combinations of having both # and + wildcards.

Obviously this would be super inefficient.

[ptescher]

I guess I assumed that peer discovery would be based on the hash and include an array of subscription strings.

From a publisher perspective I would need to keep track of more peers (all peers for a given hash), but applications would be less likely to need to transmit unnecessary information.

[RangerMauve]

That might work! I'm a huge fan of MQTT so I'd love to be able to bring my preferred method of working with pubsub to apps using IPFS. 😄

[mitra42]

@pat2man - the problem with /rooms/random etc as the identifier is that it assumes each room has the same hash, while in our model each room is separately authenticated, this is probably different from a chat-room where multiple 'people' post to the same chat.

[ptescher]

@mitra42 having a hash prefix and a topic would allow both.

Essentially hashes would create "namespaces" for topics. Messages with the same topic but different hashes would not be compatible. No way around that.

If you want to have private chat rooms you can just ignore the topic.