Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

Cannot subscribe more than 5-6 topics [ipfs-http-client] #3741

Closed
TheDiscordian opened this issue Jul 9, 2021 · 7 comments
Closed

Cannot subscribe more than 5-6 topics [ipfs-http-client] #3741

TheDiscordian opened this issue Jul 9, 2021 · 7 comments
Labels
need/triage Needs initial labeling and prioritization

Comments

@TheDiscordian
Copy link
Member

TheDiscordian commented Jul 9, 2021

  • Version:
ipfs-core
{
  version: '0.8.0',
  repo: '10',
  commit: '81f944163f5a78ef15fdc23b2c488ba7f44af0be',
  'interface-ipfs-core': '^0.147.0'
}
ipfs-http-client
{
  version: '0.8.0',
  commit: 'ce693d7e8',
  repo: '11',
  system: 'amd64/linux',
  golang: 'go1.16.2'
}
  • Platform:
    Linux TheDesktop 5.10.42-1-MANJARO #1 SMP PREEMPT Thu Jun 3 14:37:11 UTC 2021 x86_64 GNU/Linux

  • Subsystem:
    ipfs-http-client

Severity:

High

Description:

Issue found at: https://stackoverflow.com/questions/68257274/cannot-subscribe-to-more-than-6-ipfs-pubsub-channels-with-ipfs-http-client

  1. Subscribe to 20 topics
  2. Publish to all 20 topics
  3. Only first 5-6 subs will get messages

Steps to reproduce the error:

First subscribe to the topics:

const { create } = require('ipfs-http-client');

async function echo(msg) {
	console.log(`TopicID: ${msg.topicIDs[0]}, Msg: ${new TextDecoder().decode(msg.data)}`);
}

async function run() {
	// connect to the default API address http://localhost:5001
	const client = create();
	console.log(await client.version());
	
	for (let i = 0; i < 20; i++) {
		await client.pubsub.subscribe(parseInt(i), echo);
	}
}

run();

Then publish:

#!/bin/bash
for i in {1..20}
do
   ipfs pubsub pub $i $i
done

Output is only:

{
  version: '0.8.0',
  commit: 'ce693d7e8',
  repo: '11',
  system: 'amd64/linux',
  golang: 'go1.16.2'
}
TopicID: 1, Msg: 1
TopicID: 2, Msg: 2
TopicID: 3, Msg: 3
TopicID: 4, Msg: 4
TopicID: 5, Msg: 5
@TheDiscordian TheDiscordian added the need/triage Needs initial labeling and prioritization label Jul 9, 2021
@vasco-santos
Copy link
Member

Given it is not the same behaviour all the time, I would say the previous subscriptions are still being propagated in the network while the publish happens.

When a pubsub subscription is done, it will take a bit before nodes exchange their subscription messages around. So, I suspect this is the problem. If so, we should improve our docs to make the expectations clear.

@TheDiscordian could you wrap your bash script with an extra loop that sends the messages for the 20 topics multiple times to confirm these assumptions?

@TheDiscordian
Copy link
Member Author

@TheDiscordian could you wrap your bash script with an extra loop that sends the messages for the 20 topics multiple times to confirm these assumptions?

Sure thing! I'll make it loop 20 times, with a delay between each attempt.

@TheDiscordian
Copy link
Member Author

TheDiscordian commented Jul 20, 2021

@vasco-santos no change, still just first 5 subs using:

 #!/bin/bash
for ii in {1..20}
do
	echo "BATCH "$ii
	for i in {1..20}
	do
		ipfs pubsub pub $i $i
	done
	sleep 10
done

FWIW I've never personally seen 6, I'm reporting 5-6 because the SO user reported 6 (here). It's possible they were mistaken.

Edit: Log: https://ipfs.io/ipfs/QmdFNKLtp2RUheirioNFURfzRF9BwCbjM9Pf5T5c9Q3qgA

achingbrain added a commit that referenced this issue Aug 12, 2021
Browsers can only have six concurrently open connections to a host name.

Pubsub works over HTTP by holding a connection open per subscription, which
means you can only subscribe six times before things start to hang.

gRPC runs over websockets so doesn't have this limitation.  This PR adds
pubsub support to the gRPC server and `ipfs-client` module so you can subscribe
to lots and lots of channels concurrently, working around the browser connection
limitation.

Refs: #3741
@achingbrain
Copy link
Member

achingbrain commented Aug 12, 2021

The problem here is that browsers limit the number of simultaneously open connections to a given hostname to six.

In node you can increase this limit by configuring an agent for the http client to use but browsers are stuck with max six per hostname.

Each subscription uses one connection so that's why you hit the limit.

js-IPFS has an experimental gRPC-over-websockets server though, and it doesn't have this limitation.

It only supports a few methods, the rest are all patched in by ipfs-http-client (so you still have the full IPFS core-api available).

I've opened #3813 which implements ipfs.pubsub.subscribe and ipfs.pubsub.unsubscribe over gRPC - once it's merged & released, just switch out ipfs-http-client for ipfs-client, ensure the Addresses.RPC config key is configured in your ~/.jsipfs/config file and you should be good to go.

@vasco-santos
Copy link
Member

The problem here is that browsers limit the number of simultaneously open connections to a given hostname to six.

Is this in the browser? Per the information, in the issue and stackoverflow this does not seem to be browser environment?

@achingbrain
Copy link
Member

In node the default agent used by the http client limits connections to 6 so it's consistent with the browser. You can configure your own agent to change this:

const { create } = require('ipfs-http-client');
const http = require('http')

async function echo(msg) {
	console.log(`TopicID: ${msg.topicIDs[0]}, Msg: ${new TextDecoder().decode(msg.data)}`);
}

async function run() {
	// connect to the default API address http://localhost:5001
	const client = create({
		agent: http.Agent({ keepAlive: true, maxSockets: Infinity })
	});
	console.log(await client.version());

	for (let i = 0; i < 20; i++) {
		await client.pubsub.subscribe(parseInt(i), echo);
	}
}

run();
{
  version: '0.8.0',
  commit: '',
  repo: '11',
  system: 'amd64/darwin',
  golang: 'go1.15.8'
}
TopicID: 1, Msg: 1
TopicID: 2, Msg: 2
TopicID: 3, Msg: 3
TopicID: 4, Msg: 4
TopicID: 5, Msg: 5
TopicID: 6, Msg: 6
TopicID: 7, Msg: 7
TopicID: 8, Msg: 8
TopicID: 9, Msg: 9
TopicID: 10, Msg: 10
TopicID: 11, Msg: 11
TopicID: 12, Msg: 12
TopicID: 13, Msg: 13
TopicID: 14, Msg: 14
TopicID: 15, Msg: 15
TopicID: 16, Msg: 16
TopicID: 17, Msg: 17
TopicID: 18, Msg: 18
TopicID: 19, Msg: 19

achingbrain added a commit that referenced this issue Aug 16, 2021
* feat: pubsub over gRPC

Browsers can only have six concurrently open connections to a host name.

Pubsub works over HTTP by holding a connection open per subscription, which
means you can only subscribe six times before things start to hang.

gRPC runs over websockets so doesn't have this limitation.  This PR adds
pubsub support to the gRPC server and `ipfs-client` module so you can subscribe
to lots and lots of channels concurrently, working around the browser connection
limitation.

Refs: #3741
@achingbrain
Copy link
Member

This looks like it's been resolved, please re-open or create a new issue if you're still having problems.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
need/triage Needs initial labeling and prioritization
Projects
No open projects
Archived in project
Development

No branches or pull requests

3 participants