[Streams] Sending and receiving data

[odesenfans]

Hello,

I'm currently trying to write a simple messaging protocol over P2P. Under this protocol, we send a message to a peer and wait for the peer to reply using the stream. My issue is that the first message is sent and the response received without any issue, but any further message is never seen by the peer. I simplified the problem and tried to create a simple echo server to demonstrate the issue.

I modified this test of the JS client by adding a second send to reproduce the issue. The initial test case is a one-shot echo server, now it should send two messages in a row while reusing the stream.

    ...
    // First send and receive
    const source = require('it-pushable')()
    source.push(data)

    const output = await pipe(
      source,
      stream,
      take(1),
      toBuffer,
      collect
    )

    source.end()
    expect(output).to.eql([data])

    // Second send and receive
    const source2 = require('it-pushable')()
    source2.push(data)

    const output2 = await pipe(
        source2,
        stream,
        take(1),
        toBuffer,
        collect
    )

    source2.end()
    expect(output2).to.eql([data])
  })

The second send and receive fails. I'm no JS expert so this may be wrong, let me know. Anyway, I made a similar test using the Python client library here. The test has two variants, one that tries to reuse the same stream and one that closes the stream and opens a new one. The variant that reuses the stream fails on both the JS and Go daemons, and the one that opens a new stream works on the Go daemon but fails on the JS one.

Any opinion on what I'm doing wrong / should be fixed?

[odesenfans]

After a bit of exploring, I'm suspecting that the issue lies within this snipper:

pipe(
  concat([encodedMessage], stream.source),
  clientConnection,
  stream.sink
)

I tested a bypass of the clientConnection stream to just connect the input and output, effectively forcing the daemon to act as an echo server without sending data to the user code.

pipe(
  stream.source,
  stream.sink
)

I'm suspecting there's an hypothesis on the read/write abilities of the clientConnection stream that is not respected in practice.

[odesenfans]

Working on the example I gave a few days ago, I think there's at least one issue with the usage of concat in the piping of streams.

Taking the example above:

pipe(
  stream.source,
  stream.sink
)

makes the daemon a perfect echo server. But trying to add an additional message with concat does not work as expected:

pipe(
  concat(["response: "], stream.source),
  stream.sink
)

In the client code, this only results in "response: " being received. Since the protobuf data is sent using the same idea, this makes me think there's an issue with the way async iterables are used. I tried a few variations with no success so far.