no disconnection after a re-connection

[tomgrossman]

Do you want to request a feature or report a bug?
bug

What is the current behavior?
After the connection is re-connected and DB goes down again, there is no disconnection event and the readyState stays in 1 - connected

If the current behavior is a bug, please provide the steps to reproduce.

1. start the DB
2. start the app - make sure the readyState is 1
3. stop the DB
4. wait for disconnection - readyState is 0
5. start the DB
6. wait for reconnection - readyState is 1
7. stop the DB
8. the connection never gets a disconnected event and the readyState is 1

using the below script, the console output is:

connected
open
createConnection
ping: true readyState: 1
ping: true readyState: 1
ping: false readyState: 1
disconnected
ping: false readyState: 0
ping: false readyState: 0
ping: false readyState: 0
ping: false readyState: 0
ping: false readyState: 0
ping: true readyState: 0
ping: true readyState: 0
ping: true readyState: 0
ping: true readyState: 0
connected
reconnected
ping: true readyState: 1
ping: true readyState: 1
ping: false readyState: 1
ping: false readyState: 1
ping: false readyState: 1
ping: false readyState: 1
ping: false readyState: 1
ping: false readyState: 1
ping: false readyState: 1
ping: false readyState: 1
...

you can see that the ping to the server turns to false but there is no disconnection.
the script that is used:

const tcpp          = require('tcp-ping');
const Promise       = require('bluebird');
const mongoose      = require('mongoose');
mongoose.Promise = Promise;

const host = 'localhost';
const port = '2244';
const DB_URL = `mongodb://${host}:${port}/test_database`;
const ConnectionOptions = {
    promiseLibrary: Promise,
    reconnectTries: Number.MAX_VALUE
};

const DbConnection = mongoose.createConnection(DB_URL, ConnectionOptions, (err) => {
    console.log('createConnection');
    if (err) {
        console.log('createConnection', err.message);
    }
});

DbConnection.on('error', function (error) {
    console.log('error: ', error.message)
});

DbConnection.on('disconnected', function () {
    console.log('disconnected');
});

DbConnection.on('close', function () {
    console.log('close');
});

DbConnection.on('open', function () {
    console.log('open');
});

DbConnection.on('connected', function () {
    console.log('connected');
});

DbConnection.on('connecting', function () {
    console.log('connecting');
});

DbConnection.on('reconnected', function () {
    console.log('reconnected');
});

process.on('unhandledRejection', function (err) {
    if ('MongoError' === err.name) {
        console.log('unhandledRejection: ' + err.message);
    }
});

setInterval(() => {
    tcpp.probe(host, port, (err, available) => {
        console.log('ping:', available, 'readyState:', DbConnection.readyState);
    })
}, 3000);

module.exports = DbConnection;

What is the expected behavior?
When there is a connection loss, there should be a disconnected event and the readyState should update accordingly, like in the first disconnection.
I've tested almost the same script with the latest native driver using the isConnected function instead of the readyState and it works fine there. whenever the ping is gone, the isConnected function returns false.

Please mention your node.js, mongoose and MongoDB version.
node.js - 8.10.0
mongoose - 5.0.10
MongoDB - 3.4.10

[lineus]

@tomgrossman you make incredibly detailed bug reports. Thank you.

I started your code without modification (using @5.0.10), then stopped and started the server several times. I see the disconnect event each time I stop the server and the ready states line up with:

connectionstate.js:

STATES[0] = disconnected;
STATES[1] = connected;

here is my output:

mongoose5: ./6249.js 
connected
open
createConnection
ping: true readyState: 1
ping: true readyState: 1
disconnected
ping: false readyState: 0
ping: false readyState: 0
ping: false readyState: 0
connected
reconnected
ping: true readyState: 1
ping: true readyState: 1
ping: true readyState: 1
disconnected
ping: false readyState: 0
ping: false readyState: 0
ping: false readyState: 0
connected
reconnected
ping: true readyState: 1
ping: true readyState: 1
disconnected
ping: false readyState: 0
ping: false readyState: 0
ping: false readyState: 0
ping: false readyState: 0
connected
reconnected
ping: true readyState: 1
ping: true readyState: 1
ping: true readyState: 1
ping: true readyState: 1
ping: true readyState: 1
ping: true readyState: 1
^C
mongoose5: 

here is the contents of the file I ran above:

#!/usr/bin/env node
'use strict'

const db = require('./dep6249') // your repro code

[tomgrossman]

@lineus Mmm weird, it happens to me all the time. The only information I left out is that it's a mongos server connected to 2 shards. But I didn't think it matters since it's a single mongos server. But maybe it does matter.
As you can see in my case, when the ping is up, it still takes few seconds until re-connection, since it takes time for the mongos to initialize. So maybe this issue effects also the disconnection.

[lineus]

I think the mongos piece is the difference between our results. I'll tinker with it over the weekend and see what if anything I can find. this issue against the native driver calls out event weirdness in mongos, and the last response was that they're going to refactor the connection code in a few months.

[vkarpov15]

Yep this only happens with mongos. Investigating now.

[vkarpov15]

I put in a PR mongodb/node-mongodb-native#1685 ☝️ that looks like it fixes this issue. Will hustle to get it out, thanks for your patience @tomgrossman

[tomgrossman]

@vkarpov15 thanks for updating! I had many re-connections problems to non-consistent mongos instances when I tried to upgrade to mongoose 5. Both this and 6244 issues were forced me to rollback to 4.13. After this one is fixed I will be able to try again.
But is there any chance you will add a "re-connection" tests? Especially to mongos instances?

[vkarpov15]

Yes that's something we need to add. We normally don't do that because we rely on the mongodb driver to handle the differences between individual topologies, but I think its worth having a high-level check.