FTP module stability

[raboof]

We recently did a lot of changes in the FTP module, including #373 and #407.

Things look fairly good, though the tests can be quite slow and "should fail and report the exception in the result status if connection fails" fails suspiciously often on travis (#410).

Do we feel this is ready to be released or does someone want to take a closer look first?

@svezfaz @juanjovazquez @argast @nktpro @eharik @rafacm @neowulf33 @frj

[juanjovazquez]

As mentioned in #407, the current issues have to do with the testing infrastructure and the fact that they need real FTP servers running to do the job. We're currently starting three different kinds of servers/configurations (FTP, FTPs and sFTP), one instance per Spec class. I think that one of the last tests added is running and stopping additional server instances, again one per protocol.

So, IMHO the component is usable but we should invest some time in making the testing infra more robust. Maybe it's not neccesary to run all test cases for all configurations. More ideas are welcome ;-).

[raboof]

The should fail and report the exception in the result status if connection fails test indeed stops and re-starts the test FTP server, so that's one extra cycle per protocol (of which there are 6: FTP, FTPs and 4 different configurations of sFTP).

I'm not sure starting (in-process Java) FTP servers for testing should be a big deal - it would be cool if we could make that run smoothly without disabling tests :).

[juanjovazquez]

Of course, that would be the best case scenario. I'll try to find the time to take a closer look at it. Very busy now though...

[raboof]

But we're good to release alpakka 0.11 in this state?

[juanjovazquez]

IMHO yes, we're safe releasing. I'll spend some time on the testing infra ASAP. Last weekend was impossible as I had to work on other projects :-(.

[juanjovazquez]

It seems we might be facing a problem of low entropy on Travis VMs. We might need to configure jvm opts to use /dev/urandom instead of /dev/random as a random generation device. Still working on it.

[timothyklim]

After updating alpakka with new sftp dependency, I've got race conditions and strange sshj exceptions:

net.schmizz.sshj.transport.TransportException: Server closed connection during identification exchange
	at net.schmizz.sshj.transport.TransportImpl.init(TransportImpl.java:181) ~[com.hierynomus.sshj-0.21.1.jar:?]
	at net.schmizz.sshj.SSHClient.onConnect(SSHClient.java:770) ~[com.hierynomus.sshj-0.21.1.jar:?]
	at net.schmizz.sshj.SocketClient.connect(SocketClient.java:127) ~[com.hierynomus.sshj-0.21.1.jar:?]
	at akka.stream.alpakka.ftp.impl.SftpOperations.$anonfun$connect$1(SftpOperations.scala:31) ~[com.lightbend.akka.akka-stream-alpakka-ftp_2.12-0.11.jar:0.11]
	at scala.util.Try$.apply(Try.scala:209) ~[org.scala-lang.scala-library-2.12.3.jar:?]
	at akka.stream.alpakka.ftp.impl.SftpOperations.connect(SftpOperations.scala:23) ~[com.lightbend.akka.akka-stream-alpakka-ftp_2.12-0.11.jar:0.11]
	at akka.stream.alpakka.ftp.impl.SftpOperations.connect$(SftpOperations.scala:23) ~[com.lightbend.akka.akka-stream-alpakka-ftp_2.12-0.11.jar:0.11]
	at akka.stream.alpakka.ftp.impl.FtpLike$$anon$2.connect(FtpLike.scala:33) ~[com.lightbend.akka.akka-stream-alpakka-ftp_2.12-0.11.jar:0.11]
	at akka.stream.alpakka.ftp.impl.FtpLike$$anon$2.connect(FtpLike.scala:33) ~[com.lightbend.akka.akka-stream-alpakka-ftp_2.12-0.11.jar:0.11]
	at akka.stream.alpakka.ftp.impl.FtpGraphStageLogic.preStart(FtpGraphStageLogic.scala:25) ~[com.lightbend.akka.akka-stream-alpakka-ftp_2.12-0.11.jar:0.11]
	at akka.stream.impl.fusing.GraphInterpreter.init(GraphInterpreter.scala:290) ~[com.typesafe.akka.akka-stream_2.12-2.5.3.jar:?]
	at akka.stream.impl.fusing.GraphInterpreterShell.init(ActorGraphInterpreter.scala:540) ~[com.typesafe.akka.akka-stream_2.12-2.5.3.jar:?]
	at akka.stream.impl.fusing.ActorGraphInterpreter.tryInit(ActorGraphInterpreter.scala:659) ~[com.typesafe.akka.akka-stream_2.12-2.5.3.jar:?]
	at akka.stream.impl.fusing.ActorGraphInterpreter.preStart(ActorGraphInterpreter.scala:707) ~[com.typesafe.akka.akka-stream_2.12-2.5.3.jar:?]
	at akka.actor.Actor.aroundPreStart(Actor.scala:521) ~[com.typesafe.akka.akka-actor_2.12-2.5.3.jar:?]
	at akka.actor.Actor.aroundPreStart$(Actor.scala:521) ~[com.typesafe.akka.akka-actor_2.12-2.5.3.jar:?]
	at akka.stream.impl.fusing.ActorGraphInterpreter.aroundPreStart(ActorGraphInterpreter.scala:650) ~[com.typesafe.akka.akka-stream_2.12-2.5.3.jar:?]
	at akka.actor.ActorCell.create(ActorCell.scala:591) ~[com.typesafe.akka.akka-actor_2.12-2.5.3.jar:?]
	at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462) ~[com.typesafe.akka.akka-actor_2.12-2.5.3.jar:?]
	at akka.actor.ActorCell.systemInvoke(ActorCell.scala:484) ~[com.typesafe.akka.akka-actor_2.12-2.5.3.jar:?]
	at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:282) ~[com.typesafe.akka.akka-actor_2.12-2.5.3.jar:?]
	at akka.dispatch.Mailbox.run(Mailbox.scala:223) ~[com.typesafe.akka.akka-actor_2.12-2.5.3.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_141]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_141]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_141]
Caused by: net.schmizz.sshj.transport.TransportException: Server closed connection during identification exchange
	at net.schmizz.sshj.transport.TransportImpl.receiveServerIdent(TransportImpl.java:201) ~[com.hierynomus.sshj-0.21.1.jar:?]
	at net.schmizz.sshj.transport.TransportImpl.init(TransportImpl.java:174) ~[com.hierynomus.sshj-0.21.1.jar:?]
	... 24 more

We've faced this every 5 minutes after node restart. We're using alpakka-ftp to uploading image thumbnails through sftp, 5-10 files per second. Sshd logs are clean, when sshj is failed.

Is new ssh library stable enough to use in alpakka?

[juanjovazquez]

@timothyklim According to the stack trace, this seems to be an application error on the transport layer. The client is no receiving a proper identification message from server. Do you have access to the server log?. Maybe this server is limiting the number of allowed connections. If the problem persists, you might want to open an independent ticket providing enough information to diagnose it.

[ennru]

The stability has improved since this issue was created. We still see FTP tests failing from time to time.
See #991, #1047

Closing this.