[build] Verify SHA checksums of downloaded node executables

[ycombinator]

While testing #6267, @LeeDr noticed that the build ended up packaging a bad node.exe file. It turned out that the node.exe file downloaded by the build process into the .node_binaries folder was bad.

I looked at the code where the various node executables are downloaded; it appears we only check for a HTTP 200 status code to determine success. Perhaps we should also compare the SHA checksums (example for v4.3.2) of the downloaded node executables before proceeding further in the build?

[LeeDr]

+1 on the P1. For whatever reason, this download is very unreliable at the moment. I'm trying multiple times and not getting a good download of node.exe on Windows.

[LeeDr]

this might be a clue. As I keep retrying the build to get a good node.exe, this time I got;

Warning: Client request error: getaddrinfo ENOTFOUND nodejs.org nodejs.org:443 Use --force to continue.

[ycombinator]

That looks like a DNS lookup failure, which should result in no download, not a partial download. Also, if its a DNS lookup failure, I'd expect this callback to be fired with an error. Strange.

[epixa]

We should definitely be checking shas.

[LeeDr]

There's this method function validateDownload(path, expectedHash, success)
here;
https://github.com/elastic/kibana/blob/master/tasks/download_selenium.js#L18

[epixa]

To clarify, I didn't mean to imply that I think we are checking for shas. We should just start if we aren't already.

[LeeDr]

Apparently wreck.request is not very robust for download (on Windows). It's almost ALWAYS failing to properly download node.exe compared to wget https://nodejs.org/dist/v4.3.2/win-x86/node.exe which just worked perfectly 5 times in a row.

[LeeDr]

I'll suggest we change from using wreck.request('GET',... to request.get(URL) like we use when we download the selenium jar. I don't think I've ever had problems with that.

[bevacqua]

Made a utility module to help with this: check-hash

[ppisljar]

@bevacqua utility only returns md5 ... we would need sha256 ... would you mind adding an option to define hashing algorithm used ? ( i submited a pull request against your repo)