Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jib may be unable to send client certificate #2226

Closed
ST-DDT opened this issue Jan 10, 2020 · 16 comments · Fixed by #2241
Closed

Jib may be unable to send client certificate #2226

ST-DDT opened this issue Jan 10, 2020 · 16 comments · Fixed by #2241
Milestone

Comments

@ST-DDT
Copy link

ST-DDT commented Jan 10, 2020

UPDATE(@chanseokoh): this issue was re-purposed to track the potential problem that Jib may be unable to send client certificate in secure TLS connection: #2226 (comment)

Original comment follows below.


Environment:

  • Jib version: 1.8.0
  • Build tool: Maven 3.6.2
  • OS: Windows 10

Description of the issue:
When the build fails, the actual error message is null. This causes an NPE when the exception is wrapped thus hiding the original exception.

Expected behavior:
The actual error is shown

Steps to reproduce:

mvn clean package jib:build

Additional Information:

The target docker registry is spawned from minishift (on the same host).
Using a different docker registry works fine.

String message = Verify.verifyNotNull(ex.getCause().getMessage()); // keep null-away happy

Log
[INFO] --- jib-maven-plugin:1.8.0:build (default-cli) @ dummy-service ---
[WARNING] Setting image creation time to current time; your image may not be reproducible.
[INFO]
[INFO] Containerizing application to 192.168.1.2:2376/dummy-service:1.0.0-SNAPSHOT...
[WARNING] Base image 'adoptopenjdk/openjdk11:alpine-jre' does not use a specific image digest - build may not be reproducible
[WARNING] The credential helper (docker-credential-desktop) has nothing for server URL: 192.168.1.2:2376

Got output:

credentials not found in native keychain

[WARNING] Cannot verify server at https://192.168.1.2:2376/v2/. Attempting again with no TLS verification.
[WARNING] Failed to connect to https://192.168.1.2:2376/v2/ over HTTPS. Attempting again with HTTP.
[ERROR] I/O error for image [192.168.1.2:2376/dummy-service]:
[ERROR]     null
[WARNING] Cannot verify server at https://registry-1.docker.io/v2/adoptopenjdk/openjdk11/manifests/alpine-jre. Attempting again with no TLS verification.
[INFO] Executing tasks:
[INFO] [========                      ] 27,5% complete
[INFO] > pulling base image manifest
[INFO] > building dependencies layer
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  46.389 s
[INFO] Finished at: 2020-01-10T15:00:00+00:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.8.0:build (default-cli) on project dummy-service: Execution default-cli of goal co
m.google.cloud.tools:jib-maven-plugin:1.8.0:build failed: expected a non-null reference -> [Help 1]
Debug-Log
[DEBUG] TIMING        Pulling base image manifest
[DEBUG] TIMING        Preparing application layer builders
[DEBUG] TIMED Preparing application layer builders : 0.976 ms
[DEBUG] TIMING        Building dependencies layer
[DEBUG] TIMING        Building resources layer
[DEBUG] TIMING        Building snapshot dependencies layer
[DEBUG] TIMING        Building classes layer
[DEBUG] TIMED Building classes layer : 227.477 ms
[DEBUG] TIMED Building snapshot dependencies layer : 230.405 ms
[DEBUG] Building resources layer built sha256:e1197e6b2e8ff0d348266e2cd02d484818ca148f868b20eb411b7b7dff330459
[DEBUG] TIMED Building resources layer : 303.636 ms
[WARNING] The credential helper (docker-credential-desktop) has nothing for server URL: 192.168.1.2:2376

Got output:

credentials not found in native keychain

[DEBUG] No credentials could be retrieved for registry 192.168.1.2:2376
[DEBUG] TIMED Retrieving registry credentials for 192.168.1.2:2376 : 366.123 ms
[DEBUG] TIMING        Authenticating push to 192.168.1.2:2376
[WARNING] Cannot verify server at https://192.168.1.2:2376/v2/. Attempting again with no TLS verification.
[WARNING] Failed to connect to https://192.168.1.2:2376/v2/ over HTTPS. Attempting again with HTTP.
[ERROR] I/O error for image [192.168.1.2:2376/dummy-service]:
[ERROR]     null
[DEBUG] TIMED Authenticating push to 192.168.1.2:2376 : 135.71 ms
[DEBUG] TIMED Building and pushing image : 523.297 ms
[WARNING] Cannot verify server at https://registry-1.docker.io/v2/adoptopenjdk/openjdk11/manifests/alpine-jre. Attempting again with no TLS verification.
[INFO] Executing tasks:
[INFO] [========                      ] 27,5% complete
[INFO] > pulling base image manifest
[INFO] > building dependencies layer
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  47.025 s
[INFO] Finished at: 2020-01-10T15:00:00+00:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.8.0:build (default-cli) on project dummy-service: Execution default-cli of goal co
m.google.cloud.tools:jib-maven-plugin:1.8.0:build failed: expected a non-null reference -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.8.0:build (default-cli) on project du
mmy-service: Execution default-cli of goal com.google.cloud.tools:jib-maven-plugin:1.8.0:build failed: expected a non-null reference
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-cli of goal com.google.cloud.tools:jib-maven-plugin:1.8.0:build failed: expected a non-null reference
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: com.google.common.base.VerifyException: expected a non-null reference
    at com.google.common.base.Verify.verify (Verify.java:124)
    at com.google.common.base.Verify.verifyNotNull (Verify.java:500)
    at com.google.common.base.Verify.verifyNotNull (Verify.java:477)
    at com.google.cloud.tools.jib.plugins.common.JibBuildRunner.runBuild (JibBuildRunner.java:275)
    at com.google.cloud.tools.jib.maven.BuildImageMojo.execute (BuildImageMojo.java:99)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

PS: Do you have any hint what the original error could have been?

@chanseokoh
Copy link
Member

chanseokoh commented Jan 10, 2020

The original exception is of IOException. I'm curious what the exception class is. If you modify the following code in RegistryEndpointCaller.java, you'll be able to check what it is.

    } catch (IOException ex) {
      logError("I/O error for image [" + serverUrl + "/" + imageName + "]:");
      logError("    " + ex.getMessage());
      logErrorIfBrokenPipe(ex);

      if (ex instanceof SSLException) {
        throw new InsecureRegistryException(url, ex);
      }
      throw ex;
    }

For us, I think we should also log the exception class there and do not assume verifyNotNull() in JibBuildRunner.

@ST-DDT
Copy link
Author

ST-DDT commented Jan 10, 2020

[ERROR] I/O error for image [192.168.1.2:2376/dummy-service]:
[ERROR]     null
[ERROR]     null
[ERROR]     org.apache.http.client.ClientProtocolException
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at com.google.api.client.http.apache.v2.ApacheHttpRequest.execute(ApacheHttpRequest.java:73)
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
        at com.google.cloud.tools.jib.http.FailoverHttpClient.call(FailoverHttpClient.java:310)
        at com.google.cloud.tools.jib.http.FailoverHttpClient.call(FailoverHttpClient.java:252)
        at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:141)
        at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:117)
        at com.google.cloud.tools.jib.registry.RegistryClient.callRegistryEndpoint(RegistryClient.java:473)
        at com.google.cloud.tools.jib.registry.RegistryClient.getRegistryAuthenticator(RegistryClient.java:282)
        at com.google.cloud.tools.jib.builder.steps.AuthenticatePushStep.call(AuthenticatePushStep.java:66)
        at com.google.cloud.tools.jib.builder.steps.StepsRunner.lambda$authenticatePush$3(StepsRunner.java:229)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.http.ProtocolException: The server failed to respond with a valid HTTP response
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:149)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        ... 19 more

This looks like a bug(?)/imperfection in the apache http client library as ClientProtocolException seems to swallow the message.

But I don't understand that cause either. Does it attempt to connect to the registry using HTTP instead of HTTPS? (The registry uses a self-signed certificate + allowInsecureRegistries=true).

@chanseokoh
Copy link
Member

chanseokoh commented Jan 10, 2020

Jib (JVM) failed to verify the server over HTTPS:

[WARNING] Cannot verify server at https://192.168.1.2:2376/v2/. Attempting again with no TLS verification.

So it attempts the port again with no TLS verification. But it also failed even when not verifying the server:

[WARNING] Failed to connect to https://192.168.1.2:2376/v2/ over HTTPS. Attempting again with HTTP.

So Jib attempted speaking plain-HTTP on the port 2376. I suspect the port 2376 doesn't speak HTTP, so that's why you get ClientProtocolException and the message "(t)he server failed to respond with a valid HTTP response."

What does curl -v -k https://192.168.1.2:2376/v2/ say? Maybe write a simple Java program to connect to https://192.168.1.2:2376/v2/ and see if it works. I wonder if the server uses some kind of SSL-related algorithm that the JVM cannot handle. Another possibility is that the port 2376 doesn't speak TLS/SSL at all.

BTW, note that in general, if you installed the self-signed certificate to the JVM truststore correctly, you don't even need to set allowInsecureRegistries.

@ST-DDT
Copy link
Author

ST-DDT commented Jan 10, 2020

$ curl -v -k https://192.168.1.2:2376/v2/
*   Trying 192.168.1.2:2376...
* TCP_NODELAY set
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connected to 192.168.1.2 (192.168.1.2) port 2376 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [64 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [792 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
{ [15380 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
} [7 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS alert, bad certificate (554):
{ [2 bytes data]
* error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection 0
curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate

@ST-DDT
Copy link
Author

ST-DDT commented Jan 10, 2020

BTW, note that in general, if you installed the self-signed certificate to the JVM truststore correctly, you don't even need to set allowInsecureRegistries.

I know. It was supposed to be a quick test...

@chanseokoh
Copy link
Member

I think there's a problem.

curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate

@chanseokoh
Copy link
Member

Ah, I don't know much about this stuff, but reading https://community.letsencrypt.org/t/curl-35-errorssl-routinessslv3-alert-bad-certificate/85239/5, the client has to present a certificate too? (I'm just talking gibberish here.)

@ST-DDT
Copy link
Author

ST-DDT commented Jan 10, 2020

You are right, seems to be a missing client certificate.

$ curl --cacert ca.pem --key key.pem --cert cert.pem https://192.168.99.100:2376/v2/
{"message":"page not found"}

I will try to setup the truststore/keystore appropriately. If that doesn't work I will just build the tars in maven and then load them using the docker cmd (Which is what I'm currently doing/using).

Feel free to close this issue or use it for tracking purposes of the "NPE hides Exception" bug.

@chanseokoh
Copy link
Member

Please update here how it goes. I may be wrong, but maybe the client certificate should be added to the keystore (while the server certificate to the truststore): https://blog.kunicki.org/blog/2015/09/10/ssl-client-certificates-on-the-jvm/ Hope it works.

@ST-DDT
Copy link
Author

ST-DDT commented Jan 13, 2020

I added the certs/keys to my store and run it again:

[ERROR] I/O error for image [192.168.99.100:2376/adoptopenjdk/openjdk11]:
[ERROR]     readHandshakeRecord
[ERROR]     javax.net.ssl.SSLException: readHandshakeRecord
        at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1072)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at com.google.api.client.http.apache.v2.ApacheHttpRequest.execute(ApacheHttpRequest.java:73)
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
        at com.google.cloud.tools.jib.http.FailoverHttpClient.call(FailoverHttpClient.java:310)
        at com.google.cloud.tools.jib.http.FailoverHttpClient.call(FailoverHttpClient.java:237)
        at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:141)
        at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:117)
        at com.google.cloud.tools.jib.registry.RegistryClient.callRegistryEndpoint(RegistryClient.java:473)
        at com.google.cloud.tools.jib.registry.RegistryClient.pullManifest(RegistryClient.java:303)
        at com.google.cloud.tools.jib.registry.RegistryClient.pullManifest(RegistryClient.java:307)
        at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.pullBaseImage(PullBaseImageStep.java:205)
        at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call(PullBaseImageStep.java:126)
        at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call(PullBaseImageStep.java:59)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
        Suppressed: java.net.SocketException: Software caused connection abort: socket write error
                at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
                at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
                at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
                at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)
                at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:351)
                at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
                at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:405)
                ... 29 more
Caused by: java.net.SocketException: Software caused connection abort: socket write error
        at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
        at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
        at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeChangeCipherSpec(SSLSocketOutputRecord.java:221)
        at java.base/sun.security.ssl.OutputRecord.changeWriteCiphers(OutputRecord.java:162)
        at java.base/sun.security.ssl.ChangeCipherSpec$T10ChangeCipherSpecProducer.produce(ChangeCipherSpec.java:118)
        at java.base/sun.security.ssl.Finished$T12FinishedProducer.onProduceFinished(Finished.java:395)
        at java.base/sun.security.ssl.Finished$T12FinishedProducer.produce(Finished.java:379)
        at java.base/sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:436)
        at java.base/sun.security.ssl.ServerHelloDone$ServerHelloDoneConsumer.consume(ServerHelloDone.java:182)
        at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
        at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:177)
        at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
        at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
        at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
        ... 30 more

Maybe java doesn't support this kind of SSL impl?
Unfortunately this error message is relatively inexpressive.

@chanseokoh
Copy link
Member

chanseokoh commented Jan 13, 2020

Sorry, no idea. Maybe try simple code like here or SSLPoke against https://192.168.1.2:2376/v2/. And perhaps worth trying different JVMs (Java 8, Java 11, ...) on different machines.

Using jshell is another way.

@ST-DDT
Copy link
Author

ST-DDT commented Jan 22, 2020

I didn't manage to get it running. I have no more time to try this (I'm not even sure this is related to Java at all). Thanks for your help.

Please fix the NPE in the error message anyway.

@chanseokoh
Copy link
Member

chanseokoh commented Jan 29, 2020

Actually, I remembered someone told us an issue that Jib isn't sending a client certificate a while ago. Although I haven't verified myself, this may be the cause: googleapis/google-http-java-client#904

@chanseokoh
Copy link
Member

@ST-DDT 2.0.0 was released and fixes the NPE. But it's very possible that Jib is unable to send a client certificate due to some problem in the Google HTTP Client. I'll re-purpose this issue for tracking.

@chanseokoh chanseokoh reopened this Jan 29, 2020
@chanseokoh chanseokoh changed the title NPE when throwing an Exception Jib may be unable to send client certificate Jan 29, 2020
@chanseokoh chanseokoh removed this from the v2.0.0 milestone Jan 29, 2020
@chanseokoh chanseokoh added this to the v2.5.0 milestone Jul 16, 2020
@chanseokoh
Copy link
Member

Fixed by #2592. See #2585 for more details.

@chanseokoh
Copy link
Member

chanseokoh commented Aug 7, 2020

@ST-DDT we released Jib Gradle 2.5.0 and Jib Maven 2.5.2 and believe this is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants