Java 9 long classpath support to enable deterministic class loading order

[chanseokoh]

Fixes #1871 (where #2733 and #1907 are duplicates).

Also addresses #2471 (AppCDS), #894, #1640, and #3147.

---

As discussed in the previous PR (#2866 (comment)) , for Java 9+, we can always achieve explicitly listing all the dependencies by leveraging the argument file support in Java 9+, without the risk that some OS may not be able to handle an exceptionally long command. This mode will be the default for Java 9+ apps.

But ideally, for Java 9+, I think we should really use the @-argument file feature; we can always preserve the classpath order while never hitting the system argument length limit. This is going to be a lot bigger change.

In sum, perhaps

• Java 9+: always use a @-argument file. (expandClasspathDependencies is irrelevant.)
• Java <=8: expand or not based on expandClasspathDependencies. Not decisive on if it should be true by default.

With this, we can always preserve the exact dependency loading order defined by the build system. Going forward, the expandClasspathDependencies option will have no effect with Java 9+. OTOH, Java <=8 will have to continue to set expandClasspathDependencies if the user wants to

Notable implementation details

The design has changed from the initial one.

• It creates a new layer named "jvm arg files".

• Two argument files will be created inside the layer:

  • classpath: /app/jib-classpath-file
  • main class: /app/jib-main-class-file

  Note that the main class file is not used by the implementation and thus unnecessary in this sense. (Keep reading.)

• The arg files are always created, whether it's Java 8 or 9+, or even when the user sets a custom entrypoint.

• The arg files will be very useful for people who want to get the classpath and the main class inferred by Jib at runtime. Being able to do so has been one of the most requested features that will eliminate the ugly workaround where the user had to hard-code the values that may change in newer Jib versions (e.g., by this PR). It'll be easy to read these files in an entrypoint shell script. Some notable examples:

  • JHipster launch script
  • Stack Overflow answer with the workaround
  • AppCDS: Gradle extension release v0.2.0 #2477
  • Expose classpath, main class, and other variables for reference from an explicit entrypoint #894
  • Expose jib classpath and mainClass as environment variable #3147 and Jib extension user contribution

[mpeddada1]

Looks good! Only minor comments. We may want to update the issues listed above (or add an entry in the FAQ) with the new workaround once this feature is released.

[chanseokoh]

Will merge this today.

[chanseokoh]

Hmm.... actually I think I found an issue with this. Looking into it.

[chanseokoh]

Hmm.... actually I think I found an issue with this. Looking into it.

Nope, false alarm. It was that I wanted to clean up the code a bit last week.

[mpeddada1]

Took another look. LGTM