Jib and Gradle rebuild with arbitrary file change.

[forgo]

Expected behavior

I have a series of custom gradle tasks that leverage different built-in tasks. It goes something like this:

assemble -> tar (custom) -> copy (custom) -> jib*

I'm leveraging Jib to easily containerize/serve my JavaScript using an nginx:latest image.

Everything works great, but I'd like to be able to trigger a skaffold rebuild when I change a non-Java related file under my gradle project.

For example:

My copy tasks takes an Nginx config from my project's nginx/default.conf and preps it into build/jib-extra-dir/... along with my exploded web content bundle.

I'd like to be able to trigger a skaffold deployment when something in default.conf or my tar.gz changes.

Is there a way to accomplish this via the skaffold.yaml?

I noticed other types of build artifacts (e.g. - buildpack) offer a dependencies: section which:

file dependencies that skaffold should watch for both rebuilding and file syncing for this artifact.

Why is it that jib artifacts do not also have this feature?

Actual behavior

Changes to Nginx config file don't redeploy my client container.

Information

• Skaffold version: 1.1.0
• Operating system: macOS Catalina 10.15.1 (19B88)
• Contents of skaffold.yaml:

build:
  artifacts:
  - image: forgo/forgo-client
    jib:
      project: client

[loosebazooka]

Yeah this isn't currently possible mostly because jib can't know how those files are getting there. I think it makes sense for us to add a dependencies to the jib config block.

[nkubala]

we could also address this by adding support for a trigger file. see #1954

[balopat]

I agree that something here would be interesting.

Now Jib itself is providing dependencies automatically currently (based on maven src, resources definitions), similar to Docker builder (we calculate it based COPY and ADD commands) that's why we didn't have that here in the first place.

I'm thinking about calling this extraBuildTriggers?

@nkubala WDYT?

We'll also need a mini design doc here to explore the cases when these overlap with jib dependencies and/or sync definitions.

[nkubala]

@balopat that seems fine. it might be nice to somehow say these are files and not "triggers" themselves, though now we're getting pretty verbose so I'm not sure it matters much.

I'd love to see a small design writeup here if anyone is interested!

[lightoze]

@balopat @nkubala For many projects Jib plugin makes a decent job but there are special use cases. I've encountered one of them with a module utilizing generated resources. Current implementation has two drawbacks:

1. Jib does not report any inputs of Gradle resource generation task to Skaffold, so rebuild is not triggered during "skaffold dev" and caching is not working correctly.
2. Jib reports generated resources folder. That may be desirable sometimes, e.g. when generated resources are stored in version control. When resources are always generated during the build this creates another caching problem - I encountered a situation where it picked up cached image for empty (not yet generated) resource directory and skipped the build. It also may trigger unnecessary redeploys when resource generation is not entirely reproducible (e.g. some contents are randomly reordered).

So my design suggestion is to have same style dependencies { paths: [], ignore: [] } configuration for Jib as for buildpack and others, so that dependencies reported by _jibSkaffoldFilesV2 task are merged with manual ones and ignores are applied afterwards.

Another option would be to handle that in Jib itself but it's Skaffold-specific configuration anyways and having it in skaffold.yaml seems to be a simpler solution.

[loosebazooka]

@lightoze is your generated resources folder in the build directory?

Perhaps there's a way to query the runtimeClasspath to get the inputs that generate it? didn't realize you're referencing resources, not classes

[lightoze]

@loosebazooka yes it is in build directory. Also generated sources probably will cause the same problem.

[loosebazooka]

@lightoze I think this is better solved on the jib side -- in the linked proposal: GoogleContainerTools/jib#2290

[lightoze]

@loosebazooka ultimately either way is fine for me. Having it on jib side is much more flexible, though.

[loosebazooka]

I would probably actually prefer it happens on the jib side.

[loosebazooka]

@lightoze any chance you can try out this branch? GoogleContainerTools/jib#2292

[briandealwis]

Closing as fixed in upstream: see the Jib docs for the skaffold watch configuration (maven, gradle).