docker assets: cannot build docker images outside the source tree (i.e. against a cdk.out directory)

[eladb]

The feature to support customizing the docker file name introduced in #5652 will pass the full absolute path of the docker file name to docker build. This happens both for the case where a custom docker file name is specified and when it is not specified (defaults to /full/path/to/Dockerfile).

However, since we are staging the build directory into cdk.out (which can potentially be ported to a different system, e.g. as part of a CI/CD process), we need the name of the custom docker file to be relative and not absolute.

See https://github.com/aws/aws-cdk/pull/5652/files#r366863153

---

This is 🐛 Bug Report

[MrJonBirch]

I am not convinced this has been completely resolved. Please see this repo. This is a sample repo I have built that is structured exactly the same a real project I am working on where all projects should share the same docker context but each project has its own or multiple docker files.

Note that the cdk.out folder does not contain any docker files but running the AwsStack project does not throw any errors.

Also note these lines of code from the project.
AwsCdkDockerSample/AwsStack/Program.cs : Lines 16-23

AwsCdkDockerSample/AwsStack/DevStack.cs: Lines 54-60

Thanks, any insight would be much appreciated.

[ric-sapasap]

I am also having a problem with this after upgrading to 1.21.1 from 1.20.

Below is a the snippet of what I'm trying to do:

import cdk = require('@aws-cdk/core');
import ecr = require("@aws-cdk/aws-ecr");
import ecrAssets = require("@aws-cdk/aws-ecr-assets");
import path = require('path');

export class EcrStack extends cdk.Stack {
    public readonly ecrRepository: ecr.IRepository;

    constructor(scope: cdk.Construct, id: string) {
        super(scope, id);

        const defaultImage = new ecrAssets.DockerImageAsset(this, `${id}-image`, {
            file: 'Dockerfile.ecr.default',
            directory: path.join(__dirname, '..', 'assets'),
        });

        this.ecrRepository = defaultImage.repository;
    }
}

Trying to deploy the stack results in an error with docker build:

myproject-test-cbs-ecr failed: Error: docker build --tag XXXXXXXXXXX.dkr.ecr.ap-southeast-1.amazonaws.com/cdk/myprojecttestcbsecrmyprojecttestcbsecrimage27470c3a:latest cdk.out/asset.3411e74ed8a24a7db917b49f0b1b9f5bb516d211f90359cb2872ff6b4f69c096 --file cdk.out/asset.3411e74ed8a24a7db917b49f0b1b9f5bb516d211f90359cb2872ff6b4f69c096/home/ric/projects/myproject-backend/myproject-cdk/assets/Dockerfile.ecr.default exited with error code 1

The --file parameter passed to docker build appended the absolute path of the docker file to the relative cdk.out path.

[iliapolo]

Hi @MrJonBirch. You wrote:

Note that the cdk.out folder does not contain any docker files but running the AwsStack project does not throw any errors.

I see that Program.cs eventually runs the following:

CloudAssembly appAssembly = app.Synth();

Did you expect the synth operation to fail? Or are you running cdk deploy and its not failing?
Note that the missing Dockerfile would cause an error upon image building, since synth doesn't attempt to build the image, it actually passes.

Follwing that, a good addition would be to validate the existence of the Dockerfile in the cdk.out directory, post synth. Noted :)

The reason the Dockerfile is missing is because of the .dockerignore entry you have in the project root (which is also the directory used in the definition of the asset).

Is there a specific reason you explicitly ignore Dockerfile? Wondering what the use-case is so we can better understand the scenario.

Finally, if you remove that entry from .dockerignore, everything should work properly.

[iliapolo]

Hi @ric-sapasap. You wrote:

The --file parameter passed to docker build appended the absolute path of the docker file to the relative cdk.out path.

This behavior implies that you are using a cdk.out directory that was synthesized with version 1.20.0 (or 1.21.0) of the @aws-cdk/aws-ecr-assets package. Is that indeed the case?

If so, to apply the fix, you need to re-synthesize your app using version 1.21.1. Don't hesitate to share if this process poses a problem or complexity, so that we can offer some workarounds.

If not, can you perhaps share a snippet from the cdk.out/manifest.json file? Specifically, look for the "packaging": "container-image" string, and take the encolsing object. It should looks something like this:

{
  "type": "aws:cdk:asset",
  "data": {
    "id": "79452e39b3d4fc2df04cf69f52a637fe5822ba5ef5b86733e471b5089017057d",
    "packaging": "container-image",
    "path": "asset.79452e39b3d4fc2df04cf69f52a637fe5822ba5ef5b86733e471b5089017057d",
    "sourceHash": "79452e39b3d4fc2df04cf69f52a637fe5822ba5ef5b86733e471b5089017057d",
    "imageNameParameter": "AssetParameters79452e39b3d4fc2df04cf69f52a637fe5822ba5ef5b86733e471b5089017057dImageName3509FBEE",
    "repositoryName": "cdk/{repo-name}",
    "file": "{some-path-to-a-docker-file}"
   }
}

[ric-sapasap]

Hi @iliapolo,

Upgrading @aws-cdk/aws-ecr-assets did solve the problem.

I thought upgrading the cdk cli also upgraded all the @aws-cdk packages.

Thank you for the help!

[MrJonBirch]

Hi @iliapolo

Yes, the error I was getting was at the deploy phase. I did not think it worth mentioning as I was able to trace it back to synth not putting the docker files in the asset folders.

CloudAssembly appAssembly = app.Synth(); is only in program.cs for the purpose of debugging with visual studio.

The reason I was expecting an error or thought it might be a bug was because if you point to a file that does not exists it fails so it seemed like the validation check was working but the copy or asset creation stage was pointing to the wrong directory.

As far as the .dockerignore, it is the default one provided by visual studio when you either create a project with docker support or add docker support to existing project.

You are right removing Line 18 "**/Dockerfile*" fixed my issue. However I am a little confused how. Does the ContainerImage.FromAsset process the .dockerignore file? I thought that would have been done at deploy time when the container is actually built. Is there somewhere in the documentation that expands on this?

Thank You

[iliapolo]

@ric-sapasap Excellent 👍

Glad the issue is resolved, we will be sure to make this upgrade process clearer going forward.

Happy to help :)

[iliapolo]

Hi @MrJonBirch. Let me clarify what exactly is happening.

Like you mentioned, during synth, the ContainerImage validates that the Dockerfile actually exists. After this validation passes, the cdk.out directory is created by copying the necessary files from the asset configuration. Its here where the .dockerignore file is processed, which caused the Dockerfile not to be copied into cdk.out.

During cdk deploy, the cdk cli does not use the original path to the Dockerfile, but rather assumes that the Dockerfile exists inside the cdk.out directory. This behavior is intentional, to make the cdk.out directory self-contained and portable across machines and environments.

You are right that this is actually an undocumented feature/behavior. I can point you to the following issues to get more context, and will make sure we add this to the documentation.

• feat(ecr-assets): Support .dockerignore (faster Docker builds) #4104
• ecr-assets: incomplete .dockerignore support #4450

Hope this helps, and thanks!

[MrJonBirch]

Hi @iliapolo

That clears things up beautifully. Thank you for all your help on this.

[iliapolo]

@MrJonBirch Perfect, my pleasure 😊

[thylux]

@iliapolo Been around the .dockerignore issue for 3 days.
Thanks a lot for the help!

[mimozell]

@iliapolo thanks for the explanation above. I haven't seen anything like it in the documentation. It helped me focus on cdk.out for the problem, but I wasn't able to solve mine with just that information. My .dockerignore file looks like this:

**
!build/libs/*.jar

so I don't want to copy anything else over to the docker context besides the jar, but it is actually not copied over to cdk.out.

I have tried different variations to get the jar copied over, but nothing worked besides moving the jar to the root of the project instead of having it in a directory which is not realistic. Because of this the COPY command in the Dockerfile fails.

Any ideas how I could get the jar copied over so docker build completes fine?