Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added --oci-layout-path flag to save image in OCI layout. #744

Merged
merged 4 commits into from
Sep 9, 2019

Conversation

chhsia0
Copy link

@chhsia0 chhsia0 commented Aug 20, 2019

Fixed #296.

The output manifests may have application/vnd.docker.distribution.manifest.v2+json
as their media types instead of application/vnd.oci.image.manifest.v1+json.

@chhsia0
Copy link
Author

chhsia0 commented Aug 20, 2019

@priyawadhwa Could you take a look? Thanks!

Copy link
Collaborator

@priyawadhwa priyawadhwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chhsia0 thanks for working on this! A couple suggestions:

  1. Could we name the flag --oci-layout-path
  2. Could you add a unit test for this?

@chhsia0
Copy link
Author

chhsia0 commented Aug 21, 2019

Sure! Is there a reference unit test I can look at as a sample? I tried to find if there's a unit test for --digest-file but haven't found one.

@chhsia0 chhsia0 changed the title Added --layout-path flag to save image in OCI layout. Added --oci-layout-path flag to save image in OCI layout. Aug 21, 2019
@chhsia0
Copy link
Author

chhsia0 commented Aug 21, 2019

@priyawadhwa Done PTAL!

@chhsia0
Copy link
Author

chhsia0 commented Aug 22, 2019

Hmm seems this is not enough for using Kaniko in Tekton, since Tekton exports the digest of the image index itself instead of the image inside the manifest. Let me chat with the folks there to see what's the appropriate solution for this.

@priyawadhwa
Copy link
Collaborator

Sounds good!

@chhsia0 chhsia0 force-pushed the layout-path branch 2 times, most recently from 2f64f59 to 451fa87 Compare August 23, 2019 04:06
chhsia0 pushed a commit to chhsia0/pipeline that referenced this pull request Aug 23, 2019
…mage.

Currently the image digest exporter does not implemented the behavior described
in the resources doc, which says "if there are multiple versions of the image,
the latest will be used." Instead, it reports the digest of `index.json`, which
is an image index. This behavior introduces a usability issue: one of the major
public container registry --- dockerhub --- does not support OCI image indices,
and there are very few tools (if any) that support converting OCI image indices
to docker manifest lists. Skopeo currently only support pushing an OCI image
index that contain only one image. If the index has more than one images, it
requires the user to specify one:
containers/skopeo#107
containers/image#400

Essentially, these limitations make the image digest exporter useless. To make
this feature useful, the exporter could instead implement the following behavior:

1. If there is only one image in `index.json`, report the image's digest.

2. If there are multiple images, report the digest of the full index.

The advantage of this behavior is that, we can immediately use it (in conjunction
of GoogleContainerTools/kaniko#744), yet if multi-image
manifests are more widely supported, the image digest exporter can still support
that without any modification.
chhsia0 pushed a commit to chhsia0/pipeline that referenced this pull request Aug 23, 2019
…ne image.

Currently the image digest exporter does not implemented the behavior described
in the resources doc, which says "if there are multiple versions of the image,
the latest will be used." Instead, it reports the digest of `index.json`, which
is an image index. This behavior introduces a usability issue: one of the major
public container registry --- dockerhub --- does not support OCI image indices,
and there are very few tools (if any) that support converting OCI image indices
to docker manifest lists. Skopeo currently only support pushing an OCI image
index that contain only one image. If the index has more than one images, it
requires the user to specify one:
containers/skopeo#107
containers/image#400

Essentially, these limitations make the image digest exporter useless. To make
this feature useful, the exporter could instead implement the following behavior:

1. If there is only one image in `index.json`, report the image's digest.

2. If there are multiple images, report the digest of the full index.

The advantage of this behavior is that, we can immediately use it (in conjunction
of GoogleContainerTools/kaniko#744), yet if multi-image
manifests are more widely supported, the image digest exporter can still support
that without any modification.
@tejal29
Copy link
Member

tejal29 commented Aug 23, 2019

@chhsia0 Can you please rebase?

chhsia0 added 3 commits August 24, 2019 01:04
Fixed GoogleContainerTools#296.

The output manifests may have `application/vnd.docker.distribution.manifest.v2+json`
as their media types instead of `application/vnd.oci.image.manifest.v1+json`.
@chhsia0 chhsia0 force-pushed the layout-path branch 2 times, most recently from 53fe819 to ba9b616 Compare August 24, 2019 09:29
@chhsia0
Copy link
Author

chhsia0 commented Aug 25, 2019

Done.

tekton-robot pushed a commit to tektoncd/pipeline that referenced this pull request Sep 6, 2019
…ne image.

Currently the image digest exporter does not implemented the behavior described
in the resources doc, which says "if there are multiple versions of the image,
the latest will be used." Instead, it reports the digest of `index.json`, which
is an image index. This behavior introduces a usability issue: one of the major
public container registry --- dockerhub --- does not support OCI image indices,
and there are very few tools (if any) that support converting OCI image indices
to docker manifest lists. Skopeo currently only support pushing an OCI image
index that contain only one image. If the index has more than one images, it
requires the user to specify one:
containers/skopeo#107
containers/image#400

Essentially, these limitations make the image digest exporter useless. To make
this feature useful, the exporter could instead implement the following behavior:

1. If there is only one image in `index.json`, report the image's digest.

2. If there are multiple images, report the digest of the full index.

The advantage of this behavior is that, we can immediately use it (in conjunction
of GoogleContainerTools/kaniko#744), yet if multi-image
manifests are more widely supported, the image digest exporter can still support
that without any modification.
@chhsia0
Copy link
Author

chhsia0 commented Sep 6, 2019

The related PR in the Tekton project just got merged: tektoncd/pipeline#1237
Now we can use this new flag to make Tekton tasks report digests of images built by Kaniko.
@priyawadhwa Can you take another look at this PR? Thank you very much!

Copy link
Collaborator

@priyawadhwa priyawadhwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@priyawadhwa priyawadhwa merged commit 38c05ba into GoogleContainerTools:master Sep 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add flag to save image in OCI image layout form
4 participants