Re-enable cross-repository blob mounts

[briandealwis]

Re-enables mount/from for #169. The speed-up can be immense. Building and pushing an image with openjdk:8 as the base image to a new Docker Hub image takes more than 3 minutes, whereas with mount/from it takes about 8 seconds.

This implementation is is similar in spirit to #1024, but we do some additional checking to ensure we have permission to request a blob mount. Basically the Docker Registry Bearer Token includes the set of repositories and their granted permission. Private repositories won't be included if we don't have permission.

High-level summary of the changes:

• Authorization now parses the provided Bearer Token to maintain a list of repositories and their granted permissions. It provides a new method, canAccess(repository, access-scope). For non-bearer tokens, canAccess() returns true.
• RegistryClient.pushBlob() checks that the Authorization allows a cross-repository mount and ignores the mount point if not (this avoids the @chanseokoh's francium25 and francium25paran permission problem discovered previously)
• BuildConfiguration.newTargetImageRegistryClientFactory() configures the target RegistryClient to request permissions for the base and target repositories when base and target registries are the same, which is recorded in the RegistryEndpointRequestProperties object as a source image name
• RegistryAuthenticator requests permission for the source repository when set. A key change here is that if authorizing against the additional repository fails, we retry with only the target repository. NOTE: this doesn't seem to happen in practice with Docker Hub¸, and hence the need to check the resulting of the Docker Registry Bearer Token's access grants instead

I debated whether I should provide a property to disable blob mounts or if I should rework RegistryClient.pushBlob() to retry if the blob-mount fails. The Registry Spec says

If a mount fails due to invalid repository or digest arguments, the registry will fall back to the standard upload behavior and return a 202 Accepted with the upload URL in the Location header:

Fixes #169

[chanseokoh]

I think this looks good.

[chanseokoh]

Worth explaining this in the Javadoc too.

[loosebazooka]

Yeah I don't understand why "null" is valid

[briandealwis]

The other non-Docker Container Registry Token methods don't provide this information.

[chanseokoh]

I think the method name and the Javadoc can still be confusing. I think we just need to clarify (and justify). For example, I can create an Authorization instance with fromBasicCredentials("my-username", "my-passwd"). Then canAccess("any repository", "pull or push or anything") will always return true. I feel this is not intuitive.

[briandealwis]

What about making this specific to the blob-mount: canAttemptBlobMount(repository)

[chanseokoh]

That technically works. Man, naming is hard. I see depending on the name, we may want to relocate this class from the .http package. (Relocation can be done later.) This class is meant for holding the very string value for the Authorization HTTP header (i.e., for Authorization: <some schema> <token value>) as explained in the class Javadoc, hence was holding the two fields schema and token. I thought repositoryGrants was OK, because it is actually information in token. But if we incorporate the notions of BLOb or cross-repo mount, it may make sense to get this out of .http.

That said, I don't have a strong opinion about what the name should be. Maybe the logic for canAttemptBlobMount can be outside of this method, I don't know.

[briandealwis]

Hmm good point. I'll move this into RegistryClient — it's only called from pushBlobStep.

[chanseokoh]

Overall, LGTM.

[chanseokoh]

I think the method name and the Javadoc can still be confusing. I think we just need to clarify (and justify). For example, I can create an Authorization instance with fromBasicCredentials("my-username", "my-passwd"). Then canAccess("any repository", "pull or push or anything") will always return true. I feel this is not intuitive.

[briandealwis]

Moved the Docker Registry Bearer Token related code into RegistryClient, though I kept the test separate as DockerRegistryBearerTokenTest.

[loosebazooka]

Curious as to why we are "re-enabling", did it work in the past?

[loosebazooka]

Do you think we should hide this behind a flag for now?

[briandealwis]

We had partially implemented it in 0.9.8 but it broke some people so we backed it out for 0.9.9: #808 #1654

I will add a flag to disable it. The wins from having it enabled are immense (8s vs 3:34m).

[loosebazooka]

okay, so makes sense to enable by default

[briandealwis]

I somehow fat-fingered the commit button before I could edit that mess of a commit message. Blech.

[loosebazooka]

Life goes on