Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add regression testing and reorganize auth precedence levels httpOCIRegistry #482

Merged
merged 18 commits into from
Apr 14, 2023

Conversation

joshspicer
Copy link
Member

@joshspicer joshspicer commented Apr 5, 2023

related: #473, #468

Changes here are primarily around re-structuring the different auth strategies to rely more on the standard docker auth strategies (docker config file and docker credential helper), and less on the registry-specific strategies that previously held higher precedence.

The precedence order is now (encoded in getCredential(...):

  • parsed out of a special DEVCONTAINERS_OCI_AUTH environment variable
  • Read from a docker credential helper indicated in config
  • Read from a docker cred store indicated in config (https://docs.docker.com/engine/reference/commandline/login/#credentials-store)
  • Read from a docker config file (flat file with base64 encoded credentials)
  • Read from the platform's default credential helper
  • Crafted from the GITHUB_TOKEN environment variable

Two new minor changes have also been added:


This change also introduces significant changes to the registryCompatibilityOCI.test.ts test, introducing a way to quickly add "testPlans" (new registries) with different forms of provided auth. This PR introduces four "testPlans", testing both AzureCR and GHCR both anonymous and with some form of auth.

Since we want folks to be able to fork this repo and run all the tests without failure, the tests will automatically skip if there is a set useAuthStrategy and no accompanying authStrategyKey (environment variable, containing a credential if needed for the test and parsed account according to the useAuthStrategy enum). This lets us add in the necessary secrets to our GitHub Action CI tests without introducing flakey tests to other contributors who may have forks of this repo. Mocha has a way to mark tests as "skipped", which is what we use here.

const registryCompatibilityTestPlan: TestPlan[] = [
	{
		name: 'Anonymous access of Azure Container Registry',
		configName: 'azure-anonymous',
		testFeatureId: 'devcontainercli.azurecr.io/features/color',
	},
	{
		name: 'Anonymous access of GHCR',
		configName: 'github-anonymous',
		testFeatureId: 'ghcr.io/devcontainers/feature-starter/color',
	},
	// https://learn.microsoft.com/en-us/azure/container-registry/container-registry-repository-scoped-permissions
	{
		name: 'Authenticated access of Azure Container Registry with registry scoped token',
		configName: 'azure-registry-scoped',
		testFeatureId: 'privatedevcontainercli.azurecr.io/features/rabbit',
		useAuthStrategy: AuthStrategy.DockerConfigAuthFile,
		authStrategyKey: 'FEATURES_TEST__AZURE_REGISTRY_SCOPED_CREDENTIAL',
		testCommand: 'rabbit',
		testCommandResult: /rabbit-is-the-best-animal/,
	},
	// Via GHCR visibility settings, this repo's GitHub Actions CI should be able to access this Feature via its GITHUB_TOKEN.
	{
		name: 'Private access of GHCR via an environment GITHUB_TOKEN',
		configName: 'github-private',
		testFeatureId: 'ghcr.io/devcontainers/private-feature-set-for-tests/color',
		useAuthStrategy: AuthStrategy.GitHubToken,
		authStrategyKey: 'RUNNING_IN_DEVCONTAINERS_CLI_REPO_CI'
	}
];

In CI on this repository the above authStrategyKey variables are set, so no tests should be skipped (as seen here)

@joshspicer joshspicer changed the title Reorganize auth precedence levels in httpOCIRegistry Add regression testing and reorganize auth precedence levels for httpOCIRegistry Apr 10, 2023
@joshspicer joshspicer self-assigned this Apr 10, 2023
@joshspicer joshspicer changed the title Add regression testing and reorganize auth precedence levels for httpOCIRegistry Add regression testing, reorganize auth precedence levels, and fix behavior bugs for httpOCIRegistry Apr 10, 2023
@joshspicer joshspicer changed the title Add regression testing, reorganize auth precedence levels, and fix behavior bugs for httpOCIRegistry Add regression testing and reorganize auth precedence levels httpOCIRegistry Apr 10, 2023
@joshspicer joshspicer marked this pull request as ready for review April 10, 2023 20:04
@joshspicer joshspicer requested a review from a team as a code owner April 10, 2023 20:04
chrmarti
chrmarti previously approved these changes Apr 14, 2023
src/spec-configuration/httpOCIRegistry.ts Outdated Show resolved Hide resolved
@joshspicer joshspicer merged commit f6bde3e into main Apr 14, 2023
@joshspicer joshspicer deleted the joshspicer/reorganize-auth branch April 14, 2023 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants