PyOCI

Publish and download (private) python packages using an OCI registry for storage.

Why PyOCI

As part of my job we create private python packages used in the main application.
To not have to rely on yet-another-cloud-provider, instead I built PyOCI, making ghcr.io act like a python index.
This also completely removed the need for separate access management as now GitHub Packages access control applies.

Most subscriptions with cloud providers include an OCI (docker image) registry where private containers can be published and distributed from.

PyOCI allows using any (private) OCI registry as a python package index, as long as it implements the OCI distribution specification. It acts as a proxy between pip and the OCI registry.

An instance of PyOCI is available at https://pyoci.allexveldman.nl, to use this proxy, please see the Getting started.

Tested registries:

• ghcr.io

Published packages will show up in the OCI registry UI:

Getting started

To install a package with pip using PyOCI:

pip install --index-url="http://<username>:<password>@<pyoci-url>/<OCI-registry-url>/<namespace>/" <package-name>

• <pyoci-url>: https://pyoci.allexveldman.nl
• <OCI-registry-url>: URL of the OCI registry to use.
• <namespace>: namespace within the registry, for most registries this is the username or organization name.

Example installing package hello-world from organization allexveldman using ghcr.io as the registry:

pip install --index-url="https://$GITHUB_USER:$GITHUB_TOKEN@pyoci.allexveldman.nl/ghcr.io/allexveldman/" hello-world

Warning

If the package contains dependencies from regular pypi, these will not resolve.

Pip does not have a proper way of indicating you only want to resolve <package-name> through PyOCI and it's dependencies through pypi. Poetry does provide you with a way to do this.

For more examples, including how to publish a package, see the examples.

Authentication

Pip's Basic authentication is forwarded as-is to the target registry as part of the token authentication flow.

Changing a package

PyOCI will refuse to upload a package file if the package name, version and architecture already exist. To update an existing file, delete it first and re-publish it.

Deleting a package

PyOCI does not provide a way to delete a package, instead you can use the OCI registry provided methods to delete your package.

Renovate + ghcr.io

As PyOCI acts as a private pypi index, Renovate needs to be configured to use credentials for your private packages. (https://docs.renovatebot.com/getting-started/private-packages/) To prevent having to check-in encrypted secrets you can:

1. Self-host renovate as a github workflow
2. Set package: read permissions for the workflow
3. Pass the GITHUB_TOKEN as an environment variable to Renovate
4. Add a hostRule for the Renovate runner to apply basic auth for pyoci using the environment variable
5. In the package settings of the private package give the repository running renovate read access.

Note that at the time of writing, GitHub App Tokens can't be granted read:package permissions, this is why you'll need to use the GITHUB_TOKEN.

.github/workflows/renovate.yaml

...
concurrency:
  group: Renovate

# Allow the GITHUB_TOKEN to read packages
permissions:
  contents: read
  packages: read

jobs:
  renovate:
    ...
      - name: Self-hosted Renovate
        uses: renovatebot/github-action@v40.2.4
        with:
          configurationFile: config.js
          token: '${{ steps.get_token.outputs.token }}'
        env:
          RENOVATE_PYOCI_USER: pyocibot
          RENOVATE_PYOCI_TOKEN: ${{ secrets.GITHUB_TOKEN }}

config.js

module.exports = {
  ...
  hostRules: [
    {
      matchHost: "pyoci.allexveldman.nl",
      hostType: "pypi",
      username: process.env.RENOVATE_PYOCI_USER,
      password: process.env.RENOVATE_PYOCI_TOKEN
    },
  ],
};