Dependabot adding index to Pipfile.lock causes issue in newest version of pipenv

[juanitosvq]

Is there an existing issue for this?

• I have searched the existing issues

Package ecosystem

pipenv

Package manager version

No response

Language version

python 3.9

Manifest location and content before the Dependabot update

No response

dependabot.yml content

We have a private repo where we pull all our dependencies from:

version: 2
registries:
  python-codeartifact:
    type: python-index
    url: https://REDACTED.d.codeartifact.us-east-1.amazonaws.com/pypi/REDACTED/simple/
    username: aws
    password: ${{ secrets.CODE_ARTIFACT_TOKEN }}
    replaces-base: true
updates:
  - package-ecosystem: 'pip'
    directory: '/'
    insecure-external-code-execution: allow
    open-pull-requests-limit: 0
    schedule:
      interval: 'weekly'
    registries:
      - python-codeartifact
    reviewers:
      - org/security-team

Updated dependency

No response

What you expected to see, versus what you actually saw

This is a copy of an issue that was closed recently.

Dependabot created a PR fixing some vulnerability, and as per #7744, we saw a new behaviour where it included an index called dependabot-inserted-index-0 in some of the transitive dependencies in our Pipfile.lock, for example:

 "freezegun": {
            "hashes": [
                "sha256:cd22d1ba06941384410cd967d8a99d5ae2442f57dfafeff2fda5de8dc5c05446",
                "sha256:ea1b963b993cb9ea195adbd893a48d573fda951b0da64f60883d7e988b606c9f"
            ],
            "index": "dependabot-inserted-index-0",
            "version": "==1.2.2"
        },

It added that index to packages that were updated, but also to some that were not updated. It didn't add the index to all the packages. See diff below:

Our build pipelines were able to sync the dependencies when they were using an old version of pipenv, for example v2022.1.8 (we haven't found which version is the newest that still works):

Successfully installed pipenv-2022.1.8

..........................................................................................................

·✔ Successfully created virtual environment!
--
179 | Virtualenv location: /root/.local/share/virtualenvs/authorizer-0svWqLcN
180 | Installing dependencies from Pipfile.lock (923e0b)...
181 | To activate this project's virtualenv, run pipenv shell.
182 | Alternatively, run a command inside the virtualenv with pipenv run.
183 | All dependencies are now up-to-date!

However, when we updated to the latest version of pipenv (v2023.8.28), we encountered this error:

Successfully installed pipenv-2023.8.28

..........................................................................................................

✔ Successfully created virtual environment!
--
123 | Virtualenv location: /root/.local/share/virtualenvs/authorizer-jszj7Vks
124 | Installing dependencies from Pipfile.lock (923e0b)...
125 | Unable to find dependabot-inserted-index-0 in sources, please check
126 | dependencies: ['boto3==1.26.126
127 | --hash=sha256:a0a049d16dabf559bd86d4622a985549154c6918f779f373fb7588bd3cf1272d
128 | --hash=sha256:ea7dc518399106ee300b867596c92d2b7b498ee7721db935915cdd53339a9f92',
129 | 'cachetools==5.3.0
130 | --hash=sha256:13dfddc7b8df938c21a940dfa6557ce6e94a2f1cdfa58eb90c805721d58f2c14
131 | --hash=sha256:429e1a1e845c008ea6c85aa35d4b98b65d6a9763eeef3e37e92728a12d1de9d4',
132 | 'evertz-io-observability==3.0.0
133 | --hash=sha256:aa088284d216affc085b18dfb1ea0c68db05e49d1f7708287c472ba46c47cca1
134 | --hash=sha256:f8221c9d487792a8317872c03ad10580a4a0420044e3b8f1bb47df1ace99173a',
135 | 'opentelemetry-api==1.16.0
136 | --hash=sha256:4b0e895a3b1f5e1908043ebe492d33e33f9ccdbe6d02d3994c2f8721a63ddddb
137 | --hash=sha256:79e8f0cf88dbdd36b6abf175d2092af1efcaa2e71552d0d2b3b181a9707bf4bc',
138 | 'opentelemetry-sdk==1.16.0
139 | --hash=sha256:15f03915eec4839f885a5e6ed959cde59b8690c8c012d07c95b4b138c98dc43f
140 | --hash=sha256:4d3bb91e9e209dbeea773b5565d901da4f76a29bf9dbc1c9500be3cabb239a4e',
141 | 'pytest==7.2.1
142 | --hash=sha256:c7c6ca206e93355074ae32f7403e8ea12163b1163c976fee7d4d84027c162be5
143 | --hash=sha256:d45e0952f3727241918b8fd0f376f5ff6b301cc0777c6f9a556935c92d8a7d42',
144 | 'python-jose==3.3.0
145 | --hash=sha256:55779b5e6ad599c6336191246e95eb2293a9ddebd555f796a65f838f07e5d78a
146 | --hash=sha256:9b1376b023f8b298536eedd47ae1089bcdb848f1535ab30555cd92002d78923a',
147 | 'requests==2.31.0
148 | --hash=sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f
149 | --hash=sha256:942c5a758f98d790eaed1a29cb6eefc7ffb0d1cf7af05c3d2791656dbd6ad1e1']

We are trying to reproduce locally using the Dependabot CLI as requested in #7936 (comment), but we are unable to authenticate the dependabot CLI against our private CodeArtifact repository: dependabot/cli#196

Reopening this to hopefully get assistance reproducing this issue with the CLI.

Thanks!

Native package manager behavior

After manually updating the dependencies with the latest version of pipenv, that index added by dependabot disappeared and our build pipelines were able to sync again. This is the change after doing a pipenv update manually:

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

[averypelle]

Also experiencing a similar issue using a private codeartifact + latest version of Dependabot

[deivid-rodriguez]

The CLI issue has been fixed, so this may be easier to reproduce now!

[juanitosvq]

This is a duplicate of #7936, I couldn't reopen the original issue so I created this one. Closing, keeping the other one that has more details.