Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed pypi scraping issue causing issue #598 #605

Merged
merged 3 commits into from
Jul 12, 2022
Merged

Conversation

lgan-czi
Copy link
Contributor

@lgan-czi lgan-czi commented Jul 12, 2022

Background:

We were unable to update the napari-hub due to #598, which happened due to pypi deprecating the "releases" field when querying with a version, causing our code to fail.

Changes:

I prevented use of versions by setting version=None whenever get_plugin_pypi_metadata gets called. This fixes issue #598 because not asking for a version will allow us to access past releases while still retaining the same information as if we ask for the latest version. This should work because we query for the latest version anyways when gathering metadata from pypi.

Testing:

Changes were tested using unit tests and then tested in the dev environment to see if the plugins will populate, which they did.
Screen Shot 2022-07-12 at 11 52 46 AM

@lgan-czi
Copy link
Contributor Author

@potating-potato @richaagarwal pinging for visibility

@@ -148,7 +148,7 @@ def build_plugin_metadata(plugin: str, version: str) -> Tuple[str, dict]:
cached_plugin = get_cache(f'cache/{plugin}/{version}.json')
if cached_plugin:
return plugin, cached_plugin
metadata = get_plugin_pypi_metadata(plugin, version)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is minor, but is there a reason not to pass version = None here and not make any of the following changes? (It could be nice to support versions depending on what we decide for follow-up work, but it'd be easy to put back so it's definitely a minor design decision for now).

Copy link
Contributor Author

@lgan-czi lgan-czi Jul 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, that would work too, and it's probably a better way to do this to minimize side effects.

@lgan-czi
Copy link
Contributor Author

Yeah it looks like the cache populates with version=None too, should be able to merge whenever

Copy link
Collaborator

@richaagarwal richaagarwal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your quick work on this!

@lgan-czi lgan-czi merged commit 79ca5f1 into main Jul 12, 2022
@lgan-czi lgan-czi deleted the dev-fix-pypi-scraping branch July 12, 2022 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants