Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for potentially unused test data files and check file sizes only for changed files #4077

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

bernt-matthias
Copy link
Contributor

@bernt-matthias bernt-matthias commented Oct 13, 2021

Raise linter error-level to warn. This will require galaxyproject/planemo-ci-action#23 and a new release of the action.

The check for unused test data is clearly not fool proof, e.g. index files. But maybe it helps.

The change in the file size job is to check only changed files. Seems to work, since find tools/anndata -type f -size +500k would return

tools/anndata/test-data/tl.diffmap.h5ad
tools/anndata/test-data/tl.draw_graph.h5ad
tools/anndata/test-data/tl.umap.h5ad
tools/anndata/test-data/pp.neighbors_umap_euclidean.recipe_weinreb17.paul15_subsample.h5ad
tools/anndata/test-data/tl.paga.neighbors_gauss_braycurtis.recipe_weinreb17.paul15_subsample.h5ad

But since none of these files was changed they are not reported.

FOR CONTRIBUTOR:

  • - I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
  • - License permits unrestricted use (educational + commercial)
  • - This PR adds a new tool or tool collection
  • - This PR updates an existing tool or tool collection
  • - This PR does something else (explain below)

@bernt-matthias bernt-matthias changed the title check for potentially unused test data files Check for potentially unused test data files and check file sizes only for changed files Oct 19, 2021
as pointed out here: galaxyproject#3973 (comment)

reducing the size of test data does not decrease the repo size
if the files are already in. so we should only check for the sizes
of changed files and be strict with them

this also splits the job into two: one for the file size check
and one for finding potentially unused test data files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant