Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a
aws
and alocal
decorator to the tests so that tests now run on the local datasets.By default, the
aws
is deactivated andlocal
is activated andslow
is deactivated, so that only 1 test per dataset runs on circle ci.When local is activated all folders in
./datasets
are tested.Important When adding a dataset, we should no longer upload it to AWS. The steps are:
datasets/README.md
Currently we have 49 functional datasets in our code base.
We have 6 datasets "under-construction" that don't pass the tests - so I put them in a folder "datasets_under_construction" - it would be nice to open a PR to fix them and put them in the
datasets
folder.Important when running tests locally, the datasets are cached so to rerun them delete your local cache via:
rm -r ~/.cache/huggingface/datasets/*
@thomwolf @mariamabarham @lhoestq