Add MetaShift dataset

[osanseviero]

Adding a Dataset

• Name: MetaShift
• Description: collection of 12,868 sets of natural images across 410 classes-
• Paper: https://arxiv.org/abs/2202.06523v1
• Data: https://github.com/weixin-liang/metashift

Instructions to add a new dataset can be found here.

[dnaveenr]

I would like to take this up and give it a shot. Any image specific - dataset guidelines to keep in mind ? Thank you.

[dnaveenr]

#self-assign

[dnaveenr]

I've started working on adding this dataset. I require some inputs on the following :

Ref for the initial draft here

1. The dataset does not have a typical - train/test/val split. What do we do for the _split_generators() function ? How do we go about this ?
2. This dataset builds on the Visual Genome dataset, using a metadata file. The dataset is generated using generate_full_MetaShift.py script. By default, the authors choose to generate the dataset only for a SELECTED_CLASSES. The following script is used :
   Code : https://github.com/Weixin-Liang/MetaShift/blob/main/dataset/generate_full_MetaShift.py
   Info : https://metashift.readthedocs.io/en/latest/sub_pages/download_MetaShift.html#generate-the-full-metashift-dataset
   Can I just copy over the required functions into the metashift.py to generate the dataset ?
3. How do we complete the _generate_examples for this dataset ?

The user has the ability to use default selected classes, get the complete dataset or add more specific additional classes. I think config would be a good option here.

Inputs, suggestions would be helpful. Thank you.

[osanseviero]

I think @mariosasko and @lhoestq should be able to help here 😄

[lhoestq]

Hi ! Thanks for adding this dataset :) Let me answer your questions:

1. in this case you can put everything in the "train" split
2. Yes you can copy the script (provided you also include the MIT license of the code in the file header for example). Though we ideally try to not create new directories nor files when generating dataset, so if possible this script should be adapted to not create the file structure they mentioned, but instead yield the images one by one in _generate_examples. Let me know if you think this is feasible
3. see point 2 haha

The user has the ability to use default selected classes, get the complete dataset or add more specific additional classes. I think config would be a good option here.

Yup ! We can also define a selected_classes parameter such that users can do

load_dataset("metashift", selected_classes=["cat", "dog", ...])

[dnaveenr]

Great. This is helpful. Thanks @lhoestq .
Regarding Point 2, I'll try using yield instead of creating the directories and see if its feasible. selected_classes config sounds good.

[mariosasko]

Closed via #3900