Add summarization template

[lewtun]

This PR adds a task template for text summarization. As far as I can tell, we do not need to distinguish between "extractive" or "abstractive" summarization - both can be handled with this template.

Usage:

from datasets import load_dataset
from datasets.tasks import Summarization

ds = load_dataset("xsum", split="train")
# Dataset({
#     features: ['document', 'summary', 'id'],
#     num_rows: 204045
# })

summarization = Summarization(text_column="document", summary_column="summary")
ds.prepare_for_task(summarization)
# Dataset({
#     features: ['text', 'summary'],
#     num_rows: 204045
# })

[lewtun]

it seems that whatever magic @lhoestq did with windows now allows me to avoid using the annoying context manager that we have in many unit tests:

with tempfile.TemporaryDirectory() as tmp_dir, Dataset.from_dict(data) as dset:
    with self._to(in_memory, tmp_dir, dset) as dset:
        # do operations on `dset`

i never really understood why we had to do this so please correct me if it's actually needed 😄

[lhoestq]

The with statement are used to properly close open arrow files on windows (it really doesn't like keeping things open).
What you mention though is the call to self._to call that moves a dataset on disk or in memory, which can be useful in some tests. Here you don't need to do this indeed.

Since the in_memory parameter is not used in this test, you can move this test outside of the BaseDatasetTest class and have it as a regular pytest test case.
This way it won't be run twice (once for in_memory=True and once for in_memory=False)

[SBrandeis]

LGTM 🚀

[lhoestq]

Nice thanks !
Could you just move the test outside of the BaseDatasetTest class please ? Otherwise it will unnecessarily be run twice.

[lhoestq]

The with statement are used to properly close open arrow files on windows (it really doesn't like keeping things open).
What you mention though is the call to self._to call that moves a dataset on disk or in memory, which can be useful in some tests. Here you don't need to do this indeed.

Since the in_memory parameter is not used in this test, you can move this test outside of the BaseDatasetTest class and have it as a regular pytest test case.
This way it won't be run twice (once for in_memory=True and once for in_memory=False)

[lewtun]

Nice thanks !
Could you just move the test outside of the BaseDatasetTest class please ? Otherwise it will unnecessarily be run twice.

sure, on it! thanks for the explanations about the self._to method :)

[lewtun]

@lhoestq i've moved all the task template tests outside of BaseDatasetTest and collected them in their dedicated test case. (at some point i'll revisit this so we can just use pytest natively, but the PR is already getting out-of-scope :))

[lhoestq]

Looks all good, thanks :)