Feature: enrich MetricsInfo with more meta information

[lvwerra]

Add more meta information to the MetricsInfo. The existing fields are listed here:

evaluate/src/evaluate/metric.py

Lines 76 to 135 in f884495

	class MetricInfoMixin:
	"""This base class exposes some attributes of MetricInfo
	at the base level of the Metric for easy access.
	"""
	def __init__(self, info: MetricInfo):
	self._metric_info = info
	@property
	def info(self):
	""":class:`datasets.MetricInfo` object containing all the metadata in the metric."""
	return self._metric_info
	@property
	def name(self) -> str:
	return self._metric_info.metric_name
	@property
	def experiment_id(self) -> Optional[str]:
	return self._metric_info.experiment_id
	@property
	def description(self) -> str:
	return self._metric_info.description
	@property
	def citation(self) -> str:
	return self._metric_info.citation
	@property
	def features(self) -> Features:
	return self._metric_info.features
	@property
	def inputs_description(self) -> str:
	return self._metric_info.inputs_description
	@property
	def homepage(self) -> Optional[str]:
	return self._metric_info.homepage
	@property
	def license(self) -> str:
	return self._metric_info.license
	@property
	def codebase_urls(self) -> Optional[List[str]]:
	return self._metric_info.codebase_urls
	@property
	def reference_urls(self) -> Optional[List[str]]:
	return self._metric_info.reference_urls
	@property
	def streamable(self) -> bool:
	return self._metric_info.streamable
	@property
	def format(self) -> Optional[str]:
	return self._metric_info.format

Things that could be useful to add:

• requirements: we could warn and instruct on what is required if not already installed.
• test_cases: a list of input-output pairs to test whether the metrics works as expected. this could also be used to populate a metric widget with examples where the user can preselect a few choices before testing a custom input. Could also trigger automatic tests of metrics.
• score_range: what are the possible values for the metric score. E.g. [0, 1] for accuracy or [-1, 1] for cosine similarity.
• score_order: is larger or smaller better. Could be useful for automatic ordering of several results.
• task_id: a list of tasks this metric is applicable (e.g. "binary-classification"

cc @sashavor @lhoestq

[lhoestq]

I would put the requirements in an actual requirements.txt file next to the metric script. Same for test_cases, you can have a dedicated test python file I think (we'll have to see what we add in there, compared to the doctests we already have though).

task_ids can be in the YAML part of the readme, so that moon-landing can parse it and index it on the website.

score_range and score_order are nice additions too. But not sure if it would be best in YAML or in the MetricInfo (the MetricInfo can also parse the YAML if we want)

[sashavor]

also, can we add linked_dataset to reflect cases when a metric is limited to a specific dataset?

[julien-c]

@sashavor For this you could just piggy-back on this existing Space feature maybe:

[lvwerra]

We now have requirements as well as test cases in the template #14 and can add datasets in the yaml of the README.md. Score ranges are also described in the README.md template. Closing this for now.