Skip to content

Issues: huggingface/evaluate

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
148 Open 134 Closed
148 Open 134 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Feature: type hints for metric inputs enhancement New feature or request
#9 by lvwerra was closed May 11, 2022
2
[request] make load_metric api intutive
#110 by stas00 was closed Aug 3, 2022
2
How to Add New Metrics Guide enhancement New feature or request
#112 by ncoop57 was closed Jun 14, 2022
2
Trying to use metric.compute but get OSError
#113 by hyusterr was closed Jun 14, 2022
7
the meteor metric seems not consist with the official version bug Something isn't working
#115 by jianguda was closed Jul 7, 2022
2
Only retain relevant statistics in certain metrics enhancement New feature or request
#116 by ZhaofengWu was closed Aug 3, 2022
1
Rouge and Meteor for multiple references bug Something isn't working
#118 by avinashsai was closed Aug 5, 2022
1
8
Calling existing metrics from other metrics enhancement New feature or request
#119 by sashavor was closed Aug 3, 2022
@lvwerra @lhoestq
2
Feature: add community metrics enhancement New feature or request
#4 by lvwerra was closed May 11, 2022
6
Feature: enrich MetricsInfo with more meta information enhancement New feature or request
#5 by lvwerra was closed May 17, 2022
5 tasks
4
Feature: push_to_hub function for metrics. enhancement New feature or request
#6 by lvwerra was closed May 31, 2022
@NimaBoscarino
5
Feature: save metrics results locally enhancement New feature or request
#7 by lvwerra was closed May 23, 2022
1
6
Seq2Seq Metrics QOL: Bleu, Rouge enhancement New feature or request
#107 by sshleifer was closed Jun 3, 2022
5
Feature: compose multiple metrics into single object enhancement New feature or request
#8 by lvwerra was closed Jul 8, 2022
5
Defining and standardizing metric input structures enhancement New feature or request
#15 by sashavor was closed May 26, 2022
@lvwerra @sashavor
19
Connect evaluate with the Data Measurements Tool enhancement New feature or request
#18 by sashavor was closed May 26, 2022
3
Docs: Add README documentation Improvements or additions to documentation
#21 by lvwerra was closed May 11, 2022
@sashavor
Docs: add documentation page documentation Improvements or additions to documentation
#22 by lvwerra was closed May 27, 2022
3 tasks
@sashavor
1
Add Evaluator class to easily evaluate a combination of (model, dataset, metric) enhancement New feature or request
#23 by lvwerra was closed May 30, 2022
1
@ola13
12
Setup script action to push "canonical" metrics to hub enhancement New feature or request
#24 by lvwerra was closed May 20, 2022
1
Load metrics from Hub instead of GitHub repo enhancement New feature or request
#25 by lvwerra was closed May 11, 2022
1
Refactor for loading multiple evaluation categories
#38 by lvwerra was closed May 17, 2022
@lvwerra
2
BLEURT has no README.md documentation Improvements or additions to documentation
#43 by lvwerra was closed May 18, 2022
1
Issue with Perplexity metric bug Something isn't working
#51 by prabhakar267 was closed Jun 8, 2022
@TristanThrush
4
How to customize my own evaluator and metrics?
#611 by Kami-chanw was closed Aug 15, 2024
ProTip! Add no:assignee to see everything that’s not assigned.