Skip to content

Issues: huggingface/evaluate

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
148 Open 134 Closed
148 Open 134 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Problems during run initial step
#590 opened May 7, 2024 by simplelifetime updated May 15, 2024
12
Module 'accuracy' doesn't exist on the Hugging Face Hub either.
#456 opened May 6, 2023 by AceCHQ updated May 11, 2024
17
Evaluate SyntaxError: invalid syntax
#460 opened May 16, 2023 by NLBONU1 updated May 11, 2024
1
MSE not accepting (n_samples, n_ouptuts) despite docs stating so
#550 opened Feb 18, 2024 by bauersimon updated May 5, 2024
2
[Question]Shall we adding a faster BLEU score calculator?
#586 opened Apr 30, 2024 by shenxiangzhuang updated Apr 30, 2024
Is perplexity correctly computed?
#560 opened Mar 15, 2024 by halixness updated Apr 28, 2024
4
Unable to run pip install evaluate[template]
#576 opened Apr 17, 2024 by saicharan2804 updated Apr 26, 2024
1
1
How to pass generation_kwargs to the TextGeneration evaluator ?
#582 opened Apr 25, 2024 by swarnava112 updated Apr 25, 2024
the difference of your bleu and sacrebleu
#558 opened Mar 7, 2024 by cooper12121 updated Apr 24, 2024
1
[FR] Confidence intervals for metrics
#581 opened Apr 22, 2024 by NightMachinery updated Apr 22, 2024
Perplexity metric does not apply batching correctly to tokenization
#573 opened Apr 14, 2024 by ChengSashankh updated Apr 14, 2024
1
HuggingFace evaluate-cli create command results Error: "datasets[0]" must be a string
#514 opened Nov 7, 2023 by shivanraptor updated Apr 11, 2024
2
[Question] How to have no preset values sent into .compute()
#570 opened Apr 8, 2024 by alvations updated Apr 8, 2024
Allow for specify coda device in perplexity evaluation
#568 opened Apr 8, 2024 by manuelbrack updated Apr 8, 2024
Bug in BLEU calculation - BLEU of identical strings should be 1.0, not 0.0
#529 opened Dec 27, 2023 by MislavJuric updated Apr 6, 2024
2
Cannot use it offline!
#567 opened Apr 5, 2024 by SirryChen updated Apr 5, 2024
1
evaluate.load loads indefinitely offline
#473 opened Jul 18, 2023 by Virgile-Richard1 updated Apr 4, 2024
1
ValueError: Predictions and/or references don't match the expected format.
#563 opened Mar 20, 2024 by antopost updated Mar 20, 2024
1
ImportError: To be able to use evaluate-metric/rouge, you need to install the following dependencies['nltk'] using 'pip install # Here to have a nice missing dependency error message early on' for instance'
#434 opened Mar 23, 2023 by shuxiaobo updated Mar 19, 2024
2
It seems like evaluate.load doesnt use
#561 opened Mar 17, 2024 by anhq-nguyen updated Mar 17, 2024
Add DER metric metric request
#117 opened Oct 22, 2021 by albertvillanova updated Mar 4, 2024
@anton-l
1
Add FID score metric request
#276 opened Aug 29, 2022 by lvwerra updated Mar 2, 2024
2
After fine-tuning Gemma and want to evaluate performance: AttributeError: module 'keras._tf_keras.keras' has no attribute '__internal__'
#555 opened Mar 2, 2024 by XinyueZ updated Mar 2, 2024
Add Precision@k and Recall@k metrics
#554 opened Mar 2, 2024 by Andron00e updated Mar 2, 2024
need evaluator code reference for object detector & zero shot object dector task
#552 opened Feb 27, 2024 by solomonmanuelraj updated Feb 27, 2024
ProTip! Type g i on any issue or pull request to go back to the issue listing page.