You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
$$\text{Acc}(ans) = \min{ \left\{ \frac{\text{\# humans that said } ans }{3}, 1 \right\} }$$
I have following questions:
Do I need to customize my own metric? If so, can I only create metrics/vqa_accuracy/vqa_accuracy.py without other operations, such as running evaluate-cli create "accuracy name" --module_type "metric"?
I found that there is no suitable evaluator for my task, and I'm not sure if it is possible to customize my own evaluator, since I didn't find any document on creating new evaluator.
The text was updated successfully, but these errors were encountered:
I'm facing a task on VQA, where I need to compute VQA accuracy](https://visualqa.org/evaluation.html) as follows:
I have following questions:
metrics/vqa_accuracy/vqa_accuracy.py
without other operations, such as runningevaluate-cli create "accuracy name" --module_type "metric"
?evaluator
for my task, and I'm not sure if it is possible to customize my ownevaluator
, since I didn't find any document on creating newevaluator
.The text was updated successfully, but these errors were encountered: