How to customize my own evaluator and metrics?

[Kami-chanw]

I'm facing a task on VQA, where I need to compute VQA accuracy](https://visualqa.org/evaluation.html) as follows:

$$\text{Acc}(ans) = \min{ \left\{ \frac{\text{\# humans that said } ans }{3}, 1 \right\} }$$

I have following questions:

1. Do I need to customize my own metric? If so, can I only create metrics/vqa_accuracy/vqa_accuracy.py without other operations, such as running evaluate-cli create "accuracy name" --module_type "metric"?
2. I found that there is no suitable evaluator for my task, and I'm not sure if it is possible to customize my own evaluator, since I didn't find any document on creating new evaluator.