-
Notifications
You must be signed in to change notification settings - Fork 248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CER Normalization #161
Comments
Hi @Coliverfelt, not sure what you mean. You can find the implementation details here: |
Hi @lvwerra! My point is that when CER metric is above 1.0, like in this example: reference: 2.49 substitute= 1 CER formula is substitute + delete + insert / reference length and the result is equal to 1.25. But in this case that CER is greater than 1.00 we should apply normalization. So the normalized result should be: CER normalization formula is substitute + delete + insert / substitute + delete + insert + hits so the result is equal to 0.7142857142857143. And when I'm running:
The result isn't normalized. Did I make myself clearer? |
I couldn't find any reference for normalized CER do you have a link? We could add this as an optional config. E.g. cer.compute(references=references, predictions=predictions, normalized=True) What do you think? |
I think this way could work well! |
Sounds good, do you want to take a stab at it? |
I'm not finding in the documentation a method to normalize CER metric. Do you have this implemented?
The text was updated successfully, but these errors were encountered: