Issue in Quantizer decoeder_model_merged.onnx of MT5

[jhpassion0621]

System Info

optimum 1.16.2
onnx 1.15.0
onnxruntime 1.17.0

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

I got onnx files of mt5-large model and when I tried to convert that to quantized files, I got errors at decoder_model_merged.onnx.

from optimum.onnxruntime.configuration import AutoQuantizationConfig
from optimum.onnxruntime import ORTQuantizer

qconfig = AutoQuantizationConfig.arm64(is_static=False, per_channel=False)
quantizer = ORTQuantizer.from_pretrained("/content/drive/MyDrive/mt5-large", file_name="decoder_model_merged.onnx")

quantizer.quantize(save_dir="/content/drive/MyDrive/mt5-large", quantization_config=qconfig)

Expected behavior

decoder_model_merged_quantized.onnx

[fxmarty]

Hi @jhpassion0621 this is a bug introduced in onnxruntime 1.17, please downgrade to onnxruntime 1.16 to wait for a fix to be released: microsoft/onnxruntime#19421

[jhpassion0621]

Thank you very much. it works well on 1.16. @fxmarty