[ORT 1.17 regression] AttributeError: 'NoneType' object has no attribute 'HasField'

[fxmarty]

Describe the issue

Hi, a test in our CI is failing with ORT 1.17 and not with previous ORT versions. I believe it is a regression.

The culprint is likely: #18043

To reproduce

pip uninstall onnx onnxruntime && pip install onnx onnxruntime
git clone https://github.com/huggingface/optimum.git && cd optimum
pytest tests/onnxruntime/test_quantization.py -k "test_dynamic_quantization_subgraphs" -s -vvvvv

fails with 1.17 while the test passes with previous versions:

______________________________________________ ORTDynamicQuantizationTest.test_dynamic_quantization_subgraphs _______________________________________________

self = <test_quantization.ORTDynamicQuantizationTest testMethod=test_dynamic_quantization_subgraphs>

    @unittest.skipIf(parse(ort_version) == Version("1.16.0"), "not supported with this onnxruntime version")
    def test_dynamic_quantization_subgraphs(self):
        qconfig = AutoQuantizationConfig.avx512(is_static=False, per_channel=True)
        tmp_dir = tempfile.mkdtemp()
        output_dir = Path(tmp_dir)
        model = ORTModelForCausalLM.from_pretrained("fxmarty/onnx-tiny-random-gpt2-with-merge", use_merged=True)
        self.assertTrue(model.use_merged)
        model.save_pretrained(tmp_dir)
    
        quantizer = ORTQuantizer.from_pretrained(model)
>       quantizer.quantize(save_dir=output_dir, quantization_config=qconfig)

tests/onnxruntime/test_quantization.py:127: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
optimum/onnxruntime/quantization.py:417: in quantize
    quantizer.quantize_model()
../../anaconda3/envs/hf-inf/lib/python3.9/site-packages/onnxruntime/quantization/onnx_quantizer.py:399: in quantize_model
    node = self.quantize_node_with_sub_graph(node)  # noqa: PLW2901
../../anaconda3/envs/hf-inf/lib/python3.9/site-packages/onnxruntime/quantization/onnx_quantizer.py:310: in quantize_node_with_sub_graph
    kv = {attr.name: self.quantize_subgraph(attr.g, f"{node_name}:{attr.name}")}
../../anaconda3/envs/hf-inf/lib/python3.9/site-packages/onnxruntime/quantization/onnx_quantizer.py:291: in quantize_subgraph
    sub_quantizer.quantize_model()
../../anaconda3/envs/hf-inf/lib/python3.9/site-packages/onnxruntime/quantization/onnx_quantizer.py:403: in quantize_model
    op_quantizer.quantize()
../../anaconda3/envs/hf-inf/lib/python3.9/site-packages/onnxruntime/quantization/operators/base_operator.py:21: in quantize
    dequantize_node = self.quantizer._dequantize_value(node_input)
../../anaconda3/envs/hf-inf/lib/python3.9/site-packages/onnxruntime/quantization/onnx_quantizer.py:1341: in _dequantize_value
    assert onnx.numpy_helper.to_array(scale_init).size == 1
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

tensor = None, base_dir = ''

    def to_array(tensor: TensorProto, base_dir: str = "") -> np.ndarray:  # noqa: PLR0911
        """Converts a tensor def object to a numpy array.
    
        Args:
            tensor: a TensorProto object.
            base_dir: if external tensor exists, base_dir can help to find the path to it
    
        Returns:
            arr: the converted array.
        """
>       if tensor.HasField("segment"):
E       AttributeError: 'NoneType' object has no attribute 'HasField'

../../anaconda3/envs/hf-inf/lib/python3.9/site-packages/onnx/numpy_helper.py:184: AttributeError

Urgency

medium, will probably pin to onnxruntime<1.17 meanwhile

Platform

Linux

OS Version

Ubuntu 22.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[fxmarty]

Smaller repro:
decoder_model_merged.onnx.zip

&

import onnxruntime
from onnxruntime.quantization import quantize_dynamic

quantize_dynamic(
    model_input="/home/fxmarty/hf_internship/optimum/onnx_subgraphs/decoder_model_merged.onnx",
    model_output="/home/fxmarty/hf_internship/optimum/onnx_subgraphs/decoder_model_merged_quantized.onnx",
    per_channel=True,
    op_types_to_quantize=['Conv', 'MatMul', 'Attention', 'LSTM', 'Gather', 'Transpose', 'EmbedLayerNormalization'],
    extra_options={"EnableSubgraph": True}
)

Edit: seems to me that find_by_name(quantized_value.scale_name, self.model.initializer()) somehow does not find the initializer (while it is indeed there in the quantized ONNX with ORT 1.16.3)

[ryouchinsa]

Thanks for writing the issue.

This issue is causing the error when exporting the quantized SAM model.
facebookresearch/segment-anything#699 (comment)

Installing the old onnxruntime version solved the problem in our environment.
pip3 install onnxruntime==1.16.0