Skip to content

Commit

Permalink
[Fix] Throwes one excepiton while Llama2 parity_check fails (#21160)
Browse files Browse the repository at this point in the history
### Description



### Motivation and Context
The pipeline is green even Llama2 parity_check fails.

The PR should be merged after the below exception is solved.

'''
2024-06-25 03:49:43.621298481 [E:onnxruntime:,
sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned
while running Expand node. Name:'/model/Expand' Status Message:
/model/Expand: left operand cannot broadcast on dim 3 LeftShape:
{1,1,9,9}, RightShape: {2,1,9,17}
An error occurred while verifying parity: Error in execution: Non-zero
status code returned while running Expand node. Name:'/model/Expand'
Status Message: /model/Expand: left operand cannot broadcast on dim 3
LeftShape: {1,1,9,9}, RightShape: {2,1,9,17}
Traceback (most recent call last):
File
"/workspace/onnxruntime/python/tools/transformers/models/llama/convert_to_onnx.py",
line 1043, in main
    parity_check(parity_cmd)
File
"/workspace/onnxruntime/python/tools/transformers/models/llama/llama_parity.py",
line 298, in main
verify_parity(args, location, use_auth_token, kv_cache_ortvalues,
pytorch_model=llama, config=config)
File
"/workspace/onnxruntime/python/tools/transformers/models/llama/llama_parity.py",
line 137, in verify_parity
    ort_model.run_with_iobinding(io_binding)
File
"/home/onnxruntimedev/.local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py",
line 331, in run_with_iobinding
    self._sess.run_with_iobinding(iobinding._iobinding, run_options)
RuntimeError: Error in execution: Non-zero status code returned while
running Expand node. Name:'/model/Expand' Status Message: /model/Expand:
left operand cannot broadcast on dim 3 LeftShape: {1,1,9,9}, RightShape:
{2,1,9,17}
'''

The exception looks caused by #19832
  • Loading branch information
mszhanyi committed Jun 27, 2024
1 parent b49788e commit 8f738d8
Showing 1 changed file with 2 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -1052,7 +1052,8 @@ def main():
logger.info(f"check parity with cmd: {parity_cmd}")
parity_check(parity_cmd)
except Exception as e:
logger.warning(f"An error occurred while verifying parity: {e}", exc_info=True)
logger.exception(f"An error occurred while verifying parity: {e}")
sys.exit(-1)


if __name__ == "__main__":
Expand Down

0 comments on commit 8f738d8

Please sign in to comment.