Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OVWeightUpdateCommand copies a constant #2277

Merged
merged 8 commits into from
Nov 23, 2023

Conversation

andrey-churkin
Copy link
Contributor

@andrey-churkin andrey-churkin commented Nov 17, 2023

Changes

  • Update an implementation of get_const_value() method. This method was initially implemented as a workaround in PR 1654, but it is no longer needed and can be replaced with const_op.data.
  • Replace const_op.get_data() with const_op.data because the const_op.get_data() legacy method and .data property should be used.
  • Set the shared_memory parameter to True during constant creation to avoid copying the constant.

Reason for changes

  • The const_od.get_data() method copies constant.
  • Do not copy the constant value during constant creation in opset.constant(...).

Related tickets

Ref: 122922

Tests

N/A

@andrey-churkin andrey-churkin requested a review from a team as a code owner November 17, 2023 11:17
@github-actions github-actions bot added NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ labels Nov 17, 2023
Copy link

codecov bot commented Nov 17, 2023

Codecov Report

Merging #2277 (f2cd4ef) into develop (5eee3bc) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #2277   +/-   ##
========================================
  Coverage    90.76%   90.76%           
========================================
  Files          486      486           
  Lines        43763    43764    +1     
========================================
+ Hits         39722    39723    +1     
  Misses        4041     4041           
Flag Coverage Δ
COMMON 15.81% <0.00%> (-0.01%) ⬇️
ONNX 33.82% <0.00%> (-0.01%) ⬇️
OPENVINO 38.68% <100.00%> (+<0.01%) ⬆️
TENSORFLOW 30.02% <0.00%> (-0.01%) ⬇️
TORCH 62.68% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
nncf/openvino/graph/model_transformer.py 97.08% <100.00%> (+<0.01%) ⬆️
nncf/openvino/graph/node_utils.py 98.66% <100.00%> (ø)

Copy link
Contributor

@alexsu52 alexsu52 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you provide performance results?

@andrey-churkin
Copy link
Contributor Author

Could you provide performance results?

I have used the databricks/dolly-v2-3b model and got the following result.

Quantization Wall Time Before (NNCF develop): 401.24 sec
Quantization Wall Time After: 279.26 sec

nncf/openvino/graph/model_transformer.py Show resolved Hide resolved
nncf/openvino/graph/node_utils.py Outdated Show resolved Hide resolved
@github-actions github-actions bot removed the NNCF PTQ Pull requests that updates NNCF PTQ label Nov 22, 2023
Copy link
Collaborator

@KodiaqQ KodiaqQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, fill the ticket (or create PR) with the migrating to the latest opset.

@andrey-churkin
Copy link
Contributor Author

Please, fill the ticket (or create PR) with the migrating to the latest opset.

I've filled the ticket. Ref: 125777

const_value = np.reshape(const_value, const_shape).astype(const_dtype)

# TODO(andrey-churkin): Replace on opset13.constant() in a future release
new_const_node = ov.op.Constant(const_value, shared_memory=True)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexsu52 @KodiaqQ The shared_memory parameter is not accessible in opset13.constant() for the our CI version of OV. It is available only on the master branch. Therefore, I am using ov.op.Constant() here. This will be updated in a future release.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just adding a note that using ov.op.Constant is not consistent with our current approach of using opset from Python API.

@KodiaqQ KodiaqQ self-requested a review November 23, 2023 12:41
@KodiaqQ KodiaqQ merged commit 9479779 into openvinotoolkit:develop Nov 23, 2023
9 checks passed
@andrey-churkin andrey-churkin deleted the ac/const branch November 24, 2023 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NNCF OpenVINO Pull requests that updates NNCF OpenVINO
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants