Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

polish high frequency enforce error message #12591

Merged
merged 3 commits into from
Aug 16, 2018

Conversation

chenwhql
Copy link
Contributor

@chenwhql chenwhql commented Aug 8, 2018

To resolve problems:

  1. enforce compare message isn't easy to understand
paddle.fluid.core.EnforceNotMet: enforce x_dims[i + axis] == y_dims[i] failed, 1 != 13

after changing:

EnforceNotMet: Data check failed. Expected x_dims[i + axis] == y_dims[i], but received x_dims[i + axis]:1 != y_dims[i]:13.
  1. alloc gpu memory error message isn't easy to understand
paddle.fluid.core.EnforceNotMet: enforce allocating <= available failed, 1822361026 > 409206528

after changing:

EnforceNotMet: Data check failed. Expected allocating_memory <= available_memory, but received allocating_memory:20998685852 > available_memory:1645149952.

@chenwhql chenwhql requested a review from reyoung August 8, 2018 11:21
Copy link
Collaborator

@reyoung reyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enforce does not always mean Data check

PADDLE_THROW("enforce %s " #__CMP " %s failed, %s " #__INV_CMP \
" %s\n%s", \
#__VAL0, #__VAL1, paddle::string::to_string(__VAL0), \
PADDLE_THROW("Data check failed. Expected %s " #__CMP \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data check failed. --> Enforce failed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks


GpuMemoryUsage(&available, &total);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The older version is better than the new version. Since this function is GpuMaxChunkSize, using available_memory is wordiness.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks


PADDLE_ENFORCE_LE(allocating, available);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can write an extra message here if you think PADDLE_ENFORCE_LE is not good.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks

Copy link
Collaborator

@reyoung reyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data check failed. --> Enforce failed

@chenwhql chenwhql merged commit 1e961b1 into PaddlePaddle:develop Aug 16, 2018
kuke pushed a commit to kuke/Paddle that referenced this pull request Aug 25, 2018
polish high frequency enforce error message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants