fix g_param shape mismatch in WeightNormParamAttr #18940
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
guoshengCS
reviewed
Aug 1, 2019
| transpose, shape=[transpose.shape[0], -1], block=block) | ||
| norm = __norm_op(reshape, dim=[1], block=block) | ||
| reshape2 = __reshape_op(norm, shape=out_shape, block=block) | ||
| __transpose_op(reshape2, perm, out=out, block=block) |
There was a problem hiding this comment.
Since the shape of output is a vector, maybe we can remove this line __transpose_op(reshape2, perm, out=out, block=block) by redefining out_shape in reshape2 as [1...1, transpose.shape[0], 1...1] .
| their direction. Weight Norm has been implemented as discussed in this | ||
| paper: `Weight Normalization: A Simple Reparameterization to Accelerate | ||
| Training of Deep Neural Networks | ||
| <https://arxiv.org/pdf/1602.07868.pdf>`_. | ||
|
|
||
| Args: | ||
| dim(list): The parameter's name. Default None. | ||
| dim(int): Dimension over which to compute the norm. Default None. |
There was a problem hiding this comment.
Might it be the Dimension except which
There was a problem hiding this comment.
this is referred to pytorch, which said 'dim (int, optional): dimension over which to compute the norm
', just keep consistent with other framework
guoshengCS
approved these changes
Aug 5, 2019
heavengate
approved these changes
Aug 5, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
1, fix g_param shape mismatch between startup_program and main_program
2, fix 'dim' attributes from int to list(int), when append reduce_sum op