Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

softmax setting adjoint instead of incrementing adjoint #2048

Closed
bbbales2 opened this issue Aug 28, 2020 · 1 comment · Fixed by #2050
Closed

softmax setting adjoint instead of incrementing adjoint #2048

bbbales2 opened this issue Aug 28, 2020 · 1 comment · Fixed by #2050

Comments

@bbbales2
Copy link
Member

Description

@SteveBronder found that softmax isn't quite working right. The reverse mode softmax code here sets the adjoint instead of incrementing it: https://github.com/stan-dev/math/blob/develop/stan/math/rev/fun/softmax.hpp#L39

Changing the = to += fixes the problem.

I think this could be tested in the autodiff test framework. The test would be set the adjoints of the inputs to something non-zero, do the reverse mode pass, and then check the adjoints have been incremented by what you'd expect if the adjoints had been zero.

If this is fixed without the more general fix to the autodiff tester, make a new issue for that.

Current Version:

v3.3.0

@bob-carpenter
Copy link
Contributor

more general fix to the autodiff tester

+1 to that. The testing for nested use (where you need to increment) is sketchy at best, but could be easily incorporated into the test framework.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants