Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clang tidy cleanup and using std algorithms #1373

Closed
wants to merge 17 commits into from
Closed

Clang tidy cleanup and using std algorithms #1373

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
b9f545b
Clang tidy with modernize-use-equals-default,readability-braces-aroun…
SteveBronder Sep 30, 2019
4064902
Adds range based for loops via clang tidy, accumulates in log_sum_exp…
SteveBronder Sep 30, 2019
23706ed
[Jenkins] auto-formatting by clang-format version 6.0.0 (tags/google/…
stan-buildbot Sep 30, 2019
bade075
include <numeric> in dot_self header
SteveBronder Sep 30, 2019
8a2dff8
Merge branch 'clang-tidy/braces-defaults_constructor-range_for_loops'…
SteveBronder Sep 30, 2019
2407b7a
copy-pasted an accumulate without changing the variable names from on…
SteveBronder Sep 30, 2019
8bb5458
[Jenkins] auto-formatting by clang-format version 6.0.0 (tags/google/…
stan-buildbot Sep 30, 2019
0c31b37
fix headers
SteveBronder Sep 30, 2019
d68129f
forgot accumulate is in numeric and not algorithm
SteveBronder Sep 30, 2019
f9275d1
test in scal was using unitialized vector?
SteveBronder Sep 30, 2019
06c465a
merge to develop
SteveBronder Oct 8, 2019
bdadd8e
cleanups for promote_elements
SteveBronder Oct 8, 2019
8656836
[Jenkins] auto-formatting by clang-format version 5.0.0-3~16.04.1 (ta…
stan-buildbot Oct 8, 2019
d716461
remove tbb stuff that should be ignored
SteveBronder Oct 8, 2019
3266349
Use auto in gp_exp_quad_cov, add rule of 5 to accumulator, and pass b…
SteveBronder Nov 3, 2019
90be53e
[Jenkins] auto-formatting by clang-format version 6.0.0 (tags/google/…
stan-buildbot Nov 3, 2019
c9fef6c
merge to develop
SteveBronder Nov 5, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions stan/math/prim/arr/fun/log_sum_exp.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,9 @@ inline double log_sum_exp(const std::vector<double>& x) {
using std::log;
using std::numeric_limits;
double max_val = *std::max_element(x.begin(), x.end());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very neat!

double sum = std::accumulate(x.begin(), x.end(), 0.0,
[&max_val](auto& acc, auto&& x_i) {
return acc + exp(x_i - max_val);
});
double sum = std::accumulate(
x.begin(), x.end(), 0.0,
[&max_val](auto& acc, auto&& x_i) { return acc + exp(x_i - max_val); });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this generate code that's as efficient as before? It will come down to how efficiently it can compile that closure.

How do we test?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just did this on godbolt, lhs is the code (bottom is current (labled editor 1) and top is the new one (editor 2) middle is the output from the new stuff and far right is the output from the current stuff. You can highlight certain instructions and it usually pops up a little 'heres what this does'. You can click 'add' in the top right to get a diff view of the two outputs, though it usually looks wonky at O3. You can click and drag any of the tabs for each little block to move stuff. If you right click the highlighted code on the lhs it should have an options to take you to where that line is happening in whichever of the bottom two outputs, though it's not always exact.

I like to look at -O0 to see where stuff is then looking at -O3. About lines 40-60'ish is where the loop and exp calculation happen. The code is super similar, the lambda version removes a compare and a few moves. But those are mostly because we don't do the if statement in there anymore. I can look tmrw at just removing that check there with the old version.

https://godbolt.org/z/Xe8ev_

godbolt is pretty neat! I learned last night you can also get a real graph of the call graph!

https://godbolt.org/z/cCqIAH

There's a way to make a PR on their repo so we can get Stan up there, would like to find time for that in the next week or so

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another cool internet benchmark thing!

http://quick-bench.com/3Wdd56xscm20sShrc0xZx2qgdsE

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the rules for capture are like argument passing, so that primitives like max_val should be captured by value, not by reference.

return max_val + log(sum);
}

Expand Down
20 changes: 10 additions & 10 deletions stan/math/prim/mat/fun/gp_exp_quad_cov.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ gp_exp_quad_cov(const std::vector<T_x> &x, const T_sigma &sigma,
return cov;
}

for (auto&& x_i : x) {
for (auto &&x_i : x) {
check_not_nan("gp_exp_quad_cov", "x", x_i);
}

Expand Down Expand Up @@ -275,10 +275,10 @@ gp_exp_quad_cov(const std::vector<T_x1> &x1, const std::vector<T_x2> &x2,
return cov;
}

for (auto&& x1_i : x1) {
for (auto &&x1_i : x1) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As is, I think these can be const.

These should be using a vectorized check_not_nan so that the index can also be printed and we don't have all this boilerplate looping.

Another alternative would be a for-each loop, which doesn't actually simplify things here, especiallyw ith explicit capture of the function name.

std::for_each(x1.begin(), x1.end(),
              [&function_name](double x) { return check_not_nan(function_name, "x", x); });

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be using a vectorized check_not_nan so that the index can also be printed and we don't have all this boilerplate looping.

Agree this should use a vectorized check_nan, but the vectorized version of check_not_nan does not work for vectors of eigen matrices atm :-(

After Andrew and I sort out the more generic templating discussion in #1425 then I'm going to come back to these check functions and clean them up so we can do that.

check_not_nan(function_name, "x1", x1_i);
}
for (auto&& x2_i : x2) {
for (auto &&x2_i : x2) {
check_not_nan(function_name, "x2", x2_i);
}

Expand Down Expand Up @@ -325,10 +325,10 @@ gp_exp_quad_cov(const std::vector<Eigen::Matrix<T_x1, -1, 1>> &x1,
}

const char *function_name = "gp_exp_quad_cov";
for (auto&& x1_i : x1) {
for (auto &&x1_i : x1) {
check_not_nan(function_name, "x1", x1_i);
}
for (auto&& x2_i : x2) {
for (auto &&x2_i : x2) {
check_not_nan(function_name, "x2", x2_i);
}
check_positive_finite(function_name, "magnitude", sigma);
Expand Down Expand Up @@ -415,7 +415,7 @@ inline Eigen::MatrixXd gp_exp_quad_cov(const std::vector<Eigen::VectorXd> &x,
const size_t inner_x1_size = x[0].size();
const auto total_size = x_size * inner_x1_size + cov.size();
if (total_size < opencl_context.tuning_opts().gp_exp_quad_cov_complex) {
for (auto&& x_i : x) {
for (auto &&x_i : x) {
check_not_nan("gp_exp_quad_cov", "x", x_i);
}
cov = internal::gp_exp_quad_cov(x, square(sigma),
Expand Down Expand Up @@ -560,10 +560,10 @@ inline typename Eigen::MatrixXd gp_exp_quad_cov(
const auto total_size
= x1_size * x1_inner_size + x2_size * x2_inner_size + cov.size();
if (total_size < opencl_context.tuning_opts().gp_exp_quad_cov_complex) {
for (auto&& x_i : x1) {
for (auto &&x_i : x1) {
check_not_nan(function_name, "x1", x_i);
}
for (auto&& x_i : x2) {
for (auto &&x_i : x2) {
check_not_nan(function_name, "x2", x_i);
}

Expand Down Expand Up @@ -621,10 +621,10 @@ inline typename Eigen::MatrixXd gp_exp_quad_cov(
const auto total_size
= x1_size * x1_inner_size + x2_size * x2_inner_size + l_size + cov.size();
if (total_size < opencl_context.tuning_opts().gp_exp_quad_cov_complex) {
for (auto&& x1_i : x1) {
for (auto &&x1_i : x1) {
check_not_nan(function_name, "x1", x1_i);
}
for (auto&& x2_i : x2) {
for (auto &&x2_i : x2) {
check_not_nan(function_name, "x1", x2_i);
}
cov = internal::gp_exp_quad_cov(divide_columns(x1, length_scale),
Expand Down
6 changes: 3 additions & 3 deletions stan/math/rev/arr/fun/log_sum_exp.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ inline double log_sum_exp_as_double(const std::vector<var>& x) {
using std::numeric_limits;
double max_val = std::max_element(x.begin(), x.end())->val();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[optional]
This is soooo close to the double version, the only difference being the ->val() pulling out the double based value. Could the (recursive?) value_of for max_val computation allow these to be combined into a single implementation? Maybe not worth it given again how complicated the indirection would be.

Copy link
Collaborator Author

@SteveBronder SteveBronder Nov 3, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, it's so close! I think for a v v clean version of this we need a vectorized value_of. Then in the constructor for log_sum_exp_vector_vari we could just call op_vector_vari(log_sum_exp(value_of(x)), x).

I put a comment above log_sum_exp_as_double about this and can do those value_of's in a separate PR

double sum = std::accumulate(x.begin(), x.end(), 0.0,
[&max_val](auto& acc, auto&& x_i) {
return acc + exp(x_i.val() - max_val);
});
[&max_val](auto& acc, auto&& x_i) {
return acc + exp(x_i.val() - max_val);
});
return max_val + log(sum);
}

Expand Down
7 changes: 3 additions & 4 deletions stan/math/rev/arr/fun/sum.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,9 @@ class sum_v_vari : public vari {
size_t length_;

inline static double sum_of_val(const std::vector<var>& v) {
double result = std::accumulate(x.begin(), x.end(), 0.0,
[](auto& acc, auto&& x_i) {
return acc + x_i.val();
});
double result = std::accumulate(
x.begin(), x.end(), 0.0,
[](auto& acc, auto&& x_i) { return acc + x_i.val(); });
return result;
}

Expand Down
7 changes: 3 additions & 4 deletions stan/math/rev/mat/fun/gp_exp_quad_cov.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,14 +208,13 @@ class gp_exp_quad_cov_vari<T_x, double, T_l> : public vari {
* @throw std::domain_error if sigma <= 0, l <= 0, or
* x is nan or infinite
*/
template <typename T_x,
typename = enable_if_arithmetic<scalar_type_t<T_x>>>
template <typename T_x, typename = enable_if_arithmetic<scalar_type_t<T_x>>>
inline Eigen::Matrix<var, -1, -1> gp_exp_quad_cov(const std::vector<T_x> &x,
const var &sigma,
const var &length_scale) {
check_positive("gp_exp_quad_cov", "sigma", sigma);
check_positive("gp_exp_quad_cov", "length_scale", length_scale);
for (auto&& x_i : x) {
for (auto &&x_i : x) {
check_not_nan("gp_exp_quad_cov", "x", x_i);
}
const size_t x_size = x.size();
Expand Down Expand Up @@ -258,7 +257,7 @@ inline Eigen::Matrix<var, -1, -1> gp_exp_quad_cov(const std::vector<T_x> &x,
const var &length_scale) {
check_positive("gp_exp_quad_cov", "marginal variance", sigma);
check_positive("gp_exp_quad_cov", "length-scale", length_scale);
for (auto&& x_i : x) {
for (auto &&x_i : x) {
check_not_nan("gp_exp_quad_cov", "x", x_i);
}

Expand Down