Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

var matrix specializations for a-c unary functions in rev #2256

Merged
merged 30 commits into from
Dec 19, 2020

Conversation

SteveBronder
Copy link
Collaborator

Summary

This is just going through rev and adding var matrix functions for a bunch of functions.

Tests

expect_ad_matvar tests added for all of the functions in here. Though I still need to write tests for std::vector<var<matrix>>

Side Effects

I wrote this by specializing a function for each rev version. But that means we have to write the scalar_unary version like

template <typename Container,
          require_not_container_st<std::is_arithmetic, Container>* = nullptr,
          require_not_var_matrix_t<Container>* = nullptr>
inline auto fabs(const Container& x) {
  return apply_scalar_unary<fabs_fun, Container>::apply(x);
}

Though there may be some other way to do this where instead we specialize *_fun

struct fabs_fun {
  template <typename T>
  static inline T fun(const T& x) {
    using std::fabs;
    return fabs(x);
  }
};

As it's written right now it's nice because it works with std::vector<var<matrix>>. eod we need to specialize somewhere so I don't mind having it like we do now. We could also make var<matrix> not a container type so then we can remove the require_not_var_matrix_t<Container>

Release notes

Adds arc trig functions for var<matrix> along several other unary operators

Checklist

  • Math issue How to add static matrix? #1805

  • Copyright holder: Steve Bronder

    The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
    - Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
    - Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

  • the basic tests are passing

    • unit tests pass (to run, use: ./runTests.py test/unit)
    • header checks pass, (make test-headers)
    • dependencies checks pass, (make test-math-dependencies)
    • docs build, (make doxygen)
    • code passes the built in C++ standards checks (make cpplint)
  • the code is written in idiomatic C++ and changes are documented in the doxygen

  • the new changes are tested

@bbbales2
Copy link
Member

THere are a few functions like this in #2254. I was stuck there on what to do with std::vector<varmat> but sounds like you solved that.

Does what you did here also work with std::vector<std::vector<varmat>>, etc?

I asked @andrjohns about this in the other thread but maybe you have already solved it (in which case don't worry about it @andrjohns !)

Though I still need to write tests for std::vector<var>

That should be testable now. I had to add these tests for log_softmax. I didn't add tests for std::vector<std::vector<varmat>> though.

@SteveBronder
Copy link
Collaborator Author

Lemme test that. It should at least be close to working. All this does is overload each function so when apply scalar iterates over the elements it should just do the normal thing and differ to the other specializations until it hits varmat and then just do the varmat thing

@stan-buildbot
Copy link
Contributor


Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.43 3.43 1.0 -0.01% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 1.04 3.89% faster
eight_schools/eight_schools.stan 0.12 0.12 1.02 1.76% faster
gp_regr/gp_regr.stan 0.15 0.15 0.99 -0.59% slower
irt_2pl/irt_2pl.stan 5.48 5.51 0.99 -0.58% slower
performance.compilation 88.16 86.31 1.02 2.11% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.38 8.36 1.0 0.23% faster
pkpd/one_comp_mm_elim_abs.stan 29.3 30.59 0.96 -4.39% slower
sir/sir.stan 144.3 140.09 1.03 2.92% faster
gp_regr/gen_gp_data.stan 0.04 0.05 0.96 -4.18% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 2.94 2.92 1.01 0.78% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.39 0.38 1.02 1.94% faster
arK/arK.stan 2.47 2.48 1.0 -0.1% slower
arma/arma.stan 0.59 0.6 0.99 -1.14% slower
garch/garch.stan 0.6 0.61 0.99 -0.51% slower
Mean result: 1.00190292775

Jenkins Console Log
Blue Ocean
Commit hash: 91d1472


Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Comment on lines 44 to 49
template <typename Container,
require_not_container_st<std::is_arithmetic, Container>* = nullptr>
require_not_container_st<std::is_arithmetic, Container>* = nullptr,
require_not_var_matrix_t<Container>* = nullptr>
inline auto acos(const Container& x) {
return apply_scalar_unary<acos_fun, Container>::apply(x);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrjohns I'm having a hard time removing this require_not_var_matrix_t<Container> and I'd like to not have to put these everywhere. Right now this function works for any stan scalar or container that does not have an arithmetic scalar type. So it's kind of a mixed bag of types this takes in. Though specializations for each seems like overkill as well so I'm not really sure if we can make this look nicer

@SteveBronder
Copy link
Collaborator Author

@bbbales2 fyi I removed var matrix from the container types. It feels like a var matrix is a single unit and we are pretty much always going to have specializations for var matrix. With the container checks we're normally checking whether it's an eigen vector so we can do some specialization or a std vector so we can recursivly go through it (or to map the std vector to an eigen matrix to use the Eigen specialization). So like I think with var matrix it makes sense to not have it count as a container. Does that make sense?

@rok-cesnovar
Copy link
Member

But if its not a container, is it a scalar? I dont think that make sense.

@bbbales2
Copy link
Member

Yeah I'm not totally sure what it is either, but I'm also not using it like the other things we call containers. Popping it out seems fine with me.

Seems like whenever we use var_value, it is require_var_vector_t or require_var_matrix_t or require_rev_matrix_t or something like that.

I guess we could change require_container_t to require_std_vector_or_eigen_t lol but that's getting kinda long.

@stan-buildbot
Copy link
Contributor


Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.65 3.43 1.06 5.93% faster
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 1.0 0.02% faster
eight_schools/eight_schools.stan 0.11 0.11 1.0 -0.03% slower
gp_regr/gp_regr.stan 0.15 0.15 1.0 -0.12% slower
irt_2pl/irt_2pl.stan 5.45 5.48 0.99 -0.61% slower
performance.compilation 88.41 85.95 1.03 2.79% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.39 8.39 1.0 0.1% faster
pkpd/one_comp_mm_elim_abs.stan 30.18 28.99 1.04 3.92% faster
sir/sir.stan 138.88 143.1 0.97 -3.03% slower
gp_regr/gen_gp_data.stan 0.04 0.04 0.99 -1.51% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 2.92 2.91 1.0 0.36% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.38 0.39 0.97 -3.23% slower
arK/arK.stan 2.48 2.48 1.0 -0.01% slower
arma/arma.stan 0.58 0.58 1.0 -0.35% slower
garch/garch.stan 0.61 0.61 1.0 -0.29% slower
Mean result: 1.00316857358

Jenkins Console Log
Blue Ocean
Commit hash: 010ebaf


Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

@bbbales2
Copy link
Member

@SteveBronder this ready to review? I want to just copy whatever you do here for the scalar functions in #2254 so if it's done that's convenient lol.

@SteveBronder
Copy link
Collaborator Author

@bbbales2 this should be good to go

Copy link
Member

@bbbales2 bbbales2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

* @return absolute value of scalar
* @tparam Container type of container
* @param x argument
* @return Arc cosine of each variable in the container, in radians.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolute value

@@ -39,8 +39,7 @@ struct atan_fun {
*/
template <typename Container,
require_not_container_st<std::is_arithmetic, Container>* = nullptr,
require_all_not_nonscalar_prim_or_rev_kernel_expression_t<
Container>* = nullptr>
require_not_var_matrix_t<Container>* = nullptr>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not container

* <code>std::abs(int)</code>.
* @tparam T type of variable
* @param x argument
* @return Arc cosine of variable in radians.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolute value

*
* @tparam Container Type of x
* @param x argument
* @return Arc cosine of each variable in the container, in radians.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolute value

@@ -83,8 +83,9 @@ struct acosh_fun {
* @param x container
* @return Elementwise acosh of members of container.
*/
template <typename T, require_all_not_nonscalar_prim_or_rev_kernel_expression_t<
T>* = nullptr>
template <
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not require_container_st?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is the other thing I'd expect:

template <typename Container,
          require_container_st<std::is_arithmetic, Container>* = nullptr>
inline auto acosh(const Container& x) {
  return apply_vector_unary<Container>::apply(
      x, [](const auto& v) { return v.array().acosh(); });
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this can just be template <typename T, require_container_t<T>* = nullptr>

Copy link
Member

@bbbales2 bbbales2 Dec 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were some functions here that didn't have apply_vector_unary/Container implementations. I'm assuming it's because the Eigen array functions didn't exist for them to be written (cbrt, this, and others). I don't think cleanup is what we're going for in this pull really, so it's fine to leave it.

Edit: didn't exist at the time these functions were written but do exist now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(But if you want to clean things, feel free, happy to review)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ups actually ignore that, this needs to be able to take in scalar types so that's why we can't use require_constainer_st

stan/math/rev/fun/tanh.hpp Show resolved Hide resolved
template <typename F, typename T>
struct apply_scalar_unary<F, T, require_var_matrix_t<T>> {
/**
* Function return type, which is <code>var</code>.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite var

A << -2.2, -0.8, 0.5, 1 + std::numeric_limits<double>::epsilon(), 1.5, 3, 3.4,
4;
expect_ad_matvar(f, A);
std::vector<Eigen::MatrixXd> A_vec;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The easier way to build these is with initializer lists. Something like:

std::vector<Eigen::MatrixXd> A_vec = { A, A, A };

@@ -16,3 +16,18 @@ TEST(mathMixCore, atan2) {
stan::test::expect_ad(f, 1.2, 3.9);
stan::test::expect_ad(f, 0.5, 2.3);
}
/* Not sure how to test this so I'll come back to it.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now iz the time

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only for unary stuff so another time

using stan::math::asinh;
return asinh(x1);
};
auto com_args = stan::test::internal::common_args();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all these tests should be for varmat with vector, row vector and Matrix vector arguments.

How about:

expect_ad_common_vector_matvar(f);

And it does:

expect_ad_vector_matvar(f, stan::math::to_vector(stan::test::internal::common_args()));
  // not sure if `common_args` is std::vector or not

And that function is something like:

expect_ad_vector_matvar(F& f, const auto& x) {
  Eigen::VectorXd v = x;
  Eigen::RowVectorXd r = x.transpose();
  Eigen::MatrixXd m(x.size(), 2);
  com_args_m.block(0, 0, com_args_v.size(), 1) = x;
  com_args_m.block(0, 1, com_args_v.size(), 1) = x.reverse(); // reverse just to mix stuff up
  
  expect_ad_matvar(f, v);
  expect_ad_matvar(f, r);
  expect_ad_matvar(f, m);

  std::vector<Eigen::VectorXd> vv = { v, v };
  std::vector<Eigen::RowVectorXd> rv = { r, r };
  std::vector<Eigen::MatrixXd> mv = { m, m };

  expect_ad_matvar(f, vv);
  expect_ad_matvar(f, rv);
  expect_ad_matvar(f, mv);
}

Also does std::vector<std::vector<...>> work? Don't think it needs tested I'm just curious.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think it makes sense to test matrix vector row vector at least to make sure they compile. I'll write something like this up

SteveBronder and others added 2 commits December 17, 2020 19:17
…unary. Uses .adj() method instead of accessing .adj_ directly for vari
@stan-buildbot
Copy link
Contributor


Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.44 3.44 1.0 0.0% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.96 -3.64% slower
eight_schools/eight_schools.stan 0.11 0.11 1.02 1.59% faster
gp_regr/gp_regr.stan 0.15 0.15 0.99 -0.81% slower
irt_2pl/irt_2pl.stan 5.38 5.39 1.0 -0.24% slower
performance.compilation 90.28 88.4 1.02 2.08% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.41 8.39 1.0 0.16% faster
pkpd/one_comp_mm_elim_abs.stan 28.41 30.64 0.93 -7.85% slower
sir/sir.stan 133.34 132.63 1.01 0.54% faster
gp_regr/gen_gp_data.stan 0.04 0.04 1.0 -0.49% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 3.0 2.92 1.03 2.85% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.39 0.41 0.94 -6.39% slower
arK/arK.stan 2.47 2.47 1.0 0.06% faster
arma/arma.stan 0.89 0.88 1.02 1.62% faster
garch/garch.stan 0.53 0.53 1.0 -0.28% slower
Mean result: 0.993654976218

Jenkins Console Log
Blue Ocean
Commit hash: a7deb36


Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Copy link
Member

@bbbales2 bbbales2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the fabs/abs tests to work like the rest of them, and also a couple things with docs.

Let me know if this works for you and I'll merge when we get greenlight.

template <typename F, typename T>
struct apply_scalar_unary<F, T, require_var_matrix_t<T>> {
/**
* Function return type, which is `var_value<plain_type_t<T>>`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite correct. I'm not sure exactly how to say this, but it's a var_value with a plain inner type?

I went with:

Function return type, which is a `var_value` with plain value type.

@stan-buildbot
Copy link
Contributor


Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.48 3.38 1.03 2.76% faster
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.99 -1.2% slower
eight_schools/eight_schools.stan 0.11 0.11 0.97 -2.99% slower
gp_regr/gp_regr.stan 0.15 0.16 0.98 -2.51% slower
irt_2pl/irt_2pl.stan 5.39 5.38 1.0 0.31% faster
performance.compilation 89.16 87.88 1.01 1.43% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.37 8.41 1.0 -0.5% slower
pkpd/one_comp_mm_elim_abs.stan 28.94 29.04 1.0 -0.34% slower
sir/sir.stan 133.13 128.57 1.04 3.42% faster
gp_regr/gen_gp_data.stan 0.04 0.04 0.99 -1.09% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 2.94 2.93 1.0 0.19% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.39 0.4 0.96 -3.65% slower
arK/arK.stan 2.49 2.47 1.01 0.62% faster
arma/arma.stan 0.89 0.88 1.01 1.15% faster
garch/garch.stan 0.54 0.54 0.99 -0.58% slower
Mean result: 0.998388526662

Jenkins Console Log
Blue Ocean
Commit hash: 7e41553


Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

@SteveBronder
Copy link
Collaborator Author

Ty! Yes I just went through and removed the .noalias() call where we didn't need them so should be cool and good

@stan-buildbot
Copy link
Contributor


Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.42 3.5 0.98 -2.11% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.96 -4.29% slower
eight_schools/eight_schools.stan 0.11 0.11 0.98 -2.19% slower
gp_regr/gp_regr.stan 0.15 0.15 0.99 -0.82% slower
irt_2pl/irt_2pl.stan 5.47 5.49 0.99 -0.52% slower
performance.compilation 90.26 88.18 1.02 2.3% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.41 8.41 1.0 -0.04% slower
pkpd/one_comp_mm_elim_abs.stan 29.16 30.41 0.96 -4.29% slower
sir/sir.stan 128.46 138.9 0.92 -8.13% slower
gp_regr/gen_gp_data.stan 0.04 0.04 0.99 -0.95% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 2.92 2.92 1.0 0.01% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.42 0.39 1.06 5.63% faster
arK/arK.stan 2.47 2.46 1.0 0.46% faster
arma/arma.stan 0.88 0.9 0.98 -2.19% slower
garch/garch.stan 0.54 0.54 1.0 -0.13% slower
Mean result: 0.989491849075

Jenkins Console Log
Blue Ocean
Commit hash: c5caa59


Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

@bbbales2 bbbales2 merged commit a87459f into develop Dec 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants