Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Minor] Implement active set tracking in the inner coordinate descent loop #211

Open
tbenthompson opened this issue Jun 17, 2020 · 1 comment
Labels
on hold not now, maybe never

Comments

@tbenthompson
Copy link
Collaborator

This is a leftover task from #92

For narrow problems where we compute a full hessian, the inner CD loop is so fast that this would not be helpful to do. However, for future solvers for wide problems where we can't precompute a full hessian, I suspect that implementing inner loop active set logic would be very helpful!

@tbenthompson tbenthompson added the on hold not now, maybe never label Jul 7, 2020
@tbenthompson
Copy link
Collaborator Author

The result of #424 makes me think that this issue might have a substantial impact in certain narrow cases if someone ever wants to implement it.

This will have a large impact if:

  1. The problem is coordinate descent dominated. This is normally only true for gaussian distribution and l1 penalty settings (LASSO).
  2. The problem has a large penalty so that many coeffs will be zero.

But, in that situation, I could see it potentially leading to decent performance improvements because the innermost loop of the CD solver will iterate over a much smaller list of coefficients.

I'm leaving this "on hold" because it's not a high priority right now but I just wanted to share my thoughts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
on hold not now, maybe never
Projects
None yet
Development

No branches or pull requests

1 participant