Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis stream - use streams XLEN instead of XPENDING #4277

Closed
jkbmdk opened this issue Feb 24, 2023 · 3 comments
Closed

Redis stream - use streams XLEN instead of XPENDING #4277

jkbmdk opened this issue Feb 24, 2023 · 3 comments
Assignees
Labels
feature-request All issues for new features that have not been committed to needs-discussion

Comments

@jkbmdk
Copy link
Contributor

jkbmdk commented Feb 24, 2023

Proposal

When it comes to using redis streams as a message queue, as php symfony messenger does, declaring the XPENDING parameter as a scaler attribute will not be useful in some scenarios, because it usually returns the current number of subscribers.
The solution I would propose is to make the redis stream scaler optionally dependent on the XLEN parameter.

Use-Case

First of all we are using php symfony messenger and redis. The only way to combine those two is by using redis streams. In our case, we publish many messages once a day and much less throughout the day. Our goal is to quickly subscribe messages from the initial feed (say, within a few hours), and then maintain enough subscribers to handle incoming messages.

Is this a feature you are interested in implementing yourself?

Maybe

Anything else?

No response

@jkbmdk jkbmdk added feature-request All issues for new features that have not been committed to needs-discussion labels Feb 24, 2023
@JorTurFer
Copy link
Member

Hello,
This makes sense totally. Are you willing to contribute?

@jkbmdk
Copy link
Contributor Author

jkbmdk commented Feb 25, 2023

Sure, should the solution be implemented for redis streams scaler as an additional configuration parameter and when it would be set there would be no need to set pendingEntriesCount or consumerGroup?

@JorTurFer
Copy link
Member

I think so, you can make them optional and raise an error in case of they are required and they are missing (and properly document it xD)

jkbmdk pushed a commit to jkbmdk/keda that referenced this issue Mar 20, 2023
jkbmdk pushed a commit to jkbmdk/keda that referenced this issue Mar 20, 2023
Signed-off-by: jmadajczak <jmadajczak@transparentdata.pl>
jkbmdk pushed a commit to jkbmdk/keda that referenced this issue Mar 20, 2023
Signed-off-by: jmadajczak <jmadajczak@transparentdata.pl>
jkbmdk pushed a commit to jkbmdk/keda that referenced this issue Mar 20, 2023
Signed-off-by: jmadajczak <jmadajczak@transparentdata.pl>
jkbmdk pushed a commit to jkbmdk/keda that referenced this issue Mar 20, 2023
Signed-off-by: jmadajczak <jmadajczak@transparentdata.pl>
JorTurFer added a commit that referenced this issue Mar 28, 2023
Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
Co-authored-by: jmadajczak <jmadajczak@transparentdata.pl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request All issues for new features that have not been committed to needs-discussion
Projects
Status: Ready To Ship
Development

No branches or pull requests

2 participants