Make GCP PubSub Scaler more flexible

[ken8203]

Proposal

Recently we are adopting KEDA due to its rich scalers, which are fantastic.

There are a bunch of workloads that need to be scaled by GCP PubSub. One of metrics we used is pubsub.googleapis.com/subscription/pull_ack_request_count, which is not supported by KEDA.

According to the current design, if we want to add support to new metrics¹, what we do is mostly like #4239.

Thus, in order to make it more flexible, I would like to propose a new way for pubsubScaler. Perhaps we don't need to hardcode metrics, instead, we convert the mode defined in the ScaledObject CRD.

For example, it is the scaled object manifest below.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: example
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: example
  triggers:
  - type: gcp-pubsub
    metricType: Value
    metadata:
      mode: PullAckRequestCount
      subscriptionName: example
      value: "10"

Simply convert PullAckRequestCount (camel case) to pull_ack_request_count (snake case), and concatenate it with pubsub.googleapis.com/subscription/, then we get the full metric name.

I'm not sure if it's a good design. Need to discuss this with the community.

BTW, another concern is that it's hard to define default values in this way, people won't use default values in real cases though. Maybe it's okay to give an arbitrary number, e.g. 5.

Use-Case

No response

Is this a feature you are interested in implementing yourself?

Yes

Anything else?

No response

Footnotes

1. https://cloud.google.com/monitoring/api/metrics_gcp#gcp-pubsub ↩

[tomkerkhove]

I see #4239 is already open - Just for future reference; you might want to wait until we give the go in case we want another approach

[ken8203]

I see #4239 is already open - Just for future reference; you might want to wait until we give the go in case we want another approach

Thanks. I just come up with this idea after #4239. : D

[zroubalik]

Hi, in general I like the idea!

Thus, in order to make it more flexible, I would like to propose a new way for pubsubScaler. Perhaps we don't need to hardcode metrics, instead, we convert the mode defined in the ScaledObject CRD.
I concerned that this convertion would cause problems. Maybe we can introduce a new field and slowly deprecate the current mode?

BTW, another concern is that it's hard to define default values in this way, people won't use default values in real cases though. Maybe it's okay to give an arbitrary number, e.g. 5.

which default values do you mean?

[ken8203]

Hi, in general I like the idea!

Thus, in order to make it more flexible, I would like to propose a new way for pubsubScaler. Perhaps we don't need to hardcode metrics, instead, we convert the mode defined in the ScaledObject CRD.
I concerned that this convertion would cause problems. Maybe we can introduce a new field and slowly deprecate the current mode?

BTW, another concern is that it's hard to define default values in this way, people won't use default values in real cases though. Maybe it's okay to give an arbitrary number, e.g. 5.

which default values do you mean?

There are default values for each metric currently. But I think it's no big deal. We could probably set an arbitrary number as the default value.

keda/pkg/scalers/gcp_pubsub_scaler.go

Lines 20 to 21 in 38a0e1c

	defaultTargetSubscriptionSize = 5
	defaultTargetOldestUnackedMessageAge = 10

[ken8203]

Hi @JorTurFer,

I see you've moved this issue to the To Do section. Am I able to draft a PR now? 😄

[JorTurFer]

Hi @JorTurFer,

I see you've moved this issue to the To Do section. Am I able to draft a PR now? 😄

Hey,
Yes, sure! 😄 But you don't need to wait until the issues change their status because sometimes they aren't up-to-date. The state is for internal managing (like knowing in progress/done actions).
After you are sure that the feature/fix is useful, you can draft a PR whenever you want, the status change isn't required :)