-
Notifications
You must be signed in to change notification settings - Fork 390
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flow doesn't follow Queue's default job options #1034
Comments
hi @hicaro, for a flow producer, you could pass flowQueueOpts when adding a flow, that flowQueueOpts could handle the defaultJobOpts: |
I just ran into the same issue. Yes, I can pass the options into the job, but is that really the way to go? I'm expecting the job to always respect the default queue options. Why not for flows? |
@vuhrmeister ok, there seems to be some confusion here. So when you want to work with a flow you need to use the FlowProducer class to add new jobs to the Queue https://docs.bullmq.io/guide/flows |
Yes, I do use the FlowProducer and that works all fine as described. My point is: Even the FlowProducer doesn't work for itself but uses defined queues to process a job, being the parent or the children. And queues are defined somewhere. And queues have some defaultJobOptions. And even a Flow job is a job which gets processed by the queue. And that's why I expect the executed job to respect the queues defaultJobOptions. |
@vuhrmeister could you provide a small piece of code that illustrates what you mean that can help me understand? |
@manast, I agree with @vuhrmeister. If the target queue in which the flow producer will place the job has default job options, can't those options be observed if none were provided when adding the job through the flow producer? For instance, taking the queue from the initial example of this issue, if we add a job as (without the await flowProducer.add({
name: 'a-job',
queueName: 'a-queue',
}); Why couldn't it use the default options as: {
attempts: 3,
delay: 500,
removeOnComplete: 25,
removeOnFail: 25,
backoff: {
type: 'exponential',
delay: 5000,
},
} |
@hicaro can you write a complete example on what you expect? I still don't get what you mean. |
@manast take this as the queue: const queue = new Queue('a-queue', {
connection: redisClient,
defaultJobOptions: {
attempts: 3,
delay: 500,
removeOnComplete: 25,
removeOnFail: 25,
backoff: {
type: 'exponential',
delay: 5000,
},
},
}); What the point of this issue is allowing developers to be able to do this: await flowProducer.add({
name: 'a-job',
queueName: 'a-queue',
}); Instead of this since we have a await flowProducer.add({
name: 'a-job',
queueName: 'a-queue',
opts: {
attempts: 3,
delay: 500,
removeOnComplete: 25,
removeOnFail: 25,
backoff: {
type: 'exponential',
delay: 5000,
},
}
}); |
@hicaro |
Okay, I was under the impression that those options would be persisted somewhere. Would it be possible to have the flow producer instance have its own default job options? It gets very repetitive to copy and paste the same options every time a new flow is created. |
Can't you just store the default options in a const and just use the same const on all the calls to flowProducer.add? |
@hicaro actually it is possible to pass default opts in the constructor to the FlowProducer by passing an object mapping the queue name to the options as @roggervalf pointed out earlier: #1034 (comment) |
Thx @hicaro for posting some details. That's exactly my issue. @manast For me it's fine passing the options to the flow options when calling Anyway, it's just confusing since you still run those jobs on a queue. It's still a job. And that's why it's not self explanatory that the default options on the queue doesn't count for the flow. So if it's really not possible then it should be properly documented. I'm sure that this doesn't only confuse me. |
@manast I see what you mean, but it is still tied to an individual call of I agree with @vuhrmeister, we might not be the only ones confused by this. |
🎉 This issue has been resolved in version 1.74.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
The default job options set on the queue level are not obeyed when enqueuing jobs with the flow producer.
For instance, creating a queue like this:
The job dispatched by the flow producer will have the following properties:
Is there anything else I should be doing for it to work correctly?
The text was updated successfully, but these errors were encountered: