Skip to content

Commit

Permalink
GitBook: [taskforcesh#104] No subject
Browse files Browse the repository at this point in the history
  • Loading branch information
manast authored and gitbook-bot committed Mar 11, 2022
1 parent b60ac86 commit f718340
Show file tree
Hide file tree
Showing 7 changed files with 79 additions and 12 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion docs/gitbook/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,8 @@

* [Introduction](bullmq-pro/introduction.md)
* [Install](bullmq-pro/install.md)
* [Observables](bullmq-pro/observables.md)
* [Observables](bullmq-pro/observables/README.md)
* [Cancelation](bullmq-pro/observables/cancelation.md)
* [Groups](bullmq-pro/groups/README.md)
* [Rate limiting](bullmq-pro/groups/rate-limiting.md)
* [Concurrency](bullmq-pro/groups/concurrency.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/gitbook/bull/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ Bull is the legacy version of BullMQ. As it is still heavily used today, it is a

Bull has been a part of the NodeJS ecosystem for a long time and is used by many organizations both in commercial and open-source projects. A few special mentions:

![](<../.gitbook/assets/Screenshot 2022-02-15 at 11.32.39 (1).png>)![](../.gitbook/assets/mozilla-logo-bw-rgb.png)![](../.gitbook/assets/autodesk-logo-white.png)![](../.gitbook/assets/Atlassian-horizontal-blue-rgb.webp)
![](<../.gitbook/assets/Screenshot 2022-02-15 at 11.32.39 (1).png>)![](<../.gitbook/assets/mozilla-logo-bw-rgb (2).png>)![](../.gitbook/assets/autodesk-logo-white.png)![](<../.gitbook/assets/Atlassian-horizontal-blue-rgb (1).webp>)

![](../.gitbook/assets/midwayjs-logo.png)![](<../.gitbook/assets/salesforce-logo (1).png>)
![](../.gitbook/assets/midwayjs-logo.png)![](../.gitbook/assets/salesforce-logo.png)

![](<../.gitbook/assets/entethalliance-logo (1).png>)![](../.gitbook/assets/kisspng-logo-retail-target-corporation-advertising-5ae5ef43944c89.3404142515250184356074.png)

8 changes: 0 additions & 8 deletions docs/gitbook/bullmq-pro/observables.md

This file was deleted.

62 changes: 62 additions & 0 deletions docs/gitbook/bullmq-pro/observables/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Observables

Instead of returning regular promises, your workers can also return an Observable, this allows for some more advanced uses cases:

* It makes it possible to cleanly cancel a running job.
* You can define a "Time to live" (TTL) so that jobs that take too long time will be automatically canceled.
* Since the last value returned by the observable is persisted, you could retry a job and continue where you left of, for example, if the job implements a state machine or similar.

If you are new to Observables you may want to read this [introduction](https://www.learnrxjs.io/learn-rxjs/concepts/rxjs-primer). The two biggest advantages that Observables have over Promises are that they can emit more than 1 value and that they are cancelable.

Let's see a silly example of a worker making use of Observables:

```typescript
import { WorkerPro } from "@taskforcesh/bullmq-pro"
importObservable } from "rxjs"

const processor = async () => {
return new Observable<number>(subscriber => {
subscriber.next(1);
subscriber.next(2);
subscriber.next(3);
const intervalId = setTimeout(() => {
subscriber.next(4);
subscriber.complete();
}, 500);

// Provide a way of canceling and disposing the interval resource
return function unsubscribe() {
clearInterval(intervalId);
};
});
};

const worker = new WorkerPro(queueName, processor, { connection });

```

In the example above, the observable will emit 4 values, the first 3 directly and then a 4th after 500 ms. Also note that the "subscriber" returns a "unsubscribe" function. This is the function that will be called if the Observable is cancelled, so this is where you would do the necessary clean up.

You may be asking whats the use of returning several values for a worker. One case that comes to mind is if you have a larger processor and you want to make sure that if the process crashes you can continue from the latest value. You could do this with a simple switch-case on the return value, something like this:

```typescript
import { WorkerPro } from "@taskforcesh/bullmq-pro"
importObservable } from "rxjs"

const processor = async (job) => {
return new Observable<number>(subscriber => {
switch(job.returnvalue){
default:
subscriber.next(1);
case 1:
subscriber.next(2);
case 2:
subscriber.next(3);
case 3:
subscriber.complete();
}
});
};

const worker = new WorkerPro(queueName, processor, { connection });
```
12 changes: 12 additions & 0 deletions docs/gitbook/bullmq-pro/observables/cancelation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Cancelation

As mentioned, Observables allows for clean cancelation. Currently we support a TTL value that defines the maximum processing time before the job is finally cancelled:

```typescript
import { WorkerPro } from "@taskforcesh/bullmq-pro"

const worker = new WorkerPro(queueName, processor, {
ttl: 100,
connection,
});
```
2 changes: 1 addition & 1 deletion docs/gitbook/guide/jobs/getters.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

When jobs are added to a queue, they will be in different statuses during their lifetime. BullMQ provides methods to retrieve information and jobs from the different statuses.

![Lifecycle of a job](../../.gitbook/assets/architecture.png)
![Lifecycle of a job](../../.gitbook/assets/complete-architecture.png)

#### Job Counts

Expand Down

0 comments on commit f718340

Please sign in to comment.