This tool allows to execute a function inside a thread and provides an easy way to manage dependencies beetween a functions pool. Any function can also be run in multiple parallel copies. It's possible to define one or more dependencies for each function and Jobber will buid and optimize the dependy tree calculating which functions can be executed in parallel.
Take a look at jobber_test.py
to have an example of the power
of Jobber!
from jobber import Jobber
Set concunrrency (best is number of cores) and initialize Jobber
concurrency = 3
jobber = Jobber(concurrency)
Get the decorator
jobberd = jobber.decorator
@jobberd([])
def fn1():
pass
@jobberd([])
def fn2():
pass
@jobberd(['fn1','fn2'])
def fn3():
pass
fn1
and fn2
run in parallel and fn3
will run only when the
first two are completed.
@jobberd([],4)
def fn():
pass
Function fn
is queued 4 times.
Because concurrency is set to 3, Jobber will run 3 fn in parallel
and than the fourth.
Parallelism between dependency is mandatory and has to equal.
@jobberd([])
def fn1():
pass
@jobberd([],2)
def fn2():
pass
@jobberd(['fn2'],2)
def fn3():
pass
fn1
run in parallel with fn2-0
and fn2-1
.
fn3-0
depends from fn2-0
and fn3-1
depends from fn2-1