Skip to content

viktorvia/python-multi-issue

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Minimal code to reproduce a weird issue with leftover defunct processes from python's multiprocess.pool() in spawn mode.

The code generating the defunct processes is in multi.py and is the actual pool() example just with added code to enable spawn mode.

To actually see the zombies the whole thing is run from another python program.

Things work correctly in a normal environment or when the running process is aware of SIGCHLD and handles it by wait(2)ing.

Build

docker build -t multi .

Run without issues form bash

$ docker run -it multi bash main.sh
[1, 4, 9]
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  3 11:31 pts/0    00:00:00 bash main.sh
root        10     1  4 11:31 pts/0    00:00:00 /usr/local/bin/python -c from multiprocessing.resource_tracker import main;main(5)
root        19     1  0 11:32 pts/0    00:00:00 ps -ef --forest
[1, 4, 9]
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  1 11:31 pts/0    00:00:00 bash main.sh
root        22     1  0 11:32 pts/0    00:00:00 /usr/local/bin/python -c from multiprocessing.resource_tracker import main;main(5)
root        31     1  0 11:32 pts/0    00:00:00 ps -ef --forest
[1, 4, 9]
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  1 11:31 pts/0    00:00:00 bash main.sh
root        34     1  0 11:32 pts/0    00:00:00 /usr/local/bin/python -c from multiprocessing.resource_tracker import main;main(5)
root        43     1  0 11:32 pts/0    00:00:00 ps -ef --forest
...

Running from python produces defunct processes

$ docker run -it multi python main.py
[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0 11 11:33 pts/0    00:00:00 python main.py
root         8     1  0 11:33 pts/0    00:00:00 [python] <defunct>
root        17     1  0 11:33 pts/0    00:00:00 ps -ef --forest

[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  6 11:33 pts/0    00:00:00 python main.py
root         8     1  3 11:33 pts/0    00:00:00 [python] <defunct>
root        19     1  0 11:33 pts/0    00:00:00 [python] <defunct>
root        28     1  0 11:33 pts/0    00:00:00 ps -ef --forest

[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  4 11:33 pts/0    00:00:00 python main.py
root         8     1  1 11:33 pts/0    00:00:00 [python] <defunct>
root        19     1  3 11:33 pts/0    00:00:00 [python] <defunct>
root        30     1  0 11:33 pts/0    00:00:00 [python] <defunct>
root        39     1  0 11:33 pts/0    00:00:00 ps -ef --forest

[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  3 11:33 pts/0    00:00:00 python main.py
root         8     1  1 11:33 pts/0    00:00:00 [python] <defunct>
root        19     1  1 11:33 pts/0    00:00:00 [python] <defunct>
root        30     1  4 11:33 pts/0    00:00:00 [python] <defunct>
root        41     1  0 11:33 pts/0    00:00:00 [python] <defunct>
root        50     1  0 11:33 pts/0    00:00:00 ps -ef --forest
...

Running from SIGCHLD-aware python program avoids the issue

$ docker run -it multi python main-sig.py
[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0 11 11:34 pts/0    00:00:00 python main-sig.py
root        18     1  0 11:34 pts/0    00:00:00 ps -ef --forest

[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  5 11:34 pts/0    00:00:00 python main-sig.py
root        29     1  0 11:34 pts/0    00:00:00 ps -ef --forest

[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  3 11:34 pts/0    00:00:00 python main-sig.py
root        40     1  0 11:34 pts/0    00:00:00 ps -ef --forest

[1, 4, 9]

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  3 11:34 pts/0    00:00:00 python main-sig.py
root        51     1  0 11:34 pts/0    00:00:00 ps -ef --forest
...

About

Minimal case for a multiprocessing issue with python

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published