Skip to content

Commit

Permalink
Add README
Browse files Browse the repository at this point in the history
  • Loading branch information
gnossen committed Mar 8, 2019
1 parent b2a75fd commit 67ca10b
Show file tree
Hide file tree
Showing 3 changed files with 51 additions and 181 deletions.
54 changes: 51 additions & 3 deletions examples/python/multiprocessing/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,51 @@
TODO: Describe the example.
TODO: Describe how to run the example.
TODO: Describe how to run the test.
## Multiprocessing with gRPC Python

Multiprocessing allows application developers to sidestep the Python global
interpreter lock and achieve true concurrency on multicore systems.
Unfortunately, using multiprocessing and gRPC Python is not yet as simple as
instantiating your server with a `futures.ProcessPoolExecutor`.

The library is implemented as a C extension, maintaining much of the state that
drives the system in native code. As such, upon calling
[`fork`](http://man7.org/linux/man-pages/man2/fork.2.html), much of the
state copied into the child process is invalid, leading to hangs and crashes.

However, calling `fork` without `exec` in your python process is supported
*before* any gRPC servers have been instantiated. Application developers can
take advantage of this to parallelize their CPU-intensive operations.

## Running the Example

This example calculates the first 10,000 prime numbers as an RPC. We instantiate
one server per subprocess, balancing requests between the servers using the
[`SO_REUSEPORT`](https://lwn.net/Articles/542629/) socket option.

To run the server,
[ensure `bazel` is installed](https://docs.bazel.build/versions/master/install.html)
and run:

```
bazel run //examples/python/multiprocessing:server &
```

Note the address at which the server is running. For example,

```
...
[PID 107153] Binding to '[::]:33915'
[PID 107507] Starting new server.
[PID 107508] Starting new server.
...
```

Now, start the client by running

```
bazel run //examples/python/multiprocessing:client -- [SERVER_ADDRESS]
```

For example,

```
bazel run //examples/python/multiprocessing:client -- [::]:33915
```
132 changes: 0 additions & 132 deletions examples/python/multiprocessing/prime_pb2.py

This file was deleted.

46 changes: 0 additions & 46 deletions examples/python/multiprocessing/prime_pb2_grpc.py

This file was deleted.

0 comments on commit 67ca10b

Please sign in to comment.