Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement shortest-path routing for inference #362

Merged
merged 13 commits into from
Jul 18, 2023
Merged

Implement shortest-path routing for inference #362

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
2d04c79
Draft shortest-path routing
borzunov Jul 17, 2023
8ae9d9a
Add debug outputs
borzunov Jul 17, 2023
4bd3c17
Update comment in _has_cache_for()
borzunov Jul 18, 2023
9e21cca
Go to the end of the server to avoid O(N^2) graphs for N servers
borzunov Jul 18, 2023
cd4b374
Hide debug logging
borzunov Jul 18, 2023
5e8be84
Merge branch 'main' into shortest-path
borzunov Jul 18, 2023
fd4cb43
Update config.show_route param
borzunov Jul 18, 2023
7c0e679
Fix bug with incomplete first update()
borzunov Jul 18, 2023
59b7f2a
Decrease DHT's num_workers
borzunov Jul 18, 2023
cb4e6ef
Remove unused arg
borzunov Jul 18, 2023
7e33661
Take into account empirically measured overheads
borzunov Jul 18, 2023
f614702
Increase dev version
borzunov Jul 18, 2023
ede942f
Decrease default alloc_timeout
borzunov Jul 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Decrease DHT's num_workers
  • Loading branch information
@borzunov
borzunov committed Jul 18, 2023
commit 59b7f2acab76b370a85b6f741a1fc86ebe20a026
2 changes: 1 addition & 1 deletion src/petals/client/routing/sequence_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def __init__(
dht = DHT(
initial_peers=config.initial_peers,
client_mode=True,
num_workers=config.num_hidden_layers,
num_workers=32,
startup_timeout=config.daemon_startup_timeout,
start=True,
)
Expand Down
Loading