Skip to content

Commit

Permalink
If speedtest fails, assume network speed of 100 Mbit/s (#404)
Browse files Browse the repository at this point in the history
The value is chosen as some safe value below average at https://health.petals.dev/

Note that if a server uses relays, the effective throughput will be further divided by 2 (see #399).
  • Loading branch information
borzunov committed Jul 22, 2023
1 parent 8666653 commit 30b94ef
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/petals/server/throughput.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ def measure_throughput_info(


def measure_network_rps(
config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 25e6
config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 100e6 # 100 Mbit/s
) -> Optional[float]:
bits_per_request = config.hidden_size * 16 # Clients usually send 16-bit tensors for forward/backward
try:
Expand Down

0 comments on commit 30b94ef

Please sign in to comment.