Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Llama, rebalancing, throughput eval, and all CLI scripts #452

Merged
merged 32 commits into from
Aug 8, 2023
Merged
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
69abacc
Show argparse defaults, fix docstring
borzunov Aug 8, 2023
ca2850e
Test petals.cli.run_dht
borzunov Aug 8, 2023
816401e
Increase mean_block_selection_delay
borzunov Aug 8, 2023
7330653
Test rebalancing
borzunov Aug 8, 2023
a00e79d
Add help to benchmark argparse
borzunov Aug 8, 2023
5b3d4c4
Use less RAM
borzunov Aug 8, 2023
2b765b9
Don't set default model in benchmarks
borzunov Aug 8, 2023
fae58d9
Fix sleep time
borzunov Aug 8, 2023
856f53f
Test --throughput eval
borzunov Aug 8, 2023
05dc383
Fix flapping test
borzunov Aug 8, 2023
18e5b00
Use AutoDistributed{Config,Model} in tests
borzunov Aug 8, 2023
168e478
Add Maykeye/TinyLLama-v0 to tests
borzunov Aug 8, 2023
5760b15
Test using includes only
borzunov Aug 8, 2023
015238a
Adjust --num_blocks and --block_indices for 8-layer TinyLlama-v0
borzunov Aug 8, 2023
17cae64
Refactor matrix
borzunov Aug 8, 2023
b7b7464
Fix commands
borzunov Aug 8, 2023
c907990
Skip TP tests for llama
borzunov Aug 8, 2023
0040539
Fix test_greedy_generation() for llama
borzunov Aug 8, 2023
a5a95c4
Fix commands
borzunov Aug 8, 2023
c3e7638
Fix test_server_info()
borzunov Aug 8, 2023
b622a14
Fix server layout
borzunov Aug 8, 2023
8a379aa
Try reducing RAM usage
borzunov Aug 8, 2023
ecd7d3f
Check if benchmarks work
borzunov Aug 8, 2023
6ffbc28
Watch free RAM (common issue in CI)
borzunov Aug 8, 2023
033a3ca
Reduce RAM further
borzunov Aug 8, 2023
f06cebd
Tune constants to save RAM
borzunov Aug 8, 2023
47d2d53
Speed benchmark tests
borzunov Aug 8, 2023
d8e08e6
Fix flapping test
borzunov Aug 8, 2023
315c5c6
Try --no_relay
borzunov Aug 8, 2023
5cbb33b
Increase swap space
borzunov Aug 8, 2023
54cd213
Fix flapping test
borzunov Aug 8, 2023
1e34dfd
Fix flapping test
borzunov Aug 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix test_greedy_generation() for llama
  • Loading branch information
borzunov committed Aug 8, 2023
commit 0040539202ce2f3a41301ea5983341ae8100e511
3 changes: 3 additions & 0 deletions tests/test_full_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,12 @@ def test_greedy_generation(tokenizer, max_new_tokens=4):
hf_outputs = HfGenerationMixin.greedy_search(model, input_ids=inputs, max_length=inputs.size(1) + max_new_tokens)
assert torch.allclose(remote_outputs, hf_outputs), "Greedy search results are not identical to HF"

if tokenizer.pad_token_id is None:
tokenizer.pad_token_id = tokenizer.eos_token_id
inputs_batch = tokenizer(["A cat sat on a mat", "A dog sat on a mat"], return_tensors="pt", padding=True)[
"input_ids"
]

remote_outputs_batch = model.generate(
inputs_batch,
max_new_tokens=max_new_tokens,
Expand Down
Loading