Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add LLaMA end-to-end benchmarking #19985

Merged
merged 18 commits into from
Mar 22, 2024
Merged

Add LLaMA end-to-end benchmarking #19985

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
cad5aab
Add E2E benchmarking
kunal-vaishnavi Mar 17, 2024
3abc6fe
Add license to files and changes suggested by linter
kunal-vaishnavi Mar 18, 2024
72c92b8
Fix runtime errors
kunal-vaishnavi Mar 18, 2024
ac1f266
Update README
kunal-vaishnavi Mar 18, 2024
f3d9036
Update CSV labels
kunal-vaishnavi Mar 19, 2024
45774c2
Add cache dir
kunal-vaishnavi Mar 19, 2024
8172c28
Add changes suggested by linter
kunal-vaishnavi Mar 19, 2024
5ff17cf
Fix comment formatting
kunal-vaishnavi Mar 20, 2024
33897a2
Fix linter error in CI that differs locally
kunal-vaishnavi Mar 20, 2024
5736e14
Update README
kunal-vaishnavi Mar 20, 2024
559a738
Fix bug with input shapes
kunal-vaishnavi Mar 20, 2024
c44005b
Merge branch 'main' into kvaishnavi/llama-e2e
kunal-vaishnavi Mar 20, 2024
f0ebf4d
Add changes suggested by linter
kunal-vaishnavi Mar 20, 2024
b3971d1
Add longer prompts
kunal-vaishnavi Mar 21, 2024
0b21f14
Add error message for unrecognized prompt lengths
kunal-vaishnavi Mar 21, 2024
30a5973
Add local directory option for loading Hugging Face models
kunal-vaishnavi Mar 21, 2024
2dc0762
Clarify arg descriptions
kunal-vaishnavi Mar 21, 2024
34110f5
Update local directory arg description
kunal-vaishnavi Mar 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update README
  • Loading branch information
@kunal-vaishnavi
kunal-vaishnavi committed Mar 20, 2024
commit 5736e140fb51792bb490386639e7e610595bc946
6 changes: 3 additions & 3 deletions onnxruntime/python/tools/transformers/models/llama/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,7 @@ CUDA_VISIBLE_DEVICES=0 python3 -m models.llama.benchmark_e2e \
--prompts-file ./models/llama/prompts.json \
--precision fp16 \
--batch-sizes "1 2" \
--sequence-lengths "16 64" \
--prompt-lengths "16 64" \
--device cuda \
--auth
```
Expand All @@ -473,7 +473,7 @@ CUDA_VISIBLE_DEVICES=0 python3 -m models.llama.benchmark_e2e \
--prompts-file ./models/llama/prompts.json \
--precision fp32 \
--batch-sizes "1 2" \
--sequence-lengths "16 64" \
--prompt-lengths "16 64" \
--device cpu \
--auth
```
Expand All @@ -488,7 +488,7 @@ CUDA_VISIBLE_DEVICES=0 python3 -m models.llama.benchmark_e2e \
--prompts-file ./models/llama/prompts.json \
--precision fp16 \
--batch-sizes "1 2" \
--sequence-lengths "16 64" \
--prompt-lengths "16 64" \
--device cuda \
--use_buffer_share \
--auth
Expand Down
Loading