Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inclusion of InternVLChatModel In PP_SUPPORTED_MODELS(Pipeline Parallelism) #7860

Merged
merged 56 commits into from
Sep 5, 2024
Merged
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
4e260ae
Merge pull request #1 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 21, 2024
dbf6ee2
changes for internvl pipeline parallelism
Manikandan-Thangaraj-ZS0321 Aug 21, 2024
e36dfd2
Merge pull request #2 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 21, 2024
08b8538
Inclusion of InternVLChatModel in PP_SUPPORTED_MODELS
Manikandan-Thangaraj-ZS0321 Aug 22, 2024
5129c87
Merge pull request #3 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 22, 2024
5efea42
Merge pull request #4 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 22, 2024
3aca806
Merge pull request #5 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 23, 2024
c7ef1aa
Merge pull request #6 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
d7f2d54
refactor
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
a20827e
Merge remote-tracking branch 'origin/main' into main
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
caaca9a
refactor
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
2137631
refactor
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
927e3f8
refactor
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
7e8ef5c
refactor
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
c891114
refactor
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
85168ed
Merge pull request #7 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 26, 2024
80fa9dd
Merge pull request #8 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 27, 2024
654248c
Added the InternVL2-8B for testing the pipeline parallelism in test_p…
Manikandan-Thangaraj-ZS0321 Aug 27, 2024
a2fd2b6
Merge remote-tracking branch 'origin/main' into main
Manikandan-Thangaraj-ZS0321 Aug 27, 2024
6923683
Merge pull request #9 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 27, 2024
cf155c2
updating branch
Manikandan-Thangaraj-ZS0321 Aug 28, 2024
8ca55fa
Merge pull request #10 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Aug 28, 2024
4ae7573
updation
Manikandan-Thangaraj-ZS0321 Aug 28, 2024
595bd82
Merge remote-tracking branch 'origin/main' into main
Manikandan-Thangaraj-ZS0321 Aug 28, 2024
7aefcbf
updation
Manikandan-Thangaraj-ZS0321 Aug 28, 2024
4f7b9a5
updating branch
Manikandan-Thangaraj-ZS0321 Aug 28, 2024
86fb726
Merge pull request #12 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Sep 2, 2024
af98e95
Merge pull request #13 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Sep 2, 2024
530c8a3
Updating Branch
Manikandan-Thangaraj-ZS0321 Sep 2, 2024
88186f4
Merge remote-tracking branch 'origin/main' into main
Manikandan-Thangaraj-ZS0321 Sep 2, 2024
0d6ac3a
Updating Branch
Manikandan-Thangaraj-ZS0321 Sep 2, 2024
caaffb9
Updating Branch
Manikandan-Thangaraj-ZS0321 Sep 2, 2024
831d447
Refactor
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
a3e9a98
Refactor
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
e218d07
Refactor
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
ca6e920
test case completion
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
7591225
test case completion
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
85504b1
fixing imports
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
d14d6b6
fixing imports in utils.py
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
63039c2
Merge pull request #16 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
41f83dd
optional settings for tokeniser for trust_remote_code
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
3a658ea
Merge remote-tracking branch 'origin/main' into main
Manikandan-Thangaraj-ZS0321 Sep 3, 2024
0034663
InternLM2ForCausalLM and internlm/internlm2_5-7b-chat inclusion
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
6a78c24
Merge pull request #17 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
0ebf029
line formatting
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
f689444
line formatting
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
0cd208c
sorting in utils.py
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
9c3ef5f
formatting
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
7e32e34
formatting
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
468c994
formatting
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
92cfe53
formatting _PP_SUPPORTED_MODELS list
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
9a31c28
removal of internlm/internlm2_5-7b-chat in test_pipeline_parallel.py
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
158bf06
Merge pull request #18 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Sep 4, 2024
91dde87
Merge pull request #19 from vllm-project/main
Manikandan-Thangaraj-ZS0321 Sep 5, 2024
cb3f602
increasing TP_SIZE in test_pipeline_parallel.py
Manikandan-Thangaraj-ZS0321 Sep 5, 2024
5d9bc77
Merge remote-tracking branch 'origin/main' into main
Manikandan-Thangaraj-ZS0321 Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
line formatting
  • Loading branch information
Manikandan-Thangaraj-ZS0321 committed Sep 4, 2024
commit 0ebf029c3c108c72ee697de0c0cfd1974ed29efd
36 changes: 20 additions & 16 deletions tests/distributed/test_pipeline_parallel.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,22 +18,26 @@
VLLM_MULTI_NODE = os.getenv("VLLM_MULTI_NODE", "0") == "1"


@pytest.mark.parametrize(("TP_SIZE, PP_SIZE, EAGER_MODE, CHUNKED_PREFILL, TRUST_REMOTE_CODE, "
"MODEL_NAME, DIST_BACKEND"),
[
(2, 2, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(2, 2, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 3, 0, 0, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 4, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 4, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 3, 0, 0, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(1, 4, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(1, 4, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(2, 2, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(2, 2, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(1, 2, 1, 1, 1, "OpenGVLab/InternVL2-8B", "ray"),
(1, 2, 1, 1, 1, "internlm/internlm2_5-7b-chat", "ray")
])
@pytest.mark.parametrize(
(
"TP_SIZE, PP_SIZE, EAGER_MODE, CHUNKED_PREFILL, TRUST_REMOTE_CODE, "
"MODEL_NAME, DIST_BACKEND"
),
[
(2, 2, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(2, 2, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 3, 0, 0, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 4, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 4, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "mp"),
(1, 3, 0, 0, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(1, 4, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(1, 4, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(2, 2, 1, 0, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(2, 2, 0, 1, 0, "meta-llama/Meta-Llama-3-8B", "ray"),
(1, 2, 1, 1, 1, "OpenGVLab/InternVL2-8B", "ray"),
(1, 2, 1, 1, 1, "internlm/internlm2_5-7b-chat", "ray"),
],
)
@fork_new_process_for_each_test
def test_compare_tp(TP_SIZE, PP_SIZE, EAGER_MODE, CHUNKED_PREFILL, TRUST_REMOTE_CODE, MODEL_NAME,
DIST_BACKEND):
Expand Down
Loading