Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/kan 162 public ip #582

Closed
wants to merge 35 commits into from
Closed

Feature/kan 162 public ip #582

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f02bd57
Fix dummy cache allocation
artek0chumak Apr 11, 2024
e5dddfe
Try mps device selecting
artek0chumak Apr 11, 2024
0ca54a5
Rechain reloc
artek0chumak Apr 11, 2024
c76e447
infrastructure
jmikedupont2 Apr 15, 2024
661545f
tiny mixtral running locally
Apr 16, 2024
be69f31
grok
Apr 16, 2024
c98b532
docker changes
Apr 17, 2024
6124622
mixtral working
Apr 17, 2024
3365e2e
Merge branch 'feature/docker-compose'
Apr 17, 2024
1644e89
reformat black
Apr 17, 2024
db5cb42
update config
Apr 29, 2024
a13e098
Merge pull request #1 from jmikedupont2/feature/dht1
Bakobiibizo Apr 29, 2024
f2092ac
remove deps
Apr 29, 2024
5744205
added setup.sh and added .env.example
jmikedupont2 Apr 29, 2024
41cc828
Merge pull request #2 from jmikedupont2/main
Bakobiibizo Apr 29, 2024
e01d410
Merge branch 'main' into setup-and-env.example
Bakobiibizo Apr 29, 2024
1fcc81b
update
Apr 29, 2024
0bfd0cb
updat
Apr 29, 2024
d9ba0ef
onlineer
Apr 29, 2024
2679513
Merge pull request #4 from jmikedupont2/setup-and-env.example
Bakobiibizo Apr 29, 2024
b84507c
changing defaults to set disk space
May 2, 2024
804de46
sauerkraut
May 8, 2024
41a4aba
remove options
May 8, 2024
e59cc85
block count
May 8, 2024
1615f5e
Update dht1.cillium.prod.compute.agentartificial.com.txt
jmikedupont2 May 9, 2024
90c454c
updated
May 9, 2024
04132bd
update port
jmikedupont2 Apr 30, 2024
dea21b5
changing defaults to set disk space
May 2, 2024
cf70fc1
sauerkraut
May 8, 2024
7f90c06
changing setings
May 9, 2024
fa9ffaa
Merge pull request #7 from jmikedupont2/feature/sauerkraut
Bakobiibizo May 10, 2024
e4581cf
Update docker-compose.yml
jmikedupont2 May 13, 2024
82a84bd
Merge pull request #8 from jmikedupont2/patch-1
jmikedupont2 May 16, 2024
9caa5b6
bugfix
May 25, 2024
17287db
public ip
Jun 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rechain reloc
  • Loading branch information
@artek0chumak
artek0chumak committed Apr 11, 2024
commit 0ca54a5e76c0fa73522198cbeb742ad1e0e8d948
4 changes: 2 additions & 2 deletions .github/workflows/run-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ jobs:
- { model: 'bigscience/bloom-560m', os: 'ubuntu', python-version: '3.11' }
- { model: 'Maykeye/TinyLLama-v0', os: 'ubuntu', python-version: '3.8' }
- { model: 'Maykeye/TinyLLama-v0', os: 'ubuntu', python-version: '3.11' }
- { model: 'Maykeye/TinyLLama-v0', os: 'macos-14', python-version: '3.10' }
- { model: 'Maykeye/TinyLLama-v0', os: 'macos-14', python-version: '3.11' }
- { model: 'Maykeye/TinyLLama-v0', os: 'macos', python-version: '3.10' }
- { model: 'Maykeye/TinyLLama-v0', os: 'macos', python-version: '3.11' }
- { model: 'artek0chumak/TestMixtral', os: 'ubuntu', python-version: '3.8' }
- { model: 'artek0chumak/TestMixtral', os: 'ubuntu', python-version: '3.11' }
fail-fast: false
Expand Down
2 changes: 1 addition & 1 deletion src/petals/server/throughput.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ def measure_compute_rps(
block = block.to(dtype)
block = convert_block(block, 0, config, tensor_parallel_devices, device, quant_type=quant_type, freeze=True)

cache = (DUMMY_KEY_PAST.to(dtype).to(device), DUMMY_KEY_PAST.to(dtype).to(device))
cache = (DUMMY_KEY_PAST.to(dtype=dtype, device=device), DUMMY_KEY_PAST.to(dtype=dtype, device=device))
elapsed = 0
dummy_input = torch.randn(1, n_tokens, config.hidden_size, device=device, dtype=dtype)

Expand Down
Loading