Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

请问能给一点实现动态batch size的建议吗？ #489

Closed
shushanxingzhe opened this issue Mar 4, 2024 · 3 comments
Closed

请问能给一点实现动态batch size的建议吗？ #489

shushanxingzhe opened this issue Mar 4, 2024 · 3 comments
Assignees
@Jintao-Huang
Labels
enhancement New feature or request

Comments

@shushanxingzhe
Copy link

现在训练时样本长短不一，如果batch size设的过大会导致OOM，设的过小GPU算力有没有被利用起来，我想把数据根据大小排序或分组，然后希望训练时能根据根据当前batch的长度来动态调节batch size，达到算力的最优利用，气温能给我一些实现的建议吗？
@Jintao-Huang Jintao-Huang self-assigned this Mar 4, 2024
@Jintao-Huang
Copy link
Collaborator

可以设计这样的算法:

  1. 将数据按长度进行排序.
  2. 将数据按batch_size进行分组, 即每个组batch_size个.
  3. 对组进行随机排序.

@Jintao-Huang Jintao-Huang added the enhancement New feature or request label Mar 4, 2024
@shushanxingzhe
Copy link
Author

shushanxingzhe commented Mar 5, 2024 •
edited
Loading

@Jintao-Huang 谢谢你的回复，我的意思是，在代码的大概什么位置，可以动态的设置每个批次的batch size或者说可以调节每个批次取到的样本数，比如说哪个文件或哪个类

@tastelikefeet
Copy link
Collaborator

动态batchsize已经在transformers最新版中支持

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Jintao-Huang Jintao-Huang

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@shushanxingzhe @Jintao-Huang @tastelikefeet