Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

swift可以训练量化之后的模型吗，比如modelscope里面的awq或者gptq量化之后的模型 #1472

Closed
LIUKAI0815 opened this issue Jul 23, 2024 · 6 comments
Closed

swift可以训练量化之后的模型吗，比如modelscope里面的awq或者gptq量化之后的模型 #1472

LIUKAI0815 opened this issue Jul 23, 2024 · 6 comments

Comments

@LIUKAI0815
Copy link

因为在训练的时候，加载没有量化的模型，训练完再量化资源不够。可以直接训练官方量化后的模型吗
@LIUKAI0815
Copy link
Author

如果训练完之后，可以merge lora 吗

@Jintao-Huang
Copy link
Collaborator

可以训练，但是训练完之后没办法merge-lora.

@JiaXinLI98
Copy link

可以训练，但是训练完之后没办法merge-lora.

能用full的方式训练吗

@LIUKAI0815
Copy link
Author

这种情况下我理解应该就只能用QLORA训练了。因为单纯的lora训练就已经是加载16位了，而量化后的awq就是int4了，这样训练的精度就对应不上了。full全量训练应该就不更行了，这是我的理解

@LIUKAI0815
Copy link
Author

@Jintao-Huang 直接用默认的lora训练awq模型吗，默认的lora加载的是16/32位把？

@tastelikefeet
Copy link
Collaborator

支持的，QLoRA训练

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@JiaXinLI98 @Jintao-Huang @LIUKAI0815 @tastelikefeet