Q-GaLore support

[wenmengzhou]

Describe the feature

Q-GaLore is a memory-efficient training methodology for pre-training and fine-tuning. It enables full-parameter training while requiring less memory. Fine-tune 7B model with only 16GB of memory

https://github.com/VITA-Group/Q-GaLore

Paste any useful information

Additional context
Add any other context or information here(其他信息可以写在这里)

[tastelikefeet]

Supported