本项目旨在开源Llama2各个版本的中文补丁模型,以及其训练代码,参数配置等信息,从而进一步促进大模型在中文NLP社区的开放研究,争取做到人人有丹炼。
基于本项目已开源的模型有:
- 基于Llama2-chat-7B的中文对话LoRA补丁及合并后的模型:Llama2-chat-Chinese-50W
- 基于Llama2-chat-13B的中文对话LoRA补丁及合并后的模型:Llama2-chat-13B-Chinese-50W
- 基于Llama2-base-7B的全参数微调中文对话模型(1 epoch 预发布版本):Llama2-base-7B-Chinese-50W-pre_release
- 基于Llama2-base-7B的全参数微调转LoRA训练中文对话模型:Llama2-base-7B-Chinese-50W-Full2LoRA
- 基于Llama2-base-7B的全参数微调中文对话模型(3 epoch 标准版本):Llama2-base-7B-Chinese-50W-fullTune
- 基于Llama2-base-7B的中文对话LoRA补丁及合并后的模型:Llama2-base-7B-Chinese-50W-LoRA
- 基于Llama2-base-13B的中文对话LoRA补丁及合并后的模型:Llama2-base-7B-Chinese-50W-LoRA
The main purpose of this project is to open-source Chinese dialogue patch models for various versions of Llama2, along with their training code, hyper-parameter configurations, and related information. The ultimate goal is to further promote the accessibility of large-scale models in the Chinese NLP community, aiming to make advanced research available to everyone.
Currently, the following open-resource models are trained based on this project:
- The Chinese dialogue LoRA patch based on Llama2-chat-7B, along with the merged model: Llama2-chat-Chinese-50W
- The Chinese dialogue LoRA patch based on Llama2-chat-13B, along with the merged model: Llama2-chat-13B-Chinese-50W
- The full-parameter fine-tuned Chinese dialogue model based on Llama2-base-7B (1 epoch pre-release version): Llama2-base-7B-Chinese-50W-pre_release
- The Chinese dialogue model based on Llama2-base-7B, first adopted full-parameter fine-tuned, then converted to LoRA training: Llama2-base-7B-Chinese-50W-Full2LoRA
- The full-parameter fine-tuned Chinese dialogue model based on Llama2-base-7B (3 epoch standard version): Llama2-base-7B-Chinese-50W-fullTune
- The Chinese dialogue LoRA patch based on Llama2-base-7B, along with the merged model: Llama2-base-7B-Chinese-50W-LoRA
- The Chinese dialogue LoRA patch based on Llama2-base-13B, along with the merged model: Llama2-base-7B-Chinese-50W-LoRA