manual_conversion_en

Manual Model Merging and Conversion

This section describes the process of manually merging LoRA with the original Llama-2 to obtain a complete model. If you have sufficient network bandwidth, it is recommended to download the complete model directly.

Preparation

1. Before running, ensure that the latest version of the repository is pulled: git pull
2. Ensure that your machine has enough memory to load the full model (for example, the 7B model requires 13-15G) for the model merge operation
3. Install dependencies (requirements.txt in the root of this project):

$ pip install -r requirements.txt

Step 1: Obtain the Original Llama-2-hf Model

Original Llama-2-hf address: https://huggingface.co/meta-llama/Llama-2-7b-hf

HF format model-related files (you don't need to download the safetensors format model weights):

config.json
generation_config.json
pytorch_model-00001-of-00002.bin
pytorch_model-00002-of-00002.bin
pytorch_model.bin.index.json
special_tokens_map.json
tokenizer_config.json
tokenizer.json
tokenizer.model

Step 2: Merge LoRA Weights to Generate Full Model Weights

This step merges LoRA weights to generate full model weights. You can choose to output PyTorch version weights (.pth file) or output HuggingFace version weights (.bin file). Execute the following command:

$ python scripts/merge_llama2_with_chinese_lora_low_mem.py \
    --base_model path_to_original_llama2_hf_dir \
    --lora_model path_to_chinese_llama2_or_alpaca2_lora \
    --output_type huggingface \
    --output_dir path_to_output_dir 

Parameter Description:

• --base_model: The directory where the HF format Llama-2 model weights and configuration files are stored
• --lora_model: Directory where the decompressed files of Chinese LLaMA-2/Alpaca-2 LoRA are located, or the model call name on the HuggingFace Model Hub (will be downloaded automatically)
• --output_type: Specify the output format, which can be pth or huggingface. If not specified, the default is huggingface
• --output_dir: Specify the directory to save the full model weights, the default is ./
• (Optional) --verbose: Display detailed information during the merge process