1 Release published by 1 person

• v2.4.2

  published Sep 18, 2024

23 Pull requests merged by 5 people

• fix qwen2.5 template

  #2081 merged Sep 20, 2024

• dynamic vit gradient_checkpointing

  #2071 merged Sep 20, 2024

• Support Mistral-small-inst-2409

  #2077 merged Sep 20, 2024

• fix RLHF & max_length

  #2075 merged Sep 19, 2024

• Update qwen2-vl最佳实践.md

  #2058 merged Sep 19, 2024

• fix rlhf zero3

  #2072 merged Sep 19, 2024

• Fix yi template

  #2067 merged Sep 19, 2024

• fix win32 quote

  #2065 merged Sep 18, 2024

• update qwen2-vl docs

  #2063 merged Sep 18, 2024

• fix notebook gradio

  #2062 merged Sep 18, 2024

• support qwen2.5-coder

  #2061 merged Sep 18, 2024

• vllm support mutli image

  #2059 merged Sep 18, 2024

• support qwen2-vl -72b/qwen2.5-math/qwen2.5-coder

  #2056 merged Sep 18, 2024

• Support qwen2.5

  #2054 merged Sep 18, 2024

• support qwen2-vl-base

  #2052 merged Sep 18, 2024

• fix qwen2vl position_ids

  #2051 merged Sep 18, 2024

• update docs

  #2050 merged Sep 17, 2024

• llama3 tool calling

  #2048 merged Sep 15, 2024

• Fix multi coordinate grounding

  #2047 merged Sep 15, 2024

• support multi bbox grounding

  #2045 merged Sep 15, 2024

• fix mplug-owl3

  #2042 merged Sep 14, 2024

• Add longwriter filtered dataset

  #2037 merged Sep 14, 2024

• fix rlhf & zero3

  #2034 merged Sep 14, 2024

1 Pull request opened by 1 person

• update qwen2.5 best practices

  #2080 opened Sep 20, 2024

23 Issues closed by 8 people

• OOM when tokenizing datasets

  #1971 closed Sep 20, 2024

• DPO training error `RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!`

  #2044 closed Sep 20, 2024

• Swift DPO Template 格式问题

  #2023 closed Sep 20, 2024

• 模型推理保存路径 (result_path)

  #2026 closed Sep 20, 2024

• qwen2 vl 72B 微调报错

  #2068 closed Sep 20, 2024

• internvl2-8b lora微调后 merge lora出错（if llm_config['architectures'][0] == 'LlamaForCausalLM':）

  #2073 closed Sep 20, 2024

• 请问DPO微调 Prompt部分会计算Loss么？

  #2070 closed Sep 19, 2024

• swift2.4.2版本更新支持qwen2.5之后，但是yi-1.5-34b推理报错(更新之前是支持的)

  #2066 closed Sep 19, 2024

• llava video SFT BUG

  #2060 closed Sep 19, 2024

• windows使用webui训练报错 invalid float value: "'1e-4'"

  #2036 closed Sep 18, 2024

• chatglm4v-9b全量微调full报错：File "/usr/local/Python3.10.0/lib/python3.10/site-packages/deepspeed/runtime/zero/stage3.py", line 149, in __init__ [rank0]: self.dtype = self.optimizer.param_groups[0]['params'][0].dtype [rank0]: IndexError: list index out of range

  #2043 closed Sep 18, 2024

• 换了多个模型，都报一个错误：sh:1:syntax

  #1994 closed Sep 17, 2024

• dpo微调与zero3不兼容

  #1899 closed Sep 17, 2024

• DPO微调InternVL2-2B时报错

  #1979 closed Sep 17, 2024

• eval_acc 是如何计算的？

  #2006 closed Sep 17, 2024

• Does DPO/RLHF tuning support internVL2 video models?

  #2015 closed Sep 17, 2024

• 使用cogvlm2在rlaif-v数据集上做DPO训练报错

  #2025 closed Sep 17, 2024

• Merge Lora后为什么sft_type被设置为full，不应该是lora么

  #2041 closed Sep 17, 2024

• internlm-xcomposer2-7b-chat 使用 use_flash_attn 出现错误

  #2046 closed Sep 17, 2024

• [Re-appeared] DPO training error UnboundLocalError: local variable 'num_patches' referenced before assignment

  #2012 closed Sep 14, 2024

• transformers>=4.45.0.dev0

  #2018 closed Sep 14, 2024

• DPO support resume_from_checkpoint

  #2031 closed Sep 14, 2024

• Merge Lora后为什么stf被设置为full

  #2040 closed Sep 14, 2024

16 Issues opened by 16 people

• swift升级到2.4.0及以上版本后加载训练数据集报错

  #2083 opened Sep 20, 2024

• dpo InternVL2-8B meets OOM

  #2082 opened Sep 20, 2024

• I tried to fine tune the instructions for qwen2-7B instruction, qwen2.5-7B instruction, and llama3 7B instruction using the latest version of Swift, but the loss decreased to 2.1~2.2 and could not be reduced further.

  #2079 opened Sep 20, 2024

• 希望支持训练user部分或者同时训练user和assistant部分

  #2078 opened Sep 20, 2024

• MLLM支持KTO训练

  #2076 opened Sep 20, 2024

• Training llama 3.1 70B using 4 A6000

  #2074 opened Sep 19, 2024

• internvl2+lmdeploy部署推理视频，有没有控制采样频率，前处理的参数？目前的推理时间过长了

  #2069 opened Sep 19, 2024

• Fine-tuning best practices for qwen2.5-72b-instruct and qwen2-vl-72b-instruct.

  #2064 opened Sep 18, 2024

• swift更新到最新版后无法使用多个节点训练

  #2057 opened Sep 18, 2024

• 数据预处理报错

  #2055 opened Sep 18, 2024

• Pixtral 12B

  #2053 opened Sep 17, 2024

• 请问如何freeze一部分pretrain的模型后，接入自定义的pytorch model并进行训练？

  #2049 opened Sep 16, 2024

• AttributeError: 'Qwen2VLForConditionalGeneration' object has no attribute 'quantize'. Did you mean: 'dequantize'?

  #2039 opened Sep 14, 2024

• 使用OpenAI API方式启动部署报错：ImportError: cannot import name 'prepare_vllm_engine_template' from 'swift.llm.utils'

  #2038 opened Sep 14, 2024

• LISA训练要么OOM，要么使用deepseed就报错

  #2035 opened Sep 14, 2024

• 关于 微调+量化 的一些经验求教

  #2033 opened Sep 13, 2024

12 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• train_on_input

  #2004 commented on Sep 14, 2024 • 0 new comments

• Best Practices for Inference and Fine-Tuning with MiniCPM-V 2.6

  #1613 commented on Sep 15, 2024 • 0 new comments

• DPO训练的时候grad_norm出现nan值

  #923 commented on Sep 17, 2024 • 0 new comments

• LLaVA-NeXT-Video model configuration initialize error

  #1986 commented on Sep 17, 2024 • 0 new comments

• 训练中途突然报错 NCCL watchdog thread terminated with exception

  #1817 commented on Sep 18, 2024 • 0 new comments

• qwen2-vl-7b-instruct 以VLLM形式启动推理引擎失败“ assert "factor" in rope_scaling”

  #1954 commented on Sep 18, 2024 • 0 new comments

• Deployment or Export

  #2014 commented on Sep 18, 2024 • 0 new comments

• swift不能做模型的二次预训练？具体怎么做啊？

  #2022 commented on Sep 18, 2024 • 0 new comments

• 🎉Support for finetuning of Qwen2-VL-Chat series models

  #1857 commented on Sep 18, 2024 • 0 new comments

• mplug-owl3-7b-chat fine-tuning document

  #1969 commented on Sep 19, 2024 • 0 new comments

• internvl2-8b在docker里训练OOM

  #2027 commented on Sep 19, 2024 • 0 new comments

• [WIP]Feat/refactor3

  #2030 commented on Sep 18, 2024 • 0 new comments