使用cogvlm2在rlaif-v数据集上做DPO训练报错

[kaka-Cao]

在训练前，使用modelscope download --dataset swift/RLAIF-V-Dataset 命令下载了数据集，保存路径为/home/xxx/.cache/modelscope/datasets/swift/RLAIF-V-Dataset，刚刚才拉取了最新的swift main分支。运行脚本如下：
CUDA_VISIBLE_DEVICES=0
swift rlhf
--rlhf_type dpo
--model_type cogvlm2-19b-chat
--model_id_or_path /home/data3/zys/MLLM/pretrained_models/cogvlm2-llama3-chinese-chat-19B/
--beta 0.1
--sft_beta 0.1
--sft_type lora
--dataset rlaif-v#1000
--num_train_epochs 2
--lora_target_modules DEFAULT
--gradient_checkpointing false
--batch_size 1
--learning_rate 5e-5
--gradient_accumulation_steps 16
--warmup_ratio 0.03
--save_total_limit 2
报错如下：
Original Traceback (most recent call last): File "/home/zhangyusi/.conda/envs/MLLM/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
File "/home/zhangyusi/.conda/envs/MLLM/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch return self.collate_fn(data)
File "/home/zhangyusi/swift/swift/trainers/utils.py", line 424, in new_call
padded_batch[k] = [torch.Tensor(ex[k]) for ex in features]
ValueError: only one element tensors can be converted to Python scalars
版本：
torch 2.4.0
CUDA 12.1
transformers 4.41.0

[SparrowZheyuan18]

I encounter the same issue with my own dataset

[Jintao-Huang]

#1975