ImportError: cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'

[programmeraudrey]

System Info / 系統信息

huggingface_hub==0.19.4 其他配置和官方一致

Who can help? / 谁可以帮助到您？

@abmfy

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

请问 我已经将huggingface_hub的版本设置为0.19.4 但是streamlit run ChatGLM3-main/composite_demo/main.py时仍然遇到以下问题
——————————————————————报错信息——————————————————————
You can now view your Streamlit app in your browser.

Local URL: http://localhost:8506
Network URL: http://172.17.0.11:8506
External URL: http://58.144.141.203:8506

2024-07-07 19:54:10.565 Uncaught app exception
Traceback (most recent call last):
File "/root/miniconda3/envs/chatglm3-demo/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 589, in _run_script
exec(code, module.dict)
File "/root/autodl-tmp/ChatGLM3-main/composite_demo/main.py", line 10, in
import demo_chat, demo_ci, demo_tool
File "/root/autodl-tmp/ChatGLM3-main/composite_demo/demo_chat.py", line 4, in
from client import get_client
File "/root/autodl-tmp/ChatGLM3-main/composite_demo/client.py", line 12, in
from transformers.generation.utils import LogitsProcessorList
File "/root/miniconda3/envs/chatglm3-demo/lib/python3.10/site-packages/transformers/generation/utils.py", line 94, in
from accelerate.hooks import AlignDevicesHook, add_hook_to_module
File "/root/miniconda3/envs/chatglm3-demo/lib/python3.10/site-packages/accelerate/init.py", line 16, in
from .accelerator import Accelerator
File "/root/miniconda3/envs/chatglm3-demo/lib/python3.10/site-packages/accelerate/accelerator.py", line 34, in
from huggingface_hub import split_torch_state_dict_into_shards
ImportError: cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' (/root/miniconda3/envs/chatglm3-demo/lib/python3.10/site-packages/huggingface_hub/init.py)

Expected behavior / 期待表现

请问应该如何解决呢？感谢！

[YamMangoTea]

pip install -U transformers
pip install -U huggingface_hub

[whueric]

Change the huggingface_hub version in requirements.txt:
huggingface_hub<0.22.0

Then pip install -r requirements.txt

[mausch]

In my case I had to pin "accelerate" to 0.31.0
accelerate 0.32.1 triggers this error.