[Bug] qwen 2 issue when transformers>4.41.2 for PyTorch Engine

[zhyncs]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.

Describe the bug

After version 4.41.2 https://pypi.org/project/transformers/4.41.2/#history, Transformers added the cache_position parameter to qwen 2. https://github.com/huggingface/transformers/blob/e65502951593a76844e872fee9c56b805598538a/src/transformers/models/qwen2/modeling_qwen2.py#L728-L771

Currently, the PyTorch Engine in LMDeploy has not been adapted, so the TypeError issue occurs.

TypeError: PatchedQwen2Attention.forward() got an unexpected keyword argument 'cache_position'

Currently, LMDeploy does not restrict the version of transformers. If a new version is released, the above problems will occur.

lmdeploy/requirements/runtime.txt

Line 18 in e820b56

transformers

There are currently two ways to fix this issue: one is to restrict transformers<=4.41.2, and the other is to adapt the cache_position parameter.

Do you have any suggestions? Currently, this issue seems very serious and needs to be fixed before the release of v0.5.0.

@lvhan028 @grimoire @RunningLeon @AllentDan

Reproduction

upgrade transformers>4.41.2

Environment

as described above

Error traceback

No response

[zhyncs]

I'll land the code to fix this.

[Volta-lemon]

I had this problem and I saw you solved it. Now do I just need to re-execute pip install lmdeploy[all]==0.4.2?

[Volta-lemon]

exec：pip install lmdeploy[all]==0.4.2 --force-reinstall --no-deps

[zhyncs]

exec：pip install lmdeploy[all]==0.4.2 --force-reinstall --no-deps

https://github.com/InternLM/lmdeploy/releases/tag/v0.5.0

v0.5.0 is a new release, you may try it. Cheers.