Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix qwen2 cache_position for PyTorch Engine when transformers>4.41.2 #1886

Merged
merged 2 commits into from
Jul 1, 2024

Conversation

zhyncs
Copy link
Collaborator

@zhyncs zhyncs commented Jun 29, 2024

Motivation

fix #1885

Hi @lvhan028 @grimoire @RunningLeon @AllentDan May you help review this pr? Thanks.

Modification

as titled

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@zhyncs
Copy link
Collaborator Author

zhyncs commented Jun 29, 2024

ref #1852

@lvhan028 lvhan028 requested a review from grimoire June 29, 2024 14:03
@lvhan028 lvhan028 added the Bug:P1 label Jul 1, 2024
@lvhan028 lvhan028 merged commit 1f5dd4e into InternLM:main Jul 1, 2024
5 checks passed
@zhyncs zhyncs deleted the fix-qwen2 branch July 1, 2024 03:47
@Volta-lemon
Copy link

I had this problem and I saw you solved it. Now do I just need to re-execute pip install lmdeploy[all]==0.4.2?

@zhyncs
Copy link
Collaborator Author

zhyncs commented Jul 1, 2024

@Volta-lemon This PR has been merged and I manually triggered the compilation at https://github.com/zhyncs/lmdeploy-build/actions/runs/9737958121 and https://github.com/zhyncs/lmdeploy-build/actions/runs/9737959536. After the compilation is completed, it can be installed directly.

@zhyncs
Copy link
Collaborator Author

zhyncs commented Jul 1, 2024

@Volta-lemon https://github.com/zhyncs/lmdeploy-build/releases/tag/1f5dd4e
You may add extra args when installing with pip3 using --force-reinstall --no-deps

@Volta-lemon
Copy link

Thank you for solving my problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] qwen 2 issue when transformers>4.41.2 for PyTorch Engine
5 participants