Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[LLM][NPU] fix on readme #8659

Merged
merged 1 commit into from
Jun 26, 2024
Merged

[LLM][NPU] fix on readme #8659

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion llm/npu/llama/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## 🚣‍♂️ 使用PaddleNLP在NPU下跑通llama2-13b模型 🚣
PaddleNLP在昇腾NPU（[了解昇腾](https://www.hiascend.com/zh/ecosystem/industry)）上对llama2-13B模型进行了深度适配和优化，该套件实现了昇腾NPU和GPU的训推入口基本统一，达到了『无缝切换』的效果。
在技术领先性上：
- **训练策略完全适配** 支持3D混合并行，灵活适应多种训练策略。
- **训练策略完全适配** 支持4D混合并行，灵活适应多种训练策略。
- **训练性能极致优化** 95%的通信被掩盖在计算中，软硬结合提供极致性能。
- **低门槛性能调优** 分布式策略自动寻优能力打通多硬件，完全屏蔽硬件复杂性的同时，使用户可以轻松挖掘算力极限。
- **推理成本极致压缩** 推理支持 Layer 级算子融合，且融合算子已支持动态插入功能
Expand Down
2 changes: 1 addition & 1 deletion paddlenlp/transformers/mc2_parallel_linear.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
from paddlenlp.utils.tools import get_env_device

__all_gather_recomputation__ = False
if int(os.getenv("MC2_Recompute", 0)):
if int(os.getenv("FLAGS_NPU_MC2_Recompute", 0)):
__all_gather_recomputation__ = True


Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ tool_helpers ; platform_system == "Linux"
aistudio-sdk>=0.1.3
jinja2
regex
numpy==1.26.4
numpy<=1.26.4
Loading