Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat all quants ci #231

Merged
merged 30 commits into from
Sep 18, 2024
Merged

Feat all quants ci #231

merged 30 commits into from
Sep 18, 2024

Conversation

nguyenhoangthuan99
Copy link
Contributor

@nguyenhoangthuan99 nguyenhoangthuan99 commented Sep 16, 2024

Fix janhq/cortex.cpp#1155
Summary changes:

  • update model yml script to download model.yml in cortexso hugging face, then update any field, keep all comments from original file.
  • CI pipeline for update model.yml, can input quantization level, model size, and model repo_id
  • CI pipeline for convert and quantize model from huggingface format to gguf format. Can input quantization level, model size, and model repo_id

@nguyenhoangthuan99 nguyenhoangthuan99 marked this pull request as ready for review September 17, 2024 01:14
@@ -0,0 +1,92 @@
import argparse
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use the same name for update model? I see we have update-model-yml.yml and update_model_yaml.py

Copy link

@dan-homebrew dan-homebrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nguyenhoangthuan99 Thanks. I've given a preliminary approval to unblock.

@dan-homebrew
Copy link

@hiento09 @nguyenhoangthuan99 For converter and quantization, would it make sense for us to pull it out into a separate repo?

  • llama.cpp engine is meant to be for engines
  • I see our CI running for to build this for every Hardware and OS, when it doesn't really make sense

@nguyenhoangthuan99 nguyenhoangthuan99 merged commit e91a763 into main Sep 18, 2024
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

epic: Jan and Cortex's Built-in Model Library has latest models + pipeline
3 participants