GGUF Support

[abetlen]

No description provided.

[TheBloke]

FYI I will be starting my uploads of GGUF models as soon as the new k-quant PR is merged, so hopefully in the next day or two.

It'd be great if someone could ping me when there's a definite ETA for the merging of your GGUF support, so I know what to write in my GGUF READMEs regarding llama-cpp-python compatibility, once I start uploading those models.

[rlancemartin]

Add Support for the new GGUF format which replaces GGML oobabooga/text-generation-webui#3676

FYI I will be starting my uploads of GGUF models as soon as the new k-quant PR is merged, so hopefully in the next day or two.

It'd be great if someone could ping me when there's a definite ETA for the merging of your GGUF support, so I know what to write in my GGUF READMEs regarding llama-cpp-python compatibility, once I start uploading those models.

Thanks! I noticed the new format when downloading one of the new CodeLlama models.

Looks like support is underway.

I assume we will also need to update / re-build llama.cpp?