New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

localAI run on GPU #123

Closed

shengkaixuan opened this issue Apr 29, 2023 · 1 comment

shengkaixuan commented Apr 29, 2023

https://github.com/ggerganov/llama.cpp#blas-build
seems like llama.cpp can run models on GPU,will localai support that ??

Owner

mudler commented May 9, 2023

this should be definetly possible, and technically it's just about wiring up compilations options. this is discussed also in #69 , closing as a dup

mudler closed this as completed

JimHeo mentioned this issue

generation eval time is slower than llama-cli in pure llama.cpp #2780

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment