Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Anthropic models #105

Open
pyamin1878 opened this issue Jun 27, 2024 · 3 comments
Open

Add support for Anthropic models #105

pyamin1878 opened this issue Jun 27, 2024 · 3 comments

Comments

@pyamin1878
Copy link

Would be nice to use the latest and greatest Claude models with this project :)

@OKUA1
Copy link
Collaborator

OKUA1 commented Jun 27, 2024

Yes, I agree that Anthropic should be the first candidate for the next native backend.

FYI: you can use Claude even now with a GPT backend and a proxy server.

@AndreasKarasenko
Copy link
Contributor

AndreasKarasenko commented Jul 9, 2024

For people that use Ollama it would look like this (see also here):

from skllm.config import SKLLMConfig
SKLLMConfig.set_gpt_url("http://localhost:11434/v1/")
from skllm.models.gpt.classification.zero_shot import ZeroShotGPTClassifier
from skllm.datasets import get_classification_dataset


X, y = get_classification_dataset()
clf = ZeroShotGPTClassifier(model="custom_url::llama3", key="ollama")
clf.fit(X,y)
labels = clf.predict(X)

Even though we don't technically need a key, any string is still required.

Edit:
Additionally /v1/embeddings is currently not supported in Ollama so you can't really use it for stuff like DynamicFewShot (I think). But they're working on a fix.

@AndreasKarasenko
Copy link
Contributor

FYI I fully integrated Ollama into my branch now.
I need to fix the docstrings a bit and change the OllamaVectorizer but it works very nicely for me.

The main reason for the integration was that I cannot pass optional parameters (such as context size) via OpenAI, and I absolutely need that control for my work. I know you guys want to use llama.cpp instead, just figured others may be are interested in the branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants