Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[functionality] Implement completion() methods #168

Open
ashwinb opened this issue Oct 2, 2024 · 6 comments
Open

[functionality] Implement completion() methods #168

ashwinb opened this issue Oct 2, 2024 · 6 comments
Labels
good first issue Good for newcomers

Comments

@ashwinb
Copy link
Contributor

ashwinb commented Oct 2, 2024

Most of the current inference providers only implement the chat_completion() method. The completion() method raises a NotImplementedError. We should implement this method for all the inference providers:

  • meta-reference
  • fireworks
  • together
  • ollama
  • bedrock
@ashwinb ashwinb added the good first issue Good for newcomers label Oct 2, 2024
@niltonheck
Copy link

Would love to help. Will check the missing implementations.

@C0deXG
Copy link

C0deXG commented Oct 2, 2024

Am currently working on the issue will update in hour

@C0deXG
Copy link

C0deXG commented Oct 2, 2024

Would love to help. Will check the missing implementations.

am working on the issue too, should i stop?

@niltonheck
Copy link

Oh no, please continue! Are you going to implement all missing implementations? Maybe I can help with some of those.

@C0deXG
Copy link

C0deXG commented Oct 2, 2024

Oh no, please continue! Are you going to implement all missing implementations? Maybe I can help with some of those.

i am will try to do some but i may need help with the other thanks!

@niltonheck
Copy link

niltonheck commented Oct 2, 2024 •
edited
Loading

@ashwinb are the kvstore_impl and EmbeddingIndex implementations being handled by the team internally? Both raise NotImplementedError given some conditions.

C0deXG pushed a commit to C0deXG/llama-stack that referenced this issue Oct 2, 2024
@ggroup3
[functionality] Implement completion() methods meta-llama#168
abe32f5 
   - Implemented completion() method for meta-reference, fireworks, together, ollama, and bedrock providers
   - Each implementation handles both streaming and non-streaming responses
   - Converted single content to messages when necessary
   - Ensured proper yield of CompletionResponse or CompletionResponseStreamChunk objects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ashwinb @niltonheck @C0deXG