Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Chore!] Add error verification on callback from stream #40

Merged
merged 1 commit into from
Nov 27, 2023

Conversation

letschers
Copy link
Contributor

I was trying to implement a similar feature like the "Stop Generating" from ChatGPT and I noticed that there's no mechanism to stop the stream call coming from the OpenAI. I had the problem where even if I return from my callback, it will be called again and again. The most simple way I found was to return an error from the callback and validate this error.

This is a breaking change for every application that uses this lib (but also pretty simple to refactor), but I guess is important as much as necessary; if my callback is returning an error and anything is being done from the clientside, the server will keep generating chunks and wasting tokens.

I made in only one stream function to see what you guys think about it, but if it's relevant I'm willing to change the remaining stream methods

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ This pull request was sent to the PullRequest network.


@letschers you can click here to see the review status or cancel the code review job.

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PullRequest Breakdown

Reviewable lines of change

+ 5
- 3

100% Go

Type of change

Minor Update - These changes appear to be a minor update to existing functionality and features.

Copy link

@pullrequest pullrequest bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is simple and straightforward with the explanation from the author. But, as stated by the author, this is a breaking change that need to be taking care of by any other code using this

Image of Xiaoyong W Xiaoyong W


Reviewed with ❤️ by PullRequest

@letschers
Copy link
Contributor Author

Hey guys, any news?

Copy link
Contributor

@tylermann tylermann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change makes sense to me.

I could see a non-breaking change variant where we allow something like this:

onData = func (resp *ChatCompletionStreamResponse) {
    resp.Stop(errors.New('some error))
}

Although I think your proposed change here is more idiomatic and it is a fairly easy refactor as you mentioned. So lets just go with this.

Thanks for the contribution and sorry for the delay to getting around to this.

@tylermann tylermann merged commit c6cd599 into PullRequestInc:main Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants