Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cancel the async openAI.chatsStream(query: query) #198

Open
thekoc opened this issue Apr 22, 2024 · 3 comments · May be fixed by #214
Open

Cancel the async openAI.chatsStream(query: query) #198

thekoc opened this issue Apr 22, 2024 · 3 comments · May be fixed by #214

Comments

@thekoc
Copy link

thekoc commented Apr 22, 2024

Is your feature request related to a problem? Please describe.

Simple use case:

let task = Task {
    for try await result in openAI.chatsStream(query: query) {
        print(result)
    }
}

During the streaming, if I cancel the task manually (e.g. The stop generation button in the official ChatGPT app), I expect the stream to be stopped.

task.cancel() // Expecting the network connection to be cancelled too.

Unexpectedly, by monitoring the network activity, the generation does not stop even if the task has been cancelled.

This is due to (by my poor understanding) the underlying StreamingSession does not stop alongside with the task accordingly.

// func performStreamingRequest<ResultType: Codable>(request: any URLRequestBuildable, onResult: @escaping (Result<ResultType, Error>) -> Void, completion: ((Error?) -> Void)?)
let session = StreamingSession<ResultType>(urlRequest: request)
// ...
session.perform() // Once initialized never stopped

Describe the solution you'd like
Use the onTermination method to handle cancellation from the outside task.

This pull request might also help

Describe alternatives you've considered
Empty

Additional context
Empty

@thekoc thekoc changed the title Cancel during openAI.chatsStream(query: query) Cancel the async openAI.chatsStream(query: query) Apr 22, 2024
@longseespace
Copy link
Contributor

This might be useful to you @thekoc

longseespace@a290a1a

@tisfeng
Copy link

tisfeng commented May 11, 2024

I also need this feature, and would like to be able to proactively cancel stream requests.

Why hasn't the developer handled this issue?

@tisfeng
Copy link

tisfeng commented Jun 3, 2024

I needed this feature, but nobody responded, so I implemented it myself, thanks longseespace@a290a1a .

In case others need it too, I submitted a PR #214 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants