You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In our new and fancy world of LLMs and OpenAI, we have rate limits by number of message-tokens per minute, which means the bucket needs to be increased by a somewhat arbitrary number. Obviously, I could just call "check_rate" sequentially in a loop a few hundred times, but that seems... A little silly.
Would it be possible to expand the public API a little, and allow us to pass in an optional number-by-which-to-increase-the-number-of-tokens-in-the-bucket?
The text was updated successfully, but these errors were encountered:
In our new and fancy world of LLMs and OpenAI, we have rate limits by number of message-tokens per minute, which means the bucket needs to be increased by a somewhat arbitrary number. Obviously, I could just call "check_rate" sequentially in a loop a few hundred times, but that seems... A little silly.
Would it be possible to expand the public API a little, and allow us to pass in an optional number-by-which-to-increase-the-number-of-tokens-in-the-bucket?
The text was updated successfully, but these errors were encountered: