You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
However, they have to be kept maintained and up to date with the official tiktoken. It feels very natural that such a tool be apart of this repository and updated regularly by the openai team. I feel that pre-request token estimation is vital to ensuring hobbyist developers don't unknowingly succumb to financial demise.
const response = await openai.chat.completions.create({
model: "gpt-3.5-turbo",
messages: messages,
functions: functions,
function_call: "auto", // auto is default, but we'll be explicit
});
I could intercept this function call above to view what data is sent to chatGPT so I can figure out how many tokens this call takes with the message + the functions, pipe them into gpt-tokenizer and see how many tokens are used. But if it were to be built into the API itself then you can have a dry run option on your nodejs api functions that estimate cost and then the developer could determine in the logic if they want to proceed with the actual run.
What else could be very helpful would be to include some parameter whether in the configuration or when calling api functions such that if the tokens and/or usd cost exceeds a certain limit the call auto fails and doesn't actually get executed.
Additional context
No response
The text was updated successfully, but these errors were encountered:
I use function calling extensively. I'd love a simple function that returned a token count given completion input, e.g. ChatCompletionCreateParamsNonStreaming.
I want to ensure that I respect rate limits before I ever hit them. Rate limiting libraries such as Bottleneck can be configured to respect both the request and token limits, but I need the token counts before I make the request
Confirm this is a feature request for the Node library and not the underlying OpenAI API.
Describe the feature or improvement you're requesting
There a couple nodejs tokenizers laying around npm land:
However, they have to be kept maintained and up to date with the official tiktoken. It feels very natural that such a tool be apart of this repository and updated regularly by the openai team. I feel that pre-request token estimation is vital to ensuring hobbyist developers don't unknowingly succumb to financial demise.
Why apart of this library?
Take function calling for example:
I could intercept this function call above to view what data is sent to chatGPT so I can figure out how many tokens this call takes with the
message
+ thefunctions
, pipe them into gpt-tokenizer and see how many tokens are used. But if it were to be built into the API itself then you can have a dry run option on your nodejs api functions that estimate cost and then the developer could determine in the logic if they want to proceed with the actual run.What else could be very helpful would be to include some parameter whether in the configuration or when calling api functions such that if the tokens and/or usd cost exceeds a certain limit the call auto fails and doesn't actually get executed.
Additional context
No response
The text was updated successfully, but these errors were encountered: