Rate limiter

[hkirsman]

Hi,

thanks for great work. I'm not yet sure if to be scared or amazed of this software! Litte bit of both I guess.

I've had issue that I've put iteration to 100 and at 28 I hit rate limiter. On the https://console.anthropic.com/ rate limits page it's says 50 for all models. From the docs I see it's actually depending on something called tier https://docs.anthropic.com/en/api/rate-limits

So I think one solution would be pause when 429 error code is hit and wait for 1 minute. Then measure how many requests were done in last minute and figure what's the rate limit for user. On next round when it's hitting the limit, wait with last request so that minute is over.

Optionally user could set their limit in config (.env file?)

[hkirsman]

Ok, with second iteration I also hit the daily limit now :)

Error calling Claude API: Error code: 429 - {'type': 'error', 'error': {'type': 'rate_limit_error', 'message': 'Number of request tokens has exceeded your daily rate limit (https://docs.anthropic.com/en/api/rate-limits); see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase.'}}
I'm sorry, there was an error communicating with the AI. Please try again.

[ophoria]

+1

[couturelp]

+1