-
-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Raise error for long prompt #273
Conversation
Hi Lily, please check all the possible situations here and make sure you can handle all of them. Thanks! |
I move forward with the length check. The original code should already handle the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey Lily, thanks for the work! Please check my comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the hard work! In general LGTM! Will merge it after the following small comments are fixed.
…project#273) SUMMARY: * update benchmarking, testing, and accuracy jobs to run on label `aws-test-a10g-24G` or `aws-test-a10-96G` which is based on "vanilla deeplearning" AMI * update relevant GHA actions and workflows to not be dependent on `pyenv` virtualenv * update "model cache" to use local disk as opposed to "EFS" TEST PLAN: runs on remote push --------- Co-authored-by: andy-neuma <andy@neuralmagic.com> Co-authored-by: Domenic Barbuzzi <domenic@neuralmagic.com>
This is a fix for #113. The program hangs when the input prompt is too long because the check will always succeed and the request will always in the waiting queue and will never be added to the running queue.
Add a check before the if statement, if the prompt length is too long, it will raise ValueError directly.