Unable to manage to maximum context length
Frequently getting the following error…
This model's maximum context length is 114688 tokens. However, your prompt contains at least 114689 input tokens, which exceeds this limit even without any output tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=114689)
Raw API response: {"error":{"message":"This model's maximum context length is 114688 tokens. However, your prompt contains at least 114689 input tokens, which exceeds this limit even without any output tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=114689)","type":"BadRequestError","param":"input_tokens","code":400}}This seems to correlate with tool use generally and search or web fetch doing research. I’ve tried compaction but it doesn’t help because this failure is often occurring after the first prompt in a new chat. I can cause this error with one research prompt that fails at ~20-something tool calls right now.
There seemingly needs to be a control on the model configuration to set a maximum instead of failing? I know what the maximum is… but there’s nowhere to set it to prevent this failure?
Log in to comment and vote
No comments yet
Be the first to share your thoughts.