Azure foundry models comms are broken

I have to disable streaming responses when using Azure foundry and mistral medium 3.5. Ideally it shouldn’t be required. The sequence that is occurring is that the stream delivers the reply normally across 7 chunks

  1. The endpoint sends a final chunk: {"choices":[], "usage":{...}}

  2. BoltAI reads choices[0].delta without checking the array is non-empty → undefined is not an object (evaluating 'g.delta')

That trailing usage-only chunk is standard OpenAI streaming behaviour, and I confirmed the Foundry Mistral deployment emits it unconditionally on all three endpoint shapes — /openai/v1/, and the deployment path on both 2024-10-21 and 2025-01-01-preview. So there's no URL or api-version that avoids it; disabling streaming is the only client-side workaround.

The means replies appear all at once rather than typing out. Everything else is unaffected.

Please authenticate to join the conversation.

Upvoters
Status

In Progress

Board
🐛

Bug Reports

Date

9 days ago

Author

Nick Dabner

Subscribe to post

Get notified by email when there are changes.