Skip to main content

Ability to specify endpoint provider for a model on OpenRouter

A lot of the coding extensions give a UI option to select the API-call-level variable of which provider you want: eg: novita-fp8, deepinfra-fp4, etc.

This would be highly useful on Bolt as OpenRouter has the following non-ideal behavior when this variable is not set:

1) Flitting between different providers even with a heavy cache load. Today, four requests went to four different providers in immediate succession.

2) OpenRouter’s autorouting (or even what is possible by manually configuring guardrails) is limited and often leaves you with endpoints that are more expensive or with a worse quantization than is ideal.

This is best solved by utilizing the API-call level config OpenRouter allows.

This could be implemented with an optional “choose endpoint” in the model configuration, and/or the ability to switch endpoints in the right chat sidebar.

Log in to comment and vote

No comments yet

Be the first to share your thoughts.