Skip to main content

Current context window usage

Feature Requests

The “Usage (Estimated)” section shows input and output tokens, total tokens, and the number of requests. However, these are cumulative: they don’t reset when the chat context is cleared, but keep increasing with each new request. This is useful for tracking overall usage, but it doesn’t show how much of the selected model’s context window is currently occupied by the active chat.
Could you add a separate indicator for current context usage? For example: 10% (100K / 1M)
Ideally, the indicator would update when the chat context is cleared, condensed, or otherwise changed, as well as when switching models in a chat: showing the new model’s context window size and recalculating usage if possible.

Posted by tngnnj ·
Status: In QA4 comments

Log in to comment and vote

Comments4

  • Daniel Nguyen

    Team•

    Nov 10


    I added the Context popover view in v3.1.0. You can set the Compact Threshold (default 32k tokens). Detecting a model’s max context window is very hard. I tried previously but it caused more issues than it helps. I think giving you this Compact Threshold option is the better option and it works with all providers / models.

    • tngnnj

      •

      Nov 10

      Thanks! That's exactly what I asked for. The only thing I’m wondering about is changing the Compact Threshold value. This option is only visible when Context Policy is set to Auto-compact. After changing the Compact Threshold value and setting Context Policy to something other than Auto-compact, the Compact Threshold option disappears, but the saved value is still retained. Is that expected behavior?

      • Daniel Nguyen

        Team•

        Nov 11

        That is expected as I don’t know what to put there when you change the context policy. I’d use this threshold as the universal threshold that I generally want to keep the context under that number - regardless of the auto-compaction settings.

        What would you show there?

        • tngnnj

          •

          Nov 11

          When Context Policy is set to Auto-compact - everything exactly as it should.

          The only thing that feels a bit odd is that the Compact Threshold option is only visible in Auto-compact mode. Since its value affects not only the compact threshold but also the context window value in the Context popover, I suggest showing it all the time, regardless of which mode is selected in Context Policy. And maybe to avoid confusion, in non-Auto-compact mode the option could be renamed from “Compact Threshold” to “Context Window"