36 releases 42 tags

RSS feed

• v2.2.2-ew-2 9b8c373cb0

  Compare

  v2.2.2-ew-2 Stable

  engel75 released this 2026-09-24 15:23:47 +02:00 | -3417 commits to 1.4.23-ew since this release

  Fix: sglang usage.reasoning_tokens reach chat/completions responses again.

  SGLang upstreams report reasoning tokens as a TOP-LEVEL usage.reasoning_tokens
  field (sglang protocol.py UsageInfo) and send no completion_tokens_details
  object at all. Bifrost only modeled the OpenAI-spec nested shape, so the value
  was silently dropped at the first JSON decode and never reached clients — for
  the SGL and EW providers, streaming and non-streaming alike.

  • BifrostLLMUsage.UnmarshalJSON folds the top-level field into
    usage.completion_tokens_details.reasoning_tokens (ChatPromptTokensDetails
    cached_tokens alias precedent). Nested OpenAI-spec values keep precedence;
    zero values never synthesize an empty details object; the top-level key
    never leaks to the wire.
  • The fold flows into every decode path of the shared struct: streaming
    chunks, non-streaming responses, passthrough usage, and the Responses API
    (ToResponsesResponseUsage). Reasoning-model cost accounting in governance
    is now correct for sglang backends (previously under-billed).

  Verified: red/green unit tests (core/schemas/usage_test.go), full schemas
  suite, go vet, consumer suites (openai provider incl. xAI nested-shape tests,
  streaming, mocker, semanticcache, governance), provider-harness augment/filter
  checks with new 8.5.R EW wire case (run locally: FEATURE="reasoning_tokens").

  Downloads

  • Source code (ZIP)
    1 download
  • Source code (TAR.GZ)
    0 downloads