IndexThinking.Sqlite 0.33.0

IndexThinking

Working Memory Manager for Reasoning-capable LLMs

What It Does

IndexThinking handles the repetitive-but-hard parts of LLM integration:

• Truncation Recovery - Auto-continue when responses hit token limits (GetResponseAsync; a truncated stream is reported as WasTruncated)
• Reasoning Extraction - Unified API for provider-specific thinking formats
• Token Management - Token usage tracking (IBudgetTracker) and complexity estimation (IComplexityEstimator), reported in turn metrics — informational, not limits. To cap how much a model thinks, set it on the request: ChatOptions.Reasoning (effort) and ChatOptions.MaxOutputTokens; IndexThinking extracts the reasoning, it does not throttle it
• Content Recovery - Repair truncated JSON/code blocks
• Thinking State Stores - IThinkingStateStore implementations: in-memory (AddIndexThinkingInMemoryStorage) and distributed-cache (AddIndexThinkingDistributedStorage) in the core package, persistent SQLite in IndexThinking.Sqlite. All are opt-in — AddIndexThinkingAgents registers no store, and without one no per-session state is saved

Features

Each row: what it does · entry point · how to turn it on. "Default" means the Quick Start registration (AddIndexThinkingAgents() + UseIndexThinking()) already enables it.

Scope

IndexThinking shapes one model call — it fits what goes in to the context window and reads what comes out — not multi-step workflows.

Token counting role boundary — IndexThinking owns token counting (ITokenCounter, framework-neutral; IChatMessageTokenCounter for M.E.AI ChatMessage counting). Model metadata (context window, pricing) belongs to TokenMeter; the consumer picks the budget from the window it knows, and ContextBudgetChatReducer fits a request into it.

Long-term memory boundary — IndexThinking keeps no long-term memory and calls no memory store. To give a turn recalled memories, recall them before the turn and add them to the messages you send.

Packages

Quick Start

dotnet add package IndexThinking

// Register services
services.AddIndexThinkingAgents();

// Wrap any IChatClient
var client = new ChatClientBuilder(innerClient)
    .UseIndexThinking()
    .Build(serviceProvider);

// Use normally
var response = await client.GetResponseAsync(messages);

// Access metadata
var thinking = response.GetThinkingContent();
var metrics = response.GetTurnMetrics();

Conversation History

The client is stateless: send the conversation you want the model to see, and keep it yourself (ChatMessage list, a store of your own). To fit a long history into the window, register ContextBudgetChatReducer (see What It Does). A session id (ThinkingChatClientExtensions.WithSession(id) or ChatOptions key IndexThinking.SessionId) only keys the thinking state store and the per-session metrics.

Streaming with Thinking Orchestration

Streaming uses a Collect-and-Yield pattern: chunks are yielded to the caller immediately while buffered internally. After the stream completes, the buffered response is processed through the full orchestration pipeline (reasoning parsing, budget tracking). A stream cut off at the output limit is not continued — the caller already has every chunk — and is reported as TurnResult.WasTruncated. The output-token boost and the context-window cap of GetResponseAsync do not apply either: a streamed request is sent with the caller's own MaxOutputTokens.

await foreach (var update in client.GetStreamingResponseAsync(messages))
{
    // Real-time chunks arrive here
    Console.Write(update.Text);

    // The final update contains orchestration metadata
    if (update.AdditionalProperties?.ContainsKey(ThinkingChatClient.TurnResultKey) == true)
    {
        var result = update.AdditionalProperties[ThinkingChatClient.TurnResultKey] as TurnResult;
        Console.WriteLine($"\nTokens: {result?.Metrics.TotalTokens}");
    }
}

Live reasoning separation (opt-in)

Open-source / local providers (DeepSeek, Qwen3, vLLM-served) emit reasoning inline in the text delta as <think>…</think> rather than as a separate channel. Enable SeparateReasoningInStream to have IndexThinking split it live — answer text arrives as TextContent, reasoning as TextReasoningContent — even when a tag is split across chunk boundaries. No bespoke tag parser needed. Default is off (raw pass-through). Native reasoning providers (OpenAI o-series, Anthropic, Gemini) already emit a separate channel, so this is a no-op for them.

Migration note: with separation enabled, reasoning spans are no longer part of ChatResponseUpdate.Text (which concatenates TextContent only). Consumers reading update.Text must switch to iterating update.Contents (as below) or reasoning deltas will be silently dropped.

var client = new ChatClientBuilder(innerClient)
    .UseIndexThinking(o =>
    {
        o.SeparateReasoningInStream = true;         // default false
        // o.StreamingReasoningStartTag = "<think>";  // defaults shown
        // o.StreamingReasoningEndTag   = "</think>";
    })
    .Build(serviceProvider);

await foreach (var update in client.GetStreamingResponseAsync(messages))
{
    foreach (var content in update.Contents)
    {
        if (content is TextReasoningContent reasoning)
            RenderThinking(reasoning.Text);   // live "💭 thinking…" UI
        else if (content is TextContent answer)
            RenderAnswer(answer.Text);
    }
}

Supported Providers

Enabling Reasoning for DeepSeek/vLLM/Qwen

Some providers require explicit reasoning activation:

var client = new ChatClientBuilder(innerClient)
    .UseIndexThinking(o => o.EnableReasoning = true)  // adds the provider's reasoning flag to requests
    .Build(serviceProvider);

Inline Reasoning Stripping (Automatic)

When reasoning is disabled on continuation requests (to avoid double-billing thinking tokens), open-source models sometimes output inline reasoning text instead of properly tagged content. IndexThinking automatically strips these fragments:

• StripLeadingUntaggedReasoning — removes reasoning paragraphs that appear at the start of a continuation fragment (≥2 consecutive reasoning paragraphs required before stripping)
• StripUntaggedReasoning — removes trailing reasoning that appears after the actual answer content (requires ≥200 chars of content before the marker and ≥200 chars of trailing reasoning)

Both methods are called automatically by the continuation pipeline. They are also available as static methods on OpenSourceReasoningParser for manual use.

Documentation

• Design

License

MIT License - See LICENSE for details.