[Bug] Chat template redacting reasoning traces for non-tool related use case

#24
by vibhas09 - opened

I’m seeing that the current chat template does not render reasoning (reasoning / reasoning_content) for standard assistant messages unless tool_calls are present.

This makes it unclear whether this is intended behavior or a limitation/bug in the template logic.

Minimal Repro:

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Write a short joke about saving RAM."},
    {
        "role": "assistant",
        "reasoning": "Need a short memory pun.",
        "content": "I wrote a RAM joke, but I forgot to cache it."
    },
]

print(tokenizer.apply_chat_template(
    conversation=messages,
    tokenize=False,
    add_generation_prompt=False,
    enable_thinking=True
))

Observed Behavior:
Output does not include the reasoning:

<bos><|turn>system
<|think|> 
You are a helpful assistant.
<turn|> 
<|turn>user Write a short joke about saving RAM.
<turn|> 
<|turn>model I wrote a RAM joke, but I forgot to cache it.<turn|>

Expected Behavior:

If reasoning (or reasoning_content) is present, I would expect it to be rendered, e.g.:

<bos><|turn>system
<|think|>
You are a helpful assistant.<turn|>
<|turn>user
Write a short joke about saving RAM.<turn|>
<|turn>model
<|channel>thought
Need a short memory pun.
<channel|>
I wrote a RAM joke, but I forgot to cache it.<turn|>

This bevaviour is however common in HF as well as unsloth.

Relevant Template Logic:

{%- if thinking_text and loop.index0 > ns_turn.last_user_idx and message.get('tool_calls') -%}

This effectively prevents reasoning from being emitted for:

non-tool-call assistant messages
standard chat use cases

Questions:

Is reasoning intentionally restricted to tool-call flows?
Should enable_thinking=True also enable reasoning rendering for normal assistant messages?
Would it be acceptable to relax the condition to something like:

{%- if role == 'model' and thinking_text -%}

or gate it via enable_thinking instead?

Why this matters:

Limits usability of reasoning for standard chat scenarios
Makes enable_thinking behavior unclear/incomplete
Prevents consistent debugging/inspection of model reasoning

The behavior you hit in April was real, but the current chat template no longer does that for the exact repro you posted.

The old gate required message.get('tool_calls'), so reasoning on a normal assistant turn was dropped. The canonical template now in this repo is dated 9 Jul 2026 and the gate is:

{%- set thinking_gate = (loop.index0 > ns_turn.last_user_idx) or (preserve_thinking and message.get('tool_calls')) -%}
{%- if thinking_text and thinking_gate -%}

I ran your messages against that template. The thought channel is rendered:

<|turn>model
<|channel>thought
Need a short memory pun.
<channel|>I wrote a RAM joke, but I forgot to cache it.<turn|>

enable_thinking=True is not what flips this. That flag only inserts <|think|> in the system turn (and opens a thought channel after a tool response when you are generating). If the assistant turn is after the last user message, current-turn reasoning / reasoning_content is rendered even with enable_thinking=False.

What is still dropped, by design, is thinking on earlier turns — once a later user message exists, loop.index0 > last_user_idx is false. The model card says historical thoughts should not be added before the next user turn, except on tool-call turns, and those are only kept if you pass preserve_thinking=True.

So: non-tool reasoning in the current turn should work on the current template; historical non-tool reasoning is still omitted on purpose. If you are still seeing the April behavior, the runtime is probably not on this 9 Jul template.

Sign up or log in to comment