D-Edge-Decision

A "System One" typed-decision model: one forward pass, zero generated tokens.

A 3.93B full fine-tune of domyn/Domyn-Edge-SFT. It answers a typed decision (yes/no, multiple-choice, graded score) by reading the next-token distribution over single-token option codes (A, B, …): a native probability distribution, no generated tokens. One run, one epoch, no ablations.

JevBench (231 public items)

# System Composite Intelligence Speed Cost
1 Plumb-4B 76.57 84.77 92.61 60.30
2 H2O-Lightning-4B † 76.40 84.14 92.61 60.30
3 Imajev-4B 75.13 79.70 92.61 60.30
4 JevK5 v0.2.0 74.75 78.43 92.61 60.30
5 decider-4b v2 73.97 75.89 92.61 60.30
6 D-Edge-Decision (ours) 73.56 74.62 92.61 60.30
7 SemIf (Qwen3.5-4B) 72.72 72.09 92.61 60.30
8 local-jev Qwen3.5-4B 72.50 71.45 92.61 60.30
9 Jev 1.13.0 (API) 72.03 80.33 91.50 54.73
10 reflex 4B 71.84 69.55 92.61 60.30
Tier Correct Accuracy
easy 48/48 1.000
medium 70/72 0.972
hard 73/111 0.658
all 191/231 0.827

Ours: self-measured with JevBench + PR #241 adapter; Calibration 77.92, Capability 76.27. † self-reported. Speed/Cost: one value for every 4B model (H2O-Lightning-4B's board row); Jev: its own board row. Composite = harmonic mean of the three.

Inference — Transformers (HF)

Python 3.12, transformers==5.22.1, torch==2.15.0, einops. Load in fp16, no autocast:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "deshwalmahesh/d-edge-decision"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.float16
).to("cuda").eval()

# fp32 LM head — the option logits sit ~30–36, where one fp16 step moves a prob by a point
model.get_output_embeddings().register_forward_hook(lambda m, i, o: o.float())

def decide(state, question, options):
    lines = "\n".join(f"{chr(65+i)}: {o}" for i, o in enumerate(options))
    user = (f"Answer the question about the state with exactly one option code.\n\n"
            f"State:\n{state}\n\nQuestion: {question}\n\nOptions:\n{lines}\n\n"
            f"Reply with only the option code.")
    text = tok.apply_chat_template([{"role": "user", "content": user}], tokenize=False,
                                   add_generation_prompt=True, enable_thinking=False)
    enc = tok(text, return_tensors="pt").to("cuda")
    with torch.no_grad():  # no autocast: the attention needs q/k/v in one dtype
        logits = model(**enc, logits_to_keep=1).logits[:, -1, :].float()
    codes = [tok.encode(" " + chr(65+i), add_special_tokens=False)[-1]
             for i in range(len(options))]
    probs = torch.softmax(logits[0, codes], -1).tolist()
    return dict(zip(options, probs))

print(decide("The customer was charged twice for one order.",
             "Which team should handle this?",
             ["billing", "shipping"]))
# -> {'billing': 0.98…, 'shipping': 0.01…}

Training

Full-parameter SFT on tasksource/tasksource-jev-typed-decisions and caiovicentino1/eikos-decisions (crh225/plumb-decisions held out for test). Soft cross-entropy + 1.0 × Brier over the options, fp32 LM head and RoPE, 16k-token sequences, 1 epoch, cosine LR 3.5e-6, checkpoint selected by eval NLL.

License

Apache-2.0, inherited from the base model domyn/Domyn-Edge-SFT.

Disclaimer

This is a personal and independent experiment by the author. This experiment is not affiliated with, endorsed by, or related to any company or organization.

Downloads last month
2,393
Safetensors
Model size
4B params
Tensor type
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for deshwalmahesh/d-edge-decision

Finetuned
(6)
this model

Datasets used to train deshwalmahesh/d-edge-decision