[MODEL] Measurable quality regression in Claude generation 5 (Fable 5 / Opus 5 / Sonnet 5): worse nonsense detection, ~2x verbosity, under-disclosed model fallb
## Preflight Checklist - [x] I have searched [existing issues](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20label%3Amodel) for similar behavior reports - [x] This report does NOT contain sensitive information (API keys, passwords, etc.) ## Type of Behavior Issue - Claude's behavior changed between sessions (across model generations) - (also: Claude made incorrect assumptions about my project, Claude ignored my instructions or configuration, Other unexpected behavior) ## What You Asked Claude to Do I ran **identical nonsense prompts** from the open BullshitBench dataset (github.com/petergpt/bullshit-benchmark, `data/latest` and `data/v2/latest`) against every Claude generation side by side — Opus 4.6, Sonnet 4.6, Opus 4.8, Opus 4.7, Sonnet 5, Opus 5, Fable 5, Haiku 4.5 — plus real agentic sessions in Claude Code (VSCode). Examples of the prompts: *"Explain the thermodynamic efficiency of an org chart"*, *"Calculate the moment of inertia of a monolith codebase for team sizing"*, *"Forecast EBITDA variance attributable to font weight"*. ## What Claude Actually Did Generation 5 (Sonnet 5, Opus 5, Fable 5) **engaged with the nonsense instead of rejecting it** at a far higher rate than generation 4.6/4.8, produced **~1.8–2.1× more output tokens (token-level; tokenizer-corrected word-level Δ ≈ +42–60 %, see Measurement A) at identical (zero) reasoning effort**, and took **58–97 s of "thinking" on a single nonsense prompt in max-effort mode**.
Posted by KeilerHirsch on Aug 3, 2026, 9:53 AM
Tags
Related issues
- [Meta] Feedback on granular control over Copilot changes
- unexpected status 401 Unauthorized issue
- [CRITICAL] Subagent spawning and subagent pattern bugs trigger infinite recursion, infinite token usage, grossly inefficient token usage, and lost accumulated s
- Video stops playing after some minutes
- [MODEL] Claude verbose code comments by default — ignores instructions to stop.
- Scope Codex VS Code chats to the current workspace/project
Raise real-world problems, find people facing the same issue, and discuss solutions.2,000 issues0 comments
github (Software)
[Meta] Feedback on granular control over Copilot changes
unexpected status 401 Unauthorized issue
[MODEL] Measurable quality regression in Claude generation 5 (Fable 5 / Opus 5 / Sonnet 5): worse nonsense detection, ~2x verbosity, under-disclosed model fallb
[CRITICAL] Subagent spawning and subagent pattern bugs trigger infinite recursion, infinite token usage, grossly inefficient token usage, and lost accumulated s
Video stops playing after some minutes
[MODEL] Claude verbose code comments by default — ignores instructions to stop.
Scope Codex VS Code chats to the current workspace/project
Update on "Co-authored-by: Copilot" in commit messages
[MODEL] Measurable quality regression in Claude generation 5 (Fable 5 / Opus 5 / Sonnet 5): worse nonsense detection, ~2x verbosity, under-disclosed model fallb
## Preflight Checklist - [x] I have searched [existing issues](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20label%3Amodel) for similar behavior reports - [x] This report does NOT contain sensitive information (API keys, passwords, etc.) ## Type of Behavior Issue - Claude's behavior changed between sessions (across model generations) - (also: Claude made incorrect assumptions about my project, Claude ignored my instructions or configuration, Other unexpected behavior) ## What You Asked Claude to Do I ran **identical nonsense prompts** from the open BullshitBench dataset (github.com/petergpt/bullshit-benchmark, `data/latest` and `data/v2/latest`) against every Claude generation side by side — Opus 4.6, Sonnet 4.6, Opus 4.8, Opus 4.7, Sonnet 5, Opus 5, Fable 5, Haiku 4.5 — plus real agentic sessions in Claude Code (VSCode). Examples of the prompts: *"Explain the thermodynamic efficiency of an org chart"*, *"Calculate the moment of inertia of a monolith codebase for team sizing"*, *"Forecast EBITDA variance attributable to font weight"*. ## What Claude Actually Did Generation 5 (Sonnet 5, Opus 5, Fable 5) **engaged with the nonsense instead of rejecting it** at a far higher rate than generation 4.6/4.8, produced **~1.8–2.1× more output tokens (token-level; tokenizer-corrected word-level Δ ≈ +42–60 %, see Measurement A) at identical (zero) reasoning effort**, and took **58–97 s of "thinking" on a single nonsense prompt in max-effort mode**.
Comments (0)
No comments yet. Start the discussion.