Show HN: Klarity – Open-source tool to analyze uncertainty/entropy in LLM output
We've open-sourced Klarity - a tool for analyzing uncertainty and decision-making in LLM token generation. It provides structured insights into how models choose tokens and where they show uncertainty. What Klarity does: - Real-time analysis of model uncertainty during generation - Dual analysis combining log probabilities and semantic understanding - Structured JSON output with actionable insights - Fully self-hostable with customizable analysis models The tool works by analyzing each step of text generation and returns a structured JSON: - uncertainty_points: array of {step, entropy, options[], type} - high_confidence: array of {step, probability, token, context} - risk_areas: array of {type, steps[], motivation} - suggestions: array of {issue, improvement} Currently supports hugging face transformers (more frameworks coming), we tested extensively with Qwen2.5 (0.5B-7B) models, b
Why it failed
We've open-sourced Klarity - a tool for analyzing uncertainty and decision-making in LLM token generation. It provides structured insights into how models choose tokens and where they show uncertainty. What Klarity does: - Real-time analysis of model uncertainty during gener
- Kind
- startup
- Status
- Shut down
- Inventor
- mrciffa
- Ended
- 2025
- Source
- hackernews