CLUE: Hidden-State Clustering for Non-parametric Verification

The October 2, 2025 technical report from **Tencent AI Lab** introduces **CLUE (Clustering and Experience-based Verification)**, a novel, non-parametric method for assessing the correctness of solutions generated by **Large Language Models (LLMs)**. The authors argue that a solution's quality is geometrically encoded in the LLM's **internal hidden state trajectories**, specifically using the **activation delta** (the difference in hidden states before and after the reasoning block) as a robust signal. CLUE is a **training-free** approach that establishes **success and failure centroids** from past labeled experience and classifies new solutions by their proximity to these clusters. Empirical results demonstrate that CLUE significantly **outperforms traditional LLM-as-a-judge and confidence-based baselines** in both binary classification and solution reranking across mathematical and general reasoning benchmarks. The research highlights that models fine-tuned with **Reinforcement Learning (RL)** exhibit superior geometric separation of correct and incorrect reasoning, making them inherently stronger verifiers.Source:https://arxiv.org/pdf/2510.01591

There are no comments yet.

Please log in to write the first comment.