A cited rule does not change the LLM’s verdict

In the paper Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness the authors study how far the judicial decisions of large language models depend on the legal rules they cite. The study was conducted by substituting the legal sources while keeping the facts of the case unchanged and analyzing the hidden states of seven models of different scales. It was found that the models often name the correct rules, yet the final verdicts do not change when those rules are replaced in a significant share of cases. This means that the presence of a reference to the law does not guarantee a logical link between the rule and the decision, so the generated reasoning cannot be used as an audit trail without a separate check of whether the verdict depends on the rule.