PARCEL: checking a claim against its source

The paper When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection presents the PARCEL benchmark for checking whether a legal claim is supported by the source it cites. The authors collected 3,396 claims in parenthetical format based on recent decisions of the New York Court of Appeals, labeled “supported”, “contradicted” and “not found”, and tested current language models in zero-shot mode as a three-class inference task. The best models reach an accuracy of 0.97, yet they label unsupported claims as supported even when given the full text of the decision. The models detect the absence of support worse than a direct contradiction, and accuracy drops most on fabricated but plausible citations. The benchmark provides a way to measure the groundedness of answers from legal RAG systems at the level of an individual claim.