LegalReasoner: verifying reasoning steps

In the paper LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning, predicting the outcome of a case is built as a verifiable process rather than classification of the result. The model identifies the disputed issues, builds a chain of reasoning and scores each step for correctness, progress and potential. For training, the LegalHK dataset of 58,130 Hong Kong cases was assembled. Accuracy of the conclusion on LegalHK was 80.27 versus 72.37 for a fine-tuned Llama-3.1-70B and 77.15 for GPT-o1; on the Chinese CMDL it was 76.95 versus 72.03. Without step-by-step reasoning the result dropped to 69.49, without verification and correction of steps to 74.49. The authors point to the cost of verifying each step and the noise of automatic labeling. For the Lexometrica leaderboard this is an argument to score Issue, Rule, Application and Conclusion separately rather than with a single overall score.