Court-Ready: evaluating LLMs on Indian law

The paper Are LLMs Court-Ready? Evaluating Frontier Models on Indian Legal Reasoning builds an Indian dataset of 6,218 multiple-choice questions from the CLAT exams, the Delhi judicial service and the Supreme Court of India’s Advocate-on-Record exam, plus 35 long-form tasks blind-graded by five certified Advocates-on-Record. Gemini 2.5 Pro exceeded the average results of the top candidates on the multiple-choice tests: by 7.6 points on CLAT UG and by 25.3 on DHJS. On the long-form tasks the model remained below the human results for each of the five graders; errors were noted in citation discipline, procedural requirements and document structure. For Lexometrica this confirms that multiple-choice tests and the accuracy of the conclusion are not enough: separate assessments of procedure, relevance of sources and IRAC structure are needed.