Where legal factors live inside LLMs

The factor-based approach to precedent (the HYPO and CATO systems) describes a case through typical circumstances that matter for the outcome and compares cases by them. For trade secret disputes CATO defines 26 factors, each favoring the plaintiff or the defendant: F6 - the plaintiff took measures to protect the information, F16 - the information can be obtained by reverse engineering, F25 - the defendant actually obtained it that way. The authors ask whether signals corresponding to these factors can be found inside LLMs. The corpus consists of episodes of 2-4 sentences: 6,992 episodes with one of the factors (from 217 to 324 per factor) and 901 without factors, across different industries, types of information and writing styles. The texts were generated by one model, labeled by another, and disputed cases were resolved by a third; in a spot check of 220 episodes a human agreed with the labels in 98.6% of cases.

Activations of three open models - Qwen3-1.7B, Qwen3-8B and Gemma-2 2B - are taken at three depths (roughly a quarter, a half and three quarters of the layers) and averaged over the episode’s tokens; the model weights are not changed. The first question, whether a factor can be read from activations, is answered by a linear classifier over the whole vector. For the second, whether the signal is concentrated in a few places, published sparse autoencoders (SAE) are used, which decompose activations into tens of thousands of features: one or five features that best separate episodes with the factor from the rest are selected, and a small classifier is trained on them. For comparison, five random features and a TF-IDF classifier over words and word pairs are taken. Quality is measured by AUC on a separate test part of the corpus of 3,961 episodes.

The whole-vector classifier recognizes each of the 26 factors in all three models with AUC 0.998-1.000. At the middle layer, five random features give on average 0.534-0.578, close to random ranking; one selected feature gives 0.944-0.954, five selected ones 0.986-0.988, almost as good as the whole vector. If the best layer is taken for each factor, the AUC of a single feature is no lower than 0.86, with a median of 0.961-0.972. The authors list 78 “layer - feature number” coordinates, one per factor in each model. Some features are shared by related factors: in Qwen3-1.7B the same feature best predicts both F16 and F25, that is, the possibility of reverse engineering and the fact of it. Under repeated sampling the selected sets of five features do not always coincide, but the quality and the classifier weights are preserved.

The authors name the limitations themselves and consider the work a proof of concept. The episodes are short, synthetic, in English and only about US trade secrets; in real cases factors occur together and facts are contested, and testing on the texts of decisions is still ahead. The TF-IDF word classifier on this corpus is also almost error-free, so the result shows that factors can be read from activations but does not separate them from wording; some episodes with and without a factor also differ in length. The models are small, from 1.7 to 8 billion parameters. For court analytics the paper offers a way to check whether a model distinguishes the legally relevant circumstances of a case, not only whether it gives the right answer: the features found can be used to look for facts that the model confuses or misses, but that requires experiments on real cases, which the authors plan.