Cross-Architecture LLM Ensembles, Reranking, and Prompting for Legal Information Processing

arXiv CS · · 3 min read · Engineering & Technology

Read research and analysis on Cross-Architecture LLM Ensembles, Reranking, and Prompting for Legal Information Processing published by ICANEWS, a global research journal for emerging researchers.

Key Takeaways

  • Cross-architecture ensemble achieved 96.3% accuracy in Task 4 (statute entailment), ranking first.
  • Multi-view system for the Pilot Task scored 73.1% TP accuracy and 68.2% RE F1, exceeding official entries on TP.
  • Changing prompt from single- to multi-selection raised F1 from 0.343 to 0.555 for Task 2 (legal case entailment) post-competition.
  • Replacing entailment model with Qwen3-235B and structured prompt raised Task 3 (statute retrieval and entailment) accuracy from 79.3% to 91.5% post-competition.
  • Learning-to-rank system for Task 1 (legal case retrieval) achieved F1 = 0.314, ranking 11th.

Why This Matters

The research demonstrates the effectiveness of specific AI strategies, such as ensemble models and prompt engineering, in improving accuracy for various legal information processing tasks. These results suggest potential for advanced computational tools to support legal analysis and decision-making.

Overview

Team DU participated in all five tasks of COLIEE 2026, focusing on legal information processing challenges including retrieval, entailment, and judgment prediction. The team employed open-weight systems. Key strategies included cross-architecture LLM ensembles, feature-based reranking, and retrieval-augmented prompting. The research indicated that different inductive biases are beneficial across varied legal tasks, with the effectiveness of specific methods varying by context.

Research Context

Legal information processing encompasses problems such as legal case retrieval, legal case entailment, statute retrieval and entailment, and legal judgment prediction. These problems require text matching, reasoning capabilities, and robust generalization, often with limited supervision. Team DU's participation in COLIEE 2026 addressed these challenges using systems that adhered to the competition's cutoff date for model development, specifically 15 July 2025 for Tasks 3 and 4.

Approach

Team DU adopted task-specific methodologies across the COLIEE 2026 competition:

  • Task 4 (Statute Entailment): A cross-architecture ensemble was developed, comprising nine models from three distinct architectural families.
  • Pilot Task (Tort Prediction and Rationale Extraction): A multi-view system was implemented. This system combined five claim-level models and refined verdict predictions using features derived from the initial claim predictions.
  • Task 2 (Legal Case Entailment): Initial evaluation involved a prompt design, which was later modified from a single-selection to a multi-selection approach during post-competition analysis.
  • Task 3 (Statute Retrieval and Entailment): Post-competition analysis involved replacing the original entailment model with Qwen3-235B and integrating a structured legal reasoning prompt.
  • Task 1 (Legal Case Retrieval): A learning-to-rank system was designed. This system integrated lexical and semantic retrieval methods with 34 structural, citation authority, and temporal features.

Findings

  • Task 4 (Statute Entailment): The cross-architecture ensemble of nine models achieved an accuracy of 96.3%. This placed Team DU first among 33 submissions from 11 teams.
  • Pilot Task (Tort Prediction and Rationale Extraction): The multi-view system, submitted unofficially, attained 73.1% TP accuracy and 68.2% RE F1. These scores exceeded all official entries in TP accuracy and matched the highest official entry in RE F1.
  • Task 2 (Legal Case Entailment): A post-competition evaluation using released gold labels showed that changing the prompt from single-selection to multi-selection increased the F1 score from 0.343 to 0.555. This revised F1 score exceeded the best official submission's F1 of 0.490.
  • Task 3 (Statute Retrieval and Entailment): Post-competition analysis indicated that replacing the entailment model with Qwen3-235B and applying a structured legal reasoning prompt raised accuracy from 79.3% to 91.5%.
  • Task 1 (Legal Case Retrieval): The learning-to-rank system achieved an F1 score of 0.314, ranking 11th out of 54 submissions from 22 teams.
  • Overall Effectiveness of Strategies: Cross-architecture ensembling, feature-based reranking, and retrieval-augmented prompting each demonstrated effectiveness in different specific settings within legal information processing tasks.

Why This Matters

The findings indicate that tailored computational approaches can enhance performance in specific legal information processing tasks. The observed improvements through ensemble methods, feature engineering, and targeted prompting suggest pathways for developing more robust and accurate systems for legal practitioners and researchers. The varying efficacy of different techniques across tasks highlights the need for specialized rather than monolithic solutions in legal AI.

Research Information

Institution
Team DU
Original Study
View Publication
Source
arXiv CS

About ICANEWS

ICANEWS is a global research journal for emerging researchers, publishing student and emerging researcher work across all fields.