Overview
Team DU participated in all five tasks of COLIEE 2026, focusing on legal information processing challenges including retrieval, entailment, and judgment prediction. The team employed open-weight systems. Key strategies included cross-architecture LLM ensembles, feature-based reranking, and retrieval-augmented prompting. The research indicated that different inductive biases are beneficial across varied legal tasks, with the effectiveness of specific methods varying by context.
Research Context
Legal information processing encompasses problems such as legal case retrieval, legal case entailment, statute retrieval and entailment, and legal judgment prediction. These problems require text matching, reasoning capabilities, and robust generalization, often with limited supervision. Team DU's participation in COLIEE 2026 addressed these challenges using systems that adhered to the competition's cutoff date for model development, specifically 15 July 2025 for Tasks 3 and 4.
Approach
Team DU adopted task-specific methodologies across the COLIEE 2026 competition:
- Task 4 (Statute Entailment): A cross-architecture ensemble was developed, comprising nine models from three distinct architectural families.
- Pilot Task (Tort Prediction and Rationale Extraction): A multi-view system was implemented. This system combined five claim-level models and refined verdict predictions using features derived from the initial claim predictions.
- Task 2 (Legal Case Entailment): Initial evaluation involved a prompt design, which was later modified from a single-selection to a multi-selection approach during post-competition analysis.
- Task 3 (Statute Retrieval and Entailment): Post-competition analysis involved replacing the original entailment model with Qwen3-235B and integrating a structured legal reasoning prompt.
- Task 1 (Legal Case Retrieval): A learning-to-rank system was designed. This system integrated lexical and semantic retrieval methods with 34 structural, citation authority, and temporal features.
Findings
- Task 4 (Statute Entailment): The cross-architecture ensemble of nine models achieved an accuracy of 96.3%. This placed Team DU first among 33 submissions from 11 teams.
- Pilot Task (Tort Prediction and Rationale Extraction): The multi-view system, submitted unofficially, attained 73.1% TP accuracy and 68.2% RE F1. These scores exceeded all official entries in TP accuracy and matched the highest official entry in RE F1.
- Task 2 (Legal Case Entailment): A post-competition evaluation using released gold labels showed that changing the prompt from single-selection to multi-selection increased the F1 score from 0.343 to 0.555. This revised F1 score exceeded the best official submission's F1 of 0.490.
- Task 3 (Statute Retrieval and Entailment): Post-competition analysis indicated that replacing the entailment model with Qwen3-235B and applying a structured legal reasoning prompt raised accuracy from 79.3% to 91.5%.
- Task 1 (Legal Case Retrieval): The learning-to-rank system achieved an F1 score of 0.314, ranking 11th out of 54 submissions from 22 teams.
- Overall Effectiveness of Strategies: Cross-architecture ensembling, feature-based reranking, and retrieval-augmented prompting each demonstrated effectiveness in different specific settings within legal information processing tasks.
Why This Matters
The findings indicate that tailored computational approaches can enhance performance in specific legal information processing tasks. The observed improvements through ensemble methods, feature engineering, and targeted prompting suggest pathways for developing more robust and accurate systems for legal practitioners and researchers. The varying efficacy of different techniques across tasks highlights the need for specialized rather than monolithic solutions in legal AI.