Overview
The paper introduces Traceable Scholarship as a minimum normative condition for AI-assisted humanistic research. This framework addresses the challenge posed by generative AI, where large language models can produce scholarly-looking text rapidly. The core concern is not solely factual error but the appearance of established explanations without explicit sourcing, such as page numbers or specific editions. The concept likens a page anchor to Ariadne's thread, guiding scholars back to original sources within the generated output.
Research Context
The rise of generative AI allows for the rapid production of scholarly-looking text by large language models. The problem identified is that fluency in generated text does not equate to valid explanation. A key risk is the presentation of explanations as established without clear attribution to sources, page numbers, editions, or evidence. The paper situates Traceable Scholarship within three historical knowledge infrastructure revolutions: print, digital, and generative AI.
Approach
The researchers propose several mechanisms to establish Traceable Scholarship. These include page anchors, dual page numbers, citation-first generation, and explicit indicators like NO_EVIDENCE. The framework also incorporates human verification and a four-level compliance system. A concept called Scope Contract is also introduced. To illustrate these principles, the paper presents AIH-Infra, a three-layer reference implementation. The layers of AIH-Infra are Contexture, designed for document structuring; Open WebUI AIH-Infra, which functions as a traceable knowledge base; and AIH-Infra MCP Server, acting as an agent gateway.
Findings
Traceable Scholarship is presented as the minimum normative condition for humanistic research supported by AI. The study's approach incorporates specific methodological components:
- Page Anchors: These function as a mechanism to link generated text directly to its source.
- Dual Page Numbers: This system aids in precise referencing.
- Citation-First Generation: This approach prioritizes the integration of citations early in the text generation process.
- NO_EVIDENCE Indicator: This explicit signal is used to denote statements lacking supporting evidence.
- Human Verification: An integral step ensuring accuracy and adherence to scholarly standards.
- Four-Level Compliance: A systematic framework for evaluating the adherence to traceability principles.
- Scope Contract: A defined agreement on the scope of AI assistance and accountability.
A case study involving a 29-volume Kant Akademie-Ausgabe knowledge base demonstrated the utility of traceability. In this application, traceability supported several functions:
- Retrieval Correction: The ability to identify and rectify errors in information retrieval.
- Evidence Grading: The systematic evaluation of the quality and reliability of evidence.
- Judgment Downgrading: The re-evaluation and potential reduction in confidence of conclusions based on the quality of supporting evidence.
Why This Matters
Traceability is defined not merely as a software feature but as the fundamental condition under which humanistic research can maintain its public and refutable nature in the era of generative AI. Its implementation is critical for ensuring that scholarship remains verifiable and grounded in explicit evidence, preventing the uncritical acceptance of AI-generated content lacking clear sources.