Overview
Research indicates that artificial intelligence (AI) tools performing evidence synthesis on fertility decline exhibit a significant metadata bias depending on the terminology used. The same phenomenon, when framed clinically, generates a substantially more complete and machine-legible metadata catalogue compared to a social framing. This discrepancy suggests that the cataloguing process, which AI synthesis initiates with, acts as a filter on the available evidence base, potentially impacting policymakers' understanding of fertility-related issues.
Research Context
The declining fertility rate is identified as a critical policy question for the coming decade. AI tools are increasingly shaping what policymakers understand about this issue by synthesizing available evidence. Previous research has already established that databases often under-index materials from the social sciences, as well as books and grey literature (Visser et al., 2021). This study specifically explores whether metadata gaps affect the comprehensiveness of AI synthesis on the determinants of (in)fertility by holding the topic constant and varying its framing.
Approach
The researchers utilized OpenAlex queries to investigate the phenomenon of fertility decline. They constructed two distinct query baskets to represent different framings of the same underlying topic: a 'clinical basket' and a 'social basket'.
- The 'clinical basket' included terms such as "infertility," "subfertility," "ART" (Assisted Reproductive Technology), "IVF" (In Vitro Fertilization), and "fecundity." This basket yielded 101,645 results.
- The 'social basket' included terms such as "childlessness," "social infertility," "fertility intentions," and "reproductive decision-making." This basket generated 3,646 results.
These two sets of results were then compared across several metadata completeness metrics, including open access rates, output types, and institutional provenance.
Findings
The study observed consistent differences in metadata completeness and characteristics between the two framings:
- The 'social basket' framing was found to be consistently less 'machine-legible' compared to the 'clinical basket'. This suggests that metadata associated with socially framed fertility research is less amenable to automated processing by AI tools.
- Output from the 'social basket' skewed towards books and dissertations. In contrast, authorship for the 'clinical basket' tended to be university-based rather than healthcare-based.
- Open access rates were found to be essentially equal between the two baskets, with the clinical basket showing 43.1% open access and the social basket showing 41.3% open access. This indicates that the observed metadata gap is primarily related to indexing depth rather than paywall restrictions. Consequently, simple mandates for open access alone would not resolve the identified indexing discrepancy.
- The bias documented in this study further complicates the extraction stage for AI tools. Prior research (Uprety et al., 2025) suggests that even before any coverage bias, AI language models (LLMs) already miss a significant portion of hypotheses and claims when asked to extract them from mixed literature. The metadata bias compounds this existing challenge, making comprehensive evidence synthesis more difficult.
Why This Matters
The observed metadata bias has implications for how AI tools interpret and synthesize evidence on fertility decline. If policymakers rely on AI-synthesized evidence, the differential completeness of metadata between clinical and social framings could lead to an incomplete or skewed understanding of the determinants of (in)fertility, potentially influencing policy decisions.