Overview
Research into visualization often yields recommendations for practitioners and researchers regarding action or design decisions, frequently labeled as "guidelines." However, the term "guideline" exhibits ambiguity and a loose definition, leading to varied interpretations within the visualization community. What one researcher considers a guideline, another might find too broad, too loose, or too strict. This work investigates a wider array of terms capable of expressing desirable outcomes in visualization research, aiming to clarify how these terms are understood within the community and in relation to similar concepts.
Research Context
The foundational expectation in visualization research is that its outcomes should inform how researchers and practitioners proceed or make design choices. These outcomes are commonly communicated as "guidelines." The inherent ambiguity and imprecise definition of "guideline" present a challenge, as community members may hold divergent understandings of what constitutes an effective or appropriate guideline. This disparity underscores a necessity to examine the terminology used to describe research outcomes beyond the singular term "guideline."
Approach
The study employed a multi-stage approach to characterize terminology around visualization design guidelines:
- An initial exploratory study was conducted with experts.
- This was followed by a crowdsourcing study, which involved two distinct phases:
- A mapping phase with 30 participants.
- A rating phase with 42 participants.
- An analysis of terminology use was performed on a dataset comprising 3,877 papers published in IEEE VIS from 1990 to 2024.
Findings
The research identified that the term "guideline" is both ambiguous and loosely defined within the visualization research community. Different researchers hold varying interpretations regarding its scope and rigor. Based on the findings derived from the expert study, crowdsourcing study, and analysis of IEEE VIS papers, the authors advocate for more nuanced and precise discussions concerning research outcomes and their dissemination. This call for precision extends to communication targeted at the broader community, encompassing practitioners and students.
Why This Matters
The observed ambiguity around the term "guideline" indicates a potential for misinterpretation of research outcomes by both researchers and practitioners. Clarifying and expanding the terminology used to describe visualization research results can lead to more effective communication and application of these findings, benefiting students, practitioners, and researchers by providing a more precise understanding of actionable recommendations.