Increasing the Utility of Quantitative
Empirical Studies for Meta-analysis
Heidi Lam
and Tamara Munzner
In Proceedings of the 2008 CHI Workshop on BEyond time and errors: novel
evaLuation methods for Information Visualization, pp. 21-27.
PDF | Abstract
| Figures | Talk
Paper
Abstract
Despite the long history and consistent
use of quantitative empirical methods to evaluate information visualization techniques
and systems, our understanding of interface use remains incomplete. While there
are inherent limitations to the method, such as the choice of task and data, we
believe the utility of study results can be enhanced if they were amenable to
meta-analysis. Based on our experience in extracting design guidelines from existing
quantitative studies, we recommend improvements to both study design and reporting
to promote meta-analysis: (1) Use comparable interfaces in terms of visual elements,
information content and amount displayed, levels of data organization displayed,
and interaction complexity; (2) Capture usage patterns in addition to overall
performance measurements to better identify design tradeoffs; (3) Isolate and
study interface factors instead of overall interface performance; and (4) Report
more study details, either within the publications, or as supplementary materials.
Figures
Talk
Back
to Imager pubs
Last modified: Sun April 13 17:30:55 PDT 2008