Pubs: InfoVis @ UBC CS

Dirty Data in the Newsroom: Comparing Data Preparation in Journalism and Data Science

Stephen Kasica, Charles Berret, and Tamara Munzner
Proc. CHI Conf. Human Factors in Computing Systems (CHI), 2023

The work involved in gathering, wrangling, cleaning, and otherwise preparing data for analysis is often the most time consuming and tedious aspect of data work. Although many studies describe data preparation within the context of data science workflows, there has been little research on data preparation in data journalism. We address this gap with a hybrid form of thematic analysis that combines deductive codes derived from existing accounts of data science workflows and inductive codes arising from an interview study with 36 professional data journalists. We extend a previous model of data science work to incorporate detailed activities of data preparation. We synthesize 60 dirty data issues from 16 taxonomies on dirty data and our interview data, and we provide a novel taxonomy to characterize these dirty data issues as discrepancies between mental models. We also identify four challenges faced by journalists: diachronic, regional, fragmented, and disparate data sources.

[Pre-Print PDF]

VizSnippets: Compressing Visualization Bundles Into Representative Previews for Browsing Visualization Collections

Michael Oppermann and Tamara Munzner
IEEE Transactions on Visualization and Computer Graphics (TVCG), Proc. IEEE VIS 2021, 2022.

Visualization collections, accessed by platforms such as Tableau Online or Power BI, are used by millions of people to share and access diverse analytical knowledge in the form of interactive visualization bundles. Result snippets, compact previews of these bundles, are presented to users to help them identify relevant content when browsing collections. Our engagement with Tableau product teams and review of existing snippet designs on five platforms showed us that current practices fail to help people judge the relevance of bundles because they include only the title and one image. Users frequently need to undertake the time-consuming endeavour of opening a bundle within its visualization system to examine its many views and dashboards. In response, we contribute the first systematic approach to visualization snippet design. We propose a framework for snippet design that addresses eight key challenges that we identify. We present a computational pipeline to compress the visual and textual content of bundles into representative previews that is adaptive to a provided pixel budget and provides high information density with multiple images and carefully chosen keywords.

[Pre-Print PDF]

TimeElide: Visual Analysis of Non-Contiguous Time Series Slices

Michael Oppermann, Luce Liu, and Tamara Munzner
IEEE VIS Short Papers 2021

We introduce the design and implementation of TimeElide, a visual analysis tool for the novel data abstraction of non-contiguous time series slices, namely time intervals that contain a sequence of time-value pairs but are not adjacent to each other. We present a visual encoding design space as an underpinning of TimeElide, and the new sparkbox technique for visualizing fine and coarse grained temporal structures within one view. Datasets from different domains and with varying characteristics guided the development and their analysis provides preliminary evidence of TimeElide's utility.

[Pre-Print PDF]

GEViTRec: Data Reconnaissance Through Recommendation Using a Domain-Specific Visualization Prevalence Design Space

Anamaria Crisan, Shannah Elizabeth Fisher, Jennifer L. Gardy, and Tamara Munzner
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021, Early Access.

Genomic Epidemiology (genEpi) is a branch of public health that uses many different data types including tabular, network, genomic, and geographic, to identify and contain outbreaks of deadly diseases. Due to the volume and variety of data, it is challenging for genEpi domain experts to conduct data reconnaissance; that is, have an overview of the data they have and make assessments toward its quality, completeness, and suitability. We present an algorithm for data reconnaissance through automatic visualization recommendation, GEViTRec. Our approach handles a broad variety of dataset types and automatically generates visually coherent combinations of charts, in contrast to existing systems that primarily focus on singleton visual encodings of tabular datasets. We automatically detect linkages across multiple input datasets by analyzing non-numeric attribute fields, creating a data source graph within which we analyze and rank paths. For each high-ranking path, we specify chart combinations with positional and color alignments between shared fields, using a gradual binding approach to transform initial partial specifications of singleton charts to complete specifications that are aligned and oriented consistently. A novel aspect of our approach is its combination of domain-agnostic elements with domain-specific information that is captured through a domain-specific visualization prevalence design space. Our implementation is applied to both synthetic data and real Ebola outbreak data. We compare GEViTRec's output to what previous visualization recommendation systems would generate, and to manually crafted visualizations used by practitioners. We conducted formative evaluations with ten genEpi experts to assess the relevance and interpretability of our results.

[Pre-Print PDF]

Visualizing Graph Neural Networks with CorGIE: Corresponding a Graph to Its Embedding

Zipeng Liu, Yang Wang, Jürgen Bernard, and Tamara Munzner
IEEE TVCG 2022, to appear

Graph neural networks (GNNs) are a class of powerful machine learning tools that model node relations for making predictions of nodes or links. GNN developers rely on quantitative metrics of the predictions to evaluate a GNN, but similar to many other neural networks, it is difficult for them to understand if the GNN truly learns characteristics of a graph as expected. We propose an approach to corresponding an input graph to its node embedding (aka latent space), a common component of GNNs that is later used for prediction. We abstract the data and tasks, and develop an interactive multi-view interface called CorGIE to instantiate the abstraction. As the key function in CorGIE, we propose the K-hop graph layout to show topological neighbors in hops and their clustering structure. To evaluate the functionality and usability of CorGIE, we present how to use CorGIE in two usage scenarios, and conduct a case study with two GNN experts.

[Pre-Print PDF]

Data-First Visualization Design Studies

Michael Oppermann and Tamara Munzner
IEEE VIS Workshop Evaluation and Beyond - Methodological Approaches for Visualization (BELIV), 2020

We introduce the notion of a data-first design study which is triggered by the acquisition of real-world data instead of specific stakeholder analysis questions. We propose an adaptation of the design study methodology framework to provide practical guidance and to aid transferability to other data-first design processes. We discuss opportunities and risks by reflecting on two of our own data-first design studies. We review 64 previous design studies and identify 16 of them as edge cases with characteristics that may indicate a data-first design process in action.

[Pre-Print PDF]

VizCommender: Computing Text-Based Similarity in Visualization Repositories for Content-Based Recommendations

Michael Oppermann, Robert Kincaid, and Tamara Munzner
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2020.

Cloud-based visualization services have made visual analytics accessible to a much wider audience than ever before. Systems such as Tableau have started to amass increasingly large repositories of analytical knowledge in the form of interactive visualization workbooks. When shared, these collections can form a visual analytic knowledge base. However, as the size of a collection increases, so does the difficulty in finding relevant information. Content-based recommendation (CBR) systems could help analysts in finding and managing workbooks relevant to their interests. Toward this goal, we focus on text-based content that is representative of the subject matter of visualizations rather than the visual encodings and style. We discuss the challenges associated with creating a CBR based on visualization specifications and explore more concretely how to implement the relevance measures required using Tableau workbook specifications as the source of content data. We also demonstrate what information can be extracted from these visualization specifications and how various natural language processing techniques can be used to compute similarity between workbooks as one way to measure relevance. We report on a crowd-sourced user study to determine if our similarity measure mimics human judgement. Finally, we choose latent Dirichlet allocation (LDA) as a specific model and instantiate it in a proof-of-concept recommender tool to demonstrate the basic function of our similarity measure.

[Pre-Print PDF] [Video Preview] [Supp. Materials PDF]

A systematic method for surveying data visualizations and a resulting genomic epidemiology visualization typology: GEViT

Anamaria Crisan, Jennifer L. Gardy, and Tamara Munzner
In Bioinformatics, May 2019

Data visualization is an important tool for exploring and communicating findings from genomic and healthcare datasets. Yet, without a systematic way of organizing and describing the design space of data visualizations, researchers may not be aware of the breadth of possible visualization design choices or how to distinguish between good and bad options. We have developed a method that systematically surveys data visualizations using the analysis of both text and images. Our method supports the construction of a visualization design space that is explorable along two axes: why the visualization was created and how it was constructed. We applied our method to a corpus of scientific research articles from infectious disease genomic epidemiology and derived a Genomic Epidemiology Visualization Typology (GEViT) that describes how visualizations were created from a series of chart types, combinations, and enhancements. We have also implemented an online gallery that allows others to explore our resulting design space of visualizations. Our results have important implications for visualization design and for researchers intending to develop or use data visualization tools. Finally, the method that we introduce is extensible to constructing visualizations design spaces across other research areas.

[DOI] [Paper Project Page]

Adjutant: an R-based tool to support topic discovery for systematic and literature reviews

Anamaria Crisan, Tamara Munzner, and Jennifer L. Gardy
In Bioinformatics, March 2019

Adjutant is an open-source, interactive, and R-based application to support mining PubMed for literature reviews. Given a PubMed-compatible search query, Adjutant downloads the relevant articles and allows the user to perform an unsupervised clustering analysis to identify data-driven topic clusters. Following clustering, users can also sample documents using different strategies to obtain a more manageable dataset for further analysis. Adjutant makes explicit trade-offs between speed and accuracy, which are modifiable by the user, such that a complete analysis of several thousand documents can take a few minutes. All analytic datasets generated by Adjutant are saved, allowing users to easily conduct other downstream analyses that Adjutant does not explicitly support.

[DOI] [Supplemental Materials] [Paper Project Page]

Ocupado: Visualizing Location-Based Counts Over Time Across Buildings

Michael Oppermann and Tamara Munzner
Computer Graphics Forum (Proc. EuroVis 2020).

We present a multi-year design study that resulted in Ocupado, a set of visual decision-support tools centered around occupancy data for stakeholders in facilities management and planning. Ocupado uses WiFi devices as a proxy for human presence, capturing location-based counts that preserve privacy without trajectories. We contribute data and task abstractions for studying space utilization for combinations of data granularities in both space and time. In addition, we contribute generalizable design choices for visualizing location-based counts relating to indoor environments.

[pre-print PDF]

Data-driven Multi-level Segmentation of Image Editing Logs

Zipeng Liu, Zhicheng Liu, Tamara Munzner
Proc. CHI Conf. Human Factors in Computing Systems (CHI), 2020

We propose a multi-level segmentation model that works for many image editing tasks including poster creation, portrait retouching, and special effect creation. Our model takes into account features derived from four event attributes collected in realistically complex Photoshop sessions with expert users: command, timestamp, image content, and artwork layer.

[pre-print PDF]

The Sprawlter Graph Readability Metric: Combining Sprawl and Area-aware Clutter

Zipeng Liu, Takayuki Itoh, Jessica Q. Dawson, Tamara Munzner
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2020.

We propose the sprawlter metric for node-link graph readability to measure both sprawl, which is the sparsity of graph elements, and clutter in a more detailed way, which goes beyond traditional counts and handles multi-level graph layout.

[pre-print PDF]

Uncovering Data Landscapes through Data Reconnaissance and Task Wrangling.

Anamaria Crisan and Tamara Munzner
IEEE VIS

Domain experts are inundated with new and heterogeneous types of data and require better and more specific types of data visualization systems to help them. In this paper, we consider the data landscape that domain experts seek to understand, namely the set of datasets that are either currently available or could be obtained. Experts need to understand this landscape to triage which data analysis projects might be viable, out of the many possible research questions that they could pursue. We identify data reconnaissance and task wrangling as processes that experts undertake to discover and identify sources of data that could be valuable for some specific analysis goal. These processes have thus far not been formally named or defined by the research community. We provide formal definitions of data reconnaissance and task wrangling and describe how they relate to the data landscape that domain experts must uncover. We propose a conceptual framework with a four-phase cycle of acquire, view, assess, and pursue that occurs within three distinct chronological stages, which we call fog and friction, informed data ideation, and demarcation of final data. Collectively, these four phases embedded within three temporal stages delineate an expert’s progressively evolving understanding of the data landscape. We describe and provide concrete examples of these processes within the visualization community through an initial systematic analysis of previous design studies, identifying situations where there is evidence that they were at play. We also comment on the response of domain experts to this framework, and suggest design implications stemming from these processes to motivate future research directions. As technological changes will only keep adding unknown terrain to the data landscape, data reconnaissance and task wrangling are important processes that need to be more widely understood and supported by the data visualization tools. By articulating a concrete understanding of this challenge and its implications, our work impacts the design and evaluation of data visualization systems.

[PDF]

How to Evaluate an Evaluation Study? Comparing and Contrasting Practices in Vis with Those of Other Disciplines.

Ana Crisan and Madison Elliott
BELIV'18 - an IEEE VIS affilitated workshop

Evaluative practices within vis research are not routinely comparedto those of psychology, sociology, or other areas of empirical study,leaving vis vulnerable to the replicability crisis that has embroiledscientific research more generally. In this position paper, we com-pare contemporary vis evaluative practices against those in thoseother disciplines, and make concrete recommendations as to how visevaluative practice can be improved through the use of quantitative,qualitative, and mixed research methods. We summarize our discus-sion and recommendations as a checklist, that we intend to be useda resource for vis researchers conducting evaluative studies, and forreviewers evaluating the merits of such studies.

[PDF]

Segmentifier: Interactive Refinement of Clickstream Data.

Kimberly Dextras-Romagnino, and Tamara Munzner,
Computer Graphics Forum (Proceedings of EuroVis 2019).

Clickstream data has the potential to provide insights into e-commerce consumer behavior, but previous techniques fall short of handling the scale and complexity of real-world datasets because they require relatively clean and small input. We present Segmentifier, a novel visual analytics interface that supports an iterative process of refining collections of action sequences into meaningful segments. We present task and data abstractions for clickstream data analysis, leading to a high-level model built around an iterative view-refine-record loop with outcomes of conclude with an answer, export segment for further analysis in downstream tools, or abandon the question for a more fruitful analysis path. Segmentifier supports fast and fluid refinement of segments through tightly coupled visual encoding and interaction with a rich set of views that show evocative derived attributes for segments, sequences, and actions in addition to underlying raw sequences. These views support fast and fluid refinement of segments through filtering and partitioning attribute ranges. Interactive visual queries on custom action sequences are aggregated according to a three-level hierarchy. Segmentifier features a detailed glyph-based visual history of the automatically recorded analysis process showing the provenance of each segment as an analysis path of attribute constraints. We demonstrate the effectiveness of our approach through a usage scenario with real-world data and a case study documenting the insights gained by a corporate e-commerce analyst.

[pre-print PDF]

Aggregated Dendrograms for Visual Comparison Between Many Phylogenetic Trees.

Zipeng Liu, Shing Hei Zhan, and Tamara Munzner,
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2019.

We present the new visual encoding idiom of aggregated dendrograms to concisely summarize the topological relationships between interactively chosen focal subtrees according to biologically meaningful criteria, and provide a layout algorithm that automatically adapts to the available screen space.

[pre-print PDF]

Uncovering Spatiotemporal Dynamics From Non-Trajectory Data.

Michael Oppermann and Tamara Munzner
CityVis Workshop at IEEE VIS 2018.

We discuss general implications of spatiotemporal non-trajectory data in terms of ethics, data preprocessing, tasks, and visual encodings.

[pre-print PDF] [ Talk]

Evidence-Based Design and Evaluation of a Whole Genome Sequencing Clinical Report for the Reference Microbiology Laboratory.

Anamaria Crisan, Geoff McKee, Tamara Munzner, and Jennifer L. Gardy

Microbial genomics is playing an increasingly important role in public health microbiology, and its successful implementation in the clinic will rely not just on validation and accreditation of WGS-based tests, but also in how effective the resulting reports are to stakeholders, including clinicians, nurses, laboratory staff, researchers, and surveillance expert. Using design study methodology, we developed a two-page report template to communicate WGS-derived test results related to TB diagnosis, drug susceptibility testing, and clustering. The process and findings for this WGS report design project target domain specialists in the microbial genomics community who typically develop such reports and also more complex bioinformatics software, however we also present the aspects of our work that we believe are of interest to the Information Visualization research community.

[pre-print PDF] [ Talk] [ Additional Details]

Bridging From Goals to Tasks with Design Study Analysis Reports.

Heidi Lam, Melanie Tory, and Tamara Munzner
In IEEE Transactions on Visualization and Computer Graphics (TVCG, Proceedings of InfoVis 2017).

Framework based on analysis reports derived from open-coding 20 design study papers published at IEEE InfoVis 2009-2015, to build on the previous work of abstractions that collectively encompass a broad variety of domains.

[pre-print PDF]

Timelines Revisited: A Design Space and Considerations for Expressive Storytelling.

Matthew Brehmer, Bongshin Lee, Benjamin Bach, Nathalie Henry Riche, and Tamara Munzner
In IEEE Transactions on Visualization and Computer Graphics (TVCG).

There are many ways to visualize event sequences as timelines. In a storytelling context where the intent is to convey multiple narrative points, a richer set of timeline designs may be more appropriate than the narrow range that has been used for exploratory data analysis by the research community. Informed by a survey of 263 timelines, we present a design space for storytelling with timelines that balances expressiveness and effectiveness, identifying 14 design choices characterized by three dimensions: representation, scale, and layout. Twenty combinations of these choices are viable timeline designs that can be matched to different narrative points, while smooth animated transitions between narrative points allow for the presentation of a cohesive story, an important aspect of both interactive storytelling and data videos. We further validate this design space by realizing the full set of viable timeline designs and transitions in a proof-of-concept sandbox implementation that we used to produce seven example timeline stories. Ultimately, this work is intended to inform and inspire the design of future tools for storytelling with timelines.

[pre-print PDF] [IEEE DL Link]

On Regulatory and Organizational Constraints in Visualization Design and Evaluation.

Anamaria Crisan, Jennifer Gardy, and Tamara Munzner
In Proceedings of the ACM Workshop on Beyond time and errors: novel evaluation methods for Information Visualization (BELIV), 2016, p. 1-9.

Problem-based visualization research provides explicit guidance toward identifying and designing for the needs of users, but absent is more concrete guidance toward factors external to a user's needs that also have implications for visualization design and evaluation. This lack of more explicit guidance can leave visualization researchers and practitioners vulnerable to unforeseen constraints beyond the user's needs that can affect the validity of evaluations, or even lead to the premature termination of a project. Here we explore two types of external constraints in depth, regulatory and organizational constraints, and describe how these constraints impact visualization design and evaluation. By borrowing from techniques in software development, project management, and visualization research we recommend strategies for identifying, mitigating, and evaluating these external constraints through a design study methodology. Finally, we present an application of those recommendations in a healthcare case study. We argue that by explicitly incorporating external constraints into visualization design and evaluation, researchers and practitioners can improve the utility and validity of their visualization solution and improve the likelihood of successful collaborations with industries where external constraints are more present.

[pre-print PDF] [ ACM DL Link]

PaperQuest

PaperQuest: A Visualization Tool to Support Literature Review, a poster by Antoine Ponsard, Francisco "Pax" Escalona, and Tamara Munzner was accepted at ACM CHI 2016, May 7 - 12, 2016, in San Jose CA, USA.

[PaperQuest CHI Extended Abstract][PaperQuest prototype]

SEQIT: Visualizing Sequences of Interest in Eye Tracking Data.

Mike Wu and Tamara Munzner.
Poster Proceedings of IEEE Conference on Information Visualization (InfoVis) 2015.

A visualization system designed for sequence analysis of eye tracking data.

[pre-print PDF]

Matches, Mismatches, and Methods: Multiple-View Workflows for Energy Portfolio Analysis.

Matthew Brehmer, Jocelyn Ng, Kevin Tate, and Tamara Munzner.
IEEE Transactions on Visualization and Computer Graphics (TVCG, Proceedings of InfoVis 2015), 22(1), p. 449-458, 2016.

A design study in the domain of commercial energy management and conservation.

[pre-print PDF]

TimeLineCurator: Interactive Authoring of Visual Timelines from Unstructured Text.

Johanna Fulda, Matthew Brehmer, and Tamara Munzner.
IEEE Transactions on Visualization and Computer Graphics (TVCG, Proceedings of VAST 2015), 22(1), p. 300-309, 2016.

Browser-based vis tool for curating timelines generated from freeform text.

[pre-print PDF]

Detangler: Visual Analytics for Multiplex Networks.

Benjamin Renoust, Guy Melançon, and Tamara Munzner.
Computer Graphics Forum (Proceedings of EuroVis 2015).

A multiplex network has links of different types, allowing it to express many overlapping types of relationships. A core task in network analysis is to evaluate and understand group cohesion; that is, to explain why groups of elements belong together based on the underlying structure of the network. We present Detangler, a system that supports visual analysis of group cohesion in multiplex networks through dual linked views. These views feature new data abstractions derived from the original multiplex network: the substrate network and the catalyst network. We contribute two novel techniques that allow the user to analyze the complex structure of the multiplex network without the extreme visual clutter that would result from simply showing it directly. The harmonized layout visual encoding technique provides spatial stability between the substrate and catalyst views. The pivot brushing interaction technique supports linked highlighting between the views based on computations in the underlying multiplex network to leapfrog between subsets of catalysts and substrates. We present results from the motivating application domain of annotated news documents with a usage scenario and preliminary expert feedback. A second usage scenario presents group cohesion analysis of the social network of the early American independence movement.

[pre-print pdf] [supplemental material]

Visualizing Dimensionally-Reduced Data: Interviews with Analysts and a Characterization of Task Sequences.

Matthew Brehmer, Michael Sedlmair, Stephen Ingram, and Tamara Munzner.
In Proceedings of the ACM Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization (BELIV), 2014, p. 1-8.

We characterize five task sequences related to visualizing dimensionally-reduced data, drawing from data collected from interviews with ten data analysts spanning six application domains, and from our understanding of the technique literature.

[pre-print pdf]

A search-set model of path tracing in graphs.

Jessica Q. Dawson, Tamara Munzner, and Joanna McGrenere.
Information Visualization, 14(4):308–338, 2015.

We present a predictive model of human behaviour when tracing paths through a node-link graph, a low-level abstract task that feeds into many other visual data analysis tasks that require understanding topological structure. Results of a user study show that our approach provides modest improvements for predictions of response time and error using search-set factors.

[pre-print pdf] [ DOI]

Visualization Analysis and Design

Tamara Munzner.
AK Peters Visualization Series (CRC Press), 2014.

Visualization Analysis and Design provides a systematic, comprehensive framework for thinking about visualization in terms of principles and design choices. The book features a unified approach encompassing information visualization techniques for abstract data, scientific visualization techniques for spatial data, and visual analytics techniques for interweaving data transformation and analysis with interactive visual exploration. It emphasizes the careful validation of effectiveness and the consideration of function before form. The book is suitable for a broad set of readers, from beginners to more experienced visualization designers. It does not assume any previous experience in programming, mathematics, human–computer interaction, or graphic design and can be used in an introductory visualization course at the graduate or undergraduate level.

[pre-order (CRC)] [ pre-order (amazon)]

Overview: The Design, Adoption, and Analysis of a Visual Document Mining Tool For Investigative Journalists.

Matthew Brehmer, Stephen Ingram, Jonathan Stray, and Tamara Munzner.
In IEEE Trans. Visualization and Computer Graphics (Proc. InfoVis), 20(12), p. 2271-2280, 2014.

A design study about Overview, an application for the systematic analysis of large document collections based on document clustering, visualization, and tagging.

publications: papers, chapters, books, and monographs

technical reports, posters, notes