UBC’s Dr. Munzner wins Test of Time Award at IEEE VIS

UBC Computer Science Professor Dr. Tamara Munzner, and her co-authors, are being recognized with a 10-year Test of Time Award from the IEEE VIS conference this October for their paper, “Design Study Methodology: Reflections from the Trenches and the Stacks,” published in 2012.

Professor Munzner wrote the paper with Dr. Michael Sedlmair, now Head of the Augmented and Virtual Reality Group Visualization Research Center (VISUS) at the University of Stuttgart, and Dr. Miriah Meyer, now a professor in the Department of Science and Technology at Linköping University.

Meyer had just completed her postdoc at Harvard with Munzner as a co-supervisor, and Sedlmair was a postdoc at UBC with Munzner.

The paper received a Best Paper Honorable Mention Award in its year of publication.

Visualizing the gap

The team recognized there was a serious lack of published direction about the methodology of conducting problem-driven work in their field. In fact, in total there were only really three paragraphs on the subject at the time.

Munzner explained, “There was not enough guidance on how to solve real-world problems with real-world people, using design visualization systems to solve these problems. It’s easy to say but methodologically difficult to do in practice.”

She went on to explain how the visualization field had been progressing. “The earliest papers focused on developing new techniques. Researchers were starting to run controlled experiments on human subjects, and test whether these new techniques do the right thing. Then there was another surge of papers that really focused on solving real-world problems. We started thinking, both as authors and as reviewers, how could we advise people on how to do this better? After a lot of reading of existing literature and looking at other fields, we developed a new model of conducting problem-driven research, which included a list of pitfalls to avoid.”

Tamara said the list of 32 pitfalls seemed to resonate well with the community and proved to be extremely useful. “This paper was specifically written to provide guidance and advice to both authors and reviewers on how to do this kind of work, and what criteria one should use for judging. It’s quite different than designing in a vacuum without considering exactly what people are trying to accomplish.”

Dr. Munzner explained it’s the methodology behind the design process rather than the methods themselves they tackled, using an analogy of ingredients (methods) vs. a recipe (methodology). She stated that many people had already provided methods, which are essentially directive tools. “In design studies, your recipe should support humans in making the decisions, typically where you're augmenting rather than replacing human judgment. And that's really the sweet spot of visualization.”

Dr. Sedlamir reflected, saying, “I hope this design study methodology paper spared many subsequent PhD students from falling into the same pitfalls we had to struggle with. Together with Tamara and Miriah, we were the perfect team to work on this project, and it was a lot of fun.”

Miriah echoes Michael’s sentiments, “This paper was the result of a really wonderful, productive, and joyful collaboration with Tamara and Michael. I am thrilled that the work that felt so important to us has had a clear impact on the visualization community and beyond.”

Building in the long-term value

Tamara summarized, “One thing that's interesting about this kind of work is that you're solving a very specific problem that maybe nobody else in the world has. It's a particular style of qualitative research to reflect on what lessons you've learned, that are transferable to other kinds of problems. The research community can reflect on what you've done and amend the guidelines for their own kind of work. We are trying to emphasize that inherent long-term value.”

Well, it sure seems the community recognizes the long-lasting value of this paper, as it has stood the test of time for ten years.

Hearty congratulations, Tamara, Michael and Miriah, and thank you for this very valuable contribution to the computer visualization world. This is actually Tamara’s second go-round of receiving a Test of Time Award. In 2019 she received the same award from IEEE VIS for a 2009 paper: “A Nested Model for Visualization Design and Validation.”

At this year’s IEEE VIS 2022 conference, Tamara and her research group will also be presenting two papers:

GEViTRec: Data Reconnaissance Through Recommendation Using a Domain-Specific Visualization Prevalence Design Space, Anamaria Crisan, Shannah Elizabeth Fisher, Jennifer L. Gardy, Tamara Munzner

Visualizing Graph Neural Networks with CorGIE: Corresponding a Graph to Its Embedding, Zipeng Liu, Yang Wang, Jürgen Bernard, Tamara Munzner

About the IEEE VIS Test of Time Awards:

The award recognizes the most vibrant and influential IEEE VIS papers published in both 2002 and 2012. They are the papers that are most relevant today, having had major impact and influence within and beyond the visualization community. The organizers say, “During the IEEE VIS conference, we encourage the researchers of today to use the best articles from the past to inform their work in the future through the Test of Time Awards. We're trying to build on our heritage to establish an ambitious future by making it clear at the very start of the conference that we want current participants to aspire to be writing the papers that will be relevant in 10 or 20 years.”