I am an Assistant Professor in the Department of Computer Sciences at the University of Wisconsin-Madison. My research interests lie in the intersection between HCI, Visualization, and Data Science. I am interested in developing tools to help people interact with computational models and AI systems by leveraging visualization techniques. To better design the tools, I study and model how people perceive the model's prediction and its uncertainty.
A Bayesian view of data interpretation suggests that a visualization user should update their existing beliefs about a pa-rameter's value in accordance with the amount of information about the parameter value captured by the new observations. Extendingrecent work applying Bayesian models to understand and evaluate belief updating from visualizations, we show how the predictionsof Bayesian inference can be used to guide more rational belief updating. We design a Bayesian inference-assisteduncertaintyanalogythat numerically relates uncertainty in observed data to the user's subjective uncertainty, and aposterior visualizationthatprescribes how a user should update their beliefs given their prior beliefs and the observed data. In a pre-registered experiment on4,800 people, we find that when a newly observed data sample is relatively small (N=158), both techniques reliably improve people'sBayesian updating on average compared to the current best practice of visualizing uncertainty in the observed data. For large datasamples (N=5208), where people's updated beliefs tend to deviate more strongly from the prescriptions of a Bayesian model, we findevidence that the effectiveness of the two forms of Bayesian assistance may depend on people's proclivity toward trusting the sourceof the data. We discuss how our results provide insight into individual processes of belief updating andsubjectiveuncertainty, andhow understanding these aspects of interpretation paves the way for more sophisticated interactive visualizations for analysis andcommunication.
Externalizing one's thoughts can be helpful during data analysis, such as which one marks interesting data, notes hypotheses, and draws diagrams. In this paper, we present two exploratory studies conducted to investigate types and use of externalizations during the analysis process. We first studied how people take notes during different stages of data analysis using VoyagerNote, a visualization recommendation system augmented to support text annotations, and coupled with participants' favorite external note-taking tools (e.g., word processor, pen & paper). Externalizations manifested mostly as notes written on paper or in a word processor, with annotations atop views used almost exclusively in the initial phase of analysis. In the second study, we investigated two specific opportunities: (1) integrating digital pen input to facilitate the use of free-form externalizations and (2) providing a more explicit linking between visualizations and externalizations. We conducted the study with VoyagerInk, a visualization system that enabled free-form externalization with a digital pen as well as touch interactions to link externalizations to data. Participants created more graphical externalizations with VoyagerInk and revisited over half of their externalizations via the linking mechanism. Reflecting on the findings from these two studies, we discuss implications for the design of data analysis tools.
Visualization evaluation guidelines rarely account for the influence of users' prior beliefs before encountering a visualization. We demonstrate a Bayesian cognitive model for understanding how people interpret visualizations in light of prior beliefs and show how this model provides a guide for improving visualization designs. In a first study, we show how applying a Bayesian cognition model to a simple visualization scenario indicates that people's judgments are consistent with a hypothesis that they are doing approximate Bayesian inference. In a second study, we evaluate how sensitive our observations of Bayesian behavior are to different techniques for eliciting people subjective distributions, and to different datasets. We find that people don't behave consistently with Bayesian predictions for large sample size datasets, and this difference cannot be explained by elicitation technique. In a final study, we show how normative Bayesian inference can be used as an evaluation metric for visualizations of uncertainty.
Journalists often use visualizations and other interactive representations to support stories they convey in articles. While readers bring their prior beliefs to interpret these representations, typical models of designing them do not consider the readers' beliefs. We propose "Belief-driven data journalism" as a framework for integrating readers' beliefs in designing and supporting interaction with data-driven articles." We present four case studies to illustrate how belief-driven data journalism can serve journalistic goals and reflect on design considerations.We describe an authoring tool that we are developing to help journalists and others with varying technical expertise create belief-driven data journalism pieces.
It can be difficult to understand physical measurements (e.g., 28 lb, 600 gallons) that appear in news stories, data reports, and other documents. We develop tools that automatically re-express unfamiliar measurements using the measurements of familiar objects. Our work makes three contributions: (1) we identify effectiveness criteria for objects used in concrete measurement re-expressions; (2) we operationalize these criteria in a scalable method for mining a large dataset of concrete familiar objects with their physical dimensions from Amazon and Wikipedia; and (3) we develop automated concrete reexpression tools that implement three common re-expression strategies (adding familiar context, reunitization and proportional analogy) as energy minimization algorithms. Crowdsourced evaluations of our tools indicate that people find news articles with re-expressions more helpful and re-expressions help them to better estimate new measurements.
In addition to visualizing input data, interactive visualizations have the potential to be social artifacts that reveal other people's perspectives on the data. However, how such social information embedded in a visualization impacts a viewer's interpretation of the data remains unknown. Inspired by recent interactive visualizations that display people's expectations of data against the data, we conducted a controlled experiment to evaluate the effect of showing social information in the form of other people's expectations on people's ability to recall the data, the degree to which they adjust their expectations to align with the data, and their trust in the accuracy of the data. We found that social information that exhibits a high degree of consensus lead participants to recall the data more accurately relative to participants who were exposed to the data alone. Additionally, participants trusted the accuracy of the data less and were more likely to maintain their initial expectations when other people's expectations aligned with their own initial expectations but not with the data. We conclude by characterizing the design space for visualizing others' expectations alongside data.
People often have erroneous intuitions about the results of uncertain processes, such as scientific experiments. Many uncertainty visualizations assume considerable statistical knowledge, but have been shown to prompt erroneous conclusions even when users possess this knowledge. Active learning approaches as well as discrete (frequency) formats for probability information have been shown to improve statistical reasoning, but are rarely applied in visualizing uncertainty in scientific reports. We present a controlled study to evaluate the impact of an alternative, interactive graphical prediction technique for communicating uncertainty in experiment results. Using our technique, users sketch their prediction of the uncertainty in experimental effects prior to viewing the true sampling distribution from an experiment. We find that having a user graphically predict the possible effects from experiment replications is an effective way to improve one's ability to make predictions about replications of new experiments. Additionally, visualizing uncertainty as a set of discrete outcomes, as opposed to a continuous probability distribution, can improve recall of a sampling distribution from a single experiment. Our work has implications for various applications where it is important to elicit peoples' estimates of probability distributions and to communicate uncertainty effectively.
Information visualizations use interactivity to enable userdriven querying of visualized data. However, users' interactions with their internal representations, including their expectations about data, are also critical for a visualization to support learning. We present multiple graphically-based techniques for eliciting and incorporating a user's prior knowledge about data into visualization interaction. We use controlled experiments to evaluate how graphically eliciting forms of prior knowledge and presenting feedback on the gap between prior knowledge and the observed data impacts a user's ability to recall and understand the data. We find that participants who are prompted to reflect on their prior knowledge by predicting and self-explaining data outperform a control group in recall and comprehension. These effects persist when participants have moderate or little prior knowledge on the datasets. We discuss how the effects differ based on text versus visual presentations of data. We characterize the design space of graphical prediction and feedback techniques and describe design recommendations.
Lexical simplification of scientific terms represents a unique challenge due to the lack of a standard parallel corpora and fast rate at which vocabulary shift along with research. We introduce SimpleScience, a lexical simplification approach for scientific terminology. We use word embeddings to extract simplification rules from a parallel corpora containing scientific publications and Wikipedia. To evaluate our system we construct SimpleSciGold, a novel gold standard set for science-related simplifications. We find that our approach outperforms prior context-aware approaches at generating simplifications for scientific terms.
Distances and areas frequently appear in text articles. However, people struggle to understand these measurements when they cannot relate them to measurements of locations that they are personally familiar with. We contribute tools for generating personalized spatial analogies: re-expressions that contextualize spatial measurements in terms of locations with similar measurements that are more familiar to the user. Our automated approach takes a user's location and generates a personalized spatial analogy for a target distance or area using landmarks. We present an interactive application that tags distances, areas, and locations in a text article and presents personalized spatial analogies using interactive maps. We find that users who view a personalized spatial analogy map generated by our system rate the helpfulness of the information for understanding a distance or area 1.9 points higher (on a 7 pt scale) than when they see the article with no spatial analogy and 0.7 points higher than when they see generic spatial analogy.
User studies in the music information retrieval (MIR) domain tend to be exploratory and qualitative in nature, involving a small number of users, which makes it difficult to derive broader implications for system design. In order to fill this gap, we conducted a large-scale user survey questioning various aspects of people's music information needs and behaviors. In particular, we investigate if general music users' needs and behaviors have significantly changed over time by comparing our current survey result with a similar survey conducted in 2004. In this paper, we present the key findings from the survey data and discuss 4 emergent themes (a) the shift in access and use of personal music collections; (b) the growing need for tools to support collaborative music seeking, listening, and sharing; (c) the importance of visual music experiences; and (d) the need for ontologies for providing rich contextual information. We conclude by making specific recommendations for improving the design of MIR systems and services.
Prior user studies in the music information retrieval field have identified different personas representing the needs, goals, and characteristics of specific user groups for a usercentered design of music services. However, these personas were derived from a qualitative study involving a small number of participants and their generalizability has not been tested. The objectives of this study are to explore the applicability of seven user personas, developed in prior research, with a larger group of users and to identify the correlation between personas and the use of different types of music services. In total, 962 individuals were surveyed in order to understand their behaviors and preferences when interacting with music streaming services. Using a stratified sampling framework, key characteristics of each persona were extracted to classify users into specific persona groups. Responses were also analyzed in relation to gender, which yielded significant differences. Our findings support the development of more targeted approaches in music services rather than a universal service model.
Despite the increasing popularity of cloud-based music services, few studies have examined how users select and utilize these services, how they manage and access their music collections in the cloud, and the issues or challenges they are facing within these services. In this paper, we present findings from an online survey with 198 responses collected from users of commercial cloud music services, exploring their selection criteria, use patterns, perceived limitations, and future predictions. We also investigate differences in these aspects by age and gender. Our results elucidate previously under-studied changes in music consumption, music listening behaviors, and music technology adoption. The findings also provide insights into how to improve the future design of cloud-based music services, and have broader implications for any cloudbased services designed for managing and accessing personal media collections.
Complex jargon often makes scientific work less accessible to the general public. By employing a set of specific reporting strategies, journalists bridge these groups by delivering information about scientific advances in a readable, engaging way. One such strategy is using simpler terms in place of complex jargon. To assist in this process, we introduce DeScipher, a text editor application that suggests and ranks possible simplifications of complex terminology to a journalist while she is authoring an article. DeScipher applies simplification rules derived from a large collection of scientific abstracts and associated author summaries, and accounts for textual context in making suggestions to the journalist. In evaluating our system, we show that DeScipher is a viable application for producing useful simplifications of scientific and other terms by comparing to prior techniques used on other corpora. We also propose concrete opportunities for future development of 'journalist-in-the-loop' tools for aiding journalists in enacting science reporting strategies.
Despite the growing interests in video games as consumer products as well as objects of research, current methods for accessing video games are limited. We present Vizmo as a new way of browsing video games based on their visual style and mood. In order to test the usability and usefulness of Vizmo, we asked 19 video game experts to evaluate their interaction with the tool. The results show that experts perceived Vizmo as a novel and aesthetically pleasing game discovery tool which would be most useful for game research on historical and aesthetic aspects. We discuss five key points for improving the design of Vizmo as well as our future plan for the next iteration of this prototype game browser.
Most geographic maps visually represent physical distance; however, travel time can in some cases be more important than distance because it directly indicates availability. The technique of creating maps from temporal data is known as isochronal cartography, and is a form of distortion for clarification. In an isochronal map, congestion expands areas, while ideal travel conditions make the map shrink in comparison to the actual distance scale of a traditional map. Although there have been many applications of this technique, detailed user studies of its efficacy remain scarce, and there are conflicting views on its practical value. To attempt to settle this issue, we utilized a usercentered design process to determine which features of isochronal cartography might be most usable in practice. We developed an interactive cartographic visualization system, Traffigram, that features a novel combination of efficient isochronal map algorithms and an interface designed to give map users a quick and seamless experience while preserving geospatial integrity and aesthetics. We validated our design choices with multiple usability studies. We present our results and discuss implications for design.