• 2021 Fall Data Visualization
  • 2021 Spring Modeling User Interaction
  • 2020 Fall Data Visualization

I am an Assistant Professor in the Department of Computer Sciences at the University of Wisconsin-Madison. My research interests lie in the intersection between HCI, Visualization, and Data Science. I am interested in developing tools and algorithms to help people with varying abilities interact with data and visualizations.

Best Paper Honorable Mention
Communicating Visualizations without Visuals: Investigation of Visualization Alternative Text for People with Visual Impairments
Crescentia Jung, Shubham Mehta Atharva Kulkarni, Yuhang Zhao, Yea-Seul Kim
Alternative text is critical in communicating graphics to people who are blind or have low vision. Especially for graphics that contain rich information, such as visualizations, poorly written or an absence of alternative texts can worsen the information access inequality for people with visual impairments. Read More In this work, we consolidate existing guidelines and survey current practices to inspect to what extent current practices and recommendations are aligned. Then, to gain more insight into what people want in visualization alternative texts, we interviewed 22 people with visual impairments regarding their experience with visualizations and their information needs in alternative texts. The study findings suggest that participants actively try to construct an image of visualizations in their head while listening to alternative texts and wish to carry out visualization tasks (e.g., retrieve specific values) as sighted viewers would. The study also provides ample support for the need to reference the underlying data instead of visual elements to reduce users' cognitive burden. Informed by the study, we provide a set of recommendations to compose an informative alternative text. Read Less
Accessible Visualization: Design Space, Opportunities, and Challenges
Nam Wook Kim, Shakila Joyner, Amalia Riegelhuth, Yea-Seul Kim
Visualizations are now widely used across disciplines to understand and communicate data. The benefit of visualizations lies in leveraging our natural visual perception. However, the sole dependency on vision can produce unintended discrimination against people with visual impairments. While the visualization field has seen enormous growth in recent years, supporting people with disabilities is much less explored. In this work, we examine approaches to support this marginalized user group, focusing on visual disabilities. We collected and analyzed papers published for the last 20 years on visualization accessibility. Read More We mapped a design space for accessible visualization that includes seven dimensions: user group, literacy task, chart type, interaction, information granularity, sensory modality, assistive technology. We described the current knowledge gap in light of the latest advances in visualization and presented a preliminary accessibility model by synthesizing findings from existing research. Finally, we reflected on the dimensions and discussed opportunities and challenges for future research. Read Less
Bayesian-Assisted Inference from Visualized Data
Yea-Seul Kim, Paula Kayongo, Madeleine Grunde-McLaughlin, Jessica Hullman
A Bayesian view of data interpretation suggests that a visualization user should update their existing beliefs about a parameter's value in accordance with the amount of information about the parameter value captured by the new observations. Extending recent work applying Bayesian models to understand and evaluate belief updating from visualizations, we show how the predictions of Bayesian inference can be used to guide more rational belief updating. We design a Bayesian inference-assisted uncertainty analogy that numerically relates uncertainty in observed data to the user's subjective uncertainty, and a posterior visualization that prescribes how a user should update their beliefs given their prior beliefs and the observed data. Read More In a pre-registered experiment on 4,800 people, we find that when a newly observed data sample is relatively small (N=158), both techniques reliably improve people's Bayesian updating on average compared to the current best practice of visualizing uncertainty in the observed data. For large data samples (N=5208), where people's updated beliefs tend to deviate more strongly from the prescriptions of a Bayesian model, we find evidence that the effectiveness of the two forms of Bayesian assistance may depend on people's proclivity toward trusting the source of the data. We discuss how our results provide insight into individual processes of belief updating and extit{subjective} uncertainty, and how understanding these aspects of interpretation paves the way for more sophisticated interactive visualizations for analysis and communication. Read Less
ISS 2019
Inking Your Insights: Investigating Digital Externalization Behaviors During Data Analysis
Yea-Seul Kim, Nathalie Henry Riche, Bongshin Lee, Matthew Brehmer, Michel Pahud, Ken Hinckley, Jessica Hullman
Externalizing one's thoughts can be helpful during data analysis, such as which one marks interesting data, notes hypotheses, and draws diagrams. In this paper, we present two exploratory studies conducted to investigate types and use of externalizations during the analysis process. We first studied how people take notes during different stages of data analysis using VoyagerNote, a visualization recommendation system augmented to support text annotations, and coupled with participants' favorite external note-taking tools (e.g., word processor, pen & paper). Externalizations manifested mostly as notes written on paper or in a word processor, with annotations atop views used almost exclusively in the initial phase of analysis. Read More In the second study, we investigated two specific opportunities: (1) integrating digital pen input to facilitate the use of free-form externalizations and (2) providing a more explicit linking between visualizations and externalizations. We conducted the study with VoyagerInk, a visualization system that enabled free-form externalization with a digital pen as well as touch interactions to link externalizations to data. Participants created more graphical externalizations with VoyagerInk and revisited over half of their externalizations via the linking mechanism. Reflecting on the findings from these two studies, we discuss implications for the design of data analysis tools. Read Less
CHI 2019
A Bayesian Cognition Approach to Improve Data Visualization
Yea-Seul Kim, Logan A Walls, P. M. Krafft, Jessica Hullman
Visualization evaluation guidelines rarely account for the influence of users' prior beliefs before encountering a visualization. We demonstrate a Bayesian cognitive model for understanding how people interpret visualizations in light of prior beliefs and show how this model provides a guide for improving visualization designs. Read More In a first study, we show how applying a Bayesian cognition model to a simple visualization scenario indicates that people's judgments are consistent with a hypothesis that they are doing approximate Bayesian inference. In a second study, we evaluate how sensitive our observations of Bayesian behavior are to different techniques for eliciting people subjective distributions, and to different datasets. We find that people don't behave consistently with Bayesian predictions for large sample size datasets, and this difference cannot be explained by elicitation technique. In a final study, we show how normative Bayesian inference can be used as an evaluation metric for visualizations of uncertainty. Read Less
CHI 2019
Vocal Shortcuts for Creative Experts
Yea-Seul Kim, Mira Dontcheva, Eytan Adar, Jessica Hullman
Vocal shortcuts, short spoken phrases to control interfaces, have the potential to reduce cognitive and physical costs of interactions. They may benefit expert users of creative applications (e.g., designers, illustrators) by helping them maintain creative focus. To aid the design of vocal shortcuts and gather use cases and design guidelines for speech interaction, we interviewed ten creative experts. Read More Based on our findings, we built VoiceCuts, a prototype implementation of vocal shortcuts in the context of an existing creative application. In contrast to other speech interfaces, VoiceCuts targets experts’ unique needs by handling short and partial commands and leverages document model and application context to disambiguate user utterances. We report on the viability and limitations of our approach based on feedback from creative experts. Read Less
CHI 2018
Improving Comprehension of Measurements Using Concrete Re-expression Strategies
Jessica Hullman, Yea-Seul Kim, Francis Nguyen, Lauren Speers, Maneesh Agrawala
It can be difficult to understand physical measurements (e.g., 28 lb, 600 gallons) that appear in news stories, data reports, and other documents. We develop tools that automatically re-express unfamiliar measurements using the measurements of familiar objects. Our work makes three contributions: (1) we identify effectiveness criteria for objects used in concrete measurement re-expressions; (2) we operationalize these criteria in a scalable method for mining a large dataset of concrete familiar objects with their physical dimensions from Amazon and Wikipedia; and (3) we develop automated concrete reexpression tools that implement three common re-expression strategies (adding familiar context, reunitization and proportional analogy) as energy minimization algorithms. Read More Crowdsourced evaluations of our tools indicate that people find news articles with re-expressions more helpful and re-expressions help them to better estimate new measurements. Read Less
Data Through Others' Eyes: The Impact of Visualizing Others' Expectations on Visualization Interpretation
Yea-Seul Kim, Katharina Reinecke, Jessica Hullman
In addition to visualizing input data, interactive visualizations have the potential to be social artifacts that reveal other people's perspectives on the data. However, how such social information embedded in a visualization impacts a viewer's interpretation of the data remains unknown. Inspired by recent interactive visualizations that display people's expectations of data against the data, we conducted a controlled experiment to evaluate the effect of showing social information in the form of other people's expectations on people's ability to recall the data, the degree to which they adjust their expectations to align with the data, and their trust in the accuracy of the data. Read More We found that social information that exhibits a high degree of consensus lead participants to recall the data more accurately relative to participants who were exposed to the data alone. Additionally, participants trusted the accuracy of the data less and were more likely to maintain their initial expectations when other people's expectations aligned with their own initial expectations but not with the data. We conclude by characterizing the design space for visualizing others' expectations alongside data. Read Less
Imagining Replications: Graphical Prediction & Discrete Visualizations Improve Recall & Estimation of Effect Uncertainty
Jessica Hullman, Matthew Kay, Yea-Seul Kim, Samana Shrestha
People often have erroneous intuitions about the results of uncertain processes, such as scientific experiments. Many uncertainty visualizations assume considerable statistical knowledge, but have been shown to prompt erroneous conclusions even when users possess this knowledge. Active learning approaches as well as discrete (frequency) formats for probability information have been shown to improve statistical reasoning, but are rarely applied in visualizing uncertainty in scientific reports. We present a controlled study to evaluate the impact of an alternative, interactive graphical prediction technique for communicating uncertainty in experiment results. Read More Using our technique, users sketch their prediction of the uncertainty in experimental effects prior to viewing the true sampling distribution from an experiment. We find that having a user graphically predict the possible effects from experiment replications is an effective way to improve one's ability to make predictions about replications of new experiments. Additionally, visualizing uncertainty as a set of discrete outcomes, as opposed to a continuous probability distribution, can improve recall of a sampling distribution from a single experiment. Our work has implications for various applications where it is important to elicit peoples' estimates of probability distributions and to communicate uncertainty effectively. Read Less
CHI 2017
Best Paper Award
Explaining the Gap: Visualizing One's Predictions Improves Recall and Comprehension of Data
Yea-Seul Kim, Katharina Reinecke, Jessica Hullman
Information visualizations use interactivity to enable userdriven querying of visualized data. However, users' interactions with their internal representations, including their expectations about data, are also critical for a visualization to support learning. We present multiple graphically-based techniques for eliciting and incorporating a user's prior knowledge about data into visualization interaction. Read More We use controlled experiments to evaluate how graphically eliciting forms of prior knowledge and presenting feedback on the gap between prior knowledge and the observed data impacts a user's ability to recall and understand the data. We find that participants who are prompted to reflect on their prior knowledge by predicting and self-explaining data outperform a control group in recall and comprehension. These effects persist when participants have moderate or little prior knowledge on the datasets. We discuss how the effects differ based on text versus visual presentations of data. We characterize the design space of graphical prediction and feedback techniques and describe design recommendations. Read Less
EMNLP 2016
SimpleScience: Lexical Simplification of Scientific Terminology
Yea-Seul Kim, Jessica Hullman, Matthew Burgess, Eytan Adar
Lexical simplification of scientific terms represents a unique challenge due to the lack of a standard parallel corpora and fast rate at which vocabulary shift along with research. We introduce SimpleScience, a lexical simplification approach for scientific terminology. Read More We use word embeddings to extract simplification rules from a parallel corpora containing scientific publications and Wikipedia. To evaluate our system we construct SimpleSciGold, a novel gold standard set for science-related simplifications. We find that our approach outperforms prior context-aware approaches at generating simplifications for scientific terms. Read Less
CHI 2016
Generating Personalized Spatial Analogies for Distances and Areas
Yea-Seul Kim, Jessica Hullman, Maneesh Agrawala
Distances and areas frequently appear in text articles. However, people struggle to understand these measurements when they cannot relate them to measurements of locations that they are personally familiar with. We contribute tools for generating personalized spatial analogies: re-expressions that contextualize spatial measurements in terms of locations with similar measurements that are more familiar to the user. Read More Our automated approach takes a user's location and generates a personalized spatial analogy for a target distance or area using landmarks. We present an interactive application that tags distances, areas, and locations in a text article and presents personalized spatial analogies using interactive maps. We find that users who view a personalized spatial analogy map generated by our system rate the helpfulness of the information for understanding a distance or area 1.9 points higher (on a 7 pt scale) than when they see the article with no spatial analogy and 0.7 points higher than when they see generic spatial analogy. Read Less
Users' Music Information Needs and Behaviors: Design Implications for Music Information Retrieval Systems
Jin Ha Lee, Hyerim Cho, Yea-Seul Kim
User studies in the music information retrieval (MIR) domain tend to be exploratory and qualitative in nature, involving a small number of users, which makes it difficult to derive broader implications for system design. In order to fill this gap, we conducted a large-scale user survey questioning various aspects of people's music information needs and behaviors. In particular, we investigate if general music users' needs and behaviors have significantly changed over time by comparing our current survey result with a similar survey conducted in 2004. Read More In this paper, we present the key findings from the survey data and discuss 4 emergent themes (a) the shift in access and use of personal music collections; (b) the growing need for tools to support collaborative music seeking, listening, and sharing; (c) the importance of visual music experiences; and (d) the need for ontologies for providing rich contextual information. We conclude by making specific recommendations for improving the design of MIR systems and services. Read Less
ISMIR 2016
Elucidating User Behavior in Music Services through Persona and Gender
John Fuller, Lauren Hubener, Yea-Seul Kim, Jin Ha Lee
Prior user studies in the music information retrieval field have identified different personas representing the needs, goals, and characteristics of specific user groups for a usercentered design of music services. However, these personas were derived from a qualitative study involving a small number of participants and their generalizability has not been tested. Read More The objectives of this study are to explore the applicability of seven user personas, developed in prior research, with a larger group of users and to identify the correlation between personas and the use of different types of music services. In total, 962 individuals were surveyed in order to understand their behaviors and preferences when interacting with music streaming services. Using a stratified sampling framework, key characteristics of each persona were extracted to classify users into specific persona groups. Responses were also analyzed in relation to gender, which yielded significant differences. Our findings support the development of more targeted approaches in music services rather than a universal service model. Read Less
ISMIR 2016
A Look at the Cloud from Both Sides Now: An Analysis of Cloud Music Service Usage
Jin Ha Lee, Yea-Seul Kim, Chris Hubbles
Despite the increasing popularity of cloud-based music services, few studies have examined how users select and utilize these services, how they manage and access their music collections in the cloud, and the issues or challenges they are facing within these services. Read More In this paper, we present findings from an online survey with 198 responses collected from users of commercial cloud music services, exploring their selection criteria, use patterns, perceived limitations, and future predictions. We also investigate differences in these aspects by age and gender. Our results elucidate previously under-studied changes in music consumption, music listening behaviors, and music technology adoption. The findings also provide insights into how to improve the future design of cloud-based music services, and have broader implications for any cloudbased services designed for managing and accessing personal media collections. Read Less
Computation + Journalism 2015
DeScipher: A Text Simplification Tool for Science Journalism
Yea-Seul Kim, Jessica Hullman, Eytan Adar
Complex jargon often makes scientific work less accessible to the general public. By employing a set of specific reporting strategies, journalists bridge these groups by delivering information about scientific advances in a readable, engaging way. One such strategy is using simpler terms in place of complex jargon. To assist in this process, we introduce DeScipher, a text editor application that suggests and ranks possible simplifications of complex terminology to a journalist while she is authoring an article. Read More DeScipher applies simplification rules derived from a large collection of scientific abstracts and associated author summaries, and accounts for textual context in making suggestions to the journalist. In evaluating our system, we show that DeScipher is a viable application for producing useful simplifications of scientific and other terms by comparing to prior techniques used on other corpora. We also propose concrete opportunities for future development of 'journalist-in-the-loop' tools for aiding journalists in enacting science reporting strategies. Read Less
CHI 2015
VIZMO Game Browser: Accessing Video Games by Visual Style and Mood
Jin-Ha Lee, Sungsoo Hong, Hyerim Cho, Yea-Seul Kim
Despite the growing interests in video games as consumer products as well as objects of research, current methods for accessing video games are limited. We present Vizmo as a new way of browsing video games based on their visual style and mood. Read More In order to test the usability and usefulness of Vizmo, we asked 19 video game experts to evaluate their interaction with the tool. The results show that experts perceived Vizmo as a novel and aesthetically pleasing game discovery tool which would be most useful for game research on historical and aesthetic aspects. We discuss five key points for improving the design of Vizmo as well as our future plan for the next iteration of this prototype game browser. Read Less
CHI 2014
Traffigram: Distortion for Clarification via Isochronal Cartography
Sungsoo Hong, Yea-Seul Kim, Jong-Chul Yoon, Cecilia R. Aragon
Most geographic maps visually represent physical distance; however, travel time can in some cases be more important than distance because it directly indicates availability. The technique of creating maps from temporal data is known as isochronal cartography, and is a form of distortion for clarification. Read More In an isochronal map, congestion expands areas, while ideal travel conditions make the map shrink in comparison to the actual distance scale of a traditional map. Although there have been many applications of this technique, detailed user studies of its efficacy remain scarce, and there are conflicting views on its practical value. To attempt to settle this issue, we utilized a usercentered design process to determine which features of isochronal cartography might be most usable in practice. We developed an interactive cartographic visualization system, Traffigram, that features a novel combination of efficient isochronal map algorithms and an interface designed to give map users a quick and seamless experience while preserving geospatial integrity and aesthetics. We validated our design choices with multiple usability studies. We present our results and discuss implications for design. Read Less