Data Quality Challenges in Community Systems
AnHai Doan (University of Wisconsin)

Over the past three years, in Cimple, a joint effort between Wisconsin
and Yahoo! Research, we have been trying to build community
systems. Such systems employ automatic data management techniques,
such as information extraction and integration, as well as
user-centric Web 2.0-style technologies, to build structured data
portals for online communities. As the work progresses, we have
encountered a broad range of fascinating data cleaning
challenges. Some of these (e.g., data quality evaluation, record
reconciliation) also arise in traditional ETL processes. But here they
become exacerbated, take on new nuances, or are amenable to novel
solutions that exploit community characteristics. Many other
challenges however are new, and arise due to the fact that community
systems engage a multitude of users of varying skills and
knowledge. Examples include how to entice users to collaboratively
clean data, how to handle "noisy" users, and how to make certain
cleaning tasks easy for "the masses". We describe the challenges and
our initial solutions. We also describe the infrastructure support
(code, data, etc.) that we can provide, in the hope that other
researchers will join and help us address these problems.