Models and algorithms for crowdsourcing discovery

Abstract

We start by looking at the CONE Welder project which uses a robotic camera in a remote location to study the effect of climate change on the migration of birds. In CONE, an amateur birdwatcher can operate a robotic camera at a remote location from within her web browser. She can take photos of different bird species and classify different birds using the user interface in CONE. This allowed us to compare the species presented in the area from 2008 to 2011 with the species presented in the area that are reported by Blacklock in 1984. Citizen scientists found eight avian species previously unknown to have breeding populations within the region. CONE is an example of using crowdsourcing for discovering new migration patterns. Crowdsourcing can also be used to collect data on human motor movement. Fitts' law is a classical model to predict the average movement time for a human motor motion. It has been traditionally used in the field of human-computer interaction (HCI) as a model that explains the movement time from an origin to a target by a pointing device and it is a logarithmic function of the width of the target (W) and the distance of the pointer to the target (A). In the next project we first present the square-root variant of the Fitts' law similar to Meyer et al. To evaluate this model we performed two sets of studies, one uncontrolled and crowdsourced study and one in-lab controlled study with 46 participants. We show that the data collected from the crowdsourced experiment accurately follows the results from the in-lab experiments. For Homogeneous Targets the Square-Root model (T = a + b AW ) results in a smaller ERMS error than the two other control models, LOG (T = a + b log 2AW ) and LOG' (T = a + b log AW + 1) for A/W 15. In the Heterogeneous Targets the LOG' model consistently resulted in a significantly smaller error for 0 < A/W ≤ 24. These sets of experiments showed that the crowdsourced and uncontrolled experiment was consistent with the controlled in-lab experiment. To the best of our knowledge this is the largest experiment for evaluating a Fitts' law model. The project demonstrates that in-the-wild experiments, when constructed properly, can be used to validate scientific findings. Opinion Space is a system that directly elicits opinions from participants for idea generation. It uses both numerical and textual data and we look at methods to combine these two sets of data. Canonical Correlation Analysis, CCA, is used as a method to combine both the textual and numerical inputs from participants. CCA seeks to find linear transformation matrices that maximize the lower dimension correlation between the projection of numerical rating (Xwx) and textual comments onto the two dimensional space (Ywy). In other words it seeks to solve the following problem argmaxwx,wy corr(Xwx, Yw y) in which X and Y are representations of the numerical rating and textual comments of participants in high dimensions and Xwx and Ywy are their lower dimension representations. By using participants' numerical feedbacks on each others' comments, we then develop an evaluation framework to compare different dimensionality reduction methods. In this dissertation we provide supporting argument as to why this evaluation framework is appropriate for Opinion Space. We have compared different variations of CCA and PCA dimensionality reductions on different datasets. Our results suggests that the γ values for CCA are at least %169 larger than the γ values of PCA, making CCA a more appropriate dimensionality reduction model for Opinion Space. (Abstract shortened by UMI.)