Veconlab Classroom Clicker Games: The Wisdom of Crowds and the Winner's Curse

Abstract

The authors present a classroom “clicker” exercise in which students are asked to guess the number of items in a clear container before bidding on a money prize worth a penny for each item. Even if the distribution of guesses is unbiased, the highest bidder is likely to have overestimated the true number and ended up with a loss. Because winning becomes an increasingly rare and informative event as the number of bidders increases, the exercise generates a dramatic “winner's curse” in large classes where clicker systems are commonly used. Initial guesses about the number of items tend to suffer from perception bias, and this partial failure of the “wisdom of the crowds” can make bids even less rational.

Keywords:

• classroom game
• classroom response system
• perception bias
• winner's curse
• wisdom of the crowds

Keywords:

• JEL codes A20
• G02

Acknowledgments

This work was partially supported by a grant from the National Science Foundation (DUE-0737472).

The authors thank Sean Sullivan for providing helpful suggestions and references on perception bias. They also thank Ed Burton (University of Virginia) and Moriah Bostian (Lewis & Clark College) for inviting us to conduct this exercise in their classes. The authors benefited from comments made by the editor and two anonymous referees.

Notes

1. The interested instructor can visit http://clickers.veconlab.com/ to browse the collection of Veconlab Clickers exercises and to read tutorials on how to conduct an exercise. Instructors and students can also register for free accounts on the site, which will allow both parties to review the results of exercises after class.

2. We observe that about 5–10 percent of clicker responses are spurious, containing problems like negative bids, outrageous guesses, or non-numeric responses. These errors can wreak havoc with the presentation of results if the data are not cleaned first.

3. Many wisdom-of-crowds-style questions involve a physical prop such as a jar of coins or jelly beans. While we personally prefer the marshmallow prop, this exercise does not hinge on the presence of a prop. One can easily think of prop-less questions that fit into this framework: “What year was Mozart born?” “What percentage of students have majored in economics at our university in the past decade?” “How many people does the United Nations employ?” The main considerations when choosing a question are scaling the correct answer into a reasonable prize value and ensuring that the question does not generate guesses that are substantially lower than the correct answer. And, the instructor may wish to discuss some psychological biases that are more closely related to that question than the ones we provide here.

4. Jelly beans are similar in size to mini-marshmallows but are much heavier. Both will go over famously with the students and the administrative staff in your department, particularly if they are eye-catching colors such as Easter-egg pastels. If you do allow anyone to eat them, make sure to keep your hands clean during construction of the prop. (This is probably good advice in any case, because there is always one heedless student who will grab a handful of candy on the way out the door.)

5. The video is titled, “Cake on the Wisdom of the Crowds” and is available on YouTube as of the time of this writing. The video was produced by a social media company (now defunct) whose product allowed people to share their financial portfolios with each other, enabling information aggregation in a manner similar to the wisdom of crowds.

6. As a further example, one of this article's authors (Holt) once asked a group of CEOs attending a workshop on behavioral economics to privately guess the number of jelly beans in a container, with the understanding that all who guessed within 50 beans of the correct number would earn $10. There were 550 beans in the container, and the guesses were 180, 350, 560, 275, 260, 200, 333, 250, 178, and 395, for an average of 297. The group then made guesses aloud in sequence. The guesses were 500, 280, 600, 290, 358, 458, 305, 827, 400, 800, 237, and 220, for an average of 440. Although making guesses public improved the group's accuracy, guessing sequentially could still induce a bias if there is a tendency to follow the leaders in the sequence, and the leaders are significantly off the mark (e.g., as in the “information cascade” experiments in Anderson and Holt [1997]).

7. Veconlab Clickers requires the instructor to specify upper and lower bounds for the bids and guesses. These bounds eliminate the most extreme responses that are the least believable, but the instructor should be mindful not to choose bounds that are suggestive of the correct answer. Figure 2(a) indicates that there is still quite a bit of variation in guesses and bids even with the bounds in place, indicating that they were not too informative.

8. Some high bids are indeed mistakes. Forgetting to include a decimal place in the bid is a common one, so that an intended 4.01 is reported as 401. (Entering a numeric answer on an i>clicker response pad is much like mobile-phone texting, and we are suspicious that the general lack of punctuation in texts is related to this issue.) Some students may bid a multiple of their guess rather than a fraction if they invert the conversion rate.

9. Most spreadsheet programs have slope and intercept formulas for simple regression, which would allow the instructor to add regression results during discussion.

10. The null hypothesis on β represents the case in which participants bid exactly their estimates of the prize value, and the instructor must remember to apply the prize conversion factor. In our case, the factor is $0.01 per marshmallow, and so β = 0.01 corresponds to bidding the perceived value of the prize.

11. As an example of correlation between the guess and the deviation, consider a group of students who talk to each other about their guesses between the two stages of the experiment. (This can occur quite easily in large classes.) Those with the most extreme guesses will probably realize that they are in the upper tail of the class distribution of perceived values and adjust their bids substantially downward during the bidding stage to avoid the winner's curse. This adjustment, of course, is unobservable to those running the regression.

12. There is some experimental evidence suggesting that bid functions are empirically concave when they should be linear. Dorsey and Razzolini (2003) examine a pay-as-bid, private-value auction with a uniform distribution of values. The theoretical bid function in their design is linear, but they observe concave bid functions instead.