Analyzing Behavioral Big Data: Methodological, practical, ethical, and moral issues

ABSTRACT

The term “big data” evokes emotions ranging from excitement to exasperation in the statistics community. Looking beyond these emotions reveals several important changes that affect us as statisticians and as humans. I focus on Behavioral Big Data (BBD), or very large and rich multidimensional datasets on human behaviors, actions, and interactions, which have become available to companies, governments, and researchers. This article describes the BBD landscape and examines opportunities and critical issues that arise when applying statistical and data mining approaches to Behavioral Big Data, including the move from macro- to micro-decisioning and its implications.

KEYWORDS:

• big data
• data mining
• human behavior
• social interactions
• statistical methods

Funding

This work was supported in part by grant 104-2410-H-007-001-MY2 from the Ministry of Science and Technology in Taiwan.

Notes

1 iChef and NTHU’s Center for Service Innovation & Analytics have recently partnered on a large-scale research project called “small restaurant, big data.”

2 In the acknowledgments, the authors thank Sulake Corporation for the provision of data.

3 Despite the authors claim “We find that teacher VA has substantial impacts on a broad range of outcomes,” the reported magnitudes are practically insignificant and in my opinion far from substantial.

4 The breadth of applications is clearly limited by my own work environment, and therefore might exclude other areas where BBD are used for answering social science type questions.

5 This study was not described earlier, for brevity.

6 www.adweek.com/news/technology/connect-thoughts-99712

7 “Netflix Settles Privacy Lawsuit, Cancels Prize Sequel”, Forbes.com, March 12, 2010.

8 “Streaming has not only changed the way our members interact with the service, but also the type of data available to use in our algorithms.” Netflix blog, April 6, 2012, http://techblog.netflix.com/2012/04/netflix-recommendations-beyond-5-stars.html

9 www.dailymail.co.uk/news/article-2102859/, from Hardoon and Shmueli (2013).

10 “Amazon backs away from test prices”, www.cnet.com, January 2, 2002.

11 “Amazon’s business strategy and revenue model: A history and 2014 update”, www.smartinsights.com, June 30, 2014.

12 “Code of Federal Regulations”, www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html

13 From “Amazon’s business strategy and revenue model: A history and 2014 update”, www.smartinsights.com, June 30, 2014.

14 http://wiki.wearedynamo.org/index.php/Guidelines_for_Academic_Requesters

15 www.oecd.org/site/piaac/surveyofadultskills.htm