Events Data as Bismarck's Sausages? Intercoder Reliability, Coders' Selection, and Data Quality

REFERENCES

• Alvarez , Michael and John , Brehm . 1995 . American Ambivalence Towards Abortion Policy: Development of a Heteroskedastic Probit Model of Competing Values . American Journal of Political Science , 39 ( 4 ) : 1055 – 1082 .
   Web of Science ®Google Scholar
• Beardsley , Kyle and Holger , Schmidt . 2007 . UN Data Codebook , Unpublished manuscript, Emory University and George Washington University .
   Google Scholar
• Baugh , Frank . 2003 . “ Correcting Effect Sizes for Score Reliability ” . In Score Reliability: Contemporary Thinking on Reliability Issues , Edited by: Thompson , Bruce . Thousand Oaks, CA : Sage .
   Google Scholar
• Bollen , Kenneth . 1989 . Structural Equations with Latent Variables , New York : Wiley .
   Google Scholar
• Carmines , Edward and Richard , Zeller . 1979 . Reliability and Validity Assessment , Beverly Hills : Sage .
   Google Scholar
• Cederman , Lars-Erik , Luc , Girardin and Andreas , Wimmer . Getting Ethnicity Right: An Expert Survey on Power Distributions among Ethnic Groups . Paper presented at the 102th Annual meeting of the American Political Science Association . 31 August–3 September , Philadelphia, PA.
   Google Scholar
• Dorussen , Han . Introducing PKOLED: Peacekeeping Operations Locations and Event Dataset . Paper presented at the Annual GROW-Net Conference on Disaggregating the Study of Civil War and Transnational Violence . November 24–25 . University of Essex .
   Google Scholar
• Dorussen , Han , Spyros , Blavoukos and Hartmut , Lenz . 2005 . Assessing the Reliability and Validity of Expert Interviews . European Union Politics , 6 ( 3 ) : 315 – 338 .
   Web of Science ®Google Scholar
• Dorussen , Han and Theodora-Ismene , Gizelis . Into the Lion's Den. The Local Reception of UN Peacekeeping . Paper presented at the 49th Annual ISA Convention . San Francisco, CA.
   Google Scholar
• Franzosi , Roberto . 1994 . From Words to Numbers . Sociological Methodology , 24 : 105 – 136 .
   Web of Science ®Google Scholar
• Gerner , Deborah J. , Schrodt , Philip A. , Omur , Yilmaz and Rajaa , Abu-Jabr . The Creation of CAMEO (Conflict and Mediation Event Observations): An Event Data Framework for a Post Cold War World . Presented at the annual meeting of the American Political Science Association . 29 August–1 September , San Francisco.
   Google Scholar
• Gleditsch , Kristian Skrede and Michael , Ward . 1997 . Double Take: A Reexamination of Democracy and Autocracy in Modern Polities . Journal of Conflict Resolution , 41 ( 3 ) : 361 – 383 .
   Web of Science ®Google Scholar
• Hensel , Paul . Reliability and Validity Issues in the ICOW Project . Paper prepared for presentation at the 39th Annual Meeting of the International Studies Association . 17–21 March , Minneapolis.
   Google Scholar
• Hruschka , Daniel , Deborah , Schwartz , Daphne Cobb , St. John , Erin , Picone-Decaro , Jenkins , Richard A. and Carey , James W. 2004 . Reliability in Coding Open-Ended Data . Field Methods , 16 ( 3 ) : 307 – 331 .
   Google Scholar
• Jaggers , Keith and Gurr , Ted R. 1995 . Tracking Democracy's Third Wave with the Polity III Data . Journal of Peace Research , 32 ( 4 ) : 469 – 482 .
   Web of Science ®Google Scholar
• Kalyvas , Stathis . 2006 . The Logic of Violence in Civil War , New York : Cambridge University Press .
   Google Scholar
• King , Gary and Will , Lowe . 2003 . An Automated Information Extraction Tool for International Conflict Data with Performance as Good as Human Coders . International Organization , 57 ( 3 ) : 617 – 642 .
   Web of Science ®Google Scholar
• Landis , Richard and Gary , Koch . 1977 . The Measurement of Observer Agreement for Categorical Data . Biometrics , 33 ( 1 ) : 159 – 174 .
   PubMed Web of Science ®Google Scholar
• Leng , Russell J. and David Singer , J. 1988 . Militarized Interstate Crises. The BCOW Typology and its Applications . International Studies Quarterly , 32 ( 2 ) : 155 – 173 .
   Web of Science ®Google Scholar
• Liebetrau , Albert M. 1983 . Measures of Association , Thousand Oaks, CA : Sage .
   Google Scholar
• Mitchell , Ronald and Steven , Rothman . Creating Large-N datasets from Quality Information . Paper presented at the 102th Annual meeting of the American Political Science Association . 31 August – 3 September , Philadelphia, PA.
   Google Scholar
• Neuendorf , Kimberly A. 2002 . The Content Analysis Guidebook , Thousand Oaks : Sage Publications .
   Google Scholar
• Olzak , Susan . 1992 . The Dynamics of Ethnic Competition and Conflict , Stanford : Stanford University Press .
   Google Scholar
• Raleigh , Clionadh and Havard , Hegre . Introducing ACLED: An Armed Conflict Location and Event Database . Paper presented to the conference, “Disaggregating the Study of Civil War and Transnational Violence,” University of California Institute of Global Conflict and Cooperation . March 7–8 , San Diego, CA.
   Google Scholar
• Ratner , Steven R. 1996 . The New UN Peacekeeping. Building Peace in Lands of Conflict after the Cold War , New York : St Martin's Press .
   Google Scholar
• Rothman , Steven . 2007 . Understanding Data Quality through Reliability . International Studies Review , 9 ( 3 ) : 437 – 456 .
   Google Scholar
• Schrodt , Philip A. 2006 . Twenty Years of the Kansas Event Data System Project . The Political Methodologist , 14 ( 1 ) : 2 – 8 .
   Google Scholar
• Sim , Julius and Chris , Wright . 2005 . The Kappa Statistics in Reliability Studies . Physical Therapy , 85 ( 3 ) : 257 – 268 .
   PubMed Web of Science ®Google Scholar
• Singer , J. David . 1984 . Variables, Indicator, and Data . Social Science History , 6 ( 2 ) : 181 – 217 .
   Google Scholar
• Treier , Shawn and Simon , Jackman . 2008 . Democracy as a Latent Variable . American Journal of Political Science , 52 ( 1 ) : 201 – 217 .
   Web of Science ®Google Scholar