Data Collection in Online Terrorism and Extremism Research: Strengths, Limitations, and Future Directions

Figures & data

Figure 1. Overview of TDC.

Table 1. Strengths and limitations of data collection methods in online terrorism and extremism research.

Technique	Approach	Strengths	Limitations
Computational	Third party APIs	Time-efficient, large-scale data collection Relative ease of data collection Standard, simplified data collection Minimal technical skills required for data collection, formatting, or archiving Minimal need to clean, transform, or prepare collected data	Can be costly Restrictions on data collection Black box problem
	Commercial crawlers	Time-efficient, large-scale data collection Relative ease of data collection Technical skills not required for data collection, formatting, or archiving Minimal need to clean, transform, or prepare collected data Flexible data filtering options via interactive interfaces	Can be costly Restrictions on data collection Black box problem
	Custom-written crawlers	Time-efficient, large-scale data collection Source code available for data collection Ability to collect and then merge disparate data sources to be analyzed as one Filtering tools to extract specific information of interest Customizable for various platforms	Technical skills required for data collection, formatting, or archiving Specific parameters required for data collection Rate-limiting techniques from websites can slow data collection Software, which can be expensive, is needed to format and archive collected data
Manual	Database development	Relevant and quality information is collected Data is focused on the specific problem Follow up on uncovered leads to track down specific materials or variables Targeted searches to fill in missing values Better positioned to capture latent constructs and improve validity	Labor-intensive and time-consuming data collection Coder training is critical to guard against reliability issues Amount and type of information available about different types of events vary considerably A universal protocol for data collection does not exist across databases Inclusion/exclusion criteria for databases are often not transparent Missing data