Agent-based Testing: An Automated Approach toward Artificial Reactions to Human Behavior

ABSTRACT

Vast amounts of news are consumed through algorithmically curated media environments, such as search engines, social networking sites, or news aggregators. This renders algorithmic content curation with much societal relevance and highlights the urgent need for independent and resilient academic research. Therefore, a plethora of methodological approaches have been applied, such as case studies, expert interviews, observations, or agent-based approaches. The paper discusses the applicability of these methodological efforts for journalism studies, showing that all of these approaches face their limitations, especially with regard to external validity, recruitment difficulties, and data reliability. Thereby, agent-based testing represents one of the most promising approaches to overcome plenty of these methodological limitations. Agent-based testing is a systematic and experimental approach that emulates online human behavior to test algorithmically curated media environments under various conditions. For this to be achieved properly, this paper suggests a multitude of settings and requirements to adequately face the technological, legal, and ethical challenges, which come with the empirical investigation of algorithmic content curation. Ultimately, the paper presents both general considerations and practical instructions (using the “ScrapeBot”) to employ agent-based testing for journalism studies.

KEYWORDS:

• Algorithmic content curation
• personalization
• quantitative methods
• qualitative methods
• computational journalism
• data collection
• computational social science

Disclosure Statement

No potential conflict of interest was reported by the author.

Notes

1 The Hypertext Transfer Protocol (HTTP) specifies how linked and structured “hypertext” should be exchanged. In order to establish an encrypted communication, the protocol uses the cryptographic standard TLS (Transport Layer Security), the combination of which is often referred to as HTTPS.

2 TCP (Transmission Control Protocol) specifies the reliable exchange of a stream of data packages via IP networks. In contrast to UDP (User Datagram Protocol), a non-reliable protocol of data exchange where packages are transmitted without verification whether they actually reached their target, each TCP connection requires a bidirectional confirmed establishment through a so-called three-way handshake.

3 The Internet Protocol (IP) regulates the delivery of individual data packages by using IP addresses.

4 The Domain Name System (DNS) represents a hierarchical directory of combinations between domain names (e.g., google.com) and IP addresses (e.g., 172.217.23.142) for easier communication.

5 Installation of the “ScrapeBot” requires to run various Python scripts. Detailed instructions are provided on the project’s GitHub repository at https://github.com/MarHai/ScrapeBot.