Uncovering the Sources of Machine-Learning Mistakes in Advertising: Contextual Bias in the Evaluation of Semantic Relatedness

Abstract

Imagine a beer advertisement next to an article about drunk driving, or a coupon for a free dinner embedded in an article about food poisoning. While humans are quite good at seeing the error in these examples, the machine-learning algorithms that place advertisements online continue to struggle with this type of contextual nuance. We argue that this shortcoming stems from the manner in which these machines are taught about semantic relatedness—the conceptual distance between words in the human mind. Specifically, we hypothesize that there is a difference in how humans view semantic relatedness when context is present versus when it is absent and that this difference is missing from the data used by machines to place advertisements online. To test this hypothesis, we adapt existing best practices to create a new, context-aware database and then compare it to the current state of the art. We find substantial differences in the distribution of semantic relatedness scores for context-aware versus context-free databases. We also find that the nature and scope of these differences are likely to lead to the types of mistakes observed in practice.

Additional information

Notes on contributors

Jameson Watts

Jameson Watts (PhD, University of Arizona) is an assistant professor of marketing, Atkinson Graduate School of Management, Willamette University, Salem, Oregon, USA.

Anastasia Adriano

Anastasia Adriano (MBA, Willamette University) is a graduate research assistant, Atkinson Graduate School of Management, Willamette University, Salem, Oregon, USA.