Only Holistic and Iterative Change Will Fix Digital Technology Research

The study of digital technologies and how they influence adolescents is flourishing, informed by a steady stream of empirical studies, evidence reviews and meta-analyses (Dickson et al., 2018; Orben, 2020). Yet the scope of these studies is limited. Most of the investigations pursued in social, psychological and health sciences subscribe to a so-called “concern-centric” approach. In other words, they frame their research questions in terms of whether and if so, how, various digital technologies might negatively impact their users. To this end, most adopt an implicit dose-response perspective: they treat technology use as the dose, for which response outcomes need to be quantified (e.g., decreases in mental health or increases in obesity). Thus, an exact “dose” of technology use must be measured. Addressing this challenge, the broad majority of research uses as its dose the time spent interacting with a technology or multiple technologies, i.e., so-called “screen time.” By using screen time as their explanatory anchor, current research therefore reduces the diverse, dynamic, nuanced and pervasive role digital technologies play in contemporary life to a series of oversimplified time estimates.

Studying engagement through this lens is problematic. A growing body of work suggests that the concern-centric modus operandi of technology effects research is not fit for purpose, and will not help researchers understand digital technology effects nor allow them to craft effective public health policy (Granic, Morita, & Scholten, this issue; Odgers & Jensen, 2020; Orben, 2020). It is in this light that we welcome the contribution by Granic and colleagues, who thoughtfully review the shortcomings of this current research approach and highlight the value of theoretical frameworks to better investigate the function of technology use (Granic et al., this issue). With that understood, we believe that only a holistic approach, combining both theoretical and methodological development, will lead to the long-lasting, sustainable and positive change Granic et al. (this issue) are calling for. Such an approach needs to be anchored in iterative development: probing, testing, improving and replacing past theoretical and methodological approaches to successfully advance both in parallel.

Theoretical models are powerful forces for progressing research but they are not, by themselves, sufficient. Their enduring predictive and proscriptive viability is necessarily limited by the quality of the empirical tools at hand to test and improve their explanatory power. Current empirical tools and methodologies are, however, outdated and ineffective. Indeed, a limited range of methodologies might be part of the reason researchers have retained the concern-centric approach even when it has been widely critiqued by media scholars and policymakers (Blum-Ross & Livingstone, 2018; Hiniker, Radesky, Livingstone, & Blum-Ross, 2019; Kaye, Orben, Ellis, Hunter, & Houghton, 2020). While we agree that theory of function is more informative than the predominant concern-focused approach, adopting new theory without simultaneously improving methodology will leave the field no better off in a decade’s time. If we are to gain meaningful and reliable insights into the roles that digital technologies play in the lives of adolescents, we need to move toward replicable and methodologically robust studies that inform, and are informed by, empirically-based theoretical frameworks.

Such concurrent development of theory and methods presents a formidable challenge. To allow for this level of holistic change to progress and to encourage the best possible research, it is crucial that researchers embrace iterative approaches to modify and improve their research field. New theoretical approaches should thus be based on, or at minimum address, existing theoretical approaches that explain the functions of technology use by adolescents, among other populations. Such iteration will allow programs of research to build on, and inform, one another, resulting in a robust and self-correcting body of evidence rather than fragmented studies selling novelty at the cost of replicability. We therefore also use this commentary to highlight what we believe is a lack of iteration in the theoretical improvements suggested by Granic and colleagues (this issue), as they partially disregard theoretical approaches that have been previously developed and implemented successfully. If our goal is to progress academic understanding of new technologies, we believe a holistic approach focused on careful iteration will prove most fruitful.

In this commentary we therefore argue that the field of adolescent technology research must pivot away from approaches focused on reactively patching the obviously failing parts of a collapsing system focused on concern-based approaches. Instead, scholars investigating teenage technology use must engage in, and advocate for, holistic change by recognizing that different parts of the research endeavor—ranging from theory to methodology—are intricately connected and can only be advanced in parallel. Thoughtful ventures to improve both theory and methods, and the combination of the two, need to take into account past work to successfully locate the points of “breakdown” and addresses these in a systematic way. Such a recalibration toward parallel theoretical and methodological development guided by iterative change will allow for the development of a digital research ecosystem fit for the twenty-first century. To this end, we first examine the concern-centric approach and highlight why we agree with Granic et al. (this issue) that it needs to be replaced. We then detail the theoretical and methodological developments that need to occur in parallel to implement such change successfully. In doing so, our commentary closes by detailing our reasons for thinking that Granic et al. (this issue) need to embrace more iteration on previous theory.

The outdated concern-centric approach

To understand the shortcomings of the concern-centric approach, and the need for combined and holistic theoretical and methodological development to overcome its strong hold on the research area, it is important to remind ourselves of the broader and rich historical context surrounding the study of media and technology effects (Grimes, Anderson, & Bergen, 2008; Neuman, 1988). In the previous century, technologies like movies, radio and television caused concern in the population and were investigated by researchers using the same approach as is used currently (Orben, in press). Yet the use of this concern-centric approach then was not as problematic as it is now, mainly because past technologies presented very different challenges and affordances to the digital innovations that researchers are studying today (Wartella & Robb, 2008). Notably, in comparison to more uniform technologies of the past, the unique diversity of modern digital technologies makes the use of time spent on them as a measurable “dose” increasingly untenable.

Nearly a century ago, research considering the effects of films and radio on children and adolescents needed to understand the impact of a limited engagement with a predominately uniform and stable medium. First, children and young people still engaged at a relatively small-scale with these new technologies, for example around 10 h a week in 1934 (Wartella & Robb, 2008). Second, these technologies had limited diversity and were showing relatively uniform content. Consumers thus had little choice about what they did with the technology and even the differences in content consumed were comparatively small. Measuring the effects of such a technology using a dosage of “time spent” as an anchor and calculating treatment effects, could therefore still be relatively effective: the “dose” of time spent using a technology like the radio will not show extreme variation between individuals and different time points. Furthermore, the empirical tools available were still adequate to support such research. To examine the potential effects of radio, for example, researchers shadowed children or asked them to estimate the amount of time they spend interacting with this technology while also tracking their outcomes of interest (Preston, 1941). This provided them with the necessary information to operationalize a blunt, but workable, concern-centric study.

In the decades that followed, use of more sophisticated digital technologies has steadily progressed: devices can now be upgraded after they are purchased, augmented with new apps, and new features can be launched and distributed instantly via the Internet. As noted in Granic et al. (this issue), as the web 2.0 matured, society became increasingly digitalized and media use became increasingly—explosively—diverse. While in the 1930s, 20 min listening to the radio could mean 20 min listening to music, news or crime drama (Preston, 1941), now 20 min on a smartphone could mean 20 min spent video chatting with grandparents, reading a book, looking at self-harm images, or solving sudokus. Granic et al. (this issue) provide a compelling overview of how this is complicated even further. For example, much of online media are now encouraging co-creation and production, allowing children and young people to interact with media and even produce it themselves. Modern video games and social media foster and allow for a sense of agency, moving beyond sole consumption of media and breaking long-held boundaries (Przybylski, Rigby, & Ryan, 2010). Lastly, engagement is now multi-dimensional, taking place over multiple devices often at the same time (Ofcom, 2019), while the engagement has also spread to ever more life domains, meaning it is becoming more difficult to dissociate from other, offline, activities (Jurgenson, 2011).

All this leads to the concern-centric approach, looking at discrete dose-response relationships, to now be destined for failure. As highlighted by Granic et al. (this issue), the use of modern digital technologies is so diverse that each “dose” of use might represent entirely different reactions and interactions that need to be accounted for on an individual level. With no unfirom “dose” to relate to the outcomes of interest, concern-centric theoretical approaches have ultimately broken down. What started as an adequate approach to a research question, with adequate methodology to support it, almost a century ago has developed into something that is not fit for purpose.

The need to develop theory and methods in parallel

Granic et al. (this issue) thus correctly identify that the concern-centric approach, and with it the focus on dose-response relationships, screen time and negative outcomes cannot hope to capture the rich, complex, and individually unfolding relationships that characterize modern technology use and its influence on teenage life. They aptly argue for the need to pivot to theory concerning the function, not just the quantity, of technology use. While this message resonates with us, we do not believe it goes far enough. Rather, we argue that newly applied theory is not sufficient to engender long-term change in research and ensure an accurate understanding of technology effects; changing theory on its own will risk proposing new ideas that cannot be adequately implemented, probed and developed. Instead, the challenge will require simultaneously addressing underlying theory and the need for adequate empirical methods and tools.

Such a need arises because the shortcomings of theories and methodologies applied to media and technology are inseparable. If one tries to decouple theoretical and empirical tools, to locate which one should be invested in, one is confronted with their inherent symbiosis. While there are problems with entirely atheoretical approaches (Granic et al., this issue), there are also clear shortcomings in research tools and methodology that new theory cannot fix.

Our empirical tools have not developed much since the earlier media effects studies (Orben, in press), and are therefore far from able to accommodate the explosion in diversity, individuality and agency that we noted above. If we want to track the effects of a certain technology, we would need to rely on asking children how much they use certain forms of technology, something we know is ostensibly difficult to recall (Boase & Ling, 2013; Grondin, 2010; Schwarz & Oyserman, 2001). Indeed, it has been shown that self-reported estimates of digital technology engagement, probing time spent on digital screens in general or social media specifically, are only modestly associated with objectively recorded measures of use (Davidson, Shaw, & Ellis, 2020; Ellis, Davidson, Shaw, & Geyer, 2019; Scharkow, 2016). Furthermore, it would be even more difficult to get further details about nuanced use, motivation, co-creation and agency, as these have fewer concrete operationalisations that can steer a measurement and empirical strategy. In face of this difficulty, the only other research option would be to shadow a small population of children and take note of their detailed usage. However, this would not only limit sample size, but also curtail the practical scope of the research as many subtle but important concepts such as motivation and representativeness will be missed. Furthermore, such methodology could interfere with how participants use technologies during the study because of the demands of continuous shadowing.

With the unavailability of granular data reflecting the digital technologies teenagers use and how they use them, researchers’ empirical tools are forced to measure outdated conceptualisations of technology use. This lack of empirical tools and methodologies to address better research questions forces researchers to divert their research efforts to the lowest common denominator. This, in turn, exacerbates the wealth of disjointed theory in the area, which then drives low-quality research that is again affected by the lack of empirical tools. While Granic et al. (this issue) new theoretical approach might provide a much needed jolt to a stagnant area, we would note that the methodology available to implement and iterate on a promising approach is most likely to be absent.

For example, Granic et al. (this issue) outline potential studies that could be conducted in future guided by their new framework. It is noteworthy however that many of these depend on nuanced or retrospective technology data that is difficult or impossible to collect given existing empirical tools (Granic et al., this issue). While they highlight that researchers could use apps for detailed screen time tracking, for example EARS (Lind, Byrne, Wicks, Smidt, & Allen, 2018), these still predominately only log time spent using certain apps, on a single device and therefore cannot capture multi-devise and multi-dimensional usage. While there are now more intensive approaches available to harvest data from participants’ phones (e.g., the Screenome project, similar to the BlackBerry project launched a decade earlier, see Underwood, Rosen, More, Ehrenreich, & Gentsch, 2012), these come with deeply-rooted ethical issues that make such approaches highly problematic if applied to large-scale populations (Reeves, Robinson, & Ram, 2020). Without progress on improving empirical tools such as gaining access to the necessary industry data and the development of better ethical and data sharing frameworks, the necessary research to investigate the improved theory will not be able to be enacted.

The need for iterative theoretical development

While we thus believe that there needs to be holistic theoretical and methodological development, we strongly agree with Granic and colleagues (this issue) that research needs to be grounded in suitable theoretical frameworks that can inform and are informed by reliable data. Adapting and applying quantitative tools without a sensitive theory can be counterproductive, for no reason other than it undermines a researcher’s confidence they are applying an approach well-suited to guide meaningful inquiry (Dienes, 2008). The consequences of rushing ahead without theory are routinely apparent in studies of technology effects using large-scale social data (see review in Orben & Przybylski, 2019). That said, we are unconvinced by the argument that Granic et al. (this issue) makes in laying out their formula for theoretical advances. We do not believe it is consonant with the spirit of theoretical and methodological iteration that we have highlighted above. This is most pronounced if one reflects on preexisting and preceding approaches already applied to understand motivational and psychological dynamics of technology engagement outside the concern-focused literature.

Put briefly, the appeal for new theory advanced by Granic et al. (this issue) identifies two types of psychological constructs key to future studies of adolescent technology use, their functions, consequences for emotional well-being, and development of personal and social identity. The first set of constructs are a constellation of needs, things that adolescents require for well-being and are drawn to. We can take away from Granic et al. (this issue) that intrapersonal needs include agency, communion, coherence, and that a societal-level version of the communion need exists in the form of social integration. The second feature highlighted by Granic and colleagues are narratives, stories that adolescents tell themselves and others that facilitate their personal growth and search for need satisfaction: including contamination and redemptive narratives, master and alternative narratives. We agree both of these factors provide tantalizing alternatives to a concern-focused approach. We however take issue with the presentation of these constructs as novel, as they have a clear place with respect to a fairly developed technology literature. Presenting them in isolation risks undermining the iterative approach the research area desperately needs.

When approaching a broad and important domain of human experience, such as the human interaction with the digital world, the urge to create a new theoretical framework to describe and study such a broadly emergent phenomena is strong. Doing so, however, runs the risk of discarding valuable philosophical and empirical starting positions already present in previous work. For example, Granic and colleagues (this issue) highlight work by the founders of a macro-theoretical model called Self-Determination Theory (SDT: Ryan & Deci, 2000) when making the case for needs in digital environments. The commentary however does not recognize the contributions of the theory to understanding adolescents or technology effects. This omission is noteworthy as SDT is, on its own, well-suited as a lens for understanding and improving life in the digital age and has been applied to explain the function, costs, and benefits of technology use for the last 15 years. The theory is an empirically-grounded perspective that integrates motivations, individual differences, and human development and is compatible with the model proposed by Granic et al. (this issue). Further, like Granic and colleagues’ model, SDT discusses the needs of adolescents, developing identity, and implications for well-being, psychopathology, and behavior (Griffin, Adams, & Little, 2017; La Guardia & Ryan, 2002; Ryan, Deci, & Vansteenkiste, 2016; Weinstein, Przybylski, & Ryan, 2012).

As part of the SDT framework, the well-studied construct of “psychological needs” postulates three fundamental needs that underly psychological well-being and behavioral engagement: relatedness—feeling close and connected to others, autonomy—feeling volitional in one’s actions, self-congruent, and, free to self-express, and competence—feeling effective in one’s actions (Deci & Ryan, 2000). Decades of research in digital and analogue contexts provides a ready set of plausible and testable research questions concerning how the fulfillment of such psychological needs contributes to healthy development and emergence of psychopathology (Luyckx, Vansteenkiste, Goossens, & Duriez, 2009; Vansteenkiste & Ryan, 2013). Indeed the SDT framework has been applied to probe the effects of technologies such as digital gaming: research has found that the three basic psychological needs—relatedness, autonomy, and competence—independently explain substantial variance in the “pull” of videogames (i.e., why they are played and when are they experienced as fun) in laboratory studies and community samples of young adults (Ryan, Rigby, & Przybylski, 2006). Of the 2329 present citations of this research, many provide additional validation that the three psychological needs explain the motivation for and benefits of videogames (for example, when it is considered “fun” to play games). Considering the “dark side” of such games as described in Granic et al. (this issue), the autonomy and competence needs described by SDT, in particular, have provided a better account for the motivational pull of videogames than factors such as their violent content, which have been considered the driving pull and satisfaction present within these games (Przybylski, Ryan, & Rigby, 2009). Further, depletion of a psychological need, e.g., competence, has been shown in experimental work to provide a better account of why videogames produce aggression (Przybylski, Deci, Deci, Rigby, & Ryan, 2014).

Similarly, SDT has been applied to explain other digital activities also discussed by Granic et al. (this issue). Studies have shown that social media use satisfies the psychological needs of those who are otherwise socially anxious (Casale & Fioravanti, 2015), and that Internet Gaming Disorder is linked to a paucity of psychological need satisfaction in people’s lives (Scerri, Anderson, Stavropoulos, & Hu, 2019). Longitudinal models have demonstrated a causal relationship between the psychological needs of relatedness, competence, and autonomy in daily life and lower symptoms of Internet Gaming Disorder (Przybylski & Weinstein, 2019; Weinstein, Przybylski, & Murayama, 2017). Further, in a series of naturalistic and experimental studies, Sheldon, Abad, and Hinsch (2011) have shown that frustration of the relatedness need motivates Facebook use, and that Facebook use can satisfy the relatedness need, leading to further well-being. Alongside this work, many studies have explored the role of interpersonal relatedness in digital technology use (Gross, Juvonen, & Gable, 2002; Mesch & Talmud, 2006; Spies Shapiro & Margolin, 2014; Valkenburg & Peter, 2011). Research informed by SDT has thus moved beyond videogame play to explain other forms and functions of technology use.

There is a striking resemblance between aspects of identity theory described in Granic et al. (this issue) and the theorizing present within SDT. The communion needs and person-society integration proposed in Granic et al. (this issue) map closely onto relatedness need satisfaction in SDT: both fundamentally involve feeling connected to others. Thus the communion needs of Granic et al. (this issue) are arguably interchangeable with relatedness needs satisfaction in SDT (Deci & Ryan, 2000; Sheldon & Kasser, 1995). In fact, were Granic et al. (this issue) to begin from that recognition, they could iterate on SDT and ask the more elaborate research question of whether relatedness needs are satisfied equally through adolescents’ feeling connected to close others or whether their fitting into the community is additionally important. Rather than beginning a new exploration of communion needs, such an approach would iterate on the existing state of the art literature to test, develop and improve it.

Similarly, the definition of agency given in Granic et al. (this issue): the “need to assert oneself and make decisions based on personal interests and values” ( Granic et al., this issue; p. 200; Locke, 2015) and to act “more in accordance with [their] own interests and values” (Erikson, 1959; Kroger, Martinussen, & Marcia, 2010) maps beautifully onto existing definitions and operationalizations of the autonomy need in SDT (Ryan & Deci, 2000). However, as Granic et al. (this issue) continue to describe occurrences of agency within technology use, they describe it in terms of a conceptually and operationally different psychological need within the SDT framework, that of competence: the “motivational "sweet spot" which most good video games target, where challenge and frustration are balanced with eventual success and accomplishment (Granic et al., this issue; p. 203; Sweetser & Wyeth, 2005).” Were we to explicitly recognize that agency involves having both autonomy and competence needs met, we would already benefit from an existing empirical base; researchers could attempt to replicate and expand on what is already known about autonomy, competence, or the combination of these two needs, by using ever improving methods and analytic approaches.

Beyond psychological needs, we can further apply qualities of motivation defined within SDT to understand adolescents’ use of digital technologies. For example, motivational principles based on supporting the need for autonomy in adolescents have been shown to be effective in shaping adolescent technology use. Legate, Weinstein, and Przybylski (2019) identified that autonomy-supportive parenting styles were linked to adolescents’ engaging in less online bullying behaviors, and that pressuring and shaming parenting styles that undermined autonomy were linked to adolescents engaging in more peer bullying. In a similar vein, adolescents whose parents are supportive of autonomy when restricting their technology use are more likely to cooperate and intend to self-disclose their future technology use (Van Petegem, de Ferrerre, Soenens, van Rooij, & Van Looy, 2019; Weinstein & Przybylski, 2019). Further, alongside predicting adolescents’ behavior within the context of digital technology use, these same motivational principles have been applied to explain how adolescents are drawn to digital engagement in healthy and unhealthy ways. When adolescents and young adults have a more harmonious passion that comes from an autonomous self-regulation and autonomy-supportive environments, their technology engagement is better balanced with other aspects of their lives and feels empowering and positive (Przybylski, Weinstein, Ryan, & Rigby, 2009; Wang, Liu, Chye, & Chatzisarantis, 2011). This technology literature also further fits nicely within a broader theoretical literature that describes the role of psychological needs and motivation in adolescents’ development, behavior, and well-being outside the context of technology use (e.g., Ryan et al., 2016; Van Petegem, Soenens, Vansteenkiste, & Beyers, 2015; Vansteenkiste, Simons, Lens, Soenens, & Matos, 2005).

It is therefore clear that SDT provides a valuable starting point for iteration, critique and discussion. Indeed, if we evaluate the arguments of Granic et al. (this issue) starting from this existing literature, there are multiple research questions that arise from their novel contributions that could provide a valuable starting point for iteration. For example:

1. Does social integration and communal need activities satisfy relatedness the same way in adolescents than adults; do the two types of relatedness activities show similar or different themes?

2. Does agency in gaming compromise autonomy, competence, or both? One could test whether adolescents’ need for agency is driven by competence/mastery pursuits, or autonomy, self-expression pursuits.

3. How do narratives reflect psychological need satisfactions? Do redemptive narratives, etc. map onto these?

We therefore argue that by acknowledging the body of work based around SDT, Granic et al. (this issue) could build on a more unified body of work to answer concrete and directed questions that take into account both identity theory’s and SDT’s expectations for the digital context. The new theoretical approach set out by Granic et al. (this issue) could therefore be informed by and directly related to existing theory by testing related constructs systematically and carefully defining each new idea that expands the current understanding. Such an approach based in iteration would allow the new work to speak to the existing substantial literature, minimizing the waste of previous scientific efforts and ensuring a harmonized body of scientific work. Furthermore, it will allow for incremental and successful improvement, as the results will be more interpretable with respect to previous work that has already established baselines for quality and understanding. There will be more ways of vetting ideas to avoid duplication and overcomplication, as any addition to previous theory will need to offer explanation of its utility beyond the existing work. Lastly, iteration on theories like SDT will allow for a more rapid bridging of digital and offline worlds, as such theoretical approaches have already been used to describe both successfully. We therefore think a re-description of Granic et al. (this issue) valuable ideas in an iterative light—for example with SDT as a foundation—could provide a more meaningful and successful proposal for widespread change in the field of adolescent technology research.

Conclusion: The need for a system-level approach

The concern-centric approach to digital technology research, with its simplistic focus on dose-response relationships where screen time affects various outcomes of interest, has long been ready for retirement. Granic et al. (this issue) cements this perspective on the research literature, and suggests a new theoretical framework to guide the field into a generative period of digital technology research. However, we argue this renaissance will never occur if we do not take a holistic approach that in addition to iterating and building on coherent and extensible theory also modernizes the empirical ecosystem, delivering modern methodology and adequate tools to put the new theoretical approaches into practice.

Firstly, the paper by Granic et al. (this issue) needs to be the start of a much more concerted debate about the theoretical basis for technology research. The field needs to promote the production and testing of flexible theories that have high explanatory power and are receptive to being continuously updated and falsified. Adopting an iterative approach working from the basis of preexisting and successful theoretical approaches will prove valuable here. Only if iteration is adopted, will new literature speak to previous work, and will unnecessary overcomplication and hype be avoided.

A twenty-first century technology research framework will however need more than iteration on theory; it will need to create a holistic research ecosystem that is able to develop in the agile, rapid and nuanced way necessary to keep up with digital technological development. To do this it will need to bring together both theory and methodology as a combined whole. This can be achieved by promoting large multidisciplinary teams, with industry and policy support that can move fast when new technologies are introduced into society. Further, there needs to be an increased ability to iterate and learn from mistakes, both in order to support the aforementioned agility, but also to allow for the rapid creation of policies and interventions both on a micro- (design) and macro- (population) level. Lastly, there needs to be a more sustained interest in ethics. In particular, we need to consider ethics of participant data in the digital age to ensure that public-facing research can keep up with private research enterprises: allowing more bridges to be built between technology companies and research.

This change will be difficult to manage, however not having such a research ecosystem in place is already causing harm on a daily basis. Research is slow and seems years behind the technological development it is supposed to keep in check; public conversation is misled by research routinely failing to emphasize the diversity of technology use; policy is slow and does not have the necessary evidence to give much needed advice (Hawkes, 2019); and such a vacuum of evidence is opening the space for misleading evidence and moral entrepreneurs. An improved research ecosystem will therefore not only generate reliable insights that span disciplines, but will also allow psychological research to create a workflow that can inform society about newly emergent technologies with an agility and speed that can keep up with an accelerating industry. Thus focus on holistic and iterative change will truly allow technology research to graduate into the twenty-first century.