Taking on sacred cows: openness, fair critique, and retaining value when revising classics

Most experienced authors know that getting papers through the revision process on the way to publication is challenging (Mingers, 2002), but fewer may have confronted the challenges of trying to significantly revise their works post publication. When required changes are small, they can easily be accommodated in a new paper that extends the earlier article and pushes forward in new directions. When an author feels the need for more substantial changes, however, a direct confrontation of the earlier work would seem to be in order. We have found that such substantial changes can be even more challenging than making a new contribution, particularly when the original paper has become a community classic. Our experience: 12 years ago, we took on a simple, straightforward project: Updating a classic paper – Markus and Robey (1988) – and putting it on a sounder philosophical and theoretical footing. Today, the paper has been accepted (Markus & Rowe, 2018, hereafter M&R18), but the project was by no means simple and straightforward, and the paper looks nothing like its early incarnations. As a result, the direct confrontation is missing from M&R18 (and we present it here). On our longer-than-expected journey to M&R18, we’ve learned some lessons about cumulative knowledge in our field, and we’d like to start a dialogue about them. Since EJIS prides itself on providing “critical view[s] on technology, development, implementation, strategy, management and policy,” it is only logical to initiate such a dialogue here.

1. Our confessional tale

In 2006, Frantz came to Bentley for a sabbatical year. After numerous discussions about shared research interests he approached Lynne about a project he’d long envisioned. Since his doctoral student days, Frantz had made productive use of Lynne’s decades-old paper with Dan Robey, “Information Technology and Organizational Change: Causal Structure in Theory and Research” (Management Science, 1988, hereafter M&R88). At the time, the paper was still cited, used in doctoral student education, and appeared to have made a number of contributions to the IS field. For example, the paper has been substantively cited in the following works, among others: Orlikowski and Robey (1991), Kurnia and Johnston (2000), Besson & Rowe (2001), Gregor (2006), Hovorka, Germonprez, and Larsen (2008), Orlikowski and Scott (2008), Paré, Bourdeau, Marsan, Nach, and Shuraida (2008). But, as a journal editor, he had become frustrated with submissions that cited M&R88 incorrectly, in part because of errors in the text that Frantz had identified on several re-readings. Furthermore, the issues and examples in M&R88 were well out-of-date and did not support contemporary readers in applying its concepts.

At first, Lynne was reluctant. It struck her as a vanity project – something she was loathe to undertake. But Frantz eventually convinced her that she would learn a lot by exploring the philosophical and social theory literatures in search of justification and give her an opportunity to clarify her own current philosophical and theoretical commitments. The goal would be not just to update her and Dan’s old causal structure framework, but also to arrive at the best foundations for her own evolving research. Lynne also took heart from something Mary Douglas wrote in the preface to one of her later books: “This is the first book I should have written after writing on African fieldwork. Instead I wrote Purity and Danger (1966) … . My friends told me at the time that Purity and Danger was obscure, intuitive, and ill-prepared. They were right, and I have been trying ever since to understand the theoretical and logical anchoring that I would have needed to present a coherent argument …” (Douglas, 1986), p. ix.

And so, we started to read. And read. And soon, we encountered the problem of how to present what we were beginning to learn.

1.1. Our early approach

Given our starting point, it was obvious to us that the way to pitch our paper was to explain the problems in Markus and Robey’s framework. The problems, as we saw them, were several.

First, M&R88’s conceptions of causality were far from mutually exclusive and collectively exhaustive. They packed too many different issues into their first dimension (causal agency) and did not provide enough depth of treatment to their third (levels of analysis). Their second dimension (logical structure) was misleading on the topic of necessary and sufficient conditions and did not consider a broad enough range of theoretical possibilities.

Second, M&R88’s analysis of “causal agency” was ambiguous and confusing. One of their positions on that dimension was technological imperative, defined in terms of determinism. But they defined the similarly labelled organisational imperative in terms of rational actors free to choose the technology, features, and desired outcomes that best meet their needs. Thus stated, the organisational imperative is a voluntarist position, which Barley (1998) argued to be the opposite of determinism. Therefore, use of the term “imperative” in M&R88 was quite misleading.

Third, M&R88’s treatment of process and variance theory was incomplete at best, and incorrect at worst. Those authors had based their discussion on Mohr (1982), who had associated variance theory with “an invariant, necessary and sufficient relationship” (p. 584), and process theory with “a recipe of sufficient [only] conditions occurring over time”. In fact, most variance theories eschew causality and rely on probabilistic associations. Further, there are multiple kinds of process theory (Van de Ven & Poole, 2005), only one of which (historical explanation) makes use of sequences of sufficient conditions (George & Bennett, 2005); some ways of testing process theory assume invariant, necessary and sufficient relationships (George & Bennett, 2005); and some process theories involve complex combinations of necessary and/or sufficient conditions (George & Bennett, 2005).

Lastly, M&R88 gave only the sketchiest discussion of levels of analysis. Furthermore, their treatment of that idea appeared to have more to do with methods than with causality and theory.

On the basis of this analysis, we had the intuition that we could recast M&R88’s three causal structure dimensions in terms of different connotations of the term “determinism”: 1) deterministic invariance versus statistical indeterminism, 2) individualism or voluntarism versus social forces or institutional determination, and 3) human agency versus technological determinism. We started the paper with a thorough deconstruction of the M&R88 framework and followed that with a reconstruction of the framework, in which on each dimension formed a continuum of positions that were more or less deterministic.

1.2. Pushbacks and responses

The reactions of readers and reviewers to this approach were universally negative. The main reaction seemed to be outrage at our taking on a work that had provided value to readers in the past. We found it troubling to think that one’s previous best efforts might be frozen in stone. But on reflection we realised that the real issue was that we had not provided enough new value. Our “new improved” framework was not all that new and improved. It mainly restated old ideas in new language and disambiguated concepts at the margins. The best indicator of this shortcoming was our conceptualisation of the dimensions as continua, which did not account for radical innovations in social theory related to social constructionism and postmodern thinking. Our revised framework stayed mired in the paradigm that had spawned the original and did not add enough value to contemporary readers.

In the next iteration we tried to address this shortcoming by radically revising our trajectory dimension. It moved much closer to its current state. The positions were presented as distinctly different, such that they could not be reduced to, or stated in terms of, the others. All mentions of continuua were banished. But in responding to requests to justify our dimensions without relying on M&R88, we overcorrected in other ways. We greatly expanded the number of positions we presented, making it more difficult for readers to distinguish among them. We infelicitously (in retrospect) decided to construct a detailed historical evolution of philosophical and social theoretic thinking related to the framework. This not only taxed readers’ patience, it failed to show the necessary connections between the philosophical ideas and the dimensions and positions they were supposed to underpin.

Readers advised us to limit our efforts to a single dimension, arguing that we had little to contribute to scholars who had done the most to develop certain positions. But we resisted that suggestion as defeating our whole purpose of trying to expose and explicate a range of positions in IS causal reasoning that was comparable to that addressed by M&R88.

As we strove to simplify our framework, improve its justification, and streamline its exposition without cutting scope, we realised we had not yet broken free from one of the old framework’s anchors. As is clear from its title, M&R88 set out to address causal agency in both “theory and [empirical] research”. Their discussion of the causal structure and levels of analysis dimensions implied methodological choices as much as theoretical ones. For our framework, we had intended from the outset to confine ourselves to theory. The reason is our belief that theory should drive choice of methods, whereas it seems to us that in many cases choices of methods constrain the theoretical statements scholars make. It was clear to us now that, despite our intentions, one of our three dimensions was still infused with methodological concerns. So, we had to go back to the drawing board. An observation of one reviewer put us on a better track for our Ontology dimension. That, in turn, helped us find the justification we were looking for in (Lakoff & Johnson, 1999), a work we had consulted early in our project but long since set aside.

The final, and most challenging, hurdle we faced was trying to address readers’ requests for a theory instead of a framework. By now we were confident that, with considerable help from our friends, we had accomplished all but two of our original goals. We had started with the M&R88 framework, updated it to reflect accumulated IS research, and grounded the framework in the literatures of philosophy and social theory. However, we had not corrected its errors through a direct confrontation. Justification of the novel framework was built on different foundations and its novelty (constitutive causality on the first dimension, trajectory dimension as a whole, and the revival of technology autonomy) left no space for a formal comparison of M&R18 to M&R88. Accordingly, we present it here in Table 1.

Table 1. The Markus and Robey (1988) and Markus and Rowe (2018) Causal Structure Frameworks Compared.

Markus and Robey (1988) Framework	Markus and Rowe (2018) Framework	Comparative Comments
Logical structure: “the time span of the theory and … the hypothesized relationships between antecedents and outcomes” (p. 584) Variance theory: “causes are related to outcomes in an invariant, necessary and sufficient relationship …” (p. 584); “concerned with predicting levels of outcomes from levels of contemporaneous predictor variables” (p. 589) Process theory: causes are related to outcomes … in a recipe of sufficient conditions occurring over time” (p. 584); “chance and random events play a role” (p. 590)	Causal Ontology: A theorist’s views about the reality of causality Directional association: Causality refers to regular associations among certain hypothetical or observed conditions, possibly including temporal precedence Causal mechanism: Causality involves real physical, psychological and/or social processes that connect inputs and outputs under certain conditions Constitutive causality: “Causality” is a human belief about how meanings are enacted in highly situated social interactions (eg, practices) and about how people dynamically construct such interactions	Logical structure (1988) included some ideas that are incorporated in the causal trajectory and causal autonomy dimensions of the 2018 framework. The concepts of necessity and sufficiency in the 1988 positions have been deleted from the 2018 framework because of errors in the definitions of variance and process theories, which had been derived from Mohr (1982). Variance theory (1988) as commonly understood today is more consonant with the definition of directional association (2018) than it is with the definitions in the 1988 framework. Causal mechanism (2018) is similar to process theory (1988) with an important qualification. Causal mechanism (2018) covers only one of the several senses in which the term process theory is commonly used today. Causal mechanism (2018) does not include: 1) process theory as historical explanation (a sequence of consequential empirical events connecting an initial state with an outcome (cf. George & Bennett, 2005; Van de Ven & Poole, 2005) or 2) Process theorising as understood by constitutive causality theorists. Constitutive causality (2018) was not covered in the 1988 framework. M&R88 valorized process theory, whereas M&R18 cited advantages of all three causal ontology positions.
Levels of analysis: “the entities about which the theory poses concepts and relationships, such as individuals, collectives, or both” (p. 584) Macro versus Micro level: Referring to studies in which concepts relate to higher-level entities (eg, organisations or society) vs. to individuals (p. 593) Mixed level: (inferred) Referring to studies that examine top-down (organisation to individuals) or bottom-up (individual to organisational) influences	Causal Trajectory: A theorist’s views about the causal movements of an affected entity in space and time Cross-boundary change: Change involves influences across the boundaries of natural systems with environments or embedded units Upward initiative: Change involves movement from internal (lower level) to external (higher level) Downward influence: Change involves movement from external (higher level) to internal (lower level) Self-organisation: Change involves movements back-and-forth across system boundaries Indwelling change: Change occurs through interactions within an analytically defined entity. The affected entity is not differentiated into levels. There are no relevant external influences. Evolving interlinkage: Change occurs through the importation of heterogeneous new elements (eg, ideas, actors, resources) into an entity (or loss of elements pre-existing in the entity) and by the creation of new linkages among elements. The affected entity changes qualitatively in composition.	Levels of analysis (1988) was not clearly explicated. It appears to refer to “units of analysis” (eg, individuals, groups, organisations), that is, entities, whereas the causal trajectory dimension (2018) emphasises changes in entities over space and time. Moreover, selection of unit of analysis is a methodological choice as much as a theoretical choice. By contrast, the 2018 framework was explicitly restricted to theoretical choices. Theoretically, levels of analysis (1988) assumes stratified natural systems and thus only addresses cross-boundary change (2018). Causal trajectory (2018) includes two positions not found in levels of analysis (1988)—indwelling change and evolving interlinkage. M&R88 privileged the mixed level, whereas M&R18 noted the advantages of all causal trajectory positions.
Causal agency: “analysts’ beliefs about the identity of the causal agent, the nature of causal action, and the direction of causal influence” (p. 584) Technological imperative: “external forces cause change” (p. 584); “information technology is … a cause of organizational change” (p. 585); “information technology constrains or determines human and organizational behavior” (p. 587) Organisational imperative: “people act purposefully to accomplish intended objectives” (p. 584); “the motives and actions of designers of information technologies are a cause of organizational change” (p. 585) Emergent perspective: “the uses and consequences of information technology emerge unpredictably from complex social interactions” (p. 588); the diagram on p. 586 clearly depicts bottom-up emergence from individuals to organisations	Causal Autonomy: A theorist’s beliefs about the movements of causal effects between human (or social) actors and technology Human sovereignty: Technology is an inanimate product of intentional human actions and therefore only people (or social actors) can be viewed as causal Technology autonomy: Technologies can affect humans (or social actors) and (post development) can sometimes operate with limited human intervention Relational synergy: The outcomes of technology use are the product of interaction between people (or social actors) and technologies	Causal agency (1988) invokes “external forces” and “emergence”—which are addressed in the 2018 causal trajectory dimension—and “determinism”—which is addressed in the 2018 causal ontology dimension. Unlike technological imperative (1988), the technological autonomy position (1988) does not equate technology with external force. The organisational imperative position (1988) was ambiguous, because, while the name implies downward influence (2018), the definition of organisational imperative is more consistent with that of human sovereignty (2018). The emergent perspective (1988) included some ideas that are addressed in constitutive causality (one of the 2018 causal ontology positions) and in the self-organisation, indwelling change, and evolving interlinkage positions on the 2018 causal trajectory dimension. By contrast, relational synergy and the other two positions on causal autonomy (2018) focus less ambiguously solely on causal relations between humans (or social actors) and technology. M&R88 preferred the emergent perspective. By contrast, M&R18 noted the advantages of all causal autonomy positions. In particular, they highlighted the possibilities of the technology autonomy position, which has largely been avoided in sociotechnical research since 1988.

Note: The 2018 framework attempted to reduce the conceptual overlap among dimensions of M&R88. Accordingly, direct comparisons of positions across the frameworks are not possible. Notes in the Comparative Comments column aim to identify key differences in both dimensions and positions.

Furthermore, we had failed to fulfil one early, if largely unacknowledged, goal – justifying a single coherent platform for future IS research projects, no matter what they were. We couldn’t have arrived further from that goal! We learned that the key philosophical and theoretical questions about causality remain unsettled, that all the positions we articulated were supported by credible authorities, and that every position has advantages for certain research objectives or phenomena. There was and is, to our minds, no single best approach to causal reasoning for all IS investigations. All theorising involves choices about what to focus on and what to exclude from focus. Privileging one theory over all others means limiting what one can look at and how one can think about it. We believe that IS scholars should not have to accept such blanket limits in our dynamic field.

But this conclusion is not easy to take. It means that the phase of research before methods design is fraught with uncertainty. Our readers wanted us to reduce this uncertainty by providing guidance on how and when to combine the positions, or limit the possible combinations, and to support this guidance with numerous examples. While we offered our thoughts on possible and impossible combinations, we also explained that the impossible ones might become possible to the extent that positions themselves were redefined. If M&R18 appears as some kind of “box-breaking research” (Alvesson & Sandberg, 2014) with respect to M&R88, M&R18 should not become fixed only to generate future boxed-in research! We did the best we could with the requests of our readers, but all parties will undoubtedly agree that we’ve come up short. However, we are not short of observations on the follies of projects like ours, and we have some advice for others who embark on a similar path.

2. Building a cumulative tradition: openness, fair critique, and value retention

At the end of our 12-year journey, we are happy with the outcome – and also with the process, although it was painful at times. The paper would never become so satisfying to us at present! if we had not continued to labour at it in the face of very challenging comments. At the same time, we can’t help but have unanswered questions, also raised by others (Koufaris, 2010; Olbrich, Frank, Gregor, Niederman, & Rowe, 2017; Orlikowski, 2000), about this kind of endeavour. Is it acceptable, not to mention useful, to attempt to revise and correct errors in past IS research, as opposed to pursuing new studies that add to our body of knowledge? Should an author admit to finding fault in her own prior work? And, if classics are amended, does this mean that the original works no longer have value?

In retrospect, we can see that revisiting early works is a fraught undertaking. It’s one thing when a non-author scholar challenges some aspect of one’s theoretical foundation. The opposing point of view can easily be dismissed or taken as a positive incentive to improve theoretical argumentation. When, however, the author or key proponent of some aspect of one’s own theoretical worldview later repudiates it, this can be felt as a personal affront and be more difficult to reconcile. From this point of view, it may be better to leave the past alone and allow new readers to make their own appropriations and adaptations to earlier writings.

On the other hand, leaving it up to readers to take what they like from a classic and ignore the rest involves the risk of misunderstandings that inhibit theoretical progress in our field. When divergent interpretations are never clearly articulated, miscommunication can follow and missed opportunities can result. If M&R88 and other frequently cited works become a Rorshach test, open to any and all interpretations, we as a field may fail to question assumptions and fail to direct our theorising mindfully in valuable new directions. This is what we concluded after railing against what we perceived to be others’ misinterpretations of M&R88: Unless its errors were identified and vague concepts clarified in writing, any reinterpretation was as good as any other, for who could say different? (This calls to mind a review Lynne once received in which the reviewer contended that what Lynne had written in her submission was not what Markus (19..) meant!)

So, we come down on the side of encouraging our colleagues to revise our knowledge foundations in light of what we now know. But we have learned how hard it is to do so convincingly. In that spirit we offer the following suggestions to those who would try it.

First, critique alone is not enough. Highlighting the errors in a classic does little justice to the fruitful contributions it has made to a community. That which has stood the test of time should be honoured. In addition, scholars who take the path of critique must add to the body of knowledge in significant ways. As everyday wisdom has it: “You can’t fight something with nothing.” If you wish to unseat older ideas, you must offer something attractive in return. This may not mean complete replacement of older ideas (as we discuss below). Rather, it means engaging openly with new literatures and particularly with new theories and philosophy in some kind of box-breaking research (Alvesson & Sandberg, 2014). While useful in any critique (Rowe, 2018) or hermeneutic exercise (Gadamer, 1975), such engagement is essential when dealing with the general categories of language such as space, time, knowledge, or causality.

Second, criticism should not be avoided, but it should be fair. Informed and balanced critique of past works, whether or not it forms the motivation for an article, is essential for identifying the requirements that new theoretical contributions need to meet. At the same time, it is easy to criticise works unfairly. For example, it is unfair to criticise older works for failing to anticipate later phenomena, empirical research findings, or theoretical developments. Being “outdated” may be a fault in a literature review, but a theoretical work can still offer insights long past its sell-by date.

Fair critique focuses on authors’ claims and identifies logical inconsistencies. It homes in on the nuances of conceptualisations. And it highlights alternative interpretations and contextual variations. In other words, fair critique is balanced.

Third, amendments do not necessarily replace classic works; originals may retain value for certain readers. M&R88 focused their framework on IT and organisational change. M&R18 broadened their scope to include all sociotechnical research whether it concerns individuals, inter-organisational relations, or societal change. Accordingly, M&R18 covers more territory and has more positions than M&R88, and the philosophical and societal theoretic warrants in M&R18 make it a longer and more difficult read. For scholars who are mainly interested in IT and organisational change, M&R88 may provide ideas of sufficient value without unnecessary complications.

Similarly, classics can serve as useful tools when new frameworks do not illuminate the issues at hand. Old frameworks are often useful in metatriangulation studies, particularly when the literature being synthesised has been slow to incorporate more recent theories or has not yet undertaken fine-grained conceptual analysis (Besson & Rowe, 2012; Jasperson et al., 2002). Finally, scholars can breathe new life into older frameworks without replacing them by presenting alternative ways to conceptualise existing ideas (Burton-Jones, McLean, & Monod, 2015). The resulting dialogue may prove to be more productive than an alternative theoretical statement.

In short, we remain convinced that revisiting and revising classic papers can offer value for our field. But we are much less naïve now about the challenges of doing so. Despite that reservation, we hope to stimulate dialogue about contribution expectations (Ågerfalk, 2014; Te’eni, Rowe, Agerfalk, & Lee, 2015) by showing that, while innovation through (re)-problematisation is most important (Sandberg & Alvesson, 2011), fair critique is also much needed. Such critical confrontation (Sandberg & Alvesson, 2011) may call for bringing back the commentary genre in IS research, whether built on formal comparisons or not. We therefore hope to encourage others to take on tasks of pruning and shaping the IS body of knowledge – in EJIS and beyond.

Acknowledgements

We are most grateful to Pär Agerfalk, Gabe Piccoli, Dov Te’eni and Jason Thatcher for their feedback and comments.

Disclosure statement

No potential conflict of interest was reported by the authors.