Stimulating academic discourse: a call for response

1. Better to be criticised than ignored

As academics, we love it when others use our work in their research. After all, this is what generates citations and proof of impact. It is perhaps especially rewarding when someone makes an effort to analyse our work and publish a commentary, even if they do not agree with a single word we said. It is better to be criticised than ignored. Of course, they probably misunderstood the point of our paper altogether and refused to accept the better argument. Or, maybe not. A few years back, two colleagues and I published an article critical of a particular body of work, which was well-regarded in its field (Eriksson, Henderson-Sellers, & Ågerfalk, 2013). It did not take long before the key authors behind the approach we had criticised published a rebuttal of our work (Atkinson & Kühne, 2015). In doing so, they scrutinised our article, tore it to pieces, and rejected every single argument and new idea we had introduced. Although my colleagues and I regretted that these authors did not appreciate how a different perspective can make you see things differently, it felt rewarding to know that two highly esteemed scholars had gone through our work and found it important enough to trash publicly. It occurred to me that, in addition to being gratifying to the ego (in some bizarre way), the rebuttal also highlighted one of the foundational principles of scientific progression. Frantz Rowe recently emphasised the importance of being critical (Rowe, 2018). If there was one thing Atkinson and Kühne (2015) succeeded in, it was just that. They were critical. Importantly, though, my co-authors and I had set the discourse in motion by first being critical to the body of work with which these authors associated themselves. With the risk of sounding pretentious, this had me thinking about the nature of scientific development and Kuhn’s structure of scientific revolutions. In a simple form, we here had a body of knowledge, B, that was criticised on reasonable grounds, c(B). This critique was then taken as an object of critical analysis, c(c(B)), which ultimately led to a different understanding of B. B had then been extended and transformed into Bʹ with new arguments for and against its validity, appropriateness, and boundary conditions. In the process, we had also learnt more about the sociology and politics of B and Bʹ – what arguments work and to what extent is B allowed to be challenged, for instance. Being an editor, this made me think about how we could become better at stimulating such critical discourse in our field.

One may object and say we do this all the time. Every paper published in a good journal would have to relate explicitly and critically to previous research in its domain. However, this is not the same as carefully analysing a particular piece of research, following the consistent line of argumentation we expect from a top-journal publication. Such analysis is usually confined to the review process, before a manuscript is accepted for publication, in what Te’eni (2012) referred to as the author–reviewer circle of discourse. Despite increasing interest in opening up the review process (Ross-Hellauer, Deppe, & Schmidt, 2017), peer-review reports largely belong to an “occluded genre” (Swales, 1996) – they are not part of the research record and are not open for public scrutiny (Paltridge, 2017). This is true for most highly ranked IS journals, including EJIS.

When authors submit a paper to EJIS, they are asked to choose a submission genre (what some other journals call “submission category”). One of these genres is the response genre. A paper in the response genre is a paper that comments on another EJIS paper (Te’eni, Rowe, Ågerfalk, & Lee, 2015). The response genre is a way to promote scientific development through critical analysis and argumentative rationality. In this editorial, I discuss how response articles can help stimulate discourse, including bringing the “occluded arguments” of the review process into the open. In explaining the role and purpose of the response genre, I will argue that journals and their editors should see themselves not primarily as gatekeepers but as moderators of a kind of Socratic debate. For this purpose, I use the current issue of EJIS as an example to discuss some principles for writing constructive response articles.

2. Getting responsive

One can distinguish between two different circles of discourse in the journal context (Te’eni, 2012): the author–reviewer circle (as mentioned above) and the reader–author circle. The first circle is defined by the review process of a particular manuscript. The second cycle kicks in when a manuscript is eventually published and made available to readers of a journal. Although response articles may intuitively be primarily associated with the latter, they can also serve to bridge between the two. This issue of EJIS provides an example of such bridging.

The first paper of the current EJIS issue is an elaboration by Matt Mullarkey and Alan Hevner (2018) of the Action Design Research (ADR) approach (Sein, Henfridsson, Purao, Rossi, & Lindgren, 2011). This paper presents eADR as an evolution of ADR that marries ADR with the multiple entry-points Design Science Research (DSR) process described by Peffers, Tuunanen, Rothenberger, and Chatterjee (2007). To someone that has followed the DSR discourse in our field, it is probably not surprising that such an elaboration could be somewhat controversial.

During the review process, it became clear that some of the reviewers found the paper hard to accept. After a few review rounds, one of these reviewers announced that we had reached an impasse. At that point, I decided to reject the paper and invite a new submission that addressed the points I believed to be at the core of the disagreement. Importantly, some reviewers of the original submission were indeed happy with the rejected version, so the material had merit and also the potential to spark an engaging discussion, let alone controversy. As we moved forward with the paper, using a different review panel, I turned to the most sceptical of the previous reviewers and asked whether they would be interested in capturing some of their critique in a response piece. I was delighted when Maung Sein and Matti Rossi agreed to accept the challenge, the result of which is the second paper of this issue (Sein & Rossi, 2018). Although this response piece also had to go through revisions before it could be accepted, it still very much captures the essence of the disagreement that materialised during the review of the Mullarkey and Hevner (2018) paper.

These two papers, and the process in which they developed, make an illustrative example of how journals can help bring some of the occluded arguments of the author–reviewer circle into the reader–author circle and thus allow also for others, subsequently to engage in the discourse. In the process, fundamental disagreement about epistemological assumptions was discovered and articulated. The disagreement concerned how Mullarkey and Hevner had merged a multiple entry-points DSR process with ADR and how this elaboration was potentially epistemologically incommensurate with ADR.

Below, I use the eADR case as a backdrop to discuss the response genre. First, however, I would like to take the opportunity to push this example discourse a notch further. As part of writing this editorial, I seized the opportunity to continue the dialogue with Mullarkey and Hevner. Table 1 shows the outcome of our interaction about Sein and Rossi’s critique.

Table 1. A dialogue with Matt Mullarkey and Al Hevner.

Ågerfalk: You have seen the response paper by Sein and Rossi. What is your initial reaction to their critique? Mullarkey & Hevner: We welcome the comments from Sein and Rossi. Healthy disagreements and give-and-take discussions are constructive activities in an open intellectual environment. Our goal in proposing the elaborated ADR (eADR) process model is to expand the reach of ADR while maintaining its essential components of design science research and action research. We disagree with the wording in the title that we are moving away from the essential nature of ADR. Ågerfalk: One of the main criticisms by Sein and Rossi concerns your decision to include multiple entry points in the elaborated ADR model. Could you perhaps elaborate a bit on this decision? Mullarkey & Hevner: From the time when we presented a preliminary version of the eADR model at DESRIST 2015 in Dublin, we have been actively engaged in refining the model and applying it to facilitate real-world projects. Over this time, we have made research seminar presentations to international academic and industrial audiences, introduced the model in numerous industrial projects, and advised many doctoral students in their practice-oriented dissertations. Ågerfalk: That’s impressive. So you would argue that this was a well-informed design decision and also one that has been positively received? Mullarkey & Hevner: Yes, absolutely. Conservatively, over one thousand individuals have now been introduced to the model with widespread acceptance and use in academic and industrial projects. Ågerfalk: I suppose this provides substantial proof-of-use, but Sein and Rossi were not convinced. Could you provide further justification for the inclusion of multiple entry points? Mullarkey & Hevner: As we discuss in Section 6 of our paper, every ADR project faces the important question of “Where do we start?” In our experience, the answer varies depending on the project context and the research question addressed. We have worked with projects in which (1) the problem is not understood and must be investigated; (2) the problem is known but there exists no design yet for a solution; (3) a potential solution design is known but the “last mile” details of implementing and evaluating the solution have not been done; and (4) an existing solution is in place but issues of evolving the solution to a new environment and an improved state must be addressed. Our conclusion is that a single “problem centred” entry point to an eADR project is inadequate. Not all projects have “green field” contexts that allow the project to start at the beginning with problem definition. Ågerfalk: It seems to me that you may have a different understanding of “problem” compared to Sein and Rossi. This is certainly worthy of further discussion, but I understand that your empirical experience suggests that users of the elaborated ADR model do not see this as a significant problem (no pun intended). Mullarkey & Hevner: Our audiences respond very well to the need for multiple entry points and show the ability to identify appropriate entry points to their projects. In fact, many project teams did not believe that ADR was an appropriate research paradigm for their problem until we pointed out the possibilities of entering the process at the current stage in their project. Ågerfalk: So, how did this lead you to adopt, or perhaps rather adapt, the Peffers et al. model? Mullarkey & Hevner: Early in our investigations, we found that our thinking on eADR entry points aligned well with the entry points proposed in the DSRM process model of Peffers et al. Rather than reinventing the wheel, we adapted their entry point names and definitions to the eADR process model. Table 1 and Figure 4 in our paper provide the details of this adaptation. Ågerfalk: Sein and Rossi apparently disagree with these entry points. Could it be primarily a terminology problem or are there deeper concerns? Mullarkey & Hevner: In hindsight, we agree with the commentary’s critique that better names for the entry points could have been employed. Our attempt to distinguish the entry point names from the ADR stage names perhaps led to some confusion. Thus, while the entry point names may be slightly unclear, the definitions in the tables should provide sufficient clarity to the users of the model. Thus, we believe this is largely an issue of getting the terminology right. Ågerfalk: Apart from the entry point issue, Sein and Rossi leverage a severe critique on more philosophical grounds when stating that you are “combining two approaches that are epistemologically incommensurate.” What is your reaction to this accusation? Mullarkey & Hevner: The commentary authors state that the essence of ADR is the design and implementation of a solution to a specific problem where the reflection during the ADR process leads to generalised learning. This is inductive reasoning from the specific to the general. Whereas, they claim the DSR process draws from general design rules to build a solution. This is deductive reasoning from general to specific. Thus, combining features from the two process models are incompatible from their viewpoint. Ågerfalk: So you agree with the critique? Mullarkey & Hevner: No, we do not see the world of DSR and ADR in quite the same dichotomous way. We disagree with the premise that process models of DSR or ADR must be epistemologically pure in terms of logical reasoning. From a pragmatic viewpoint, we find mixtures of inductive, deductive, and abductive reasoning across all of the eADR projects we have seen. We encourage project teams to find applicable design theories and principles to inform the artefact design, but we never derivate from the message that the solution artefact is the primary goal of the research project with subsequent reflection on nascent design theories. We also observe that Iivari (2015, p. 113) states that “one cannot conclude that Strategy 1 relies on deductive reasoning only and Strategy 2 on inductive reasoning only.” Ågerfalk: Thank you so much, Matt and Al. I am sure that your paper, Sein and Rossi’s response, and this follow-up discussion will be useful to the community in reflecting on the epistemological foundations of ADR and DSR. Any final words before I send this piece to the publisher? Mullarkey & Hevner: In conclusion, we appreciate the sincerity and the rigour that Sein and Rossi bring to their ADR commentary as two of the principal authors of the seminal ADR paper. However, we stand firm with our convictions that multiple entry points in the eADR process model are helpful to support ADR project teams with the needed flexibility to establish well-defined project starting points. From our experiences, we also believe that the incorporation of entry points in the eADR process model is compatible with the use of multiple types of logical reasoning as rigorous bases in the research. Finally, we disagree with the critique’s closing comments that our eADR ideas provide “backward” steps. Project teams have greater freedom to apply eADR as appropriate to support the relevant and rigorous objectives of their research project. Adding to the richness and openness of ADR thinking should not be considered backward steps. Our goal, one which I am sure that we share with Sein and Rossi, is to support and encourage more research teams in the effective use of ADR to achieve improved solutions to challenging problems and opportunities. Ågerfalk: I am sure we all share that vision. Thank you once again and let us now all move forward.

The dialogue in Table 1 illustrates the iterative nature of the academic discourse. Indeed, a response paper is unlikely to bring closure to debate. On the contrary, being subject of scrutiny, critique, and, quite possibly, misreading, authors may want to clarify their position. Maybe a continued discourse can resolve outstanding issues, or maybe participants have to agree to disagree at some point. In any case, the dialogue in Table 1 indicates the kind of continued discourse that response papers can trigger. In the case of eADR, we are yet to see what steps will be taken next, and in what direction.

3. Criticising constructively

Although the academic discourse may take many different forms, the Socratic dialogue remains at its core (Rowe, 2018). Such dialogue can be characterised as a cross-examination in which participants raise critical questions about the actions or position of an interlocutor. Engaging in such dialogue “clarifies discourse, avoids dead-ends and opens up new perspectives” (Rowe, 2018, p. 389). Typically, repetitive questioning is used “to force people to admit their ignorance … and thus rely on logic instead of pride or faith when deciding which beliefs are valid” (Overholser, 1993, p. 67).

A Socratic dialogue aims to be as concrete as possible and avoids abstract argumentation in the pursuit of the stronger argument in the discourse that forms through the debate – discourses are “practices that systematically form the objects of which they speak” (Foucault, 2002, p. 54). Discourse can be thought of as “dialog with others in the community expressing thoughts and emotions, generating perspectives of situations and phenomena, moving across levels of context of a perspective, moving from simplifications to complications and from concrete to abstract descriptions, and using representations with various formats and lexicons to do so” (Te’eni, 2012, p. 341). By cross-examining these “discourse objects”, personal biases and prejudices are identified and challenged to enable intersubjective understanding.

In a response paper, it is difficult fully to apply the Socratic method. Nevertheless, when crafting a response, it can be a useful model to have in mind. For instance, returning to the eADR case, we can see how Sein and Rossi formulated a number of claims about eADR “discourse objects” (inductive vs. deductive reasoning in DSR and ADR, for instance) with which Mullarkey and Hevner did not agree. When asked specifically about these claims (Table 1), Mullarkey and Hevner could clarify their perspective. Such clarification can facilitate further analysis and perhaps trigger further interrogation (which regrettably is beyond the scope of this editorial). One way to think about discourse is to view it as a sequence of arguments exchanged to convince an interlocutor. Toulmin (2003) suggests that valid arguments consist of a set of key components, including claims, grounds, warrants, and backing. Claims assert something for an audience to believe. Grounds are the evidence produced in support of the claims. Warrants provide the logical connection between a claim and its grounds. The backing is the reasoning that supports a warrant. Strong backing is particularly important if the receiver find it hard to accept the warrant. To develop well-argued criticism, one may use Toulmin’s key components as discourse objects to structure the analysis.

Moreover, as Baron (1988, p. 199) points out, “Recipients seem to prefer feedback that is specific, is delivered promptly, and is considerate in nature, to feedback that is general, is delivered only after a delay, and is not considerate in tone.” Thus, although the Socratic method is confrontative, we need to be careful to deliver criticism that is constructive and considerate (Rowe & Markus, 2018). The aim of a response paper is not to prove one’s point but rather to highlight logical flaws, implicit assumptions, and biases in the original paper. For the response to be effective, the feedback needs to be perceived as constructive. Said Saar-Tsechansky (2015, p. iii), “Perhaps the most basic element of a critique is that it is well-founded and clearly explained.” Additionally, in order to provide constructive feedback, both what went wrong and what went right needs to be communicated (Bee & Bee, 1998).

Thus, when writing response papers, I would suggest authors keep the Socratic principles in mind when (a) drawing out the underlying assumptions, which could be used as a basis for interrogation, and (b) avoiding abstract argumentation by instead discussing specific discourse objects focused on (c) the key components of the arguments. I believe we see evidence of all these elements in the eADR case, and we would like to see evidence of them in future response paper submissions as well. For instance, Mullarkey and Hevner (M&H) claimed that multiple entry points and ADR could be effectively integrated. Sein and Rossi (S&R) questioned the grounds for this claim by referring to the idea that all ADR processes are problem driven (a warrant). They also provided their constructive and (reasonably) considerate feedback promptly. Nevertheless, M&H found this critique invalid (Table 1) because, in their view, it lacked evidence (grounds) and sufficient backing. To counter the critique, they provided empirically derived grounds to warrant their original claim.

Interestingly, M&H and S&R used different strategies and arrived at different conclusions. S&R focused on theoretical and philosophical grounding. Although responding to this critique, M&H primarily turned to empirical grounding and proof-of-use. A continued dialogue might challenge S&R’s warrant by examining their interpretation of the underlying philosophy. Similarly, M&H could be challenged to present data (grounds) that support their claims about the reception of eADR. The lines of argument could potentially converge, and discourse could be a way of resolving the controversy. Alternatively, discourse could prove this is indeed an impasse. In any case, breaking a chain of argument down into its components (discourse objects) can help identify weak arguments and areas of disagreement. The analysis can also be used to ascertain that all bases are covered when crafting responses.

4. A call for response (and constructive reviews)

In this editorial, I have shared some thoughts on what to think about when writing response articles. At EJIS, we believe such articles are essential and would like to encourage the community to contribute to this genre. I have used the current issue of EJIS as an example and also included an additional “post response” dialogue. Usually, such dialogue would happen elsewhere, maybe as a panel at a conference or as part of a more elaborate theory development submission. In keeping with the ideas presented by Te’eni (2012), it could certainly also happen online.

It should be noted that an academic discourse on a particular topic is to some extent unbounded. It is usually difficult to say when it started and when it has ended. A response article is therefore always situated in a historical context of arguments and thoughts. Analogous to the structure outlined in the introduction above, Mullarkey and Hevner’s (2018) work can be seen as an empirically grounded response to the original ADR paper by Sein et al. (2011), which, in turn, was a reaction to the then current DSR discourse. At EJIS, we ask that papers in the response genre focus on a single earlier EJIS paper. This way, the response authors can focus their attention and use one particular previous publication as a peg on which to hang their arguments. If the response needs to broaden its focus and engage with a larger body of research, it has shifted into a theory development, research essay, or literature review and should be submitted and evaluated as such.

Although this editorial has focused on the response genre, and thus primarily the reader–author cycle of discourse, most of the advice also applies to the author–reviewer cycle. The latter is perhaps even closer to the Socratic dialogue in that it may extend over several rounds of interaction (i.e. revision cycles). It is, therefore, my hope that the ideas presented here will be useful also to authors and reviewers more generally.

Acknowledgment

I thank Frantz Rowe, Dov Te’eni, Jonas Sjöström, and Amir Haj-Bolouri for constructive criticism of a draft version of this editorial.

Disclosure statement

No potential conflict of interest was reported by the author.