Preparing Manuscripts that Report Qualitative Research: Avoiding Common Pitfalls and Illegitimate Questions

Abstract

The growing scholarship about qualitative inquiry means it has become an umbrella term for a diverse range of studies with differing theoretical underpinnings and methods. Nevertheless qualitative studies share a common purpose of representing or interpreting complex and situated social processes and experiences and continue to be the most common type of manuscript published by Australian Social Work. Telling a clear and concise story in a 6,000 word journal article about the what, how, and why aspects of the social world requires well-honed writing skills and good judgment about what must be included and what can be left out. This paper provides guidance to preparation of qualitative manuscripts drawing on the authors’ experience as an academic author and past editor of Australian Social Work.

由于学术上的日积月累，定性研究已成为一个涵盖广泛、包括了不同理论及方法研究的概念。不过，定性研究有共同的目的，那就是要表达或阐释复杂而具体的社会过程及经历。它一直是《澳大利亚社会工作》发表最多的一类稿件。在一篇6000词的杂志文章里要把事情从内容到方式到原因讲得简明扼要，需要出色的写作技巧，知道该去什么该留什么。作者根据自己学术写作的经验以及担任《澳大利亚社会工作》主编的经历，对如何写作定性文章做了指点。

Keywords:

• Qualitative Research
• Writing for Publication
• Social Work Scholarship

Research published in Australian Social Work (ASW) furthers the purpose of social work: producing knowledge that enhances understanding of social problems, the organisation and practice of human services and development of equitable social policy (Orme & Powell, 2008). It demonstrates the many truths and ways of knowing in social work (Hartmann, 1990). A majority of papers fall under the umbrella of qualitative research (Simpson & Lord, 2015). There may no longer be a need to establish the value of qualitative research per se (Patton, 2002), but the increasingly diverse kinds of inquiry that shelter under the qualitative umbrella pose new challenges for authors. This paper discusses some of these, particularly the task of demonstrating rigour, by balancing sufficient detail about theoretical underpinnings and evidence to support findings, with editorial imperatives for conciseness. It aims to guide preparation of manuscripts and complement some of the more detailed guidelines for publication and evaluation of qualitative research (Australian Journal of Social Issues, 2011; Critical Appraisal Skills Programme, 2013; Letts et al., 2007; Social Science and Medicine, 2010).

Imperative to Articulate Theoretical Underpinnings

Traditionally qualitative research was regarded as a field of inquiry in its own right traversing disciplines, fields, and subject matter (Denzin & Lincoln, 2011). Many texts use qualitative and quantitative as the organising frame to discuss different types of research (e.g., Alston & Bowles, 2012). Such distinctions carry implicit assumptions that, for example, quantitative research is underpinned by a positivist theoretical perspective and uses “quantitative” methods such as content analysis or statistical analysis, while qualitative research has an interpretive perspective and uses “qualitative” methods such as participant observation or interviews. However, the distinction is not clear-cut as many methods are used in both qualitative and quantitative research.

Qualitative research has become a generic term incorporating a broad range of methods and theoretical stances, and there is “as much variation among qualitative researchers as there is between qualitatively and quantitatively orientated scholars” (Staller, 2013, p. 403). Staller's (2013) ideas of “vertical integration” and having to “moor” methods are useful in thinking about the need for authors to be explicit about the theoretical underpinnings of their research. This has not always been the case, as Patton (2002) explained: the first edition of his book on qualitative research methods did not include discussion of epistemology, as this had not been done in texts about survey research. The tradition of “unconscious epistemology” (i.e., failing to make epistemology explicit), alluded to by Patton, continues to be reflected in both guidelines and papers reporting on quantitative methods. However, this is no longer an option for qualitative researchers.

Qualitative researchers assemble differing combinations of epistemology (theory of knowledge, e.g., objectivism, constructivism); theoretical perspectives (philosophical stance, e.g., positivism, interpretivism); methodology (the strategy for action or design that shapes choice of methods, e.g., grounded theory, action research, ethnography); and methods (e.g., purposive sampling, participant observation, comparative analysis) (Crotty, 1998). For example, grounded theory methods are used in studies with quite different epistemological or theoretical perspectives: Craig and Bigby (2015b) used grounded theory methods in a study with a critical realist epistemology, Charmaz (2014) used these methods in research with a social constructionist perspective, and Strauss used them in research with a positivistic perspective (Birks & Mills, 2011). Similarly Braun and Clarke (2006) illustrated thematic analysis as a method independent of any epistemology that can be applied across a range of theoretical perspectives. Anfara and Mertz (2006) gave many examples of inserting into qualitative research methods theoretical lenses from different fields of study, such as job choice theory or learning theory that are unconnected with particular methodologies but shape the direction of data analysis. Ensuring a manuscript is vertically integrated will capture the theoretical assumptions embedded in the study, as well as ensure that they cohere and are reflected in the level below—epistemology is reflected in theoretical perspective, which is reflected in methodology, which is reflected in the methods.

Making Explicit Criteria for Quality and Rigour

Explication of the method and findings is the means by which rigour is judged. There are no definitive criteria, as these depend upon the perspectives used by the particular scholars on whom an author has drawn to inform the design or analysis. As Patton (2002, p. 266) suggested, competing philosophical and theoretical underpinnings will “generate different criteria for judging quality and credibility”. He proposed contrasting sets of criteria including traditional scientific research criteria and trustworthiness that stems from social constructivist criteria (Lincoln & Guba, 1986). Thus depending on theoretical underpinnings, analysis of qualitative interview data might be evaluated in terms coding inter-rater reliability, or evidence of the analyst's reflexivity and capacity to understand how their own experiences and location have affected how the data were constructed and interpreted. Simply describing methods and a study as being qualitative, as has been the case in many papers in the past, gives little insight into its theoretical underpinnings and thus the criteria by which to judge it. “Illegitimate questions” (Guba & Lincoln, 2005), about rigour—ones that have no meaning as they are drawn from an inappropriate theoretical frame of reference—can be avoided, or at least dealt with by specifically referencing and addressing the criteria relevant to the study.

Concisely Capturing Complexity

The purpose of qualitative inquiry and its suggested characteristics provides a glimpse of the complexity in terms of social theory, processes, and the context it sets out to represent or interpret. Definitions such as Creswell's (2013) recognise this complexity and help to lay out the tasks involved in writing up qualitative research for publication:

Qualitative research begins with assumptions and the use of interpretative/theoretical frameworks that inform the study of research problems addressing the meaning individuals or groups ascribe to a social or human problem … qualitative researchers use an emerging qualitative approach … the collection of data in its natural settings sensitive to people and places under study … data analysis that is both inductive and deductive and establishes patterns or themes. The final written report … includes the voices of participants, the reflexivity of the researcher, a complex description and interpretation of the problem, and its contribution to the literature or a call for change. (Creswell, 2013, p. 44)

The author must explain not only their original theoretical perspective but also any changes in design that occur during the study, details about the context in which data was collected (cultural, geographic, historic, etc.), the participants’ characteristics and the processes used in analysis, as well as representing the complex and perhaps differing realities about the problem at hand. As Becker (1986) illustrated, the rich representations of social problems, lived experiences and practice, and the new concepts and theories produced by qualitative research make reporting a study in a 6,000 word journal article a daunting task.

Telling a clear and concise story about the what, how, and why of aspects of the social world requires judgment and prioritisation. Authors must let go of interesting details about the context in which data were collected that are not central to the main story. They must capture messy, creative, and iterative processes by reconstructing reality into a linear sequence. Attention must also be paid to language conventions, such as use of people-first language for people with intellectual disability irrespective of the impact on word count.

Common Issues Arising in Preparing Manuscripts

The APA style of ASW applies to the form of a manuscript, which is similar in all types of research, as well as citation and referencing conventions. The Introduction describes the background and rationale for the study, establishing its aims and reviewing existing knowledge about the problem. Method explicates and defends the theoretical underpinnings and describes the design, and methods used to collect or create and analyse data. Results, more often called “findings” in qualitative manuscripts, distil and provide evidence for what the study has found, and Discussion draws parallels with existing literature, and discusses the significance of the study and its implications for practice or policy. The following sections highlight issues specific to qualitative research in each manuscript part.

Abstract and Title

The abstract is a précis or summary, not a menu of what is to come. Do not write “and the implications of the study will be discussed” but rather “the study showed supported living is a viable option for people with intellectual disability in group homes”. The abstract and title are now crucial to the discoverability of papers in a digital world. Snappy quotes, so often used in titles, should take second place or be omitted. On reflection for instance, “It's pretty hard with our ones, they can’t talk, the more able bodied can participate” (Bigby, Clement, Mansell, & Beadle‐Brown, 2009) was not a good title as it conveyed almost nothing about the paper's content. A more pedestrian title using key words will pay off in the long run.

Introduction

The introduction creates the rationale for the paper, by establishing the problem or issue being addressed, situating it in the existing literature, and identifying gaps in knowledge that exist. It acts as a funnel, starting from the bigger picture and narrowing down to the paper's specific aim and research questions. How wide to start and whether or not to write a few sentences of overarching introduction are dilemmas. I avoid introductory sentences, and rely on readers being introduced to the paper through the abstract.

Scope is a common issue, an entire doctoral thesis or contract research report cannot be squeezed into a journal article. Several considerations should guide authors in splitting up larger studies for publication. Every paper must have substance and make a contribution to knowledge in its own right. “Salami slicing” a project into many “thin” papers of little substance will be identified by reviewers and frowned upon by editors. Every paper must stand alone with its own aims and research questions. The data may be flagged as being drawn from a larger study but the methods used to create them must be articulated and “unlinked” from a bigger study.

This is not straightforward as a manuscript may rely on a key concept or finding from the bigger study. Useful strategies are to reference the bigger study or think carefully about the order in which papers are prepared so a cumulative set of evidence is built for citation in later papers. For example, Craig's doctoral thesis (2013) developed the concept of Active Participation. In writing a paper about the factors that detracted from or supported it, she faced the dilemma of referring to a concept that did not appear in the literature. Rather than explaining its analytical development she simply defined the concept and cited her thesis (Craig, 2013; Craig & Bigby, 2015a). However, relying too much on unpublished work is problematic.

From my own PhD thesis, I wrote several papers about informal and formal support available to older people with intellectual disability which set the groundwork in the literature for a broader paper about policy issues. Looking at an author's body of work may provide examples of how bigger studies have been split up. It is also useful to craft phrases such as, this paper “drew on a subset of data from … The data presented in this paper relate to the siblings…associated with the 13 older group home residents from the larger sample of 17 … The research questions were…” (Bigby, Webber, & Bowers, 2014).

A second issue particularly in evaluation or action research studies is muddling together the background or context of the study, such as the program being evaluated with the findings about it. Failing to separate these elements can mean that findings contain lots of descriptive information about the program that obscures the study's central message. On the other hand, if insufficient information is given about the program it is difficult for the reader to know what is being evaluated. This can be managed by inserting a subsection in the introduction describing the context or program that was central to the study (e.g., Bigby & Frawley, 2010; Wendt & Baker, 2013).

Methods

As already discussed, the theoretical underpinning of a study should be set out as the first part of the methods section or at the end of the introduction, and used to justify the design and methods. Be cognisant of inconsistent terminology (e.g., Creswell, 2007, p. 19) by selecting, citing, and using consistently one source through the paper. Staller illustrated the necessity of this in her discussion of the differences between Crotty (1998), whose terms I have used in this paper, and Padgett (2008) in her book Qualitative Methods in Social Work Research:

There is some disagreement between Padgett and Crotty at the level of methodology and methods (with Padgett introducing another layer labelled “approach”). So while Crotty would call ethnography, grounded theory, phenomenology and action research methodologies, Padgett labels these “approaches”. While Crotty and Padgett agree that discourse is placed at the methodology level, they disagree (for example) whether participatory observation constitutes a methodology (Padgett) or merely a method (Crotty). (Staller, 2013, p. 405)

While mixing methods to create a unique design may be a feature of qualitative research, mixing terms to classify and describe the theoretical underpinnings of a study should be avoided.

Similar imperatives to provide sufficient detail about the design and conduct of the study apply to qualitative and quantitative research. Qualitative researchers may use different terms and order of presentation. Rather than sample, they often refer to participants and settings, first describing their approach to sampling—such as purposive or theoretical sampling in the case of grounded theory—and then detailing the method or methods of recruitment and participants’ characteristics. Details of all participants and their relationships, if any, to each other must be included together with data collection methods. The reverse also applies: do not include details of data or participants that were included in a bigger study but not reported in the manuscript.

The analysis section must explain the processes used to develop reported themes or categories. The author should use their own words and illustrate, for example, the process of moving from concrete descriptive coding to higher levels of abstraction. There are many and diverse approaches to coding qualitative data (e.g., Saldaña, 2009), so name the approach and cite the sources drawn on, ensuring analytical method is vertically integrated into the study. For example, was a content analysis informed by a specific theory that counted the number of times a word or concept was mentioned as might occur in a positivistic study, or was it an analysis that used an interpretative inductive approach to conceptualise recurring themes across all or a subset of the participants associated with particular conditions? Avoid cutting and pasting nicely written statements about analysis from texts, or claiming to have taken a specific approach unless this is reflected in the findings. Too often what is claimed to be a thematic or interpretative analysis turns out to be simply a description of the topics that were discussed in interviews or focus groups.

As already discussed, the relevant criteria by which to judge the study should be named, and the way methods have addressed at least some of these described (Lincoln & Guba, 1986; Patton, 2002). Allude to both the strengths and limitations of the study in an honest and straightforward way.

Finally decisions about data deidentification made at the ethical approval and consent stages of the study must be respected and flow through into the manuscript. For example, whether the names of services involved or the location of a study have been disguised, and pseudonyms used for participants. Any coding schema to identify the source of quotes used in the findings should be explained.

Findings

Findings are usually presented in the form of the central themes or categories derived from the analysis. Do not start each theme or paragraph with the phrase “analysis of the focus groups found that”, rather be definite and direct and go straight into describing the theme. Theoretical underpinnings and methods dictate how to present findings and particularly whether to give counts of “how many” participants’ views for example, were reflected in a particular theme. For example, research that uses a constructivist grounded theory methodology will focus on reporting identifiable patterns in the data; it’s the qualitative nature of categories and the differences between them rather than number of participants in each category that is significant. As Sandelowski (2001, p. 238) illustrated in reporting a study of HIV and women, what was important was the discovery of their different orientations and demonstrating “there were different and mutually exclusive patterns of response, which the analysis labelled future orientations … and the consequences of having each orientation”, rather than showing the prevalence of each orientation that would hold little meaning given the small purposeful sample.

The “evidence” is usually in the form of quotes from participants or texts rather than tables and statistical analysis. Importantly, a strong narrative thread must run through the findings to act as the glue that holds the evidence together. Choosing quotes is difficult, particularly when there is thick data and saturated themes. Ideally, but not for each theme or subtheme, quotes should come from across the data set, and reinforce any claims about saturation. Presenting quotes extracted from long transcripts is an art when words are at a premium. A quote must be long enough to be convincing, and set within sufficient context to make sense for the reader. The general convention, in some methods but not all, is to “tidy up” quotes improving flow and readability, by removing “ums and aahs” or “you knows”, correcting grammar and removing repeated words. You may also splice parts of a transcript together using ellipses.

Discussion

The discussion summarises the findings, makes connections between these and existing knowledge in the literature, and draws out the implications for practice and policy. It is fairly similar for all types of research, other than perhaps studies using an objectivist epistemology that are more likely to include limitations in the discussion rather than methods section. Alarm bells will ring for reviewers if an author using a constructionist theoretical perspective and grounded theory or thematic analytical methods states that findings from the study cannot be generalised, as the sample was not big or representative enough as this is an “illegitimate” and unnecessary statement.

Dealing with the convention that no new data or literature should be mentioned in the discussion presents a quandary for qualitative researchers, particularly those working in grounded theory methods where the analysis leads to new ideas and exploration of new literatures. Although debatable, my preference is to follow the convention and not introduce new literature, which may require reconstructing an iterative process for the sake of producing a logical and concise manuscript.

Conclusion

Overclaiming the significance and reach of a qualitative study is a common downfall that behoves authors to remember the purpose of qualitative research is to represent or interpret complex and situated social processes and experiences, rather than explaining the nature of an entire social system or perspectives of populations. Research is not research until it is published and good qualitative research manuscripts will generate key ideas and evidence about important social processes that will further social work practice and the development of social policy. The best and most concise writing is always a collaborative process. Do not do it alone, fresh eyes along the way will identify leaps of logic, things that are muddled together, and missing elements. Editors and peer reviewers should be the last of many who will have read and reread a manuscript.