![Sample our Social Sciences journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/110822/TOC-Subject-Sample-2021-Social-Sciences.jpg"})
Abstract
The methods used when samples of journal articles are treated as the basis for measuring levels of quantification in national sociologies are examined. The data come from every identified example in English. It is found that operational definitions of ‘quantitative’ vary, while ‘qualitative’ work is characterised simply by the absence of sophisticated quantification, which is sometimes taken to imply quantitative incompetence rather than methodological choice. The articles used as data on the state of a whole national sociology are usually drawn from elite general journals in which many sociologists have never published, and ignore the authors’ national backgrounds and publications elsewhere. It is concluded that there is a gap between concepts and operational definitions which it would be desirable to fill.
Acknowledgement
Thanks to the referee whose painstakingly detailed work required a response which improved the precision with which the data are reported.
Notes
1. One from a now-defunct Australian social work journal proved impossible to obtain.
2. When the comparisons are of widely separated periods, it might seem appropriate to take into account the ways in which quantitative techniques invented, data availability, or physical technologies, have changed over time; tests which were advanced and entailed a lot of work at an earlier period now look quite different. No paper found has done this.
3. Abend (Citation2006, p. 4) used a definition that he thought informally conventional: ‘ whereas articles that run OLS [ordinary least squares] regressions are generally seen as quantitative, articles that present cross-tabulations are generally seen as nonquantitative’.
4. Not all the papers confine themselves to these rather limited and mechanical categories. Two feminist authors, MacKie (Citation1985) and Collyer (Citation2013b), are led by their qualitative methodological commitments to classify papers not just on the basis of the appearance or not of numbers in them, but of the general thrust of their argument; MacInnes, Whybrow, and Eichhorn (Citationin press) have a more sophisticated coding scheme than most of the others, distinguishing how the data are used in the argument and what forms of qualitative data are used. The rural sociology papers also enrich their lists of statistical tests by having more on features of the whole argument and the way in which it feeds into an assumed continuing research programme. All of these approaches make for a more interesting and empathetic discussion.
5. In addition, some authors omitted special issues from the count on the ground that they would bias it; unfortunately, the omission of course also biases the count.
6. The category of ‘all relevant articles’ is not always as broad a representation as it sounds; it includes both one paper [on women’s publications] with every article by a woman published in the period covered (Platt, Citation2007) and another [on the relation between theory and methodology] which limited itself to articles in particular topic areas which were of interest to the graduate students doing the work (Snizek, Citation1975).
7. Green, Feinerer, and Burman (Citation2013, p. 187) discuss the same problem in a slightly different context. They found that ‘Fashionable topics and approaches … experienced a “flash” and then quickly faded away …’. This meant that if many years were combined interesting detail was lost, while if the detail was recovered by looking at years separately this risked creating artefactual gaps.
8. There is something questionable, too, in maintaining the same criteria for measuring the level of quantification deployed over a period which for a number of the studies lasts more than 30 years, and includes the transformation brought about by the diffusion of computer technology and SPSS, and the invention or introduction of novel statistical methods. What the answer to the question might be is not obvious, but it surely merits some methodological discussion.
9. Fleck (Citation2011, p. 116) addresses directly the issue of who to count as a sociologist – and (in a case where little other data are available) uses having published an article in a sociological journal as one criterion.
10. It is evident that foreign authors come into British journals by a variety of routes: distinguished ones are invited, emigrants submit to journals from their country of origin, foreigners who have done graduate work in Britain retain a sentimental link, foreigners who have collected data on Britain write a paper on it for the British readership, foreigners now hold jobs in Britain, papers rejected by prominent US journals try their luck elsewhere …
11. Béland and Blais do actually draw on some data on publications elsewhere by members of their constituency.
12. US rural sociology was a rather special intellectual community, not just like another ASA section – it had its own association, and was largely located in Agricultural Experiment Stations and land-grant universities; in this case, it may indeed be appropriate to treat the journal Rural Sociology as representing the field academically (Platt, Citation1996b, pp. 150–152).
13. Turner (Citation1989) looks like an exception, presenting data only on AJS, but the argument also uses Wells and Picou’s data on ASR, so this is treated here as using two cases.
14. However, there are good practical reasons for some influence from date of foundation: only the older ones have been around long enough to provide data on historical trends.
15. Data directly comparing the characteristics of authors with different publication patterns are provided by Shrum (Citation1997), who found that in developing world countries studied members of international centres published more than others, but not in the national journals of the countries where they were located, and actual research productivity was higher than shown by international sources.
16. What if the specialism is methodological? Travers (Citation2009, p. 164) is, for instance, able to list eight journals whose titles include ‘Qualitative’ or ‘Ethnography’; presumably, these include many papers which in their absence would have had to be submitted to a general journal, which would thus have been more qualitative.
17. No author has chosen journals from more than one country unless an explicitly comparative question is addressed, so none can claim data on world sociology.
18. All which journals? Kelly and Burrows (Citation2011) found that in sociology submissions to the RAE in 2008, the journal articles came from 847 journals; Piriou and Cibois (Citation2009) found that the articles of a sample of about 300 French sociologists had appeared in 735 journals, few of them the national general ones and many with little obvious connection with sociology.
19. Cronin, Snyder, and Atkins (Citation1997) found little overlap between lists of highly cited authors from book and journal sources.
20. Some of our authors also collected data on conference papers. That usefully widens the range of coverage, but some of them later become journal articles, with potential double counting. However, this could provide interesting data if the conference papers were followed through to see which ones got published, or perhaps even how they had been modified when finally published. An article on the fate of 1966 ASA conference papers (Lin, Garvey, & Nelson, Citation1970) found that after two years a fairly small proportion had been published, and numbers of those were in journals relatively obscure to sociologists.
21. He shares with Gold (Citation1957) perception of many contemporary uses of quantitative methods as incompetent and inappropriate, and that makes part of his reasons against the tendency.
22. Sewell (Citation1965) did impressively list a number of generally agreed criteria for adequacy of research articles, and applied them to his set of rural-sociology articles. This enabled him to reach conclusions on such matters as how many of the articles employing significance tests used them appropriately.
23. It is notable, too, that Erikson (Citation2005), a great proponent of quantification, makes something of the contents of a single issue of Sociology in his argument.
24. Béland and Blais (Citation1989) are a valuable exception here.
25. Moody’s (Citation2004) sample, of all English journal articles listed in Sociological Abstracts from 1963 to 1999, found that the modal author appearing there had only one publication.
26. Ennis’s data (1992, p. 261) show that in American Sociological Association section affiliations there have been marked differences between the clusters with which quantitative and qualitative methodology memberships are associated, which is suggestive here.
27. A good example of this is provided by the University of Liverpool’s Social Research Series in the 1950–1960s, where the research teams listed in the books regularly included Elizabeth Gittus, who held mathematical/statistical qualifications, as the statistician.
28. May (Citation2005) suggests some practical career reasons for choosing qualitative strategies.
29. Some of those expressing alarm about the current British situation have also collected useful survey data which does throw direct light on matters such as respondents’ claimed statistical competence.