A conceptual-empirical typology of social science research methods pedagogy

ABSTRACT

The challenge of research methods teaching is gaining attention among policy-makers keen to build social science research capacity and, critically, among educationalists keen to enhance the pedagogy. This paper addresses pedagogy, presenting a new conceptual-empirical typology of pedagogy for social science research methods teaching. Taking a sociocultural perspective, pedagogy is seen as encompassing both actions and underlying values. A mix of qualitative methods was used to engage more than 100 methods teachers (plus students) from diverse UK and international contexts. An expert panel method and focus groups helped elucidate pedagogical knowledge. Video-stimulated reflective dialogue added detail to that knowledge. Thematic analysis was used to make sense of teaching practice with individuals and across the dataset. A typology of research methods teaching developed iteratively across this process, proposing the core categories of approach, strategy, tactics and tasks. In-depth case studies helped to gain nuance and test the emergent typology in situ. The paper argues that the typology contributes a dynamic tool for developing practice. It transforms the way we think about teaching and can be applied in any social science research method teaching context, benefitting the pedagogic community by enabling greater focus in planning and reflection.

KEYWORDS:

• Research methods pedagogy
• typology
• teaching approach
• teaching strategy

Introduction: research methods pedagogy

This paper reports on the findings and outcome of a five-year study with the purpose of examining the pedagogic practices of research methods teachers in the social sciences, opening them up for discussion with a view to building the pedagogic culture in research methods training. An outcome – or product of the research – is a conceptual-empirical typology authentically grounded in data that we argue is a valuable tool for the development of methods teaching and reflexive pedagogic practice. This argument is based on a mix of our intentions for the typology, our efforts to make it accessible and useable and the responses (of participants, teachers and wider audiences) during the process of conducting and sharing the research.

The study investigated the pedagogies at work in advanced (postgraduate) research methods teaching and training, primarily in the UK. Research methods is an area of education where previous pedagogic research has been limited (Wagner, Garner, and Kawulich 2011; Earley 2014; Kilburn, Nind, and Wiles 2014). What research there is, is built primarily on individual reflective teacher inquiry with a single class or cohort. There have been occasional studies (Hurworth 2008) or meta-analyses of qualitative methods pedagogy (Cooper, Chenail, and Fleming 2012) and there have been studies of the research methods teaching literature more widely (Wagner, Garner, and Kawulich 2011; Earley 2014; Kilburn, Nind, and Wiles 2014). But there has been little first-hand, primary, cross-case work on what methods teachers across disciplines and paradigms value in their teaching or undertake in practice. In this respect, our study breaks important ground. This scarcity in research may reflect discourses that argue qualitative methods cannot be taught (Parry, Atkinson, and Delamont 1994; Hammersley 2012) or the idea that proficiency in research is best learned through apprenticeship (Bourdieu 1992). In effect, methods teachers have, therefore, been largely unsupported by research in their pedagogic decision-making and practice. Methods teaching is difficult, sometimes unwelcome (Daniel 2018) and stakes are high – due in part to the foundational role that methods have in the building of empirical knowledge across the social sciences, and in part to the value placed on research competence by governmental funding bodies globally.

From our sociocultural perspective, pedagogy is ‘concerned not just with what people do in teaching and learning situations but with what they perceive to be meaningful, important and relevant’ (Nind, Kilburn, and Luff 2015, 2). It includes the hidden pedagogy of resilient pragmatic beliefs based on classroom experience (Densgombe 1982). It includes both what is and isn’t visible (Gamble 2001), embracing tacit knowledges – the knowing-in-practice (Polanyi 1958) that recognises that skilled teachers know much more than they can tell. This relates to the intertwined know that and know how (Ryle 1949) of being a teacher. These layers, plus the struggle to articulate something that is non-linear, relational and emergent (Sellar 2009), can make pedagogy ‘hard to know’, as Nind, Curtin, and Hall (2016) argue. Teachers’ craft knowledge (Brown and McIntyre 1993) is both complex, and elusive. To develop practice in the teaching and learning of social science research methods, it is important to tease out how practice is understood and rationalised and to explore moment-by-moment pedagogic (inter)actions and decisions in situ. In our work, we have done this (see methods section) and sought to develop a typology to represent teachers’ explicit and tacit processes.

Typology as research work

A typology of pedagogy can offer teachers a way of seeing and thinking about what other teachers do and why, and how this can be related to their own practice. This is not about providing a normative steer, rather a self-analytic tool. A typology is a classification of practice that distils complex data – in our case on complex pedagogies – through the identification of key parts or categories. As Bailey (1994, 1) argues, ‘without classification, there could be no advanced conceptualisation, reasoning, language, data analysis or, for that matter, social science research’. We have been looking to communicate with social science researchers about the teaching of research methods in a way that makes sense to them; this means naming the distinct aspects of that teaching (Bailey’s (1994, 6) ‘nomenclature’) to support new ways of knowing:

… a well-constructed typology can be very effective in bringing order out of chaos. It can transform the complexity of apparently … diverse cases into well-ordered sets of a few rather homogeneous types, clearly situated in a property space of a few important dimensions. A sound typology forms a solid foundation for both theorizing and empirical research. Perhaps no other tool has such power to simplify life for the social scientist. (Bailey 1994, 33)

In essence, we want to move from being able to say that everyone has their own way of tackling methods teaching, to being able to say that there types of practice that we can identify through research, categories constructed through a combination of data analysis and theoretical knowledge.

Typologies have been constructed in education for different purposes including to map types of education provision (Rix and Twining 2007), to clarify approaches to particular education challenges (Rao and Stupans 2012) and to classify the impacts of education interventions (Lunsford et al. 2013). Typologies rely on the ‘reduction’ of data and concepts (Lewis-Beck 1994, v) to identify parts, properties and their dimensions (Kluge 2000). This process usually involves arranging ‘entities into groups’ that are internally homogenous but distinct from each other (Bailey 1994, 1). Importantly, unlike taxonomies, typologies are understood to be non-hierarchical. Categories should relate to one another rather than being dominant or subsidiary (Ayres and Knafle 2008). This fit with our desire to resist an evaluative perspective that values some aspects of teaching above others in ways that might subject participants to normative scrutiny. A sophisticated typology of pedagogy requires engagement with teachers to capture nuance and diversity and ensure that any abstraction and categorisation makes space for these.

Making sense of the social environment through classification is the basis of what we do as social scientists (Lewis-Beck 1994; Bowker and Star 1999). Classifying is also what we do as human beings (Bowker and Star 1999), but we argue that it is important to move from informal classification of pedagogies – as good or bad, effective or ineffective, possible or unrealistic – to a typology that would be useful both as a ‘descriptive tool’ (Bailey 1994, 12) and a thinking tool. Typologies in education research have been developed conceptually from the literature (see, for example Hoggan 2016) or are prospective for empirical testing (see for example Rao and Stupans 2012). Often though, classifying practice to form a typology has both a conceptual as well as empirical dimension (Bailey 1994). There have been classifications of methods/higher education pedagogy as student-centred versus teacher-centred (Barraket 2005), surface versus deep learning (Haggis 2003) etc, but it is rare though to find classifications presented bounded by a specific pedagogy as Alexander (2018) offers for dialogic teaching in primary education. In the area of research methods teaching, where expertise often resides in the discipline or research method rather than in pedagogical practice, a typology of research methods pedagogy has not been attempted before and necessitated empirical, reflective and conceptual work.

Methods

We have constructed the typology using data generated from a bespoke combination of methods within one study involving teachers of quantitative, qualitative, mixed and digital methods. The Pedagogy of Methodological Learning study was conducted in two phases from 2013 to 2017 as shown in Table 1. At the heart of the study was the principle the research itself could be developmental for those involved. This meant that the methods – like the participants – needed to be in dialogue (see Nind and Lewthwaite 2018a). In the research design the starting point for the dialogue was a conversation using expert panel method. Panel participants were leaders in the sense of having extensive teaching experience and commitment to reflection (Lucas and Claxton 2013) and experts in that they had garnered ‘significant experience over time of advanced methods teaching at a postgraduate level’ (Lewthwaite and Nind 2016, 417); they had often also written methods textbooks or papers on pedagogy for methods teaching or had senior transnational roles teaching research methods. Because they could offer rich and diverse perspectives we went to them to begin a series of dialogues before observing teaching in situ. Here, as in other papers, experts are named with their consent, as anonymising them would be impossible. Nationally (phase 1, UK) expert participants comprised: Andy Field, John MacInnes, Malcolm Williams, Julia Brannen, Pauline Leonard, Pat Sikes, Harry Torrance and Amanda Coffey. Internationally (phase 2), panellists were Andrew Gelman, Anne Porter, W.Paul Vogt, Chris Wild, Sharlene Hesse-Biber, Pat Bazeley, John Cresswell, Manfred Max Bergman, Richard Rogers, Bagele Chilisa, César Cisneros-Puebla, Johnny Saldaña and Yvonna Lincoln.

Table 1. Participants, methods and dataset.

Starting the dialogue: Expert panel (transcripts and online data)
Phase 1 October– November 2013		Phase 2 May–July 2015
Interviews with 8 UK experts (3 quantitative methods, 3 qualitative methods, 2 mixed methods)	n.8	Interviews with international experts (4 quantitative, 4 qualitative, 4 mixed methods, 1 digital methods, spanning USA, South America, Europe & Africa)	n.13
		Phase 2 July–September 2015
		Online expert forum	n.4
Deepening & broadening the conversation: Focus groups with teachers (transcripts)
Phase 1 October– November 2013		Phase 2 April–October 2016
Focus group 1: Teachers of qualitative methods, natural grouping from 1 institution and department	n.3	Focus group 4: Teachers of qualitative, quantitative and mixed methods, new grouping from 1 institution but different departments	n.5
Focus group 2: Teachers of quantitative methods, natural grouping from 1 institution and department	n.3	Focus group 5: Teachers of qualitative, quantitative and mixed methods, new grouping from universities and social research organisations	n.7
Focus group 3: Teachers of narrative methods, natural grouping from linked set of colleagues	n.8	Focus group 6: Conducted via Skype, new grouping of teachers of research methods online	n.2
		Focus group 7: Conducted via Skype, new grouping of teachers of research methods online	n.3
Moving closer to the action Video stimulated dialogue (focus groups with teachers & learners) (transcripts and observational fieldnotes)
Phase 1 February- April 2014		Phase 2 April 2016–May 2017
Re 1-day course on an aspect of multi-modal analysis	n.13	Re 1-day course on an aspect of survey design	n.10
Re 1-day course on using a computer-assisted qualitative analysis software package	n.5	Re 1 day of 3-day course on data linkage	n.7
Re 3-day course on multi-level modelling	n.7	Re 8-week course on ethnographic technique	n.6
Re 2-day course systematic review	n.7
In-depth case studies (Phase 2 only, March–July 2017) (transcripts and detailed observational fieldnotes)
		1: Teaching ethnographic technique
		2: Teaching advanced computational quantitative methods

Following individual semi-structured interviews, transcripts were thematically analysed and, in Phase 2, the experts were brought into dialogue with each other in an online forum to discuss emergent themes (see Lewthwaite and Nind 2016). The interviews probed the participants’ pedagogy, including theories and experiences that had influenced them; the culture of paradigms and pedagogies, including sociocultural and geopolitical factors; innovation in methods and data landscape; and innovation in teaching. The topics pursued in forum dialogue were favoured approaches, the roots of the pedagogic practice, use of pedagogic resources, pedagogic challenges, national and international contexts, and controversies and gaps in pedagogic culture.

To deepen and broaden the conversation we added focus groups with methods teachers in varied UK contexts (university, social research organisation, teaching face-to-face and online) to find out whether the emergent themes from the experts resonated with practitioners, and what else was going on. The topic guide for the focus groups covered the same topics discussed in the expert forum, and ended with a discussion of approaches, strategies, tactics and tasks, probing whether these were concepts that were core to our emerging typology were ones they would use or could identify with. Both these data generation methods probed knowledge about action in that the data represented narrative accounts of practice and the thinking behind that practice.

To get closer to ‘knowledge in action’ (Nind, Kilburn, and Wiles 2015, 564) we used video stimulated recall, reflection and – most important – dialogue. This involved observing and video-recording a diverse selection of methods teaching (see Table 1) and using excerpts from the recording to stimulate dialogue between teachers, learners and researchers about the pedagogy that had just played out. This method held a mirror to the minutiae of practice and process with discussion about such topics as: ‘did you try anything new today? Does that relate to conscious decisions you’ve made to do it that way?’ and ‘Are there particular points in the day – teaching moments, learning moments – that you would like to review and discuss?’ This created data that was grounded in what we had all observed or experienced.

The iterative nature of this process meant that as the analysis gained momentum our vision of the typology emerged and became enriched as the dataset grew. To test out how the typology held up and to explore pedagogic themes in detail and in context we ended with two contrasting in-depth case studies. One involved doctoral researchers learning about ethnographic technique over a number of weeks. The other was a more intensive summer school for researchers wanting to apply advanced computational quantitative techniques in social research. In these we observed (for 35 h) and talked with teachers and learners about their practice thereby generating rich fieldnotes as well as interview transcripts.

Data analysis in the study was primarily a thematic, flexible and iterative approach to identifying patterns of meaning in the data, a ‘way of seeing’ and ‘making sense of’ (Boyatzis 1998, 4) data. We looked for themes in participants’ experiences, accounts of their practice and rationales for action that we could further explore in successive stages of data generation and analysis. The analysis of professionally transcribed data began with listening to gain familiarity, pre-coding (Saldaña 2016) and memoing thoughts about what the data were telling us. We then independently engaged in a process of inductive, free coding (what Boyatzis (1998) would see as recognising the codable moments) before coming together to refine codes and agree categories into a harmonised codebook. This was a key stage for constructing themes, not as found gems but as ‘meaningful entities’ (Braun and Clarke 2016, 740). Phase 1 clarified leitmotifs such as the challenges involved in teaching research methods. In phase 2, categories and codes were developed in tandem rather than sequentially as the conceptual ground was already familiar.

Data were initially hand-coded in MS Word, moving to using NVivo (versions 10 and 11) when we had sufficient feel for the data. This software, chosen for its familiarity to us, facilitated the management, linkage and retrieval of data within our substantial dataset. As we moved into analysis of the focus group data in Phase 2, we affirmed the main categories we would use for the typology and we could begin to see the layers we were coding as core categories: approach, strategy, tactics, task. Hypothesis coding enabled us ‘to confirm or disconfirm any assertion or theories developed thus far’ (Saldaña 2016, 171); by returning to using the facilities of MS Word, we colour-coded the categories of the typology in the transcripts to create visual reference points. This illuminated where there was sometimes a gap and some patterns also began to be visible, such as how the nature of the categories of the typology differed for the teaching of quantitative methods and qualitative methods. This prompted the analytic process of seeking out counter-evidence to prevent misreading patterns in the data.

While the horizontal analysis (Borkan 1999) cut across the dataset, we also conducted more vertical analyses of single transcripts to retain a sense of the conversational moves and of the individual, dyad or group constructing – and making sense of – their teaching practice

Findings

It was in Phase 2 that we constructed the themes of pedagogic approach, strategy, tactics and tasks in the process of organising and interpreting codes and categories. These themes became the structuring concepts for our typology – our categories and essentially the typology itself. We understood that the militaristic origin of ‘tactics’ sat uncomfortably with some teachers in the study – but for us, the notion of a tactical response, that articulates strategy in a dynamic, field situation, is not adequately expressed by alternatives such as technique, which has more performative connotations.

Our typology classifies pedagogy in four core categories from the abstract to the specific – a series of linked layers getting closer to philosophical foundations at one end and to classroom action at the other as shown in Table 2.

Table 2. Typology classifications.

Category	Characteristic
Approach	How the teacher goes about their pedagogic work in a way that coheres around a theory, set of values or principles
Strategy	Goal directed planning for implementing an approach
Tactics	Translation of strategies when the planning becomes procedural and specific to the context
Tasks	What learners (or teachers) are required to do, or actually do

Teachers often carefully think through their approaches and, regardless of how they name them, their approaches relate to their pedagogic aspirations or to their identity as a particular kind of teacher. Strategies similarly cohere around a purpose: They may be, for example, strategies to motivate and engage, to manage cognitive load, or to facilitate reflection. Methods teachers have strategies concerning where to start and how to hook learners in based on an identifiable rationale, and they have tactics for managing the realities of the various pedagogic situations. It is the set tasks, however, that are most transparent and accessible. This is evident in Dawson’s (2016) book, 100 Activities for Teaching Research Methods. Tasks may be more or less strongly framed in relation to their strategic or tactical function and even retained or dropped dependent on trial and error.

The expert panel was particularly useful for uncovering the values underpinning the practices of these experienced, thoughtful teachers and therefore most effectively highlighted the approaches and their flow into strategies. Experts sometimes also used re-voicing, rehearsal or re-enactment of teachable moments as illustrations, showing how tactics are responsive to situations as well as pedagogic values and methodological priorities. Focus group participants were concerned with the gritty realities and so illuminated particularly well the middle ground of the strategies and tactics. By their very focus on the classroom or technology lab, the video stimulated dialogue worked out from tasks to make visible the tactics and strategies behind them. The case studies brought the holistic quality they are renowned for (Stake 2000). The process of defining approach, strategy, tactics and tasks happened over several iterations of going into the data and stepping back from it. We were conscious of assigning symbolic meaning additional to that intended by the participants as we gathered a picture that was wider and deeper than that which any of the individuals could have.

We checked our understanding of our definitions of pedagogic approach, strategy, tactics and task by each of us independently applying them to one particularly rich expert interview transcript. We also identified inclusion and exclusion criteria and examples from the data that matched the definition and those that were close but not quite to clarify our thinking (as suggested in Saldaña 2016). This is shown in Table 3.

Table 3. Coding principles and definitions for typological categories.

Category	Characteristic	Coding principles
Approach	How the teacher goes about their pedagogic work in a way that coheres around a theory, set of values or principles	Including: established named approaches (e.g. active learning) or emergent unnamed approaches (e.g. visual approach to statistics) Excluding: collected strategies that do not cohere in some clear way Close but not quite: reference to an approach that is one of several strategies employed as part of bigger approach
Strategy	Goal directed planning for implementing an approach	Including: strategies to motivate, engage, manage cognitive load, facilitate reflection, select which data to use as a teaching resource (depending on their approach, purpose and priority) Including: starting points and pedagogic hooks, often with a rationale Excluding: where the purpose/goal is unclear to the teacher Excluding: more generic/abstract descriptions (likely to be approach)
Tactics	Translation of strategies when the planning becomes procedural and specific to the context	Including: context-specific decisions about teaching activities
Tasks	What learners (or teachers) are required to do, or actually do	Including: activities within the pedagogic context that may or may not have pedagogic value or purpose

We have imposed tight definitions rather than use participants’ everyday terms, which, like the use of the terms in the literature, could be quite loose and lead to conceptual confusion. A series of dialogues with teachers about the data and the definitions helped to validate them, and assure us that this process was not distortive. From reading the dataset we argue that strategies and tactics relate directly to the perceived challenges and that teachers face and their aspirations. For example, in focus Group 2, the experienced qualitative methods teachers shared an aspiration to ‘try to ground what you’re doing in what they’re [the students] doing’. This student-centred approach was articulated through the strategy of teaching using students’ own data. However, the focus group also recognised that ‘if you ask people to bring their data, they often bring massively inappropriate amounts, or sometimes there are ethical issues in what they bring’. They therefore developed the tactic of seeing and discussing the learner data in advance of the class, prior to mobilising data-based tasks. Teachers’ accounts of their practice data show how they hone their strategies as they gain experience and how they adjust them in light of changing technology.

Teacher tactics can be impromptu, responsive to new challenges and include an element of risk-taking. In one of the video-stimulated dialogue focus groups for instance, Nadia, who had been teaching the application of a qualitative data analysis package, described how spontaneously drawing a diagram on the whiteboard was something she not had not done before. ‘I wasn’t planning to do that, it was just because of the conversation we were having, I thought okay, this is what this is isn’t it, … but I wasn’t really sure how it [the diagram] was going to turn out’. One of the learners confirmed the utility of the tactic, observing ‘it really helped clarify things for me’. In expert interviews, Sharlene Hesse-Biber highlighted the questioning tactics that she deploys in qualitative and mixed methods teaching. She re-voiced many of dynamic ways she prompts, probes and responds to student thinking to elicit reflection, using specific examples and vignettes in the interview.

And they [students] go, ‘[…] my professor was a positivist, I didn’t know there was something else out there, I guess I’m a subjectivist’. I go ‘well, what kind?’. They go, ‘I didn’t know they came in flavours’. […] It’s a teachable moment.

For Sharlene Hesse-Biber these tactics were situated within an overall commitment to a critical feminist teaching approach that engages reflexivity and standpoints. The tactics sit among strategies such as ‘experience sampling’ and they link to tasks such as student research journaling.

The grounding of the typology in research data generated a kind of matrix connecting the categories related to important approaches found in our observational and reflective data as shown in Table 4. There is considerable overlap in the literature between the pedagogic approaches of student-centred, active and experiential learning in terms of ways of working (see e.g. Hsuing 2008; Galliers and Huang 2012) and this was apparent in how they were evidenced in the data. The matrix is intended to be illustrative, not exhaustive. The typology is an overarching frame that can incorporate new approaches, strategies, tactics and tasks as these are developed to meet (for example) new methodological challenges.

Table 4. Example of typologically generated matrix of methods teaching.

Approach	Strategy	Tactics	Tasks
Student-centred Putting the student at the heart of pedagogic decisions	Teachers start with where the learners are comfortable and motivated, using themes that connect with the group Teachers show relevance of data and methods to students’ professions/disciplines	Teachers pick up on anything in class that indicates what is meaningful to students personally Build a common vocabulary Use expertise in the room	Students work on their own data Students critique papers in their own discipline
Active learning Valuing learning by doing and application of knowledge	Alternate lecture and exercises so students apply what they hear and learn by doing Work through the statistical knowledge and the software simultaneously	Teachers use learning glitches to reinforce key concepts Teachers often choose their own data for exercises so they can be responsive to queries about it	Students work hands on with any data as long as they can ‘have a play’ and gain ‘flying time’
Experiential approach to teaching qualitative methods Valuing the power of authentic experience	Require students enter the field And attend to the sensory	Teachers exploit opportunities for reflexivity	Students embody data by reading it aloud
Problem-based learning approach Valuing the motivational benefits and cognitive process of approaching tangible research problems	Require students to respond to analysis of research needs by devising projects or action to solve them	Teachers provide support as needed: time for mapping the challenge, collaborating peers, an audience for ideas, tools and materials	Students work collaboratively and intensively on the problem (in parts and holistically) and present their solution
Standpoint-led approach to teaching qualitative methods Valuing reflexivity and critical engagement	Bring teachers’ and students’ standpoints to the foreground for examination	Teachers encourage dialogue and model reflexivity	Students reflect upon, articulate, share and defend standpoints
Visual approach to teaching statistics Valuing the power of the visual, putting it up front	Teachers use visual scaffolds to reduce the cognitive load	Students see data-related things quicker	Students work with visual metaphors and visual software
Verbal approach to teaching statistics Cohering premise that learning stats is like learning a foreign language	Teachers make the concepts understandable and backfill technical skills later	Teachers translate between statistical and non-technical terms	Students develop glossaries

Some approaches in Table 4 will be more familiar than others. Familiar approaches (e.g. active learning, experiential learning) have attained a certain ‘level of emergence’ (Foucault 1972, 186) in pedagogical discourse, arguably meeting thresholds that allow them to be recognised at a community level as meeting ‘norms of verification and coherence’ (see Foucault 1972, in Olssen 2006, 22). Within our data, we observed approaches that have been individually developed, and were at an earlier level of emergence. These are functional for that teacher, but yet to be realised at a community level or fitted into wider pedagogical discourse. Given the lack of pedagogic culture in social science research methods, and often the individualised experience of developing methods teaching in the social sciences, this is unsurprising.

Strategies evident in the data largely reflected the teachers’ approaches. For example, favouring active learning meant that Nita, a teacher of narrative methods didn’t mind which data she used in her teaching as long as students could ‘have a play’, play being the key strategy and active learning the dominant approach.

Vertical analysis of the interview data with members of the expert panel often illuminated an ease with the pedagogic rationale for their practices. Johnny Saldaña, a qualitative methods expert spoke of his teaching emerging from his perspective and ‘signature beliefs’. Believing that ‘we teach who we are’, he explained

Because I’m a theatre and drama educator, my profession demands that we be on our feet for studio work, and so again I transfer that same pedagogical practice into my research methods classroom. On-your-feet work might consist of such things as … improvisation perhaps … role play.

Adopting a student-centred approach, he explained, meant he was ‘conscious of trying to make my own teaching as relevant to my students and participants as possible’, ‘and so I try to find themes that may cut across as many different audiences as possible’. He outlined how, ‘I feel I have to start from ground zero to make sure that we’re all sharing common vocabulary…’. Johnny Saldaña could move swiftly from articulating a student-centred approach to the associated task: ‘advanced methods require authentic data to be collected by the students to gain a sense of ownership’; moreover there was purposeful strategy in this, ‘the fact that they’re using their own data gives them a sense a personal ownership, and so they’re better able to analyse the data, because it comes from them’. Another of his strategies in teaching qualitative methods was using particular film clips: ‘We show that clip because it is exactly parallel to what qualitative data analysts try to do with their own data corpus’ and he identified related tasks such as, ‘I take for example Kinsey [2004, dir. Bill Condon], … I show the first 90 s … Kinsey talking to his research assistant about how to interview’. Saldaña’s creative, arts-informed approach is also very experiential and he has a broad strategy of using embodiment. Specific tactics and tasks here include getting ‘students to read the data. …With talking the data you get to embody it right, you take cognitive ownership of it’. This one very experienced teacher showed an ability to move fluidly but coherently between approaches and their associated strategies, tactics and tasks in his talk.

The typology does not indicate that the starting point for teachers has to be an approach. In interview, Richard Rogers, a digital ‘quali-quanti’ methods expert, described his digital methods summer schools starting with outlining his strategies in which groups of students hear from subject matter experts on their analytical needs within their fields and how internet analysis might add something; ‘The group then operationalise the subject experts’ analytical needs into sort of research projects for groups of students to tackle.’ The interviewer engaged in a form of scaffolding to help this teacher articulate the approach or strategy behind this, suggesting ‘it appears to me that the teaching is kind of blurring the line between real world active research and the kind of sandpit of the learning environment’. On affirmation of this, she offered a pedagogical label of problem-based learning approach that might fit what has been described. While unfamiliar with this pedagogic language, the concept helped Richard Rogers to reflect and articulate further, first reiterating the students’ task in addressing ‘some sort of real-world problem’ and then addressing its purpose, what it is a tactic for doing. Analysis of the dialogue shows that questioning about ‘working approaches’ enabled the process of making explicit the strategies that had seemed ‘hard to know’. This gestures both to the implicit nature of much pedagogic knowledge and to the importance of creating a typology in which emergent pedagogic knowledge can be recognised as foundational for individuals on the way to entering wider discourses.

Our reading of this participant’s pedagogy is that this is someone with expert in-class tactics, (a hackathon and data sprint), and honed tasks, (covering the full range of an authentic digital research project activity). Taken together, these articulate implicit strategies that take the students through a steep learning curve, expressing an approach that resembles problem-based learning and that holds transcultural learning and interdisciplinarity as a core value. In sum, it is an active and immersive approach built around working authentically with data and it shows the implicit nature of teachers’ craft knowledge and pedagogical content knowledge.

Exploring the typology in relation to the teaching of qualitative (and mixed) methods more widely the picture is dominated by approaches that are student-centred, active and experiential, employed strategically to facilitate deep, ethical engagement with methods, standpoints and data. Teachers of qualitative methods affirmed the conceptually difficult content of their teaching, particularly facilitating and provoking necessary reflexivity. Examples include the consensus among teachers in focus group 1 that the best courses have ‘lots of space for people to talk about their own research and tie it … whatever the focus is, to their research’, and that this ‘will include a sort of reflexive moment, thinking about their position within that’. They observed too that, ‘Sometimes really simples exercises can be effective’, even ‘mind-blowing’. Similarly, Amanda Coffey, a qualitative expert interviewee argued that ‘you cannot teach someone to become a qualitative research practitioner, actually to be able to do it and do it well, without them actually practicing’; ‘we have to get them into the field’. This view was echoed emphatically by Yvonna Lincoln in arguing, ‘you can’t teach fieldwork methods as a theoretical course’. Within a US context, Lincoln endorsed the idea that teaching required finding ways for the learners to ‘inspect their own interests in a very deep way, before they choose a lens’, which might be an indigenous lens or another examined standpoint. Bagele Chilisa, a qualitative expert based in Botswana, argued that concepts are learned through application, students must ‘apply what they have learned’. Experiential approaches were associated with strategies for attending to the sensory, the tacit and the embodied.

The teachers of quantitative methods in the study often foregrounded their tactics for addressing widely perceived anxiety among students. This influenced their choice of data and disciplinary examples to work with, leading them towards towards student-centred approaches combined with humour and enthusiasm. Chris Wild, expert interviewee from New Zealand had devised a strongly visual approach and W. Paul Vogt from the USA had developed a ‘translational’ approach, using non-technical language. These approaches were designed to make difficult content accessible (see Nind and Lewthwaite 2018b). In this vein, strategies included chunking material, complexity reduction strategies such as bootstrapping (a statistical operation also used as a pedagogic imperative (Wild et al. 2017)), removing obstacles, stream-lined software, translating terms, backfilling with technical skill and scaffolding concepts.

Some participating teachers did not identify their teaching in terms of any known teaching approach, though from other things they said the approach might be interpreted as one of active learning, experiential learning, student-centred learning, peer/interactive/collaborative/dialogical learning, problem-based or independent learning. When participating teachers did define their pedagogic approach in pedagogic language, the labels of experiential, active or interactive learning were used. The collaborative dimension to learning was important to teachers of qualitative, digital and mixed methods, but learning within quantitative methods was more often individualised. Some methods experts articulated a whole approach, with or without reference to theory, but representing a holistic response to the many challenges. One example was the dialogical standpoint-led approach discussed by Yvonna Lincoln, Sharlene Hesse-Biber, Bagele Chilisa and teachers in focus Groups 4 and 5. This approach was accompanied by tasks of articulating and defending standpoints (see Table 4).

The strategies that participating teachers depicted spanned strategies to structure the sequencing of content, to build bridges into new knowledge and skills, to balance breadth and depth and data and theory, and strategies to maintain engagement. Within these, they had tactics for finding out about student needs (judicious questioning including probing, prompting, ‘experience sampling’ and polling) and connecting with their interests (chiming in, storytelling, data-based vignettes, research examples). They had tactics for making data and methods relevant (seizing opportunities as they arose, ‘teachable moments’), and for enhancing understanding (modelling decision-making in action). Many tactics became most apparent through case studies, where they could be observed by the research team. In interviews and focus groups, reporting these tactical activities in the abstract is difficult, since tactics are immediate and responsive to feedback in situ. Through case study, for example, rhetorical devices that express tacit pedagogic knowledge become visible.

In observation, Larry, a teacher of computational social science demonstrated rich rhetorical pedagogical strategies and tactics. In his classroom talk threshold concepts¹ were outlined on the way to more advanced and complex material, ‘It’s very important to understand that basic point’. Difficulty and confounding factors were highlighted: ‘It’s very difficult to identify…’. Larry used specifying language to suggest overall trends that allow confidence in the methods chosen, ‘With very few exceptions…’, and he gestured to the influence of different international contexts. Perhaps most importantly (across all these indices), Larry explicitly modelled ways of thinking: ‘You can think about [X] by [doing Y]’, ‘This is a good way to think about the research design …’. This strategy shows how he ascribes value as an active researcher and teacher, modelling where reflection and attention should be placed, ‘You must always ask yourself …’. Taken together, these examples show how an approach that is nominally didactic in exposition-based teaching (the quantitative lecture) is rich in pedagogic strategies and tactics.

Discussion

While the categories in an ideal typology are mutually exclusive with total coverage in a system that is complete, we recognise the need for ‘dynamic compromise’ (Bowker and Star 1999, 55) in classifying teaching and learning. In the process of coding, and in talking with teachers it has sometimes been useful to think in the broader categorical terms of approach-strategy and strategy-tactics because of the way that each permeates the other. In this respect, our categories are not mutually exclusive. Teachers and researchers wanting to use the typology are encouraged to build in the flexibility they need. The porous nature of the typology we have generated is intentional. It allows for fluidity between its parts and also works against creating standardised expectations. This is important, as any typology will valorise some cases, categories or points of view and obscure others (Bowker and Star 1999). Even a descriptive typology soon takes on prescriptive qualities if the categories become naturalised into what people should do. In weighing all these factors, we have retained the simplicity of the central nomenclature of our categories as it provides a structure that has utility. This is to reduce complexity so that comparisons and relationships can be explored and made visible (Bailey 1994; Bowker and Star 1999; Kluge 2000). In the field of pedagogy, it should give practitioners conceptual tools to think with and a language with which to share practices. This language needs to have texture and depth as well as some degree of precision and simplicity to allow it to travel well, so that pedagogical content knowledge can be more readily passed on and so that the pedagogical culture can build. In presenting the categories of the typology as layers from the abstract to the concrete, we have tried to articulate how the approach, strategy, tactics and tasks look for individual research methods teachers, for different types of research methods or particular challenges. The more data we draw in the more we can populate the typology with examples. Ultimately, the scope for combinations is vast.

Alternative formulations of approach, strategy and tactic in education

The typology presented here comprises a holistic combination of approach, strategy, tactics and task. This is new but the categories have been discussed previously in the pedagogic literature, singly and in combination. We discuss various usages here looking at parallels and disjuncture with literature, focusing on the meaning of the terms. Historically, Foshay (1975) positioned teaching strategies as the new thing in the 1970s. He noted the anomaly that ‘in education we are not engaging an adversary’ and argued that the term was inflated as educationalists need ‘only one teaching strategy. It is to induce a situation in which the conditions for learning are met’ (373). For Foshay, this one strategy needed to be supported by multiple teaching tactics. Such pedagogical conceptualisations have often echoed militaristic use of the terms, with strategy related to deployment of resources to meet objectives and tactics seen as smaller movement. More recently, in research methods, Silver and Woolf (2015) have seen the relevance of military analysis in Luttwak’s (2001) fulcrum between strategies and tactics for teaching qualitative data analysis and using supporting software packages. Silver and Woolf’s model positions strategies as ‘what you plan to do’, which may be iterative and emergent, and tactics as ‘how you plan to do it’ with the software (535); they stress the need to attend to both.

In terms of the language we have selected we notice, as Garcia (1989) argues, that in practice teachers have used approach, methodology and technique interchangeably. Acero, Javier, and Castro (2000) find most classifications in the literature have approaches as most general in nature, encompassing the overall orientation or viewpoint on it, with methods and techniques positioned as more specific parts of this. In contrast to our position, some academic writers move between educational theory and strategy without depicting an approach at all. For example, Brown (2004, 77–78), refers to the Adult Learning Theory, Transformative Learning Theory, and others as ‘theoretical perspectives’ which ‘are interwoven with the pedagogical strategies of critical reflection, rational discourse, and policy praxis’.

Tangible support for interpreting pedagogic approaches into action is evident in the grey literature surrounding areas where the pedagogical culture is strong, such as the field of Teaching English as a Foreign Language (for example Telfpedia²). In e-Learning, Salmon (2005), discussing the translation of pedagogies for online contexts, suggests that strategy builds from where we are and what we know. She stresses the value of surfacing teachers’ strategy and tactics, unpicking that which appears natural to build confidence in new pedagogical contexts. This resonates with our determination that a typology must be of use to methods teachers. Elsewhere, strategies are seen as being about ‘various aspects of sequencing and organising the content, specifying learning activities, and deciding how to deliver the content and activities’ (Dick, Carey, and Carey 2001, 184), but they are linked to the big theories of behaviourism, cognitivism, constructionism, constructivism and so on.

Within open and distance learning Goodyear (1999) presents a pedagogical framework which has similarities with our typology, but lacks the grounding in research. In this formulation activities and tasks sit within the educational environment. Working from the specific to the general he positions the learning task as ‘a specification for learner activity’. This is designed based on ‘the best of what we know about how people learn, on a deep knowledge of academic subject matter and/or vocational competences, and on knowledge of the learners’ (n.p.). Above these procedural or operational levels in this framework sit the more conceptual ‘philosophy’ and ‘high level pedagogy’ (exemplified by what we might call the approaches of ‘guided discovery learning’ and ‘problem-based learning’). Goodyear illuminates the need to turn philosophy into action and, like many writers, stresses the role of goal-directed strategy as the link. For Goodyear though, ‘The only difference between pedagogical strategy and pedagogical tactics is one of grain size. Tactics are the detailed moves through which strategy is effected’ (n.p.).

Again the need to balance the conceptual world and lived reality is important. Goodyear argues that in the real world, strategy does not always determine tactics and is not always driven by philosophy, suggesting a classification whereby the types are loosely coupled. Apart from these examples, and more often through the literature, examples show how the concepts of pedagogic approach, strategy and method are used loosely or interchangeably (e.g. Lewis and Dehler 2000; Perry and Paterson 2005). While Goodyear’s framework is frequently cited and used in e-learning, it has reached little into wider pedagogical research. Our typology is more careful with language and we propose it as a conceptual-empirical tool for methods teachers to work with.

Evaluating the typology of research methods pedagogy

At best, typologies are intuitive and understandable, explanatory and principled (Bowker and Star 1999). However, these implicit elements are not guaranteed in research contexts where conceptual-empirical typologies are hard to design. Gregor (2006) argues that the evaluation of the success of a typology should be based on ‘category labels being meaningful, the logic of the dimensions being clear and the ability to completely and exhaustively classify being demonstrable’. This definition, generated within Information Systems, is less plastic than our usage. Approach, strategy, tactics and tasks may be exhaustive in one sense, but there will be more specifiable pedagogic approaches, strategies, tactics and tasks than we have included in this paper. While the range of contexts we could explore in the research was limited, the data come from diverse pedagogical actors drawn from multiple institutions, paradigms and backgrounds. The data generated through multiple methods was sufficient for the analytic process to identify the four key processes of approach, strategy, tactic and task around which we are currently fostering engaged pedagogical dialogue in workshops and seminars with methods teachers. The process has enabled us to understand the main approaches evident among around 100 research methods teachers in various social science disciplines as well as what is important about them. It has been possible to elucidate associated strategies, tactics and tasks for how these translate in teaching quantitative, qualitative, mixed and digital methods. At an individual level there may be gaps, with teachers unable to articulate their approach for example. Exposing these gaps can be constructive and by working at a communal level, across methods teachers, a whole picture is generated that is more complete. The typology then may be sufficient without being exhaustive.

The meaningfulness and logic of the typology has been explored within the research through its iterative nature. Vincent, a teacher of ethnographic technique in case study 1, could identify with the in-class, quick thought nature of tactics, connecting this with ‘busking’ or ‘relying on embodied expertise … to know how to judge a class’. Steve, a teacher of quantitative methods in education in focus group 4, used the typology meaningfully when he spoke of the value of thinking beyond the task, ‘the what, and the how’ to ‘looking at why we would teach these in this way, and why would we sequence it in that way’. Thelma, a qualitative methods teacher in health, though, expressed some discomfort with tactics as a label, ‘it always sounds a bit malevolent to me, “tactics”. It’s trying to do something without people knowing’. This focus group concurred with Melanie, who, when facilitating, admitted that the categories of the typology have ‘blurry edges’. Yet Steve stressed the usefulness of the typology for thinking about ‘how we would help realise a particular strategy’ and Nita recognised that ‘the approaches [are] what you have internally in your head’ and the importance of retaining these in challenging teaching contexts. Nicci, who was responsible for the research methods training of health professionals and part of focus Group 5, used the typology to reflect, ‘where does my overall orientation, or approach, interface with tactics that I need, or strategy that I might need to use because of context, student members, etcetera’. While the concepts were meaningful to this group of methods teachers working inside and outside academia, Amy, who had crossed disciplines and methods, summed up the desire to retain dynamism in any typology: ‘what people have been describing here is something that is much more fluid and it’s contingent’.

Dialogue with participants showed the importance of the typology not just being read from general to specific, that teachers using it do not always start with their approach and end with the task. There was recognition of the role of the content for shaping tactics and tasks with values influencing their approaches. Similarly, there was recognition that years of teaching meant building up a useful ‘bank of ideas’, but that there were dangers if drawing on this became unreflective. One experienced teachers reflected that ‘a less experienced teacher I think would gain from being much more specific, much more aware of that typology and apply it in a very kind of an intentional kind of fashion’.

Conclusion

We have presented a typology of research methods pedagogy that has been generated from research with teachers. It was created out of analysis of dialogic and observational data and we have checked back that the categories make sense for teachers. The typology came out of – and can be related to – diverse contexts and perspectives. It offers ‘situated knowledges’ whilst also abstracting from what teachers do and value. We contend that the conceptual (logic) and data-generated (empirical) nature of the typology invites use and offers rich grounds for individual pedagogic reflection and development. Any static typology (or categorical, classification system) fossilises (Bowker and Star 1999) as frontiers of knowledge change. Users of typologies frequently subvert formal category schemes, using work-arounds (Atran 1990) and informal, ‘vernacular (folk)’ classifications (Bowker and Star 1999, 54). Moreover, where people and behaviours are described, ‘looping effects’ (Hacking 1995), whereby the subjects of the typology resist or negotiate the ascriptions of typology, can render a typology obsolete. It is for these reasons that we have kept the core of the typology simple, arguing that the approaches, strategies, tactics and tasks need to be generated afresh by teachers of research methods in new digital contexts for example, or working with the changing data landscape.

In light of the debates about classification, we recognise that the typology might be sufficient for a moment in time in the pedagogical, methods and data landscape, but ultimately that this moment may pass. For now, it can be used to prompt deep pedagogical thought, to spur dialogue and to invite refinement. Using the data, we could illustrate each part of the typology for a multitude of research methods and teaching contexts, but this is not the point. The danger in such an endeavour is that it is trying to catalogue that which cannot be catalogued or contained; it is that the descriptive could leak into the prescriptive. We have seen though that the categories of the typology both capture and stimulate pedagogical thinking. We are not arguing that methods teachers have never previously distinguished between approach, strategy, tactics and task, or done so sufficiently. Rather we maintain that there are benefits to thinking with these concepts when planning, doing and reflecting on such teaching. Teachers exposed to the typology have enjoyed grappling with applying it to excerpts from our data or to their own situations; the porous quality and inter-relational aspects facilitate critical engagement. This paper will, we hope, lead to far-reaching pedagogic work of this kind.

Both Bowker and Star (1999) and Foucault (1970) (in his work on genealogies of knowledge and the social sciences) identify and discuss the power that categories and organising systems of thought exert upon people. We have already described the ‘perils of reification’ (Bailey 1994, 15) and we recognise the risk that each category ‘valorises some point of view and silences another’ (Bowker and Star 1999, 15). We have sought to temper these issues through multi-method, dialogic methods that co-construct knowledge with our participants and, through case study, with our readers. The intention of generating a conceptual-empirical typology is to describe – and provoke – but not to evaluate and prescribe or proscribe. This typology represents knowledge moving ‘from practice to research’ (Levin 2013, 10) as much as the reverse.

Typological knowledge-work requires an active, reflexive negotiation of power and knowledge-making that accords with the ‘watching brief on modes of socialisation’ (Boyne 1990, 134) that Foucault encourages as he reflects on categorical influence in the social sciences. We have sought to avoid ‘inertia’ (Bowker and Star 1999, 14) by keeping the typology dynamic enough to engage new perspectives and practices in the future. Looking forward, this typology will benefit from engagement, and expansion to ensure continuing credibility. Important starting points for interrogating the typology of methods pedagogy will include richer and more textured multi-perspective ethnographic accounts of teaching in practice. These could articulate conceptual formulations of approach/strategy/tactic/task more holistically in the narrative of experience. Further to this, exploration of the typology through visual modes that facilitate exploration and express the interrelationships of categories (for example, where one task serves several pedagogic aims) and the value of blurring edges of the typologies categories could hold value, promoting more nuanced rather than consolidated understanding. Additional modes of expansion and exploration could be facilitated by social indexing, or collaborative ‘tagging’ associated with folksonomy, to ‘facilitate the introduction of emerging concepts’ (Luff, Byatt, and Martin 2015), alongside more conventional research. In sum, addressing the quality of social science research/methods cannot be fast-tracked through understanding, typologising and debating methods pedagogy, but a useable framework, which methods teachers can use to surface implicit and unnamed pedagogic practice for reflection and development, offers a significant step forward for methods education research and practice.

Acknowledegments

We thank our participants for their contribution to the knowledge shared here, and we gratefully acknowledge Daniel Kilburn and Rose Wiles for their research work in the first phase of this project. We also thank our critical readers Debbie Collins, Tony Kelly, Christina Silver, Raphael Susewind and Mike Wallace for their important formative feedback on the paper. This research was funded by the UK Economic and Social Research Council (ESRC) - National Centre for Research Methods grant number ES/L/008351/1.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was funded by the UK Economic and Social Research Council (ESRC) - National Centre for Research Methods grant number ES/L/008351/1.

Notes on contributors

Melanie Nind

Melanie Nind is director of the Centre for Research in Inclusion in the Southampton Education School and a co-director of the ESRC National Centre for Research Methods where she led the Pedagogy of Methodology Learning study. She is co-editor of International Journal of Research and Method in Education and editor of Bloomsbury Research Methods for Education book series. Melanie’s research interests lie in pedagogy, methodology, learning disability and inclusion, and particularly the intersections between these.

Sarah Lewthwaite

Sarah Lewthwaite is a Research Fellow at the Centre for Research in Inclusion, and the ESRC National Centre for Research Methods, where she is researching the pedagogy of innovative and advanced research methods. She holds a PhD in Learning Sciences (disability and social media) from the University of Nottingham and maintains a strong interest in the intersections between critical disability theory, web accessibility, social media and pedagogy. Sarah has over ten years’ experience in digital and inclusive research. She has published widely on education and accessibility. 

Notes

1. Threshold concepts (Meyer and Land 2005) represent those points of understanding that must be gained before a learner can transition into further knowledge.

2. http://teflpedia.com/ ‘A wiki for the English-teaching community to share knowledge’.