Identifying and promoting qualitative methods for impact assessment

Figures & data

Figure 1. An example of data coded using the hybrid deductive-inductive coding approach.

Table 1. Qualitative methods used in impact assessment (adapted from Walker et al. 2023, pp. 38–39).

Method category	Description
Deliberative methods	Rely on discussion-based approaches to engage the public in collaborative problem solving and decision-making (Gastil 2009). Deliberative methods are founded on the understanding that ordinary citizens provided with appropriate information and led through a process of deliberation can make sound, well-informed decisions (Wiklund and Viklund 2006). These methods aim for representativeness, including people with diverse backgrounds, viewpoints, and values. Three common deliberative methods include deliberative polling (Fishkin 2021), world cafés (Schieffer et al. 2004), and citizens’ juries (Crosby 1995).
Delphi method	A technique for systematically eliciting advice, and ultimately consensus, from a panel of anonymous experts through iterative rounds of questionnaires (Richey et al. 1985; Egan and Jones 1997). It is particularly useful where an evaluation is required but only incomplete information is available or where the experts are geographically distributed (e.g. Hanna and Noble 2015) or have diverse expertise in relation to the problem. A minimum of two rounds are conducted in Delphi studies (Richey et al. 1985), with additional rounds of questionnaires developed if consensus is not yet reached.
Document analysis	This was the most used method by the survey respondents (83%). It involves systematically identifying, selecting, evaluating, analysing, and synthesizing information about a specific topic from print and electronic documents (Bowen 2009; Savin-Baden and Major 2013). The information is usually organised into major themes, categories, and case examples through thematic or content analysis (Bowen 2009). Common types of document analysis include literature reviews, systematic reviews and meta-analyses, secondary data analysis, archival and historical research, and policy research (Tight 2019).
Focus groups	Identified by 30% of survey respondents as being commonly used in IA, focus groups are facilitator-moderated group discussions that explore participant experiences, perspectives, and opinions about a specific topic (Morgan 1996; Toth 2001). In contrast to research interviews, which elicit individually held knowledge, focus groups are used to understand shared and collectively held knowledge, and the range of perspectives held by certain groups or communities (Creswell 2007). Although a moderator guides the discussion, knowledge is generated through the conversation among focus group participants (typically 6–8) rather than through interaction directly between the moderator and individual participants.
Fuzzy sets	Fuzzy set theory enables the quantitative processing of imprecise qualitative data represented by linguistic variables (Zadeh 1965). This involves transforming qualitative, descriptive data into a form that can be mathematically described and manipulated in a rigorous way that accounts for the subjective nature of the descriptors. Fuzzy set theory is a potentially powerful tool for IA because linguistic variables that are open to different interpretations (e.g. severe/major/minor/negligible) are often used to describe the significance of impacts. Wood et al. (2007) used fuzzy set theory within IA to define degrees of significance of noise and visual impacts of a proposed wind farm development.
Interviews	The second most used method by the survey respondents (61%), interviews allow for in-depth exploration of individuals’ experiences, perspectives, and opinions. Interviews can be unstructured, semi-structured, or highly structured. According to our respondents, interviews in IA are most often semi-structured, combining structure with flexibility by using an established set of questions to guide the conversation, but also allowing for follow-up and exploration of new topics raised by the interviewees. In such interviews, the participant shares knowledge and experiences in their own words and the interviewer probes and re-directs discussion as needed (Morris 2015). Interviews can take place face-to-face, online, or via telephone.
Matrices	Matrices—grids that link system components with project activities—are among the oldest and most familiar methods for IA. Originally focused on environmental systems (Fischer and Davies 1973), their use has expanded to encompass social-environmental systems. Multiple matrix methods can be used in IA (Bisset 1980; Wathern 1984; Shopley et al. 1990). While these often rely primarily on expert judgement to determine the most relevant system components and conduct significance evaluations, participatory applications have been developed to include a wider range of values and perspectives (e.g. Satterfield et al. 2013; Nchanji et al. 2017).
Multi-criteria analysis	Multi-criteria analysis, or multi-criteria decision analysis, is a family of mathematical techniques that support decision-making by assessing and aggregating performance of options (such as alternative development proposals) against multiple, often conflicting, criteria (Dodgson et al. 2009; Geneletti and Ferretti 2015; Te Boveldt et al. 2021). Participatory or qualitative approaches may be used to collect and integrate qualitative data into the analysis. Such approaches are helpful in the weighting step, which involves assessing the relative importance of each criterion and allows for integration of the subjective views and values of different stakeholders, and allows for sensitivity testing (of the sensitivity of conclusions to different value-sets).
Narrative methods	Narrative research involves the telling and interpretation of people’s experiences through storytelling. It helps in ordering and interpreting events and circumstances experienced by individuals, while also revealing broader cultural, social, and institutional contexts in which people live. Narrative research can involve various data collection techniques, such as oral histories, interviews, journal entries, and digital recordings (Ross 1989; Moen 2006). Narrative data can be presented through vignettes (e.g. Vanclay 2015), through detailed contextualizing information interspersed with passages directly from data (e.g. Goater et al. 2012), or, in the case of digital storytelling, through short audio-video compilations (e.g. Gislason et al. 2018).
Q methodology	Developed to enable the systematic study of subjectivity (Brown 1993), Q methodology uses statistical analysis to identify dominant perspectives/discourses around a specific issue by having participants sort and rank a set (called a concourse) of qualitative statements representing a full range of opinions. It can generate insights beyond a polarised description (e.g. jobs versus economy), whereby there are often three, four or five distinct points of view on an issue. It has potential application in IA where the goal is to understand different perspectives or worldviews that may be held by different groups (e.g. Jenkins, 2017).
Qualitative data analysis	Often used by 37% of the survey respondents, qualitative data analysis is a systematic process of organizing and interpreting text and image data. A wide range of data sources and types can be qualitatively analysed (e.g. interview and focus group transcripts, written documents, photographs, and workshop notes). Five generic steps of qualitative data analysis include: 1) organizing and preparing the data; 2) an initial review of the data; 3) data coding; 4) categorizing and theming; and 5) interpretation (Leavy 2017; Creswell and Creswell 2018). These are often managed using computer-assisted data analysis software (e.g. NVivo, ATLAS-ti).
Scenario-based methods	Scenario-based methods integrate qualitative future-oriented scenarios (i.e. plausible or desired pathways by which the future could unfold) into IA analysis. They encompass various techniques that facilitate strategic mid-range and long-term planning when a certain degree of uncertainty exists (e.g. Rounsevell and Metzger 2010; UK Government 2017). These methods are typically used for either risk management (i.e. by testing decisions against desired futures) or for generating creative new ideas (Duinker and Greig 2007). Two main forms of scenario-based methods were identified in this research for use in IA: scenario analysis (Reed et al. 2013) and gaming (UK Government 2017).
Participatory spatial methods	These diverse methods integrate geospatial, rich, and in-depth qualitative information, and can rely on basic materials in situ, printed maps, or digital technologies and global positioning systems. Prominent forms include participatory geographic information systems (PGIS, or PPGIS) (Sieber 2006; González et al. 2008; Alagan and Aladuwaka 2012; Van Riper et al. 2020) and variants, e.g. web-based GIS (Kwan and Ding 2008; Tang and Liu 2016), volunteered geographic information (Brown et al. 2014), geo-visualization and geo-narratives (Kwan and Ding 2008). Other important forms include land use and occupancy mapping (or traditional land use mapping) (Tobias 2014) and community mapping (Chambers 2006; Spaling et al. 2011).
Surveys	Surveys use highly structured questionnaires to examine individuals’ experiences, perspectives, attitudes, or opinions about a specific topic (Neuman 2014). Many of the respondents (38%) indicated they often use qualitative surveys in their IA practice. By obtaining comparable information from respondents, surveys can allow standardised measurement across the answers received. Surveys may be undertaken as a quantitative method when they rely on closed-ended questions and statistical analysis of the data. However, they can also include qualitative open-ended questions, which allow respondents to expand on their answers in their own words.
Systems/network analysis	Systems analysis involves representation and analysis of relationships between systems components relevant to an IA. The analysis may include one or more systems (e.g. ecological, social, economic, institutional). Network analysis is closely related and maps the relationships among stakeholders and/or impact chains relevant to the IA. A broad distinction between the two is that network analysis, is based upon linear chains of causality, while systems analysis also incorporates circular causality and feedback loops (European Commission 1999; Perdicoúlis and Glasson 2006).
Visual methods	These methods collect and analyze visual or audio-visual images as data. Prominent visual methods include photo-elicitation, photovoice, and seasonal calendars. Photo-based methods involve the collection of photographs as data or that use photographs to facilitate data collection through other methods (Vitous and Zarger 2020). Seasonal calendars (also called seasonal activity calendars, community calendars or Indigenous calendars) involve collecting qualitative data that are used to develop diagrams depicting important environmental conditions and community activities, events, issues, and opportunities over an annual cycle (Narayanasamy 2008). Other important visual methods include video narratives (Gislason et al. 2018) and social media image analysis (Chen et al. 2019, 2020).
Workshops	Workshops are facilitated participatory sessions in which participants discuss, brainstorm, and identify solutions for a specific problem (Ørngreen and Levinsen 2017). They are versatile and can integrate a wide range of qualitative and quantitative data gathering techniques, and were one of the more commonly used qualitative methods with 38% of survey respondents indicating they often use them in their IA practice. Workshops typically run longer (three hours to multiple days) and include more participants (10+) than a focus group.

Figure 2. Impact assessment steps for which each qualitative method has been used or suggested as relevant by research participants (adapted from Walker et al. 2023, p. 42).

Table 2. Matching qualitative methods with particular IA needs (adapted from Walker et al. 2023, p. 44).

If an IA seeks to…	The possible methods include…
Gather external expert knowledge on a specific topic	document analysis (particularly academic literature, organizational reports), fuzzy sets, Delphi method, interviews, matrices, multi-criteria analysis
Understand the broad range of public perspectives, concerns, or opinions related to an IA	deliberative methods, document analysis (particularly news media analysis), fuzzy sets, surveys, Q methodology, visual methods (particularly photo preference surveys, photo visioning, and social media image analysis), workshops
Understand local values associated with place and land	document analysis, focus groups, interviews, (participatory) matrices, narrative methods, participatory spatial methods, visual methods
Understand perspectives of specific demographic groups related to specific topics (e.g. such as for gender-based analysis plus [GBA+] processes)	focus groups, (participatory) matrices, scenario-based methods, visual methods, workshops
Gather information to inform community-led IAs	focus groups, narrative methods, participatory spatial methods, surveys, visual methods, workshops
Integrate qualitative and quantitative indicators/assessment criteria and related data	matrices, multi-criteria analysis, fuzzy sets
Identify impact pathways and relationships among impacts and valued components in complex systems	matrices, multi-criteria analysis, systems/network analysis
Understand potential impacts on intangible social, cultural, spiritual, environmental values (e.g. connection to place, social cohesion, mental and emotional well-being, spirituality)	interviews, narrative methods, participatory spatial methods, visual methods
Understand the historical, environmental, social, cultural, economic, and political contexts in which a proposed project is located	document analysis, focus groups, narrative methods, scenario-based methods, visual methods

Figure 3. A basic classification of qualitative methods for IA.

Walker H, Pope J, Sinclair J, Bond A, Diduck A. 2023. Qualitative methods for the next generation of impact assessment. Report submitted to: impact assessment agency of Canada, University of Manitoba; [accessed 11 Jan 2024]. https://mspace.lib.umanitoba.ca/items/40910784-63ad-4b1a-a330-158929aa7a6d.

 Google Scholar

Gastil J. 2009. A comprehensive approach to evaluating deliberative public engagement. In: MASS LBP Engaging with impact: targets and indicators for successful community engagement by Ontario’s local health integration networks, Ontario ministry of health and long term care, health system strategy division, and the central, north west and south east LHINs. Toronto (ON): Ontario Ministry of Health and Long Term Care; p. 15–27.

 Google Scholar

Wiklund H, Viklund P. 2006. Public deliberation in strategic environmental assessment: an experiment with citizens’ juries in energy planning. In: Emmelin L, editor. Effective environmental assessment tools-critical reflections on concepts and practice. Karlskrona, Sweden: Blekinge Institute of Technology; p. 44–59.

 Google Scholar

Fishkin JS. 2021. Deliberative public consultation via deliberative polling: criteria and methods. Hastings Cent Rep. 51(S2):S19–S24. doi: 10.1002/hast.1316.

 PubMedGoogle Scholar

Schieffer A, Isaacs D, Gyllenpalm B. 2004. The world café: part one. World. 18(8):1–9.

 Google Scholar

Crosby N. 1995. Citizens juries: one solution for difficult environmental questions. In: Renn O, Webler T Wiedemann P, editors. Fairness and competence in citizen participation: evaluating models for environmental discourse. Springer Netherlands; p. 157–174. 10.1007/978-94-011-0131-88.

 Google Scholar

Richey JS, Horner RR, Mar BW. 1985. The Delphi technique in environmental assessment II. Consensus on critical issues in environmental monitoring program design. J Environ Manag. 21(2):147–159.

 Web of Science ®Google Scholar

Egan AF, Jones SB. 1997. Determining forest harvest impact assessment criteria using expert opinion: a Delphi study. North J Appl For. 14(1):20–25. doi: 10.1093/njaf/14.1.20.

 Google Scholar

Hanna K, Noble B. 2015. Using a Delphi study to identify effectiveness criteria for environmental assessment. Impact Assess Proj Apprais. 33(2):116–125. doi: 10.1080/14615517.2014.992672.

 Web of Science ®Google Scholar

Bowen GA. 2009. Document analysis as a qualitative research method. Qual Res J. 9(2):27–40. doi: 10.3316/QRJ0902027.

 Google Scholar

Savin-Baden M, Major CH. 2013. Qualitative research: the essential guide to theory and practice. London: Routledge.

 Google Scholar

Tight M. 2019. Documentary research in the social sciences. London: SAGE Publications.

 Google Scholar

Morgan DL. 1996. Focus groups. Annu Rev Sociol. 22(1):129–152. doi: 10.1146/annurev.soc.22.1.129.

 Web of Science ®Google Scholar

Toth FL. 2001. Participatory integrated assessment methods: an assessment of their usefulness to the European environmental agency. European Environmental Agency; [accessed 12 Jan 2024]. https://www.eea.europa.eu/publications/Technical_report_no_64/page001.html.

 Google Scholar

Creswell JW. 2007. Qualitative inquiry & research design: choosing among five approaches. 2nd ed. London: Sage.

 Google Scholar

Zadeh LA. 1965. Fuzzy sets. Inf Control. 8(3):338–353. doi: 10.1016/S0019-9958(65)90241-X.

 Google Scholar

Wood G, Rodriguez-Bachiller A, Becker J. 2007. Fuzzy sets and simulated environmental change: evaluating and communicating impact significance in environmental impact assessment. Environ Plan A. 39(4):810–829. doi: 10.1068/a3878.

 Web of Science ®Google Scholar

Morris A. 2015. A practical guide to in-depth interviewing. London: SAGE Publications.

 Google Scholar

Fischer DW, Davies GS. 1973. An approach to assessing environmental impacts. J Environ Manag. 1:207–227.

 Web of Science ®Google Scholar

Bisset R. 1980. Methods for environmental impact analysis: recent trends and future prospects. J Environ Manag. 11(1):27–43.

 Web of Science ®Google Scholar

Wathern P. 1984. Ecological modelling in impact analysis. In: Roberts RD Roberts TM, editors. Planning and ecology. London and (NY): Chapman and Hall; p. 80–98.

 Google Scholar

Shopley J, Sowman M, Fuggle R. 1990. Extending the capability of the component interaction matrix as a technique for addressing secondary impacts in environmental assessment. J Environ Manag. 31(3):197–213. doi: 10.1016/S0301-4797(05)80034-9.

 Web of Science ®Google Scholar

Satterfield T, Gregory R, Klain S, Roberts M, Chan KM. 2013. Culture, intangibles and metrics in environmental management. J Environ Manag. 117:103–114. doi: 10.1016/j.jenvman.2012.11.033.

 PubMed Web of Science ®Google Scholar

Nchanji YK, Levang P, Jalonen R. 2017. Learning to select and apply qualitative and participatory methods in natural resource management research: self-critical assessment of research in Cameroon. Forests Trees Livelihoods. 26(1):47–64. doi: 10.1080/14728028.2016.1246980.

 Google Scholar

Dodgson JS, Spackman M, Pearman A, Phillips LD. 2009. Multi-criteria analysis: a manual. UK Department of Communities and Local Government; [accessed 12 Jan 2024]. https://eprints.lse.ac.uk/12761/1/Multi-criteria_Analysis.pdf.

 Google Scholar

Geneletti D, Ferretti V. 2015. Multicriteria analysis for sustainability assess-ment: concepts and case studies. In: Morrison-Saunders A, Pope J Bond A, editors. Handbook of sustainability assessment. Cheltenham, (UK): Edward Elgar; p. 235–264.

 Google Scholar

Te Boveldt G, Keseru I, Macharis C. 2021. How can multi- criteria analysis support deliberative spatial planning? A critical review of methods and participatory frameworks. Evaluation. 27(4):492–509. doi: 10.1177/13563890211020334.

 Web of Science ®Google Scholar

Ross H. 1989. Community social impact assessment: a cumulative study in the Turkey Creek area, Western Australia. East Kimberley working paper No. 27. [accessed 12 Jan 2024]. https://library.dbca.wa.gov.au/static/Journals/081519/081519-27.pdf.

 Google Scholar

Moen T. 2006. Reflections on the narrative research approach. Int J Qual. 5(4):56–59. doi: 10.1177/160940690600500405.

 Google Scholar

Vanclay F. 2015. Qualitative methods in regional program evaluation: an examination of the story-based approach. In: Karlsson C, Andersson M Norman T, editors. Handbook of research methods and applications in economic geography. Cheltenham: Edward Elgar Publishing; p. 544–570.

 Google Scholar

Goater S, Goater R, Goater I, Kirsch P. 2012. This life of mine – personal reflections on the well-being of the contracted fly-in, fly-out workforce. Proceedings from the Eighth AUSIMM Open Pit Operators’ Conference; [accessed 12 Jan 2024]; Perth, Western Australia. https://www.academia.edu/download/33505950/Goater_2012_This_life_of_mine_AUSIMM.pdf.

 Google Scholar

Gislason MK, Morgan VS, Mitchell-Foster K, Parkes MW. 2018. Voices from the landscape: storytelling as emergent counter-narratives and collective action from northern BC watersheds. Health & Place. 54:191–199. doi: 10.1016/j.healthplace.2018.08.024.

 PubMed Web of Science ®Google Scholar

Brown SR. 1993. A primer on Q methodology. Operant Subjectivity. 16(3/4):91–138. doi: 10.22488/okstate.93.100504.

 Google Scholar

Jenkins J. 2017. Rare earth at Bearlodge: anthropocentric and biocentric perspectives of mining development in a multiple use landscape. J Environ Stud Sci. 7:189–199.

 Google Scholar

Leavy P. 2017. Research design: quantitative, mixed methods, arts-based, and community-based participatory research approaches. (NY): The Gilford Press.

 Google Scholar

Creswell JW, Creswell JD. 2018. Research design: qualitative, quantitative, and mixed methods approaches. 5th ed. London: SAGE Publications.

 Google Scholar

Rounsevell MDA, Metzger MJ. 2010. Developing qualitative scenario storylines for environmental change assessment. Wiley Interdiscip Rev Clim Change. 1(4):606–619. doi: 10.1002/wcc.63.

 Web of Science ®Google Scholar

UK Government. 2017. Futures toolkit: tools for strategic futures for policy-makers and analysts. Cabinet Office and Government Office for Science; [accessed 12 Jan 2024]. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/674209/futures-toolkit-edition-1.pdf.

 Google Scholar

Duinker PN, Greig LA. 2007. Scenario analysis in environmental impact assessment: improving explorations of the future. Environ Impact Assess Rev. 27(3):206–219. doi: 10.1016/j.eiar.2006.11.001.

 Web of Science ®Google Scholar

Reed MS, Kenter J, Bonn A, Broad K, Burt TP, Fazey IR, Fraser EDG, Hubacek K, Nainggolan D, Quinn CH, et al. 2013. Participatory scenario development for environmental management: a methodological framework illustrated with experience from the UK uplands. J Environ Manag. 128:345–362. doi: 10.1016/j.jenvman.2013.05.016.

 PubMed Web of Science ®Google Scholar

Sieber R. 2006. Public participation geographic information systems: a literature review and framework. Ann Assoc Am Geogr. 96(3):491–507. doi: 10.1111/j.1467-8306.2006.00702.x.

 Web of Science ®Google Scholar

González A, Gilmer A, Foley R, Sweeney J, Fry J. 2008. Technology-aided participative methods in environmental assessment: an international perspective. Comput Environ Urban Syst. 32(4):303–316. doi: 10.1016/j.compenvurbsys.2008.02.001.

 Web of Science ®Google Scholar

Alagan R, Aladuwaka S. 2012. Innovative public participatory GIS methodologies adopted to deal with the social impact assessment process challenges: a Sri Lankan experience. J Urban Reg Inf Syst Assoc. 24(2):19–32.

 Google Scholar

Van Riper CJ, Foelske L, Kuwayama SD, Keller R, Johnson D. 2020. Understanding the role of local knowledge in the spatial dynamics of social values expressed by stakeholders. Appl Geogr. 123:102279. doi: 10.1016/j.apgeog.2020.102279.

 Web of Science ®Google Scholar

Kwan M-P, Ding G. 2008. Geo-narrative: extending geographic information systems for narrative analysis in qualitative and mixed-method research. Prof Geogr. 60(4):443–465. doi: 10.1080/00330120802211752.

 Web of Science ®Google Scholar

Tang Z, Liu T. 2016. Evaluating internet-based public participation GIS (PPGIS) and volunteered geographic information (VGI) in environmental planning and management. J Environ Plann Manag. 59(6):1073–1090. doi: 10.1080/09640568.2015.1054477.

 Web of Science ®Google Scholar

Brown G, Kelly M, Whitall D. 2014. Which ‘public’? Sampling effects in public participation GIS (PPGIS) and volunteered geographic information (VGI) systems for public lands management. J Environ Plann Manage. 57(2):190–214. doi: 10.1080/09640568.2012.741045.

 Web of Science ®Google Scholar

Tobias TN. 2014. Research design and data collection for land use and occupancy mapping. SPC Tradit Mar Resource Manag Knowl Inf Bull. 33:13–25.

 Google Scholar

Chambers R. 2006. Participatory mapping and geographic information systems: whose map? Who is empowered and who disempowered? Who gains and who loses? Electron J Info Sys Dev Countries. 25(1):1–11. doi: 10.1002/j.1681-4835.2006.tb00163.x.

 Google Scholar

Spaling H, Montes J, Sinclair AJ. 2011. Best practices for promoting participation and learning for sustainability: lessons from community-based environmental assessment in Kenya and Tanzania. J Environ Assess Policy Manag. 13(3):343–366. doi: 10.1142/S1464333211003924.

 Google Scholar

Neuman WL. 2014. Social research methods: qualitative and quantitative approaches. 7th ed. Harlow: Pearson Education Limited.

 Google Scholar

European Commission. 1999. Guidelines for the assessment of indirect and cumulative impacts as well as impact interactions. Office for Official Publications of the European Communities; [accessed 14 Jan 2024]. https://ec.europa.eu/environment/archives/eia/eia-studies-and-reports/pdf/guidel.pdf.

 Google Scholar

Perdicoúlis A, Glasson J. 2006. Causal networks in EIA. Environ Impact Assess Rev. 26(6):553–569. doi: 10.1016/j.eiar.2006.04.004.

 Web of Science ®Google Scholar

Vitous CA, Zarger R. 2020. Visual narratives: exploring the impacts of tourism development in Placencia, Belize. Ann Anthropol Pract. 44(1):104–118. doi: 10.1111/napa.12135.

 Web of Science ®Google Scholar

Narayanasamy N. 2008. Participatory rural appraisal principles, methods and application. London: SAGE Publications.

 Google Scholar

Chen Y, Parkins JR, Sherren K. 2019. Leveraging social media to understand younger people’s perceptions and use of hydroelectric energy landscapes. Soc Natur Resour. 32(10):1114–1122. doi: 10.1080/08941920.2019.1587128.

 Web of Science ®Google Scholar

Chen Y, Caesemaecker C, Rahman HMT, Sherren K. 2020. Comparing cultural ecosystem service delivery in dykelands and marshes using Instagram: a case of the Cornwallis (Jijuktu’kwejk) river, Nova Scotia, Canada. Ocean Coast Manag. 193:105254. doi: 10.1016/j.ocecoaman.2020.105254.

 Web of Science ®Google Scholar

Ørngreen R, Levinsen K. 2017. Workshops as a research methodology. The Electron J eLearning. 15(1):70–81.

 Web of Science ®Google Scholar