Translation and validation of a questionnaire for measuring teachers’ views on nature of science

ABSTRACT

Background

Teachers’ views of nature of science (NOS) play a key-role in how they teach science and can significantly impact their students’ understanding of and achievement in science. NOS has been a curriculum goal in science education for several decades. Therefore, there is a need for assessment tools that can examine views of NOS in the language spoken by teachers and used in their teaching.

Purpose

This paper reports the development, through translation and validation of a measure for assessing NOS understanding in the Arabic language, specifically, the Reconceptualised Family Resemblance Approach to Nature of Science Questionnaire (RFNQ). The RFNQ has already been validated for use in English and Turkish languages; however, a validated version of the questionnaire in Arabic is lacking.

Method

The RFNQ was adapted to Arabic and validated according to widely accepted guidelines for cross-cultural adaptation of questionnaires for different languages. These incorporated forward translation, critical assessment by a panel of Arabic native experts and experienced teachers, and back-translation.

Sample

Subsequently, a total of 130 pre-service teachers completed the questionnaire to assess the internal consistency of the constructed Arabic version (RFNQ-AR).

Results

The RNFQ-AR was developed through a comprehensive procedure to ensure accurate translation and meaning. Further, the RFNQ-AR questionnaire reliability coefficient for the whole items found to be acceptable (α = 0.89) and varied per scale between 0.72 and 0.74, which is standard for internal consistency reliability.

Conclusions

Current estimates of 422 million Arabic speakers worldwide indicate the need for assessment tools to be available in Arabic. Questionnaires such as the RFNQ-AR will facilitate research into teacher understanding of NOS and enable evidence-based training in this area to ensure optimal student performance.

KEYWORDS:

• Nature of science (NOS)
• family resemblance approach to nature of science (FRA)
• Reconceptualised Family Resemblance Approach to Nature of Science (RFN)
• science education

Introduction

Nature of science (NOS) has been a focus in the literature over several years, and recently there has been renewed interest in understanding teachers’ and students’ conceptualizations of NOS-related content (Abd-El-Khalick and Lederman 2000; Erduran et al. 2020). Underlying this interest was the desire to examine the relationship between teachers’ fundamental conceptions of NOS and their classroom practice, and whether these beliefs might significantly impact their students’ conceptions and understanding of NOS as research indicated that such relational assumption did exist (Abell & Lederman 2007; Clough and Olson 2012; Nur and Fitnat 2015; Sahin and Deniz 2016). Hence, it is that teachers’ views of NOS play a key-role in how they teach science and can potentially influence their instructional decisions. Critically, research into perceptions of NOS indicated that both teachers and students possess inadequate understanding of NOS (Bilican, Cakiroglu, and Oztekin 2015; Karaman 2018). Moreover, many studies have found teachers having misconceptions about NOS (Abd-El-Khalick and Lederman 2000). Thereby, diagnosing and altering perceptions of NOS is considered, indeed, a major aim in Science Education for effective science teaching and learning (Erduran et al. 2020).

For the purpose of assessing those conceptions, various tools were developed and used, both quantitatively and qualitatively, including questionnaires, surveys, interviews, small group discussions, classroom observations and writing tasks. Among the earliest, self-reported measures were the Science Attitude Questionnaire (Wilson’s 1954), the Test on Understanding Science (TOUS) (Klopfer and Cooley 1961), the Nature of Scientific Knowledge Scale (NSKS) (Rubba 1976) and the View on Nature of Science (VNOS-A) (Lederman and O’Malley 1990). Much more recently, various theoretical approaches to NOS have been presented in literature, such as Allchin’s ‘Whole Science’ (Allchin 2011) and Matthew’s ‘Features of Science’ (Matthews 2012). Comprehensive reviews and synthesis for various assessments of NOS can be found in Abd-El-Khalick (2014), Lederman (2007) and Lederman et al. (2002).

The primary objective of this paper is to report about the validation of a NOS questionnaire based on the family resemblance approach (FRA) (Erduran 2014; Irzik and Nola 2014). FRA is a relatively new approach to the characterization of NOS and as such, there is limited set of tools to investigate NOS views. One questionnaire based on FRA has already been validated for use in English (Erduran and Kaya 2018) and Turkish languages (Erduran et al. 2020). However, a validated version of the questionnaire in Arabic is lacking. Studies focusing on validation of instruments is an important element of educational research as illustrated by articles in this journal (e.g. Schelfhout et al. 2006). Existence of a tool in Arabic is useful not only for tracing teachers’ NOS views in Arabic-speaking countries but also across Europe and elsewhere where there are native Arabic speakers. Therefore, this validation study reports the process of translating and validating an extended version of the original tool in the Arabic language context. The paper is presented as part of a funded project with the aim of designing, implementing and evaluating the impact of pedagogical strategies to teach about NOS. Following a review of the literature on FRA to NOS, the empirical study is presented illustrating the translation and validation of the questionnaire.

Family resemblance approach to nature of science

One of the many initiatives to assess and improve teachers’ views and understanding of NOS, is the family resemblance approach (FRA) to nature of science (NOS). The concept of family resemblance was originally described by Wittgenstein and then adapted by the philosophers of science Irzik and Nola (2011, 2014). It initially focused on scientific activities, aims/values, methodologies/methodological rules and products of science. FRA to NOS characterizes both the domain-general and the domain-specific features of science. In this sense, the various branches of science are conceptualized as a ‘family’ consisting of members who exemplify some general shared characteristics, but yet do have other characteristics that are specific to each sub-domain. Later on, Irzik and Nola (2014) broadened the FRA scope to include the social-institutional dimensions of science.

Building on their work, Erduran and Dagher (2014), Erduran (2014) reconceptualized Irzik and Nola’s FRA to NOS (Irzik and Nola 2014, 2011) while adapting it into the science education, resulting with an expanded version of the original FRA. The newly developed version (RFN) expanded the social-institutional categories to include political, financial and organizational aspects as these factors greatly impact how science is functioning and undertaken and, therefore, are important with respect to science curriculum. Moreover, educational applications from science education research were also incorporated providing an empirical adaptation for teacher education. Hence, RFN reflects a more holistic view of NOS as it inclusively accounted for the pedagogical, instructional, curricular and assessment issues in science education (Erduran et al. 2020).

The RFN model used in the current study consists of 11 NOS categories contained in two major dimensions: (1) cognitive-epistemic system encompassing category including aims and values, scientific practices, methods and methodological rules, scientific knowledge, and (2) social institutional system encompassing category including professional activities, scientific ethos, social certification and dissemination, social values of science, social organizations and interactions, political power structures, and financial systems. Table 1, adapted from Erduran (2014) and McDonald (2017), represents a detailed account on these categorizations of NOS.

Table 1. A detailed account on the categorizations of NOS (adapted from Erduran 2014; McDonald 2017).

Category	Description
Cognitive-epistemic system dimension
Aims and values	The scientific enterprise is underpinned by adherence to a set of values that guide scientific practices. These aims and values are often implicit and they may include accuracy, objectivity, consistency, skepticism, rationality, simplicity, empirical adequacy, prediction, testability, novelty, fruitfulness, commitment to logic, viability and explanatory power.
Scientific practices	The scientific enterprise encompasses a wide range of cognitive, epistemic and discursive practices. Scientific practices such as observation, classification, and experimentation utilize a variety of methods to gather observational, historical or experimental data. Cognitive practices, such as explaining, modeling and predicting, are closely linked to discursive practices involving argumentation and reasoning.
Methods and methodological rules	Scientists engage in disciplined inquiry by utilizing a variety of observational, investigative and analytical methods to generate reliable evidence and construct theories, laws and models in a given science discipline, which are guided by particular methodological rules. Scientific methods are revisionary in nature, with different methods producing different forms of evidence, leading to clearer understandings and more coherent explanations of scientific phenomena.
Scientific knowledge	Theories, laws and models (TLM) are inter-related products of the scientific enterprise that generate and/or validate scientific knowledge and provide logical and consistent explanations to develop scientific understanding. Scientific knowledge is holistic and relational, and TLM are conceptualized as a coherent network, not as discrete and disconnected fragments of knowledge.
Social-institutional system dimensions
Professional activities	Scientists engage in a number of professional activities to enable them to communicate their research, including conference attendance and presentation, writing manuscripts for peer-reviewed journals, reviewing papers, developing grant proposals and securing funding.
Scientific ethos	Scientists are expected to abide by a set of norms both within their own work, and during their interactions with colleagues and scientists from other institutions. These norms may include organized skepticism, universalism, communalism and disinterestedness, freedom and openness, intellectual honesty, respect for research subjects, and respect for the environment.
Social certification and dissemination	By presenting their work at conferences, and writing manuscripts for peer-reviewed journals, scientists’ work is reviewed and critically evaluated by their peers. This form of social quality control aids in the validation of new scientific knowledge by the broader scientific community.
Social values of science	The scientific enterprise embodies various social values including social utility, respecting the environment, freedom, decentralizing power, addressing human needs and equality of intellectual authority.
Social organizations and interactions	Science is socially organized in various institutions including universities and research centers. The nature of social interactions among members of a research team working on different projects is governed by an organizational hierarchy. In a wider organizational context, the institute of science has been linked to industry and the defense force.
Political power structures	The scientific enterprise operates within a political environment that influences the direction and use of science. The outcomes of science are not always beneficial for individuals, groups, communities or cultures.
Financial systems	The scientific enterprise is mediated by economic factors. Scientists require funding in order to carry out their work, and state and national level governing bodies provide significant levels of funding to universities and research centers. As such, these organizations have an influence on the types of scientific research funded, and ultimately conducted.

Taken as a whole, this approach and its applications have been widely considered for science education (Akbayrak and Kaya 2020, 2018; Akgun and Kaya 2020; Erduran and Kaya 2018;BouJaoude, Dagher, and Refai 2017; Dagher and Erduran 2017; Erduran 2014; Erduran and Dagher 2014; Akbayrak and Kaya 2020; Erduran et al. 2020; Erduran and Kaya 2018, 2019; Kampourakis 2016; Kaya and Erduran 2016; McDonald 2017). Erduran, Dagher and McDonald’s 2019 work is instructive in this regard especially in providing a comprehensive account of the contributions of FRA to NOS in science education including an overview of research and development efforts aimed at utilizing the FRA along with a synthesis of relevant empirical studies. One of the issues in this line of scholarship is that there is limited set of research instruments designed from the FRA perspective.

The Reconceptualised Family Resemblance Approach to Nature of Science Questionnaire (RFNQ)

Building on RFN model, Erduran et al. (2020) developed a reliable and valid instrument based on all categories of RFN, namely, the Reconceptualised Family Resemblance Approach to Nature of Science Questionnaire (RFNQ). The RFNQ is a self-reported measure to assess commonly held beliefs about NOS. The measure has a strong base in the literature, and validity is supported by multiple quantitative and qualitative studies. It comprises 70 items grouped into three major scales ‘Cognitive-Epistemic System’, ‘Social-Institutional System’ and ‘Educational Applications’, denoting all the RFN categories, with both positive and negative items for each category. These are presented in Table 2.

Table 2. Scales, sub-scales and items of RFNQ.

Major Scales	Sub-scales	Number of items	Positive Items	Negative Items
Cognitive-epistemic system encompassing	Aims and Values	7	2, 20, 40, 51, 69	46, 56
	Scientific Practices	13	4, 5, 15, 19, 23, 33, 38, 57,61, 63	26, 52, 64
	Scientific Methods	9	11, 22, 24, 28	8, 25, 37, 49, 60
	Scientific Knowledge	9	10, 30, 44, 50, 54	3, 16, 43, 66
Social-institutional system	Social-institutional Aspects	16	7, 9, 14, 32, 34, 41, 45, 48, 53, 58, 67, 70	13, 18, 36, 39
Educational Applications		16	1, 6, 12, 17, 21, 27, 29, 31, 42, 55, 59, 62, 65	35, 47, 68

A major advantage of the RFNQ is the inclusion of items relating to ‘Educational Applications’ for each category of the RFN. These were included to gain insight into how teachers consider the teaching aspects of RFN. One example is an item on whether teachers consider teaching scientific aims and values in science classes and how this might affect students’ cognitive ability levels. It is also worth noting that no bias due to gender, ethnicity or level of education was identified in the original constructed items. Furthermore, items are measured against a 5-point Likert scale where respondents are asked to determine their views about each statement by selecting from the given five options: ‘totally disagree’, ‘disagree’, ‘not sure’, ‘agree’ and ‘totally agree’. It takes approximately 7–15 minutes to complete.

In attempting to assure the validity of the original questionnaire, the items were checked by two experts in science education for appropriateness in NOS context and were refined accordingly. For the reliability check, the questionnaire was pilot tested to a total of 222 pre-service science teachers across two public universities. Results show that the questionnaire has demonstrated excellent psychometric properties yielding a Cronbach's alpha of 0.8, which is considered to be highly reliable.

The aforementioned RFNQ has become a notable tool for the measurement of beliefs relating to NOS and has already been used by other researchers (e.g. Azninda and Sunarti 2021). The questionnaire has already been validated and used in English, as well as adapted to Turkish languages (Kaya et al. 2017). In terms of the studies to date, the process of adaptation and validation of an Arabic language version has not been undertaken, and no official Arabic version of the questionnaire exists. Arabic-speaking people may have difficulties in understanding the English items of the original RFNQ and may interpret the terms used in slightly different ways than intended, hence, adapting it to the first language of Arabic group of respondents is believed more appropriate. This is especially true given that the total population of the Arab world in 2019 amounted to approximately 427.87 million inhabitants (Arab world: Total population from 2009 to 2019, 2021), and there are around 422 million speakers of Arabic language and its dialects (native and non-native) across both the Arab nations whose official language is Arabic (e.g. the Middle East) and countries whose co-official language is Arabic (e.g. Somalia, South Sudan and Eritrea), making it one of the five spoken languages in the world (List of countries where Arabic is an official language, 2021).

At the time of the current reported study, no attention was paid to adapt the original RFNQ tool to the Arabic language. For this purpose, it is of ultimate importance to ensure the validity of the translated measures in the Arabic language context in order to confirm integrity of any relevant reported research outcomes. As such, without a sufficient pre-data collection to assess the correct representation of the original instrument, any further data collection will be flawed and less rigorous (Hall et al. 2018). Accordingly, and to complement this need, the current paper, though, reports on the validation of the Arabic version of the RFNQ, standing for the RFNQ-AR.

Methodology

Setting

The paper is presented as part of a funded research project examining the Early Years’ Science Education in a university in the Middle East with the aim of designing, implementing and evaluating the impact of RFN strategies. The current study was conducted by a public university in the UAE and the target sample consisted of pre-service teachers at the college of Education. Data collection took place between March and April 2021.

Ethical considerations and consent to participate

For the ethical considerations, permission to translate and validate the Reconceptualised Family Resemblance Approach to Nature of Science Questionnaire (RFNQ) was obtained from the original designers of the questionnaire (Erduran et al. 2020). Moreover, research ethics approval was granted by the Social Science & Humanities Ethics Committee at the United Arab Emirates University. Information about the study and its purpose along with participant’s role and rights were given, and informed consent from participants was obtained via the administered online questionnaires.

Procedures for developing the translated version and its validation

The translation of the RFNQ to Arabic was guided by the review of guidelines for cross-cultural adaptation of questionnaires (Epstein, Santo, and Guillemin 2015) and the good practice guide for translating and adapting questionnaires for different languages and cultures reported by Hall et al. (2018) and Gjersing, Caplehorn, and Clausen (2010). This process was completed through a series of rigorous steps. Figure 1 represents a flow chart of the administered procedures for developing the translated version and its validation.

Figure 1. Procedures for developing the translated version (RFNQ-AR) and its validation.

Questionnaire forward translation

The original 70-items of the RFNQ were first reviewed by two bilingual content experts to establish the content equivalence in terms of relevance and sensitivity for the culture being targeted. The content of the items was considered to be adaptable for translation and implementation in Arabic. As such, the items were then independently forward-translated to the Arabic language by two native Arabic researchers who are proficient in English and who both had academic and teaching experience. The translation was more conceptual, meaning not a word-for-word translation in order to best convey the same meaning as that in the English items by using the most reflective Arabic wordings (Hall et al. 2018). The used Arabic language included no jargon, unfamiliar technical terminologies, or any terms that might be considered sensitive or offensive to the target population. The two translated versions were cross-checked for consistency. Variances were resolved upon a consensus discussion between the two translators, along with an early communication with the original questionnaire developers to reduce any risk of interpretation in later stages of translation. The two initial translated versions were then consolidated into a single version (the RFNQ-AR).

Review of the translated version by Arabic native experts

To assess for clarity and to check for potential ambiguities of the constructed translated version, a panel of two experts in the Arabic language and two bi-lingual (English/Arabic) professors with experience in science education was formed to review the forward translations of the RFNQ. The reviewers were asked to assess each of the 70 translated items in terms of ‘linguistic check’ (if the item is correctly stated in Arabic) and for ‘clarity check’ (if the item is clear, not confusing and can be clearly read and understood) in corresponding to two responses ‘valid’ or ‘not valid’. Moreover, they were asked to provide their own comments beside each item where necessary, to identify items that are unfamiliar or with inappropriate grammar and syntax. An agreement was reached with regard to the semantics of meanings that were addressed by the reviewer’s comments on certain expressions and terms while negotiating the corresponded suggested alternatives. Subsequently, the initially constructed version was modified accordingly.

Review of the translated version by Arabic native teachers

In order to examine the readability and comprehension of the produced RFNQ-AR, the Arabic questionnaire was further reviewed and assessed by a purposefully selected group of four teachers. The teachers were qualified to at least Bachelor’s level and had between 7 and 15 years teaching experience. This step was included to ensure that all translated items were clear, can be easily understood, and that there are no terms considered to be ambiguous. Respondents were asked to assess whether each Arabic statement was readable, clear and understandable in corresponding to two responses ‘valid’ or ‘not valid’, as well as to provide their own comments beside each item where necessary. The respondents' input was critically reviewed. Where necessary, the respondents were communicated by the researchers to fully understand the nature of certain concerns. The comments and feedback received from the teachers were taken into consideration, and refinements were made accordingly.

Questionnaire back-translation

In an attempt to assure the accuracy of the constructed RFNQ-AR, a bilingual independent translator who is a native speaker of Arabic holding a Master’s degree in Science Education, therefore being familiar with terminologies of the area covered in the instrument, performed a back-translation from Arabic to English. The English items were not shared with the translator during this procedure, and thus, the translator was blinded to the original RFNQ. The back-translation was then cross-checked with the original English items to identify potential discrepancies and check for consistency in meanings while focusing mainly on conceptual consistency but not linguistic equivalence. There were a number of instances where phrases required verification to achieve the same meaning as expressed in English. In addition, minor semantic discrepancies from the original items were found and reconciled with both, the translator and the researchers, from where final amendments were made.

Further validation of the questionnaire

All the prior assessments helped improve the item performance in the next revision at which the updated RFNQ-AR was further reviewed and approved by a panel of research experts from the Abu Dhabi Department of Education and Knowledge at the UAE Ministry of Education (MoE). The original English RFNQ and the RFNQ-AR are available in the appendix (see Appendix A).

Pilot testing and data collection

Online questionnaires for the final translated version of RFNQ-AR were developed and administered via electronic Google form. All the sampled pre-service teachers were invited to participate in the pilot study (Purposive sample). The responses for the online questionnaires were gathered from March 2021 to April 2021. A total of 130 completed questionnaires were included in the pilot study.

Data analysis

The participant’s responses across the 5-point Likert scale options for each item were coded. The options of ‘totally disagree’, ‘disagree’, ‘not sure’, ‘agree’ and ‘totally agree’ were coded as 1, 2, 3, 4, and 5, respectively. The scores of the negative items were then reversed in order to have all scores coded in the same direction (Items 46, 56, 26, 52, 64, 8, 25, 37, 49, 60, 3, 16, 43, 66, 13, 18, 36, 39, 35, 47, 68). For these negative items, the codes of ‘1’, ‘2’, ‘4’ and ‘5’ were re-coded as ‘5’, ‘4’, ‘2’ and ‘1’, respectively. To test reliability, Cronbach’s Alpha (internal consistency) was calculated for the whole measure and scales with Cronbach’s α reliability coefficient at or above 0.7. Reliability result of the new translated version was compared with the reported reliability of the original English version.

Results

Overall, the results of forward translation process yielded a number of linguistic considerations. These required rewording of some statements, selecting alternatives in Arabic to best convey the exact meaning intended in English, and merging some English phrases into one when translated into Arabic. All decisions made in this regard were in accordance with the linguistic context of each individual statement. Furthermore, the reviewers of the translated version of the questionnaire indicated generally no issues with understanding the Arabic items, except for certain translated content which revealed a potential misunderstanding that the Arabic reader might be having, and the backward translation indicated minor semantic discrepancies from the original questionnaire, all which were negotiated and adjusted accordingly. Finally, the reliability coefficient of the RFNQ-AR questionnaire was 0.89 and varied per scale from 0.72 to 0.74 which is considered to be acceptable. These results are further detailed below.

Results for questionnaire validity

Across the different stages of the translation and validation process, reviewers provided their own comments beside each item where necessary along with suggested alternatives which were all agreed upon and considered in the final refinements.

The forward translation yielded a number of linguistic considerations, these included, for example, the careful distinction between the word ‘should’ and ‘must’, and thus, the use of Arabic words that clearly distinguish between both while reflecting the right degree of the expressed obligation ‘ينبغي’ and ‘يجب’ (‘Yanbaghi’ and ‘Yajib’). Similarly, a great deal of care was given to the choice of Arabic words in order to reflect the exact meanings as precise as possible especially that there are several Arabic synonyms available. For example, the word ‘include’ has several synonyms like ‘يتضمن’, ‘يشمل’, and ‘يحتوي’ (‘Yatadamman’, ‘Yashmal’, ‘Yahtawi’) and the word ‘progress’ could be translated into Arabic as ‘تطور’, ‘تحسن’, ‘تقدم’, and ‘ارتقاء’ (‘Tatawwor’, ‘Tahasson’, ‘Taqaddom’, ‘Irtiqaa’). Thus, the choice was much dependent on the linguistic context of each individual statement. Consistent with this point, and when looking at the conceptual equivalence, some of the English terms found to reflect a broad spectrum of understanding when translated to Arabic. For instance, it was found that the term ‘method’ in items (24) and (69) could be understood differently by some of the target population either as ‘the overall methodological approach for conducting a scientific experiment/investigation’ or as ‘the specific employed ways, techniques, or instruments’. Therefore, two terms were used in the Arabic version to reflect both meanings ‘الوسائل والأساليب المنهجية’ (‘Al-Wasael Wa Al-Asaleeb Al-Manhajiyya’) .

Another notable consideration relates to the term ‘scientific investigation’. The literal Arabic translation for this term ‘التحقيق العلمي’ (‘Al-Tahqeeq Al-Elmi’) is interchangeably used with the term ‘الاستقصاء العلمي’ (‘Al-Istiqsaa Al-Elmi’) which denotes in English the word ‘scientific inquiry’. This observation was further supported by the teachers’ input along with the follow-up revision of relevant documents (e.g. the Arabic version of science investigation rubrics) revealing an inconsistent overlap in the usage of the two terms. Although both generally reflect a similar meaning to some extent, this observation attracted attention to the very precise difference between the two practices from a professional scientific perspective. Nevertheless, both words were used in the Arabic version where needed (e.g. items 12 and 42) to avoid any confusion. Apart from that, in some other cases, the English statement was rephrased when translated to Arabic to convey the exact meaning as possible (e.g. item 20).

In order to maintain brevity in expression without detracting the exact essence of the meaning, some Arabic statements merged two English phrases into one. For instance, the phrase ‘change their mind’ and ‘change their ideas’ in item (40) reflects no considerable different meanings when translated to Arabic. Accordingly, the Arabic translation incorporated the two into one phrase. A similar scenario applied in the use of the two words ‘race and ethnicity’ in item (48) as they were translated into one single common Arabic terminology ‘الانتماءات العرقية’ (‘Al-Intimaat Al-Irqiyya’) that reflects the entire meaning of the two words as a whole. As such, no issues were addressed with respect to cross-cultural adaptation in term of cultural validation as the translated expressions and wordings were found to be appropriate to the context of the target groups, and hence, the comprehensibility of the translated statements was not problematic during the forward and backward translation process, as well as the oral discussion with the reviewers.

Unexpectedly, and given the operational equivalences between the source- and target-language, comments made by the expert Arabic reviewers helped identify selected words and phrases that seemed problematic. For example, the point that the noun ‘science’ can be countable or uncountable and thus might reflect the plural form in some contexts. This was also evident in the backward translation, wherein some incidences, the word ‘science’ was understood from the learning context as ‘the science subject’ rather than ‘the pursuit and application of knowledge’ when perceived from the macro-context. Building on that, in item 7, the word ‘science’ was translated in a plural form as to reflect the intended meaning in a better way (reflecting the various types or disciplines of sciences). It was also noted that the phrase ‘doing science’ when literally translated to Arabic ‘عمل العلم’ (‘Amal Al-ilm’), might be difficult to comprehend, though, it was replaced with ‘لإجراء الممارسات العلمية’ (‘Li Ijraa Al-Momarasat Al-Ilmitta’) meaning in English to ‘undertake scientific practices’ (e.g. items 22 and 36). Likewise, the concept ‘representations’ was quite challenging as the Arabic translation ‘التمثيلات التوضيحية’ (‘Al-Tamtheelat Al-Tawdeehiyya’) is not commonly used in the local context of practice; hence, illustrative examples of the concept were provided in brackets to better clarify its meaning ‘models and equations’. Also, a number of changes have been applied to the translated version based on the recommended better fit alternatives, both for the prepositions and the words used. Nevertheless, the analysis identified no issues with the negatively worded questions when translated from English to Arabic.

The subsequent review by the teachers provided further assurance of the understanding and readability of the modified translated items. Participants’ written and oral comments indicated that for the most part, there was no issue with understanding the Arabic items, as well as the questionnaire introduction and its instructions. However, this step revealed a misconception that Arabic reader might have over certain translated content. Particularly, the phrase ‘use observation’ in item (23), when literally translated to Arabic, can be limitedly understood as ‘using the notes that are written on papers’. Given that the meaning of the original English phrase could mean both the act of observation, as well as the act of annotating or recording down the observed notes; consequently, an alternative term was used in Arabic ‘بإجراء الملاحظات’ (‘Bi Ijraa Al-Molahathat’). This term is similar to ‘make observations’ in English reflecting the general meaning of the term ‘observation’ as a science practice. This ensured that participants are exposed for no possibility of misinterpretation due to the different ways of thinking about the term or expression. Two participants suggested that presenting the items with breaks (separated by sections) was better to maintain the reader attention than listing all in one flow.

Finally, the backward translation indicated that there were generally no significant differences between the original and the translated version. Minor semantic discrepancies from the original questionnaire were found and were discussed between the translator and the researchers. These mainly resulted from having alternative words reported by the translator, which in many cases were slightly less formal when considered in the science learning context. These are presented in Table 3.

Table 3. Semantic discrepancies reported in the backward translation.

Original Term	Back Translated
Assess	Evaluate
Classification	Taxonomy
Society	Community
Build	Create
Representations	Illustrations
Stress	Emphasize
Interpret	Explain
Experimental	Empirical
Changing	Switching
Debated	Under discussion
Scientists	Scholars
Standards	Criteria

Interestingly, a second closer analysis of the forward translation also detected very precise differences in the used Arabic wordings. One example was for the word ‘determine’ in the original item (12) ‘Students should determine the methods of their science investigations themselves’ which was back-translated as ‘decide’ and, thus, was then changed in the Arabic item with another word that exactly means ‘to determine or identify’. Similarly, the word ‘specify’ in item (47) which was back-translated as ‘clarify’, as well as for the phrase ‘work together’ in item (50) which was replaced by an Arabic alternative word meaning ‘contribute to’ as it better fits with the original intended meaning. Although such differences, ultimately carried somehow similar meanings, changes were made to the Arabic wording to assure the highest degree of accuracy. Other changes concerning the order of the translated words against those in the original items were also applied. Taken collectively, and by completion of the above explained procedures, and upon the final revisions, data collection using the RFNQ-AR for pilot testing proceeded without no further adjustments to the translation.

Pilot testing results for reliability

A total of 130 pre-service education students from one public university in the UAE completed the questionnaire. The age of participants ranged from 17 to 26 years. The majority of the participants were UAE nationals (96%), along with (4%) from Oman, Syria and Jordan. Table 4 presents descriptive statistics for the RFNQ-AR questionnaire scales. Overall, the average summary score for the whole questionnaire items was M = 3.46 (SD = 20.33).

Table 4. Descriptive statistics for RFNQ-AR questionnaire scales.

Scale	Mean	SD
Whole Questionnaire	3.46	20.33
Scale 1 – Cognitive-Epistemic System	3.37	9.60
Scale 2 – Social-Institutional System	3.60	6.31
Scale 3 – Educational Applications	3.55	6.00

The reliability of the RFNQ-AR questionnaire scales and items was investigated with Cronbach’s α reliability coefficient at or above 0.70 as shown in Table 5. The reliability coefficient of the whole questionnaire was 0.89, which is considered to be acceptable (Nunnally 1978). The internal reliability coefficients for the first scale ‘Cognitive-Epistemic System’ yielded a coefficient of (0.74). Likewise, the second scale ‘Social-Institutional System’ yielded a coefficient of (0.74). Reliability coefficient was α = 0.72 for the third scale ’Educational Applications’ suggesting that different items within the scales are closely related to each other and, therefore, may cover similar facets of the same construct. Accordingly, Cronbach’s α of the scales were all greater than 0.70 and varied per scale from 0.72 to 0.74 which is standard for internal consistency reliability. Together, the reliability results showed that the items correlated substantially with each other and, therefore, are reliable measures and are likely to provide similar results when administered to the actual comparable sample. Accordingly, the RFNQ-AR questionnaire believed to be reliable tool for measuring the examined constructs.

Table 5. Reliability coefficients for RFNQ-AR questionnaire items.

Scale	Cronbach’s Alpha	Items No.
Whole Questionnaire	0.89	70
Scale 1 – Cognitive-Epistemic System	0.74	38
Scale 2 – Social-Institutional System	0.74	16
Scale 3 – Educational Applications	0.72	16

Cronbach’ s α > 0.7 was taken as reliable.

Discussion and conclusions

The process of English-to-Arabic translation of the original RFNQ and its validation involved a number of modifications, basically relating to linguistic considerations in terms of the item wording. This was presented and discussed in detail in the results section. The decision on the made changes was dependent on the linguistic context of each individual statement in order to reflect the right degree of the expressions used in the original items, as well as to maintain brevity without detracting the exact essence of the meaning. This was very true especially for items that were translated easily but yet, the language did not capture the exact original meaning. In doing so, the very precise differences in the wording between both versions were dealt with cautiously. For example, the teachers’ input revealed that there was an inconsistent overlap in the usage of the two terms ‘scientific investigation’ and ‘scientific inquiry’ when translated into Arabic.

This was also captured by the traditional forward- and reverse-translation process. Another example of a seemingly discrete issue had to do with the focal word ‘science’. Although it exists in Arabic, the participants had difficulty understanding the context of the item due to the multiple uses of this word in Arabic. Both cases, thus, required multiple conversations during the process between Arabic experts and teacher’s reviews in order to generate agreement. Additionally, the item analysis revealed some translational issues relating to specific words or phrases which when translated from English to Arabic did not convey similar meaning and, therefore, could result in respondents developing different understanding other than the intended meaning. Those terms which have been repeatedly addressed were identified as problematic during the validation procedures were improved with relatively minor adjustments through rewording based on the recommended better fit alternatives, both for the prepositions and the terms used. In addition, there were some changes to reconcile the semantic discrepancies which had emerged from the backward translation.

Furthermore, the word order for some items in the final instrument has been rearranged, and there was inclusion of specific words in combination to satisfy the differences in vocabulary meaning and, therefore, to ensure minimal confusion when considering the average level of comprehension across the different potential target participants who may have somehow more or less knowledge about the central examined topic (NOS). Nevertheless, no issues were detected with the negatively worded items translated from American English to Arabic, and thus, no serious damages to the internal consistency of scales in Arabic language instruments existed.

Ultimately, the scale consisted of all the 70 original items with almost all words and concepts having been translated relatively easily due to being general and universally transferable. The comprehensibility of the final version was not problematic based on the validation input obtained from the Arabic expertise reviewer’s & teacher’s comments, forward and backward translation process, as well as from the follow-up discussions. Hence, the RFNQ-AR questionnaire was found to transmit a similar meaning and was acceptably readable and well understood in the target language. Moreover, the RFNQ-AR questionnaire reliability coefficient for the whole items found to be acceptable (0.89) with a good internal consistency for all the three scales ‘Cognitive-Epistemic System’, ‘Social-Institutional System’ and ’Educational Applications’ (Cronbach’s α was greater than 0.70 for the three scales and varied per scale from 0.72 to 0.74) and, therefore, the RFNQ-AR questionnaire is believed to be reliable tool for measuring the examined constructs.

Compared to the original validation RFNQ study (Erduran and Kaya 2018), Cronbach’s alpha value of the current translated version is similar to that reported for in the English version (Cronbach’s α in the original study r = 0.8). Also, it is comparable to reliability results of other translated versions as such for the Turkish Version (0.77) (Erduran et al. 2020). There may be some possible limitations in this study that warrant mention while interpreting the results. Firstly, the participated pre-service teachers were sampled using a purposive technique. This may restrict the generalizability of the results. Secondly, the validity was assessed with a self-reported measure of the participated reviews, more objective outcome measures, as well as quantitative analysis of response agreement percentages needs to be further included.

Acknowledgments

The authors would like to acknowledge the following who are all co-investigators on the grant.

Mrs Lindsay Schofield (UAEU), Dr Hala Elhoweris (UAEU), Dr Olga Ioannidou (OU).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The datasets used and/or analysed during the current study are not available.

Supplementary material

Supplemental data for this article can be accessed at https://doi.org/10.1080/02635143.2022.2138846

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The project was funded by United Arab Emirates University (UAEU) Grant Number G00003448 on a joint collaboration project with Oxford University.