Psychometric properties of the Persian version of the Ambulatory Care Learning Educational Environment Measure (ACLEEM) questionnaire, Shiraz, Iran

Abstract

Purpose

Evaluation is the main component in design and implementation of educational activities and rapid growth of educational institution programs. Outpatient medical education and clinical training environment is one of the most important parts of training of medical residents. This study aimed to determine the validity and reliability of the Persian version of Ambulatory Care Learning Educational Environment Measure (ACLEEM) questionnaire, as an instrument for assessment of educational environments in residency medical clinics.

Materials and methods

This study was performed on 180 residents in Shiraz University of Medical Sciences, Shiraz, Iran, in 2014–2015. The questionnaire designers’ electronic permission (by email) and the residents’ verbal consent were obtained before distributing the questionnaires. The study data were gathered using ACLEEM questionnaire developed by Arnoldo Riquelme in 2013. The data were analyzed using the SPSS statistical software, version 14, and MedCalc^® software. Then, the construct validity, including convergent and discriminant validities, of the Persian version of ACLEEM questionnaire was assessed. Its internal consistency was also checked by Cronbach’s alpha coefficient.

Results

Five team members who were experts in medical education were consulted to test the cultural adaptation, linguistic equivalency, and content validity of the Persian version of the questionnaire. Content validity indexes were >0.9 in all items. In factor analysis of the instrument, the Kaiser–Meyer–Olkin index was 0.928 and Barlett’s sphericity test yielded the following results: X²=6,717.551, df =1,225, and P≤0.001. Besides, Cronbach’s alpha coefficient of ACLEEM questionnaire was 0.964. Cronbach’s alpha coefficients were also >0.80 in all the three domains of the questionnaire. Overall, the Persian version of ACLEEM showed excellent convergent validity and acceptable discriminant validity, except for the clinical training domain.

Conclusion

According to the results, the Persian version of ACLEEM questionnaire was a valid and reliable instrument for Iranian residents to assess specialized clinics and residency ambulatory settings.

Keywords:

• ACLEEM
• postgraduate medical education
• ambulatory medicine
• educational environment
• psychometric studies
• validation
• Iran

Introduction

As one of the key contributors to educational development, educational environment is definitely one of the most valuable assets of an academic setting.¹ Based on its quality, this environment can either support students or impede them from obtaining the desirable outcome of education, which is learning.² This impact is probably more pronounced in medical education settings, where the environment that engulfs students plays an important role in their skill acquirement and, consequently, patient care. Therefore, to guarantee a high-quality medical education, careful and continuous observation, assessment, and revision of the existing educational environment are mandated.^1,3

Although many use the words teaching and learning environments synonymously and interchangeably,⁴ learning environment applies mostly to the physical locations, contexts, and cultures in which students learn. In other words, it usually refers to the quality of individuals’ interactions with one another as well as the educational settings designed to facilitate learning.⁵ These also include policies and governance structures. Overall, a standard learning and teaching environment provides students with sufficient opportunities to acquire theoretical and practical skills as well as social responsibility that in turn prepares them for their future professional lives.^5–8

Nowadays, the main environment influencing medical students is the clinical learning environment, which helps them merge their theoretical training with clinical skills.^9,10 Since the learning environment can only be assessed through its outcomes, the learning process, as the most important outcome, seems to be the perfect choice for its evaluation.^11,12 This process is influenced by a wide variety of factors. Therefore, all the effective factors must be considered for its complete evaluation.¹³ These factors themselves can be categorized into internal and external factors. The first refers to mental qualities, objective and intrinsic motivation, and sentiment, while the second includes educational facilities, methods, and external stimulators of the educational environment.^6,11

In the recent years, several studies have been conducted to develop and validate new instruments for assessment of learning environments in medical education, such as Dundee Ready Education Environment Measure (DREEM), Postgraduate Hospital Educational Environment Measure (PHEEM), Clinical Learning Environment Scale, and Dutch Residency Education Environments Test questionnaire.^14–21 Review of the literature indicated that all these tools were more suitable for inpatient medical learning.^{16,18,21–24}

Ambulatory or outpatient care is the medical care given to the patients who do not need hospital admission. In addition to diagnosis, follow-up, consultation, interventional treatments, and rehabilitation services, the ambulatory care may also include medical procedures.^25–28 As suggested by World Health Organization in 2008, the ambulatory care has been considered as an important part of medical residency programs.²⁹ Thus, more attention has been paid to outpatient medical education and its learning environment in the recent years. Ambulatory Care Learning Education Environment Measure (ACLEEM) questionnaire was designed by Riquelme et al²³ in 2013 to assess the educational environment of residency medical clinics in Pontifical Catholic University of Chile in Latin America.

Considering the importance of utilization of new and improved evaluation methods, such as ACLEEM, in outpatient medical education enhancement and lack of studies based on this method in Iran, an attempt was made to adapt a valid and reliable Persian translation of this questionnaire in Shiraz University of Medical Sciences (SUMS).

Materials and methods

Design and setting of the study

This cross-sectional study was conducted on Persian-speaking medical residents of SUMS, Shiraz, Iran, in 2014–2015.

Sample size and participants

Several researchers have provided conflicting viewpoints and rules of thumb for sampling. Most of the researchers have recommended that the sample size ranging from 100 to 400 is adequate for testing the construct validity. On the other hand, others have suggested five or ten times the number of the items.³⁰ Therefore, because of the limited number of residents in SUMS, all of them were invited to participate in the study. The inclusion criteria of the study were being Iranian, speaking Persian, and having experience of teaching and learning in the ambulatory setting in different specialty fields, including psychiatry, pediatrics, surgery, orthopedics, dermatology, otorhinolaryngology, ophthalmology, neurology, internal medicine, obstetrics and gynecology, anesthesia, pathology, traditional Persian medicine, cardiology, and rehabilitation and physical medicine, in their residency period at least for one course (up to 5 months). Overall, 180 residents aged 25–46 years participated in the study.

Instrument

ACLEEM questionnaire was administered by the researcher and members of the Education Development Office (EDO) in the educational groups with the support of the Education Developmental Center, Quality Improvement in Clinical Education Research Center, special departments, and heads of each educational group of SUMS.

This instrument contains 50 questions scored based on a five-point Likert scale ranging from 4 (strongly agree) to 0 (strongly disagree). The 50 items were embedded in three domains, including clinical teaching (items 1–16), clinical training (items 17–38), and support (items 39–50). Because items 24 and 27 contained negative statements, we had to reverse code the scores for these questions. The maximum score of the questionnaire was 200.²³

Development of the Persian version of ACLEEM questionnaire

After taking consent from Arnold Riquelme, the author of the original version of ACLEEM questionnaire, forward–backward method was used to translate this questionnaire from English into Persian. First, the questionnaire was translated into Persian by two experts in English language at EDO of SUMS. Then, backward translation was done by another linguist. After that, the final English version was compared to the original version of the questionnaire and confirmed by an expert linguist. After all, a professional bilingual translator made the final Persian version of the questionnaire. The last confirmed version of ACLEEM questionnaire was used for evaluation of the psychometric properties.

Data collection

The obtained questionnaire was administered in printed format. Before distributing the questionnaires, the objectives of the project were explained to the residents. In addition, the residents were ascertained that they were free to leave the study at any time. It should be noted that each questionnaire took ~10–15 minutes to be filled.

Research ethics

Arnoldo Riquelme’s license to use this tool was received by email. This study was approved by the ethics committee of SUMS. The main idea, purpose, and benefits of the study were described for the medical residents. Then, the participants announced their verbal consent to take part in the study on a voluntary basis. Additionally, all the residents filled the questionnaires anonymously.

Reliability and validity assessment and statistical analysis

In order to assess and confirm the cultural adaptation, linguistic equivalency, and content validity of the final Persian version of ACLEEM questionnaire, the instrument was given to expert team members involving five faculty members of Education Developmental Center who were proficient in medical education. Moreover, the reliability of the ACLEEM domains was tested using Cronbach’s alpha coefficient. Internal consistency was considered to be acceptable if the coefficient was at least 0.7.³¹ Furthermore, appropriate statistical analyses, including minimum and maximum scores, mean ± SD, corrected item-total correlation, items’ scores, and Cronbach’s alpha with and without any specific items, were applied for determination of the properties of the items.

Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity were used to specify whether factor analysis could be useful with our data due to proportion of variance in the variables.

Moreover, exploratory factor analysis with Varimax rotation was utilized to determine the construct validity of the Persian version of ACLEEM questionnaire. Convergent and discriminant validities, the two aspects of construct validity, were examined using Spearman’s correlation. Accordingly, values ≥0.40 represented appropriate convergent validity. Besides, discriminant validity was supported whenever the correlation between an item and its hypothesized domain was higher than its correlation with the other domains. Convergent and discriminant validities were computed using the following formulas:³¹ $\text{Convergent validity}=\frac{\text{Number of correlations between}\text{items and hypothesized scale}\text{corrected for overlap}\ge 0.4}{\text{Total numbe of convergent validity tests}}$(1) $\text{Discriminated validity}=\frac{\text{Number of convergent correlations significantly}\text{higher than discriminant correlations}}{\text{Total numbe of correlations}}$(2)

All the statistical analyses were performed using the SPSS statistical software, version 14 and MedCalc^® statistical software.

Results

In total, 174 of the 180 residents (96.66%) returned the questionnaires. Six questionnaires were excluded (3.33%) since they were incomplete. The participants included 98 (56.3%) males and 76 (43.7%) females. The mean ± SD of the residents’ age was 31.89±4.08 years. Besides, the mean ± SD of the male and female residents’ age was 32.43±4.45 years and 31.04±3.27 years, respectively, and the difference was not statistically significant (P=0.06). The characteristics of the residents have been presented in Table 1.

Table 1 The characteristics of the residents who completed ACLEEM questionnaire

Sex: n (%)	Male: 98 (56.3%)
	Female: 76 (43.7%)
Age: mean ± SD (years)	Male: 32.43±4.45
	Female: 31.04±3.27
	P=0.06
Training level of the residents: n (%)
Postgraduate year 1	25 (14.4%)
Postgraduate year 2	53 (30.5%)
Postgraduate year 3	53 (30.5%)
Postgraduate year 4	43 (24.7%)

Abbreviations: ACLEEM, Ambulatory Care Learning Educational Environment Measure; SD, standard deviation.

Cultural adaptation, linguistic equivalency, and content validity

Cultural adaptation, linguistic equivalency, and content validity of the Persian version of ACLEEM questionnaire were confirmed by all the five faculty members who were expert in medical education in EDO. The content validity indexes of all items in the scale were >0.90, and none of the questions were removed.

Reliability

The reliability of the 50-item questionnaire was confirmed with Cronbach’s alpha of 0.964. All the three domains of the Persian version of ACLEEM questionnaire also showed sufficient reliability (Cronbach’s alpha coefficients >0.7). Minimum and maximum scores, mean ± SD scores, and Cronbach’s alpha coefficients of each domain have been shown in Table 2.

Table 2 Minimum and maximum scores, mean scores, and Cronbach’s alpha of the dimensions of the Persian version of ACLEEM questionnaire

Domains	Number of items	N	Minimum score (%)a	Maximum score (%)b	Mean score ± SDc	Cronbach’s alpha
Clinical teaching	16	174	0/64 (0.6)	64/64 (2.3)	2.48±0.77	0.947
Clinical training	22	174	12/88 (0.6)	84/88 (0.6)	2.40±0.65	0.924
Support	12	174	0/48 (1.1)	48/48 (1.1)	2.01±0.75	0.895
Total	50	174	12/200 (0.6)	192/200 (0.6)	2.34±0.64	0.964

Notes:

a The number in parentheses indicates the percentage of the minimum score.

b The number in parentheses indicates the percentage of the maximum score.

c Sum of the scores in each dimensions/number of questions of each dimensions.

Abbreviations: ACLEEM, Ambulatory Care Learning Educational Environment Measure; SD, standard deviation.

Construct validity

Using factor analysis for assessment of sample adequacy, KMO index was found to be 0.928 and Bartlett’s test yielded the following results: X ²=6,717.551, df =1,225, and P≤0.001. All the initial communalities were equal to 1, all the extracted communalities were ≥0.4, and only two questions (24 and 27) had extracted communalities <0.3. Moreover, all the questions of the clinical teaching domain, 63.63% of the questions of the clinical training domain, and 75% of those of the support domain showed strong correlation coefficients (>0.4) and had clear loading for the main domains. The results of the factor analysis with Varimax rotation used to test the construct validity of the Persian version of ACLEEM questionnaire have been presented in the three last columns of Table 3.

Table 3 Items analysis and factor loading (Varimax rotation) of the three-factor solutions for the Persian version of ACLEEM questionnaire

Item	N	Minimum score (score=0), n (%)	Maximum score (score=4), n (%)	Mean	SD	Mean score of the scale if item deleted	Variance of the scale if item deleted	Corrected item-total correlation	Cronbach’s alpha if item deleted	Factor analysis (Varimax rotation)
										Clinical teaching	Clinical training	Support
Q1	174	2 (1.1)	41 (23.6)	2.96	0.842	114.41	1,014.012	0.551	0.965	0.441	0.483	−0.023
Q2	174	2 (1.1)	40 (23)	2.87	0.887	114.50	1,006.899	0.651	0.964	0.596	0.349	0.017
Q3	174	2 (1.1)	33 (19)	2.75	0.888	114.61	1,008.631	0.618	0.964	0.594	0.379	−0.001
Q4	174	14 (8)	23 (13.2)	2.30	1.145	115.06	990.256	0.732	0.964	0.764	0.141	0.175
Q5	174	9 (5.2)	27 (15.5)	2.48	1.068	114.89	993.987	0.730	0.964	0.662	0.357	0.224
Q6	174	9 (5.2)	28 (16.1)	2.39	1.126	114.98	991.109	0.732	0.964	0.743	0.165	0.204
Q7	174	3 (1.7)	47 (27)	2.87	0.989	114.49	1,004.159	0.625	0.964	0.599	0.477	−0.112
Q8	174	8 (4.6)	24 (13.8)	2.29	1.080	115.08	995.404	0.701	0.964	0.724	0.338	0.053
Q9	174	8 (4.6)	18 (10.3)	2.14	1.057	115.22	997.366	0.686	0.964	0.658	0.220	0.299
Q10	174	4 (2.3)	25 (14.4)	2.49	0.996	114.88	999.552	0.695	0.964	0.627	0.400	0.127
Q11	174	7 (4)	18 (10.3)	2.25	1.060	115.12	995.598	0.711	0.964	0.724	0.253	0.121
Q12	174	9 (5.2)	32 (18.4)	2.37	1.154	115.00	1,000.624	0.580	0.964	0.609	0.200	0.020
Q13	174	8 (4.6)	20 (11.5)	2.24	1.069	115.13	992.331	0.754	0.964	0.814	0.144	0.199
Q14	174	10 (5.7)	23 (13.2)	2.37	1.104	114.99	998.318	0.642	0.964	0.667	0.155	0.249
Q15	174	9 (5.2)	26 (14.9)	2.36	1.123	115.01	991.960	0.722	0.964	0.716	0.196	0.247
Q16	174	6 (3.4)	32 (18.4)	2.59	1.043	114.78	994.475	0.741	0.964	0.813	0.086	0.180
Q17	174	5 (2.9)	35 (20.1)	2.71	1.025	114.66	995.117	0.744	0.964	0.610	0.546	0.007
Q18	174	7 (4)	34 (19.5)	2.67	1.038	114.70	999.970	0.659	0.964	0.523	0.604	−0.123
Q19	174	7 (4)	37 (21.3)	2.70	1.045	114.67	1,000.545	0.645	0.964	0.630	0.471	−0.047
Q20	174	5 (2.9)	40 (23)	2.84	0.952	114.52	1,006.274	0.614	0.964	0.682	0.405	−0.085
Q21	174	6 (3.4)	36 (20.7)	2.63	1.050	114.74	998.563	0.673	0.964	−0.729	0.222*	0.107
Q22	174	14 (8)	20 (11.5)	2.20	1.173	115.17	1,002.429	0.545	0.965	0.505	0.184*	0.345
Q23	174	23 (13.2)	13 (7.5)	1.88	1.179	115.49	998.552	0.595	0.964	0.450	0.179*	0.512
Q24	174	14 (8)	32 (18.4)	1.80	1.238	115.56	1,048.698	−0.074	0.967	0.092	–0.339*	−0.144
Q25	174	15 (8.6)	16 (9.2)	2.01	1.138	115.36	994.705	0.673	0.964	0.666	0.206*	0.294
Q26	174	8 (4.6)	24 (13.8)	2.39	1.084	114.98	1,000.965	0.615	0.964	0.357	0.557	0.113
Q27	174	14 (8)	13 (7.5)	1.87	1.070	115.50	1,049.939	−0.098	0.967	−0.064	–0.193*	−0.034
Q28	174	3 (1.7)	20 (11.5)	2.36	0.992	115.01	1,001.890	0.660	0.964	0.524	0.276*	0.333
Q29	174	4 (2.3)	29 (16.7)	2.68	0.936	114.68	1,008.680	0.584	0.964	0.341	0.599	0.209
Q30	174	2 (1.1)	22 (12.6)	2.51	0.923	114.86	1,001.611	0.716	0.964	0.516	0.521	0.160
Q31	174	2 (1.1)	18 (1.3)	2.57	0.876	114.80	1,004.809	0.697	0.964	0.556	0.480	0.073
Q32	174	21 (12.1)	23 (13.2)	2.13	1.243	115.24	996.381	0.591	0.964	0.395	0.235*	0.432
Q33	174	6 (3.4)	30 (17.2)	2.76	0.955	114.61	1,002.494	0.676	0.964	0.437	0.616	0.103
Q34	174	3 (1.7)	24 (13.8)	2.41	0.986	114.95	1,002.634	0.652	0.964	0.472	0.533	0.137
Q35	174	5 (2.9)	18 (10.3)	2.53	0.923	114.84	1,001.812	0.713	0.964	0.461	0.621	0.076
Q36	174	4 (2.3)	21 (12.1)	2.60	0.931	114.77	1,003.820	0.672	0.964	0.433	0.682	0.003
Q37	174	12 (6.9)	22 (12.6)	2.33	1.103	115.04	1,001.704	0.593	0.964	0.418	0.639	0.143
Q38	174	6 (3.4)	27 (15.5)	2.54	1.006	114.83	999.230	0.692	0.964	0.418	0.639	0.143
Q39	174	9 (5.2)	22 (12.6)	2.30	1.067	115.06	994.233	0.727	0.964	0.695	0.079	0.408
Q40	174	15 (8.6)	16 (9.2)	2.05	1.137	115.32	999.225	0.609	0.964	0.404	0.188	0.580
Q41	174	18 (10.3)	16 (9.2)	2.13	1.128	115.24	996.482	0.654	0.964	0.362	0.327	0.577
Q42	174	16 (9.2)	22 (12.6)	2.29	1.158	115.07	1,000.035	0.586	0.964	0.415	0.289	0.400
Q43	174	9 (5.2)	20 (11.5)	2.51	1.001	114.86	1,001.992	0.651	0.964	0.234	0.637	0.276*
Q44	174	9 (5.2)	21 (12.1)	2.49	0.996	114.87	1,012.897	0.480	0.965	0.036	0.617	0.356*
Q45	174	12 (6.9)	19 (10.9)	2.32	1.052	115.05	1,006.535	0.549	0.964	0.112	0.586	0.477
Q46	174	30 (17.2)	18 (10.3)	1.97	1.292	115.40	1,012.946	0.361	0.965	0.054	0.215	0.620
Q47	174	56 (32.2)	10 (5.7)	1.37	1.251	115.99	1,014.514	0.354	0.965	0.117	−0.077	0.799
Q48	174	64 (36.8)	8 (4.6)	1.20	1.176	116.17	1,019.415	0.313	0.965	0.124	−0.133	0.831
Q49	174	20 (11.5)	13 (7.5)	1.87	1.117	115.50	1,011.974	0.437	0.965	0.098	0.102	0.751
Q50	174	18 (10.3)	11 (6.3)	2.02	1.109	115.35	1,016.911	0.370	0.965	0.119	0.312	0.600

Notes: Bold values show items belonging to subscales in the original version of ACLEEM questionnaire.

* These items have coefficient less than 0.40 in own subscales.

Abbreviations: ACLEEM, Ambulatory Care Learning Educational Environment Measure; SD, standard deviation.

Our findings showed that the scaling success rate for convergent validity was 100% for teaching and support domains and 95% for the training domain. Additionally, the success rates for discriminant validity were 87.5% (28/32), 54.54% (24/44), and 79.16% (19/24) for teaching, training, and support domains, respectively (Table 4).

Table 4 Item scaling test: convergent and discriminant validities of the Persian version of ACLEEM questionnaire

Domains	Number of items	Convergent validitya		Discriminant validityb
		Range of correlation	Scaling success (%)	Range of correlation	Scaling success (%)
Clinical teaching	16	0.62–0.85	16/16 (100)	0.28–0.96	28/32 (87.5)
Clinical training	22	0.20–0.75	21/22 (95)	0–0.74	24/44 (54.54)
Support	12	0.55–0.75	12/12 (100)	0.14–0.70	19/24 (79.16)

Notes:

a Number of correlations between the items and the hypothesized scale corrected for overlap ≥0.4/total number of convergent validity tests.

b Number of convergent correlations significantly higher than discriminant correlations/total number of correlations.

Abbreviation: ACLEEM, Ambulatory Care Learning Educational Environment Measure.

Items analysis

The mean score of the total instrument was 2.34±0.64. The subscale mean scores for clinical teaching, clinical training, and support were 2.48±0.77, 2.40±0.65, and 2.01±0.75, respectively (Table 2).

The lowest mean score of the items was 1.20 (for item 48: “there are adequate bathroom facilities in the outpatient clinic [OPC]”), while the highest mean score was 2.96 (for item 1: “working in the OPC enables me to develop my problem solving skill”). Overall, none of the mean scores of the items were >2.96, and the mean scores of the seven items (23, 24, 27, 46, 47, 48, and 49) were <2. The results of items analysis indicated that removing or retaining a special item did not change the Cronbach’s alpha. The items analysis for the Persian version of ACLEEM questionnaire has been presented in Table 3.

Discussion

Learning environment is one of the curtail factors, which has a significant impact on medical students’ success and satisfaction.^9,32 Therefore, educational systems need a certain instrument to assess the quality of the medical environment given for this goal.³³ The ambulatory medicine program is suggested as an important section of the educational course for development of medical residents’ clinical skills by providing focused teaching with increased exposure to patients with a wide range of chronic medical conditions in a dedicated learning environment.³⁴ ACLEEM questionnaire was the first specialized instrument developed to evaluate various aspects of the learning environment in outpatient clinics. It is also a unique instrument in this respect.²³ Therefore, this study was designed to assay the validity and reliability of the Persian version of ACLEEM questionnaire to determine whether this version is appropriate for measurement of the residency medical ambulatory environments in Iran.

Reliability and repeatability discuss the casual variability associated with measurements.³⁵ In this study, the Cronbach’s alpha coefficient of the instrument was 0.964, which is in line with the results of the study by Riquelme et al³⁶ in 2015. Additionally, Cronbach’s alpha coefficients were >0.70 for all the three subscales. Thus, the reliability of the Persian version of ACLEEM questionnaire was confirmed.

Content validity is related to the proportion to which the items are sensible and reflect the conscious domain of interest.³⁰ None of the items in the Persian version of ACLEEM instrument were removed due to confirmation of content validity by an expert five-member team with excellent content validity indexes.

Construct validity illustrates the ability of the scale to measure what it is assumed to measure. Convergent and discriminant validities are the two important aspects of construct validity.³⁰ The acceptable values of KMO and Bartlett’s sphericity test indicated that our study sample was sufficient and factor analysis could be done for evaluating the construct validity. In this study, scaling success of convergent validity was excellent for all the three dimensions, as shown in Table 4. Besides, discriminant validity was >70% in clinical teaching and support domains, but not in the clinical training domain. This implies that the questions of clinical teaching and support domains, but not those of the clinical training domain, were well understood by the residents to assess the relevant aspects. Up to now, no studies have assessed the convergent and discriminant validities of this questionnaire to compare the results.

In our study, the mean score of total questionnaire and all three subscales were >2, which reveals a more positive than negative environment in all dimensions. Clinical teaching dimension had the highest mean score among the others. The mean scores of seven items (23, 24, 27, 46, 47, 48, and 49) were <2, the expected mean score of each item. This can be attributed to inadequate supplies and instruments at the clinics of SUMS as well as to Iranian culture. For instance, “opportunity for visiting and following up the patients” in items 23 and 24, “adequacy of the number of supervisors in clinics” in item 27, “computer access” in item 46, and “provide lockers to keep the residents’ belongings safe” in item 47 were the components focused on instruments and supplies, and “adequate bathroom facilities” in item 48 might refer to Iranian culture and religious practices related to bathing in private homes rather than clinics, in addition to instrument inadequacy.

The results of factor analysis revealed that the majority of the questions were included in the questionnaire.³⁰ Riquelme et al²³ divided the original ACLEEM questionnaire into three domains, including clinical teaching, clinical training, and support. These three domains were also used in factor analysis in this study. However, the items were distributed differently in the domains of the Persian questionnaire. The results of factor analysis also indicated that the Persian version of ACLEEM questionnaire did not exactly evaluate the three underlying constructs reported in the original version.²³ Accordingly, all the 16 items (100%) of the clinical teaching domain were clearly loaded in the expected domain. All the items of the support domain, except for items 43 “I feel part of the team in the OPC” and 44 “I receive support from other OPC residents when I need it”, were also clearly loaded in the appropriate territory in factor analysis. Items 43 and 44 were loaded for the clinical training domain in factor analysis with Varimax rotation. However, eight of the 22 items (36%) of the clinical training domain (21, 22, 23, 24, 25, 27, 28, and 31) had no clear loading for this domain. On the other hand, ten of the 22 items (45.45%) of the clinical training domain (17, 19, 20, 22, 23, 25, 26, 28, 29, and 31) had higher coefficients in the clinical teaching domain in factor analysis. This may imply that the study residents had a different perception and attitude toward some items. In other words, the Persian version of this questionnaire could have a different structure. Thus, another factor analysis could be performed for finding more domains in the Persian version of this instrument. The results also indicated that items 24 and 27 had a low coefficient in all the domains in factor analysis with Varimax rotation. Therefore, these items had a little capability for assessment of the medical ambulatory setting in Iranian residents.

Jalili et al¹⁸ conducted a study in 2014 for validating the Persian version of PHEEM questionnaire for measurement of the educational environment in academic emergency department. The results revealed that this instrument had a good reliability with Cronbach’s alpha of 0.86. However, the Persian version of PHEEM had a different structure in factor analysis. The results of this study are also approximately similar to those obtained by Jalili et al. In another study performed by Khan et al³⁷ in Pakistan in 2011, the internal consistency of DREEM instrument was confirmed with Cronbach’s alpha of 0.91. However, a large mismatch in structure was observed compared to the original version.³⁷ This similarity regarding structural differences in ACLEEM, PHEEM, and DREEM in different nationalities could be associated with the participants’ cultural diversity.

Our study had some limitations, the first of which being the small sample size, which was due to the small number of individuals passing the residency examination for SUMS each year as well as to the diversity at the time of beginning the residency training in ambulatory care setting in different specialty fields. Moreover, in order to have a more uniform sample, the residents had to have at least 5 months of experience upon entrance and all the residents who met this inclusion criterion were entered into the study. Another study limitation was that the residents filled the questionnaires anonymously. Therefore, it was not possible to assess all the residents for the second time, and consequently, the temporal stability of the instrument could not be assayed. Third, because ACLEEM questionnaire is the only available instrument for evaluating the ambulatory care learning educational environment,²³ it is not possible to test the criterion validity to assess whether it has empirical association with external criteria, such as other established instruments. Finally, our participants were distributed in different levels of residency course from first to fourth years. Therefore, the results might have been affected by the heterogeneity in their medical experiences and skills.

Overall, our study findings revealed excellent internal consistency, excellent convergent validity, and good discriminant validity of the Persian version of ACLEEM questionnaire for evaluation of the residency outpatient clinics, environments. Therefore, this valid and reliable instrument can be useful to improve the medical ambulatory settings, eventually leading to progress of teaching and training practices. This instrument can also help managers monitor the performance of the educational systems, faculty members of medical schools, and residents. Furthermore, this tool lets educational systems identify the weak points of medical ambulatory settings and improve processing of the goals. Future studies are recommended to assess the quality of clinical teaching and training and support the residents of SUMS to detect and resolve the gaps in learning programs and medical ambulatory settings.

Conclusion

Based on our findings, the Persian version of ACLEEM questionnaire was a multidimensional, simple, practical, valid, and reliable instrument for assessing the environment of outpatient clinics of medical residents. It also provided a useful index of the priorities for curriculum improvement. Yet, ACLEEM questionnaire had a different structure from the original version and measured more than three aspects of outpatient clinics for Persian residents.

Author contributions

MMP, MA, MRD: collected the data, wrote the manuscript, conceived, designed, and did statistical analysis, and revised the manuscript. MA, PJ: conceived, designed, and did statistical analysis and revised the manuscript. ZP: collected the data and wrote the manuscript.

Acknowledgments

This article was extracted from the thesis written by Mohammad Mahdi Parvizi as the partial fulfilment of the requirements for obtaining Master’s degree in Medical Education in Shiraz University of Medical Sciences (proposal number 93-6964). Hereby, the authors wish to thank the residents who participated in this study and all the individuals who helped in performing the project. They are also grateful for Ms A. Keivanshekouh at the Research Improvement Center of Shiraz University of Medical Sciences for improving the use of English in the manuscript.

Disclosure

The authors declare that they have no conflicts of interest in this work.