A validity and reliability study of the Attitudes toward Sustainable Development scale

Abstract

This article describes the development and validation of the Attitudes toward Sustainable Development scale, a quantitative 20-item scale that measures Italian university students’ attitudes toward sustainable development. A total of 484 undergraduate students completed the questionnaire. The validity and reliability of the scale was statistically tested by computing the KMO and Bartlett tests and via an exploratory factor analysis, descriptive statistics, Cronbach’s alpha, a confirmatory factor analysis and a multi-group invariance testing. The results of the principal components factor analysis show that the scale consists of the following four dimensions, with five items in each: environment, economy, society and education. The overall structure and measurement of the scale are confirmed by the confirmatory factor analysis and by the multi-group invariance testing. Internal reliability, which was found using Cronbach’s alpha, varies between .660 and .854. The results show that the instrument meets the validity and reliability criteria. To demonstrate its utility, the scale was applied to detect differences in sustainable development attitudes among students pursuing degrees in psychology and in agriculture. Relevant differences were detected for the dimensions of environment and society. The Attitudes toward Sustainable Development scale could be useful for understanding the ways in which students think about sustainability issues and could be used to investigate the relationship between sustainability attitudes and other variables.

Keywords:

• education for sustainable development
• sustainability assessment
• higher education
• sustainability attitudes
• instrument validation

Introduction

Over the past few decades, a broad body of research has been conducted to respond to environmental issues that induce a change of view about the relationships between humans and nature. Education was considered to be a core discipline for disseminating sustainable development principles, and increased attention was dedicated to Education for Sustainable Development (ESD) (Cotton et al. 2007; Michalos et al. 2012; Olsson, Gericke, and Chang Rundgren 2015; Tilbury 2012). ESD refers to educational programs and experiences that are designed to allow people to acquire the knowledge, skills and values that are necessary to shape a sustainable future. Chapter 36 of Agenda 21 (UNESCO 1992) was one of the first calls for action on education for sustainability and provided a basis for developing international networks on ESD that address the following three purposes: re-orienting education toward Sustainable Development (SD), increasing public awareness and promoting training. Several other initiatives were implemented by UNESCO, such as the United Nations’ decade of education for SD (UNESCO 2005), to internationally support and improve the integration of ESD into educational strategies and educational action plans in all of the member countries. The purposes of the decade included the following (UNESCO 2014):

•	Incorporating quantitative and qualitative ESD indicators into the on-going monitoring and evaluation of education for all and into the United Nations’ literacy decade.
•	Monitoring the progress of activities undertaken by the United Nations agencies, governments and NGOs in observance of the decade and facilitating their implementation and follow-up.
•	Evaluating the achievement of measurable results in pursuing the aims and objectives of the decade, particularly with regard to the integration of ESD into national educational policies, programs and systems.
•	Making recommendations to further promote ESD based on the results and lessons that are learned from the decade.

These purposes demonstrate that there is a need for tools and measures to assess ESD. During the UNESCO decade, several initiatives and projects have been carried out in higher education institutions, such as the Reorient University Curricula to Address Sustainability (RUCAS) project, an EU-Tempus initiative. The main objective of the RUCAS project was to reorient toward SD the curricula of several courses of 11 European and Middle East universities, providing knowledge, skills, perspectives and values of sustainability. Several initiatives were carried out in the framework of the RUCAS project to develop resources, revise and foster new curriculum initiatives, build capacity and strengthen national and regional networks (Kostoulas-Makrakis and Makrakis 2012). The following main actions were undertaken: an ESD student competence framework was developed, validated and implemented, and a curricula revision process was realised to infuse sustainable development principles in university courses in schools of economics, education, engineering, applied sciences and social sciences (Makrakis and Kostoulas-Makrakis 2012). The professors participating in RUCAS revised their courses to address sustainability and implemented the revised courses into their classes. The activities of revising the curricula induced the adoption of transformative teaching methods, often resulting in a change in teaching style. As an effect of the curricula revision, the professors moved from lectures to more student-centred teaching methods based on having clarified their own values and critical thinking, as reported by Kostoulas-Makrakis and Makrakis (2012).

The current study presents a quantitative tool that could be used for assessing the effects of curricula revision after having infused SD principles. Moreover, the focus is on the development and validation of this scale, which measures SD attitudes in Italian university students. In addition, the scale was applied to detect differences in SD attitudes among university students pursuing different degrees. Differences are expected according to the diverse values based on major. The purpose of the comparison was to demonstrate the utility of the scale. The theoretical background considers previous studies on the construction and validation of tools for measuring competences, attitudes and beliefs in Environmental Education (EE) and ESD.

Literature review

There is a growing interest in quantitative research that measures competences, attitudes and beliefs in EE and ESD, and researchers have considered several issues that are involved in the construction of these assessment tools (Michalos et al. 2012; Olsson, Gericke, and Chang Rundgren 2015; Powell et al. 2011; Schneller, Johnson, and Bogner 2015; Vagias et al. 2012). The first issue relates to the definition of environmental attitudes, because several approaches and theoretical backgrounds were used to define this concept. Clear and widely shared definitions of attitudes have not yet been developed for SD, and psychosocial variables have emerged as a major component in the literature. In addition, there is no agreement on the use of a common tool that measures environmental and SD attitudes, due to the different theoretical backgrounds that underlie the various approaches (Schneller, Johnson, and Bogner 2015). Different theoretical backgrounds, such as the Model of Ecological Values (2-MEV) (Bogner and Wiseman 2006; Schneller, Johnson, and Bogner 2015) and the new environmental paradigm (NEP) (Dunlap et al. 2000; Dunlap and van Liere 1978; Fleury-Bahia et al. 2015; Lundmark 2007; Shephard et al. 2011), were used to develop the different instruments in EE. A few of the most frequently applied scales include the Children’s Environmental Attitudes and Knowledge Scale (Leeming, Dwyer, and Bracken 1995) and the revised NEP scale (Dunlap and van Liere 1978; Dunlap et al. 2000). Other tools considered environmental attitudes (Milfont and Duckitt 2010) and students’ environmental attitudes, awareness, and intention to act (Bergman 2015). Most of the tools were developed in an educational setting, primarily for elementary or secondary schools (Dijkstra and Goedhart 2012; Karpudewan, Roth, and Chandrakesan 2015; Olsson, Gericke, and Chang Rundgren 2015; Powell et al. 2011) However, there are few studies about the environmental attitudes and knowledge of college students (Biasutti 2015; Shephard et al. 2011).

Regarding the assessment of SD beliefs and attitudes, there are scales measuring students’ competences, attitudes and behaviours that are based on several UNESCO documents (Biasutti and Surian 2012; Michalos et al. 2012; Olsson, Gericke, and Chang Rundgren 2015). Biasutti and Surian (2012) applied the student survey of ESD competences to compare university students’ beliefs and attitudes. The participants were 467 bachelor students in the following five areas: social sciences, educational sciences, applied sciences, engineering and health sciences. The questionnaire comprises several parts, including attitudes toward ESD and ESD competences regarding the following five fundamental types of learning: learning to be, learning to live together sustainably, learning to know, learning to do and learning to transform oneself and society. Significant differences between students of each area were found: a more enhanced pro-sustainability attitude was determined among engineering students who showed a significant advantage over applied sciences, health sciences and social sciences students. Social sciences students showed an enhanced pro-social profile, although the only group of students who were significantly less socially oriented when compared with social sciences students were health sciences students. Educational sciences students seem to prefer the social dimension over the knowledge and the ‘to do’ dimensions of sustainability when compared with engineering and applied sciences students, who scored significantly higher on the learning to know and learning to do scales.

Other tools were related to the UNESCO (2005) definition of SD, including environmental, economic, and social dimensions. Michalos et al. (2012) developed a tool for measuring tenth-grade students’ knowledge, attitudes and behaviours concerning SD. This tool comprises 50 items divided into the following three indexes: index of knowledge of SD, index of favourable attitudes toward SD and index of favourable behaviours toward SD. Olsson, Gericke, and Chang Rundgren (2015) developed a Likert-scale questionnaire to evaluate the sustainability consciousness in pupils from sixth to ninth grades. The environmental, economic, and social dimensions of SD, in terms of sustainability knowingness, attitudes, and behaviours, have been considered. These last two tools have been conceived for school pupils and not for university students. The Olsson, Gericke, and Chang Rundgren (2015) scale is based on environment, economy and society, which are considered the pillars of SD, as affirmed by UNESCO (2005). The environment refers to the development of an awareness of the resources and of the vulnerability of the physical environment. The economy concerns an awareness of limits, the potential of economic growth, and how they could impact on the environment and society. Society is considered to be a system based on democracy, which offers to citizens the possibility of actively participating in the policy life, expressing different opinions and electing governments. Environment, economy and society are viewed as interrelated, and this framework was used by several research studies on SD (Gough 2002; Giddings, Hopwood, and O’Brien 2002; Olsson, Gericke, and Chang Rundgren 2015; Walshe 2008). However, many studies lack another important dimension – education – that is transversal to these SD pillars. Education is a fundamental component of ESD and of the UNESCO mission and is a core aspect of Agenda 21 (UNESCO 1992). The role of education is considered in many chapters of Agenda 21, with a specific focus in Chapter 36 (UNESCO 1992; Section 36.3): education is considered crucial for supporting sustainable development and for advancing the ability of the people to address sustainable development issues. Education must be considered equally as it relates to the other components of SD and is essential for developing environmental and ethical awareness in mankind, including values and attitudes that are consistent with sustainable development. The aim is to promote an effective public participation in decision-making and in the policy life. To produce effective improvement in the quality of knowledge and attitudes toward sustainability, constructivist learning theories and learner-centred methodologies should be considered (Biasutti 2015; Corney and Reid 2007; McNaughton 2012; Scoullos 2013). These theories should be based on innovative teaching methods, the promotion of future-oriented thinking and higher order thinking skills, interdisciplinarity and the linking of local and global issues. Scoullos (2013, 110) outlined the following characteristics of ESD learning methods:

•	Interdisciplinary and holistic.
•	Learner-centred and participatory.
•	Values-driven, promoting critical thinking and exploring all interested ‘sides’.
•	Forward-looking, promoting medium and long-term planning.
•	Locally relevant, encouraging multilateral collaborations among schools, local actors and authorities, scientific communities, the private sector and NGOs, etc., and,
•	Revealing global issues and connections as part of everyday life, whether in a small village or a large city.

These teaching/learning methods promote changes in behaviour and ways of thinking and relate not only to knowledge but also to processes, because these methods teach learners how to think – not what to think (Biasutti 2015).

In the reviewed literature, several methods have been adopted for assessing environmental attitudes. The theoretical approaches that were used to develop the questionnaires were based on models such as the Model of Ecological Values and the NEP. Many studies added further knowledge to this literature, but we might wonder how these scales are different theoretically and conceptually from SD: a scale that measures environmental attitudes does not necessarily mean that it measures attitudes toward the environmental dimension of sustainability. Regarding the SD questionnaires, only a limited number of tools were developed, mainly concerning primary and secondary school students’ knowledge, attitudes and behaviours (Michalos et al. 2012; Olsson, Gericke, and Chang Rundgren 2015). These SD tools were based on the three dimensions – environment, economy and society – but they lack a focus on education, which is a crucial component of ESD. The current research aims to address these gaps by developing a quantitative scale that measures SD attitudes in university students, adding the new dimension ‘education’ to the three pillars of SD.

Purposes of the current study

The purpose of the current study is to develop and validate a quantitative scale for measuring SD attitudes in Italian university students, the Attitudes toward Sustainable Development scale (ASD). This scale is based on four dimensions – environment, economy, society and education – and the following leading questions were considered:

(1)	Are the four dimensions of the tool confirmed by the exploratory and confirmatory factor analysis (CFA)?
(2)	Does the tool meet the reliability (Cronbach’s alpha) and stability criteria?
(3)	Can the ASD scale detect difference in SD attitudes among university students pursuing different degrees?

Method

Participants and procedure

Five hundred and five students were enrolled to complete the questionnaire. Twenty-one of these questionnaires were not considered because some data were missing; the questionnaires used for the statistical analyses totalled 484 (N = 128 male, N = 356 female). The participants were undergraduate students at a university located in northeast Italy. They were enrolled in the first year (mean age = 20.1, minimum age-maximum age = 18–36) of the following degrees: agriculture (N = 67) engineering (N = 34), primary education (N = 30), and psychology (N = 353). The scale was administered in Italian to the students at the beginning or the end of a lesson. No teaching that was relevant to SD took place during these courses. The data were collected over five months and were randomly separated into two subsamples: one subsample was assigned to the exploratory factor analysis group, and the other subsample was assigned to the CFA group. A part of the original sample of 484 (97 participants) completed the questionnaire at two different times (approximately two-and-a-half months apart).

Theoretical framework and item development of the ASD

The theoretical framework of the ASD considered the following four dimensions: environment, economy, society and education, which were previously discussed. The current research differs from other studies because previous research was focused on the five types of learning: learning to be, learning to live together sustainably, learning to know, learning to do and learning to transform oneself and society (Biasutti and Surian 2012). Alternatively, the previous research was focused on the following three UNESCO dimensions – environment, economy and society – without considering education (Michalos et al. 2012; Olsson, Gericke, and Chang Rundgren 2015). It was considered crucial to ask Italian university students to reflect also on the role of education in SD to produce awareness about SD.

The procedures outlined by DeVellis (2003) were followed to develop the scale: first a literature review was carried out to determine the goals of the measurement, and then an item pool and the format for the measurement were generated. The item pool was reviewed by two experts to foster the validity of the scale. The final steps included administering the scale to a sample, evaluating the items through statistical procedures and investigating the reliability of the scale. After having defined the four dimensions – environment, economy, society and education – as a theoretical framework for constructing ASD, the research team worked on developing the questionnaire items. A literature review was performed before developing the items of the scale. Relevant related questionnaires that measured interests, attitudes and beliefs toward environmental and SD were examined, such as the revised NEP (Dunlap et al. 2000), the Revised 2-MEV scale by Schneller, Johnson, and Bogner (2015) and the scales by Michalos et al. (2012) and Olsson, Gericke, and Chang Rundgren (2015). Moreover, the following topics of the UNESCO (2005) and UN (2012) documents were considered:

• Environment: natural resources, climate change, rural development, sustainable urbanization, disaster prevention and mitigation.

• Economy: poverty reduction, corporate responsibility, market economy.

• Society: human rights, gender equity, peace and human security, health, HIV/AIDS, governance, cultural diversity and inter-cultural understanding.

Regarding education, the following skills and characteristics of ESD learning methods outlined by Scoullos (2013) were considered: student-centred teaching methods, future-oriented thinking, higher order thinking skills, critical thinking, interdisciplinarity, and linking local and global issues.

This literature review showed that there are several topics that relate to the dimensions of environment, economy, society and education. These topics informed the development of the scales, meaning that, e.g. there were items related to the fragility of the physical environment and items to measure awareness of resources. In constructing the scale, the numbers of items (five per factor) were balanced. Summaries of the definitions and topics of the four constructs of ASD are reported in Table 1.

Table 1. Definitions and topics of the four constructs of ASD.

Constructs	UNESCO (1992, Section 36.3; and 2005, 5) definitions	Topics
Environment	The environment regards the development of an awareness of the resources, the fragility of the physical environment, and how human activity and decisions affect it, with a commitment to factoring environmental concerns into social and economic policy development	Natural resources, climate change, rural development, sustainable urbanization, disaster prevention and mitigation
Economy	The economy regards a sensitivity to limits, the potential of economic growth, and their impact on society and on the environment, with a commitment to assessing the personal and societal levels of consumption, out of concern for the environment and for social justice	Poverty reduction, corporate responsibility, market economy
Society	The society, as well as the democratic and participatory systems, provide an opportunity for the expression of opinions, the selection of governments, the forging of consensus and the resolution of differences	Human rights, gender equity, peace and human security, health, HIV/AIDS, governance, cultural diversity and inter-cultural understanding
Education	Education is critical for promoting sustainable development and improving the capacity of the people to address environment and development issues. (…) It is also critical for achieving environmental and ethical awareness, values and attitudes, skills and behaviour consistent with sustainable development and for effective public participation in decision-making	Student-centred teaching methods, future-oriented thinking skills, higher order thinking skills, interdisciplinarity, linking local and global issues

To provide validation, a panel of two ESD experts later examined the created item pool in terms of its content validity. Each expert had a PhD. One was a UNESCO programme specialist working with this organisation for 20 years; the other was a university professor for 12 years, and had more than 20 years of experience in global education, awareness on development, human rights and global issues, gained through continuous involvement in EC-funded programs, projects and institutions. The experts were asked to check for ambiguous statements and to comment on the questionnaire about the conceptual validity and the formulation of the items. These comments were considered when revising the scale, and any suggested changes were made to the items.

The validated questionnaire contains 20 items and is a self-reported scale used to measure students’ attitudes toward SD. A set of statements was presented, and participants were asked to express their agreement on a five-point Likert scale with the following answer choices: ‘strongly disagree’, ‘disagree’, ‘neutral’, ‘agree’, and ‘strongly agree’. A sample of the scale can be found in Appendix 1.

Data analysis and results

The data were analysed with IBM SPSS Statistics 20 and Lisrel 8.80 to statistically test the validity and reliability of the scale. In line with previous research (Erdogan, Ok, and Marcinkowski 2012; Ugulu 2015), the KMO and Bartlett tests, an exploratory factor analysis, descriptive statistics, Cronbach’s alpha, and a CFA were computed. The stability of the scale was assessed by using multi-group invariance testing. In addition, a group comparison was performed with a T-test to compare the students who were pursuing degrees in agriculture and psychology. These two groups were considered because they were the most representative – psychology had 353 participants, agriculture had 67 participants, while engineering and primary education had only 34 and 30 participants, respectively.

Research question one

Psychometric properties and factorial structure of the scale

The first research question asked about the validity of the four-dimension model of ASD. The KMO and Bartlett tests were the first statistical analyses performed to verify suitability of the data for an exploratory factor analysis (Ugulu 2015). A KMO value over .90 is optimal (Russell 2002), and the values of the Bartlett test suggest that the null hypothesis must be rejected when there is a significance level of .05 (Snedecor and Cochran 1989). The results included the following: KMO = .830; Bartlett test: χ² = 1338.83, df = 190 (p = .000), which indicates that an additional factor analysis on the ASD can be conducted.

The second step was to perform an exploratory factor analysis that used a Varimax rotation method to determine the links between the observed variables and underlying factors (Byrne 1998). The Kaiser criterion (Kaiser 1960) and the Scree test were used to determine the number of factors, and the factors with eigenvalues equal or superior to one were considered. A structure of four factors, with five items for each factor, was found. A name for each factor was given, as follows:

(1)	Environment (item 1–5; e.g. ‘Environmental protection is more important than industrial growth’ and ‘Building development is less important than environmental protection’).
(2)	Economy (item 6–10; e.g. ‘People should make more sacrifices in order to reduce the economic differences between populations’ and ‘Government economic policies should increase fair trade’). In spite of item 9 loading slightly higher on factor 3 than on factor 2, it was grouped with factor 2 because it is about economy rather than education.
(3)	Society (item 11–15; e.g. ‘Society should provide free basic health services’ and ‘Society should take responsibility for the welfare of individuals and families’). In spite of item 13 loading slightly higher on factor 2 than on factor 4, it was grouped with factor 4 because it is about society rather than economy.
(4)	Education (item 16–20; e.g. ‘Teachers in college should promote future-oriented thinking, in addition to historical knowledge’ and ‘Teachers in college should promote critical thinking, rather than lecturing’).

The rotated factor values ranged between .327 and .780, as reported in Table 2. The rotation was unconstrained and items with factor loadings lower than .30 are not reported. In the factors where one item loaded in other factors, the higher value was considered, with the exceptions of items 13 and 9, as reported above. The factors explained 51.68% of the total variance, as indicated in Table 3. The results of the item loadings per factor, the eigenvalues and the variance that explains the percentages of the factors confirm the four factor structure. Descriptive statistics, eigenvalues, percentages of variance and Cronbach’s alphas are reported in Table 3.

Table 2. Mean (M), standard deviation (SD), and rotated factor matrix (exploratory factor analysis) for the ASD.

ASD items	M (SD)	Factors^a
		1	2	3	4
1) When people interfere with the environment, they often produce disastrous consequences	2.92 (.90)	.599
2) Environmental protection and people’s quality of life are directly linked	4.06 (.95)	.604
3) Biodiversity should be protected at the expense of industrial agricultural production	3.32 (1.04)	.678
4) Building development is less important than environmental protection	3.82 (1.03)	.780
5) Environmental protection is more important than industrial growth	3.66 (.94)	.752	.329
6) Government economic policies should increase sustainable production even if it means spending more money	3.80 (1.00)	.386	.554
7) People should sacrifices more to reduce economic differences between populations	3.77 (.98)		.747
8) Government economic policies should increases fair trade	3.85 (.91)		.647
9) Government economic policies should act if a country is wasting its natural resources	3.56 (1.03)		.327	.459
10) Reducing poverty and hunger in the world is more important than increasing the economic well-being of the industrialized countries	4.12 (.95)		.538		.350
11) Each country can do a lot to keep the peace in the world	4.08 (.96)				.543
12) The society should further promote equal opportunities for males and females	4.38 (.85)				.479
13) The contact between cultures is stimulating and enriching	4.37 (.88)		.619		.409
14) The society should provide free basic health services	4.52 (.78)				.710
15) The society should take responsibility for the welfare of individuals and families	4.16 (.87)				.651
16) Teachers in college should use student-centred teaching methods	3.72 (.97)			.615
17) Teachers in college should promote future-oriented thinking in addition to historical knowledge	4.10 (.88)			.704
18) Teachers in college should promote interdisciplinarity between subjects	4.31 (.76)		.341	.589	.371
19) Teachers in college should promote the connection between local and global issues	4.12 (.82)		.396	.531	.304
20) Teachers in college should promote critical thinking rather than lecturing	4.35 (.86)			.667

^a Factors: 1 environment; 2 economy; 3 education; 4 society. (N = 216).

Table 3. Descriptive statistics mean (M) and standard deviation (SD), eigenvalue, percentage of variance, Cronbach’s alpha (reliability).

ASD factors	M (SD) (N = 216)	Eigenvalue (N = 216)	%Variance (N = 216)	Cronbach’s α (N = 484)
1. Environment	3.56 (.70)	5.342	14.734	.743
2. Economy	3.82 (.67)	2.582	13.376	.737
3. Education	4.12 (.61)	1.273	12.406	.757
4. Society	4.30 (.56)	1.139	11.167	.660
Total	3.95 (.46)		51.683	.854

Confirmatory factor analysis

The factor framework that was derived from the exploratory factor analysis was applied to the second study group of 268 participants so that the CFA could be performed by using the maximum likelihood method. In the CFA, all the adaptive values are reported because it is generally recommended to report more than one adaptive value (Thompson 2000). The worth of the fit values is reported in Table 4. As the values suggest, there is an acceptable fit for RMSEA (values less than 0.5 indicate good fit), S-RMR, CFI, NNFI, and IFI, and there is a perfect fit for χ²/d (Byrne 1998; Schreiber et al. 2006). The factorial model of the scale is represented in Figure 1. The CFA indicates that the four factors are confirmed, even if the GFI and AGFI are slightly lower than the middle values but close to the value 1, which is indicative of a good fit (Byrne 1998).

Table 4. Goodness of fit of CFA of ASD (N = 268) and multi-group invariance (MGI) configural and metric of pre- (N = 97) and post-test (N = 97) groups.

Model	N	χ^2(df)	RMSEA	SRMR	GFI	AGFI	CFI	NNFI	IFI
CFA	268	238.82(164)	.041	.053	.92	.89	.97	.97	.97
MGI configural	97	418.23(328)	.054				.94	.93	.94
Pre	97			.086	.83	.79
Post	97			.091	.81	.76
MGI metrical	97	425.59(348)	.048				.94	0.94	.94
Pre	97			.094	.83	.79
Post	97			.091	.81	.76

Figure 1. Confirmatory factor analysis model of the ASD (N = 268).

Research question two

Reliability and stability of the scale

The second research question asked about the reliability and stability of the scale. To determine the scale’s reliability and internal consistency, the Cronbach’s alpha reliability coefficient was calculated for each factor (values ranged between .660 and .757) and for the total score (.854). The Cronbach’s alpha of .660 for the society factor was also accepted if it was low, because some reliability values lower than .70 were reported in other preliminary studies (Biasutti and Frezza 2009; Liu 2003), and also for factors with less than six items (Kyle, Graefe, and Manning 2005). For these reasons, the Cronbach’s alpha of value of .660 is considered acceptable for this research (Ugulu 2015). The results are shown in Table 3, which indicate that the scale has good internal consistency.

The stability of the scale was tested by using the multi-group invariance testing to compute the invariance between the two scale applications on a subsample of 97 participants, who responded to the scale after about two and half months. Data were computed using the multi-group configural and metric invariance testing. The multi-group configural test produced statistics indicative of a good fitting model, comparing the factor structure and factor-loading patterns (Powell et al. 2011). The measured invariance was tested, leaving the factor loadings free. The values shown in Table 4 suggest that RMSEA has a good fit, and the CFI, IFI and NNFI are acceptable (Byrne 1998; Schreiber et al. 2006). Regarding the multi-group metric test, the relationships were verified between factors by constraining them to be equal across the two samples. The results of the analysis provided evidence that the structure of the ASD scale is the same in the two samples (RMSEA is a good fit, and the CFI, IFI and NNFI are acceptable). These findings confirmed the stability of the scale. The results are reported in Table 4.

Research question three

Group comparison

The third research question asked about the differences in SD attitudes among university students pursuing different degrees. A group comparison was performed with an independent sample T-test that compared the students who were pursuing degrees in agriculture with those studying psychology and that included Cohen’s d as the effect size index. These two groups were selected because they were the most representative; for the other groups of students, there were only a few participants who could be used to perform a comparison (e.g. primary education). Levene’s test for testing the equality of variance was computed to determine when use an equal or unequal means estimates of t. When the F was significant with p < 0.05 the unequal estimate of t was selected. The statistical analysis showed a significant difference between the ASD factors of environment, Levene’s test (F = 5.506, p = .019) with t(106.59) = –6.518, p < .001, d = 1.26, Levene’s test (F = 4.763, p = .030) and society, with t(82,26) = 4.089, p < .001, d = .90. Mean values for psychology and agriculture students for the factor environment of the ASD were M = 3.521 SD = .673 and M = 4.020 SD = .554, respectively, and for the factor society, M = 4.376 SD = .502 and M = 4.038 SD = .637. Agriculture students performed better on the environment factor than psychology students, thus demonstrating stronger attitudes toward environmental care, whereas psychology students performed better on the society factor, thus demonstrating more sensibility toward social issues.

Discussion and further developments

The current research answers the call to create evaluation tools that assess SD (UNESCO 2014) through the development and validation of a quantitative scale that measures SD attitudes in Italian university students. The results of the principal components factor analysis show that the scale consists of the following four dimensions: environment, economy, society and education, which are confirmed by the CFA and by the multi-group invariance testing. These findings provide evidence that the structure of the items lends support to the UNESCO framework of sustainability, which includes the environment, the economy and society (UNESCO 2005), plus education. The reliability and stability analyses show that the instrument meets the validity criteria quite well, and the ASD seems to be appropriate for measuring SD attitudes in university students in Italy.

The group comparison analysis provided an idea of the possible applications of the ASD in higher education by comparing agriculture students and psychology students. The findings highlighted a different trend regarding the students’ backgrounds: the agriculture students performed better on the environmental factor than did the psychology students, whereas the psychology students performed better on the society factor than did the agriculture students. These findings demonstrated that agriculture students had a greater pro-environmental attitude, whereas psychology students were more oriented toward social issues.

This study fits into the existing literature on EE (Dijkstra and Goedhart 2012; Schneller, Johnson, and Bogner 2015) and ESD assessment (Biasutti and Surian 2012; Michalos et al. 2012; Olsson, Gericke, and Chang Rundgren 2015). Previous instruments for EE are based on the beliefs about the relationship between the environment and humans, and the ESD tools focused on SD knowledge, attitudes and behaviours (Biasutti and Surian 2012; Michalos et al. 2012; Olsson, Gericke, and Chang Rundgren 2015). The ASD contributes to the current knowledge base and focuses on the UNESCO dimensions of sustainability, which include the environment, the economy and society. In addition, the ASD fills the gap in the educational dimension, because few previous tools considered education to be a main factor. The ASD education factor dedicates a special focus to the methodological issues by combining teaching approaches and the basic principles of how education could contribute to ESD.

There are limitations of the current study regarding the characteristics of the group of subjects. The approach has restrictions due to the limited number of participants involved in the research. In addition, the participants attended the same university in Italy, which has drastically reduced the generalizability of the study. Caution is advised when generalising the results to other university students who are different from those who participated in the current research. The results, however, are a platform for developing future research on SD attitudes. It would be helpful to develop further work to validate the ASD. For instance, it could be given to known groups to verify whether people with known attitudes toward SD score differently on the ASD. The appropriateness of the ASD could also be tested when evaluating programs and detecting changes in SD attitudes in Italian university students after they have attended programs and courses that are focused on ESD. The ASD could also be useful for understanding the ways in which students think about sustainability issues and for measuring the effects of curricula revision after having infused SD principles. In addition, ASD can be used to investigate the relationship between SD attitudes and other variables. Another issue concerns the examination of the relationships between SD attitudes and behaviours, and whether changes in attitudes could influence pro-sustainability behaviours. However, more research is needed to further test the scale with these different aspects, including students from different backgrounds who attend a variety of SD programs and degrees.

Notes on contributors

Michele Biasutti, PhD, associate professor at Padova University, had experience in ESD, awareness on development, both as researcher and consultant. Deep understanding of EC policies, methodologies and working methods gained through an involvement in EC founded programs, project and institutions. He is scientific director of projects financed by Italian and EU Institutions and director of conferences. He has published articles in international peer-reviewed journals and author of seven books.

Sara Frate is a PhD candidate in Pedagogical, Educational and Training Sciences at the Department of Philosophy, Sociology, Pedagogy and Applied Psychology at Padova University. She has a Master Degree in Clinical-Dinamic Psychology. Her research projects focus on ESD and social metacognition in collaborative e-learning environments.

Disclosure statement

No potential conflict of interest was reported by the authors.

Acknowledgements

Individual contribution: MB was the idea originator of the paper, decided the method of study, collected the data, contributed with results interpretation and wrote the paper. SF contributed with the literature review, data collection and the statistical analysis of the data.

Appendix 1.

The Attitudes toward Sustainable Development scale

Please indicate the extent of your agreement/disagreement with the statements by using the following scale:

Table

	1	2	3	4	5
	Strongly disagree	Disagree	Neutral	Agree	Strongly Agree
1) When people interfere with the environment, they often produce disastrous consequences	1	2	3	4	5
2) Environmental protection and people’s quality of life are directly linked	1	2	3	4	5
3) Biodiversity should be protected at the expense of industrial agricultural production	1	2	3	4	5
4) Building development is less important than environmental protection	1	2	3	4	5
5) Environmental protection is more important than industrial growth	1	2	3	4	5
6) Government economic policies should increase sustainable production even if it means spending more money	1	2	3	4	5
7) People should sacrifice more to reduce economic differences between populations	1	2	3	4	5
8) Government economic policies should increases fair trade	1	2	3	4	5
9) Government economic policies should act if a country is wasting its natural resources	1	2	3	4	5
10) Reducing poverty and hunger in the world is more important than increasing the economic well-being of the industrialized countries	1	2	3	4	5
11) Each country can do a lot to keep the peace in the world	1	2	3	4	5
12) The society should further promote equal opportunities for males and females	1	2	3	4	5
13) The contact between cultures is stimulating and enriching	1	2	3	4	5
14) The society should provide free basic health services	1	2	3	4	5
15) The society should take responsibility for the welfare of individuals and families	1	2	3	4	5
16) Teachers in college should use student-centred teaching methods	1	2	3	4	5
17) Teachers in college should promote future-oriented thinking in addition to historical knowledge	1	2	3	4	5
18) Teachers in college should promote interdisciplinarity between subjects	1	2	3	4	5
19) Teachers in college should promote the connection between local and global issues	1	2	3	4	5
20) Teachers in college should promote critical thinking rather than lecturing	1	2	3	4	5