Introducing the Scale of Perceived Affect Response to Online Worship (SPAROW): a psychometric assessment of ritual innovation during the pandemic

ABSTRACT

The COVID-19 pandemic led to an increase in the use of online platforms for Christian worship and emphasised the need for a simple psychometric instrument that is sufficiently general to assess affect responses to a range worship services. This paper reports on the development of the six-item Scale of Perceived Affect Response to Online Worship (SPAROW) during the third UK pandemic lockdown in 2021. The scale items were included in an online survey completed from January to July 2021 by 2,017 Anglicans living in England and 1713 Roman Catholics from the UK or the Republic of Ireland. Exploratory Factor Analysis (principal components extraction and varimax rotation) in the Anglican sample indicated a single-dimensional scale that had excellent internal consistency reliability for those who accessed pre-recorded services (α = .90, n = 1238) and live-streamed services (α = .91, n = 1492). Confirmatory Factor Analysis on the Roman Catholic sample using a Structural Equation Model showed a good fit to a single-dimensional scale.

KEYWORDS:

• Affect response
• Church of England
• COVID-19
• online worship
• Roman Catholic
• ritual

Introduction

In Christian thought, worship is connected intrinsically with the deep affective response of the creature to the transcendent creator. Underhill (1962), in her classic book, Worship, writes as follows about “the nature of worship”:

Worship may be overt or direct, unconscious or conscious. Where conscious, its emotional colour can range from fear through reverence … For worship is an acknowledgement of transcendence; that is to say, of a reality independent of the worshipper, which is always more or less deeply coloured by mystery. (p. 130)

Underhill’s definition of worship stands in the tradition of Otto’s (1959) analysis in his book, The idea of the Holy. Otto defines his analysis as an enquiry into the “non-rational”, and identifies the affective depth of the human response as both tremendum (mystery that repels) and fascinans (mystery that attracts). Otto focuses on the affect response and speaks of the “association of feelings”. In popular literature, this experience is most powerfully captured by Grahame (1961) in The Wind in the Willows, when Ratty and Mole came face-to-face with the “august Presence”.

Then suddenly the Mole felt a great Awe fall upon him, an awe that turned his muscles to water … It was no panic terror – indeed he felt wonderfully at peace and happy – but it was an awe that smote and held him … Then the two animals, crouching to the earth, bowed their heads and did worship. (pp. 134–136)

The social scientific study of religion has largely failed to identify this conceptualisation of worship as the primary focus of research. For example, within the sociology of religion, Ammerman (2005), Chaves (2004), and Francis and Lankshear (2021) focus on the dynamics of congregations and the social character of congregational life. Yet within congregational life, there is often much to enhance affective responses, including music, oratory, and ritual (Pecklers, 2003), especially when located within architect-designed sacred space (Whyte, 2017) and reinforced by community expectations.

With the advent of the pandemic, with national lockdowns, and with the widespread closure of churches and the suspension of congregational life, worship migrated to an online and virtual environment. Against this background, the present paper reports on the development of the six-item Scale of Perceived Affect Response to Online Worship (SPAROW) designed to test the effectiveness of online worship in connecting with the deep affective response of the creature to the transcendent creator.

Online worship

There has been a long-standing increase in worshippers accessing religious ritual online, which has followed the increasing power of hardware and software, and the growth of access to the internet (Campbell, 2010; Hojsgaard & Warburg, 2005; Hutchings, 2020). Online worship varies from “cyber-churches”, which exist only online and where participants take on virtual identities as avatars, to the recording or live streaming of worship that is based on traditional church services and where the participants are largely passive observers (Hutchings, 2010). This latter form of online worship was relatively unusual before the COVID-19 pandemic of 2020, but the lockdowns introduced to curb the spread of the virus made worship in buildings difficult or impossible, which then spawned many initiatives by mainstream churches offering worship online (Campbell, 2020a, 2020b; Phillips, 2021a). In the Church of England, the complete closure of churches from March to July 2020 (McGowan, 2020) led to a rapid growth of both pre-recorded and live-streamed worship online (Village & Francis, 2020).

It was realised early in the pandemic that religious rituals might suffer during lockdowns, and there were a number of UK studies instigated by researchers to examine responses of worshippers to online church experiences, especially worship (Bryson et al., 2020; Davis et al., 2020; Edelman et al., 2021; Lovell et al., 2022; Village & Francis, 2020). Most studies of online worship have been case studies by academics (for example, see Alderson & Davie, 2021; Campbell, 2020c; Hutchings, 2007, 2010) or, especially during the pandemic, reports from religious ministers and practitioners (for example, Campbell, 2020b). Where there have been surveys in the UK and elsewhere, they have tended to look at attitudes towards virtual worship or other aspects of lockdown (Francis & Village, 2021; Kruger, 2021; Village & Francis, 2021a, 2021b).

Assessing the effects of services on worshippers can be difficult because such effects may be specific and linked to the content in ways that make comparisons between individuals or between services hard to make or interpret (for example, Busman, 2021; Nelson, 1996; Price et al., 1980). Theologically “good” worship might be seen as that which brings people closer to God (Monro, 2014). However, Ellens (1973) pointed out some years ago that worship also involves complex interactions between worshippers that are likely to produce changes in affect. A recent edition of the journal Liturgy was dedicated to the theme of worship and emotion and included both positive affirmations and negative warnings about the place of emotion in worship (Phillips, 2021b). Where attention has been paid to responses to worship, it has often been focused on the place of music within ritual (Astley et al., 2000; Mall et al., 2021). At the turn of this century, (Sloboda, 2000) pointed to the paucity of psychological studies of music in worship, and the need to assess different kinds of psychological responses that might be important.

There have been some attempts to assess emotional responses to worship, either qualitatively (Edie, 2021; Nelson, 1996) or using psychometric scales (Miller & Strongman, 2002). The latter study included an eight-item Religious Service Experience Scale, which asked general questions about the importance of various aspects of services such as music, sermons or an “experience of the Holy Spirit”. Emotional responses to specific types of music (secular versus religious) were measured outside a service using a 41-item scale developed for more general use with music (Asmus, 1985).

There has been a little quantitative study to date of experiences of online worship. The largest study in the Church of England involved a survey of 607 clergies and laypeople in the Church of England during the third national lockdown in 2021 (Lovell et al., 2022). Respondents were asked to rate a range of aspects of “distanced-church” experience against pre-pandemic experience of face-to-face church on a five-point scale ranging from “much better” to “much worse”. The items were mostly general (for example, “Caring for other people at church” or “Growing into who I want to be”), but there were a few focused especially on worship. Thus 51% rated Holy Communion as much worse, and 26% rated “Worshipping God with others” as much worse. This study defined a distanced-church as “church which did not generally feature face-to-face gatherings of more than six people” (p. 9), so did not fully distinguish online from face-to-face worship. The measure was also relative to pre-pandemic experience, so it was not suitable for a scale that might be used in general circumstances.

The rise in importance of online worship among mainstream churches during the COVID-19 pandemic encouraged us to include a short measure of affect responses to online worship in the Covid-19 & Church-21 survey, which ran from January to July 2021 during the third lockdown in England. In the absence of a suitable existing instrument, this paper reports the development of a new instrument, the SPAROW, which was designed to be used on pre-recorded and live-streamed worship and which could, in principle, be used for in-church services as well. The assessment of affect response is important in light of the association between negative affect and church leaving (see Francis & Richter, 2007).

Method

Procedure

During the third lockdown in England, an online survey, named Covid-19 & Church-21, was delivered through the Qualtrics XM platform from 22 January to 23 July 2021. It was designed to be used by various denominations and was promoted through the online and paper versions of the Church Times, the main newspaper of the Church of England, through Catholic Voices in the UK, and in the Republic of Ireland through the Mater Dei Centre for Catholic Education, Dublin City University. The total response from those who completed sufficient components of the survey to be included in the analyses was 5853, of whom 2017 were Anglicans living in England and 1713 were Roman Catholics from the UK or the ROI. The scale was developed using Exploratory Factor Analysis (EFA) with the sample from the Church of England and then tested using Confirmatory Factor Analysis (CFA) with a Structural Equation Model (SEM) on the sample of Roman Catholics.

Sample profiles

The Church of England sample comprised 55% women and 45% men, the majority (54%) were in their 50s or 60s, and 36% were ordained (Table 1). The Roman Catholic sample had a slightly higher proportion of women than the Church of England sample (66%), 48% were in their 50s or 60s, and only 7% were ordained (Table 1). A higher proportion of the Church of England sample (37%) lived in rural areas compared with the Roman Catholic sample (17%).

Table 1. Sample profiles.

	Church of England	Roman Catholic
	N = 2017	N = 1713
	%	%
Sex
Male	44.6	34.0
Female	55.4	66.0
Age
20s	1.5	6.3
30s	4.1	8.7
40s	9.5	14.3
50s	20.1	24.0
60s	33.8	24.4
70s	26.2	19.0
80s+	4.8	3.4
Location
Rural	36.8	16.5
Town	31.4	34.8
Suburban	24.2	37.3
Inner city	7.6	11.3
Status
Lay	64.0	93.5
Ordained	36.0	6.5

Instrument

Scale of Perceived Affect Response to Online Worship (SPAROW)

The survey contained two blocks of items related to accessing pre-recorded online and live-streamed online worship services. Each block was headed by a question that asked respondents if they had accessed these particular sorts of services, and only those that had were shown the corresponding items. This meant that the sample sizes varied between the two types of online services, with some respondents accessing neither, some accessing only one type of online worship, and some accessing both.

Each block contained a set of Likert items assessing perceived affect response to participating in online worship. Each item had a five-point response scale ranging from strongly agree to strongly disagree. Each set of items was introduced by the statement “During or after these [pre-recorded/live-streamed] services I usually felt: (Please click a button for EACH item)”. There followed six affect items, three positive (“Energised”, “Inspired”, and “Fulfilled”) and three negative (“Detached”, “Unmoved”, and “Distracted”). These items were derived from a wide assessment of the literature. Both positive and negative items were included to safeguard against response setting.

Analysis

The EFA employed procedures in SPSS 28 (IBM_Corporation, 2021) and the CFA employed AMOS 28 (Arbuckle, 2021). The EFA used the Church of England sample; extraction was principal components, with the default setting of extracting components with a minimum eigenvalue of one, followed by a varimax rotation. This procedure tends to maximise the number of orthogonal factors identified in the data (O’Connor, 2000). The CFA applied the SEM specified in Figure 1 to the Roman Catholic sample. Model fit was tested using chi-squared adjusted for degrees of freedom (CMIN/DF), comparative fit index (CFI), and the root mean square error of approximation (RMSEA).

Figure 1. Model diagram for CFA using Structural Equation Modelling.

Note: *Items reverse coded in the scale.

The internal reliability of the scales was tested on both samples for both types of service using Cronbach’s alpha (Cronbach, 1951), with negative items reverse coded. Summated rating scales were then created from the items and distribution properties reported.

Results

Exploratory Factor Analysis

For the Church of England sample, 1238 of 2017 (61%) reported responses to pre-recorded services. Factor analysis extracted a single component that accounted for 68% of the variance in the sample. In the same sample, 1492 (74%) reported responses to live-streamed services. Factor analysis also extracted a single component which accounted for 70% of the variance in the sample. For both sub-samples, the six-item scale show high internal reliability (pre-recorded services, alpha = .90; live-streamed services, alpha = .91), with item versus rest-of-scale correlations ranging from .63 to .78 for pre-recorded and .68 to .79 for live-streamed (Table 2).

Table 2. Scale of Perceived Affect Response for Online Worship (SPAROW) properties for Church of England sample.

	Pre-recorded				Live-streamed
	N = 1238				N = 1492
	CITC	% AG	% NC	% DA	CITC	% AG	% NC	% DA
Energised	.78	45	28	27	.79	50	25	25
Inspired	.76	51	27	22	.78	50	28	22
Fulfilled	.75	36	35	30	.76	38	33	29
Detached^a	.75	41	19	41	.73	33	20	47
Unmoved^a	.76	24	22	54	.79	23	20	57
Distracted^a	.63	33	25	43	.68	31	24	45

^a These items were reverse coded. CITC = Corrected Item-Total Correlation; AG = Agree (combining Strongly agree and Agree); NC = Not certain; DA = Disagree (combining Strongly disagree and Disagree).

Confirmatory factor analysis

In the Roman Catholic sample, 411 of 1713 (23%) reported responses to pre-recorded services and 1248 (73%) reported responses to live-streamed services. CFA models fitted using SEM (Figure 1) to both sets of data were improved by allowing error terms e5–e7 to covary. Fit indices (Table 3) suggested the model was reasonably well fitted to the Roman Catholic pre-recorded sample and well-fitted for the live-stream sample.

Table 3. Model fit indices for the CFA using Roman Catholics who accessed pre-recorded (n = 411) or live-streamed (n = 1284) online worship.

	CMIN/DF	CFI	RMSEA	LO90	HI90	PCLOSE
Pre-recorded	3.298	.989	.076	.041	.115	.103
Live-streamed	2.426	.998	.034	.012	.057	.864

Note: CMIN/DF = Chi-squared adjusted for degrees of freedom, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, LO90 and HI90 = 90% confidence limits, PCLOSE = probability that RMSEA > .05. Well-fitted models are indicated by CMIN/DF < 5, CFI close to 1, and RMSEA < .05 (Byrne, 2010).

Scale properties

In both samples, the scale showed very low levels of negative skew and low levels of kurtosis (Table 4). The levels suggest a distribution that is reasonably evenly distributed around the mean, but platykurtic, with few extreme outliers and a less pronounced peak around the mean. Mean scores were higher among Roman Catholics than among Anglicans for both types of online service (pre-recorded mean (SD): 15.3 (5.1) versus 13.2 (5.2), df = 1659, p < .001; live-streamed: 14.9 (5.4) versus 13.6 (5.4), df = 2790, p < .001).

Table 4. SPAROW scale properties for four sub-samples used in the analyses.

	N	Min.	Max.	Mean (SD)	Skewness	Kurtosis
Church of England
Pre-recorded	1238	0	28	13.2 (5.2)	−.21	−.39
Live-streamed	1492	0	29	13.6 (5.4)	−.28	−.42
Roman Catholic
Pre-recorded	411	1	28	15.3 (5.1)	−.33	−.10
Live-streamed	1284	0	29	14.9 (5.4)	−.31	−.29

Discussion and conclusion

During the COVID-19 lockdown in 2021, 2017 Anglicans living in England and 1713 Roman Catholics living in the United Kingdom and Republic of Ireland completed the Covid-19, Church-21 survey. The survey included a six-item instrument designed to measure affect response to pre-recorded and live-streamed online worship. EFA of the Anglican sample (principal components extraction and varimax rotation) identified a single factor composed of three positive (“Energised”, “Inspired”, and “Fulfilled”) and three negative (“Detached”, “Unmoved”, and “Distracted”) items that accounted for 68% of the variation in pre-recorded responses, and 70% of the variation in live-streamed responses. Alpha reliabilities were high (.90 and .91 respectively). CFAs using Structural Equation Modelling applied to the Roman Catholic sample confirmed the scale was unidimensional.

The SPAROW is a short but reliable instrument that assesses experiences of online services. The affect items cover a range of the sorts of emotions or reactions that are likely during or after worship, and the fact that the scale worked in a similar fashion for pre-recorded and live-streamed online worship in two different religious traditions suggests it is likely to have widespread utility as a survey instrument. It is likely also to be applicable to church-based services and may offer a means of comparing worshipper responses to a range of different rituals.

Limitation of the study

The range of affect used was relatively small, and a more focused study might be needed to see if different kinds of affect are important responses to some services. The study asked those who had accessed particular types of online service to complete the instrument, but this was an overall response to possibly several different experiences over a period of some months. The SPAROW needs to be employed in response to specific worship services to test if affect response varies in ways that might be predicted from the specific content of a service. This would enable the face validity of the items to be tested more thoroughly. The development and initial assessment of the SPAROW have also been limited by the nature of the survey that generated these data. The survey was focused on two denominations (Anglican or Catholic) and attracted responses mainly from people aged 50 and over. These two limitations need to be addressed by future studies involving younger participants and involving members from other denominations.

Ethical approval

Ethical approval was granted by the Research Ethics Committee for the School of Humanities, Religion and Philosophy at York St John University (approval code: HRP-RS-AV-0420-01). All participants had to affirm they were 18 or over and give their informed consent by ticking a box that gave access to the rest of the survey.

Disclosure statement

No potential conflict of interest was reported by the author(s).