‘Student experience’ surveys: some methodological considerations and an empirical investigation

References

• Albanese , M. , Prucha , C. , Barnet , J.H. and Gjerde , C.L. 1997 . The effect of right or left placement of the positive response on Likert‐type scales used by medical students for rating instruction . Academic Medicine , 72 ( 7 ) : 627 – 30 .
   PubMed Web of Science ®Google Scholar
• Attwood , R. 2007 . Anglia v‐c rallies troops . The Times Higher Education Supplement ,
   Google Scholar
• Attwood , R. 2008 . Probe ordered into ‘manipulation’ . The Times Higher Education ,
   Google Scholar
• Barnette , J.J. Likert response alternative direction: SA to SD or SD to SA: Does it make a difference? . Paper presented at the annual meeting of the American Educational Research Association . April 19–23 , Montreal, Canada. ERIC ED 429125
   Google Scholar
• Benson , J. and Hocevar , D. 1985 . The impact of item phrasing on the validity of attitude scales for elementary school children . Journal of Educational Measurement , 22 ( 3 ) : 213 – 40 .
   Web of Science ®Google Scholar
• BERA . 2004 . Revised ethical guidelines for educational research (2004) www.bera.ac.uk/publications/pdfs/ETHICA1.PDF (accessed April 9, 2008)
   Google Scholar
• Bergstrom , B.A. and Lunz , M.E. Rating scale analysis: Gauging the impact of positively and negatively worded items . Paper presented at the annual meeting of the American Educational Research Association . April 13–17 , San Diego, CA. ERIC ED 423289
   Google Scholar
• Billiet , J.B. and McClendon , M.J. 2000 . Modeling acquiescence in measurement models for two balanced sets of items . Structural Equation Modeling: A Multidisciplinary Journal , 7 ( 4 ) : 608 – 28 .
   Web of Science ®Google Scholar
• Campbell , D.T. , Siegman , C.R. and Rees , M.B. 1967 . Direction‐of‐wording effects in the relationship between scales . Psychological Bulletin , 68 ( 5 ) : 293 – 303 .
   PubMed Web of Science ®Google Scholar
• Dolnicar , S. 2006 . Are we drawing the right conclusions? The dangers of answer format effects in empirical tourism research . Tourism Analysis , 11 ( 3 ) : 199 – 209 .
   Google Scholar
• Eley , M. 2001 . The Course Experience Questionnaire: Altering question format and phrasing could improve the CEQ’s effectiveness . Higher Education Research and Development , 20 ( 3 ) : 293 – 312 .
   Google Scholar
• Goldberg , L.R. and Kilkowski , J.M. 1985 . The prediction of semantic consistency in self‐description: Characters of persons and terms that affect the consistency of responses to synonym and antonym pairs . Journal of Personality and Social Psychology , 48 ( 1 ) : 82 – 98 .
   PubMed Web of Science ®Google Scholar
• Graduate Careers Australia . 2007 . Graduate Course Experience 2006: The report of the Course Experience Questionnaire , Carlton, Victoria : Author .
   Google Scholar
• Green , R.F. and Goldfried , M.R. 1965 . On the bipolarity of semantic space . Psychological Monographs: General and Applied , 79 (Whole No. 599)
   Google Scholar
• Hamilton , J.C. and Shumansky , T.R. 1990 . Self‐awareness mediated the relationship between serial position and item reliability . Journal of Personality and Social Psychology , 59 ( 6 ) : 1301 – 7 .
   Web of Science ®Google Scholar
• Javeline , D. 1999 . Response effects in polite cultures: A test of acquiescence in Kazakhstan . Public Opinion Quarterly , 63 ( 1 ) : 1 – 28 .
   Web of Science ®Google Scholar
• Johanson , G.A. and Osborn , C.J. 2004 . Acquiescence as differential person functioning . Assessment & Evaluation in Higher Education , 29 ( 5 ) : 535 – 48 .
   Google Scholar
• Knight , R.G. , Chisholm , B.J. , Marsh , N.V. and Godfrey , H.P.D. 1988 . Some normative, reliability, and factor analytic data for the revised UCLA Loneliness Scale . Journal of Clinical Psychology , 44 ( 2 ) : 203 – 6 .
   PubMed Web of Science ®Google Scholar
• Knowles , E.S. 1988 . Item context effects on personality scales: Measuring changes the measure . Journal of Personality and Social Psychology , 55 ( 2 ) : 312 – 20 .
   Web of Science ®Google Scholar
• Krause , K.‐L. , Hartley , R. , James , R. and McInnis , C. 2005 . The first year experience in Australian universities: Findings from a decade of national studies , Canberra : Department of Education, Science and Training .
   Google Scholar
• Krosnick , J.A. 1999 . Survey research . Annual Review of Psychology , 50 : 537 – 67 .
   PubMed Web of Science ®Google Scholar
• Macnee , C.L. and McCabe , S. 2004 . Satisfaction with care among homeless patients: Development and testing of a measure . Journal of Community Health Nursing , 21 ( 3 ) : 167 – 78 .
   PubMedGoogle Scholar
• McClendon , M.J. 1991 . Acquiescence and recency response‐order effects in interview surveys . Sociological Methods & Research , 20 ( 1 ) : 60 – 103 .
   Web of Science ®Google Scholar
• Moser , C. and Kalton , G. 1971 . Survey methods in social investigation. , 2nd ed , London : Heinemann .
   Google Scholar
• Nunnally , J.C. and Bernstein , I.H. 1994 . Psychometric theory. , 3rd ed , New York : McGraw‐Hill .
   Google Scholar
• O’Muircheartaigh , C. , Krosnick , J.A. and Helic , A. 2000 . Middle alternatives, acquiescence, and the quality of questionnaire data http://harrisschool.uchicago.edu/about/publications/working-papers/pdf/wp_01_3.pdf (accessed April 15, 2008)
   Google Scholar
• Oppenheim , A.N. 1992 . Questionnaire design, interviewing and attitude measurement , London : Pinter .
   Google Scholar
• Osborn , C.J. and Johanson , G.A. 2001 . Development and validation of the Theoretical Orientation Scale for Clinicians: Reflecting solution‐focused principles . Assessment , 8 ( 3 ) : 301 – 13 .
   PubMed Web of Science ®Google Scholar
• Osgood , C.E. , Suci , G.J. and Tannenbaum , P.H. 1957 . The measurement of meaning , London : University of Illinois Press .
   Google Scholar
• Paulhus , D.L. 1991 . “ Measurement and control of response bias ” . In Measures of personality and social psychological attitudes , Edited by: Robinson , J.P. , Shaver , P.R. and Wrightsman , L.S. 17 – 59 . San Diego, CA : Academic Press .
   Google Scholar
• Payne , S.L. 1951 . The art of asking questions , Princeton, NJ : Princeton University Press (reprinted 1980) .
   Google Scholar
• Quinn , S.B. and Belson , W.A. 1969 . The effects of reversing the order of presentation of verbal rating scales in survey interviews , London : Survey Research Centre, London School of Economics and Political Science .
   Google Scholar
• Ramsden , P. 1991 . “ Report on the Course Experience Questionnaire trial ” . In Performance indicators in higher education: Report of a trial evaluation study commissioned by the Commonwealth Department of Employment, Education and Training, Volume 2 (the Linke Report) , 1 – 85 . Canberra : Australian Government Publishing Service .
   Google Scholar
• Ray , J.J. 1990 . Acquiescence and problems with forced‐choice scales . Journal of Social Psychology , 130 ( 3 ) : 397 – 99 .
   Web of Science ®Google Scholar
• Richardson , J.T.E. 2004 . The National Student Survey: Final report from the 2003 pilot project , Milton Keynes : Institute for Educational Technology, Open University (mimeo .
   Google Scholar
• Richardson , J.T.E. and IPSOS UK Public Affairs . 2005 . National Student Survey: Interim assessment of the 2005 questionnaire http://www.hefce.ac.uk/pubs/rdreports/2005/rd20_05/ (accessed April 15, 2008)
   Google Scholar
• Rorer , L.G. 1965 . The great response‐style myth . Psychological Bulletin , 63 ( 3 ) : 129 – 56 .
   PubMed Web of Science ®Google Scholar
• Sanders , C. 2006 . Students, think twice before you tick us off . The Times Higher Education Supplement ,
   Google Scholar
• Schommer , M. 1998 . The influence of age and schooling on epistemological beliefs . British Journal of Educational Psychology , 68 ( 4 ) : 551 – 62 .
   Google Scholar
• Schriesheim , C.A. , Eisenbach , R.J. and Hill , K.D. 1991 . The effect of negation and polar opposite item reversals on questionnaire reliability and validity: An experimental investigation . Educational and Psychological Measurement , 51 ( 1 ) : 67 – 78 .
   Web of Science ®Google Scholar
• Schuman , H. and Presser , S. 1996 . Questions and answers in attitude surveys: Experiments on question form, wording, and context , London : Sage . (reprint of 1981 edition, Academic Press, London)
   Google Scholar
• Schwarz , N. and Hippler , H.‐J. 1995 . Subsequent questions may influence answers to preceding questions in mail surveys . Public Opinion Quarterly , 59 ( 1 ) : 93 – 7 .
   Web of Science ®Google Scholar
• Shepherd , J. 2006 . Student threat to boycott survey . The Times Higher Education Supplement ,
   Google Scholar
• Siegel , S. and Castellan , N.J. Jr. 1988 . Nonparametric statistics for the behavioral sciences , New York : McGraw‐Hill .
   Google Scholar
• Simon , H.A. 1957 . Models of man , New York : Wiley .
   Google Scholar
• Smith , T.W. 1983 . An experimental comparison of clustered and scattered scale items . Social Psychology Quarterly , 46 ( 2 ) : 163 – 68 .
   Web of Science ®Google Scholar
• Smith , T.W. 2004 . “ Developing and evaluating cross‐national survey instruments ” . In Methods for testing and evaluating survey questionnaires , Edited by: Presser , S. , Rothgeb , J.M. , Couper , M.P. , Lessler , J.T. , Martin , E. , Martin , J. and Singer , E. 431 – 52 . Hoboken, NJ : John Wiley & Sons .
   Google Scholar
• Steinberg , L. 1994 . Context and serial‐order effects in personality measurement: Limits on the generality of measuring changes the measure . Journal of Personality and Social Psychology , 66 ( 2 ) : 341 – 49 .
   Web of Science ®Google Scholar
• Surridge , P. 2006 . The National Student Survey 2005: Findings http://www.hefce.ac.uk/pubs/rdreports/2006/rd22_06/ (accessed April 10, 2008)
   Google Scholar
• Surridge , P. 2007 . The National Student Survey 2006 http://www.hefce.ac.uk/pubs/rdreports/2007/rd14_07/ (accessed April 10, 2008)
   Google Scholar
• Tourangeau , R. , Rips , L.J. and Rasinski , K. 2000 . The psychology of survey response , Cambridge : Cambridge University Press .
   Google Scholar
• van Herk , H. , Poortinga , Y.H. and Verhallen , T.M.M. 2004 . Response styles in rating scales: Evidence of method bias in data from six EU countries . Journal of Cross‐Cultural Psychology , 35 ( 3 ) : 346 – 60 .
   Web of Science ®Google Scholar
• Watson , D. 1992 . Correcting for acquiescent response bias in the absence of a balanced scale: An application to class consciousness . Sociological Methods & Research , 21 ( 1 ) : 52 – 88 .
   Web of Science ®Google Scholar
• Weems , G.H. , Onwuegbuzie , A.J. , Schreiber , J.B. and Eggers , S.J. 2003 . Characteristics of respondents who respond differently to positively and negatively worded items on rating scales . Assessment & Evaluation in Higher Education , 28 ( 6 ) : 587 – 607 .
   Google Scholar
• Wembridge , E.R. and Means , E.R. 1918 . Obscurities in voting upon measures due to double‐negative . Journal of Applied Psychology , 2 ( 2 ) : 156 – 63 .
   Google Scholar
• Winkler , J.D. , Kanouse , D.E. and Ware , J.E. 1982 . Controlling for attitude response set in scale development . Journal of Applied Psychology , 67 ( 5 ) : 555 – 61 .
   Web of Science ®Google Scholar
• Yorke , M. 2001 . Bipolarity … or not? Some conceptual problems relating to bipolar rating scales . British Education Research Journal , 27 ( 2 ) : 171 – 86 .
   Web of Science ®Google Scholar
• Yorke , M. and Knight , P. 2007 . Evidence‐informed pedagogy and the enhancement of student employability . Teaching in Higher Education , 12 ( 2 ) : 157 – 70 .
   Web of Science ®Google Scholar
• Yorke , M. and Longden , B. 2007 . The first‐year experience in higher education in the UK: Report on Phase 1 of a project funded by the Higher Education Academy http://www.heacademy.ac.uk/assets/York/documents/ourwork/research/FYE/FirstYearExperienceRevised.pdf (accessed July 3, 2008)
   Google Scholar