Evaluating a Brief Big Five Personality Test in a Diverse Chinese Sample: The Role of Midpoint Designs and Reversely-Worded Items

Abstract

Response styles present a threat to the validity of data from Likert scales. This study evaluates a brief personality test administered by interviewers to a diverse Chinese sample, focusing on two factors with direct impacts on response styles, the midpoint designs and the inclusion of reversely-worded items. We randomly assign respondents into three midpoint designs: masked midpoint, explicit midpoint and removed midpoint. Cronbach’s alpha tends to be low and is not drastically affected by midpoint designs. Score reliability estimates from the removed and explicit midpoint designs are above or close to .60 along the whole score range for openness, but not the other subscales. The masked midpoint design presents some advantages in criterion relationship validity evidence based on correlations with demographic, behavioral, subjective and cognitive variables. Dropping reversely-worded items increases the estimates of Cronbach’s alpha, but shows no positive effects on the criterion relationship validity. In conclusion, for interviewer-administered brief personality scales applied to Chinese respondents, masked midpoint design that aims to reduce a possible overuse of midpoint presents some advantages in the overall psychometric properties, but the inclusion of reversely-worded items shows mixed effects.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Open Scholarship

This article has earned the Center for Open Science badges for Open Materials. The materials are openly accessible at http://dx.doi.org/10.18170/DVN/45LCSO.

Data availability statement

The data that support the findings of this study are openly available in Peking University Open Research Data at http://dx.doi.org/10.18170/DVN/45LCSO. Data used in this study is part of CFPS restricted data for experimental studies that are available to the users through a restricted data application. Codes for the paper is available from figshare: https://doi.org/10.6084/m9.figshare.23853549.v2.

Additional information

Funding

The research was supported by the Philosophy and Social Science Foundation of Ministry of Education of China under Grant No. 20JZD032.