Investigating the reliability of second-order formative measurement in information systems research

Abstract

Recently, a few studies empirically explored the stability of first-order formative measurement, and raised concerns with its estimation reliability. Interpretational confounding, the disparity in the nominal and empirical meaning of a formatively measured construct, is at the center stage of the concern. Our study examines the issue in the context of the higher-order abstraction, focusing on the formatively defined relationship between the second-order construct and its indicators (i.e., first-order latent variables). Although the second-order formative abstraction is a widely accepted practice in structural equation modeling, the estimation results have been given a blind faith with no attempt to evaluate their integrity. Our empirical test, therefore, constitutes an attempt to fill the void. This study observed moderations of the theoretical relationship between reflectively designed first-order constructs and formatively defined second-order constructs when there is a change of endogenous variables. For this, two different formatively defined second-order constructs (i.e., IT management capabilities and IT personnel expertise) are utilized for the empirical testing. The estimation reveals that, while there was a considerable moderation of weights between IT management capabilities and its first-order constructs, those between IT personnel expertise and its first-order constructs remained relatively stable. These results demonstrate that the formatively defined relationship between the first- and second-order constructs can be precarious depending on the choice of the dependent variables. The analysis, therefore, revealed a significant presence of interpretational confounding and a higher chance of Type 1 error in model estimation. This implies that it becomes difficult to retain the construct validity and external validity of a formatively defined second-order construct. Thus, researchers are encouraged to exercise caution in mobilizing the formatively defined second-order measurement.

Keywords:

• structural equation modeling
• formative measurement
• formative indicator
• second-order construct
• interpretational confounding
• construct validity

Acknowledgements

This research is supported by the ubiquitous Computing and Network (UCN) Project, the Ministry of Knowledge and Economy(MKE) Knowledge and Economy Frontier R&D Program in Korea as a result of UCN's subproject 11C3-T2-10M. Bongsik Shin's research was supported in part by a grant from the San Diego State University.

Notes

1 For a formative construct coefficients are ‘weights’ and for a reflective construct these are ‘loadings’.

Additional information

Notes on contributors

Bongsik Shin

About the Authors

Bongsik Shin is a Professor at the Department of Information and Decision Systems at San Diego State University. He earned a Ph.D. from the University of Arizona and has taught computer networking, electronic commerce, IT management/strategy, data warehousing, and statistics. His research interests include IT-driven business models, electronic and mobile commerce, and research methodology. His research has been published in journals such as MISQ, JMIS, JAIS, ISJ, IEEE Transactions, CACM, JOCEC, I&M and DSS.

Gimun Kim

Gimun Kim is an Assistant Professor of Information Systems at the College of Business Administration, Konyang University in Korea. He received his Ph.D. from Yonsei University, Korea, and M.S. from Georgia State University. His research interests are in the business value of information technology capabilities, user behavior in electronic commerce, and research methodology related to structural equation modeling. He has published articles in journals such as MIS Quarterly, Information Systems Journal, and Information & Management.