780
Views
12
CrossRef citations to date
0
Altmetric
Original Articles

Using Score Equating and Measurement Invariance to Examine the Flynn Effect in the Wechsler Adult Intelligence Scale

, &

REFERENCES

  • Albano, A. (2011). equate: Statistical methods for test score equating. http://CRAN.R-project.org/package=equate
  • Ashton, M.C., & Lee, K. (2005). Problems with the method of correlated vectors. Intelligence, 33, 431–444doi: 10.1016/j.intell.2004.12.004
  • Asparouhov, T., & Muthén, B.O. (2005, November). Multivariate statistical modeling with survey data. Paper presented at the annual meeting of the Federal Committee on Statistical Methodology Research Conference, Arlington, VA.
  • Beaujean, A.A. (2014). Latent variable modeling using R: A step by step guide. New York, NY: Routledge/Taylor and Francis.
  • Beaujean, A., & Osterlind, S.J. (2008). Using item response theory to assess the Flynn effect in the National Longitudinal Study of Youth 79 Children and Young Adults' data. Intelligence, 36, 455–463doi: 10.1016/j.intell.2007.10.004
  • Beaujean, A.A., Parkin, J., & Parker, S. (2014). Comparing Cattell-Horn-Carroll models for predicting language achievement: Differences between bi-factor and higher-order factor models. Psychological Assessment, 26, 789–805doi: 10.1037/a0036745
  • Beaujean, A.A., & Sheng, Y. (2010). Examining the Flynn effect in the general social survey vocabulary test using item response theory. Personality and Individual Differences, 48(3), 294–298doi: 10.1016/j.paid.2009.10.019
  • Beaujean, A.A., & Sheng, Y. (2014). Assessing the Flynn effect in the Wechsler scales. Journal of Individual Differences, 35, 63–78doi: 10.1027/1614-0001/a000128
  • Blair, C., Gamson, D., Thorne, S., & Baker, D. (2005). Rising mean IQ: Cognitive demand of mathematics education for young children, population exposure to formal schooling, and the neurobiology of the prefrontal cortex. Intelligence, 33(1), 93–106doi: 10.1016/j.intell.2004.07.008
  • Boake, C. (2002). From the Binet-Simon to the Wechsler-Bellevue: Tracing the history of intelligence testing. Journal of Clinical and Experimental Neuropsychology, 24(3), 383–405doi: 10.1076/jcen.24.3.383.981
  • Brunner, M. (2008). No g in education? Learning and Individual Differences, 18(2), 152–165doi: 10.1016/j.lindif.2007.08.005
  • Camara, W.J., Nathan, J.S., & Puente, A.E. (2000). Psychological test usage: Implications in professional psychology. Professional Psychology: Research and Practice, 31(2), 141154doi: 10.1037/0735-7028.31.2.141
  • Carroll, J.B. (1997). Theoretical and technical issues in identifying a factor of general intelligence. In B. Devlin, S.E. Fienberg, D.P. Resnick, & K. Roeder (Eds.), Intelligence, genes, and success: Scientists respond to the bell curve (pp. 125–156). New York, NY: Springer-Verlag.
  • Carroll, J.B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York, NY: Cambridge University.
  • Chen, F.F., West, S.G., & Sousa, K.H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41(2), 189–225doi: 10.1207/s15327906mbr4102_5
  • Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255doi: 10.1207/S15328007SEM0902_5
  • Colom, R., Lluis-Font, J.M., & Andrés-Pueyo, A. (2005). The generational intelligence gains are caused by decreasing variance in the lower half of the distribution: Supporting evidence for the nutrition hypothesis. Intelligence, 33, 83–91doi: 10.1016/j.intell.2004.07.010
  • Dolan, C.V., & Hamaker, E.L. (2001). Investigating Black-White differences in psychometric IQ: Multi-group confirmatory factor analyses of the WISC-R and K-ABC, and a critique of the method of correlated vectors. In F. Columbus (Ed.), Advances in psychological research (Vol. 6, pp. 31–60). Huntington, NY: Nova Science.
  • Enders, C.K. (2001). The impact of nonnormality on full information maximum-likelihood estimation for structural equation models with missing data. Psychological Methods, 6(4), 352–370doi: 10.1037/1082-989X.6.4.352
  • Enders, C.K., & Bandalos, D.L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 8(3), 430–457doi: 10.1207/S15328007SEM0803_5
  • Floyd, R.G., Bergeron, R., McCormack, A.C., Anderson, J.L., & Hargrove-Owens, G.L. (2005). Are Cattell-Horn-Carroll (CHC) broad ability composite scores exchangeable across batteries? School Psychology Review, 34(3), 386–341.
  • Floyd, R.G., Clark, M.H., & Shadish, W.R. (2008). The exchangeability of IQs: Implications for professional psychology. Professional Psychology: Research and Practice, 39(4), 414–423doi: 10.1037/0735-7028.39.4.414
  • Floyd, R.G., Reynolds, M.R., Farmer, R.L., & Kranzler, J.H. (2013). Are the general factors from different child and adolescent intelligence tests the same? Results from a five-sample, six-test analysis. School Psychology Review, 42(4), 383–401.
  • Floyd, R.G., Shands, E.I., Rafael, F.A., Bergeron, R., & McGrew, K.S. (2009). The dependability of general-factor loadings: The effects of factor-extraction methods, test battery composition, test battery size, and their interactions. Intelligence, 37(5), 453–465doi: 10.1016/j.intell.2009.05.003
  • Flynn, J.R. (1984). The mean IQ of Americans: Massive gains 1932 to 1978. Psychological Bulletin, 95, 29–51doi: 10.1037/0033-2909.95.1.29
  • Flynn, J.R. (1996). What environmental factors affect intelligence: The relevance of IQ gains over time. In D. Detterman (Ed.), Current topics in human intelligence (vol. 5): The environment. Norwood, NJ: Ablex.
  • Flynn, J.R. (1998). WAIS-III and WISC-III gains in the United States from 1972 to 1995: How to compensate for obsolete norms. Perceptual and Motor Skills, 86(31231–1239doi: 10.2466/pms.1998.86.3c.1231
  • Flynn, J.R. (2006). Tethering the elephant: Capital cases, IQ, and the Flynn effect. Psychology, Public Policy, And Law, 12(2), 170–189doi: 10.1037/1076-8971.12.2.170
  • Flynn, J.R. (2009a). The WAIS-III and WAIS-IV: Daubert motions favor the certainly false over the approximately true. Applied Neuropsychology, 16(2), 98–104doi: 10.1080/09084280902864360
  • Flynn, J.R. (2009b). What is intelligence? Beyond the Flynn effect. Cambridge UK: Cambridge University [Expanded paperback edition].
  • Flynn, J.R. (2012). Are we getting smarter? Rising IQ in the twenty-first century. New York, NY: Cambridge University.
  • Flynn, J.R., & Rossi-Casé, L. (2012). IQ gains in Argentina between 1964 and 1998. Intelligence, 40(2), 145–150doi: 10.1016/j.intell.2012.01.006
  • Fox, M.C., & Mitchum, A.L. (2013). A knowledge-based theory of rising scores on “culture-free” tests. Journal of Experimental Psychology: General, 142(3), 979–1000doi: 10.1037/a0030155
  • Gignac, G.E. (2005). Revisiting the factor structure of the WAIS-R: Insights through nested factor modeling. Assessment, 12, 320–329.
  • Gignac, G.E. (2006). The WAIS-III as a nested factors model: A useful alternative to the more conventional oblique and higher-order models. Journal of Individual Differences, 27, 73–86.
  • Gignac, G.E. (2008). Higher-order models versus direct hierarchical models: g as superordinate or breadth factor? Psychology Science Quarterly, 50(1), 21–43.
  • Gignac, G.E., & Watkins, M.W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48(5), 639–662doi: 10.1080/00273171.2013.804398
  • Golembiewski, R.T., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. The Journal of Applied Behavioral Science, 12(2), 133–157doi: 10.1177/002188637601200201
  • Greshman, F., & Reschly, D.J. (2011). Standard of practice and Flynn effect testimony in death penalty cases. Intellectual and Developmental Disabilities, 49(3), 131–140doi: 10.1352/1934-9556-49.3.131
  • Herrnstein, R.J., & Murray, C. (1996). The bell curve: Intelligence and class structure in American life. New York, NY: Free Press.
  • Holland, P.W., & Thayer, D.T. (2000). Univariate and bivariate loglinear models for discrete test score distributions. Journal of Educational and Behavioral Statistics, 25(2), 133–183doi: 10.3102/10769986025002-133
  • Holland, P.W., & Thayer, D.T. (1987). Notes on the use of log-linear models for fitting discrete probability distributions (Technical Report 87–79)Princeton, NJ: Educational Testing Service.
  • Horn, J.L., & McArdle, J.J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3–4), 117–144.
  • Irwing, P. (2012). Sex differences in g: An analysis of the US standardization sample of the WAIS-III. Personality and Individual Differences, 53, 126–131doi: 10.1016/j.paid.2011.05.001
  • Jensen, A.R. (1992). Spearman's hypothesis: Methodology and evidence. Multivariate Behavioral Research, 27, 225–233doi: 10.1207/s15327906mbr2702_5
  • Jensen, A.R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.
  • Jensen, A.R. (2004). Obituary: John Bissell Carroll. Intelligence, 32, 1–5doi: 10.1016/j.intell.2003.10.001
  • Johnson, W., Bouchard, T.J., Krueger, R.F., McGue, M., & Gottesman, I.I. (2004). Just one g: Consistent results from three test batteries. Intelligence, 32(1), 95–107doi: 10.1016/S0160-2896(03)00062-X
  • Johnson, W., te Nijenhuis, J., & Bouchard, T.J. (2008). Still just 1 g: Consistent results from five test batteries. Intelligence, 36(1), 81–95doi: 10.1016/j.intell.2007.06.001
  • Kanaya, T., Scullin, M.H., & Ceci, S.J. (2003) The Flynn effect and U.S. policies: The impact of rising IQs on American society via mental retardation diagnoses. American Psychologist, 58, 778–790doi: 10.1037/0003-066X.58.10.778
  • Kanaya, T., Ceci, S.J., & Scullin, M.H. (2005). Age differences within secular IQ trends: An individual growth modeling approach. Intelligence, 33(6), 613–621doi: 10.1016/j.intell.2005.08.001
  • Kane, H.D., & Oakland, T.D. (2000). Secular declines in Spearman's g: Some evidence from the United States. The Journal of Genetic Psychology, 161, 337–345.
  • Kaufman, A.S. (2010). “In what way are apples and oranges alike?” A critique of Flynn's interpretation of the Flynn effect. Journal of Psychoeducational Assessment, 28, 382–398doi: 10.1177/0734282910373346
  • Kaufman, A.S., & Weiss, L.G. (2010). Flynn effect [Special issue]. Journal of Psychoeducational Assessment, 28(5).
  • Kolen, M.J., & Brennan, R.L. (2014). Test equating, scaling, and linking: Methods and practices ( ed.)3rd. New York, NY: Springer-Verlag.
  • Linn, R.L. (1993). Linking results of distinct assessments. Applied Measurement in Education, 6, 83–102doi: 10.1207/s15324818ame0601_5
  • Little, T.D., & Slegers, D.W. (2005). Factor analysis: Multiple groups. In B.S. Everitt & D.C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 2, pp. 617–623). Chichester, UK: Wiley.
  • Lord, F.M. (1982). The standard error of equipercentile equating. Journal of Educational and Behavioral Statistics, 7(3), 165–174doi: 10.3102/10769986007003165
  • Lynn, R. (1998). In support of the nutrition theory. In U. Nesser (Ed.), The rising curve: Long-term gains in IQ and related measures (pp. 207–215). Washington, DC: American Psychological Association.
  • Lynn, R. (2009). What has caused the Flynn effect? Secular increases in the Development Quotients of infants. Intelligence, 37(1), 16–24doi: 10.1016/j.intell.2008.07.008
  • Lynn, R. (2013). Who discovered the Flynn effect? A review of early studies of the secular increase of intelligence. Intelligence, 41(6), 765–769doi: 10.1016/j.intell.2013.03.008
  • Major, J.T., Johnson, & Bouchard, W. T. J. (2011). The dependability of the general factor of intelligence: Why small, single-factor models do not adequately represent g. Intelligence, 39(5), 418–433doi: 10.1016/j.intell.2011.07.002
  • McArdle, J.J. (2009). Latent variable modeling of differences and changes with longitudinal data. Annual Review of Psychology, 60(1), 577–605doi: 10.1146/annurev.psych.60.110707.163612
  • McArdle, J.J., & Cattell, R.B. (1994). Structural equation models of factorial invariance in parallel proportional profiles and oblique confactor problems. Multivariate Behavioral Research, 29(1), 63–113doi: 10.1207/s15327906mbr2901_3
  • McArdle, J.J., & Prindle, J.J. (2013). Basic issues in the measurement of change. In K.F. Geisinger, B.A. Bracken, J.F. Carlson, J.I. C. Hansen, N.R. Kuncel, S.P. Reise, & M.C. Rodriguez (Eds.), APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology (pp. 223–243). Washington, DC: American Psychological Association.
  • McClain, A.L. (1996). Hierarchical analytic methods that yield different perspectives on dynamics: Aids to interpretation. In B. Thompson (Ed.), Advances in social science methodology (Vol. 4, pp. 229–240). Bingley, England: Emerald Group.
  • McDermott, P.A., Watkins, M.W., & Rhoad, A.M. (2014). Whose IQ is it? Assessor bias variance in high-stakes psychological assessment. Psychological Assessment, 26(1), 207–214doi: 10.1037/a0034832
  • Meade, A.W., Johnson, E.C., & Braddy, P.W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal Of Applied Psychology, 93(3), 568–592doi: 10.1037/0021-9010.93.3.568
  • Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543doi: 10.1007/BF0229-4825
  • Millsap, R.E., & Hartog, S.B. (1988). Alpha, beta, and gamma change in evaluation research: A structural equation approach. Journal of Applied Psychology, 73(3), 574–584doi: 10.1037/0021-9010.73.3.574
  • Mingroni, M.A. (2004). The secular rise in IQ: Giving heterosis a closer look. Intelligence, 32(1), 65–83.doi: 10.1016/S0160-2896(03)000-58-8
  • Mislevy, R.J. (1992). Linking educational assessments: Concepts, issues, methods, and prospects. Princeton, NJ: Educational Testing Service.
  • Moses, T., & Holland, P.W. (2009). Selection strategies for univariate loglinear smoothing models and their effect on equating function accuracy. Journal of Educational Measurement, 46(2), 159–176doi: 10.1111/j.1745-3984.2009.00075.x
  • Murray, A. L., & Johnson, W. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 41(5), 407–422. doi: 10.1016/j.intell.2013.06.004
  • Must, O., & Must, A. (2013). Changes in test-taking patterns over time. Intelligence, 41(6), 780–790doi: 10.1016/j.intell.2013.04.005
  • Must, O., Must, A., & Raudik, V. (2003). The secular rise in IQs: In Estonia, the Flynn effect is not a Jensen effect. Intelligence, 31(5), 461–471doi: 10.1016/S0160-2896(03)00013-8
  • Must, O., te Nijenhuis, J., Must, A., & van Vianen, A.E. M. (2009). Comparability of IQ scores over time. Intelligence, 37(1), 25–33doi: 10.1016/j.intell.2008.05.002
  • Nettelbeck, T., & Wilson, C. (2004). The Flynn effect: Smarter not faster. Intelligence, 32(1), 85–93doi: 10.1016/S0160-2896(03)00060-6
  • Parkin, J., & Beaujean, A.A. (2012). The effects of Wechsler Intelligence Scale for Children Fourth Edition cognitive abilities on math achievement. Journal of School Psychology, 50, 113–128doi: 10.1016/j.jsp.2011.08.003
  • Pietschnig, J., Tran, U.S., & Voracek, M. (2013). Item-response theory modeling of IQ gains (the Flynn effect) on crystallized intelligence: Rodgers' hypothesis yes, Brand's hypothesis perhaps. Intelligence, 41(6), 791–801doi: 10.1016/j.intell.2013.06.005
  • Reise, S.P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696doi: 10.1080/00273171.2012.715555
  • Reynolds, M.R., & Keith, T.Z. (2013). Measurement and statistical issues in child assessment research. In D.H. Saklofske, V.L. Schwean, & C.R. Reynolds (Eds.), The Oxford Handbook of Child Psychological Assessment (pp. 48–83). New York, NY: Oxford University.
  • Reynolds, M.R., Keith, T.Z., Flanagan, D.P., & Alfonso, V.C. (2013). A cross-battery, reference variable, confirmatory factor analytic investigation of the CHC taxonomy. Journal of School Psychology, 51(4), 535–555doi: 10.1016/j.jsp.2013.02.003
  • Rodgers, J.L. (1998). A critique of the Flynn Effect: Massive IQ gains, methodological artifacts, or both? Intelligence, 26(4), 337–356doi: 10.1016/S0160-2896(99)00004-5
  • Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. Retrieved from http://www.jstatsoft.org/v48/i02/
  • Schaie, K.W., Willis, S.L., & Pennak, S. (2005). An historical framework for cohort differences in intelligence. Research in Human Development, 2(1), 43–67.
  • Schmiedek, F., & Li, S.-C. (2004). Toward an alternative representation for disentangling age-associated differences in general and specific cognitive abilities. Psychology and Aging, 19(1), 40–56doi: 10.1037/0882-7974.19.1.40
  • Shiu, W., Beaujean, A., Must, O., te Nijenhuis, J., & Must, A. (2013). An item-level examination of the Flynn effect on the National Intelligence Test in Estonia. Intelligence, 41(6), 770–779.doi: 10.1016/j.intell.2013.05.
  • Shiu, W., Beaujean, A.A., & Wells, K. (2015). A meta-analysis of the Flynn effect in the United States, United Kingdom, and Australia [Manuscript submitted for publication].
  • Sinharay, S. (2010). How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement, 47(2), 150–174doi: 10.1111/j.1745-3984.2010.00106.x
  • Sinharay, S., Puhan, G., & Haberman, S.J. (2011). An NCME instructional module on subscores. Educational Measurement: Issues and Practice, 30(3), 29–40doi: 10.1111/j.1745-3992.2011.00208.x
  • Spearman, C. (1904). “General intelligence” objectively determined and measured. American Journal of Psychology, 15, 201–293doi: 10.2307/1412107
  • Steinmetz, H. (2013). Analyzing observed composite differences across groups: Is partial measurement invariance enough? Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 9(1), 1–12doi: 10.1027/1614-2241/a000049
  • te Nijenhuis, J., & van der Flier, H. (2013). Is the Flynn effect on g?: A meta-analysis. Intelligence, 41(6), 802–807doi: 10.1016/j.intell.2013.03.001
  • te Nijenhuis, J., van Vianen, A.M., & van der Flier, H. (2007). Score gains on g-loaded tests: No g. Intelligence, 35(3), 283–300doi: 10.1016/j.intell.2006.07.006
  • Wai, J., & Putallaz, M. (2011). The Flynn effect puzzle: A 30-year examination from the right tail of the ability distribution provides some missing pieces. Intelligence, 39(6), 443–455doi: 10.1016/j.intell.2011.07.006
  • Wechsler, D. (1981).Wechsler Adult Intelligence Scale—Revised: Manual. San Antonio, TX: Psychological Corporation.
  • Wechsler, D. (1997). Wechsler Adult Intelligence Scale—Third Edition. San Antonio, TX: Psychological Corporation.
  • Wechsler, D. (2008). Wechsler Adult Intelligence Scale—Fourth Edition. San Antonio, TX: Pearson Assessment.
  • Weiss, L.G. (2010). Considerations on the Flynn Effect. Journal of Psychoeducational Assessment, 28, 482–493doi: 10.1177/0734282910373572
  • Weiss, L.G., Keith, T.Z., Zhu, J., & Chen, H. (2013). Technical and practical issues in the structure and clinical invariance of the Wechsler scales: A rejoinder to commentaries. Journal of Psychoeducational Assessment, 31(2), 235–243doi: 10.1177/0734282913478050
  • West, S.G., Taylor, A.B., & Wu, W. (2012). Model fit and model selection in structural equation modeling. In R.H. Hoyle (Ed.),Handbook of structural equation modeling (pp. 209–231). New York, NY: Guilford.
  • Wicherts, J.M., Dolan, C.V., Hessen, D.J., Oosterveld, P, van Baal, G.C. M., Boomsma, D.I., & Span, M.M. (2004). Are intelligence tests measurement invariant over time? Investigating the nature of the Flynn effect. Intelligence, 32, 509–537doi: 10.1016/j.intell.2004.07.002
  • Williams, R.L. (2013). Overview of the Flynn effect. Intelligence, 41(6), 753–764doi: 10.1016/j.intell.2013.04.010
  • Woodley, M.A., te Nijenhuis, J., & Murphy, R. (2013). Were the Victorians cleverer than us? The decline in general intelligence estimated from a meta-analysis of the slowing of simple reaction time. Intelligence, 41(6), 843–850doi: 10.1016/j.intell.2013.04.006
  • Zhou, X., Zhu, J., & Weiss, L.G. (2010). Peeking inside the “black box” of the Flynn effect: Evidence from three Wechsler instruments. Journal of Psychoeducational Assessment, 28(5), 399–411doi: 10.1177/0734282910373340

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.