1,391
Views
27
CrossRef citations to date
0
Altmetric
Articles

When attaining the best sample is out of reach: Nonprobability alternatives when engaging in public administration research

References

  • Baker, R., Brick, J. M., Bates, N. A., Battaglia, M., Couper, M. P., Dever, J. A., … Tourangeau, R. (2013a). Report of the AAPOR task force on non-probability sampling. Retrieved September 28, 2016, from http://www.aapor.org/Education-Resources/Reports/Non-Probability-Sampling.aspx
  • Baker, R., Brick, J. M., Bates, N. A., Battaglia, M., Couper, M. P., Dever, J. A., … Tourangeau, R. (2013b). Summary report of the AAPOR task force on non-probability sampling. Journal of Survey Statistics and Methodology, 1, 90–143. doi:10.1093/jssam/smt008
  • Battaglia, M. P. (2008). Nonprobability sampling. In P. J. Lavrakas (ed.), Encyclopedia of survey research methods (pp. 523–526). Los Angeles, CA: SAGE.
  • Berk, R. A. (1983). An introduction to sample selection bias in sociological data. American Sociological Review, 48(3), 386–398. doi:10.2307/2095230
  • Bernard, H. R. (2002). Research methods in anthropology: Qualitative and quantitative approaches (2nd ed.). Creek, CA: Alta Mira Press.
  • Boruch, R. E. (ed.). (1997). Randomized experiments for planning and evaluation: A practical guide (Vol. 44). Thousand Oaks, CA: SAGE Publications.
  • Brick, J. M. (2011). The future of survey sampling. Public Opinion Quarterly, 75(5), 872–888. doi:10.1093/poq/nfr045
  • Brick, J. M., & Williams, D. (2013). Explaining rising nonresponse rates in cross-sectional. The ANNALS of the American Academy of Political and Social Science, 645(1), 36–59. doi:10.1177/0002716212456834
  • Camerer, C. F. (2003). Behavioral game theory: Experiments in strategic interaction. Princeton, NJ: Princeton University Press.
  • Cepeda, M. S., Boston, R., Farrar, J. T., & Strom, B. L. (2003). Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. American Journal of Epidemiology, 158(3), 280–287. doi:10.1093/aje/kwg115
  • Chistensen, L. B., Johnson, R. B., & Turner, L. A. (2011). Research methods, design and analysis. Boston, MA: Pearson.
  • Collins, K. M. T., Onwuegbuzie, A. J., & Jiao, Q. G. (2006). Prevalence of mixed-methods sampling designs in social science research. Evaluation and Research in Education, 19(2), 83–101. doi:10.2167/eri421.0
  • Couper, M. P. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64(4), 464–494. doi:10.1086/318641
  • Creswell, J. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Los Angeles, CA: SAGE.
  • Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed method research (2nd ed.). Thousand Oaks, CA: SAGE.
  • Curtin, R., Presser, S., & Singer, E. (2005). Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly, 69(1), 87–98. doi:10.1093/poq/nfi002
  • Devore, J. L., & Berk, K. N. (2007). Modern mathematical statistics with applications. Belmont, CA: Thomson Brooks/Cole.
  • Diamond, S. S. (2000). Reference guide on survey research. Washington, DC: Federal Judicial Center.
  • Dörnyei, Z. (2007). Research methods in applied linguistics. New York, NY: Oxford University Press.
  • Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4. doi:10.11648/j.ajtas.20160501.11
  • Fenwick, J. (1992). Some problems of teaching research method in public policy. Teaching Public Administration, 7(2), 77–81. doi:10.1177/014473949201200209
  • Ferreira, L. O. C., de Oliveira, E. S., Raymond, H. F., Chen, S. Y., & McFarland, W. (2008). Use of time-location sampling for systematic behavioral surveillance of truck drivers in Brazil. AIDS and Behavior, 12(1), 32–38. doi:10.1007/s10461-008-9386-0
  • Fitzpatrick, J. (2000). What are our goals in teaching research methods to public administrators? Journal of Public Affairs Education, 6(3), 173–181.
  • Flick, U. (1998). An introduction to qualitative research. Thousand Oaks, CA: Sage.
  • Frankel, M. R., & Frankel, L. R. (1987). Fifty years of survey sampling in the United States. Public Opinion Quarterly, 51, S127–S138. doi:10.1093/poq/51.4_PART_2.S127
  • Franklin, T. W. (2015). Race and ethnicity effects in federal sentencing: A propensity score analysis. Justice Quarterly, 32(4), 653–679. doi:10.1080/07418825.2013.790990
  • Goertz, G., & Mahoney, J. (2012). A tale of two cultures: Qualitative and quantitative research in the social sciences. Princeton, NJ: Princeton University Press.
  • Gomm, R., Hammersley, M., & Foster, P. (2000). Case study methods: Key issues, key texts. Thousand Oaks, CA: Sage.
  • Goves, R., & Couper, M. (1998). Household survey nonresponse. New York, NY: John Wiley & Sons, Inc.
  • Groenwold, R. H. H., Donders, A. R. T., Roes, K. C. B., Harrell, Jr., F. E., & Moons, K. G. M. (2012). Dealing with missing outcome data in randomized trials and observational studies. American Journal of Epidemiology, 175(3), 210–217. doi:10.1093/aje/kwr302
  • Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72(2), 167–189. doi:10.1093/poq/nfn011
  • Hahn, G. J., & Meeker, W. Q. (1993). Assumptions for statistical inference. The American Statistician, 47(1), 1–11.
  • Hampton, R. L., & Newberger, E. H. (1985). Child abuse incidence and reporting by hospitals: Significance of severity, class, and race. American Journal of Public Health, 75(1), 56–60. doi:10.2105/AJPH.75.1.56
  • Hansen, B. B. (2004). Full matching in an observational study of coaching for the SAT. Journal of the American Statistical Association, 99(467), 609–618. doi:10.1198/016214504000000647
  • Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234–249. doi:10.1037/a0019623
  • Heckman, J. J. (1979). Sample bias as a specification of error. Econometrica, 47(1), 153–162. doi:10.2307/1912352
  • Henrich, J., Boyd, R., Bowels, S., Camerer, C., Fehr, E., Gintis, H., & Tracer, D. (2005). “Economic Man” in cross-cultural perspective: Behavioral experiments in 15 small-scale societies. Behavioral and Brain Sciences, 28(6), 795–815. doi:10.1017/S0140525X05000142
  • Hibberts, M., Johnson, R. B., & Hudson, K. (2012). Common survey sampling techniques. In L. Gideon (Ed.), Handbook of survey methodology for the social sciences (pp. 53–74). New York, NY: Springer.
  • Holtzer, M., Warner, L. M., & Lee, D. B. (2017). Teaching resources guide for public affairs and administration (4th ed.). Published by NASPAA and the School of Public Affairs and Administration (SPAA), Rutgers University – Newark. Retrieved August 6, 2017, from https://spaa.newark.rutgers.edu/teaching-resources-guide
  • Im, T., Porumbescu, G., & Lee, H. (2013). ICT as a buffer to change: A case study of the Seoul Metropolitan Government’s Dasan Call Center. Public Performance & Management Review, 36(3), 436–455. doi:10.2753/PMR1530-9576360303
  • Imai, K., King, G., & Stuart, E. A. (2008). Misunderstandings among experimentalists and observationalists in causal inference. Journal of the Royal Statistical Society, Series A, 481–502. doi:10.1111/j.1467-985X.2007.00527.x
  • Jargowsky, P. A. (2005). Omitted variable bias. In K. Kempf-Leonard (Ed), Encyclopedia of social measurement (pp. 919–924). Philadelphia, PA: Elsevier.
  • Kalton, G. (1983). Introduction to survey sampling. Beverly Hills, CA: SAGE Publications.
  • Kemper, E. A., Stringfield, S., & Tefflie, C. (2003). Mixed methods sampling strategies in social science research. In A. Tashakkori & C. Teddlie (eds.), Handbook of mixed methods in social and behavioral research (pp. 273–296). Thousand Oaks, CA: Sage.
  • Kiaer, A. N. (1895). Observations et Expériences Concernant des Dénobrements Représenatifs. Bulletin of the International Statistical Institute, 9(2), 176–183.
  • Kish, L. (1987). Statistical design for research. New York, NY: John Wiley & Sons.
  • Kish, L. (1991). Taxonomy of elusive populations. Journal of Official Statistics, 7(3), 339–347.
  • Kish, L. (1995). The hundred years’ war of survey sampling. Statistics in Transition, 2(5), 813–830.
  • Kral, A., Malekinejad, M., Vaudrey, J., Martinez, A. N., Lorvick, J., McFarland, W., & Raymond, H. F. (2010). Comparing respondent-driven sampling and targeted sampling methods of recruiting injection drug users in San Francisco. Journal of Urban Health, 87(5), 839–850. doi:10.1007/s11524-010-9486-9
  • Kuzel, A. J. (1992). Sampling in qualitative inquiry. In B. F. Crabtree & W. L. Miller (Eds), Doing qualitative research (pp. 31–44). Newbury Park, CA: Sage.
  • Lavrakas, P. J., Shuttles, C. D., Steeh, C., & Fienberg, H. (2007). The state of surveying cell phone numbers in the United States – 2007 special issue. Public Opinion Quarterly, 71(5), 840–854. doi:10.1093/poq/nfm054
  • Lucas, S. R. (2016). Where the rubber meets the road: Probability and nonprobability moments in experiment, interview, archival, administrative, and ethnographic data collection. Socius: Sociological Research for a Dynamic World, 2, 1–24. doi:10.1177/2378023116634709
  • Mackellar, D. A., Gallagher, K. M., Finlayson, T., Sanchez, T., Lansky, A., & Sullivan, P. S. (2007). Surveillance of HIV risk and prevention behaviors of men who have sex with men – A National Application of Venue-Based, Time-Space Sampling. Public Health Reports, 122(S1), 39–47. doi:10.1177/00333549071220S107
  • Mayes, L. C., Horwitz, R. I., & Feinstein, A. R. (1988). A collection of 56 topics with contradictory results in case-control research. International Journal of Epidemiology, 17(3), 680–685. doi:10.1093/ije/17.3.680
  • McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403–425. doi:10.1037/1082-989X.9.4.403
  • Meier, K. J., Brudney, J. L., & Bohte, J. (2015). Applied statistics for public and nonprofit administration (9th ed.). Stamford, CT: Cengage Learning.
  • Ming, K., & Rosenbaum, P. R. (2001). A note on optimal matching with variable controls using the assignment algorithm. Journal of Computational and Graphical Statistics, 10(3), 455–463. doi:10.1198/106186001317114938
  • Moser, C. A. (1952). Quota sampling. Journal of the Royal Statistical Society, 115(3), 411–423. doi:10.2307/2980740
  • Moser, C. A., & Stuart, A. (1953). An experimental study of quota sampling. Journal of the Royal Statistical Society, 116(4), 349–405. doi:10.2307/2343021
  • Neyman, J. (1934). On the two different aspects of the representative method: The method of stratified sampling and the method of purposive sampling. Journal of the Royal Statistics Society, 97(4), 558–625. doi:10.2307/2342192
  • O’Sullivan, E., Rassel, G., Berner, M., & Taliaferro, J. D. (2016). Research methods for public administrators (6th ed.). New York, NY: Routledge.
  • Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12(2), 281–316.
  • Overton, M. (2016). Sorting through the determinants of local government competition. American Review of Public Administration. doi:10.1177/0275074016651143
  • Palinkas, L. A., Green, C. A., Wisdom, J. P., & Hoagwood, K. E. (2013). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544. doi:10.1007/s10488-013-0528-y
  • Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.
  • Perterson, R. A., & Merunka, D. R. (2014). Convenience samples of college students and research reproducibility. Journal of Business Research, 67(5), 1035–1041. doi:10.1016/j.jbusres.2013.08.010
  • Platt, L., Wall, M., Rhodes, T., Judd, A., Hickman, M., Johnston, L. G., … Sarang, A. (2006). Methods to recruit hard-to-reach groups: Comparing two chain referral sampling methods of recruiting injecting drug users across nine studies in Russia and Estonia. Journal of Urban Health, 83(7), i39–i53. doi:10.1007/s11524-006-9101-2
  • Pohlman, J. T., & Leitner, D. W. (2003). A comparison of ordinary least squares and logistic regression. Ohio Journal of Science, 103(5), 118–125.
  • Pruchno, R. A., Brill, J. E., Shands, Y., Gordon, J. R., Genderson, M. W., Rose, M., & Cartwright, F. (2008). Convenience samples and caregiving research: How generalizable are findings. The Gerontologist, 48(6), 820–827. doi:10.1093/geront/48.6.820
  • Reinisch, J., Sanders, S., Mortensen, E., & Rubin, D. B. (1995). In utero exposure to phenobarbital and intelligence deficits in adult men. Journal of American Medical Association, 274(19), 1518–1525. doi:10.1001/jama.1995.03530190032031
  • Remafedi, G., Jurek, A. M., & Oakes, J. M. (2008). Sexual identity and tobacco use in a venue-based sample of adolescents and young adults. American Journal of Preventative Medicine, 35(6), S463–S470. doi:10.1016/j.amepre.2008.09.002
  • Remler, D., & Van Ryzin, G. G. (2014). Research methods in practice: Strategies for description and causation (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.
  • Riccucci, N. M. (2007). The ethical responsibilities of street-level bureaucrats under welfare reform. Public Integrity, 9(2), 155–173. doi:10.2753/PIN1099-9922090203
  • Ridgeway, G. (2006). Assessing the effect of race bias in post-traffic stop outcomes using propensity scores. Journal of Quantitative Criminology, 22(1), 1–29. doi:10.1007/s10940-005-9000-9
  • Rivera, J. D. (2016a). Acquiring federal disaster assistance: Investigating equitable resource distribution within FEMA’s home assistance program (Doctoral dissertation). Rutgers, The State University of New Jersey.
  • Rivera, J. D. (2016b). Accessing disaster recovery resource information: Reliance on social capital in the aftermath of hurricane sandy. In M. Companion & M. Chaiken (eds.), Responses to disasters and climate change: Understanding vulnerability and fostering resilience (pp. 61–70). Boca Raton, FL: CRC Press.
  • Rosenbaum, P., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. doi:10.1093/biomet/70.1.41
  • Rosenbaum, P., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. American Statistician, 39(1), 33–38.
  • Rosenbaum, P. R. (1991). A characterization of optimal designs for observational studies. Journal of the Royal Statistical Society, Series B, 597–610.
  • Rothman, K. J., Greenland, S., & Lash, T. L. (2008). Modern epidemiology (3rd ed.). Philadelphia, PA: Lippincott Williams & Wilkins.
  • Roy, K. M. (2012). In search of a culture: Navigating the dimensions of qualitative research. Journal of Marriage and the Family, 74(4), 660–665. doi:10.1111/j.1741-3737.2012.00981.x
  • Rubin, D. B. (1973). The use of matched sampling and regression adjustment to remove bias in observational studies. Biometrics, 29(1), 185–203. doi:10.2307/2529685
  • Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581–592. doi:10.1093/biomet/63.3.581
  • Rubin, D. B., & Stuart, E. A. (2006). Affinely invariant matching methods with discriminant mixtures of proportional ellipsoidally symmetric distributions. The Annals of Statistics, 34(4), 1814–1826. doi:10.1214/009053606000000407
  • Rubin, D. B., & Thomas, N. (1996). Matching using estimated propensity scores, relating theory to practice. Biometrics, 52(1), 249–264. doi:10.2307/2533160
  • Rubin, D. B., & Thomas, N. (2000). Combining propensity score matching with additional adjustments for prognostic covariates. Journal of the American Statistical Association, 95(450), 573–585. doi:10.1080/01621459.2000.10474233
  • Sainani Kristin, L. (2015). Dealing with missing data. PM&R: the Journal of Injury, Function and Rehabilitation, 7(9), 990–994. doi:10.1016/j.pmrj.2015.07.011
  • Saumure, K., & Given, L. M. (2008). Convenience sample. In L. M. Given (Ed.), The SAGE encyclopedia of qualitative research methods (pp. 125–126). Thousand Oaks, CA: SAGE.
  • Schafer, J. L. (1997). The analysis of incomplete multivariate data. Boca Raton, FL: Chapman & Hall.
  • Sears, D. O. (1986). College sophomores in the laboratory: Influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology, 51(3), 515–530. doi:10.1037/0022-3514.51.3.515
  • Semaan, S. (2010). Time-space sampling and respondent-driven sampling with hard-to-reach populations. Methodological Innovations Online, 5(2), 60–75. doi:10.4256/mio.2010.0019
  • Shadish, W., Lullen, J., & Clark, M. (2006). Propensity scores and quasi-experiments: A testimony to the practical side of Lee Sechrest. In R. R. Bootzin & P. E. McKnight (eds), Strengthening research methodology: Psychological measurement and evaluation (pp. 143–157). Washington, D.C.: American Psychological Association.
  • Shadish, W. R., & Steiner, P. M. (2010). A primer on propensity score analysis. Newborn & Infant Nursing Reviews, 10(1), 19–26. doi:10.1053/j.nainr.2009.12.010
  • Simon, H. A. (2000). Bounded rationality in social science: Today and tomorrow. Mind & Society, 1(1), 25–39. doi:10.1007/BF02512227
  • Small, M. L. (2009). How many cases do i need? On science and the logic of case selection in field-based research. Ethnography, 10(1), 5–38. doi:10.1177/1466138108099586
  • Smith, H. (1997). Matching with multiple controls to estimate treatment effects in observational studies. Sociological Methodology, 27(1), 325–353. doi:10.1111/1467-9531.271030
  • Spradley, J. P. (1979). The ethnographic interview. New York, NY: Holt, Rinehart & Winston.
  • Stallings, R. A. (2002). Methods of disaster research. International Research Committee on Disasters.
  • Stoop, I. (2012). Unit non-response due to refusal. In L. Gideon (Ed.), Handbook of survey methodology for the social sciences (pp. 121–147). New York, NY: Springer.
  • Stuart, E. A., & Green, K. M. (2008). Using full matching to estimate causal effects in non-experimental studies: Examining the relationship between adolescent marijuana use and adult outcomes. Development Psychology, 44(2), 395–406. doi:10.1037/0012-1649.44.2.395
  • Stuart, E. A. (2010). Matching methods for casual inference: A review and a look forward. Statistical Science, 25(1), 1–21. doi:10.1214/09-STS313
  • Sudman, S., Sirken, M., & Cowan, C. (1988). Sampling rare ad elusive populations. Science, 240, 991–996. doi:10.1126/science.240.4855.991
  • Sudman, S., & Blair, E. (1999). Sampling in the Twenty-First Century. Journal of the Academy of Marketing Science, 27(2), 269–277. doi:10.1177/0092070399272010
  • Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social & behavioral research. Thousand Oaks, CA: Sage.
  • Teddlie, C., & Yu, F. (2007). Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research, 1(1), 77–100.
  • Teng, P., Hall, B. J., & Li, L. (2014). The association between social resources and depression among female migrants affected by domestic violence. European Journal of Psycho-Traumatology, 5, 26528. doi:10.3402/ejpt.v5.26528
  • Terhanian, G., & Bremer, J. (2012). A smarter way to select respondents for survey? International Journal of Market Research, 54(6), 751–780. doi:10.2501/IJMR-54-6-751-780
  • Terman, J. (2014). Evaluating political signals: The nature of bureaucratic response in minority preference purchasing. American Review of Public Administration, 44(5), 522–549. doi:10.1177/0275074012471890
  • Thiel, S. V. (2014). Research methods in public administration and public management: An introduction. New York, NY: Routledge.
  • Thoemmes, F. J., & Kim, E. S. (2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research, 46(1), 90–118. doi:10.1080/00273171.2011.540475
  • Thompson, S. K. (2012). Sampling (3rd ed.). Hoboken, NJ: John Wiley & Sons, Inc.
  • Vavreck, L., & Rivers, D. (2008). The 2006 cooperative congressional election study. Journal of Elections, Public Opinions & Parties, 18(4), 355–366. doi:10.1080/17457280802305177
  • Yates, F. (1946). A review of recent statistical developments in sampling and sampling surveys. Journal of the Royal Statistical Society, 109(1), 12–43. doi:10.2307/2981390
  • Yin, R. (1989). Case study research: Design and methods (2nd Ed). Newbury Park, CA: Sage.
  • Young, S. S., & Karr, A. (2011). Deming, data and observational studies. Significance, 8(3), 116–120. doi:10.1111/j.1740-9713.2011.00506.x

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.