Strengthening Evaluation in Nonprofit Human Service Organizations: Results of a Capacity-Building Experiment

References

• Abt Associates. (2009). Assessing changes in nonprofit capacity: Outcome study of the Compassion Capital Fund demonstration program (Technical Appendix Report prepared for Office of Community Services, Administration for Children and Families, U.S. Department of Health and Human Services). Retrieved from the Administration for Children and Families website: http://www.acf.hhs.gov/sites/default/files/opre/outcome_study_tech_appndx_rprt.pdf
   Google Scholar
• Abt Associates. (2010). Improving capacity among non-profits: Impact study of the Compassion Capital Fund demonstration program (Technical Appendix Report prepared for Office of Community Services, Administration for Children and Families, U.S. Department of Health and Human Services). Retrieved from the Administration for Children and Families website: http://www.acf.hhs.gov/sites/default/files/opre/tech_apndx_final.pdf
   Google Scholar
• Andrews, A. B., Motes, P. S., Floyd, A. G., Flerx, V. C., & Fede, A. L. D. (2005). Building evaluation capacity in community-based organizations: Reflections of an empowerment evaluation team. Journal of Community Practice, 13(4), 85–104. doi:10.1300/J125v13n04_06
   Google Scholar
• Augsberger, A., Schudrich, W., McGowan, B. G., & Auerbach, C. (2012). Respect in the workplace: A mixed methods study of retention and turnover in the voluntary child welfare sector. Children and Youth Services Review, 34, 1222–1229. doi:10.1016/j.childyouth.2012.02.016
   Web of Science ®Google Scholar
• Austin, M. J. (2008). Strategies for transforming human service organizations into learning organizations: Knowledge management and the transfer of learning. Journal of Evidence-Based Social Work, 5, 569–596. doi:10.1080/15433710802084326
   PubMedGoogle Scholar
• Austin, M. J., Regan, K., Samples, M. W., Schwartz, S. L., & Carnochan, S. (2011). Building managerial and organizational capacity in nonprofit human service organizations through a leadership development program. Administration in Social Work, 35, 258–281. doi:10.1080/03643107.2011.575339
   Web of Science ®Google Scholar
• Barak, M. E. M., Nissly, J. A., & Levin, A. (2001). Antecedents to retention and turnover among child welfare, social work, and other human service employees: What can we learn from past research? A review and meta-analysis. Social Service Review, 75, 625–661. doi:10.1086/323166
   Web of Science ®Google Scholar
• Barney, J. B., & Hesterly, W. S. (2012). Strategic management and competitive advantage (4th ed.). Boston, MA: Pearson.
   Google Scholar
• Benjamin, L. M. (2013). The potential of outcome measurement for strengthening nonprofits’ accountability to beneficiaries. Nonprofit and Voluntary Sector Quarterly, 42, 1224–1244. doi:10.1177/0899764012454684
   Web of Science ®Google Scholar
• Bertram, R. M., Blase, K. A., & Fixsen, D. L. (2015). Improving programs and outcomes: Implementation frameworks and organization change. Research on Social Work Practice, 25, 477–487. doi:10.1177/1049731514537687
   Web of Science ®Google Scholar
• Besel, K., Williams, C. L., & Klak, J. (2011). Nonprofit sustainability during times of uncertainty. Nonprofit Management and Leadership, 22, 53–65. doi:10.1002/nml.20040
   Web of Science ®Google Scholar
• Bess, G. (1998). A first stage organization life cycle study of six emerging nonprofit organizations in Los Angeles. Administration in Social Work, 22, 35–52. doi:10.1300/J147v22n04_03
   Web of Science ®Google Scholar
• Bogenschutz, M., Nord, D., & Hewitt, A. (2015). Competency-based training and worker turnover in community supports for people with IDD: Results from a group randomized controlled study. Intellectual and Developmental Disabilities, 53, 182–195. doi:10.1352/1934-9556-53.3.182
   PubMed Web of Science ®Google Scholar
• Bowman, W. (2011). Financial capacity and sustainability of ordinary nonprofits. Nonprofit Management and Leadership, 22, 37–51. doi:10.1002/nml.20039
   Web of Science ®Google Scholar
• Bradach, J. L., Tierney, T. J., & Stone, N. (2008). Delivering on the promise of nonprofits. Harvard Business Review, 86(12), 88–97. Retrieved from https://oldfirst.hypermart.net/i-mission/R0812G-PDF-ENG.pdf
   Web of Science ®Google Scholar
• Brown, M. (2008). Improving organizational capacity of faith-based and community-based domestic violence service providers. In P. Joshi, S. Hawkins, & J. Novey (Eds.), Compendium of research papers presented at the faith-based and community initiatives conference on research, outcomes, and evaluation. Washington, DC: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation. Retrieved from http://aspe.hhs.gov/whatsnew/posted2008.shtml
   Google Scholar
• Bryan, T. K., & Brown, C. H. (2015). The individual, group, organizational, and community outcomes of capacity-building programs in human service nonprofit organizations: Implications for theory and practice. Human Service Organizations: Management, Leadership and Governance, 1–18. doi:10.1080/23303131.2015.1063555
   Web of Science ®Google Scholar
• Campbell, D. (2002). Outcomes assessment and the paradox of nonprofit accountability. Nonprofit Management and Leadership, 12, 243–259. doi:10.1002/nml.12303
   Google Scholar
• Carman, J. G. (2007). Evaluation practice among community-based organizations: Research into the reality. American Journal of Evaluation, 28, 60–75. doi:10.1177/1098214006296245
   Web of Science ®Google Scholar
• Carman, J. G., & Fredericks, K. A. (2008). Nonprofits and evaluation: Empirical evidence from the field. New Directions for Evaluation, 2008(119), 51–71. doi:10.1002/ev.268
   Google Scholar
• Carman, J. G., & Fredericks, K. A. (2010). Evaluation capacity and nonprofit organizations: Is the glass half-empty or half-full? American Journal of Evaluation, 31, 84–104. doi:10.1177/1098214009352361
   Web of Science ®Google Scholar
• Cherin, D., & Meezan, W. (1998). Evaluation as a means of organizational learning. Administration in Social Work, 22(2), 1–21. doi:10.1300/J147v22n02_01
   Web of Science ®Google Scholar
• Courtney, R. (2002). Strategic management for nonprofit organizations. New York: Routledge.
   Google Scholar
• Cousins, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Framing the capacity to do and use evaluation. New Directions for Evaluation, 2014(141), 7–23. doi:10.1002/ev.20076
   Google Scholar
• De Vita, C. J., & Fleming, C. (Eds.). (2001). Building capacity in nonprofit organizations. Washington, DC: Urban Institute. Retrieved from http://www.urban.org/UploadedPDF/building_capacity.PDF
   Google Scholar
• Despard, M. R. (in press). Challenges in implementing evidence-based practices and programs in nonprofit human service organizations. Journal of Evidence-Based Social Work.
   Google Scholar
• Ebrahim, A., & Rangan, V. K. (2010). The limits of nonprofit impact: A contingency framework for measuring social performance (Working Paper 10–099). Retrieved from the Harvard Business School website: http://www.hbs.edu/research/pdf/10-099.pdf
   Google Scholar
• Fine, A. H., Thayer, C. E., & Coghlan, A. (2000). Program evaluation practice in the nonprofit sector. Nonprofit Management and Leadership, 10, 331–339. doi:10.1002/nml.10309
   Google Scholar
• Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19, 531–540. doi:10.1177/1049731509335549
   Web of Science ®Google Scholar
• Francis, K., Minzner, A., Fisman, L., & Fox, L. (2011). Building non-profit capacity and community partnerships: Findings from the communities empowering youth (CEY) evaluation (OPRE Report #2011–35). Retrieved from the Administration for Children and Families website: http://www.acf.hhs.gov/sites/default/files/opre/cey_final_report.pdf
   Google Scholar
• Fraser, M. W., & Galinsky, M. J. (2010). Steps in intervention research: Designing and developing social programs. Research on Social Work Practice, 20, 459–466. doi:10.1177/1049731509358424
   Web of Science ®Google Scholar
• Froelich, K. A. (1999). Diversification of revenue strategies: Evolving resource dependence in nonprofit organizations. Nonprofit and Voluntary Sector Quarterly, 28, 246–268. doi:10.1177/0899764099283002
   Web of Science ®Google Scholar
• Garrow, E. E. (2011). Receipt of government revenue among nonprofit human service organizations. Journal of Public Administration Research and Theory, 21, 445–471. doi:10.1093/jopart/muq074
   Web of Science ®Google Scholar
• Gilmer, S. D. (2012). A description of nonprofit executive leaders’ perceptions of human resource capacity building within their organizations ( Doctoral dissertation). Available from ProQuest Dissertations and Theses. (1013852299)
   Google Scholar
• Grantmakers for Effective Organizations. (2011). Smarter relationships, better results: Making the most of grantmakers’ work with intermediaries. Retrieved from http://docs.geofunders.org/?filename=geo_intermediary_briefing_final.pdf
   Google Scholar
• Guo, S., & Fraser, M. W. (2010). Propensity score analysis: Statistical methods and applications. Thousand Oaks, CA: Sage.
   Google Scholar
• Hasenfeld, Y., & Schmid, H. (1989). The life cycle of human service organizations: An administrative perspective. Administration in Social Work, 13, 243–269. doi:10.1300/J147v13n03_12
   Web of Science ®Google Scholar
• Hendricks, M., Plantz, M. C., & Pritchard, K. J. (2008). Measuring outcomes of United Way–funded programs: Expectations and reality. New Directions for Evaluation, 2008(119), 13–35. doi:10.1002/ev.266
   Google Scholar
• Hill, C. W. L., & Jones, G. R. (2008). Strategic management: An integrated approach (8th ed.). Boston, MA: Houghton-Mifflin.
   Google Scholar
• Hillman, A. J., Withers, M. C., & Collins, B. J. (2009). Resource dependence theory: A review. Journal of Management, 35, 1404–1427. doi:10.1177/0149206309343469
   Web of Science ®Google Scholar
• Hoefer, R. (2000). Accountability in action? Program evaluation in nonprofit human service agencies. Nonprofit Management and Leadership, 11, 167–177. doi:10.1002/nml.11203
   Google Scholar
• Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). Hoboken, NJ: Wiley & Sons.
   Google Scholar
• Hunter, D. E. (2006). Using a theory of change approach to build organizational strength, capacity and sustainability with not-for-profit organizations in the human services sector. Evaluation and Program Planning, 29, 193–200. doi:10.1016/j.evalprogplan.2005.10.003
   Web of Science ®Google Scholar
• Innovation Network. (2012). State of evaluation 2012: Evaluation practice and capacity in the nonprofit sector. Retrieved from the Innovation Network website: http://www.innonet.org/client_docs/innonet-state-of-evaluation-2012.pdf
   Google Scholar
• Kapucu, N., Healy, B. F., & Arslan, T. (2011). Survival of the fittest: Capacity building for small nonprofit organizations. Evaluation and Program Planning, 34, 236–245. doi:10.1016/j.evalprogplan.2011.03.005
   PubMed Web of Science ®Google Scholar
• Kutner, M. H., Nachtsheim, C. J., & Neter, J. (2004). Applied linear regression models (4th ed.). New York: McGraw Hills/Irwin.
   Google Scholar
• Leake, R., Green, S., Marquez, C., Vanderburg, J., Guillaume, S., & Gardner, V. A. (2007). Evaluating the capacity of faith-based programs in Colorado. Research on Social Work Practice, 17, 216–228. doi:10.1177/1049731506296408
   Web of Science ®Google Scholar
• LeRoux, K., & Wright, N. S. (2010). Does performance measurement improve strategic decision making? Findings from a national survey of nonprofit social service agencies. Nonprofit and Voluntary Sector Quarterly, 39, 571–587. doi:10.1177/0899764009359942
   Web of Science ®Google Scholar
• Light, P. C. (2004). Sustaining nonprofit performance: The case for capacity building and the evidence to support it. Washington, DC: Brookings Institution Press.
   Google Scholar
• Liket, K. C., Rey-Garcia, M., & Maas, K. E. (2014). Why aren’t evaluations working and what to do about it: A framework for negotiating meaningful evaluation in nonprofits. American Journal of Evaluation, 35, 171–188. doi:10.1177/1098214013517736
   Web of Science ®Google Scholar
• Lynch-Cerullo, K., & Cooney, K. (2011). Moving from outputs to outcomes: A review of the evolution of performance measurement in the human service nonprofit sector. Administration in Social Work, 35, 364–388. doi:10.1080/03643107.2011.599305
   Web of Science ®Google Scholar
• Markovitz, C., Magged, L., Florez, M., & Klein, S. (2008). HOPE II: Faith-based and community organization program evaluation study, outcomes evaluation report (U.S. Department of Justice Document No. 224988). Retrieved from the U.S. Department of Justice website: https://www.ncjrs.gov/pdffiles1/nij/grants/224988.pdf
   Google Scholar
• Maynard, B. R. (2010). Social service organizations in the era of evidence-based practice: The learning organization as a guiding framework for bridging science to service. Journal of Social Work, 10, 301–316. doi:10.1177/1468017309342520
   Web of Science ®Google Scholar
• Mayne, J. (2007). Challenges and lessons in implementing results-based management. Evaluation, 13, 87–109. doi:10.1177/1356389007073683
   Google Scholar
• McBeath, B., Briggs, H. E., & Aisenberg, E. (2010). Examining the premises supporting the empirically supported intervention approach to social work practice. Social Work, 55, 347–357. doi:10.1093/sw/55.4.347
   PubMed Web of Science ®Google Scholar
• McKeever, B., & Pettijohn, S. L. (2014). The nonprofit sector in brief 2014: Public charities, giving, and volunteering. Washington, DC: Urban Institute. Retrieved from http://www.urban.org/UploadedPDF/413277-Nonprofit-Sector-in-Brief-2014.pdf
   Google Scholar
• McKinsey and Company. (2001). Effective capacity building in nonprofit organizations. Retrieved from http://www.vppartners.org/sites/default/files/reports/full_rpt.pdf
   Google Scholar
• Minzner, A., Jastrzab, J., Klerman, J., Markovitz, C., Williams, J., Fink, B. … Ryan, W. (2010). Improving capacity among nonprofits: Impact study of the Compassion Capital Fund demonstration program. Retrieved from the Administration for Children and Families website: http://www.acf.hhs.gov/sites/default/files/opre/ccf_impact_rprt_final.pdf
   Google Scholar
• Minzner, A., Klerman, J. A., Markovitz, C. E., & Fink, B. (2014). The impact of capacity-building programs on nonprofits: A random assignment evaluation. Nonprofit and Voluntary Sector Quarterly, 43, 547–569. doi:10.1177/0899764013491013
   Web of Science ®Google Scholar
• Newcomer, K. E. (1997). Using performance measurement to improve programs. New Directions for Evaluation, 1997(75), 5–14. doi:10.1002/ev.1076
   Google Scholar
• Nichols, A., & Schaffer, M. (2007, September). Clustered errors in Stata. Paper presented at a meeting of the United Kingdom Stata Users’ Group. Retrieved from http://www.stata.com/meeting/13uk/nichols_crse.pdf
   Google Scholar
• Oster, S. M. (1995). Strategic management for nonprofit organizations: Theory and cases. New York: Oxford University Press.
   Google Scholar
• Patrizi, P. A., Gross, E. K., & Freedman, S. (2006). Strength in flexibility: Lessons from a cluster of capacity building grants in the juvenile justice field. Evaluation and Program Planning, 29, 162–170. doi:10.1016/j.evalprogplan.2006.01.010
   Web of Science ®Google Scholar
• Pejsa, L. J. (2011). Improving evaluation in the nonprofit sector: The promise of evaluation capacity building for nonprofit social service organizations in an age of accountability ( Doctoral dissertation). Available from ProQuest Dissertations and Theses. (885876409)
   Google Scholar
• Peng, C. Y. J., Lee, K. L., & Ingersoll, G. M. (2002). An introduction to logistic regression analysis and reporting. Journal of Educational Research, 96, 3–14. doi:10.1080/00220670209598786
   Web of Science ®Google Scholar
• Pfeffer, J., & Salancik, G. R. (1978). The external control of organizations: A resource dependence perspective. New York: Harper & Row.
   Google Scholar
• Poister, T. H. (2010). Performance measurement: Monitoring program outcomes. In J. S. Wholey, H. P. Hatry, & K. E. Newcomer (Eds.), Handbook of Practical Program Evaluation (3rd ed., pp. 100–124). San Francisco: Jossey-Bass.
   Google Scholar
• Popescu, M., & Dewan, S. (2009). Esperanza Trabajando gains, losses, lessons learned and steps forward: An evaluation report. International NGO Journal, 4, 403–420. Retrieved from http://www.academicjournals.org/article/article1381741271_Popescu%20and%20Dewan.pdf
   Google Scholar
• Raynor, J., Cardona, C., Knowlton, T., Mittenthal, R., & Simpson, J. (2014). Capacity building 3.0: How to strengthen the social ecosystem (TCC Group Briefing Paper). Retrieved from http://www.tccgrp.com/pdfs/11-18-14_TCC_Capacity_3.pdf
   Google Scholar
• Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. doi:10.1093/biomet/70.1.41
   Web of Science ®Google Scholar
• Ryan, W. P., & Taylor, B. E. (2012). An experiment in scaling impact: Assessing the growth capital aggregation pilot. Retrieved from the Edna McConnell Clark Foundation website: http://www.emcf.org/fileadmin/media/PDFs/GCAPReport_Final.pdf
   Google Scholar
• Salamon, L. M., & Geller, S. L. (2007). Nonprofit fiscal trends and challenges (No. 6). Baltimore, MD: Johns Hopkins University, Center for Civil Society Studies, Institute for Policy Studies. Retrieved from http://ccss.jhu.edu/wp-content/uploads/downloads/2011/09/LP_Communique6_2007.pdf
   Google Scholar
• Schmid, H. (2013). Nonprofit human services: Between identity blurring and adaptation to changing environments. Administration in Social Work, 37, 242–256. doi:10.1080/03643107.2012.676611
   Web of Science ®Google Scholar
• Simon, J. S. (2001). The 5 life stages of nonprofit organizations. St. Paul, MN: Amherst Wilder Foundation.
   Google Scholar
• Sobeck, J., & Agius, E. (2007). Organizational capacity building: Addressing a research and practice gap. Evaluation and Program Planning, 30, 237–246. doi:10.1016/j.evalprogplan.2007.04.003
   PubMed Web of Science ®Google Scholar
• Sobeck, J. L. (2008). How cost-effective is capacity building in grassroots organizations? Administration in Social Work, 32, 49–68. doi:10.1300/J147v32n02_04
   Web of Science ®Google Scholar
• Sontag-Padilla, L. M., Staplefoote, L., & Gonzalez Morganti, K. (2012). Financial sustainability for nonprofit organizations: A review of the literature. Retrieved from the Rand Corporation website: http://www.rand.org/content/dam/rand/pubs/research_reports/RR100/RR121/RAND_RR121.pdf
   Google Scholar
• StataCorp. (2013). Stata statistical software: Release 13. College Station, TX: StataCorp LP.
   Google Scholar
• Stern, K. (2013). With charity for all: Why charities are failing and a better way to give. New York: Random House.
   Google Scholar
• Stevens, S. (2002). Nonprofit lifecycles: Stage-based wisdom for nonprofit capacity. St. Paul, MN: Stagewise Enterprises.
   Google Scholar
• TCC Group. (2010). Fortifying L.A.’s nonprofit organizations: Capacity-building needs and services in Los Angeles County. Retrieved from the Weingart Foundation website: http://www.weingartfnd.org/files/Capacity-Report-Final.pdf
   Google Scholar
• Thurston, W. E., & Potvin, L. (2003). Evaluability assessment: A tool for incorporating evaluation in social change programmes. Evaluation, 9, 453–469. doi:10.1177/1356389003094006
   Google Scholar
• Van Slyke, D. M. (2007). Agents or stewards: Using theory to understand the government-nonprofit social service contracting relationship. Journal of Public Administration Research and Theory, 17(2), 157–187. doi:10.1093/jopart/mul012
   Web of Science ®Google Scholar
• Vittinghoff, E., & McCulloch, C. E. (2007). Relaxing the rule of ten events per variable in logistic and Cox regression. American Journal of Epidemiology, 165, 710–718. doi:10.1093/aje/kwk052
   PubMed Web of Science ®Google Scholar
• W. K. Kellogg Foundation. (2004). Logic model development guide. Retrieved from http://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide
   Google Scholar
• Weerawardena, J., McDonald, R. E., & Mort, G. S. (2010). Sustainability of nonprofit organizations: An empirical investigation. Journal of World Business, 45, 346–356. doi:10.1016/j.jwb.2009.08.004
   Web of Science ®Google Scholar
• Wells, R., Jolles, M. P., Chuang, E., McBeath, B., & Collins-Camargo, C. (2014). Trends in local public child welfare agencies 1999–2009. Children and Youth Services Review, 38, 93–100. doi:10.1016/j.childyouth.2014.01.015
   PubMed Web of Science ®Google Scholar
• Wetta-Hall, R., Ablah, E., Oler-Manske, J., Berry, M., & Molgaard, C. (2004). Strategies for community-based organization capacity building: Planning on a shoestring budget. Health Care Manager, 23, 302–309. doi:10.1097/00126450-200410000-00003
   PubMedGoogle Scholar
• Wing, K. T. (2004). Assessing the effectiveness of capacity-building initiatives: Seven issues for the field. Nonprofit and Voluntary Sector Quarterly, 33, 153–160. doi:10.1177/0899764003261518
   Web of Science ®Google Scholar
• Witesman, E. M., & Fernandez, S. (2013). Government contracts with private organizations: Are there differences between nonprofits and for-profits? Nonprofit and Voluntary Sector Quarterly, 42, 689–715. doi:10.1177/0899764012442592
   Web of Science ®Google Scholar
• Yung, B., Leahy, P., Deason, L. M., Fischer, R. L., Perkins, F., Clasen, C., & Sharma, M. (2008). Capacity-building needs of minority health nonprofits. Evaluation and Program Planning, 31, 382–391. doi:10.1016/j.evalprogplan.2008.08.001
   PubMed Web of Science ®Google Scholar
• Zietlow, J. (2010). Nonprofit financial objectives and financial responses to a tough economy. Journal of Corporate Treasury Management, 3, 238–248. Retrieved from http://henrystewart.metapress.com/link.asp?id=717321tt51533124
   Google Scholar