Evaluating concordance between government administrative data and externally collected data among high-volume government health facilities in Uttar Pradesh, India

ABSTRACT

Background: Globally, opportunities to validate government reports through external audits are rare, notably in India. A cross-sectional maternal health study in Uttar Pradesh, India’s most populous state, compares government administrative data and externally collected data on maternal health service indicators.

Objectives: Our study aims to determine the level of concordance between government-reported health facility data compared to externally collected health facility data on the same maternal healthcare quality indicators. Second, our study aims to explore whether the level of agreement between government administrative data versus the externally collected data differs by level of facility or by type of maternal health service.

Methods: Facility assessment surveys were administered to key health staff by government-hired enumerators from January 2017 to March 2017 at nearly 750 government health facilities across UP. The same survey was re-conducted by external data collectors from August 2017 to October 2017 at 40 of the same facilities. We conduct comparative analyses of the two datasets for agreement among the same measures of maternal healthcare quality.

Results: The findings indicate concordance between most indicators across government administrative data and externally collected health facility data. However, when stratified by facility-level or service type, results suggest significant over-reporting in the government administrative data on indicators that are incentivized. This finding is consistent across all levels of facilities; however, the most significant disparities appear at higher-level facilities, namely District Hospitals.

Conclusion: This study has a number of important programmatic and policy implications. Government administrative health data have the potential to be highly critical in informing large-scale quality improvements in maternal healthcare quality, but its credibility must be readily verifiable and accessible to politicians, researchers, funders, and most importantly, the public, to improve the overall health, patient experience, and well-being of women and newborns.

KEYWORDS:

• Quality of care
• external audit
• government health system
• incentives
• India

Responsible Editor Stig Wall, Umeå University, Sweden

Responsible Editor Stig Wall, Umeå University, Sweden

Acknowledgments

We are grateful to all the members of the SPARQ team at the University of California, San Francisco, National Health Mission of Uttar Pradesh (Mission Director, Mr. Pankaj Kumar), and Community Empowerment Lab (Mr. Vinay Pratap Singh). We thank all of our local data collectors and study participants in Uttar Pradesh, India.

Disclosure statement

No potential conflict of interest was reported by the authors.

Ethics and consent

Ethical review and clearance were provided by the Institutional Review Boards of the University of California, San Francisco in California and the Community Empowerment Lab in Lucknow, Uttar Pradesh, India.

Paper context

Government administrative data have the potential to be highly critical in informing large-scale quality improvement programs in maternal health in India and globally. Our study findings from India suggest significant over-reporting in government administrative data on maternal health indicators that are incentivized, with the largest discordance at the highest-level facilities. The credibility of government health facility data must be readily verifiable and accessible to politicians, funders, and the public to ensure robust health policies.

Additional information

Funding

This research was funded by the Bill and Melinda Gates Foundation (OPP1127467). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Notes on contributors

Beth S. Phillips

BP conceived the manuscript, refined the tools, conducted the data analysis, and led the writing. SS and SM supported data collection and preparation, analysis, and writing. SYC provided editorial review and assisted with data interpretation. MS and FK designed the study, developed the tools, and supported the writing of the manuscript. All authors read and approved the final manuscript.