Is Textual Information Contained in Financial Documents Informative? Assessing Transparency using Textual Analysis

Abstract

Despite abundant research on government transparency, there have been a limited number of empirical studies that assess government transparency by directly examining the content of information. Using 50 U.S. states from 2002 to 2016, this study examines the relationship between the tone of textual parts in financial documents and financial condition to understand whether governments provide important information about their operations in an easily understandable format (i.e., textual format). Conducting machine-aided textual analysis on Annual Comprehensive Financial Reports (ACFRs), this study finds that state governments ensure effective transparency by making what textual information indicates consistent with objective measures of financial condition. As financial conditions change, states adjust the tone of textual parts accordingly, which can be more easily understood by laypeople than complicated numbers in financial statements. The findings also suggest that declining financial conditions do not lead states to increase the use of uncertainty words to mask negative financial information. This study enhances our understanding of the meaning of transparency, and thereby contributes to the development of accountable public management practices and good governance.

Keywords:

• Accountability
• financial reporting
• textual analysis
• transparency

Acknowledgements

The author would like to thank Jaegyeong Jo for her help with data collection. The author also appreciates feedback from the three anonymous reviewers.

Notes

1 The literature on political communication (more broadly, information communication) can also inform us on how public managers/leaders communicate financial conditions of their governments to the public. It can also provide an insight into why they interpret the financial positions of their governments in the way that is communicated in the MD&A section. Generally, the citizens “have a limited amount of information about politics, a limited knowledge of how government works, and a limited understanding of how governmental actions are connected to consequences of immediate concern to them” (Popkin, 1991, p. 8). Thus, prior studies find that the citizens’ perceptions of and attitudes toward how their governments and politicians perform can be influenced by the information that they obtain from diverse sources, including the mass media (Barabas & Jerit, 2009; Gilens, 2001). The citizens also engage and participate in various civic activities using the information and knowledge about government programs and policies (Barabas & Jerit, 2009; Gilens, 2001; Kuklinski et al., 2000).

2 It is also important to understand how public officials assess financial conditions of their governments. While there have been many studies that use financial indicators to evaluate governments’ financial conditions, limited attention has been given to how public officials assess the financial conditions of their governments (Leiser & Mills, 2019). Among a few, both Leiser and Mills (2019) and Maher and Deller (2011) find that public officials’ self-assessment of financial condition is not associated with objective financial condition that is measured by widely used financial indicators. Using open-ended survey questions, Leiser and Mills (2019, pp. 76–77) find that “local officials—especially those that rate their fiscal health worse—place the most emphasis on long-term issues, such as the need for infrastructure funding, and external stressors, such as lack of intergovernmental revenue.” They also find that “local governments reporting high versus low stress tend to emphasize different factors in explaining their overall stress levels, suggesting that high stress and low stress are not just two ends of a continuum but rather two fundamentally different states” (p. 77).

3 Loughran and McDonald (2011) developed a dictionary in corporate finance contexts. Their dictionary includes six different word categories – negative, positive, uncertainty, litigious, strong modal, and weak modal words. The first three categories of this dictionary are used in this study. This study especially uses a revised version of the Loughran and McDonald (2011) dictionary that was updated in 2018 to better capture unique features of finance contexts.

4 Examples of each word group are given in parentheses. More examples can be found in Appendix Table A1. Examples of how these words are used in ACFRs and MD&As are also provided in Appendix Table A2.

5 Other studies conducting textual analysis on corporate financial documents also conducted analysis on the MD&A section because of the possibility that “the MD&A section is where management is most likely to reveal information through the tone that they use” (Loughran & McDonald, 2011, p. 40). The MD&A section has also attracted attention from public financial management research (e.g., Guo et al., 2009; Yusuf & Jordan, 2017). However, it is also possible that the MD&A is too short to include enough information about government operations and financial condition. Thus, this study also conducts textual analysis on the whole ACFR in addition to the MD&A section.

6 Alternatively, n-grams could have been used, but this study prefers the “bag of words” approach over n-grams for the following reasons. First, there is a specific dictionary developed in finance contexts (i.e., the Loughran and McDonald, (2011) dictionary). Second, “using n-grams is more computationally demanding, since there are many more unique sequences of words than individual words” (Welbers et al., 2017, p. 261). Most importantly, prior studies find that n-grams do not significantly improve the performance of measuring the tone of text (Grimmer & Stewart, 2013).

7 For interested readers, Welbers et al. (2017) could be a good source to understand the standard procedures of textual analysis in R.

8 Since this study uses dictionary-based textual analysis instead of other methods of textual analysis such as supervised or unsupervised machine learning, it does not conduct a test to validate the dictionary and understand to what extent these words capture the positive/negative words in financial reports. Instead, this study follows “the standard practice in using dictionaries,” “which assume[s] the measures created from a dictionary are correct and then apply them to the problem” due to “the exceptional difficulties in validating dictionaries” (Grimmer & Stewart, 2013, p. 275). Following this practice, prior studies that use dictionary-based textual analysis, such as Allee and DeAngelis (2015), Chen et al. (2018), Golder and Macy (2011), Rich et al. (2018), did not perform validity tests.

9 Whereas a government’s ability to manage liquidity can be better measured by dividing cash assets by short-term liabilities, this study calculates the cash position by dividing cash assets by total expenditures because the U.S. Census State and Local Government Finances do not provide any data for short-term liabilities.

10 For example, state ACFRs include a section named “Economic Factors Affecting the State.”

11 Note that singular and plural forms of the same word (e.g., gain and gains) are treated as different words in the Loughran and McDonald (2011) finance dictionary.

12 Additional analyses were conducted by separating total debt into short-term and long-term debt and including only one of them at a time. The results show that only the long-term debt has positive and statistically significant relationships with the net positivity variables.

Additional information

Notes on contributors

Youngsung Kim

Youngsung Kim is an Assistant Professor of Public Policy and Administration in the Department of Political Science at Colorado State University. His research focuses on public budgeting and financial management, computational social science, education finance and policy, and local government management.