The Critical Role of Implementing Partners: Evidence From Training Micro and Small Enterprises Across 12 Financial Service Providers

Abstract

Despite access to financial resources, many micro and small-scale entrepreneurs struggle to grow their businesses due to management inefficiencies. This study analyses a globally recognized ILO business training initiative in Indonesia, involving 12 financial service providers in 2018. Using an RCT and panel data from 3,975 clients to study the impacts of the business training we find largely null results across a large range of outcomes. Exploiting variation across financial institutions implementing the program, we show that one financial service provider demonstrated significant improvements in its clients’ behaviours. This uneven outcome suggests that the efficacy of large-scale interventions critically depends on the quality of implementation by individual partners. The overarching conclusions advise restraint in broadly expanding these programs, while simultaneously highlighting the crucial role of partner selection in the successful implementation of these initiatives.

Keywords:

• entrepreneurship training
• management skills
• microfinance
• micro and small enterprises
• program evaluation
• randomised control trial

JEL Codes:

• O17
• L26
• O12
• D22

Acknowledgements

Funding for this work was provided by the State Secretariat for Economic Affairs (SECO) of Switzerland, administered through the International Labour Organization (ILO). The views expressed here are our own and do not necessarily reflect the views of any of the funders.

The data collection was commissioned by ILO, which hired independent parties to collect the relevant information from the beneficiaries. We commit to provide anonymised data to bona fide researchers upon request, after notifying ILO if no objection from ILO was received. Replication do-files are readily available in the Supplementary Material.

We would like to thank David McKenzie, Nathan Fiala, Albrecht Bohne, seminar participants at C4ED (internal seminar), and conference participants at PegNet in Bonn and the DIW in Berlin (VfS annual conference on Development Economics and Policy) as well as two anonymous referees for valuable comments and feedback on earlier versions of this work. This work would not have been possible without the key contributions by the ILO team, in particular Owais Parray, Muce Mochtar, Yanis Saputra, Agustinus Simon Petrus Siregar, and Yousra Hamed. We thank the financial institutions that participated in the pilot project for their cooperation, and the University of Padjajaran (UNPAD) and Institut Teknologi Sepuluh Nopember (ITS) for conducting the baseline and endline survey. Adelina Gamarow, Mridhula Mohan, Tatjana Kulp, and Ulugbek Aminjonov provided excellent research assistance.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 To comply with best research standards, we uploaded a Pre-Analysis Plan (PAP) on the AEA website (RCT ID: AEARCTR-0003625) before the endline data was collected. The PAP includes 16 hypotheses and a detailed description of the outcomes of interest. The PAP builds on our original intention to analyse data from 13 participating FSPs. One FSP, a development bank, changed its offer of treatment arms and it is unclear what services were offered to whom at what time. Therefore, we disregard the data from this FSP in this paper. A detailed analysis by providers, as part of the heterogeneity analysis, was requested by the financial institutions and agreed upon prior to the endline data collection. All banks additionally received individual reports indicating key descriptive statistics and impacts.

2 For example, Karlan and Valdivia (2011) find no overall impact of training on profits or employment, Drexler et al. (2014) find a significant impact in a rule-of-thumb training, Lafortune, Riutort, and Tessada (2017) find that visits of role models can increase training impact, and Seitz, Menkhoff, and Grohmann (2020) find providing feedback can make a difference. McKenzie and Woodruff (2014) share a comprehensive overview of similar early evaluations and the general caveats in providing classroom training.

3 Please refer to related work by Bloom et al. (2013); Bruhn, Karlan, and Schoar (2013); Iacovone, Maloney, and Mckenzie (2022); Karlan, Knight, and Udry (2015); Lafortune et al. (2017) for the effects of counselling interventions.

4 See also relevant discussions in Banerjee et al. (2017) and Bird et al. (2021). Related, Swain and Varghese (2013) show that training delivery mechanisms affect the impact of the training with training provided by non-governmental organisations achieving greater impact than training provided by government officials in bank linkage groups in India. Their setting is however non-experimental and might be suffering from self-selection.

5 The intervention of interest in this study was a pilot within the broader Promoting Micro and Small Enterprises through Improved Entrepreneurs’ Access to Financial Services (PROMISE-IMPACT) initiative of the ILO in Indonesia.

6 At the time of our study, by law 20/2008, the Ministry of Cooperatives and SMEs characterised micro (small) enterprises as having net assets below 50 (500) mln IDR or annual revenues below 300 mln (2.5 bln) IDR. The Central Bureau of Statistics (Badan Pusat Statistik, BPS) follows an employment-based definition, with microenterprises employing 1–4 people and small enterprises 5–19 people. The clients in our study fall under MSEs in both definitions.

7 Shedding some more light on this, Cole, Sampson, and Zia (2009) investigate whether a low demand for financial services in Indonesia is rational (prices are higher than productivity) or constrained by information asymmetries (lack of financial literacy). Their results rather favour the former, given that financial literacy training only marginally affected demand.

8 The ILO supported the FSPs in conducting (not necessarily representative) surveys with a total of 2,405 clients in autumn 2016 in order to understand and prioritise the needs of their clients. A staggering 82 per cent believed business support services to be important, yet only a small part had received training and mostly only related to repayment of loans – a topic they themselves deemed of little importance. In contrast, clients prioritised business support services regarding marketing, product quality improvement, and financial management. The self-reported willingness to pay for these services is, however, low, with more than half of the respondents stating that they would not pay any fee.

9 20 per cent of clients report five or more visits, but at 25 per cent the mode of the distribution is two counselling sessions.

10 The attrition rate is 5.3 per cent in Field, Jayachandran, and Pande (2010), 8 per cent in de Mel et al. (2008), 24 per cent in Karlan and Valdivia (2011), and 26 per cent in Calderon, Cunha, and De Giorgi (2013) as presented in McKenzie and Woodruff (2014).

11 Note on vector of randomisation strata as covariates: we have used 65 baseline variables for the randomisation, that is we ensured that treatment and control groups are balanced on these variables. However, this was done by batch, so when combining the whole sample together and excluding attrited households, there may be some imbalances in baseline characteristics. We cannot control for all of them due to their large number. Instead, we automatically select those 20 covariates which have the greatest normalised mean difference between treatment and control group. This implies that different covariates X_i are used for different sub-samples. For our full sample, which pools all FSPs these variables are: indicator for clients having add. income from full-time job, average number of loans per year, indicator for stating tough competition as a main barrier, last loan amount in mln Indonesian Rupiah (IDR), indicator for having last loan as business/individual loan, indicator for stating that business brings high income, indicator for not stating any business barrier, indicator for having written contracts with the workers, indicator for having positive spending on durables, indicator for having no additional income, indicator for wanting to borrow more, indicator for business not being registered, indicator for business being active throughout the year, indicator for business being part of the household, cost per day for workers’ salaries, indicator for stating that business brings respect, indicator for having a university degree, indicator for keeping records of all transactions. It is also well noted that bias might also arise from near-balanced variables if they have predictive power for future outcomes. We therefore use a PDS lasso algorithm, which selects the baseline controls that best predict either the outcome of interest or treatment. The point estimates for all but the quantitative outcomes on cost and profits remain unchanged, and for the latter inference is unaffected. However, our base model is more conservative and avoids selecting covariates based on outcomes. Results with varying control variables are available upon request (for tested model specifications see Supplementary Material).

12 Note on indicators for imputed values as covariates: there are a few cases of item non-responses in some baseline randomisation variables in the raw data. We impute the missing values by estimating a regression model and include the imputed values in the vector X_i. To account for this imputation, we include a set of indicator variables MissX_i which are equal to 1 if a variable in a specific group has been imputed (to account for collinearity).

13 Note on FSP fixed effects: these account for differences between FSPs which stem from systematic differences in the client base, differential commitment to the intervention, loan officer qualifications, or other unobservable factors.

14 This model slightly deviates from the one specified in the PAP. Here, we had included batch fixed effects and also intended to use the full set of randomisation variables. We calculated these alternative models as robustness checks and find inference to remain unchanged. The reason we had to deviate from the PAP model is the stark, not anticipated variation in the implementation of the treatment between the FSPs. By controlling for 12-1 FSP fixed effects instead of only 3-1 batch fixed effects, we believe that we can capture more variation within each FSP, reducing a potential omitted variable bias, and thereby estimating a more restrictive model.

15 Our model furthermore deviates from our Working Paper version by no longer including enumerator fixed effects. We thank an anonymous referee for pointing out that enumerator fixed effects are not warranted in our context without random assignment of enumerators. Results remain largely unchanged, except that we no longer find significant increases in the share of entrepreneurs including cash flow analysis in their business plan.

16 We conduct robustness checks, such as omitting FSP fixed effects, including batch instead of FSP fixed effects, using 10 or 30 most imbalanced control variables, using lasso methods to select control variables which best predict assignment to treatment, adding further control variables, and estimating effects on unwinsorized data. We also repeat the analysis on the sub-sample of observations with only non-missing values, in which case MissX_i is dropped. Additionally, we estimate a model with enumerator fixed effects and one where we drop data from five enumerators who report at least one outlier value in more than 40 per cent of their respondents. While inference on weakly significant estimates changes with the model, our highly significant results remain robust.

17 To present results more concisely, we deviate from the PAP by neglecting the following outcomes: whether the business is exporting outside of Indonesia, is registered, and is paying tax. We also do not show estimates on the content of business plans other than cash flow. There are no effects on any of these outcomes. Similarly, we abstain from presenting estimates on individual indicators that we have consolidated into knowledge and business practice indices.

18 We asked all clients: ‘Are you aware of the existence of services such as classroom training and individual counselling that [FSP name] is providing?’, where we inserted the FSP name from the baseline survey. Only if the answer was affirmative, we asked the next question: ‘In the following, we will refer to this programme as business development programme offered by your financial institution. Were you offered such training or counselling on business development as aforementioned?’ Only if the answer was affirmative, we asked: ‘Did you participate in classroom training or individual counselling sessions or both?’ For all three questions, the answer categories were ‘Yes’, ‘No’, ‘Refused to Answer’, ‘Do not know’/’Not applicable’.

19 The most common reasons for non-participation were that the client could not leave the business unattended (40.5%), or had to attend to household and child care duties (18.8%). Note that female clients were much more likely to participate in treatment when offered: 75 per cent of female clients invited to treatment took up the offer, as opposed to only 52 per cent among male clients. This finding might also be due to the differential reporting as cooperatives offering three treatment arms also have a larger share of female clients. Unaffected by this, we also cannot confirm the finding by Valdivia (2015), where women with young children are less likely to participate than women without young children.

20 Save the mediation analysis, we had specified all other regressions in the PAP and hence only here face the risk associated with multiple hypotheses testing.

21 We also estimate the average causal mediation effect of the three knowledge outcomes on the six business practices for FSP 8. We present results in the Supplementary Material.

22 The experience of varying implementation fidelity is similar to that presented in the work by Karlan and Valdivia (2011): In an entrepreneurship training intervention at different village banks of FINCA Peru only half of the partner banks reached 17 out of 22 envisioned sessions within two years. The authors also report that these delays are typical for similar interventions and conclude that analysis should focus on intention-to-treat effects to avoid selection bias. We agree with this conclusion and also used the initial assignment as the main treatment indicator for our main analysis which hence focuses solely on ITT effects.