Improving Tax Audit Efficiency Using Machine Learning: The Role of Taxpayer’s Network Data in Fraud Detection

Figures & data

Table 1. Number of audits and incidence of fraud by main audit types and in total

	2018			2019
Audit	Audit	Fraud	Fraud rate, %	Audit	Fraud	Fraud rate, %
Tax receipt audits	1,561	1,197	77%	1473	986	67%
Unregistered employee audits	374	291	78%	500	341	68%
Budget (comprehensive) audits	978	762	78%	605	284	47%
Other audit types	2,064	1,252	61%	2,673	1,144	43%
Overall total	4,977	3,502	70%	5251	2755	52%

Note: Here, we include the number of audits and not the unique number of companies that a given company can appear several times.

Figure 1. Fraud prediction approach using machine learning.

Note: Numbering corresponds to the steps outlined in the Methodology and Data section.

Table 2. Mean values of selected predictors of audit and fraud used in the study

Features	Positive Fraud					No Fraud
	Min	Max	Median	Mean	St. Dev	Min	Max	Median	Mean	St. Dev
Historical fraud frequency	0.00	65.00	8.00	12.80	13.20	0.00	46.00	4.00	7.30	8.90
Admin cost (%)	0.00	1.00	0.00	0.10	0.10	0.00	1.00	0.10	0.10	0.20
VAT ratio 5	0.00	7303.90	1.30	16.30	236.40	0.00	42059.30	1.60	85.40	1535.20
Supplier invoices count	0.00	53491.20	17.90	585.70	2730.70	0.00	18950.50	6.00	146.10	874.70
Tax receipt audit frequency	0.00	44.00	2.00	2.90	4.00	0.00	23.00	0.00	1.60	2.80
Profitability	−0.90	1.00	0.00	0.10	0.10	−1.00	1.00	0.00	0.10	0.20
VAT ratio 1	0.00	1.00	0.00	0.10	0.30	0.00	1.00	0.00	0.20	0.40
Productivity (mln. AMD)	0.00	–	13.90	26.32	47.63	0.00	–	12.79	47.10	262.36
Employee growth (2017–2018)	−0.90	120.90	0.10	0.40	3.50	−1.00	73.70	0.00	0.40	3.00
Small firm dummy	0.00	1.00	0.00	0.40	0.50	0.00	1.00	0.00	0.40	0.50

Note: Productivity is measured in annual terms and local currency unit. Maximum probability is not reported as the large firms can be identifiable.

Figure 2. Fraud distribution by taxpayer tenure (2019).

Figure 3. Fraud distribution by the taxpayer sector (NACE) of operation (2019).

Note: The key to NACE Rev. 2 sector classification is available in appendix Table A1.

Figure 4. Effect of historical audits and frauds on probability of fraud in 2019.

Table 3. Results of Audit and Fraud classification models

Model	Audit Model	Fraud Models
	ROC AUC on the 2018 test	ROC AUC on the 2018 test	ROC AUC on the 2019 test	2018 test LIFT_1	2019 LIFT_1
Logistic regression	0.91	0.69	0.69	1.26	1.49
Decision tree	0.94	0.67	0.65	1.13	1.49
Random forest	0.94	0.69	0.7	1.39	1.62
Gradient boosting	0.95	0.71	0.73	1.37	1.85

Note: ROC AUC, accuracy, precision, and recall of both train and test performances for the Fraud model are reported in Table A2, for the winner (Gradient Boosting) model for both 2018 and 2019 years.

Figure 5. SHAP values for the variables used in the baseline fraud model.

Note: variables are ranked in descending order in terms of their predictive power. Positive and negative values along the horizontal axis represent the direction and size of effect on the probability of fraud. The color shows whether the value of a given variable is high (red) or low (blue) for that observation. For example, a high level of the “historic fraud” has a high and positive impact on the probability of fraud.

Table 4. Audit model predictions and Fraud model actual accuracy by deciles

Prediction probability decile	Audit model results*			Fraud model results
	Number of taxpayers	Decile probability average	Audited companies in 2020		Fraud detected	Audit accuracy by deciles
1	580	0.677	1125		696	62%
2	138	0.265	386		178	46%
3	49	0.097	236		93	39%
4	23	0.04	48		21	44%
5	7	0.019	5		1	20%
6	0	0.01	66		23	35%
7	1	0.008	78		22	28%
8	0	0.008	190		63	33%
9	0	0.007	134		20	15%
10	0	0.007	273		36	13%
Total taxpayers:	798		2541		1153	45%

*Audit model presents predictions only for comprehensive (budget) audit plans.

Table 5. Taxpayer’s network (supplier and buyer) characteristics used in the fraud model

Feature	Value by
	Fraud	No Fraud
Average audit among buyer partners	0.35	0.32
Average fraud among supplier partners	0.29	0.21
Total number of audits among supplier partners	22.75	14.82
Average audit among supplier partners	0.48	0.40
Maximum number of supplier transactions	78.14	58.76
Average number of supplier transactions	7.45	5.45
Minimum amount of buyer transactions	2,129,537.20	5,856,382.51
Number of buyer partners	194.41	71.35
Average amount of supplier transactions	3,689,081.28	5,763,190.09
Maximum amount of buyer transactions	89,163,224.52	173,719,256.30

Note: Transaction amounts are in Local currency units.

Table 6. Comparison of fraud model performance with the exclusion of historical data and inclusion of taxpayer’s network information

	Baseline Fraud model	Baseline Fraud model without historic features	Model with network features
ROC AUC	0.730	0.712	0.724
First decile lift score	1.85	1.75	1.79
Number of features	29	21	31
Features removed/added		Historical audit and fraud, number of employees annual growth	Supplier/buyer network audit, fraud, and transaction characteristics

Table A1. NACE Rev. 2 economic activity code key for Figure 3

B	Mining and quarrying
C	Manufacturing
D	Electricity, gas, steam, and air conditioning
E	Water supply, sewerage, waste management, and remediation activities
F	Construction
G	Wholesale and retail trade and repair of motor vehicles and motorcycles
I	Accommodation and food service activities
H	Transportation and storage
J	Information and communication
K	Financial and insurance activities
L	Real estate sctivities
M	Professional, scientific, and technical activities
N	Administrative and support service activities
O	Public administration and defense and compulsory social security
P	Education
Q	Human health and social work activities
R	Arts, entertainment, and eecreation
S	Other service activities
T	Activities of households as employers; undifferentiated goods- and services- producing activities of households for own usese
U	Activities of extraterritorial organizations and bodies

Table A2. Feature definitions¹³

Feature name	Feature description or calculation formula
Historical fraud frequency	The number of historical cases that the taxpayer was fined as a result of audit
Admin cost (%)	The ratio of administrative costs over total costs of the taxpayer
Tax receipts audit frequency	The number of historical tax receipt audits of the taxpayer
VAT ratios (1, 4, and 5)	Ratios derived from VAT return form that show various relationships between export, import, total sales, and refunds claimed thereon.
Supplier invoices count	Number of invoices that the taxpayer provided to its buyers
Profitability	The profitability of the taxpayer as the ratio of the profit over total revenue
Employee growth	The change of the number of employees in (year on year)
Productivity	The ratio of the revenue over the number of employees, showing the average revenue generated by an employee
Buyer invoices count	The number of invoices that the taxpayer was provided for its purchases
St. deviation of monthly receipts count	The standard deviation of the number of the monthly tax receipts
Small firm dummy	If the number of employees is between 10 and 50 then 1 and otherwise 0
Tax receipts average monthly count	The average of the number of monthly tax receipts
Budget audit frequency	The number of historical budget audits of the taxpayer
Average monthly receipts	The average of monthly tax receipts amount
Trade sector	If the taxpayer is in the trade sector, then 1 otherwise 0
Buyer invoice average amount	The average amount of the invoices that the taxpayer provided for its purchases
St. deviation of monthly receipts amount	The standard deviation of monthly tax receipts amount

Table A3. Results of Audit Classification models

	Audit				Fraud
Metric	Logistic Regression	Decision Tree	Random Forest	Gradient Boosting	Gradient Boosting 2018	Gradient Boosting 2019
ROC AUC on train	0.920	0.948	0.960	0.969	0.72	0.77
ROC AUC on test	0.908	0.935	0.939	0.948	0.71	0.73
Recall on train	0.872	0.896	0.931	0.650	0.90	0.68
Recall on test	0.853	0.864	0.838	0.541	0.89	0.61
Precision on train	0.428	0.478	0.521	0.836	0.70	0.64
Precision on test	0.432	0.470	0.491	0.733	0.71	0.59
Train accuracy	0.834	0.861	0.881	0.939	0.69	0.70
Test accuracy	0.832	0.853	0.864	0.913	0.69	0.65
LIFT:1st decile	1.34	–	1.289	1.34	–	–

Table A4. Robustness: Alternative definitinos of target fraud variable

Robustness	Fraud >1%of monthly revenue		Fraud for sales receipt audit only
	Train	Test	Train	Test
ROC-AUC	0.79	0.73	0.64	0.61
Precision	0.71	0.62	0.57	0.58
Recall	0.4	0.34	0.83	0.85
Accuracy	0.74	0.71	0.57	0.58