Fraudulent Firm Classification: A Case Study of an External Audit

Figures & data

Figure 1. Audit work-flow.

Table 1. Target sectors.

Sector ID	Target sector	Information	Number of target firms
1	IR	Irrigation	114
2	P	Public Health	77
3	BR	Buildings and Roads	82
4	FO	Forest	70
5	CO	Corporate	47
6	AH	Animal Husbandry	95
7	C	Communication	1
8	E	Electrical	4
9	L	Land	5
10	S	Science and Technology	3
11	T	Tourism	1
12	F	Fisheries	41
13	I	Industries	37
14	A	Agriculture	200

Table 2. Risk factors classification and other features in model.

Inherent risk factors		Control risk factors
Feature	Information	Feature	Information
Para A value	Discrepancy found in the planned-expenditure of inspection and summary report A in Rs (in crore).	Sector score	Historical risk score value of the target-unit in the Table 1 using analytical procedure.
Para B value	Discrepancy found in the unplanned-expenditure of inspection and summary report B in Rs (in crore).	Loss	Amount of loss suffered by the firm last year.
Total	Total amount of discrepancy found in other reports Rs (in crore).	History	Average historical loss suffered by firm in the last 10 years.
Number	Historical discrepancy score.	District score	Historical risk score of a district in the last 10 years.
Money value	Amount of money involved in misstatements in the past audits.
Other features
Feature	Information	Feature	Information
Sector ID	Unique ID of the target sector.	Location ID	Unique ID of the city/province.
ARS	Total risk score using analytical procedure.	Audit ID	Unique Id assigned to an audit case.
Risk class	Risk Class assigned to an audit-case. (Target Feature)

Table 3. Sample data of the corporate sector unit.

Audit ID	Loc ID	Para A	Para B	Total	Sector score	Loss Score	Num.	Money value	History score	District score	ARS	Risk class
26	4	5.78	57.92	73.70	3.89	0	5	11.16	1	2	4	F
27	4	7.42	2.24	19.66	3.89	1	1	1.25	2	2.5	2.4	F
28	4	0	1.10	4.11	3.89	0	3	0.007	2	2	2	N
29	14	6.85	31.76	58.61	3.89	2	5	1.46	1	4	3.6	F
30	14	0	1.03	5.03	3.89	0	5	0	2	2	1	N
31	37	0	0.75	3.75	3.89	0	5	6.78	2	2	2.2	F
32	37	2.4	16.63	29.73	3.89	0	3	1.16	0	4	3.6	F
33	5	0	0.05	1.23	3.89	1.3	5	152.41	2	2	2.4	F
34	5	0	1.76	4.76	3.89	0	2	1.08	2	2	2	N
35	5	0	2.97	6.97	3.89	0	5	2.84	1	2	2	N

Figure 2. Proposed framework for an audit field work decision-making.

Table

Algorithm 1 Risk Assessment Algorithm

Input: Agenda list A.

Output: Risk class of each sample in the agenda list A. Comment: {Let X denote the number of offices to be examined. Assume X is finite. Let A is a list of nominated set of offices called Agenda A, extracted from set X (Here Agenda is a set of all offices that are not being visited since last three years).} 1. Data collection: Collect the unstructured data of all the offices under Agenda A. 2. Feature extraction: Examine k features (risk factors) that may be needed for the inherent and control risk assessment. 3. Risk calculation: - Calculate the loss that may be involved for each risk factor. - Calculate the probability of loss for each risk factor . - Calculate the risk for each risk factor as . 4. Risk classification: Classify the risk factors into inherent-risk class(IN) and control risk class (CO) and calculate the sum of risk for each class as and , respectively. 5. Combined risk: Calculate the combined risk of class CO and IN as 6. Detection risk: Define the detection risk value DR. 7. Audit risk score: Calculate the audit risk as the 8. Risk assessment: Calculate the average of audit risk as . Classify the audit risk for each audit case as high (fraud class) and low (no fraud class) by the following rules: • if the audit risk , label it as No Fraud. • else label it as Fraud.

Figure 3. Prediction model.

Table 4. Risk factors weight assignment using Particle Swarm Optimization (PSO).

Feature ID	Feature name	PSO weight	Feature rank
F1	Para A value	0.18	5
F2	Para B value	0.16	6
F3	Total	0	9
F4	Money value	0.01	8
F5	Number	0.07	7
F6	Sector score	0.21	4
F7	History score	0.69	2
F8	District score	0.50	3
F9	Loss score	0.93	1

Table 5. Confusion matrix.

	True reference
Predicted condition	Fraud	No fraud
Fraud	True positive X	False negative Z
No fraud	False positive Q	True negative Y

Table 6. Performance evaluation metrics.

Performance metric	Formula
Type-I error	Q
Type-II error	Z
Sensitivity
Specificity
Accuracy
Error rate
F1 score
F2 score
MCC

Table 7. Machine learning classification methods.

Model	Method	Package	Tuning parameter	Reference
DT	rpart	rpart	MinSplit = 20, MaxDepth = 30, MinBucket = 7	(Quinlan 1986)
AB	adaboost	fastAdaboost	Default	(Schapire 1999)
RF	rf	randomForest	mtry = 500, sampling = bagging	(Liaw and Wiener 2002)
SVM	ksvm	e1071	nu = 10, epsilon = 0.5	(Keerthi and Gilbert 2002)
PLM	bayesglm	arm	Default	(Chambers 1977, Finney 1992)
NN	neuralnet	nnet	size = 10, linout = TRUE, skip = TRUE, MaxNWts = 10000,trace = FALSE, maxit = 100	(Russell et al. 2003)
DSM	decision stump	RWeka	rules = 6,pruned = 25,smoothed = 0.9	(Iba and Langley 1992)
J48	J48	RWeka	Default	(Quinlan 1996)
NB	NaiveBayes	RWeka	Default	(Rish 2001)
BN	BayesNet	RWeka	Default	(Pearl 1985, Neapolitan et al. 2004)

Table 8. Average performance comparison of machine learning methods for the prediction of an audit risk on testing dataset.

Classifier	Type-I error	Type-II error	Sens.	Spec.	Acc.	Error rate	F1 score	F2 score	MCC	AUC
DT	0.09	0.04	0.88	0.91	0.89	0.11	0.91	0.89	0.78	0.95
AB	0.05	0.11	0.87	0.95	0.91	0.09	0.92	0.89	0.81	0.94
RF	0.01	0.12	0.88	0.99	0.92	0.08	0.93	0.90	0.85	0.96
SVM	0.10	0.14	0.85	0.90	0.87	0.13	0.89	0.86	0.73	0.92
PLM	0.14	0.11	0.84	0.86	0.85	0.15	0.87	0.84	0.68	0.91
NN	0.07	0.12	0.84	0.93	0.88	0.12	0.89	0.86	0.76	0.78
DSM	0.08	0.22	0.78	0.92	0.84	0.16	0.85	0.81	0.69	0.89
J48	0.01	0.09	0.91	0.99	0.94	0.06	0.95	0.92	0.88	0.96
NB	0.05	0.40	0.60	0.95	0.74	0.26	0.73	0.65	0.55	0.93
BN	0.01	0.09	0.91	0.99	0.94	0.06	0.95	0.92	0.88	0.97

Figure 4. Performance score of SAW and TOPSIS MCDM methods.

Figure 5. Ten-fold cross validation of Type-II error, sensitivity, accuracy, and AUC on the testing dataset in the audit risk prediction using Bayes Net, J48, and Random Forest.

Quinlan, J. R. 1986. Induction of decision trees. Machine Learning 1 (1):81–106. doi:10.1007/BF00116251.

 Google Scholar

Freund, Y., R. Schapire,and N. Abe, 1999. A short introduction to boosting. Journal of Japanese Society For Artificial Intelligence 14:771–80.

 Google Scholar

Liaw, A., and M. Wiener. 2002. Classification and regression by randomforest. R News 2 (3):18–22.

 Google Scholar

Keerthi, S. S., and E. G. Gilbert. 2002. Convergence of a generalized smo algorithm for svm classifier design. Machine Learning 46 (1–3):351–60. doi:10.1023/A:1012431217818.

 Web of Science ®Google Scholar

Chambers, J. M. 1977. Computational methods for data analysis. Technical report, New York.

 Google Scholar

Finney, D. J. 1992. Miscellaneous problems. Probit Analysi 4:140–45. JSTOR.

 Google Scholar

Russell, S. J., P. Norvig, J. F. Canny, J. M. Malik, and D. D. Edwards. 2003. Artificial intelligence: A modern approach, Vol. 2, 65–71. Malaysia: Pearson Education Limited

 Google Scholar

Iba, W., and P. Langley 1992. Induction of one-level decision trees. In Proceedings of the ninth international conference on machine learning, Moffett Field, California. 233–40.

 Google Scholar

Quinlan, J. R. 1996. Improved use of continuous attributes in C4.5. Journal of Artificial Intelligence Research 4:77–90.

 Web of Science ®Google Scholar

Rish, I. 2001. An empirical study of the naive bayes classifier. In IJCAI workshop on empirical methods in artificial intelligence 3 (22):41–46. Seattle, Washington: IBM.

 Google Scholar

Pearl, J. 1985. Bayesian networks: A model CF self-activated memory for evidential reasoning. In Proceedings of the 7th Conference of the Cognitive Science Society, Irvine, California, Pp. 329–334.

 Google Scholar

Neapolitan, R. E., et al. 2004. Introduction to Bayesian Networks. In Learning Bayesian Networks, 40. Chicago: Pearson.

 Google Scholar