An Event Log Repair Method Based on Masked Transformer Model

Figures & data

Figure 1. The model architecture of Transformer Encoder (based on(Vaswani, Shazeer, and Parmar et al. 2017)).

Figure 2. The architecture of the event log repair method.

Figure 3. Masking strategies for different noise additions.

Figure 4. An example of an input feature.

Table 1. Information of event log.

Datasets	Cases	Events	Activities	Max. case length
BPIC2012	13,087	262,200	36	175
BPIC2013	7,554	65,533	13	123
BPIC2017	31,509	1,202,267	66	180
Helpdesk	4580	21,348	14	15
Small	2,000	28,000	14	14
BPIC2015_1	1,199	52,217	398	101
BPIC2015_2	829	44,347	410	132
BPIC2015_3	1,377	59,533	383	124
BPIC2015_4	1,046	47,271	356	116
BPIC2015_5	1,153	59,068	389	154

Table 2. Details of masked transformer-based event log repair method.

Module	Hyperparameter	Values
Masked Transformer-Based Event Log Repair method	Epochs	100
	Batch size	32
	Number of Transformer layers	5
	Number of attention heads	8
	Embedding size	64
	Learning rate	0.1
	Dropout	0.1

Table 3. Confusion matrix(Sokolova and Lapalme 2009).

Data class	Classified as positive	Classified as negative
Positive	True positive ($t_p$)	False negative ($f_n$)
Negative	False positive ($f_p$)	True negative ($t_n$)

Table 4. Evaluation metrics(Sokolova and Lapalme 2009).

Measure	Formula	Evaluation focus
Accuracy	$\frac{t_p+t_n}{t_p+t_n+f_p+f_n}$	Overall effectiveness of the classifier
Precision	$\frac{t_p}{t_p+f_p}$	Class agreement of the positive labels being predicted
Recall	$\frac{t_p}{t_p+f_n}$	How well does the classifier predict positive labels
F1-Score	$\frac{t_p}{t_p+\frac{1}{2}(f_p+f_n)}$	Relations between the data’s positive labels and those given by the classifier

Figure 5. Performance of the model with different noise addition schemes (fixed 15% noise inclusion rate).

Table 5. Execution time reported in milliseconds(Fixed 15% noise content).

Data	Our method(8-1-1)	Our method(Mask_All)	Our method(Random_All)
BPIC2012	137	137	133
BPIC2013	61	60	61
BPIC2017	147	147	147
Helpdesk	45	44	45
SMALL	39	40	39

Table 6. Details of hyperparameters for the Baseline Methods(Nguyen et al. 2019).

Data	Hidden size [AE, VAE, LAE]
BPIC2012	[300, 100, Fani Sani, Zelst, and van der Aalst 2018]
BPIC2013	(Nguyen et al. 20190, Fischer, Goel, and Andrews et al. 2020, 10)
SMALL	(Guo, Lin, and Yang et al. 2021, Fani Sani, Zelst, and van der Aalst 2018, Fischer, Goel, and Andrews et al. 2020)

Table 7. Accuracy of missing noise repair.

Event log	Missingness	VAE(Nguyen et al. 2019)	AE(Nguyen et al. 2019)	LAE(Nguyen et al. 2019)	Ours
BPIC2012	30%	0.691	0.792	0.489	0.957
	35%	0.650	0.787	0.280	0.945
	40%	0.643	0.760	0.320	0.947
	50%	0.608	0.750	0.370	0.936
BPIC2013	30%	0.730	0.786	0.743	0.758
	35%	0.749	0.753	0.737	0.757
	40%	0.737	0.788	0.766	0.756
	50%	0.715	0.763	0.720	0.765
SMALL	30%	0.947	0.964	0.619	0.987
	35%	0.940	0.970	0.674	0.978
	40%	0.911	0.954	0.660	0.967
	50%	0.901	0.937	0.659	0.932

Table 8. Performance of the model on the BPIC2015 dataset.

Event log	Precision	Recall	F1_score	Accuracy	Val_Accuracy
BPIC2015_1	0.933	0.865	0.881	0.309	0.942
BPIC2015_2	0.933	0.839	0.861	0.209	0.941
BPIC2015_3	0.937	0.861	0.880	0.379	0.959
BPIC2015_4	0.932	0.860	0.877	0.359	0.950
BPIC2015_5	0.950	0.864	0.890	0.355	0.957

Figure 6. Process models mined using the small artificial event log.

Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and L. Polosukhin. 2017. Attention is all you need[J]. Advances in Neural Information Processing Systems:30.

 Google Scholar

Sokolova, M., and G. Lapalme. 2009. A systematic analysis of performance measures for classification tasks. Information Processing and Management 45 (4):427–37. doi:10.1016/j.ipm.2009.03.002.

 Web of Science ®Google Scholar

Nguyen, H. T. C., S. Lee, J. Kim, J. Ko, and M. Comuzzi. 2019. Autoencoders for improving quality of process event logs[J]. Expert Systems with Applications 131:132–47. doi:10.1016/j.eswa.2019.04.052.

 Web of Science ®Google Scholar

Fani Sani, M., S. J. Zelst, and W. M. P. van der Aalst. 2018. Repairing outlier behaviour in event logs[C]//International Conference on Business Information Systems, 115–31. Cham: Springer.

 Google Scholar

Fischer, D. A., K. Goel, R. Andrews, C.G.J. van Dun, M.T. Wynn, and M. Röglinger. 2020. Enhancing Event Log Quality: Detecting and Quantifying Timestamp Imperfections. In Business Process Management. BPM 2020, ed. D.Fahland, C.Ghidini, J.Becker, and M.Dumas. Lecture Notes in Computer Science, Vol. 12168.

 Google Scholar

Guo, H., X. Lin, J. Yang, J. Bai, T. Zheng, B. Zhang, and Z. Li. 2021. Translog: A unified transformer-based framework for log anomaly detection[J]. arXiv preprint arXiv:2201.00016.

 Google Scholar