The effect of class-balance and class-overlap in the training set for multivariate and product-adapted grading of Scots pine sawn timber

Figures & data

Table 1. Results of the product-grade reference and image-grade reference of the baseline training data set consisting of 251 planks.

	Grades
Grade reference	A	Not-clearly A	Not-clearly B	B	Totals
Product	157	–	–	94	251
Image	123	46	18	64	251

Figure 1. Flow-chart showing the different training scenarios and prediction models. Solid boxes represent models trained on the product-grade (P) or the image-grade (I), respectively. Dotted boxes show the different training scenarios where the two models are compared with the baseline training scenario.

Figure 2. The observed-predicted plot of the baseline training data, using the model trained on the baseline training data with the product-grade reference. The upper observations (1) represents grade A, and the lower (0) represents grade B. The y-axis shows the grade of each plank as the actual binary grade, and the x-axis shows the continuous grade predicted by the model. The encircled observations have a weak correlation between their measured features and their assigned grade, i.e. an observation in the bottom right looks to the model as, and would have been predicted as, a plank of grade A (1) while the product-grade was grade B (0).

Table 2. Misclassification tables for the three training scenarios: baseline, class-overlap, and class-balance.

(a) Baseline training scenario
		Predicted by product-trained		Grading accuracy	Predicted by image-trained		Grading accuracy
Observed	Grade	A	B		A	B
	A	314	72	81%	313	73	81%
	B	73	136	65%	72	137	66%
	Total			76%			76%
(b) Class-overlap training scenario
		Predicted by product-trained		Grading accuracy	Predicted by image-trained		Grading accuracy
Observed	Grade	A	B		A	B
	A	302	84	78%	314	72	81%
	B	84	125	60%	73	136	65%
	Total			72%			76%
(c) Class-balance training scenario
		Predicted by product-trained		Grading accuracy	Predicted by image-trained		Grading accuracy
Observed	Grade	A	B		A	B
	A	310	76	80%	307	79	80%
	B	77	132	63%	78	131	63%
	Total			74%			74%

Note: Each sub-table presents the grading results of the two models trained on the product-grade reference (product-trained) and image-grade reference (image-trained) for that training scenario. The test set contained 386 planks of grade A and 209 planks of grade B.

Table 3. Grading agreement of all models, showing the proportion of the test set graded identically by two models measured in percent (%). The headers show the training scenario and the two corresponding models of that scenario, trained on the reference product-grade (P) and image-grade (I), respectively.

		Baseline		Class-overlap		Class-balance
Scenario	Reference	P	I	P	I	P	I
baseline	P	100
	I	88	100
Class-overlap	P	93	86	100
	I	88	96	85	100
Class-balance	P	95	87	94	87	100
	I	87	92	85	93	87	100