Estimation of log-gripping position using instance segmentation for autonomous log loading

Figures & data

Figure 1. The outline of the algorithm for estimating the gripping position and orientation of a log.

Figure 2. Examples of labeled images.

Figure 3. The outline of the estimated gripping position and evaluation axis in the log coordinate system.

Here, l is length of the log, c is the lengthwise center of the log, g is the center of gravity of the log, and g_s is the gripping point in this study.

Figure 4. Experimental setup.

The grapple loader is equipped with a stereo camera to estimate the log gripping position. The total station was used to measure the positions and orientations of the logs.

Figure 5. Trajectory of the machine chassis in the experiment by using simultaneous localization and mapping (SLAM).

(a) Sparse logs. (b) Dense logs. (c) Unorganized logs.

Table 1. Log-shape statistics.

	Large-end diameter (m)	Small-end diameter (m)	Log length (m)	Num. of logs
Experiment 1: Sparse logs
Max	0.251	0.220	4.161	5
Min	0.192	0.132	4.068
Average	0.229	0.185	4.121
S.D.	0.020	0.029	0.038
Experiment 2: Dense logs
Max	0.251	0.235	4.165	10
Min	0.141	0.126	4.068
Average	0.203	0.184	4.120
S.D.	0.033	0.034	0.035
Experiment 3: Unorganized logs
Max	0.255	0.220	4.165	16
Min	0.144	0.111	4.073
Average	0.203	0.164	4.125
S.D.	0.031	0.033	0.030

Figure 6. A three-dimensional diagram of the grapple head.

Figure 7. Examples of log detection in the test dataset.

Figure 8. Examples of log detection during the experiments.

(a) Successful detection in sparse logs. (b) Misdetection in sparse logs. (c) Successful detection in dense logs. (d) Misdetection in dense logs. (e) Successful detection in unorganized logs. (f) Misdetection in unorganized logs.

Figure 9. Estimated positions of gripping logs.

(a) Sparse logs. (b) Dense logs. (c) Unorganized logs.

Figure 10. Estimated positions of gripping logs rotated through π/4 radians around the X-axis.

(a) Sparse logs. (b) Dense logs. (c) Unorganized logs.

Figure 11. Estimated positions of gripping logs rotated through π/4 radians around the Y-axis.

(a) Sparse logs. (b) Dense logs. (c) Unorganized logs.

Table 2. Gripping position estimation results.

	Radial (m)	Axial (m)	Vertical (m)	Estimated num. of points
Sparse logs
Max	0.353	2.052	0.251	142
RMSE	0.162	1.526	0.140
S.D.	0.162	0.785	0.058
Dense logs
Max	0.822	0.676	0.337	108
RMSE	0.384	0.271	0.119
S.D.	0.384	0.271	0.104
Unorganized logs
Max	4.633	3.175	1.520	86
RMSE	0.764	1.022	0.194
S.D.	0.764	0.813	0.191

Table 3. Gripping orientation estimation results.

	Roll (rad)	Pitch (rad)	Yaw (rad)
Sparse logs
Max	1.561	1.526	1.538
RMSE	0.249	0.388	0.533
S.D.	0.249	0.388	0.533
Dense logs
Max	0.685	1.412	1.557
RMSE	0.232	0.594	0.617
S.D.	0.232	0.589	0.614
Unorganized logs
Max	0.977	1.571	1.138
RMSE	0.274	0.736	0.288
S.D.	0.274	0.736	0.286

Rotations around the radial, axial, and vertical axes of the log in Figure 3 were referred to as roll, pitch, and yaw, respectively.

Figure 12. Evaluation outline of the log-camera rotational errors in the camera coordinate system.

The rotational errors α between the X-axis of the camera and the orientation of the log were evaluated.

Figure 13. The positional relations of the absolute rotational error between the X-axis of the camera and the log orientation around the Z-axis of the camera.

(a) Sparse logs. (b) Dense logs. (c) Unorganized logs.

Figure 14. The orientational relations of absolute rotational error between the X-axis of the camera and the log orientation around the Z-axis of the camera.

(a) Sparse logs. (b) Dense logs. (c) Unorganized logs.

Figure 15. Points in the graspable range in each experiment.

(a) Sparse logs. (b) Dense logs. (c) Unorganized logs.

Table 4. Correlation coefficients.

	Radial (m)	Axial (m)	Vertical (m)
Positional errors
Sparse logs	−0.050	0.055	0.081
Dense logs	−0.538***	−0.457***	−0.754***
Unorganized logs	−0.139	−0.640***	−0.024
	Orientational errors
	Roll (rad)	Pitch (rad)	Yaw (rad)
Sparse logs	0.645***	0.196*	0.796***
Dense logs	−0.115	−0.033	−0.009
Unorganized logs	0.031	−0.128	0.005

*p < 0.05, ** p < 0.01, and *** p < 0.001.