Behaviour recognition of housed sheep based on spatio-temporal information

Figures & data

Figure 1. Video data collection.

Table 1. The basic behaviours of sheep in a house.

Behaviour	Status
Rumination	Limbs curled, back arched upward, head extended and mouth in a chewing state.
Lying	Limbs curled, head pressed against the body or in contact with the ground.
Other	Changing postures, walking, rubbing against walls, defecating, etc.

Table 2. The model training environment and parameter configuration.

Environment	Parameter
Computer operating system	Ubuntu 20.04
GPU	RTX 2060s
CPU	i5-10400
Framework	torch 1.13
GPU configuration	Cuda 11.7;cudnn 8.4
Visual Studio 2015 Community
Python 3.8
YOLOv5 6.0 + DeepSORT v3.0

Figure 2. Flow chart of sheep behaviour identification.

Figure 3. Flowchart of multi-target detection and tracking algorithm.

Figure 4. Selection of the sheep’s keypoints. (a) Schematic of the sheep’s keypoints Selection (b) Example of the sheep’s keypoints labelling 1. Nose; 2. Lower lip; 3. Head; 4. Neck; 5. Back; 6. Rump; 7. The elbow of the left front leg; 8. The elbow of the right front leg; 9. The elbow of the left back leg; 10. The elbow of the right back leg; 11. The knee joint of the left front leg; 12. The knee joint of the right front leg; 13. The knee joint of the left back leg; 14. The knee joint of the right back leg; 15. The toe of the left front leg; 16. The toe of the right front leg; 17. The toe of the left back leg; 18. The toe of the right back leg.

Table 3. The basic behaviours of sheep in a house.

Behaviour	Amount of data
Rumination	119
Lying	168
Other	190

Figure 5. The flow of the Alphapose algorithm.

Figure 6. The illustration of SSTN and parallel SEEP model training strategy.

Figure 7. The result of zeroing of pixel values of non-object area.

Figure 8. Alphapose model and sequence of skeleton keypoints.

Figure 9. ST-GCN network structure.

Figure 10. Graph of the change in the curve of the training indicators.

Table 4. The test results of Yolo V5 for multi-target tracking.

Training		Validation
Evaluation indicators	Result	Evaluation indicators	Result
box_loss	0.017	box_loss	0.159
obj_loss	0.021	obj_loss	0.015
Precision	0.997	mAP_0.5	0.994
Recall	0.996	mAP_0.5:0.95	0.886

Table 5. The average accuracy of Alphapose for different key points.

Keypoints	Average accuracy
Nose	96.5%
Lower lip	98.2%
Head	93.0%
Neck	77.2%
Back	78.8%
Rump	80.7%
Elbow (on four legs)	83.3%
Knee joint (on four legs)	92.1%
Toe (on four legs)	88.2%

Figure 11. The result of the test. (a) Graph of precision; (b) Graph of loss; (c) Confusion matrix on test set.

Figure 12. Sheep’s behaviour recognition results. (a) Rumination; (b) Lying; (c) Other.

Figure 13. Examples of recognition errors.

Figure 14. Improved selection of the sheep’s keypoints. 1. Nose; 2. Lower lip; 3. head; 4. neck; 5. The elbow of the left front leg; 6. The elbow of the right front leg; 7. The knee joint of the left front leg; 8. The knee joint of the right front leg.

Figure 15. Improved spatial temporal graph of sheep skeldom.

Table 6. The test results of Alphapose for posture estimation.

Evaluation indicators	Result/%
Accuracy of the training set	96.9	↑6.7%
Accuracy of the test set	95.6	↑6.1%

Table 7. The average accuracy of Alphapose for different key points.

Location of keypoints	Average accuracy of model/%
Nose	98.2	↑1.7%
Lower lip	98.2	No change
head	96.5	↑3.5%
neck	94.7	↑17.5%
(L/R) Elbow	94.7	↑11.4%
(L/R) Knee joint	93.9	↑1.8%

Figure 16. Confusion matrix on the test set.

Figure 17. Behaviour recognition results of the improved model. (a) Rumination; (b) Lying; (c) Other.

Table 8. The test results of Yolo V8n for multi-target tracking.

Algorithm	Precision	Recall	mAP_0.5	mAP_0.5:0.95
YoloV8n	0.995	0.982	0.991	0.9