Taxon and trait recognition from digitized herbarium specimens using deep convolutional neural networks

Figures & data

Figure 1. Image processing: herbarium scans as downloaded have been cropped and reduced to standard size in order to prepare them for treatment in deep learning algorithms.

Table 1. Sequence of layers in the ResNet and dimensions of features for each layer. Since the images for herbarium scans are resized to dimensions of 196 by 292, the last average pooling layer had to be modified to cater for the change in image size.

Layer Type	Filter Size / Stride	Output Size
Convolution	7 × 7 / 2	146 × 98 × 64
Max Pool	3 × 3 / 2	72 × 48 × 64
Convolution	[1 × 1, 3 × 3, 1 × 1] × 3	72 × 48 × 256
Average Pool	2 × 2 / 2	36 × 24 × 256
Convolution	[1 × 1, 3 × 3, 1 × 1] × 4	36 × 24 × 512
Average Pool	2 × 2 / 2	18 × 12 × 512
Convolution	[1 × 1, 3 × 3, 1 × 1] × 6	18 × 12 × 1024
Average Pool	2 × 2 / 2	9 × 6 × 1024
Convolution	[1 × 1, 3 × 3, 1 × 1] × 3	9 × 6 × 2048
Average Pool	9 × 6 / 9 × 6	1 × 1 × 2048
Fully Connected (Softmax)	1000 dense	1000

Figure 2. A bottleneck ResNet block.

Figure 3. Species recognition from herbarium scans: performance measures depending on number of herbarium images (the frequency class includes all species with a number of images between × and ×+500): (a) boxplots of accuracy depending on number of test images per species . While the best-recognized species in each frequency class all have values > 0.9, median values increase slightly, but especially minimum values increase strongly with number of images. (b) boxplots of probability values for scans to be assigned to the correct taxon (mean probability of all test images of a taxon).

Table 2. Leaf traits used in this study with OBO ID and number of herbarium scans labelled to have this trait. The last column shows the accuracy of predicted scans for each trait.

trait	state	OBO ID	# scans	accuracy(%)
Leaf - Arrangement	alternate	FLOPO:0001032	120,514	92.98
Leaf - Arrangement	opposite	FLOPO:0000420	34,262	36.8
Leaf - Arrangement	rosulate	FLOPO:0900066	37,459	63
Leaf - Arrangement	whorled	FLOPO:0002264	7550	44.61
Leaf - Form	cordate	FLOPO:0900069	10,378	29.81
Leaf - Form	deeply lobed	FLOPO:0006834	28,900	59.79
Leaf - Form	oblong to linear	FLOPO:0000103	86,644	81
Leaf - Form	orbicular	FLOPO:0017811	8032	23.78
Leaf - Form	ovate or elliptic etc.	FLOPO:0000286	91,954	89.83
Leaf - Margin	entire	FLOPO:0900073	118,297	87.3
Leaf - Margin	not entire (serrate or crenate etc.)	FLOPO:0900074	59,148	72.5
Leaf - Structure	palmately compound	FLOPO:0018499	2268	46.42
Leaf - Structure	pinnately compound	FLOPO:0907004	46,827	68.62
Leaf - Structure	simple	FLOPO:0000693	128,391	97
Leaf - Structure	trifoliolate	FLOPO:0900067	8711	9.1
Leaf Venation	palmate	FLOPO:0900070	17,275	48.11
Leaf Venation	parallel	FLOPO:0900072	40,710	89.57
Leaf Venation	pinnate	FLOPO:0000561	102,663	90.35
Leaf Venation	triplinerve	FLOPO:0900071	7372	21

Figure 4. Accuracy of species recognition depending on the number of images available for training (25%, 50%, 75%, 100%).