Deep learning for real-time social media text classification for situation awareness – using Hurricanes Sandy, Harvey, and Irma as case studies

Figures & data

Figure 1. Overall architecture of the CNN.

Table 1. Dimensions of layers and operations.

Layer	Operator	Output height	Output width	Output depth
Input	–	1	50	–
Embedding		1	50	300
Dropout	Rate = 0.25	1	50	300
Convolution	Stride = 1, zero padding = 0, depth = 250, filter size = 3; activation = ReLU	1	50	250
Global max pooling	–	1	1	250
Fully connected	Output depth = 250; activation = ReLU	1	1	250
Dropout	Rate = 0.25	1	1	250
Fully connected	Output depth = 5; activation = Softmax	1	1	5

Table 2. Social media classification scheme.

	Class	Description	Example
1	Caution and Advice (CA)	Warning given about a related incident	Flooded neighborhoods in Norfolk and its approaching low tide
2	Casualties and Damage (CD)	Information about casualties or infrastructure damage	This tree and power lines are down at the corner of Station Road and Bethlehem Pike in Quakertown
3	Information Sources (IS)	A message from an official news source, media or government	@NYCMayorsOffice: Mayor: All @NYCSchools are closed tomorrow
4	Infrastructure and Resource (IR)	Information about IR that is not reported from an official news source and is not related to donation or aid	Two gas stations at my hotel are out of gas. Had to buy potato bread at the store … Train services are being shut down #HurricaneSandy
5	Donation and Aid (DA)	Goods or services offered or needed by victims	I do not have any money to donate but I have lots of time, where can I help/volunteer in #Hoboken? Who do I call?

Figure 2. Tweet topic over time.

Figure 3. Spatial distribution of Hurricane Sandy tweet topic.

Figure 4. Spatial distribution of Hurricane Harvey tweet topic.

Figure 5. Spatial distribution of Hurricane Irma tweet topic.

Figure 6. Learning curve of the CNN classifier.

Figure 7. Performance scores comparison among CNN, SVM, and LR.

Table 3. Overall accuracy scores for single event experiments.

	(a) Hurricane Sandy			(b) Hurricane Harvey			(c) Hurricane Irma
	CNN	SVM	LR	CNN	SVM	LR	CNN	SVM	LR
Precision	0.81	0.72	0.56	0.81	0.63	0.44	0.80	0.64	0.6
Recall	0.80	0.70	0.53	0.81	0.64	0.51	0.80	0.63	0.53
F1-score	0.80	0.70	0.50	0.81	0.63	0.40	0.79	0.63	0.51

Table 4. Overall accuracy scores for cross-event experiments.

	(a) Train Sandy tweets to classify Harvey tweets			(b) Train Sandy tweets to classify Irma tweets			(c) Train Harvey tweets to classify Irma tweets			(d) Train Sandy and Harvey tweets to classify Irma tweets
	CNN	SVM	LR	CNN	SVM	LR	CNN	SVM	LR	CNN	SVM	LR
Precision	0.79	0.65	0.58	0.79	0.61	0.56	0.76	0.64	0.56	0.83	0.67	0.67
Recall	0.79	0.64	0.50	0.78	0.59	0.46	0.65	0.51	0.37	0.8	0.65	0.65
F1-score	0.78	0.62	0.45	0.78	0.58	0.4	0.66	0.51	0.31	0.8	0.65	0.65

Figure 8. Cross-event classification accuracy.