Multi-Paragraph Machine Reading Comprehension with Hybrid Reader over Tables and Text

Figures & data

Figure 1. Example of a paragraph from a full Wikipedia document with a title, text paragraph, and a table.

Figure 2. Architecture of (a) pipeline models with a separated reader, (b) pipeline models with hybrid reader.

Figure 3. Example of generating negative and positive paragraph table from the original paragraph.

Figure 4. Architecture of a hybrid reader with projection.

Figure 5. Architecture of a hybrid reader with adapter.

Figure 6. Architecture of adapter layer; row-wise, direct, column-wise projection is concatenated and passed to adapter transformer layer.

Figure 7. The architecture of hybrid reader model with projection using long sequence transformer.

Figure 8. The architecture of adapter layer for tables.

Table 1. Result comparison between separated model and hybrid model with and without paragraph ranking.

		With paragraph ranking			oracle
		All	Text	Table	All	Text	Table
Model		EM/F1	EM/F1	EM/F1	EM/F1	EM/F1	EM/F1
Separated Model	Vanilla	58.32/78.95		38.36/61.59	65.04/84.79		66.68/77.15
	TAPAS	60.03/80.25	63.60/83.52	46.29/67.59	66.80/86.19	78.25/90.00	77.50/85.70
	Projection	59.67/79.98		44.85/66.62	66.66/86.04		76.63/84.78
	Adapter	59.21/79.53		42.45/64.33	66.82/86.26		77.62/86.16
Hybrid Model	Vanilla	56.90/77.93	63.00/83.05	33.75/58.51	64.81/84.71	77.80/89.76	67.11/77.61
	TAPAS	57.18/77.98	63.64/83.34	32.69/57.68	67.32/83.90	78.87/86.41	78.12/86.41
	Projection	59.64/79.82	63.45/83.35	45.19/66.41	66.80/83.40	78.37/85.95	77.00/85.27
	Adapter	59.87/80.13	63.71/83.49	45.28/67.36	67.88/86.99	79.94/91.15	77.13/85.91

The values in bold indicate the highest performance achieved in each category.

Table 2. Result of the new architecture of BigBird model with our modifications.

Model	ALL	Table	Text
(BigBird)	EM/F1	EM/F1	EM/F1
Vanilla	58.15/80.60	51.20/64.01	74.31/86.87
Projection Layer	58.80/81.01	59.89/70.58	73.17/85.89
Adapter	58.55/80.99	54.83/67.33	74.02/86.65

The values in bold indicate the highest performance achieved in each category.

Table 3. Results for experiments using the English MRC dataset, SQuAD 1.1, WikiSQL, and the NQ-Tables on hybrid reader model.

	Datasets
	SQuAD v1.1	WikiSQL	NQ-Tables
Model	EM/F1	Accuracy	EM/F1
Vanilla	80.84/93.05	56.78	69.59/81.12
Tapas	46.34/71.78	81.79	54.94/73.65
Projection	78.27/91.65	80.73	76.16/85.59
Adapter	80.19/92.77	80.65	74.45/84.49

The values in bold indicate the highest performance achieved in each category.

Table 4. Result for experiments using TAT-QA dataset.

	Table	Table-Text	Text	Total
Model	EM/F1	EM/F1	EM/F1	EM/F1
TAGOP (Zhu et al. 2021)	44.62/46.81	54.58/59.05	48.01/72.56	48.70/58.81
Tapas	56.47/57.83	59.15/63.58	20.85/44.43	49.18/56.66
Projection	55.37/57.17	61.90/66.51	47.75/73.05	55.78/63.91
Adapter	57.43/59.33	63.00/67.11	45.64/71.20	56.57/64.59

The values in bold indicate the highest performance achieved in each category.

Figure 9. The performance comparison of each vanilla, projection, TAPAS, and adapter.

Zhu, F., W. Lei, Y. Huang, C. Wang, S. Zhang, J. Lv, F. Feng, and T.-S. Chua. 2021. TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance. arXiv Preprint arXiv:150302531 2 Abs/2105.07624. https://api.semanticscholar.org/CorpusID:234741852.

 Google Scholar

Data availability statement

Source codes and dataset used in this paper can be accessed in our github page.⁵