TopoBERT: a plug and play toponym recognition module harnessing fine-tuned BERT

References

• Abdul-Rahman, M., E. H. Chan, M. S. Wong, V. E. Irekponor, and M. O. Abdul-Rahman. 2021. “A Framework to Simplify pre-Processing Location-Based Social Media big Data for Sustainable Urban Planning and Management.” Cities 109:102986. https://doi.org/10.1016/j.cities.2020.102986.
   Web of Science ®Google Scholar
• Al-Olimat, H., K. Thirunarayan, V. Shalin, and A. Sheth. 2018. “Location Name Extraction from Targeted Text Streams using Gazetteer-based Statistical Language Models.” In Proceedings of the 27th International Conference on Computational Linguistics, 1986–1997. https://aclanthology.org/C18-1169.
   Google Scholar
• Alam, F., U. Qazi, M. Imran, and F. Ofli. 2021. “HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks.” Proceedings of the International AAAI Conference on Web and Social Media 15:933–942. https://doi.org/10.1609/icwsm.v15i1.18116.
   Google Scholar
• Avvenuti, M., S. Cresci, M. N. La Polla, A. Marchetti, and M. Tesconi. 2014. “Earthquake Emergency Management by Social Sensing.” In 2014 IEEE International Conference on Pervasive Computing and Communication Workshops (PERCOM WORKSHOPS), 587–592. Budapest, Hungary: IEEE.
   Google Scholar
• Bahdanau, D., K. Cho, and Y. Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. Preprint, arXiv:1409.0473.
   Google Scholar
• Brown, T., B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, … D. Amodei. 2020. “Language Models are few-Shot Learners.” Advances in Neural Information Processing Systems 33:1877–1901. https://doi.org/10.48550/arXiv.2005.14165.
   Google Scholar
• Cao, R., and R. K. W. Lee. 2020. “Hategan: Adversarial Generative-Based Data Augmentation for Hate Speech Detection.” In Proceedings of the 28th International Conference on Computational Linguistics, 6327–6338.
   Google Scholar
• Cardoso, A. B., B. Martins, and J. Estima. 2022. “A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution.” ISPRS International Journal of Geo-Information 11 (1): Article 1. https://doi.org/10.3390/ijgi11010028.
   Web of Science ®Google Scholar
• Cervone, G., E. Schnebele, N. Waters, M. Moccaldi, and R. Sicignano. 2017. “Using Social Media and Satellite Data for Damage Assessment in Urban Areas During Emergencies.” In Seeing Cities Through big Data, edited by Piyushimita (Vonu) Thakuriah, Nebiyou Tilahun, and Moira Zellner, 443–457. Tilahun: Springer.
   Google Scholar
• Derczynski, L., E. Nichols, M. van Erp, and N. Limsopatham. 2017. “Results of the WNUT2017 Shared Task on Novel and Emerging Entity Recognition.” In Proceedings of the 3rd Workshop on Noisy User-generated Text, 140–147.
   Google Scholar
• Devlin, J., M. W. Chang, K. Lee, and K. Toutanova. 2018. Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding. Preprint, arXiv:1810.04805.
   Google Scholar
• Dutt, R., K. Hiware, A. Ghosh, and R. Bhaskaran. 2018. “Savitr: A System for Real-Time Location Extraction from Microblogs During Emergencies.” In Companion Proceedings of the Web Conference 2018, 1643–1649. https://doi.org/10.1145/3184558.3191623.
   Google Scholar
• Feng, S. Y., V. Gangal, J. Wei, S. Chandar, S. Vosoughi, T. Mitamura, and E. Hovy. 2021. A Survey of Data Augmentation Approaches for NLP. Preprint, arXiv:2105.03075.
   Google Scholar
• Fernández-Martínez, N. J., and C. Periñán-Pascual. 2021. “nLORE: A Linguistically Rich Deep-Learning System for Locative-Reference Extraction in Tweets.” In Intelligent environments 2021: Workshop proceedings of the 17th international conference on intelligent environments. Vol. 29, edited by Engie Bashir and Mitja Luštrek, 243. Dubai: IOS Press. https://doi.org/10.3233/AISE210103.
   Google Scholar
• Finkel, J. R., T. Grenager, and C. D. Manning. 2005. “Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling.” In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), 363–370.
   Google Scholar
• Giridhar, P., T. Abdelzaher, J. George, and L. Kaplan. 2015. “On Quality of Event Localization from Social Network Feeds.” In 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), 75–80. St. Louis, MO: IEEE.
   Google Scholar
• Hochreiter, S., and J. Schmidhuber. 1997. “Long Short-Term Memory.” Neural Computation 9 (8): 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.
   PubMed Web of Science ®Google Scholar
• Hu, X., H. S. Al-Olimat, J. Kersten, M. Wiegmann, F. Klan, Y. Sun, and H. Fan. 2022a. “GazPNE: Annotation-Free Deep Learning for Place Name Extraction from Microblogs Leveraging Gazetteer and Synthetic Data by Rules.” International Journal of Geographical Information Science 36 (2): 310–337. https://doi.org/10.1080/13658816.2021.1947507.
   Web of Science ®Google Scholar
• Hu, X., Z. Zhou, H. Li, Y. Hu, F. Gu, J. Kersten, H. Fan, and F. Klan. 2022b. Location Reference Recognition from Texts: A Survey and Comparison. arXiv:2207.01683. https://doi.org/10.48550/arXiv.2207.01683
   Google Scholar
• Hu, X., Z. Zhou, Y. Sun, J. Kersten, F. Klan, H. Fan, and M. Wiegmann. 2022c. “GazPNE2: A General Place Name Extractor for Microblogs Fusing Gazetteers and Pretrained Transformer Models.” IEEE Internet of Things Journal 9 (17): 16259–16271. https://doi.org/10.1109/JIOT.2022.3150967.
   Web of Science ®Google Scholar
• Huang, X., Z. Li, Y. Jiang, X. Li, and D. Porter. 2020. “Twitter Reveals Human Mobility Dynamics During the COVID-19 Pandemic.” PloS one 15 (11): e0241957. https://doi.org/10.1371/journal.pone.0241957.
   PubMed Web of Science ®Google Scholar
• Kiranyaz, S., O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj, and D. J. Inman. 2021. “1D Convolutional Neural Networks and Applications: A Survey.” Mechanical Systems and Signal Processing 151:107398. https://doi.org/10.1016/j.ymssp.2020.107398.
   Web of Science ®Google Scholar
• Labusch, K., P. Kulturbesitz, C. Neudecker, and D. Zellhöfer. 2019. “BERT for named entity recognition in contemporary and historical German.” In Proceedings of the 15th Conference on Natural Language Processing, Erlangen, Germany, 8–11.
   Google Scholar
• Lan, Z., M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. arXiv:1909.11942. https://doi.org/10.48550/arXiv.1909.11942
   Google Scholar
• Lee, J. Y., and F. Dernoncourt. 2016. Sequential Short-text Classification with Recurrent and Convolutional Neural Networks. Preprint, arXiv:1603.03827.
   Google Scholar
• Li, W., C.-Y. Hsu, and M. Hu. 2021. “Tobler’s First Law in GeoAI: A Spatially Explicit Deep Learning Model for Terrain Feature Detection Under Weak Supervision.” Annals of the American Association of Geographers 111 (7): 1887–1905. https://doi.org/10.1080/24694452.2021.1877527.
   Web of Science ®Google Scholar
• Lin, B., L. Zou, N. Duffield, A. Mostafavi, H. Cai, B. Zhou, J. Tao, M. Yang, D. Mandal, and J. Abedin. 2022. “Revealing the Linguistic and Geographical Disparities of Public Awareness to Covid-19 Outbreak Through Social Media.” International Journal of Digital Earth 15 (1): 868–889. https://doi.org/10.1080/17538947.2022.2070677.
   Web of Science ®Google Scholar
• Liu, Z., K. Janowicz, L. Cai, R. Zhu, G. Mai, and M. Shi. 2022. “Geoparsing: Solved or Biased? An Evaluation of Geographic Biases in Geoparsing.” AGILE: GIScience Series 3:1–13. https://doi.org/10.5194/agile-giss-3-9-2022.
   Google Scholar
• Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692. https://doi.org/10.48550/arXiv.1907.11692.
   Google Scholar
• Luoma, J., and S. Pyysalo. 2020. Exploring Cross-sentence Contexts for Named Entity Recognition with BERT. arXiv:2006.01563. https://doi.org/10.48550/arXiv.2006.01563.
   Google Scholar
• Ma, K., Y. Tan, Z. Xie, Q. Qiu, and S. Chen. 2022. “Chinese toponym recognition with variant neural structures from social media messages based on BERT methods.” Journal of Geographical Systems 24 (2): 143–169. https://doi.org/10.1007/s10109-022-00375-9.
   Web of Science ®Google Scholar
• Middleton, S. E., G. Kordopatis-Zilos, S. Papadopoulos, and Y. Kompatsiaris. 2018. “Location Extraction from Social Media: Geoparsing, Location Disambiguation, and Geotagging.” ACM Transactions on Information Systems 36 (4): 1–27. https://doi.org/10.1145/3202662.
   Web of Science ®Google Scholar
• Mikolov, T., I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. “Distributed Representations of Words and Phrases and Their Compositionality.” Advances in Neural Information Processing Systems 26. https://doi.org/10.48550/arXiv.1310.4546.
   Google Scholar
• Milusheva, S., R. Marty, G. Bedoya, S. Williams, E. Resor, and A. Legovini. 2021. “Applying Machine Learning and Geolocation Techniques to Social Media Data (Twitter) to Develop a Resource for Urban Planning.” PloS One 16 (2): e0244317. https://doi.org/10.1371/journal.pone.0244317.
   PubMed Web of Science ®Google Scholar
• Peters, M., M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018. “Deep Contextualized Word Representations.” In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2227–2237. New Orleans, Louisiana: Association for Computational Linguistics.
   Google Scholar
• Pushp, P. K., and M. M. Srivastava. 2017. Train Once, Test Anywhere: Zero-shot Learning for Text Classification. Preprint, arXiv:1712.05972.
   Google Scholar
• Qi, P., Y. Zhang, Y. Zhang, J. Bolton, and C. D. Manning. 2020. Stanza: A Python Natural Language Processing Toolkit for Many Human Languages. Preprint, arXiv:2003.07082.
   Google Scholar
• Quezada, M., V. Peña-Araya, and B. Poblete. 2015. “Location-Aware Model for News Events in Social Media.” In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, 935–938.
   Google Scholar
• Radford, A., J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. 2019. “Language Models are Unsupervised Multitask Learners.” OpenAI Blog 1 (8): 9. https://doi.org/10.48550/arXiv.2005.14165.
   Google Scholar
• Ritter, A., S. Clark, Mausam, and O. Etzioni. 2011. “Named Entity Recognition in Tweets: An Experimental Study.” In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 1524–1534. https://aclanthology.org/D11-1141.
   Google Scholar
• Rolnick, D., A. Veit, S. Belongie, and N. Shavit. 2017. Deep Learning Is Robust to Massive Label Noise. Preprint, arXiv:1705.10694.
   Google Scholar
• Sang, Erik F., Tjong Kim, and F. Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. https://doi.org/10.48550/arXiv.cs/0306050.
   Google Scholar
• Sanh, V., L. Debut, J. Chaumond, and T. Wolf. 2020. DistilBERT, A Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter. arXiv:1910.01108. https://doi.org/10.48550/arXiv.1910.01108.
   Google Scholar
• Shorten, C., T. M. Khoshgoftaar, and B. Furht. 2021. “Text Data Augmentation for Deep Learning.” Journal of Big Data 8 (1): 101. https://doi.org/10.1186/s40537-021-00492-0.
   PubMedGoogle Scholar
• Shu, K., X. Zhou, S. Wang, R. Zafarani, and H. Liu. 2019. “The Role of User Profiles for Fake News Detection.” In Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining, 436–439.
   Google Scholar
• Souza, F., R. Nogueira, and R. Lotufo. 2020. Portuguese Named Entity Recognition using BERT-CRF. arXiv:1909.10649. https://doi.org/10.48550/arXiv.1909.10649.
   Google Scholar
• Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, … I. Polosukhin. 2017. “Attention is all you Need.” Advances in Neural Information Processing Systems 30. https://doi.org/10.48550/arXiv.1706.03762.
   Google Scholar
• Wallgrün, J. O., M. Karimzadeh, A. M. MacEachren, and S. Pezanowski. 2018. “GeoCorpora: Building a Corpus to Test and Train Microblog Geoparsers.” International Journal of Geographical Information Science 32 (1): 1–29. https://doi.org/10.1080/13658816.2017.1368523.
   Web of Science ®Google Scholar
• Wang, J., Y. Hu, and K. Joseph. 2020. “NeuroTPR: A Neuro-net Toponym Recognition Model for Extracting Locations from Social Media Messages.” Transactions in GIS 24 (3): 719–735. https://doi.org/10.1111/tgis.12627.
   Web of Science ®Google Scholar
• Wang, S., X. Zhang, P. Ye, and M. Du. 2018. “Deep Belief Networks Based Toponym Recognition for Chinese Text.” ISPRS International Journal of Geo-Information 7 (6): Article 6. https://doi.org/10.3390/ijgi7060217.
   Web of Science ®Google Scholar
• Wei, J., M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, … Q. V. Le. 2021. Finetuned Language Models Are Zero-shot Learners. Preprint, arXiv:2109.01652.
   Google Scholar
• Wortsman, M., G. Ilharco, J. W. Kim, M. Li, S. Kornblith, R. Roelofs, R. G. Lopes, et al. 2021. Robust Fine-Tuning of Zero-Shot Models. https://arxiv.org/pdf/2109.01903.
   Google Scholar
• Zhao, J., X. Mao, and L. Chen. 2019. “Speech Emotion Recognition Using Deep 1D & 2D CNN LSTM Networks.” Biomedical Signal Processing and Control 47:312–323. https://doi.org/10.1016/j.bspc.2018.08.035.
   Web of Science ®Google Scholar
• Zhou, Y., and J. Luo. 2012. “Geo-location Inference on News Articles via Multimodal pLSA.” In MM’12: The Proceedings of the 20th ACM International Conference on Multimedia, co-Located with ACM Multimedia 2012, October 29-November 2, 2012, Nara, Japan, edited by N. Babaguchi, K. Aizawa, J. Smith, S. Satoh, T. Plagemann, X.-S. Hua, and R. Yan, 741. Nara, Japan: Association for Computer Machinery. https://doi.org/10.1145/2393347.2396301.
   Google Scholar
• Zhou, B., L. Zou, A. Mostafavi, B. Lin, M. Yang, N. Gharaibeh, H. Cai, J. Abedin, and D. Mandal. 2022. “VictimFinder: Harvesting Rescue Requests in Disaster Response from Social Media with BERT.” Computers, Environment and Urban Systems 95:101824. https://doi.org/10.1016/j.compenvurbsys.2022.101824.
   Web of Science ®Google Scholar
• Zou, L., N. S. N. Lam, H. Cai, and Y. Qiang. 2018. “Mining Twitter Data for Improved Understanding of Disaster Resilience.” Annals of the American Association of Geographers 108 (5): 1422–1441. https://doi.org/10.1080/24694452.2017.1421897.
   Web of Science ®Google Scholar
• Zou, L., N. S. N. Lam, S. Shams, H. Cai, M. A. Meyer, S. Yang, K. Lee, S.-J. Park, and M. A. Reams. 2019. “Social and Geographical Disparities in Twitter use During Hurricane Harvey.” International Journal of Digital Earth 12 (11): 1300–1318. https://doi.org/10.1080/17538947.2018.1545878.
   Web of Science ®Google Scholar
• Zou, L., D. Liao, N. S. Lam, M. Meyer, N. G. Gharaibeh, H. Cai, B. Zhou, and D. Li. 2023. “Social Media for Emergency Rescue: An Analysis of Rescue Requests on Twitter During Hurricane Harvey.” International Journal of Disaster Risk Reduction 85:103513. https://doi.org/10.1016/j.ijdrr.2022.103513.
   Web of Science ®Google Scholar