A New Adapter Tuning of Large Language Model for Chinese Medical Named Entity Recognition

References

• Ashok, D., and Z. C. Lipton. 2023. PromptNER: Prompting for named entity recognition. ArXiv abs/2305.15444:1–22. https://api.semanticscholar.org/CorpusID:258887456.
   Google Scholar
• Ayesha, J., N. Pipil, S. Santra, S. Mondal, J. Kumar Behera, H. Mondal, N. Pipil Sr, S. Santra Sr, and J. K. Behera IV. 2023. The capability of ChatGPT in predicting and explaining common drug-drug interactions. Cureus 15:3. doi:10.7759/cureus.36272.
   Web of Science ®Google Scholar
• Chen, J., A. Zhang, X. Shi, M. Li, A. Smola, and D. Yang. 2023. Parameter-efficient fine-tuning design spaces. ArXiv abs/2301.01821 (16): 1–18. https://api.semanticscholar.org/CorpusID:255440621.
   Google Scholar
• Du, Z., Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, and J. Tang. 2022. GLM: General language Model pretraining with autoregressive blank infilling. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland, 320–35.
   Google Scholar
• Geva, M., R. Schuster, J. Berant, and O. Levy. 2021. Transformer feed-forward layers are key-value memories. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic, 5484–95.
   Google Scholar
• He, J., C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig. 2022. Towards a unified view of parameter-efficient transfer learning. International Conference on Learning Representations. https://openreview.net/forum?id=0RDcd5Axok.
   Google Scholar
• Houlsby, N., A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly. 2019. Parameter-efficient transfer learning for NLP. International Conference on Machine Learning, Long Beach, California, USA, 2790–99.
   Google Scholar
• Hu, Z., Y. Lan, L. Wang, W. Xu, E.-P. Lim, R. Ka-Wei Lee, L. Bing, and S. Poria. 2023. LLM-Adapters: An adapter family for parameter-efficient fine-tuning of large language models. ArXiv abs/2304.01933:1–21. https://api.semanticscholar.org/CorpusID:257921386.
   Google Scholar
• Huang, X., K. Han, Y. Yang, D. Bao, Q. Tao, Z. Chai, and Q. Zhu. 2024. GNNs as adapters for LLMs on text-attributed graphs. The Web Conference 2024. https://openreview.net/forum?id=AFJYWMkVCh.
   Google Scholar
• Keloth, V. K., Y. Hu, Q. Xie, X. Peng, Y. Wang, A. Zheng, M. Selek, K. Raja, C. H. Wei, Q. Jin, et al. 2024. Advancing entity recognition in biomedicine via instruction tuning of large language models. Bioinformatics 40 (4):btae163. doi:10.1093/bioinformatics/btae163.
   PubMed Web of Science ®Google Scholar
• Kenton, J. D. M.-W. C., and L. Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT, Minneapolis, MN, USA, 4171–86.
   Google Scholar
• Li, L., Y. Dai, D. Tang, X. Qiu, Z. Xu, and S. Shi. 2023. Markbert: Marking word boundaries improves Chinese bert. CCF International Conference on Natural Language Processing and Chinese Computing, Foshan, China, 325–36.
   Google Scholar
• Li, X., H. Yan, X. Qiu, and X.-J. Huang. 2020. FLAT: Chinese NER using flat-lattice transformer. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, Washington, USA, 6836–42.
   Google Scholar
• Li, X., H. Zhang, and X.-H. Zhou. 2020. Chinese clinical named entity recognition with variant neural structures based on BERT methods. Journal of Biomedical Informatics 107:103422. doi:10.1016/j.jbi.2020.103422.
   PubMed Web of Science ®Google Scholar
• Li, Y., J. Li, J. He, and C. Tao. 2024. AE-GPT: Using large language models to extract adverse events from surveillance reports-A use case with influenza vaccine adverse events. PLOS ONE 19 (3):1–16. doi:10.1371/journal.pone.0300919.
   Web of Science ®Google Scholar
• Li, Y., Z. Li, K. Zhang, R. Dan, S. Jiang, and Y. Zhang. 2023. ChatDoctor: A medical chat Model fine-tuned on a large language Model Meta-ai (LLaMA) using medical domain knowledge. Cureus 15 (6). doi:10.7759/cureus.40895.
   Web of Science ®Google Scholar
• Lialin, V., V. Deshpande, and A. Rumshisky. 2023. Scaling down to scale up: A guide to parameter-efficient fine-tuning. ArXiv abs/2303.15647:1–21. https://api.semanticscholar.org/CorpusID:257771591.
   Google Scholar
• Liu, T., J. Gao, W. Ni, and Q. Zeng. 2023. A multi-granularity word fusion method for Chinese NER. Applied Sciences 13 (5):2789. doi:10.3390/app13052789.
   Google Scholar
• Luo, C., Y. Shen, Z. Zhu, Q. Zheng, Z. Yu, and C. Yao. 2024. LayoutLLM: Layout instruction tuning with large language models for document understanding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Washington, USA, 15630–40.
   Google Scholar
• Luo, L., J. Ning, Y. Zhao, Z. Wang, Z. Ding, P. Chen, W. Fu, Q. Han, G. Xu, Y. Qiu, et al. 2024. Taiyi: A bilingual fine-tuned large language model for diverse biomedical tasks. Journal of the American Medical Informatics Association ocae37. doi:10.1093/jamia/ocae037.
   Google Scholar
• Peng, H., Z. Zhang, D. Liu, and X. Qin. 2023. Chinese medical entity recognition based on the dual-branch TENER model. BMC Medical Informatics & Decision Making 23 (1):136. doi:10.1186/s12911-023-02243-y.
   PubMed Web of Science ®Google Scholar
• Pfeiffer, J., A. Kamath, A. Rücklé, K. Cho, and I. Gurevych. 2021. AdapterFusion: Non-destructive task composition for transfer learning. 16th Conference of the European Chapter of the Associationfor Computational Linguistics, EACL 2021, Online Conference, 487–503.
   Google Scholar
• Qin, C., A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang. 2023. Is ChatGPT a general-purpose natural language processing task solver? ArXiv abs/2302.06476:1–47. https://api.semanticscholar.org/CorpusID:256827430.
   Google Scholar
• Qu, L., S. Wu, H. Fei, L. Nie, and T.-S. Chua. 2023. Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, Canada, 643–54.
   Google Scholar
• Shah, A., S. Thapa, A. Jain, and L. Huang. 2023. Adept: Adapter-based efficient prompt tuning approach for language models. Proceedings of The Fourth Workshop on Simple and Efficient Natural Language Processing (SustaiNLP), Toronto, Canada, 121–28.
   Google Scholar
• Singh, S., A. Djalilian, and M. Javed Ali. 2023. ChatGPT and ophthalmology: Exploring its potential with discharge summaries and operative notes. Seminars in Ophthalmology 38 (5):503–07. doi:10.1080/08820538.2023.2209166.
   PubMed Web of Science ®Google Scholar
• Tian, X., X. Bu, and L. He. 2023. Multi-task learning with helpful word selection for lexicon-enhanced Chinese NER. Applied Intelligence 53 (16):19028–43. doi:10.1007/s10489-023-04464-0.
   Web of Science ®Google Scholar
• Touvron, H., T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. 2023. Llama: Open and efficient foundation language models. ArXiv abs/2302.13971:1–27. https://api.semanticscholar.org/CorpusID:257219404.
   Google Scholar
• Villa, L., C.-P. David, S.-M. Adrián, C. D. Cosmin, and H. Ramón. 2023. Conversational agent development through large language models: Approach with GPT. Proceedings of the 15th International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023), Riviera Maya, Mexico, ed. J. Bravo and G. Urzáiz Cham, 286–97. Springer Nature Switzerland.
   Google Scholar
• Wang, S., X. Sun, X. Li, R. Ouyang, F. Wu, T. Zhang, J. Li, and G. Wang. 2023. Gpt-ner: Named entity recognition via large language models. ArXiv abs/2304.10428:1–21. https://api.semanticscholar.org/CorpusID:258236561.
   Google Scholar
• Wang, Y., B. Yu, Y. Zhang, T. Liu, H. Zhu, and L. Sun. 2020. Tplinker: Single-stage joint extraction of entities and relations through token pair linking. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain, 1572–82.
   Google Scholar
• Wei, J., X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (NeurIPS 2022), New Orleans, Louisiana, USA 35:24824–37.
   Google Scholar
• Wei, Z., J. Su, Y. Wang, Y. Tian, and Y. Chang. 2020. A novel cascade binary tagging framework for relational triple extraction. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online Conference, 1476–88.
   Google Scholar
• Wu, T., S. He, J. Liu, S. Sun, K. Liu, Q.-L. Han, and Y. Tang. 2023. A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica 10 (5):1122–36. doi:10.1109/JAS.2023.123618.
   Google Scholar
• Xia, C., C. Zhang, T. Yang, Y. Li, N. Du, X. Wu, W. Fan, F. Ma, and P. Yu. 2020. Multi-grained named entity recognition. 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019, Florence, Italy, 1430–40.
   Google Scholar
• Xiang, Y., W. Liu, J. Guo, and L. Zhang. 2023. Local and global character representation enhanced model for Chinese medical named entity recognition. Journal of Intelligent & Fuzzy Systems 45 (3):3779–90. doi:10.3233/JIFS-231554.
   Web of Science ®Google Scholar
• Yang, N., S. Hang Pun, M. I. Vai, Y. Yang, and Q. Miao. 2022. A unified knowledge extraction method based on BERT and Handshaking tagging scheme. Applied Sciences 12 (13):6543. doi:10.3390/app12136543.
   Google Scholar
• Zhang, N., M. Chen, Z. Bi, X. Liang, L. Li, X. Shang, K. Yin, C. Tan, J. Xu, F. Huang, et al. 2022. CBLUE: A Chinese biomedical language understanding evaluation benchmark. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland, ed. S. Muresan, P. Nakov, and A. Villavicencio, 7888–915. Association for Computational Linguistics. doi:10.18653/v1/2022.acl-long.544.
   Google Scholar
• Zhang, S., S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al. 2022. Opt: Open pre-trained transformer language models. ArXiv abs/2205.01068:1–30. https://api.semanticscholar.org/CorpusID:248496292.
   Google Scholar
• Zhang, X., and J. Wu. 2024. Dissecting learning and forgetting in language model finetuning. The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=tmsqb6WpLz.
   Google Scholar
• Zhang, Z., T. Chuanqi, X. Haiyang, W. Chengyu, H. Jun, and H. Songfang. 2023. Towards adaptive prefix tuning for parameter-efficient language Model fine-tuning. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 2023, ed. A. Rogers, J. L. Boyd-Graber, and N. Okazaki, 1239–48, Toronto, Canada, July 9-14, 2023.
   Google Scholar
• Zhao, Y., W. Zhang, H. Wang, K. Kawaguchi, and L. Bing. 2024. AdaMergeX: Cross-lingual transfer with large language models via adaptive adapter merging. ArXiv abs/2402.18913:1–15. https://api.semanticscholar.org/CorpusID:268063729.
   Google Scholar
• Zhou, W., S. Zhang, Y. Gu, M. Chen, and H. Poon. 2024. UniversalNER: Targeted distillation from large language models for open named entity recognition. The Twelfth International Conference on Learning Representations, Vienna, Austria. https://openreview.net/forum?id=r65xfUb76p.
   Google Scholar
• Zou, B., C. Yang, Y. Qiao, C. Quan, and Y. Zhao. 2024. LLaMA-excitor: General instruction tuning via indirect feature interaction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, Washington, USA, 14089–99, June.
   Google Scholar