An enhanced dynamic hash TRIE algorithm for lexicon search

Abstract

Information retrieval (IR) is essential to enterprise systems along with growing orders, customers and materials. In this article, an enhanced dynamic hash TRIE (eDH-TRIE) algorithm is proposed that can be used in a lexicon search in Chinese, Japanese and Korean (CJK) segmentation and in URL identification. In particular, the eDH-TRIE algorithm is suitable for Unicode retrieval. The Auto-Array algorithm and Hash-Array algorithm are proposed to handle the auxiliary memory allocation; the former changes its size on demand without redundant restructuring, and the latter replaces linked lists with arrays, saving the overhead of memory. Comparative experiments show that the Auto-Array algorithm and Hash-Array algorithm have better spatial performance; they can be used in a multitude of situations. The eDH-TRIE is evaluated for both speed and storage and compared with the naïve DH-TRIE algorithms. The experiments show that the eDH-TRIE algorithm performs better. These algorithms reduce memory overheads and speed up IR.

Keywords:

• machine retrieval
• hash
• TRIE
• uniqueness
• Java
• natural language processing
• intelligent information processing

Notes

1. ‘CArray Classes,’ http://msdn.microsoft.com/en-us/library/4h2f09ct (VS.80).aspx ed: Microsoft Corporation, 2008.

2. ‘Borland C++ Builder Help,’ C++ Builder Version 6.0 (Build 10.157) ed: Borland Software Corporation, 2006.