A Case Study on Context-Centered Lexicon Construction

Abstract

An approach is proposed for extracting Chinese words or phrases to allow construction of a lexical database for processing the language. The lack of explicit word boundaries has been an obstacle to reliable collection of unknown Chinese lexical items. A context-centered template pattern matching strategy involves left & right strings of predefined elements surrounding a token string of fixed or variable length. An algorithm was tried using a template where four-gram tokens between a fixed string & a comma were extracted; the 205 matches from a 4,000,000-character corpus are analyzed. The algorithm is judged productive, & application of syntactic constraints would make it even more effective.