Using a Dependency Structure Parser without any Grammar Formalism to Analyse a Software Manual Corpus

Abstract

DESPAR is discussed, a hybrid approach to parsing that is based on an enhanced hidden Markov model & relies on no grammar formalism. The approach is corpus-based & statistical. Implementation builds on the insight of M. Liberman (1993) that dependency parsing is a kind of tagging for parts of speech. DESPAR takes tagged sentences as input, seeks candidate governors for each part of speech, eliminates invalid candidates for governor, & returns a likely dependency structure as output. The enhanced hidden Markov model operates with bigrams & uses a dynamic context algorithm & dependency axioms. The statistical part-of-speech tagger is based on the Brown & Wall Street Journal corpora, totaling almost 180,000 sentences. A module to handle unknown words effectively gives the parser unlimited vocabulary. A divide-and-conquer module simplifies complex sentences before parsing. DESPAR was applied to the software manual corpus in two stages; with original grammar & vocabulary & with added vocabulary. Preprocessing consisted of tokenization. it is concluded that no grammar formalism is required to analyze the dependency structure of a sentence. The performance of the parser could be improved by providing more corpora & by refining the enhanced hidden Markov model to use trigram transitions. L. Lagerquist.