A data compression scheme for Chinese text files using Huffman coding and a two-level dictionary

Abstract

This paper presents a data compression scheme for Chinese text files. Due to the skewness of the distribution of Chinese ideograms, the Huffman coding method is adopted. By storing the frequencies of the encoding symbols rather than their Huffman codes in a dictionary, applying differential coding where it saves space, and structuring the dictionary in the Huffman coding scheme into a two-level dictionary structure, the algorithm produces significant improvement on the compression results. The proposed method is evaluated by comparing its performance with three well-known compression algorithms. This algorithm should also be applicable to other ideogram-based or oriental-language texts. Also, it has the potential to reduce the dictionary size in a bigram- or trigram-based semi-adaptive compression scheme for English texts. © 1995.