Model assessment through renormalization group in statistical learning

Abstract

This paper proposes a new method for model assessment based on Renormalization Group (RG). RG is applied to the original data set to obtain the transformed data set with the majority rule to set its labels. The assessment is first performed on the data level without invoking any learning method, and the consistency and non-randomness indices are defined by comparing two data sets to reveal informative content of the data. When the indices indicate informative data, the next assessment is carried out at the model level, and the predictions are compared between two models learnt from the original and transformed data sets, respectively. The model consistency and reliability indices are introduced accordingly. Unlike cross-validation and other standard methods in the literature, the proposed method creates a new data set and data assessment. Besides, it requires only two models and thus less computational burden for model assessment. The proposed method is illustrated with academic and practical examples.