Cluster weighted error rate control on datasets with multi-level structures

Abstract

Modern technology has resulted in hypothesis testing on massive datasets. When the fraction of signals is small, useful signals are easily missed when applying the classical family-wise error rate criterion. Benjamini and Hochberg proposed a more lenient false discovery rate (FDR) error controlling criterion and showed how Simes procedure can be calibrated to control FDR at a given level. We propose a multi-level BH procedure for large sample testing that utilizes multi-level structure of the dataset. We prove that the procedure provides cluster weighted FDR control and show that it has better signal detection properties when the false null hypotheses are clustered. We show in simulation studies that a refinement of the procedure using false null proportion estimation improves performance. A second method that we apply uses a scoring device that is robust against model deviations. Renewal and boundary-crossing theories are used to compute exceedance probabilities of the scores.