A Brief Theoretical Overview of Random Forests

Random Forests is an important ensemble learning method and it is widely used in data classification and nonparametric regression. In this paper, we review three main theoretical issues of random forests, i.e., the convergence theorem, the generalization error bound and the out-of-bag estimation. In the end, we present an improved Random Forests algorithm, which uses a feature weighting sampling method to sample a subset of features at each node in growing trees. The new method is suitable to solve classification problems of very high dimensional data.