Robust Minimum Density Estimators and Stochastic Resonance for Classification Algorithms

Abstract

The class of Robust Minimum Density Estimators (RMDE’s) are a subset of the Minimum Density Estimators (MDE). Unlike most statistical techniques, RMDE’s treat a sample as a single observation of a random distribution function. The deviance of a small number of observations does not change the general shape of the random distribution function. As the RMDE finds estimators based on the general shape of the random distribution function, the RMDE has a great resistance to outliers. Asymptotic results of the RMDE are presented including consistency and bounds on the variance function.

Once the asymptotic results are presented, the generality of the estimator is presented. Techniques of parameter estimation and regression specific to the RMDE are developed. Simulations are presented to compare the RMDE estimator with standard estimation methods with and without the addition of outliers. The methods are then extended to regression problems which does not differ for linear, nonlinear regression problems or even heteroscedastic errors.

Leveraging the capabilities of the RMDE is the adaptation of Bayesian analysis to create an alternative posterior distribution. By exploiting a density associated with the RMDE estimator, a posterior distribution can be created which is incredibly robust to outliers in datasets. Simulations are used to compare the regular Bayesian posterior distribution with the RMDE posterior distribution. Techniques to implement standard Bayesian methods using the RMDE posterior distribution are described. A discussion of simulating from the posterior distribution, sequential updating of the posterior, and creation of Bayesian credible regions is presented.