Dimensionality Reduction and Feature Selection using a Mixed-norm Penalty Function
No Thumbnail Available
Files
Date
2006-03-13
Authors
Journal Title
Series/Report No.
Journal ISSN
Volume Title
Publisher
Abstract
Dimensionality reduction, which is the process of mapping high-dimension patterns to lower dimension subspaces, is a key issues in enhancing the processing efficiency of high dimensional data such as hyperspectral images. Dimensionality reduction has been widely discussed in the areas of data mining, image processing, pattern recognition, etc. Because in most situations, many of the dimensions are redundant or unnecessary for the tasks of interest, removing those dimensionality will produce more efficient computation while maintaining the original performance. Dimensionality reduction also reduces the measurement and storage requirements, reduces training and utilization times and it defies the curse of dimensionality to improve classification performance.
Feature selection, the process of constructing and selecting the subsets of features that are useful to build a good predictor is of interest for many years. Before Kohavi and John published a special issue on feature selection in 1997, usually no more than 40 features are studied. Ever since then, people started looking at problems with hundreds to tens of thousands of features. Like dimensionality reduction, feature selection reduces the measurement and storage requirements, reduces training and utilization times, and it facilitates data visualization and data understanding.
In this work, popular methods for dimensionality reduction and feature selection, such as vector space method, penalty function and support vector machine (SVM) are reviewed and compared. A novel penalty function called the mixed-norm penalty function is proposed. It minimizes the 1-norm of the weight vector while keeping the 2-norm constant. Both dimensionality reduction and feature selection in this work are realized via artificial neural networks (ANNs). Together with Bi-level optimization (BLO) technique, the mixed-norm penalty establishes great performance for both the synthetic data and hyperspectral images.
Description
Keywords
dimensionality reduction, feature selection, neural networks, machine learning, penalty function, mixed-norm penalty function
Citation
Degree
PhD
Discipline
Electrical Engineering
