Dimensionality reduction is becoming increasingly important in the field of machine learning. In this thesis, we examine several traditional methods of dimensionality reduction, which include random projections, principal component analysis, singular value decomposition, kernel principal component analysis and discrete cosine transform. We also examine several existing applications of random projections (or dimensionality reduction, in general).
In their paper, Random projections in dimensionality reduction: Applications to image and text data (2001), Bingham and Manilla suggest the use of random projections for query matching in a situation where a set of documents, instead of one particular one, were searched for. This suggests another application of random projections, namely to reduce the complexity of the query process. In this thesis, we explain why this approach fails, and suggest three alternative approaches to reducing the complexity of the query process using dimensionality reduction. We also outline query-based dimensionality reduction methods that can be used for image and web data.
In each of the traditional approaches to dimensionality reduction (named above), each attribute in the reduced set is actually a linear combination of the attributes in the original data set. In this thesis, we take the position that true dimensionality reduction is obtained when the set of attributes in the reduced set is a proper subset of the attributes in the original data set, and we discuss seven novel approaches which satisfy this requirement. Using these seven approaches, as well as the RP and PCA approaches, we discuss several ways in which dimensionality reduction can be used for high dimensional clustering and classification.