Spectral Regression: A Regression Framework for Efficient Regularized Subspace Learning

[摘要] Spectral methods have recently emerged as a powerful tool fordimensionality reduction and manifold learning. These methods useinformation contained in the eigenvectors of a data affinity (\ie,item-item similarity) matrix to reveal the low dimensional structurein the high dimensional data. The most popular manifold learning algorithmsinclude Locally Linear Embedding, ISOMAP, and Laplacian Eigenmap.However, these algorithms only provide the embedding results oftraining samples. There are many extensions of these approacheswhich try to solve the out-of-sample extension problem by seeking anembedding function in reproducing kernel Hilbert space. However, adisadvantage of all these approaches is that their computationsusually involve eigen-decomposition of dense matrices which isexpensive in both time and memory. In this thesis, we introduce anovel dimensionality reduction framework, called {\bf SpectralRegression} (SR). SR casts the problem of learning an embeddingfunction into a regression framework, which avoidseigen-decomposition of dense matrices. Also, with the regressionas a building block, different kinds of regularizers can be naturallyincorporated into our framework which makes it more flexible. SR canbe performed in supervised, unsupervised and semi-supervisedsituation. It can make efficient use of both labeled and unlabeledpoints to discover the intrinsic discriminant structure in the data.We have applied our algorithms to several real world applications,e.g. face analysis, document representation and content-based imageretrieval.

[关键词] Machine Learning