Dimension Reduction and Classification with Weighted Supervised PCA

Supervised principal component analysis (PCA) is a useful dimension reduction method which is able to induce more discriminative features compared to the conventional PCA in regression or classification problems. This is because supervised PCA tries to extract principal directions by considering not only data variance in input space but also the information of output variable. In this paper, we introduce a simple weighted supervised PCA and apply it to a multi-category classification problem. The weights in the proposed method represent some importance or discriminative power of each input variable for classifying the labels of the output variable. Therefore, we modify directly the correlation matrix using the weights. PCA from this adjusted correlation matrix can produce principal directions that are more useful for classifying the output labels. Some data analyses verify that the proposed method is simple to apply and show good prediction accuracy.