Recent Advances in Discriminant Analysis for High-dimensional DataClassification

[摘要] There are serious challenges posed by high-dimensional data sets.With the arrival of new technologies, high-throughput modeling isbecoming a norm in many disciplines such as statistical genetics,epidemiology, astronomy, high energy physics, and ecology. Highdimensional data have emerged from various sources such as digitalimages, documents, next-gen sequencing, mass spectrometry,metabolomics, microarray, proteomics, online videos and web pages.One area with a growing need for new statistical methods and theory forhigh-dimensional data is the classification of subgroups. For example,cancer classification has primarily been based on histopathologicalappearance of tumor. However, patients with similar tumor appearancecan have different prognosis and response to treatment. The traditionalway to classify cancer by pathological review may cause biased resultsand misclassify the tumor subtypes for patients. The availability ofmicroarray data allows simultaneous measures of thousands of genes.These high-dimensional data have become a standard tool for biomedicalstudies and are now commonly collected from patients in clinicaltrials. The identification of informative genes may result in potentialmolecular markers for tumor class prediction. Correct classificationscan help practitioners identify the right treatment for patients. Dueto the cost and/or experimental difficulties in obtaining sufficientbiological materials, it is common to see studies with sample size muchsmaller than the number of dimensions. These problems are referredto as “large p small n” issues, where p is the number of dimensions(or say genes) and n is the sample size. High-dimensional data posechallenges to traditional statistical methods. For instance, owing tosmall n, there are increased uncertainties in the standard estimations ofparameters such as means and variances. As a consequence, statisticalanalyses based on such parameters estimation are usually unreliable. Tohave improved parameters estimation, researchers have come up withinnovative ways to deal with this.