| Paper ID | D6-S1-T3.3 |
| Paper Title |
Linear Discriminant Analysis under f-divergence Measures |
| Authors |
Anmol Dwivedi, Sihui Wang, Ali Tajer, RPI, United States |
| Session |
D6-S1-T3: Classification II |
| Chaired Session: |
Monday, 19 July, 22:00 - 22:20 |
| Engagement Session: |
Monday, 19 July, 22:20 - 22:40 |
| Abstract |
In statistical inference, the information-theoretic performance limits can often be expressed in terms of a notion of divergence between the underlying statistical models (e.g., in binary hypothesis testing, the total error probability is equal to the total variation between the models). As the data dimension grows, computing the statistics involved in decision-making and the attendant performance limits (divergence measures) face complexity and stability challenges. Dimensionality reduction addresses these challenges at the expense of compromising the performance (divergence reduces due to the data processing inequality for divergence). This paper considers linear dimensionality reduction such that the divergence between the models is \emph{maximally} preserved. Specifically, the paper focuses on the Gaussian models and characterizes an optimal projection of the data onto a lower-dimensional subspace with respect to four $f$-divergence measures (Kullback-Leibler, $\chi^2$, Hellinger, and total variation). There are two key observations. First, projections are not necessarily along the dominant modes of the covariance matrix of the data, and even in some situations, they can be along the least dominant modes. Secondly, under specific regimes, the optimal design of subspace projection is identical under all the $f$-divergence measures considered, rendering a degree of universality to the design independent of the inference problem of interest.
|
