The recent success of neural networks in pattern recognition and classification problems suggests that neural networks possess qualities distinct from other more classical classifiers, such as SVMs or boosting classifiers. This paper studies the performance of plug-in classifiers based on neural networks in a binary classification setting as measured by their excess risks. Compared to the typical settings imposed in the literature, we consider a more general scenario that resembles actual practice in two respects: first, the function class to be approximated includes the Barron functions as a proper subset, hence smooth functions, and second, the neural network classifier constructed is the minimizer of a surrogate loss instead of the 0-1 loss so that gradient descent-based numerical optimizations can be easily applied. We study the estimation and approximation properties of neural networks to obtain a dimension-free, uniform rate of convergence. In the analysis of the estimation error, we obtain a novel result that relates the approximate excess risk to the approximate excess ϕ-risk, which is of interest on its own. Finally, we show that the rate obtained is, in fact, minimax optimal up to a logarithmic factor, and the lower bound obtained shows the effect of the margin assumption in this regime.

Speaker: Professor Xiaoming Huo
Date: 15 November 2023 (Wednesday)
Time: 10:00am – 11:00am
Poster: Click here