Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other practical domains. The involved deep neural network architectures and computational issues have been well considered in the community of machine learning. But there lacks a theoretical foundation for understanding the modelling, approximation or generalization ability of deep learning models and algorithms. We are interested in deep convolutional neural networks (CNNs) which are practically powerful in processing natural images and speeches. The convolutional architecture and structures give essential differences between the deep CNNs and fully-connected deep neural networks, and the classical theory for fully connected networks developed around 30 years ago does not apply. This talk describes a theory of deep CNNs associated with the rectied linear unit (ReLU) activation function. In particular, we give the first proof for the universality of deep CNNs, meaning that a deep CNN can be used to approximate any continuous function to an arbitrary accuracy when the depth of the neural network is large enough. We also give explicit rates of approximation, and show that the approximation ability of deep CNNs is at least as good as that of fully-connected multi-layer neural networks. Our quantitative estimates, given tightly in terms of the number of free parameters to be trained, verify the effciency of deep learning algorithms in dealing with large dimensional data.

Speaker: Professor Ding-Xuan ZHOU 
Date: 3 June 2020 (Wed)
Time: 11:00am - 12:00pm
Poster: Click here