Randomized first-order optimization algorithms have become the de-facto techniques for solving large-scale and high-dimensional composite optimization problems which appear increasingly more frequent in modern data science and engineering applications. In the past decade, researchers have made key progresses along this line and developed efficient algorithms which have fast convergence rates matching the complexity lower bounds of generic classes. However, while focusing on the generic optimizers, the classical paradigms for algorithmic design have a limitation of ignoring the inherent data structure of the optimization problem, leading to sub-optimal algorithms in practice.

In this talk, we will start by presenting our new paradigms for randomized first-order optimization and showing that the intrinsic low-dimensional structure of the composite optimization problem's solution (such as sparsity, group-sparsity, piece-wise smoothness, or low-rank structure, etc), can be actively exploited by some purposefully tailored optimization algorithms to achieve even faster convergence rates. The proposed methods are provably equipped with the desirable structure-adaptive property leading to significantly improved convergence rates in practice, including the sketched gradient algorithms, the adaptive stochastic variance-reduced gradient methods and randomized proximal alternating minimization algorithms, etc. We demonstrate the effectiveness of our algorithms in various machine learning and computational imaging tasks such as sparse regression, matrix factorization, blind deconvolution, and tomographic medical image reconstruction.

Moreover, since the practitioners in different scientific areas usually only need to focus on a very narrow subclass of problems with a certain distribution of data, further accelerations can be achieved if we could also enable the optimization algorithms to exploit the prior information of the data distribution, leading to truly optimal optimizers specialized for the application. In the second part of the talk, we will present our on-going works towards achieving this goal, where we propose new theoretically-principled paradigms of intelligent data-driven optimizers by leveraging deep-learning schemes in our algorithmic design.

Speaker: Dr Junqi TANG
Date: 16 September 2022 (Friday)
Time: 2:30pm – 3:30pm
Poster: Click here