In this talk, I will discuss briefly the theoretical advances of non-convex optimization methods stemmed from machine learning practice. I will begin with (perhaps the simplest) PCA model and show that scalable algorithms can achieve a rate that matches minimax information lower bound. Then, I will briefly discuss the recent progress on escaping from saddle points, and the importance of noise therein. Finally, I will introduce our new SPIDER technique which can be used to design an algorithm achieving a (perhaps surprisingly) $O(\varepsilon^{-3})$ convergence rate for finding an $(\varepsilon,O(\varepsilon^{0.5}))$- approximate second-order stationary point.
Speaker: Dr Junchi LI
Date: 19 February 2019
Time: 10:00am - 11:00am
Poster: Click here
Latest Seminar
Biography
Dr Junchi Li obtained his B.S. in Mathematics and Applied Mathematics at Peking University in 2009, and his Ph.D. in Mathematics at Duke University in 2014. He has since held several research positions, including the role of visiting postdoctoral research associate at Department of Operations Research and Financial Engineering, Princeton University. His research interests include statistical machine learning and optimization, scalable online algorithms for big data analytics, and stochastic dynamics on graphs and social networks. He has published original research articles in both top optimization journals and top machine learning conferences, including an oral presentation paper (1.23%) at NIPS 2017 and a spotlight paper (4.08%) at NIPS 2018.