About Myself

Hi, I am a Ph.D. student of the joint Ph.D. Program of USTC and Microsoft Research (Asia). I am very fortunate to be advised by Prof. Wei Chen and Prof. Zhi-Ming Ma. I also work closely with Dr. Qi Meng, Dr. Huishuai Zhang, and Prof. Ruoyu Sun.

Research Interests

My current research interest is optimization in deep learning. I am eager to understand the underlying mechanism of optimizers’ behavior over deep neural networks. My ultimate goal is to build effective and stable optimizers for deep learning, especially for large models. Please don’t hesitate to drop an email if you are interested to collaborate!

News

• Temporarily leave MSRA and start an internship @ Bytedance AI lab
• Invited to give a talk on the KAUST Rising Stars in AI Symposium 2024!
• One paper is accepted to NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning as Oral!
• Three papers are accepted by NeurIPS 2023! Thanks to all the collaborators!
• I will visit Prof. Ruoyu Sun at The Chinese University of Hong Kong (Shenzhen), starting from June 11th!

Prepints

• Provable Adaptivity in Adam
  Bohan Wang, Yushun Zhang, Huishuai Zhang, Qi Meng, Zhi-Ming Ma, Tie-Yan Liu, Wei Chen
  A brief introduction (in Chinese) can be found here.

Publication (* denotes equal contribution)

• Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study
  Prin Phunyaphibarn, Junghyun Lee, Bohan Wang, Huishuai Zhang, Chulhee Yun
  NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral)
• Closing the gap between the upper bound and lower bound of Adam’s iteration complexity
  *Bohan Wang, *Jingwen Fu, Huishuai Zhang, Nanning Zheng, Wei Chen
  Neurips 2023.
• On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-training
  *Jieyu Zhang, *Bohan Wang, Zhengyu Hu, Pang Wei Koh, Alexander Ratner
  Neurips 2023.
• Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions
  *Xiang Cheng, *Bohan Wang, *Jingzhao Zhang, *Yusong Zhu
  Neurips 2023.
• Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions
  Bohan Wang, Huishuai Zhang, Zhi-Ming Ma, Wei Chen
  COLT 2023.
• $\mathcal{O}$-GNN: incorporating ring priors into molecular modeling
  Jinhua Zhu, Kehan Wu, Bohan Wang, Yingce Xia, Shufang Xie, Qi Meng, Lijun Wu, Tao Qin, Wengang Zhou, Houqiang Li, Tie-Yan Liu
  ICLR 2023.
• Does Momentum Change the Implicit Regularization on Separable Data?
  Bohan Wang, Qi Meng, Huishuai Zhang, Ruoyu Sun, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu
  Neurips 2022 (Spotlight).
• Creating Training Sets via Weak Indirect Supervision
  Jieyu Zhang, Bohan Wang, Xiangchen Song, Yujing Wang, Yaming Yang, Jing Bai, Alexander Ratner.
  ICLR 2022.
• Machine-Learning Non-Conservative Dynamics for New-Physics Detection
  Ziming Liu, Bohan Wang, Qi Meng, Wei Chen, Max Tegmark, Tie-Yan Liu.
  Physics Review E.
• Optimizing Information-theoretical Generalization Bound via Anisotropic Noise in SGLD.
  Bohan Wang, Huishuai Zhang, Jieyu Zhang, Qi Meng, Wei Chen, Tie-Yan Liu.
  NeurIPS 2021.
• The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks
  Bohan Wang, Qi Meng, Wei Chen, Tie-Yan Liu.
  ICML 2021 (Long Presentation).
• Tighter generalization bounds for iterative differentially private learning algorithms
  *Fengxiang He, *Bohan Wang, Dacheng Tao.
  UAI 2021.
• Piecewise linear activations substantially shape the loss surfaces of neural networks
  *Fengxiang He, *Bohan Wang, Dacheng Tao.
  ICLR 2020.

Education

• University of Science and Technology of China - Joint Ph.D. Program of USTC & MSR (Asia) (2021 - Present).
  Supervised by Prof. Wei Chen and Prof. Zhi-Ming Ma.
• University of Science and Technology of China - Sc.B in Computational Mathematics.
  Supervised by Prof. Jie Wang.

Internship

• Research Assistant - University of Sydney (2019.06-2019.09).
  Supervised by Prof. Dacheng Tao
• Star Bridge Research Assistant - Microsoft Research (Asia) (2020.09-2021.08)
  Supervised by Prof. Wei Chen.

Awards

• Travel Award, Neurips 2021.

Professional Service

• Reviewer for AISTATS, ICML, Neurips, ACML, ICLR, ALT.
• I cofound the Theory Lecture Series of Microsoft Research Asia Theory Center, featuring the latest theoretical advances in big data, artificial intelligence, and related areas.
• I am an organizer of FAI-Seminar, featuring the recent advantages in Foundational Artificial Intelligence. The speakers are mostly Ph.D. students. Drop me an email (or send it to tjy20@mails.tsinghua.edu.cn) if you want to become a speaker!

Miscellaneous

The photo was taken by my girlfriend Xianyi Tao. We have met since we were high school classmates.

Contact

Email: bhwangfy@gmail.com