About Myself

Hi, I am a Ph.D. student of the joint Ph.D. Program of USTC and Microsoft Research (Asia). I am very fortunate to be advised by Prof. Wei Chen and Prof. Zhi-Ming Ma. I also work closely with Dr. Qi Meng, Dr. Huishuai Zhang, and Prof. Ruoyu Sun.

Research Interests

I am deeply interested in the interplay between deep learning and mathematics. Specifically, my research focuses on two main areas:

• Mathematics for Deep Learning: I am fascinated by how humans can understand deep neural networks and, more importantly, leverage this understanding to develop better models. My particular interest lies in optimization for deep learning, which I believe is one of the most critical areas where mathematics can make a significant impact. The ultimate goal in this direction is designing the next Adam.

• Deep Learning for Mathematics (AI for Math): The impressive capabilities of large language models as natural language chatbots naturally lead to the question of whether these models can also serve as chatbots for mathematics. I am especially interested in how we can enhance language models using formal verifiers (such as Lean 4). I am a firm believer that AI will revolutionize the way human mathematicians develop mathematics and will eventually replace human mathematicians.

News

• Temporarily leave MSRA and start an internship @ Bytedance AI lab
• Invited to give a talk on the KAUST Rising Stars in AI Symposium 2024!
• One paper is accepted to NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning as Oral!
• Three papers are accepted by NeurIPS 2023! Thanks to all the collaborators!
• I will visit Prof. Ruoyu Sun at The Chinese University of Hong Kong (Shenzhen), starting from June 11th!

Prepints

Publication (* denotes equal contribution)

• Provable Adaptivity in Adam
  *Bohan Wang, *Yushun Zhang, Huishuai Zhang, Qi Meng, Zhi-Ming Ma, Tie-Yan Liu, Wei Chen
  A brief introduction (in Chinese) can be found here
  KDD 2024
• Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study
  Prin Phunyaphibarn, Junghyun Lee, Bohan Wang, Huishuai Zhang, Chulhee Yun
  NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral)
• Closing the gap between the upper bound and lower bound of Adam’s iteration complexity
  *Bohan Wang, *Jingwen Fu, Huishuai Zhang, Nanning Zheng, Wei Chen
  Neurips 2023.
• On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-training
  *Jieyu Zhang, *Bohan Wang, Zhengyu Hu, Pang Wei Koh, Alexander Ratner
  Neurips 2023.
• Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions
  *Xiang Cheng, *Bohan Wang, *Jingzhao Zhang, *Yusong Zhu
  Neurips 2023.
• Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions
  Bohan Wang, Huishuai Zhang, Zhi-Ming Ma, Wei Chen
  COLT 2023.
• $\mathcal{O}$-GNN: incorporating ring priors into molecular modeling
  Jinhua Zhu, Kehan Wu, Bohan Wang, Yingce Xia, Shufang Xie, Qi Meng, Lijun Wu, Tao Qin, Wengang Zhou, Houqiang Li, Tie-Yan Liu
  ICLR 2023.
• Does Momentum Change the Implicit Regularization on Separable Data?
  Bohan Wang, Qi Meng, Huishuai Zhang, Ruoyu Sun, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu
  Neurips 2022 (Spotlight).
• Creating Training Sets via Weak Indirect Supervision
  Jieyu Zhang, Bohan Wang, Xiangchen Song, Yujing Wang, Yaming Yang, Jing Bai, Alexander Ratner.
  ICLR 2022.
• Machine-Learning Non-Conservative Dynamics for New-Physics Detection
  Ziming Liu, Bohan Wang, Qi Meng, Wei Chen, Max Tegmark, Tie-Yan Liu.
  Physics Review E.
• Optimizing Information-theoretical Generalization Bound via Anisotropic Noise in SGLD.
  Bohan Wang, Huishuai Zhang, Jieyu Zhang, Qi Meng, Wei Chen, Tie-Yan Liu.
  NeurIPS 2021.
• The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks
  Bohan Wang, Qi Meng, Wei Chen, Tie-Yan Liu.
  ICML 2021 (Long Presentation).
• Tighter generalization bounds for iterative differentially private learning algorithms
  *Fengxiang He, *Bohan Wang, Dacheng Tao.
  UAI 2021.
• Piecewise linear activations substantially shape the loss surfaces of neural networks
  *Fengxiang He, *Bohan Wang, Dacheng Tao.
  ICLR 2020.

Education

• University of Science and Technology of China - Joint Ph.D. Program of USTC & MSR (Asia) (2021 - Present).
  Supervised by Prof. Wei Chen and Prof. Zhi-Ming Ma.
• University of Science and Technology of China - Sc.B in Computational Mathematics.
  Supervised by Prof. Jie Wang.

Internship

• Research Assistant - University of Sydney (2019.06-2019.09).
  Supervised by Prof. Dacheng Tao
• Star Bridge Research Assistant - Microsoft Research (Asia) (2020.09-2021.08)
  Supervised by Prof. Wei Chen.

Awards

• Travel Award, Neurips 2021.

Professional Service

• Reviewer for AISTATS, ICML, Neurips, ACML, ICLR, ALT.
• I cofound the Theory Lecture Series of Microsoft Research Asia Theory Center, featuring the latest theoretical advances in big data, artificial intelligence, and related areas.
• I am an organizer of FAI-Seminar, featuring the recent advantages in Foundational Artificial Intelligence. The speakers are mostly Ph.D. students. Drop me an email (or send it to tjy20@mails.tsinghua.edu.cn) if you want to become a speaker!

Miscellaneous

The photo was taken by my girlfriend Xianyi Tao. We have met since we were high school classmates.

Contact

Email: bhwangfy@gmail.com