Tan Minh Nguyen

Assistant Professor (Presidential Young Professor) Department of Mathematics National University of Singapore Email: tanmn@nus.edu.sg Office: 10 Lower Kent Ridge Road, S17-08-20, Singapore 119076 Education Ph.D. in Electrical and Computer Engineering, Rice University, 2020 M.S. in Electrical and Computer Engineering, Rice University, 2016 B.S. in Electrical and Computer Engineering, Rice University, 2014 CV, Github, Google Scholar

Brief Biography

I am currently an Assistant Professor of Mathematics (Presidential Young Professor) at the National University of Singapore (NUS). Before joining NUS, I was a postdoctoral scholar in the Department of Mathematics at the University of California, Los Angeles, working with Dr. Stanley J. Osher. I obtained my Ph.D. in Machine Learning from Rice University, where I was advised by Dr. Richard G. Baraniuk. I gave an invited talk in the Deep Learning Theory Workshop at NeurIPS 2018 and organized the 1st Workshop on Integration of Deep Neural Models and Differential Equations at ICLR 2020. I also had two awesome long internships with Amazon AI and NVIDIA Research, during which I worked with Dr. Anima Anandkumar. I am the recipient of the prestigious Computing Innovation Postdoctoral Fellowship (CIFellows) from the Computing Research Association (CRA), the NSF Graduate Research Fellowship, and the IGERT Neuroengineering Traineeship. I received my M.S. and B.S. in Electrical and Computer Engineering from Rice in May 2018 and May 2014, respectively.

I am also writing about simple ideas and principled approaches that lead to working machine learning algorithms on my blog, Almost Convergent.

Research Interests

My research focuses on the interplay of the interpretability, robustness, and efficiency of machine learning models from three principled approaches:

• Optimization (primal-dual frameworks for deep learning models, momentum-based neural networks, fast multipole transformers)

• Differential equations (Nesterov neural ordinary differential equations, graph neural diffusion)

• Statistical modeling (mixture and nonparametric kernel regression frameworks for transformers, deep generative models)

Publications

Journal Publications

• A Bayesian Perspective of Convolutional Neural Networks through a Deconvolutional Generative Model. Journal of Machine Learning Research, 2023.
  Tan M. Nguyen*, Nhat Ho*, Ankit B. Patel, Anima Anandkumar, Michael I. Jordan, Richard G. Baraniuk.

• Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent. SIAM Journal on Imaging Sciences, 2022.
  Bao Wang*, Tan M. Nguyen*, Andrea L. Bertozzi, Richard G. Baraniuk, Stanley J. Osher.

• How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies. Research in the Mathematical Sciences, 2022.
  Bao Wang, Hedi Xia, Tan M. Nguyen, Stanley J. Osher.

• Dual Dynamic Inference: Enabling More Efficient, Adaptive, and Controllable Deep Inference. IEEE Journal of Selected Topics in Signal Processing, 2020.
  Yue Wang, Jianghao Shen, Ting-Kuei Hu, Pengfei Xu, Tan M. Nguyen, Richard G. Baraniuk, Zhangyang Wang, Yingyan Lin.

Conference Publications

• MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts. Conference on Neural Information Processing Systems (NeurIPS), 2024.
  Rachel S.Y. Teo, Tan M. Nguyen.

• Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysis. Conference on Neural Information Processing Systems (NeurIPS), 2024.
  Rachel S.Y. Teo, Tan M. Nguyen.

• Monomial Matrix Group Equivariant Neural Functional Networks. Conference on Neural Information Processing Systems (NeurIPS), 2024.
  Hoang V. Tran, Thieu N. Vo, Tho H. Tran, An T. Nguyen, Tan M. Nguyen.

• Elliptical Attention. Conference on Neural Information Processing Systems (NeurIPS), 2024.
  Stefan K. Nielsen, Laziz U. Abdullaev, Rachel S.Y. Teo, Tan M. Nguyen.

• PIDformer: Transformer Meets Control Theory. International Conference on Machine Learning (ICML), 2024.
  Tam Nguyen, Cesar A. Uribe, Tan M. Nguyen**, Richard Baraniuk**.

• Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features Model. International Conference on Machine Learning (ICML), 2024.
  Hien Dang, Tho Tran Huu, Tan M. Nguyen**, Nhat Ho**.

• Revisiting Kernel Attention with Correlated Gaussian Process Representation. Conference on Uncertainty in Artificial Intelligence (UAI), 2024.
  Long Minh Bui, Tho Tran Huu, Duy Dinh, Tan M. Nguyen**, Trong Nghia Hoang**.

• Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders. International Conference on Learning Representations (ICLR), 2024.
  Hien Dang, Tho Tran Huu, Tan M. Nguyen**, Nhat Ho**.

• From Coupled Oscillators to Graph Neural Networks: Reducing Over-smoothing via a Kuramoto Model-based Approach. International Conference on Artificial Intelligence and Statistics (AISTATS), 2024.
  Tuan Nguyen, Hirotada Honda, Takashi Sano, Vinh Nguyen*, Shugo Nakamura*, Tan M. Nguyen*.

• Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionals. Conference on Neural Information Processing Systems (NeurIPS), 2023.
  Tam Nguyen, Tan M. Nguyen, Richard G. Baraniuk.

• Designing Robust Transformers using Robust Kernel Density Estimation. Conference on Neural Information Processing Systems (NeurIPS), 2023.
  Xing Han*, Tongzheng Ren*, Tan M. Nguyen*, Khai Nguyen, Joydeep Ghosh, Nhat Ho.

• Revisiting Over-smoothing and Over-squashing using Ollivier-Ricci Curvature. International Conference on Machine Learning (ICML), 2023.
  Khang Nguyen, Hieu Nong, Vinh Nguyen, Nhat Ho, Stanley J. Osher, Tan M. Nguyen.

• Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Data. International Conference on Machine Learning (ICML), 2023.
  Hien Dang*, Tho Tran Huu*, Stanley J. Osher, Hung Tran-The, Nhat Ho**, Tan M. Nguyen**.

• A Primal-Dual Framework for Transformers and Neural Networks. International Conference on Learning Representations (ICLR) (notable-top-25%), 2023.
  Tan M. Nguyen*, Tam Nguyen*, Nhat Ho, Andrea L. Bertozzi, Richard G. Baraniuk, Stanley J. Osher.

• Hierarchical Sliced Wasserstein Distance. International Conference on Learning Representations (ICLR), 2023.
  Khai Nguyen, Tongzheng Ren, Huy Nguyen, Litu Rout, Tan M. Nguyen, Nhat Ho.

• DeepGRAND: Deep Graph Neural Diffusion. Asilomar Conference, 2023.
  Khang Nguyen, Hieu Nong, Khuong Nguyen, Tan M. Nguyen**, Vinh Nguyen**.

• A Probabilistic Framework for Pruning Transformers via a Finite Admixture of Keys. International Conference on Acoustics, Speech, and Signal Processing (ICASSP, notable-top-3%), 2023.
  Tan M. Nguyen*, Tam Nguyen*, Long Bui, Hai Do, Dung Le, Hung Tran-The, Khuong Nguyen, Richard G. Baraniuk, Nhat Ho, Stanley J. Osher.

• FourierFormer: Transformer Meets Generalized Fourier Integral Theorem. Conference on Neural Information Processing Systems (NeurIPS), 2022.
  Tan M. Nguyen*, Minh Pham*, Tam Nguyen, Khai Nguyen, Stanley J. Osher, Nhat Ho.

• Improving Transformer with an Admixture of Attention Heads. Conference on Neural Information Processing Systems (NeurIPS), 2022.
  Tan M. Nguyen*, Tam Nguyen*, Hai Do, Khai Nguyen, Vishwanath Saragadam, Minh Pham, Khuong Nguyen, Nhat Ho**, Stanley J. Osher**.

• Improving Neural Ordinary Differential Equations with Nesterov’s Accelerated Gradient Method. Conference on Neural Information Processing Systems (NeurIPS), 2022.
  Nghia Nguyen*, Tan M. Nguyen*, Huyen Vo, Stanley J. Osher, Thieu Vo.

• Improving Transformers with Probabilistic Attention Keys. International Conference on Machine Learning (ICML), 2022.
  Tam Nguyen*, Tan M. Nguyen*, Dung Le, Khuong Nguyen, Anh Tran, Richard G. Baraniuk, Nhat Ho**, Stanley J. Osher**.

• GRAND++: Graph Neural Diffusion with a Source Term. International Conference on Learning Representations (ICLR), 2022.
  Matthew Thorpe*, Tan M. Nguyen*, Hedi Xia*, Thomas Strohmer, Andrea Bertozzi, Stanley J. Osher, Bao Wang.

• Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization. Mathematical and Scientific Machine Learning (MSML), 2022.
  Tan M. Nguyen, Richard G. Baraniuk, Mike Kirby, Stanley J. Osher, Bao Wang.

• InfoCNF: An Efficient Conditional Continuous Normalizing Flow with Adaptive Solvers. Asilomar Conference, 2022.
  Tan M. Nguyen, Animesh Garg, Richard G Baraniuk, Anima Anandkumar.

• Probabilistic Framework for Pruning Transformers via a Finite Admixture of Keys. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023.
  Tan M. Nguyen*, Tam Nguyen*, Long Bui*, Hai Do, Dung Le, Hung Tran-The, Khuong Nguyen, Richard G. Baraniuk, Nhat Ho**, Stanley J. Osher**.

• FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention. Conference on Neural Information Processing Systems (NeurIPS), 2021.
  Tan M. Nguyen, Vai Suliafu, Stanley J. Osher, Long Chen, Bao Wang.

• Heavy Ball Neural Ordinary Differential Equations. Conference on Neural Information Processing Systems (NeurIPS), 2021.
  Hedi Xia, Vai Suliafu, Hangjie Ji, Tan M. Nguyen, Andrea Bertozzi, Stanley J. Osher, Bao Wang.

• MomentumRNN: Integrating Momentum into Recurrent Neural Networks. Conference on Neural Information Processing Systems (NeurIPS), 2020.
  Tan M. Nguyen, Richard G. Baraniuk, Andrea Bertozzi, Stanley J. Osher, Bao Wang.

• Neural Networks with Recurrent Generative Feedback. Conference on Neural Information Processing Systems (NeurIPS), 2020.
  Yujia Huang, James Gornet, Sihui Dai, Zhiding Yu, Tan M. Nguyen, Doris Tsao, Anima Anandkumar.

• Neural Rendering Model: Joint Generation and Prediction for Semi-Supervised Learning. Deep Math Conference (DeepMath), 2019. (Oral presentation)
  Tan M. Nguyen*, Nhat Ho*, Ankit B. Patel, Anima Anandkumar, Michael I. Jordan, Richard G. Baraniuk.

• Towards a Cortically Inspired Deep Learning Model: Semi-Supervised Learning, Divisive Normalization, and Synaptic Pruning. Conference on Cognitive Computational Neuroscience (CCN), 2017.
  Tan M. Nguyen, Wanjia Liu, Fabian Sinz, Richard G. Baraniuk, Andreas S. Tolias, Xaq Pitkow, Ankit B. Patel.

• Probabilistic Framework for Deep Learning. Conference on Neural Information Processing Systems (NeurIPS), 2016.
  Ankit B. Patel, Tan M. Nguyen, Richard G Baraniuk.

Papers on Application

• Turbulence Forecasting via Neural ODE. NeurIPS Workshop on Machine Learning and the Physical Sciences, 2019.
  Gavin D. Portwood, Peetak P. Mitra, Mateus Dias Ribeiro, Tan M. Nguyen, Balasubramanya T. Nadiga, Juan A. Saenz, Michael Chertkov, Animesh Garg, Anima Anandkumar, Andreas Dengel, Richard G. Baraniuk, David P. Schmidt.