UT Shield

Aryan Mokhtari

Publications

Preprints

  • Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks [pdf]
      • L. Collins, H. Hassani, M. Soltanolkotabi, A. Mokhtari, S. Shakkottai
  • An Inexact Conditional Gradient Method for Constrained Bilevel Optimization [pdf]
      • N. Abolfazli, R. Jiang, A. Mokhtari, E. Yazdandoost Hamedani
  • Limited-Memory Greedy Quasi-Newton Method with Non-asymptotic Superlinear Convergence Rate [pdf]
      • Z. Gao, A. Mokhtari, A. Koppel
  • Statistical and Computational Complexities of BFGS Quasi-Newton Method for Generalized Linear Models [pdf]
      • Q. Jin, T. Ren, N. Ho, A. Mokhtari
  • Generalized Optimistic Methods for Convex-Concave Saddle Point Problems [pdf]
      • R. Jiang, A. Mokhtari

2023

  • Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization [pdf]
      • R. Jiang, A. Mokhtari. Neural Information Processing Systems (NeurIPS), 2023. (Spotlight)
  • Greedy Pruning with Group Lasso Provably Generalizes for Matrix Sensing [pdf]
      • N. Rajaraman, Devvrit, A. Mokhtari, K. Ramchandran. Neural Information Processing Systems (NeurIPS), 2023.
  • Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem [pdf]
      • J. Cao, R. Jiang, N. Abolfazli, E. Yazdandoost Hamedani, A. Mokhtari. Neural Information Processing Systems (NeurIPS), 2023.
  • Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergence [pdf]
      • R. Jiang, Q. Jin, A. Mokhtari. Conference on Learning Theory (COLT), 2023.
  • InfoNCE Loss Provably Learns Cluster-Preserving Representations [pdf]
      • A. Parulekar, L. Collins, K. Shanmugam, A. Mokhtari, S. Shakkottai. Conference on Learning Theory (COLT), 2023.
  • A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem [pdf]
      • R. Jiang, N. Abolfazli, A. Mokhtari, E. Yazdandoost Hamedani. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2023.
  • Network Adaptive Federated Learning: Congestion and Lossy Compression [pdf]
      • P. Hedge, G. de Veciana, A. Mokhtari. Int. Conf. on Computer Communications (INFOCOM), 2023.
  • Provably Private Distributed Averaging Consensus: An Information-Theoretic Approach [pdf]
      • M. Fereydounian, A. Mokhtari, R. Pedarsani, H. Hassani. IEEE Transactions on Information Theory, 2023.
  • Straggler-Resilient Personalized Federated Learning [pdf]
      • I. Tziotis, Z. Shen, R. Pedarsani, H. Hassani, A. Mokhtari. Transactions on Machine Learning Research (TMLR), 2023. 

2022

  • Non-asymptotic Superlinear Convergence of Standard Quasi-Newton Methods [pdf]
      • Q. Jin, A. Mokhtari, Mathematical Programming (MAPR), 2022.
  • FedAvg with Fine Tuning: Local Updates Lead to Representation Learning [pdf]
      • L. Collins, H. Hassani, A. Mokhtari, S. Shakkottai, Neural Information Processing Systems (NeurIPS), 2022.
  • MAML and ANIL Provably Learn Representations [pdf]
      • L. Collins, A. Mokhtari, S. Oh, S. Shakkottai. Int. Conference on Machine Learning (ICML), 2022.
  • Sharpened Quasi-Newton Methods: Faster Superlinear Rate and Larger Local Convergence Neighborhood [pdf]
      • Q. Jin, A. Koppel, K. Rajawat, A. Mokhtari. Int. Conference on Machine Learning (ICML), 2022.
  • The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance [pdf]
      • M. Faw, I. Tziotis, C. Caramanis, A. Mokhtari, S. Shakkottai, R. Ward. Conference on Learning Theory (COLT), 2022.
  • Minimax Optimization: The Case of Convex-Submodular [pdf] (Oral Presentation)
      • A. Adibi, A. Mokhtari, H. Hassani. International Conference on Artificial Intelligence and Statistics (AISTATS) 2022.
  • Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation System [pdf]
      • M. Ye, R. Jiang, H. Wang, D. Choudhary, X. Du, B. Bhushanam, A. Mokhtari, A. Kejariwal, Q. Liu. Conf. on Uncertainty in Artificial Intelligence (UAI) 2022.
  • Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity [pdf]
      • A. Reisizadeh, I. Tziotis, H. Hassani, A. Mokhtari, R. Pedarsani. IEEE Journal on Selected Areas in Information Theory (JSAIT), 2022.
  • How Does the Task Landscape Affect MAML Performance? [pdf]
      • L. Collins, A. Mokhtari, S. Shakkottai. Conference on Lifelong Learning Agents (CoLLAs) 2022.

2021

  • Exploiting Local Convergence of Quasi-Newton Methods Globally: Adaptive Sample Size Approach [pdf]
      • Q. Jin, A. Mokhtari. Neural Information Processing Systems (NeurIPS), 2021.
  • Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks [pdf]
      • A. Fallah, A. Mokhtari, A. Ozdaglar. Neural Information Processing Systems (NeurIPS), 2021.
  •  On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning [pdf]
      • A. Fallah, A. Mokhtari, A. Ozdaglar. Neural Information Processing Systems (NeurIPS), 2021.
  • Exploiting Shared Representations for Personalized Federated Learning [pdf]
      • L. Collins, H. Hassani , A. Mokhtari, S. Shakkottai. Int. Conference on Machine Learning (ICML), 2021.
  • Federated Learning with Compression: Unified Analysis and Sharp Guarantees [pdf]
      • F. Haddadpour, M. M. Kamani, A. Mokhtari, M. Mahdavi. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2021.

2020

  • Task-Robust Model-Agnostic Meta-Learning [pdf]
      • L. Collins, A. Mokhtari, S. Shakkottai. Neural Information Processing Systems (NeurIPS), 2020.
  • Second Order Optimality in Decentralized Non-Convex Optimization via Perturbed Gradient Tracking [pdf]
      • I. Tziotis,  C. Caramanis, A. Mokhtari. Neural Information Processing Systems (NeurIPS), 2020.
  • Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach [pdf]
      • A. Fallah, A. Mokhtari, A. Ozdaglar. Neural Information Processing Systems (NeurIPS), 2020.
  • Submodular Meta-Learning [pdf]
      • A. Adibi, A. Mokhtari, H. Hassani. Neural Information Processing Systems (NeurIPS), 2020.
  • Stochastic Quasi-Newton Methods [pdf]
      • A. Mokhtari, A. Ribeiro. Proceedings of the IEEE (PIEEE), 2020. [Invited survey paper]
  • Convergence Rate of O(1/k) for Optimistic Gradient and Extra-gradient Methods in Smooth Convex-Concave Saddle Point Problems [pdf]
      • (α-β) A. Mokhtari, A. Ozdaglar, S. Pattathil. SIAM Journal on Optimization (SIOPT), 2020.
  • Stochastic Conditional Gradient++: (Non-)Convex Minimization and Continuous Submodular Maximization [pdf]
      • (α-β) H. Hassani, A. Karbasi, A. Mokhtari, Z. Shen. SIAM Journal on Optimization (SIOPT), 2020.
  • High-Dimensional Nonconvex Stochastic Optimization by Doubly Stochastic Successive Convex Approximation [pdf]
      • A. Mokhtari, A. Koppel. IEEE Transactions on Signal Processing (TSP), 2020.
  • Quantized Push-sum for Gossip and Decentralized Optimization over Directed Graph. [pdf]
      • H. Taheri, A. Mokhtari, H. Hassani, R. Pedarsani. Int. Conference on Machine Learning (ICML), 2020.
  • Stochastic Conditional Gradient Methods: From Convex Minimization to Submodular Maximization [pdf]
      • A. Mokhtari, H. Hassani, A. Karbasi. Journal of Machine Learning Research (JMLR), 2020.
  • A Class of Parallel Doubly Stochastic Algorithms for Large-Scale Learning [pdf]
      • A. Mokhtari, A. Koppel, M. Takac, A. Ribeiro. Journal of Machine Learning Research (JMLR), 2020.
  • On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms [pdf]
      • A. Fallah, A. Mokhtari, A. Ozdaglar. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2020.
  • One Sample Stochastic Frank-Wolfe [pdf]
      • M. Zhang, Z. Shen, A. Mokhtari, H. Hassani, A. Karbasi. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2020.
  • FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization [pdf]
      • A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, R. Pedarsani. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2020.
  • Quantized Frank-Wolfe: Communication-Efficient Distributed Optimization [pdf]
      • M. Zhang, L. Chen, A. Mokhtari, H. Hassani, A. Karbasi. Int. Conf.on Artificial Intelligence and Statistics (AISTATS), 2020.
  • A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach [pdf]
      • (α-β) A. Mokhtari, A. Ozdaglar, S. Pattathil. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2020.
  • Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy [pdf]
      • M. Jahani, X. He, C. Ma, A. Mokhtari, D. Mudigere, A. Ribeiro, M. Takac. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2020.
  • DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate [pdf]
      • S. Soori, K. Mischenko, A. Mokhtari, M. Dehnavi, M. Gurbuzbalaban. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2020.

2019

  • Stochastic Continuous Greedy++: When Upper and Lower Bounds Match
      • (α-β order) H. Hassani, A. Karbasi, A. Mokhtari, Z. Shen. Neural Information Processing Systems (NeurIPS), 2019.
  • Robust and Communication-Efficient Collaborative Learning [pdf]
      • A. Reisizadeh, H. Taheri, A. Mokhtari, H. Hassani, R. Pedarsani. Neural Information Processing Systems (NeurIPS), 2019.
  • Efficient Nonconvex Empirical Risk Minimization via Adaptive Sample Size Methods [pdf]
      • A. Mokhtari, A. Ozdaglar, A. Jadbabaie. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2019.
  • A Newton-based Method for Nonconvex Optimization with Fast Evasion of Saddle Points [pdf]
      • S. Paternain, A. Mokhtari, A. Ribeiro. SIAM Journal on Optimization (SIOPT), 2019.
  • An Exact Quantized Decentralized Gradient Descent Algorithm [pdf]
      • A. Reisizadeh, A. Mokhtari, H. Hassani, R. Pedarsani. IEEE Transactions on Signal Processing (TSP), 2019.
  • A Primal-Dual Quasi-Newton Method for Exact Consensus Optimization [pdf]
      • M. Eisen, A. Mokhtari, A. Ribeiro. IEEE Transactions on Signal Processing (TSP), 2019.
  • Achieving Acceleration in Distributed Optimization via Direct Discretization of the Heavy-Ball ODE [pdf]
      • J. Zhang, C. Uribe, A. Mokhtari, A. Jadbabaie. American Control Conference (ACC), 2019.

2018

  • Escaping Saddle Points in Constrained Optimization [pdf] (Spotlight: Top 4% of the submitted papers)
      • A. Mokhtari, A. Ozdaglar, A. Jadbabaie. Neural Information Processing Systems (NeurIPS), 2018.
  • Direct Runge-Kutta Discretization Achieves Acceleration [pdf]  (Spotlight: Top 4% of the submitted papers)
      • J. Zhang, A. Mokhtari, S. Sra, A. Jadbabaie. Neural Information Processing Systems (NeurIPS), 2018.
  • Decentralized Submodular Maximization: Bridging Discrete and Continuous Settings [pdf] [Supplementary material]
      • A. Mokhtari, H. Hassani, A. Karbasi. Int. Conference on Machine Learning (ICML), 2018. (Long talk)
  • Towards More Efficient Stochastic Decentralized Learning: Faster Convergence and Sparse Communication [pdf] [Supp. material]
      • Z. Shen, A. Mokhtari, H. Qian, P. Zhao, T. Zhou. Int. Conference on Machine Learning (ICML), 2018.
  • Conditional Gradient Method for Stochastic Submodular Maximization: Closing the Gap [pdf] [Supplementary material]
      • A. Mokhtari, H. Hassani, A. Karbasi. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2018.
  • Large Scale Empirical Risk Minimization via Truncated Adaptive Newton Method [pdf] [Supplementary material]
      • M. Eisen, A. Mokhtari, A. Ribeiro. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), 2018.
  • IQN: An Incremental Quasi-Newton Method with Local Superlinear Convergence Rate [pdf]
      • A. Mokhtari, M. Eisen, A. Ribeiro, SIAM Journal on Optimization (SIOPT), 2018.
  • Surpassing Gradient Descent Provably: A Cyclic Incremental Method with Linear Convergence Rate [pdf]
      • A. Mokhtari, M. Gürbüzbalaban, A. Ribeiro. SIAM Journal on Optimization (SIOPT), 2018.
  • Quantized Decentralized Consensus Optimization [pdf]
      • A. Reisizadeh, A. Mokhtari, H. Hassani, R. Pedarsani. IEEE Conference on Decision and Control (CDC), 2018.
  • A Newton Method for Faster Navigation in Cluttered Environments [pdf]
      • S. Paternain, A. Mokhtari, A. Ribeiro. IEEE Conference on Decision and Control (CDC), 2018.
  • Parallel Stochastic Successive Convex Approximation Method for Large-Scale Dictionary Learning
      • A. Koppel, A. Mokhtari, A. Ribeiro. Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2018.

2017

  • Efficient Methods for Large-Scale Empirical Risk Minimization [pdf]
      • A. Mokhtari. Ph.D. Dissertation, University of Pennsylvania, 2017. (Joseph and Rosaline Wolf Award for Best Doctoral Dissertation)
  • First-Order Adaptive Sample Size Methods to Reduce Complexity of Empirical Risk Minimization [pdf] [Supplementary material]
      • A. Mokhtari, A. Ribeiro. Advances in Neural Information Processing Systems (NeurIPS), 2017.
  • Decentralized Prediction-Correction Methods for Networked Time-Varying Convex Optimization [pdf]
      • A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, A. Ribeiro. IEEE Transactions on Automatic Control (TAC), 2017.
  • Stochastic Averaging for Constrained Optimization with Application to Online Resource Allocation [pdf]
      • T. Chen, A. Mokhtari, X. Wang, A. Ribeiro, G. B. Giannakis. IEEE Transactions on Signal Processing (TSP), 2017.
  • Decentralized Quasi-Newton Methods [pdf]
      • M. Eisen, A. Mokhtari, A. Ribeiro. IEEE Transactions on Signal Processing (TSP), 2017.
  • Network Newton Distributed Optimization Methods [pdf]
      • A. Mokhtari, Q. Ling, A. Ribeiro. IEEE Transactions on Signal Processing (TSP), 2017.
  • A Primal-Dual Quasi-Newton Method for Consensus Optimization [pdf]
      • M. Eisen, A. Mokhtari, A. Ribeiro. Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2017.
  • An Incremental Quasi-Newton Method with a Local Superlinear Convergence Rate [pdf]
      • A. Mokhtari, M. Eisen, A. Ribeiro. Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.
  • A Double Incremental Aggregated Gradient Method with Linear Convergence Rate for Large-Scale Optimization [pdf]
      • A. Mokhtari, M. Gürbüzbalaban, A. Ribeiro. Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.
  • Large-Scale NonConvex Stochastic Optimization by Doubly Stochastic Successive Convex Approximation [pdf]
      • A. Mokhtari, A. Koppel, G. Scutari, A. Ribeiro. Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.
  • A Diagonal-Augmented Quasi-Newton Method with Application to Factorization Machines [pdf]
      • A. Mokhtari, A. Ingber. Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.

2016

  • DSA: Decentralized Double Stochastic Averaging Gradient Algorithm [pdf]
      • A. Mokhtari, A. Ribeiro. Journal of Machine Learning Research (JMLR), 2016.
  • Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy [pdf] [Supplementary Material]
      • A. Mokhtari, H. Daneshmand, A. Lucchi, T. Hofmann, A. Ribeiro. Advances in Neural Information Processing Systems (NeurIPS), 2016.
  • A Decentralized Second-Order Method with Exact Linear Convergence Rate for Consensus Optimization [pdf]
      • A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro. IEEE Transactions on Signal and Information Processing over Networks (TSIPN), 2016
  • DQM: Decentralized Quadratically Approximated Alternating Direction Method of Multipliers [pdf]
      • A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro. IEEE Transactions on Signal Processing (TSP), 2016.
  • A Class of Prediction-Correction Methods for Time-Varying Convex Optimization [pdf]
      • A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, A. Ribeiro. IEEE Transactions on Signal Processing (TSP), 2016.
  • Online Optimization in Dynamic Environments: Improved Regret Rates for Strongly Convex Problems [pdf]
      • A. Mokhtari, S. Shahrampour, A. Jadbabaie, A. Ribeiro. IEEE Conference on Decision and Control (CDC), 2016.
  • A Decentralized Second-Order Method for Dynamic Optimization [pdf]
      • A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro. IEEE Conference on Decision and Control (CDC), 2016.
  • A Decentralized Quasi-Newton Method for Dual Formulations of Consensus Optimization [pdf]
      • M. Eisen, A. Mokhtari, A. Ribeiro. IEEE Conference on Decision and Control (CDC), 2016.
  • A Quasi-Newton Prediction-Correction Method for Decentralized Dynamic Convex Optimization [pdf]
      • A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, A. Ribeiro. European Control Conference (ECC), 2016.
  • Doubly Random Parallel Stochastic Methods for Large Scale Learning [pdf]
      • A. Mokhtari, A. Koppel, A. Ribeiro. American Control Conference (ACC), 2016.
  • A Data-driven Approach to Stochastic Network Optimization [pdf]
      • T. Chen, A. Mokhtari, X. Wang, A. Ribeiro, G. B. Giannakis. IEEE Global Conf. on Signal and Information Processing (GlobalSIP), 2016.
  • Decentralized Constrained Consensus Optimization with Primal-Dual Splitting Projection [pdf]
      • H. Zhang, W. Shi, A. Mokhtari, A. Ribeiro, Q. Ling.. IEEE Global Conf. on Signal and Information Processing (GlobalSIP), 2016.
  • An Asynchronous Quasi-Newton Method for Consensus Optimization [pdf]
      • M. Eisen, A. Mokhtari, A. Ribeiro. IEEE Global Conf. on Signal and Information Processing (GlobalSIP), 2016.
  • ESOM: Exact Second-Order Method for Consensus Optimization [pdf]
      • A. Mokhtari, W. Shi, Qing Ling. Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2016.
  • Doubly Stochastic Algorithms for Large-Scale Optimization [pdf]
      • A. Koppel, A. Mokhtari, A. Ribeiro. Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2016.

2015

  • Global Convergence of Online Limited Memory BFGS [pdf]
      • A. Mokhtari, A. Ribeiro. Journal of Machine Learning Research (JMLR), 2015.
  • A Decentralized Prediction-Correction Method for Networked Time-Varying Convex Optimization [pdf]
      • A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, A. Ribeiro. IEEE Int. Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), 2015.
  • Decentralized Quadratically Approximated Alternating Direction Method of Multipliers [pdf]
      • A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro. IEEE Global Conf. on Signal and Information Processing (GlobalSIP), 2015.
  • Target Tracking with Dynamic Convex Optimization [pdf]
      • A. Koppel, A. Simonetto, A. Mokhtari, G. Leus, A. Ribeiro. IEEE Global Conf. on Signal and Information Processing (GlobalSIP), 2015.
  • Decentralized Double Stochastic Averaging Gradient [pdf]
      • A. Mokhtari, A. Ribeiro. Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2015.
  • Prediction-Correction Methods for Time-Varying Convex Optimization [pdf]
      • A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, A. Ribeiro. Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2015.
  • An Approximate Newton Method for Distributed Optimization [pdf]
      • A. Mokhtari, Q. Ling, A. Ribeiro. Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2015.

2014

  • RES: Regularized Stochastic BFGS Algorithm [pdf]
      • A. Mokhtari, A. Ribeiro. IEEE Transactions on Signal Processing (TSP), 2014.
  • Network Newton [pdf]
      • A. Mokhtari, Q. Ling, A. Ribeiro. Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2014.
  • A Quasi-Newton Method for Large Scale Support Vector Machines [pdf]
      • A. Mokhtari, A. Ribeiro. Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2014.

2013

  • Regularized Stochastic BFGS algorithm [pdf]
      • A. Mokhtari, A. Ribeiro. IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2013.
  • A Dual Stochastic DFP algorithm for Optimal Resource Allocation in Wireless Systems [pdf]
      • A. Mokhtari, A. Ribeiro. IEEE Workshop on Signal Process. Advances in Wireless Communication (SPAWC), 2013.
×