UT Shield

Aryan Mokhtari

Publications

Preprints

  • Straggler-Resilient Personalized Federated Learning[pdf] I. Tziotis, Z. Shen, R. Pedarsani, H. Hassani, A. Mokhtari
  • Statistical and Computational Complexities of BFGS Quasi-Newton Method for Generalized Linear Models [pdf] Q. Jin, T. Ren, N. Ho, A. Mokhtari
  • Generalized Optimistic Methods for Convex-Concave Saddle Point Problems [pdf] R. Jiang, A. Mokhtari
  • Provably Private Distributed Averaging Consensus: An Information-Theoretic Approach [pdf] M. Fereydounian, A. Mokhtari, R. Pedarsani, H. Hassani.

 

2023

  • A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem [pdf] R. Jiang, N. Abolfazli, A. Mokhtari, E. Yazdandoost Hamedani. Int. Conference on Artificial Intelligence and Statistics (AISTATS) 2023.

2022

  • FedAvg with Fine Tuning: Local Updates Lead to Representation Learning[pdf] L. Collins, H. Hassani, A. Mokhtari, S. Shakkottai, Neural Information Processing Systems (NeurIPS), 2022.
  • Non-asymptotic Superlinear Convergence of Standard Quasi-Newton Methods [pdf] Q. Jin, A. Mokhtari, Mathematical Programming (MAPR), 2022.
  • MAML and ANIL Provably Learn Representations [pdf] L. Collins, A. Mokhtari, S. Oh, S. Shakkottai. International Conference on Machine Learning (ICML), 2022.
  • Sharpened Quasi-Newton Methods: Faster Superlinear Rate and Larger Local Convergence Neighborhood [pdf] Q. Jin, A. Koppel, K. Rajawat, A. Mokhtari. International Conference on Machine Learning (ICML), 2022.
  • The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance [pdf] M. Faw, I. Tziotis, C. Caramanis, A. Mokhtari, S. Shakkottai, R. Ward. Conference on Learning Theory (COLT), 2022.
  • Minimax Optimization: The Case of Convex-Submodular [pdf] (Oral Presentation)

    A. Adibi, A. Mokhtari, H. Hassani. International Conference on Artificial Intelligence and Statistics (AISTATS) 2022.

  • Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation System [pdf] M. Ye, R. Jiang, H. Wang, D. Choudhary, X. Du, B. Bhushanam, A. Mokhtari, A. Kejariwal, and Q. Liu. Conference on Uncertainty in Artificial Intelligence (UAI) 2022.
  • Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity [pdf] A. Reisizadeh, I. Tziotis, H. Hassani, A. Mokhtari, R. Pedarsani. IEEE Journal on Selected Areas in Information Theory (JSAIT), 2022.
  • How Does the Task Landscape Affect MAML Performance? [pdf] L. Collins, A. Mokhtari, S. Shakkottai. Conference on Lifelong Learning Agents (CoLLAs) 2022.

 

2021

  • Exploiting Local Convergence of Quasi-Newton Methods Globally: Adaptive Sample Size Approach [pdf]
    Q. Jin, A. Mokhtari. Neural Information Processing Systems (NeurIPS), 2021.
  • Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks [pdf]
    A. Fallah, A. Mokhtari, A. Ozdaglar. Neural Information Processing Systems (NeurIPS), 2021.
  •  On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning [pdf]
    A. Fallah, A. Mokhtari, A. Ozdaglar. Neural Information Processing Systems (NeurIPS), 2021.
  • Exploiting Shared Representations for Personalized Federated Learning [pdf]
    L. Collins, H. Hassani , A. Mokhtari, S. Shakkottai. International Conference on Machine Learning (ICML), 2021.
  • Federated Learning with Compression: Unified Analysis and Sharp Guarantees [pdf]
    F. Haddadpour, M. M. Kamani, A. Mokhtari, M. Mahdavi. International Conference on Artificial Intelligence and Statistics (AISTATS), 2021.

 

2020

  • Task-Robust Model-Agnostic Meta-Learning [pdf] L. Collins, A. Mokhtari, S. Shakkottai. Neural Information Processing Systems (NeurIPS), 2020.
  • Second Order Optimality in Decentralized Non-Convex Optimization via Perturbed Gradient Tracking [pdf] I. Tziotis,  C. Caramanis, A. Mokhtari. Neural Information Processing Systems (NeurIPS), 2020.
  • Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach [pdf] A. Fallah, A. Mokhtari, A. Ozdaglar. Neural Information Processing Systems (NeurIPS), 2020.
  • Submodular Meta-Learning [pdf] A. Adibi, A. Mokhtari, H. Hassani. Neural Information Processing Systems (NeurIPS), 2020.
  • Stochastic Quasi-Newton Methods [pdf] A. Mokhtari, A. Ribeiro. Proceedings of the IEEE (PIEEE), 2020. [Invited survey paper]
  • Convergence Rate of O(1/k) for Optimistic Gradient and Extra-gradient Methods in Smooth Convex-Concave Saddle Point Problems [pdf] (α-β order) A. Mokhtari, A. Ozdaglar, S. Pattathil. SIAM Journal on Optimization (SIOPT), 2020.
  • Stochastic Conditional Gradient++: (Non-)Convex Minimization and Continuous Submodular Maximization [pdf] (α-β order) H. Hassani, A. Karbasi, A. Mokhtari, Z. Shen. SIAM Journal on Optimization (SIOPT), 2020.
  • High-Dimensional Nonconvex Stochastic Optimization by Doubly Stochastic Successive Convex Approximation [pdf] A. Mokhtari, A. Koppel. IEEE Transactions on Signal Processing (TSP), 2020.
  • Quantized Push-sum for Gossip and Decentralized Optimization over Directed Graph. [pdf] H. Taheri, A. Mokhtari, H. Hassani, R. Pedarsani. Int. Conference on Machine Learning (ICML), 2020.
  • Stochastic Conditional Gradient Methods: From Convex Minimization to Submodular Maximization [pdf] A. Mokhtari, H. Hassani, A. Karbasi. Journal of Machine Learning Research (JMLR), 2020.
  • A Class of Parallel Doubly Stochastic Algorithms for Large-Scale Learning [pdf] A. Mokhtari, A. Koppel, M. Takac, A. Ribeiro. Journal of Machine Learning Research (JMLR), 2020.
  • On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms [pdf] A. Fallah, A. Mokhtari, A. Ozdaglar. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
  • One Sample Stochastic Frank-Wolfe [pdf] M. Zhang, Z. Shen, A. Mokhtari, H. Hassani, A. Karbasi. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
  • FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization [pdf] A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, R. Pedarsani. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.

  • Quantized Frank-Wolfe: Communication-Efficient Distributed Optimization [pdf] M. Zhang, L. Chen, A. Mokhtari, H. Hassani, A. Karbasi. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
  • A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach [pdf] (α-β order) A. Mokhtari, A. Ozdaglar, S. Pattathil. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
  • Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy [pdf] M. Jahani, X. He, C. Ma, A. Mokhtari, D. Mudigere, A. Ribeiro, M. Takac. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
  • DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate [pdf] S. Soori, K. Mischenko, A. Mokhtari, M. Dehnavi, M. Gurbuzbalaban. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2020.

2019

  • Stochastic Continuous Greedy++: When Upper and Lower Bounds Match
    (α-β order)
    H. Hassani, A. Karbasi, A. Mokhtari, Z. Shen. Advances in Neural Information Processing Systems (NeurIPS), 2019.
  • Robust and Communication-Efficient Collaborative Learning [pdf] A. Reisizadeh, H. Taheri, A. Mokhtari, H. Hassani, R. Pedarsani. Advances in Neural Information Processing Systems (NeurIPS), 2019.
  • Efficient Nonconvex Empirical Risk Minimization via Adaptive Sample Size Methods [pdf] A. Mokhtari, A. Ozdaglar, A. Jadbabaie. Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2019.

  • A Newton-based Method for Nonconvex Optimization with Fast Evasion of Saddle Points [pdf] S. Paternain, A. Mokhtari, A. Ribeiro. SIAM Journal on Optimization (SIOPT), 2019.
  • An Exact Quantized Decentralized Gradient Descent Algorithm [pdf] A. Reisizadeh, A. Mokhtari, H. Hassani, R. Pedarsani. IEEE Transactions on Signal Processing (TSP), 2019.
  • A Primal-Dual Quasi-Newton Method for Exact Consensus Optimization [pdf] M. Eisen, A. Mokhtari, A. Ribeiro. IEEE Transactions on Signal Processing (TSP), 2019.
  • Achieving Acceleration in Distributed Optimization via Direct Discretization of the Heavy-Ball ODE [pdf] J. Zhang, C. Uribe, A. Mokhtari, A. Jadbabaie. American Control Conference (ACC), 2019.

2018

  • Escaping Saddle Points in Constrained Optimization [pdf] A. Mokhtari, A. Ozdaglar, A. Jadbabaie
    Advances in Neural Information Processing Systems (NeurIPS), 2018. (Spotlight: Top 4% of the submitted papers)
  • Direct Runge-Kutta Discretization Achieves Acceleration [pdf] J. Zhang, A. Mokhtari, S. Sra, A. Jadbabaie
    Advances in Neural Information Processing Systems (NeurIPS), 2018. (Spotlight: Top 4% of the submitted papers)
  • Decentralized Submodular Maximization: Bridging Discrete and Continuous Settings [pdf] [Supplementary material] A. Mokhtari, H. Hassani, A. Karbasi.
    Int. Conference on Machine Learning (ICML), 2018. (Long talk)
  • Towards More Efficient Stochastic Decentralized Learning: Faster Convergence and Sparse Communication [pdf] [Supp. material]Z. Shen, A. Mokhtari, H. Qian, P. Zhao, T. Zhou
    Int. Conference on Machine Learning (ICML), 2018.
  • Conditional Gradient Method for Stochastic Submodular Maximization: Closing the Gap [pdf] [Supplementary material]A. Mokhtari, H. Hassani, A. Karbasi
    Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2018.
  • Large Scale Empirical Risk Minimization via Truncated Adaptive Newton Method [pdf] [Supplementary material]M. Eisen, A. Mokhtari, A. Ribeiro
    Int. Conference on Artificial Intelligence and Statistics (AISTATS), 2018.
  • IQN: An Incremental Quasi-Newton Method with Local Superlinear Convergence Rate [pdf]A. Mokhtari, M. Eisen, A. Ribeiro
    SIAM Journal on Optimization (SIOPT), 2018.
  • Surpassing Gradient Descent Provably: A Cyclic Incremental Method with Linear Convergence Rate [pdf]

    A. Mokhtari, M. Gürbüzbalaban, A. Ribeiro
    SIAM Journal on Optimization (SIOPT), 2018.
  • Quantized Decentralized Consensus Optimization [pdf]A. Reisizadeh, A. Mokhtari, H. Hassani, R. Pedarsani
    IEEE Conference on Decision and Control (CDC), 2018.
  • A Newton Method for Faster Navigation in Cluttered Environments [pdf] S. Paternain, A. Mokhtari, A. Ribeiro
    IEEE Conference on Decision and Control (CDC), 2018.
  • Parallel Stochastic Successive Convex Approximation Method for Large-Scale Dictionary Learning,
    A. Koppel, A. Mokhtari, A. Ribeiro
    Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2018.

2017

  • Efficient Methods for Large-Scale Empirical Risk Minimization [pdf]A. Mokhtari
    Ph.D. Dissertation, University of Pennsylvania, 2017. (Joseph and Rosaline Wolf Award for Best Doctoral Dissertation)
  • First-Order Adaptive Sample Size Methods to Reduce Complexity of Empirical Risk Minimization [pdf] [Supplementary material] A. Mokhtari, A. Ribeiro
    Advances in Neural Information Processing Systems (NeurIPS), 2017.
  • Decentralized Prediction-Correction Methods for Networked Time-Varying Convex Optimization [pdf]
    A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, A. Ribeiro
    IEEE Transactions on Automatic Control (TAC), 2017.
  • Stochastic Averaging for Constrained Optimization with Application to Online Resource Allocation [pdf]
  • T. Chen, A. Mokhtari, X. Wang, A. Ribeiro, G. B. Giannakis
  • IEEE Transactions on Signal Processing (TSP), 2017.
  • Decentralized Quasi-Newton Methods [pdf]
    M. Eisen, A. Mokhtari, A. Ribeiro
    IEEE Transactions on Signal Processing (TSP), 2017.
  • Network Newton Distributed Optimization Methods [pdf]
    A. Mokhtari, Q. Ling, A. Ribeiro
    IEEE Transactions on Signal Processing (TSP), 2017.
  • A Primal-Dual Quasi-Newton Method for Consensus Optimization [pdf] M. Eisen, A. Mokhtari, A. Ribeiro
    Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2017.
  • An Incremental Quasi-Newton Method with a Local Superlinear Convergence Rate [pdf] A. Mokhtari, M. Eisen, A. Ribeiro
    Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.
  • A Double Incremental Aggregated Gradient Method with Linear Convergence Rate for Large-Scale Optimization [pdf] A. Mokhtari, M. Gürbüzbalaban, A. Ribeiro
    Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.
  • Large-Scale NonConvex Stochastic Optimization by Doubly Stochastic Successive Convex Approximation [pdf] A. Mokhtari, A. Koppel, G. Scutari, A. Ribeiro
    Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.
  • A Diagonal-Augmented Quasi-Newton Method with Application to Factorization Machines [pdf] A. Mokhtari, A. Ingber
    Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2017.

2016

  • DSA: Decentralized Double Stochastic Averaging Gradient Algorithm [pdf] A. Mokhtari, A. Ribeiro
    Journal of Machine Learning Research (JMLR), 2016.
  • Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy [pdf] [Supplementary Material] A. Mokhtari, H. Daneshmand, A. Lucchi, T. Hofmann, A. Ribeiro
    Advances in Neural Information Processing Systems (NeurIPS), 2016.
  • A Decentralized Second-Order Method with Exact Linear Convergence Rate for Consensus Optimization [pdf] A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro.
    IEEE Transactions on Signal and Information Processing over Networks (TSIPN), 2016
  • DQM: Decentralized Quadratically Approximated Alternating Direction Method of Multipliers [pdf] A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro
    IEEE Transactions on Signal Processing (TSP), 2016.
  • A Class of Prediction-Correction Methods for Time-Varying Convex Optimization [pdf] A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, A. Ribeiro.
    IEEE Transactions on Signal Processing (TSP), 2016.
  • Online Optimization in Dynamic Environments: Improved Regret Rates for Strongly Convex Problems [pdf] A. Mokhtari, S. Shahrampour, A. Jadbabaie, A. Ribeiro.
    IEEE Conference on Decision and Control (CDC), 2016.
  • A Decentralized Second-Order Method for Dynamic Optimization [pdf] A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro
    IEEE Conference on Decision and Control (CDC), 2016.
  • A Decentralized Quasi-Newton Method for Dual Formulations of Consensus Optimization [pdf] M. Eisen, A. Mokhtari, A. Ribeiro
    IEEE Conference on Decision and Control (CDC), 2016.
  • A Quasi-Newton Prediction-Correction Method for Decentralized Dynamic Convex Optimization [pdf] A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, A. Ribeiro
    European Control Conference (ECC), 2016.
  • Doubly Random Parallel Stochastic Methods for Large Scale Learning [pdf] A. Mokhtari, A. Koppel, A. Ribeiro
    American Control Conference (ACC), 2016.
  • A Data-driven Approach to Stochastic Network Optimization [pdf] T. Chen, A. Mokhtari, X. Wang, A. Ribeiro, G. B. Giannakis
    IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2016.
  • Decentralized Constrained Consensus Optimization with Primal-Dual Splitting Projection [pdf] H. Zhang, W. Shi, A. Mokhtari, A. Ribeiro, Q. Ling.
    IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2016.
  • An Asynchronous Quasi-Newton Method for Consensus Optimization [pdf] M. Eisen, A. Mokhtari, A. Ribeiro
    IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2016.
  • ESOM: Exact Second-Order Method for Consensus Optimization [pdf] A. Mokhtari, W. Shi, Qing Ling
    Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2016.
  • Doubly Stochastic Algorithms for Large-Scale Optimization [pdf] A. Koppel, A. Mokhtari, A. Ribeiro
    Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2016.

2015

  • Global Convergence of Online Limited Memory BFGS [pdf] A. Mokhtari, A. Ribeiro
    Journal of Machine Learning Research (JMLR), 2015.
  • A Decentralized Prediction-Correction Method for Networked Time-Varying Convex Optimization [pdf] A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, A. Ribeiro
    IEEE Int. Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), 2015.
  • Decentralized Quadratically Approximated Alternating Direction Method of Multipliers [pdf] A. Mokhtari, W. Shi, Q. Ling, A. Ribeiro
    IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2015.
  • Target Tracking with Dynamic Convex Optimization [pdf] A. Koppel, A. Simonetto, A. Mokhtari, G. Leus, A. Ribeiro
    IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2015.
  • Decentralized Double Stochastic Averaging Gradient [pdf] A. Mokhtari, A. Ribeiro
    Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2015.
  • Prediction-Correction Methods for Time-Varying Convex Optimization [pdf] A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, A. Ribeiro
    Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2015.
  • An Approximate Newton Method for Distributed Optimization [pdf] A. Mokhtari, Q. Ling, A. Ribeiro
    Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2015.

2014

  • RES: Regularized Stochastic BFGS Algorithm [pdf] A. Mokhtari, A. Ribeiro.
    IEEE Transactions on Signal Processing (TSP), 2014.
  • Network Newton [pdf] A. Mokhtari, Q. Ling, A. Ribeiro.
    Asilomar Conference on Signals, Systems, and Computers (Asilomar), 2014.
  • A Quasi-Newton Method for Large Scale Support Vector Machines [pdf] A. Mokhtari, A. Ribeiro
    Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2014.

2013

  • Regularized Stochastic BFGS algorithm [pdf] A. Mokhtari, A. Ribeiro.
    IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2013.
  • A Dual Stochastic DFP algorithm for Optimal Resource Allocation in Wireless Systems [pdf] A. Mokhtari, A. Ribeiro.
    IEEE Workshop on Signal Process. Advances in Wireless Communication (SPAWC), 2013.
×