121 citations · 413 across the 32 of their papers we have counts for
33 papers
Random projections beyond zero overlap
Timothy L. H. Wee, Sekhar Tatikonda
A random vector whose norm and overlap (inner product with an independent copy) concentrates is shown to have random low-dimensional projections that are approximately random Gauss…
Local independence in mean-field spin glasses
Timothy L. H. Wee, Sekhar Tatikonda
We present a new approach to local independence in spin glasses, i.e. the phenomenon that any fixed subset of coordinates is asymptotically independent in the thermodynamic limit.…
Surrogate Gap Minimization Improves Sharpness-Aware Training
Juntang Zhuang, Boqing Gong, Liangzhe Yuan +6
The recently proposed Sharpness-Aware Minimization (SAM) improves generalization by minimizing a \textit{perturbed loss} defined as the maximum loss within a neighborhood in the pa…
DESTRESS: Computation-Optimal and Communication-Efficient Decentralized Nonconvex Finite-Sum Optimization
Boyue Li, Zhize Li, Yuejie Chi
Emerging applications in multi-agent environments such as internet-of-things, networked sensing, autonomous systems and federated learning, call for decentralized algorithms for fi…
MALI: A memory efficient and reverse accurate integrator for Neural ODEs
Juntang Zhuang, Nicha C. Dvornek, Sekhar Tatikonda +1
Neural ordinary differential equations (Neural ODEs) are a new family of deep-learning models with continuous depth. However, the numerical estimation of the gradient in the contin…
Multiple-shooting adjoint method for whole-brain dynamic causal modeling
Juntang Zhuang, Nicha Dvornek, Sekhar Tatikonda +3
Dynamic causal modeling (DCM) is a Bayesian framework to infer directed connections between compartments, and has been used to describe the interactions between underlying neural p…
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
Juntang Zhuang, Tommy Tang, Yifan Ding +4
Most popular optimizers for deep learning can be broadly categorized as adaptive methods (e.g. Adam) and accelerated schemes (e.g. stochastic gradient descent (SGD) with momentum).…
Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE
Juntang Zhuang, Nicha Dvornek, Xiaoxiao Li +3
Neural ordinary differential equations (NODEs) have recently attracted increasing attention; however, their empirical performance on benchmark tasks (e.g. image classification) are…
Sparse Regression Codes
Ramji Venkataramanan, Sekhar Tatikonda, Andrew Barron
Developing computationally-efficient codes that approach the Shannon-theoretic limits for communication and compression has long been one of the major goals of information and codi…
Zero-shot Transfer Learning for Semantic Parsing
Javid Dadashkarimi, Alexander Fabbri, Sekhar Tatikonda +1
While neural networks have shown impressive performance on large datasets, applying these models to tasks where little data is available remains a challenging problem. In this pape…
Sequence to Logic with Copy and Cache
Javid Dadashkarimi, Sekhar Tatikonda
Generating logical form equivalents of human language is a fresh way to employ neural architectures where long short-term memory effectively captures dependencies in both encoder a…
Identifiability in Gaussian Graphical Models
De Wen Soh, Sekhar Tatikonda
In high-dimensional graph learning problems, some topological properties of the graph, such as bounded node degree or tree structure, are typically assumed to hold so that the samp…
The Time-Invariant Multidimensional Gaussian Sequential Rate-Distortion Problem Revisited
Photios A. Stavrou, Takashi Tanaka, Sekhar Tatikonda
We revisit the sequential rate-distortion (SRD) trade-off problem for vector-valued Gauss-Markov sources with mean-squared error distortion constraints. We show via a counterexampl…
Accelerated Consensus via Min-Sum Splitting
Patrick Rebeschini, Sekhar Tatikonda
We apply the Min-Sum message-passing protocol to solve the consensus problem in distributed optimization. We show that while the ordinary Min-Sum algorithm does not converge, a mod…
A New Approach to Laplacian Solvers and Flow Problems
Patrick Rebeschini, Sekhar Tatikonda
This paper investigates the behavior of the Min-Sum message passing scheme to solve systems of linear equations in the Laplacian matrices of graphs and to compute electric flows. V…
Scale-free network optimization: foundations and algorithms
Patrick Rebeschini, Sekhar Tatikonda
We investigate the fundamental principles that drive the development of scalable algorithms for network optimization. Despite the significant amount of work on parallel and decentr…
Locality in Network Optimization
Patrick Rebeschini, Sekhar Tatikonda
In probability theory and statistics notions of correlation among random variables, decay of correlation, and bias-variance trade-off are fundamental. In this work we introduce ana…
Scatter Matrix Concordance: A Diagnostic for Regressions on Subsets of Data
Michael J. Kane, Bryan Lewis, Sekhar Tatikonda +1
Linear regression models depend directly on the design matrix and its properties. Techniques that efficiently estimate model coefficients by partitioning rows of the design matrix…
The Rate-Distortion Function and Excess-Distortion Exponent of Sparse Regression Codes with Optimal Encoding
Ramji Venkataramanan, Sekhar Tatikonda
This paper studies the performance of sparse regression codes for lossy compression with the squared-error distortion criterion. In a sparse regression code, codewords are linear c…
Loopy Belief Propogation and Gibbs Measures
Sekhar Tatikonda, Michael I. Jordan
We address the question of convergence in the loopy belief propagation (LBP) algorithm. Specifically, we relate convergence of LBP to the existence of a weak limit for a sequence o…
Sparse Regression Codes for Multi-terminal Source and Channel Coding
Ramji Venkataramanan, Sekhar Tatikonda
We study a new class of codes for Gaussian multi-terminal source and channel coding. These codes are designed using the statistical framework of high-dimensional linear regression…
Message-Passing Algorithms for Quadratic Minimization
Nicholas Ruozzi, Sekhar Tatikonda
Gaussian belief propagation (GaBP) is an iterative algorithm for computing the mean of a multivariate Gaussian distribution, or equivalently, the minimum of a multivariate positive…
Lossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decoding
Ramji Venkataramanan, Tuhin Sarkar, Sekhar Tatikonda
We propose computationally efficient encoders and decoders for lossy compression using a Sparse Regression Code. The codebook is defined by a design matrix and codewords are struct…
Rewritable storage channels with hidden state
Ramji Venkataramanan, Sekhar Tatikonda, Luis Lastras +1
Many storage channels admit reading and rewriting of the content at a given cost. We consider rewritable channels with a hidden state which models the unknown characteristics of th…