12 citations · 53 across the 37 of their papers we have counts for
39 papers
Decoupling Memory from Context: Structured Memory for Token-Efficient Test-Time Continual Learning
Yehya Farhat, Michael Desmond, Anastasios Kyrillidis
Large language models (LLMs) are increasingly deployed in enterprise, scientific, and medical applications, where agents must incorporate domain-specific knowledge and adapt from e…
Understanding the Subspace Stabilization of the Hessian and Gradient Covariance Matrix
Fangshuo Liao, Anastasios Kyrillidis
The phenomenon of the top subspace stabilization of the Hessian matrix is an surprising and critical aspect in study of the second-order information of neural network training. Pri…
Identity-Paired Progressive Depth Training: When Trainability Persists Beyond Expressibility
Athanasios Hadjidimoulas, Tirthak Patel, Anastasios Kyrillidis
Variational Quantum Algorithms (VQAs) are a leading paradigm for near-term quantum computing, yet their training suffers from sensitivity to circuit depth, initialization, and land…
SuperThoughts: Reasoning Tokens in Superposition
Zheyang Xiong, Shivam Garg, Max Yu +4
Long Chain-of-Thought (CoT) reasoning improves LLM problem-solving but is computationally expensive due to sequential token generation. While recent works explore reasoning in cont…
One Model, Two Roles: Emergent Specialization in a Shared Recurrent Transformer
Jucheng Shen, Barbara Su, Anastasios Kyrillidis
Can a shared-weight recurrent Transformer develop distinct internal roles without being partitioned into separate modules? We study this in Asymmetric Input Recurrence (AIR), a min…
AdaPaD: Adaptive Parallel Deflation for PEFT with Self-Correcting Rank Discovery
Barbara Su, Fangshuo Liao, Anastasios Kyrillidis
Fine-tuning large language models with LoRA requires choosing a rank r before training starts. Existing approaches either extract rank-1 components sequentially, freezing each comp…
SGD at the Edge of Stability: The Stochastic Sharpness Gap
Fangshuo Liao, Afroditi Kolomvaki, Anastasios Kyrillidis
When training neural networks with full-batch gradient descent (GD) and step size , the largest eigenvalue of the Hessian -- the sharpness -- rises to an…
Convergence Analysis of Two-Layer Neural Networks under Gaussian Input Masking
Afroditi Kolomvaki, Fangshuo Liao, Evan Dramko +2
We investigate the convergence guarantee of two-layer neural network training with Gaussian randomly masked inputs. This scenario corresponds to Gaussian dropout at the input level…
GHOST: Unmasking Phantom States in Mamba2 via Grouped Hidden-state Output-aware Selection & Truncation
Michael Menezes, Anastasios Kyrillidis
While Mamba2's expanded state dimension enhances temporal modeling, it incurs substantial inference overhead that saturates bandwidth during autoregressive generation. Standard pru…
On The Finetuning of MLIPs Through the Lens of Iterated Maps With BPTT
Evan Dramko, Yizhi Zhu, Aleksandar Krivokapic +4
Accurate structural relaxation is critical for advanced materials design. Traditional approaches built on physics-derived first-principles calculations are computationally expensiv…
Completion of partial structures using Patterson maps with the CrysFormer machine learning model
Tom Pan, Evan Dramko, Mitchell D. Miller +2
Protein structure determination has long been one of the primary challenges of structural biology, to which deep machine learning (ML)-based approaches have increasingly been appli…
TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Training
Michael Menezes, Barbara Su, Xinze Feng +3
We introduce TwIST, a distributed training framework for efficient large language model (LLM) sparsification. TwIST trains multiple subnetworks in parallel, periodically aggregates…
Three Birds with One Stone: Improving Performance, Convergence, and System Throughput with Nest
Yuqian Huo, David Quiroga, Anastasios Kyrillidis +1
Variational quantum algorithms (VQAs) have the potential to demonstrate quantum utility on near-term quantum computers. However, these algorithms often get executed on the highest-…
Guided by the Experts: Provable Feature Learning Dynamic of Soft-Routed Mixture-of-Experts
Fangshuo Liao, Anastasios Kyrillidis
Mixture-of-Experts (MoE) architectures have emerged as a cornerstone of modern AI systems. In particular, MoEs route inputs dynamically to specialized experts whose outputs are agg…
ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs
Evan Dramko, Yihuang Xiong, Yizhi Zhu +4
Point defects play a central role in driving the properties of materials. First-principles methods are widely used to compute defect energetics and structures, including at scale f…
One Rank at a Time: Cascading Error Dynamics in Sequential Learning
Mahtab Alizadeh Vandchali, Fangshuo, Liao +1
Sequential learning -- where complex tasks are broken down into simpler, hierarchical components -- has emerged as a paradigm in AI. This paper views sequential learning through th…
Exploring How LLMs Capture and Represent Domain-Specific Knowledge
Mirian Hipolito Garcia, Camille Couturier, Daniel Madrigal Diaz +5
We study whether Large Language Models (LLMs) inherently capture domain-specific nuances in natural language. Our experiments probe the domain sensitivity of LLMs by examining thei…
Quantum EigenGame for excited state calculation
David Quiroga, Jason Han, Anastasios Kyrillidis
Computing the excited states of a given Hamiltonian is computationally hard for large systems, but methods that do so using quantum computers scale tractably. This problem is equiv…
RecCrysFormer: Refined Protein Structural Prediction from 3D Patterson Maps via Recycling Training Runs
Tom Pan, Evan Dramko, Mitchell D. Miller +2
Determining protein structures at an atomic level remains a significant challenge in structural biology. We introduce , a hybrid model that exploits the str…
Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflation
Fangshuo Liao, Wenyi Su, Anastasios Kyrillidis
We study a distributed Principal Component Analysis (PCA) framework where each worker targets a distinct eigenvector and refines its solution by updating from intermediate solution…
A Catalyst Framework for the Quantum Linear System Problem via the Proximal Point Algorithm
Junhyung Lyle Kim, Nai-Hui Chia, Anastasios Kyrillidis
Solving systems of linear equations is a fundamental problem, but it can be computationally intensive for classical algorithms in high dimensions. Existing quantum algorithms can a…
Better Schedules for Low Precision Training of Deep Neural Networks
Cameron R. Wolfe, Anastasios Kyrillidis
Low precision training can significantly reduce the computational overhead of training deep neural networks (DNNs). Though many such techniques exist, cyclic precision training (CP…
Grover-QAOA for 3-SAT: Quadratic Speedup, Fair-Sampling, and Parameter Clustering
Zewen Zhang, Roger Paredes, Bhuvanesh Sundar +5
The SAT problem is a prototypical NP-complete problem of fundamental importance in computational complexity theory with many applications in science and engineering; as such, it ha…
CrysFormer: Protein Structure Prediction via 3d Patterson Maps and Partial Structure Attention
Chen Dun, Qiutai Pan, Shikai Jin +4
Determining the structure of a protein has been a decades-long open question. A protein's three-dimensional structure often poses nontrivial computation costs, when classical simul…