1 citations · 1 across the 19 of their papers we have counts for
20 papers
An Identifiability Theory of Masked Prediction: Mode Blindness and Mask Schedules
Yichao Cai, Javen Qinfeng Shi
Masked prediction learns to infer missing variables from visible context selected by a mask schedule. When does small excess risk guarantee recovery of the true joint distribution?…
Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs
Hamed Damirchi, Ignacio Meza De la Jara, Damith Ranasinghe +2
As language models are increasingly used for tasks that require verifiable reasoning, reliably distinguishing sound reasoning from flawed reasoning has become an important practica…
Link prediction on multi-relational graphs from an influence propagation perspective
Zidu Yin, Yuankai Qi, Dong Gong +3
Predicting the existence and type of links (edges) between nodes in a multi-relational graph is key for applications from social interaction prediction to knowledge relationship id…
Boundary Embedding Shaping with Adaptive Contrastive Learning for Graph Structural Disentanglement
Jiaqing Chen, Zidu Yin, Yichao Cai +4
Graph neural networks (GNNs) excel at aggregating neighbor information for classification, yet their performance is hindered by graph structural entanglement, where spurious correl…
Learning Latent Dynamical Causal Processes for Single-Cell Perturbation Prediction
Wenkang Jiang, Yuhang Liu, Erdun Gao +3
Single-cell perturbation prediction aims to infer how cells respond to unseen interventions and to achieve out-of-distribution (OOD) generalization, providing a computational route…
What Makes a Representation Good for Single-Cell Perturbation Prediction?
Wenkang Jiang, Yuhang Liu, Yichao Cai +5
Single-cell perturbation modeling is fundamental for understanding and predicting cellular responses to genetic perturbations. However, existing approaches, from causal representat…
Chem4DLLM: 4D Multimodal LLMs for Chemical Dynamics Understanding
Xinyu Li, Zhen Zhang, Qi Chen +3
Existing chemical understanding tasks primarily rely on static molecular representations, limiting their ability to model inherently dynamic phenomena such as bond breaking or conf…
Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
Hamed Damirchi, Ignacio Meza De la Jara, Ehsan Abbasnejad +3
Existing explainability methods for Large Language Models (LLMs) typically treat hidden states as static points in activation space, assuming that correct and incorrect inferences…
Concept Component Analysis: A Principled Approach for Concept Extraction in LLMs
Yuhang Liu, Erdun Gao, Dong Gong +2
Developing human understandable interpretation of large language models (LLMs) becomes increasingly critical for their deployment in essential domains. Mechanistic interpretability…
The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence
Yichao Cai, Zhen Zhang, Yuhang Liu +1
While InfoNCE underlies modern contrastive learning, its geometric mechanisms remain under-characterized beyond the canonical alignment--uniformity decomposition. We develop a meas…
Decomposing Task Vectors for Refined Model Editing
Hamed Damirchi, Ehsan Abbasnejad, Zhen Zhang +1
Large pre-trained models have transformed machine learning, yet adapting these models effectively to exhibit precise, concept-specific behaviors remains a significant challenge. Ta…
The Quest for Winning Tickets in Low-Rank Adapters
Hamed Damirchi, Cristian Rodriguez-Opazo, Ehsan Abbasnejad +2
The Lottery Ticket Hypothesis (LTH) suggests that over-parameterized neural networks contain sparse subnetworks ("winning tickets") capable of matching full model performance when…
Causal Disentanglement and Cross-Modal Alignment for Enhanced Few-Shot Learning
Tianjiao Jiang, Zhen Zhang, Yuhang Liu +1
Few-shot learning (FSL) often requires effective adaptation of models using limited labeled data. However, most existing FSL methods rely on entangled representations, requiring th…
Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models
Bahram Mohammadi, Ehsan Abbasnejad, Yuankai Qi +3
The remote embodied referring expression (REVERIE) task requires an agent to navigate through complex indoor environments and localize a remote object specified by high-level instr…
On the Value of Cross-Modal Misalignment in Multimodal Representation Learning
Yichao Cai, Yuhang Liu, Erdun Gao +4
Multimodal representation learning, exemplified by multimodal contrastive learning (MMCL) using image-text pairs, aims to learn powerful representations by aligning cues across mod…
Analytic DAG Constraints for Differentiable DAG Learning
Zhen Zhang, Ignavier Ng, Dong Gong +6
Recovering the underlying Directed Acyclic Graph (DAG) structures from observational data presents a formidable challenge, partly due to the combinatorial nature of the DAG-constra…
I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
Yuhang Liu, Dong Gong, Yichao Cai +6
Recent empirical evidence shows that LLM representations encode human-interpretable concepts. Nevertheless, the mechanisms by which these representations emerge remain largely unex…
MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models
Hongrong Cheng, Miao Zhang, Javen Qinfeng Shi
As Large Language Models (LLMs) grow dramatically in size, there is an increasing trend in compressing and speeding up these models. Previous studies have highlighted the usefulnes…
Augmented Commonsense Knowledge for Remote Object Grounding
Bahram Mohammadi, Yicong Hong, Yuankai Qi +3
The vision-and-language navigation (VLN) task necessitates an agent to perceive the surroundings, follow natural language instructions, and act in photo-realistic unseen environmen…
Highway Graph to Accelerate Reinforcement Learning
Zidu Yin, Zhen Zhang, Dong Gong +2
Reinforcement Learning (RL) algorithms often struggle with low training efficiency. A common approach to address this challenge is integrating model-based planning algorithms, such…