1 citations · 1 across the 18 of their papers we have counts for
19 papers
Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers
Ligong Han, Kai Xu, Hao Wang +3
Fully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlin…
sGPO: Trading Inference FLOPs for Training Efficiency in RLVR
Shivchander Sudalairaj, Kai Xu, Akash Srivastava +1
Standard Reinforcement Learning with Verifiable Rewards (RLVR) training allocates a fixed rollout budget to every query, without regard for what each query's difficulty means for t…
Intrinsic Selection and Particle Resampling for Inference-Time Scaling Beyond Domain Verifiability
Giorgio Giannone, Mustafa Eyceoz, Shabana Baig +5
Inference-Time Scaling (ITS) has largely succeeded in verifiable domains like math and coding, where cheap verification enables scalable output selection. However, extending ITS to…
Turning Intent into Specifications: A Benchmark and an Interactive User-Assistant Agent
Hao Wang, Ligong Han, Kai Xu +1
Today's agents are highly effective at implementing well-scoped software design plans, but user intent is often vague and admits multiple equally valid solutions. In this paper, we…
SNLP: Layer-Parallel Inference via Structured Newton Corrections
Ligong Han, Kai Xu, Hao Wang +1
Autoregressive language models execute Transformer layers sequentially, creating a latency bottleneck that is not removed by conventional tensor or pipeline parallelism. We study w…
GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback
Giorgio Giannone, Anna Clare Doris, Amin Heyrani Nobari +3
Generating executable CAD programs from images requires alignment between visual geometry and symbolic program representations, a capability that current methods fail to learn reli…
Few-Step Diffusion Language Models via Trajectory Self-Distillation
Tunyu Zhang, Xinxi Zhang, Ligong Han +9
Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding. However, realizing this poten…
Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling
Giorgio Giannone, Guangxuan Xu, Nikhil Shivakumar Nayak +4
Inference-Time Scaling (ITS) improves language models by allocating more computation at generation time. Particle Filtering (PF) has emerged as a strong ITS method for complex math…
Inference-Time Scaling of Diffusion Language Models via Trajectory Refinement
Meihua Dang, Jiaqi Han, Minkai Xu +3
Discrete diffusion models have recently emerged as strong alternatives to autoregressive language models, matching their performance through large-scale training. However, inferenc…
Hopscotch: Discovering and Skipping Redundancies in Language Models
Mustafa Eyceoz, Nikhil Shivakumar Nayak, Hao Wang +2
Modern causal language models stack many attention blocks to improve performance, but not all blocks are necessary for every task. We propose Hopscotch, a simple yet effective meth…
Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
Nikhil Shivakumar Nayak, Krishnateja Killamsetty, Ligong Han +8
Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. E…
SQuat: Subspace-orthogonal KV Cache Quantization
Hao Wang, Ligong Han, Kai Xu +1
The key-value (KV) cache accelerates LLMs decoding by storing KV tensors from previously generated tokens. It reduces redundant computation at the cost of increased memory usage. T…
Activation-Informed Merging of Large Language Models
Amin Heyrani Nobari, Kaveh Alim, Ali ArjomandBigdeli +3
Model merging, a method that combines the parameters and embeddings of multiple fine-tuned large language models (LLMs), offers a promising approach to enhance model performance ac…
Dr. SoW: Density Ratio of Strong-over-weak LLMs for Reducing the Cost of Human Annotation in Preference Tuning
Guangxuan Xu, Kai Xu, Shivchander Sudalairaj +2
Preference tuning relies on high-quality human preference data, which is often expensive and time-consuming to gather. In this paper, we introduce Dr.SoW (Density Ratio of Strong o…
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models
Xiaoxiao He, Quan Dao, Ligong Han +14
Discrete diffusion models have achieved success in tasks like image generation and masked language modeling but face limitations in controlled content editing. We introduce DICE (D…
Urban context and delivery performance: Modelling service time for cargo bikes and vans across diverse urban environments
Maxwell Schrader, Navish Kumar, Esben Sørig +5
Light goods vehicles (LGV) used extensively in the last mile of delivery are one of the leading polluters in cities. Cargo-bike logistics and Light Electric Vehicles (LEVs) have be…
Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models
Xinxi Zhang, Song Wen, Ligong Han +6
Adapting large-scale pre-trained generative models in a parameter-efficient manner is gaining traction. Traditional methods like low rank adaptation achieve parameter efficiency by…
Differentially Private Synthetic Data Generation for Relational Databases
Kaveh Alimohammadi, Hao Wang, Ojas Gulati +2
Existing differentially private (DP) synthetic data generation mechanisms typically assume a single-source table. In practice, data is often distributed across multiple tables with…
Private Synthetic Data Meets Ensemble Learning
Haoyuan Sun, Navid Azizan, Akash Srivastava +1
When machine learning models are trained on synthetic data and then deployed on real data, there is often a performance drop due to the distribution shift between synthetic and rea…