196 citations · 450 across the 15 of their papers we have counts for
17 papers
P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
Mude Hui, Xin Huang, Jaime Campos Salas +5
Reasoning LLMs produce longer outputs, requiring speculative decoding drafters trained on extended sequences. Parallel drafting - predicting multiple tokens per forward pass - offe…
PPLqa: An Unsupervised Information-Theoretic Quality Metric for Comparing Generative Large Language Models
Gerald Friedland, Xin Huang, Yueying Cui +3
We propose PPLqa, an easy to compute, language independent, information-theoretic metric to measure the quality of responses of generative Large Language Models (LLMs) in an unsupe…
Representation Projection Invariance Mitigates Representation Collapse
Anastasia Razdaibiedina, Ashish Khetan, Zohar Karnin +3
Fine-tuning contextualized representations learned by pre-trained language models remains a prevalent practice in NLP. However, fine-tuning can lead to representation degradation (…
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection
Xin Huang, Ashish Khetan, Rene Bidart +1
Transformer-based language models such as BERT have achieved the state-of-the-art performance on various NLP tasks, but are computationally prohibitive. A recent line of works use…
TabTransformer: Tabular Data Modeling Using Contextual Embeddings
Xin Huang, Ashish Khetan, Milan Cvitkovic +1
We propose TabTransformer, a novel deep tabular data modeling architecture for supervised and semi-supervised learning. The TabTransformer is built upon self-attention based Transf…
PruneNet: Channel Pruning via Global Importance
Ashish Khetan, Zohar Karnin
Channel pruning is one of the predominant approaches for accelerating deep neural networks. Most existing pruning methods either train from scratch with a sparsity inducing term su…