196 citations · 450 across the 15 of their papers we have counts for
17 papers
P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
Mude Hui, Xin Huang, Jaime Campos Salas +5
Reasoning LLMs produce longer outputs, requiring speculative decoding drafters trained on extended sequences. Parallel drafting - predicting multiple tokens per forward pass - offe…
PPLqa: An Unsupervised Information-Theoretic Quality Metric for Comparing Generative Large Language Models
Gerald Friedland, Xin Huang, Yueying Cui +3
We propose PPLqa, an easy to compute, language independent, information-theoretic metric to measure the quality of responses of generative Large Language Models (LLMs) in an unsupe…
Representation Projection Invariance Mitigates Representation Collapse
Anastasia Razdaibiedina, Ashish Khetan, Zohar Karnin +3
Fine-tuning contextualized representations learned by pre-trained language models remains a prevalent practice in NLP. However, fine-tuning can lead to representation degradation (…
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection
Xin Huang, Ashish Khetan, Rene Bidart +1
Transformer-based language models such as BERT have achieved the state-of-the-art performance on various NLP tasks, but are computationally prohibitive. A recent line of works use…
TabTransformer: Tabular Data Modeling Using Contextual Embeddings
Xin Huang, Ashish Khetan, Milan Cvitkovic +1
We propose TabTransformer, a novel deep tabular data modeling architecture for supervised and semi-supervised learning. The TabTransformer is built upon self-attention based Transf…
PruneNet: Channel Pruning via Global Importance
Ashish Khetan, Zohar Karnin
Channel pruning is one of the predominant approaches for accelerating deep neural networks. Most existing pruning methods either train from scratch with a sparsity inducing term su…
schuBERT: Optimizing Elements of BERT
Ashish Khetan, Zohar Karnin
Transformers \citep{vaswani2017attention} have gradually become a key component for many state-of-the-art natural language representation models. A recent Transformer based model-…
Robust conditional GANs under missing or uncertain labels
Kiran Koshy Thekumparampil, Sewoong Oh, Ashish Khetan
Matching the performance of conditional Generative Adversarial Networks with little supervision is an important task, especially in venturing into new domains. We design a new trai…
DARC: Differentiable ARchitecture Compression
Shashank Singh, Ashish Khetan, Zohar Karnin
In many learning situations, resources at inference time are significantly more constrained than resources at training time. This paper studies a general paradigm, called Different…
Number of Connected Components in a Graph: Estimation via Counting Patterns
Ashish Khetan, Harshay Shah, Sewoong Oh
Due to the limited resources and the scale of the graphs in modern datasets, we often get to observe a sampled subgraph of a larger original graph of interest, whether it is the wo…
Robustness of Conditional GANs to Noisy Labels
Kiran Koshy Thekumparampil, Ashish Khetan, Zinan Lin +1
We study the problem of learning conditional generators from noisy labeled samples, where the labels are corrupted by random noise. A standard training of conditional GANs will not…
Learning From Noisy Singly-labeled Data
Ashish Khetan, Zachary C. Lipton, Anima Anandkumar
Supervised learning depends on annotated examples, which are taken to be the \emph{ground truth}. But these labels often come from noisy crowdsourcing platforms, like Amazon Mechan…
PacGAN: The power of two samples in generative adversarial networks
Zinan Lin, Ashish Khetan, Giulia Fanti +1
Generative adversarial networks (GANs) are innovative techniques for learning generative models of complex data distributions from samples. Despite remarkable recent improvements i…
Spectrum Estimation from a Few Entries
Ashish Khetan, Sewoong Oh
Singular values of a data in a matrix form provide insights on the structure of the data, the effective dimensionality, and the choice of hyper-parameters on higher-level data anal…
Computational and Statistical Tradeoffs in Learning to Rank
Ashish Khetan, Sewoong Oh
For massive and heterogeneous modern datasets, it is of fundamental interest to provide guarantees on the accuracy of estimation when computational resources are limited. In the ap…
Achieving Budget-optimality with Adaptive Schemes in Crowdsourcing
Ashish Khetan, Sewoong Oh
Crowdsourcing platforms provide marketplaces where task requesters can pay to get labels on their data. Such markets have emerged recently as popular venues for collecting annotati…
Data-driven Rank Breaking for Efficient Rank Aggregation
Ashish Khetan, Sewoong Oh
Rank aggregation systems collect ordinal preferences from individuals to produce a global ranking that represents the social preference. Rank-breaking is a common practice to reduc…