19 papers
Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning
Soumajyoti Sarkar, Leonard Lausen, Volkan Cevher +3
Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. These models use conditionally activated feedforward subnetworks…
Learning to Generate Answers with Citations via Factual Consistency Models
Rami Aly, Zhiqiang Tang, Samson Tan +1
Large Language Models (LLMs) frequently hallucinate, impeding their reliability in mission-critical situations. One approach to address this issue is to provide citations to releva…
Inference Optimization of Foundation Models on AI Accelerators
Youngsuk Park, Kailash Budhathoki, Liangfu Chen +7
Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI across various industries. Industry an…
Parameter-Efficient Tuning Large Language Models for Graph Representation Learning
Qi Zhu, Da Zheng, Xiang Song +4
Text-rich graphs, which exhibit rich textual information on nodes and edges, are prevalent across a wide range of real-world business applications. Large Language Models (LLMs) hav…
Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
Costas Mavromatis, Petros Karypis, George Karypis
Fusing knowledge from multiple Large Language Models (LLMs) can combine their diverse strengths to achieve improved performance on a given task. However, current fusion approaches…
OmniMatch: Effective Self-Supervised Any-Join Discovery in Tabular Data Repositories
Christos Koutras, Jiani Zhang, Xiao Qin +5
How can we discover join relationships among columns of tabular data in a data repository? Can this be done effectively when metadata is missing? Traditional column matching works…