614 citations · 917 across the 13 of their papers we have counts for
13 papers
KernelSight-LM: A Kernel-Level LLM Inference Simulator
Xiteng Yao, Taeho Kim, Hengzhi Pei +7
As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to m…
The Amazon Nova Family of Models: Technical Report and Model Card
Amazon AGI, Aaron Langford, Aayush Shah +783
We present Amazon Nova, a new generation of state-of-the-art foundation models that deliver frontier intelligence and industry-leading price performance. Amazon Nova Pro is a highl…
Understanding Silent Data Corruption in LLM Training
Jeffrey Ma, Hengzhi Pei, Leonard Lausen +1
As the scale of training large language models (LLMs) increases, one emergent failure is silent data corruption (SDC), where hardware produces incorrect computations without explic…
Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning
Soumajyoti Sarkar, Leonard Lausen, Volkan Cevher +3
Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. These models use conditionally activated feedforward subnetworks…
Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Tasks
Soumajyoti Sarkar, Leonard Lausen
Tables stored in databases and tables which are present in web pages and articles account for a large part of semi-structured data that is available on the internet. It then become…
HYTREL: Hypergraph-enhanced Tabular Data Representation Learning
Pei Chen, Soumajyoti Sarkar, Leonard Lausen +4
Language models pretrained on large collections of tabular data have demonstrated their effectiveness in several downstream tasks. However, many of these models do not take into ac…
Large Language Models of Code Fail at Completing Code with Potential Bugs
Tuan Dinh, Jinman Zhao, Samson Tan +4
Large language models of code (Code-LLMs) have recently brought tremendous advances to code completion, a fundamental feature of programming assistance and code intelligence. Howev…
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
Hengzhi Pei, Jinman Zhao, Leonard Lausen +2
Pretrained code language models have enabled great progress towards program synthesis. However, common approaches only consider in-file local context and thus miss information and…
Parameter and Data Efficient Continual Pre-training for Robustness to Dialectal Variance in Arabic
Soumajyoti Sarkar, Kaixiang Lin, Sailik Sengupta +3
The use of multilingual language models for tasks in low and high-resource languages has been a success story in deep learning. In recent times, Arabic has been receiving widesprea…
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning
Vishakh Padmakumar, Leonard Lausen, Miguel Ballesteros +3
Recent work has found that multi-task training with a large number of diverse tasks can uniformly improve downstream performance on unseen target tasks. In contrast, literature on…
GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing
Jian Guo, He He, Tong He +13
We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating). These toolkits provide state-of-…
NSML: A Machine Learning Platform That Enables You to Focus on Your Models
Nako Sung, Minkyu Kim, Hyunwoo Jo +8
Machine learning libraries such as TensorFlow and PyTorch simplify model implementation. However, researchers are still required to perform a non-trivial amount of manual tasks suc…
Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
Xingjian Shi, Zhihan Gao, Leonard Lausen +4
With the goal of making high-resolution forecasts of regional rainfall, precipitation nowcasting has become an important and fundamental technology underlying various public servic…