35 citations · 82 across the 23 of their papers we have counts for
23 papers
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +572
We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 t…
Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
Hayate Iso, Tiyasa Mitra, Sudipta Mondal +15
RL post-training of frontier language models is increasingly bottlenecked by autoregressive rollout generation, making rollout acceleration a central systems challenge. Many existi…
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani +544
We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemo…
The Rarity Blind Spot: A Framework for Evaluating Statistical Reasoning in LLMs
Seiji Maekawa, Hayate Iso, Nikita Bhutani
Effective decision-making often relies on identifying what makes each candidate distinctive. While existing benchmarks for LLMs emphasize retrieving or summarizing information rele…
Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education
Hayate Iso, Pouya Pezeshkpour, Nikita Bhutani +1
Large Language Models (LLMs) offer the potential to automate hiring by matching job descriptions with candidate resumes, streamlining recruitment processes, and reducing operationa…
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization
Catarina G. Belem, Pouya Pezeshkpour, Hayate Iso +3
Although many studies have investigated and reduced hallucinations in large language models (LLMs) for single-document tasks, research on hallucination in multi-document summarizat…
Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data
Seiji Maekawa, Hayate Iso, Nikita Bhutani
The rapid increase in textual information means we need more efficient methods to sift through, organize, and understand it all. While retrieval-augmented generation (RAG) models e…
A Blueprint Architecture of Compound AI Systems for Enterprise
Eser Kandogan, Sajjadur Rahman, Nikita Bhutani +11
Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goa…
Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models
Seiji Maekawa, Hayate Iso, Sairam Gurajada +1
While large language models (LMs) demonstrate remarkable performance, they encounter challenges in providing accurate responses when queried for information beyond their pre-traine…
AmbigNLG: Addressing Task Ambiguity in Instruction for NLG
Ayana Niwa, Hayate Iso
We introduce AmbigNLG, a novel task designed to tackle the challenge of task ambiguity in instructions for Natural Language Generation (NLG). Ambiguous instructions often impede th…
Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks
Pouya Pezeshkpour, Hayate Iso, Thom Lake +2
Numerous HR applications are centered around resumes and job descriptions. While they can benefit from advancements in NLP, particularly large language models, their real-world ado…
XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updates
Haopeng Zhang, Hayate Iso, Sairam Gurajada +1
Text editing is a crucial task of modifying text to better align with user intents. However, existing text editing benchmark datasets contain only coarse-grained instructions and l…
Less is More for Long Document Summary Evaluation by LLMs
Yunshu Wu, Hayate Iso, Pouya Pezeshkpour +2
Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational costs and the Lost-in-the-Middle prob…
Zero-shot Triplet Extraction by Template Infilling
Bosung Kim, Hayate Iso, Nikita Bhutani +3
The task of triplet extraction aims to extract pairs of entities and their corresponding relations from unstructured text. Most existing methods train an extraction model on traini…
Noisy Pairing and Partial Supervision for Stylized Opinion Summarization
Hayate Iso, Xiaolan Wang, Yoshi Suhara
Opinion summarization research has primarily focused on generating summaries reflecting important opinions from customer reviews without paying much attention to the writing style.…
AutoTemplate: A Simple Recipe for Lexically Constrained Text Generation
Hayate Iso
Lexically constrained text generation is one of the constrained text generation tasks, which aims to generate text that covers all the given constraint lexicons. While the existing…
Comparative Opinion Summarization via Collaborative Decoding
Hayate Iso, Xiaolan Wang, Stefanos Angelidis +1
Opinion summarization focuses on generating summaries that reflect popular subjective information expressed in multiple online reviews. While generated summaries offer general and…
Biomedical Entity Linking with Contrastive Context Matching
Shogo Ujiie, Hayate Iso, Eiji Aramaki
We introduce BioCoM, a contrastive learning framework for biomedical entity linking that uses only two resources: a small-sized dictionary and a large number of raw biomedical arti…
End-to-end Biomedical Entity Linking with Span-based Dictionary Matching
Shogo Ujiie, Hayate Iso, Shuntaro Yada +2
Disease name recognition and normalization, which is generally called biomedical entity linking, is a fundamental process in biomedical text mining. Recently, neural joint learning…
Convex Aggregation for Opinion Summarization
Hayate Iso, Xiaolan Wang, Yoshihiko Suhara +2
Recent advances in text autoencoders have significantly improved the quality of the latent space, which enables models to generate grammatical and consistent text from aggregated l…
Fact-based Text Editing
Hayate Iso, Chao Qiao, Hang Li
We propose a novel text editing task, referred to as \textit{fact-based text editing}, in which the goal is to revise a given document to better describe the facts in a knowledge b…
Learning to Select, Track, and Generate for Data-to-Text
Hayate Iso, Yui Uehara, Tatsuya Ishigaki +6
We propose a data-to-text generation model with two modules, one for tracking and the other for text generation. Our tracking module selects and keeps track of salient information…
Density Estimation for Geolocation via Convolutional Mixture Density Network
Hayate Iso, Shoko Wakamiya, Eiji Aramaki
Nowadays, geographic information related to Twitter is crucially important for fine-grained applications. However, the amount of geographic information avail- able on Twitter is lo…