15 citations · 20 across the 18 of their papers we have counts for
20 papers
EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation
Zeru Shi, Jindong Jiang, Ryo Hachiuma +10
Complex and long-horizon agent tasks often rely on frontier models deployed in the cloud to achieve strong performance. However, using these models for every step of a long-horizon…
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani +544
We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemo…
GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
Shih-Yang Liu, Xin Dong, Ximing Lu +10
As language models become increasingly capable, users expect them to provide not only accurate responses but also behaviors aligned with diverse human preferences across a variety…
ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
Hongjin Su, Shizhe Diao, Ximing Lu +13
Large language models are powerful generalists, yet solving deep and complex problems such as those of the Humanity's Last Exam (HLE) remains both conceptually challenging and comp…
Universal Deep Research: Bring Your Own Model and Strategy
Peter Belcak, Pavlo Molchanov
Deep research tools are among the most impactful and most commonly encountered agentic systems today. We observe, however, that each deep research agent introduced so far is hard-c…
Small Language Models are the Future of Agentic AI
Peter Belcak, Greg Heinrich, Shizhe Diao +5
Large language models (LLMs) are often praised for exhibiting near-human performance on a wide range of tasks and valued for their ability to hold a general conversation. The rise…
Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
Peter Belcak, Greg Heinrich, Jan Kautz +1
Finetuning language models for a new domain inevitably leads to the deterioration of their general performance. This becomes more pronounced the more limited the finetuning data re…
Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
Shizhe Diao, Yu Yang, Yonggan Fu +11
Pre-training datasets are typically collected from web content and lack inherent domain divisions. For instance, widely used datasets like Common Crawl do not include explicit doma…
Text Compression for Efficient Language Generation
David Gu, Peter Belcak, Roger Wattenhofer
We challenge the prevailing assumption that LLMs must rely fully on sub-word tokens for high-quality text generation. To this end, we propose the "Generative Pretrained Thoughtform…
Tiny Transformers Excel at Sentence Compression
Peter Belcak, Roger Wattenhofer
It is staggering that words of the English language, which are on average represented by 5--6 bytes of ASCII, require as much as 24 kilobytes when served to large language models.…
Exponentially Faster Language Modelling
Peter Belcak, Roger Wattenhofer
Language models only really need to use an exponential fraction of their neurons for individual inferences. As proof, we present UltraFastBERT, a BERT variant that uses 0.3% of its…
Fast Feedforward Networks
Peter Belcak, Roger Wattenhofer
We break the linear link between the layer size and its inference cost by introducing the fast feedforward (FFF) architecture, a log-time alternative to feedforward networks. We de…
Examining the Emergence of Deductive Reasoning in Generative Language Models
Peter Belcak, Luca A. Lanzendörfer, Roger Wattenhofer
We conduct a preliminary inquiry into the ability of generative transformer models to deductively reason from premises provided. We observe notable differences in the performance o…
Neural Combinatorial Logic Circuit Synthesis from Input-Output Examples
Peter Belcak, Roger Wattenhofer
We propose a novel, fully explainable neural approach to synthesis of combinatorial logic circuits from input-output examples. The carrying advantage of our method is that it readi…
A Neural Model for Regular Grammar Induction
Peter Belcák, David Hofer, Roger Wattenhofer
Grammatical inference is a classical problem in computational learning theory and a topic of wider influence in natural language processing. We treat grammars as a model of computa…
Periodic Extrapolative Generalisation in Neural Networks
Peter Belcák, Roger Wattenhofer
The learning of the simplest possible computational pattern -- periodicity -- is an open problem in the research of strong generalisation in neural networks. We formalise the probl…
FACT: Learning Governing Abstractions Behind Integer Sequences
Peter Belcák, Ard Kastrati, Flavio Schenker +1
Integer sequences are of central importance to the modeling of concepts admitting complete finitary descriptions. We introduce a novel view on the learning of such concepts and lay…
Deterministic Graph-Walking Program Mining
Peter Belcak, Roger Wattenhofer
Owing to their versatility, graph structures admit representations of intricate relationships between the separate entities comprising the data. We formalise the notion of connecti…
The LL(finite) strategy for optimal LL(k) parsing
Peter Belcak
The LL(finite) parsing strategy for parsing of LL(k) grammars where k needs not to be known is presented. The strategy parses input in linear time, uses arbitrary but always minima…
Fast Agent-Based Simulation Framework with Applications to Reinforcement Learning and the Study of Trading Latency Effects
Peter Belcak, Jan-Peter Calliess, Stefan Zohren
We introduce a new software toolbox for agent-based simulation. Facilitating rapid prototyping by offering a user-friendly Python API, its core rests on an efficient C++ implementa…