243 citations · 461 across the 25 of their papers we have counts for
28 papers
Bookkeeping, Composition, or Unreachable Gold? Reading MemoryAgentBench's Conflict-Resolution Scores Against a Frozen Last-Write Resolver
Egor Pakhomov, Erik Nijkamp
MemoryAgentBench's Conflict Resolution split is read as measuring "selective forgetting". We execute the benchmark's own rule - the newest statement about a fact wins - as a zero-l…
Does an Agent's History Tell You When Compaction Will Hurt? A Modest, Bounded Effect on the TRACE Paired-Replay Corpus
Egor Pakhomov, Erik Nijkamp
Many long-horizon agents compact their context on a global rule, usually a token budget, blind to what the agent was doing. We ask whether the agent's recent behaviour predicts whe…
An Architecture for Long-Horizon Agents: Levels, Ticks and Cascaded Intelligence
Erik Nijkamp, Anurag Koul, Egor Pakhomov +1
Language-model agents are increasingly asked to carry out work spanning days or weeks, such as an operations remediation or a research programme. Such a task outlives any context w…
Parsing the Stream: A Live Trace Model for Long-Horizon Agents and Their Observers
Egor Pakhomov, Erik Nijkamp
A long-horizon agent's trace outgrows both of its consumers: the human observer monitoring the run, and the agent itself, whose bounded context the trace must be folded back into.…
Learning from Language Feedback via Variational Policy Distillation
Yang Li, Erik Nijkamp, Semih Yavuz +1
Reinforcement learning from verifiable rewards (RLVR) suffers from sparse outcome signals, creating severe exploration bottlenecks on complex reasoning tasks. Recent on-policy self…
Convomem Benchmark: Why Your First 150 Conversations Don't Need RAG
Egor Pakhomov, Erik Nijkamp, Caiming Xiong
We introduce a comprehensive benchmark for conversational memory evaluation containing 75,336 question-answer pairs across diverse categories including user facts, assistant recall…
xGen-small Technical Report
Erik Nijkamp, Bo Pang, Egor Pakhomov +5
We introduce xGen-small, a family of 4B and 9B Transformer decoder models optimized for long-context applications. Our vertically integrated pipeline unites domain-balanced, freque…
XGen-7B Technical Report
Erik Nijkamp, Tian Xie, Hiroaki Hayashi +22
Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LL…
CodeGen2: Lessons for Training LLMs on Programming and Natural Languages
Erik Nijkamp, Hiroaki Hayashi, Caiming Xiong +2
Large language models (LLMs) have demonstrated remarkable abilities in representation learning for program synthesis and understanding tasks. The quality of the learned representat…
Generating High Fidelity Synthetic Data via Coreset selection and Entropic Regularization
Omead Pooladzandi, Pasha Khosravi, Erik Nijkamp +1
Generative models have the ability to synthesize data points drawn from the data distribution, however, not all generated samples are high quality. In this paper, we propose using…
Learning Probabilistic Models from Generator Latent Spaces with Hat EBM
Mitch Hill, Erik Nijkamp, Jonathan Mitchell +2
This work proposes a method for using any generator network as the foundation of an Energy-Based Model (EBM). Our formulation posits that observed images are the sum of unobserved…
BigIssue: A Realistic Bug Localization Benchmark
Paul Kassianik, Erik Nijkamp, Bo Pang +2
As machine learning tools progress, the inevitable question arises: How can machine learning help us write better code? With significant progress being achieved in natural language…
ProGen2: Exploring the Boundaries of Protein Language Models
Erik Nijkamp, Jeffrey Ruffolo, Eli N. Weinstein +2
Attention-based models trained on protein sequences have demonstrated incredible success at classification and generation tasks relevant for artificial intelligence-driven protein…
Long Document Summarization with Top-down and Bottom-up Inference
Bo Pang, Erik Nijkamp, Wojciech Kryściński +3
Text summarization aims to condense long documents and retain key information. Critical to the success of a summarization model is the faithful inference of latent representations…
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Erik Nijkamp, Bo Pang, Hiroaki Hayashi +5
Program synthesis strives to generate a computer program as a solution to a given problem specification, expressed with input-output examples or natural language descriptions. The…
Generative Text Modeling through Short Run Inference
Bo Pang, Erik Nijkamp, Tian Han +1
Latent variable models for text, when trained successfully, accurately model the data distribution and capture global semantic and syntactic features of sentences. The prominent ap…
Semi-supervised Learning by Latent Space Energy-Based Model of Symbol-Vector Coupling
Bo Pang, Erik Nijkamp, Jiali Cui +2
This paper proposes a latent space energy-based prior model for semi-supervised learning. The model stands on a generator network that maps a latent vector to the observed example.…
Joint Training of Variational Auto-Encoder and Latent Energy-Based Model
Tian Han, Erik Nijkamp, Linqi Zhou +3
This paper proposes a joint training method to learn both the variational auto-encoder (VAE) and the latent energy-based model (EBM). The joint training of VAE and latent EBM are b…
Learning Latent Space Energy-Based Prior Model
Bo Pang, Tian Han, Erik Nijkamp +2
We propose to learn energy-based model (EBM) in the latent space of a generator model, so that the EBM serves as a prior model that stands on the top-down network of the generator…
MCMC Should Mix: Learning Energy-Based Model with Neural Transport Latent Space MCMC
Erik Nijkamp, Ruiqi Gao, Pavel Sountsov +4
Learning energy-based model (EBM) requires MCMC sampling of the learned model as an inner loop of the learning algorithm. However, MCMC sampling of EBMs in high-dimensional data sp…
Learning Multi-layer Latent Variable Model via Variational Optimization of Short Run MCMC for Approximate Inference
Erik Nijkamp, Bo Pang, Tian Han +3
This paper studies the fundamental problem of learning deep generative models that consist of multiple layers of latent variables organized in top-down architectures. Such models h…
Flow Contrastive Estimation of Energy-Based Models
Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma +3
This paper studies a training method to jointly estimate an energy-based model and a flow-based model, in which the two models are iteratively updated based on a shared adversarial…
Representation Learning: A Statistical Perspective
Jianwen Xie, Ruiqi Gao, Erik Nijkamp +2
Learning representations of data is an important problem in statistics and machine learning. While the origin of learning representations can be traced back to factor analysis and…
A Generative Model for Sampling High-Performance and Diverse Weights for Neural Networks
Lior Deutsch, Erik Nijkamp, Yu Yang
Recent work on mode connectivity in the loss landscape of deep neural networks has demonstrated that the locus of (sub-)optimal weight vectors lies on continuous paths. In this wor…