47 citations · 262 across the 33 of their papers we have counts for
20 papers · 1 filter
The Emergent Symbolic Structure of Artificial Neural Networks
R. Thomas McCoy, Paul Soulos, Tal Linzen +1
Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modeled as operating over structured…
GFlowRL: Scaling Distribution-Matching RL to Large Language Models
Xiaodong Liu, Michael Xu, Jack W. Stokes +3
Generative Flow Networks (GFlowNets) offer a promising alternative to reward-maximizing reinforcement learning (RL) for large reasoning models, encouraging diverse reasoning paths…
Implicit Chain of Thought Reasoning via Knowledge Distillation
Yuntian Deng, Kiran Prasad, Roland Fernandez +3
To augment language models with the ability to reason, researchers usually prompt or finetune them to produce chain of thought reasoning steps before producing the final answer. Ho…
Differentiable Tree Operations Promote Compositional Generalization
Paul Soulos, Edward Hu, Kate McCurdy +4
In the context of structure-to-structure transformation tasks, learning sequences of discrete symbolic operations poses significant challenges due to their non-differentiability. T…
Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models
Najoung Kim, Tal Linzen, Paul Smolensky
Human linguistic capacity is often characterized by compositionality and the generalization it enables -- human learners can produce and comprehend novel complex expressions by com…
Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages
Paul Soulos, Sudha Rao, Caitlin Smith +9
Machine translation has seen rapid progress with the advent of Transformer-based models. These models have no explicit linguistic structure built into them, yet they may still impl…
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN
R. Thomas McCoy, Paul Smolensky, Tal Linzen +2
Current language models can generate high-quality text. Are they simply copying text they have seen before, or have they learned generalizable linguistic abstractions? To tease apa…
Distributed neural encoding of binding to thematic roles
Matthias Lalisse, Paul Smolensky
A framework and method are proposed for the study of constituent composition in fMRI. The method produces estimates of neural patterns encoding complex linguistic structures, under…
Scalable knowledge base completion with superposition memories
Matthias Lalisse, Eric Rosen, Paul Smolensky
We present Harmonic Memory Networks (HMem), a neural architecture for knowledge base completion that models entities as weighted sums of pairwise bindings between an entity's neigh…
Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization
Yichen Jiang, Asli Celikyilmaz, Paul Smolensky +7
Abstractive summarization, the task of generating a concise summary of input documents, requires: (1) reasoning over the source document to determine the salient pieces of informat…
Universal linguistic inductive biases via meta-learning
R. Thomas McCoy, Erin Grant, Paul Smolensky +2
How do learners acquire languages from the limited data available to them? This process must involve some inductive biases - factors that affect how a learner generalizes - but it…
HUBERT Untangles BERT to Improve Transfer across NLP Tasks
Mehrad Moradshahi, Hamid Palangi, Monica S. Lam +2
We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional Transformer language model.…
Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
Kezhen Chen, Qiuyuan Huang, Hamid Palangi +3
Generating formal-language programs represented by relational tuples, such as Lisp programs or mathematical operations, to solve problems stated in natural language is a challengin…
RNNs Implicitly Implement Tensor Product Representations
R. Thomas McCoy, Tal Linzen, Ewan Dunbar +1
Recurrent neural networks (RNNs) can learn continuous vector representations of symbolic structures such as sequences and sentences; these representations often exhibit linear regu…
Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations
Matthias Lalisse, Paul Smolensky
Neural models of Knowledge Base data have typically employed compositional representations of graph objects: entity and relation embeddings are systematically combined to evaluate…
Predicting the Argumenthood of English Prepositional Phrases
Najoung Kim, Kyle Rawlins, Benjamin Van Durme +1
Distinguishing between arguments and adjuncts of a verb is a longstanding, nontrivial problem. In natural language processing, argumenthood information is important in tasks such a…
Discrete symbolic optimization and Boltzmann sampling by continuous neural dynamics: Gradient Symbolic Computation
Paul Tupper, Paul Smolensky, Pyeong Whan Cho
Gradient Symbolic Computation is proposed as a means of solving discrete global optimization problems using a neurally plausible continuous stochastic dynamical system. Gradient sy…
A Neural-Symbolic Approach to Design of CAPTCHA
Qiuyuan Huang, Paul Smolensky, Xiaodong He +2
CAPTCHAs based on reading text are susceptible to machine-learning-based attacks due to recent significant advances in deep learning (DL). To address this, this paper promotes imag…
Question-Answering with Grammatically-Interpretable Representations
Hamid Palangi, Paul Smolensky, Xiaodong He +1
We introduce an architecture, the Tensor Product Recurrent Network (TPRN). In our application of TPRN, internal representations learned by end-to-end optimization in a deep neural…
Reasoning in Vector Space: An Exploratory Study of Question Answering
Moontae Lee, Xiaodong He, Wen-tau Yih +3
Question answering tasks have shown remarkable progress with distributed vector representation. In this paper, we investigate the recently proposed Facebook bAbI tasks which consis…