4 papers
When Load-Balancing Goes Too Far: Expert Pruning in Over-Dispersed Mixture-of-Experts Models
Berkcan Kapusuzoglu, Connor Pryor, Sangwoo Cho +4
Expert pruning reduces the memory and serving cost of Mixture-of-Experts (MoE) models by removing low-importance experts identified by the router, assuming router probabilities pro…
A Mathematical Framework and a Suite of Learning Techniques for Neural-Symbolic Systems
Charles Dickens, Connor Pryor, Changyu Gao +5
The field of Neural-Symbolic (NeSy) systems is growing rapidly. Proposed approaches show great promise in achieving symbiotic unions of neural and symbolic methods. However, a unif…
Using Domain Knowledge to Guide Dialog Structure Induction via Neural Probabilistic Soft Logic
Connor Pryor, Quan Yuan, Jeremiah Liu +4
Dialog Structure Induction (DSI) is the task of inferring the latent dialog structure (i.e., a set of dialog states and their temporal transitions) of a given goal-oriented dialog.…
Convex and Bilevel Optimization for Neuro-Symbolic Inference and Learning
Charles Dickens, Changyu Gao, Connor Pryor +2
We leverage convex and bilevel optimization techniques to develop a general gradient-based parameter learning framework for neural-symbolic (NeSy) systems. We demonstrate our frame…