26 citations · 212 across the 74 of their papers we have counts for
4 papers · 2 filters
A Scaled-Up Empirical Study of Syntactic Language Models
Pengcheng Wang, Zhexuan Zhang, Chao Lou +1
Syntactic language models (SLMs) that generate sentences along with their syntactic tree structures have shown promise in both syntactic generalization and downstream tasks. Prior…
Does Traversal Order Matter? A Systematic Study of Tree Traversal Methods in Transformer Grammars
Zongru Liu, Pengyu Ji, Pengcheng Wang +1
Transformer Grammars (TGs) enhance language modeling by incorporating syntactic tree structures. Despite the potentially significant impact on model performance of how syntactic tr…
GiLT: Augmenting Transformer Language Models with Dependency Graphs
Tianyu Huang, Yida Zhao, Chuyan Zhou +1
Augmenting Transformers with linguistic structures effectively enhances the syntactic generalization performance of language models. Previous work in this direction focuses on synt…
Scaling Probabilistic Transformer via Efficient Cross-Scale Hyperparameter Transfer
Penghao Kuang, Haoyi Wu, Kewei Tu
Probabilistic Transformer (PT), a white-box probabilistic model for contextual word representation, has demonstrated substantial similarity to standard Transformers in both computa…