2 citations · 2 across the 2 of their papers we have counts for
7 papers
Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewards
Yuxin Zhang, Meihao Fan, Ju Fan +5
Recent advances in large language models (LLMs) trained with reinforcement learning (RL) have improved Text-to-SQL performance. However, RL-based approaches still struggle with com…
Trainable Dynamic Mask Sparse Attention
Jingze Shi, Yifan Wu, Yiran Peng +4
The increasing demand for long-context modeling in large language models (LLMs) is bottlenecked by the quadratic complexity of the standard self-attention mechanism. The community…
TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding
Bingheng Wu, Jingze Shi, Yifan Wu +2
Transformers exhibit proficiency in capturing long-range dependencies, whereas State Space Models (SSMs) facilitate linear-time sequence modeling. Notwithstanding their synergistic…
Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting
Yifan Wu, Jingze Shi, Bingheng Wu +4
Existing chain-of-thought (CoT) distillation methods can effectively transfer reasoning abilities to base models but suffer from two major limitations: excessive verbosity of reaso…
Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture
Jingze Shi, Bingheng Wu
In order to make the foundation model more efficient and effective, our idea is combining sequence transformation and state transformation. First, we prove the availability of rota…
Wonderful Matrices: More Efficient and Effective Architecture for Language Modeling Tasks
Jingze Shi, Bingheng Wu, Lu He +1
We prove the availability of inner product form position encoding in the state space dual algorithm and study the effectiveness of different position embeddings in the hybrid quadr…