5 citations · 27 across the 27 of their papers we have counts for
27 papers
Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models
Zhenghao Lin, Zihao Tang, Xiao Liu +31
We introduce Sigma, an efficient large language model specialized for the system domain, empowered by a novel architecture including DiffQKV attention, and pre-trained on our metic…
Semantic Alignment for Multimodal Large Language Models
Tao Wu, Mengze Li, Jingyuan Chen +6
Research on Multi-modal Large Language Models (MLLMs) towards the multi-image cross-modal instruction has received increasing attention and made significant progress, particularly…
Xinyu: An Efficient LLM-based System for Commentary Generation
Yiquan Wu, Bo Tang, Chenyang Xi +13
Commentary provides readers with a deep understanding of events by presenting diverse arguments and evidence. However, creating commentary is a time-consuming task, even for skille…
Causal Inference with Complex Treatments: A Survey
Yingrong Wang, Haoxuan Li, Minqin Zhu +4
Causal inference plays an important role in explanatory analysis and decision making across various fields like statistics, marketing, health care, and education. Its main task is…
Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning
Ziyu Zhao, Leilei Gan, Guoyin Wang +5
Low-Rank Adaptation (LoRA) offers an efficient way to fine-tune large language models (LLMs). Its modular and plug-and-play nature allows the integration of various domain-specific…
Stable Heterogeneous Treatment Effect Estimation across Out-of-Distribution Populations
Yuling Zhang, Anpeng Wu, Kun Kuang +3
Heterogeneous treatment effect (HTE) estimation is vital for understanding the change of treatment effect across individuals or subgroups. Most existing HTE estimation methods focu…