Publications (83)
Next Block Prediction: Video Generation via Semi-Autoregressive Modeling
Shuhuai Ren, Shuming Ma, Xu Sun +1
CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation
Jian Yang, Shaohan Huang, Shuming Ma +6
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
Shuming Ma, Li Dong, Shaohan Huang +6
On the Pareto Front of Multilingual Neural Machine Translation
Liang Chen, Shuming Ma, Dongdong Zhang +2
On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation
Liang Chen, Shuming Ma, Dongdong Zhang +2
A Generic Online Parallel Learning Framework for Large Margin Models
Shuming Ma, Xu Sun
Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization
Shuming Ma, Xu Sun, Junyang Lin +1
HanoiT: Enhancing Context-aware Translation via Selective Context
Jian Yang, Yuwei Yin, Shuming Ma +7
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Jian Yang, Shuming Ma, Li Dong +7
UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation
Jian Yang, Yuwei Yin, Shuming Ma +5
Retentive Network: A Successor to Transformer for Large Language Models
Yutao Sun, Li Dong, Shaohan Huang +5
MH-MoE: Multi-Head Mixture-of-Experts
Shaohan Huang, Xun Wu, Shuming Ma +1
A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model
Xin Sun, Tao Ge, Shuming Ma +3
Unsupervised Machine Commenting with Neural Variational Topic Model
Shuming Ma, Lei Cui, Furu Wei +1
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Zewen Chi, Li Dong, Shuming Ma +3
BitNet: Scaling 1-bit Transformers for Large Language Models
Hongyu Wang, Shuming Ma, Li Dong +7
GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation
Jian Yang, Yuwei Yin, Liqun Yang +5
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Shuming Ma, Jian Yang, Haoyang Huang +10
A New Recurrent Neural CRF for Learning Non-linear Edge Features
Shuming Ma, Xu Sun
Auto-ICL: In-Context Learning without Human Supervision
Jinghan Yang, Shuming Ma, Furu Wei
Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data?
Yi Zhang, Xu Sun, Shuming Ma +2
LongNet: Scaling Transformers to 1,000,000,000 Tokens
Jiayu Ding, Shuming Ma, Li Dong +5
A Bilingual Parallel Corpus with Discourse Annotations
Yuchen Eleanor Jiang, Tianyu Liu, Shuming Ma +3
Global Encoding for Abstractive Summarization
Junyang Lin, Xu Sun, Shuming Ma +1
Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation
Shuming Ma, Xu Sun, Wei Li +3
How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?
Weijia Xu, Shuming Ma, Dongdong Zhang +1
RedStone: Curating General, Code, Math, and QA Data for Large Language Models
Yaoyao Chang, Lei Cui, Li Dong +13
PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation
Juncheng Wan, Jian Yang, Shuming Ma +4
Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method
Xu Sun, Xuancheng Ren, Shuming Ma +5
LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts
Shuming Ma, Lei Cui, Damai Dai +2
KOSMOS-2.5: A Multimodal Literate Model
Tengchao Lv, Yupan Huang, Jingye Chen +13
Kosmos-2: Grounding Multimodal Large Language Models to the World
Zhiliang Peng, Wenhui Wang, Li Dong +4
Improving Semantic Relevance for Sequence-to-Sequence Learning of Chinese Social Media Text Summarization
Shuming Ma, Xu Sun, Jingjing Xu +3
Bitnet.cpp: Efficient Edge Inference for Ternary LLMs
Jinheng Wang, Hansong Zhou, Ting Song +7
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
Zewen Chi, Shaohan Huang, Li Dong +8
Deconvolution-Based Global Decoding for Neural Machine Translation
Junyang Lin, Xu Sun, Xuancheng Ren +3
A Semantic Relevance Based Neural Network for Text Summarization and Text Simplification
Shuming Ma, Xu Sun
Language Is Not All You Need: Aligning Perception with Language Models
Shaohan Huang, Li Dong, Wenhui Wang +15
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
Jian Yang, Shuming Ma, Haoyang Huang +8
Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation
Shuming Ma, Pengcheng Yang, Tianyu Liu +3
BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
Hongyu Wang, Shuming Ma, Furu Wei
Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation
Guanhua Chen, Shuming Ma, Yun Chen +4
DeepNet: Scaling Transformers to 1,000 Layers
Hongyu Wang, Shuming Ma, Li Dong +3
1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
Jinheng Wang, Hansong Zhou, Ting Song +5
Bag-of-Words as Target for Neural Machine Translation
Shuming Ma, Xu Sun, Yizhong Wang +1
Complex Structure Leads to Overfitting: A Structure Regularization Decoding Method for Natural Language Processing
Xu Sun, Weiwei Sun, Shuming Ma +4
Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
Hongyu Wang, Shuming Ma, Ruiping Wang +1
When an Image is Worth 1,024 x 1,024 Words: A Case Study in Computational Pathology
Wenhui Wang, Shuming Ma, Hanwen Xu +4
Are More Layers Beneficial to Graph Transformers?
Haiteng Zhao, Shuming Ma, Dongdong Zhang +2
BitNet b1.58 2B4T Technical Report
Shuming Ma, Hongyu Wang, Shaohan Huang +5
SGM: Sequence Generation Model for Multi-label Classification
Pengcheng Yang, Xu Sun, Wei Li +3
Revamping Multilingual Agreement Bidirectionally via Switched Back-translation for Multilingual Neural Machine Translation
Hongyuan Lu, Haoyang Huang, Shuming Ma +3
LongReasonArena: A Long Reasoning Benchmark for Large Language Models
Jiayu Ding, Shuming Ma, Lei Cui +2
On the Representation Collapse of Sparse Mixture of Experts
Zewen Chi, Li Dong, Shaohan Huang +9
A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification
Pengcheng Yang, Shuming Ma, Yi Zhang +3
Foundation Transformers
Hongyu Wang, Shuming Ma, Shaohan Huang +12
Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
Junyang Lin, Qi Su, Pengcheng Yang +2
A Length-Extrapolatable Transformer
Yutao Sun, Li Dong, Barun Patra +6
SMDT: Selective Memory-Augmented Neural Document Translation
Xu Zhang, Jian Yang, Haoyang Huang +4
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Yuchen Eleanor Jiang, Tianyu Liu, Shuming Ma +3
meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting
Xu Sun, Xuancheng Ren, Shuming Ma +1
StableMoE: Stable Routing Strategy for Mixture of Experts
Damai Dai, Li Dong, Shuming Ma +4
A Hierarchical End-to-End Model for Jointly Improving Text Summarization and Sentiment Classification
Shuming Ma, Xu Sun, Junyang Lin +1
BitNet a4.8: 4-bit Activations for 1-bit LLMs
Hongyu Wang, Shuming Ma, Furu Wei
Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation
Junyang Lin, Shuming Ma, Qi Su +1
Multimodal Matching Transformer for Live Commenting
Chaoqun Duan, Lei Cui, Shuming Ma +3
Recursive Graphical Neural Networks for Text Classification
Wei Li, Shuheng Li, Shuming Ma +3
Lock-Free Parallel Perceptron for Graph-based Dependency Parsing
Xu Sun, Shuming Ma
Automatic Academic Paper Rating Based on Modularized Hierarchical Convolutional Neural Network
Pengcheng Yang, Xu Sun, Wei Li +1
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
Wenkai Yang, Shuming Ma, Yankai Lin +1
Language Models are General-Purpose Interfaces
Yaru Hao, Haoyu Song, Li Dong +5
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation
Yuchen Eleanor Jiang, Tianyu Liu, Shuming Ma +7
Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks
Xu Sun, Bingzhen Wei, Xuancheng Ren +1
HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation
Jian Yang, Yuwei Yin, Shuming Ma +3
TorchScale: Transformers at Scale
Shuming Ma, Hongyu Wang, Shaohan Huang +8
Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt
Lianzhe Huang, Shuming Ma, Dongdong Zhang +2
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Shuming Ma, Hongyu Wang, Lingxiao Ma +7
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
Damai Dai, Yutao Sun, Li Dong +4
You Only Cache Once: Decoder-Decoder Architectures for Language Models
Yutao Sun, Li Dong, Yi Zhu +6
Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Models
Hongyuan Lu, Haoyang Huang, Shuming Ma +3
Towards Easier and Faster Sequence Labeling for Natural Language Processing: A Search-based Probabilistic Online Learning Framework (SAPO)
Xu Sun, Shuming Ma, Yi Zhang +1
Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Guanhua Chen, Shuming Ma, Yun Chen +5
Identifying High-Quality Chinese News Comments Based on Multi-Target Text Matching Model
Deli Chen, Shuming Ma, Pengcheng Yang +1