2 citations · 3 across the 15 of their papers we have counts for
18 papers
Meta Expert: Cross-Layer Expert Sharing for Storage- and Retrieval-Efficient MoE
Ermo Hua, Xiangyu Hong, Baiting Wu +11
As model parameters scale up, sparsely activated Mixture-of-Experts (MoE) approaches have become essential for model scaling. However, mainstream models currently adopt hierarchica…
Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond
Zhiyuan Ma, Jiabao Wei, Tianyu Wang +21
Diffusion models (DMs) have steadily demonstrated outstanding performance in high-fidelity, controllable generation across diverse scenarios, following the "pretraining-finetuning"…
Improving Test-Time Scaling with Adaptive Looped Transformers
Yichen You, Tianyu Fu, Aosong Feng +4
Looped transformers have demonstrated promising parameter efficiency by reusing layers for latent computation. Prior studies compare looped and non-looped models at matched paramet…
Safin-1: Safety from Within through Memory-Native State Evolution
Ming Zhang, Kaisen Yang, Shu Yu +15
Long-horizon complex tasks require foundation models to accumulate information, maintain internal states, and adapt over extended interactions. Safety should be an intrinsic proper…
Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning
Kai Chen, Jifeng Ding, Ning Ding +44
We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iteratively achieve comp…
SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning
Haonan He, Haodi Lei, Yun Luo +13
On-policy distillation (OPD) offers a promising way to transfer reasoning capabilities from stronger teacher models, but applying it to long-context reasoning teachers and short-co…
MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
Ming Zhang, Kaisen Yang, Shu Yu +6
Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length. This flexibility, however, incurs a quadratic comput…
Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering
Junlin Yang, Che Jiang, Yu Fu +21
Recursive self-improvement (RSI) requires AI systems that improve the process of building AI (i.e., AI4AI); machine learning engineering (MLE) offers a concrete, executable testbed…
Draft-OPD: On-Policy Distillation for Speculative Draft Models
Haodi Lei, Yafu Li, Haoran Zhang +8
Speculative decoding accelerates large language model inference by pairing a target model with a lightweight draft model whose proposed tokens are verified in parallel. A common wa…
Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
Qianjia Cheng, Yuchen Zhang, Zhilin Wang +9
Tool-integrated reasoning (TIR) offers a direct way to extend thinking models beyond the limits of text-only reasoning. Paradoxically, we observe that tool-enabled evaluation can d…
Emotion-Director: Bridging Affective Shortcut in Emotion-Oriented Image Generation
Guoli Jia, Junyao Hu, Xinwei Long +5
Image generation based on diffusion models has demonstrated impressive capability, motivating exploration into diverse and specialized applications. Owing to the importance of emot…
Automating Exploratory Multiomics Research via Language Models
Shang Qu, Ning Ding, Linhai Xie +13
This paper introduces PROTEUS, a fully automated system that produces data-driven hypotheses from raw data files. We apply PROTEUS to clinical proteogenomics, a field where effecti…
Self-Reflective Reinforcement Learning for Diffusion-based Image Reasoning Generation
Jiadong Pan, Zhiyuan Ma, Kaiyan Zhang +2
Diffusion models have recently demonstrated exceptional performance in image generation task. However, existing image generation methods still significantly suffer from the dilemma…
LFQA-E: Carefully Benchmarking Long-form QA Evaluation
Yuchen Fan, Chen Lin, Xin Zhong +11
Long-Form Question Answering (LFQA) involves generating comprehensive, paragraph-level responses to open-ended questions, which poses a significant challenge for evaluation due to…
EVA-Score: Evaluating Abstractive Long-form Summarization on Informativeness through Extraction and Validation
Yuchen Fan, Yazhe Wan, Xin Zhong +3
Since LLMs emerged, more attention has been paid to abstractive long-form summarization, where longer input sequences indicate more information contained. Nevertheless, the automat…
Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process
Ermo Hua, Biqing Qi, Kaiyan Zhang +4
Supervised Fine-Tuning (SFT) and Preference Optimization (PO) are key processes for aligning Language Models (LMs) with human preferences post pre-training. While SFT excels in eff…
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following
Kaiyan Zhang, Jianyu Wang, Ermo Hua +3
With the advancement of language models (LMs), their exposure to private data is increasingly inevitable, and their deployment (especially for smaller ones) on personal devices, su…
CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model
Kaiyan Zhang, Ning Ding, Biqing Qi +3
Instruction tuning has recently been recognized as an effective way of aligning Large Language Models (LLMs) to enhance their generalization ability across various tasks. However,…