5 citations · 14 across the 12 of their papers we have counts for
13 papers
DivMoE: Fine-Grained MoE Upcycling via Cross-Domain Expert Composition
Yuxuan Lou, Kai Yang, Geng Zhang +2
Mixture-of-Experts (MoE) architectures have become essential for scaling large language models, with recent work demonstrating the benefits of fine-grained expert designs. Training…
LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes
Chuyan Chen, Haoxing Chen, Kun Chen +27
We introduce LLaDA-Image, a unified framework that pairs a 6B Diffusion Transformer (DiT) trained from scratch with a frozen vision-language understanding module built on the LLaDA…
AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decoding
Yingxuan Ren, Yuxuan Lou, Yong Liu +4
Block-wise semi-autoregressive decoding is the standard inference paradigm for diffusion large language models (DLMs), but it imposes a strict dependency between blocks: the next b…
OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling
Yuxuan Lou, Yang You
Muon fixes the \emph{direction} of every matrix-valued update at the polar factor of its momentum, while each layer's step \emph{magnitude} is addressed only by a static shape corr…
Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoning
Peihao Wang, Shan Yang, Xijun Wang +8
Associative memory has long underpinned the design of sequential models. Beyond recall, humans reason by projecting future states and selecting goal-directed actions, a capability…
DiffuSpeech: Silent Thought, Spoken Answer via Unified Speech-Text Diffusion
Yuxuan Lou, Ziming Wu, Yaochen Wang +6
Current speech language models generate responses directly without explicit reasoning, leading to errors that cannot be corrected once audio is produced. We introduce \textbf{``Sil…
MoST: Mixing Speech and Text with Modality-Aware Mixture of Experts
Yuxuan Lou, Kai Yang, Yang You
We present MoST (Mixture of Speech and Text), a novel multimodal large language model that seamlessly integrates speech and text processing through our proposed Modality-Aware Mixt…
Memory Transfer Planning: LLM-driven Context-Aware Code Adaptation for Robot Manipulation
Tomoyuki Kagaya, Subramanian Lakshmi, Yuxuan Lou +6
Large language models (LLMs) are increasingly explored in robot manipulation, but many existing methods struggle to adapt to new environments. Many systems require either environme…
MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE
Geng Zhang, Yuxuan Han, Yuxuan Lou +3
Mixture-of-Experts (MoE) enables efficient scaling of large language models by activating only a subset of experts per input token. However, deploying MoE-based models incurs signi…
EnvBridge: Bridging Diverse Environments with Cross-Environment Knowledge Transfer for Embodied AI
Tomoyuki Kagaya, Yuxuan Lou, Thong Jing Yuan +8
In recent years, Large Language Models (LLMs) have demonstrated high reasoning capabilities, drawing attention for their applications as agents in various decision-making processes…
RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agents
Tomoyuki Kagaya, Thong Jing Yuan, Yuxuan Lou +6
Owing to recent advancements, Large Language Models (LLMs) can now be deployed as agents for increasingly complex decision-making applications in areas including robotics, gaming,…
One Student Knows All Experts Know: From Sparse to Dense
Fuzhao Xue, Xiaoxin He, Xiaozhe Ren +2
Human education system trains one student by multiple experts. Mixture-of-experts (MoE) is a powerful sparse architecture including multiple experts. However, sparse MoE model is e…
Cross-token Modeling with Conditional Computation
Yuxuan Lou, Fuzhao Xue, Zangwei Zheng +1
Mixture-of-Experts (MoE), a conditional computation architecture, achieved promising performance by scaling local module (i.e. feed-forward network) of transformer. However, scalin…