24 citations · 51 across the 27 of their papers we have counts for
28 papers
On-Policy Distillation Teaches New Skills but Not New Knowledge
Yixuan Tang, Yi Yang
On-policy distillation (OPD) strengthens language-model reasoning, yet whether students acquire new factual knowledge or compositional skill for multi-step reasoning remains unknow…
DREAM: Dense Retrieval Embeddings via Autoregressive Modeling
Yixuan Tang, Yi Yang
Dense retrieval embedding models are a fundamental component of modern retrieval-based AI systems. Most dense retrievers are trained with contrastive objectives, which require labe…
FLARE: Task-agnostic embedding model evaluation through a normalization process
Jingzhou Jiang, Yixuan Tang, Yi Yang +1
When task-specific labels are not available, it becomes difficult to select an embedding model for a specific target corpus. Existing labelless measures based on kernel estimators…
MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Models
Zhaokang Liao, Yingguo Gao, Yi Yang +2
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising approach to improve the reasoning abilities of Large Language Models (LLMs). Among RLVR algorithms,…
Crowded in B-Space: Calibrating Shared Directions for LoRA Merging
Yixuan Tang, Yi Yang
Merging separately trained LoRA adapters is a practical alternative to joint multi-task training, but it often hurts performance. Existing methods usually treat the LoRA update $ΔW…
Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design
Bojian Hou, Xiaolong Liu, Xiaoyi Liu +26
Deriving predictable scaling laws that govern the relationship between model performance and computational investment is crucial for designing and allocating resources in massive-s…
Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers
Zecheng Tang, Quantong Qiu, Yi Yang +6
The quadratic complexity of standard attention mechanisms poses a significant scalability bottleneck for large language models (LLMs) in long-context scenarios. While hybrid attent…
KV-Embedding: Training-free Text Embedding via Internal KV Re-routing in Decoder-only LLMs
Yixuan Tang, Yi Yang
While LLMs are powerful embedding backbones, their application in training-free settings faces two structural challenges: causal attention restricts early tokens from accessing sub…
ACE-Sync: An Adaptive Cloud-Edge Synchronization Framework for Communication-Efficient Large-Scale Distributed Model Training
Yi Yang, Ziyu Lin, Liesheng Wei
Large-scale deep learning models impose substantial communication overh ead in distributed training, particularly in bandwidth-constrained or heterogeneous clo ud-edge environments…
SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment
Yixuan Tang, Yi Yang
Aligning Large Language Models (LLMs) with human preferences typically relies on external supervision, which faces critical limitations: human annotations are scarce and subjective…
GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings
Yixuan Tang, Yi Yang
Domain-specific embedding models have shown promise for applications that require specialized semantic understanding, such as coding agents and financial retrieval systems, often a…
PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions
Yixuan Tang, Yi Yang, Ahmed Abbasi
Recent advancements in Large Language Models (LLMs) demonstrate remarkable capabilities across various fields. These developments have led to more direct communication between huma…
Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models
Ningyuan Deng, Hanyu Duan, Yixuan Tang +1
Text embedding models are widely used in natural language processing applications. However, their capability is often benchmarked on tasks that do not require understanding nuanced…
Achieving binary weight and activation for LLMs using Post-Training Quantization
Siqing Song, Chuang Wang, Ruiqi Wang +2
Quantizing large language models (LLMs) to 1-bit precision significantly reduces computational costs, but existing quantization techniques suffer from noticeable performance degrad…
Adversarial Mixup Unlearning
Zhuoyi Peng, Yixuan Tang, Yi Yang
Machine unlearning is a critical area of research aimed at safeguarding data privacy by enabling the removal of sensitive information from machine learning models. One unique chall…
FinMTEB: Finance Massive Text Embedding Benchmark
Yixuan Tang, Yi Yang
Embedding models play a crucial role in representing and retrieving information across various NLP applications. Recent advances in large language models (LLMs) have further enhanc…
Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?
Yixuan Tang, Yi Yang
The significant advancements of Large Language Models (LLMs) in generative tasks have led to a growing body of work exploring LLM-based embedding models. While these models, employ…
Do We Need Domain-Specific Embedding Models? An Empirical Investigation
Yixuan Tang, Yi Yang
Embedding models play a crucial role in representing and retrieving information across various NLP applications. Recent advancements in Large Language Models (LLMs) have further en…
Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning
Jiaqi Li, Yixuan Tang, Yi Yang
Large language models (LLMs) demonstrate remarkable capabilities but face challenges from hallucinations, which typically arise from insufficient knowledge or context. While instru…
Revealing the Two Sides of Data Augmentation: An Asymmetric Distillation-based Win-Win Solution for Open-Set Recognition
Yunbing Jia, Xiaoyu Kong, Fan Tang +3
In this paper, we reveal the two sides of data augmentation: enhancements in closed-set recognition correlate with a significant decrease in open-set recognition. Through empirical…
CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers
Longwei Zou, Qingyang Wang, Han Zhao +3
The fast-growing large scale language models are delivering unprecedented performance on almost all natural language processing tasks. However, the effectiveness of large language…
Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives
Jiaxin Liu, Yi Yang, Kar Yan Tam
In this paper, we introduce the Financial-STS task, a financial domain-specific NLP task designed to measure the nuanced semantic similarity between pairs of financial narratives.…
Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States
Hanyu Duan, Yi Yang, Kar Yan Tam
Large Language Models (LLMs) can make up answers that are not real, and this is known as hallucination. This research aims to see if, how, and to what extent LLMs are aware of hall…
MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries
Yixuan Tang, Yi Yang
Retrieval-augmented generation (RAG) augments large language models (LLM) by retrieving relevant knowledge, showing promising potential in mitigating LLM hallucinations and enhanci…