24 citations · 51 across the 27 of their papers we have counts for
20 papers · 1 filter
On-Policy Distillation Teaches New Skills but Not New Knowledge
Yixuan Tang, Yi Yang
On-policy distillation (OPD) strengthens language-model reasoning, yet whether students acquire new factual knowledge or compositional skill for multi-step reasoning remains unknow…
DREAM: Dense Retrieval Embeddings via Autoregressive Modeling
Yixuan Tang, Yi Yang
Dense retrieval embedding models are a fundamental component of modern retrieval-based AI systems. Most dense retrievers are trained with contrastive objectives, which require labe…
Crowded in B-Space: Calibrating Shared Directions for LoRA Merging
Yixuan Tang, Yi Yang
Merging separately trained LoRA adapters is a practical alternative to joint multi-task training, but it often hurts performance. Existing methods usually treat the LoRA update $ΔW…
Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers
Zecheng Tang, Quantong Qiu, Yi Yang +6
The quadratic complexity of standard attention mechanisms poses a significant scalability bottleneck for large language models (LLMs) in long-context scenarios. While hybrid attent…
KV-Embedding: Training-free Text Embedding via Internal KV Re-routing in Decoder-only LLMs
Yixuan Tang, Yi Yang
While LLMs are powerful embedding backbones, their application in training-free settings faces two structural challenges: causal attention restricts early tokens from accessing sub…
SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment
Yixuan Tang, Yi Yang
Aligning Large Language Models (LLMs) with human preferences typically relies on external supervision, which faces critical limitations: human annotations are scarce and subjective…
GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings
Yixuan Tang, Yi Yang
Domain-specific embedding models have shown promise for applications that require specialized semantic understanding, such as coding agents and financial retrieval systems, often a…
PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions
Yixuan Tang, Yi Yang, Ahmed Abbasi
Recent advancements in Large Language Models (LLMs) demonstrate remarkable capabilities across various fields. These developments have led to more direct communication between huma…
Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models
Ningyuan Deng, Hanyu Duan, Yixuan Tang +1
Text embedding models are widely used in natural language processing applications. However, their capability is often benchmarked on tasks that do not require understanding nuanced…
FinMTEB: Finance Massive Text Embedding Benchmark
Yixuan Tang, Yi Yang
Embedding models play a crucial role in representing and retrieving information across various NLP applications. Recent advances in large language models (LLMs) have further enhanc…
Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?
Yixuan Tang, Yi Yang
The significant advancements of Large Language Models (LLMs) in generative tasks have led to a growing body of work exploring LLM-based embedding models. While these models, employ…
Do We Need Domain-Specific Embedding Models? An Empirical Investigation
Yixuan Tang, Yi Yang
Embedding models play a crucial role in representing and retrieving information across various NLP applications. Recent advancements in Large Language Models (LLMs) have further en…
Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning
Jiaqi Li, Yixuan Tang, Yi Yang
Large language models (LLMs) demonstrate remarkable capabilities but face challenges from hallucinations, which typically arise from insufficient knowledge or context. While instru…
CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers
Longwei Zou, Qingyang Wang, Han Zhao +3
The fast-growing large scale language models are delivering unprecedented performance on almost all natural language processing tasks. However, the effectiveness of large language…
Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives
Jiaxin Liu, Yi Yang, Kar Yan Tam
In this paper, we introduce the Financial-STS task, a financial domain-specific NLP task designed to measure the nuanced semantic similarity between pairs of financial narratives.…
Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States
Hanyu Duan, Yi Yang, Kar Yan Tam
Large Language Models (LLMs) can make up answers that are not real, and this is known as hallucination. This research aims to see if, how, and to what extent LLMs are aware of hall…
MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries
Yixuan Tang, Yi Yang
Retrieval-augmented generation (RAG) augments large language models (LLM) by retrieving relevant knowledge, showing promising potential in mitigating LLM hallucinations and enhanci…
Exploring the Relationship between In-Context Learning and Instruction Tuning
Hanyu Duan, Yixuan Tang, Yi Yang +2
In-Context Learning (ICL) and Instruction Tuning (IT) are two primary paradigms of adopting Large Language Models (LLMs) to downstream applications. However, they are significantly…
Bias A-head? Analyzing Bias in Transformer-Based Language Model Attention Heads
Yi Yang, Hanyu Duan, Ahmed Abbasi +2
Transformer-based pretrained large language models (PLM) such as BERT and GPT have achieved remarkable success in NLP tasks. However, PLMs are prone to encoding stereotypical biase…
FinEntity: Entity-level Sentiment Classification for Financial Texts
Yixuan Tang, Yi Yang, Allen H Huang +2
In the financial domain, conducting entity-level sentiment analysis is crucial for accurately assessing the sentiment directed toward a specific financial entity. To our knowledge,…