4 papers · 1 filter
Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
Ximing Dong, Shaowei Wang, Dayi Lin +2
Large Language Models (LLMs) achieve strong performance across many tasks but suffer from high inference latency due to autoregressive decoding. The issue is exacerbated in Large R…
Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization
Ximing Dong, Shaowei Wang, Dayi Lin +1
Optimizing Large Language Model (LLM) performance requires well-crafted prompts, but manual prompt engineering is labor-intensive and often ineffective. Automated prompt optimizati…
A Framework for Real-time Safeguarding the Text Generation of Large Language Model
Ximing Dong, Dayi Lin, Shaowei Wang +1
Large Language Models (LLMs) have significantly advanced natural language processing (NLP) tasks but also pose ethical and societal risks due to their propensity to generate harmfu…
PromptExp: Multi-granularity Prompt Explanation of Large Language Models
Ximing Dong, Shaowei Wang, Dayi Lin +4
Large Language Models excel in tasks like natural language understanding and text generation. Prompt engineering plays a critical role in leveraging LLM effectively. However, LLMs…