5 papers · 1 filter
On Temperature-Constrained Non-Deterministic Machine Translation: Potential and Evaluation
Weichuan Wang, Mingyang Liu, Linqi Song +1
In recent years, the non-deterministic properties of language models have garnered considerable attention and have shown a significant influence on real-world applications. However…
Activation-Guided Consensus Merging for Large Language Models
Yuxuan Yao, Shuqi Liu, Zehua Liu +6
Recent research has increasingly focused on reconciling the reasoning capabilities of System 2 with the efficiency of System 1. While existing training-based and prompt-based appro…
Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations
Guanzhi Deng, Mingyang Liu, Dapeng Wu +2
Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient fine-tuning method for large language models. However, its linear nature limits expressiveness. We propose LoRAN,…
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling
Yuxuan Yao, Han Wu, Mingyang Liu +5
Large language models (LLMs) exhibit varying strengths and weaknesses across different tasks, prompting recent studies to explore the benefits of ensembling models to leverage thei…
DiffETM: Diffusion Process Enhanced Embedded Topic Model
Wei Shao, Mingyang Liu, Linqi Song
The embedded topic model (ETM) is a widely used approach that assumes the sampled document-topic distribution conforms to the logistic normal distribution for easier optimization.…