2 citations · 2 across the 4 of their papers we have counts for
5 papers
Rethinking LoRA Initialization for Robust Asymmetric Learning Rates
Disen Liao, Yaoliang Yu
Low-Rank Adaptation (LoRA) is often highly sensitive to learning-rate (LR) choices. Recent empirical studies suggest that, once LRs are carefully tuned, vanilla LoRA can remain com…
Cross-Session Decomposition Attacks: Scaling Risk and Intent-Aligned Retrieval Defense
Disen Liao, Yihan Wang, Freda Shi +1
Scaling laws are usually read as a capability story: lower language-modeling loss yields more useful models. We study a safety consequence of this mechanism in \emph{cross-session…
How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them
Disen Liao, Freda Shi
Tokenization is the first step in every language model (LM), yet it never takes the sounds of words into account. We investigate how tokenization influences text-only LMs' ability…
Efficient Bilevel Optimization with KFAC-Based Hypergradients
Disen Liao, Felix Dangel, Yaoliang Yu
Bilevel optimization (BO) is widely applicable to many machine learning problems. Scaling BO, however, requires repeatedly computing hypergradients, which involves solving inverse…
StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
Jialin Yang, Dongfu Jiang, Lipeng He +17
As Large Language Models (LLMs) become integral to software development workflows, their ability to generate structured outputs has become critically important. We introduce Struct…