9 papers
SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Residual-stream analysis asks how language-model computation evolves across depth, but intermediate decoding requires comparable readout coordinates across layers. If embedding anc…
Beyond Neural Incompatibility: Cross-Scale Knowledge Transfer in Language Models through Latent Semantic Alignment
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Language Models (LMs) encode substantial knowledge in their parameters, yet it remains unclear how to transfer such knowledge in a fine-grained manner, namely parametric knowledge…
Rethinking Weight Tying: Pseudo-Inverse Tying for LM Stable Training and Updates
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Weight tying is widely used in compact language models to reduce parameters by sharing the token table between the input embedding and the output projection. However, parameter sha…
ViBR: Automated Bug Replay from Video-based Reports using Vision-Language Models
Sidong Feng, Dingbang Wang, Nikola Tomic +3
Bug reports play a critical role in software maintenance by helping users convey encountered issues to developers. Recently, GUI screen capture videos have gained popularity as a b…
A Semantic-based Optimization Approach for Repairing LLMs: Case Study on Code Generation
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Language Models (LMs) are widely used in software engineering for code generation, but they may produce erroneous code. Rather than repairing outputs, a more thorough remedy is to…
A Semantic-Aware Layer-Freezing Approach to Computation-Efficient Fine-Tuning of Language Models
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Finetuning language models (LMs) is crucial for adapting the models to downstream data and tasks. However, full finetuning is usually costly. Existing work, such as parameter-effic…