72 citations · 167 across the 14 of their papers we have counts for
17 papers
Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics
Jian Gu, Hongyu Zhang, Chunyang Chen +1
The vocabulary of a language model is its native semantics, and the embedding and unembedding fix the meaning of a state at the two ends of the residual stream. Between them, exist…
Mind the Gaps: From Failure Attribution to Closed-Form Repair of Code Language Models
Jian Gu, Hongyu Zhang, Chunyang Chen +1
Code language models must be maintained like the software around them: when a library evolves, a model keeps writing the interface that it saw during training. Repairing the model…
SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Residual-stream analysis asks how language-model computation evolves across depth, but intermediate decoding requires comparable readout coordinates across layers. If embedding anc…
Source-Free Detection and Impact Analysis of Compiler Optimization Problems in Mobile Applications
Han Hu, Xiaoheng Xie, Bo Sun +3
Mobile apps frequently suffer from frame drops, overheating, and excessive power consumption. While developers optimize algorithms and debug code, a critical bottleneck often goes…
Rethinking Weight Tying: Pseudo-Inverse Tying for LM Stable Training and Updates
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Weight tying is widely used in compact language models to reduce parameters by sharing the token table between the input embedding and the output projection. However, parameter sha…
Beyond Neural Incompatibility: Cross-Scale Knowledge Transfer in Language Models through Latent Semantic Alignment
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Language Models (LMs) encode substantial knowledge in their parameters, yet it remains unclear how to transfer such knowledge in a fine-grained manner, namely parametric knowledge…
SeMe: Training-Free Language Model Merging via Semantic Alignment
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Despite the remarkable capabilities of Language Models (LMs) across diverse tasks, no single model consistently outperforms others, necessitating efficient methods to combine their…
A Semantic-based Optimization Approach for Repairing LLMs: Case Study on Code Generation
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Language Models (LMs) are widely used in software engineering for code generation, but they may produce erroneous code. Rather than repairing outputs, a more thorough remedy is to…
A Semantic-Aware Layer-Freezing Approach to Computation-Efficient Fine-Tuning of Language Models
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Finetuning language models (LMs) is crucial for adapting the models to downstream data and tasks. However, full finetuning is usually costly. Existing work, such as parameter-effic…
Vocabulary-Defined Semantics: Latent Space Clustering for Improving In-Context Learning
Jian Gu, Aldeida Aleti, Chunyang Chen +1
In-context learning enables language models (LM) to adapt to downstream data or tasks by incorporating few samples as demonstrations within the prompts. It offers strong performanc…
Neuron Patching: Semantic-based Neuron-level Language Model Repair for Code Generation
Jian Gu, Aldeida Aleti, Chunyang Chen +1
Language Models (LMs) have become widely used in software engineering, especially for tasks such as code generation, where they are referred to as code LMs. These models have prove…
Towards Top-Down Automated Development in Limited Scopes: A Neuro-Symbolic Framework from Expressibles to Executables
Jian Gu, Harald C. Gall
Deep code generation is a topic of deep learning for software engineering (DL4SE), which adopts neural models to generate code for the intended functions. Since end-to-end neural m…
Assemble Foundation Models for Automatic Code Summarization
Jian Gu, Pasquale Salza, Harald C. Gall
Automatic code summarization is beneficial to daily software development since it could help reduce the requirement of manual writing. Currently, artificial intelligence is undergo…
On the Effectiveness of Transfer Learning for Code Search
Pasquale Salza, Christoph Schwizer, Jian Gu +1
The Transformer architecture and transfer learning have marked a quantum leap in natural language processing, improving the state of the art across a range of text-based tasks. Thi…
Multimodal Representation for Neural Code Search
Jian Gu, Zimin Chen, Martin Monperrus
Semantic code search is about finding semantically relevant code snippets for a given natural language query. In the state-of-the-art approaches, the semantic similarity between co…
Superoptimization of WebAssembly Bytecode
Javier Cabrera-Arteaga, Shrinish Donde, Jian Gu +4
Motivated by the fast adoption of WebAssembly, we propose the first functional pipeline to support the superoptimization of WebAssembly bytecode. Our pipeline works over LLVM and S…
Automated Classification of Overfitting Patches with Statically Extracted Code Features
He Ye, Jian Gu, Matias Martinez +2
Automatic program repair (APR) aims to reduce the cost of manually fixing software defects. However, APR suffers from generating a multitude of overfitting patches, those patches t…