332 citations · 895 across the 68 of their papers we have counts for
69 papers
From Detection to Refusal: Safer LLMs via Circuit-Guided Weight Scaling
Kuan-Lin Chu, Chung-En Sun, Tsui-Wei Weng
Despite extensive alignment efforts, Large Language Models (LLMs) remain vulnerable to generating unsafe content under adversarial prompting, yet the internal mechanisms by which s…
Data-Efficient Adaptation of LLMs via Attention Head Reweighting
Tuomas Oikarinen, Zixiao Chen, Charlotte Siska +3
Learning effectively from limited data is critical in domains like security where labeled examples are scarce. Large language models (LLMs) have demonstrated some capabilities for…
Multimodal Concept Bottleneck Models
Tongqing Shi, Ge Yan, Tuomas Oikarinen +1
Concept Bottleneck Models (CBMs) enhance the interpretability of deep learning networks by aligning the features extracted from images with natural concepts. However, existing CBMs…
The Cold-Start Safety Gap in LLM Agents
Chung-En Sun, Linbo Liu, Tsui-Wei Weng
Are tool-calling LLM agents equally safe throughout a conversation? We discover they are not: agents are most vulnerable at the very start of a session and become substantially saf…
LLM Agents Already Know When to Call Tools -- Even Without Reasoning
Chung-En Sun, Linbo Liu, Ge Yan +2
Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and latency, yet no existing benchmark…
CI-CBM: Class-Incremental Concept Bottleneck Model for Interpretable Continual Learning
Amirhosein Javadi, Tuomas Oikarinen, Tara Javidi +1
Catastrophic forgetting remains a fundamental challenge in continual learning, in which models often forget previous knowledge when fine-tuned on a new task. This issue is especial…
Steer2Edit: From Activation Steering to Component-Level Editing
Chung-En Sun, Ge Yan, Zimo Wang +1
Steering methods influence Large Language Model behavior by identifying semantic directions in hidden representations, but are typically realized through inference-time activation…
Distance Marching for Generative Modeling
Zimo Wang, Ishit Mehta, Haolin Lu +5
Modern generative models based on diffusion or flow matching transport noise toward the data manifold, without representing the data manifold explicitly. We propose Distance Marchi…
Faithful and Stable Neuron Explanations for Trustworthy Mechanistic Interpretability
Ge Yan, Tuomas Oikarinen, Tsui-Wei +1
Neuron identification is a popular tool in mechanistic interpretability, aiming to uncover the human-interpretable concepts represented by individual neurons in deep networks. Whil…
ReflCtrl: Controlling LLM Reflection Efficiently via Representation Engineering
Ge Yan, Chung-En Sun, Linbo Liu +1
Large reasoning models achieve strong performance on diverse tasks by producing extended chains of thought. Self-reflection, the ability to review and revise prior reasoning steps,…
Resting Neurons, Active Insights: Robustifying Activation Sparsity in LLMs via Spontaneity
Haotian Xu, Jiannan Yang, Tian Gao +2
Activation sparsity offers a compelling route to accelerate large language model (LLM) inference by selectively suppressing hidden activations, yet existing approaches exhibit seve…
Interpretable and Steerable Concept Bottleneck Sparse Autoencoders
Akshay Kulkarni, Tsui-Wei Weng, Vivek Narayanaswamy +3
Sparse autoencoders (SAEs) promise a unified approach for mechanistic interpretability, concept discovery, and model steering in LLMs and LVLMs. However, realizing this potential r…
ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretability
Chung-En Sun, Ge Yan, Akshay Kulkarni +1
Recent advances in long chain-of-thought (CoT) reasoning have largely prioritized answer accuracy and token efficiency, while overlooking aspects critical to trustworthiness. We ar…
Graph Concept Bottleneck Models
Haotian Xu, Tsui-Wei Weng, Lam M. Nguyen +1
Concept Bottleneck Models (CBMs) provide explicit interpretations for deep neural networks through concepts and allow intervention with concepts to adjust final predictions. Existi…
Responsiveness Verification: Will Predictions Change? How Much? How Often?
Harry Cheon, Meredith Stewart, Bogdan Kulynych +2
Machine learning models are often used in applications where their inputs change due to routine interactions, strategic manipulation, or noise. In such settings, models can undermi…
Evaluating Neuron Explanations: A Unified Framework with Sanity Checks
Tuomas Oikarinen, Ge Yan, Tsui-Wei Weng
Understanding the function of individual units in a neural network is an important building block for mechanistic interpretability. This is often done by generating a simple text e…
Beyond Top Activations: Efficient and Reliable Crowdsourced Evaluation of Automated Interpretability
Tuomas Oikarinen, Ge Yan, Akshay Kulkarni +1
Interpreting individual neurons or directions in activation space is an important topic in mechanistic interpretability. Numerous automated interpretability methods have been propo…
Anti-Sensing: Defense against Unauthorized Radar-based Human Vital Sign Sensing with Physically Realizable Wearable Oscillators
Md Farhan Tasnim Oshim, Nigel Doering, Bashima Islam +2
Recent advancements in Ultra-Wideband (UWB) radar technology have enabled contactless, non-line-of-sight vital sign monitoring, making it a valuable tool for healthcare. However, U…
Effective Skill Unlearning through Intervention and Abstention
Yongce Li, Chung-En Sun, Tsui-Wei Weng
Large language Models (LLMs) have demonstrated remarkable skills across various domains. Understanding the mechanisms behind their abilities and implementing controls over them is…
Interpretable Generative Models through Post-hoc Concept Bottlenecks
Akshay Kulkarni, Ge Yan, Chung-En Sun +2
Concept bottleneck models (CBM) aim to produce inherently interpretable models that rely on human-understandable concepts for their predictions. However, existing approaches to des…
RAT: Boosting Misclassification Detection Ability without Extra Data
Ge Yan, Tsui-Wei Weng
As deep neural networks(DNN) become increasingly prevalent, particularly in high-stakes areas such as autonomous driving and healthcare, the ability to detect incorrect predictions…
ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models
Chung-En Sun, Ge Yan, Tsui-Wei Weng
Recent studies have shown that Large Language Models (LLMs) augmented with chain-of-thought (CoT) reasoning demonstrate impressive problem-solving abilities. However, in this work,…
Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data
Pei-Yau Weng, Minh Hoang, Lam M. Nguyen +3
Fine-tuning pre-trained models is a popular approach in machine learning for solving complex tasks with moderate data. However, fine-tuning the entire pre-trained model is ineffect…
Understanding Fixed Predictions via Confined Regions
Connor Lawless, Tsui-Wei Weng, Berk Ustun +1
Machine learning models can assign fixed predictions that preclude individuals from changing their outcome. Existing approaches to audit fixed predictions do so on a pointwise basi…