81 citations · 108 across the 29 of their papers we have counts for
19 papers · 1 filter
Domain Generalization under Sampling Pattern Shifts in Irregular Time Series
Changhun Kim, Joohyung Lee, Kwanhyung Lee +3
Irregularly sampled multivariate time series (ISMTS) are prevalent in real-world applications, where both observation times and available measurements can vary substantially across…
Why DDIM Hallucinates More Than DDPM: A Theoretical Analysis of Reverse Dynamics
Muhammad H. Ashiq, Samanyu Arora, Abhinav N. Harish +3
We theoretically study the hallucination phenomena in two canonical diffusion samplers: the stochastic Denoising Diffusion Probabilistic Model (DDPM) and the deterministic Denoisin…
LJ-Bench: Ontology-Based Benchmark for U.S. Crime
Hung Yun Tseng, Wuzhen Li, Blerina Gkotse +1
The potential of Large Language Models (LLMs) to provide harmful information remains a significant concern due to the vast breadth of illegal queries they may encounter. Unfortunat…
Corrective Diffusion Language Models
Shuibai Zhang, Fred Zhangzhi Peng, Yiheng Zhang +2
While Diffusion Language Models (DLMs) are theoretically well-suited for iterative refinement due to their non-causal structure, they often fail to reliably revise incorrect tokens…
Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders
James Oldfield, Shawn Im, Sharon Li +3
Multilayer perceptrons (MLPs) are an integral part of large language models, yet their dense representations render them difficult to understand, edit, and steer. Recent methods le…
Hadamard product in deep learning: Introduction, Advances and Challenges
Grigorios G Chrysos, Yongtao Wu, Razvan Pascanu +2
While convolution and self-attention mechanisms have dominated architectural design in deep learning, this survey examines a fundamental yet understudied primitive: the Hadamard pr…
Single-pass Detection of Jailbreaking Input in Large Language Models
Leyla Naz Candogan, Yongtao Wu, Elias Abad Rocamora +2
Defending aligned Large Language Models (LLMs) against jailbreaking attacks is a challenging problem, with existing approaches requiring multiple requests or even queries to auxili…
Certified Robustness Under Bounded Levenshtein Distance
Elias Abad Rocamora, Grigorios G. Chrysos, Volkan Cevher
Text classifiers suffer from small perturbations, that if chosen adversarially, can dramatically change the output of the model. Verification methods can provide robustness certifi…
Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
Zheyang Xiong, Ziyang Cai, John Cooper +11
Large Language Models (LLMs) have demonstrated remarkable in-context learning (ICL) capabilities. In this study, we explore a surprising phenomenon related to ICL: LLMs can perform…
Revisiting Character-level Adversarial Attacks for Language Models
Elias Abad Rocamora, Yongtao Wu, Fanghui Liu +2
Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based met…
Robust NAS under adversarial training: benchmark, theory, and beyond
Yongtao Wu, Fanghui Liu, Carl-Johann Simon-Gabriel +2
Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence o…
Generalization of Scaled Deep ResNets in the Mean-Field Regime
Yihang Chen, Fanghui Liu, Yiping Lu +2
Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate \…
Efficient local linearity regularization to overcome catastrophic overfitting
Elias Abad Rocamora, Fanghui Liu, Grigorios G. Chrysos +2
Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-ste…
Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks
Yixin Cheng, Markos Georgopoulos, Volkan Cevher +1
Large Language Models (LLMs) are susceptible to Jailbreaking attacks, which aim to extract harmful information by subtly modifying the attack query. As defense mechanisms evolve, d…
On the Convergence of Encoder-only Shallow Transformers
Yongtao Wu, Fanghui Liu, Grigorios G Chrysos +1
In this paper, we aim to build the global convergence theory of encoder-only shallow Transformers under a realistic setting from the perspective of architectures, initialization, a…
Maximum Independent Set: Self-Training through Dynamic Programming
Lorenzo Brusca, Lars C. P. M. Quaedvlieg, Stratis Skoulakis +2
This work presents a graph neural network (GNN) framework for solving the maximum independent set (MIS) problem, inspired by dynamic programming (DP). Specifically, given a graph,…
Multilinear Latent Conditioning for Generating Unseen Attribute Combinations
Markos Georgopoulos, Grigorios Chrysos, Maja Pantic +1
Deep generative models rely on their inductive bias to facilitate generalization, especially for problems with high dimensional data, like images. However, empirical studies have s…
Deep Polynomial Neural Networks
Grigorios Chrysos, Stylianos Moschoglou, Giorgos Bouritsas +3
Deep Convolutional Neural Networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. T…
PolyGAN: High-Order Polynomial Generators
Grigorios Chrysos, Stylianos Moschoglou, Yannis Panagakis +1
Generative Adversarial Networks (GANs) have become the gold standard when it comes to learning generative models for high-dimensional distributions. Since their advent, numerous va…