27 citations · 40 across the 32 of their papers we have counts for
16 papers · 1 filter
LEMON: Equivariant Learning over Attention Maps for Zero-Shot Misbehavior Monitoring on New LLMs
Guy Bar-Shalom, Mor Geva, Fabrizio Frasca +1
Large language models exhibit misbehaviors that hinder deployment in high-stakes settings. One approach to monitor such failures is to use an external LLM as a monitor, but this is…
Nova shells in 3D: I. A catalog of integral field spectroscopic observations
Janis B. Rodríguez-González, Edgar Santamaría, Martin A. Guerrero +1
Classical novae eject material during thermonuclear eruptions on accreting white dwarfs, forming expanding shells whose structures and kinematics provide insights into the eruption…
Graph Metanetworks for Transformer Based Language Models
Yonathan Wolanowsky, Guy Bar-Shalom, Fabrizio Frasca +1
Weight-space learning treats the weights of neural networks as data, processed by other models called metanetworks. A key challenge is that different weights can realize the same f…
CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space networks operate directly on parameters of other neural networks, enabling tasks such as predicting model properties, editing trained models, and generating weights. W…
Transferable Graph Metanetworks
Yuxin Ma, Adir Dayan, Yam Eitan +2
A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of…
Finite Probes Suffice: Identifiability and Universality for Weight-Space Learning
Soutrik Sarangi, Yonatan Sverdlov, Adir Dayan +2
Learning properties of neural networks has recently attracted growing interest, with existing approaches operating either directly on network parameters or through probe-based repr…
Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation
Arkadi Piven, Yam Eitan, Guy Bar-Shalom +5
A trained neural network can be represented by a parameter vector in high dimensions. Learning distributions over these vectors enables the generation of new models across various…
UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations
Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca +4
Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input. Effective mi…
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aaron Blakeman +572
We introduce Nemotron 3 Ultra, a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model. We pre-trained Nemotron 3 Ultra on 20 t…
Training Transformers for KV Cache Compressibility
Yoav Gelberg, Yam Eitan, Michael Bronstein +2
Long-context language modeling is increasingly constrained by the Key-Value (KV) cache, whose memory and decode-time access costs scale linearly with the prefix length. This bottle…
A Survey of Weight Space Learning: Understanding, Representation, and Generation
Xiaolong Han, Zehong Wang, Bo Zhao +8
Neural network weights are typically viewed as the end product of training, while most deep learning research focuses on data, features, and architectures. However, recent advances…
A Graph Meta-Network for Learning on Kolmogorov-Arnold Networks
Guy Bar-Shalom, Ami Tavory, Itay Evron +3
Weight-space models learn directly from the parameters of neural networks, enabling tasks such as predicting their accuracy on new datasets. Naive methods -- like applying MLPs to…
Spanning the Visual Analogy Space with a Weight Basis of LoRAs
Hila Manor, Rinon Gal, Haggai Maron +2
Visual analogy learning enables image editing via demonstration rather than textual description, allowing users to specify complex transformations difficult to articulate in words.…
SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
Yewei Liu, Xiyuan Wang, Yansheng Mao +4
We propose SHINE (Scalable Hyper In-context NEtwork), a scalable hypernetwork that can map diverse meaningful contexts into high-quality LoRA adapters for large language models (LL…
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
Adir Dayan, Yam Eitan, Haggai Maron
Weight-space learning studies neural architectures that operate directly on the parameters of other neural networks. Motivated by the growing availability of pretrained models, rec…
Learning from Historical Activations in Graph Neural Networks
Yaniv Galron, Hadar Sinai, Haggai Maron +1
Graph Neural Networks (GNNs) have demonstrated remarkable success in various domains such as social networks, molecular chemistry, and more. A crucial component of GNNs is the pool…