From the 1 of 10 linked papers with an AI index.
10 papers
-OPSD: Deriving with Policy Optimization, Training with Self-Distillation
Jiawei Xu, Minghui Liu, Juzheng Zhang +2
The paper proposes β‑OPSD, a generalized on‑policy self‑distillation method that treats the KL regularization weight as a tunable parameter, enabling a controlled interpolation bet…
Not All LLM Reasoning is Visible in the Chain-of-Thought
Vatsal Baherwani, Tom Goldstein, Ashwinee Panda
A key question for AI safety is whether a language model expresses all of its reasoning in its output tokens. We demonstrate a concrete failure mode where frontier models exhibit i…
Speculating Experts Accelerates Inference for Mixture-of-Experts
Vivan Madan, Prajwal Singhania, Abhinav Bhatele +2
Mixture-of-Experts (MoE) models have gained popularity as a means of scaling the capacity of large language models (LLMs) while maintaining sparse activations and reduced per-token…
GATES: Self-Distillation under Privileged Context with Consensus Gating
Alex Stein, Furong Huang, Tom Goldstein
We study self-distillation in settings where supervision is unreliable: there are no ground truth labels, verifiable rewards, or external graders to evaluate answers. We focus on d…
Image Generation with a Sphere Encoder
Kaiyu Yue, Menglin Jia, Ji Hou +1
We introduce the Sphere Encoder, an efficient generative framework capable of producing images in a single forward pass and competing with many-step diffusion models using fewer th…
FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges
Kevin David Hayes, Micah Goldblum, Vikash Sehwag +3
Text-to-image (T2I) models are capable of generating visually impressive images, yet they often fail to accurately capture specific attributes in user prompts, such as the correct…