32 citations · 49 across the 11 of their papers we have counts for
11 papers
StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
Tianyi Chen, Sihan Chen, Xiaoyi Qu +5
Quantization-aware training (QAT) is essential for deploying large models under strict memory and latency constraints, yet achieving stable and robust optimization at ultra-low bit…
Large Language Model Scaling Laws for Neural Quantum States in Quantum Chemistry
Oliver Knitter, Dan Zhao, Stefan Leichenauer +1
Scaling laws have been used to describe how large language model (LLM) performance scales with model size, training data size, or amount of computational resources. Motivated by th…
The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
Wei Du, Branislav Kisacanin, George Armstrong +22
Reasoning-capable language models achieve state-of-the-art performance in diverse complex tasks by generating long, explicit Chain-of-Thought (CoT) traces. While recent works show…
Comparative study of the ansätze in quantum language models
Jordi Del Castillo, Dan Zhao, Zongrui Pei
Quantum language models are the alternative to classical language models, which borrow concepts and methods from quantum machine learning and computational linguistics. While sever…
Layer by Layer: Uncovering Hidden Representations in Language Models
Oscar Skean, Md Rifat Arefin, Dan Zhao +4
From extracting features to generating text, the outputs of large language models (LLMs) typically rely on the final layers, following the conventional wisdom that earlier layers c…
Retentive Neural Quantum States: Efficient Ansätze for Ab Initio Quantum Chemistry
Oliver Knitter, Dan Zhao, James Stokes +3
Neural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative…
Improvements & Evaluations on the MLCommons CloudMask Benchmark
Varshitha Chennamsetti, Laiba Mehnaz, Dan Zhao +2
In this paper, we report the performance benchmarking results of deep learning models on MLCommons' Science cloud-masking benchmark using a high-performance computing cluster at Ne…
Entangling Machine Learning with Quantum Tensor Networks
Constantijn van der Poel, Dan Zhao
This paper examines the use of tensor networks, which can efficiently represent high-dimensional quantum states, in language modeling. It is a distillation and continuation of the…
Sustainable Supercomputing for AI: GPU Power Capping at HPC Scale
Dan Zhao, Siddharth Samsi, Joseph McDonald +5
As research and deployment of AI grows, the computational burden to support and sustain its progress inevitably does too. To train or fine-tune state-of-the-art models in NLP, comp…
From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference
Siddharth Samsi, Dan Zhao, Joseph McDonald +7
Large language models (LLMs) have exploded in popularity due to their new generative capabilities that go far beyond prior state-of-the-art. These technologies are increasingly bei…
Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks
Dan Zhao
Works on implicit regularization have studied gradient trajectories during the optimization process to explain why deep networks favor certain kinds of solutions over others. In de…