4 citations · 4 across the 6 of their papers we have counts for
6 papers
Making the Most of your Model: Methods for Finetuning and Applying Pretrained Transformers
Davis Yoshida
This thesis provides methods and analysis of models which make progress on this goal. The techniques outlined are task agnostic, and should provide benefit when used with nearly an…
Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizers
Lütfi Kerem Senel, Besnik Fetahu, Davis Yoshida +5
Recommender systems are widely used to suggest engaging content, and Large Language Models (LLMs) have given rise to generative recommenders. Such systems can directly generate ite…
MAP's not dead yet: Uncovering true language model modes by conditioning away degeneracy
Davis Yoshida, Kartik Goyal, Kevin Gimpel
It has been widely observed that exact or approximate MAP (mode-seeking) decoding from natural language generation (NLG) models consistently leads to degenerate outputs (Holtzman e…
NF4 Isn't Information Theoretically Optimal (and that's Good)
Davis Yoshida
This note shares some simple calculations and experiments related to absmax-based blockwise quantization, as used in Dettmers et al., 2023. Their proposed NF4 data type is said to…
Reconsidering the Past: Optimizing Hidden States in Language Models
Davis Yoshida, Kevin Gimpel
We present Hidden-State Optimization (HSO), a gradient-based method for improving the performance of transformer language models at inference time. Similar to dynamic evaluation (K…
Adding Recurrence to Pretrained Transformers for Improved Efficiency and Context Size
Davis Yoshida, Allyson Ettinger, Kevin Gimpel
Fine-tuning a pretrained transformer for a downstream task has become a standard method in NLP in the last few years. While the results from these models are impressive, applying t…