Publications (9)
Homogeneous Vector Capsules Enable Adaptive Gradient Descent in Convolutional Neural Networks
Adam Byerly, Tatiana Kalganova
Capsules are the name given by Geoffrey Hinton to vector-valued neurons. Neural networks traditionally produce a scalar value for an activated neuron. Capsules, on the other hand,…
Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell
Taiming Lu, Muhan Gao, Kuai Yu +2
Large Language Models (LLMs) exhibit positional bias, struggling to utilize information from the middle or end of long contexts. Our study explores LLMs' long-context reasoning by…
On the Importance of Capturing a Sufficient Diversity of Perspective for the Classification of micro-PCBs
Adam Byerly, Tatiana Kalganova, Anthony J. Grichnik
We present a dataset consisting of high-resolution images of 13 micro-PCBs captured in various rotations and perspectives relative to the camera, with each sample labeled for PCB t…
Towards an Analytical Definition of Sufficient Data
Adam Byerly, Tatiana Kalganova
We show that, for each of five datasets of increasing complexity, certain training samples are more informative of class membership than others. These samples can be identified a p…
Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
Adam Byerly, Daniel Khashabi
Self-consistency (SC) improves the performance of large language models (LLMs) across various tasks and domains that involve short content. However, does this support its effective…
Tur[k]ingBench: A Challenge Benchmark for Web Agents
Kevin Xu, Yeganeh Kordi, Tanay Nayak +7
Can advanced multi-modal models effectively tackle complex web-based tasks? Such tasks are often found on crowdsourcing platforms, where crowdworkers engage in challenging micro-ta…
No Routing Needed Between Capsules
Adam Byerly, Tatiana Kalganova, Ian Dear
Most capsule network designs rely on traditional matrix multiplication between capsule layers and computationally expensive routing mechanisms to deal with the capsule dimensional…
GOLD PANNING: Strategic Context Shuffling for Needle-in-Haystack Reasoning
Adam Byerly, Daniel Khashabi
Large language models (LLMs) exhibit pronounced position bias in long-context needle-in-haystack problems, systematically prioritizing the location of information over its relevanc…
Class Density and Dataset Quality in High-Dimensional, Unstructured Data
Adam Byerly, Tatiana Kalganova
We provide a definition for class density that can be used to measure the aggregate similarity of the samples within each of the classes in a high-dimensional, unstructured dataset…