164 citations · 605 across the 16 of their papers we have counts for
16 papers
The Bias is in the Details: An Assessment of Cognitive Bias in LLMs
R. Alexander Knipper, Charles S. Knipper, Kaiqi Zhang +3
As Large Language Models (LLMs) are increasingly embedded in real-world decision-making processes, it becomes crucial to examine the extent to which they exhibit cognitive biases.…
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
NVIDIA, :, Johan Bjorck +40
General-purpose robots need a versatile body and an intelligent mind. Recent advancements in humanoid robots have shown great promise as a hardware platform for building generalist…
Scaling Optimal LR Across Token Horizons
Johan Bjorck, Alon Benhaim, Vishrav Chaudhary +2
State-of-the-art LLMs are powered by scaling -- scaling model size, dataset size and cluster size. It is economically infeasible to extensively tune hyperparameter for the largest…
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Jyoti Aneja, Hany Awadalla +126
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal test…
Language Is Not All You Need: Aligning Perception with Language Models
Shaohan Huang, Li Dong, Wenhui Wang +15
A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence. In this work, we introduce Kosmos-1, a Multim…
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
Wenhui Wang, Hangbo Bao, Li Dong +8
A big convergence of language, vision, and multimodal pretraining is emerging. In this work, we introduce a general-purpose multimodal foundation model BEiT-3, which achieves state…
Is High Variance Unavoidable in RL? A Case Study in Continuous Control
Johan Bjorck, Carla P. Gomes, Kilian Q. Weinberger
Reinforcement learning (RL) experiments have notoriously high variance, and minor details can have disproportionately large effects on measured outcomes. This is problematic for cr…
Towards Deeper Deep Reinforcement Learning with Spectral Normalization
Johan Bjorck, Carla P. Gomes, Kilian Q. Weinberger
In computer vision and natural language processing, innovations in model architecture that increase model capacity have reliably translated into gains in performance. In stark cont…
Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precision
Johan Bjorck, Xiangyu Chen, Christopher De Sa +2
Low-precision training has become a popular approach to reduce compute requirements, memory footprint, and energy consumption in supervised learning. In contrast, this promising ap…
Understanding Decoupled and Early Weight Decay
Johan Bjorck, Kilian Weinberger, Carla Gomes
Weight decay (WD) is a traditional regularization technique in deep learning, but despite its ubiquity, its behavior is still an area of active research. Golatkar et al. have recen…
Automatic Detection and Compression for Passive Acoustic Monitoring of the African Forest Elephant
Johan Bjorck, Brendan H. Rappazzo, Di Chen +3
In this work, we consider applying machine learning to the analysis and compression of audio signals in the context of monitoring elephants in sub-Saharan Africa. Earth's biodivers…
Understanding Batch Normalization
Johan Bjorck, Carla Gomes, Bart Selman +1
Batch normalization (BN) is a technique to normalize activations in intermediate layers of deep neural networks. Its tendency to improve accuracy and speed up training have establi…
Scalable Relaxations of Sparse Packing Constraints: Optimal Biocontrol in Predator-Prey Network
Johan Bjorck, Yiwei Bai, Xiaojian Wu +3
Cascades represent rapid changes in networks. A cascading phenomenon of ecological and economic impact is the spread of invasive species in geographic landscapes. The most promisin…
Surface acoustic wave unidirectional transducers for quantum applications
Maria K. Ekström, Thomas Aref, Johan Runeson +3
The conversion efficiency of electric microwave signals into surface acoustic waves in different types of superconducting transducers is studied with the aim of quantum application…
Phase-Mapper: An AI Platform to Accelerate High Throughput Materials Discovery
Yexiang Xue, Junwen Bai, Ronan Le Bras +8
High-Throughput materials discovery involves the rapid synthesis, measurement, and characterization of many different but structurally-related materials. A key problem in materials…
Automated Phase Mapping with AgileFD and its Application to Light Absorber Discovery in the V-Mn-Nb Oxide System
Santosh K. Suram, Yexiang Xue, Junwen Bai +8
Rapid construction of phase diagrams is a central tenet of combinatorial materials science with accelerated materials discovery efforts often hampered by challenges in interpreting…