4.3k citations · 4.8k across the 15 of their papers we have counts for
16 papers
Path Independent Equilibrium Models Can Better Exploit Test-Time Computation
Cem Anil, Ashwini Pokle, Kaiqu Liang +5
Designing networks capable of attaining better performance with an increased inference budget is important to facilitate generalization to harder problem instances. Recent efforts…
Stability of Weighted Majority Voting under Estimated Weights
Shaojie Bai, Dongxia Wang, Tim Muller +2
Weighted Majority Voting (WMV) is a well-known optimal decision rule for collective decision making, given the probability of sources to provide accurate information (trustworthine…
Deep Equilibrium Optical Flow Estimation
Shaojie Bai, Zhengyang Geng, Yash Savani +1
Many recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms by encouraging iterative refinements toward a…
Joint inference and input optimization in equilibrium networks
Swaminathan Gurumurthy, Shaojie Bai, Zachary Manchester +1
Many tasks in deep learning involve optimizing over the \emph{inputs} to a network to minimize or maximize some objective; examples include optimization over latent spaces in a gen…
On Training Implicit Models
Zhengyang Geng, Xin-Yu Zhang, Shaojie Bai +2
This paper focuses on training implicit models of infinite layers. Specifically, previous works employ implicit differentiation and solve the exact gradient for the backward propag…
AHEAD: Adaptive Hierarchical Decomposition for Range Query under Local Differential Privacy
Linkang Du, Zhikun Zhang, Shaojie Bai +4
For protecting users' private data, local differential privacy (LDP) has been leveraged to provide the privacy-preserving range query, thus supporting further statistical analysis.…
Stabilizing Equilibrium Models by Jacobian Regularization
Shaojie Bai, Vladlen Koltun, J. Zico Kolter
Deep equilibrium networks (DEQs) are a new class of models that eschews traditional depth in favor of finding the fixed point of a single nonlinear layer. These models have been sh…
SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models
Zaccharie Ramzi, Florian Mannel, Shaojie Bai +3
In recent years, implicit deep learning has emerged as a method to increase the effective depth of deep neural networks. While their training is memory-efficient, they are still si…
A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning
Yao-Hung Hubert Tsai, Shaojie Bai, Louis-Philippe Morency +1
In this report, we relate the algorithmic design of Barlow Twins' method to the Hilbert-Schmidt Independence Criterion (HSIC), thus establishing it as a contrastive learning approa…
A community-powered search of machine learning strategy space to find NMR property prediction models
Lars A. Bratholm, Will Gerrard, Brandon Anderson +26
The rise of machine learning (ML) has created an explosion in the potential strategies for using data to make scientific predictions. For physical scientists wishing to apply ML st…
Multiscale Deep Equilibrium Models
Shaojie Bai, Vladlen Koltun, J. Zico Kolter
We propose a new class of implicit networks, the multiscale deep equilibrium model (MDEQ), suited to large-scale and highly hierarchical pattern recognition domains. An MDEQ direct…
Deep Equilibrium Models
Shaojie Bai, J. Zico Kolter, Vladlen Koltun
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models co…
Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel
Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada +2
Transformer is a powerful architecture that achieves superior performance on various sequence learning tasks, including neural machine translation, language understanding, and sequ…
Multimodal Transformer for Unaligned Multimodal Language Sequences
Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang +3
Human language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors. However, two major challenges in modeling such multimo…
Trellis Networks for Sequence Modeling
Shaojie Bai, J. Zico Kolter, Vladlen Koltun
We present trellis networks, a new architecture for sequence modeling. On the one hand, a trellis network is a temporal convolutional network with special structure, characterized…
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Shaojie Bai, J. Zico Kolter, Vladlen Koltun
For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent…