4k citations · 8.3k across the 28 of their papers we have counts for
6 papers · 1 filter
OptiSeq: Ordering Examples On-The-Fly for In-Context Learning
Rahul Atul Bhope, Praveen Venkateswaran, K. R. Jayaram +3
Developers using LLMs and LLM-based agents in their applications have provided plenty of anecdotal evidence that in-context-learning (ICL) is fragile. In this paper, we show that i…
LLM-Mixer: Multiscale Mixing in LLMs for Time Series Forecasting
Md Kowsher, Md. Shohanur Islam Sobuj, Nusrat Jahan Prottasha +3
Time series forecasting remains a challenging task, particularly in the context of complex multiscale temporal patterns. This study presents LLM-Mixer, a framework that improves fo…
Mixtral of Experts
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux +23
We introduce Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model. Mixtral has the same architecture as Mistral 7B, with the difference that each layer is composed of 8…
Addressing Some Limitations of Transformers with Feedback Memory
Angela Fan, Thibaut Lavril, Edouard Grave +2
Transformers have been successfully applied to sequential, auto-regressive tasks despite being feedforward networks. Unlike recurrent neural networks, Transformers use attention to…
TorchBeast: A PyTorch Platform for Distributed RL
Heinrich Küttler, Nantas Nardelli, Thibaut Lavril +4
TorchBeast is a platform for reinforcement learning (RL) research in PyTorch. It implements a version of the popular IMPALA algorithm for fast, asynchronous, parallel training of R…
Efficient keyword spotting using dilated convolutions and gating
Alice Coucke, Mohammed Chlieh, Thibault Gisselbrecht +3
We explore the application of end-to-end stateless temporal modeling to small-footprint keyword spotting as opposed to recurrent networks that model long-term temporal dependencies…