5 citations · 11 across the 10 of their papers we have counts for
10 papers
Shieldstral
Antonia Calvi, Avinash Sooriyarachchi, Giada Pistilli +273
We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7 its size on text safety benchmarks and set…
Voxtral Realtime
Mistral-AI, :, Alexander H. Liu +166
We introduce Voxtral Realtime, a natively streaming automatic speech recognition model that matches offline transcription quality at sub-second latency. Unlike approaches that adap…
Ministral 3
Alexander H. Liu, Kartik Khandelwal, Sandeep Subramanian +116
We introduce the Ministral 3 series, a family of parameter-efficient dense language models designed for compute and memory constrained applications, available in three model sizes:…
Devstral: Fine-tuning Language Models for Coding Agent Applications
Abhinav Rastogi, Adam Yang, Albert Q. Jiang +100
We introduce Devstral-Small, a lightweight open source model for code agents with the best performance among models below 100B size. In this technical report, we give an overview o…
Tensor-Train Point Cloud Compression and Efficient Approximate Nearest-Neighbor Search
Georgii Novikov, Alexander Gneushev, Alexey Kadeishvili +1
Nearest-neighbor search in large vector databases is crucial for various machine learning applications. This paper introduces a novel method using tensor-train (TT) low-rank tensor…
Inverted Activations: Reducing Memory Footprint in Neural Network Training
Georgii Novikov, Ivan Oseledets
The scaling of neural networks with increasing data and model sizes necessitates the development of more efficient deep learning algorithms. A significant challenge in neural netwo…
Efficient GPT Model Pre-training using Tensor Train Matrix Representation
Viktoriia Chekalina, Georgii Novikov, Julia Gusak +2
Large-scale transformer models have shown remarkable performance in language modelling tasks. However, such models feature billions of parameters, leading to difficulties in their…
Optimization of Functions Given in the Tensor Train Format
Andrei Chertkov, Gleb Ryzhakov, Georgii Novikov +1
Tensor train (TT) format is a common approach for computationally efficient work with multidimensional arrays, vectors, matrices, and discretized functions in a wide range of appli…
Few-Bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction
Georgii Novikov, Daniel Bershatsky, Julia Gusak +3
Memory footprint is one of the main limiting factors for large neural network training. In backpropagation, one needs to store the input to each operation in the computational grap…
Tensor-Train Density Estimation
Georgii S. Novikov, Maxim E. Panov, Ivan V. Oseledets
Estimation of probability density function from samples is one of the central problems in statistics and machine learning. Modern neural network-based models can learn high dimensi…