Mistral 7B
arXiv:2310.06825
Abstract
We introduce Mistral 7B v0.1, a 7-billion-parameter language model engineered for superior performance and efficiency. Mistral 7B outperforms Llama 2 13B across all evaluated benchmarks, and Llama 1 34B in reasoning, mathematics, and code generation. Our model leverages grouped-query attention (GQA) for faster inference, coupled with sliding window attention (SWA) to effectively handle sequences of arbitrary length with a reduced inference cost. We also provide a model fine-tuned to follow instructions, Mistral 7B -- Instruct, that surpasses the Llama 2 13B -- Chat model both on human and automated benchmarks. Our models are released under the Apache 2.0 license.
arXiv commentModels and code are available at https://mistral.ai/news/announcing-mistral-7b/
Cited by in corpus (15)
- Large Language Models (LLMs) as Agents for Augmented Democracy
- Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review
- Challenges for Monocular 6D Object Pose Estimation in Robotics
- Domain Terminology Integration into Machine Translation: Leveraging Large Language Models
- The Last JITAI? Exploring Large Language Models for Issuing Just-in-Time Adaptive Interventions: Fostering Physical Activity in a Conceptual Cardiac Rehabilitation Setting
- HiPerRAG: High-Performance Retrieval Augmented Generation for Scientific Insights
- Large Language Models for Combinatorial Optimization: A Systematic Review
- Artificial Intelligence for the Internal Democracy of Political Parties
- Helping LLMs Improve Code Generation Using Feedback from Testing and Static Analysis
- Infrastructure Ombudsman: Mining Future Failure Concerns from Structural Disaster Response
- Don't Get Too Excited -- Eliciting Emotions in LLMs
- Evaluating the Reliability of Self-Explanations in Large Language Models
- Wrong Answers Can Also Be Useful: PlausibleQA -- A Large-Scale QA Dataset with Answer Plausibility Scores
- BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotations
- A Roadmap for Tamed Interactions with Large Language Models