5 citations · 12 across the 13 of their papers we have counts for
14 papers
HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model
Noam Kayzer, Dan Revital, Ori Bar Joseph +10
We present Hebatron, a Hebrew-specialized open-weight large language model built on the NVIDIA Nemotron-3 sparse Mixture-of-Experts architecture. Training employs a three-phase eas…
Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs
Shaltiel Shmidman, Avi Shmidman, Amir DN Cohen +1
Open-weight LLMs have been released by frontier labs; however, sovereign Large Language Models (for languages other than English) remain low in supply yet high in demand. Training…
Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces
Shaltiel Shmidman, Asher Fredman, Oleg Sudakov +1
Test-time scaling, which leverages additional computation during inference to improve model accuracy, has enabled a new class of Large Language Models (LLMs) that are able to reaso…
NeoDictaBERT: Pushing the Frontier of BERT models for Hebrew
Shaltiel Shmidman, Avi Shmidman, Moshe Koppel
Since their initial release, BERT models have demonstrated exceptional performance on a variety of tasks, despite their relatively small size (BERT-base has ~100M parameters). Neve…
Splintering Nonconcatenative Languages for Better Tokenization
Bar Gazit, Shaltiel Shmidman, Avi Shmidman +1
Common subword tokenization algorithms like BPE and UnigramLM assume that text can be split into meaningful units by concatenative measures alone. This is not true for languages su…
Diversity Over Quantity: A Lesson From Few Shot Relation Classification
Amir DN Cohen, Shauli Ravfogel, Shaltiel Shmidman +1
In few-shot relation classification (FSRC), models must generalize to novel relations with only a few labeled examples. While much of the recent progress in NLP has focused on scal…
Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities
Shaltiel Shmidman, Avi Shmidman, Amir DN Cohen +1
Training large language models (LLMs) in low-resource languages such as Hebrew poses unique challenges. In this paper, we introduce DictaLM2.0 and DictaLM2.0-Instruct, two LLMs der…
MRL Parsing Without Tears: The Case of Hebrew
Shaltiel Shmidman, Avi Shmidman, Moshe Koppel +1
Syntactic parsing remains a critical tool for relation extraction and information extraction, especially in resource-scarce languages where LLMs are lacking. Yet in morphologically…
Introducing DictaLM -- A Large Generative Language Model for Modern Hebrew
Shaltiel Shmidman, Avi Shmidman, Amir David Nissan Cohen +1
We present DictaLM, a large-scale language model tailored for Modern Hebrew. Boasting 7B parameters, this model is predominantly trained on Hebrew-centric data. As a commitment to…
DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew
Shaltiel Shmidman, Avi Shmidman, Moshe Koppel
We present DictaBERT, a new state-of-the-art pre-trained BERT model for modern Hebrew, outperforming existing models on most benchmarks. Additionally, we release three fine-tuned v…
Large Pre-Trained Models with Extra-Large Vocabularies: A Contrastive Analysis of Hebrew BERT Models and a New One to Outperform Them All
Eylon Gueta, Avi Shmidman, Shaltiel Shmidman +6
We present a new pre-trained language model (PLM) for modern Hebrew, termed AlephBERTGimmel, which employs a much larger vocabulary (128K items) than standard Hebrew PLMs before. W…
Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language
Avi Shmidman, Joshua Guedalia, Shaltiel Shmidman +3
We present a new pre-trained language model (PLM) for Rabbinic Hebrew, termed Berel (BERT Embeddings for Rabbinic-Encoded Language). Whilst other PLMs exist for processing Hebrew t…
A Novel Challenge Set for Hebrew Morphological Disambiguation and Diacritics Restoration
Avi Shmidman, Joshua Guedalia, Shaltiel Shmidman +2
One of the primary tasks of morphological parsers is the disambiguation of homographs. Particularly difficult are cases of unbalanced ambiguity, where one of the possible analyses…
Nakdan: Professional Hebrew Diacritizer
Avi Shmidman, Shaltiel Shmidman, Moshe Koppel +1
We present a system for automatic diacritization of Hebrew text. The system combines modern neural models with carefully curated declarative linguistic knowledge and comprehensive…