4 papers
JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures
Ariel Larey, Elay Dahan, Amit Bleiweiss +15
Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local s…
ProteinJEPA: Latent prediction complements protein language models
Dan Ofer, Dafna Shahaf, Michal Linial
Protein language models are trained primarily with masked language modeling (MLM), which predicts amino-acid identities at masked positions. We ask whether latent-space prediction…
ProtSent: Protein Sentence Transformers
Dan Ofer, Oriel Perets, Michal Linial +1
Protein language models (pLMs) produce per-residue representations that capture evolutionary and structural information, yet their mean-pooled sequence embeddings are not explicitl…
InterFeat: A Pipeline for Finding Interesting Scientific Features
Dan Ofer, Michal Linial, Dafna Shahaf
Finding interesting phenomena is the core of scientific discovery, but it is a manual, ill-defined concept. We present an integrative pipeline for automating the discovery of inter…