6 papers
Efficient Pre-Training of LLMs through Truncated SVD Layers
Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3
The massive scaling of Large Language Models (LLMs) has made pretraining increasingly cost-prohibitive. While low-rank representation and orthonormal weight matrices could in princ…
Evolution With Purpose: Hierarchy-Informed Optimization of Whole-Brain Models
Hormoz Shahrzad, Niharika Gajawelli, Kaitlin Maile +2
Evolutionary search is well suited for large-scale biophysical brain modeling, where many parameters with nonlinear interactions and no tractable gradients need to be optimized. St…
TerraLingua: Emergence and Analysis of Open-endedness in LLM Ecologies
Giuseppe Paolo, Jamieson Warner, Hormoz Shahrzad +3
As autonomous agents increasingly operate in real-world digital ecosystems, understanding how they coordinate, form institutions, and accumulate shared culture becomes both a scien…
Solving a Million-Step LLM Task with Zero Errors
Elliot Meyerson, Giuseppe Paolo, Roberto Dailey +6
LLMs have achieved remarkable breakthroughs in reasoning, insights, and tool use, but chaining these abilities into extended processes at the scale of those routinely executed by h…
GPU-Accelerated Rule Evaluation and Evolution
Hormoz Shahrzad, Risto Miikkulainen
This paper introduces an innovative approach to boost the efficiency and scalability of Evolutionary Rule-based machine Learning (ERL), a key technique in explainable AI. While tra…
EVOTER: Evolution of Transparent Explainable Rule-sets
Hormoz Shahrzad, Babak Hodjat, Risto Miikkulainen
Most AI systems are black boxes generating reasonable outputs for given inputs. Some domains, however, have explainability and trustworthiness requirements that cannot be directly…