5 citations · 6 across the 7 of their papers we have counts for
9 papers
BitSparse: Compact Activation Storage for ReLU Feed-Forward Networks
Max Zhu, Bohang Sun, Pietro Lio +1
Reducing memory use in large language model (LLM) training is an ongoing challenge. Activation checkpointing saves memory by recomputation, while lossy compression changes the stor…
LCT: Losslessly Compressed Tensors with GPU friendly execution for Training
Max Zhu, Bohang Sun, Pietro Lio +1
Lossless tensor compression could reduce memory usage during training, but training requires tensors to be repeatedly re-encoded. We introduce Losslessly Compressed Tensors (LCT),…
The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
Bohang Sun, Max Zhu, Francesco Caso +5
Diffusion large language models promise faster generation by refining many token positions in parallel, but this parallelism introduces a hidden control problem: which proposed tok…
Challenges and Guidelines in Deep Generative Protein Design: Four Case Studies
Tianyuan Zheng, Alessandro Rondina, Gos Micklem +1
Deep generative models show promise for protein design, yet reliably producing designs that are geometrically plausible, evolutionarily consistent, functionally…
FLUID-LLM: Learning Computational Fluid Dynamics with Spatiotemporal-aware Large Language Models
Max Zhu, Adrián Bazaga, Pietro Liò
Learning computational fluid dynamics (CFD) traditionally relies on computationally intensive simulations of the Navier-Stokes equations. Recently, large language models (LLMs) hav…
HyperBERT: Mixing Hypergraph-Aware Layers with Language Models for Node Classification on Text-Attributed Hypergraphs
Adrián Bazaga, Pietro Liò, Gos Micklem
Hypergraphs are characterized by complex topological structure, representing higher-order interactions among multiple entities through hyperedges. Lately, hypergraph-based deep lea…
Language Model Knowledge Distillation for Efficient Question Answering in Spanish
Adrián Bazaga, Pietro Liò, Gos Micklem
Recent advances in the development of pre-trained Spanish language models has led to significant progress in many Natural Language Processing (NLP) tasks, such as question answerin…
SQLformer: Deep Auto-Regressive Query Graph Generation for Text-to-SQL Translation
Adrián Bazaga, Pietro Liò, Gos Micklem
In recent years, the task of text-to-SQL translation, which converts natural language questions into executable SQL queries, has gained significant attention for its potential to d…
Unsupervised Pretraining for Fact Verification by Language Model Distillation
Adrián Bazaga, Pietro Liò, Gos Micklem
Fact verification aims to verify a claim using evidence from a trustworthy knowledge base. To address this challenge, algorithms must produce features for every claim that are both…