37 citations · 91 across the 22 of their papers we have counts for
12 papers · 1 filter
A Comparative Analysis of LLM Adaptation: SFT, LoRA, and ICL in Data-Scarce Scenarios
Bernd Bohnet, Rumen Dangovski, Kevin Swersky +4
The remarkable capabilities of Large Language Models (LLMs) often need to be tailored for specific applications, requiring the integration of new knowledge or the acquisition of ne…
QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation
Zhuo Chen, Rumen Dangovski, Charlotte Loh +3
We propose Quantum-informed Tensor Adaptation (QuanTA), a novel, easy-to-implement, fine-tuning method with no inference overhead for large-scale pre-trained language models. By le…
Multimodal Foundation Models for Material Property Prediction and Discovery
Viggo Moro, Charlotte Loh, Rumen Dangovski +7
Artificial intelligence is transforming computational materials science, improving the prediction of material properties, and accelerating the discovery of novel materials. Recentl…
Model Stitching: Looking For Functional Similarity Between Representations
Adriano Hernandez, Rumen Dangovski, Peter Y. Lu +1
Model stitching (Lenc & Vedaldi 2015) is a compelling methodology to compare different neural network representations, because it allows us to measure to what degree they may be in…
Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries
Charlotte Loh, Seungwook Han, Shivchander Sudalairaj +5
Deep ensembles (DE) have been successful in improving model performance by learning diverse members via the stochasticity of random initialization. While recent works have attempte…
On the Importance of Calibration in Semi-supervised Learning
Charlotte Loh, Rumen Dangovski, Shivchander Sudalairaj +5
State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency…