4 citations · 10 across the 10 of their papers we have counts for
11 papers
Contrastive Network Representation Learning
Zihan Dong, Xin Zhou, Ryumei Nakada +2
Network representation learning seeks to embed networks into a low-dimensional space while preserving the structural and semantic properties, thereby facilitating downstream tasks…
Residual Feature Integration is Sufficient to Prevent Negative Transfer
Yichen Xu, Ryumei Nakada, Linjun Zhang +1
Transfer learning has become a central paradigm in modern machine learning, yet it suffers from the long-standing problem of negative transfer, where leveraging source representati…
A Theoretical Framework for Prompt Engineering: Approximating Smooth Functions with Transformer Prompts
Ryumei Nakada, Wenlong Ji, Tianxi Cai +2
Prompt engineering has emerged as a powerful technique for guiding large language models (LLMs) toward desired responses, significantly enhancing their performance across diverse t…
A Statistical Hypothesis Testing Framework for Data Misappropriation Detection in Large Language Models
Yinpeng Cai, Lexin Li, Linjun Zhang
Large Language Models (LLMs) are rapidly gaining enormous popularity in recent years. However, the training of LLMs has raised significant privacy and legal concerns, particularly…
SFT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
Xinyu Yang, Jixuan Leng, Geyang Guo +5
Current PEFT methods for LLMs can achieve either high quality, efficient training, or scalable serving, but not all three simultaneously. To address this limitation, we investigate…
PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers
Yibo Zhong, Haoxiang Jiang, Lincan Li +5
Fine-tuning large pre-trained foundation models often yields excellent downstream performance but is prohibitively expensive when updating all parameters. Parameter-efficient fine-…
Synthetic Oversampling: Theory and A Practical Approach Using LLMs to Address Data Imbalance
Ryumei Nakada, Yichen Xu, Lexin Li +1
Imbalanced classification and spurious correlation are common challenges in data science and machine learning. Both issues are linked to data imbalance, with certain groups of data…
Contrastive Learning on Multimodal Analysis of Electronic Health Records
Tianxi Cai, Feiqing Huang, Ryumei Nakada +2
Electronic health record (EHR) systems capture a wealth of multimodal clinical data, encompassing both structured clinical codes and unstructured clinical notes. Yet, many EHR-focu…
Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training
Alyssa Huang, Peihan Liu, Ryumei Nakada +2
The surge in multimodal AI's success has sparked concerns over data privacy in vision-and-language tasks. While CLIP has revolutionized multimodal learning through joint training o…
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data
Ryumei Nakada, Halil Ibrahim Gulluk, Zhun Deng +3
Language-supervised vision models have recently attracted great attention in computer vision. A common approach to build such models is to use contrastive learning on paired data a…
The Power of Contrast for Feature Learning: A Theoretical Analysis
Wenlong Ji, Zhun Deng, Ryumei Nakada +2
Contrastive learning has achieved state-of-the-art performance in various self-supervised learning tasks and even outperforms its supervised counterpart. Despite its empirical succ…