56 citations · 67 across the 12 of their papers we have counts for
13 papers
Contrastive Network Representation Learning
Zihan Dong, Xin Zhou, Ryumei Nakada +2
Network representation learning seeks to embed networks into a low-dimensional space while preserving the structural and semantic properties, thereby facilitating downstream tasks…
Residual Feature Integration is Sufficient to Prevent Negative Transfer
Yichen Xu, Ryumei Nakada, Linjun Zhang +1
Transfer learning has become a central paradigm in modern machine learning, yet it suffers from the long-standing problem of negative transfer, where leveraging source representati…
A Theoretical Framework for Prompt Engineering: Approximating Smooth Functions with Transformer Prompts
Ryumei Nakada, Wenlong Ji, Tianxi Cai +2
Prompt engineering has emerged as a powerful technique for guiding large language models (LLMs) toward desired responses, significantly enhancing their performance across diverse t…
SFT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
Xinyu Yang, Jixuan Leng, Geyang Guo +5
Current PEFT methods for LLMs can achieve either high quality, efficient training, or scalable serving, but not all three simultaneously. To address this limitation, we investigate…
PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers
Yibo Zhong, Haoxiang Jiang, Lincan Li +5
Fine-tuning large pre-trained foundation models often yields excellent downstream performance but is prohibitively expensive when updating all parameters. Parameter-efficient fine-…
Synthetic Oversampling: Theory and A Practical Approach Using LLMs to Address Data Imbalance
Ryumei Nakada, Yichen Xu, Lexin Li +1
Imbalanced classification and spurious correlation are common challenges in data science and machine learning. Both issues are linked to data imbalance, with certain groups of data…
Differentially Private Federated Learning: Servers Trustworthiness, Estimation, and Statistical Inference
Zhe Zhang, Ryumei Nakada, Linjun Zhang
Differentially private federated learning is crucial for maintaining privacy in distributed environments. This paper investigates the challenges of high-dimensional estimation and…
Contrastive Learning on Multimodal Analysis of Electronic Health Records
Tianxi Cai, Feiqing Huang, Ryumei Nakada +2
Electronic health record (EHR) systems capture a wealth of multimodal clinical data, encompassing both structured clinical codes and unstructured clinical notes. Yet, many EHR-focu…
Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training
Alyssa Huang, Peihan Liu, Ryumei Nakada +2
The surge in multimodal AI's success has sparked concerns over data privacy in vision-and-language tasks. While CLIP has revolutionized multimodal learning through joint training o…
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data
Ryumei Nakada, Halil Ibrahim Gulluk, Zhun Deng +3
Language-supervised vision models have recently attracted great attention in computer vision. A common approach to build such models is to use contrastive learning on paired data a…
The Power of Contrast for Feature Learning: A Theoretical Analysis
Wenlong Ji, Zhun Deng, Ryumei Nakada +2
Contrastive learning has achieved state-of-the-art performance in various self-supervised learning tasks and even outperforms its supervised counterpart. Despite its empirical succ…
Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networks
Ryumei Nakada, Masaaki Imaizumi
We investigate the asymptotic risk of a general class of overparameterized likelihood models, including deep models. The recent empirical success of large-scale models has motivate…
Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionality
Ryumei Nakada, Masaaki Imaizumi
In this study, we prove that an intrinsic low dimensionality of covariates is the main factor that determines the performance of deep neural networks (DNNs). DNNs generally provide…