Synergistic Learning in Domain Adaptation with a Known Link: A Minimax Understanding

Abstract

When can combining source and target data yield much better performance than using either alone? From a minimax viewpoint, writing $R(n,n_T)$ for the minimax risk from $n$ source and $n_T$ target observations, we ask when $R(n,n_T)$ decays strictly faster than both $R(n,0)$ and $R(0,n_T)$. We call this the Synergistic Learning Phenomenon (SLP) and quantify it by the Synergistic Advantage Ratio $\mathrm{SAR}=\min (R(n,0),R(0,n_T))/R(n,n_T)$. We study domain adaptation with a known link: source regression $f$, target regression $g=\mathcal{A}f$ for a known operator $\mathcal{A}$, and possibly different designs. Three links---identity, integration, and differentiation---and the density ratio $\rho =h_T/h_S$ determine whether synergy is possible, at what rate, and in which sample-size window. For the identity link (covariate shift) with a Beta$(a,1)$ source and uniform target, a known necessary condition for SLP is $a\ge 2+1/(2\beta )$, where $\beta$ is the smoothness of $f$ (Result I). An integral link $g(x)={\int }_0^{x}f(t)dt$ can yield the joint rate $(n{n}_T{)}^{-1/2}$ even for a bounded ratio (Result II), and under a Beta$(a,1)$ source has a phase diagram with cuts at $a=2$ and $a=3$: integral SLP for $1<a<2$, an extra level-split rate $n_T^{-1}$ for $2<a<3$, and covariate-shift-type spatial SLP for $a>3$ (Result III). A derivative link $g=\dot{f}$ synergizes only under a stronger singularity (Result IV), giving a sharp integration--differentiation asymmetry under a bounded ratio. Across all three links, synergy arises from role separation: each dataset resolves a distinct bottleneck associated with the link and designs. We characterize the sample-size windows for synergy and how SAR varies with $n$ and $n_T$. Simulations support the predicted phase structure, and a mirrored integration--differentiation analysis of real vehicle and smartphone signals illustrates the asymmetry.