3.4k citations · 5k across the 19 of their papers we have counts for
19 papers
Spontaneous symmetry breaking and Goldstone modes for deep information propagation
Nabil Iqbal, T. Anderson Keller, Yue Song +2
In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation o…
Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization
Shota Takashiro, Masanori Koyama, Takeru Miyato +3
We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.…
Artificial Kuramoto Oscillatory Neurons
Takeru Miyato, Sindy Löwe, Andreas Geiger +1
It has long been known in both neuroscience and AI that ``binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represe…
GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers
Takeru Miyato, Bernhard Jaeger, Max Welling +1
As transformers are equivariant to the permutation of input tokens, encoding the positional information of tokens is necessary for many tasks. However, since existing positional en…
Neural Fourier Transform: A General Approach to Equivariant Representation Learning
Masanori Koyama, Kenji Fukumizu, Kohei Hayashi +1
Symmetry learning has proven to be an effective approach for extracting the hidden structure of data, with the concept of equivariance relation playing the central role. However, m…
Invariance-adapted decomposition and Lasso-type contrastive learning
Masanori Koyama, Takeru Miyato, Kenji Fukumizu
Recent years have witnessed the effectiveness of contrastive learning in obtaining the representation of dataset that is useful in interpretation and downstream tasks. However, the…
Unsupervised Learning of Equivariant Structure from Sequences
Takeru Miyato, Masanori Koyama, Kenji Fukumizu
In this study, we present meta-sequential prediction (MSP), an unsupervised framework to learn the symmetry from the time sequence of length at least three. Our method leverages th…
Contrastive Representation Learning with Trainable Augmentation Channel
Masanori Koyama, Kentaro Minami, Takeru Miyato +1
In contrastive representation learning, data representation is trained so that it can classify the image instances even when the images are altered by augmentations. However, depen…
Robustness to Adversarial Perturbations in Learning from Incomplete Data
Amir Najafi, Shin-ichi Maeda, Masanori Koyama +1
What is the role of unlabeled data in an inference problem, when the presumed underlying distribution is adversarially perturbed? To provide a concrete answer to this question, thi…
Spatially Controllable Image Synthesis with Internal Representation Collaging
Ryohei Suzuki, Masanori Koyama, Takeru Miyato +2
We present a novel CNN-based image editing strategy that allows the user to change the semantic information of an image over an arbitrary region by manipulating the feature-space r…
Neural Multi-scale Image Compression
Ken Nakanishi, Shin-ichi Maeda, Takeru Miyato +1
This study presents a new lossy image compression method that utilizes the multi-scale features of natural images. Our model consists of two networks: multi-scale lossy autoencoder…
Spectral Normalization for Generative Adversarial Networks
Takeru Miyato, Toshiki Kataoka, Masanori Koyama +1
One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called sp…
cGANs with Projection Discriminator
Takeru Miyato, Masanori Koyama
We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the unde…
Parameter Reference Loss for Unsupervised Domain Adaptation
Jiren Jin, Richard G. Calland, Takeru Miyato +2
The success of deep learning in computer vision is mainly attributed to an abundance of data. However, collecting large-scale data is not always possible, especially for the superv…
Spectral Norm Regularization for Improving the Generalizability of Deep Learning
Yuichi Yoshida, Takeru Miyato
We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades t…
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama +1
We propose a new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input. Virtual adversarial l…
Learning Discrete Representations via Information Maximizing Self-Augmented Training
Weihua Hu, Takeru Miyato, Seiya Tokui +2
Learning discrete representations of data is a central machine learning task because of the compactness of the representations and ease of interpretation. The task includes cluster…
Evaluating Distributed Execution of Workloads
Matteo Turilli, Yadu Nand Babuji, Andre Merzky +4
Resource selection and task placement for distributed execution poses conceptual and implementation difficulties. Although resource selection and task placement are at the core of…
Distributional Smoothing with Virtual Adversarial Training
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama +2
We propose local distributional smoothness (LDS), a new notion of smoothness for statistical model that can be used as a regularization term to promote the smoothness of the model…