4 citations · 6 across the 8 of their papers we have counts for
9 papers
Robustifying Asynchronous SGD via Soft Throttling
Kaoru Otsuka, Maxime Meyer, Yuki Takezawa +2
Asynchronous SGD is a popular algorithm for distributed learning where each client's gradient update is applied on arrival. This leads to a speed-up, but also an increased vulnerab…
Effects of width-dependent model hyperparameters and -regularization on the loss landscape of two-layer ReLU networks
Haruka Eshima, Makoto Yamada
Understanding deep neural networks remains a central challenge in machine learning. In particular, the theoretical properties of even two-layer ReLU networks, especially in the pre…
Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation
Kaoru Otsuka, Yuki Takezawa, Makoto Yamada
Partial participation is essential for communication-efficient federated learning at scale, yet existing Byzantine-robust methods typically assume full client participation. In the…
Many-to-Many Matching via Sparsity Controlled Optimal Transport
Weijie Liu, Han Bao, Makoto Yamada +3
Many-to-many matching seeks to match multiple points in one set and multiple points in another set, which is a basis for a wide range of data mining problems. It can be naturally r…
Parameter-free Clipped Gradient Descent Meets Polyak
Yuki Takezawa, Han Bao, Ryoma Sato +2
Gradient descent and its variants are de facto standard algorithms for training machine learning models. As gradient descent is sensitive to its hyperparameters, we need to tune th…
PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis
Satoki Ishikawa, Makoto Yamada, Han Bao +1
Predictive coding is a theory which hypothesises that cortex predicts sensory inputs at various levels of abstraction to minimise prediction errors. Inspired by predictive coding,…
Embarrassingly Simple Text Watermarks
Ryoma Sato, Yuki Takezawa, Han Bao +2
We propose Easymark, a family of embarrassingly simple yet effective watermarks. Text watermarking is becoming increasingly important with the advent of Large Language Models (LLM)…
An Empirical Study of Self-supervised Learning with Wasserstein Distance
Makoto Yamada, Yuki Takezawa, Guillaume Houry +4
In this study, we delve into the problem of self-supervised learning (SSL) utilizing the 1-Wasserstein distance on a tree structure (a.k.a., Tree-Wasserstein distance (TWD)), where…
Necessary and Sufficient Watermark for Large Language Models
Yuki Takezawa, Ryoma Sato, Han Bao +2
In recent years, large language models (LLMs) have achieved remarkable performances in various NLP tasks. They can generate texts that are indistinguishable from those written by h…