4 citations · 21 across the 14 of their papers we have counts for
15 papers
Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation
Kaoru Otsuka, Yuki Takezawa, Makoto Yamada
Partial participation is essential for communication-efficient federated learning at scale, yet existing Byzantine-robust methods typically assume full client participation. In the…
Any-stepsize Gradient Descent for Separable Data under Fenchel-Young Losses
Han Bao, Shinsaku Sakaue, Yuki Takezawa
The gradient descent (GD) has been one of the most common optimizer in machine learning. In particular, the loss landscape of a neural network is typically sharpened during the ini…
Parameter-free Clipped Gradient Descent Meets Polyak
Yuki Takezawa, Han Bao, Ryoma Sato +2
Gradient descent and its variants are de facto standard algorithms for training machine learning models. As gradient descent is sensitive to its hyperparameters, we need to tune th…
PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis
Satoki Ishikawa, Makoto Yamada, Han Bao +1
Predictive coding is a theory which hypothesises that cortex predicts sensory inputs at various levels of abstraction to minimise prediction errors. Inspired by predictive coding,…
Embarrassingly Simple Text Watermarks
Ryoma Sato, Yuki Takezawa, Han Bao +2
We propose Easymark, a family of embarrassingly simple yet effective watermarks. Text watermarking is becoming increasingly important with the advent of Large Language Models (LLM)…
An Empirical Study of Self-supervised Learning with Wasserstein Distance
Makoto Yamada, Yuki Takezawa, Guillaume Houry +4
In this study, we delve into the problem of self-supervised learning (SSL) utilizing the 1-Wasserstein distance on a tree structure (a.k.a., Tree-Wasserstein distance (TWD)), where…
Necessary and Sufficient Watermark for Large Language Models
Yuki Takezawa, Ryoma Sato, Han Bao +2
In recent years, large language models (LLMs) have achieved remarkable performances in various NLP tasks. They can generate texts that are indistinguishable from those written by h…
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence
Yuki Takezawa, Ryoma Sato, Han Bao +2
Decentralized learning has recently been attracting increasing attention for its applications in parallel computation and privacy preservation. Many recent studies stated that the…
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data
Yuki Takezawa, Han Bao, Kenta Niwa +2
SGD with momentum is one of the key components for improving the performance of neural networks. For decentralized learning, a straightforward approach using momentum is Distribute…
Approximating 1-Wasserstein Distance with Trees
Makoto Yamada, Yuki Takezawa, Ryoma Sato +3
Wasserstein distance, which measures the discrepancy between distributions, shows efficacy in various types of natural language processing (NLP) and computer vision (CV) applicatio…
Communication Compression for Decentralized Learning with Operator Splitting Methods
Yuki Takezawa, Kenta Niwa, Makoto Yamada
In decentralized learning, operator splitting methods using a primal-dual formulation (e.g., the Edge-Consensus Learning (ECL)) has been shown to be robust to heterogeneous data an…
Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization
Yuki Takezawa, Kenta Niwa, Makoto Yamada
In recent years, decentralized learning has emerged as a powerful tool not only for large-scale machine learning, but also for preserving privacy. One of the key challenges in dece…
Improving the Robustness to Variations of Objects and Instructions with a Neuro-Symbolic Approach for Interactive Instruction Following
Kazutoshi Shinoda, Yuki Takezawa, Masahiro Suzuki +2
An interactive instruction following task has been proposed as a benchmark for learning to map natural language instructions and first-person vision into sequences of actions to in…
Fixed Support Tree-Sliced Wasserstein Barycenter
Yuki Takezawa, Ryoma Sato, Zornitsa Kozareva +2
The Wasserstein barycenter has been widely studied in various fields, including natural language processing, and computer vision. However, it requires a high computational cost to…
Supervised Tree-Wasserstein Distance
Yuki Takezawa, Ryoma Sato, Makoto Yamada
To measure the similarity of documents, the Wasserstein distance is a powerful tool, but it requires a high computational cost. Recently, for fast computation of the Wasserstein di…