Publications (74)
A Tuned and Scalable Fast Multipole Method as a Preeminent Algorithm for Exascale Systems
Rio Yokota, Lorena Barba
FedPM: Federated Learning Using Second-order Optimization with Preconditioned Mixing of Local Parameters
Hiro Ishii, Kenta Niwa, Hiroshi Sawada +3
DGEMM on Integer Matrix Multiplication Unit
Hiroyuki Ootomo, Katsuhisa Ozaki, Rio Yokota
Communication Complexity of the Fast Multipole Method and its Algebraic Variants
Rio Yokota, George Turkiyyah, David Keyes
Informative Sample-Aware Proxy for Deep Metric Learning
Aoyu Li, Ikuro Sato, Kohta Ishikawa +2
Improving LoRA with Variational Learning
Bai Cong, Nico Daheim, Yuesong Shen +3
An FMM Based on Dual Tree Traversal for Many-core Architectures
Rio Yokota
Proxy-surface-based fast direct solver for TE-mode scattering problems on distributed memory systems
Yasuhiro Matsumoto, Rio Yokota
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
Kazuki Fujii, Yukito Tajima, Sakae Mizuki +14
Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
Taishi Nakamura, Takuya Akiba, Kazuki Fujii +3
Reducing shared memory footprint to leverage high throughput on Tensor Cores and its flexible API extension library
Hiroyuki Ootomo, Rio Yokota
Variational Learning Finds Flatter Solutions at the Edge of Stability
Avrajit Ghosh, Bai Cong, Rio Yokota +5
Recovering single precision accuracy from Tensor Cores while surpassing the FP32 theoretical peak performance
Hiroyuki Ootomo, Rio Yokota
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
Kazuki Osawa, Yohei Tsuji, Yuichiro Ueno +3
A Performance Model for the Communication in Fast Multipole Methods on HPC Platforms
Huda Ibeid, Rio Yokota, David Keyes
Asynchronous Execution of the Fast Multipole Method Using Charm++
Mustafa AbdulJabbar, Rio Yokota, David Keyes
Treecode and fast multipole method for N-body simulation with CUDA
Rio Yokota, Lorena Barba
Mixed-Precision Random Projection for RandNLA on Tensor Cores
Hiroyuki Ootomo, Rio Yokota
Data-Driven Execution of Fast Multipole Methods
Hatem Ltaief, Rio Yokota
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Kazuki Osawa, Yohei Tsuji, Yuichiro Ueno +3
Formula-Supervised Visual-Geometric Pre-training
Ryosuke Yamada, Kensho Hara, Hirokatsu Kataoka +4
Scaling Backwards: Minimal Synthetic Pre-training?
Ryo Nakamura, Ryu Tadokoro, Ryosuke Yamada +6
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Kazuki Osawa, Satoki Ishikawa, Rio Yokota +2
Fast Multipole Preconditioners for Sparse Matrices Arising from Elliptic Equations
Huda Ibeid, Rio Yokota, Jennifer Pestana +1
PowerCLIP: Powerset Alignment for Contrastive Pre-Training
Masaki Kawamura, Nakamasa Inoue, Rintaro Yanagi +2
$O(N)$ distributed direct factorization of structured dense matrices using runtime systems
Sameer Deshmukh, Qinxiang Ma, Rio Yokota +1
Quantum Turbulence Coupled with Externally Driven Normal-Fluid Turbulence in Superfluid $^4$He
Satoshi Yui, Hiromichi Kobayashi, Makoto Tsubota +1
Practical Deep Learning with Bayesian Principles
Kazuki Osawa, Siddharth Swaroop, Anirudh Jain +4
Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime
Hiroki Naganuma, Taiji Suzuki, Rio Yokota +3
Epipolar-Guided Deep Object Matching for Scene Change Detection
Kento Doi, Ryuhei Hamaguchi, Shun Iwase +3
On the Optimal Reasoning Length for RL-Trained Language Models
Daisuke Nohara, Taishi Nakamura, Rio Yokota
Effect of Mixed Precision Computing on H-Matrix Vector Multiplication in BEM Analysis
Rise Ooi, Takeshi Iwashita, Takeshi Fukaya +2
On the Relationship Between Double Descent of CNNs and Shape/Texture Bias Under Learning Process
Shun Iwase, Shuya Takahashi, Nakamasa Inoue +3
Quantum Circuit Simulation by SGEMM Emulation on Tensor Cores and Automatic Precision Selection
Hiroyuki Ootomo, Hidetaka Manabe, Kenji Harada +1
OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
Hana Hoshino, Kei Ota, Asako Kanezaki +1
Extreme Scale FMM-Accelerated Boundary Integral Equation Solver for Wave Scattering
Mustafa Abduljabbar, Mohammed Al Farhan, Noha Al-Harthi +4
Multi-Level Restricted Maximum Likelihood Covariance Estimation and Kriging for Large Non-Gridded Spatial Datasets
Julio E. Castrillon-Candas, Marc G. Genton, Rio Yokota
Masked Gated Linear Unit
Yukito Tajima, Nakamasa Inoue, Yusuke Sekikawa +2
Parallel QR Factorization of Block Low-Rank Matrices
M. Ridwan Apriansyah, Rio Yokota
Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation
Satoki Ishikawa, Rio Yokota, Ryo Karakida
Cache Optimization and Performance Modeling of Batched, Small, and Rectangular Matrix Multiplication on Intel, AMD, and Fujitsu Processors
Sameer Deshmukh, Rio Yokota, George Bosilca
Computing the k-th Eigenvalue of Symmetric $H^2$-Matrices
M. Ridwan Apriansyah, Rio Yokota
A Matrix-free Preconditioner for the Helmholtz Equation based on the Fast Multipole Method
Huda Ibeid, Rio Yokota, David Keyes
Accelerating Large Language Model Training with 4D Parallelism and Memory Consumption Estimator
Kazuki Fujii, Kohei Watanabe, Rio Yokota
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
LLM-jp, :, Akiko Aizawa +80
Communication Reducing Algorithms for Distributed Hierarchical N-Body Problems with Boundary Distributions
Mustafa Abduljabbar, George Markomanolis, Huda Ibeid +2
SegRCDB: Semantic Segmentation via Formula-Driven Supervised Learning
Risa Shinoda, Ryo Hayamizu, Kodai Nakashima +3
Beyond pass@k: Redundancy-Aware RLVR for Multi-Sample Code Generation
Le Bronnec Florian, Alexandre Verine, Rio Yokota +1
Variational Learning is Effective for Large Deep Networks
Yuesong Shen, Nico Daheim, Bai Cong +8
Fast Multipole Method as a Matrix-Free Hierarchical Low-Rank Approximation
Rio Yokota, Huda Ibeid, David Keyes
Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks
Taishi Nakamura, Satoki Ishikawa, Masaki Kawamura +4
Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMs
Koshiro Saito, Sakae Mizuki, Masanari Ohi +11
FMM-based vortex method for simulation of isotropic turbulence on GPUs, compared with a spectral method
Rio Yokota, L. A. Barba
Pre-training Vision Transformers with Very Limited Synthesized Images
Ryo Nakamura, Hirokatsu Kataoka, Sora Takashima +3
NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA
Marlon Tobaben, Mohamed Ali Souibgui, Rubèn Tito +24
Hierarchical N-body simulations with auto-tuning for heterogeneous systems
Rio Yokota, Lorena A. Barba
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
Youmi Ma, Sakae Mizuki, Kazuki Fujii +12
Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves
Sora Takashima, Ryo Hayamizu, Nakamasa Inoue +2
Scaling Large-scale GNN Training to Thousands of Processors on CPU-based Supercomputers
Chen Zhuang, Lingqi Zhang, Du Wu +8
RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering
Shun Iwase, Xingyu Liu, Rawal Khirodkar +2
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
Taishi Nakamura, Mayank Mishra, Simone Tedeschi +42
Evolutionary Context Search for Automated Skill Acquisition
Qi Sun, Stefan Nielsen, Rio Yokota +1
Building a Large Japanese Web Corpus for Large Language Models
Naoaki Okazaki, Kakeru Hattori, Hirai Shota +7
Variational Low-Rank Adaptation Using IVON
Bai Cong, Nico Daheim, Yuesong Shen +4
Scalable Linear Time Dense Direct Solver for 3-D Problems Without Trailing Sub-Matrix Dependencies
Qianxiang Ma, Sameer Deshmukh, Rio Yokota
An inherently parallel H2-ULV factorization for solving dense linear systems on GPUs
Qianxiang Ma, Rio Yokota
Rethinking Image Super-Resolution from Training Data Perspectives
Go Ohtani, Ryu Tadokoro, Ryosuke Yamada +7
Pre-training Vision Transformers with Formula-driven Supervised Learning
Hirokatsu Kataoka, Sora Takashima, Ryo Hayamizu +6
Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
Kazuki Fujii, Taishi Nakamura, Mengsay Loem +7
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Satoki Ishikawa, Tal Ben-Nun, Brian Van Essen +2
PetRBF--A parallel O(N) algorithm for radial basis function interpolation
Rio Yokota, L. A. Barba, Matthew G. Knepley
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization
Hiroki Naganuma, Kartik Ahuja, Shiro Takagi +5
Balancing Speed and Stability: The Trade-offs of FP8 vs. BF16 Training in LLMs
Kazuki Fujii, Taishi Nakamura, Rio Yokota
Biomolecular electrostatics using a fast multipole BEM on up to 512 GPUs and a billion unknowns
Rio Yokota, Jaydeep P. Bardhan, Matthew G. Knepley +2