27 citations · 61 across the 15 of their papers we have counts for
12 papers · 1 filter
EOE: Evolutionary Optimization of Experts for Training Language Models
Yingshi Chen
This paper presents an evolutionary framework for the training of large language models(LLM). The models are divided into several experts(sub-networks), which have the same structu…
Fast Block Linear System Solver Using Q-Learning Schduling for Unified Dynamic Power System Simulations
Yingshi Chen, Xinli Song, HanYang Dai +3
We present a fast block direct solver for the unified dynamic simulations of power systems. This solver uses a novel Q-learning based method for task scheduling. Unified dynamic si…
The Brownian motion in the transformer model
Yingshi Chen
Transformer is the state of the art model for many language and visual tasks. In this paper, we give a deep analysis of its multi-head self-attention (MHSA) module and find that: 1…
An iterative K-FAC algorithm for Deep Learning
Yingshi Chen
Kronecker-factored Approximate Curvature (K-FAC) method is a high efficiency second order optimizer for the deep learning. Its training time is less than SGD(or other first-order m…
A short note on the decision tree based neural turing machine
Yingshi Chen
Turing machine and decision tree have developed independently for a long time. With the recent development of differentiable models, there is an intersection between them. Neural t…
Attention augmented differentiable forest for tabular data
Yingshi Chen
Differentiable forest is an ensemble of decision trees with full differentiability. Its simple tree structure is easy to use and explain. With full differentiability, it would be t…
Learning Unsplit-field-based PML for the FDTD Method by Deep Differentiable Forest
Yingshi Chen, Naixing Feng
Alternative unsplit-filed-based absorbing boundary condition (ABC) computation approach for the finite-difference time-domain (FDTD) is efficiently proposed based on the deep diffe…
Deep differentiable forest with sparse attention for the tabular data
Yingshi Chen
We present a general architecture of deep differentiable forest and its sparse attention mechanism. The differentiable forest has the advantages of both trees and neural networks.…
LiteMORT: A memory efficient gradient boosting tree system on adaptive compact distributions
Yingshi Chen
Gradient boosted decision trees (GBDT) is the leading algorithm for many commercial and academic data applications. We give a deep analysis of this algorithm, especially the histog…
Express Wavenet -- a low parameter optical neural network with random shift wavelet pattern
Yingshi Chen
Express Wavenet is an improved optical diffractive neural network. At each layer, it uses wavelet-like pattern to modulate the phase of optical waves. For input image with n2 pixel…
An optical diffractive deep neural network with multiple frequency-channels
Yingshi Chen, Jinfeng Zhu
Diffractive deep neural network (DNNet) is a novel machine learning framework on the modulation of optical transmission. Diffractive network would get predictions at the speed of l…
A novel guided deep learning algorithm to design low-cost SPP films
Yingshi Chen, Jinfeng Zhu
The design of surface plasmon polaritons (SPP) films is an ill-posed inverse problem. There are many-to-one correspondence between the structures and user needs. We present a novel…