Boltzmann machines for time-series
arXiv:1708.06004
Abstract
We review Boltzmann machines extended for time-series. These models often have recurrent structure, and back propagration through time (BPTT) is used to learn their parameters. The per-step computational complexity of BPTT in online learning, however, grows linearly with respect to the length of preceding time-series (i.e., learning rule is not local in time), which limits the applicability of BPTT in online learning. We then review dynamic Boltzmann machines (DyBMs), whose learning rule is local in time. DyBM's learning rule relates to spike-timing dependent plasticity (STDP), which has been postulated and experimentally confirmed for biological neural networks.
arXiv commentVersion 1.0.1. 33 pages. The topics covered in this paper are presented in Part III of IJCAI-17 tutorial on energy-based machine learning. https://researcher.watson.ibm.com/researcher/view_group.php?id=7834
References in corpus (5)
- STDP as presynaptic activity times rate of change of postsynaptic activity
- Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation
- Learning dynamic Boltzmann machines with spike-timing dependent plasticity
- Boltzmann machines and energy-based models
- Learning binary or real-valued time-series via spike-timing dependent plasticity