Binarized Knowledge Graph Embeddings
arXiv:1902.02970
Abstract
Tensor factorization has become an increasingly popular approach to knowledge graph completion(KGC), which is the task of automatically predicting missing facts in a knowledge graph. However, even with a simple model like CANDECOMP/PARAFAC(CP) tensor decomposition, KGC on existing knowledge graphs is impractical in resource-limited environments, as a large amount of memory is required to store parameters represented as 32-bit or 64-bit floating point numbers. This limitation is expected to become more stringent as existing knowledge graphs, which are already huge, keep steadily growing in scale. To reduce the memory requirement, we present a method for binarizing the parameters of the CP tensor decomposition by introducing a quantization function to the optimization problem. This method replaces floating point-valued parameters with binary ones after training, which drastically reduces the model size at run time. We investigate the trade-off between the quality and size of tensor factorization models for several KGC benchmark datasets. In our experiments, the proposed method successfully reduced the model size by more than an order of magnitude while maintaining the task performance. Moreover, a fast score computation technique can be developed with bitwise operations.
References in corpus (11)
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- Complex Embeddings for Simple Link Prediction
- SimplE Embedding for Link Prediction in Knowledge Graphs
- Analogical Inference for Multi-Relational Embeddings
- A Survey on Methods and Theories of Quantized Neural Networks
- Question Answering with Subgraph Embeddings
- Canonical Tensor Decomposition for Knowledge Base Completion
- Logistic Tensor Factorization for Multi-Relational Data
- On the Equivalence of Holographic and Complex Embeddings for Link Prediction
- Word2Bits - Quantized Word Vectors