Incremental Knowledge Graphs
Pic source: Abelson Taylor’s blog
This project was directed towards the final course project requirement for COMP 550: Natural Language Processing course at McGill University.
Knowledge graphs (KGs) succinctly represent real-world facts as multi-relational graphs. A plethora of work exists in embedding the information in KG to a continuous vector space in order to obtain new facts and facilitate multiple down-stream NLP tasks.
Despite the popularity of the KG embedding problem, to the best of our knowledge, we find that no existing work handles dynamic/evolving knowledge graphs that incorporates facts about new entities.
In this project, we propose this problem as an incremental learning problem and propose solutions to obtain representations for new entities and also update the representations of old entities that share facts with these newer entities. The primary motive of this setup is to avoid relearning the knowledge graph embedding altogether with the occurrence of every new set of facts (triplets).
We build our solutions with TransE(Bordes et al.) as our base KG embedding model and evaluate the learned embeddings on facts associated with these new entities.
To this aim, we formulated two solutions; the first approach followed a finetuning based transfer-learning solution, and the second followed a model-agnostic meta-learning based approach with Graph Convolutional Networks (GCN). While our model-specific finetuning approach fared well, the proposed model independent approach failed to learn representations for a new entity.
We used OpenKE’s implementation for setting our model. For our task, we made changes to the TransE model, so that it can learn the representations of the new entities. We employed the FB20K dataset ( Xie et al., 2016 ) for our task. In addition to containing all the entities and relations from the FB15K dataset, this dataset also contains new entities which was required for our setup. We evaluate the models for link prediction, which aims to predict the missing h or t for a relation fact (h, r, t).
