Knowledge Tree
Home Research Docs About
BERT ↔ Knowledge Graph

Relations (1)

related 1.58 — strongly supporting 2 facts

Knowledge Graphs are integrated with BERT architectures to enhance representation learning, as seen in the KnowBERT model's KAR component [1] and the KRISP model's use of explicit knowledge graphs alongside BERT-pretrained transformers [2].

Facts (2)

Sources
Combining Knowledge Graphs and Large Language Models - arXiv arxiv.org arXiv 2 facts
referenceKnowBERT includes a Knowledge Attention and Recontextualization (KAR) component within the BERT architecture that computes a knowledge-enhanced representation using entity links from a Knowledge Graph and passes it to the next transformer block.
referenceKRISP uses a multimodal BERT-pretrained transformer to process question and image pairs in an implicit knowledge model, while a separate explicit knowledge model constructs a Knowledge Graph from question and image symbols to predict answers.