DESCRIPTION: Image-based DBpedia Knowledge Graph
Goal: To create a knowledge graph consisting of images from articles and link it with the existing DBpedia knowledge graph enabling it to perform multiple image-based retrieval tasks.
Impact: The following tasks can be achieved using the image-based knowledge graph:
- Given an input image, one can find relevant articles with similar images and related text entities.
- The input images can be complemented with text embeddings that will enable showing images upon searching DBpedia using text.
- The generated knowledge graph will enable DBpedia to show related articles based on semantic similarity between the images.
- Current DBpedia articles contain few images. Upon using the knowledge graph generated, relevant images for an article can be searched from the web to populate the article.
Warm-up tasks: Create the knowledge graph for a subset of the articles and check the idea’s feasibility.
Project size (175h or 350h): This project broadly consists of two parts: a) creating the image-based knowledge graph and b) using the knowledge graph to advance the functionality of DBpedia. Part (a) should be done in 175h. However, to explore various usage of the knowledge graph (part (b)), more time will be required (350h).
Keywords: knowledge graph, images, image embeddings, text embeddings, retrieval, multi-modal knowledge graph, image-based search.