Image-based DBpedia Knowledge Graph - GSoC2022

DESCRIPTION: Image-based DBpedia Knowledge Graph

Goal: To create a knowledge graph consisting of images from articles and link it with the existing DBpedia knowledge graph enabling it to perform multiple image-based retrieval tasks.

Impact: The following tasks can be achieved using the image-based knowledge graph:

  • Given an input image, one can find relevant articles with similar images and related text entities.
  • The input images can be complemented with text embeddings that will enable showing images upon searching DBpedia using text.
  • The generated knowledge graph will enable DBpedia to show related articles based on semantic similarity between the images.
  • Current DBpedia articles contain few images. Upon using the knowledge graph generated, relevant images for an article can be searched from the web to populate the article.

Warm-up tasks: Create the knowledge graph for a subset of the articles and check the idea’s feasibility.

Project size (175h or 350h): This project broadly consists of two parts: a) creating the image-based knowledge graph and b) using the knowledge graph to advance the functionality of DBpedia. Part (a) should be done in 175h. However, to explore various usage of the knowledge graph (part (b)), more time will be required (350h).

Keywords: knowledge graph, images, image embeddings, text embeddings, retrieval, multi-modal knowledge graph, image-based search.

Hi sid2697,

Thank you for your interest in DBpedia. Image based and multimodal retrieval/similarity is an interesting new project idea.

For text based retrieval and/or embeddings based projects, I suggest please check the following existing works of DBpedia. As a warm up task, please clone them and try them out. You may want to use existing framework/architecture to build upon your idea:

You will require to craft a more detailed/specific proposal. And we are happy to help you build one. I suggest the following steps:

  1. Prepare a Google Docs draft. This is an example of a very nice proposal that was accepted a few years ago.
  2. Share the proposal with me over mail (fatma.nausheen@gmail.com) .
  3. Address the comments that the other mentors and I will leave.
  4. Submit the proposal to the official GSoC platform.

Cheers,
Nausheen

1 Like