Image-based DBpedia Knowledge Graph - GSoC2022

Projects GSOC (Google Summer of Code)
1

DESCRIPTION: Image-based DBpedia Knowledge Graph

Goal: To create a knowledge graph consisting of images from articles and link it with the existing DBpedia knowledge graph enabling it to perform multiple image-based retrieval tasks.

Impact: The following tasks can be achieved using the image-based knowledge graph:

  • Given an input image, one can find relevant articles with similar images and related text entities.
  • The input images can be complemented with text embeddings that will enable showing images upon searching DBpedia using text.
  • The generated knowledge graph will enable DBpedia to show related articles based on semantic similarity between the images.
  • Current DBpedia articles contain few images. Upon using the knowledge graph generated, relevant images for an article can be searched from the web to populate the article.

Warm-up tasks: Create the knowledge graph for a subset of the articles and check the idea’s feasibility.

Project size (175h or 350h): This project broadly consists of two parts: a) creating the image-based knowledge graph and b) using the knowledge graph to advance the functionality of DBpedia. Part (a) should be done in 175h. However, to explore various usage of the knowledge graph (part (b)), more time will be required (350h).

Keywords: knowledge graph, images, image embeddings, text embeddings, retrieval, multi-modal knowledge graph, image-based search.

2

Hi sid2697,

Thank you for your interest in DBpedia. Image based and multimodal retrieval/similarity is an interesting new project idea.

For text based retrieval and/or embeddings based projects, I suggest please check the following existing works of DBpedia. As a warm up task, please clone them and try them out. You may want to use existing framework/architecture to build upon your idea:

You will require to craft a more detailed/specific proposal. And we are happy to help you build one. I suggest the following steps:

  1. Prepare a Google Docs draft. This is an example of a very nice proposal that was accepted a few years ago.
  2. Share the proposal with me over mail (fatma.nausheen@gmail.com) .
  3. Address the comments that the other mentors and I will leave.
  4. Submit the proposal to the official GSoC platform.

Cheers,
Nausheen

2 Likes
3

Hello everyone!

My name is Liang Runying, a student majoring in Automation from Xi 'an Jiaotong University, China. I have participated in two knowledge graph related projects (" multi-granularity evaluation technology of trial quality based on semantic analysis of case files “and” Knowledge fusion, security control and service technology oriented to intelligent factory "). The current research direction is construction and application of multi-modal knowledge graph.

I observed that this project mainly builds multimodal knowledge graph based on images and achieves mutual retrieval of images and text modes together with DBpedia, which is very relevant to my research direction. I am very interested in it and have the ability to contribute to DBpedia as a student of GSoC 22’.

Looking forward to discussing project ideas with you!

Liangrunying

4

Hi,

This is really interesting. I’d like to contribute as well. I’m working on a project which makes use of dbpedia and wikidata and aims to enrich semantic data by providing users with a convenient gui for entering new attributes. I’m also working on semantic search and this project relates to it.

Regards.