HTTPS support for DBpedia APIs

doctorbud · July 15, 2019, 7:32am

I would like to be able to access DBpedia’s API services from my HTTPS-hosted web page, but it appears that there are only HTTP versions of the DBpedia API exposed. This causes a Mixed-Content security error in most modern browsers, and the typical solution is for the API provider to make their service available via HTTPS.

I’m trying to access the LDF service as described here:
http://fragments.dbpedia.org (note that this is HTTP)

But my website is HTTPS, so my browser prevents this security violation (Mixed Content). https://developer.mozilla.org/en-US/docs/Web/Security/Mixed_content#Mixed_active_content

I would like to know whether there is a HTTPS version of the above LDF API to DBpedia, and if someone could provide me the proper URL to access it.

If not, is there a plan to address this limitation? While I could use a proxy, it seems an unnecessary burden considering most browsers encourage HTTPS nowadays.

Thanks

kurzum · July 15, 2019, 6:38am

Thanks for spotting this, we are trying to consolidate research prototypes at the moment into these categories:

applications for download and self-deploy
shared services: high quality free services with no guaranteed sla
dedicated (paid)

I will ping Ruben who hosts LDF to get a statement.
Sebastian

RubenVerborgh · July 15, 2019, 2:06pm

Indeed. Good news is that we have scheduled HTTPS porting for all of our services this summer, including http://fragments.dbpedia.org.

kurzum · July 16, 2019, 10:51am

@RubenVerborgh We are collecting services around the data currently manually.

Could you complete your record here:

github.com

dbpedia/databus-content/blob/master/services.ttl#L98


      databus-sys:banner          <https://raw.githubusercontent.com/dbpedia/databus-content/master/img/dbpedia_spotlight_logo.jpg> ;
      # Your databus account uri
      databus-sys:publisher       <https://databus.dbpedia.org/dbpedia> ;
      # The URL that points to your service info page or signup form
      databus-sys:homepage        <https://www.dbpedia-spotlight.org/demo/> ;
      # Databus groups that can be used with your service
      databus-sys:targetGroup     <https://databus.dbpedia.org/dbpedia/mappings> ;
      databus-sys:targetGroup     <https://databus.dbpedia.org/marvin/wiki> ;
      databus-sys:targetGroup     <https://databus.dbpedia.org/marvin/mappings> .


<https://databus.dbpedia.org/system/service/imec/fragments>
      a                           databus-sys:Service ;
      a 				    databus-sys:SharedService ;
      dct:title                   "Linked Data Fragments over DBpedia"@en ;
      dct:description             "<h1>TODO</h1> <p>TODO</p>"^^rdf:HTML ;
      dct:abstract                "TODO"@en ;
      # TODO A small icon 128x128 px for your service
      databus-sys:icon            <https://raw.githubusercontent.com/dbpedia/databus-content/master/img/sparql.png> ;
      # TODO A banner for your service (640x320 px)
      databus-sys:banner          <https://raw.githubusercontent.com/dbpedia/databus-content/master/img/dbpedia_spotlight_logo.jpg> ;
      # TODO imec or RV? Databus account needs foaf profile

You can check how it looks here: https://databus.dbpedia.org/dbpedia/mappings/mappingbased-literals
updated every 30 min.

doctorbud · August 30, 2019, 9:05pm

The examples below display the Beatles’ members using the formerBandMember predicate…

It looks like the TPF server now works with https:
https://fragments.dbpedia.org/2016-04/en?subject=http%3A%2F%2Fdbpedia.org%2Fresource%2FThe_Beatles&predicate=http%3A%2F%2Fdbpedia.org%2Fontology%2FformerBandMember&object=
The Comunica Sparql client (and the old ldf-client) still fail when I use https:
https://query.linkeddatafragments.org/#transientDatasources=https%3A%2F%2Ffragments.dbpedia.org%2F2016-04%2Fen&query=SELECT%20%3Fbeatle%20WHERE%20{ %20%20<http%3A%2F%2Fdbpedia.org%2Fresource%2FThe_Beatles>%20<http%3A%2F%2Fdbpedia.org%2Fontology%2FformerBandMember>%20%3Fbeatle%20. }

My hypothesis is that even if I access query.linkeddatafragments.org via https, it is returning metadata that points to the http version of query.linkeddatafragments.org, so that the subsequent requests from within comunica-browser.js will be to the http version, which will fail due to mixed-content errors.

I’m going to explore hacking the proxy option to the Comunica engine, so that it takes a function (optionally) instead of a prefix string. This function can rewrite the http into https prior to the subsequent requests. @RubenVerborgh if there is a better way, please let me know (or just tell me to wait longer for y’all to fix it).

Thanks

RubenVerborgh · August 30, 2019, 9:18pm

This is indeed a problem on the server side, which we will fix.

doctorbud · August 30, 2019, 9:31pm

Thanks…

In the meantime, the httpProxyHandler option to Comunica.newEngine().query() is a workaround I can use to get my work done when I’m using the programmatic API

const httpProxyHandler = {
  getProxy: function getProxy(request) {
    request.input = request.input.replace('http://fragments.dbpedia.org', 'https://fragments.dbpedia.org');
    return request;
  },
};

…

Comunica.newEngine().query(
  query,
  {
    httpProxyHandler: httpProxyHandler,
    sources: [
      {
        type: '',
        value: 'https://fragments.dbpedia.org/2016-04/en'
      }
    ]
  })

RubenVerborgh · September 12, 2019, 11:33am

The server has been updated, so queries will work as expected.
Thanks again for reporting!

anandcv · December 14, 2019, 8:21am

Hi,
Same is happening with https://live.dbpedia.org/sparql/
In chrome it gives the error : net::ERR_CERT_DATE_INVALID

I think it is a certificate problem. In main domain, it is working without any error. (i.e. https://dbpedia.org/sparql/)

I hope this will be fixed…

valexande · December 14, 2019, 6:58pm

They have same maintenance issues try to access all day.

anandcv · December 15, 2019, 10:09am

The dbpedia SPARQL endpoint service is up now. But the problem persists…
live.dbpedia/sparql is still having problems…

pkleef · December 16, 2019, 10:38am

There was a hardware failure on live.dbpedia.org over the weekend . New part will be installed tonight which should restore operations. I apologize for any inconvenience.

anandcv · December 16, 2019, 12:27pm

Thanks for looking into it.
Please also look into the certificate issue…
Thanks in advance.

pkleef · December 17, 2019, 7:54am

@anandcv @valexande /CC @kurzum

Maintenance on https://live.dbpedia.org/sparql has been completed successfully.

Issues with the certificate have been resolved as well.

anandcv · December 17, 2019, 11:35am

Thank you for the quick resolution!
Works without any issues…

timbl · December 6, 2020, 2:34pm

Were all the APIs not moved to https:? I see the linked data one seems to redirect from https: to http:

$ curl -I  https://dbpedia.org/data/Massachusetts_Institute_of_Technology
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.18.0
Date: Sun, 06 Dec 2020 14:33:20 GMT
Content-Type: text/html
Content-Length: 145
Connection: keep-alive
Location: http://dbpedia.org/data/Massachusetts_Institute_of_Technology

timbl · December 6, 2020, 2:35pm

This means it can’t be accessed by browser scripts, as @doctorbud mentioned as the original post in this thread.

pkleef · December 8, 2020, 4:15pm

Note that the original forum post talked about fragments.dbpedia.org which is a different API than the dbpedia.org pages that @timbl checked.

On the current dbpedia.org instance we only enabled https support on the /sparql endpoint to deal with web applications which need an https endpoint in order to use Ajax / CORS to get results from queries.

The DBpedia Association and OpenLink Software are currently testing a new database based on the new Databus technology which will soon replace the dbpedia.org content. With the new database content, OpenLink will also enable https access on all endpoints under https://dbpedia.org/

A demo is currently available on https://dbpedia.demo.openlinksw.com/sparql/

$ curl -I  https://dbpedia.demo.openlinksw.com/data/Massachusetts_Institute_of_Technology
HTTP/2 200 
date: Tue, 08 Dec 2020 15:25:52 GMT
content-type: application/rdf+xml; charset=UTF-8
content-length: 961476
vary: Accept-Encoding
server: Virtuoso/08.03.3319 (Linux) x86_64-generic-linux-glibc25  VDB
accept-ranges: bytes
expires: Tue, 15 Dec 2020 16:25:51 GMT
content-location: Massachusetts_Institute_of_Technology.xml
link: <http://creativecommons.org/licenses/by-sa/3.0/>;rel="license",<http://dbpedia.demo.openlinksw.com/data/Massachusetts_Institute_of_Technology>; rel="alternate"; type="application/rdf+xml"; title="Structured Descriptor Document (RDF/XML format)", <http://dbpedia.demo.openlinksw.com/data/Massachusetts_Institute_of_Technology>; rel="alternate"; type="text/n3"; title="Structured Descriptor Document (N3/Turtle format)", <http://dbpedia.demo.openlinksw.com/data/Massachusetts_Institute_of_Technology>; rel="alternate"; type="application/json"; title="Structured Descriptor Document (RDF/JSON format)", <http://dbpedia.demo.openlinksw.com/data/Massachusetts_Institute_of_Technology>; rel="alternate"; type="application/atom+xml"; title="OData (Atom+Feed format)", <http://dbpedia.demo.openlinksw.com/data/Massachusetts_Institute_of_Technology>; rel="alternate"; type="application/odata+json"; title="OData (JSON format)", <http://dbpedia.demo.openlinksw.com/page/Massachusetts_Institute_of_Technology>; rel="alternate"; type="text/html"; title="XHTML+RDFa", <http://dbpedia.demo.openlinksw.com/resource/Massachusetts_Institute_of_Technology>; rel="http://xmlns.com/foaf/0.1/primaryTopic", <http://dbpedia.demo.openlinksw.com/resource/Massachusetts_Institute_of_Technology>; rev="describedby", <http://dbpedia.mementodepot.org/timegate/http://dbpedia.demo.openlinksw.com/data/Massachusetts_Institute_of_Technology>; rel="timegate"
x-sparql-default-graph: http://dbpedia.org
content-disposition: filename=sparql_2020-12-08_15-25-52Z.rdf
strict-transport-security: max-age=15768000