I update FR mapping and I want to check its impact on a dedicated Wikipedia page where I know this mapping applies. For that I launch triple extraction but I see no difference. This result is ambigous: I dont know if I am faulty somewhere, or if the triples shown no longer correspond to the new defined mapping and are obsolete needing to be refreshed. Should I wait next day to get the updated triples ? or may we force the analysis of a Wikipedia page in someway ?
@jimkont do you know something about this - how is this “live testing” supposed to work? it says “test mappings” in the mappings wiki but this uses the dief tools deployment which is restarted every day. so only over midnight the mappings and ontologies are fetched again. is there a way to fetch this on-demand instead without a performing a restart?
@christianwia in the meanwhile you can try your changes live locally just call https://github.com/dbpedia/extraction-framework/blob/master/redeploy-server after every change and use the web UI on http://localhost:9999/server/
Hi, can you explain a bit what you are trying to map and what would be the expected output of this mapping in the Men in Black article?
Asking because the template you are mapping is not a top-level infobox, it is usually nested in the main infobox or defined later in the page. It has been a while since I defined complex mappings like this but many an intermediate node mapping (similar to what you defined at the first versions) would fit better? (but I could be wrong, needs some testing)
wrt to updating, I think there was a callback function that was triggering the server to refresh on each change in the mappings or the ontology in the wiki, if we moved the server to a different location this could have been affected and not refreshing.
Testing this locally should be easy, just check out the code and run cd server && ../run server which should open the server on http://localhost:9999/server/ as @jfrey suggested
thx @jimkont. any pointers on this callback? I am a bit lost here. I tried to run it without mappings loaded locally although docu says # mappings dir. if it doesn't exist, load from server
https://github.com/dbpedia/extraction-framework/blob/4f3e6f1dfd9fe1b06f072bb2efcb7927cb5f8f4d/server/server.default.properties#L17 it just crashes and complains about missing mappings. so I don’t know about this callback. is the callback just trigerring a refresh of these files and then DIEF server will do a hot-reload automatically, because then I could just write a cronjob that fetches the mappings every minute.
That was created way before I was involved but from what I recall the mediawiki was patched to hit some server url whenever there was a change.
Looking at the Extraction Server we have these functions defined:
def updateOntologyPage(page : WikiPage)
def removeOntologyPage(title : WikiTitle)
def updateMappingPage(page : WikiPage, language : Language)
def removeMappingPage(title : WikiTitle, language : Language)
def updateAll()
for each there is a corresponding resource definition, e.g. in Mappings.scala we have:
@PUT
@Path("pages/{title: .+$}")
@Consumes(Array("application/xml"))
def putPage(@PathParam("title") title : String, pageXML : Elem)
my guess is that the mediawiki is set to hit these endpoints whenever there is a change in a mappings/ontology page. Adjusting the PHP code and changing the host to the new server should do the trick (or something similar)
Doing a reload would also work but would waste a lot of resources when the wiki is idle. Some optimization like checking the last wiki modification date should be better
1 Like
thanks this helped a lot. as a resultI found this here, and will check with the admins.
Hi all, nice to have all these informations but to progress I must say I enter the subject of mapping and I use the provided web interfaces on Windows.I am still wondering to guess the correct form I must provide for the mapping and if the system is not reactive enough (statistics frozen 2 days) it becomes hopeless. Documentation does not speak about restrictions concerning such a delay and suggests changes are immediately apply.
The windows I have provide a bash interface under cygwin:
$ bash --version
GNU bash, version 4.4.12(3)-release (i686-pc-cygwin)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
but is not strong enough to start servers or wikipedia on my pc.
So the best way for me would be as possible if anyone could deblock directly on the server (via cron, manual command on demand, ssh…) to get rid of the refresh problem and allow to concentrate on the mapping by itself
Thanks.
Good news I understood the functionality on the Extraction tool side, I am back from vacation and sent an email to the admin hosting the mappings wiki server to hopefully recover the callbacks.
In the meanwhile you can manually trigger the updates using HTTP e.g. with curl I managed to the following
curl -X PUT -H "Content-Type: application/xml" -d @mapping.xml http://dief.tools.dbpedia.org/server/mappings/en/pages/Mapping_en:Infobox_country
you can download the mapping.xml with curl like this
curl http://mappings.dbpedia.org/index.php/Special:Export/Mapping_en:Infobox_country > mapping.xml
1 Like
good news will try. thanks.
C:\Users… \curl-7.80.0-win32-mingw\bin>curl Mapping en:Infobox country > mapping.xml
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 27257 0 27257 0 0 112k 0 --:–:-- --:–:-- --:–:-- 113kC:\Users…\curl-7.80.0-win32-mingw\bin>ls
curl-ca-bundle.crt curl.exe gui_curl.exe libcurl.def libcurl.dll mapping.xml proxy.cfg target.cfgC:\Users…\curl-7.80.0-win32-mingw\bin>curl -X PUT -H “Content-Type: application/xml” -d @mapping.xml Mapping en:Infobox country
And xml contents with mapping rules for this infobox :
…
| defaultMappings =
{{PropertyMapping | templateProperty = name | ontologyProperty = foaf:name}}
{{PropertyMapping | templateProperty = common_name | ontologyProperty = foaf:name }}
{{PropertyMapping | templateProperty = linking_name | ontologyProperty = foaf:name }}
{{PropertyMapping | templateProperty = native_name | ontologyProperty = originalName }}
{{PropertyMapping | templateProperty = conventional_long_name | ontologyProperty = foaf:name }}
{{PropertyMapping | templateProperty = named_after | ontologyProperty = namedAfter }}
{{PropertyMapping | templateProperty = abbreviation | ontologyProperty = abbreviation}}
{{PropertyMapping | templateProperty = formerly | ontologyProperty = alternativeName}}
{{PropertyMapping | templateProperty = affiliations | ontologyProperty = affiliation}}
{{PropertyMapping | templateProperty = organization_name | ontologyProperty = foaf:name }}
…
But this is what we already can do via the Web interface without export/import ing the xml file. So what is the added value ?.
this is a workaround to actually try out your latest change(s) to the mapping, by syncing it manually and then using the “triple extraction GUI”.
so this answers your question: “how do I test them”
update: the callbacks seem to work for ontology changes (e.g. i added a label to a class and it was updated live) but it does not work for Mappings for some reason.
So unfortunately the workaround still needs to be applied.
Hi all, nice to have all these informations but to progress I must say I enter the subject of mapping and I use the provided web interfaces on Windows.I am still wondering to guess the correct form I must provide for the mapping and if the system is not reactive enough ([statistics frozen 2 days] it becomes hopeless. Documentation does not speak about restrictions concerning such a delay and suggests changes are immediately apply.
The windows I have provide a bash interface under cygwin:
$ bash --version
GNU bash, version 4.4.12(3)-release (i686-pc-cygwin)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
but is not strong enough to start servers or wikipedia on my pc.
So the best way for me would be as possible if anyone could deblock directly on the server (via cron, manual command on demand, ssh…) to get rid of the refresh problem and allow to concentrate on the mapping by itself
Thanks.