Hi everyone,
I will monitor this thread to answer questions concerning the deployment of DBpedia Chapter services and applications.
I also invite the maintainers of successfully deployed Chapters and anyone who knows their way around the DBpedia Stack to stick around share their tips and tricks!
Typical hardware requirements for a national chapter?
Hi,
I would be interested in the recommended amount of disk space, CPU, GPU, RAM, etc. for running a smaller national chapter.
Hi micsik,
The amount of disk space really depends on the chapter data. The latest core collection in English has a size of 10 GB, so the index structure might be well over 200 GB or more. At least 16-32 GB of RAM won’t hurt for caches. I currently do not have any super precise values here.
The DBpedia Lookup only requires around 4-5 GB of RAM when running and 5 GB of Disk Space for the index structures
The DBpedia Spotlight requires around 50 GB of RAM and around 30 GB of Disk Space.
The whole Stack profits from multi-threading, so a lot of CPU cores an threads are helpful here.
Maybe someone can give more precise values but I would suggest
> 128 GB of RAM
Any modern CPU with 8 cores or more
> 500 GB of Disk space
There is no heavy rendering done in the stack, so a strong GPU is not required. If you intend to build your own lookup index structure (which might be required for your chapter), you can either build the index structure on disk (which is slow) or build in memory (a lot faster) - this will however require up to 200 GBs of RAM, so only choose this option if you have that much available.
Hi, I am trying to install the endpoint with the instructions on GitHub - dbpedia/virtuoso-sparql-endpoint-quickstart: creates a docker image with Virtuoso preloaded with the latest DBpedia dataset
But every time it ends with
load_1 |
load_1 | Done. -- 198 msec.
load_1 |
load_1 | [INFO] update/filling of geo index
load_1 |
load_1 | Done. -- 19 msec.
load_1 |
load_1 | [INFO] making checkpoint...
store_1 | 12:45:32 Checkpoint started
store_1 | 12:45:32 Checkpoint finished, log reused
load_1 |
load_1 | Done. -- 47 msec.
load_1 |
load_1 | [INFO] bulk load done; terminating loader
virtuoso-sparql-endpoint-quickstart_load_1 exited with code 0
I have tried three times and it always shows this and I check that the data is not incompletely loaded.
What can I do with it?
Ubuntu 20.4 with docker compose 1.29.2 and Docker version 20.10.7
Thanks a lot!
Hi, I can not reproduce this. Please make sure that you use latest compose from git and latest images with docker-compose pull . If this is still happening provide full logs, +command, +config, + content (ls -l) of the download folder please.
I recheck the logs and find that it might be the downloading progress and I rerun it to see whether it could happens again. sorry to say, but it happens again.
wget https://databus.dbpedia.org/dbpedia/generic/redirects/2021.05.01/redirects_lang=en_transitive.ttl.bz2"but it seems that it will change the file name when saving.
So the question is:
Is it possible to run “wget” to download rather than “download” progress in docker-compose.yml?If so, what is the naming rules when saving?
Thanks a lot!
the hashes appended to the file names are only used to prevent file name clashes.
yes you can copy RDF files “manually” to the download folder. you can then run docker-compose up store load no need to modify the compose file for this.
Given the screenshots you provide, I think, there could be a problem with the proxy, could you try without?
What collection did you use?
Hi! I’m trying to set up a new DBpedia chapter server. To do that, I’ve launched docker-compose up on Virtuoso quickstart.
How can I access the web / Linked Data view of DBpedia resources published by this VOS instance? Do I understand it correctly that VOS DBpedia Plugin provides a human-friendly HTML view of DBpedia resources?
I can access the VOS Conductor UI (on port 8890) but that’s not what I am looking for here.
