Curl error 52: Empty reply from server when trying to annotate with dbpedia spotlight docker

Support DBpedia Databus Support
,
27

Increasing the memory on the docker application to 12GB while also setting the cache to 4GB allows the model to start and the curl requests to work properly. That is a solution.

I just want to know if there are other solutions that do not demand such massive amount of memory to be allocated to a docker image, since I will be using that image along with other applications.

Thanks for the time and attention demanded.
Best Regards,
Lucas.

1 Like
28

Hi @lucas0,

Thanks for the updates!!, they are really helpful.

It seems that the problem is not related to Java memory. The problem is more related to Docker memory and how Mac OS handles it. Here is the link to docker documentation which mentions something about this (related to what you already found :+1: ):

  • Memory: By default, Docker Desktop is set to use 2 GB runtime memory, allocated from the total available memory on your Mac. To increase the RAM, set this to a higher number. To decrease it, lower the number.

On the contrary, for a Linux system, the limited memory is the memory of the host. Also, the Docker documentation for Linux does not mention anything about the limit of memory used for docker. That explains why in an 8 GB system, I was able to run the English model (by the way, it consumes all the RAM but it was able to run it).

Then, the amount of memory needed to run the English model is >8GB, and, in the case of Mac OS, the docker configuration for RAM is important to take into consideration.

Thanks again for the feedback, I hope this information helps you.

1 Like
29

Hey I am experiencing a similar issue.

I followed your advice on using the spotlight-compose.yml file. I also increased the RAM to 14 GB. ( Also used the lt-model because its the smallest for testing). The download seems to work according to the logs below:

$ docker logs dbpedia-spotlight.lt
Selected language: lt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
100  1410  100   119  100  1291     44    483  0:00:02  0:00:02 --:--:--   529
http://downloads.dbpedia.org/repo/dbpedia/spotlight/spotlight-model/2021.09.01/spotlight-model_lang=lt.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
100 34.5M  100 34.5M    0     0  3983k      0  0:00:08  0:00:08 --:--:-- 5087k
lt/
lt/spotter_thresholds.txt
lt/fsa_dict.mem
lt/model.properties
lt/model/
lt/model/sf.mem
lt/model/context.mem
lt/model/candmap.mem
lt/model/res.mem
lt/model/quantized_counts.mem
lt/model/tokens.mem
lt/stopwords.list
/opt/spotlight/models/lt http://0.0.0.0:80/rest/

But after this the rest of the logs are empty. According to your previous conversation the

Loading MemoryContextStore...

should start. But it does not.

You also said that once the log: dbpedia-spotlight.es | /opt/spotlight/es http://0.0.0.0:80/rest/ appears I am good to send a request. But the request still returns an empty reply from the server.

$ curl http://localhost:2224/rest/annotate \
   --data-urlencode \
   "text=President Obama called Wednesday on Congress to extend a tax break for students included in last year's economic stimulus package, arguing that the policy provides more generous assistance." \
--data "confidence=0.35" -H "Accept: text/turtle"
curl: (52) Empty reply from server

I really do not know what I am doing wrong. Help would be greatly appreciated since I spend two days on this problem.
Best wishes!

30

Hi @mingaflo,

Thanks for writing. My first question: are you using a Mac OS?. The output from the log must be something like this:

docker-compose -f spotlight-compose-it.yml up
Creating dbpedia-spotlight.it ... done
Attaching to dbpedia-spotlight.it
dbpedia-spotlight.it | Selected language: it
dbpedia-spotlight.it |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dbpedia-spotlight.it |                                  Dload  Upload   Total   Spent    Left  Speed
100  1410  100   119  100  1291     51    559  0:00:02  0:00:02 --:--:--   611
dbpedia-spotlight.it | http://downloads.dbpedia.org/repo/dbpedia/spotlight/spotlight-model/2021.09.01/spotlight-model_lang=it.tar.gz
dbpedia-spotlight.it |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dbpedia-spotlight.it |                                  Dload  Upload   Total   Spent    Left  Speed
100  576M  100  576M    0     0  1885k      0  0:05:13  0:05:13 --:--:-- 1126k
dbpedia-spotlight.it | it/
dbpedia-spotlight.it | it/spotter_thresholds.txt
dbpedia-spotlight.it | it/fsa_dict.mem
dbpedia-spotlight.it | it/model.properties
dbpedia-spotlight.it | it/model/
dbpedia-spotlight.it | it/model/sf.mem
dbpedia-spotlight.it | it/model/context.mem
dbpedia-spotlight.it | it/model/candmap.mem
dbpedia-spotlight.it | it/model/res.mem
dbpedia-spotlight.it | it/model/quantized_counts.mem
dbpedia-spotlight.it | it/model/tokens.mem
dbpedia-spotlight.it | it/stopwords.list
dbpedia-spotlight.it | /opt/spotlight/models/it http://0.0.0.0:80/rest/
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (38 ms)
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (2391 ms)
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (7710 ms)
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (5066 ms)
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (2606 ms)
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryContextStore...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (27716 ms)
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading FSADictionary...
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (510 ms)
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 1 disambiguators.
dbpedia-spotlight.it | [main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 2 spotters.

If you are using Mac OS the recommendation from Lucas could help:

Increasing the memory on the docker application to 12GB while also setting the cache to 4GB allows the model to start and the curl requests to work properly. That is a solution.

If any of this help, please let me know your system configuration and the example you are running. Thanks, have a great day.

My best regards

Julio

31

Hi Julio,

thank you so much for the reply.
Yes I am on MacOS (MacBook Pro with a M1 Chip if that matters) and I managed to find a way to make some progress just now. But only without docker:

 # download main jar
wget https://sourceforge.net/projects/dbpedia-spotlight/files/spotlight/dbpedia-spotlight-1.0.0.jar
# download latest model (assuming en model)
wget -O en.tar.gz http://downloads.dbpedia.org/repo/dbpedia/spotlight/spotlight-model/2020.11.18/spotlight-model_lang%3den.tar.gz
# extract model
tar xzf en.tar.gz
# run server
java -jar dbpedia-spotlight-1.0.0.jar en http://localhost:2222/rest

Doing that I encountered the issue that java was not installed so i got that and set the memory to export JAVA_OPTS=-Xmx16g. I was then able to at least not get an empty response but a javax.servlet.ServletException which I don’t understand.

But I am actually interested to get it running as a docker!
I set the docker application on Mac to this:

Bildschirmfoto 2021-12-09 um 14.35.53
Bildschirmfoto 2021-12-09 um 14.35.531002×936 46.3 KB

That should be more than enough.
I tried adding -e JAVA_OPTS='-Xmx16g' as well to the docker run command, with no success.

docker run -ti \
 -e JAVA_OPTS='-Xmx16g' --restart unless-stopped \
 --name dbpedia-spotlight.de \
 --mount source=spotlight-models,target=/opt/spotlight \
 -p 2222:80 \
 dbpedia/dbpedia-spotlight \
 spotlight.sh de

With docker-compose I managed to get part of the logs you are talking about. I got everything until dbpedia-spotlight.it | /opt/spotlight/models/it http://0.0.0.0:80/rest/ but everything following just does not show up.

But I do get them when running the above java-command:
java -jar dbpedia-spotlight-1.0.0.jar de http://localhost:2222/rest

This is the output of docker-compose (when running it again):

docker-compose -f spotlight-compose.yml up
[+] Running 2/2
 ⠿ Network spotlight-docker_default  Created                               0.0s
 ⠿ Container dbpedia-spotlight.lt    C...                                  0.1s
Attaching to dbpedia-spotlight.lt
dbpedia-spotlight.lt  | Selected language: lt
dbpedia-spotlight.lt  | /opt/spotlight/lt http://0.0.0.0:80/rest/

I am not able to insert anything suggesting something is happening in the background, but the logs only show this:

Selected language: lt
/opt/spotlight/lt http://0.0.0.0:80/rest/

Sending a curl just returns an empty reply.

Any ideas?

32

ok yeah the curl- Error looks like something is not downloading.

~ docker-compose -f spotlight.yml up
[+] Running 1/0
 ⠿ Container dbpedia-spotlight.de2  Created                                                                                              0.1s
Attaching to dbpedia-spotlight.de2
dbpedia-spotlight.de2  | Selected language: de
dbpedia-spotlight.de2  |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dbpedia-spotlight.de2  |                                  Dload  Upload   Total   Spent    Left  Speed
100  1410  100   119  100  1291    262   2849 --:--:-- --:--:-- --:--:--  3154
dbpedia-spotlight.de2  | http://downloads.dbpedia.org/repo/dbpedia/spotlight/spotlight-model/2021.09.01/spotlight-model_lang=de.tar.gz
dbpedia-spotlight.de2  |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
dbpedia-spotlight.de2  |                                  Dload  Upload   Total   Spent    Left  Speed


 95 1003M   95  959M    0     0  1525k      0  0:11:13  0:10:43  0:00:30  706k
dbpedia-spotlight.de2  | curl: (18) transfer closed with 46329422 bytes remaining to read
dbpedia-spotlight.de2  | de/
dbpedia-spotlight.de2  | tar: unexpected end of file
dbpedia-spotlight.de2  | tar: short read
dbpedia-spotlight.de2  | de/spotter_thresholds.txt
dbpedia-spotlight.de2  | de/fsa_dict.mem
dbpedia-spotlight.de2  | de/model.properties
dbpedia-spotlight.de2  | de/model/
dbpedia-spotlight.de2  | de/model/sf.mem
dbpedia-spotlight.de2  | de/model/context.mem
dbpedia-spotlight.de2  | de/model/candmap.mem
dbpedia-spotlight.de2  | de/model/res.mem
dbpedia-spotlight.de2  | /opt/spotlight/models/de http://0.0.0.0:80/rest/
33

Hi @mingaflo,

Thank you so much for your postings. According to your feedback, and considering the experience with Lucas, sometimes the download process fails. Then, it is necessary to alert if something wrong happens with the model downloading process. Thanks again for the feedback and the support information.

34

@JulioNoe What do you mean with “it is necessary to alert if something wrong happens”?

For smaller languages like it or lt the curl works always. But it still does not run the java commands I checked spotlight.sh in the docker. It seams like the error happens in there.
But because of the lack of echos its hard to track where to programm is.

#!/bin/sh

LANG=$1

MODELFOLDER=/opt/spotlight
cd $MODELFOLDER

DIRECTORY=/opt/spotlight/models/$LANG
echo "Selected language: $LANG"
if [ -d "$DIRECTORY" ]
then
     echo "/opt/spotlight/$LANG http://0.0.0.0:80/rest/"
     if [[ $LANG == "en" ]]
     then
	 java -Dfile.encoding=UTF-8 -Xmx15G -jar /opt/spotlight/dbpedia-spotlight.jar /opt/spotlight/models/$LANG http://0.0.0.0:80/rest
     else
	 java -Dfile.encoding=UTF-8 -Xmx10G -jar /opt/spotlight/dbpedia-spotlight.jar /opt/spotlight/models/$LANG http://0.0.0.0:80/rest
     fi

else
      QUERY="PREFIX dataid: <http://dataid.dbpedia.org/ns/core#>
      PREFIX dataid-cv: <http://dataid.dbpedia.org/ns/cv#>
      PREFIX dct: <http://purl.org/dc/terms/>
      PREFIX dcat:  <http://www.w3.org/ns/dcat#>

      SELECT DISTINCT ?file WHERE {
 	?dataset dataid:artifact <https://databus.dbpedia.org/dbpedia/spotlight/spotlight-model> .
	?dataset dcat:distribution ?distribution .
	{
		?distribution dct:hasVersion ?latestVersion
		{
			SELECT (?version as ?latestVersion) WHERE {
				?dataset dataid:artifact <https://databus.dbpedia.org/dbpedia/spotlight/spotlight-model> .
				?dataset dct:hasVersion ?version .
			} ORDER BY DESC (?version) LIMIT 1
		}
                ?distribution dataid:contentVariant '$LANG'^^xsd:string .
	}
	?distribution dcat:downloadURL ?file .
       }"

      RESULT=`curl --data-urlencode query="$QUERY" --data-urlencode format="text/tab-separated-values" https://databus.dbpedia.org/repo/sparql | sed 's/"//g' | grep -v "^file$" | head -n 1 `
      echo $RESULT
      curl -O  $RESULT
      tar -C /opt/spotlight/models -xvf spotlight-model_lang=$LANG.tar.gz
      rm spotlight-model_lang=$LANG.tar.gz
      echo "/opt/spotlight/models/$LANG http://0.0.0.0:80/rest/"
      if [[ $LANG == "en" ]]
      then
	java -Dfile.encoding=UTF-8 -Xmx15G -jar /opt/spotlight/dbpedia-spotlight.jar /opt/spotlight/models/$LANG http://0.0.0.0:80/rest
       else
	 java -Dfile.encoding=UTF-8 -Xmx10G -jar /opt/spotlight/dbpedia-spotlight.jar /opt/spotlight/models/$LANG http://0.0.0.0:80/rest
      fi

fi
/bin #

Since the directory does not exist during the first run, it goes into the else statement and echos
dbpedia-spotlight.it | /opt/spotlight/models/it http://0.0.0.0:80/rest/ but then the java call just doesn’t do anything. And without any error-messages I am quite lost here.

Why is that?

Also after that the CPU Usage is at 100%.

Bildschirmfoto 2021-12-09 um 17.15.45
Bildschirmfoto 2021-12-09 um 17.15.451814×986 47 KB

running java inside the docker cli also doesn’t do anything.
Update:
Scratch that ps in the CLI tells me that java is running:

$ ps
PID   USER     TIME  COMMAND
    1 root      0:00 {spotlight.sh} /usr/bin/qemu-x86_64 /bin/sh /bin/spotlight.sh lt
   36 root      1:49 {java} /usr/bin/qemu-x86_64 /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Dfile.encoding=UTF-8 -X
   41 root      0:00 {sh} /usr/bin/qemu-x86_64 /bin/sh
   66 root      0:00 /bin/ps
35

Hi @mingaflo,

About the:

What do you mean with “it is necessary to alert if something wrong happens”?

Prompt a message about the error (the download fails), specifically in the part:

curl -O $RESULT

Even when the download fails: the script continues with the following lines

tar -C /opt/spotlight/models -xvf spotlight-model_lang=$LANG.tar.gz
rm spotlight-model_lang=$LANG.tar.gz
echo "/opt/spotlight/models/$LANG http://0.0.0.0:80/rest/"

The following command is also executed (considering that the language model is not “en”)

 java -Dfile.encoding=UTF-8 -Xmx10G -jar /opt/spotlight/dbpedia-spotlight.jar /opt/spotlight/models/$LANG http://0.0.0.0:80/rest

after that, the DBpedia Spotlight service is started with errors… (that why you saw java running)

In summary, a model download failure will result in a running service with errors.

I will update the script to avoid the execution of the DBpedia Spotlight service when the model download fails. Thanks again for the feedback… have a great day.

36

The problem persist even if the download does not fail.
As I mentioned: For small languages the download succeeds but the java download still fails / does not print any updates. Even though it is running.

Any ideas why that is? It’s alright if you don’t have any :slight_smile:
I just wanted to clarify what is happening if someone has a similar issue. I didn’t find a fix. I will update if I find something.

37

Hi @mingaflo,

Please, could you share your YML file to reproduce the process?.

In the meantime, please could you try the following:

docker run -tid --restart unless-stopped --name dbpedia-spotlight.lt --mount source=models,target=/opt/spotlight -p 2222:80 dbpedia/dbpedia-spotlight spotlight.sh lt

This is the desired output:

$ docker logs dbpedia-spotlight.lt
Selected language: lt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1410  100   119  100  1291    351   3808 --:--:-- --:--:-- --:--:--  4159
http://downloads.dbpedia.org/repo/dbpedia/spotlight/spotlight-model/2021.09.01/spotlight-model_lang=lt.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 34.5M  100 34.5M    0     0  9222k      0  0:00:03  0:00:03 --:--:-- 9219k
lt/
lt/spotter_thresholds.txt
lt/fsa_dict.mem
lt/model.properties
lt/model/
lt/model/sf.mem
lt/model/context.mem
lt/model/candmap.mem
lt/model/res.mem
lt/model/quantized_counts.mem
lt/model/tokens.mem
lt/stopwords.list
/opt/spotlight/models/lt http://0.0.0.0:80/rest/
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (15 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (81 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (279 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (79 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (91 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryContextStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (480 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading FSADictionary...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (18 ms)
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 1 disambiguators.
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 2 spotters.
Grd 10, 2021 3:58:04 PM com.sun.grizzly.Controller logVersion
INFO: GRIZZLY0001: Starting Grizzly Framework 1.9.48 - 21.12.10 15.58
Server started in /opt/spotlight listening on http://0.0.0.0:80/rest
[main] INFO org.dbpedia.spotlight.web.rest.Server -  Core threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server -   Max threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server - Max POST size: 2097152

After that:

curl http://localhost:2222/rest/annotate \
    --data-urlencode "text=President Obama called Wednesday on Congress to extend a tax break for students included in last year's economic stimulus package, arguing that the policy provides more generous assistance." \
    --data "confidence=0.35" \
    -H "Accept: text/turtle"

the desired output:

@prefix xsd:   <http://www.w3.org/2001/XMLSchema#> .
@prefix itsrdf: <http://www.w3.org/2005/11/its/rdf#> .
@prefix nif:   <http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core#> .

<http://www.dbpedia-spotlight.com/#collection>
        a               nif:ContextCollection ;
        nif:hasContext  <http://www.dbpedia-spotlight.com/#offset_0_189> ;
        <http://purl.org/dc/terms/conformsTo>
                <http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core/2.1> .

<http://www.dbpedia-spotlight.com/#offset_0_189>
        a               nif:OffsetBasedString , nif:Context ;
        nif:beginIndex  "0"^^xsd:nonNegativeInteger ;
        nif:endIndex    "189"^^xsd:nonNegativeInteger ;
        nif:isString    "President Obama called Wednesday on Congress to extend a tax break for students included in last year's economic stimulus package, arguing that the policy provides more generous assistance." .

Thanks and have a great day.

38

When running the above docker run-command I get the following:

$ docker run -tid --restart unless-stopped --name dbpedia-spotlight.lt --mount     source=models,target=/opt/spotlight -p 2222:80 dbpedia/dbpedia-spotlight spotlight.sh lt
Unable to find image 'dbpedia/dbpedia-spotlight:latest' locally
latest: Pulling from dbpedia/dbpedia-spotlight
e7c96db7181b: Pull complete
f910a506b6cb: Pull complete
b6abafe80f63: Pull complete
54cd1b8a02b8: Pull complete
8e0905f6c0f0: Pull complete
cc062f7b308d: Pull complete
45efe250a884: Pull complete
Digest: sha256:89cd13d0ea29b7a0869ead2553d652219552b2d6bb2e85780ed6d9fb3ed42a0b
Status: Downloaded newer image for dbpedia/dbpedia-spotlight:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
1dcf8eb49fbade126ec843a76c4cfe773e7a6daa1ee8c8e3825741fd74467acdt by 4 spaces

Regarding the Warning. I tried adding --platform linux/amd64 but the resulting logs are the same.
The logs look like this:

$ docker logs dbpedia-spotlight.lt

Selected language: lt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1410  100   119  100  1291    293   3179 --:--:-- --:--:-- --:--:--  3507
http://downloads.dbpedia.org/repo/dbpedia/spotlight/spotlight-model/2021.09.01/spotlight-model_lang=lt.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 34.5M  100 34.5M    0     0  1972k      0  0:00:17  0:00:17 --:--:-- 2538k
lt/
lt/spotter_thresholds.txt
lt/fsa_dict.mem
lt/model.properties
lt/model/
lt/model/sf.mem
lt/model/context.mem
lt/model/candmap.mem
lt/model/res.mem
lt/model/quantized_counts.mem
lt/model/tokens.mem
lt/stopwords.list
/opt/spotlight/models/lt http://0.0.0.0:80/rest/

The curl afterwards returns nothing:

$ curl http://localhost:2222/rest/annotate \
    --data-urlencode "text=President Obama called Wednesday on Congress to extend a tax break for students included in last year's economic stimulus package, arguing that the policy provides more generous assistance." \
    --data "confidence=0.35" \
    -H "Accept: text/turtle"
curl: (52) Empty reply from server

My .yml file looks like this:

version: '3.5'
services:
    spotlight.lt:
        image: dbpedia/dbpedia-spotlight
        container_name: dbpedia-spotlight.lt2
        volumes:
            - spotlight-models:/opt/spotlight/models
        restart: unless-stopped
        ports:
            - "0.0.0.0:2223:80"
        command: /bin/spotlight.sh lt

volumes:
    spotlight-models:
        external: true
39

Hi @mingaflo,

It seems this warning is the main problem and according to the following quote it could be solved by changing the docker version (solution for Mac):

Source: https://github.com/google/cadvisor/issues/2763#issuecomment-757421498:
This is a docker problem in version 20.
Version 19 should work fine.
see docker/for-linux#1170 (comment)

But, this is just one solution. The other solution is to build the docker image in your own machine:

$ git clone https://github.com/dbpedia-spotlight/spotlight-docker.git
$ cd spotlight-docker
$ docker build -t dbpedia-spotlight-local --platform linux/amd64 .
$ docker run -tid --restart unless-stopped --name dbpedia-spotlight.lt --mount     source=models,target=/opt/spotlight -p 2222:80 dbpedia-spotlight-local spotlight.sh lt

and the output will be:

$ docker logs dbpedia-spotlight.lt 
Selected language: lt
/opt/spotlight/lt http://0.0.0.0:80/rest/
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (16 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (83 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (289 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (86 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (103 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryContextStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (661 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading FSADictionary...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (23 ms)
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 1 disambiguators.
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 2 spotters.
Grd 15, 2021 9:54:50 AM com.sun.grizzly.Controller logVersion
INFO: GRIZZLY0001: Starting Grizzly Framework 1.9.48 - 21.12.15 09.54
Server started in /opt/spotlight listening on http://0.0.0.0:80/rest
[main] INFO org.dbpedia.spotlight.web.rest.Server -  Core threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server -   Max threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server - Max POST size: 2097152

and the test:

$ curl http://localhost:2222/rest/annotate \
>     --data-urlencode "text=President Obama called Wednesday on Congress to extend a tax break for students included in last year's economic stimulus package, arguing that the policy provides more generous assistance." \
>     --data "confidence=0.35" \
>     -H "Accept: text/turtle"

@prefix xsd:   <http://www.w3.org/2001/XMLSchema#> .
@prefix itsrdf: <http://www.w3.org/2005/11/its/rdf#> .
@prefix nif:   <http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core#> .

<http://www.dbpedia-spotlight.com/#collection>
        a               nif:ContextCollection ;
        nif:hasContext  <http://www.dbpedia-spotlight.com/#offset_0_189> ;
        <http://purl.org/dc/terms/conformsTo>
                <http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core/2.1> .

<http://www.dbpedia-spotlight.com/#offset_0_189>
        a               nif:OffsetBasedString , nif:Context ;
        nif:beginIndex  "0"^^xsd:nonNegativeInteger ;
        nif:endIndex    "189"^^xsd:nonNegativeInteger ;
        nif:isString    "President Obama called Wednesday on Congress to extend a tax break for students included in last year's economic stimulus package, arguing that the policy provides more generous assistance." .

I hope the building process works for you. Have a great day.

My best regards

Julio

40

Hi Julio

thanks again for your reply.
Sorry for being unclear. If I add --platform linux/amd64 the warning disappears but the logs still stop after /opt/spotlight/lt http://0.0.0.0:80/rest/.

I also tried building the docker image in my own machine but with no success. The logs stay the same. I also tried it with adding --platform linux/amd64 to the run command as well since I got the warning there again but the result wasn’t any different.

I don’t know what I am missing.

41

When going into the CLI I notice that java -version does not do anything. The cursor just blinks. Maybe that’s a hint?

42

Hi @mingaflo,

Maybe the doc “Docker Desktop for Apple silicon” helps. Also, in this Stackoverflow question I found some interesting solutions but I am not sure how accurate they are since I do not have a Mac OS. Sorry, please if you find any solution, I will appreciate if you can post it for future references. Thanks.

43

Ok the solution is building the image locally. Since I am a noob when it comes to dockers this took me a while.
For the future interested reader that made the horrible decision of getting a MacBook Pro with M1 chip, here is the solution:

  1. Make a dockerfile that looks like this:

     FROM --platform=linux/arm64/v8 ubuntu:21.04
 MAINTAINER  DBpedia Spotlight Team <dbp-spotlight-developers@lists.sourceforge.net>

 ENV SPOTLIGHT  https://sourceforge.net/projects/spotlight-multilingual-docker/files/dbpedia-spotlight-1.1.jar

 # adding required packages
 RUN apt update && apt upgrade -y && \
     apt install -y bash && apt install -y openjdk-8-jre && \
     apt install -y tshark && apt install -y wget && \
     apt install -y curl && apt clean

 # downloading spolight model and dbpedia spotlight
 RUN mkdir -p /opt/spotlight/models && \
    cd /opt/spotlight && \
    wget -O dbpedia-spotlight.jar $SPOTLIGHT && \
    mkdir -p src/main/resources/templates/

 # adding the script to the container
 ADD spotlight.sh /bin/spotlight.sh
 COPY nif-21.vm /opt/spotlight/src/main/resources/templates/nif-21.vm
 RUN chmod +x /bin/spotlight.sh

 EXPOSE 80

    I could not manage for the life of me to make it run with the pretty outdated FROM openjdk:8-jre-alpine call (I am referring to the github-dockerfile) . I think this is used because it is smaller but anyway.
    If we do this we need to update all apk commands to apt. I am not sure if apt install -y openjdk-8-jre is needed but I can’t be asked to test this any further 🥲
    The rest stays the same.

  2. We call the docker build:

    docker build -t dbpedia-spotlight-local --platform linux/amd64 .

I am quite sure that --platform linux/amd64 is needed. Better be save than sorry.

  1. Finally run the docker:
    Swap the language of your choosing and add --platform linux/amd64 one last time. @JulioNoe has missed that in a previous comment.

     docker run -tid --platform linux/amd64 --restart unless-stopped --name dbpedia-spotlight.lt --mount     source=models,target=/opt/spotlight -p 2222:80 dbpedia-spotlight-local spotlight.sh lt

Swap the language of your choosing and add --platform linux/amd64 one last time. @JulioNoe has missed that in a previous comment.

And finally, you can make curl-requests to the server and get a response.

@JulioNoe Thanks again for the help. If I might add: Docker have multi-arch support. Maybe dbpedia considers releasing a newer version of the image to have better support for all systems and to avoid future problems? Shouldn’t take too long for someone who knows what he is doing (not me :upside_down_face:).

1 Like
44

Hi @mingaflo,

I appreciate your solution a lot. I take note of your steps for future reference and for the next releases. Thanks again, have a great day.

My best regards

Julio

1 Like
45

@JulioNoe @mingaflo I had the same problem today and was able to get a local docker build running on M1 thanks to @mingaflo instructions, however, Spotlight throws these warnings:

WARNING: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS

does it matter? anyway to fix?

46

Hi,

I had to the version of ubuntu to 22.04 to get it running on my M1.

    FROM --platform=linux/arm64/v8 ubuntu:22.04
    MAINTAINER  DBpedia Spotlight Team <dbp-spotlight-developers@lists.sourceforge.net>

    ENV SPOTLIGHT  https://sourceforge.net/projects/spotlight-multilingual-docker/files/dbpedia-spotlight-1.1.jar

    # adding required packages
    RUN apt update && apt upgrade -y && \
        apt install -y bash && apt install -y openjdk-8-jre && \
        apt install -y tshark && apt install -y wget && \
        apt install -y curl && apt clean

    # downloading spolight model and dbpedia spotlight
    RUN mkdir -p /opt/spotlight/models && \
       cd /opt/spotlight && \
       wget -O dbpedia-spotlight.jar $SPOTLIGHT && \
       mkdir -p src/main/resources/templates/

    # adding the script to the container
    ADD spotlight.sh /bin/spotlight.sh
    COPY nif-21.vm /opt/spotlight/src/main/resources/templates/nif-21.vm
    RUN chmod +x /bin/spotlight.sh

    EXPOSE 80
1 Like