"The IRI is not in Unicode Normal Form KC" from Jena for generic english labels 2020.06.01

KonradHoeffner · July 3, 2020, 8:26am

I downloaded and extracted https://downloads.dbpedia.org/repo/dbpedia/generic/labels/2020.06.01/labels_lang=en.ttl.bz2 but when it gets parsed by Apache though LIMES, there are a bunch of warnings like the ones below.

As far as I understand this problem, that is not an Apache Bug but determines that there are some problems with how the URIs are represented in the Turtle file. I’m not a unicode expert, so I don’t know if that is a major problem or just a preference of Jena.

23:54.658 [main] [] WARN  org.apache.jena.riot:95 - [line: 772507, col: 1 ] Bad IRI: <http://dbpedia.org/resource/ACR_Alvorense_1º_Dezembro> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.658 [main] [] WARN  org.apache.jena.riot:95 - [line: 772507, col: 1 ] Bad IRI: <http://dbpedia.org/resource/ACR_Alvorense_1º_Dezembro> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.704 [main] [] WARN  org.apache.jena.riot:95 - [line: 779858, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Cheers> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.704 [main] [] WARN  org.apache.jena.riot:95 - [line: 779858, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Cheers> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779859, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Heroes_&_Villains> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779859, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Heroes_&_Villains> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779860, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Laughs> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779860, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Laughs> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779861, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Movie_Quotes> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779861, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Movie_Quotes> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779862, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Movies> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779862, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Movies> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779863, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Movies_(10th_Anniversary_Edition)> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.705 [main] [] WARN  org.apache.jena.riot:95 - [line: 779863, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Movies_(10th_Anniversary_Edition)> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.706 [main] [] WARN  org.apache.jena.riot:95 - [line: 779864, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Passions> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.706 [main] [] WARN  org.apache.jena.riot:95 - [line: 779864, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Passions> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.706 [main] [] WARN  org.apache.jena.riot:95 - [line: 779865, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Songs> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.706 [main] [] WARN  org.apache.jena.riot:95 - [line: 779865, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Songs> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.706 [main] [] WARN  org.apache.jena.riot:95 - [line: 779866, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Stars> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.706 [main] [] WARN  org.apache.jena.riot:95 - [line: 779866, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Stars> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.706 [main] [] WARN  org.apache.jena.riot:95 - [line: 779867, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Thrills> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779867, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years_…_100_Thrills> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779868, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Cheers> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779868, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Cheers> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779869, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Heroes_and_Villains> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779869, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Heroes_and_Villains> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779870, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Laughs> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779870, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Laughs> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779871, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Movie_Quotes> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779871, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Movie_Quotes> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779872, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Movies> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779872, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Movies> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779873, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Movies_(10th_Anniversary_Edition)> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779873, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Movies_(10th_Anniversary_Edition)> Code: 56/COMPATIBILITY_CHARACTER in PATH: TODO
15:23:54.707 [main] [] WARN  org.apache.jena.riot:95 - [line: 779874, col: 1 ] Bad IRI: <http://dbpedia.org/resource/AFI's_100_Years…100_Passions> Code: 47/NOT_NFKC in PATH: The IRI is not in Unicode Normal Form KC.

KonradHoeffner · July 3, 2020, 8:40am

P.S.: There are also many identical triples with seemingly empty labels at the end:

<http://dbpedia.org/resource/󠇯> <http://www.w3.org/2000/01/rdf-schema#label> "󠇯"@en .

There is also some strange unicode magic going on, because I cannot search for the first " character there in vim, as the two quotation marks have constituent codepoints:

0022 QUOTATION MARK
E01E9 VARIATION SELECTOR-250
0022 QUOTATION MARK

jfrey · July 8, 2020, 8:59am

Hi @KonradHoeffner,
we do not care about unicode normal forms, in order to keep the mapping from wikipedia to dbpedia iris as simple as possible. You can verify the triples with DBpedia Databus Ntriples parser
http://akswnc7.informatik.uni-leipzig.de:8088/report (based on Jena). If they succeed there they will end up in DBpedia releases. There will be new Jena releases soon which detects some invalid unicode chars https://tools.ietf.org/html/rfc3987#section-2.2 which where not spotted yet, but your ‎E01EF VARIATION SELECTOR-256 (at least this is what I can read from the string you copied above (maybe it was modified by the forum or your clipboard) seems to be valid in IRIs.

kurzum · August 2, 2020, 12:31pm

@KonradHoeffner in the first place the Jena message is more an internal message to warn about a legacy incompatibility to previous jena versions.
Secondly, I asked this on the Jena user mailing list:
http://mail-archives.apache.org/mod_mbox/jena-users/201909.mbox/<1947b288-4c43-b42d-8198-c439723ab483@informatik.uni-leipzig.de>

Andy Seaborne removed the message in Jena 3.13.0 onwards.