DBpedia Release Evaluation: Difference between revisions
From DBpedia Mappings
Jump to navigationJump to search
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
== DBpedia Release Evaluation == | == DBpedia Release Evaluation == | ||
The [http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/dbpedia/raw-file/946fc9d51783/qualityAssessmentFramework/Kreis2011__Design_of_a_Quality_Assessment_Framework.pdf Quality Assessment Framework (QAF)] is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of | The [http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/dbpedia/raw-file/946fc9d51783/qualityAssessmentFramework/Kreis2011__Design_of_a_Quality_Assessment_Framework.pdf Quality Assessment Framework (QAF)] is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of the QAF is a comparison between a manually created best-case dataset and the output from DBpedia's ontology based extraction. The QAF estimates the precision of the extraction framework and the completeness (recall) of DBpedia compared to its source Wikipedia. | ||
== Evaluation Results == | == Evaluation Results == | ||
{| class="wikitable" | {| class="wikitable" | ||
|+ Completeness (fixed mappings from 3.5.1) | |+ Completeness / Recall (fixed mappings from 3.5.1) | ||
|- | |- | ||
! Category | ! Category | ||
| Line 72: | Line 72: | ||
| 57,1% | | 57,1% | ||
| 71,4% | | 71,4% | ||
|} | |||
{| class="wikitable" | |||
|+ Precision (fixed mappings from 3.5.1) | |||
|- | |||
! Category | |||
! DBpedia 3.5.1 | |||
! DBpedia 3.6 | |||
! DBpedia 3.7 | |||
|- | |||
| Total | |||
| 91,2% | |||
| 92,3% | |||
| 92,4% | |||
|- | |||
| Plain Property | |||
| 96,3% | |||
| 96,6% | |||
| 97,4% | |||
|- | |||
| Number-Unit | |||
| 85,4% | |||
| 85,4% | |||
| 85,0% | |||
|- | |||
| Coordinate | |||
| 100,0% | |||
| 100,0% | |||
| 100,0% | |||
|- | |||
| Interval | |||
| 100,0% | |||
| 100,0% | |||
| 88,9% | |||
|- | |||
| List | |||
| 91,5% | |||
| 92,8% | |||
| 93,2% | |||
|- | |||
| One-Property-Table | |||
| 32,5% | |||
| 36,8% | |||
| 34,1% | |||
|- | |||
| Multi-Poprty-Table | |||
| na | |||
| na | |||
| na | |||
|- | |||
| Open Property | |||
| 100,0% | |||
| 100,0% | |||
| 100,0% | |||
|- | |||
| Open Property Table | |||
| na | |||
| na | |||
| na | |||
|- | |||
| Internal Template | |||
| 83,3% | |||
| 75,0% | |||
| 75,0% | |||
|- | |||
| Merged Properties | |||
| 80,0% | |||
| 80,0% | |||
| 100,0% | |||
|} | |} | ||
Revision as of 14:15, 14 September 2011
DBpedia Release Evaluation
The Quality Assessment Framework (QAF) is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of the QAF is a comparison between a manually created best-case dataset and the output from DBpedia's ontology based extraction. The QAF estimates the precision of the extraction framework and the completeness (recall) of DBpedia compared to its source Wikipedia.
Evaluation Results
| Category | DBpedia 3.5.1 | DBpedia 3.6 | DBpedia 3.7 |
|---|---|---|---|
| Total | 45,7% | 60,2% | 61,8% |
| Plain Property | 80,5% | 83,7% | 86,0% |
| Number-Unit | 68,6% | 68,6% | 66,7% |
| Coordinate | 100,0% | 100,0% | 100,0% |
| Interval | 72,7% | 68,2% | 72,7% |
| List | 33,9% | 75,7% | 77,8% |
| One-Property-Table | 5,4% | 5,8% | 5,8% |
| Multi-Poprty-Table | 0,0% | 0,0% | 0,0% |
| Open Property | 23,1% | 23,1% | 30,8% |
| Open Property Table | na | na | na |
| Internal Template | 8,6% | 10,3% | 10,3% |
| Merged Properties | 57,1% | 57,1% | 71,4% |
| Category | DBpedia 3.5.1 | DBpedia 3.6 | DBpedia 3.7 |
|---|---|---|---|
| Total | 91,2% | 92,3% | 92,4% |
| Plain Property | 96,3% | 96,6% | 97,4% |
| Number-Unit | 85,4% | 85,4% | 85,0% |
| Coordinate | 100,0% | 100,0% | 100,0% |
| Interval | 100,0% | 100,0% | 88,9% |
| List | 91,5% | 92,8% | 93,2% |
| One-Property-Table | 32,5% | 36,8% | 34,1% |
| Multi-Poprty-Table | na | na | na |
| Open Property | 100,0% | 100,0% | 100,0% |
| Open Property Table | na | na | na |
| Internal Template | 83,3% | 75,0% | 75,0% |
| Merged Properties | 80,0% | 80,0% | 100,0% |
