10.5281/ZENODO.3840646
Cutrona, Vincenzo
Vincenzo
Cutrona
0000-0001-7830-7596
University of Milano - Bicocca
Bianchi, Federico
Federico
Bianchi
0000-0003-0776-361X
Bocconi University
Jiménez-Ruiz, Ernesto
Ernesto
Jiménez-Ruiz
0000-0002-9083-4599
City, University of London
Palmonari, Matteo
Matteo
Palmonari
0000-0002-1801-5118
University of Milano - Bicocca
Tough Tables: Carefully Evaluating Entity Linking for Tabular Data
Zenodo
2021
entity linking
entity resolution
tabular data annotation
CEA
cell entity annotation
CTA
column type annotation
SemTab 2019
SemTab 2020
SemTab 2021
SemTab 2022
2021-06-30
https://zenodo.org/record/7419275
10.5281/zenodo.3840647
10.5281/zenodo.4246370
10.5281/zenodo.6211551
10.5281/zenodo.7419275
https://zenodo.org/communities/linkeddata
3.0
Creative Commons Attribution 4.0 International
Open Access
<p>Tough Tables (2T) is a dataset designed to evaluate table annotation approaches in solving the CEA and CTA tasks.<br>
The dataset is compliant with the data format used in <a href="https://www.cs.ox.ac.uk/isg/challenges/sem-tab/2019/index.html">SemTab 2019</a>, and it can be used as an additional dataset without any modification. The target knowledge graph is DBpedia 2016-10.<br>
Check out the <a href="https://github.com/vcutrona/tough-tables">2T GitHub repository</a> for more details about the dataset generation.</p>
<p><strong>New in v3.0:</strong> We release the updated version of 2T! The target knowledge graphs are DBpedia <a href="http://downloads.dbpedia.org/wiki-archive/downloads-2016-10.html">2016-10</a> and Wikidata <a href="https://zenodo.org/record/6643443">20220521</a>. Starting from this version, the dataset is split into valid and test sets.</p>
<p>This work is based on the following paper:</p>
<blockquote>
<p>Cutrona, V., Bianchi, F., Jimenez-Ruiz, E. and Palmonari, M. (2020). Tough Tables: Carefully Evaluating Entity Linking for Tabular Data. ISWC 2020, LNCS 12507, pp. 1–16.</p>
</blockquote>
<p>Note on License: This dataset includes data from the following sources. Refer to each source for license details:<br>
- Wikipedia https://www.wikipedia.org/<br>
- DBpedia https://dbpedia.org/<br>
- Wikidata https://www.wikidata.org/<br>
- SemTab 2019 https://doi.org/10.5281/zenodo.3518539<br>
- GeoDatos https://www.geodatos.net<br>
- The Pudding https://pudding.cool/<br>
- Offices.net https://offices.net/<br>
- DATA.GOV https://www.data.gov/</p>
<p>THIS DATA IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.<br>
<br>
<strong>Changelog:</strong></p>
<p><em><strong>v3.0</strong></em></p>
<ul>
<li>Both datasets require SemTab2020 CEA format (tab_id, row_id, col_id, entity).
<ul>
</ul>
</li>
<li>Tables IDs and artificial noise values differ from previous versions.</li>
<li>Datasets are split into Valid and Test sets of tables.</li>
<li>New GT for ToughTables-WD (2T_WD)
<ul>
<li>Entities Q23772518 and Q7327323 have been removed because they are no longer represented in WD</li>
<li>Updated ancestor/descendant hierarchies to evaluate CTA.</li>
</ul>
</li>
<li>Evaluation scripts are provided with the data sets.</li>
</ul>
<p><em><strong>v2.0</strong></em></p>
<ul>
<li>New GT for 2T_WD
<ul>
<li>A few entities have been removed from the CEA GT, because they are no longer represented in WD (e.g., dbr:Devonté points to wd:Q21155080, which does not exist)</li>
<li>Tables codes and values differ from the previous version, because of the random noise.</li>
<li>Updated ancestor/descendant hierarchies to evaluate CTA.</li>
</ul>
</li>
</ul>
<p><em><strong>v1.0</strong></em></p>
<ul>
<li>New Wikidata version (2T_WD)</li>
<li>Fix header for tables CTRL_DBP_MUS_rock_bands_labels.csv and CTRL_DBP_MUS_rock_bands_labels_NOISE2.csv (column 2 was reported with id 1 in target - NOTE: the affected column has been removed from the SemTab2020 evaluation)</li>
<li>Remove duplicated entries in tables</li>
<li>Remove rows with wrong values (e.g., the Kazakhstan entity has an empty name "''")</li>
<li>Many rows and noised columns are shuffled/changed due to the random noise generator algorithm</li>
<li>Remove row "Florida","Floorida","New York, NY" from TOUGH_WEB_MISSP_1000_us_cities.csv (and all its NOISE1 variants)</li>
<li>Fix header of tables:
<ul>
<li>CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states.csv</li>
<li>CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states_NOISE2.csv</li>
<li>TOUGH_T2D_BUS_29414811_2_4773219892816395776___videogames_developers.csv</li>
<li>TOUGH_T2D_BUS_29414811_2_4773219892816395776___videogames_developers_NOISE2.csv</li>
</ul>
</li>
</ul>
<p><em><strong>v0.1-pre</strong></em></p>
<ul>
<li>First submission. It contains only tables, without GT and Targets.</li>
</ul>
Visit the SemTab challenge website (https://sem-tab-challenge.github.io/2022/) for more details.