Explanatory and actionable debugging for machine learning: A tableQA demonstration

Minseok Cho, Gyeongbok Lee, Seung Won Hwang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Question answering from tables (TableQA) extracting answers from tables from the question given in natural language, has been actively studied. Existing models have been trained and evaluated mostly with respect to answer accuracy using public benchmark datasets such as WikiSQL. The goal of this demonstration is to show a debugging tool for such models, explaining answers to humans, known as explanatory debugging. Our key distinction is making it “actionable" to allow users to directly correct models upon explanation. Specifically, our tool surfaces annotation and models errors for users to correct, and provides actionable insights.

Original languageEnglish
Title of host publicationSIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages1333-1336
Number of pages4
ISBN (Electronic)9781450361729
DOIs
Publication statusPublished - 2019 Jul 18
Event42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019 - Paris, France
Duration: 2019 Jul 212019 Jul 25

Publication series

NameSIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

Conference42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019
CountryFrance
CityParis
Period19/7/2119/7/25

Fingerprint

Debugging
Learning systems
Machine Learning
Demonstrations
Model Error
Question Answering
Natural Language
Annotation
Model
Benchmark

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Applied Mathematics
  • Software

Cite this

Cho, M., Lee, G., & Hwang, S. W. (2019). Explanatory and actionable debugging for machine learning: A tableQA demonstration. In SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1333-1336). (SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval). Association for Computing Machinery, Inc. https://doi.org/10.1145/3331184.3331404
Cho, Minseok ; Lee, Gyeongbok ; Hwang, Seung Won. / Explanatory and actionable debugging for machine learning : A tableQA demonstration. SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc, 2019. pp. 1333-1336 (SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval).
@inproceedings{8adf93ea14c5436993c20459abe208cd,
title = "Explanatory and actionable debugging for machine learning: A tableQA demonstration",
abstract = "Question answering from tables (TableQA) extracting answers from tables from the question given in natural language, has been actively studied. Existing models have been trained and evaluated mostly with respect to answer accuracy using public benchmark datasets such as WikiSQL. The goal of this demonstration is to show a debugging tool for such models, explaining answers to humans, known as explanatory debugging. Our key distinction is making it “actionable{"} to allow users to directly correct models upon explanation. Specifically, our tool surfaces annotation and models errors for users to correct, and provides actionable insights.",
author = "Minseok Cho and Gyeongbok Lee and Hwang, {Seung Won}",
year = "2019",
month = "7",
day = "18",
doi = "10.1145/3331184.3331404",
language = "English",
series = "SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval",
publisher = "Association for Computing Machinery, Inc",
pages = "1333--1336",
booktitle = "SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval",

}

Cho, M, Lee, G & Hwang, SW 2019, Explanatory and actionable debugging for machine learning: A tableQA demonstration. in SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery, Inc, pp. 1333-1336, 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, 19/7/21. https://doi.org/10.1145/3331184.3331404

Explanatory and actionable debugging for machine learning : A tableQA demonstration. / Cho, Minseok; Lee, Gyeongbok; Hwang, Seung Won.

SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc, 2019. p. 1333-1336 (SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Explanatory and actionable debugging for machine learning

T2 - A tableQA demonstration

AU - Cho, Minseok

AU - Lee, Gyeongbok

AU - Hwang, Seung Won

PY - 2019/7/18

Y1 - 2019/7/18

N2 - Question answering from tables (TableQA) extracting answers from tables from the question given in natural language, has been actively studied. Existing models have been trained and evaluated mostly with respect to answer accuracy using public benchmark datasets such as WikiSQL. The goal of this demonstration is to show a debugging tool for such models, explaining answers to humans, known as explanatory debugging. Our key distinction is making it “actionable" to allow users to directly correct models upon explanation. Specifically, our tool surfaces annotation and models errors for users to correct, and provides actionable insights.

AB - Question answering from tables (TableQA) extracting answers from tables from the question given in natural language, has been actively studied. Existing models have been trained and evaluated mostly with respect to answer accuracy using public benchmark datasets such as WikiSQL. The goal of this demonstration is to show a debugging tool for such models, explaining answers to humans, known as explanatory debugging. Our key distinction is making it “actionable" to allow users to directly correct models upon explanation. Specifically, our tool surfaces annotation and models errors for users to correct, and provides actionable insights.

UR - http://www.scopus.com/inward/record.url?scp=85073771814&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073771814&partnerID=8YFLogxK

U2 - 10.1145/3331184.3331404

DO - 10.1145/3331184.3331404

M3 - Conference contribution

AN - SCOPUS:85073771814

T3 - SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

SP - 1333

EP - 1336

BT - SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

PB - Association for Computing Machinery, Inc

ER -

Cho M, Lee G, Hwang SW. Explanatory and actionable debugging for machine learning: A tableQA demonstration. In SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc. 2019. p. 1333-1336. (SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval). https://doi.org/10.1145/3331184.3331404