Skill ontology-based model for quality assurance in crowdsourcing

Kinda El Maarry, Wolf Tilo Balke, Hyunsouk Cho, Seung Won Hwang, Yukino Baba

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

Crowdsourcing continues to gain more momentum as its potential becomes more recognized. Nevertheless, the associated quality aspect remains a valid concern, which introduces uncertainty in the results obtained from the crowd. We identify the different aspects that dynamically affect the overall quality of a crowdsourcing task. Accordingly, we propose a skill ontology-based model that caters for these aspects, as a management technique to be adopted by crowdsourcing platforms. The model maintains a dynamically evolving ontology of skills, with libraries of standardized and personalized assessments for awarding workers skills. Aligning a worker's set of skills to that required by a task, boosts the ultimate resulting quality. We visualize the model's components and workflow, and consider how to guard it against malicious or unqualified workers, whose responses introduce this uncertainty and degrade the overall quality.

Original languageEnglish
Title of host publicationDatabase Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops
Subtitle of host publicationBDMA, DaMEN, SIM3, UnCrowd, Revised Selected Papers
PublisherSpringer Verlag
Pages376-387
Number of pages12
ISBN (Print)9783662439838
DOIs
Publication statusPublished - 2014 Jan 1
Event19th International Conference on Database Systems for Advanced Applications, DASFAA 2014 - Bali, Indonesia
Duration: 2014 Apr 212014 Apr 24

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8505 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other19th International Conference on Database Systems for Advanced Applications, DASFAA 2014
CountryIndonesia
CityBali
Period14/4/2114/4/24

Fingerprint

Quality Assurance
Quality assurance
Ontology
Uncertainty
Momentum
Component Model
Model
Work Flow
Continue
Valid
Skills
Crowdsourcing

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Maarry, K. E., Balke, W. T., Cho, H., Hwang, S. W., & Baba, Y. (2014). Skill ontology-based model for quality assurance in crowdsourcing. In Database Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops: BDMA, DaMEN, SIM3, UnCrowd, Revised Selected Papers (pp. 376-387). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8505 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-662-43984-5_29
Maarry, Kinda El ; Balke, Wolf Tilo ; Cho, Hyunsouk ; Hwang, Seung Won ; Baba, Yukino. / Skill ontology-based model for quality assurance in crowdsourcing. Database Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops: BDMA, DaMEN, SIM3, UnCrowd, Revised Selected Papers. Springer Verlag, 2014. pp. 376-387 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e72d774957224065886cb093360ebd06,
title = "Skill ontology-based model for quality assurance in crowdsourcing",
abstract = "Crowdsourcing continues to gain more momentum as its potential becomes more recognized. Nevertheless, the associated quality aspect remains a valid concern, which introduces uncertainty in the results obtained from the crowd. We identify the different aspects that dynamically affect the overall quality of a crowdsourcing task. Accordingly, we propose a skill ontology-based model that caters for these aspects, as a management technique to be adopted by crowdsourcing platforms. The model maintains a dynamically evolving ontology of skills, with libraries of standardized and personalized assessments for awarding workers skills. Aligning a worker's set of skills to that required by a task, boosts the ultimate resulting quality. We visualize the model's components and workflow, and consider how to guard it against malicious or unqualified workers, whose responses introduce this uncertainty and degrade the overall quality.",
author = "Maarry, {Kinda El} and Balke, {Wolf Tilo} and Hyunsouk Cho and Hwang, {Seung Won} and Yukino Baba",
year = "2014",
month = "1",
day = "1",
doi = "10.1007/978-3-662-43984-5_29",
language = "English",
isbn = "9783662439838",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "376--387",
booktitle = "Database Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops",
address = "Germany",

}

Maarry, KE, Balke, WT, Cho, H, Hwang, SW & Baba, Y 2014, Skill ontology-based model for quality assurance in crowdsourcing. in Database Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops: BDMA, DaMEN, SIM3, UnCrowd, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8505 LNCS, Springer Verlag, pp. 376-387, 19th International Conference on Database Systems for Advanced Applications, DASFAA 2014, Bali, Indonesia, 14/4/21. https://doi.org/10.1007/978-3-662-43984-5_29

Skill ontology-based model for quality assurance in crowdsourcing. / Maarry, Kinda El; Balke, Wolf Tilo; Cho, Hyunsouk; Hwang, Seung Won; Baba, Yukino.

Database Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops: BDMA, DaMEN, SIM3, UnCrowd, Revised Selected Papers. Springer Verlag, 2014. p. 376-387 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8505 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Skill ontology-based model for quality assurance in crowdsourcing

AU - Maarry, Kinda El

AU - Balke, Wolf Tilo

AU - Cho, Hyunsouk

AU - Hwang, Seung Won

AU - Baba, Yukino

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Crowdsourcing continues to gain more momentum as its potential becomes more recognized. Nevertheless, the associated quality aspect remains a valid concern, which introduces uncertainty in the results obtained from the crowd. We identify the different aspects that dynamically affect the overall quality of a crowdsourcing task. Accordingly, we propose a skill ontology-based model that caters for these aspects, as a management technique to be adopted by crowdsourcing platforms. The model maintains a dynamically evolving ontology of skills, with libraries of standardized and personalized assessments for awarding workers skills. Aligning a worker's set of skills to that required by a task, boosts the ultimate resulting quality. We visualize the model's components and workflow, and consider how to guard it against malicious or unqualified workers, whose responses introduce this uncertainty and degrade the overall quality.

AB - Crowdsourcing continues to gain more momentum as its potential becomes more recognized. Nevertheless, the associated quality aspect remains a valid concern, which introduces uncertainty in the results obtained from the crowd. We identify the different aspects that dynamically affect the overall quality of a crowdsourcing task. Accordingly, we propose a skill ontology-based model that caters for these aspects, as a management technique to be adopted by crowdsourcing platforms. The model maintains a dynamically evolving ontology of skills, with libraries of standardized and personalized assessments for awarding workers skills. Aligning a worker's set of skills to that required by a task, boosts the ultimate resulting quality. We visualize the model's components and workflow, and consider how to guard it against malicious or unqualified workers, whose responses introduce this uncertainty and degrade the overall quality.

UR - http://www.scopus.com/inward/record.url?scp=84958538712&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84958538712&partnerID=8YFLogxK

U2 - 10.1007/978-3-662-43984-5_29

DO - 10.1007/978-3-662-43984-5_29

M3 - Conference contribution

AN - SCOPUS:84958538712

SN - 9783662439838

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 376

EP - 387

BT - Database Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops

PB - Springer Verlag

ER -

Maarry KE, Balke WT, Cho H, Hwang SW, Baba Y. Skill ontology-based model for quality assurance in crowdsourcing. In Database Systems for Advanced Applications - 19th International Conference, DASFAA 2014, International Workshops: BDMA, DaMEN, SIM3, UnCrowd, Revised Selected Papers. Springer Verlag. 2014. p. 376-387. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-662-43984-5_29