Skip to content

Research at St Andrews

Parallel and streaming truth discovery in large-scale quantitative crowdsourcing

Research output: Contribution to journalArticlepeer-review

Author(s)

Robin Wentao Ouyang, Lance M. Kaplan, Alice Toniolo, Mani Srivastava, Timothy J. Norman

School/Research organisations

Abstract

To enable reliable crowdsourcing applications, it is of great importance to develop algorithms that can automatically discover the truths from possibly noisy and conflicting claims provided by various information sources. In order to handle crowdsourcing applications involving big or streaming data, a desirable truth discovery algorithm should not only be effective, but also be scalable. However, with respect to quantitative crowdsourcing applications such as object counting and percentage annotation, existing truth discovery algorithms are not simultaneously effective and scalable. They either address truth discovery in categorical crowdsourcing or perform batch processing that does not scale. In this paper, we propose new parallel and streaming truth discovery algorithms for quantitative crowdsourcing applications. Through extensive experiments on real-world and synthetic datasets, we demonstrate that 1) both of them are quite effective, 2) the parallel algorithm can efficiently perform truth discovery on large datasets, and 3) the streaming algorithm processes data incrementally, and it can efficiently perform truth discovery both on large datasets and in data streams.

Close

Details

Original languageEnglish
Pages (from-to)2984-2997
Number of pages14
JournalIEEE Transactions on Parallel and Distributed Systems
Volume27
Issue number10
Early online date6 Jan 2016
DOIs
Publication statusPublished - 1 Oct 2016

    Research areas

  • Crowdsourcing, Truth discovery, Quantitative task, Big data, Parallel algorithm, Streaming algorithm

Discover related content
Find related publications, people, projects and more using interactive charts.

View graph of relations

Related by author

  1. Argumentation-based explanations of multimorbidity treatment plans

    Shaheen, Q., Toniolo, A. & Kuster Filipe Bowles, J., 2021, PRIMA 2020: Principles and Practice of Multi-Agent Systems: 23rd International Conference, Nagoya, Japan, November 18–20, 2020, Proceedings. Uchiya, T., Bai, Q. & Maestre, I. M. (eds.). Cham: Springer, p. 394-402 9 p. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); vol. 12568 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

  2. Responsible agent deliberation

    Walton, D. & Toniolo, A., 1 Jun 2020, Reason to Dissent: Proceedings of the 3rd European Conference on Argumentation, Volume . Novaes, C. D., Jansen, H., van Laar, J. A. & Verheij, B. (eds.). College Publications, p. 391-405 15 p.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

  3. Dialogue games for explaining medication choices

    Shaheen, Q., Toniolo, A. & Kuster Filipe Bowles, J., 2020, Rules and Reasoning: 4th International Joint Conference, RuleML+RR 2020, Oslo, Norway, June 29–July 1, 2020, Proceedings. Gutiérrez Basulto, V., Kliegr, T., Soylu, A., Giese, M. & Roman, D. (eds.). Cham: Springer, p. 97-111 15 p. (Lecture Notes in Computer Science (Programming and Software Engineering); vol. 12173 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

  4. On natural language generation of formal argumentation

    Cerutti, F., Toniolo, A. & Norman, T. J., 27 Dec 2019, Proceedings of the 3rd Workshop on Advances In Argumentation In Artificial Intelligence co-located with the 18th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2019): Rende, Italy, November 19-22, 2019. Santini, F. & Toniolo, A. (eds.). Sun SITE Central Europe, p. 15-29 15 p. (CEUR Workshop Proceedings; vol. 2528).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

  5. Dialectical models of deliberation, problem solving and decision making

    Walton, D., Toniolo, A. & Norman, T. J., 13 Sep 2019, In: Argumentation. First Online

    Research output: Contribution to journalArticlepeer-review

Related by journal

  1. Collaborative heterogeneity-aware OS scheduler for asymmetric multicore processors

    Yu, T., Zhong, R., Janjic, V., Petoumenos, P., Zhai, J., Leather, H. & Thomson, J. D., 1 May 2021, In: IEEE Transactions on Parallel and Distributed Systems. 32, 5, p. 1224-1237 14 p.

    Research output: Contribution to journalArticlepeer-review

  2. Large-scale automatic k-means clustering for heterogeneous many-core supercomputer

    Yu, T., Zhao, W., Liu, P., Janjic, V., Yan, X., Wang, S., Fu, H., Yang, G. & Thomson, J. D., May 2020, In: IEEE Transactions on Parallel and Distributed Systems. 31, 5, p. 997-1008 12 p.

    Research output: Contribution to journalArticlepeer-review

ID: 247864392

Top