Skip to content

Research at St Andrews

Discovering topic structures of a temporally evolving document corpus

Research output: Research - peer-reviewArticle

DOI

Open Access permissions

Open

Author(s)

Adham Beykikhoshk, Ognjen Arandelovic, Dinh Phung, Svetha Venkatesh

School/Research organisations

Abstract

In this paper we describe a novel framework for the discovery of the topical content of a data corpus, and the tracking of its complex structural changes across the temporal dimension. In contrast to previous work our model does not impose a prior on the rate at which documents are added to the corpus nor does it adopt the Markovian assumption which overly restricts the type of changes that the model can capture. Our key technical contribution is a framework based on (i) discretization of time into epochs, (ii) epoch-wise topic discovery using a hierarchical Dirichlet process-based model, and (iii) a temporal similarity graph which allows for the modelling of complex topic changes: emergence and disappearance, evolution, splitting, and merging. The power of the proposed framework is demonstrated on two medical literature corpora concerned with the autism spectrum disorder (ASD) and the metabolic syndrome (MetS)—both increasingly important research subjects with significant social and healthcare consequences. In addition to the collected ASD and metabolic syndrome literature corpora which we made freely available, our contribution also includes an extensive empirical analysis of the proposed framework. We describe a detailed and careful examination of the effects that our algorithms’s free parameters have on its output and discuss the significance of the findings both in the context of the practical application of our algorithm as well as in the context of the existing body of work on temporal topic analysis. Our quantitative analysis is followed by several qualitative case studies highly relevant to the current research on ASD and MetS, on which our algorithm is shown to capture well the actual developments in these fields.
Close

Details

Original languageEnglish
Pages (from-to)599-632
Number of pages34
JournalKnowledge and Information Systems
Volume55
Issue number3
Early online date10 Aug 2017
DOIs
StatePublished - Jun 2018

    Research areas

  • Data mining, Non-parametric, Bayesian, Autism, ASD, Metabolic syndrome

Discover related content
Find related publications, people, projects and more using interactive charts.

View graph of relations

Related by author

  1. Reimagining the central challenge of face recognition: turning a problem into an advantage

    Arandelovic, O. 22 Jun 2018 In : Pattern Recognition. 83, p. 388-400 13 p.

    Research output: Research - peer-reviewArticle

  2. Machine learning based prediction of consumer purchasing decisions: the evidence and its significance

    Stubseid, S. & Arandelovic, O. 2 Feb 2018 Proceedings AI and Marketing Science workshop at AAAI-2018. 7 p.

    Research output: ResearchConference contribution

  3. Baseline fusion for image an pattern recognition - what not to do (and how to do better)

    Arandelovic, O. Dec 2017 In : Journal of Imaging. 3, 4, 16 p., 44

    Research output: Research - peer-reviewArticle

ID: 250119430