Semi-supervised Dirichlet-Hawkes process with applications of topic detection and tracking in Twitter

Wanying Ding, Yue Zhang, Chaomei Chen, Xiaohua Hu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)


Understanding ongoing topics and their evolutions in social media is of great importance. Although topic analysis is not a novel research question, social media environment has presented new challenges. First, with insufficient co-occurrence information, short text have undermined many word co-occurrence oriented topic models' applicability. Second, real time message streams make traditional discretized topic tracking methods hard to function. Third, topics' evolution mechanisms are of great importance in social media context, but many studies have ignored them. Forth, topics have more complicated correlation among each other. Considering the existing problems, this paper has proposed a Semi-Supervised Dirichlet-Hawkes Process (SDHP) to deal with topic detection and tracking from social media. The main contributions of this paper are reflected in: (1) SDHP can handle short text problem efficiently; (2) SDHP can track topics from continuous message stream; (3) SDHP can reveal topics' underlying evolution patterns; and (4) SDHP can capture topics' correlations We have evaluated SDHP's ability in both topic detection and tracking in 8 real datasets from Twitter, and the algorithm's performances are very promising.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
EditorsRonay Ak, George Karypis, Yinglong Xia, Xiaohua Tony Hu, Philip S. Yu, James Joshi, Lyle Ungar, Ling Liu, Aki-Hiro Sato, Toyotaro Suzumura, Sudarsan Rachuri, Rama Govindaraju, Weijia Xu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781467390040
Publication statusPublished - 2016
Event4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States
Duration: 2016 Dec 52016 Dec 8

Publication series

NameProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016


Other4th IEEE International Conference on Big Data, Big Data 2016
Country/TerritoryUnited States

Bibliographical note

Publisher Copyright:
© 2016 IEEE.

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Hardware and Architecture

Cite this