Detecting duplicate biological entities using Markov random field-based edit distance

Min Song, Alex Rudniy

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Detecting duplicate entities in biological data is an important research task. In this paper, we propose a novel and context-sensitive Markov random field-based edit distance (MRFED) for this task. We apply the Markov random field theory to the Needleman-Wunsch distance and combine MRFED with TFIDF, a token-based distance algorithm, resulting in SoftMRFED. We compare SoftMRFED with other distance algorithms such as Levenshtein, SoftTFIDF, and Monge-Elkan for two matching tasks: biological entity matching and synonym matching. The experimental results show that SoftMRFED significantly outperforms the other edit distance algorithms on several test data collections. In addition, the performance of SoftMRFED is superior to token-based distance algorithms in two matching tasks.

Original languageEnglish
Pages (from-to)371-387
Number of pages17
JournalKnowledge and Information Systems
Volume25
Issue number2
DOIs
Publication statusPublished - 2010 Nov 1

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems
  • Human-Computer Interaction
  • Hardware and Architecture
  • Artificial Intelligence

Cite this