Evaluating interactive bibliographic information retrieval systems: A user-centered approach

Yongjun Zhu, Meen Chul Kim, Erjia Yan

Research output: Contribution to journalArticlepeer-review


As an integral part of research activities, scholars frequently interact with bibliographic information retrieval systems to acquire scholarly information. With the dramatic increase of scientific literature, there is the pressing need of building effective and efficient bibliographic information retrieval systems that support more granular and complex bibliographic information needs. Investigating users' interaction with these systems plays an important role in understanding the systems' features and usability issues, which contributes to better system design and implementation. In this paper, we evaluate three interactive bibliographic information retrieval systems proposed in our previous studies using a user-centered approach, including form-, natural language-, and visual graph-based systems. This study recruits 20 participants to evaluate the three systems from the aspects of success rate, search time, query size, usefulness, ease of use, ease of learning, and satisfaction. Results showed that: the form-based system needed the least time to formulate simple bibliographic queries and allowed the participants to make the least mistakes; the natural language-based system needed the least time to formulate high-complexity bibliographic queries and was rated as the most useful, easy-to-use, and easy-to-learn system; and the visual graph-based system's strengths lie in the support of complex queries as shown that it got better ratings of usefulness and satisfaction by the high-complexity task group.

Original languageEnglish
Pages (from-to)628-637
Number of pages10
JournalProceedings of the Association for Information Science and Technology
Issue number1
Publication statusPublished - 2018 Jan

Bibliographical note

Funding Information:
This project was made possible in part by the Institute of Museum and Library Services (Grant Award Number: RE-07-15-0060-15), for the project titled “Building an entity-based research framework to enhance digital services on knowledge discovery and delivery.”

Publisher Copyright:
Copyright © 2018 by Association for Information Science and Technology

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Library and Information Sciences


Dive into the research topics of 'Evaluating interactive bibliographic information retrieval systems: A user-centered approach'. Together they form a unique fingerprint.

Cite this