Evaluating visual analytics systems for investigative analysis: Deriving design principles from a case study

Youn Ah Kang, Carsten Görg, John Stasko

Research output: Chapter in Book/Report/Conference proceedingConference contribution

51 Citations (Scopus)

Abstract

Despite the growing number of systems providing visual analytic support for investigative analysis, few empirical studies of the potential benefits of such systems have been conducted, particularly controlled, comparative evaluations. Determining how such systems foster insight and sensemaking is important for their continued growth and study, however. Furthermore, studies that identify how people use such systems and why they benefit (or not) can help inform the design of new systems in this area. We conducted an evaluation of the visual analytics system Jigsaw employed in a small investigative sensemaking exercise, and we compared its use to three other more traditional methods of analysis. Sixteen participants performed a simulated intelligence analysis task under one of the four conditions. Experimental results suggest that Jigsaw assisted participants to analyze the data and identify an embedded threat. We describe different analysis strategies used by study participants and how computational support (or the lack thereof) influenced the strategies. We then illustrate several characteristics of the sensemaking process identified in the study and provide design implications for investigative analysis tools based thereon. We conclude with recommendations for metrics and techniques for evaluating other visual analytics investigative analysis tools.

Original languageEnglish
Title of host publicationVAST 09 - IEEE Symposium on Visual Analytics Science and Technology, Proceedings
Pages139-146
Number of pages8
DOIs
Publication statusPublished - 2009 Dec 1
EventVAST 09 - IEEE Symposium on Visual Analytics Science and Technology - Atlantic City, NJ, United States
Duration: 2009 Oct 122009 Oct 13

Publication series

NameVAST 09 - IEEE Symposium on Visual Analytics Science and Technology, Proceedings

Other

OtherVAST 09 - IEEE Symposium on Visual Analytics Science and Technology
CountryUnited States
CityAtlantic City, NJ
Period09/10/1209/10/13

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Information Systems

Cite this

Kang, Y. A., Görg, C., & Stasko, J. (2009). Evaluating visual analytics systems for investigative analysis: Deriving design principles from a case study. In VAST 09 - IEEE Symposium on Visual Analytics Science and Technology, Proceedings (pp. 139-146). [5333878] (VAST 09 - IEEE Symposium on Visual Analytics Science and Technology, Proceedings). https://doi.org/10.1109/VAST.2009.5333878