The open connectome project data cluster: Scalable analysis and vision for high-throughput neuroscience

Randal Burns, William Gray Roncal, Dean Kleissas, Kunal Lillaney, Priya Manavalan, Eric Perlman, Daniel R. Berger, Davi D. Bock, Kwanghun Chung, Logan Grosenick, Narayanan Kasthuri, Nicholas C. Weiler, Karl Deisseroth, Michael Kazhdan, Jeff Lichtman, R. Clay Reid, Stephen J. Smith, Alexander S. Szalay, Joshua T. Vogelstein, R. Jacob Vogelstein

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)

Abstract

We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes- neural connectivity maps of the brain-using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems-reads to parallel disk arrays and writes to solid-state storage-to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effectiveness of spatial data organization.

Original languageEnglish
Title of host publicationSSDBM 2013 - Proceedings of the 25th International Conference on Scientific and Statistical Database Management
DOIs
Publication statusPublished - 2013
Event25th International Conference on Scientific and Statistical Database Management, SSDBM 2013 - Baltimore, MD, United States
Duration: 2013 Jul 292013 Jul 31

Publication series

NameACM International Conference Proceeding Series

Conference

Conference25th International Conference on Scientific and Statistical Database Management, SSDBM 2013
CountryUnited States
CityBaltimore, MD
Period13/7/2913/7/31

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'The open connectome project data cluster: Scalable analysis and vision for high-throughput neuroscience'. Together they form a unique fingerprint.

Cite this