A Scalable Inference Pipeline for 3D Axon Tracing Algorithms

Benjamin Fenelon; Lars A. Gjesteby; Webster Guan; Juhyuk Park; Kwanghun Chung; Laura J. Brattain

doi:10.1109/HPEC55821.2022.9926403

A Scalable Inference Pipeline for 3D Axon Tracing Algorithms

Benjamin Fenelon, Lars A. Gjesteby, Webster Guan, Juhyuk Park, Kwanghun Chung, Laura J. Brattain

Yonsei Advanced Science Institute

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

High inference times of machine learning-based axon tracing algorithms pose a significant challenge to the practical analysis and interpretation of large-scale brain imagery. This paper explores a distributed data pipeline that employs a SLURM-based job array to run multiple machine learning algorithm predictions simultaneously. Image volumes were split into N (1-16) equal chunks that are each handled by a unique compute node and stitched back together into a single 3D prediction. Preliminary results comparing the inference speed of 1 versus 16 node job arrays demonstrated a 90.95% decrease in compute time for 32 GB input volume and 88.41% for 4 GB input volume. The general pipeline may serve as a baseline for future improved implementations on larger input volumes which can be tuned to various application domains.

Original language	English
Title of host publication	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665497862
DOIs	https://doi.org/10.1109/HPEC55821.2022.9926403
Publication status	Published - 2022
Event	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 - Virtual, Online, United States Duration: 2022 Sept 19 → 2022 Sept 23

Publication series

Name	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

Conference

Conference	2022 IEEE High Performance Extreme Computing Conference, HPEC 2022
Country/Territory	United States
City	Virtual, Online
Period	22/9/19 → 22/9/23

Bibliographical note

Funding Information:
The authors would like to acknowledge Adam Michaleas and the MIT Lincoln Laboratory Supercomputing Center (LLSC) for their support of high performance computing tasks

Publisher Copyright:
© 2022 IEEE.

All Science Journal Classification (ASJC) codes

Artificial Intelligence
Computer Science Applications
Hardware and Architecture
Software
Computational Mathematics
Numerical Analysis

Access to Document

10.1109/HPEC55821.2022.9926403

Cite this

Fenelon, B., Gjesteby, L. A., Guan, W., Park, J., Chung, K., & Brattain, L. J. (2022). A Scalable Inference Pipeline for 3D Axon Tracing Algorithms. In 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 (2022 IEEE High Performance Extreme Computing Conference, HPEC 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/HPEC55821.2022.9926403

@inproceedings{1d73b87f8aae4935b4c8f229117e9840,

title = "A Scalable Inference Pipeline for 3D Axon Tracing Algorithms",

abstract = "High inference times of machine learning-based axon tracing algorithms pose a significant challenge to the practical analysis and interpretation of large-scale brain imagery. This paper explores a distributed data pipeline that employs a SLURM-based job array to run multiple machine learning algorithm predictions simultaneously. Image volumes were split into N (1-16) equal chunks that are each handled by a unique compute node and stitched back together into a single 3D prediction. Preliminary results comparing the inference speed of 1 versus 16 node job arrays demonstrated a 90.95% decrease in compute time for 32 GB input volume and 88.41% for 4 GB input volume. The general pipeline may serve as a baseline for future improved implementations on larger input volumes which can be tuned to various application domains.",

author = "Benjamin Fenelon and Gjesteby, {Lars A.} and Webster Guan and Juhyuk Park and Kwanghun Chung and Brattain, {Laura J.}",

note = "Funding Information: The authors would like to acknowledge Adam Michaleas and the MIT Lincoln Laboratory Supercomputing Center (LLSC) for their support of high performance computing tasks Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022 ; Conference date: 19-09-2022 Through 23-09-2022",

year = "2022",

doi = "10.1109/HPEC55821.2022.9926403",

language = "English",

series = "2022 IEEE High Performance Extreme Computing Conference, HPEC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2022 IEEE High Performance Extreme Computing Conference, HPEC 2022",

address = "United States",

}

Fenelon, B, Gjesteby, LA, Guan, W, Park, J, Chung, K & Brattain, LJ 2022, A Scalable Inference Pipeline for 3D Axon Tracing Algorithms. in 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022. 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022, Institute of Electrical and Electronics Engineers Inc., 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022, Virtual, Online, United States, 22/9/19. https://doi.org/10.1109/HPEC55821.2022.9926403

A Scalable Inference Pipeline for 3D Axon Tracing Algorithms. / Fenelon, Benjamin; Gjesteby, Lars A.; Guan, Webster et al.

2022 IEEE High Performance Extreme Computing Conference, HPEC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. (2022 IEEE High Performance Extreme Computing Conference, HPEC 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A Scalable Inference Pipeline for 3D Axon Tracing Algorithms

AU - Fenelon, Benjamin

AU - Gjesteby, Lars A.

AU - Guan, Webster

AU - Park, Juhyuk

AU - Chung, Kwanghun

AU - Brattain, Laura J.

N1 - Funding Information: The authors would like to acknowledge Adam Michaleas and the MIT Lincoln Laboratory Supercomputing Center (LLSC) for their support of high performance computing tasks Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - High inference times of machine learning-based axon tracing algorithms pose a significant challenge to the practical analysis and interpretation of large-scale brain imagery. This paper explores a distributed data pipeline that employs a SLURM-based job array to run multiple machine learning algorithm predictions simultaneously. Image volumes were split into N (1-16) equal chunks that are each handled by a unique compute node and stitched back together into a single 3D prediction. Preliminary results comparing the inference speed of 1 versus 16 node job arrays demonstrated a 90.95% decrease in compute time for 32 GB input volume and 88.41% for 4 GB input volume. The general pipeline may serve as a baseline for future improved implementations on larger input volumes which can be tuned to various application domains.

AB - High inference times of machine learning-based axon tracing algorithms pose a significant challenge to the practical analysis and interpretation of large-scale brain imagery. This paper explores a distributed data pipeline that employs a SLURM-based job array to run multiple machine learning algorithm predictions simultaneously. Image volumes were split into N (1-16) equal chunks that are each handled by a unique compute node and stitched back together into a single 3D prediction. Preliminary results comparing the inference speed of 1 versus 16 node job arrays demonstrated a 90.95% decrease in compute time for 32 GB input volume and 88.41% for 4 GB input volume. The general pipeline may serve as a baseline for future improved implementations on larger input volumes which can be tuned to various application domains.

UR - http://www.scopus.com/inward/record.url?scp=85142216060&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85142216060&partnerID=8YFLogxK

U2 - 10.1109/HPEC55821.2022.9926403

DO - 10.1109/HPEC55821.2022.9926403

M3 - Conference contribution

AN - SCOPUS:85142216060

T3 - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

BT - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 IEEE High Performance Extreme Computing Conference, HPEC 2022

Y2 - 19 September 2022 through 23 September 2022

ER -

A Scalable Inference Pipeline for 3D Axon Tracing Algorithms

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this