Thanks to the recent advances in Deep Neural Networks (DNNs), DNN-based object detection systems become highly accurate and widely used in real-time environments such as autonomous vehicles, drones and security robots. Although the systems should detect objects within a certain time limit that can vary depending on their execution environments such as vehicle speeds, existing systems blindly execute the entire long-latency DNNs without reflecting the time-varying time limits, and thus they cannot guarantee real-time constraints. This work proposes a novel real-time object detection system that employs multipath neural networks based on a new worst-case execution time (WCET) model for DNNs on a GPU. This work designs the WCET model for a single DNN layer analyzing processor and memory contention on GPUs, and extends the WCET model to the end-to-end networks. This work also designs the multipath networks with three new operators such as skip, switch, and dynamic generate proposals that dynamically change their execution paths and the number of target objects. Finally, this work proposes a path decision model that chooses the optimal execution path at run-time reflecting dynamically changing environments and time constraints. Our detailed evaluation using widely-used driving datasets shows that the proposed real-time object detection system performs as good as a baseline object detection system without violating the time-varying time limits. Moreover, the WCET model predicts the worst-case execution latency of convolutional and group normalization layers with only 27% and 81% errors on average, respectively.
|Title of host publication||Proceedings - 2020 IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2020|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||14|
|Publication status||Published - 2020 Apr|
|Event||26th IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2020 - Sydney, Australia|
Duration: 2020 Apr 21 → 2020 Apr 24
|Name||Proceedings of the IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS|
|Conference||26th IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2020|
|Period||20/4/21 → 20/4/24|
Bibliographical noteFunding Information:
We thank the anonymous reviewers and shepherd for their valuable feedback. This research was supported by Sam-sung Research Funding Center of Samsung Electronics under Project Number SRFC-TB1703-03. This research was also partly supported by the Yonsei university research fund of 2019. Hanjun Kim is the corresponding author of this paper.
© 2020 IEEE.
All Science Journal Classification (ASJC) codes