We address the problem of highlight detection from a 360 ◦ video by summarizing it both spatially and temporally. Given a long 360 ◦ video, we spatially select pleasantly-looking normal field-of-view (NFOV) segments from unlimited field of views (FOV) of the 360 ◦ video, and temporally summarize it into a concise and informative highlight as a selected subset of subshots. We propose a novel deep ranking model named as Composition View Score (CVS) model, which produces a spherical score map of composition per video segment, and determines which view is suitable for highlight via a sliding window kernel at inference. To evaluate the proposed framework, we perform experiments on the Pano2Vid benchmark dataset (Su, Jayaraman, and Grauman 2016) and our newly collected 360 ◦ video highlight dataset from YouTube and Vimeo. Through evaluation using both quantitative summarization metrics and user studies via Amazon Mechanical Turk, we demonstrate that our approach outperforms several state-of-the-art highlight detection methods. We also show that our model is 16 times faster at inference than AutoCam (Su, Jayaraman, and Grauman 2016), which is one of the first summarization algorithms of 360 ◦ videos.
|Title of host publication||32nd AAAI Conference on Artificial Intelligence, AAAI 2018|
|Number of pages||9|
|Publication status||Published - 2018|
|Event||32nd AAAI Conference on Artificial Intelligence, AAAI 2018 - New Orleans, United States|
Duration: 2018 Feb 2 → 2018 Feb 7
|Name||32nd AAAI Conference on Artificial Intelligence, AAAI 2018|
|Other||32nd AAAI Conference on Artificial Intelligence, AAAI 2018|
|Period||18/2/2 → 18/2/7|
Bibliographical noteFunding Information:
helpful discussion about the model. This work was supported by the Visual Display Business (RAK0117ZZ-21RF) of Samsung Electronics. Gunhee Kim is the corresponding author.
Copyright © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
All Science Journal Classification (ASJC) codes
- Artificial Intelligence