Learning from PhotoShop Operation Videos: The PSOV Dataset

Jingchun Cheng; Han Kai Hsu; Chen Fang; Hailin Jin; Shengjin Wang; Ming Hsuan Yang

doi:10.1007/978-3-030-20870-7_14

Learning from PhotoShop Operation Videos: The PSOV Dataset

Jingchun Cheng, Han Kai Hsu, Chen Fang, Hailin Jin, Shengjin Wang, Ming Hsuan Yang

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we present the PhotoShop Operation Video (PSOV) dataset, a large-scale, densely annotated video database designed for the development of software intelligence. The PSOV dataset consists of 564 densely-annotated videos for Photoshop operations, covering more than 500 commonly used commands in the Photoshop software. Videos in this dataset are obtained from YouTube, manually watched and annotated precisely to seconds by experts. There are more than 74Â h of videos with 29,204 labeled commands. To the best of our knowledge, the PSOV dataset is the first large-scale software operation video database with high-resolution frames and dense annotations. We believe that this dataset can help advance the development of intelligent software, and has extensive application aspects. In this paper, we describe the dataset construction procedure, data attributes, proposed tasks and their corresponding evaluation metrics. To demonstrate that the PSOV dataset has sufficient data and labeling for data-driven methods, we develop a deep learning based algorithm for the command classification task. We also carry out experiments and analysis with the proposed method to encourage better understanding and usage of the PSOV dataset.

Original language	English
Title of host publication	Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
Editors	Greg Mori, Hongdong Li, C.V. Jawahar, Konrad Schindler
Publisher	Springer Verlag
Pages	223-239
Number of pages	17
ISBN (Print)	9783030208691
DOIs	https://doi.org/10.1007/978-3-030-20870-7_14
Publication status	Published - 2019
Event	14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia Duration: 2018 Dec 2 → 2018 Dec 6

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11364 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th Asian Conference on Computer Vision, ACCV 2018
Country/Territory	Australia
City	Perth
Period	18/12/2 → 18/12/6

Bibliographical note

Funding Information:
This work is supported in part by the NSF CAREER Grant #1149783, and gifts from Adobe.

Publisher Copyright:
© 2019, Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-20870-7_14

Cite this

Cheng, J., Hsu, H. K., Fang, C., Jin, H., Wang, S., & Yang, M. H. (2019). Learning from PhotoShop Operation Videos: The PSOV Dataset. In G. Mori, H. Li, C. V. Jawahar, & K. Schindler (Eds.), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers (pp. 223-239). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-20870-7_14

Cheng, Jingchun ; Hsu, Han Kai ; Fang, Chen et al. / Learning from PhotoShop Operation Videos : The PSOV Dataset. Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. editor / Greg Mori ; Hongdong Li ; C.V. Jawahar ; Konrad Schindler. Springer Verlag, 2019. pp. 223-239 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{377c346740054c56a60be5d27ad31bf9,

title = "Learning from PhotoShop Operation Videos: The PSOV Dataset",

abstract = "In this paper, we present the PhotoShop Operation Video (PSOV) dataset, a large-scale, densely annotated video database designed for the development of software intelligence. The PSOV dataset consists of 564 densely-annotated videos for Photoshop operations, covering more than 500 commonly used commands in the Photoshop software. Videos in this dataset are obtained from YouTube, manually watched and annotated precisely to seconds by experts. There are more than 74{\^A} h of videos with 29,204 labeled commands. To the best of our knowledge, the PSOV dataset is the first large-scale software operation video database with high-resolution frames and dense annotations. We believe that this dataset can help advance the development of intelligent software, and has extensive application aspects. In this paper, we describe the dataset construction procedure, data attributes, proposed tasks and their corresponding evaluation metrics. To demonstrate that the PSOV dataset has sufficient data and labeling for data-driven methods, we develop a deep learning based algorithm for the command classification task. We also carry out experiments and analysis with the proposed method to encourage better understanding and usage of the PSOV dataset.",

author = "Jingchun Cheng and Hsu, {Han Kai} and Chen Fang and Hailin Jin and Shengjin Wang and Yang, {Ming Hsuan}",

note = "Funding Information: This work is supported in part by the NSF CAREER Grant #1149783, and gifts from Adobe. Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 14th Asian Conference on Computer Vision, ACCV 2018 ; Conference date: 02-12-2018 Through 06-12-2018",

year = "2019",

doi = "10.1007/978-3-030-20870-7_14",

language = "English",

isbn = "9783030208691",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "223--239",

editor = "Greg Mori and Hongdong Li and C.V. Jawahar and Konrad Schindler",

booktitle = "Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers",

address = "Germany",

}

Cheng, J, Hsu, HK, Fang, C, Jin, H, Wang, S & Yang, MH 2019, Learning from PhotoShop Operation Videos: The PSOV Dataset. in G Mori, H Li, CV Jawahar & K Schindler (eds), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11364 LNCS, Springer Verlag, pp. 223-239, 14th Asian Conference on Computer Vision, ACCV 2018, Perth, Australia, 18/12/2. https://doi.org/10.1007/978-3-030-20870-7_14

Learning from PhotoShop Operation Videos : The PSOV Dataset. / Cheng, Jingchun; Hsu, Han Kai; Fang, Chen et al.

Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. ed. / Greg Mori; Hongdong Li; C.V. Jawahar; Konrad Schindler. Springer Verlag, 2019. p. 223-239 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11364 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Learning from PhotoShop Operation Videos

T2 - 14th Asian Conference on Computer Vision, ACCV 2018

AU - Cheng, Jingchun

AU - Hsu, Han Kai

AU - Fang, Chen

AU - Jin, Hailin

AU - Wang, Shengjin

AU - Yang, Ming Hsuan

PY - 2019

Y1 - 2019

N2 - In this paper, we present the PhotoShop Operation Video (PSOV) dataset, a large-scale, densely annotated video database designed for the development of software intelligence. The PSOV dataset consists of 564 densely-annotated videos for Photoshop operations, covering more than 500 commonly used commands in the Photoshop software. Videos in this dataset are obtained from YouTube, manually watched and annotated precisely to seconds by experts. There are more than 74Â h of videos with 29,204 labeled commands. To the best of our knowledge, the PSOV dataset is the first large-scale software operation video database with high-resolution frames and dense annotations. We believe that this dataset can help advance the development of intelligent software, and has extensive application aspects. In this paper, we describe the dataset construction procedure, data attributes, proposed tasks and their corresponding evaluation metrics. To demonstrate that the PSOV dataset has sufficient data and labeling for data-driven methods, we develop a deep learning based algorithm for the command classification task. We also carry out experiments and analysis with the proposed method to encourage better understanding and usage of the PSOV dataset.

AB - In this paper, we present the PhotoShop Operation Video (PSOV) dataset, a large-scale, densely annotated video database designed for the development of software intelligence. The PSOV dataset consists of 564 densely-annotated videos for Photoshop operations, covering more than 500 commonly used commands in the Photoshop software. Videos in this dataset are obtained from YouTube, manually watched and annotated precisely to seconds by experts. There are more than 74Â h of videos with 29,204 labeled commands. To the best of our knowledge, the PSOV dataset is the first large-scale software operation video database with high-resolution frames and dense annotations. We believe that this dataset can help advance the development of intelligent software, and has extensive application aspects. In this paper, we describe the dataset construction procedure, data attributes, proposed tasks and their corresponding evaluation metrics. To demonstrate that the PSOV dataset has sufficient data and labeling for data-driven methods, we develop a deep learning based algorithm for the command classification task. We also carry out experiments and analysis with the proposed method to encourage better understanding and usage of the PSOV dataset.

UR - http://www.scopus.com/inward/record.url?scp=85066872618&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066872618&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-20870-7_14

DO - 10.1007/978-3-030-20870-7_14

M3 - Conference contribution

AN - SCOPUS:85066872618

SN - 9783030208691

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 223

EP - 239

BT - Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers

A2 - Mori, Greg

A2 - Li, Hongdong

A2 - Jawahar, C.V.

A2 - Schindler, Konrad

PB - Springer Verlag

Y2 - 2 December 2018 through 6 December 2018

ER -

Cheng J, Hsu HK, Fang C, Jin H, Wang S, Yang MH. Learning from PhotoShop Operation Videos: The PSOV Dataset. In Mori G, Li H, Jawahar CV, Schindler K, editors, Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Springer Verlag. 2019. p. 223-239. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-20870-7_14

Learning from PhotoShop Operation Videos: The PSOV Dataset

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this