A robust change-point detection method by eliminating sparse noises from time series

Kun Qin; Lei Sun; Bo Liu; Yuan Fan; Kar Ann Toh

doi:10.1109/DSC.2018.00029

A robust change-point detection method by eliminating sparse noises from time series

Kun Qin, Lei Sun, Bo Liu, Yuan Fan, Kar Ann Toh

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Singular Spectrum Transform (SST) is a fundamental subspace analysis technique which has been widely adopted for solving change-point detection (CPD) problems in information security applications. However, the performance of a SST based CPD algorithm is limited to the lack of robustness to corrupted observations with large noises in practice. Based on the observation that large noises in practical time series are generally sparse, in this paper, we study a combination of Robust Principal Component Analysis (RPCA) and SST to obtain a robust CPD algorithm dealing with sparse large noises. The sparse large noises are to be eliminated from observation trajectory matrices by performing a low-rank matrix recovery procedure of RPCA. The noise-eliminated matrices are then used to extract SST subspaces for CPD. The effectiveness of the proposed method is demonstrated through experiments based on both synthetic and real-world datasets. Experimental results show that the proposed method outperforms the competing state-of-the-arts in terms of detection accuracy for time series with sparse large noises.

Original language	English
Title of host publication	Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	146-152
Number of pages	7
ISBN (Electronic)	9781538642108
DOIs	https://doi.org/10.1109/DSC.2018.00029
Publication status	Published - 2018 Jul 16
Event	3rd IEEE International Conference on Data Science in Cyberspace, DSC 2018 - Guangzhou, Guangdong, China Duration: 2018 Jun 18 → 2018 Jun 21

Publication series

Name	Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018

Other

Other	3rd IEEE International Conference on Data Science in Cyberspace, DSC 2018
Country/Territory	China
City	Guangzhou, Guangdong
Period	18/6/18 → 18/6/21

Bibliographical note

Funding Information:
This work was partly supported by National NSF of China (No.61673059, No.61771051, No. U1713215).

Publisher Copyright:
© 2018 IEEE.

All Science Journal Classification (ASJC) codes

Computer Networks and Communications
Information Systems
Information Systems and Management
Safety, Risk, Reliability and Quality

Access to Document

10.1109/DSC.2018.00029

Cite this

Qin, K., Sun, L., Liu, B., Fan, Y., & Toh, K. A. (2018). A robust change-point detection method by eliminating sparse noises from time series. In Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018 (pp. 146-152). [8411850] (Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DSC.2018.00029

Qin, Kun ; Sun, Lei ; Liu, Bo ; Fan, Yuan ; Toh, Kar Ann. / A robust change-point detection method by eliminating sparse noises from time series. Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 146-152 (Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018).

@inproceedings{05a8d470372f4318bd5d65ec28c8122e,

title = "A robust change-point detection method by eliminating sparse noises from time series",

abstract = "Singular Spectrum Transform (SST) is a fundamental subspace analysis technique which has been widely adopted for solving change-point detection (CPD) problems in information security applications. However, the performance of a SST based CPD algorithm is limited to the lack of robustness to corrupted observations with large noises in practice. Based on the observation that large noises in practical time series are generally sparse, in this paper, we study a combination of Robust Principal Component Analysis (RPCA) and SST to obtain a robust CPD algorithm dealing with sparse large noises. The sparse large noises are to be eliminated from observation trajectory matrices by performing a low-rank matrix recovery procedure of RPCA. The noise-eliminated matrices are then used to extract SST subspaces for CPD. The effectiveness of the proposed method is demonstrated through experiments based on both synthetic and real-world datasets. Experimental results show that the proposed method outperforms the competing state-of-the-arts in terms of detection accuracy for time series with sparse large noises.",

author = "Kun Qin and Lei Sun and Bo Liu and Yuan Fan and Toh, {Kar Ann}",

note = "Funding Information: This work was partly supported by National NSF of China (No.61673059, No.61771051, No. U1713215). Publisher Copyright: {\textcopyright} 2018 IEEE.; 3rd IEEE International Conference on Data Science in Cyberspace, DSC 2018 ; Conference date: 18-06-2018 Through 21-06-2018",

year = "2018",

month = jul,

day = "16",

doi = "10.1109/DSC.2018.00029",

language = "English",

series = "Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "146--152",

booktitle = "Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018",

address = "United States",

}

Qin, K, Sun, L, Liu, B, Fan, Y & Toh, KA 2018, A robust change-point detection method by eliminating sparse noises from time series. in Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018., 8411850, Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018, Institute of Electrical and Electronics Engineers Inc., pp. 146-152, 3rd IEEE International Conference on Data Science in Cyberspace, DSC 2018, Guangzhou, Guangdong, China, 18/6/18. https://doi.org/10.1109/DSC.2018.00029

A robust change-point detection method by eliminating sparse noises from time series. / Qin, Kun; Sun, Lei; Liu, Bo; Fan, Yuan; Toh, Kar Ann.

Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 146-152 8411850 (Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A robust change-point detection method by eliminating sparse noises from time series

AU - Qin, Kun

AU - Sun, Lei

AU - Liu, Bo

AU - Fan, Yuan

AU - Toh, Kar Ann

PY - 2018/7/16

Y1 - 2018/7/16

N2 - Singular Spectrum Transform (SST) is a fundamental subspace analysis technique which has been widely adopted for solving change-point detection (CPD) problems in information security applications. However, the performance of a SST based CPD algorithm is limited to the lack of robustness to corrupted observations with large noises in practice. Based on the observation that large noises in practical time series are generally sparse, in this paper, we study a combination of Robust Principal Component Analysis (RPCA) and SST to obtain a robust CPD algorithm dealing with sparse large noises. The sparse large noises are to be eliminated from observation trajectory matrices by performing a low-rank matrix recovery procedure of RPCA. The noise-eliminated matrices are then used to extract SST subspaces for CPD. The effectiveness of the proposed method is demonstrated through experiments based on both synthetic and real-world datasets. Experimental results show that the proposed method outperforms the competing state-of-the-arts in terms of detection accuracy for time series with sparse large noises.

AB - Singular Spectrum Transform (SST) is a fundamental subspace analysis technique which has been widely adopted for solving change-point detection (CPD) problems in information security applications. However, the performance of a SST based CPD algorithm is limited to the lack of robustness to corrupted observations with large noises in practice. Based on the observation that large noises in practical time series are generally sparse, in this paper, we study a combination of Robust Principal Component Analysis (RPCA) and SST to obtain a robust CPD algorithm dealing with sparse large noises. The sparse large noises are to be eliminated from observation trajectory matrices by performing a low-rank matrix recovery procedure of RPCA. The noise-eliminated matrices are then used to extract SST subspaces for CPD. The effectiveness of the proposed method is demonstrated through experiments based on both synthetic and real-world datasets. Experimental results show that the proposed method outperforms the competing state-of-the-arts in terms of detection accuracy for time series with sparse large noises.

UR - http://www.scopus.com/inward/record.url?scp=85051030176&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85051030176&partnerID=8YFLogxK

U2 - 10.1109/DSC.2018.00029

DO - 10.1109/DSC.2018.00029

M3 - Conference contribution

AN - SCOPUS:85051030176

T3 - Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018

SP - 146

EP - 152

BT - Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 3rd IEEE International Conference on Data Science in Cyberspace, DSC 2018

Y2 - 18 June 2018 through 21 June 2018

ER -

Qin K, Sun L, Liu B, Fan Y, Toh KA. A robust change-point detection method by eliminating sparse noises from time series. In Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 146-152. 8411850. (Proceedings - 2018 IEEE 3rd International Conference on Data Science in Cyberspace, DSC 2018). https://doi.org/10.1109/DSC.2018.00029

A robust change-point detection method by eliminating sparse noises from time series

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this